Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

…derstands.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

…hanos-io#2697)

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

) (thanos-io#2698)

* Allow using multiple memcached clients at the same time.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Added unit test to verify that we can have multiple clients.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Added CHANGELOG.md entry.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Remove unused argument.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>
# Conflicts:
#	CHANGELOG.md

…rting for StoreAPI + Optimized iter chunk dedup. (thanos-io#2710) (thanos-io#2711)

* Deduplicate chunk dups on proxy StoreAPI level. Recommend chunk sorting for StoreAPI.

Also: Merge same series together on proxy level instead select. This allows better dedup efficiency.

Partially fixes: thanos-io#2303

Cases like overlapped data from store and sidecar and 1:1 duplicates are optimized as soon as it's possible.
This case was highly visible on GitLab repro data and exists in most of Thanos setup.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Optimized algorithm to combine series only on start.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Optimized chunk comparision for overlaps.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Optimized deduplication for deduplicated chunk on query level as well.

Never use proto .String() in fast path!

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>
# Conflicts:
#	CHANGELOG.md
#	pkg/store/storepb/custom.go
#	pkg/store/storepb/custom_test.go

…nts (thanos-io#2705) (thanos-io#2718)

* Updated minio dependency to v6.0.56 for af-south-1 region

Signed-off-by: Theunis Botha <theunisb@jurumani.com>

* Updated CHANGELOG.md

Signed-off-by: Theunis Botha <theunisb@jurumani.com>

thanos-io#2719)

See related discussion: https://cloud-native.slack.com/archives/CL25937SP/p1590408991325100

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

# Conflicts:
#	CHANGELOG.md

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

…v0.13.0-rc.2 (thanos-io#2765)

* shipper: Be strict about upload order unless it's specified so.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* Cut 0.13.0-rc.2

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

…s-io#2772)

* Add more tracing spans for receive

Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>

* Add spans for read path

Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>

* Fix context propogation issue

Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>

Signed-off-by: yeya24 <yb532204897@gmail.com>

* Implement async select for Querier

Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>

* Use prometheus/gate for now

Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>

* Add changelog and mention behaviour in documentation

Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>

* Use new gate.Gate implementation

Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>

* Preserve tracing context

Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>

Various tools in the SRE space use UTC as the default to make it
effortless during incidents to not have to calculate between timezones.
The Thanos query UI also defaults to this just like Prometheus.

Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com>

* ui: Add ErrorBoundary to the React UI

Signed-off-by: Prem Kumar <prmsrswt@gmail.com>

* query:  Return empty array instead of nil from API

Signed-off-by: Prem Kumar <prmsrswt@gmail.com>

* ui: Fix typo; remove unnecessary inline style

Signed-off-by: Prem Kumar <prmsrswt@gmail.com>

* format bindata.go

Signed-off-by: Prem Kumar <prmsrswt@gmail.com>

* Add more context for receive logs

Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>

* Add request id for receive logs

Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>

* Add memcached server address to Set() error

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Addressed review comments

Signed-off-by: Marco Pracucci <marco@pracucci.com>

Signed-off-by: Prem Kumar <prmsrswt@gmail.com>

…hanos-io#2787)

Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com>

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

Cut release v0.13.0

# Conflicts:
#	.circleci/config.yml
#	CHANGELOG.md
#	go.mod
#	go.sum
#	pkg/shipper/shipper.go
#	pkg/shipper/shipper_e2e_test.go
#	pkg/shipper/shipper_test.go
#	pkg/store/storepb/custom.go
#	pkg/store/storepb/custom_test.go
#	pkg/ui/bindata.go

Merge 0.13.0 to master

Signed-off-by: soniasingla <soniasingla.1812@gmail.com>

Signed-off-by: Marco Pracucci <marco@pracucci.com>

Signed-off-by: thisisobate <obasiuche62@gmail.com>

Signed-off-by: soniasingla <soniasingla.1812@gmail.com>

Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com>

…ease shephard (thanos-io#2804)

Also moved release date to next week as Kemal is PTO this week.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

* ui: Add store page to React UI

Signed-off-by: Prem Kumar <prmsrswt@gmail.com>

* query: Remove unnecessary field 'store_type' from api response

Signed-off-by: Prem Kumar <prmsrswt@gmail.com>

* ui: React: Show infinity icon the time is too big

Signed-off-by: Prem Kumar <prmsrswt@gmail.com>

* ui: Add tests for the Stores page in React UI

Signed-off-by: Prem Kumar <prmsrswt@gmail.com>

* Reduced memory allocations in readIndexRange()

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Updated CHANGELOG

Signed-off-by: Marco Pracucci <marco@pracucci.com>

…2808)

Add some verbiage that explains when exactly do the blocks get deleted.

Closes thanos-io#2782.

Signed-off-by: Giedrius Statkevičius <giedriuswork@gmail.com>

…anos-io#2752)

Thanos compact currently does metadata sync every minute if `--wait` is
active. Thanos store has a equivalent flag named `sync-block-duration`.
This commit introduces `--block-viewer.global.sync-block-interval` to be able to
configure the interval via a flag.

Signed-off-by: Schlotter, Christian <christi.schlotter@gmail.com>

)

* Preallocate output buffer when encoding postings.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Rename values to length.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

* Renamed count method to length.

Signed-off-by: Peter Štibraný <peter.stibrany@grafana.com>

Signed-off-by: Matthias Loibl <mail@matthiasloibl.com>

…NoUpload-improvement

Improve ThanosReceiveNoUpload to only alert on current instances

* query: Add /status/flags endpoint

Signed-off-by: Prem Kumar <prmsrswt@gmail.com>

* ui: Allow dropdowns in Navbar; enable flags page

Signed-off-by: Prem Kumar <prmsrswt@gmail.com>

* go.mod: Bump Prometheus to current latest

Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com>

* cmd/thanos/rule: Fix rule manager options field name

This change is necessary due to the Prometheus bump, which down-scopes
the interface required for the rule manager to replay ALERT_FOR_STATE.

Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com>

* pkg/block/indexheader: Fix reading from index in test

Signed-off-by: Frederic Branczyk <fbranczyk@gmail.com>

…hanos-io#2816)

startTimeMargin and StartTime is used only by Prometheus remote read, Thanos does not use it.
Fixed following race:

```
=== RUN   TestMultiTSDB/run_on_existing_storage
==================
WARNING: DATA RACE
Read at 0x00c00073ae80 by goroutine 69:
  github.com/prometheus/prometheus/tsdb.validateOpts()
      /home/bwplotka/Repos/thanosgopath/pkg/mod/github.com/prometheus/prometheus@v1.8.2-0.20200609165731-66dfb951c4ca/tsdb/db.go:510 +0x55
  github.com/prometheus/prometheus/tsdb.Open()
      /home/bwplotka/Repos/thanosgopath/pkg/mod/github.com/prometheus/prometheus@v1.8.2-0.20200609165731-66dfb951c4ca/tsdb/db.go:502 +0x61
  github.com/thanos-io/thanos/pkg/receive.(*MultiTSDB).getOrLoadTenant.func1()
      /home/bwplotka/Repos/thanos/pkg/receive/multitsdb.go:268 +0x56b
  github.com/thanos-io/thanos/pkg/receive.(*MultiTSDB).getOrLoadTenant()
      /home/bwplotka/Repos/thanos/pkg/receive/multitsdb.go:302 +0x4ef
  github.com/thanos-io/thanos/pkg/receive.(*MultiTSDB).Open.func1()
      /home/bwplotka/Repos/thanos/pkg/receive/multitsdb.go:142 +0x66
  golang.org/x/sync/errgroup.(*Group).Go.func1()
      /home/bwplotka/Repos/thanosgopath/pkg/mod/golang.org/x/sync@v0.0.0-20200317015054-43a5402ce75a/errgroup/errgroup.go:57 +0x85

Previous write at 0x00c00073ae80 by goroutine 57:
  github.com/prometheus/prometheus/tsdb.validateOpts()
      /home/bwplotka/Repos/thanosgopath/pkg/mod/github.com/prometheus/prometheus@v1.8.2-0.20200609165731-66dfb951c4ca/tsdb/db.go:511 +0x1f2
  github.com/prometheus/prometheus/tsdb.Open()
      /home/bwplotka/Repos/thanosgopath/pkg/mod/github.com/prometheus/prometheus@v1.8.2-0.20200609165731-66dfb951c4ca/tsdb/db.go:502 +0x61
  github.com/thanos-io/thanos/pkg/receive.(*MultiTSDB).getOrLoadTenant.func1()
      /home/bwplotka/Repos/thanos/pkg/receive/multitsdb.go:268 +0x56b
  github.com/thanos-io/thanos/pkg/receive.(*MultiTSDB).getOrLoadTenant()
      /home/bwplotka/Repos/thanos/pkg/receive/multitsdb.go:302 +0x4ef
  github.com/thanos-io/thanos/pkg/receive.(*MultiTSDB).Open.func1()
      /home/bwplotka/Repos/thanos/pkg/receive/multitsdb.go:142 +0x66
  golang.org/x/sync/errgroup.(*Group).Go.func1()
      /home/bwplotka/Repos/thanosgopath/pkg/mod/golang.org/x/sync@v0.0.0-20200317015054-43a5402ce75a/errgroup/errgroup.go:57 +0x85

Goroutine 69 (running) created at:
  golang.org/x/sync/errgroup.(*Group).Go()
      /home/bwplotka/Repos/thanosgopath/pkg/mod/golang.org/x/sync@v0.0.0-20200317015054-43a5402ce75a/errgroup/errgroup.go:54 +0x73
  github.com/thanos-io/thanos/pkg/receive.(*MultiTSDB).Open()
      /home/bwplotka/Repos/thanos/pkg/receive/multitsdb.go:141 +0x2af
  github.com/thanos-io/thanos/pkg/receive.TestMultiTSDB.func3()
      /home/bwplotka/Repos/thanos/pkg/receive/multitsdb_test.go:118 +0x6d3
  testing.tRunner()
      /home/bwplotka/.gvm/gos/go1.14.2/src/testing/testing.go:991 +0x1eb

Goroutine 57 (running) created at:
  golang.org/x/sync/errgroup.(*Group).Go()
      /home/bwplotka/Repos/thanosgopath/pkg/mod/golang.org/x/sync@v0.0.0-20200317015054-43a5402ce75a/errgroup/errgroup.go:54 +0x73
  github.com/thanos-io/thanos/pkg/receive.(*MultiTSDB).Open()
      /home/bwplotka/Repos/thanos/pkg/receive/multitsdb.go:141 +0x2af
  github.com/thanos-io/thanos/pkg/receive.TestMultiTSDB.func3()
      /home/bwplotka/Repos/thanos/pkg/receive/multitsdb_test.go:118 +0x6d3
  testing.tRunner()
      /home/bwplotka/.gvm/gos/go1.14.2/src/testing/testing.go:991 +0x1eb
==================
```

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

Signed-off-by: Marco Pracucci <marco@pracucci.com>

* Query: always return a string in the `lastError` field

While testing out the new React UI, I have found out that some errors
return empty dicts when marshalled into the JSON format. The response
looks like this:

```
{..., "lastError": {}}
```

And then, obviously, the UI fails to parse that. Add a
`stringifiedError` type with tests which ensures that we always get a
string.

Signed-off-by: Giedrius Statkevičius <giedriuswork@gmail.com>

* CHANGELOG: add PR number + remove whitespace

Signed-off-by: Giedrius Statkevičius <giedriuswork@gmail.com>

* query/storeset: fix a subtle bug

Signed-off-by: Giedrius Statkevičius <giedriuswork@gmail.com>

* Make changes according to Bartek's suggestions

Signed-off-by: Giedrius Statkevičius <giedriuswork@gmail.com>

…leaks ): (thanos-io#2817)

Reason: Missing (), probably we need linter for this.

Signed-off-by: Bartlomiej Plotka <bwplotka@gmail.com>

Right now the alert actually fires in wrong conditions.
We select all sum(instances  == 0) which is always 0...
Instead we want to sum(instances) == 0

Signed-off-by: Matthias Loibl <mail@matthiasloibl.com>

…hanos-io#2824)

* Soft cut changelog on master to indicate 0.14 being in progress

Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>

* Update version in tutorials

Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>

Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>

* Manually download promu in crossbuild stage

Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>

* Make sure all directories created in the path

Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>

…2848)

* correctly marshal errors to JSON and ignore if nil

Signed-off-by: Dan Potepa <dan@danpotepa.co.uk>

* LastError should be cleared if the newer update was ok

Signed-off-by: Dan Potepa <dan@danpotepa.co.uk>
Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>

Signed-off-by: Kemal Akkoyun <kakkoyun@gmail.com>

* upstream/release-0.14: (46 commits)
  Cut release v0.14.0-rc.1 (thanos-io#2853)
  Query: correctly marshal errors to JSON and ignore if nil (thanos-io#2848)
  ci: Manually download promu in crossbuild stage (thanos-io#2828)
  Cut release v0.14.0-rc.0 (thanos-io#2826)
  Soft cut changelog on master to indicate v0.14.0 being in progress (thanos-io#2824)
  Update ThanosReceiveNoUpload to select sum == 0 (thanos-io#2819)
  receive: Added more observability, fixed leaktest, to actually check leaks ): (thanos-io#2817)
  Query: always return a string in the `lastError` field (thanos-io#2809)
  Added missing CHANGELOG entry for PR 2613 (thanos-io#2820)
  receive: Fixed small options race; Removed unused StartTime feature. (thanos-io#2816)
  go.mod: Bump Prometheus to current latest (thanos-io#2814)
  Implement CLI Flags page in React UI (thanos-io#2796)
  Improve ThanosReceiveNoUpload to only alert on current instances
  store: Preallocate output buffer when encoding postings. (thanos-io#2812)
  compact: introduce flag --block-viewer.global.sync-block-interval (thanos-io#2752)
  docs: compact: add blurb about how retention policy works (thanos-io#2808)
  Reduced memory allocations in readIndexRange() (thanos-io#2807)
  ui: Add Stores page to React UI (thanos-io#2754)
  Added Kemal to Maintainer Role; Kemal is volounteering to be next release shephard (thanos-io#2804)
  proposal: Add scalable rule storage proposal (thanos-io#2661)
  ...

Currently, alert state is rendered as upper case.
In Prometheus it is lower case.
This fixes it.

Signed-off-by: Sergiusz Urbaniak <sergiusz.urbaniak@gmail.com>