The link has moved to another since Cisco bought Banzai Cloud.

Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>
Signed-off-by: Michael Hoffmann <mhoffm@posteo.de>

…se-0.34.1

Cut patch release 0.34.1

Attach object storage hash to the iter key so that it would be possible
to reuse the same cache storage e.g. Redis for different buckets.
Without this, the results are funny to say the least if you accidentally
attempt to do that. Thus, let's add the hash to reduce the possibility
of an accident for our users.

Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>

…to-main

…to-main

…34.1-to-main

Merge release 0.34.1 to main

Adds a flag to register the extended promql functions supported by the thanos
query engine when running the rule component.  This will allow rule config
files containing query expressions with (xrate / xincrease / xdelta) to pass
validation.  This will only work if the query endpoint in use is running the
thanos engine.

Signed-off-by: Samuel Dufel <samuel.dufel@shopify.com>

* Add support for extended promql functions in rule

Adds a flag to register the extended promql functions supported by the thanos
query engine when running the rule component.  This will allow rule config
files containing query expressions with (xrate / xincrease / xdelta) to pass
validation.  This will only work if the query endpoint in use is running the
thanos engine.

Signed-off-by: Samuel Dufel <samuel.dufel@shopify.com>

* Update rendered docs with added flag

Signed-off-by: Samuel Dufel <samuel.dufel@shopify.com>

---------

Signed-off-by: Samuel Dufel <samuel.dufel@shopify.com>

Similar to thanos-io#6957, we should copy
labels from remote instant queries so that memory does not get overwritten
when processing series in a central engine.

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>

Adding tests for explain/analyze with QFE. Will add fixes as separate
PR.

Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>

Add path to the hash. This allows identifying difference instances by
different config paths.

Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>

* Allow using different listing strategies

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>

* Expose flags for block list strategy

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>

* Run make docs

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>

* Fix whitespace

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>

* Add CHANGELOG entry

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>

---------

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>

…rations in the cache client (thanos-io#7010)

* Implement the circuit breaker pattern for asynchronous set operations in the cache client

Signed-off-by: Xiaochao Dong (@damnever) <the.xcdong@gmail.com>

* Add feature flag for circuitbreaker

Signed-off-by: Xiaochao Dong (@damnever) <the.xcdong@gmail.com>

* Sync docs

Signed-off-by: Xiaochao Dong (@damnever) <the.xcdong@gmail.com>

* Skip configuration validation if the circuit breaker is disabled

Signed-off-by: Xiaochao Dong (@damnever) <the.xcdong@gmail.com>

* Make lint happy

Signed-off-by: Xiaochao Dong (@damnever) <the.xcdong@gmail.com>

* Abstract the logic of the circuit breaker

Signed-off-by: Xiaochao Dong (@damnever) <the.xcdong@gmail.com>

---------

Signed-off-by: Xiaochao Dong (@damnever) <the.xcdong@gmail.com>

Fix the analysis functionality with query-frontend after the recent
changes. Added tests for this.

Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>

…on (thanos-io#7122)

* bugfix: catch lazy posting optimization using wrong cardinality for estimation

Signed-off-by: Ben Ye <benye@amazon.com>

* update changelog

Signed-off-by: Ben Ye <benye@amazon.com>

---------

Signed-off-by: Ben Ye <benye@amazon.com>

We have detected a problem in the chunk seriers merger where it will
panic in case it encounters native histogram chunks.
I am using thanos as a library for a project and wanted to use the
penalty function to dedup blocks from Prometheus instances.

Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com>

queryfrontend: fix analysis after API changes

* Query UI: Add tenant box

With this commit as tenant box is added to the query UI. It can be used
to specify which tenant to use when making a query.

Signed-off-by: Jacob Baungard Hansen <jacobbaungard@redhat.com>

* Re-compile static react app

Recompiles the static react app as now needed following:
thanos-io#6900

Signed-off-by: Jacob Baungard Hansen <jacobbaungard@redhat.com>

* Move changelog item to appropiate future release

After merging it was under the 0.34 release.

Signed-off-by: Jacob Baungard Hansen <jacobbaungard@redhat.com>

* Move query path tenancy proposal to done

Signed-off-by: Jacob Baungard Hansen <jacobbaungard@redhat.com>

---------

Signed-off-by: Jacob Baungard Hansen <jacobbaungard@redhat.com>

We want to be able to route all tenants which begin with certain letters
to some receivers so we need to have some kind of globbing/regex support
in the hashring. This PR adds that functionality. We've been using this
in prod successfully.

Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>

The current image is deprecated. See
https://discuss.circleci.com/t/remote-docker-image-deprecations-and-eol-for-2024/50176.

Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>

This commit exposes the distributed query execution mode to end-users by unhiding the
flag used to toggle this feature.

The commit also adds documentation on when the mode is appropriate to be used.

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>

Unhide distributed execution mode

We were not merging analysis properly - mergo was overwriting data.
Instead of using a whole library for this, just write two small
functions and use them. Add test to cover this.

Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>

Fixes docs formatting and updates the distributed execution link to the done proposal.

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>

Fix docs

Signed-off-by: Munir Khakhi <munir@improwised.com>

Signed-off-by: Munir Khakhi <munir@improwised.com>

This PR bumps the version of google.golang.org/protobuf to v1.33.0 fix a
potential vulnerability in the protojson.Unmarhsl function [1] that can
occure when unmarshaling a message with a protobuf value.

Even if the function isn't used directly in Thanos it would be safer to
just bump it directly.

[1] https://pkg.go.dev/vuln/GO-2024-2611

Signed-off-by: Daniel Mellado <dmellado@redhat.com>

…r logo

fix: add anchor tag to all images
Signed-off-by: Payal17122000 <raviyapayal17@gmail.com>

Signed-off-by: Vasiliy Rumyantsev <4119114+xBazilio@users.noreply.github.com>

Signed-off-by: Vasiliy Rumyantsev <4119114+xBazilio@users.noreply.github.com>

Do not turn off Ruler if resolving fails. We can still (try to) evaluate
rules even if Alertmanager is not available.

Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>

With this commit we only show the tenant-ui box when enforcement of
tenancy is on, as it is not needed otherwise.

Signed-off-by: Jacob Baungard Hansen <jacobbaungard@redhat.com>

change platform-engineer logo size and make 'https://thanos.io/support/' responsive

compact: properly treat native histogram deduplication in chunk series merger

…guration directories (thanos-io#7199)

Signed-off-by: Daniel Hrabovcak <thespiritxiii@gmail.com>

Signed-off-by: Helia Barroso <helia.barroso@hotmail.com>
Co-authored-by: Helia Barroso <helia.barroso@hotmail.com>

Signed-off-by: Michael Hoffmann <mhoffm@posteo.de>

…er-as-client-channels-unbuffered

storepb: make ServerAsClient channels unbuffered

* Add support for TSDB selector in querier

This PR allows using the query distributed mode against a set of multi-tenant receivers
as described in https://github.com/thanos-io/thanos/blob/main/docs/proposals-done/202301-distributed-query-execution.md#distributed-execution-against-receive-components.

The feature is enabled by a selector.relabel-config flag in the Query component
which allows it to select a subset of TSDBs to query based on their external labels.

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>

* Add CHANGELOG entry and fix docs

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>

* Fix tests

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>

* Add comments

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>

* Add test case for MatchersForLabelSets

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>

* Fix failing test

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>

* Use an unbuffered channel

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>

* Change flag description

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>

* Remove parameter from ServerAsClient

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>

---------

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>

* Update thanos-io/promql-engine

This commit updates the promql-engine module to latest main and modifies
to remote engine based on the breaking change.

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>

* Fix lint

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>

---------

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>

Signed-off-by: Ben Ye <benye@amazon.com>

Co-authored-by: Filip Petkovski <filip.petkovsky@gmail.com>

Signed-off-by: Ben Ye <benye@amazon.com>

* add username cfg to rueidis client

Signed-off-by: Thibault Mange <22740367+thibaultmg@users.noreply.github.com>

* update changelog

Signed-off-by: Thibault Mange <22740367+thibaultmg@users.noreply.github.com>

---------

Signed-off-by: Thibault Mange <22740367+thibaultmg@users.noreply.github.com>

* feat(ui): added BlockSizeStats calculation to blocks page

A block can have a list of contained files set in `.thanos.files`.
If the `files` array is set, all referenced files with `size_bytes` set are counted:
- sum of all `chunk/*` file sizes
- size of index file
- total size (sum of both)

Shows statistics about the selected block in the block details view:
- Total size of block
- Size of index (and percentage of total)
- Size of all chunks (and percentage of total)
- Daily growth, based on total size and block duration

Output is humanized up to Pebibytes and fixed to two decimal places;
raw bytes are accessible through mouse over / title text.

Signed-off-by: Markus Möslinger <markus.moeslinger@socra.dev>

* feat(ui): added aggregated BlockSizeStats to blocks row title

Added total size of all blocks from a source to the row title, beneath the source name.

The shown total size is humanized up to pebibytes and fixed to two decimal places;
raw bytes value is accessible through mouse over / title text.

The shown value will refresh with selected compaction levels, but doesn't take block filter into account.

I thought about showing daily growth as well, but just summing all milliseconds of all blocks doesn't work with overlapping blocks / multiple resolutions.

Signed-off-by: Markus Möslinger <markus.moeslinger@socra.dev>

* chore(docs): added UI block size PR to CHANGELOG.md

Signed-off-by: Markus Möslinger <markus.moeslinger@socra.dev>

* chore(ui): removed comments

Automatic code formatting duplicated some comments near import statements.

Signed-off-by: Markus Möslinger <markus.moeslinger@socra.dev>

---------

Signed-off-by: Markus Möslinger <markus.moeslinger@socra.dev>

…-io#7220)

* fix lazy expanded postings cache and bug of non equal matcher with non existent values

Signed-off-by: Ben Ye <benye@amazon.com>

* test case for remove keys noop

Signed-off-by: Ben Ye <benye@amazon.com>

* add promqlsmith fuzz test

Signed-off-by: Ben Ye <benye@amazon.com>

* update

Signed-off-by: Ben Ye <benye@amazon.com>

* changelog

Signed-off-by: Ben Ye <benye@amazon.com>

* fix go mod

Signed-off-by: Ben Ye <benye@amazon.com>

* rename test

Signed-off-by: Ben Ye <benye@amazon.com>

* fix series request timestamp

Signed-off-by: Ben Ye <benye@amazon.com>

* skip e2e test

Signed-off-by: Ben Ye <benye@amazon.com>

* handle non lazy expanded case

Signed-off-by: Ben Ye <benye@amazon.com>

* update comment

Signed-off-by: Ben Ye <benye@amazon.com>

---------

Signed-off-by: Ben Ye <benye@amazon.com>

* bump Prometheus version to include new label matcher regex value optimization

Signed-off-by: Ben Ye <benye@amazon.com>

* update

Signed-off-by: Ben Ye <benye@amazon.com>

* fix again

Signed-off-by: Ben Ye <benye@amazon.com>

* include latest fix

Signed-off-by: Ben Ye <benye@amazon.com>

* update go mod

Signed-off-by: Ben Ye <benye@amazon.com>

* fix explain test

Signed-off-by: Ben Ye <benye@amazon.com>

* fix test again

Signed-off-by: Ben Ye <benye@amazon.com>

* update again

Signed-off-by: Ben Ye <benye@amazon.com>

* update

Signed-off-by: Ben Ye <benye@amazon.com>

* fix tests so far

Signed-off-by: Ben Ye <benye@amazon.com>

* fix compactor tests

Signed-off-by: Ben Ye <benye@amazon.com>

* use own out of order chunk index

Signed-off-by: Ben Ye <benye@amazon.com>

---------

Signed-off-by: Ben Ye <benye@amazon.com>

Signed-off-by: Alec Rajeev <13004609+alecrajeev@users.noreply.github.com>

Signed-off-by: Michael Hoffmann <mhoffm@posteo.de>

Signed-off-by: Ben Ye <benye@amazon.com>

Signed-off-by: Ben Ye <benye@amazon.com>

Signed-off-by: roth-wine <philipp.roth@hetzner.com>

…emlimit-reference

fix(changelog): fix GOMEMLIMIT pull request reference

* Added Shield in adopters

Signed-off-by: suhas.chikkanna.shield <suhas.chikkanna@shield.com>

* Upload compatible image 

Signed-off-by: suhas-chikkanna <162577490+suhas-chikkanna@users.noreply.github.com>

---------

Signed-off-by: suhas.chikkanna.shield <suhas.chikkanna@shield.com>
Signed-off-by: suhas-chikkanna <162577490+suhas-chikkanna@users.noreply.github.com>
Co-authored-by: suhas.chikkanna.shield <suhas.chikkanna@shield.com>

* added missing sampler types

Signed-off-by: Neeraj Nagure <nagureneeraj@gmail.com>

* added changelog entry

Signed-off-by: Neeraj Nagure <nagureneeraj@gmail.com>

* fixed changelog entry

Signed-off-by: Neeraj Nagure <nagureneeraj@gmail.com>

* Fixed changelog entry conflict

Signed-off-by: Neeraj Nagure <nagureneeraj@gmail.com>

---------

Signed-off-by: Neeraj Nagure <nagureneeraj@gmail.com>

Signed-off-by: Yi Jin <yi.jin@databricks.com>

Bump promql-engine version to include samples counting.

Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>

* added UNKNOWN to TargetHealth_value at targets proto

Signed-off-by: Tidhar Klein Orbach <tizkiko@gmail.com>

* added TargetHealth_value UNKNOWN to rpc.pb.go

Signed-off-by: Tidhar Klein Orbach <tizkiko@gmail.com>

---------

Signed-off-by: Tidhar Klein Orbach <tizkiko@gmail.com>

Signed-off-by: notafile <nota@notafile.com>

Signed-off-by: notafile <nota@notafile.com>

…tion is enabled (thanos-io#7225)

Signed-off-by: Ben Ye <benye@amazon.com>

* Serialize the plan for remote executions

latest engine

Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com>

Propagate marshaled plan and introduce optimizer

Propagating the query plan in the remote engine requests and introduce new SetProjectionColumns optimizer

Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com>

* Fixing passing down of plan

Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com>

* go mod tidy

Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com>

* avoid panics

Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com>

* delete dev file

Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com>

* undo small refactor

Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com>

* improve test

Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com>

---------

Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com>

generating protos

Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com>

fixing v1

Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com>

delete unused method

Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com>

Set projection labels after distributing queries

Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com>

Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com>

Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com>

Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com>

Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com>

Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com>

Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com>

Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com>

Show the new peak/total fields in analyze output next to each operator.
Add tooltips to explain what is the meaning of each field.

Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>

…tor-docs

Clarify documentation around selecor.relabel-config option

…ize-optimize

query: forward query plan in the remote query request

* receive/handler: implement tenant label splitting

Implement splitting incoming HTTP requests along some label inside of
the timeseries themselves. This functionality is useful when you have
one big application exposing lots of series and, for instance, you have
a label `team` that identifies different owners of metrics in that
application. Then using this you can use that `team` label to have
different tenants in Thanos.

Only negative thing that I could spot is that if after splitting one of
the requests fails then that code is used for all tenants and that skews
the Receiver metrics a little bit. I think that can be left as a TODO
task.

Signed-off-by: Giedrius Statkevičius <giedriuswork@gmail.com>

* test/e2e: add more receiver tests

Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>

* thanos/receive: note that splitting takes precendence over HTTP

Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>

* thanos/receive: fix typo

Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>

---------

Signed-off-by: Giedrius Statkevičius <giedriuswork@gmail.com>
Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>

…os-io#7271)

* query: fixing dedup iterator when working on mixed sample types

There was a panic in case the dedupiterator worked on two chunks with both Native Histograms and Float (XOR encoded).

Co-authored-by: Sebastian Rabenhorst <sebastian.rabenhorst@shopify.com>
Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com>

* Adding changelog

Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com>

* fixing lint

Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com>

* removing comments

Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com>

* Fixing repro test case

Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com>

* fixing initialization

Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com>

* fixing changelog

Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com>

* adding header to new file

Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com>

* using t.Run

Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com>

* fixing ordering of samples in tests

Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com>

---------

Signed-off-by: Pedro Tanaka <pedro.tanaka@shopify.com>
Co-authored-by: Sebastian Rabenhorst <sebastian.rabenhorst@shopify.com>

- as 'reflect.String.Header' is deprecated, it is replaced with an unsafe package.

Signed-off-by: Youngjun <yj.yoo@okestro.com>

* Receive: fix issue-7248 by introducing a worker pool

Signed-off-by: Yi Jin <yi.jin@databricks.com>

* fix unit test bug

Signed-off-by: Yi Jin <yi.jin@databricks.com>

* fix CLI flags not pass into the receive handler

Signed-off-by: Yi Jin <yi.jin@databricks.com>

* address comments

Signed-off-by: Yi Jin <yi.jin@databricks.com>

* init context in constructor

Signed-off-by: Yi Jin <yi.jin@databricks.com>

---------

Signed-off-by: Yi Jin <yi.jin@databricks.com>

Signed-off-by: Magiceses <magiceses0118@gmail.com>

Signed-off-by: Nicolas Takashi <nicolas.tcs@hotmail.com>

* receive/multitsdb: do not delete not uploaded blocks

If a block hasn't been uploaded yet then tell the TSDB layer not to
delete them. This prevents a nasty race where the TSDB layer can delete
a block before the shipper gets to it. I saw this happen with a very
small block.

Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>

* receive/multitsdb: change order

Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>

* shipper/receive: just use a single lock

Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>

---------

Signed-off-by: Giedrius Statkevičius <giedrius.statkevicius@vinted.com>

…o#7283)

Compaction dowsnample metrics have too high a cardinality, causing metric
bloat on large installations. The group information is better suited to logs.
* Replace with a resolution label to reduce cardinality.

Fixes: thanos-io#5841

Signed-off-by: SuperQ <superq@gmail.com>

Warnings from remote instant queries get turned into errors, which
is a bug. It should be up to the root client to decide whether warnings
should be show as such, or converted to errors.

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>

Propagate warnings from instant queries

…/skipVerify

Signed-off-by: Guillaume Lecerf <guillaume.lecerf@iziwork.com>

…tls-external-termination

Receive: stop relying on grpc server config to set grpc client secure/skipVerify

* Show warnings in query frontend

QFE currently does not parse warnings from downstream queriers.
This commit fixes that by adding the field to proto messages and
modifies the merge function to take warnings into account.

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>

* Add CHANGELOG entry

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>

* Omit empty warnings

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>

---------

Signed-off-by: Filip Petkovski <filip.petkovsky@gmail.com>