From 9a69e9072fd7c317ace4353d06fbc6ddb496223b Mon Sep 17 00:00:00 2001 From: Dimitar Dimitrov Date: Thu, 20 Jun 2024 15:57:38 +0200 Subject: [PATCH 1/9] 2.13 release notes This is a draft PR with release notes. I've based this hugely on the changelog. The wording in the release notes is also largely influenced by the changelog as well. There are a lot of enhancements and bug fixes in 2.13, so it was difficult to include everything. I tried to focus on user-visible changes, changes that are likely to impact more Mimir installations, or changes related to new features. I'm open to feedback for mentioning other features or removing some of the ones mentioned. related to XXX Signed-off-by: Dimitar Dimitrov --- docs/sources/mimir/release-notes/v2.13.md | 90 +++++++++++++++++++++++ 1 file changed, 90 insertions(+) create mode 100644 docs/sources/mimir/release-notes/v2.13.md diff --git a/docs/sources/mimir/release-notes/v2.13.md b/docs/sources/mimir/release-notes/v2.13.md new file mode 100644 index 00000000000..3eb931fbdad --- /dev/null +++ b/docs/sources/mimir/release-notes/v2.13.md @@ -0,0 +1,90 @@ +--- +title: "Grafana Mimir version 2.13 release notes" +menuTitle: "V2.13 release notes" +description: "Release notes for Grafana Mimir version 2.13" +weight: 1100 +--- + +# Grafana Mimir version 2.13 release notes + + + + + +Grafana Labs is excited to announce version 2.13 of Grafana Mimir. + +The highlights that follow include the top features, enhancements, and bug fixes in this release. +For the complete list of changes, refer to the [CHANGELOG](https://github.com/grafana/mimir/blob/main/CHANGELOG.md). + +## Features and enhancements + +- **Store-gateway can be configured to explicitly disable or enable tenants**. This is useful in cases when compaction slows down for a tenant to temporarily exclude a tenant while compaction catches up without impact other tenants. + +- Configurable S3 bucket lookup type improves interoperability with **S3-compatible providers such as **Tencent and Alibaba\*\*\*\*. Configure lookup type via `-.s3.bucket-lookup-type` or `-common.storage.s3-bucket-lookup-type`. + +- **Mimirtool can now verify the validity of a Mimir runtime configuration** file with the `mimirtool runtime-config verify` command. + +- **Configuration of TLS for S3 buckets** is now possible via `-.s3.http.*` and `-common.storage.s3.http.*` configuration options. + +- **Improved OTLP ingestion performance** by using native translation, which reduces memory usage in the distributor by up to 30% and CPU usage by up to 8%. + +- **Active series are now updated along with owned series.** This means the number of active series for a tenant is much more accurate after adding ingesters. + +- **Remote read (`/prometheus/api/v1/read`) is becoming a first-class endpoint in Mimir** with support in `query stats` logs and query blocking in the query-frontend. More coming in future releases. + +- **Improved CPU performance in processing queries with regular expressions** that contain many alternations (for example `foo|bar|baz|...`). + +Additionally, the following previously experimental features are now considered stable: + +- Rules tenant federation via the `source_tenants` field in rule groups. +- Enabling and recording and alerting rules evaluation on a per-tenant basis. +- Limiting the number of tenants a federated query can query. + +## Important changes + +In Grafana Mimir 2.13 the following behavior has changed: + +- The default Docker image `grafana/mimir` is now based on the distroless image `gcr.io/distroless/static-debian12`. + See [Debugging distroless container images](https://grafana.com/docs/mimir/next/manage/mimir-runbooks/#debugging-distroless-container-images-in-kubernetes) for more details on how to + work with distroless images. + +- Errors logs in the ingester are now sampled at 10% by default. Sampled log lines will contain `(sampled 1/10)`. All the discarded samples will still be tracked by the `cortex_discarded_samples_total` metric. + +- Anonymous usage statistics now include actual CPU usage instead of available CPU cores. + +- Continuous-test is no longer a standalone binary and is now part of Mimir as its own target. The published Docker images has been updated to use the new packaging. + +The following deprecated configuration options are removed in Grafana Mimir 2.13: + +- The configuration option `-log.buffered`, which was deprecated in 2.11 and is now enabled by default. + +## Experimental features + +Grafana Mimir 2.13 includes new features that are considered experimental and disabled by default. +Use them with caution and report any issues you encounter: + +- **New PromQL query engine** can be configured via the `-querier.promql-engine=mimir` CLI flag on queriers. + It aims to provide better utilization than the upstream Prometheus PromQL engine when processing large sets of data. + +- Support for [**HAProxy PROXY protocol**](https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt) can now be enabled via `-server.proxy-protocol-enabled=true`. + +- Experimental support for server-side circuit breakers in ingesters on read and write requests. This can be enabled using `-ingester.push-circuit-breaker.enabled` and `-ingester.read-circuit-breaker.enabled` options and further configured via the `-ingester.push-circuit-breaker.*` and `-ingester.read-circuit-breaker.*` options. + +The following configuration options are deprecated and will be removed in a future Grafana Mimir release: + +- `evaluation_delay` field in the rule group configuration has been deprecated. Please use `query_offset` instead. + +## Bug fixes + +- OTLP ingestion: translate all HTTP 5xx errors into one of 502, 503, or 504, so that the otel-collector retries the failed requests. +- OTLP ingestion: return properly formatted protobuf error messages when ingestion fails. +- OTLP ingestion: generate `target_info` metric only when there are metrics in the request and there is at least one configured identifying label. +- OTLP ingestion: don't discard timeseries paired with invalid exemplars. Instead, try to ingest the timeseries and discard only the invalid exemplars. +- Subqueries: `@ end()` and `@ start()` now work correctly with queries split by time. +- Native histograms: order exemplars before ingestion to improve success rate when ingesting multiple exemplars. +- Native histograms: return HTTP 400 on invalid native histogram samples instead of HTTP 500. The metric `cortex_discarded_samples_total{reason="invalid-native-histogram"}` is now incremented on invalid histogram samples. + +### Helm chart improvements + +The Grafana Mimir and Grafana Enterprise Metrics Helm charts are released independently. +Refer to the [Grafana Mimir Helm chart documentation](/docs/helm-charts/mimir-distributed/latest/). From dca3c7c99aa66957650001bba57741df13af8980 Mon Sep 17 00:00:00 2001 From: Dimitar Dimitrov Date: Fri, 21 Jun 2024 11:07:31 +0200 Subject: [PATCH 2/9] Update docs/sources/mimir/release-notes/v2.13.md --- docs/sources/mimir/release-notes/v2.13.md | 1 - 1 file changed, 1 deletion(-) diff --git a/docs/sources/mimir/release-notes/v2.13.md b/docs/sources/mimir/release-notes/v2.13.md index 3eb931fbdad..0b82806d8d8 100644 --- a/docs/sources/mimir/release-notes/v2.13.md +++ b/docs/sources/mimir/release-notes/v2.13.md @@ -66,7 +66,6 @@ Use them with caution and report any issues you encounter: - **New PromQL query engine** can be configured via the `-querier.promql-engine=mimir` CLI flag on queriers. It aims to provide better utilization than the upstream Prometheus PromQL engine when processing large sets of data. -- Support for [**HAProxy PROXY protocol**](https://www.haproxy.org/download/1.8/doc/proxy-protocol.txt) can now be enabled via `-server.proxy-protocol-enabled=true`. - Experimental support for server-side circuit breakers in ingesters on read and write requests. This can be enabled using `-ingester.push-circuit-breaker.enabled` and `-ingester.read-circuit-breaker.enabled` options and further configured via the `-ingester.push-circuit-breaker.*` and `-ingester.read-circuit-breaker.*` options. From 2c4a09eb734f08572fb920f0d5c7452395531992 Mon Sep 17 00:00:00 2001 From: Dimitar Dimitrov Date: Fri, 21 Jun 2024 11:09:22 +0200 Subject: [PATCH 3/9] Update docs/sources/mimir/release-notes/v2.13.md Co-authored-by: Charles Korn --- docs/sources/mimir/release-notes/v2.13.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/sources/mimir/release-notes/v2.13.md b/docs/sources/mimir/release-notes/v2.13.md index 0b82806d8d8..d10367e8960 100644 --- a/docs/sources/mimir/release-notes/v2.13.md +++ b/docs/sources/mimir/release-notes/v2.13.md @@ -20,7 +20,7 @@ For the complete list of changes, refer to the [CHANGELOG](https://github.com/gr - **Store-gateway can be configured to explicitly disable or enable tenants**. This is useful in cases when compaction slows down for a tenant to temporarily exclude a tenant while compaction catches up without impact other tenants. -- Configurable S3 bucket lookup type improves interoperability with **S3-compatible providers such as **Tencent and Alibaba\*\*\*\*. Configure lookup type via `-.s3.bucket-lookup-type` or `-common.storage.s3-bucket-lookup-type`. +- Configurable S3 bucket lookup type improves interoperability with **S3-compatible providers such as Tencent and Alibaba**. Configure lookup type via `-.s3.bucket-lookup-type` or `-common.storage.s3-bucket-lookup-type`. - **Mimirtool can now verify the validity of a Mimir runtime configuration** file with the `mimirtool runtime-config verify` command. From ba4f536d77af5acfbbce62554ff86be119d7dd67 Mon Sep 17 00:00:00 2001 From: Dimitar Dimitrov Date: Fri, 21 Jun 2024 11:12:37 +0200 Subject: [PATCH 4/9] Update docs/sources/mimir/release-notes/v2.13.md Co-authored-by: Charles Korn --- docs/sources/mimir/release-notes/v2.13.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/sources/mimir/release-notes/v2.13.md b/docs/sources/mimir/release-notes/v2.13.md index d10367e8960..631f9560a9a 100644 --- a/docs/sources/mimir/release-notes/v2.13.md +++ b/docs/sources/mimir/release-notes/v2.13.md @@ -48,7 +48,7 @@ In Grafana Mimir 2.13 the following behavior has changed: See [Debugging distroless container images](https://grafana.com/docs/mimir/next/manage/mimir-runbooks/#debugging-distroless-container-images-in-kubernetes) for more details on how to work with distroless images. -- Errors logs in the ingester are now sampled at 10% by default. Sampled log lines will contain `(sampled 1/10)`. All the discarded samples will still be tracked by the `cortex_discarded_samples_total` metric. +- Error logs in the ingester are now sampled at 10% by default. Sampled log lines will contain `(sampled 1/10)`. All the discarded samples will still be tracked by the `cortex_discarded_samples_total` metric. - Anonymous usage statistics now include actual CPU usage instead of available CPU cores. From 7a5dd01204dd59d3a55a85274c7af10ce3fe1748 Mon Sep 17 00:00:00 2001 From: Dimitar Dimitrov Date: Fri, 21 Jun 2024 11:13:50 +0200 Subject: [PATCH 5/9] Update docs/sources/mimir/release-notes/v2.13.md Co-authored-by: Charles Korn --- docs/sources/mimir/release-notes/v2.13.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/sources/mimir/release-notes/v2.13.md b/docs/sources/mimir/release-notes/v2.13.md index 631f9560a9a..e15bb4952c1 100644 --- a/docs/sources/mimir/release-notes/v2.13.md +++ b/docs/sources/mimir/release-notes/v2.13.md @@ -52,7 +52,7 @@ In Grafana Mimir 2.13 the following behavior has changed: - Anonymous usage statistics now include actual CPU usage instead of available CPU cores. -- Continuous-test is no longer a standalone binary and is now part of Mimir as its own target. The published Docker images has been updated to use the new packaging. +- Continuous-test is no longer a standalone binary and is now part of Mimir as its own target. The published Docker image has been updated to use the new packaging. The following deprecated configuration options are removed in Grafana Mimir 2.13: From 8cce3f024c07b4c74214c41ab6566349905a43e7 Mon Sep 17 00:00:00 2001 From: Dimitar Dimitrov Date: Fri, 21 Jun 2024 11:13:55 +0200 Subject: [PATCH 6/9] Update docs/sources/mimir/release-notes/v2.13.md --- docs/sources/mimir/release-notes/v2.13.md | 2 -- 1 file changed, 2 deletions(-) diff --git a/docs/sources/mimir/release-notes/v2.13.md b/docs/sources/mimir/release-notes/v2.13.md index e15bb4952c1..b9102f888f9 100644 --- a/docs/sources/mimir/release-notes/v2.13.md +++ b/docs/sources/mimir/release-notes/v2.13.md @@ -63,8 +63,6 @@ The following deprecated configuration options are removed in Grafana Mimir 2.13 Grafana Mimir 2.13 includes new features that are considered experimental and disabled by default. Use them with caution and report any issues you encounter: -- **New PromQL query engine** can be configured via the `-querier.promql-engine=mimir` CLI flag on queriers. - It aims to provide better utilization than the upstream Prometheus PromQL engine when processing large sets of data. - Experimental support for server-side circuit breakers in ingesters on read and write requests. This can be enabled using `-ingester.push-circuit-breaker.enabled` and `-ingester.read-circuit-breaker.enabled` options and further configured via the `-ingester.push-circuit-breaker.*` and `-ingester.read-circuit-breaker.*` options. From c29fbf4fdf861a1b440f4d8edd54a59861afd260 Mon Sep 17 00:00:00 2001 From: Dimitar Dimitrov Date: Fri, 21 Jun 2024 11:19:36 +0200 Subject: [PATCH 7/9] Update wording on owned series Signed-off-by: Dimitar Dimitrov --- docs/sources/mimir/release-notes/v2.13.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/sources/mimir/release-notes/v2.13.md b/docs/sources/mimir/release-notes/v2.13.md index b9102f888f9..1b25590fa48 100644 --- a/docs/sources/mimir/release-notes/v2.13.md +++ b/docs/sources/mimir/release-notes/v2.13.md @@ -28,7 +28,7 @@ For the complete list of changes, refer to the [CHANGELOG](https://github.com/gr - **Improved OTLP ingestion performance** by using native translation, which reduces memory usage in the distributor by up to 30% and CPU usage by up to 8%. -- **Active series are now updated along with owned series.** This means the number of active series for a tenant is much more accurate after adding ingesters. +- **Active series are now updated along with owned series.** This means the number of active series for a tenant is much more accurate after scaling out ingesters. As a result ingesters now enforce more precisely tenants' series limits. This is only enabled when `-ingester.track-ingester-owned-series` or `-ingester.use-ingester-owned-series-for-limits` are enabled. - **Remote read (`/prometheus/api/v1/read`) is becoming a first-class endpoint in Mimir** with support in `query stats` logs and query blocking in the query-frontend. More coming in future releases. From 2db2de707b8b4380c876a361dce823b6f58cb59c Mon Sep 17 00:00:00 2001 From: Dimitar Dimitrov Date: Fri, 21 Jun 2024 11:20:51 +0200 Subject: [PATCH 8/9] Order feature highlights and remove space Signed-off-by: Dimitar Dimitrov --- docs/sources/mimir/release-notes/v2.13.md | 14 ++++++-------- 1 file changed, 6 insertions(+), 8 deletions(-) diff --git a/docs/sources/mimir/release-notes/v2.13.md b/docs/sources/mimir/release-notes/v2.13.md index 1b25590fa48..318c8afb731 100644 --- a/docs/sources/mimir/release-notes/v2.13.md +++ b/docs/sources/mimir/release-notes/v2.13.md @@ -18,21 +18,21 @@ For the complete list of changes, refer to the [CHANGELOG](https://github.com/gr ## Features and enhancements -- **Store-gateway can be configured to explicitly disable or enable tenants**. This is useful in cases when compaction slows down for a tenant to temporarily exclude a tenant while compaction catches up without impact other tenants. +- **Improved CPU performance in processing queries with regular expressions** that contain many alternations (for example `foo|bar|baz|...`). -- Configurable S3 bucket lookup type improves interoperability with **S3-compatible providers such as Tencent and Alibaba**. Configure lookup type via `-.s3.bucket-lookup-type` or `-common.storage.s3-bucket-lookup-type`. +- **Improved OTLP ingestion performance** by using native translation, which reduces memory usage in the distributor by up to 30% and CPU usage by up to 8%. -- **Mimirtool can now verify the validity of a Mimir runtime configuration** file with the `mimirtool runtime-config verify` command. +- Configurable S3 bucket lookup type improves interoperability with **S3-compatible providers such as Tencent and Alibaba**. Configure lookup type via `-.s3.bucket-lookup-type` or `-common.storage.s3-bucket-lookup-type`. - **Configuration of TLS for S3 buckets** is now possible via `-.s3.http.*` and `-common.storage.s3.http.*` configuration options. -- **Improved OTLP ingestion performance** by using native translation, which reduces memory usage in the distributor by up to 30% and CPU usage by up to 8%. +- **Mimirtool can now verify the validity of a Mimir runtime configuration** file with the `mimirtool runtime-config verify` command. - **Active series are now updated along with owned series.** This means the number of active series for a tenant is much more accurate after scaling out ingesters. As a result ingesters now enforce more precisely tenants' series limits. This is only enabled when `-ingester.track-ingester-owned-series` or `-ingester.use-ingester-owned-series-for-limits` are enabled. -- **Remote read (`/prometheus/api/v1/read`) is becoming a first-class endpoint in Mimir** with support in `query stats` logs and query blocking in the query-frontend. More coming in future releases. +- **Store-gateway can be configured to explicitly disable or enable tenants**. This is useful in cases when compaction slows down for a tenant to temporarily exclude a tenant while compaction catches up without impact other tenants. -- **Improved CPU performance in processing queries with regular expressions** that contain many alternations (for example `foo|bar|baz|...`). +- **Remote read (`/prometheus/api/v1/read`) is becoming a first-class endpoint in Mimir** with support in `query stats` logs and query blocking in the query-frontend. More coming in future releases. Additionally, the following previously experimental features are now considered stable: @@ -63,8 +63,6 @@ The following deprecated configuration options are removed in Grafana Mimir 2.13 Grafana Mimir 2.13 includes new features that are considered experimental and disabled by default. Use them with caution and report any issues you encounter: - - - Experimental support for server-side circuit breakers in ingesters on read and write requests. This can be enabled using `-ingester.push-circuit-breaker.enabled` and `-ingester.read-circuit-breaker.enabled` options and further configured via the `-ingester.push-circuit-breaker.*` and `-ingester.read-circuit-breaker.*` options. The following configuration options are deprecated and will be removed in a future Grafana Mimir release: From 56209c0fc779374da101de89dc838bcfb07707d6 Mon Sep 17 00:00:00 2001 From: Dimitar Dimitrov Date: Fri, 21 Jun 2024 18:43:25 +0200 Subject: [PATCH 9/9] Apply styling suggestions Co-authored-by: taylor c <41653732+tacole02@users.noreply.github.com> --- docs/sources/mimir/release-notes/v2.13.md | 12 ++++++------ 1 file changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/sources/mimir/release-notes/v2.13.md b/docs/sources/mimir/release-notes/v2.13.md index 318c8afb731..b266b5f4833 100644 --- a/docs/sources/mimir/release-notes/v2.13.md +++ b/docs/sources/mimir/release-notes/v2.13.md @@ -22,22 +22,22 @@ For the complete list of changes, refer to the [CHANGELOG](https://github.com/gr - **Improved OTLP ingestion performance** by using native translation, which reduces memory usage in the distributor by up to 30% and CPU usage by up to 8%. -- Configurable S3 bucket lookup type improves interoperability with **S3-compatible providers such as Tencent and Alibaba**. Configure lookup type via `-.s3.bucket-lookup-type` or `-common.storage.s3-bucket-lookup-type`. +- Configurable S3 bucket lookup type improves interoperability with **S3-compatible providers, such as Tencent and Alibaba**. Configure lookup type via `-.s3.bucket-lookup-type` or `-common.storage.s3-bucket-lookup-type`. - **Configuration of TLS for S3 buckets** is now possible via `-.s3.http.*` and `-common.storage.s3.http.*` configuration options. - **Mimirtool can now verify the validity of a Mimir runtime configuration** file with the `mimirtool runtime-config verify` command. -- **Active series are now updated along with owned series.** This means the number of active series for a tenant is much more accurate after scaling out ingesters. As a result ingesters now enforce more precisely tenants' series limits. This is only enabled when `-ingester.track-ingester-owned-series` or `-ingester.use-ingester-owned-series-for-limits` are enabled. +- **Active series are now updated along with owned series.** This means the number of active series for a tenant is more accurate after scaling out ingesters. As a result, ingesters now more precisely enforce tenants' series limits. This feature is only enabled when `-ingester.track-ingester-owned-series` or `-ingester.use-ingester-owned-series-for-limits` are enabled. -- **Store-gateway can be configured to explicitly disable or enable tenants**. This is useful in cases when compaction slows down for a tenant to temporarily exclude a tenant while compaction catches up without impact other tenants. +- **Store-gateway can be configured to explicitly disable or enable tenants**. This is useful in cases where compaction slows down for a tenant to temporarily exclude another tenant while compaction catches up without affecting other tenants. - **Remote read (`/prometheus/api/v1/read`) is becoming a first-class endpoint in Mimir** with support in `query stats` logs and query blocking in the query-frontend. More coming in future releases. Additionally, the following previously experimental features are now considered stable: - Rules tenant federation via the `source_tenants` field in rule groups. -- Enabling and recording and alerting rules evaluation on a per-tenant basis. +- Enabling recording, and alerting rules evaluation on a per-tenant basis. - Limiting the number of tenants a federated query can query. ## Important changes @@ -45,10 +45,10 @@ Additionally, the following previously experimental features are now considered In Grafana Mimir 2.13 the following behavior has changed: - The default Docker image `grafana/mimir` is now based on the distroless image `gcr.io/distroless/static-debian12`. - See [Debugging distroless container images](https://grafana.com/docs/mimir/next/manage/mimir-runbooks/#debugging-distroless-container-images-in-kubernetes) for more details on how to + See [Debugging distroless container images](https://grafana.com/docs/mimir/latest/manage/mimir-runbooks/#debugging-distroless-container-images-in-kubernetes) for more details on how to work with distroless images. -- Error logs in the ingester are now sampled at 10% by default. Sampled log lines will contain `(sampled 1/10)`. All the discarded samples will still be tracked by the `cortex_discarded_samples_total` metric. +- Error logs in the ingester are now sampled at 10% by default. Sampled log lines contain `(sampled 1/10)`. The `cortex_discarded_samples_total` metric still tracks all discarded samples. - Anonymous usage statistics now include actual CPU usage instead of available CPU cores.