Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Mimir query engine: expand TestConcurrentQueries to include testing queries with float samples #9194

Merged
merged 2 commits into from
Sep 4, 2024
Merged

Mimir query engine: expand TestConcurrentQueries to include testing queries with float samples #9194

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
1f8db8a
Expand TestConcurrentQueries to include testing queries with float sa…
charleskorn Sep 4, 2024
dfc546c
Add changelog entry
charleskorn Sep 4, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@
* [CHANGE] Ruler: Removed `-ruler.drain-notification-queue-on-shutdown` option, which is now enabled by default. #9115
* [CHANGE] Querier: allow wrapping errors with context errors only when the former actually correspond to `context.Canceled` and `context.DeadlineExceeded`. #9175
* [FEATURE] Alertmanager: Added `-alertmanager.log-parsing-label-matchers` to control logging when parsing label matchers. This flag is intended to be used with `-alertmanager.utf8-strict-mode-enabled` to validate UTF-8 strict mode is working as intended. The default value is `false`. #9173
* [FEATURE] Querier: add experimental streaming PromQL engine, enabled with `-querier.query-engine=mimir`. #8422 #8430 #8454 #8455 #8360 #8490 #8508 #8577 #8660 #8671 #8677 #8747 #8850 #8872 #8838 #8911 #8909 #8923 #8924 #8925 #8932 #8933 #8934 #8962 #8986 #8993 #8995 #9008 #9017 #9018 #9019 #9120 #9121 #9136 #9139 #9145 #9191
* [FEATURE] Querier: add experimental streaming PromQL engine, enabled with `-querier.query-engine=mimir`. #8422 #8430 #8454 #8455 #8360 #8490 #8508 #8577 #8660 #8671 #8677 #8747 #8850 #8872 #8838 #8911 #8909 #8923 #8924 #8925 #8932 #8933 #8934 #8962 #8986 #8993 #8995 #9008 #9017 #9018 #9019 #9120 #9121 #9136 #9139 #9145 #9191 #9194
aknuds1 marked this conversation as resolved.
Show resolved Hide resolved
* [FEATURE] Experimental Kafka-based ingest storage. #6888 #6894 #6929 #6940 #6951 #6974 #6982 #7029 #7030 #7091 #7142 #7147 #7148 #7153 #7160 #7193 #7349 #7376 #7388 #7391 #7393 #7394 #7402 #7404 #7423 #7424 #7437 #7486 #7503 #7508 #7540 #7621 #7682 #7685 #7694 #7695 #7696 #7697 #7701 #7733 #7734 #7741 #7752 #7838 #7851 #7871 #7877 #7880 #7882 #7887 #7891 #7925 #7955 #7967 #8031 #8063 #8077 #8088 #8135 #8176 #8184 #8194 #8216 #8217 #8222 #8233 #8503 #8542 #8579 #8657 #8686 #8688 #8703 #8706 #8708 #8738 #8750 #8778 #8808 #8809 #8841 #8842 #8845 #8853 #8886 #8988
* What it is:
* When the new ingest storage architecture is enabled, distributors write incoming write requests to a Kafka-compatible backend, and the ingesters asynchronously replay ingested data from Kafka. In this architecture, the write and read path are de-coupled through a Kafka-compatible backend. The write path and Kafka load is a function of the incoming write traffic, the read path load is a function of received queries. Whatever the load on the read path, it doesn't affect the write path.
Expand Down
54 changes: 54 additions & 0 deletions pkg/streamingpromql/engine_concurrency_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,10 @@ func TestConcurrentQueries(t *testing.T) {
native_histogram{group="a", instance="2"} {{schema:5 sum:8 count:7 buckets:[1 5 1]}}+{{schema:5 sum:10 count:7 buckets:[1 2 3 1]}}x9
native_histogram{group="b", instance="1"} {{schema:5 sum:10 count:7 buckets:[1 2 3 1]}}+{{schema:5 sum:8 count:7 buckets:[1 5 1]}}x9
native_histogram{group="b", instance="2"} {{schema:3 sum:4 count:4 buckets:[4]}}
float{group="a", instance="1"} 1+2x9
float{group="a", instance="2"} 3+2.5x9
float{group="b", instance="1"} 1 2 4 2 7 1 2 3 4
float{group="b", instance="2"} 1 9 2 8 11 12 13 14 15
`

startT := timestamp.Time(0)
Expand All @@ -61,6 +65,12 @@ func TestConcurrentQueries(t *testing.T) {
end: startT.Add(10 * time.Minute),
step: time.Minute,
},
{
expr: `sum(rate(native_histogram[5m]))`,
start: startT,
end: startT.Add(10 * time.Minute),
step: time.Minute,
},
{
expr: `count_over_time(native_histogram[5m])`,
start: startT,
Expand All @@ -79,6 +89,50 @@ func TestConcurrentQueries(t *testing.T) {
end: startT.Add(10 * time.Minute),
step: time.Minute,
},
{
expr: `rate(float[5m])`,
start: startT,
end: startT.Add(10 * time.Minute),
step: time.Minute,
},
{
// Sum of floats into single group
expr: `sum(float)`,
start: startT,
end: startT.Add(10 * time.Minute),
step: time.Minute,
},
{
// Sum of floats into many groups
expr: `sum by (group) (float)`,
start: startT,
end: startT.Add(10 * time.Minute),
step: time.Minute,
},
{
expr: `sum(rate(float[5m]))`,
start: startT,
end: startT.Add(10 * time.Minute),
step: time.Minute,
},
{
expr: `count_over_time(float[5m])`,
start: startT,
end: startT.Add(10 * time.Minute),
step: time.Minute,
},
{
expr: `sum_over_time(float[5m])`,
start: startT,
end: startT.Add(10 * time.Minute),
step: time.Minute,
},
{
expr: `avg_over_time(float[5m])`,
start: startT,
end: startT.Add(10 * time.Minute),
step: time.Minute,
},
}

storage := promqltest.LoadedStorage(t, data)
Expand Down
Loading