Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

First iteration of streaming PromQL engine #7693

Merged
merged 163 commits into from
Apr 9, 2024
Merged

First iteration of streaming PromQL engine #7693

merged 163 commits into from
Apr 9, 2024

Commits on Aug 10, 2023

  1. Use forked Prometheus module

    @charleskorn
    charleskorn committed Aug 10, 2023
    Configuration menu
    Copy the full SHA
    70418a3 View commit details
    Browse the repository at this point in the history

  2. Use promql.QueryEngine rather than *promql.Engine, add config option … 

    …to select engine
    @charleskorn
    charleskorn committed Aug 10, 2023
    Configuration menu
    Copy the full SHA
    09c9f1e View commit details
    Browse the repository at this point in the history

  3. Add parallel query path with streaming engine.

    @charleskorn
    charleskorn committed Aug 10, 2023
    Configuration menu
    Copy the full SHA
    e76edce View commit details
    Browse the repository at this point in the history

  4. Add IntelliJ debugging configuration for streaming querier.

    @charleskorn
    charleskorn committed Aug 10, 2023
    Configuration menu
    Copy the full SHA
    290cf6b View commit details
    Browse the repository at this point in the history

  5. Add Grafana datasource for streaming querier

    @charleskorn
    charleskorn committed Aug 10, 2023
    Configuration menu
    Copy the full SHA
    204ae00 View commit details
    Browse the repository at this point in the history

  6. Use correct query-scheduler port and enable debugging

    @charleskorn
    charleskorn committed Aug 10, 2023
    Configuration menu
    Copy the full SHA
    a3a8b89 View commit details
    Browse the repository at this point in the history

  7. Set up load test.

    @charleskorn
    charleskorn committed Aug 10, 2023
    Configuration menu
    Copy the full SHA
    86a7b11 View commit details
    Browse the repository at this point in the history

  8. Fix p99 latency panel

    @charleskorn
    charleskorn committed Aug 10, 2023
    Configuration menu
    Copy the full SHA
    9b1ebdc View commit details
    Browse the repository at this point in the history

  9. Commit settings used in previous test.

    @charleskorn
    charleskorn committed Aug 10, 2023
    Configuration menu
    Copy the full SHA
    0e08552 View commit details
    Browse the repository at this point in the history

Commits on Aug 11, 2023

  1. Enable chunks streaming.

    @charleskorn
    charleskorn committed Aug 11, 2023
    Configuration menu
    Copy the full SHA
    676b2ec View commit details
    Browse the repository at this point in the history

Commits on Sep 7, 2023

  1. Disable load generator.

    @charleskorn
    charleskorn committed Sep 7, 2023
    Configuration menu
    Copy the full SHA
    9005d2f View commit details
    Browse the repository at this point in the history

  2. Create separate load testing query paths to ensure meta-monitoring qu… 

    …eries don't interfere with load test results.
    @charleskorn
    charleskorn committed Sep 7, 2023
    Configuration menu
    Copy the full SHA
    546762e View commit details
    Browse the repository at this point in the history

  3. Add initial k6-based load test script

    @charleskorn
    charleskorn committed Sep 7, 2023
    Configuration menu
    Copy the full SHA
    8e0fa53 View commit details
    Browse the repository at this point in the history

  4. Add script to run load tests against both engines.

    @charleskorn
    charleskorn committed Sep 7, 2023
    Configuration menu
    Copy the full SHA
    acff7ff View commit details
    Browse the repository at this point in the history

  5. Add todo

    @charleskorn
    charleskorn committed Sep 7, 2023
    Configuration menu
    Copy the full SHA
    fe93410 View commit details
    Browse the repository at this point in the history

Commits on Sep 8, 2023

  1. Extract friendly name to hostname logic to separate script

    @charleskorn
    charleskorn committed Sep 8, 2023
    Configuration menu
    Copy the full SHA
    f0d562e View commit details
    Browse the repository at this point in the history

  2. Rename k6 output file

    @charleskorn
    charleskorn committed Sep 8, 2023
    Configuration menu
    Copy the full SHA
    046537c View commit details
    Browse the repository at this point in the history

  3. Check both engines produce the same result.

    @charleskorn
    charleskorn committed Sep 8, 2023
    Configuration menu
    Copy the full SHA
    55d1049 View commit details
    Browse the repository at this point in the history

  4. Summarise results after each test run.

    @charleskorn
    charleskorn committed Sep 8, 2023
    Configuration menu
    Copy the full SHA
    94b54ed View commit details
    Browse the repository at this point in the history

  5. Enable streaming chunks from store-gateways to queriers.

    @charleskorn
    charleskorn committed Sep 8, 2023
    Configuration menu
    Copy the full SHA
    7c8c136 View commit details
    Browse the repository at this point in the history

  6. Refactor test script, print max and average CPU and memory after test

    @charleskorn
    charleskorn committed Sep 8, 2023
    Configuration menu
    Copy the full SHA
    909334d View commit details
    Browse the repository at this point in the history

  7. Reorder variables to match output order

    @charleskorn
    charleskorn committed Sep 8, 2023
    Configuration menu
    Copy the full SHA
    49b17c2 View commit details
    Browse the repository at this point in the history

  8. Don't include query name in k6 output file name.

    @charleskorn
    charleskorn committed Sep 8, 2023
    Configuration menu
    Copy the full SHA
    0ee3b7b View commit details
    Browse the repository at this point in the history

  9. Add initial script to generate data.

    @charleskorn
    charleskorn committed Sep 8, 2023
    Configuration menu
    Copy the full SHA
    f7aa5b3 View commit details
    Browse the repository at this point in the history

  10. Rename script

    @charleskorn
    charleskorn committed Sep 8, 2023
    Configuration menu
    Copy the full SHA
    948ae97 View commit details
    Browse the repository at this point in the history

  11. Add script to generate data generation config.

    @charleskorn
    charleskorn committed Sep 8, 2023
    Configuration menu
    Copy the full SHA
    80852f5 View commit details
    Browse the repository at this point in the history

  12. Make it easier to test different queries.

    @charleskorn
    charleskorn committed Sep 8, 2023
    Configuration menu
    Copy the full SHA
    cee0e1b View commit details
    Browse the repository at this point in the history

Commits on Sep 11, 2023

  1. Rename PromQL engine CLI flag.

    @charleskorn
    charleskorn committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    231be86 View commit details
    Browse the repository at this point in the history

  2. Add Thanos' engine.

    @charleskorn
    charleskorn committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    6a66d0f View commit details
    Browse the repository at this point in the history

  3. Add Thanos engine to benchmarking setup.

    @charleskorn
    charleskorn committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    53f789d View commit details
    Browse the repository at this point in the history

  4. Set default test duration to 5 minutes.

    @charleskorn
    charleskorn committed Sep 11, 2023
    Configuration menu
    Copy the full SHA
    b6462d1 View commit details
    Browse the repository at this point in the history

Commits on Sep 12, 2023

  1. Rework data generation setup to work around limitation in prometheus-… 

    …toolbox.
    @charleskorn
    charleskorn committed Sep 12, 2023
    Configuration menu
    Copy the full SHA
    bd06d1e View commit details
    Browse the repository at this point in the history

  2. Load test data faster.

    @charleskorn
    charleskorn committed Sep 12, 2023
    Configuration menu
    Copy the full SHA
    16dcb15 View commit details
    Browse the repository at this point in the history

  3. Record test config for first test

    @charleskorn
    charleskorn committed Sep 12, 2023
    Configuration menu
    Copy the full SHA
    b0f8f61 View commit details
    Browse the repository at this point in the history

  4. Record test config for second test

    @charleskorn
    charleskorn committed Sep 12, 2023
    Configuration menu
    Copy the full SHA
    b103d6e View commit details
    Browse the repository at this point in the history

  5. Record test config for third test

    @charleskorn
    charleskorn committed Sep 12, 2023
    Configuration menu
    Copy the full SHA
    0be5f14 View commit details
    Browse the repository at this point in the history

  6. Disable gRPC compression for ingester client

    @charleskorn
    charleskorn committed Sep 12, 2023
    Configuration menu
    Copy the full SHA
    173a285 View commit details
    Browse the repository at this point in the history

Commits on Sep 13, 2023

  1. Record test config for re-run of second test

    @charleskorn
    charleskorn committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    2f43ca6 View commit details
    Browse the repository at this point in the history

  2. Record test config for re-run of first test

    @charleskorn
    charleskorn committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    563ced1 View commit details
    Browse the repository at this point in the history

  3. Record test config for re-run of third test

    @charleskorn
    charleskorn committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    b2016d3 View commit details
    Browse the repository at this point in the history

  4. Log URL used for test.

    @charleskorn
    charleskorn committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    ee065c2 View commit details
    Browse the repository at this point in the history

  5. Record test config for fourth test

    @charleskorn
    charleskorn committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    9ec79a3 View commit details
    Browse the repository at this point in the history

  6. Record test config for fifth test

    @charleskorn
    charleskorn committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    4146014 View commit details
    Browse the repository at this point in the history

  7. Record test config for 10k series test

    @charleskorn
    charleskorn committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    aeba5b3 View commit details
    Browse the repository at this point in the history

  8. Record test config for 1k series test

    @charleskorn
    charleskorn committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    c4fe6fc View commit details
    Browse the repository at this point in the history

  9. Record test config for 50k series test

    @charleskorn
    charleskorn committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    c67895d View commit details
    Browse the repository at this point in the history

  10. Fix configuration issue that does not affect outcome of testing. 

    This does not affect the outcome of testing because all tests use
queriers directly (ie. do not use query-frontends or query-schedulers)
    @charleskorn
    charleskorn committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    2c1934b View commit details
    Browse the repository at this point in the history

  11. Record test config for seventh test

    @charleskorn
    charleskorn committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    6578a48 View commit details
    Browse the repository at this point in the history

Commits on Sep 18, 2023

  1. Move production code to main Mimir code tree

    @charleskorn
    charleskorn committed Sep 18, 2023
    Configuration menu
    Copy the full SHA
    537ba98 View commit details
    Browse the repository at this point in the history

  2. Copy test files into repo

    @charleskorn
    charleskorn committed Sep 18, 2023
    Configuration menu
    Copy the full SHA
    9aec0be View commit details
    Browse the repository at this point in the history

  3. Add license and provenance comments

    @charleskorn
    charleskorn committed Sep 18, 2023
    Configuration menu
    Copy the full SHA
    be52a92 View commit details
    Browse the repository at this point in the history

  4. Add a readme.

    @charleskorn
    charleskorn committed Sep 18, 2023
    Configuration menu
    Copy the full SHA
    d533800 View commit details
    Browse the repository at this point in the history

  5. Update package path in scripts and remove unnecessary testing script

    @charleskorn
    charleskorn committed Sep 18, 2023
    Configuration menu
    Copy the full SHA
    b023c31 View commit details
    Browse the repository at this point in the history

  6. Add more notes to code

    @charleskorn
    charleskorn committed Sep 18, 2023
    Configuration menu
    Copy the full SHA
    313c6c4 View commit details
    Browse the repository at this point in the history

  7. Explain benefit of streaming engine.

    @charleskorn
    charleskorn committed Sep 18, 2023
    Configuration menu
    Copy the full SHA
    c859e59 View commit details
    Browse the repository at this point in the history

  8. Mention benchmarking tools in readme.

    @charleskorn
    charleskorn committed Sep 18, 2023
    Configuration menu
    Copy the full SHA
    27ace04 View commit details
    Browse the repository at this point in the history

Commits on Mar 6, 2024

  1. Remove references to Thanos' engine.

    @charleskorn
    charleskorn committed Mar 6, 2024
    Configuration menu
    Copy the full SHA
    1086168 View commit details
    Browse the repository at this point in the history

  2. Undo unnecessary go.mod changes

    @charleskorn
    charleskorn committed Mar 6, 2024
    Configuration menu
    Copy the full SHA
    1b375e9 View commit details
    Browse the repository at this point in the history

  3. Remove benchmarking tool for now 

    We can add something more robust in the future if we need it.
    @charleskorn
    charleskorn committed Mar 6, 2024
    Configuration menu
    Copy the full SHA
    d2e1cab View commit details
    Browse the repository at this point in the history

  4. Remove docker-compose changes

    @charleskorn
    charleskorn committed Mar 6, 2024
    Configuration menu
    Copy the full SHA
    aff2b72 View commit details
    Browse the repository at this point in the history

  5. Reset whitespace changes

    @charleskorn
    charleskorn committed Mar 6, 2024
    Configuration menu
    Copy the full SHA
    9ff2fd5 View commit details
    Browse the repository at this point in the history

  6. Remove readme

    @charleskorn
    charleskorn committed Mar 6, 2024
    Configuration menu
    Copy the full SHA
    a1f222f View commit details
    Browse the repository at this point in the history

  7. Merge branch 'main' into charleskorn/streaming-promql-engine-for-merging 

    # Conflicts:
#	cmd/mimir/help-all.txt.tmpl
#	pkg/mimir/mimir.go
#	pkg/querier/querier.go
    @charleskorn
    charleskorn committed Mar 6, 2024
    Configuration menu
    Copy the full SHA
    9c259f3 View commit details
    Browse the repository at this point in the history

  8. Vendor in branch of mimir-prometheus with changes from prometheus/pro… 

    …metheus#13713
    @charleskorn
    charleskorn committed Mar 6, 2024
    Configuration menu
    Copy the full SHA
    24b35d3 View commit details
    Browse the repository at this point in the history

  9. Fix issues introduced with recent Prometheus changes and fix linting … 

    …issues
    @charleskorn
    charleskorn committed Mar 6, 2024
    Configuration menu
    Copy the full SHA
    aa04c8f View commit details
    Browse the repository at this point in the history

Commits on Mar 13, 2024

  1. Detect and reject configuration not supported by streaming engine

    @charleskorn
    charleskorn committed Mar 13, 2024
    Configuration menu
    Copy the full SHA
    6c9f83d View commit details
    Browse the repository at this point in the history

  2. Remove TODOs tracked elsewhere

    @charleskorn
    charleskorn committed Mar 13, 2024
    Configuration menu
    Copy the full SHA
    a019a40 View commit details
    Browse the repository at this point in the history

  3. Bring in latest changes to QueryEngine interface

    @charleskorn
    charleskorn committed Mar 13, 2024
    Configuration menu
    Copy the full SHA
    723e5f2 View commit details
    Browse the repository at this point in the history

  4. Remove more TODOs tracked elsewhere

    @charleskorn
    charleskorn committed Mar 13, 2024
    Configuration menu
    Copy the full SHA
    de36024 View commit details
    Browse the repository at this point in the history

  5. Change signature of Next() to return an error when the stream is ex… 

    …hausted
    @charleskorn
    charleskorn committed Mar 13, 2024
    Configuration menu
    Copy the full SHA
    0dbbfcb View commit details
    Browse the repository at this point in the history

  6. Ignore warnings when comparing results from Prometheus and streaming … 

    …engines.
    @charleskorn
    charleskorn committed Mar 13, 2024
    Configuration menu
    Copy the full SHA
    15645c9 View commit details
    Browse the repository at this point in the history

  7. Correct naming

    @charleskorn
    charleskorn committed Mar 13, 2024
    Configuration menu
    Copy the full SHA
    d7edb4a View commit details
    Browse the repository at this point in the history

  8. Rename Operator to clarify that it produces instant vectors.

    @charleskorn
    charleskorn committed Mar 13, 2024
    Configuration menu
    Copy the full SHA
    a6b76d5 View commit details
    Browse the repository at this point in the history

Commits on Mar 14, 2024

  1. Rename test file

    @charleskorn
    charleskorn committed Mar 14, 2024
    Configuration menu
    Copy the full SHA
    42bbfb3 View commit details
    Browse the repository at this point in the history

  2. Reorganise test file

    @charleskorn
    charleskorn committed Mar 14, 2024
    Configuration menu
    Copy the full SHA
    761894c View commit details
    Browse the repository at this point in the history

  3. Bring in latest benchmark changes from upstream.

    @charleskorn
    charleskorn committed Mar 14, 2024
    Configuration menu
    Copy the full SHA
    c7d46ea View commit details
    Browse the repository at this point in the history

  4. Run tests and benchmarks with isolation disabled, to mirror what Mimi… 

    …r itself does
    @charleskorn
    charleskorn committed Mar 14, 2024
    Configuration menu
    Copy the full SHA
    9bcf6f3 View commit details
    Browse the repository at this point in the history

  5. Fix script permissions

    @charleskorn
    charleskorn committed Mar 14, 2024
    Configuration menu
    Copy the full SHA
    c6ce4a2 View commit details
    Browse the repository at this point in the history

  6. Simplify test setup and don't bother testing 2000 series case to spee… 

    …d up tests.
    @charleskorn
    charleskorn committed Mar 14, 2024
    Configuration menu
    Copy the full SHA
    ff23e39 View commit details
    Browse the repository at this point in the history

  7. Use assertion helpers in benchmark

    @charleskorn
    charleskorn committed Mar 14, 2024
    Configuration menu
    Copy the full SHA
    9d7354e View commit details
    Browse the repository at this point in the history

  8. Improve engine benchmark comparison script: fix issue with package na… 

    …me containing "streaming", clearly label results
    @charleskorn
    charleskorn committed Mar 14, 2024
    Configuration menu
    Copy the full SHA
    61d22a1 View commit details
    Browse the repository at this point in the history

  9. Remove unnecessary files

    @charleskorn
    charleskorn committed Mar 14, 2024
    Configuration menu
    Copy the full SHA
    eae3a5e View commit details
    Browse the repository at this point in the history

  10. Add note to engine benchmark comparison script

    @charleskorn
    charleskorn committed Mar 14, 2024
    Configuration menu
    Copy the full SHA
    cffbf10 View commit details
    Browse the repository at this point in the history

Commits on Mar 15, 2024

  1. Make test names clearer

    @charleskorn
    charleskorn committed Mar 15, 2024
    Configuration menu
    Copy the full SHA
    a2c8491 View commit details
    Browse the repository at this point in the history

  2. Ensure both engines produce the same result before benchmarking

    @charleskorn
    charleskorn committed Mar 15, 2024
    Configuration menu
    Copy the full SHA
    c23ea43 View commit details
    Browse the repository at this point in the history

  3. Add tests for unsupported expressions and improve error messages retu… 

    …rned when unsupported expression used.
    @charleskorn
    charleskorn committed Mar 15, 2024
    Configuration menu
    Copy the full SHA
    0c9bb09 View commit details
    Browse the repository at this point in the history

  4. Fix linting warnings

    @charleskorn
    charleskorn committed Mar 15, 2024
    Configuration menu
    Copy the full SHA
    ac995e1 View commit details
    Browse the repository at this point in the history

  5. Add notes for invalid cases that should be caught by the PromQL parser.

    @charleskorn
    charleskorn committed Mar 15, 2024
    Configuration menu
    Copy the full SHA
    a9d3959 View commit details
    Browse the repository at this point in the history

  6. Implement Query.Statement()

    @charleskorn
    charleskorn committed Mar 15, 2024
    Configuration menu
    Copy the full SHA
    179cf09 View commit details
    Browse the repository at this point in the history

  7. Give variable a better name

    @charleskorn
    charleskorn committed Mar 15, 2024
    Configuration menu
    Copy the full SHA
    34fdd45 View commit details
    Browse the repository at this point in the history

  8. Remove Pool interface

    @charleskorn
    charleskorn committed Mar 15, 2024
    Configuration menu
    Copy the full SHA
    944306c View commit details
    Browse the repository at this point in the history

  9. Remove unnecessary types

    @charleskorn
    charleskorn committed Mar 15, 2024
    Configuration menu
    Copy the full SHA
    7b09ed4 View commit details
    Browse the repository at this point in the history

  10. Update comments

    @charleskorn
    charleskorn committed Mar 15, 2024
    Configuration menu
    Copy the full SHA
    687f889 View commit details
    Browse the repository at this point in the history

  11. Remove unused lookback delta for range vector selector

    @charleskorn
    charleskorn committed Mar 15, 2024
    Configuration menu
    Copy the full SHA
    2815a1b View commit details
    Browse the repository at this point in the history

Commits on Mar 22, 2024

  1. Extract selector logic to a shared type

    @charleskorn
    charleskorn committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    6189f25 View commit details
    Browse the repository at this point in the history

  2. Add more TODOs

    @charleskorn
    charleskorn committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    378a8e3 View commit details
    Browse the repository at this point in the history

  3. Use Selector for range vector selectors.

    @charleskorn
    charleskorn committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    8006939 View commit details
    Browse the repository at this point in the history

  4. Compute values used on every call to Next() once in instant and range… 

    … vector selectors.

This gives a 0-7% latency improvement for queries with many series at
small absolute cost to queries with few series.

Memory consumption is slightly higher (tens of bytes), but not
concerning.
    @charleskorn
    charleskorn committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    dc292e1 View commit details
    Browse the repository at this point in the history

  5. Merge branch 'main' into charleskorn/streaming-promql-engine-for-merging 

    # Conflicts:
#	go.mod
#	go.sum
#	vendor/modules.txt
    @charleskorn
    charleskorn committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    94d29aa View commit details
    Browse the repository at this point in the history

  6. Remove unnecessary script and update comment

    @charleskorn
    charleskorn committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    03d065a View commit details
    Browse the repository at this point in the history

  7. Validate query time range

    @charleskorn
    charleskorn committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    cbdc44c View commit details
    Browse the repository at this point in the history

  8. Enforce that range query expression produces a vector or scalar result.

    @charleskorn
    charleskorn committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    017304a View commit details
    Browse the repository at this point in the history

  9. Add CLI flag to experimental features list in docs

    @charleskorn
    charleskorn committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    e427ade View commit details
    Browse the repository at this point in the history

  10. Add changelog entry

    @charleskorn
    charleskorn committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    e8ace15 View commit details
    Browse the repository at this point in the history

  11. Fix linting warnings

    @charleskorn
    charleskorn committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    333c8df View commit details
    Browse the repository at this point in the history

Commits on Mar 25, 2024

  1. Move common PromQL engine configuration to its own package

    @charleskorn
    charleskorn committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    e3933a2 View commit details
    Browse the repository at this point in the history

  2. Don't bother with 10 series benchmarks. 

    They're not very interesting, and the benchmarks take too long to run.
    @charleskorn
    charleskorn committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    7d0182b View commit details
    Browse the repository at this point in the history

  3. Sort imports

    @charleskorn
    charleskorn committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    1bbea89 View commit details
    Browse the repository at this point in the history

  4. Add missing license header.

    @charleskorn
    charleskorn committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    0058fc5 View commit details
    Browse the repository at this point in the history

  5. Use bucketed pool based on zeropool. 

    This greatly improves latency and allocations, with latency
improvements between 0 and 15% on our benchmarks.
    @charleskorn
    charleskorn committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    228548b View commit details
    Browse the repository at this point in the history

  6. Fix import sorting

    @charleskorn
    charleskorn committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    91d1cb9 View commit details
    Browse the repository at this point in the history

  7. Set upper limits for sizes of pooled slices.

    @charleskorn
    charleskorn committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    6eaec63 View commit details
    Browse the repository at this point in the history

Commits on Mar 26, 2024

  1. Set size for SeriesBatch pool

    @charleskorn
    charleskorn committed Mar 26, 2024
    Configuration menu
    Copy the full SHA
    2034c55 View commit details
    Browse the repository at this point in the history

  2. Add integration test.

    @charleskorn
    charleskorn committed Mar 26, 2024
    Configuration menu
    Copy the full SHA
    69a25d1 View commit details
    Browse the repository at this point in the history

  3. Move RingBuffer out of util package and pool slices used. 

    This improves latency of rate-related benchmarks by between 0 and 11%.
    @charleskorn
    charleskorn committed Mar 26, 2024
    Configuration menu
    Copy the full SHA
    ebe368e View commit details
    Browse the repository at this point in the history

  4. Add tests for ring buffer type.

    @charleskorn
    charleskorn committed Mar 26, 2024
    Configuration menu
    Copy the full SHA
    8a1c27a View commit details
    Browse the repository at this point in the history

  5. Return RingBuffer point slices to pool when no longer required.

    @charleskorn
    charleskorn committed Mar 26, 2024
    Configuration menu
    Copy the full SHA
    f58b52a View commit details
    Browse the repository at this point in the history

  6. Fix linting

    @charleskorn
    charleskorn committed Mar 26, 2024
    Configuration menu
    Copy the full SHA
    4b82b46 View commit details
    Browse the repository at this point in the history

  7. Remove TODOs about pooling group slices and maps. 

    Pooling these slices did not have a substantial performance impact.
    @charleskorn
    charleskorn committed Mar 26, 2024
    Configuration menu
    Copy the full SHA
    5baaab3 View commit details
    Browse the repository at this point in the history

  8. Pool groups in aggregation operator. 

    This reduces latency of aggregation operations by 0-2% in our
benchmarks.
    @charleskorn
    charleskorn committed Mar 26, 2024
    Configuration menu
    Copy the full SHA
    d7f3433 View commit details
    Browse the repository at this point in the history

  9. Remove TODOs about pooling label builders. 

    Pooling these had a negative performance impact.
    @charleskorn
    charleskorn committed Mar 26, 2024
    Configuration menu
    Copy the full SHA
    1a4dda3 View commit details
    Browse the repository at this point in the history

  10. Remove more TODOs, clarify interface expectations

    @charleskorn
    charleskorn committed Mar 26, 2024
    Configuration menu
    Copy the full SHA
    c5f7858 View commit details
    Browse the repository at this point in the history

  11. Release memory sooner after query execution

    @charleskorn
    charleskorn committed Mar 26, 2024
    Configuration menu
    Copy the full SHA
    37c958e View commit details
    Browse the repository at this point in the history

Commits on Mar 27, 2024

  1. Fix typo in comment

    @charleskorn
    charleskorn committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    c017273 View commit details
    Browse the repository at this point in the history

  2. Remove unecessary context cancellation checks from operators. 

    Context cancellation is handled by the underlying queryable, at least
for the PromQL expressions we currently support.
    @charleskorn
    charleskorn committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    c44074b View commit details
    Browse the repository at this point in the history

  3. Fix linting warnings

    @charleskorn
    charleskorn committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    c367862 View commit details
    Browse the repository at this point in the history

  4. Address remaining TODOs in Selector

    @charleskorn
    charleskorn committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    676070a View commit details
    Browse the repository at this point in the history

  5. Fix integration test flakiness

    @charleskorn
    charleskorn committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    c0dd40c View commit details
    Browse the repository at this point in the history

  6. Don't panic when encountering a native histogram

    @charleskorn
    charleskorn committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    43929c9 View commit details
    Browse the repository at this point in the history

  7. Fix issue where bucketed pool could return a slice with capacity less… 

    … than the requested size.
    @charleskorn
    charleskorn committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    458c366 View commit details
    Browse the repository at this point in the history

  8. Remove redundant checks

    @charleskorn
    charleskorn committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    4aa8f69 View commit details
    Browse the repository at this point in the history

  9. Add support for @ in instant and range vector selectors. 

    This is used extensively by Prometheus' test scripts, so it's easier
to just implement support for this rather than try to workaround it
or disable it.
    @charleskorn
    charleskorn committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    2fe1b10 View commit details
    Browse the repository at this point in the history

  10. Run Prometheus' test cases against streaming engine. 

    There are still some failing test cases, but these appear to be
genuine issues that need to be investigated.
    @charleskorn
    charleskorn committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    fc6b340 View commit details
    Browse the repository at this point in the history

Commits on Mar 28, 2024

  1. Fix failing staleness test: don't emit empty series for instant query

    @charleskorn
    charleskorn committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    52f4671 View commit details
    Browse the repository at this point in the history

  2. Upgrade mimir-prometheus

    @charleskorn
    charleskorn committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    e3f0055 View commit details
    Browse the repository at this point in the history

  3. Add test case for stale markers in a range query.

    @charleskorn
    charleskorn committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    e4e8123 View commit details
    Browse the repository at this point in the history

  4. Add test cases for @ modifier with range queries

    @charleskorn
    charleskorn committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    51ed802 View commit details
    Browse the repository at this point in the history

  5. Check for errors while reading test script

    @charleskorn
    charleskorn committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    6a2623b View commit details
    Browse the repository at this point in the history

  6. Fix whitespace

    @charleskorn
    charleskorn committed Mar 28, 2024
    Configuration menu
    Copy the full SHA
    c51b047 View commit details
    Browse the repository at this point in the history

Commits on Apr 2, 2024

  1. Remove duplicate timestamp computation in Selector and operators

    @charleskorn
    charleskorn committed Apr 2, 2024
    Configuration menu
    Copy the full SHA
    6bbc93e View commit details
    Browse the repository at this point in the history

  2. Update rate calculation to match behaviour in prometheus/prometheus… 

    …#13725
    @charleskorn
    charleskorn committed Apr 2, 2024
    Configuration menu
    Copy the full SHA
    1fd7ad5 View commit details
    Browse the repository at this point in the history

  3. Add an overview of the engine.

    @charleskorn
    charleskorn committed Apr 2, 2024
    Configuration menu
    Copy the full SHA
    b23d086 View commit details
    Browse the repository at this point in the history

  4. Fix linting issues and clarify docs.

    @charleskorn
    charleskorn committed Apr 2, 2024
    Configuration menu
    Copy the full SHA
    30ec0c5 View commit details
    Browse the repository at this point in the history

  5. Add test case for scenario where input to aggregation returns no resu… 

    …lts or no points, and remove stale TODOs
    @charleskorn
    charleskorn committed Apr 2, 2024
    Configuration menu
    Copy the full SHA
    663ec17 View commit details
    Browse the repository at this point in the history

  6. Implement Query.String(), and remove stale TODOs.

    @charleskorn
    charleskorn committed Apr 2, 2024
    Configuration menu
    Copy the full SHA
    06ebc4d View commit details
    Browse the repository at this point in the history

  7. Clarify BucketedPool contract, and add test case for case where min s… 

    …ize is greater than 1.
    @charleskorn
    charleskorn committed Apr 2, 2024
    Configuration menu
    Copy the full SHA
    7bb3ca8 View commit details
    Browse the repository at this point in the history

  8. Expand test coverage to include scenarios not covered by upstream tes… 

    …ts or those uniquely likely to cause issues in the streaming engine.
    @charleskorn
    charleskorn committed Apr 2, 2024
    Configuration menu
    Copy the full SHA
    9f9023a View commit details
    Browse the repository at this point in the history

  9. Elaborate on comment

    @charleskorn
    charleskorn committed Apr 2, 2024
    Configuration menu
    Copy the full SHA
    119160c View commit details
    Browse the repository at this point in the history

  10. Merge branch 'main' into charleskorn/streaming-promql-engine-for-merging

    @charleskorn
    charleskorn committed Apr 2, 2024
    Configuration menu
    Copy the full SHA
    d5a9127 View commit details
    Browse the repository at this point in the history

  11. Expand example in docs and fix typo

    @charleskorn
    charleskorn committed Apr 2, 2024
    Configuration menu
    Copy the full SHA
    65e323f View commit details
    Browse the repository at this point in the history

  12. Remove unnecessary sorting and use non-experimental slices package in… 

    … comparison tests.
    @charleskorn
    charleskorn committed Apr 2, 2024
    Configuration menu
    Copy the full SHA
    c86ecf9 View commit details
    Browse the repository at this point in the history

  13. Add missing comments.

    @charleskorn
    charleskorn committed Apr 2, 2024
    Configuration menu
    Copy the full SHA
    2174241 View commit details
    Browse the repository at this point in the history

Commits on Apr 3, 2024

  1. Clean up temporary files in benchmark comparison script, and improve … 

    …formatting of output
    @charleskorn
    charleskorn committed Apr 3, 2024
    Configuration menu
    Copy the full SHA
    2eadbf5 View commit details
    Browse the repository at this point in the history

Commits on Apr 5, 2024

  1. Merge branch 'main' into charleskorn/streaming-promql-engine-for-merging

    @charleskorn
    charleskorn authored Apr 5, 2024
    Configuration menu
    Copy the full SHA
    6a1af36 View commit details
    Browse the repository at this point in the history

  2. Add note explaining purpose of script.

    @charleskorn
    charleskorn committed Apr 5, 2024
    Configuration menu
    Copy the full SHA
    4419c84 View commit details
    Browse the repository at this point in the history

Commits on Apr 8, 2024

  1. Address PR feedback

    @charleskorn
    charleskorn committed Apr 8, 2024
    Configuration menu
    Copy the full SHA
    6e26ba3 View commit details
    Browse the repository at this point in the history

  2. Replace use of DurationMilliseconds with d.Milliseconds()

    @charleskorn
    charleskorn committed Apr 8, 2024
    Configuration menu
    Copy the full SHA
    ba06a32 View commit details
    Browse the repository at this point in the history

  3. Rename Series() to SeriesMetadata()

    @charleskorn
    charleskorn committed Apr 8, 2024
    Configuration menu
    Copy the full SHA
    7ecc3fe View commit details
    Browse the repository at this point in the history

Commits on Apr 9, 2024

  1. Include names of acceptable values in CLI flag description

    @charleskorn
    charleskorn committed Apr 9, 2024
    Configuration menu
    Copy the full SHA
    d2a9001 View commit details
    Browse the repository at this point in the history

  2. Remove TODOs

    @charleskorn
    charleskorn committed Apr 9, 2024
    Configuration menu
    Copy the full SHA
    95699d6 View commit details
    Browse the repository at this point in the history

  3. Add explanation for RangeVectorSelectorWithTransformation

    @charleskorn
    charleskorn committed Apr 9, 2024
    Configuration menu
    Copy the full SHA
    10bf1cd View commit details
    Browse the repository at this point in the history

  4. Move to top-level streamingpromql package

    @charleskorn
    charleskorn committed Apr 9, 2024
    Configuration menu
    Copy the full SHA
    1bdf6c6 View commit details
    Browse the repository at this point in the history

  5. Move common query engine config back to original location (revert e39… 

    …33a2)
    @charleskorn
    charleskorn committed Apr 9, 2024
    Configuration menu
    Copy the full SHA
    2dba89a View commit details
    Browse the repository at this point in the history