Support to prometheus native histogram

[jeromeinsf]

Is your feature request related to a problem? Please describe.
Experimental native histogram support was released with Prometheus v0.40.x.

Describe the solution you'd like
We should implement native histogram support (behind a feature flag)

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.

[elliesaber]

just checking if there is any plan for supporting prometheus native histogram
great news opentelemetry collector now supports the conversion as well
open-telemetry/opentelemetry-collector-contrib#17370

[friedrichg]

@amouhadi yes that is the plan

[friedrichg]

#5289 this tracks how many samples for this histogram is cortex dropping. It's a start

[hkfgo]

Hey just checking in again to see if anything's changed. Is native histogram support still on the roadmap?

[yeya24]

Yeah I believe it is still on the roadmap. We need to wait our upstream project Thanos to support native histograms first.

[yeya24]

If I understand correctly, Thanos already supports native histogram on all components other than downsampling.
We should start integrating with native histograms on Cortex

[yeya24]

Let me update the checklist for steps to enable native histograms. Feel free to leave comments about missing items.

Checklist

• Distributor
  • Distributor should send native histograms in its write request Fix distributor sends native histograms #5679 Update unit tests for distributor push native histograms #6020
• Ingester
  • Ingester is able to ingest native histograms to TSDB Allow ingesting native histograms #5986 Add more tests for native histogram push #6010
• Query Frontend
  • Include native histogram as part of Query response, merge native histogram samples correctly support querying native histograms in query frontend #5996
  • Support for native histogram in instant queries support QFE instant query parse native histograms #6043
• Querier
  • Prerequisite change and cleanup: Remove unused tombstones loader and cache gen code #5694
  • Allow querier to be able to handle native histograms from Store Gateway: Refactor: remove chunk interface #5884
  • Change querier series interface to include native histogram chunks Implement histogram iterators for batch #5944
  • More tests Add more unit tests for querying native histograms #6031
• Ruler
  • Ingest native histograms to ingester Support ingesting native histograms in Ruler appender #6029
• Store Gateway
  • Make sure Store Gateway can return native histogram in response (should work as Thanos supports it)
• Compactor
  • Make sure compactor compacts native histogram series (should work once TSDB up-to-date)

[gburton1]

How's the checklist above looking? Any recent updates?

[hkfgo]

I see an August date for native histogram support, under the cortex project page. Is that date an estimate?

[yeya24]

Thanks. I updated the checklist above.
I think the ingestion path is done. Query path are almost done. Things left:

• Ruler
• Querier queries histogram iterator when asked for float histogram

Also some more tests need to be added.
I expect it to be released as part of 1.18 release.

[yeya24]

With all the PRs merged, I think we can mark this issue as resolved.

I will create another issue to track any efforts post merge.

Updated: new tracking issue #6052