Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Handle TSDB native histogram validation errors as soft errors #7773

Merged
merged 5 commits into from
Apr 3, 2024

Conversation

krajorama
Copy link
Contributor

@krajorama krajorama commented Apr 2, 2024

What this PR does

Handle TSDB native histogram validation errors as soft errors.

Which issue(s) this PR fixes or relates to

Supersedes #7736

Checklist

  • Tests updated.
  • N/A Documentation added.
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX].
  • N/A about-versioning.md updated with experimental features.

@krajorama krajorama requested review from jdbaldry and a team as code owners April 2, 2024 13:53
@krajorama krajorama force-pushed the krajo/native-histograms-extra-validation branch from 922f112 to 5d825c7 Compare April 2, 2024 13:56
Copy link
Member

@pstibrany pstibrany left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you. Code changes make sense to me. I'd prefer to have some tests for mapping of TSDB native histogram errors to correct errors from Mimir.

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
@krajorama
Copy link
Contributor Author

Thank you. Code changes make sense to me. I'd prefer to have some tests for mapping of TSDB native histogram errors to correct errors from Mimir.

Added testcases for all scenarios.

Copy link
Member

@pstibrany pstibrany left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you very much!

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
@krajorama krajorama changed the title Handle TSDB native histogram validation errors are soft errors Handle TSDB native histogram validation errors as soft errors Apr 3, 2024
Copy link
Member

@pstibrany pstibrany left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@krajorama
Copy link
Contributor Author

Additional metrics work in production:
image

@krajorama krajorama enabled auto-merge (squash) April 3, 2024 10:23
@krajorama krajorama merged commit 51c4088 into main Apr 3, 2024
29 checks passed
@krajorama krajorama deleted the krajo/native-histograms-extra-validation branch April 3, 2024 10:37
@grafanabot
Copy link
Contributor

The backport to r284 failed:

The process '/usr/bin/git' failed with exit code 1

To backport manually, run these commands in your terminal:

# Fetch latest updates from GitHub
git fetch
# Create a new branch
git switch --create backport-7773-to-r284 origin/r284
# Cherry-pick the merged commit of this pull request and resolve the conflicts
git cherry-pick -x 51c4088cc3b5087aabf72478dcc909d011ff64c0
# Push it to GitHub
git push --set-upstream origin backport-7773-to-r284
git switch main
# Remove the local backport branch
git branch -D backport-7773-to-r284

Then, create a pull request where the base branch is r284 and the compare/head branch is backport-7773-to-r284.

krajorama added a commit that referenced this pull request Apr 3, 2024
* Revert "Distributor: add bucket count validation to native histograms (#7736)"

This reverts commit fb7dbaa.

* Handle TSDB native histogram validation errors are soft errors

* add all testcases

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>

* Count towards discarded samples metrics with new reason

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>

---------

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
(cherry picked from commit 51c4088)
krajorama added a commit that referenced this pull request Apr 3, 2024
…#7785)

* Revert "Distributor: add bucket count validation to native histograms (#7736)"

This reverts commit fb7dbaa.

* Handle TSDB native histogram validation errors are soft errors

* add all testcases

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>

* Count towards discarded samples metrics with new reason

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>

---------

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
(cherry picked from commit 51c4088)
grafanabot pushed a commit that referenced this pull request Apr 3, 2024
…#7785)

* Revert "Distributor: add bucket count validation to native histograms (#7736)"

This reverts commit fb7dbaa.

* Handle TSDB native histogram validation errors are soft errors

* add all testcases

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>

* Count towards discarded samples metrics with new reason

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>

---------

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
(cherry picked from commit 51c4088)
(cherry picked from commit 868d1b3)
krajorama added a commit that referenced this pull request Apr 3, 2024
* Revert "Distributor: add bucket count validation to native histograms (#7736)"

This reverts commit fb7dbaa.

* Handle TSDB native histogram validation errors are soft errors

* add all testcases

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>

* Count towards discarded samples metrics with new reason

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>

---------

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
(cherry picked from commit 51c4088)
krajorama added a commit that referenced this pull request Apr 3, 2024
…#7785) (#7786)

* Revert "Distributor: add bucket count validation to native histograms (#7736)"

This reverts commit fb7dbaa.

* Handle TSDB native histogram validation errors are soft errors

* add all testcases

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>

* Count towards discarded samples metrics with new reason

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>

---------

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
(cherry picked from commit 51c4088)
(cherry picked from commit 868d1b3)

Co-authored-by: George Krajcsovits <krajorama@users.noreply.github.com>
krajorama added a commit that referenced this pull request Apr 3, 2024
…#7787)

* Revert "Distributor: add bucket count validation to native histograms (#7736)"

This reverts commit fb7dbaa.

* Handle TSDB native histogram validation errors are soft errors

* add all testcases

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>

* Count towards discarded samples metrics with new reason

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>

---------

Signed-off-by: György Krajcsovits <gyorgy.krajcsovits@grafana.com>
(cherry picked from commit 51c4088)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants