Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ci(instrumentation-http): improve metrics test stability #3242

Conversation

pichlermarc
Copy link
Member

Which problem is this PR solving?

Currently the tests for metrics in the @opentelemetry/http-instrumentation package are failing every once in a while. This PR aims to improve test stability by introducing a custom MetricReader with a export method that does not rely on time passed.

Fixes #3238

Type of change

  • Bug fix (non-breaking change which fixes an issue)

How Has This Been Tested?

  • Existing unit tests

Checklist:

  • Followed the style guidelines of this project
  • Unit tests have been added
  • Documentation has been updated

@codecov
Copy link

codecov bot commented Sep 9, 2022

Codecov Report

Merging #3242 (5ef3a6f) into main (bd0a77f) will decrease coverage by 0.01%.
The diff coverage is n/a.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3242      +/-   ##
==========================================
- Coverage   93.30%   93.29%   -0.02%     
==========================================
  Files         203      203              
  Lines        6606     6606              
  Branches     1389     1389              
==========================================
- Hits         6164     6163       -1     
- Misses        442      443       +1     
Impacted Files Coverage Δ
...-trace-base/src/platform/node/RandomIdGenerator.ts 87.50% <0.00%> (-6.25%) ⬇️

@pichlermarc pichlermarc marked this pull request as ready for review September 9, 2022 08:01
@pichlermarc pichlermarc requested a review from a team September 9, 2022 08:01
experimental/CHANGELOG.md Outdated Show resolved Hide resolved
@pichlermarc pichlermarc changed the title fix(instrumentation-http): improve metrics test stability ci(instrumentation-http): improve metrics test stability Sep 12, 2022
@dyladan
Copy link
Member

dyladan commented Sep 12, 2022

I think the changes in #3225 broke this PR and now you have to use the override modifier on the testing class methods

Copy link
Member

@legendecas legendecas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm in favor of this explicit collection than the periodic metric reader based approach!

Copy link
Member

@legendecas legendecas left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

CI is complaining but the code LGTM.

@osherv
Copy link
Member

osherv commented Sep 13, 2022

Hey! That's an awesome feature!
Actually, that could be really useful in the 'contrib' package also (thanks @legendecas for pointing that out).
Does it possible to export this TestMetricReader so we can use it in the contrib repo? We already exporting InMemoryExporters that are really help :) Otherwise, we'll implement that also in the contrib package.
Here's the related contrib pr
Thanks!

@pichlermarc
Copy link
Member Author

I think the changes in #3225 broke this PR and now you have to use the override modifier on the testing class methods

Yep that's the problem. Fixed in 36eb96b 🙂

@pichlermarc
Copy link
Member Author

pichlermarc commented Sep 13, 2022

Hey! That's an awesome feature! Actually, that could be really useful in the 'contrib' package also (thanks @legendecas for pointing that out). Does it possible to export this TestMetricReader so we can use it in the contrib repo? We already exporting InMemoryExporters that are really help :) Otherwise, we'll implement that also in the contrib package. Here's the related contrib pr Thanks!

Hi, thanks! 🙂 While I do agree that this is quite useful for tests, there are no plans to export this right now (see this comment here).
For now I would recommend to copy this over to the contrib repo, but be aware that it might not work 1:1 as there are unreleased changes to the MetricReader at the moment.

@osherv
Copy link
Member

osherv commented Sep 13, 2022

@pichlermarc Thanks :)

@legendecas legendecas merged commit fade3c9 into open-telemetry:main Sep 13, 2022
@dyladan dyladan deleted the fix/instrumentation-flakyness branch September 13, 2022 18:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Flaky test in @opentelemetry/instrumentation-http
5 participants