Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Store sum of events for different metrics #1181

Closed

Conversation

hierynomus
Copy link

This is a rework of PR #1022 by @hashbrowncipher (and fixes @slovdahl's issue #712). This PR is based on the 4.0-development branch, which has evolved further than master.

The main differences are:

  • Introduction of a new Summing interface, which is implemented by Meter, Timer and Histogram
  • Reporting of the Summing.sum() is done as a long to avoid loss of precision on high values due to floating point properties
  • All reporters report the sum() property of the relevant metrics

The main purpose of storing the sum is to facilitate monitoring systems that use raw ascending counters (e.g. Prometheus), and time-series databases such as InfluxDB (see also @mevdschee comment)

@arteam arteam added this to the 5.0.0 milestone Sep 8, 2017
@vladimir-bukhtoyarov
Copy link
Contributor

Long datatype is not enough long to store sum in common case. For example when using the Long for timer you are able to store only 192 year of accumulated latency then overflow will happen

@storozhukBM
Copy link
Contributor

@vladimir-bukhtoyarov any suggestions for suitable sum storage type?

@mevdschee
Copy link

192 year of accumulated latency then overflow will happen

I'm not sure I see the issue. You need to run with many cores on a very popular task to have this duration exceed 192 years. And even if this happens, would it cause a problem? When using monotonous increasing counters you will miss just a single measurement.

@vladimir-bukhtoyarov
Copy link
Contributor

When using monotonous increasing counters you will miss just a single measurement.

Keep in mind that you are unable to achive even monotonous increasing semantic on arithmetic overflow when using LongAdder. Because LongAdder splits it state to many chunks, and arithmetic overflow can happen in one chunk, simultneously on many chunks, in sum, and due to dynamic chunk count inside LongAdder the results is totally unpredictable when overflow happen.

@vladimir-bukhtoyarov
Copy link
Contributor

vladimir-bukhtoyarov commented Nov 15, 2017

@vladimir-bukhtoyarov any suggestions for suitable sum storage type?

I argue that this kind of functionality is not needed at core level. The problem can be easy solved by decorators. Anyone who firstly wants to have sum, secondly understands the risks related to overflow, can easily decorate any timer or histogram and implement sum reporting as separated gauge.

Of course it is possible to replace LongAdder by DoubleAdder to pospone the overflow, but accuracy of sum will be significantly decreased.

@storozhukBM
Copy link
Contributor

storozhukBM commented Nov 15, 2017

I argue that this kind of functionality is not needed at core level.

OK, now it's clear.
Personally I'm totally agree with your thoughts. Core library is actually flexible enough and you can easily extend Histogram class within your project to add required functionality.

@mevdschee
Copy link

the results is totally unpredictable when overflow happen.

True, but can't we still say we miss one measurement on average per 192 year of duration data?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants