From 7f2937efbad53bb340e0f80db290e938198115a0 Mon Sep 17 00:00:00 2001 From: jmacd Date: Thu, 27 Jun 2019 14:31:04 -0700 Subject: [PATCH 01/11] Four metrics RFCs to eliminate raw statistics --- README.md | 24 ++++++++++++------------ 1 file changed, 12 insertions(+), 12 deletions(-) diff --git a/README.md b/README.md index e796449d1..3dee9ae69 100644 --- a/README.md +++ b/README.md @@ -68,26 +68,26 @@ If the rationale behind a change of state cannot be easily inferred - e.g., if a ### Submitting a new RFC 1. If you haven't done so yet, [fork](https://help.github.com/en/articles/fork-a-repo) the [repo](https://github.com/open-telemetry/rfcs) -1. Copy [`0000-template.md`] to `text/0000-my-rfc.md`, where `my-rfc` is a title relevant to your proposal. Leave the number as is for now. +1. Copy [`0000-template.md`](./0000-template.md) to `text/0000-my-rfc.md`, where `my-rfc` is a title relevant to your proposal. Leave the number as is for now. 1. Fill in the template. Do not omit things you think may be obvious to readers; it's better to be too explicit than to leave ambiguity! In particular, assume that no one else cares yet about the proposal, and that your are writing this to **convince** them that they should :) 1. Submit a [pull request](https://github.com/open-telemetry/compare). Assuming that your RFC follows the template and there are no significant omissions, the pull request will be accepted and your RFC will be assigned a number - * TODO: Should this be automated? + * TODO: Should this be automated? 1. A new "work-in-progress" (WIP) pull request will be created with that updates the RFC's status to `approved` - * TODO: This should probably be automated + * TODO: This should probably be automated 1. In the WIP pull request, your proposed RFC will now be up for feedback from the OpenTelemetry community. Others may suggest changes, raise concerns, or add new pull requests building on top of yours 1. Aim to address the discussion! While you are the champion of the RFC, the community collectively owns the result. We're all in this together :) - * Please make changes as new commits, documenting the reasoning for each in the commit message. Try to avoid squashing, rebasing, force-pushing, etc. at this point, as doing so may mess with the historical discussion on GitHub + * Please make changes as new commits, documenting the reasoning for each in the commit message. Try to avoid squashing, rebasing, force-pushing, etc. at this point, as doing so may mess with the historical discussion on GitHub 1. Once the value and trade-offs of the change have been sufficiently discussed, a non-author member of the relevant SIG will remove "WIP" from the PR's title and propose a "motion for final comment period" (FCP) - * By this point, no **new** points should have been raised in the discussion for **at least three business days** - * If substantial new points (e.g., a significant trade-off) are raised *during* the FCP, the FCP should be cancelled and discussion renewed + * By this point, no **new** points should have been raised in the discussion for **at least three business days** + * If substantial new points (e.g., a significant trade-off) are raised *during* the FCP, the FCP should be cancelled and discussion renewed 1. During the FCP, SIG members should each vote to `approve`, `reject`, or `defer` the RFC - * If the discussion has been contentious, members should explain the rationale behind their vote + * If the discussion has been contentious, members should explain the rationale behind their vote 1. The FCP will be considered closed and the RFC either approved, rejected, or approved after the following conditions have all been met: - * At least two business days have passed since the FCP began - * Either: - * A majority of non-author members have voted - * At least three non-author members have voted, and the vote is unanimous -* The RFC will be merged with the appropriate state + * At least two business days have passed since the FCP began + * Either: + * A majority of non-author members have voted + * At least three non-author members have voted, and the vote is unanimous +1. The RFC will be merged with the appropriate state ### Reviewing an RFC From 6fc43532a8acc9024c143fa85777d53d2e646208 Mon Sep 17 00:00:00 2001 From: jmacd Date: Thu, 27 Jun 2019 15:40:40 -0700 Subject: [PATCH 02/11] Create 4 RFC drafts on metrics --- 0001-metric-pre-defined-labels.md | 37 ++++++++++++++++ 0002-metric-measure.md | 37 ++++++++++++++++ 0003-eliminate-stats-record.md | 34 ++++++++++++++ 0004-metric-configurable-aggregation.md | 59 +++++++++++++++++++++++++ 4 files changed, 167 insertions(+) create mode 100644 0001-metric-pre-defined-labels.md create mode 100644 0002-metric-measure.md create mode 100644 0003-eliminate-stats-record.md create mode 100644 0004-metric-configurable-aggregation.md diff --git a/0001-metric-pre-defined-labels.md b/0001-metric-pre-defined-labels.md new file mode 100644 index 000000000..128449d5f --- /dev/null +++ b/0001-metric-pre-defined-labels.md @@ -0,0 +1,37 @@ +# Pre-defined label support for all metric operations + +Let all Metric objects (Cumulative, Gauge, ...) and raw statistics support pre-defined label values. + +## Motivation + +In the current `Metric.GetOrCreateTimeSeries` API for Gauges and Cumulatives, the caller obtains a `TimeSeries` handle for repeatedly recording metrics with certain pre-defined label values set. This is an important optimization, especially for exporting aggregated metrics. + +The use of pre-defined labels improves usability too, for working with metrics in code. Application programs with long-lived objects and associated Metrics can compute predefined label values once (e.g., in a constructor), rather than once per call site. + +The current API for recording raw statistics does not support the same optimization or usability advantage. This RFC proposes to add support for pre-defined labels on all metrics. + +## Explanation + +In the current proposal, Metrics are used for _common_ pre-aggregated metric types, whereas Raw statistics are used for uncommon and vendor-specific aggregations. The optimization and the usability advantages gained with pre-defined labels should be extended to Raw statistics because they equally important and equally applicable. + +For example, where the application wants to compute a histogram of some value (e.g., latency), there's good reason to pre-aggregate such information. In this exampler, this allows an implementation to effienctly export the histogram of latencies "grouped" into individual results by label value(s). + +## Internal details + +This RFC is accompanied by RFC 0002-metric-measure which proposes to create a new Metric type to replace raw statistics. The metric type, named "Measure", would replace the existing concept and type named "Measure" in the metrics API. The new MeasureMetric object would support a `Record` method to record measurements. + +## Trade-offs and mitigations + +This is an refactoring of the existing proposal to cover more use-cases that does not introduce new complexity. The existing mechanism for metrics is extended to raw statistics by a new metric type, which allows elimianting raw statistics, leaving equal complexity. + +## Prior art and alternatives + +This Measure Metric API is conceptually close to the Prometheus [Histogram, Summary, and Untyped metric types](https://prometheus.io/docs/concepts/metric_types/). + +## Open questions + +This RFC is co-dependent on several others; it's an open question how to address this concern if the other RFCs are not accepted. + +## Future possibilities + +This change will potentially help clarify the relationship between Metric types and Aggregation types. In a future RFC, we will propose that Measure Metrics (as introduced here) can be used to support arbitrary "advanced" aggregations including histograms and distribution summaries. diff --git a/0002-metric-measure.md b/0002-metric-measure.md new file mode 100644 index 000000000..8fb05de76 --- /dev/null +++ b/0002-metric-measure.md @@ -0,0 +1,37 @@ +# Replace Raw statistics with Measure-type Metric + +Define a new Metric type named "Measure" to cover existing "Raw" statistics uses. + +## Motivation + +The primary motivation is that raw statistics should support the optimization and usability improvements associated with pre-defined label values (0001-metric-pre-defined-labels). By elevating non-Cumulative, non-Gauge statistics to the same conceptual level as Metrics in the API, we effectively make the type of a metric independent from whether it supports pre-defined labels. + +This also makes it possible to eliminate the low-level `stats.Record` interface from the API specification entirely (0003-eliminate-stats-record). + +## Explanation + +This extends the `GetOrCreateTimeSeries` functionality supported by Metrics to what has been known as "Raw" statistics, satisfying the change in capability requested in RFC 0001-metric-pre-defined-labels. This allows programmers to predefined labels for all metrics, regardless of whether they are actually configured for pre-aggregation or not. This is not only a potential optimization for the programmer, it is a usability improvement in the code. + +Without raw statistics in the API, it becomes possible to elimiante the low-level `stats.Record` API, which may also be desireable. + +## Internal details + +The type known as `MeasureMetric` is a direct replacement for raw statistics. The `MeasureMetric.Record` method records a single observation of the metric. The `MeasureMetric.GetOrCreateTimeSeries` supports pre-defined keys as discussed in 0001-metric-pre-defined-labels. + +## Trade-offs and mitigations + +This change, while it eliminates the need for a raw statistics concept, potentially introduces new required concepts. Whereas Raw statistics have no directly-declared aggregations, introducing MeasureMetric raises the question of which aggregations apply. We will propose how a programmer can declare recommended aggregations (and good defaults) in RFC 0004-configurable-aggregation. + +## Prior art and alternatives + +The MeasureMetric type introduced here covers a related group of metric types from other systems, including Histogram metric, Summary metric, and Unknown metrics in the Prometheus system. The proposal here suggests that we think of the metric type in terms of the _action performed_ (i.e., which _verb_?). Gauges support the `Set` action. Cumulatives support an `Inc` action. Measures support a `Record` action. + +This proposal suggests we think about which aggregations apply to a metric independently. A MeasureMetric could be used to aggregate a Histogram, or a Summary, or _both_ of these aggregations simultaneously. This proposal makes metric type independent of aggregation type, whereas there is a precedent for combining these types into one. + +## Open questions + +With this proposal accepted, there would be three Metric types: Gauge, Cumulative, and Measure. This proposal does not directly address what to do over the existing, conflicting uses of "Measure" and "Measurement". + +## Future possibilities + +This change enables metrics to support configurable aggregation types, which allows the programmer to provide recommended aggregations at the point where Metrics are defined. This will allow support for good out-of-the-box behavior for metrics defined by third-party libraries, for example. diff --git a/0003-eliminate-stats-record.md b/0003-eliminate-stats-record.md new file mode 100644 index 000000000..e9b40d888 --- /dev/null +++ b/0003-eliminate-stats-record.md @@ -0,0 +1,34 @@ +# Eliminate stats.Record functionality + +Remove `stats.Record` from the specification, following the MeasureMetric type (RFC 0002-metric-measure). + +## Motivation + +This change is no longer a necessary interface. There are conceivable reasons to support it, but they are not outweighed by the cost of implementing and supporting two interfaces for recording metrics and statistics. + +## Explanation + +In RFC 0002-metric-measure, a new MeasureMetric type is introduced to replace raw statistics, with support for pre-defined label values. With the new type introduced, it's now possible to record formerly-raw statistics through a higher-level Metric interface. stats.Record is no longer a requirement and should be removed until strong evidence of its need emerges from real use-cases. + +## Internal details + +This simply involves removing the low-level `stats.Record` API from the specification. + +## Trade-offs and mitigations + +There are two reasons to maintain a low-level API that we know of: + +1. For _generality_. An application that forwards metrics from another source may need to handle metrics in generic code. For these applications, having type-specific Metric handles could actually require more code to be written, whereas the low-level `stats.Record` API is more amenable to generic use. +1. For _atomicity_. An application that wishes to record multiple statistics in a single operation can feel confident computing formulas based on multiple metrics, not worry about inconsistent views of the data. + +## Prior art and alternatives + +Raw statistics were a solution to confusion found in existing metrics APIs over Metric types vs. Aggregation types. This proposal accompanies RFC 0001-metric-pre-defined-labels and RFC 0002-metric-measure.md in proposing that we think about Metric _type_ as independent of which aggregations apply. Once we have a Metric to support histogram and summary aggregations, we no longer need raw statistics, and we no longer need `stats.Record`. This avoids introducing new concepts (Raw statistics), at the same time departs from prior art in letting one Metric type support both Histogram and Summary aggregations. + +## Open questions + +Are either of the trade-offs described above important enough to keep the low-level `stats.Record` API? + +## Future possibilities + +This restricts future possibilities for the benefit of a smaller, simpler specification. diff --git a/0004-metric-configurable-aggregation.md b/0004-metric-configurable-aggregation.md new file mode 100644 index 000000000..4dc43b869 --- /dev/null +++ b/0004-metric-configurable-aggregation.md @@ -0,0 +1,59 @@ +# Let Metrics support configrable, recommended aggregations + +Let the user configure recommended Metric aggregations (SUM, COUNT, MIN, MAX, LAST_VALUE, HISTOGRAM, SUMMARY). + +## Motivation + +In the current API proposal, Metric types like Gauge and Cumulative are mapped into specific aggregations: Gauge:LAST_VALUE and Cumulative:SUM. Depending on RFC 0002-metric-measure, which creates a new MeasureMetric type, this proposal introduces the ability to configure alternative, potentially multiple aggregations for Metrics. This allows the MeasureMetric type to support HISTOGRAM and SUMMARY aggregations, as an alternative to raw statistics. + +## Explanation + +This proposal completes the elimination of Raw statistics by recognizing that aggregations should be independent of metric type. This recognizes that _sometimes_ we have a cumulative but want to compute a histogram of increment values, and _sometimes_ we have a measure that has multiple interesting aggregations. + +Following this change, we should think of the _Metric type_ as: + +1. Indicating something about what kind of numbers are being recorded (i.e., the input domain, e.g., restricted to values >= 0?) + 1. For Gauges: Something pre-computed where rate or count is not relevant + 1. For Cumulatives: Something where rate or count is relevant + 1. For Measures: Something where individual values are relevant +1. Indicating something about the default interpretation, based on the action verb (Set, Inc, Record, etc.) + 1. For Gauges: the action is Set() + 1. For Cumulatives: the action is Inc() + 1. For Measures: the action is Record() +1. Unless the programmer declares otherwise, suggesting a default aggregation + 1. For Gauges: LAST_VALUE is interesting, SUM is likely not interesting + 1. For Cumulatives: SUM is interesting, LAST_VALUE is likely not interesting + 1. For Measures: all aggregations apply, default is MIN, MAX, SUM, COUNT. + +## Internal details + +Metric constructors should take an optional list of aggregations, to override the default behavior. When constructed with an explicit list of aggregations, the implementation may use this as a hint about which aggregations should be exported by default. However, the implementation is not bound by these recommendations in any way and is free to control which aggregations that are applied. + +The standard defined aggregations are broken into two groups, those which are "decomposable" (i.e., inexpensive) and those which are not. + +The decomposable aggregations are simple to define: + +1. SUM: The sum of observed values. +1. COUNT: The number of observations. +1. MIN: The smallest value. +1. MAX: The largest value. +1. LAST_VALUE: The latest value. + +The non-decomposable aggregations do not have standard definitions, they are purely advisory. The intention behind these are: + +1. HISTOGRAM: The intended output is a distribution summary, specifically summarizing counts into non-overlapping ranges. +1. SUMMARY: This is a more generic way to request information about a distribution, perhaps represented in some vendor-specific way / not a histogram. + +## Trade-offs and mitigations + +This avoids requiring programmers to use the `view` API, which is an SDK API, not a user-facing instrumentation API. Letting the application programmer recommend aggregations directly gives the implementation more information about the raw statistics. Letting programmers declare their intent has few downsides, since there is a well-defined default behavior. + +## Prior art and alternatives + +Existing systems generaly declare separate Metric types according to the desired aggregation. Raw statistics were invented to overcome this, and the present proposal brings back the ability to specify an Aggregation at the point where a Metric is defined. + +## Open questions + +There are questions about the value of the MIN and MAX aggregations. While they are simple to compute, they are difficult to use in practice. + +There are questions about the interpretation of HISTOGRAM and SUMMARY. The point of Raw statistics was that we shouldn't specify these aggregations because they are expensive and many implementations are possible. This is still true. What is the value in specifying HISTOGRAM as opposed to SUMMARY? How is SUMMARY different from MIN/MAX/COUNT/SUM? From 04acacb77db385f917327db12b201e99b8daa3f3 Mon Sep 17 00:00:00 2001 From: Joshua MacDonald Date: Fri, 28 Jun 2019 12:36:07 -0700 Subject: [PATCH 03/11] Update 0001-metric-pre-defined-labels.md Co-Authored-By: Isobel Redelmeier --- 0001-metric-pre-defined-labels.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/0001-metric-pre-defined-labels.md b/0001-metric-pre-defined-labels.md index 128449d5f..442d17f60 100644 --- a/0001-metric-pre-defined-labels.md +++ b/0001-metric-pre-defined-labels.md @@ -12,7 +12,7 @@ The current API for recording raw statistics does not support the same optimizat ## Explanation -In the current proposal, Metrics are used for _common_ pre-aggregated metric types, whereas Raw statistics are used for uncommon and vendor-specific aggregations. The optimization and the usability advantages gained with pre-defined labels should be extended to Raw statistics because they equally important and equally applicable. +In the current proposal, Metrics are used for _common_ pre-aggregated metric types, whereas Raw statistics are used for uncommon and vendor-specific aggregations. The optimization and the usability advantages gained with pre-defined labels should be extended to Raw statistics because they are equally important and equally applicable. For example, where the application wants to compute a histogram of some value (e.g., latency), there's good reason to pre-aggregate such information. In this exampler, this allows an implementation to effienctly export the histogram of latencies "grouped" into individual results by label value(s). From fb47377758f14c667599352d460d8c2ed59f5848 Mon Sep 17 00:00:00 2001 From: Joshua MacDonald Date: Fri, 28 Jun 2019 12:38:35 -0700 Subject: [PATCH 04/11] Update 0001-metric-pre-defined-labels.md Co-Authored-By: Isobel Redelmeier --- 0001-metric-pre-defined-labels.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/0001-metric-pre-defined-labels.md b/0001-metric-pre-defined-labels.md index 442d17f60..99ba02780 100644 --- a/0001-metric-pre-defined-labels.md +++ b/0001-metric-pre-defined-labels.md @@ -22,7 +22,7 @@ This RFC is accompanied by RFC 0002-metric-measure which proposes to create a ne ## Trade-offs and mitigations -This is an refactoring of the existing proposal to cover more use-cases that does not introduce new complexity. The existing mechanism for metrics is extended to raw statistics by a new metric type, which allows elimianting raw statistics, leaving equal complexity. +This is a refactoring of the existing proposal to cover more use-cases and does not introduce new complexity. The existing mechanism for metrics is extended raw statistics by a new metric type, which allows eliminating raw statistics, leaving equal complexity. ## Prior art and alternatives From 8f6917c5079e7705978678759633bbefde1c7564 Mon Sep 17 00:00:00 2001 From: jmacd Date: Fri, 28 Jun 2019 15:43:44 -0700 Subject: [PATCH 05/11] Refinements --- 0001-metric-pre-defined-labels.md | 16 ++++++------ 0002-metric-measure.md | 22 +++++++++------- 0003-eliminate-stats-record.md | 8 +++--- 0004-metric-configurable-aggregation.md | 34 +++++++++++++++++-------- 4 files changed, 50 insertions(+), 30 deletions(-) diff --git a/0001-metric-pre-defined-labels.md b/0001-metric-pre-defined-labels.md index 99ba02780..52217cf5b 100644 --- a/0001-metric-pre-defined-labels.md +++ b/0001-metric-pre-defined-labels.md @@ -1,6 +1,6 @@ # Pre-defined label support for all metric operations -Let all Metric objects (Cumulative, Gauge, ...) and raw statistics support pre-defined label values. +Let all Metric objects (Cumulative, Gauge, ...) and Raw statistics support pre-defined label values. ## Motivation @@ -8,25 +8,25 @@ In the current `Metric.GetOrCreateTimeSeries` API for Gauges and Cumulatives, th The use of pre-defined labels improves usability too, for working with metrics in code. Application programs with long-lived objects and associated Metrics can compute predefined label values once (e.g., in a constructor), rather than once per call site. -The current API for recording raw statistics does not support the same optimization or usability advantage. This RFC proposes to add support for pre-defined labels on all metrics. +The current API for recording Raw statistics does not support the same optimization or usability advantage. This RFC proposes to add support for pre-defined labels on all metrics. ## Explanation -In the current proposal, Metrics are used for _common_ pre-aggregated metric types, whereas Raw statistics are used for uncommon and vendor-specific aggregations. The optimization and the usability advantages gained with pre-defined labels should be extended to Raw statistics because they are equally important and equally applicable. +In the current proposal, Metrics are used for pre-aggregated metric types, whereas Raw statistics are used for uncommon and vendor-specific aggregations. The optimization and the usability advantages gained with pre-defined labels should be extended to Raw statistics because they are equally important and equally applicable. This is a new requirement. -For example, where the application wants to compute a histogram of some value (e.g., latency), there's good reason to pre-aggregate such information. In this exampler, this allows an implementation to effienctly export the histogram of latencies "grouped" into individual results by label value(s). +For example, where the application wants to compute a histogram of some value (e.g., latency), there's good reason to pre-aggregate such information. In this example, it allows an implementation to effienctly export the histogram of latencies "grouped" into individual results by label value(s). ## Internal details -This RFC is accompanied by RFC 0002-metric-measure which proposes to create a new Metric type to replace raw statistics. The metric type, named "Measure", would replace the existing concept and type named "Measure" in the metrics API. The new MeasureMetric object would support a `Record` method to record measurements. +This RFC is accompanied by RFC 0002-metric-measure which proposes to create a new Metric type to replace Raw statistics. The metric type, named "Measure", would replace the existing concept and type named "Measure" in the metrics API. The new MeasureMetric object would support a `Record` method to record measurements. ## Trade-offs and mitigations -This is a refactoring of the existing proposal to cover more use-cases and does not introduce new complexity. The existing mechanism for metrics is extended raw statistics by a new metric type, which allows eliminating raw statistics, leaving equal complexity. +This is a refactoring of the existing proposal to cover more use-cases and arguably reduces API complexity. ## Prior art and alternatives -This Measure Metric API is conceptually close to the Prometheus [Histogram, Summary, and Untyped metric types](https://prometheus.io/docs/concepts/metric_types/). +Prometheus supports the notion of vector metrics, which are those with declared dimensions. The vector-metric API supports a variety of methods like `WithLabelValues` to associate labels with a metric handle, similar to `GetOrCreateTimeSeries` in the existing proposal. As in this proposal, Prometheus supports vector metrics for all metric types. ## Open questions @@ -34,4 +34,4 @@ This RFC is co-dependent on several others; it's an open question how to address ## Future possibilities -This change will potentially help clarify the relationship between Metric types and Aggregation types. In a future RFC, we will propose that Measure Metrics (as introduced here) can be used to support arbitrary "advanced" aggregations including histograms and distribution summaries. +This change will potentially help clarify the relationship between Metric types and Aggregation types. In a future RFC, we will propose that MeasureMetrics can be used to support arbitrary "advanced" aggregations including histograms and distribution summaries. diff --git a/0002-metric-measure.md b/0002-metric-measure.md index 8fb05de76..6562abf6b 100644 --- a/0002-metric-measure.md +++ b/0002-metric-measure.md @@ -4,34 +4,38 @@ Define a new Metric type named "Measure" to cover existing "Raw" statistics uses ## Motivation -The primary motivation is that raw statistics should support the optimization and usability improvements associated with pre-defined label values (0001-metric-pre-defined-labels). By elevating non-Cumulative, non-Gauge statistics to the same conceptual level as Metrics in the API, we effectively make the type of a metric independent from whether it supports pre-defined labels. +The primary motivation is that Raw statistics should support the optimization and usability improvements associated with pre-defined label values (0001-metric-pre-defined-labels). By elevating non-Cumulative, non-Gauge statistics to the same conceptual level as Metrics in the API, we effectively make the type of a metric independent from whether it supports pre-defined labels. This also makes it possible to eliminate the low-level `stats.Record` interface from the API specification entirely (0003-eliminate-stats-record). ## Explanation -This extends the `GetOrCreateTimeSeries` functionality supported by Metrics to what has been known as "Raw" statistics, satisfying the change in capability requested in RFC 0001-metric-pre-defined-labels. This allows programmers to predefined labels for all metrics, regardless of whether they are actually configured for pre-aggregation or not. This is not only a potential optimization for the programmer, it is a usability improvement in the code. +This proposal suggests we think about which aggregations apply to a metric independently from its type. A MeasureMetric could be used to aggregate a Histogram, or a Summary, or _both_ of these aggregations simultaneously. This proposal makes metric type independent of aggregation type, whereas there is a precedent for combining these types into one. -Without raw statistics in the API, it becomes possible to elimiante the low-level `stats.Record` API, which may also be desireable. +The proposal here suggests that we think of the metric type in terms of the _action performed_ (i.e., which _verb_ applies?). Gauges support the `Set` action. Cumulatives support an `Inc` action. Measures support a `Record` action. + +This extends the `GetOrCreateTimeSeries` (pre-defined labels) functionality supported by Metrics to what has been known as Raw statistics, satisfying the change in capability requested in RFC 0001-metric-pre-defined-labels. This allows programmers to predefine labels for all metrics. This is not only an important potential optimization for the programmer, it is a usability improvement in the code. + +There are no new requirements stated in this RFC. ## Internal details -The type known as `MeasureMetric` is a direct replacement for raw statistics. The `MeasureMetric.Record` method records a single observation of the metric. The `MeasureMetric.GetOrCreateTimeSeries` supports pre-defined keys as discussed in 0001-metric-pre-defined-labels. +The type known as `MeasureMetric` is a direct replacement for Raw statistics. The `MeasureMetric.Record` method records a single observation of the metric. The `MeasureMetric.GetOrCreateTimeSeries` supports pre-defined keys as discussed in 0001-metric-pre-defined-labels. ## Trade-offs and mitigations -This change, while it eliminates the need for a raw statistics concept, potentially introduces new required concepts. Whereas Raw statistics have no directly-declared aggregations, introducing MeasureMetric raises the question of which aggregations apply. We will propose how a programmer can declare recommended aggregations (and good defaults) in RFC 0004-configurable-aggregation. +This change, while it eliminates the need for a Raw statistics concept, potentially introduces new required concepts. Whereas Raw statistics have no directly-declared aggregations, introducing MeasureMetric raises the question of which aggregations apply. We will propose how a programmer can declare recommended aggregations (and good defaults) in RFC 0004-configurable-aggregation. ## Prior art and alternatives -The MeasureMetric type introduced here covers a related group of metric types from other systems, including Histogram metric, Summary metric, and Unknown metrics in the Prometheus system. The proposal here suggests that we think of the metric type in terms of the _action performed_ (i.e., which _verb_?). Gauges support the `Set` action. Cumulatives support an `Inc` action. Measures support a `Record` action. - -This proposal suggests we think about which aggregations apply to a metric independently. A MeasureMetric could be used to aggregate a Histogram, or a Summary, or _both_ of these aggregations simultaneously. This proposal makes metric type independent of aggregation type, whereas there is a precedent for combining these types into one. +This Measure Metric API is conceptually close to the Prometheus [Histogram, Summary, and Untyped metric types](https://prometheus.io/docs/concepts/metric_types/). ## Open questions -With this proposal accepted, there would be three Metric types: Gauge, Cumulative, and Measure. This proposal does not directly address what to do over the existing, conflicting uses of "Measure" and "Measurement". +With this proposal accepted, there would be three Metric types: Gauge, Cumulative, and Measure. This proposal does not directly address what to do over the existing, conflicting uses of "Measure". ## Future possibilities This change enables metrics to support configurable aggregation types, which allows the programmer to provide recommended aggregations at the point where Metrics are defined. This will allow support for good out-of-the-box behavior for metrics defined by third-party libraries, for example. + +Without Raw statistics in the API, it becomes possible to elimiante the low-level `stats.Record` API, which may also be desireable. diff --git a/0003-eliminate-stats-record.md b/0003-eliminate-stats-record.md index e9b40d888..c8af75c1f 100644 --- a/0003-eliminate-stats-record.md +++ b/0003-eliminate-stats-record.md @@ -4,15 +4,15 @@ Remove `stats.Record` from the specification, following the MeasureMetric type ( ## Motivation -This change is no longer a necessary interface. There are conceivable reasons to support it, but they are not outweighed by the cost of implementing and supporting two interfaces for recording metrics and statistics. +`stats.Record` is no longer a necessary interface. There are conceivable reasons to support it, but they are outweighed by the cost of implementing and supporting two interfaces for recording metrics and statistics. ## Explanation -In RFC 0002-metric-measure, a new MeasureMetric type is introduced to replace raw statistics, with support for pre-defined label values. With the new type introduced, it's now possible to record formerly-raw statistics through a higher-level Metric interface. stats.Record is no longer a requirement and should be removed until strong evidence of its need emerges from real use-cases. +In RFC 0002-metric-measure, a new MeasureMetric type is introduced to replace raw statistics, with support for pre-defined label values. With the new type introduced, it's now possible to record formerly-raw statistics through a higher-level Metric interface. ## Internal details -This simply involves removing the low-level `stats.Record` API from the specification. +This simply involves removing the low-level `stats.Record` API from the specification, as it is no longer required. ## Trade-offs and mitigations @@ -32,3 +32,5 @@ Are either of the trade-offs described above important enough to keep the low-le ## Future possibilities This restricts future possibilities for the benefit of a smaller, simpler specification. + +This leaves open the possibility of adding `stats.Record` functionality later, when the need is more clearly recognized. \ No newline at end of file diff --git a/0004-metric-configurable-aggregation.md b/0004-metric-configurable-aggregation.md index 4dc43b869..a1659d6a7 100644 --- a/0004-metric-configurable-aggregation.md +++ b/0004-metric-configurable-aggregation.md @@ -13,17 +13,17 @@ This proposal completes the elimination of Raw statistics by recognizing that ag Following this change, we should think of the _Metric type_ as: 1. Indicating something about what kind of numbers are being recorded (i.e., the input domain, e.g., restricted to values >= 0?) - 1. For Gauges: Something pre-computed where rate or count is not relevant - 1. For Cumulatives: Something where rate or count is relevant - 1. For Measures: Something where individual values are relevant + 1. For Gauges: Something pre-computed where rate or count is not relevant + 1. For Cumulatives: Something where rate or count is relevant + 1. For Measures: Something where individual values are relevant 1. Indicating something about the default interpretation, based on the action verb (Set, Inc, Record, etc.) - 1. For Gauges: the action is Set() - 1. For Cumulatives: the action is Inc() - 1. For Measures: the action is Record() + 1. For Gauges: the action is Set() + 1. For Cumulatives: the action is Inc() + 1. For Measures: the action is Record() 1. Unless the programmer declares otherwise, suggesting a default aggregation - 1. For Gauges: LAST_VALUE is interesting, SUM is likely not interesting - 1. For Cumulatives: SUM is interesting, LAST_VALUE is likely not interesting - 1. For Measures: all aggregations apply, default is MIN, MAX, SUM, COUNT. + 1. For Gauges: LAST_VALUE is interesting, SUM is likely not interesting + 1. For Cumulatives: SUM is interesting, LAST_VALUE is likely not interesting + 1. For Measures: all aggregations apply, default is MIN, MAX, SUM, COUNT. ## Internal details @@ -44,6 +44,20 @@ The non-decomposable aggregations do not have standard definitions, they are pur 1. HISTOGRAM: The intended output is a distribution summary, specifically summarizing counts into non-overlapping ranges. 1. SUMMARY: This is a more generic way to request information about a distribution, perhaps represented in some vendor-specific way / not a histogram. +## Example + +To declare a MeasureMetric, + +``` + myMetric := metric.NewMeasureMetric( + "ex.com/mymetric", + metric.WithAggregations(metric.SUM, metric.COUNT), + metric.WithLabelKeys(aKey, bKey)) +) +``` + +Here, we have declared a Measure-type metric with recommended SUM and COUNT aggregations (allowing to compute the average) with `aKey` and `bKey` as recommended aggregation dimensions. While the SDK has full control over which aggregations are actually performed, the programmer has specified a good default behavior for the implementation to use. + ## Trade-offs and mitigations This avoids requiring programmers to use the `view` API, which is an SDK API, not a user-facing instrumentation API. Letting the application programmer recommend aggregations directly gives the implementation more information about the raw statistics. Letting programmers declare their intent has few downsides, since there is a well-defined default behavior. @@ -56,4 +70,4 @@ Existing systems generaly declare separate Metric types according to the desired There are questions about the value of the MIN and MAX aggregations. While they are simple to compute, they are difficult to use in practice. -There are questions about the interpretation of HISTOGRAM and SUMMARY. The point of Raw statistics was that we shouldn't specify these aggregations because they are expensive and many implementations are possible. This is still true. What is the value in specifying HISTOGRAM as opposed to SUMMARY? How is SUMMARY different from MIN/MAX/COUNT/SUM? +There are questions about the interpretation of HISTOGRAM and SUMMARY. The point of Raw statistics was that we shouldn't specify these aggregations because they are expensive and many implementations are possible. This is still true. What is the value in specifying HISTOGRAM as opposed to SUMMARY? How is SUMMARY different from MIN/MAX/COUNT/SUM, does it imply implementation-defined quantiles? From 89a8fef6b1c8cdec10218e2dceb4b56a68139f13 Mon Sep 17 00:00:00 2001 From: Josh MacDonald Date: Tue, 30 Jul 2019 21:52:25 -0700 Subject: [PATCH 06/11] Set status to proposed --- 0001-metric-pre-defined-labels.md | 2 ++ 0002-metric-measure.md | 2 ++ 0003-eliminate-stats-record.md | 2 ++ 0004-metric-configurable-aggregation.md | 2 ++ 4 files changed, 8 insertions(+) diff --git a/0001-metric-pre-defined-labels.md b/0001-metric-pre-defined-labels.md index 52217cf5b..b910d0c8b 100644 --- a/0001-metric-pre-defined-labels.md +++ b/0001-metric-pre-defined-labels.md @@ -1,5 +1,7 @@ # Pre-defined label support for all metric operations +*Status: proposed* + Let all Metric objects (Cumulative, Gauge, ...) and Raw statistics support pre-defined label values. ## Motivation diff --git a/0002-metric-measure.md b/0002-metric-measure.md index 6562abf6b..ddde3c6ea 100644 --- a/0002-metric-measure.md +++ b/0002-metric-measure.md @@ -1,5 +1,7 @@ # Replace Raw statistics with Measure-type Metric +*Status: proposed* + Define a new Metric type named "Measure" to cover existing "Raw" statistics uses. ## Motivation diff --git a/0003-eliminate-stats-record.md b/0003-eliminate-stats-record.md index c8af75c1f..5bdc88157 100644 --- a/0003-eliminate-stats-record.md +++ b/0003-eliminate-stats-record.md @@ -1,5 +1,7 @@ # Eliminate stats.Record functionality +*Status: proposed* + Remove `stats.Record` from the specification, following the MeasureMetric type (RFC 0002-metric-measure). ## Motivation diff --git a/0004-metric-configurable-aggregation.md b/0004-metric-configurable-aggregation.md index a1659d6a7..7988caa5b 100644 --- a/0004-metric-configurable-aggregation.md +++ b/0004-metric-configurable-aggregation.md @@ -1,5 +1,7 @@ # Let Metrics support configrable, recommended aggregations +*Status: proposed* + Let the user configure recommended Metric aggregations (SUM, COUNT, MIN, MAX, LAST_VALUE, HISTOGRAM, SUMMARY). ## Motivation From 3b21a6ea3ddfcfab8143afc822e39b3bc23ca221 Mon Sep 17 00:00:00 2001 From: Josh MacDonald Date: Mon, 12 Aug 2019 13:43:02 -0700 Subject: [PATCH 07/11] Assign numbers 3-4-5-6 --- .../0003-metric-pre-defined-labels.md | 0 0002-metric-measure.md => text/0004-metric-measure.md | 0 .../0005-eliminate-stats-record.md | 0 .../0006-metric-configurable-aggregation.md | 0 4 files changed, 0 insertions(+), 0 deletions(-) rename 0001-metric-pre-defined-labels.md => text/0003-metric-pre-defined-labels.md (100%) rename 0002-metric-measure.md => text/0004-metric-measure.md (100%) rename 0003-eliminate-stats-record.md => text/0005-eliminate-stats-record.md (100%) rename 0004-metric-configurable-aggregation.md => text/0006-metric-configurable-aggregation.md (100%) diff --git a/0001-metric-pre-defined-labels.md b/text/0003-metric-pre-defined-labels.md similarity index 100% rename from 0001-metric-pre-defined-labels.md rename to text/0003-metric-pre-defined-labels.md diff --git a/0002-metric-measure.md b/text/0004-metric-measure.md similarity index 100% rename from 0002-metric-measure.md rename to text/0004-metric-measure.md diff --git a/0003-eliminate-stats-record.md b/text/0005-eliminate-stats-record.md similarity index 100% rename from 0003-eliminate-stats-record.md rename to text/0005-eliminate-stats-record.md diff --git a/0004-metric-configurable-aggregation.md b/text/0006-metric-configurable-aggregation.md similarity index 100% rename from 0004-metric-configurable-aggregation.md rename to text/0006-metric-configurable-aggregation.md From 69671c402a784da31086541ebd49880428747a0a Mon Sep 17 00:00:00 2001 From: Josh MacDonald Date: Mon, 12 Aug 2019 13:46:11 -0700 Subject: [PATCH 08/11] Renumber refs --- text/0003-metric-pre-defined-labels.md | 2 +- text/0004-metric-measure.md | 10 +++++----- text/0005-eliminate-stats-record.md | 6 +++--- text/0006-metric-configurable-aggregation.md | 2 +- 4 files changed, 10 insertions(+), 10 deletions(-) diff --git a/text/0003-metric-pre-defined-labels.md b/text/0003-metric-pre-defined-labels.md index b910d0c8b..3021bf072 100644 --- a/text/0003-metric-pre-defined-labels.md +++ b/text/0003-metric-pre-defined-labels.md @@ -20,7 +20,7 @@ For example, where the application wants to compute a histogram of some value (e ## Internal details -This RFC is accompanied by RFC 0002-metric-measure which proposes to create a new Metric type to replace Raw statistics. The metric type, named "Measure", would replace the existing concept and type named "Measure" in the metrics API. The new MeasureMetric object would support a `Record` method to record measurements. +This RFC is accompanied by RFC 0004-metric-measure which proposes to create a new Metric type to replace Raw statistics. The metric type, named "Measure", would replace the existing concept and type named "Measure" in the metrics API. The new MeasureMetric object would support a `Record` method to record measurements. ## Trade-offs and mitigations diff --git a/text/0004-metric-measure.md b/text/0004-metric-measure.md index ddde3c6ea..737e06226 100644 --- a/text/0004-metric-measure.md +++ b/text/0004-metric-measure.md @@ -6,9 +6,9 @@ Define a new Metric type named "Measure" to cover existing "Raw" statistics uses ## Motivation -The primary motivation is that Raw statistics should support the optimization and usability improvements associated with pre-defined label values (0001-metric-pre-defined-labels). By elevating non-Cumulative, non-Gauge statistics to the same conceptual level as Metrics in the API, we effectively make the type of a metric independent from whether it supports pre-defined labels. +The primary motivation is that Raw statistics should support the optimization and usability improvements associated with pre-defined label values (0003-metric-pre-defined-labels). By elevating non-Cumulative, non-Gauge statistics to the same conceptual level as Metrics in the API, we effectively make the type of a metric independent from whether it supports pre-defined labels. -This also makes it possible to eliminate the low-level `stats.Record` interface from the API specification entirely (0003-eliminate-stats-record). +This also makes it possible to eliminate the low-level `stats.Record` interface from the API specification entirely (0005-eliminate-stats-record). ## Explanation @@ -16,17 +16,17 @@ This proposal suggests we think about which aggregations apply to a metric indep The proposal here suggests that we think of the metric type in terms of the _action performed_ (i.e., which _verb_ applies?). Gauges support the `Set` action. Cumulatives support an `Inc` action. Measures support a `Record` action. -This extends the `GetOrCreateTimeSeries` (pre-defined labels) functionality supported by Metrics to what has been known as Raw statistics, satisfying the change in capability requested in RFC 0001-metric-pre-defined-labels. This allows programmers to predefine labels for all metrics. This is not only an important potential optimization for the programmer, it is a usability improvement in the code. +This extends the `GetOrCreateTimeSeries` (pre-defined labels) functionality supported by Metrics to what has been known as Raw statistics, satisfying the change in capability requested in RFC 0003-metric-pre-defined-labels. This allows programmers to predefine labels for all metrics. This is not only an important potential optimization for the programmer, it is a usability improvement in the code. There are no new requirements stated in this RFC. ## Internal details -The type known as `MeasureMetric` is a direct replacement for Raw statistics. The `MeasureMetric.Record` method records a single observation of the metric. The `MeasureMetric.GetOrCreateTimeSeries` supports pre-defined keys as discussed in 0001-metric-pre-defined-labels. +The type known as `MeasureMetric` is a direct replacement for Raw statistics. The `MeasureMetric.Record` method records a single observation of the metric. The `MeasureMetric.GetOrCreateTimeSeries` supports pre-defined keys as discussed in 0003-metric-pre-defined-labels. ## Trade-offs and mitigations -This change, while it eliminates the need for a Raw statistics concept, potentially introduces new required concepts. Whereas Raw statistics have no directly-declared aggregations, introducing MeasureMetric raises the question of which aggregations apply. We will propose how a programmer can declare recommended aggregations (and good defaults) in RFC 0004-configurable-aggregation. +This change, while it eliminates the need for a Raw statistics concept, potentially introduces new required concepts. Whereas Raw statistics have no directly-declared aggregations, introducing MeasureMetric raises the question of which aggregations apply. We will propose how a programmer can declare recommended aggregations (and good defaults) in RFC 0006-configurable-aggregation. ## Prior art and alternatives diff --git a/text/0005-eliminate-stats-record.md b/text/0005-eliminate-stats-record.md index 5bdc88157..e6ef4c54e 100644 --- a/text/0005-eliminate-stats-record.md +++ b/text/0005-eliminate-stats-record.md @@ -2,7 +2,7 @@ *Status: proposed* -Remove `stats.Record` from the specification, following the MeasureMetric type (RFC 0002-metric-measure). +Remove `stats.Record` from the specification, following the MeasureMetric type (RFC 0004-metric-measure). ## Motivation @@ -10,7 +10,7 @@ Remove `stats.Record` from the specification, following the MeasureMetric type ( ## Explanation -In RFC 0002-metric-measure, a new MeasureMetric type is introduced to replace raw statistics, with support for pre-defined label values. With the new type introduced, it's now possible to record formerly-raw statistics through a higher-level Metric interface. +In RFC 0004-metric-measure, a new MeasureMetric type is introduced to replace raw statistics, with support for pre-defined label values. With the new type introduced, it's now possible to record formerly-raw statistics through a higher-level Metric interface. ## Internal details @@ -25,7 +25,7 @@ There are two reasons to maintain a low-level API that we know of: ## Prior art and alternatives -Raw statistics were a solution to confusion found in existing metrics APIs over Metric types vs. Aggregation types. This proposal accompanies RFC 0001-metric-pre-defined-labels and RFC 0002-metric-measure.md in proposing that we think about Metric _type_ as independent of which aggregations apply. Once we have a Metric to support histogram and summary aggregations, we no longer need raw statistics, and we no longer need `stats.Record`. This avoids introducing new concepts (Raw statistics), at the same time departs from prior art in letting one Metric type support both Histogram and Summary aggregations. +Raw statistics were a solution to confusion found in existing metrics APIs over Metric types vs. Aggregation types. This proposal accompanies RFC 0003-metric-pre-defined-labels and RFC 0004-metric-measure.md in proposing that we think about Metric _type_ as independent of which aggregations apply. Once we have a Metric to support histogram and summary aggregations, we no longer need raw statistics, and we no longer need `stats.Record`. This avoids introducing new concepts (Raw statistics), at the same time departs from prior art in letting one Metric type support both Histogram and Summary aggregations. ## Open questions diff --git a/text/0006-metric-configurable-aggregation.md b/text/0006-metric-configurable-aggregation.md index 7988caa5b..352ece75c 100644 --- a/text/0006-metric-configurable-aggregation.md +++ b/text/0006-metric-configurable-aggregation.md @@ -6,7 +6,7 @@ Let the user configure recommended Metric aggregations (SUM, COUNT, MIN, MAX, LA ## Motivation -In the current API proposal, Metric types like Gauge and Cumulative are mapped into specific aggregations: Gauge:LAST_VALUE and Cumulative:SUM. Depending on RFC 0002-metric-measure, which creates a new MeasureMetric type, this proposal introduces the ability to configure alternative, potentially multiple aggregations for Metrics. This allows the MeasureMetric type to support HISTOGRAM and SUMMARY aggregations, as an alternative to raw statistics. +In the current API proposal, Metric types like Gauge and Cumulative are mapped into specific aggregations: Gauge:LAST_VALUE and Cumulative:SUM. Depending on RFC 0004-metric-measure, which creates a new MeasureMetric type, this proposal introduces the ability to configure alternative, potentially multiple aggregations for Metrics. This allows the MeasureMetric type to support HISTOGRAM and SUMMARY aggregations, as an alternative to raw statistics. ## Explanation From b938b3ac217283b2d94cd3524a03440cc3dfa5ea Mon Sep 17 00:00:00 2001 From: Josh MacDonald Date: Mon, 12 Aug 2019 13:47:27 -0700 Subject: [PATCH 09/11] Update status styling --- text/0003-metric-pre-defined-labels.md | 2 +- text/0004-metric-measure.md | 2 +- text/0005-eliminate-stats-record.md | 2 +- text/0006-metric-configurable-aggregation.md | 2 +- 4 files changed, 4 insertions(+), 4 deletions(-) diff --git a/text/0003-metric-pre-defined-labels.md b/text/0003-metric-pre-defined-labels.md index 3021bf072..9d7676b2b 100644 --- a/text/0003-metric-pre-defined-labels.md +++ b/text/0003-metric-pre-defined-labels.md @@ -1,6 +1,6 @@ # Pre-defined label support for all metric operations -*Status: proposed* +**Status:** `proposed` Let all Metric objects (Cumulative, Gauge, ...) and Raw statistics support pre-defined label values. diff --git a/text/0004-metric-measure.md b/text/0004-metric-measure.md index 737e06226..ac088da68 100644 --- a/text/0004-metric-measure.md +++ b/text/0004-metric-measure.md @@ -1,6 +1,6 @@ # Replace Raw statistics with Measure-type Metric -*Status: proposed* +**Status:** `proposed` Define a new Metric type named "Measure" to cover existing "Raw" statistics uses. diff --git a/text/0005-eliminate-stats-record.md b/text/0005-eliminate-stats-record.md index e6ef4c54e..d9762c554 100644 --- a/text/0005-eliminate-stats-record.md +++ b/text/0005-eliminate-stats-record.md @@ -1,6 +1,6 @@ # Eliminate stats.Record functionality -*Status: proposed* +**Status:** `proposed` Remove `stats.Record` from the specification, following the MeasureMetric type (RFC 0004-metric-measure). diff --git a/text/0006-metric-configurable-aggregation.md b/text/0006-metric-configurable-aggregation.md index 352ece75c..a63be9879 100644 --- a/text/0006-metric-configurable-aggregation.md +++ b/text/0006-metric-configurable-aggregation.md @@ -1,6 +1,6 @@ # Let Metrics support configrable, recommended aggregations -*Status: proposed* +**Status:** `proposed` Let the user configure recommended Metric aggregations (SUM, COUNT, MIN, MAX, LAST_VALUE, HISTOGRAM, SUMMARY). From c28d3bca3e85b4ed6021edd5b1d707100e112e47 Mon Sep 17 00:00:00 2001 From: Josh MacDonald Date: Mon, 12 Aug 2019 13:49:54 -0700 Subject: [PATCH 10/11] Fix misspellings --- text/0004-metric-measure.md | 2 +- text/0006-metric-configurable-aggregation.md | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-) diff --git a/text/0004-metric-measure.md b/text/0004-metric-measure.md index ac088da68..8ee0ccf38 100644 --- a/text/0004-metric-measure.md +++ b/text/0004-metric-measure.md @@ -40,4 +40,4 @@ With this proposal accepted, there would be three Metric types: Gauge, Cumulativ This change enables metrics to support configurable aggregation types, which allows the programmer to provide recommended aggregations at the point where Metrics are defined. This will allow support for good out-of-the-box behavior for metrics defined by third-party libraries, for example. -Without Raw statistics in the API, it becomes possible to elimiante the low-level `stats.Record` API, which may also be desireable. +Without Raw statistics in the API, it becomes possible to elimiante the low-level `stats.Record` API, which may also be desirable. diff --git a/text/0006-metric-configurable-aggregation.md b/text/0006-metric-configurable-aggregation.md index a63be9879..5d6bebbce 100644 --- a/text/0006-metric-configurable-aggregation.md +++ b/text/0006-metric-configurable-aggregation.md @@ -66,7 +66,7 @@ This avoids requiring programmers to use the `view` API, which is an SDK API, no ## Prior art and alternatives -Existing systems generaly declare separate Metric types according to the desired aggregation. Raw statistics were invented to overcome this, and the present proposal brings back the ability to specify an Aggregation at the point where a Metric is defined. +Existing systems generally declare separate Metric types according to the desired aggregation. Raw statistics were invented to overcome this, and the present proposal brings back the ability to specify an Aggregation at the point where a Metric is defined. ## Open questions From bc2ce9199148896e0ed66cfd07ef0e5af66cf20f Mon Sep 17 00:00:00 2001 From: Josh MacDonald Date: Mon, 12 Aug 2019 17:16:52 -0700 Subject: [PATCH 11/11] Combine the first three into one --- text/0003-measure-metric-type.md | 51 +++++++++++++++++++ text/0003-metric-pre-defined-labels.md | 39 -------------- ...> 0004-metric-configurable-aggregation.md} | 2 +- text/0004-metric-measure.md | 43 ---------------- text/0005-eliminate-stats-record.md | 38 -------------- 5 files changed, 52 insertions(+), 121 deletions(-) create mode 100644 text/0003-measure-metric-type.md delete mode 100644 text/0003-metric-pre-defined-labels.md rename text/{0006-metric-configurable-aggregation.md => 0004-metric-configurable-aggregation.md} (93%) delete mode 100644 text/0004-metric-measure.md delete mode 100644 text/0005-eliminate-stats-record.md diff --git a/text/0003-measure-metric-type.md b/text/0003-measure-metric-type.md new file mode 100644 index 000000000..916e67dd4 --- /dev/null +++ b/text/0003-measure-metric-type.md @@ -0,0 +1,51 @@ +# Consolidate pre-aggregated and raw metrics APIs + +**Status:** `proposed` + +## Forward + +This propsal was originally split into three semi-related parts. Based on the feedback, they are now combined here into a single proposal. The original proposals were: + + 000x-metric-pre-defined-labels + 000x-metric-measure + 000x-eliminate-stats-record + +## Overview + +Introduce a `Measure` type of metric object that supports a `Record` API. Like existing `Gauge` and `Cumulative` metrics, the new `Measure` metric supports pre-defined labels. A new measurement batch API is introduced for recording multiple metric observations simultaneously. + +## Motivation + +In the current `Metric.GetOrCreateTimeSeries` API for Gauges and Cumulatives, the caller obtains a `TimeSeries` handle for repeatedly recording metrics with certain pre-defined label values set. This is an important optimization, especially for exporting aggregated metrics. + +The use of pre-defined labels improves usability too, for working with metrics in code. Application programs with long-lived objects and associated Metrics can compute predefined label values once (e.g., in a constructor), rather than once per call site. + +The current raw statistics API does not support pre-defined labels. This RFC replaces the raw statistics API by a new, general-purpose type of metric, `MeasureMetric`, generally intended for recording individual measurements the way raw statistics did, with added support for pre-defined labels. + +The former raw statistics API supported all-or-none recording for interdependent measurements. This RFC introduces a `MeasurementBatch` to support recording batches of metric observations. + +## Explanation + +In the current proposal, Metrics are used for pre-aggregated metric types, whereas Raw statistics are used for uncommon and vendor-specific aggregations. The optimization and the usability advantages gained with pre-defined labels should be extended to Raw statistics because they are equally important and equally applicable. This is a new requirement. + +For example, where the application wants to compute a histogram of some value (e.g., latency), there's good reason to pre-aggregate such information. In this example, it allows an implementation to effienctly export the histogram of latencies "grouped" into individual results by label value(s). + +The new `MeasureMetric` API satisfies the requirements of a single-argument call to record raw statistics, but the raw statistics API had secondary purpose, that of supporting recording multiple observed values simultaneously. This proposal introduces a `MeasurementBatch` API to record multiple metric observations in a single call. + +## Internal details + +The type known as `MeasureMetric` is a direct replacement for the raw statistics `Measure` type. The `MeasureMetric.Record` method records a single observation of the metric. The `MeasureMetric.GetOrCreateTimeSeries` supports pre-defined labels, just the same as `Gauge` and `Cumulative` metrics. + +## Trade-offs and mitigations + +This Measure Metric API is conceptually close to the Prometheus [Histogram, Summary, and Untyped metric types](https://prometheus.io/docs/concepts/metric_types/), but there is no way in OpenTelemetry to distinguish these cases at the declaration site, in code. This topic is covered in 0004-metric-configurable-aggregation. + +## Prior art and alternatives + +Prometheus supports the notion of vector metrics, which are those which support pre-defined labels. The vector-metric API supports a variety of methods like `WithLabelValues` to associate labels with a metric handle, similar to `GetOrCreateTimeSeries` in OpenTelemetry. As in this proposal, Prometheus supports a vector API for all metric types. + +## Open questions + +Argument ordering has been proposed as the way to pass pre-defined label values in `GetOrCreateTimeseries`. The argument list must match the parameter list exactly, and if it doesn't we generally find out at runtime or not at all. This model has more optimization potential, but is easier to misuse, than the alternative. The alternative approach is to always pass label:value pairs to `GetOrCreateTimeseries`, as opposed to an ordered list of values. + +The same discussion can be had for the `MeasurementBatch` type described here. It can be declared with an ordered list of metrics, then the `Record` API takes only an ordered list of numbers. Alternatively, and less prone to misuse, the `MeasurementBatch.Record` API could be declared with a list of metric:number pairs. diff --git a/text/0003-metric-pre-defined-labels.md b/text/0003-metric-pre-defined-labels.md deleted file mode 100644 index 9d7676b2b..000000000 --- a/text/0003-metric-pre-defined-labels.md +++ /dev/null @@ -1,39 +0,0 @@ -# Pre-defined label support for all metric operations - -**Status:** `proposed` - -Let all Metric objects (Cumulative, Gauge, ...) and Raw statistics support pre-defined label values. - -## Motivation - -In the current `Metric.GetOrCreateTimeSeries` API for Gauges and Cumulatives, the caller obtains a `TimeSeries` handle for repeatedly recording metrics with certain pre-defined label values set. This is an important optimization, especially for exporting aggregated metrics. - -The use of pre-defined labels improves usability too, for working with metrics in code. Application programs with long-lived objects and associated Metrics can compute predefined label values once (e.g., in a constructor), rather than once per call site. - -The current API for recording Raw statistics does not support the same optimization or usability advantage. This RFC proposes to add support for pre-defined labels on all metrics. - -## Explanation - -In the current proposal, Metrics are used for pre-aggregated metric types, whereas Raw statistics are used for uncommon and vendor-specific aggregations. The optimization and the usability advantages gained with pre-defined labels should be extended to Raw statistics because they are equally important and equally applicable. This is a new requirement. - -For example, where the application wants to compute a histogram of some value (e.g., latency), there's good reason to pre-aggregate such information. In this example, it allows an implementation to effienctly export the histogram of latencies "grouped" into individual results by label value(s). - -## Internal details - -This RFC is accompanied by RFC 0004-metric-measure which proposes to create a new Metric type to replace Raw statistics. The metric type, named "Measure", would replace the existing concept and type named "Measure" in the metrics API. The new MeasureMetric object would support a `Record` method to record measurements. - -## Trade-offs and mitigations - -This is a refactoring of the existing proposal to cover more use-cases and arguably reduces API complexity. - -## Prior art and alternatives - -Prometheus supports the notion of vector metrics, which are those with declared dimensions. The vector-metric API supports a variety of methods like `WithLabelValues` to associate labels with a metric handle, similar to `GetOrCreateTimeSeries` in the existing proposal. As in this proposal, Prometheus supports vector metrics for all metric types. - -## Open questions - -This RFC is co-dependent on several others; it's an open question how to address this concern if the other RFCs are not accepted. - -## Future possibilities - -This change will potentially help clarify the relationship between Metric types and Aggregation types. In a future RFC, we will propose that MeasureMetrics can be used to support arbitrary "advanced" aggregations including histograms and distribution summaries. diff --git a/text/0006-metric-configurable-aggregation.md b/text/0004-metric-configurable-aggregation.md similarity index 93% rename from text/0006-metric-configurable-aggregation.md rename to text/0004-metric-configurable-aggregation.md index 5d6bebbce..c5b6abb96 100644 --- a/text/0006-metric-configurable-aggregation.md +++ b/text/0004-metric-configurable-aggregation.md @@ -6,7 +6,7 @@ Let the user configure recommended Metric aggregations (SUM, COUNT, MIN, MAX, LA ## Motivation -In the current API proposal, Metric types like Gauge and Cumulative are mapped into specific aggregations: Gauge:LAST_VALUE and Cumulative:SUM. Depending on RFC 0004-metric-measure, which creates a new MeasureMetric type, this proposal introduces the ability to configure alternative, potentially multiple aggregations for Metrics. This allows the MeasureMetric type to support HISTOGRAM and SUMMARY aggregations, as an alternative to raw statistics. +In the current API proposal, Metric types like Gauge and Cumulative are mapped into specific aggregations: Gauge:LAST_VALUE and Cumulative:SUM. Depending on RFC 0003-measure-metric-type, which creates a new MeasureMetric type, this proposal introduces the ability to configure alternative, potentially multiple aggregations for Metrics. This allows the MeasureMetric type to support HISTOGRAM and SUMMARY aggregations, as an alternative to raw statistics. ## Explanation diff --git a/text/0004-metric-measure.md b/text/0004-metric-measure.md deleted file mode 100644 index 8ee0ccf38..000000000 --- a/text/0004-metric-measure.md +++ /dev/null @@ -1,43 +0,0 @@ -# Replace Raw statistics with Measure-type Metric - -**Status:** `proposed` - -Define a new Metric type named "Measure" to cover existing "Raw" statistics uses. - -## Motivation - -The primary motivation is that Raw statistics should support the optimization and usability improvements associated with pre-defined label values (0003-metric-pre-defined-labels). By elevating non-Cumulative, non-Gauge statistics to the same conceptual level as Metrics in the API, we effectively make the type of a metric independent from whether it supports pre-defined labels. - -This also makes it possible to eliminate the low-level `stats.Record` interface from the API specification entirely (0005-eliminate-stats-record). - -## Explanation - -This proposal suggests we think about which aggregations apply to a metric independently from its type. A MeasureMetric could be used to aggregate a Histogram, or a Summary, or _both_ of these aggregations simultaneously. This proposal makes metric type independent of aggregation type, whereas there is a precedent for combining these types into one. - -The proposal here suggests that we think of the metric type in terms of the _action performed_ (i.e., which _verb_ applies?). Gauges support the `Set` action. Cumulatives support an `Inc` action. Measures support a `Record` action. - -This extends the `GetOrCreateTimeSeries` (pre-defined labels) functionality supported by Metrics to what has been known as Raw statistics, satisfying the change in capability requested in RFC 0003-metric-pre-defined-labels. This allows programmers to predefine labels for all metrics. This is not only an important potential optimization for the programmer, it is a usability improvement in the code. - -There are no new requirements stated in this RFC. - -## Internal details - -The type known as `MeasureMetric` is a direct replacement for Raw statistics. The `MeasureMetric.Record` method records a single observation of the metric. The `MeasureMetric.GetOrCreateTimeSeries` supports pre-defined keys as discussed in 0003-metric-pre-defined-labels. - -## Trade-offs and mitigations - -This change, while it eliminates the need for a Raw statistics concept, potentially introduces new required concepts. Whereas Raw statistics have no directly-declared aggregations, introducing MeasureMetric raises the question of which aggregations apply. We will propose how a programmer can declare recommended aggregations (and good defaults) in RFC 0006-configurable-aggregation. - -## Prior art and alternatives - -This Measure Metric API is conceptually close to the Prometheus [Histogram, Summary, and Untyped metric types](https://prometheus.io/docs/concepts/metric_types/). - -## Open questions - -With this proposal accepted, there would be three Metric types: Gauge, Cumulative, and Measure. This proposal does not directly address what to do over the existing, conflicting uses of "Measure". - -## Future possibilities - -This change enables metrics to support configurable aggregation types, which allows the programmer to provide recommended aggregations at the point where Metrics are defined. This will allow support for good out-of-the-box behavior for metrics defined by third-party libraries, for example. - -Without Raw statistics in the API, it becomes possible to elimiante the low-level `stats.Record` API, which may also be desirable. diff --git a/text/0005-eliminate-stats-record.md b/text/0005-eliminate-stats-record.md deleted file mode 100644 index d9762c554..000000000 --- a/text/0005-eliminate-stats-record.md +++ /dev/null @@ -1,38 +0,0 @@ -# Eliminate stats.Record functionality - -**Status:** `proposed` - -Remove `stats.Record` from the specification, following the MeasureMetric type (RFC 0004-metric-measure). - -## Motivation - -`stats.Record` is no longer a necessary interface. There are conceivable reasons to support it, but they are outweighed by the cost of implementing and supporting two interfaces for recording metrics and statistics. - -## Explanation - -In RFC 0004-metric-measure, a new MeasureMetric type is introduced to replace raw statistics, with support for pre-defined label values. With the new type introduced, it's now possible to record formerly-raw statistics through a higher-level Metric interface. - -## Internal details - -This simply involves removing the low-level `stats.Record` API from the specification, as it is no longer required. - -## Trade-offs and mitigations - -There are two reasons to maintain a low-level API that we know of: - -1. For _generality_. An application that forwards metrics from another source may need to handle metrics in generic code. For these applications, having type-specific Metric handles could actually require more code to be written, whereas the low-level `stats.Record` API is more amenable to generic use. -1. For _atomicity_. An application that wishes to record multiple statistics in a single operation can feel confident computing formulas based on multiple metrics, not worry about inconsistent views of the data. - -## Prior art and alternatives - -Raw statistics were a solution to confusion found in existing metrics APIs over Metric types vs. Aggregation types. This proposal accompanies RFC 0003-metric-pre-defined-labels and RFC 0004-metric-measure.md in proposing that we think about Metric _type_ as independent of which aggregations apply. Once we have a Metric to support histogram and summary aggregations, we no longer need raw statistics, and we no longer need `stats.Record`. This avoids introducing new concepts (Raw statistics), at the same time departs from prior art in letting one Metric type support both Histogram and Summary aggregations. - -## Open questions - -Are either of the trade-offs described above important enough to keep the low-level `stats.Record` API? - -## Future possibilities - -This restricts future possibilities for the benefit of a smaller, simpler specification. - -This leaves open the possibility of adding `stats.Record` functionality later, when the need is more clearly recognized. \ No newline at end of file