Skip to content

Expose ExemplarReservoir as configuration option of View. #3820

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
carlosalberto merged 13 commits into from
Jan 31, 2024
Merged

Expose ExemplarReservoir as configuration option of View. #3820

Show file tree
Hide file tree
Changes from 9 commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
61b590f
Add requirement to allow SDK users to provide their own ExemplarReser…
jsuereth Jan 12, 2024
500a25a
Updated compliance matrix for exemplars.
jsuereth Jan 12, 2024
7793ba7
Update changelog.
jsuereth Jan 12, 2024
32c43de
Regenerate TOC
jsuereth Jan 12, 2024
e35bba3
Make it clear exemplars are still feature freeze.
jsuereth Jan 16, 2024
2580724
Remove unecessary part of specification.
jsuereth Jan 16, 2024
792aa84
Fixes #3812 - ExemplarFilter does not need to be exposed as an interf…
jsuereth Jan 17, 2024
8bc8cf6
Regenerate toc.
jsuereth Jan 17, 2024
d881c52
Fix markdownlint.
jsuereth Jan 17, 2024
612034d
Add exemplar-filter to changelog.
jsuereth Jan 19, 2024
d7d4324
Fixes from review.
jsuereth Jan 19, 2024
1240e8a
Merge branch 'main' into wip-exemplar-in-view
jsuereth Jan 25, 2024
eb5096e
Merge branch 'main' into wip-exemplar-in-view
jsuereth Jan 31, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,9 @@ release.

### Metrics

- Expose `ExemplarReservoir` as configuration parameter for views.
([#3820](https://github.com/open-telemetry/opentelemetry-specification/pull/3820))

### Logs

### Resource
Expand Down
8 changes: 4 additions & 4 deletions spec-compliance-matrix.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -201,17 +201,17 @@ formats is required. Implementing more than one format is optional.
| The metrics Exporter `shutdown` function do not block indefinitely. | | + | + | + | - | | | | | + | + | |
| The metrics SDK samples `Exemplar`s from measurements. | | | + | | - | | | | | | - | |
| Exemplar sampling can be disabled. | | | - | | - | | | | | | - | |
| The metrics SDK samples measurements in the context of a sampled trace by default. | | | + | | - | | | | | | - | |
| The metrics SDK supports SDK-wide exemplar filter configuration | | | + | | - | | | | | | - | |
| The metrics SDK supports `TraceBased` exemplar filter | | | + | | - | | | | | | - | |
| The metrics SDK supports `AlwaysOn` exemplar filter | | | + | | - | | | | | | - | |
| The metrics SDK supports `AlwaysOff` exemplar filter | | | + | | - | | | | | | - | |
| Exemplars retain any attributes available in the measurement that are not preserved by aggregation or view configuration. | | | + | | - | | | | | | - | |
| Exemplars contain the associated trace id and span id of the active span in the Context when the measurement was taken. | | | + | | - | | | | | | - | |
| Exemplars contain the timestamp when the measurement was taken. | | | + | | - | | | | | | - | |
| The metrics SDK provides an `ExemplarReservoir` interface or extension point. | X | | - | | - | | | + | | | - | |
| An `ExemplarReservoir` has an `offer` method with access to the measurement value, attributes, `Context` and timestamp. | X | | - | | - | | | + | | | - | |
| The metrics SDK provides a `SimpleFixedSizeExemplarReservoir` that is used by default for all aggregations except `ExplicitBucketHistogram`. | | | + | | - | | | + | | | - | |
| The default size of the `SimpleFixedSizeExemplarReservoir` is `1`. | | | | | | | | | | | | |
| The metrics SDK provides an `AlignedHistogramBucketExemplarReservoir` that is used by default for `ExplicitBucketHistogram` aggregation. | | | + | | - | | | | | | - | |
| The metrics SDK provides an `ExemplarFilter` interface or extension point. | X | | - | | - | | | + | | | - | |
| An `ExemplarFilter` has access to the measurement value, attributes, `Context` and timestamp. | X | | - | | - | | | + | | | - | |
| A metric Producer accepts an optional metric Filter | | | | | | | | | | | | |
| The metric Reader implementation supports registering metric Filter and passing them its registered metric Producers | | | | | | | | | | | | |
| The metric SDK's metric Producer implementations uses the metric Filter | | | | | | | | | | | | |
Expand Down
63 changes: 30 additions & 33 deletions specification/metrics/sdk.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,15 +50,14 @@ linkTitle: SDK
- [Attribute limits](#attribute-limits)
- [Exemplar](#exemplar)
* [ExemplarFilter](#exemplarfilter)
* [Built-in ExemplarFilters](#built-in-exemplarfilters)
+ [AlwaysOn](#alwayson)
+ [AlwaysOff](#alwaysoff)
+ [TraceBased](#tracebased)
+ [Configuration](#configuration-2)
* [ExemplarReservoir](#exemplarreservoir)
* [Exemplar defaults](#exemplar-defaults)
+ [SimpleFixedSizeExemplarReservoir](#simplefixedsizeexemplarreservoir)
+ [AlignedHistogramBucketExemplarReservoir](#alignedhistogrambucketexemplarreservoir)
* [Custom ExemplarReservoir](#custom-exemplarreservoir)
- [MetricReader](#metricreader)
* [MetricReader operations](#metricreader-operations)
+ [Collect](#collect)
Expand Down Expand Up @@ -344,7 +343,7 @@ The SDK MUST accept the following stream configuration parameters:
user does not provide an `aggregation` value, the `MeterProvider` MUST apply
a [default aggregation](#default-aggregation) configurable on the basis of
instrument type according to the [MetricReader](#metricreader) instance.
* **Status**: [Feature-freeze](../document-status.md) - `exemplar_reservoir`: A
* **Status**: [Experimental, Feature-freeze](../document-status.md) - `exemplar_reservoir`: A
functional type that generates an exemplar reservoir a `MeterProvider` will
use when storing exemplars. This functional type needs to be a factory or
callback similar to aggregation selection functionality which allows
Expand Down Expand Up @@ -500,10 +499,6 @@ meterProviderBuilder
);
```

TODO: after we release the initial Stable version of Metrics SDK specification,
we will explore how to allow configuring custom
[ExemplarReservoir](#exemplarreservoir)s with the [View](#view) API.

The SDK MUST provide the following `Aggregation` to support the
[Metric Points](./data-model.md#metric-points) in the
[Metrics Data Model](./data-model.md).
Expand Down Expand Up @@ -882,7 +877,7 @@ series and the topic requires further analysis.

## Exemplar

**Status**: [Feature-freeze](../document-status.md)
**Status**: [Experimental, Feature-freeze](../document-status.md)

Exemplars are example data points for aggregated data. They provide specific
context to otherwise general aggregations. Exemplars allow correlation between
Expand Down Expand Up @@ -937,31 +932,27 @@ A Metric SDK MUST allow exemplar sampling to leverage the configuration of
metric aggregation. For example, Exemplar sampling of histograms should be able
to leverage bucket boundaries.

A Metric SDK SHOULD provide extensible hooks for Exemplar sampling, specifically:
A Metric SDK SHOULD provide configuration for Exemplar sampling, specifically:

- `ExemplarFilter`: filter which measurements can become exemplars.
- `ExemplarReservoir`: storage and sampling of exemplars.

### ExemplarFilter

The `ExemplarFilter` interface MUST provide a method to determine if a
measurement should be sampled. Sampled here simply makes the measurement
eligible for being included as an exemplar. `ExemplarReservoir` makes the final
decision if a measurement becomes an exemplar.

This interface SHOULD have access to:
The `ExemplarFilter` configuration MUST allow users to select between one of the
built-in ExemplarFilters. While `ExemplarFilter` determines which measurements
are _eligible_ for becoming an `Exemplar`, the `ExemplarReservoir` makes the
final decision if a measurement becomes an exemplar and is stored.

- The `value` of the measurement.
- The complete set of `Attributes` of the measurement.
- The [Context](../context/README.md) of the measurement, which covers the
[Baggage](../baggage/api.md) and the current active
[Span](../trace/api.md#span).
- A `timestamp` that best represents when the measurement was taken.
The ExemplarFilter SHOULD be a configuration parameter of a `MeterProvider` for
an SDK. The default value SHOULD be `TraceBased`. The filter configuration
SHOULD follow the [environment variable specification](../configuration/sdk-environment-variables.md#exemplar).

### Built-in ExemplarFilters
An OpenTelemetry SDK MUST support the following filters:

OpenTelemetry supports a number of built-in exemplar filters to choose from.
The default is `TraceBased`.
- [AlwaysOn](#alwayson)
- [AlwaysOff](#alwaysoff)
- [TraceBased](#tracebased)

#### AlwaysOn

Expand All @@ -977,12 +968,6 @@ Using this ExemplarFilter is as good as disabling Exemplar feature.
An ExemplarFilter which makes those measurements eligible for being an
Exemplar, which are recorded in the context of a sampled parent span.

#### Configuration

The ExemplarFilter SHOULD be a configuration parameter of a `MeterProvider` for
an SDK. The default value SHOULD be `TraceBased`. The filter configuration
SHOULD follow the [environment variable specification](../configuration/sdk-environment-variables.md#exemplar).

### ExemplarReservoir

The `ExemplarReservoir` interface MUST provide a method to offer measurements
Expand Down Expand Up @@ -1033,7 +1018,7 @@ The `ExemplarReservoir` SHOULD avoid allocations when sampling exemplars.

### Exemplar defaults

The SDK SHOULD include two types of built-in exemplar reservoirs:
The SDK MUST include two types of built-in exemplar reservoirs:

1. `SimpleFixedSizeExemplarReservoir`
2. `AlignedHistogramBucketExemplarReservoir`
Expand Down Expand Up @@ -1097,6 +1082,20 @@ measurements using the equivalent of the following naive algorithm:
return boundaries.length
```

This Exemplar reservoir MAY take a configuration parameter for the bucket
boundaries used by the reserovoir. The size of the reservoir is always the
number of bucket boundaries plus one. This configuration parameter SHOULD have
the same format as specifying bucket boundaries to
[Explicit Bucket Histogram Aggregation](./sdk.md#explicit-bucket-histogram-aggregation).

### Custom ExemplarReservoir

The SDK SHOULD provide a mechanism for SDK users to provide their own
ExemplarReservoir implementation. This extension SHOULD be configurable on
a metric [View](#view), although individual reservoirs MUST still be
instantiated per metric-timeseries (see
[Exemplar Reservoir - Paragraph 2](#exemplarreservoir)).

## MetricReader

**Status**: [Stable](../document-status.md)
Expand Down Expand Up @@ -1684,8 +1683,6 @@ specific guarantees and safeties.
**MeterProvider** - Meter creation, `ForceFlush` and `Shutdown` are safe to be
called concurrently.

**ExemplarFilter** - all methods are safe to be called concurrently.

**ExemplarReservoir** - all methods are safe to be called concurrently.

**MetricReader** - `Collect`, `ForceFlush` (for periodic exporting MetricReader)
Expand Down