Skip to content

Commit d5540c6

Browse files
committed
Merge from main.
2 parents 67a2017 + 735dd0d commit d5540c6

File tree

54 files changed

+2716
-288
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

54 files changed

+2716
-288
lines changed

.gitignore

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -345,3 +345,6 @@ ASALocalRun/
345345
/.sonarqube
346346

347347
/src/LastMajorVersionBinaries
348+
349+
# Tempo files
350+
tempo-data/

OpenTelemetry.sln

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -249,6 +249,8 @@ Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "getting-started-console", "
249249
EndProject
250250
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "getting-started-jaeger", "docs\trace\getting-started-jaeger\getting-started-jaeger.csproj", "{A0C0B77C-6C7B-4EC2-AC61-EA1F489811B9}"
251251
EndProject
252+
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "stratified-sampling-example", "docs\trace\advanced\stratified-sampling-example\stratified-sampling-example.csproj", "{9C99621C-343E-479C-A943-332DB6129B71}"
253+
EndProject
252254
Global
253255
GlobalSection(SolutionConfigurationPlatforms) = preSolution
254256
Debug|Any CPU = Debug|Any CPU
@@ -523,6 +525,10 @@ Global
523525
{A0C0B77C-6C7B-4EC2-AC61-EA1F489811B9}.Debug|Any CPU.Build.0 = Debug|Any CPU
524526
{A0C0B77C-6C7B-4EC2-AC61-EA1F489811B9}.Release|Any CPU.ActiveCfg = Release|Any CPU
525527
{A0C0B77C-6C7B-4EC2-AC61-EA1F489811B9}.Release|Any CPU.Build.0 = Release|Any CPU
528+
{9C99621C-343E-479C-A943-332DB6129B71}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
529+
{9C99621C-343E-479C-A943-332DB6129B71}.Debug|Any CPU.Build.0 = Debug|Any CPU
530+
{9C99621C-343E-479C-A943-332DB6129B71}.Release|Any CPU.ActiveCfg = Release|Any CPU
531+
{9C99621C-343E-479C-A943-332DB6129B71}.Release|Any CPU.Build.0 = Release|Any CPU
526532
EndGlobalSection
527533
GlobalSection(SolutionProperties) = preSolution
528534
HideSolutionNode = FALSE
@@ -562,6 +568,7 @@ Global
562568
{DEDE8442-03CA-48CF-99B9-EA224D89D148} = {5B7FB835-3FFF-4BC2-99C5-A5B5FAE3C818}
563569
{EF4F6280-14D1-49D4-8095-1AC36E169AA8} = {5B7FB835-3FFF-4BC2-99C5-A5B5FAE3C818}
564570
{A0C0B77C-6C7B-4EC2-AC61-EA1F489811B9} = {5B7FB835-3FFF-4BC2-99C5-A5B5FAE3C818}
571+
{9C99621C-343E-479C-A943-332DB6129B71} = {5B7FB835-3FFF-4BC2-99C5-A5B5FAE3C818}
565572
EndGlobalSection
566573
GlobalSection(ExtensibilityGlobals) = postSolution
567574
SolutionGuid = {55639B5C-0770-4A22-AB56-859604650521}

build/Common.prod.props

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -6,6 +6,7 @@
66
<IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
77
<PrivateAssets>all</PrivateAssets>
88
</PackageReference>
9+
910
<PackageReference Include="Microsoft.CodeAnalysis.PublicApiAnalyzers" Version="$(MicrosoftCodeAnalysisAnalyzersPkgVer)" Condition=" $(OS) == 'Windows_NT'">
1011
<PrivateAssets>All</PrivateAssets>
1112
</PackageReference>

docs/metrics/customizing-the-sdk/README.md

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -403,6 +403,70 @@ AnotherFruitCounter.Add(4, new("name", "mango"), new("color", "yellow")); // Not
403403
streams. There is no ability to apply different limits for each instrument at
404404
this moment.
405405

406+
### Exemplars
407+
408+
Exemplars are example data points for aggregated data. They provide access to
409+
the raw measurement value, time stamp when measurement was made, and trace
410+
context, if any. It also provides "Filtered Tags", which are attributes (Tags)
411+
that are [dropped by a view](#select-specific-tags). Exemplars are an opt-in
412+
feature, and allow customization via ExemplarFilter and ExemplarReservoir.
413+
414+
#### ExemplarFilter
415+
416+
`ExemplarFilter` determines which measurements are eligible to become an
417+
Exemplar. i.e. `ExemplarFilter` determines which measurements are offered to
418+
`ExemplarReservoir`, which makes the final decision about whether the offered
419+
measurement gets stored as an exemplar. They can be used to control the noise
420+
and overhead associated with Exemplar collection.
421+
422+
OpenTelemetry SDK comes with the following Filters:
423+
424+
* `AlwaysOnExemplarFilter` - makes all measurements eligible for being an Exemplar.
425+
* `AlwaysOffExemplarFilter` - makes no measurements eligible for being an
426+
Exemplar. Use this to turn-off Exemplar feature.
427+
* `TraceBasedExemplarFilter` - makes those measurements eligible for being an
428+
Exemplar, which are recorded in the context of a sampled parent `Activity`
429+
(span).
430+
431+
`SetExemplarFilter` method on `MeterProviderBuilder` can be used to set the
432+
desired `ExemplarFilter`.
433+
434+
The snippet below shows how to set `ExemplarFilter`.
435+
436+
```csharp
437+
using OpenTelemetry;
438+
using OpenTelemetry.Metrics;
439+
440+
using var meterProvider = Sdk.CreateMeterProviderBuilder()
441+
// rest of config not shown
442+
.SetExemplarFilter(new TraceBasedExemplarFilter())
443+
.Build();
444+
```
445+
446+
> **Note**
447+
> As of today, there is no separate toggle for enable/disable Exemplar
448+
feature. It can be turned off by using `AlwaysOffExemplarFilter`.
449+
450+
If the built-in `ExemplarFilter`s are not meeting the needs, one may author
451+
custom `ExemplarFilter` as shown
452+
[here](../extending-the-sdk/README.md#exemplarfilter). A custom filter, which
453+
eliminates all un-interesting measurements from becoming Exemplar is a
454+
recommended way to control performance overhead associated with collecting
455+
Exemplars. See
456+
[benchmark](../../../test/Benchmarks/Metrics/ExemplarBenchmarks.cs) to see how
457+
much impact can `ExemplarFilter` have on performance.
458+
459+
#### ExemplarReservoir
460+
461+
`ExemplarReservoir` receives the measurements sampled in by the `ExemplarFilter`
462+
and is responsible for storing Exemplars.
463+
`AlignedHistogramBucketExemplarReservoir` is the default reservoir used for
464+
Histograms with buckets, and it stores one exemplar per histogram bucket. The
465+
exemplar stored is the last measurement recorded - i.e. any new measurement
466+
overwrites the previous one.
467+
468+
Currently there is no ability to change the Reservoir used.
469+
406470
### Instrumentation
407471

408472
// TODO

docs/metrics/exemplars/README.md

Lines changed: 99 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,99 @@
1+
# Using Exemplars in OpenTelemetry .NET
2+
3+
Exemplars are example data points for aggregated data. They provide specific
4+
context to otherwise general aggregations. One common use case is to gain
5+
ability to correlate metrics to traces (and logs). While OpenTelemetry .NET
6+
supports Exemplars, it is only useful if the telemetry backend also supports the
7+
capabilities. This tutorial uses well known open source backends to demonstrate
8+
the concept. The following are the components involved:
9+
10+
* Test App - We use existing example app from the repo. This app is already
11+
instrumented with OpenTelemetry for logs, metrics and traces, and is configured
12+
to export them to the configured OTLP end point.
13+
* OpenTelemetry Collector - An instance of collector is run, which receives
14+
telemetry from the above app using OTLP. The collector then exports metrics to
15+
Prometheus, traces to Tempo.
16+
* Prometheus - Prometheus is used as the Metric backend.
17+
* Tempo - Tempo is used as the Tracing backend.
18+
* Grafana - UI to query metrics from Prometheus, traces from Tempo, and to
19+
navigate between metrics and traces using Exemplar.
20+
21+
All these components except the test app require additional configuration to
22+
enable Exemplar feature. To make it easy for users, these components are
23+
pre-configured to enable Exemplars, and a docker-compose is provided to spun
24+
them all up, in the required configurations.
25+
26+
## Pre-requisite
27+
28+
Install docker: <https://docs.docker.com/get-docker/>
29+
30+
## Setup
31+
32+
As mentioned in the intro, this tutorial uses OTel Collector, Prometheus, Tempo,
33+
and Grafana, and they must be up and running before proceeding. The following
34+
spins all of them with the correct configurations to support Exemplars.
35+
36+
Navigate to current directory and run the following:
37+
38+
```sh
39+
docker-compose up -d
40+
```
41+
42+
If the above step succeeds, all dependencies would be spun up and ready now. To
43+
test, navigate to Grafana running at: "http://localhost:3000/".
44+
45+
## Run test app
46+
47+
Now that the required dependencies are ready, lets run the demo app.
48+
This tutorial is using the existing ASP.NET Core app from the repo.
49+
50+
Navigate to [Example Asp.Net Core App](../../../examples/AspNetCore/Program.cs)
51+
directory and run the following command:
52+
53+
```sh
54+
dotnet run
55+
```
56+
57+
Once the application is running, navigate to
58+
[http://localhost:5000/weatherforecast]("http://localhost:5000/weatherforecast")
59+
from a web browser. You may use the following Powershell script to generate load
60+
to the application.
61+
62+
```powershell
63+
while($true)
64+
{
65+
Invoke-WebRequest http://localhost:5000/weatherforecast
66+
Start-Sleep -Milliseconds 500
67+
}
68+
```
69+
70+
## Use Exemplars to navigate from Metrics to Traces
71+
72+
The application sends metrics (with exemplars), and traces to the OTel
73+
Collector, which export metrics and traces to Prometheus and Tempo
74+
respectively.
75+
76+
Please wait for 2 minutes before continuing so that enough data is generated
77+
and exported.
78+
79+
Open Grafana, select Explore, and select Prometheus as the source. Select the
80+
metric named "http_server_duration_bucket", and plot the chart. Toggle on the
81+
"Exemplar" option from the UI and hit refresh.
82+
83+
![Enable Exemplar](https://user-images.githubusercontent.com/16979322/218627781-9886f837-11ae-4d52-94d3-f1821503209c.png)
84+
85+
The Exemplars appear as special "diamond shaped dots" along with the metric
86+
charts in the UI. Select any Exemplar to see the exemplar data, which includes
87+
the timestamp when the measurement was recorded, the raw value, and trace
88+
context when the recording was done. The "trace_id" enables jumping to the
89+
tracing backed (tempo). Click on the "Query with Tempo" button next to the
90+
"trace_id" field to open the corresponding `Trace` in Tempo.
91+
92+
![Navigate to trace with exemplar](https://user-images.githubusercontent.com/16979322/218629999-1d1cd6ba-2385-4683-975a-d4797df8361a.png)
93+
94+
## References
95+
96+
* [Exemplar specification](https://github.com/open-telemetry/opentelemetry-specification/blob/main/specification/metrics/sdk.md#exemplar)
97+
* [Exemplars in Prometheus](https://prometheus.io/docs/prometheus/latest/feature_flags/#exemplars-storage)
98+
* [Exemplars in Grafana](https://grafana.com/docs/grafana/latest/fundamentals/exemplars/)
99+
* [Tempo](https://github.com/grafana/tempo)
Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
version: "3"
2+
services:
3+
4+
# OTEL Collector to receive logs, metrics and traces from the application
5+
otel-collector:
6+
image: otel/opentelemetry-collector:0.70.0
7+
command: [ "--config=/etc/otel-collector.yaml" ]
8+
volumes:
9+
- ./otel-collector.yaml:/etc/otel-collector.yaml
10+
ports:
11+
- "4317:4317"
12+
- "4318:4318"
13+
- "9201:9201"
14+
15+
# Exports Traces to Tempo
16+
tempo:
17+
image: grafana/tempo:latest
18+
command: [ "-config.file=/etc/tempo.yaml" ]
19+
volumes:
20+
- ./tempo.yaml:/etc/tempo.yaml
21+
- ./tempo-data:/tmp/tempo
22+
ports:
23+
- "3200" # tempo
24+
- "4317" # otlp grpc
25+
- "4318" # otlp http
26+
27+
# Exports Metrics to Prometheus
28+
prometheus:
29+
image: prom/prometheus:latest
30+
command:
31+
- --config.file=/etc/prometheus.yaml
32+
- --web.enable-remote-write-receiver
33+
- --enable-feature=exemplar-storage
34+
volumes:
35+
- ./prometheus.yaml:/etc/prometheus.yaml
36+
ports:
37+
- "9090:9090"
38+
39+
# UI to query traces and metrics
40+
grafana:
41+
image: grafana/grafana:9.3.2
42+
volumes:
43+
- ./grafana-datasources.yaml:/etc/grafana/provisioning/datasources/datasources.yaml
44+
environment:
45+
- GF_AUTH_ANONYMOUS_ENABLED=true
46+
- GF_AUTH_ANONYMOUS_ORG_ROLE=Admin
47+
- GF_AUTH_DISABLE_LOGIN_FORM=true
48+
- GF_FEATURE_TOGGLES_ENABLE=traceqlEditor
49+
ports:
50+
- "3000:3000"
51+
Lines changed: 33 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,33 @@
1+
apiVersion: 1
2+
3+
datasources:
4+
- name: Prometheus
5+
type: prometheus
6+
uid: prometheus
7+
access: proxy
8+
orgId: 1
9+
url: http://prometheus:9090
10+
basicAuth: false
11+
isDefault: true
12+
version: 1
13+
editable: false
14+
jsonData:
15+
httpMethod: GET
16+
exemplarTraceIdDestinations:
17+
- name: trace_id
18+
datasourceUid: Tempo
19+
- name: Tempo
20+
type: tempo
21+
access: proxy
22+
orgId: 1
23+
url: http://tempo:3200
24+
basicAuth: false
25+
isDefault: false
26+
version: 1
27+
editable: false
28+
apiVersion: 1
29+
uid: tempo
30+
jsonData:
31+
httpMethod: GET
32+
serviceMap:
33+
datasourceUid: prometheus
Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
receivers:
2+
otlp:
3+
protocols:
4+
grpc:
5+
http:
6+
7+
exporters:
8+
logging:
9+
loglevel: debug
10+
prometheus:
11+
endpoint: ":9201"
12+
send_timestamps: true
13+
metric_expiration: 180m
14+
enable_open_metrics: true
15+
otlp:
16+
endpoint: tempo:4317
17+
tls:
18+
insecure: true
19+
20+
service:
21+
pipelines:
22+
traces:
23+
receivers: [otlp]
24+
exporters: [logging,otlp]
25+
metrics:
26+
receivers: [otlp]
27+
exporters: [logging,prometheus]
28+
logs:
29+
receivers: [otlp]
30+
exporters: [logging]
Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
global:
2+
scrape_interval: 15s
3+
evaluation_interval: 15s
4+
5+
scrape_configs:
6+
- job_name: 'otel'
7+
static_configs:
8+
- targets: [ 'otel-collector:9201' ]

docs/metrics/exemplars/tempo.yaml

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
server:
2+
http_listen_port: 3200
3+
4+
distributor:
5+
receivers:
6+
otlp:
7+
protocols:
8+
http:
9+
grpc:
10+
11+
storage:
12+
trace:
13+
backend: local
14+
wal:
15+
path: /tmp/tempo/wal
16+
local:
17+
path: /tmp/tempo/blocks

0 commit comments

Comments
 (0)