Test OTEL retries works

[JamesNK]

Test https://www.nuget.org/packages/OpenTelemetry.Exporter.OpenTelemetryProtocol/1.8.0-beta.1 retry logic works.

Can test by delaying startup of dashboard and then ensuring that telemetry from apps is eventually correctly recorded.

[DamianEdwards]

rc.1 is out now https://github.com/open-telemetry/opentelemetry-dotnet/releases/tag/core-1.8.0-rc.1

[vishweshbankwar]

@JamesNK - Just the heads up. A change is coming for this open-telemetry/opentelemetry-dotnet#5495

It should be released tomorrow. I will update here, once the package is available with the change.

[vishweshbankwar]

It is available now: https://www.nuget.org/packages/OpenTelemetry.Exporter.OpenTelemetryProtocol/1.8.0

[JamesNK]

To test out retries I updated OTEL to 1.8.0, added the env var, and a 20 second delay to starting up the server.

I don't see startup logging from .NET projects in the server:

What I expected to see:

Retries don't appear to be working. How many times does it retry? How long does it try to retry?

[JamesNK]

PR to update to 1.8.0 - #3351

[vishweshbankwar]

@JamesNK - Could you try increasing the timeout period? https://github.com/open-telemetry/opentelemetry-dotnet/blob/9b246750ab76e94986fc433d5c7af046a137e477/src/OpenTelemetry.Exporter.OpenTelemetryProtocol/OtlpExporterOptions.cs#L113-L117?

How long does it try to retry? - The retries will continue until the total time does not exceed the configured timeout for each export.

How many times does it retry - There is no defined # of retires, it is only controlled by the timeout period (which is max time the exporter can spend on exporting each batch).

[JamesNK]

Thanks. I tested with a 10 second startup delay and it works with the retry env var and doesn't work without the retry env var. That means it's working as expected.

10 second timeout is fine as a default.