Skip to content

Blueprint for the path to stabilizing the OTLP log exporter #4416

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
alanwest wants to merge 7 commits into from
Closed

Blueprint for the path to stabilizing the OTLP log exporter #4416

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
5d2556c
Delete separate OTLP logs project
alanwest Apr 20, 2023
c8ec489
Add BatchExportLogRecordProcessorOptions
alanwest Apr 20, 2023
b7f1547
Add LogRecordExportProcessorOptions
alanwest Apr 20, 2023
e46c44e
Add ActivityExportProcessorOptions
alanwest Apr 20, 2023
1f15db8
Make AddOtlpExporter use LogRecordExportProcessorOptions
alanwest Apr 21, 2023
181e59d
Fix XML doc comment
alanwest Apr 21, 2023
a9edbf4
Obsolete OtlpExporterOptions.ExportProcessorType and BatchExportProce…
alanwest Apr 21, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 0 additions & 6 deletions OpenTelemetry.sln
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -205,8 +205,6 @@ Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "customizing-the-sdk", "docs
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "OpenTelemetry.Tests.Stress", "test\OpenTelemetry.Tests.Stress\OpenTelemetry.Tests.Stress.csproj", "{2770158A-D220-414B-ABC6-179371323579}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "OpenTelemetry.Exporter.OpenTelemetryProtocol.Logs", "src\OpenTelemetry.Exporter.OpenTelemetryProtocol.Logs\OpenTelemetry.Exporter.OpenTelemetryProtocol.Logs.csproj", "{6E1A5FA3-E024-4972-9EDC-11E36C5A0D6F}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "source-generation", "docs\logs\source-generation\source-generation.csproj", "{1F6CC903-04C9-4E7C-B388-C215C467BFB9}"
EndProject
Project("{9A19103F-16F7-4668-BE54-9A1E7A4F7556}") = "getting-started-prometheus-grafana", "docs\metrics\getting-started-prometheus-grafana\getting-started-prometheus-grafana.csproj", "{41B784AA-3301-4126-AF9F-1D59BD04B0BF}"
Expand Down Expand Up @@ -443,10 +441,6 @@ Global
{2770158A-D220-414B-ABC6-179371323579}.Debug|Any CPU.Build.0 = Debug|Any CPU
{2770158A-D220-414B-ABC6-179371323579}.Release|Any CPU.ActiveCfg = Release|Any CPU
{2770158A-D220-414B-ABC6-179371323579}.Release|Any CPU.Build.0 = Release|Any CPU
{6E1A5FA3-E024-4972-9EDC-11E36C5A0D6F}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{6E1A5FA3-E024-4972-9EDC-11E36C5A0D6F}.Debug|Any CPU.Build.0 = Debug|Any CPU
{6E1A5FA3-E024-4972-9EDC-11E36C5A0D6F}.Release|Any CPU.ActiveCfg = Release|Any CPU
{6E1A5FA3-E024-4972-9EDC-11E36C5A0D6F}.Release|Any CPU.Build.0 = Release|Any CPU
{1F6CC903-04C9-4E7C-B388-C215C467BFB9}.Debug|Any CPU.ActiveCfg = Debug|Any CPU
{1F6CC903-04C9-4E7C-B388-C215C467BFB9}.Debug|Any CPU.Build.0 = Debug|Any CPU
{1F6CC903-04C9-4E7C-B388-C215C467BFB9}.Release|Any CPU.ActiveCfg = Release|Any CPU
Expand Down
1 change: 0 additions & 1 deletion examples/AspNetCore/Examples.AspNetCore.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,6 @@
<ProjectReference Include="$(RepoRoot)\src\OpenTelemetry.Extensions.Hosting\OpenTelemetry.Extensions.Hosting.csproj" />
<ProjectReference Include="$(RepoRoot)\src\OpenTelemetry.Exporter.Console\OpenTelemetry.Exporter.Console.csproj" />
<ProjectReference Include="$(RepoRoot)\src\OpenTelemetry.Exporter.OpenTelemetryProtocol\OpenTelemetry.Exporter.OpenTelemetryProtocol.csproj" />
<ProjectReference Include="$(RepoRoot)\src\OpenTelemetry.Exporter.OpenTelemetryProtocol.Logs\OpenTelemetry.Exporter.OpenTelemetryProtocol.Logs.csproj" />
<ProjectReference Include="$(RepoRoot)\src\OpenTelemetry.Instrumentation.AspNetCore\OpenTelemetry.Instrumentation.AspNetCore.csproj" />
<ProjectReference Include="$(RepoRoot)\src\OpenTelemetry.Instrumentation.Http\OpenTelemetry.Instrumentation.Http.csproj" />
<ProjectReference Include="$(RepoRoot)\src\OpenTelemetry.Exporter.Jaeger\OpenTelemetry.Exporter.Jaeger.csproj" />
Expand Down
1 change: 0 additions & 1 deletion examples/Console/Examples.Console.csproj
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,5 @@
<ProjectReference Include="$(RepoRoot)\src\OpenTelemetry.Exporter.Jaeger\OpenTelemetry.Exporter.Jaeger.csproj" />
<ProjectReference Include="$(RepoRoot)\src\OpenTelemetry.Exporter.InMemory\OpenTelemetry.Exporter.InMemory.csproj" />
<ProjectReference Include="$(RepoRoot)\src\OpenTelemetry.Exporter.Prometheus.HttpListener\OpenTelemetry.Exporter.Prometheus.HttpListener.csproj" />
<ProjectReference Include="$(RepoRoot)\src\OpenTelemetry.Exporter.OpenTelemetryProtocol.Logs\OpenTelemetry.Exporter.OpenTelemetryProtocol.Logs.csproj" />
</ItemGroup>
</Project>
Empty file removed ...OpenTelemetry.Exporter.OpenTelemetryProtocol.Logs/.publicApi/net462/PublicAPI.Shipped.txt
View file
Open in desktop
Empty file.
3 changes: 0 additions & 3 deletions ...enTelemetry.Exporter.OpenTelemetryProtocol.Logs/.publicApi/net462/PublicAPI.Unshipped.txt
View file
Open in desktop

This file was deleted.

Empty file removed ...metry.Exporter.OpenTelemetryProtocol.Logs/.publicApi/netstandard2.0/PublicAPI.Shipped.txt
View file
Open in desktop
Empty file.
3 changes: 0 additions & 3 deletions ...try.Exporter.OpenTelemetryProtocol.Logs/.publicApi/netstandard2.0/PublicAPI.Unshipped.txt
View file
Open in desktop

This file was deleted.

Empty file removed ...metry.Exporter.OpenTelemetryProtocol.Logs/.publicApi/netstandard2.1/PublicAPI.Shipped.txt
View file
Open in desktop
Empty file.
3 changes: 0 additions & 3 deletions ...try.Exporter.OpenTelemetryProtocol.Logs/.publicApi/netstandard2.1/PublicAPI.Unshipped.txt
View file
Open in desktop

This file was deleted.

31 changes: 0 additions & 31 deletions src/OpenTelemetry.Exporter.OpenTelemetryProtocol.Logs/AssemblyInfo.cs
View file
Open in desktop

This file was deleted.

91 changes: 0 additions & 91 deletions src/OpenTelemetry.Exporter.OpenTelemetryProtocol.Logs/CHANGELOG.md
View file
Open in desktop

This file was deleted.

23 changes: 0 additions & 23 deletions ...orter.OpenTelemetryProtocol.Logs/OpenTelemetry.Exporter.OpenTelemetryProtocol.Logs.csproj
View file
Open in desktop

This file was deleted.

11 changes: 0 additions & 11 deletions src/OpenTelemetry.Exporter.OpenTelemetryProtocol.Logs/README.md
View file
Open in desktop

This file was deleted.

4 changes: 4 additions & 0 deletions src/OpenTelemetry.Exporter.OpenTelemetryProtocol/.publicApi/net462/PublicAPI.Unshipped.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
OpenTelemetry.Logs.OtlpLogExporterHelperExtensions
static OpenTelemetry.Logs.OtlpLogExporterHelperExtensions.AddOtlpExporter(this OpenTelemetry.Logs.OpenTelemetryLoggerOptions loggerOptions) -> OpenTelemetry.Logs.OpenTelemetryLoggerOptions
static OpenTelemetry.Logs.OtlpLogExporterHelperExtensions.AddOtlpExporter(this OpenTelemetry.Logs.OpenTelemetryLoggerOptions loggerOptions, System.Action<OpenTelemetry.Exporter.OtlpExporterOptions> configure) -> OpenTelemetry.Logs.OpenTelemetryLoggerOptions
static OpenTelemetry.Logs.OtlpLogExporterHelperExtensions.AddOtlpExporter(this OpenTelemetry.Logs.OpenTelemetryLoggerOptions loggerOptions, System.Action<OpenTelemetry.Exporter.OtlpExporterOptions> configure, System.Action<OpenTelemetry.Logs.LogRecordExportProcessorOptions> configureProcessorOptions) -> OpenTelemetry.Logs.OpenTelemetryLoggerOptions
4 changes: 4 additions & 0 deletions src/OpenTelemetry.Exporter.OpenTelemetryProtocol/.publicApi/net6.0/PublicAPI.Unshipped.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
OpenTelemetry.Logs.OtlpLogExporterHelperExtensions
static OpenTelemetry.Logs.OtlpLogExporterHelperExtensions.AddOtlpExporter(this OpenTelemetry.Logs.OpenTelemetryLoggerOptions loggerOptions) -> OpenTelemetry.Logs.OpenTelemetryLoggerOptions
static OpenTelemetry.Logs.OtlpLogExporterHelperExtensions.AddOtlpExporter(this OpenTelemetry.Logs.OpenTelemetryLoggerOptions loggerOptions, System.Action<OpenTelemetry.Exporter.OtlpExporterOptions> configure) -> OpenTelemetry.Logs.OpenTelemetryLoggerOptions
static OpenTelemetry.Logs.OtlpLogExporterHelperExtensions.AddOtlpExporter(this OpenTelemetry.Logs.OpenTelemetryLoggerOptions loggerOptions, System.Action<OpenTelemetry.Exporter.OtlpExporterOptions> configure, System.Action<OpenTelemetry.Logs.LogRecordExportProcessorOptions> configureProcessorOptions) -> OpenTelemetry.Logs.OpenTelemetryLoggerOptions
4 changes: 4 additions & 0 deletions ...elemetry.Exporter.OpenTelemetryProtocol/.publicApi/netstandard2.0/PublicAPI.Unshipped.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
OpenTelemetry.Logs.OtlpLogExporterHelperExtensions
static OpenTelemetry.Logs.OtlpLogExporterHelperExtensions.AddOtlpExporter(this OpenTelemetry.Logs.OpenTelemetryLoggerOptions loggerOptions) -> OpenTelemetry.Logs.OpenTelemetryLoggerOptions
static OpenTelemetry.Logs.OtlpLogExporterHelperExtensions.AddOtlpExporter(this OpenTelemetry.Logs.OpenTelemetryLoggerOptions loggerOptions, System.Action<OpenTelemetry.Exporter.OtlpExporterOptions> configure) -> OpenTelemetry.Logs.OpenTelemetryLoggerOptions
static OpenTelemetry.Logs.OtlpLogExporterHelperExtensions.AddOtlpExporter(this OpenTelemetry.Logs.OpenTelemetryLoggerOptions loggerOptions, System.Action<OpenTelemetry.Exporter.OtlpExporterOptions> configure, System.Action<OpenTelemetry.Logs.LogRecordExportProcessorOptions> configureProcessorOptions) -> OpenTelemetry.Logs.OpenTelemetryLoggerOptions
4 changes: 4 additions & 0 deletions ...elemetry.Exporter.OpenTelemetryProtocol/.publicApi/netstandard2.1/PublicAPI.Unshipped.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
OpenTelemetry.Logs.OtlpLogExporterHelperExtensions
static OpenTelemetry.Logs.OtlpLogExporterHelperExtensions.AddOtlpExporter(this OpenTelemetry.Logs.OpenTelemetryLoggerOptions loggerOptions) -> OpenTelemetry.Logs.OpenTelemetryLoggerOptions
static OpenTelemetry.Logs.OtlpLogExporterHelperExtensions.AddOtlpExporter(this OpenTelemetry.Logs.OpenTelemetryLoggerOptions loggerOptions, System.Action<OpenTelemetry.Exporter.OtlpExporterOptions> configure) -> OpenTelemetry.Logs.OpenTelemetryLoggerOptions
static OpenTelemetry.Logs.OtlpLogExporterHelperExtensions.AddOtlpExporter(this OpenTelemetry.Logs.OpenTelemetryLoggerOptions loggerOptions, System.Action<OpenTelemetry.Exporter.OtlpExporterOptions> configure, System.Action<OpenTelemetry.Logs.LogRecordExportProcessorOptions> configureProcessorOptions) -> OpenTelemetry.Logs.OpenTelemetryLoggerOptions
2 changes: 2 additions & 0 deletions src/OpenTelemetry.Exporter.OpenTelemetryProtocol/OtlpExporterOptions.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -150,11 +150,13 @@ public Uri Endpoint
/// <summary>
/// Gets or sets the export processor type to be used with the OpenTelemetry Protocol Exporter. The default value is <see cref="ExportProcessorType.Batch"/>.
/// </summary>
[Obsolete("Use ActivityExportProcessorOptions or LogRecordExportProcessorOptions.")]
public ExportProcessorType ExportProcessorType { get; set; } = ExportProcessorType.Batch;

/// <summary>
/// Gets or sets the BatchExportProcessor options. Ignored unless ExportProcessorType is Batch.
/// </summary>
[Obsolete("Use ActivityExportProcessorOptions or LogRecordExportProcessorOptions.")]
public BatchExportProcessorOptions<Activity> BatchExportProcessorOptions { get; set; }

/// <summary>
Expand Down
39 changes: 28 additions & 11 deletions ...l.Logs/OtlpLogExporterHelperExtensions.cs → ...otocol/OtlpLogExporterHelperExtensions.cs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@
// limitations under the License.
// </copyright>

using System.Diagnostics;
using OpenTelemetry.Exporter;

namespace OpenTelemetry.Logs
Expand All @@ -30,43 +29,61 @@ public static class OtlpLogExporterHelperExtensions
/// <remarks><inheritdoc cref="AddOtlpExporter(OpenTelemetryLoggerOptions, Action{OtlpExporterOptions})" path="/remarks"/></remarks>
/// <param name="loggerOptions"><see cref="OpenTelemetryLoggerOptions"/> options to use.</param>
/// <returns>The instance of <see cref="OpenTelemetryLoggerOptions"/> to chain the calls.</returns>
[Obsolete("We will never ship this method as it is. We will ship the one that returns LoggerProviderBuilder from main-logs branch instead of OpenTelemetryLoggerOptions.")]
public static OpenTelemetryLoggerOptions AddOtlpExporter(this OpenTelemetryLoggerOptions loggerOptions)
Comment on lines +32 to 33
Copy link
Member Author

@alanwest alanwest Apr 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This highlights one reason we should not attempt to ship a stable OTLP exporter without first integrating LoggerProviderBuilder currently on the main-logs branch.

=> AddOtlpExporterInternal(loggerOptions, configure: null);
=> AddOtlpExporter(loggerOptions, configure: null, configureProcessorOptions: null);

/// <summary>
/// Adds OTLP Exporter as a configuration to the OpenTelemetry ILoggingBuilder.
/// </summary>
/// <param name="loggerOptions"><see cref="OpenTelemetryLoggerOptions"/> options to use.</param>
/// <param name="configure">Callback action for configuring <see cref="OtlpExporterOptions"/>.</param>
/// <returns>The instance of <see cref="OpenTelemetryLoggerOptions"/> to chain the calls.</returns>
[Obsolete("We will never ship this method as it is. We will ship one that returns LoggerProviderBuilder from main-logs branch instead of OpenTelemetryLoggerOptions.")]
public static OpenTelemetryLoggerOptions AddOtlpExporter(
this OpenTelemetryLoggerOptions loggerOptions,
Action<OtlpExporterOptions> configure)
=> AddOtlpExporterInternal(loggerOptions, configure);
=> AddOtlpExporterInternal(loggerOptions, configure, configureProcessorOptions: null);

/// <summary>
/// Adds OTLP Exporter as a configuration to the OpenTelemetry ILoggingBuilder.
/// </summary>
/// <param name="loggerOptions"><see cref="OpenTelemetryLoggerOptions"/> options to use.</param>
/// <param name="configure">Callback action for configuring <see cref="OtlpExporterOptions"/>.</param>
/// <param name="configureProcessorOptions">Callback action for configuring <see cref="LogRecordExportProcessorOptions"/>.</param>
/// <returns>The instance of <see cref="OpenTelemetryLoggerOptions"/> to chain the calls.</returns>
[Obsolete("We will never ship this method as it is. We will ship one that returns LoggerProviderBuilder from main-logs branch instead of OpenTelemetryLoggerOptions.")]
public static OpenTelemetryLoggerOptions AddOtlpExporter(
this OpenTelemetryLoggerOptions loggerOptions,
Action<OtlpExporterOptions> configure,
Action<LogRecordExportProcessorOptions> configureProcessorOptions)
Comment on lines +56 to +59
Copy link
Member Author

@alanwest alanwest Apr 21, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This differs from what we did with metrics.

opentelemetry-dotnet/src/OpenTelemetry.Exporter.OpenTelemetryProtocol/OtlpMetricExporterExtensions.cs

Lines 112 to 114 in 10eb79c

public static MeterProviderBuilder AddOtlpExporter(
this MeterProviderBuilder builder,
Action<OtlpExporterOptions, MetricReaderOptions> configureExporterAndMetricReader)

That is, I've separated the delegates to configure the exporter and processor options. I'm a bit torn here, because I know we're unhappy with the pattern we used for metrics because it cannot be well supported by IOptions. But on the other hand, I don't like that we're introducing an inconsistency.

One option may be to obsolete the extensions we have for metrics and add overloads that have a separate delegate for configuring MetricReaderOptions.

=> AddOtlpExporterInternal(loggerOptions, configure, configureProcessorOptions);

private static OpenTelemetryLoggerOptions AddOtlpExporterInternal(
OpenTelemetryLoggerOptions loggerOptions,
Action<OtlpExporterOptions> configure)
Action<OtlpExporterOptions> configure,
Action<LogRecordExportProcessorOptions> configureProcessorOptions)
{
var exporterOptions = new OtlpExporterOptions();
// TODO: This has not yet been adapted to correctly utilize IConfiguration/IOptions
// It simply illustrates that the processor options will not come from
// the obsolete properties on OtlpExporterOptions.

// TODO: We are using span/activity batch environment variable keys
// here when we should be using the ones for logs.
var defaultBatchOptions = exporterOptions.BatchExportProcessorOptions;
var exporterOptions = new OtlpExporterOptions();

Debug.Assert(defaultBatchOptions != null, "defaultBatchOptions was null");
var processorOptions = new LogRecordExportProcessorOptions();

configure?.Invoke(exporterOptions);
configureProcessorOptions?.Invoke(processorOptions);

var otlpExporter = new OtlpLogExporter(exporterOptions);

if (exporterOptions.ExportProcessorType == ExportProcessorType.Simple)
if (processorOptions.ExportProcessorType == ExportProcessorType.Simple)
{
loggerOptions.AddProcessor(new SimpleLogRecordExportProcessor(otlpExporter));
}
else
{
var batchOptions = exporterOptions.BatchExportProcessorOptions ?? defaultBatchOptions;
var batchOptions = processorOptions.BatchExportProcessorOptions ?? new BatchExportLogRecordProcessorOptions();

loggerOptions.AddProcessor(new BatchLogRecordExportProcessor(
otlpExporter,
Expand Down
Loading