enhance(tracing): customizable tasks sampler #1184
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
support for customizing the tracing sampler on a per-task basis.
{ "task-name": { "active": bool } }a
taskis basically the "span name" based on the work being performed.a
taskcan be an API endpoint interaction, a gorm query, etc. see: https://opentelemetry.io/docs/specs/otel/trace/api/#spanif a
taskis not represented in the configuration file then the task will be treated normally, with tracing enabled.consider the following example configuration files:
Example 1
this will enable all tracing.
{}Example 2
this will enable all tracing except for the
/healthendpoint.{ "/health": { "active": false } }Example 3
{ "/health": { "active": false }, "/api/v1/deployments/:org/:repo": { "active": false }, "/api/v1/:worker": { "active": true }, "gorm.query": { "active": false } }/healthwithactive: falsewill disable tracing on/health./api/v1/deployments/:org/:repowithactive: falsewill disable tracing on/api/v1/deployments/:org/:repofor ALL:organd:repoparameters./api/v1/:workerwithactive: truewill do nothing at this point, because any tasks that are not present in the configuration file will automatically be sampled normally.for now this is slightly confusing, but in the future there will be more configuration fields that will determine how an
active: truetask is sampled.gorm.querywithactive: falsewould disable tracing for raw gorm queries. this is meant to show that the config applies to all trace tasks and not just API/HTTP endpoints.all other tasks will be sampled as normal using the parent rate limit sampler! this includes all api endpoints and gorm queries.
Potential Enhancements
planning future enhancements like per-task ratio-sampling and per-endpoint method restrictions.
consider the following stretch goal configuration file:
{ "/health": { "active": false }, "/api/v1/deployments/:org/:repo": { "active": true, "ratio": 0.3, // record only 30% "methods": [ "get", "put" ], // record particular methods "query_parameters": [ "event=push" ] // record particular queries },