-
Notifications
You must be signed in to change notification settings - Fork 83
fix(tracing): handle buildbot tracing integration problems #5100
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
|
This pull request adds or modifies JavaScript ( |
| @@ -1,5 +1,5 @@ | |||
| import { HoneycombSDK } from '@honeycombio/opentelemetry-node' | |||
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We don't really need the HoneycombSDK since we're using a collector. Plus it was logging warnings I couldn't supress:
| context: 'production' | string | ||
| statsdOpts: { host?: number; port: number } | ||
| bugsnagKey?: string | ||
| tracing: TracingOptions |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We actually perform the transformation and apply the default values to tracingOpts 🤷 this makes sure we use that.
| // Set the diagnostics logger to our system logger. We also need to suppress the override msg | ||
| // in case there's a default console logger already registered (it would log a msg to it) | ||
| diag.setLogger(getOtelLogger(logger), { logLevel: DiagLogLevel.INFO, suppressOverrideMessage: true }) | ||
|
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This sets the internal diagnostics logger for open telemetry to our system logger.
Running this locally renders something like this if there's any errors and we're running in debug mode (in Buildbot prod this should still land in our backend without showing up in customer's builds):
node packages/build/bin.js --tracing.enabled=true --tracing.host=127.0.0.1 --debug ../../github/www-and-blog/
Netlify Build
────────────────────────────────────────────────────────────────
❯ Version
@netlify/build 29.14.0
❯ Flags
debug: true
doest:
not:
exist: yolo
siteId: 1e16aaa6-91e2-4bbc-bbe6-134250b9d81d
tracing:
enabled: true
host: 127.0.0.1
❯ Current directory
/Users/jgantunes/workspace/netlify/build
❯ Config file
No config file was defined: using default values.
❯ Resolved config
build:
publish: /Users/jgantunes/workspace/netlify/build
publishOrigin: default
❯ Context
production
Netlify Build Complete
────────────────────────────────────────────────────────────────
(Netlify Build completed in 66ms)
Build step duration: Netlify Build completed in 66ms
[otel-traces] Service request {"resourceSpans":[{"resource":{"attributes":[{"key":"service.name","value":{"stringValue":"@netlify/build"}},{"key":"telemetry.sdk.language","value":{"stringValue":"nodejs"}},{"key":"telemetry.sdk.name","value":{"stringValue":"opentelemetry"}},{"key":"telemetry.sdk.version","value":{"stringValue":"1.14.0"}},{"key":"process.pid","value":{"intValue":80030}},{"key":"process.executable.name","value":{"stringValue":"/Users/jgantunes/.nvm/versions/node/v16.4.2/bin/node"}},{"key":"process.executable.path","value":{"stringValue":"/Users/jgantunes/.nvm/versions/node/v16.4.2/bin/node"}},{"key":"process.command_args","value":{"arrayValue":{"values":[{"stringValue":"/Users/jgantunes/.nvm/versions/node/v16.4.2/bin/node"},{"stringValue":"/Users/jgantunes/workspace/netlify/build/packages/build/bin.js"},{"stringValue":"--tracing.enabled=true"},{"stringValue":"--tracing.host=127.0.0.1"},{"stringValue":"--debug"},{"stringValue":"--token"},{"stringValue":"-AaVbRmHW9hv0iftbfZESEsxLhUpsGjj7SQWhGjIPp4"},{"stringValue":"--siteId"},{"stringValue":"1e16aaa6-91e2-4bbc-bbe6-134250b9d81d"},{"stringValue":"--doest.not.exist"},{"stringValue":"yolo"},{"stringValue":"../../github/www-and-blog/"}]}}},{"key":"process.runtime.version","value":{"stringValue":"16.4.2"}},{"key":"process.runtime.name","value":{"stringValue":"nodejs"}},{"key":"process.runtime.description","value":{"stringValue":"Node.js"}},{"key":"process.command","value":{"stringValue":"/Users/jgantunes/workspace/netlify/build/packages/build/bin.js"}},{"key":"process.owner","value":{"stringValue":"jgantunes"}}],"droppedAttributesCount":0},"scopeSpans":[{"scope":{"name":"steps"},"spans":[{"traceId":"o9LynihymCNYSXun71/HPg==","spanId":"5Z/uq69ryis=","parentSpanId":"dofsu5pRMqI=","name":"run-step-build_command","kind":1,"startTimeUnixNano":1687953946533000000,"endTimeUnixNano":1687953946533486600,"attributes":[{"key":"build.execution.step.should_run","value":{"boolValue":false}}],"droppedAttributesCount":0,"events":[],"droppedEventsCount":0,"status":{"code":0},"links":[],"droppedLinksCount":0},{"traceId":"o9LynihymCNYSXun71/HPg==","spanId":"G3vEo9dyLMY=","parentSpanId":"dofsu5pRMqI=","name":"run-step-functions_bundling","kind":1,"startTimeUnixNano":1687953946534000000,"endTimeUnixNano":1687953946534406700,"attributes":[{"key":"build.execution.step.should_run","value":{"boolValue":false}}],"droppedAttributesCount":0,"events":[],"droppedEventsCount":0,"status":{"code":0},"links":[],"droppedLinksCount":0},{"traceId":"o9LynihymCNYSXun71/HPg==","spanId":"K0ojzneQpxc=","parentSpanId":"dofsu5pRMqI=","name":"run-step-edge_functions_bundling","kind":1,"startTimeUnixNano":1687953946534000000,"endTimeUnixNano":1687953946534243300,"attributes":[{"key":"build.execution.step.should_run","value":{"boolValue":false}}],"droppedAttributesCount":0,"events":[],"droppedEventsCount":0,"status":{"code":0},"links":[],"droppedLinksCount":0},{"traceId":"o9LynihymCNYSXun71/HPg==","spanId":"nDLvgSE0q5U=","parentSpanId":"dofsu5pRMqI=","name":"run-step-deploy_site","kind":1,"startTimeUnixNano":1687953946535000000,"endTimeUnixNano":1687953946535169800,"attributes":[{"key":"build.execution.step.should_run","value":{"boolValue":false}}],"droppedAttributesCount":0,"events":[],"droppedEventsCount":0,"status":{"code":0},"links":[],"droppedLinksCount":0},{"traceId":"o9LynihymCNYSXun71/HPg==","spanId":"Urb73yGcZqQ=","parentSpanId":"dofsu5pRMqI=","name":"run-step-save_artifacts","kind":1,"startTimeUnixNano":1687953946535000000,"endTimeUnixNano":1687953946535169000,"attributes":[{"key":"build.execution.step.should_run","value":{"boolValue":false}}],"droppedAttributesCount":0,"events":[],"droppedEventsCount":0,"status":{"code":0},"links":[],"droppedLinksCount":0}]},{"scope":{"name":"core"},"spans":[{"traceId":"o9LynihymCNYSXun71/HPg==","spanId":"dofsu5pRMqI=","name":"exec-build","kind":1,"startTimeUnixNano":1687953946469000000,"endTimeUnixNano":1687953946537037300,"attributes":[{"key":"build.execution.success","value":{"boolValue":true}},{"key":"build.execution.code","value":{"intValue":0}},{"key":"build.execution.status","value":{"stringValue":"success"}}],"droppedAttributesCount":0,"events":[],"droppedEventsCount":0,"status":{"code":0},"links":[],"droppedLinksCount":0}]}]}]}
[otel-traces] 14 UNAVAILABLE: No connection established Error: 14 UNAVAILABLE: No connection established
at callErrorFromStatus (/Users/jgantunes/workspace/netlify/build/node_modules/@grpc/grpc-js/build/src/call.js:31:19)
at Object.onReceiveStatus (/Users/jgantunes/workspace/netlify/build/node_modules/@grpc/grpc-js/build/src/client.js:192:76)
at Object.onReceiveStatus (/Users/jgantunes/workspace/netlify/build/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:360:141)
at Object.onReceiveStatus (/Users/jgantunes/workspace/netlify/build/node_modules/@grpc/grpc-js/build/src/client-interceptors.js:323:181)
at /Users/jgantunes/workspace/netlify/build/node_modules/@grpc/grpc-js/build/src/resolving-call.js:94:78
at processTicksAndRejections (node:internal/process/task_queues:78:11)
for call at
Removing the debug flag clears these logs.
lukasholzer
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
THis looks awesome!
🎉 Thanks for submitting a pull request! 🎉
Summary
Related with - https://github.com/netlify/buildbot/issues/2689. This addresses a couple of issues I've seen running this in production. I've left comments on the most relevant changes but the main problems I've spotted were noisy logs and the fact that the port wasn't being correctly set.
For us to review and ship your PR efficiently, please perform the following steps:
we can discuss the changes and get feedback from everyone that should be involved. If you`re fixing a typo or
something that`s on fire 🔥 (e.g. incident related), you can skip this step.
your code follows our style guide and passes our tests.
A picture of a cute animal (not mandatory, but encouraged)