Skip to content

Conversation

@jamietanna
Copy link
Contributor

@jamietanna jamietanna commented Nov 27, 2025

Changes

Now we have instrumented our Git operations with tracing, we can make
sure that users who are not using OpenTelemetry still get insights into
their Git operation usage using a debug log output.

Following how other insights are produced, we can introduce a new type
in lib/util/stats for GitOperationStats, which will track the
overall execution for each operation type, and output them in the same
style as we use for other timing reports.

At the end of a Renovate run, we'll then make sure that this is logged
into a debug log for further investigation.

To create the report, we can make sure that we introduce a new
OpenTelemetry SpanProcessor, which will run when we end our Run, and
store any Git operations in GitOperationStats.

Context

Please select one of the below:

  • This closes an existing Issue: #
  • This doesn't close an Issue, but I accept the risk that this PR may be closed if maintainers disagree with its opening or implementation

AI assistance disclosure

Did you use AI tools to create any part of this pull request?

Please select one option and, if yes, briefly describe how AI was used (e.g., code, tests, docs) and which tool(s) you used.

  • No — I did not use AI for this contribution.
  • Yes — minimal assistance (e.g., IDE autocomplete, small code completions, grammar fixes).
  • Yes — substantive assistance (AI generated non‑trivial portions of code, tests, or documentation).
  • Yes — other (please describe):

Documentation (please check one with an [x])

  • I have updated the documentation, or
  • No documentation update is required

How I've tested my work (please select one)

I have verified these changes via:

  • Code inspection only, or
  • Newly added/modified unit tests, or
  • No unit tests but ran on a real repository, or
  • Both unit tests + ran on a real repository

The public repository:

Evidence

When tested with:

/**
 * @typedef {import('renovate/dist/config/types').AllConfig} AllConfig
 */

const fs = require('fs')

/** @type {AllConfig} */
let config = {
  onboarding: false,
  requireConfig: 'ignored',

  // branchConcurrentLimit: 100,
  // prConcurrentLimit: 100,
  // // make sure that **??**
  // prHourlyLimit: 1000,

  extends: ['config:best-practices'],
  logLevelRemap: [
    {
      matchMessage: "/Git operations statistics/",
      "newLogLevel": "info"
    },
  ]
}

module.exports = config;

With:

env npm run compile:ts && env OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318 node ../dist/renovate.js --token $(gh auth token) $repo --dry-run=full 

backstage/backstage

Screenshot 2025-12-01 at 15 43 49

A large repository, with many possible branches, let's see how long it takes to run.

 INFO: Git operations statistics (repository=backstage/backstage)
       "other": {
         "count": 29,
         "avgMs": 36,
         "medianMs": 17.72544,
         "maxMs": 359.779328,
         "totalMs": 1032
       },
       "clone": {
         "count": 1,
         "avgMs": 32537,
         "medianMs": 32537.324032,
         "maxMs": 32537.324032,
         "totalMs": 32538
       },
       "checkout": {
         "count": 1,
         "avgMs": 194,
         "medianMs": 194.338048,
         "maxMs": 194.338048,
         "totalMs": 195
       },
       "reset": {
         "count": 1,
         "avgMs": 152,
         "medianMs": 152.2048,
         "maxMs": 152.2048,
         "totalMs": 153
       },
       "oldLevel": "debug",
       "trace_id": "5c81f7938635a648ad360eef90fa7b77",
       "span_id": "ba9b6e3aa9976d0c",
       "trace_flags": "01"

JamieTanna-Mend-testing/elastic-poetry-1-forced-csp-security-policies (fork of https://github.com/elastic/csp-security-policies/)

A smaller repository, where we'd need to invoke poetry (although not tested in dry-run mode)

 INFO: Git operations statistics (repository=JamieTanna-Mend-testing/elastic-poetry-1-forced-csp-security-policies)
       "other": {
         "count": 93,
         "avgMs": 18,
         "medianMs": 12.624896,
         "maxMs": 358.847232,
         "totalMs": 1695
       },
       "clone": {
         "count": 1,
         "avgMs": 1250,
         "medianMs": 1249.728768,
         "maxMs": 1249.728768,
         "totalMs": 1250
       },
       "checkout": {
         "count": 23,
         "avgMs": 26,
         "medianMs": 25.03808,
         "maxMs": 114.301184,
         "totalMs": 595
       },
       "reset": {
         "count": 33,
         "avgMs": 18,
         "medianMs": 16.93696,
         "maxMs": 31.168256,
         "totalMs": 593
       },
       "merge": {
         "count": 20,
         "avgMs": 230,
         "medianMs": 349.608448,
         "maxMs": 443.538432,
         "totalMs": 4608
       },
       "oldLevel": "debug",
       "trace_id": "3d152da3d3ab4b97b4ae90b410cea4ef",
       "span_id": "e1ee2ef83ffdb686",
       "trace_flags": "01"

@jamietanna jamietanna force-pushed the chore/git-instrument-impl branch 4 times, most recently from 4eba28c to 6e53101 Compare November 28, 2025 16:56
@jamietanna jamietanna force-pushed the chore/git-instrument-impl-result branch from 9f74cc8 to 3c36ae1 Compare November 28, 2025 17:01
@jamietanna

This comment was marked as outdated.

@jamietanna

This comment was marked as outdated.

@jamietanna jamietanna force-pushed the chore/git-instrument-impl branch from 6e53101 to 2aca5e2 Compare November 28, 2025 17:42
@jamietanna jamietanna force-pushed the chore/git-instrument-impl branch from 2aca5e2 to 0918850 Compare December 1, 2025 10:16
Base automatically changed from chore/git-instrument-impl to main December 1, 2025 10:37
@jamietanna jamietanna force-pushed the chore/git-instrument-impl-result branch 3 times, most recently from 9bd3f58 to fa88ff4 Compare December 1, 2025 10:46
@jamietanna jamietanna marked this pull request as ready for review December 1, 2025 10:46
@jamietanna jamietanna changed the title chore(git): report stats fix(git): report Git operation statistics at end of run Dec 1, 2025
@jamietanna
Copy link
Contributor Author

Let's get #39653 in first

Now we have instrumented our Git operations with tracing, we can make
sure that users who are not using OpenTelemetry still get insights into
their Git operation usage using a debug log output.

Following how other insights are produced, we can introduce a new type
in `lib/util/stats` for `GitOperationStats`, which will track the
overall execution for each operation type, and output them in the same
style as we use for other timing reports.

At the end of a Renovate run, we'll then make sure that this is logged
into a debug log for further investigation.

To create the report, we can make sure that we introduce a new
OpenTelemetry `SpanProcessor`, which will run when we end our Run, and
store any Git operations in `GitOperationStats`.

This adapts code from GPT-4.1 and code from [0].

Co-authored-by: gpt-4.1 <[email protected]>

[0]: https://github.com/yurisouza/monitoria-sdk/blob/ca9d751fa7c3a785a04f87b8966bb782b4617021/src/adapters/nestjs/span-duration-tracker.ts#L18-L27
@jamietanna jamietanna force-pushed the chore/git-instrument-impl-result branch from fa88ff4 to e6f3c57 Compare December 1, 2025 15:47
@jamietanna jamietanna added this pull request to the merge queue Dec 1, 2025
Merged via the queue into main with commit b5cd031 Dec 1, 2025
39 of 41 checks passed
@jamietanna jamietanna deleted the chore/git-instrument-impl-result branch December 1, 2025 15:59
@github-actions
Copy link
Contributor

github-actions bot commented Dec 1, 2025

🎉 This PR is included in version 42.29.3 🎉

The release is available on:

Your semantic-release bot 📦🚀

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants