Add configurable concurrency limits to prevent OOM in findChangedDependencies #1226
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
When running chromatic --only-changed after a rebase with many dependency changes in large monorepos, the CLI encounters OOM (Out of Memory) errors.
Cause
findChangedDependencies.ts uses nested Promise.all() calls that spawn unlimited concurrent operations:
Real-world scenario
715 manifest/lockfile pairs × N baseline commits = 1,430+ concurrent getDependencies() calls, each parsing large lockfiles and building memory-intensive dependency graphs → immediate OOM crash.
Proposed solution
Cap the concurrency to prevent memory usage from growing too high.
This PR
This PR adds per-level configurable concurrency limits using p-limit (already used elsewhere in the CLI):
Default behavior: Unlimited (same as before) - no breaking changes
Opt-in limits: Users experiencing OOM can set environment variables to control concurrency
Does it work? Yes
I have tested this solution locally by doing the following:
Final solution
I don't expect this PR is the final solution, but sharing so that Chromatic can experiment. I would expect that some sensible defaults should be applied to the CLI or a single config option with a single plimit limiter is needed rather than two.