Skip to content

perf(javascript): index source-map ignore lists per leaf - #1905

Merged
morluto merged 1 commit into
morluto:mainfrom
cyanseek:codex/index-source-map-ignore-lists
Oct 11, 2026
Merged

morluto merged 1 commit into
morluto:mainfrom
cyanseek:codex/index-source-map-ignore-lists

Conversation

@cyanseek

Copy link
Copy Markdown
Contributor

Summary

Index source-map ignore lists once per leaf instead of scanning the entire list for every source declaration. This removes quadratic work from trace-web-source-location / trace_web_source_location while preserving their complete Evidence.

Problem and expected behavior

inspectSourceMapValue currently calls includes(sourceIndex) for each source. Maps with many third-party sources therefore spend O(sources × ignore-list entries) work just assigning ignored, even when tracing a single generated point. Ignore lists are used by real bundlers to identify vendor/framework sources (Angular/Chrome example).

On base 7c207c7, a generated 3,777,849-byte map with 200,000 sources and ignore-list entries takes about 10 seconds through the public CLI. It is within the existing 4 MiB and 262,144-record limits. This PR addresses that independently reproduced cost; it does not claim to resolve the broader #1065 roadmap.

Change and scope

  • Create a leaf-local Set only when projecting declarations. The browser-collection profile still skips declaration projection.
  • Preserve standard-field precedence, including an explicitly empty ignoreList, and fallback to x_google_ignoreList when the standard field is absent.
  • Preserve raw arrays, duplicate source identities, section-local indexes, output ordering, and all existing limits. No shared cache or new configuration.

Contract and boundary impact

  • Semantic owner and earliest changed stage: validated source-map declaration projection in SourceMapFormat.ts.
  • CLI and MCP/tool-catalog contract: unchanged.
  • Provider, bridge, target-format, or platform compatibility: existing local source-map codec; no added format or platform support.
  • Evidence, artifact, provenance, or reconstruction contract: unchanged; complete baseline/candidate outputs match.
  • Process execution, authorization, cleanup, or containment impact: none; the existing owned codec and its memory/deadline limits remain in use.
  • Generated metadata, package, or installation impact: none.

Evidence and regression coverage

Three alternating baseline/candidate pairs per generated input, fresh processes, Windows x64 / Node 24.18.0 / npm 11.16.0. The baseline and candidate share the same complete build except for the changed compiled module. Values below are medians, in milliseconds:

Sources / ignored entries Map bytes Declaration inspection, base → candidate Complete CLI, base → candidate
1,000 13,849 2.77 → 2.49 884 → 886
80,000 1,417,849 1,349 → 38 2,496 → 1,233
200,000 3,777,849 8,577 → 71 10,059 → 1,675

These are capacity workloads, not measurements of an Angular application. The small case has no meaningful end-to-end improvement. Each query pays this cost once; the table does not multiply local savings by a hypothetical call count.

Reproducible map construction (n selects the table row):

const text = JSON.stringify({
  version: 3,
  sources: Array.from({ length: n }, (_, i) => `m${i}.ts`),
  names: [],
  mappings: "AAAA",
  ignoreList: Array.from({ length: n }, (_, i) => i),
});

Trace generated position { line: 1, column: 0 } against a retained script containing generated, with the explicitly selected local map and URL context https://app.test/app.js.map. Direct inspectSourceMap(text) measures the declaration-inspection column; the CLI column includes startup, artifact acquisition, the actual owned codec, and complete output delivery.

  • Full inspected-structure and complete CLI Evidence hashes match for every baseline/candidate pair.

  • Actual stdio MCP and CLI outputs match on both revisions for the 200,000-source map, a 200,000-source map without an ignore list, and the unchanged published lossless-json@4.3.1 UMD source map (44,946 bytes). MCP text/structured output matches; subsequent ping succeeds; clients and transports are closed with awaited cleanup.

  • Added a semantic regression covering duplicate source URLs with different ignored states, duplicate ignore entries, independent section indexes, legacy fallback, and explicit-empty precedence. It preserves existing semantics rather than asserting fragile wall-clock thresholds.

  • Representative output remains a mapped m0.ts source with ignored: true and explicitly unavailable embedded content for the generated cases.

  • Remaining proof gaps: no workload-frequency estimate or whole-application speedup claim; no Linux/macOS/Bun timing or live-browser capture claim.

  • Observed, derived, and inferred claims remain distinguishable.

  • Artifact identity, source provenance, and failed attempts remain preserved.

  • Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.

Validation performed

  • npm run build:cached — passed.
  • npm run test:focused -- src/javascript/sourceMaps/TraceSourceMap.test.ts src/application/WebSourceLocationService.test.ts tests/boundary/javascript/sourceMapDecoder.test.ts tests/boundary/mcp/webSourceLocationMcp.test.ts tests/acceptance/webSourceLocationCli.test.ts tests/boundary/browser/webSourceMapFetcher.test.ts tests/boundary/browser/webSourceMapNestedCoordinates.test.ts tests/boundary/filesystem/webSourceLocationArtifacts.test.ts — 145 tests passed across 8 files, including the real Windows owned codec and public CLI/MCP workflow.
  • npm run check — passed, including architecture guards.
  • git diff --check — passed.

Compatibility, safety, and release

  • Breaking changes or migration steps: none.
  • Real Hopper/Ghidra, browser, or OS coverage: local source-map CLI/MCP on Windows x64; no Hopper/Ghidra or live-browser changes or verification.
  • Package or release metadata impact: none.
  • Security, privacy, process, or containment review: map text and declarations remain unchanged; the temporary index is confined to one validated leaf inside existing codec limits.

Review checklist

  • This PR addresses a concrete problem or an agreed enhancement.
  • The PR has one focused outcome and the title follows type(scope): outcome.
  • Related issue is linked, or the reason for not linking one is stated above.
  • Tests cover changed observable behavior and meaningful failure paths.
  • Owning docs, contracts, and generated metadata are updated where needed.
  • User-visible CLI/MCP changes include representative output.
  • I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 11, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-11T16:44:39.196988Z 2461100 PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@morluto
morluto merged commit 898d509 into morluto:main Oct 11, 2026
@github-actions github-actions Bot mentioned this pull request Oct 11, 2026
3 tasks done
@morluto

morluto commented Oct 11, 2026

Copy link
Copy Markdown
Owner

Thanks for the source-map optimization, @cyanseek. Building the ignore-list Set once for each projected leaf removes a repeated scan per source while preserving existing output. Your regression covers duplicate source identities and standard-versus-legacy ignore-list precedence. We’re glad to have you contributing to REA.

@morluto morluto added performance Performance regressions, scaling limits, and bounded-resource work area: native Native analysis and decompilation, including Ghidra, IDA, and Hopper analysis behavior labels Oct 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: native Native analysis and decompilation, including Ghidra, IDA, and Hopper analysis behavior performance Performance regressions, scaling limits, and bounded-resource work

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants