Skip to content

fix(managed): charge retained member facts instead of decoding bounds - #1768

Merged
morluto merged 4 commits into
morluto:mainfrom
flitzrrr:fix/managed-member-decode-budget
Oct 11, 2026
Merged

morluto merged 4 commits into
morluto:mainfrom
flitzrrr:fix/managed-member-decode-budget

Conversation

@flitzrrr

@flitzrrr flitzrrr commented Oct 11, 2026 •

Copy link
Copy Markdown
Contributor

Summary

inspect_managed_members returns no types, methods or call edges for ordinary mid-size assemblies. For example, Newtonsoft.Json 13.0.3 (lib/netstandard2.0, 705,296 bytes) reports coverage.state: "unavailable" with "Managed metadata exceeds the 67108864 byte representation budget". The budget was charged with decoding upper bounds that were never returned, rather than with the facts that are actually retained. After this change the same file yields 496 types, 4,185 methods and 18,846 call edges.

I did not open a separate issue because the reproduction uses a public NuGet package and is included below.

Problem and expected behavior

Before decoding, every method body reserved 256 bytes per IL byte (ManagedMethodBodyReader), every exception section 256 bytes per clause, and every member signature 256 bytes per blob byte (budgetedSignature). These reservations bound transient decoding bookkeeping, but they stayed charged for the whole inspection. The budget therefore scaled with the total IL and signature size, not with the output. Instrumenting the budget on the base revision for the Newtonsoft.Json file above showed 41.0 MiB from method-body bounds and 10.8 MiB from signature bounds, before any projection.

The final projection admission (admitManagedProjection) also charged every decoded string at six bytes per UTF-16 unit, every key at the same rate, and every scalar at 32 bytes. That is roughly eight times its JSON size.

Expected: the budget limits what the inspection retains and projects, so assemblies whose facts fit are inspected completely.

Change and scope

  • ManagedDecodeBudget.admitTransient checks a decoding bound against the remaining capacity without consuming it. reserve now builds on it.
  • IL, exception-clause and signature decoding use admitTransient for their bounds, then reserve the decoded body or signature that is retained, through admitManagedProjection.
  • admitManagedProjection charges the exact UTF-8 JSON bytes of decoded strings, keys and scalars. Escaped control characters and multibyte text still count at their encoded size. Heap-string reservations made before decoding keep their worst-case bound because the content is unknown at that point.

Not included: when the retained facts genuinely exceed the budget, members are still reported as unavailable rather than partial. System.Private.ServiceModel 4.7.0 (2.3 MB) remains such a case. inspect_managed_artifact similarly labels a budget stop as partial without facts. Returning partial member inventories would change what compare_managed_members and the managed application graph can assume, so it deserves its own change.

Contract and boundary impact

  • Semantic owner and earliest changed stage: managed static provider decoding (src/dotnet/ManagedDecodeBudget.ts, ManagedMethodBodyReader.ts, ManagedMemberInstructionDecoder.ts, ManagedMemberRows.ts).
  • CLI and MCP/tool-catalog contract: none; schemas and tools are unchanged. More assemblies return populated results instead of unavailable.
  • Provider, bridge, target-format, or platform compatibility: PE/CLI only; no change to admitted formats.
  • Evidence, artifact, provenance, or reconstruction contract: unchanged shapes; the representation budget and its 64 MiB value are unchanged.
  • Process execution, authorization, cleanup, or containment impact: none; no target code is loaded. Per-item decoding stays bounded before allocation.
  • Generated metadata (docs/public/product-catalog.json), package, or installation impact: none.

Evidence and regression coverage

  • Tests added or updated:
    • src/dotnet/ManagedMemberInspector.test.ts: a fixture method with 200,001 IL bytes plus an 80,003-byte signature. Each bound fits alone, but together they exceeded 64 MiB on the base revision.
    • src/dotnet/ManagedDecodeBudget.test.ts: exact charging of a 12 MiB ASCII string; escaped and multibyte text still exceeds the budget at its encoded size.
  • Base reproduction: both new "charges decoded output" and "charges decoded strings" cases fail on the base revision (the first returns unavailable) and pass with this change.
  • User-visible CLI/MCP output: rea inspect-managed-members ~/.nuget/packages/newtonsoft.json/13.0.3/lib/netstandard2.0/Newtonsoft.Json.dll --format json
    • Before: types: [], methods: [], call_edges: [], coverage: { state: "unavailable", issues: [{ code: "resource-limit", detail: "Managed metadata exceeds the 67108864 byte representation budget" }] }
    • After: 496 types, 4,185 methods, 18,846 call edges, coverage.state: "partial" with no issues. The partial state comes from member signatures or bodies that are not fully decoded, as before.
  • Remaining proof gaps: no Windows or Linux run; no real-provider lane applies.

For evidence-bearing changes:

  • Observed, derived, and inferred claims remain distinguishable.
  • Artifact identity, source provenance, and failed attempts remain preserved.
  • Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.

Validation performed

On macOS (Darwin 27.0, Apple Silicon), Node 24.18.0:

  • npm run test:fast — 376 files, 4272 passed, 7 skipped
  • npm run test:focused -- tests/boundary/providers/managed/*.test.ts tests/conformance/providers/managed/managedNativeVerification.test.ts tests/boundary/mcp/managedMcp.test.ts tests/acceptance/managedCliOutput.test.ts — 9 files, 44 passed
  • npm run check — passed
  • npm run verify:test-discovery — 814 test files discovered exactly once
  • npm run build:cached and the CLI run above — passed

Review follow-up

  • b82218d: transient decoding bounds accumulate per method or signature until released; JSON bytes of decoded strings are counted without serializing.
  • dd64d3c: derived call and field-access records are admitted as they are built.
  • New regressions (IL plus EH over budget, derived edges over budget) fail on the previous head and pass here.
  • npx vitest run src/dotnet — 92 passed; managed boundary, MCP and acceptance — 44 passed; npm run check:fast — passed. Newtonsoft.Json and the 730 KB assembly still return every member.

Compatibility, safety, and release

  • Breaking changes or migration steps: none.
  • Real Hopper/Ghidra, browser, or OS coverage: not applicable; macOS only.
  • Package or release metadata impact: none.
  • Security, privacy, process, or containment review: total retained output stays bounded by the unchanged budget. Transient decoding is still checked before each allocation, and only its capacity is no longer consumed.

Review checklist

  • This PR addresses a concrete problem or an agreed enhancement.
  • The PR has one focused outcome and the title follows type(scope): outcome.
  • Related issue is linked, or the reason for not linking one is stated above.
  • Tests cover changed observable behavior and meaningful failure paths.
  • Owning docs, contracts, and generated metadata are updated where needed.
  • User-visible CLI/MCP changes include representative output.
  • I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

🤖 Generated with Claude Code

inspect_managed_members returned no types or methods for ordinary
assemblies: Newtonsoft.Json 13.0.3 (netstandard2.0, 705 KB) exceeded
the 64 MiB representation budget. Each method body reserved 256 bytes
per IL byte and each signature 256 bytes per blob byte before decoding,
and those worst-case bounds were never returned, so the budget scaled
with total IL size rather than with the retained output. The final
projection admission also charged every decoded string at six bytes per
character and every scalar at 32 bytes, about eight times its JSON size.

Bound IL, exception-clause and signature decoding with a transient
admission that does not consume capacity, then reserve the decoded body
or signature that is actually retained. Projection admission now
charges the exact UTF-8 JSON bytes of decoded strings, keys and scalars;
escaped and multibyte text still counts at its encoded size.

Newtonsoft.Json now yields 496 types, 4185 methods and 18846 call
edges. Assemblies whose retained facts exceed the budget, such as the
2.3 MB System.Private.ServiceModel 4.7.0, remain unavailable.
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 11, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-11T10:17:50.027713Z e376cf2 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 70a862f39a

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/dotnet/ManagedDecodeBudget.ts Resolved
Comment thread src/dotnet/ManagedMethodBodyReader.ts Resolved
Review follow-up. A method's IL bound and its exception-section bounds
were each checked against the same remaining capacity although their
bookkeeping is live at the same time, so 40 MiB of IL plus 40 MiB of
exception clauses passed a 64 MiB budget. Transient admissions now
accumulate until the method body or signature is built, and reserve
respects bounds that are still live.

Projection admission also serialized each decoded string to measure it,
which could allocate about six times a near-limit string before the
check. It now counts the exact escaped UTF-8 bytes without building the
JSON text.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b82218de32

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/dotnet/ManagedMethodBodyReader.ts Resolved
Review follow-up. Released IL bounds no longer stand in for the call and field-access records that edges() derives from anchors. Each record is now admitted before the graph grows further, so a crafted assembly cannot materialize the whole derived graph before the representation guard fails.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: dd64d3c56f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/dotnet/ManagedMemberRows.ts Resolved
Review follow-up. The caller name of call and field-access records was interpolated per anchor before any admission saw it. It is now built once per method, after its exact JSON size, counted from the type and member names, has been admitted.
@morluto

morluto commented Oct 11, 2026

Copy link
Copy Markdown
Owner

Thanks for reworking the managed metadata budget, @flitzrrr. It now checks transient decoding bounds without consuming the retained-output budget, then charges the projection that is actually kept. That lets the PR’s 705 KB Newtonsoft.Json reproduction return 496 types and 4,185 methods under the existing cap.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: domain Domain logic, errors, and pure functions bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants