Skip to content

fix(ghidra): disclose missing PDB symbol coverage - #2275

Merged
morluto merged 5 commits into
morluto:mainfrom
hifzabuildsai:fix/ghidra-pdb-coverage
Oct 13, 2026
Merged

morluto merged 5 commits into
morluto:mainfrom
hifzabuildsai:fix/ghidra-pdb-coverage

Conversation

@hifzabuildsai

@hifzabuildsai hifzabuildsai commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Problem

Refs #2259. A PE's CodeView metadata can reference a PDB that Ghidra did not load. REA's private executable import omits the sibling PDB, but successful analysis Evidence does not disclose missing symbol coverage. An empty name search can therefore look complete without explaining why original names are unavailable.

Change

  • Report Ghidra's post-analysis PDB Program Information attributes in the authenticated handshake: file, GUID, hexadecimal age, legacy signature, version, and nullable loaded state. Preserve source strings and absent attributes.
  • Carry a limitation into PE health, inventory, search, and function observations when referenced symbols are not loaded or their load state is unknown. Include identity and an address-based next probe. A missing bridge report remains unknown.
  • Read the established handshake before direct PE calls; the client reuses its existing startup promise rather than launching another engine. Rename the shared handshake projection to reflect its broader ownership.
  • Add pdb_observations: program-info-v1 to PE analysis profiles so snapshots produced before this reporting change have a different identity.
  • Document the symbol-coverage boundary. A reported loaded flag does not independently prove a matching PDB identity or complete symbols. The change does not copy PDBs, contact symbol servers, change analyzers, or add mutation authority.

Validation

Published commit 5890b2a076127dd2801e8aa781928d7106a766e9 has the exact validated tree f5d14d3dc7b10ab1be8971f882ae05002318af52 of local commit 23b6b11176fefa9c94ffb16881d7feed5eef332a, based on upstream 11fe5a13f8aee87d0c0e036a9f497407e085fe48. Linux x64, repository-pinned Node 24.18.0 and npm 11.16.0:

  • npm run test:focused -- src/ghidra/GhidraSessionValues.test.ts src/ghidra/GhidraProvider.behavior.test.ts src/ghidra/GhidraInventoryValues.test.ts tests/boundary/mcp/ghidraInventoryMcp.test.ts — 4 suites, 85 tests passed.
  • npm run check — passed: runtime compilation, typecheck, formatting, lint (0 errors; warnings reported), knip, metadata, resolved module boundaries, architecture guards.
  • npm run docs:check — passed, including both README consistency tests.
  • npm run verify:test-discovery — all 950 test files discovered exactly once.
  • git diff --check — passed.

The regressions first failed on the original source: the handshake rejected PDB facts and direct searches skipped the handshake projection. After the change they pass. Public MCP testing uses the actual provider/service/server pipeline with synthetic bridge facts, a generated PE fixture, retained Evidence, and function snapshot replay. It does not establish behavior of an installed Ghidra engine.

The Java APIs and attribute meanings were checked against Ghidra 12.1.4 source:
PdbParserConstants and PdbProgramAttributes.

Remaining verification — draft

Real producer verification (2026-10-12)

Ghidra 12.1.4 and Temurin JDK 21.0.12.1+1 were downloaded from official GitHub releases, and both published SHA-256 digests matched. All packaged Java bridge sources compiled against the installed Ghidra jars.

On Linux x64, Clang/LLD 18.1.8 cross-compiled a small self-owned C fixture to x86_64-pc-windows-msvc PE32+ with a matching PDB and a relative CodeView PDB path. This is a Linux-host PE/PDB producer check, not Windows/MSVC-host verification.

  • Executable SHA-256: 3828e3c48d2670453cd054837a9a44f2ad91d95895013460de83eb82c6a89a6c.
  • PDB GUID: 7047ef95-97c7-9e00-4c4c-44205044422e; hexadecimal age 1.
  • With the sidecar, actual headless Ghidra loaded the PDB and recovered add_numbers and main. Calling the compiled production bridge's pdbObservation against this Program returned loaded: true; the production TypeScript limitation projection returned no missing-PDB warning.
  • An identical executable copied to a directory without the PDB produced FUN_140001000 and entry. Ghidra omitted the PDB Loaded attribute. The production bridge preserved that omission as loaded: null; the TypeScript projection reported the referenced file/GUID/age and unknown load state. It did not manufacture a false loaded state.
  • Producer probing used a test-only headless script invoking the production observation method on the actual analyzed Program. It did not exercise the authenticated REA socket handshake or full CLI/MCP successful Evidence path.

Remaining host/provider gap

Actual REA CLI and MCP search attempts remain blocked before engine launch by this environment's process-ownership prerequisite: Node-spawned ps -axo pid=,ppid=,pgid=,uid=,stat=,command= fails with fatal library error, lookup self. The same command works from the outer shell, and the Node subprocess failure was checked independently. Ownership checks were not bypassed, and the failed workflows are not counted as PASS.

Complete real CLI/MCP outcome propagation and lifecycle/cleanup verification still need a compatible host. Windows native ownership and MSVC-host reproduction were not exercised. Keep the PR draft until the required real-provider lane completes.

This addresses the reporting portion of #2259. Automatic or caller-selected matching-PDB admission remains outside this change; do not close the broader issue as fully resolved without the maintainer's scope decision.

The added output consists of six handshake attributes and one limitation when needed. Cold/warm real-engine elapsed time, CPU, peak process-family memory, and completed-task token costs have not been measured.

@matbrgz

matbrgz commented Oct 13, 2026

Copy link
Copy Markdown
Contributor

Adversarial Review Finding: sessionLimitations() Evaluated Statically Before Handshake

In src/ghidra/GhidraProviderClient.ts:

Problem: Static Array Capture Drops pdbLimitations

At lines ~660-675:

  const pdbLimitations: string[] = [];
  ...
  const sessionLimitations = (): readonly string[] => [
    ...providerLimitations,
    ...targetLimitations,
    ...(psp ? GHIDRA_PSP_LIMITATIONS : []),
    ...softFloatLimitations,
    ...ghidraExtensionLimitations(extensions),
    ...seedLimitations,
    ...pdbLimitations,
    ...(releaseLimitation === undefined ? [] : [releaseLimitation]),
  ];

In the PR, sessionLimitations is defined as a closure calling the array spread. However, let's examine the lifecycle and timing of tool execution vs session startup:

  1. In projectExecution:
    limitations: [
      ...limitationsFor(operation),
      ...observationLimitations,
      ...sessionLimitations(),
    ],
  2. When checkSessionObservations runs:
    const checkSessionObservations = async (
      operation: AnalysisOperation,
      info: GhidraSessionInfo,
    ): Promise<AnalysisError | undefined> => {
      if (target.format === "pe")
        pdbLimitations.splice(
          0,
          pdbLimitations.length,
          ...ghidraPdbLimitations(info.pdb),
        );
      ...
  3. However, look at how open_binary / initial operations are handled in GhidraProviderClient.ts:
    When health runs:
    const started = await client.start();
    if (!started.ok) ...
    const failed = await checkSessionObservations(operation, started.value);
    if (failed !== undefined) return err(failed);
    return ok(
      createAnalysisExecution(started.value, committedProfile.provider, {
        rawResult: started.value,
        analysisProfile: committedProfile,
        limitations: [...sessionLimitations()],
      }),
    );
    Notice that health explicitly splices pdbLimitations and then invokes sessionLimitations(). This succeeds for health.
  4. But for target-free or cached replay operations where info.pdb is undefined (such as snapshot replay or when the provider client handles an operation without re-querying checkSessionObservations, or when pdb is omitted):
    ghidraPdbLimitations(undefined) returns:
    ["Ghidra did not report PDB attributes; debug-symbol coverage is unknown."]
    If the bridge handshake returns pdb: undefined (for example, on an older or non-PE target, or if a non-PE target executes where target.format !== "pe"), pdbLimitations is empty because if (target.format === "pe") skips updating it entirely.
  5. But even more critically: Non-PE targets with CodeView-like debug metadata or other formats.
    If a PE target session fails or times out during client.start(), or an error occurs before checkSessionObservations completes, any partially projected evidence or error envelope that references sessionLimitations() will have pdbLimitations unpopulated (or stale from a previous target in cached workers).

Non-Atomic Cache Invalidation on Reconnect / Reopen

If client disconnects or restarts in a persistent provider instance (as seen in Ghidra reconnection workflows, e.g. #2014):
pdbLimitations is mutated in place via splice(). If a subsequent session handshake times out or encounters a socket failure, pdbLimitations retains the previous session's mutated values unless cleared on disconnect.
To prevent leaking PDB limitations across reconnections, pdbLimitations.length = 0 must be guaranteed on disconnect/reset.

@morluto

morluto commented Oct 13, 2026

Copy link
Copy Markdown
Owner

@greptileai review

@greptile-apps

greptile-apps Bot commented Oct 13, 2026

Copy link
Copy Markdown
Contributor

This PR is set for a Plus review. TREX isn't supported with Plus or Apex reviews yet, so Greptile reviews this PR at Base while TREX is on.

@greptile-apps

greptile-apps Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

[Medium impact] No blocking code issue was found; the missing budget tests are non-blocking.

Findings

  1. P2 PDB limits lack boundary tests ▶
Fix with agent prompt
### Issue 1
src/ghidra/GhidraSessionValues.ts:71-81
The new 4096-unit limit has no boundary tests. Existing PDB tests only use short strings and never supply `resource_limit`. A future mismatch between the Java bridge and TypeScript reader could reject valid startup reports or lose the omission warning without failing these tests.

Add tests for exactly 4096 units, combined overflow across individually valid fields, and omitted attributes reaching PE Evidence.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Summary

The PR reports Ghidra's PDB observations in PE Evidence. The latest changes bound PDB strings and retain a coverage caveat even when Ghidra reports a loaded PDB.

  • PE analysis now explains when Ghidra may lack PDB symbols.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart LR
  A[Ghidra Program Information] --> B[Bound PDB strings]
  B --> C[Authenticated handshake]
  C --> D[Read PDB observations]
  D --> E[PE coverage limitations]
  E --> F[Health, search, inventory and function Evidence]
  F --> G[Saved snapshot replay]
Loading

Reviews (6) · Last reviewed commit: "fix(ghidra): bound PDB handshake and ret..." · Reviewed by Greptile

@morluto
morluto marked this pull request as ready for review October 13, 2026 11:48
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-13T13:41:29.471623Z c84ced1 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: a6df916523

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/ghidra/GhidraSessionValues.ts Resolved

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6fc0e136bc

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread bridge/ghidra/ReaGhidraBridge.java Resolved
Comment on lines +71 to +81
.refine(
(pdb) =>
[
pdb.file,
pdb.guid,
pdb.age,
pdb.signature,
pdb.version,
].reduce<number>((sum, value) => sum + (value?.length ?? 0), 0) <=
PDB_ATTRIBUTE_UTF16_BUDGET,
"PDB attributes exceed the aggregate wire budget",

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 PDB limits lack boundary tests

The new 4096-unit limit has no boundary tests. Existing PDB tests only use short strings and never supply resource_limit. A future mismatch between the Java bridge and TypeScript reader could reject valid startup reports or lose the omission warning without failing these tests.

Add tests for exactly 4096 units, combined overflow across individually valid fields, and omitted attributes reaching PE Evidence.

Prompt To Fix With AI
This is a comment left during a code review.
Path: src/ghidra/GhidraSessionValues.ts
Line: 71-81

Comment:
**PDB limits lack boundary tests**

The new 4096-unit limit has no boundary tests. Existing PDB tests only use short strings and never supply `resource_limit`. A future mismatch between the Java bridge and TypeScript reader could reject valid startup reports or lose the omission warning without failing these tests.

Add tests for exactly 4096 units, combined overflow across individually valid fields, and omitted attributes reaching PE Evidence.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

Note: If this suggestion doesn't match your team's coding style, reply to this and let me know. I'll remember it for next time!

@morluto

morluto commented Oct 13, 2026

Copy link
Copy Markdown
Owner

Review findings addressed at their producing boundaries and consumers. Latest changes were reviewed from source; tests and provider runs were skipped at the maintainer's request. Findings requiring separate decisions or verification remain open.

  1. thread 1

A loaded PDB was incorrectly treated as complete symbol authority. ghidraPdbLimitations now emits a loaded-state caveat independently of missing/unloaded states, so the shared PE execution boundary carries it into health, inventory, search and function Evidence. Updated the existing loaded-state regression expectation and documentation; a reported load never attests full names/types or absence after an empty search.

Published in c84ced1. Validation: inspected the original finding, producing boundary and affected consumers; source formatting only. Tests, builds, checks, lint, typechecks, provider workloads and CI reruns were skipped at the user’s explicit request. Any authored regressions are unexecuted. No directly related cleanup remains identified in this bounded source pass; broader contribution-scope and real-provider verification requirements remain before integration.

  1. thread 1

The bridge serialized target-controlled PDB strings without accounting for expansion, and every PE execution could repeat them. PDB attributes now share a producer-side 4096 UTF-16-unit budget, at most 24 KiB after worst-case JSON escaping. Whole attributes that do not fit become null plus an explicit bounded resource_limit report naming omissions; values are not truncated. The handshake schema checks both individual and aggregate size and ghidraPdbLimitations propagates the omission as unknown, including when all attributes were omitted. Retained values and loaded state remain unchanged.

Published in c84ced1. Validation: inspected the original finding, producing boundary and affected consumers; source formatting only. Tests, builds, checks, lint, typechecks, provider workloads and CI reruns were skipped at the user’s explicit request. Any authored regressions are unexecuted. No directly related cleanup remains identified in this bounded source pass; broader contribution-scope and real-provider verification requirements remain before integration.

@morluto morluto added bug Something isn't working area: native Native analysis and decompilation, including Ghidra, IDA, and Hopper analysis behavior area: bridge Hopper bridge (Python) related changes area: cli CLI adapter and one-shot commands P2: medium Medium priority issue labels Oct 13, 2026
@morluto
morluto merged commit 865c35e into morluto:main Oct 13, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: bridge Hopper bridge (Python) related changes area: cli CLI adapter and one-shot commands area: native Native analysis and decompilation, including Ghidra, IDA, and Hopper analysis behavior bug Something isn't working P2: medium Medium priority issue

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants