Skip to content

fix(ghidra): preserve function facet and call coverage - #3002

Merged
morluto merged 12 commits into
mainfrom
fix/2951-ghidra-dossier-coverage
Oct 13, 2026
Merged

morluto merged 12 commits into
mainfrom
fix/2951-ghidra-dossier-coverage

Conversation

@N0zoM1z0

@N0zoM1z0 N0zoM1z0 commented Oct 13, 2026 •

Copy link
Copy Markdown
Collaborator

Ghidra dossiers omitted the availability of successfully collected function facets, so explicit function comparison could remain unknown. Call coverage also claimed completeness when its inventories omitted unresolved calls or body-and-entry references, and caller-only gaps incorrectly made forward path reconstruction incomplete.

This change declares collected facets, preserves producer-owned relationship coverage through the adapter/cache/snapshots, and records unresolved sites only when their evidence is requested. One reference visitor owns the full function body plus exact entry: inventory reads stream omission counts, while dossiers reuse their collected references. Known containing-function targets remain resolved even when Ghidra's exact-entry getter omits them. Typed limited_directions lets forward searches retain caller explanations without weakening complete callee coverage. Old call-coverage snapshot profiles are rejected under v5. Comment collection uses Ghidra's indexed comment-address iterator. Shared optional diagnostics keep three tagged coverage states instead of six duplicate Schema branches, preserving compact Evidence field types within the existing expansion budget.

Validation on 80daf923fe224ab6c9110f3756361e1b0c768705, based on main 1f122c02b71dae7f4109f35ed834f82b96fbf247:

  • Full static gate, normal pre-push checks, runtime build, and docs checks pass; 268 related source cases and 43 MCP boundary cases (including all compact profile checks) pass. The actual SDK-advertised schemas enforce direction cardinality and reject duplicates.
  • Real Linux Ghidra 12.1.4 / Java 21 producer regression checks manual references on non-call instructions, interior target ownership, omitted callers/callees, directional dossier coverage, and no-return behavior. The old bridge fails the manual-reference coverage assertion; the new bridge passes. Input bytes and owned-process cleanup are verified.
  • Public MCP direct/self-comparison, comment write/read/invalidation, and unresolved callback/path/comparison workflows pass. Incomplete calls remain unknown.
  • A real 128-call ELF returns a 7,239-byte callee inventory instead of its explicitly requested 2,048,517-byte full dossier. Repeated inventory requests take 7.37–8.25 ms with stable Evidence IDs. Cold startup is measured separately; these are shared-host client timings, not isolated producer speedups or token estimates.
  • The complete native/AArch64 lane passed on frozen pre-Schema-cleanup head 08822d779f: 384 successful public boundary calls, 52 expected rejections, CLI/MCP parity, snapshot/mutation/permission/cancellation and cleanup checks; 713.59 s wall and 1377.01 s waited-child CPU. Every recorded source/runtime hash matched after completion. Final post-cleanup/head 80daf923fe public direct/annotation, unresolved callback and real producer golden workflows pass; the exact-head full lane also passes: 384 successful boundary calls, 52 expected rejections, native plus crosscompiled AArch64 fixtures, 688.13 s wall / 1332.78 s waited-child CPU / approximately 4140 MiB summed-family RSS estimate. All 3,614 source and 1,540 runtime hashes match after completion, input immutability and owned-process cleanup are verified. Shared pages may be counted more than once; CPU/latency are shared-host observations.

The new head fixes this PR's compact input-schema CI regression. Upstream moved the JavaScript worker test into the compiled boundary lane. Additional local upstream PowerPC tests fail installation admission because their mock properties omit application.java.min; exact main/test/installation blobs match, and coverage parameters are never reached. The extended source selection has 291 passes and these three unrelated failures. No unrelated repair is included; fresh required CI remains the merge gate. Both PRs also fail the same unchanged ciTestReports.test.ts blob-report diagnostic fixture (Partial test diagnostics absent); test/report/artifact code blobs match main.

Closes #2951.

Declare successful query availability in the Java producer, including explicit unavailable pseudocode reasons. Carry observed static call-set coverage through the parsed provider result and Evidence without inventing facts for legacy omissions. Use Ghidra indexed comment addresses over the function body and entry, retaining cancellation checks and sparse collection.

Closes #2951

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 13, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-13T23:48:45.681272Z dbd3a5b New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: cd87d86dcb

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread bridge/ghidra/ReaGhidraBridge.java Resolved
@greptile-apps

greptile-apps Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

[Medium impact] The PR appears safe to merge, with no actionable defect found.

Summary

This PR preserves collected function facets and producer-owned call coverage throughout Ghidra reads and saved observations.

  • Ghidra dossiers say which function facts they actually contain.
  • Call coverage names which caller and callee lists have gaps.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Ghidra function body and entry] --> B[Shared reference visitor]
  B --> C[Compare references with returned call sets]
  D[Decoded instructions] --> E[Find calls without destinations]
  C --> F[Coverage and limited directions]
  E --> F
  F --> G[Adapter and snapshot]
  G --> H[Function comparison]
  G --> I[Forward call search]
  H --> J[Incomplete call sets remain unknown]
  I --> K[Caller-only gaps keep explanations without blocking search]
Loading

Reviews (8) · Last reviewed commit: "chore: merge main and reconcile Ghidra c..." · Reviewed by Greptile

Share decoded unresolved-call facts between dossiers, callee coverage and reference observations. Combine terminal and unresolved observations in one instruction traversal and retain site records only when requested. Keep missing procedure identities incomplete and validate unresolved locations at the provider boundary.

Bind snapshots to the corrected call coverage profile so older complete claims cannot replay under the new semantics. Add real public MCP callback trace/comparison regressions.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Comment thread bridge/ghidra/ReaGhidraBridge.java Resolved
Reuse containing-function ownership for resolved reference identities and independently compare those identities with the actual returned callee set. Keep known interior routes out of unresolved site records while retaining partial inventory coverage.

Advance the coverage profile to v3 and reject interim v2 cache bindings. Add a real Ghidra producer regression with restored reference metadata; the previous bridge fails the known-route assertion.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: af2256f84c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread bridge/ghidra/ReaGhidraBridge.java Resolved
Compare incoming reference source identities across the owned function body and entry with the actual returned caller set. Stream caller-only checks and reuse dossier reference observations. Combine caller/callee coverage from observed counts without reinterpreting serialized metadata, and share the body scope owner with comments and references.

Bind snapshots to function-body coverage semantics v4 and preserve the real off-entry caller regression.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 746a6a45bd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread bridge/ghidra/ReaGhidraBridge.java Resolved
Comment thread bridge/ghidra/ReaGhidraBridge.java Resolved
Integrate the latest captured main before required CI verification.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 3428cb101b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread bridge/ghidra/ReaGhidraBridge.java Resolved
Comment thread bridge/ghidra/ReaGhidraBridge.java Resolved
N0zoM1z0 and others added 6 commits October 13, 2026 22:17
Check body-and-entry call references independently of decoded instruction flow. Reuse collected dossier references and stream inventory-only coverage checks. Keep caller-only gaps distinct from incomplete outgoing coverage in offline path reconstruction, and invalidate prior snapshot profiles.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Share optional diagnostics across the three tagged relationship coverage states instead of advertising duplicate branches. Preserve coverage validation while keeping compact Evidence properties within the existing reference expansion budget. Verify invalid direction cardinalities against the actual SDK-advertised schema.

Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
Co-authored-by: Kaoru0822 <208281322+kaoru0822-kitauji@users.noreply.github.com>
@N0zoM1z0

Copy link
Copy Markdown
Collaborator Author

The compact input-schema regression is fixed in dd1c73b, preserving existing field-type assertions and the existing expansion budget. Current head80daf923 is based on main1f122c02. Full static/docs/normal pre-push checks and43MCP cases pass; real Ghidra direct/annotation/callback and producer goldens pass. Exact-head native/AArch64 verification is running.

The extended source check has291passes and three new upstream PowerPC mock admission failures. The fake installation properties contain only application.version=12.1.4 and omit application.java.min. Direct installation inspection returns java/version_unresolved with application.java.min is missing; test/provider/installation/policy blobs are identical to main. This happens before call-coverage parameters are produced, and no unrelated fixture repair is included.

@morluto morluto added area: native Native analysis and decompilation, including Ghidra, IDA, and Hopper analysis behavior bug Something isn't working labels Oct 13, 2026
@morluto
morluto merged commit 242e2b0 into main Oct 13, 2026
37 checks passed
@morluto
morluto deleted the fix/2951-ghidra-dossier-coverage branch October 13, 2026 23:52
morluto added a commit that referenced this pull request Oct 14, 2026
Share tagged readiness observations, retain partial provider facts, authenticate bridge identity, bound diagnostics, and preserve cancellation and lost-session reasons. Fail closed for unparsed MIPS ABI attributes and macOS saved Hopper databases without a script-capable bridge route.

Reviewed fixes: #2010, #2108, #2513, #2695, #2731, #2771, #2828, #2867, #2868, #2870, #2880, #2919, #2945, #2961, #2981, #2998, #3000, #3002.

Validation: 285 focused source/boundary tests passed. Real Ghidra, IDA, Linux Hopper, and Windows provider runs were not repeated.

BREAKING CHANGE: MIPS targets with unresolved GNU ABI attributes and saved Hopper databases on macOS are refused rather than advertised as supported.
morluto added a commit that referenced this pull request Oct 14, 2026
Preserve XML and Interface Builder identity at their parsing boundary, retain usable Objective-C slots with precise coverage, and keep native trace failures distinct from absence. Add caller-selected Interface Builder summaries backed by complete retained Evidence, and preserve comparison limitations.

Reviewed fixes: #1969, #2088, #2108, #2513, #2639, #2684, #2731, #2759, #2875, #2888, #2889, #2970, #2971, #3002.

Validation: relevant native/domain focused lane passed 142 tests; earlier Interface Builder/MCP checks passed 24 tests. Real native UI and Ghidra producer coverage was not repeated.
morluto added a commit that referenced this pull request Oct 14, 2026
Use one named input-schema projection for clients and catalog identity, preserve dialect declarations, and bound compact schema depth. Keep canonical SDK output validation when output advertisements are disabled. Release request-owned state on cancellation and carry upstream limitations through composed workflows. Preserve doctor full-output envelopes and validate navigation facts at their owner.

Reviewed fixes: #2088, #2108, #2513, #2684, #2693, #2695, #2836, #2864, #2868, #2877, #2933, #2998, #3002, #3015.

Validation: focused schema, SDK output-validation, context, and workflow tests passed; no schema dialect or canonical output validation was relaxed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: native Native analysis and decompilation, including Ghidra, IDA, and Hopper analysis behavior bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(ghidra): retain observed dossier facet and relationship coverage

2 participants