Skip to content

fix(firmware): name missing extractor dependencies in partial extraction - #1879

Merged
morluto merged 1 commit into
morluto:mainfrom
DeryFerd:fix/firmware-dependency-diagnostics
Oct 11, 2026
Merged

morluto merged 1 commit into
morluto:mainfrom
DeryFerd:fix/firmware-dependency-diagnostics

Conversation

@DeryFerd

Copy link
Copy Markdown
Contributor

Summary

Fixes #1853. When Unblob reaches partial extraction because an external extractor is absent, it reports an ExtractorDependencyNotFoundReport that already names the missing dependencies, but normalizeUnblobReport forwarded that value verbatim into diagnostics as { input_path, report }. A caller that landed on partial coverage therefore had to decode the raw producer report to learn which extractor was missing and at which stage. This PR normalizes those entries: the missing dependency names are collected from the report and any nested value, the producing handler is recorded as the stage, and the original report stays retained for provenance.

Problem and expected behavior

Smallest trigger: run extract_firmware on a firmware image whose format needs an extractor that is not on PATH. Unblob finishes its configured extraction, retains the carved region, and reports the missing extractor as a nested ExtractorDependencyNotFoundReport with dependencies: ["sasquatch"]. Prior behavior: the normalized result carried coverage: "partial" and a diagnostics entry of { input_path, report }, where report held the raw producer report, so the dependency name and the stage existed only inside an unparsed producer value. The violated invariant: a partial result should explain the exact missing dependency and extraction stage, and distinguish carving from content extraction, without making the caller reverse-engineer the producer's report shape. Expected behavior: the diagnostic names the missing dependencies and the producing stage inline, the raw report is still retained, and coverage stays partial rather than implying a corrupt target or an empty filesystem.

Change and scope

src/firmware/FirmwareReports.ts gains one normalization helper used at both places where nested extraction problems are collected: the chunk-report branch (which carries a handler_name and an extraction_reports array) and the multi-file branch. The helper walks the report and any nested value for ExtractorDependencyNotFoundReport entries, collects their dependencies in first-seen order, records the handler as stage, and keeps the original report under report. Entries that carry no dependency report keep their previous { input_path, report } shape, so unrelated diagnostics such as unknown chunk types are unchanged. Intentionally not included: no change to the extract_firmware result schema, no change to how coverage is derived, no attempt to install or resolve extractors, and no reinterpretation of a missing dependency as a corrupt target.

Contract and boundary impact

  • Semantic owner and earliest changed stage: firmware producer-report normalization in src/firmware/FirmwareReports.ts, which interprets the pinned Unblob report before it is projected into the extract_firmware result.
  • CLI and MCP/tool-catalog contract: no schema change. diagnostics is an open array of JSON values in the result contract, so the added stage and missing_dependencies fields are additive and the tool catalog does not move.
  • Provider, bridge, target-format, or platform compatibility: none beyond Unblob's own report shape; Binwalk inspection and the region-scan path are untouched, and the change is host-independent.
  • Evidence, artifact, provenance, or reconstruction contract: the original producer report is retained alongside the normalized fields, so provenance and the exact reported dependencies both remain available; carved region offsets and digests are unchanged.
  • Process execution, authorization, cleanup, or containment impact: none; this reads an already-produced report and does not change how the extractor runs, which dependencies are present, or how output is published.
  • Generated metadata (docs/public/product-catalog.json), package, or installation impact (CI generates and validates ignored catalogs and skills/; edit authored instructions in skill-src/ and never commit build outputs): none committed.

Evidence and regression coverage

  • Tests added or updated: a new unit case in src/firmware/FirmwareReports.test.ts builds a chunk report with a nested ExtractorDependencyNotFoundReport and asserts the normalized diagnostic names the dependency and the gzip stage while retaining the raw report; the existing multi-file case now asserts the same enriched shape for a nested dependency report, and the firmware integration fixture expectation for the dependency mode is updated to the enriched diagnostic.
  • Base reproduction or other evidence: on the base revision the new case fails because the diagnostic is only { input_path, report }; after the change the firmware unit suite passes with the dependency named inline.
  • User-visible CLI/MCP output: a partial extract_firmware result now lists the missing extractor and the stage directly in diagnostics instead of burying them in a raw producer report.
  • Remaining proof gaps: real Unblob extraction was not run locally because it requires the external extractor stack; the integration lane that exercises a real Unblob run is gated on that toolchain and was skipped here, so the fixture-level and unit-level coverage stand in for it.

For evidence-bearing changes:

  • Observed, derived, and inferred claims remain distinguishable.
  • Artifact identity, source provenance, and failed attempts remain preserved.
  • Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.

Validation performed

  • npx vitest run src/firmware/FirmwareReports.test.ts — 7 passed, including the new dependency-diagnostic case.
  • npx vitest run src/firmware/ tests/boundary/firmware/ — 23 passed, 30 skipped; the skipped cases are the real-toolchain lanes that need an installed Unblob.
  • npm run typecheck — exit 0.
  • npx oxlint on the changed files — 0 errors; the three warnings (complexity, max-lines-per-function, max-depth) are pre-existing on the base in normalizeUnblobReport and reproduced with the change stashed. oxfmt --check on the committed (LF) form — clean.
  • Real Hopper/Ghidra, browser, or OS lanes: not applicable; no provider, bridge, or platform-specific code is touched.

Compatibility, safety, and release

  • Breaking changes or migration steps: none. The change is additive on an open JSON-valued field; existing diagnostics keep their shape, and the enriched entries add fields rather than replacing the retained report.
  • Real Hopper/Ghidra, browser, or OS coverage: not applicable; the firmware unit and fixture suites cover the changed normalization.
  • Package or release metadata impact: none.
  • Security, privacy, process, or containment review: the change surfaces the exact missing extractor and stage for an already-produced partial result; it adds no network, process, installation, or authority surface and does not weaken the extractor sandbox.

Review checklist

  • This PR addresses a concrete problem or an agreed enhancement.
  • The PR has one focused outcome and the title follows type(scope): outcome.
  • Related issue is linked, or the reason for not linking one is stated above.
  • Tests cover changed observable behavior and meaningful failure paths.
  • Owning docs, contracts, and generated metadata are updated where needed.
  • User-visible CLI/MCP changes include representative output.
  • I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

Unblob reports a missing external extractor as an
ExtractorDependencyNotFoundReport nested inside a chunk or multi-file
report, and normalizeUnblobReport forwarded that value verbatim into
diagnostics as { input_path, report }. A caller that reached partial
coverage therefore had to decode the raw producer report to learn which
extractor was absent and at which stage, even though the report already
names the dependencies.

Extraction diagnostics now normalize those entries: the named
dependencies are collected from the report and any nested value, the
producing handler is recorded as the stage, and the original report stays
retained for provenance. Entries without a dependency report keep their
previous shape.

Validation: FirmwareReports unit cases cover the chunk-nested and
multi-file-nested dependency reports and the retained raw report;
firmware suite passes; typecheck, oxlint and oxfmt on changed files. Real
Unblob extraction is unverified here because it needs the external
extractor stack.
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 11, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ⚠️ Failed 2026-10-11T15:39:17.772620Z 150a56e PR opened
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@morluto
morluto merged commit 1769417 into morluto:main Oct 11, 2026
31 checks passed
@github-actions github-actions Bot mentioned this pull request Oct 11, 2026
3 tasks done
@morluto

morluto commented Oct 11, 2026

Copy link
Copy Markdown
Owner

Thanks for making partial firmware results easier to act on, @DeryFerd. The normalized diagnostic now names missing extractors and the handler stage inline, while retaining the raw Unblob report and keeping coverage marked partial. Callers get the cause without having to decode the producer payload.

@morluto morluto added bug Something isn't working area: native Native analysis and decompilation, including Ghidra, IDA, and Hopper analysis behavior labels Oct 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: native Native analysis and decompilation, including Ghidra, IDA, and Hopper analysis behavior bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Make partial firmware extraction recovery explicit for missing dependencies

2 participants