Repository navigation
feat(ghidra): admit relocatable MIPS32r2 ELF objects - #1950
Conversation
…at storage Soft-float o32 executables (Linux kernel builds, bare-metal firmware) were refused because their ABI flags record declares no CPR1 and FP ABI 3. Ghidra's MIPS32 compiler specifications model only hard-float o32, so typed float/double signatures (DWARF) would be bound to $f12/$f14/$f0 and the decompiler loses the real dataflow: `return sel ? a : b` decompiled to `return in_f0;`, and callers passed phantom FPU inputs. Admit ELF32 ET_EXEC o32 MIPS32r2 targets whose ABI record declares soft-float with no CPR1, and commit `mips_soft_float_storage: o32-integer-registers-v1` in their analysis profile only (hard-float profile digests are unchanged). ReaGhidraMipsSoftFloat.java runs after auto-analysis and, for each function with a float/double formal type, either keeps the types with the storage Ghidra's o32 model assigns to a same-size integer (pair alignment, stack spill), or, for auto-parameters such as hidden struct-return pointers, swaps float types for same-size integers under dynamic storage. No function is left on hard-float storage. The report is written last; the provider client requires it exactly when the profile commits the policy and refuses the session before any operation otherwise. Verification: real Ghidra 12.1.4 lane (verify:ghidra:mips) now builds hard- and soft-float fixtures in both byte orders and checks exact parameter/return storage through inspect_native_api, caller constants, snapshot profile and Evidence outcome; test:fast, verify:package and an installed-tarball soft-float run pass.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4282316342
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| for (Function function : currentProgram.getFunctionManager().getFunctions(true)) { | ||
| if (function.isThunk()) continue; |
There was a problem hiding this comment.
Remap external soft-float prototypes
For an admitted soft-float object whose DWARF/type information gives an external callee a floating parameter or return (for example, extern float helper(float)), this loop never visits that callee: the repository's no-return script and bridge iterate getExternalFunctions() separately from getFunctions(true). The external prototype, and the thunk skipped on the next line, therefore retain Ghidra's hard-float $f12/$f0 storage, so caller decompilation and inspect_native_api can report incorrect dataflow even though the handshake says the soft-float policy ran; include external callable signatures in the remap and ensure thunk signatures follow their remapped target.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Addressed by the prototype rewrite in 5965eae (#1949, included here): external functions are enumerated alongside getFunctions(true) and switched to the derived __rea_soft_float prototype, function-pointer definitions too, and thunks follow their targets.
This PR's fixture now calls powf, whose float prototype Ghidra's C library archive applies by name (cd46f16, built with -fno-builtin). With externals left out of the remap, the soft-float module variant decompiled powf(1.5f, 2.5f) as powf(param_1,param_2). Now every variant must decompile powf(1.5,2.5) without FPU inputs, and the soft-float variant must report that the prototype covered it. verify:ghidra:mips-relocatable passes for all three variants.
| external_data: false, | ||
| unplaced: 0, | ||
| }; | ||
| let pending; |
There was a problem hiding this comment.
Track every outstanding HI16 relocation
For a valid MIPS REL stream with two R_MIPS_HI16 sites before the R_MIPS_LO16 that resolves them, this single pending slot is overwritten at the second high. The later low validates only the newest site, so the first site's relocated high/carry is never compared with S + A; a bad Ghidra result can still pass the command advertised as checking every relocation. Keep all outstanding HI16 sites and resolve each matching entry before claiming complete coverage.
AGENTS.md reference: AGENTS.md:L42-L42
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Right: the single pending slot was overwritten by a second HI16 and cleared by unrelated relocations. On our 30 real soft-float kernel modules, 24 have several HI16s resolved by one LO16, so this mattered in practice.
Fixed in cd46f16. Every HI16 now waits for the next LO16 of its symbol, which checks the effective address it forms with every outstanding HI16; later LO16s of that symbol reuse those HI16s; intervening R_MIPS_26/R_MIPS_32 entries no longer reset the state; and a HI16 left unresolved at a section end fails. New unit cases cover two HI16s plus an intervening R_MIPS_26 before one LO16, corruption of the earlier HI16 (which the old code missed), and an unresolved HI16. Rerunning the real-module check: 30/30 pass, 54,972 placed relocations match, 0 failed statuses.
…lable Review feedback on morluto#1949: rewriting storage per Function left external functions and indirect calls through typed function pointers on Ghidra's hard-float o32 prototype, which still read $f12/$f14/$f0 while the session reported success. A negative control confirms it: without the change, `(*rea_mips_soft_callback)(8.5f, 9.5f, sel)` decompiles to `(*rea_mips_soft_callback)(in_f12, in_f14, sel)`. ReaGhidraMipsSoftFloat.java now derives a `__rea_soft_float` prototype from the program's own default o32 model (encoded, float pentries and float-consuming rules removed, so byte-order-specific aggregate rules and the hidden struct-return pointer are kept), installs it as a specification extension, and assigns it with dynamic storage to every non-thunk function, external function and function-pointer definition whose formal types use float or double. Thunks follow their targets. Float types are kept everywhere, so the integer-retype fallback is removed. The script fails without writing its report if the derived prototype or any reassigned function still uses an FPU register. The policy becomes `o32-soft-float-prototype-v1` with a {functions, definitions} report. The soft-float fixture adds an indirect call through a typed pointer; the real-Ghidra lane requires its constants with no FPU inputs and the struct return's `float scale` in $a1.
…rnals Review feedback on morluto#1950: - The relocation oracle kept one pending HI16, so a second HI16 before the LO16 that resolves both overwrote the first, and unrelated relocations cleared it; the earlier site's high half and carry were never compared with S + A. Every HI16 now waits for the next LO16 of its symbol, which checks all of them; later LO16s reuse those HI16s, and a HI16 left unresolved at a section end fails. On 30 real soft-float kernel modules, 24 contain several HI16s per LO16; all 54,972 placed relocations match. - External callees with float prototypes: the relocatable fixture now calls powf, whose prototype Ghidra's C library archive applies by name. Every variant must decompile `powf(1.5,2.5)` without FPU inputs, and the soft-float module variant must report that __rea_soft_float covered it (with the external step removed it decompiles as `powf(param_1,param_2)`). `-fno-builtin` keeps the call from being folded. Also updates the module variant's expected soft-float policy to `o32-soft-float-prototype-v1`.
4282316 to
cd46f16
Compare
|
@codex review |
|
Codex Review: Didn't find any major issues. Keep it up! Reviewed commit: ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. Codex can also answer questions or update the PR. Try commenting "@codex address that feedback". |
Review on morluto#1949 suggested requiring flags1 == 0 for soft-float, since MIPS_AFL_FLAGS1_ODDSPREG names CPR1 registers. GCC 5.2 kernel modules, LLVM/Rust firmware and Clang -msoft-float objects all record flags1 = 1 with FP ABI soft-float: it is the MIPS32 o32 toolchain default and selects nothing without CPR1. Keep accepting it, say why at the check, and test that soft-float with ODDSPREG is admitted while any other general flag is refused.
Linux MIPS kernel modules (.ko) and other ELF32 o32 MIPS32r2 ET_REL objects were refused by the Ghidra MIPS lane. Ghidra's ELF loader already places each allocated section and applies REL relocations; this admits ET_REL alongside ET_EXEC (ET_DYN and PRX stay refused) under its own support lane, `elf32-rel-o32-arch32r2-standard-v1`, so executable targets keep theirs. Relocatable sessions carry limitations: addresses are analysis coordinates, undefined symbols live in Ghidra's EXTERNAL block, and HI16 relocations report PARTIAL because Ghidra applies them with their paired LO16. Verification: new `verify:ghidra:mips-relocatable` real-Ghidra lane builds -c -G0 objects in both byte orders plus a soft-float -mlong-calls module variant. GNU readelf supplies sections, symbols and REL entries; every relocation in an allocated section must encode S + A in the loaded word (S from Ghidra placement, A from original bytes), with HI16/LO16 checked as the effective address each LO16 forms, and coverage of a shared HI16, a LO16 carry, external code/data and R_MIPS_32 is asserted. The executable lane, unit suites pass, and on 30 real .ko files 54,972 relocations matched (93 reference empty sections and are not checkable).
…rnals Review feedback on morluto#1950: - The relocation oracle kept one pending HI16, so a second HI16 before the LO16 that resolves both overwrote the first, and unrelated relocations cleared it; the earlier site's high half and carry were never compared with S + A. Every HI16 now waits for the next LO16 of its symbol, which checks all of them; later LO16s reuse those HI16s, and a HI16 left unresolved at a section end fails. On 30 real soft-float kernel modules, 24 contain several HI16s per LO16; all 54,972 placed relocations match. - External callees with float prototypes: the relocatable fixture now calls powf, whose prototype Ghidra's C library archive applies by name. Every variant must decompile `powf(1.5,2.5)` without FPU inputs, and the soft-float module variant must report that __rea_soft_float covered it (with the external step removed it decompiles as `powf(param_1,param_2)`). `-fno-builtin` keeps the call from being folded. Also updates the module variant's expected soft-float policy to `o32-soft-float-prototype-v1`.
cd46f16 to
1b2b98e
Compare
Resolve the relocatable MIPS conflicts with the merged soft-float lane. Share fixture compilation and ELF attestation, retain both session report failure cases in a table, and declare the relocatable readelf parser argument. Validation: check:ci, docs:check, five focused files (97 passed, one skip), and seven real Clang/LLD MIPS fixture builds. Real Ghidra was not rerun on this host.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b200b6d81b
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| byteOrder, | ||
| }); | ||
| assert.equal(coverage.unplaced, 0, "Fixture references an empty section"); | ||
| assert.ok(coverage.shared_hi16, "No HI16 shared by several LO16s"); |
There was a problem hiding this comment.
Require the real lane to cover multiple pending HI16s
When the real Ghidra path mishandles two HI16 records that are resolved by one LO16, this command can still report passed: this block requires only coverage.shared_hi16 (one high reused by multiple lows) and never requires the new coverage.multiple_hi16 flag. The synthetic oracle test cannot validate Ghidra's loader, so make the real fixture emit that stream and assert the flag here; the fresh evidence beyond the resolved single-slot review is that the fixed oracle now exposes multiple_hi16, but the final verifier never consumes it.
AGENTS.md reference: AGENTS.md:L46-L46
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Agreed. Clang's C output never produces a LO16 that resolves several pending HI16s, so the real lane could not catch Ghidra mishandling it, and 24 of the 30 GCC-built kernel modules I checked contain exactly that. Fixed in #1974: an assembly block in the fixture creates that stream (including the carry), and the lane now requires coverage.multiple_hi16. It was false on the old fixture and is true for LE, BE and the soft-float module variant on real Ghidra 12.1.4.
…locatable lane (#1974) * test(ghidra): require pending multi-HI16 coverage in the real MIPS relocatable lane Post-merge review on #1950: the relocation oracle exposes coverage.multiple_hi16, but the real-Ghidra lane never required it, and Clang's C output never produces a LO16 that resolves several pending HI16s. Ghidra's loader could therefore mishandle that stream while the lane still passed, although most GCC-built kernel modules checked contain it. The relocatable fixture adds an assembly block with two lui %hi(rea_mips_rel_carry); LLVM's relocation sort then leaves one HI16 pending until a later LO16 that resolves it together with another HI16, including the carry. The lane now requires multiple_hi16: false on the previous fixture, true for all three variants (LE, BE, soft-float module) on real Ghidra 12.1.4. * test(ghidra): make the MIPS multi-HI16 fixture independent of relocation sorting Review on #1974: the block used two %lo of rea_mips_rel_carry, a symbol with other %lo uses, so the relocation stream depended on the assembler's HI16/LO16 sort. Clang 21 happened to leave a HI16 pending; Clang 17 (per review) and GNU as 2.45 (checked here: HI@4 LO@8 HI@0 LO@10) pair each HI16 with its own LO16, so multiple_hi16 stays false and the lane fails. Use a dedicated symbol with two %hi and exactly one %lo. Any assembler that orders each HI16 before a matching LO16 must then emit both HI16s ahead of that single LO16: Clang 21 gives HI@0 HI@4 LO@8, GNU as 2.45 gives HI@4 HI@0 LO@8. The real-Ghidra lane passes for LE, BE and the soft-float module variant with multiple_hi16 true.
Stacked on #1949 (soft-float), because kernel modules are soft-float. Until #1949 merges this PR also shows its commits; review only this PR's own two,
feat(ghidra): admit relocatable MIPS32r2 ELF objects(4019966) andfix(ghidra): verify every MIPS HI16 and archive-typed soft-float externals(1b2b98e).Closes #1938.
Behavior
ET_RELalongsideET_EXEC.ET_DYNand PRX stay refused.mips_support_lane: elf32-rel-o32-arch32r2-standard-v1; executables keepelf32-exec-o32-arch32r2-standard-v3.EXTERNALblock, andinspect_native_load_imagerelocation statuses (HI16 →PARTIAL).__rea_soft_floatprototype, including archive-typed externals such aspowf.Verification
Host: Linux x64, Ghidra 12.1.4, OpenJDK 21.0.12, Clang 21, GNU readelf.
npm run verify:ghidra:mips-relocatable(real Ghidra): LE and BE-c -G0 -fno-builtinobjects, plus an LE soft-float-mlong-callsvariant matching Linux module flags. GNU readelf supplies sections, symbols and REL entries; for every relocation in an allocated section the lane reads the loaded word viaread_bytesand requiresS + A(S from Ghidra'slist_segments/EXTERNALplacement, A from original bytes). Each HI16 waits for the next LO16 of its symbol, which checks the effective address it forms with every HI16 it resolves; later LO16s reuse those HI16s; a HI16 left unresolved fails. Coverage is asserted: shared HI16, LO16 ≥ 0x8000 carry, external code and data,R_MIPS_32,R_MIPS_26only without long calls. The fixture callspowf, typed by Ghidra's C library archive: every variant must decompilepowf(1.5,2.5)without FPU inputs, and the soft-float variant must report that the soft-float prototype covered it (with externals left out it decompiles aspowf(param_1,param_2)). Plus direct calls, snapshot lane/type/policy, Evidence limitation, CLI evidence and unchanged source bytes. All 3 fixtures passed, process lineage verified.npm run verify:ghidra:mips(executables, 4 fixtures) passes on this branch.npm run test:fast: 393 files, 4620 passed, 3 skipped.check:fast,oxfmt --check .and test discovery clean.R_MIPS_26, an unresolved HI16)..kofiles on the final commit: 30/30 passed, 54,972 relocations matched, 93 references to empty sections counted as unplaced, 0 failed Ghidra statuses. 24 of the 30 modules contain several HI16s resolved by one LO16, which the first revision did not fully check. The largest module's load image and name inventory exceed the MCP stdio budget and were read from the session's exported evidence bundle.Not run: Windows lanes (MIPS is not admitted on Windows P0);
--psp.