Skip to content

feat(ghidra): admit relocatable MIPS32r2 ELF objects - #1950

Merged
morluto merged 6 commits into
morluto:mainfrom
AurelioB:feat/mips-relocatable
Oct 11, 2026
Merged

morluto merged 6 commits into
morluto:mainfrom
AurelioB:feat/mips-relocatable

Conversation

@AurelioB

@AurelioB AurelioB commented Oct 11, 2026 •

Copy link
Copy Markdown
Contributor

Stacked on #1949 (soft-float), because kernel modules are soft-float. Until #1949 merges this PR also shows its commits; review only this PR's own two, feat(ghidra): admit relocatable MIPS32r2 ELF objects (4019966) and fix(ghidra): verify every MIPS HI16 and archive-typed soft-float externals (1b2b98e).

Closes #1938.

Behavior

  • Admission: ELF32 o32 MIPS32r2 ET_REL alongside ET_EXEC. ET_DYN and PRX stay refused.
  • Profile: relocatable targets commit mips_support_lane: elf32-rel-o32-arch32r2-standard-v1; executables keep elf32-exec-o32-arch32r2-standard-v3.
  • Session limitations for relocatable targets: addresses are Ghidra analysis coordinates, undefined symbols live in the EXTERNAL block, and inspect_native_load_image relocation statuses (HI16 → PARTIAL).
  • Soft-float relocatable objects (kernel modules) get feat(ghidra): admit soft-float MIPS32r2 ELF with integer-register float storage #1949's __rea_soft_float prototype, including archive-typed externals such as powf.

Verification

Host: Linux x64, Ghidra 12.1.4, OpenJDK 21.0.12, Clang 21, GNU readelf.

  • New npm run verify:ghidra:mips-relocatable (real Ghidra): LE and BE -c -G0 -fno-builtin objects, plus an LE soft-float -mlong-calls variant matching Linux module flags. GNU readelf supplies sections, symbols and REL entries; for every relocation in an allocated section the lane reads the loaded word via read_bytes and requires S + A (S from Ghidra's list_segments/EXTERNAL placement, A from original bytes). Each HI16 waits for the next LO16 of its symbol, which checks the effective address it forms with every HI16 it resolves; later LO16s reuse those HI16s; a HI16 left unresolved fails. Coverage is asserted: shared HI16, LO16 ≥ 0x8000 carry, external code and data, R_MIPS_32, R_MIPS_26 only without long calls. The fixture calls powf, typed by Ghidra's C library archive: every variant must decompile powf(1.5,2.5) without FPU inputs, and the soft-float variant must report that the soft-float prototype covered it (with externals left out it decompiles as powf(param_1,param_2)). Plus direct calls, snapshot lane/type/policy, Evidence limitation, CLI evidence and unchanged source bytes. All 3 fixtures passed, process lineage verified.
  • npm run verify:ghidra:mips (executables, 4 fixtures) passes on this branch.
  • npm run test:fast: 393 files, 4620 passed, 3 skipped. check:fast, oxfmt --check . and test discovery clean.
  • Unit: admission/lane/limitation tests, readelf REL parser, and the relocation assertion's negative cases (missing carry, wrong jump target, rewritten opcode, unrelocated word, failed/missing Ghidra relocation, wrong file offset, empty-section symbol counted as unplaced, a bad earlier HI16 hidden behind a later HI16 and an intervening R_MIPS_26, an unresolved HI16).
  • Real modules (not committed): the same assertion over 30 soft-float X1600 .ko files on the final commit: 30/30 passed, 54,972 relocations matched, 93 references to empty sections counted as unplaced, 0 failed Ghidra statuses. 24 of the 30 modules contain several HI16s resolved by one LO16, which the first revision did not fully check. The largest module's load image and name inventory exceed the MCP stdio budget and were read from the session's exported evidence bundle.

Not run: Windows lanes (MIPS is not admitted on Windows P0); --psp.

…at storage

Soft-float o32 executables (Linux kernel builds, bare-metal firmware) were
refused because their ABI flags record declares no CPR1 and FP ABI 3.
Ghidra's MIPS32 compiler specifications model only hard-float o32, so typed
float/double signatures (DWARF) would be bound to $f12/$f14/$f0 and the
decompiler loses the real dataflow: `return sel ? a : b` decompiled to
`return in_f0;`, and callers passed phantom FPU inputs.

Admit ELF32 ET_EXEC o32 MIPS32r2 targets whose ABI record declares soft-float
with no CPR1, and commit `mips_soft_float_storage: o32-integer-registers-v1`
in their analysis profile only (hard-float profile digests are unchanged).
ReaGhidraMipsSoftFloat.java runs after auto-analysis and, for each function
with a float/double formal type, either keeps the types with the storage
Ghidra's o32 model assigns to a same-size integer (pair alignment, stack
spill), or, for auto-parameters such as hidden struct-return pointers, swaps
float types for same-size integers under dynamic storage. No function is left
on hard-float storage. The report is written last; the provider client
requires it exactly when the profile commits the policy and refuses the
session before any operation otherwise.

Verification: real Ghidra 12.1.4 lane (verify:ghidra:mips) now builds
hard- and soft-float fixtures in both byte orders and checks exact
parameter/return storage through inspect_native_api, caller constants,
snapshot profile and Evidence outcome; test:fast, verify:package and an
installed-tarball soft-float run pass.
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 11, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-11T21:08:03.856472Z b200b6d New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4282316342

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines +37 to +38
for (Function function : currentProgram.getFunctionManager().getFunctions(true)) {
if (function.isThunk()) continue;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Remap external soft-float prototypes

For an admitted soft-float object whose DWARF/type information gives an external callee a floating parameter or return (for example, extern float helper(float)), this loop never visits that callee: the repository's no-return script and bridge iterate getExternalFunctions() separately from getFunctions(true). The external prototype, and the thunk skipped on the next line, therefore retain Ghidra's hard-float $f12/$f0 storage, so caller decompilation and inspect_native_api can report incorrect dataflow even though the handshake says the soft-float policy ran; include external callable signatures in the remap and ensure thunk signatures follow their remapped target.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Addressed by the prototype rewrite in 5965eae (#1949, included here): external functions are enumerated alongside getFunctions(true) and switched to the derived __rea_soft_float prototype, function-pointer definitions too, and thunks follow their targets.

This PR's fixture now calls powf, whose float prototype Ghidra's C library archive applies by name (cd46f16, built with -fno-builtin). With externals left out of the remap, the soft-float module variant decompiled powf(1.5f, 2.5f) as powf(param_1,param_2). Now every variant must decompile powf(1.5,2.5) without FPU inputs, and the soft-float variant must report that the prototype covered it. verify:ghidra:mips-relocatable passes for all three variants.

external_data: false,
unplaced: 0,
};
let pending;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Track every outstanding HI16 relocation

For a valid MIPS REL stream with two R_MIPS_HI16 sites before the R_MIPS_LO16 that resolves them, this single pending slot is overwritten at the second high. The later low validates only the newest site, so the first site's relocated high/carry is never compared with S + A; a bad Ghidra result can still pass the command advertised as checking every relocation. Keep all outstanding HI16 sites and resolve each matching entry before claiming complete coverage.

AGENTS.md reference: AGENTS.md:L42-L42

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Right: the single pending slot was overwritten by a second HI16 and cleared by unrelated relocations. On our 30 real soft-float kernel modules, 24 have several HI16s resolved by one LO16, so this mattered in practice.

Fixed in cd46f16. Every HI16 now waits for the next LO16 of its symbol, which checks the effective address it forms with every outstanding HI16; later LO16s of that symbol reuse those HI16s; intervening R_MIPS_26/R_MIPS_32 entries no longer reset the state; and a HI16 left unresolved at a section end fails. New unit cases cover two HI16s plus an intervening R_MIPS_26 before one LO16, corruption of the earlier HI16 (which the old code missed), and an unresolved HI16. Rerunning the real-module check: 30/30 pass, 54,972 placed relocations match, 0 failed statuses.

…lable

Review feedback on morluto#1949: rewriting storage per Function left external
functions and indirect calls through typed function pointers on Ghidra's
hard-float o32 prototype, which still read $f12/$f14/$f0 while the session
reported success. A negative control confirms it: without the change,
`(*rea_mips_soft_callback)(8.5f, 9.5f, sel)` decompiles to
`(*rea_mips_soft_callback)(in_f12, in_f14, sel)`.

ReaGhidraMipsSoftFloat.java now derives a `__rea_soft_float` prototype from
the program's own default o32 model (encoded, float pentries and
float-consuming rules removed, so byte-order-specific aggregate rules and the
hidden struct-return pointer are kept), installs it as a specification
extension, and assigns it with dynamic storage to every non-thunk function,
external function and function-pointer definition whose formal types use
float or double. Thunks follow their targets. Float types are kept
everywhere, so the integer-retype fallback is removed. The script fails
without writing its report if the derived prototype or any reassigned
function still uses an FPU register.

The policy becomes `o32-soft-float-prototype-v1` with a
{functions, definitions} report. The soft-float fixture adds an indirect
call through a typed pointer; the real-Ghidra lane requires its constants
with no FPU inputs and the struct return's `float scale` in $a1.
AurelioB added a commit to AurelioB/rea that referenced this pull request Oct 11, 2026
…rnals

Review feedback on morluto#1950:

- The relocation oracle kept one pending HI16, so a second HI16 before the
  LO16 that resolves both overwrote the first, and unrelated relocations
  cleared it; the earlier site's high half and carry were never compared
  with S + A. Every HI16 now waits for the next LO16 of its symbol, which
  checks all of them; later LO16s reuse those HI16s, and a HI16 left
  unresolved at a section end fails. On 30 real soft-float kernel modules,
  24 contain several HI16s per LO16; all 54,972 placed relocations match.
- External callees with float prototypes: the relocatable fixture now calls
  powf, whose prototype Ghidra's C library archive applies by name. Every
  variant must decompile `powf(1.5,2.5)` without FPU inputs, and the
  soft-float module variant must report that __rea_soft_float covered it
  (with the external step removed it decompiles as `powf(param_1,param_2)`).
  `-fno-builtin` keeps the call from being folded.

Also updates the module variant's expected soft-float policy to
`o32-soft-float-prototype-v1`.
@AurelioB
AurelioB force-pushed the feat/mips-relocatable branch from 4282316 to cd46f16 Compare October 11, 2026 20:32
@AurelioB

Copy link
Copy Markdown
Contributor Author

@codex review

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Keep it up!

Reviewed commit: cd46f1642c

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Review on morluto#1949 suggested requiring flags1 == 0 for soft-float, since MIPS_AFL_FLAGS1_ODDSPREG names CPR1 registers. GCC 5.2 kernel modules, LLVM/Rust firmware and Clang -msoft-float objects all record flags1 = 1 with FP ABI soft-float: it is the MIPS32 o32 toolchain default and selects nothing without CPR1. Keep accepting it, say why at the check, and test that soft-float with ODDSPREG is admitted while any other general flag is refused.
Linux MIPS kernel modules (.ko) and other ELF32 o32 MIPS32r2 ET_REL objects
were refused by the Ghidra MIPS lane. Ghidra's ELF loader already places each
allocated section and applies REL relocations; this admits ET_REL alongside
ET_EXEC (ET_DYN and PRX stay refused) under its own support lane,
`elf32-rel-o32-arch32r2-standard-v1`, so executable targets keep theirs.
Relocatable sessions carry limitations: addresses are analysis coordinates,
undefined symbols live in Ghidra's EXTERNAL block, and HI16 relocations report
PARTIAL because Ghidra applies them with their paired LO16.

Verification: new `verify:ghidra:mips-relocatable` real-Ghidra lane builds
-c -G0 objects in both byte orders plus a soft-float -mlong-calls module
variant. GNU readelf supplies sections, symbols and REL entries; every
relocation in an allocated section must encode S + A in the loaded word
(S from Ghidra placement, A from original bytes), with HI16/LO16 checked as
the effective address each LO16 forms, and coverage of a shared HI16, a LO16
carry, external code/data and R_MIPS_32 is asserted. The executable lane,
unit suites pass, and on 30 real .ko files 54,972 relocations matched (93 reference empty sections and are not checkable).
…rnals

Review feedback on morluto#1950:

- The relocation oracle kept one pending HI16, so a second HI16 before the
  LO16 that resolves both overwrote the first, and unrelated relocations
  cleared it; the earlier site's high half and carry were never compared
  with S + A. Every HI16 now waits for the next LO16 of its symbol, which
  checks all of them; later LO16s reuse those HI16s, and a HI16 left
  unresolved at a section end fails. On 30 real soft-float kernel modules,
  24 contain several HI16s per LO16; all 54,972 placed relocations match.
- External callees with float prototypes: the relocatable fixture now calls
  powf, whose prototype Ghidra's C library archive applies by name. Every
  variant must decompile `powf(1.5,2.5)` without FPU inputs, and the
  soft-float module variant must report that __rea_soft_float covered it
  (with the external step removed it decompiles as `powf(param_1,param_2)`).
  `-fno-builtin` keeps the call from being folded.

Also updates the module variant's expected soft-float policy to
`o32-soft-float-prototype-v1`.
@AurelioB
AurelioB force-pushed the feat/mips-relocatable branch from cd46f16 to 1b2b98e Compare October 11, 2026 20:47
Resolve the relocatable MIPS conflicts with the merged soft-float lane. Share fixture compilation and ELF attestation, retain both session report failure cases in a table, and declare the relocatable readelf parser argument.

Validation: check:ci, docs:check, five focused files (97 passed, one skip), and seven real Clang/LLD MIPS fixture builds. Real Ghidra was not rerun on this host.
@morluto
morluto merged commit 290d301 into morluto:main Oct 11, 2026
28 of 29 checks passed
@github-actions github-actions Bot mentioned this pull request Oct 11, 2026
3 tasks done

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: b200b6d81b

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

byteOrder,
});
assert.equal(coverage.unplaced, 0, "Fixture references an empty section");
assert.ok(coverage.shared_hi16, "No HI16 shared by several LO16s");

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Require the real lane to cover multiple pending HI16s

When the real Ghidra path mishandles two HI16 records that are resolved by one LO16, this command can still report passed: this block requires only coverage.shared_hi16 (one high reused by multiple lows) and never requires the new coverage.multiple_hi16 flag. The synthetic oracle test cannot validate Ghidra's loader, so make the real fixture emit that stream and assert the flag here; the fresh evidence beyond the resolved single-slot review is that the fixed oracle now exposes multiple_hi16, but the final verifier never consumes it.

AGENTS.md reference: AGENTS.md:L46-L46

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed. Clang's C output never produces a LO16 that resolves several pending HI16s, so the real lane could not catch Ghidra mishandling it, and 24 of the 30 GCC-built kernel modules I checked contain exactly that. Fixed in #1974: an assembly block in the fixture creates that stream (including the carry), and the lane now requires coverage.multiple_hi16. It was false on the old fixture and is true for LE, BE and the soft-float module variant on real Ghidra 12.1.4.

morluto pushed a commit that referenced this pull request Oct 12, 2026
…locatable lane (#1974)

* test(ghidra): require pending multi-HI16 coverage in the real MIPS relocatable lane

Post-merge review on #1950: the relocation oracle exposes coverage.multiple_hi16, but the real-Ghidra lane never required it, and Clang's C output never produces a LO16 that resolves several pending HI16s. Ghidra's loader could therefore mishandle that stream while the lane still passed, although most GCC-built kernel modules checked contain it.

The relocatable fixture adds an assembly block with two lui %hi(rea_mips_rel_carry); LLVM's relocation sort then leaves one HI16 pending until a later LO16 that resolves it together with another HI16, including the carry. The lane now requires multiple_hi16: false on the previous fixture, true for all three variants (LE, BE, soft-float module) on real Ghidra 12.1.4.

* test(ghidra): make the MIPS multi-HI16 fixture independent of relocation sorting

Review on #1974: the block used two %lo of rea_mips_rel_carry, a symbol with other %lo uses, so the relocation stream depended on the assembler's HI16/LO16 sort. Clang 21 happened to leave a HI16 pending; Clang 17 (per review) and GNU as 2.45 (checked here: HI@4 LO@8 HI@0 LO@10) pair each HI16 with its own LO16, so multiple_hi16 stays false and the lane fails.

Use a dedicated symbol with two %hi and exactly one %lo. Any assembler that orders each HI16 before a matching LO16 must then emit both HI16s ahead of that single LO16: Clang 21 gives HI@0 HI@4 LO@8, GNU as 2.45 gives HI@4 HI@0 LO@8. The real-Ghidra lane passes for LE, BE and the soft-float module variant with multiple_hi16 true.
@morluto morluto added enhancement New feature or request area: native Native analysis and decompilation, including Ghidra, IDA, and Hopper analysis behavior labels Oct 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: native Native analysis and decompilation, including Ghidra, IDA, and Hopper analysis behavior enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Admit relocatable MIPS32r2 ELF objects (kernel modules) through Ghidra

2 participants