Skip to content

Evolve REA into an evidence-backed analysis platform - #46

Merged
morluto merged 5 commits into
mainfrom
morl/reverse_improvements
Jul 12, 2026
Merged

morluto merged 5 commits into
mainfrom
morl/reverse_improvements

Conversation

@morluto

@morluto morluto commented Jul 12, 2026

Copy link
Copy Markdown
Owner

Summary

  • make Hopper results truthful and bounded by fixing pagination, overview inputs/counts, segment permission reporting, and function dossier limitations
  • replace generic MCP outputs with exact schemas and agent-oriented descriptions across the 46-tool surface
  • add artifact SHA-256 identity, evidence envelopes, and provider-neutral analysis/session interfaces while retaining Hopper as the first provider
  • add containing-procedure resolution, bounded raw procedure references, and the literal-first trace_feature workflow
  • harden the real-Hopper verifier against embedded failures and validate structured diagnostics
  • add deterministic source-built C, Objective-C, Swift, N-API, version-pair, and large-pagination fixtures

Verification

  • npm run check — 24 files, 142 tests
  • npm run verify:package — 46 MCP tools, target-free startup
  • npm run verify:fixtures — 7 generated artifacts verified
  • real Hopper verification against:
    • C and version-pair fixtures
    • Objective-C and N-API fixtures
    • Swift fixture
    • 1,205-procedure and 1,205-string pagination fixture
  • clean Hopper shutdown verified
  • Hopper's bundled MCP server was not used
  • autoreview completed without actionable findings

Notes

Generated binaries remain ignored under build/; only fixture sources and deterministic builders are committed.

Addresses #16, #18, #20, #21, #23, #25, #28, #31, and #33.

@chatgpt-codex-connector

Copy link
Copy Markdown

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.

@morluto
morluto merged commit 9f60348 into main Jul 12, 2026
8 checks passed
@morluto
morluto deleted the morl/reverse_improvements branch July 28, 2026 17:39
darsrc pushed a commit to darsrc/rea that referenced this pull request Oct 10, 2026
Evolve REA into an evidence-backed analysis platform
kaoru0822-kitauji added a commit that referenced this pull request Oct 14, 2026
A non-ASCII special host may map under UTS #46 to an accepted host
(fullwidth digits, ideographic full stops, ignored code points), so
treat it as accepted instead of running the ASCII IPv4 check on it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
kaoru0822-kitauji added a commit that referenced this pull request Oct 14, 2026
* fix(capture): end special-scheme URL authorities at a backslash

The native mitmproxy adapter removed transport userinfo with
`^(...//)[^/?#]*@`, so the authority ran to the last "@" before "/", "?"
or "#". For `https://example.test\part@tail` it treated
`example.test\part` as userinfo, reported `https://tail` as the
Location, and withdrew the original bytes as a transport credential.
WHATWG URL parsing, used by the shared TypeScript owner
(recordedTransportUrl.ts) and by HAR captures, ends a special-scheme or
scheme-relative authority at a backslash, so that "@" is ordinary path
text.

Locate the authority with the same boundary rule before removing
userinfo. Bytes are handled as Latin-1 text so unrelated bytes stay
identical. Userinfo before the backslash is still removed and redacted.
Add source unit tests for the adapter projection to test:python.

Fixes #2671

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(capture): skip special-scheme leading slashes before userinfo

Ending the authority at a backslash exposed a credential: for
`https://\user:pass@host/path`, WHATWG parsing skips every leading "/"
or "\" after a special scheme (and in scheme-relative references), so
`user:pass` is userinfo, but the new boundary stopped at the first
backslash and found no "@". The same applied to `https:\\user:pass@host`
and `https:user:pass@host`.

Start the authority after that whole slash run for non-file special
schemes and scheme-relative references. File URLs keep their "//"
boundary because they carry no userinfo. Load the adapter in tests with
fresh mitmproxy stand-ins scoped to its import instead of leaving them
in sys.modules.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(capture): parse URL controls and mixed network authorities

* fix(capture): distinguish native paths and URL base authorities

* fix(capture): preserve file-relative path evidence after separators

* fix(comparison): recheck historical calls-only changes (#2223)

* fix(comparison): preserve incomplete function call evidence

Keep missing, unknown, partial and unresolved call relationships unknown in function comparisons and reconstruction claims. Aggregate IDA caller and callee coverage without overwriting uncertainty.

Validate with 144 focused tests, typecheck/lint, docs checks and test discovery. Real providers were not verified; the full headless IDA session suite requires unavailable Windows native controls.

* fix(comparison): recheck historical calls-only changes

---------

Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>

* fix(android): make DEX descriptor pattern portable

* fix(ci): isolate Linux Hopper process fixture inventory

* fix(capture): remove empty URL userinfo markers

* test(capture): cover empty URL userinfo markers

* test(ci): isolate Hopper process inventory fixtures

* fix(ci): declare process inventory callback

* test(ci): isolate alternate launcher process fixture

* fix(capture): keep URLs that WHATWG rejects instead of redacting userinfo

File URLs cannot carry userinfo (an "@" in a file host fails the parse),
and a special authority whose host or port WHATWG rejects fails the whole
URL. Neither has credentials, so the bridge now keeps their source text
and byte sidecar. Checked against Node's URL over 30,000 spellings: no
under-redaction, over-redaction 20,540 -> 5,865 (remaining: empty
userinfo, IPv4 range, IDNA, non-special and baseless references).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(capture): strip trailing URL whitespace and validate IP hosts and ports

The previous host check counted trailing C0 controls and spaces, which
WHATWG strips, so `https://user:pw@host ` kept its password. Trailing
C0/space is now removed from the parse view (raw offsets are unchanged).

Special hosts now also follow the WHATWG IPv4 parser (ends-in-a-number,
hex/octal parts, range) and IPv6 literal syntax (zone IDs rejected), and
ports are bounded lexically before int(), which raised ValueError past
4300 digits on Python 3.11+ and failed the whole capture.

Node URL differential over 126,000 spellings: under-redaction 8,084 -> 0,
ValueError 2,550 -> 0; remaining special-scheme over-redaction is IDNA
and the pinned empty-userinfo case. Cost: +0.6 us per URL.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

* fix(capture): redact userinfo before hosts that need Unicode mapping

A non-ASCII special host may map under UTS #46 to an accepted host
(fullwidth digits, ideographic full stops, ignored code points), so
treat it as accepted instead of running the ASCII IPv4 check on it.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
Co-authored-by: Lucien <164857476+dakjdakd@users.noreply.github.com>
@morluto morluto added enhancement New feature or request area: evidence Evidence, artifacts, and reconstruction labels Oct 14, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: evidence Evidence, artifacts, and reconstruction enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant