Repository navigation
feat(flutter): identify Dart snapshot builds and inspect AOT payloads by pure APK parsing - #1582
Conversation
|
Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits. |
f73e518 to
ecf57fd
Compare
|
TL;DR: On three real Flutter apps, the PR's snapshot hash, ELF sections and package list match my own checks. The control app gets no false Flutter claim. I found four small gaps: build-id, compiler line, empty library, and a truncated library. Tested at ResultsAll runs used Node 24.18.0, CLI and MCP. Node 22.23.1 and 26.11.1 gave byte-identical CLI output.
Version mapping is from an outside table (reFlutter's What I would fix
More findings and the other robustness cases are in the folds below. Not run
Edit: On the new head Thanks, 3QL, for a clear PR with a real-APK check script. More findings, with how I checked each
Robustness casesI made these files from PocketDhamma. All runs ended in under 2.2 seconds. None hung or crashed.
Tests, speed and memory
On The PR body says 3,632 tests. I count 3,936 in Median of 3 runs on Node 24.18.0, wall time and peak memory:
Main has neither Flutter command. It exits 1 with APK list
The first three SHA-256 values match the F-Droid index from earlier today. Coffee's chain was checked in an earlier run. Each Flutter APK holds Commands# trees: git archive <sha> | tar -x; npm ci not run (see test setup); npm run build on Node 24
env -i HOME=<fake> TMPDIR=<own> PATH=<node bin>:/usr/bin:/bin nice -n 10 /usr/bin/time -v timeout 120 \
node scripts/rea.mjs identify-flutter-build <apk> --format json
node scripts/rea.mjs inspect-dart-aot <apk> [--abi arm64-v8a|armeabi-v7a|x86_64] --format json
node scripts/rea.mjs capabilities --format json # main vs PR: only the 2 new tools differ
node scripts/rea.mjs inspect-artifact <apk> --format json
node scripts/rea.mjs inspect-android-package <apk> --format json
node scripts/verify-flutter.mjs --apk <apk> --plain-apk <coffee apk> # all checks pass on all three
# MCP: stdio, initialize, tools/list (172 tools on the PR, 170 on main), tools/call for both tools
npm run test:focused -- <7 test files> npm run test:fast npm run check:fast
# my checks
python3 -I: find f5f5dcdc, read 32 bytes at +20, header length and kind
readelf -W --dyn-syms / -l / -n strings -a -n 6 on the isolate-data sliceRaw outputPocketDhamma, flutter_detected: true
abis:
- abi: arm64-v8a
libapp: { bytes: 5309344, snapshot_hash_sources: 2, snapshot_hash: 830f4f59e7969c70b595182826435c19 }
libflutter: { bytes: 11275600, build_id: 1797803bdbdcdbd9156cf2b806201ff6b46f4a0d, dart_version: null }
coverage: completePocketDhamma,
Header kind is 3 and the magic is valid on both data sections. cone, build-ids from The real clang line is 262 characters and ends Test setup
🤖 Generated with Claude Code (Sonnet 5.5, verified by Opus 5.5) 🦊 |
ecf57fd to
5e3edff
Compare
|
TL;DR: I retested the new head What changedA rebase re-committed all six commits at 17:26 UTC, on top of main The PR touches 48 files. 47 are byte-for-byte the same as at The output of The five findings
The v-mode changeThe commit Tests
The 4 skips match the four tests that read Still not run
Edit: The four v-mode and MCP detailsI called both tools over MCP on Node 24.18.0, 8 calls in all. Valid input works. An extra key, a missing The fix was already in Notes that did not change
Broken-library casesI made these files from PocketDhamma. Each row matches
APK list
All four SHA-256 values match the ones in my first comment. I downloaded them again today. Commands# trees: git archive <sha> | tar -x; npm ci; npm run build on Node 24 (fake HOME, HUSKY=0)
node scripts/rea.mjs identify-flutter-build <apk> --format json
node scripts/rea.mjs inspect-dart-aot <apk> [--abi armeabi-v7a|x86_64] --format json
node scripts/rea.mjs inspect-artifact <apk> --format json
node scripts/rea.mjs inspect-android-package <apk> --format json # needs REA_JADX_MCP_JAR and a JDK
node scripts/verify-flutter.mjs --apk <apk> --plain-apk <coffee apk>
node scripts/rea.mjs mcp # stdio: initialize, tools/list, one tools/call per tool
npm run test:focused -- <7 test files> npm run test:fast npm run check:fast
Test setup
🤖 Generated with Claude Code (Sonnet 5.5, verified by Opus 5.5) 🦊 |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6b2f1ea564
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
Thank you for the exceptionally thorough verification — testing against three real F-Droid apps plus a control, cross-checking hashes with All five findings are fixed in
Two of your notes beyond the five:
Also re-run on your other findings' inputs: PocketDhamma's and McSweepy's hashes ( The PR body's commit count was indeed stale — it described the pre-rebase state; the branch now carries 7 commits after the merge-conflict and duplication rounds. |
Add a flutter-provider family with one operation, identify_flutter_build, that reads an APK's lib/<abi>/libapp.so and libflutter.so through the hardened ZIP reader and reports build facts as observations: - The Dart VM snapshot hash, read 20 bytes after every snapshot magic (f5f5dcdc). The vm and isolate sections are cross-checked: agreeing sections collapse to one snapshot_hash, disagreeing sections keep every candidate with snapshot_hash null and partial coverage. - Per-ABI library digests and byte sizes, plus the engine library's GNU build-id note and any labeled 'Dart VM version' or Android clang/LLVM toolchain strings, verbatim; release builds without a version string report null rather than a guess. - The APK is digested before parsing so Evidence binds the observation to the exact input, and a target with no libapp.so reports not-detected with complete coverage. The scanner is streaming with explicit pending-magic and rescan bookkeeping, so chunk boundaries cannot double-count or drop sections, and magic-free streams never accumulate. No engine, no external lookups: the hash-to-SDK mapping stays with external datasets by design. Verified against Flutter Gallery 2.9.2 (112 MB, three ABIs): every ABI reports the same agreeing hash 65817c30a78bb44c3dc3771876b6010a from two sections, per-ABI build-ids, and the clang toolchain line; a plain aapt2-built APK reports not-detected with complete coverage. Scanner goldens come from the same libraries. READMEs carry the new 'Flutter apps' row in every supported language.
Add inspect_dart_aot to the flutter family: for one ABI's libapp.so it locates the four _kDart*Snapshot* symbols through the ELF dynamic symbol table (ELF32 and ELF64 little-endian, bounds-checked reads; a payload without exported symbols is refused), reports each section's offset and size, validates data-section headers (magic, kind, length; instructions sections honestly report nulls for the fields their layout does not carry), and projects a bounded string pool from the isolate data section: distinct package: source URIs (the app's dependency inventory), dart: SDK URIs, and identifier-shaped tokens, each with counts, bounds, and partial coverage when truncated. The string pool is labeled as a byte-pattern projection, not deserialized cluster semantics: typed class and function recovery remains an SDK-specific parser, which the snapshot hash now unlocks. The ABI is caller-selected with a deterministic default (arm64-v8a, x86_64, then first available). Verified against Flutter Gallery 2.9.2: all four symbols on both ELF64 (arm64-v8a) and ELF32 (armeabi-v7a) images, header validation, hash agreement with identify_flutter_build, and a 756 package: / 201 dart: URI pool from the isolate data section; goldens in DartElf.test.ts come from the same libraries.
…tion Mirror the ownership bar the maintainers applied across the merged JEB, ADB, and Apktool PRs: - Copy the admitted APK into a provider-owned, read-only snapshot (EXCL create, chmod 0400) before digesting and parsing; every later read uses the snapshot, so a concurrently modified original cannot mix into one observation. The result reports the snapshot's byte size and digest. - Retain failed snapshot-root cleanup in a provider set and retry it on close(), throwing an aggregate error when a root is still owned; root removal is injectable so the retention is testable. - Allocate the AOT image buffer at exactly the declared library size before reading (bounded before allocation, not concat-after-stream), keep the 256 MiB ceiling, and check cancellation between digest and stream chunks through a captured abort reason. - Accept APK paths containing spaces (covered by a fixture test) and escape the ABI pattern's dash for v-mode regex compatibility. Also add tests for cleanup retention with close retry and for cancellation before any snapshot root is created.
The Italian translation landed between the merged feature PRs; its table now carries every merged row plus this one, keeping its documentation-link set identical to README.md.
Use a hex-escaped hyphen in the ABI character class, matching the form the ADB portability fix uses, so clients compiling advertised schemas in unicodeSets mode accept the inspect_dart_aot input schema.
The duplicate-detection gate (jscpd, 2.5% threshold) tipped to 2.51% with the flutter files added. Reuse the existing machinery instead of repeating it: - FlutterBuildAnalysisService delegates to executeProviderOperation, the shared admission/Evidence composition, instead of restating its body. - The CLI registers both commands from one table-driven spec. - Boundary tests use the shared serveCli and assertAdvertisedMcpContracts harnesses rather than hand-rolled equivalents. - The provider test consolidates the four payload-shape cases into one table-driven case; the service test uses one admission table and distinct phrasings. Duplicate lines fall from 2.51% to 2.48%, back under the gate.
CassieFox23 verified the PR against three real F-Droid Flutter apps and a control app, and found five gaps. All are fixed and re-verified against cone (info.tangential.cone), one of the tested apps: - Build-id: the scanner now reads the descriptor length from the note header instead of assuming 20 bytes, so 8-byte ids from older toolchains parse; the schema accepts 8-128 hex characters. Verified against cone's readelf-reported ids on all three ABIs. - Compiler line: the toolchain capture bound rises from 200 to 500 characters, keeping cone's 262-character clang line verbatim. - Zero-byte libapp.so: the extraction target is created before streaming, so an empty entry yields a clean unsupported_target refusal for inspect_dart_aot and elf_header_valid: false with partial coverage for identify_flutter_build, instead of a raw ENOENT with code UNKNOWN. - Truncated library: identify_flutter_build marks fewer than two snapshot sections as partial coverage with a limitation naming the count, and a missing ELF magic gets its own limitation. - Docs: the default ABI preference now matches the code exactly (arm64-v8a, x86_64, x86, armeabi-v7a, then the rest alphabetically). Also: every test that read the author's /tmp/opencode goldens now builds a self-contained synthetic ELF64 image with dynsym, four snapshot symbols, two snapshot sections, and a string pool, so the suite runs anywhere; the verify lane now requires a present hex build-id per ABI instead of accepting null.
Eight findings, all applied: - P1: SnapshotHashScanner keeps candidates in a Set (no quadratic Array.includes) with a 64-distinct-hash cap, so a hostile payload of magic+hash records cannot stall the scan or bloat the response. - P1: projectStringPool stops admitting new distinct entries once each category reaches twice its projection bound and reports collectionCapped; allocation and sorting now track the response, not the payload. - P2: ZIP reader rejections during entry iteration or entry streaming map into the typed error algebra (cancellation vs input refusal) instead of escaping as unhandled exceptions in both operations. - P2: magic_valid now marks coverage only for the *Data symbols; a complete four-symbol image with valid data magics reports complete coverage (the instruction symbols intentionally lack the magic). - P2: the MCP server closes the Flutter provider during shutdown, so a retained snapshot root is retried and reported, not leaked. - P2: LabeledStringScanner commits matches only when a printable run is delimiter-terminated; the carry grows to 600 bytes so a 500-character toolchain line survives any chunk boundary untruncated. - P2: both Flutter tool effects declare writesFilesystem: true; the owned APK snapshot and extracted libapp.so are temporary writes. - P2: the verify lane asserts every ABI agrees on exactly one snapshot hash instead of discarding the computed set.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c29163212f
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
c291632 to
93db928
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 93db9281bc
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Nine findings on the first-round fixes: - P1: identify_flutter_build enforces a cumulative extraction budget (768 MiB) across every library entry, not just per library, so an APK with many ABIs cannot stream an unbounded total. - P1: a single NUL-free printable run stops materializing at 1024 bytes and marks the pool capped, so an adversarial one-run section cannot allocate unbounded strings. - P1: ELF cString bounds the NUL-terminator scan to the name maximum plus one, so malformed symbol tables cannot scan the whole image per entry. - P2: mapReaderFailure distinguishes ArtifactReaderFailure texts — budget/limit rejections map to resource constraints, integrity failures (CRC/overlap/signature) to their own input refusal — instead of collapsing all reader failures into one message. - P2: the labeled-string carry is cleared when a delimiter commits a run, so consecutive toolchain lines are never concatenated. - P2: string-pool counts are renamed *_count_at_least and documented as lower bounds, because capped collection discards values beyond twice the projection bound. - P2: inspect_dart_aot marks hash disagreements, non-pair source counts, and candidate-cap saturation as partial coverage. - P2: only valid lowercase-hex hash records count as snapshot sources; complete magic records with non-hex fields do not inflate agreement. - P2: the per-entry size refusal in inspect_dart_aot moves after ABI selection, so an oversized unselected ABI no longer refuses a small request.
|
All nine round-two findings are fixed in eeb56f9. The three P1s: identify now enforces a 768 MiB cumulative extraction budget across every library entry (1), a single NUL-free printable run stops materializing at 1024 bytes and marks the pool capped (2), and ELF name scans bound the NUL-terminator search to the name maximum plus one (3). The six P2s: mapReaderFailure now distinguishes budget/limit rejections (resource constraint) from CRC/overlap/signature failures (integrity refusal) instead of collapsing them (4), the labeled-string carry clears when a delimiter commits a run so consecutive lines never concatenate (5), string-pool counts are renamed *_count_at_least with documented lower-bound semantics since capped collection discards beyond twice the projection bound (6), inspect_dart_aot marks hash disagreements, non-pair source counts, and cap saturation as partial coverage (7), only valid lowercase-hex hash records count as snapshot sources (8), and the per-entry size refusal is deferred until after ABI selection so an oversized unselected ABI no longer refuses a small request (9). Full gates green on the new head: 4,126 tests, check:ci end-to-end, and the cone verify lane. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: eeb56f9d30
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Five findings: - P2: the owned snapshot revalidates the source after the copy (size, mtime, inode), so an in-place replacement or swap mid-copy refuses the observation instead of publishing mixed bytes. - P2: a failed ZipArtifactReader close retains the snapshot root for close() retry instead of silently swallowing the descriptor leak, in both operations. - P2: snapshot headers are parsed within their symbol bounds and a symbol smaller than the 16-byte header reports magic_valid false, so trailing-image bytes can no longer fake a valid data section. - P2: the verify lane counts provider.close() cleanup failures as lane failures instead of certifying a run with retained roots. - P2: LabeledStringScanner reports toolchain_capped when further distinct matching lines are observed past the 4-line projection; identify marks such results partial with a naming limitation.
|
All five round-three findings are fixed in 87ac338: the owned snapshot revalidates the source after the copy (size, mtime, inode) so mid-copy swaps refuse the observation (1), a failed ZipArtifactReader close retains the snapshot root for close() retry instead of leaking the descriptor (2), snapshot headers parse within their symbol bounds with a minimum-16-byte requirement so trailing bytes can no longer fake a valid data section (3), the verify lane counts provider cleanup failures as lane failures (4), and a toolchain_capped flag marks results that observed more distinct lines than the 4-line projection as partial with a naming limitation (5). Full gates green on the new head. |
Round three (one missed finding) plus round four (five findings): - P2: the GNU build-id note descriptor size is decoded from all four bytes of the little-endian field, rejecting nonzero high bytes instead of masking them to 16 bits. - P1: the Flutter MCP handlers route through createContractToolHandler and the session's analysisAdmission, so close_binary and snapshot operations drain in-flight Flutter calls instead of racing them. - P2: the owned-snapshot copy is a streamed, abort-checked write (no copyFile), so cancellation stops the write promptly while the source-identity revalidation (size, mtime, inode) still guards against mid-copy replacement. - P2: identify marks any snapshot_hash_sources value other than exactly two as partial, matching the inspection path's pair requirement. - P2: the verify lane refuses to run without --apk (exit 2) instead of printing a false-green pass, and the engine check requires every libapp-carrying ABI to have a present libflutter with a valid hex build-id rather than filtering missing engines away vacuously.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 24d61ec6d6
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Six findings plus the upstream duplication baseline: - P2: the flutter-provider kind reports available in the capability inventory, so binary_session no longer lists always-registered Flutter tools as provider_missing. - P2: printable runs that exceed the 1024-byte cap are discarded at their delimiter instead of emitting their truncated prefix as a fabricated observation. - P2: ELF symbols must fit inside the file-backed portion of their PT_LOAD segment (p_filesz), so addresses in gaps or BSS are refused instead of accepting unrelated bytes. - P2: the build-id carry spans the complete 80-byte maximum note (16-byte header + 64-byte descriptor) so chunk-boundary splits cannot drop the first byte. - P2: the labeled-string carry is sized for the toolchain pattern's true 605-byte maximum, not the previous 600. - P2: the AOT completeness predicate requires exactly two sources and one candidate unconditionally, so zero-source images with valid data magics report partial instead of complete. Also: the shared ELF fixture builder and snapshot-section helper moved to a dedicated FlutterTestFixtures module, eliminating cross-file test duplication; jscpd now passes at 2.50% while main sits at 2.52%.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: dbdd7fce3b
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
… coverage CassieFox23's third corrupt-header case: an ELF with valid magic but zeroed section/program header offsets passes the 4-byte magic check and the byte scanner still finds hash patterns in the data sections, so identify_flutter_build reports complete coverage for a structurally broken file. The elf_header_valid check now also verifies beyond the magic: e_type must be a known value (ET_REL/ET_EXEC/ET_DYN), at least one of e_shoff/e_phoff must be non-zero, and at least one of e_shnum/e_phnum must be non-zero. A file with valid magic but zeroed structure reports partial coverage with the existing ELF-magic limitation.
66b0cbb to
c6e7325
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: c6e7325755
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Share bounded ELF header parsing between identification and inspection. Accumulate headers across ZIP chunks and validate table entry widths and extents before claiming complete identification coverage. Bind AOT hashes to the exported VM and isolate headers, validate stored snapshot lengths including the magic bytes, and preserve partial evidence for missing, malformed, aliased, or overlapping data headers. Preserve native stream cancellations and APK resource constraints across both operations. Bound snapshot copies while streaming and retain useful input-failure diagnostics and lower-bound count wording. Validation: 72 focused tests; check:fast; check including formatting; docs:check; test discovery; jscpd; real cone APK and non-Flutter control on Linux. Verified CLI/MCP parity for valid and corrupted ELF headers and AOT inspection across three ABIs. Native Windows and macOS were not run.
|
Thanks @CassieFox23 for the careful retesting and for catching the remaining corrupt-header case. Fixed in cf18c8c: identification now accumulates the full ELF header across chunks and validates table offsets, entry widths, and extents against the actual library size. Zeroed or out-of-bounds tables report partial coverage, while preserving the observed hashes. This also addresses the six latest Codex findings: split-header handling, table bounds, native cancellation, APK size-limit classification, VM/isolate hash-source attribution, and declared snapshot lengths. Validation: 72 focused tests passed, along with static checks, formatting, documentation checks, and duplication checks. On Linux, I verified CLI/MCP parity using a real three-ABI Flutter APK and corrupted variants, plus a non-Flutter control. Native Windows and macOS remain unverified. If you have time, a retest of your offset-zeroed case against this head would be appreciated. Thanks again for the detailed reproductions—they’ve been genuinely useful. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: cf18c8c64a
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Merge upstream main at 9d8e42b and address the three latest Codex findings: enforce library limits after ABI selection, reject undefined or unsupported ELF symbol definitions, and retain ZIP read limits as resource constraints. Also address earlier review findings: preserve complete labeled strings with separate Dart/toolchain omission flags, retain readers and file handles until cleanup succeeds, and verify snapshot hashes and source counts for every inspected ABI. Add explicit static-analysis wording exceptions for Flutter, WASM, and Harmony in the effects audit, resolving the reported CI assertion without changing their authority declarations. Validation: 96 focused tests including the failing effects audit; check:fast; check including formatting and architecture guards; docs:check; test discovery; jscpd. Real Linux verification covers the three-ABI cone APK, a non-Flutter control, CLI/MCP schema parity, undefined ELF32 exports, and verifier rejection of a damaged secondary ABI. Native Windows and macOS were not run.
|
Thanks again for the detailed reviews. Pushed 61394b6, including a merge from main. The three latest findings are fixed:
I also addressed three earlier outstanding findings: complete-label parsing with separate Dart/toolchain omission flags, retained ownership after failed resource cleanup, and hash/source-count verification for every ABI. The failing CI assertion was the effects-wording audit interpreting “no execution” as an execution claim. Explicit static-analysis exceptions now cover Flutter, WASM, and Harmony without changing their declared authority. Local validation: 96 focused tests passed, including that exact assertion. Static, formatting, documentation, and duplication checks passed. Real Linux verification covered a three-ABI Flutter APK, CLI/MCP parity, undefined ELF32 exports, and rejection of a damaged secondary ABI. Native Windows/macOS verification remains outstanding. These results are local; the fresh GitHub CI run will validate the pushed head. |
Merge upstream main at 601814e. Keep the upstream WASM wording exception alongside the Flutter execution-free parsing exceptions and the shared Harmony exception. Validation: all 96 focused Flutter, CLI/MCP, and effects-audit tests pass; check:fast, format:check, and git diff --check pass.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 248ce4c737
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
|
TL;DR: I retested the head cf18c8c. All five of my findings are fixed, and 17 new broken-header files all give The head moved to 248ce4c after my run. Your newest comment says it fixes that CI test. I have not tested the new head yet, and I can if you'd like. 3QL asked for a retest of the zeroed-offsets case. It now gives The five findings
New broken headersI made 17 more broken copies of the PocketDhamma library, plus one unmodified control. The cases include table offsets past the file end and entry width 0. Others set a count to 0 or 65,535, or break the byte order, version, header size or type. Two are cuts to 64 and 40 bytes. At 8d9fd5b, 15 of the 17 said A header split across small chunks: I could not build an APK that does this. The ZIP reader hands over 65,536-byte chunks, stored or deflated. So I re-sliced the first chunk of each library in code, as the PR's own test does. At the 9 sizes I tried, from 1 to 65 bytes, the unmodified library stays No regressions found
Mergeable, and one red checkThe PR diff applies to main dfe3117 (#1917) with The 3 failures come from one test, Still not runEdit: I tested the struck items at 248ce4c. The results are in my new comment. Still not run: a device, Windows and macOS, 4 or more ABIs, JIT builds, a real app with a library over 256 MiB, and
Thanks, @3QL, for the quick fix and for the clear note on what changed. Other changes I saw
All 18 new header cases, one row eachI changed one field at a time in PocketDhamma's
My earlier broken files give the same answers at cf18c8c as in my last comment, except these. Header offsets zeroed: now Tests and checks
CLI output is byte-identical on all three Node versions. That covers 81 files: 15 from the four apps and 66 from 33 test files, after I drop evidence ids, timings and my temp paths. I ran the
At 8d9fd5b, no test or script read APK list
All four SHA-256 values match my earlier comments. I checked the saved files again today. Commands# trees: git archive <sha> | tar -x; npm ci; npm run build; npm run test:prepare (Node 24, fake HOME, HUSKY=0)
env -i HOME=<fake> TMPDIR=<own> PATH=/usr/bin:/bin node scripts/rea.mjs identify-flutter-build <apk> --format json
node scripts/rea.mjs inspect-dart-aot <apk> [--abi armeabi-v7a|x86_64] --format json
node scripts/rea.mjs inspect-artifact <apk> --format json
node scripts/verify-flutter.mjs --apk <apk> --plain-apk <coffee apk>
npm run test:focused -- <the 6 PR test files>
npm run test:focused -- src/contracts/toolEffects.test.ts # in main dfe31172c, with and without the PR diff
git apply --check <diff from merge base 30787297d to cf18c8c64> # run against main dfe31172c
readelf -n lib/<abi>/libflutter.so strings -a lib/<abi>/libflutter.so | grep 'clang version'
# split header: wrap ZipArtifactReader.prototype.open to re-slice the first chunk, then call FlutterBuildProvider.executeTest setup
🤖 Generated with Claude Code (Sonnet 5.5, verified by Opus 5.5) 🦊 |
|
TL;DR: I retested the head 248ce4c. Nothing broke, and all three of your newest fixes hold when I test them one at a time. One new thing: an APK that holds two files with the same name is now read two different ways, with no warning. Results at 248ce4c
Your three new claims hold. One small test for each is in the first fold. For the library size claim, New finding: two files with one nameI built an APK with two
At cf18c8c all four runs failed with The cause is in the code. Main changed CI, secondhand: 29 of 31 checks pass. All 4 test shards pass, so the Not run
Thanks, 3QL, for the fast fixes. Your three new claims, one test each
CI detailsSecondhand, from Other changes I saw
Gallery, MCP, cancel, speedGallery. Hash MCP. I started this PR build's own server over stdio. The command is Cancelling. I stopped runs on a 2 GiB sparse file, 1.5 seconds in. The CLI got SIGINT. The CLI exits 130 with Speed and memory. This was a shared box. The load average counts programs that are running or waiting for a CPU, averaged over a minute. This box has 24 threads. The load was 3.84 before and 4.92 after. Three runs each, Node 24, median seconds and peak memory:
A refusal that does no parsing takes 0.8 seconds and 304 MiB, so most of the small-app time is Node start-up. APK list
Commands# trees: git archive <sha> | tar -x; npm ci; npm run build; npm run test:prepare (Node 24, fake HOME, HUSKY=0)
npm run test:focused -- src/contracts/toolEffects.test.ts
npm run test:fast
npm run test:focused -- <the 7 changed test files> # Node 22, 24 and 26
env -i HOME=<fake> TMPDIR=<own> PATH=/usr/bin:/bin node scripts/rea.mjs identify-flutter-build <apk> --format json
node scripts/rea.mjs inspect-dart-aot <apk> [--abi armeabi-v7a|x86_64] --format json
node scripts/verify-flutter.mjs --apk <apk> --plain-apk <coffee apk>
# fixtures, all built with Python's zipfile from the four apps:
# 17 header edits + control; library 257 MiB of zeros in x86_64; st_shndx set to 0, 0xfff1 or 0xfff2 on the four snapshot symbols (ELF64 and ELF32)
# 120,000-entry APK (15.7 MB central directory); label strings appended to libflutter.so; two entries named libapp.so
truncate -s 2147483648 exact.apk; truncate -s 2147483649 over.apk
# cancel: start the CLI in the background, kill -INT <pid> after 1.5 s; MCP: JSON-RPC over stdio, then notifications/cancelled
readelf -n lib/<abi>/libflutter.so strings -a lib/<abi>/libflutter.so | grep 'clang version'
git apply --check <diff from merge base 601814ed0 (#1927) to 248ce4c73> # run against main 72489a944 (#1915)Test setup
🤖 Generated with Claude Code (Sonnet 5.5, verified by Opus 5.5) 🦊 |
dandgabr
left a comment
There was a problem hiding this comment.
Consolidated review — current head 248ce4c737ca8bf6f9b34eaef1157dfbbca06e33
The following unresolved findings are already covered by open inline threads; this comment consolidates them without adding duplicate inline threads.
P2
- Reject duplicate Flutter ZIP members.
src/flutter/FlutterBuildProvider.tsselects duplicate relevant paths inconsistently: identification keeps the last entry while AOT inspection can select the first. Reject duplicate relevant paths before processing so both operations use the same bytes. - Bound stripped ELF symbol walks. In
src/flutter/DartElf.ts, thePT_DYNAMICfallback can walk dynsym to EOF whenDT_STRTABprecedesDT_SYMTAB, interpreting later bytes as symbols. Derive a trustworthy symbol bound or reject this layout. - Cap relevant ZIP entries and ABIs.
src/flutter/FlutterBuildProvider.tshas no count limit for relevant library entries. A bounded central directory can still contain tens of thousands of empty ABI libraries, expanding processing and response size. Apply an explicit entry/ABI budget. - Preserve access-denied errors. Map
EACCESandEPERMfrom file admission/reads toaccess_denied, rather thaninvalid_request. - Preserve artifact-change errors. When source identity changes during snapshotting, return the tagged
artifact_changederror instead of a genericErrorthat becomesinvalid_request. - Validate the opened source identity. The source race check compares pathname
statvalues before and after copying; a same-size write with restored mtime can evade it. Validate identity through the opened descriptor using the shared project primitive. - Keep ELF string scans inside the table.
DartElf.tsaskscStringto scan one byte beyond the remaining string-table range, so a NUL immediately after the table can terminate a fabricated symbol name. Bound the scan at the table's exclusive end. - Report candidate truncation.
FlutterPayloadScan.tscaps snapshot-hash candidates at 64 without marking the result as truncated. Expose the cap/partial status so the returned list cannot appear complete.
P3
- Include five-character class-like tokens.
FlutterBuildProvider.tsfilters for at least six characters before projection, although the token rule accepts five. Align the filter with the rule so valid tokens are not omitted undercompletecoverage. - Use the caller's APK path in diagnostics. ZIP errors currently expose the temporary snapshot path, which is removed after the operation. Report the caller-selected path while retaining the snapshot path internally for reading.
- Validate snapshot headers in the real verifier.
scripts/verify-flutter.mjschecks data-symbol magic but notkind,header_length, or the declared section extent. Assert those fields for each data symbol and ABI. - Verify the documented toolchain strings. The real verifier checks build IDs but not the documented clang/LLVM lines or the absence of a Dart label. Add those assertions, or clarify that this workflow does not verify those outputs.
No P0 or P1 findings were confirmed. Security and binary reviewers found no additional distinct issues.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 34e764db1f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Read GNU build IDs only from bounded ELF note ranges and retain conflicting candidates. Preserve source versus owned storage failures and honor cancellation through parsing and final cleanup. Separate snapshot structure from string-pool coverage. Verify every Gallery ABI with digest-bound engine labels and URI counts, and document the supported snapshot-kind layout.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ccfaec06a1
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const segment = segments.find( | ||
| (candidate) => | ||
| vaddr >= candidate.vaddr && | ||
| vaddr - candidate.vaddr <= candidate.filesz && | ||
| size <= candidate.filesz - (vaddr - candidate.vaddr), |
There was a problem hiding this comment.
Reject conflicting PT_LOAD mappings
When a malformed ELF declares overlapping PT_LOAD ranges with different file-offset deltas for a requested symbol, this find silently picks the first mapping. Valid-looking headers at that file range can then make inspect_dart_aot return all four sections and coverage: "complete" even though the symbol address has conflicting file interpretations; detect all matching segments and fail or mark coverage partial unless they map to the same offset.
AGENTS.md reference: AGENTS.md:L46-L46
Useful? React with 👍 / 👎.
| size > totalBytes - offset | ||
| ) | ||
| return failure("ELF note range is not file-backed"); | ||
| ranges.set(`${offset}:${size}`, { offset, size }); |
There was a problem hiding this comment.
Bound the number of declared ELF note ranges
When malformed engine ELFs keep their PH/SHT bytes under 8 MiB but declare tens of thousands of distinct one-byte PT_NOTE/SHT_NOTE ranges, this map retains every range and the loop below performs a separate file.read for each. Because examined charges only bytes, the 32 allowed ABIs can drive millions of reads while all byte and ABI caps pass, stalling identify_flutter_build; impose a note-range representation cap or coalesce/reject pathological overlaps before insertion.
AGENTS.md reference: AGENTS.md:L46-L46
Useful? React with 👍 / 👎.
* fix(flutter): reject ambiguous mappings and bound ELF notes Resolve PT_LOAD interpretations once for symbols and dynamic tables, including partial overlaps. Preserve equivalent mappings and leave conflicting or invalid alternate definitions unresolved. Bound distinct nonempty ELF note ranges before retention and reads, deduplicate shared ranges, preserve observed build-ID candidates and report partial coverage on exhaustion. Correct shared verifier workspace ownership and structured Windows path assertions. Addresses the two post-merge review findings on PR #1582. Validation: 147 focused cases across Linux, macOS Sonoma 14.8.9 and non-administrator Windows 11 build 26300.9457; real Gallery CLI/MCP checks on all three hosts; six compiler-produced ELF32/ELF64 hash-layout cases; static, docs, discovery and duplication checks. Full string-pool projections remain explicitly bounded. * test(flutter): parse complete facts through named result schemas --------- Co-authored-by: me <me@medz.org> Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
feat(flutter): identify Dart snapshot builds and inspect AOT payloads by pure APK parsing
Opens the untouched Flutter lane — no PR, issue, or roadmap entry had covered Dart/Flutter analysis before. Four commits: two features, one ownership hardening matching the bar applied across the merged JEB/ADB/Apktool PRs, and one inherited-main fix.
identify_flutter_build— build identificationReads an APK's
lib/<abi>/libapp.soandlibflutter.sothrough the hardened ZIP reader and reports:f5 f5 dc dc). The vm and isolate sections are both found and cross-checked: agreeing sections collapse to onesnapshot_hash; disagreeing sections keep every value insnapshot_hash_candidateswithsnapshot_hash: nulland partial coverage rather than silently picking one.Dart VM version: …, Android clang/LLVM toolchain lines) — verbatim; release builds strip the Dart string and absence is reported asnull, not guessed.libapp.soreportsflutter_detected: falsewith complete coverage — absence is an answer.inspect_dart_aot— AOT payload inspectionFor one ABI's
libapp.so(caller-selected; default arm64-v8a → x86_64 → first available):_kDart*Snapshot*symbols located through the ELF dynamic symbol table (ELF32 and ELF64 little-endian, bounds-checked; payloads without exported symbols are refused), each with file offset and size.nullfor those fields — honestly.package:source URIs (the app's dependency inventory),dart:SDK URIs, and identifier-shaped tokens — labeled a byte-pattern projection, not deserialized cluster semantics; typed recovery needs an SDK-specific parser, which the snapshot hash unlocks.Ownership hardening (mirrors the merged PRs' bar)
chmod 0400) before digesting and parsing; every read uses the snapshot, so concurrent modification cannot mix into one observation. Failed snapshot cleanup is retained and retried onclose()(aggregate error when still owned); root removal is injectable for tests.Inherited-main fix (first commit, independently cherry-pickable)
The merged ADB family advertises five patterns with unescaped dashes in character classes (
[A-Za-z0-9._-]), which unicodeSets-mode clients reject — the advertised-patterns boundary test is red on currentmainforread_adb_settingandstart_adb_activity. Escaped the same way the JEB fix did, and restored the Italian README's analysis-table rows (JEB/ADB/Apkktool/Flutter) that the recent merges added everywhere else, keeping its link set identical toREADME.md.Verification
npm run verify:flutter— agreeing hash65817c30a78bb44c3dc3771876b6010aon every ABI, per-ABI build-ids, all four snapshot symbols on both ELF64 and ELF32 images, header validation, hash agreement between both tools, and the 756/201 URI string pool; a plain aapt2-built APK reports not-detected with complete coverage.check:fast,docs:generate,docs:check, 3,632 tests across domain/services/adapters/composition, and the product catalog all pass on this branch.Towards the Flutter gap in the #717 Android roadmap discussion. No engine, SDK, or tool installation; no new runtime dependencies; pure parsing only.