Repository navigation
fix: order remaining outputs independently of host locale - #1867
Conversation
String.prototype.localeCompare without a pinned collation follows the host's ICU default locale, so the same input could be ordered differently on different machines. 614fb76 moved canonical domain projections to compareUnicodeCodePoints; provider, application, catalog, and setup orderings still used locale collation. Order every remaining source site by Unicode code point, including Mach-O symbols, firmware extraction paths, ASAR entries (previously pinned to "en" collation), evidence ID sets, browser and Electron target, script, and resource inventories, reconstruction obligations, capability and provider listings, and setup environment details. The current tool and prompt catalog names order identically under both comparators, so the catalog identity digest is unchanged.
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e16dfc71d9
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| paths = listPackage(snapshotPath, { isPack: false }).sort((left, right) => | ||
| left.localeCompare(right, "en"), | ||
| compareUnicodeCodePoints(left, right), |
There was a problem hiding this comment.
Sort ASAR entries by their portable path
On Windows, listPackage supplies host-spelled paths containing \, so applying code-point order to listed before toArtifactPath makes this result platform-dependent: an archive with a/b and a0 orders the nested entry first on POSIX (/ < 0) but the flat file first on Windows (0 < \), even though both are later exposed with /. Retain the raw spelling for statFile/adapterKey, but normalize a separate sort key before sorting.
AGENTS.md reference: AGENTS.md:L19-L19
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in de51642. orderAsarListing now pairs each host-spelled listing with its portable archive path once and sorts by that path. The host spelling is kept for @electron/asar calls and adapter keys. src/artifacts/AsarArtifactReader.test.ts covers your a/b vs a0 case with both / and \ separators; both orders are now a, a/b, a0. The previous "en" collation had the same separator dependence, so this also closes a gap that predates the PR.
@electron/asar lists host-spelled paths, so code-point order over the raw spelling still differed by host: "a\b" sorts after "a0" on Windows while "a/b" sorts before it on POSIX. Pair each listing with its portable archive path once, order by that path, and keep the host spelling for @electron/asar calls and adapter keys.
|
Welcome, @dollarsmoney! Replacing the remaining locale-dependent comparisons with REA’s code-point comparator makes machine-key ordering consistent across host locales. The regressions also cover firmware output and portable archive-path ordering. Thanks for applying it across the project. |
Summary
String.prototype.localeCompare()without a pinned collation follows the host's ICU default locale, which comes fromLANG/LC_ALL. The same input can therefore be ordered differently on different machines. 614fb76 ("fix(evidence): order canonical projections independently of locale") moved the canonical domain projections tocompareUnicodeCodePoints. Forty-twolocaleComparecalls in 30 provider, application, catalog and setup files still used locale collation. This PR orders all of them by Unicode code point, so nolocaleCompareremains in non-test source.No linked issue: I found this while reading the code, as a direct follow-up to 614fb76.
Problem and expected behavior
Smallest trigger, on unmodified
main:inventoryFirmwareOutputsorts extracted firmware paths withlocaleCompare.The violated invariant: evidence and listings derived from the same target should not depend on the analyst's locale. The same kind of drift affects Mach-O symbol lists (
_main/_Zeta/Zetainterleave differently from code-point order), and any sorted set that later feeds a digest or comparison.Change and scope
Every remaining non-test
a.localeCompare(b)becomescompareUnicodeCodePoints(a, b), the existing owner insrc/domain/unicodeCodePointOrder.ts. I applied it with a parser-based rewrite and reviewed the result by hand:NativeMachoInspection), firmware extraction paths (FirmwareFiles), ASAR entries (AsarArtifactReader, previously pinned to"en"collation, which still varies by ICU version and is not code-point order), and evidence ID sets (registerInvestigationTools/helpers).catalogIdentity.ts), client registration and setup environment details, process ownership diagnostics, and the Ghidra transport's exact-key check.Not included: test files and
PeResources.fixture.ts, which are not shipped output. Nothing here was human-facing collation; every site orders machine keys such as IDs, names, paths and environment variable names.Contract and boundary impact
compareUnicodeCodePointsowner; each producer's final ordering step.dist/contracts)._, digits, non-ASCII), listed items may appear in a different order than before. Set membership is unchanged.Evidence and regression coverage
tests/boundary/firmware/firmwareReportFiles.test.tsnow inventoriesa.bin,z.bin,ä.bin,B.binandLib/x.binon the real filesystem throughinventoryFirmwareOutputand asserts code-point order.src/firmware/FirmwareFiles.tsreverted tomain, the new test fails (1 failed | 5 passed). With the change it passes. The locale table above was produced withnode -eunder eachLC_ALL.For evidence-bearing changes:
Validation performed
macOS 15 (Darwin 24.3.0, arm64), Node 26.9.0:
npm run check:fast: typecheck and lint passed; it also passed as the pre-push hook after rebasing onto1f531fc2.npm run check:changed: 83 test files, 854 passed, 4 skipped.npm run test:focused -- <all 176 files under tests/boundary/{firmware,filesystem,browser,process}>:ibtoolrequires full Xcode and this host has only the Command Line Tools. The same 7 fail on unmodifiedmainin a separate worktree.keyedArchiveNativeIntegers(30 s timeout) andprocessFilesystemCoverageAdapters(process-ownership read) failed once under the full parallel run. Both then passed twice when rerun on this branch.Compatibility, safety, and release
Review checklist
type(scope): outcome.