Skip to content

test(http3): add real MITM capture verification lab - #1997

Merged
morluto merged 13 commits into
morluto:mainfrom
LuisTaffelli:test/http3-capture-verification
Oct 13, 2026
Merged

morluto merged 13 commits into
morluto:mainfrom
LuisTaffelli:test/http3-capture-verification

Conversation

@LuisTaffelli

@LuisTaffelli LuisTaffelli commented Oct 12, 2026 •

Copy link
Copy Markdown
Contributor

Problem and scope

REA can inspect native mitmproxy captures, but the existing capture verifier does not establish that traffic negotiated HTTP/3 through a MITM. A successful HTTP request may fall back to another protocol.

This PR adds an optional, reproducible Linux reverse-proxy laboratory using existing mitmproxy and aioquic implementations. It changes no production contracts or adapters. The ten changed files comprise the nine laboratory/guide files and one optional npm command, verify:web:http3; no HTTP/3 CI lane is added. Registering the command also resolves Knip's unused-verifier/export findings.

Two scenarios

  1. Real HTTP/3: three concurrent binary POSTs to the same URL through mitmproxy to an HTTP/3 origin, plus an independent cURL --http3-only smoke request. Responses are dispatched in reverse origin completion order; association is verified independently of client/capture arrival order.
  2. Fallback control: disable HTTP/3 in the proxy; cURL --http3 negotiates client HTTP/2 over TCP and reaches an HTTP/1.1 TLS origin. The same positive HTTP/3 assertion must reject this capture.

The positive case checks both legs' HTTP versions, transport, ALPN and TLS observations; endpoint-negotiated QUIC v1; native UUID preservation; request/response bytes, lengths and SHA-256; method, URL components and ordered duplicate regular headers; CLI/MCP parity; advertised schemas; and complete retained MCP Evidence. Both input captures must remain unchanged. Endpoint stream IDs remain scoped observations: no mitmproxy stream IDs or cross-leg mapping are invented.

UUID-specific reports and an atomic running/passed/failed marker prevent stale successes. Cleanup attempts all closes and audits while preserving the primary failure; success is published only after cleanup.

Reproduction

Follow the laboratory guide at the verified commit in a disposable Linux VM. It pins the image and mitmproxy archive by digest, cURL's HTTP/3 dependency profile and the Python runtime graph. Build from this checkout using its pinned Node/npm and locked dependencies; keep host directories unmounted and host trust/routes/packages unchanged. Before loopback traffic, ensure the guest has no IPv4 or IPv6 default route. The verified VM used IPv6 disabled in QEMU and removed its IPv4 default route. Clients/proxy receive explicit paths to generated test CAs.

REA_MITMDUMP_COMMAND=/absolute/lab/tools/mitmdump \
REA_HTTP3_PYTHON_COMMAND=/absolute/lab/venv/bin/python \
npm run verify:web:http3 -- /absolute/evidence/run-1

REA_MITMDUMP_COMMAND=/absolute/lab/tools/mitmdump \
npm run verify:web:http3 -- /absolute/evidence/run-1 --inspect-existing

The guide provides the Node/Python regression commands. Generated evidence and private keys stay outside Git. This PR publishes source and reviewed synthetic-result summaries, with no captures, certificates, VM disks or credentials attached.

Exact source and executed verification

Verified source: 891c7878dfd659437ba3c15799ef1d95abe3279c, based on upstream main e9fe6437a7da381f0aa5aa77571dcf03317122e7. Its locally retained git archive --format=tar.gz source has SHA-256 1942bc7d7f2180ddbddb074fea6466a21f18e7e42b983985a0fc20169a9dddd1. All 2,879 archived file digests matched the isolated worktree. Source bytes were compared inside the guest before execution and checked again afterward; the runtime was built inside the guest.

The exercised guest used Node 24.18.0 / npm 11.16.0, fixture Python 3.14.7, aioquic 1.2.0, cURL 8.22.0 (OpenSSL 3.6.4, ngtcp2 1.25.0, nghttp3 1.18.0), and standalone mitmproxy 12.2.3 (bundled Python 3.14.0 / OpenSSL 3.5.5).

On this source:

  • npm run check, including Knip, passed locally. Guest npm ci, npm run build:cached, and Python pip check passed.
  • npm run verify:web:http3: fresh real traffic passed with four HTTP/3 flows and one fallback flow, CLI/MCP byte/association/schema/session checks and empty owned proxy groups. Run UUID: 41849ea4-7b4a-4fb3-93f4-d583d99ff812.
  • Four Node and two Python verifier regressions passed.
  • Controlled oracle-digest and producer failures produced failed attempts without a new success, preserving original diagnostics, the earlier success and both capture digests. Cleanup had no failures or live owned groups. These additional injection drivers remain local, outside the PR.
  • Recovery through npm run verify:web:http3 -- ... --inspect-existing passed: UUID 80dc2ec6-2ae1-45c7-bfef-296ceedb5947. Final source hashes matched, the final laboratory process snapshot was empty, and the guest shut down cleanly.

The reviewed local evidence export has SHA-256 c40d80b123d573f35c993950da8407557f698e92fabb520ad030a392467141a5; every manifest digest was checked. It excludes PEM files, private keys and proxy configuration and is not attached. Another session reviewed reports and source digests, without independently rerunning the lab or exhaustively auditing every artifact.

Earlier real-provider results on production base 12ee785e5e61132898adecf9424ce6706a2e754a remain separately retained; they are not substituted for the new run. Earlier build/docs/discovery and 94 focused capture tests ran on PR head 20f763775fbc87b1fc752998ba930657495dbba7; CI for the new head is tracked separately in GitHub checks. The prior APPROVED review refers to that earlier head. The full local suite / check:pr was not rerun.

Coverage limits

This verifies only native captures, Linux reverse mode, QUIC v1 and the named clients/versions; it does not establish general HTTP/3 support in REA. Empty/absent/incomplete/redacted bodies, cancellations, resets, certificate failures, malformed/truncated captures, HAR, qlog, local/WireGuard modes, browsers, QUIC v2, migration, 0-RTT, mTLS, replay, sustained load and loss tolerance remain outside this increment.

Future options are being discussed separately in Discussion #2000, related to #774. They do not expand this PR or commit to future delivery.

Concurrent verification marker correction

Commit 8b92214ae567949227697007b9343a3cfcd5775a fixes the race reported in review comment. Attempt registration and current-owner check/replacement share a SQLite exclusive transaction lock. Only the most recently registered attempt may publish the current marker; earlier attempts retain their own reports and diagnostics.

Validation on the unchanged reviewed patch: seven lifecycle tests pass, including separate supervised processes finishing in reverse order, lock contention and lock release after an invalid marker. npm run check, npm run docs:check, targeted strict lint and the pre-push gate passed. The lock has no global wait deadline; a live stalled owner may delay publication until it exits. This is verifier bookkeeping only. Earlier real HTTP/3 evidence remains pinned to 891c7878dfd659437ba3c15799ef1d95abe3279c; no real-traffic rerun is claimed for the new commit. Fresh CI is pending.

CI baseline corrections (2026-10-13)

Run 38379313793 tested merge 4f726317d7f58c05ef6e91d78ca11b30e9fb73f5 against base d13c9f840d36a321a56917e75c48afcb8d72920e. Both the historical base and exact CI merge reproduce the same five failures (45 passing tests): four release preflight assertions and one same-named Mach-O slice fixture.

This branch now merges upstream bd2c202dd930548194cdbc55c27a797f22279db9, including release preflight fix 747e7299141cec0b03774c427f579889d5088d20. Commit 32e763a0765505ef567c565cff234c534e216ffe fixes the remaining fixture: a neutral x86_64 slice allows target admission on x64 runners, while explicit --architecture arm64e retains both ARM subtype roots and all original evidence assertions. Production target admission is unchanged.

Validated on the exact published tree 8d9f9def4878de0b9ccef889e5350febe5457216 with Node 24.18.0 / npm 11.16.0: 50/50 focused release/dylib tests, 7/7 HTTP/3 lifecycle regressions, 120/120 CI scope/required-lane tests, and typecheck/lint. Pre-push checks passed. The earlier native HTTP/3 evidence remains pinned to 891c7878dfd659437ba3c15799ef1d95abe3279c; no native traffic rerun is claimed for this CI integration. Fresh GitHub CI is pending.

Producer TLS fact regression (2026-10-13)

Commit 0f7dc00273bc469a4aa2bea7320856b8a07095bf addresses review comment 4257957176: positive records now preserve the UUID-matched producer's exact tls_version, timestamp_tls_setup, and connection error on both legs, alongside connection IDs. Producer version labels are compared as observations; timestamps are neither rounded nor given a tolerance.

Twelve mutations that previously passed are now rejected. All 20 verifier regressions, static checks, strict lint, formatting, documentation checks, and pre-push checks pass on the published tree e2f051f5bfbbbcac3cab18c7ec10fbfe9a8dfa44. The new assertions also pass on four retained CLI/MCP result views from the original real lab (runs 41849ea4-7b4a-4fb3-93f4-d583d99ff812 and 80dc2ec6-2ae1-45c7-bfef-296ceedb5947), with all six input-file digests unchanged. This is historical evidence from source 891c7878dfd659437ba3c15799ef1d95abe3279c; no new decoding or native traffic run is claimed for this commit. Fresh GitHub CI is pending.

Readable capture-field regression (2026-10-13)

Commit 095dbb00428766cb50c4c3e429b23b74ceed5b6c addresses review comment 4258507827. In addition to retained bytes, positive assertions now compare readable method/scheme/authority/path against observed pseudo-fields, ordered request/response header pairs including duplicate values, and h3 ALPN on both legs. Expected text preserves the native decoder's strict UTF-8 semantics, including null display values for non-UTF-8 bytes.

Fourteen readable-only mutations accepted before the fix are now rejected. All 34 verifier regressions pass; the guide runs both the lifecycle and newly separated assertion test files. Pinned Node 24.18.0 / npm 11.16.0 build, typecheck/lint, strict targeted lint, formatting, documentation and pre-push checks pass on tree 9626f7047c680e00d07234a308ca66ddaa65822e.

The new assertions also pass over the four retained CLI/MCP result views of the earlier real lab, with all six input digests unchanged. Historical native evidence still belongs to 891c7878dfd659437ba3c15799ef1d95abe3279c; no decoding or native traffic rerun is claimed for this fix. Fresh GitHub CI is pending.

@LuisTaffelli
LuisTaffelli marked this pull request as ready for review October 12, 2026 15:41
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Repo admins can enable using credits for code reviews in their settings.

@baron baron left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Exact-head review: 891c787. No confirmed source defect in the bounded development-only loopback capture lab; this is an evidence hold, not a request to broaden or redesign the supported capability.

Before merging, please make the nonsecret exact-head live report/manifest, source/capture digests, and final cleanup receipts reviewable. The description pins run 41849ea4-7b4a-4fb3-93f4-d583d99ff812 and inspect-existing run 80dc2ec6-2ae1-45c7-bfef-296ceedb5947, but the reported local export was unavailable to our review. Green CI and the three safe verifier regression checks do not independently establish those real-traffic claims. No real interception, trust changes, or client configuration was performed by this review.

@LuisTaffelli

Copy link
Copy Markdown
Contributor Author

Thanks, @baron — agreed that CI and verifier regressions do not independently establish the live-traffic claims.

The retained nonsecret reports, manifests, digests and cleanup receipts are available here: retained evidence package. This publishes existing evidence; no new traffic run or independent provider rerun is claimed.

All artifacts refer to source 891c7878dfd659437ba3c15799ef1d95abe3279c:

  • Fresh real-traffic report: 41849ea4-7b4a-4fb3-93f4-d583d99ff812, completed 2026-10-12T14:56:02.348Z.
  • Inspect-existing report: 80dc2ec6-2ae1-45c7-bfef-296ceedb5947, completed 2026-10-12T14:57:58.634Z; this re-inspects the same captures rather than generating new traffic.
  • The capture oracle manifest, source-file digest list, original export manifest and publication index are included. I rechecked all 73 original export entries and matched all 2,879 source archive files locally.
  • Cleanup receipts show no recorded cleanup failures, empty owned producer groups and no remaining verifier descendants. The final laboratory process snapshot is []. Reports from both controlled negative attempts retain their expected failed status and their cleanup evidence; they are not counted as successful traffic runs.

SHA-256:

source archive: 1942bc7d7f2180ddbddb074fea6466a21f18e7e42b983985a0fc20169a9dddd1
normal.mitm:    4cbb2ca9c3d6ef9d64700f37ec491d63e024fe6cadc98fc9337362c574e675f3
fallback.mitm:  e4b23b88676b95560191b33053755d41049fcfe63de6ce4c676750ca727d5633
local export:   c40d80b123d573f35c993950da8407557f698e92fabb520ad030a392467141a5

The original full export remains local; the linked publication is a documented subset. Captures, certificates, private keys, proxy configuration, VM disks and binaries are excluded. Full CLI/MCP replies and session bundles also remain local because they embed generated public certificates; the original export manifest lists their digests but does not make those files available.

Digests establish artifact identity, not independent execution attestation. Cleanup receipts describe the recorded processes and observation times, not a general containment guarantee. All coverage limits in the PR description remain in force: native captures, Linux reverse mode, QUIC v1 and the named clients/versions only. Please keep the evidence hold until you have reviewed these artifacts; this response does not claim your acceptance.

@morluto

morluto commented Oct 13, 2026

Copy link
Copy Markdown
Owner

@greptileai review

@greptile-apps

greptile-apps Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

[Medium impact] The reviewed changes appear safe to merge; no blocking issue was found.

Summary

Adds an optional Linux lab that checks real HTTP/3 captures through CLI and MCP, with a fallback case that must fail the positive HTTP/3 check.

  • The HTTP/3 lab checks real mitmproxy captures against a fallback run.
  • Function comparisons stay unknown when call coverage is incomplete.
  • Linux process checks can scan a supplied process tree.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Register attempt under lock] --> B{Inspect existing captures?}
  B -->|No| C[Run loopback HTTP/3 and fallback traffic]
  B -->|Yes| D[Read saved captures and observations]
  C --> D
  D --> E[Compare CLI and MCP results]
  E --> F[Check bytes, fields, protocols and retained Evidence]
  F --> G[Attempt every cleanup and input check]
  G --> H[Save this attempt's result]
  H --> I{Still the current attempt?}
  I -->|Yes| J[Replace current marker under lock]
  I -->|No| K[Keep result in attempt history]
Loading

Reviews (8) · Last reviewed commit: "fix(http3): retain MCP startup diagnosti..." · Reviewed by Greptile

@LuisTaffelli

Copy link
Copy Markdown
Contributor Author

CI follow-up: the five failures in run 38379313793 reproduce identically on the historical base and exact PR merge (45 pass / 5 fail each), before the HTTP/3 marker correction can affect them.

Updated this branch with upstream bd2c202 (including the four release-preflight fixes) and fixture-only commit 32e763a07 for the remaining x64 failure. The fixture now includes a neutral host-compatible slice and explicitly selects arm64e; both same-named ARM subtype roots and all original evidence assertions remain covered. No production target-admission changes.

The exact published tree passes 50 focused release/dylib tests, 7 HTTP/3 lifecycle regressions, 120 CI planning tests, typecheck/lint, and pre-push checks on the pinned toolchain. Fresh CI is pending; existing native HTTP/3 evidence remains tied to its original tested revision.

@morluto morluto added area: tests Test suite and verification area: runtime Process capture and runtime observation area: build Build, CI, packaging, and publishing P3: low Low priority, nice to have labels Oct 13, 2026
dakjdakd and others added 4 commits October 14, 2026 01:03
* fix(comparison): preserve incomplete function call evidence

Keep missing, unknown, partial and unresolved call relationships unknown in function comparisons and reconstruction claims. Aggregate IDA caller and callee coverage without overwriting uncertainty.

Validate with 144 focused tests, typecheck/lint, docs checks and test discovery. Real providers were not verified; the full headless IDA session suite requires unavailable Windows native controls.

* fix(comparison): recheck historical calls-only changes

---------

Co-authored-by: morluto <76467478+morluto@users.noreply.github.com>
@morluto
morluto merged commit ebaaa79 into morluto:main Oct 13, 2026
6 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: build Build, CI, packaging, and publishing area: runtime Process capture and runtime observation area: tests Test suite and verification P3: low Low priority, nice to have

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants