Repository navigation
feat(web): report value-free header names in the api-surface view - #9504
2SinceJuly wants to merge 10 commits into
Conversation
The api-surface view could only say that a group had query names. Analysts asking which endpoints are signed or session-bearing had to page complete capture records and compare header values by eye, which copies the values into the agent's context. Add an opt-in include_request_header_names facet that lists the request header, cookie and set-cookie names recorded for each group, with the records that presented them. Values are never read: the reader walks only name members, tolerates malformed members without dropping the rest, and reserves its scratch before collecting anything. Omitting the option leaves group output byte-identical, and the existing sensitive-parent refusal still applies.
Codex Review Summary: ✅ CompletedThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
[Medium impact] The PR appears safe to merge; both previous findings are fixed and no new blocking issue was found. SummaryAdds an opt-in header-name inventory to the
Diagram%%{init: {'theme': 'neutral'}}%%
flowchart TD
A[Retained HAR Evidence] --> B[Admit parent and check sensitivity declarations]
B --> C[Group request lines]
C --> D{Header names selected?}
D -->|No| E[Existing group output]
D -->|Yes| F[Reserve space and read name members]
F --> G[Merge names with source ordinals]
G --> H[Sort names and return selected groups]
Reviews (4) · Last reviewed commit: "Merge remote-tracking branch 'origin/mai..." · Reviewed by Greptile |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7b849fdb45
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| readonly cookie: ReadonlySet<string>; | ||
| readonly setCookie: ReadonlySet<string>; | ||
| } | ||
| const recordedName = z.string().min(1); |
There was a problem hiding this comment.
Preserve empty recorded header and cookie names
When a schema-valid HAR header or cookie has name: "", this validator discards it: the pinned HAR schema requires a string but not a nonempty one, and the new output schema likewise accepts any string. The selected facet therefore makes a recorded empty spelling indistinguishable from an omitted member despite documenting verbatim names; accept every string here and handle malformed nonstrings separately.
AGENTS.md reference: AGENTS.md:L19-L19
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in 7ce84a8, same change as greptile's empty-name finding: every string name is accepted verbatim, including "", so a recorded empty spelling is no longer indistinguishable from an omitted member.
| const parsed = recordedName.safeParse(member); | ||
| if (parsed.success) selected.add(parsed.data); |
There was a problem hiding this comment.
Surface malformed header-name members as unknown
When retained or imported capture Evidence contains a header/cookie member with a missing or non-string name (the new test exercises both), the failed parse is silently skipped and the view emits a complete-looking empty or partial name list with no unknown or limitation. Analysts can therefore read [] as observed absence even though a member was not interpretable; keep usable sibling names but propagate malformed-member coverage as an explicit unknown or limitation.
AGENTS.md reference: AGENTS.md:L37-L37
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Partly fixed in 7ce84a8. A member whose name is missing or non-string no longer silently disappears: readRecordedHeaderNames reports it as unreadableMember, and the accumulator surfaces "Some retained header, cookie or set-cookie members have no string name, so the selected name lists are incomplete." in unknowns while keeping the usable sibling names. Coverage is at the whole-view level rather than per group, because the unknown is a property of the parent capture and adding a per-group field would change the published schema; say if you want it per group and I will add the field.
| Array.isArray(value) | ||
| ? value.map((member: unknown) => | ||
| typeof member === "object" && member !== null && "name" in member | ||
| ? (member as { readonly name: unknown }).name | ||
| : undefined, | ||
| ) |
There was a problem hiding this comment.
Reserve header-array scratch before mapping it
When a selected capture contains a large headers or cookies array that would exceed the projection allowance, value.map(...) allocates a full input-sized array before the temporary(bytes) check can reject it, so a request intended to return a bounded resource error has already consumed memory proportional to the rejected input. Count and reserve from the source array before materializing the name list.
AGENTS.md reference: AGENTS.md:L97-L97
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in 7ce84a8: the arrays are charged by recordedNamesBytes before Array.map builds any name list, so an oversized headers or cookies array now returns the bounded resource error without first allocating a full input-sized array.
The branch's own change to `webNetworkRouteTemplates.ts` did not match oxfmt, which is what failed `Static checks` on this PR independently of the main-side formatting break. Merged current main and formatted the file.
Resolve the inspect_analysis_view description conflict by keeping both the header-name opt-in added here and main's note that exported symbol bindings are not module units.
# Conflicts: # src/contracts/analysisViewToolContracts.ts
readRecordedHeaderNames derived the request, cookie and set-cookie names with Array.map before the temporary-byte reservation, so a capture whose arrays exceed the projection allowance still allocated three input-sized arrays. Charge each array's names from the borrowed members first, then materialize the lists only once the reservation succeeds. Keep a recorded empty name: the pinned HAR schema accepts any string name, so dropping it made a recorded empty spelling indistinguishable from an omitted member. Report a member whose name is not a string as an explicit unknown instead.
|
Addressed the two automated findings in 7ce84a8:
Local: |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7ce84a8703
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| ? (member as { readonly name: unknown }).name | ||
| : undefined, | ||
| ) | ||
| : []; |
There was a problem hiding this comment.
Mark malformed name collections as unknown
When an admitted/imported HAR record has request.headers, request.cookies, or response.cookies omitted or set to null/another non-array, this fallback converts the malformed collection to []; because unreadableMember only scans those returned arrays, the view emits a complete-looking empty name list with no header-name unknown. The malformed-member test masks this case because another malformed header sets the flag, so track a non-array collection as unreadable instead of treating it as observed absence.
AGENTS.md reference: AGENTS.md:L37-L37
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in 523e9ef. memberNames still returns [] for a non-array collection, but readRecordedHeaderNames now tracks any present non-array request.headers / request.cookies / response.cookies as unreadable, so a record whose collections are all malformed reports the name list as incomplete instead of an observed-empty one. Omitted collections stay observed absence. Regression: treats a recorded non-array header collection as an unreadable member (headers array + non-array cookies, asserts the cookie name is absent and the unknown is present). Validation: npx vitest run src/domain/analysisView/webNetworkApiSurfaceView.test.ts 27/27, npx tsc --noEmit, npm run lint clean. Full suite not run.
…member When a recorded request.headers, request.cookies or response.cookies member is present but not an array, memberNames collapsed it to an empty list and only the per-member scan could flag the record. A record whose collections are all malformed therefore produced a complete-looking empty name list with no unknown, while only an omitted collection is observed absence. Track a non-array collection as unreadable so the name list is reported incomplete.
|
Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits. |
Upstream main now commits the admission after add() returns, so the exclude and append paths share one rollback checkpoint. Keep that structure and thread this branch's request-header names through the merged #append call.
|
Status re-checked against The one thing worth your decision: this is Part A of #3592, and Part B (the corresponding response-side header names) is not in this branch. It is a self-contained 451-line change across 6 files — 232 of those lines are the new test file — so it should merge without touching the view's existing output. Questions I cannot answer from the repo:
|
Summary
Fixes #3592 (Part A only).
The
api-surfaceview could describe a group's query parameter names but notits headers, so an analyst asking "which endpoints in this capture are signed or
session-bearing, under which header and cookie names" had to page complete
capture records and compare header values by eye — copying those values into an
agent's context. This adds an opt-in facet that reports only the recorded
header, cookie and set-cookie names.
Problem and expected behavior
api-surfacegroups retained HAR request lines and deliberately copies noheaders or bodies. The only opt-in facet was
include_query_parameter_names(#2982). Values of
Authorization,CookieandSet-Cookieare retained asnullby capture-time redaction, so even the scheme token is gone afterinspection.
Expected: selecting the new option lists the names each group presented, with
the records that presented them, and copies no name's value. Omitting the
option leaves existing group output byte-identical.
Change and scope
apiSurfaceViewSchemagainsinclude_request_header_names(default omitted),and the group schema gains the optional
request_header_namesobject withrequest_headers,cookie_namesandset_cookie_names, each an array of{name, record_ordinals}sorted by Unicode code point.readRecordedHeaderNameswalks onlynamemembers ofrequest.headers,request.cookiesandresponse.cookies. It reads novaluemember, chargesits scratch before collecting, and tolerates malformed members without
dropping the well-formed ones.
WebNetworkApiSurfaceAccumulatorretains one ordinal list per name per group,merging case-sensitively exactly as recorded (
Acceptandacceptstaydistinct).
inspect_analysis_viewcontract text are updated.Part B of #3592 (keeping a fixed-vocabulary auth scheme token from a redacted
Authorizationvalue) is not included: it changes producer redaction and theissue asks maintainers for a scope decision first.
Not included: live CDP headers, SAZ/mitmproxy surfaces beyond retained HAR,
replay, signature verification, or any inference about authentication from a
header name.
Contract and boundary impact
src/domain/analysisView/owns theHAR grouping projection; the reader lands beside the existing request-line
reader in
webNetworkRouteTemplates.ts. No producer, capture or redactioncode changed.
within an existing view and evidence boundary; not applicable.
inspect_analysis_view'sapi-surfaceview request and result. Absent whennot selected.
is started and only retained HAR Evidence is read.
retained records; a header present in a redacted credential surface is
reported as a name only.
Expected effects:
Evidence and regression coverage
src/domain/analysisView/webNetworkApiSurfaceView.test.ts— selected/omitted/explicit-
falseequivalence, per-record ordinals and case distinction acrosstwo groups, no value substring in the serialized result, and a malformed-member
case (
name: null, missing name, non-object member,cookies: null).not exist on
origin/main; there is no bug to reproduce before the change.domain test above.
inspected; fixtures are synthetic HAR entries in the repository's existing
analysis-view fixture.
For evidence-bearing changes:
Validation performed
npx tsc --noEmit -p tsconfig.json— passed.npm run check:fast(typecheck + lint) — 2 successful.npx vitest run --project domain src/domain/analysisView/webNetworkApiSurfaceView.test.ts— 25 passed.npx vitest run --project domain src/domain/analysisView/WebNetworkApiSurfaceAccumulator.test.ts— 14 passed.npx vitest run --project services src/application/analysisView/AnalysisViewService.test.ts— 69 passed.npm run build:cached— succeeded;dist/present for the boundary suites.npx vitest run --project cli-boundary tests/boundary/cli/analysisViewCli.test.ts— 31 passed.npx vitest run --project mcp-boundary tests/boundary/mcp/analysisViewMcp.test.ts— 35 passed.npm run docs:check— 9 passed.npm run check:pr, real-browser HAR capture, and any Windows orprovider-backed workflow (none applies to retained-HAR projection).
Compatibility, safety, and release
set-cookie value, prefix, hash or substring is read or emitted, including for
headers whose value capture-time redaction nulled. The existing
sensitive_values_unretainedrefusal for parents inspected with sensitivitydeclarations still applies unchanged.
Author readiness
Human owner: @2SinceJuly
Before merge
type(scope): outcome.