Skip to content

feat(web): report value-free header names in the api-surface view - #9504

Open
2SinceJuly wants to merge 10 commits into
morluto:mainfrom
2SinceJuly:feat/3592-header-facets
Open

2SinceJuly wants to merge 10 commits into
morluto:mainfrom
2SinceJuly:feat/3592-header-facets

Conversation

@2SinceJuly

@2SinceJuly 2SinceJuly commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

Summary

Fixes #3592 (Part A only).

The api-surface view could describe a group's query parameter names but not
its headers, so an analyst asking "which endpoints in this capture are signed or
session-bearing, under which header and cookie names" had to page complete
capture records and compare header values by eye — copying those values into an
agent's context. This adds an opt-in facet that reports only the recorded
header, cookie and set-cookie names.

Problem and expected behavior

api-surface groups retained HAR request lines and deliberately copies no
headers or bodies. The only opt-in facet was include_query_parameter_names
(#2982). Values of Authorization, Cookie and Set-Cookie are retained as
null by capture-time redaction, so even the scheme token is gone after
inspection.

Expected: selecting the new option lists the names each group presented, with
the records that presented them, and copies no name's value. Omitting the
option leaves existing group output byte-identical.

Change and scope

  • apiSurfaceViewSchema gains include_request_header_names (default omitted),
    and the group schema gains the optional request_header_names object with
    request_headers, cookie_names and set_cookie_names, each an array of
    {name, record_ordinals} sorted by Unicode code point.
  • readRecordedHeaderNames walks only name members of request.headers,
    request.cookies and response.cookies. It reads no value member, charges
    its scratch before collecting, and tolerates malformed members without
    dropping the well-formed ones.
  • WebNetworkApiSurfaceAccumulator retains one ordinal list per name per group,
    merging case-sensitively exactly as recorded (Accept and accept stay
    distinct).
  • One limitation sentence is added when the facet is selected; the guide and the
    inspect_analysis_view contract text are updated.

Part B of #3592 (keeping a fixed-vocabulary auth scheme token from a redacted
Authorization value) is not included: it changes producer redaction and the
issue asks maintainers for a scope decision first.

Not included: live CDP headers, SAZ/mitmproxy surfaces beyond retained HAR,
replay, signature verification, or any inference about authentication from a
header name.

Contract and boundary impact

  • Semantic owner and earliest changed stage: src/domain/analysisView/ owns the
    HAR grouping projection; the reader lands beside the existing request-line
    reader in webNetworkRouteTemplates.ts. No producer, capture or redaction
    code changed.
  • Contribution scope fit and linked maintainer decision: ordinary opt-in facet
    within an existing view and evidence boundary; not applicable.
  • CLI and MCP/tool-catalog contract: additive optional field on
    inspect_analysis_view's api-surface view request and result. Absent when
    not selected.
  • Provider, bridge, target-format, or platform compatibility: none; no provider
    is started and only retained HAR Evidence is read.
  • Evidence, artifact, provenance, or reconstruction contract: names come from
    retained records; a header present in a redacted credential surface is
    reported as a name only.
  • Target selection, authority, upstream behavior, and cleanup details: none.
  • Generated metadata: none.

Expected effects:

  • Read existing files or captured data.
  • Launch or attach to a process.
  • Inject instrumentation or run scripts/code.
  • Change process memory, registers, files, or configuration.
  • Contact network services.
  • Recover protected code or work around protections.
  • Documentation or usability only; no target interaction.
  • Not sure yet; investigation or maintainer guidance needed.

Evidence and regression coverage

  • Tests added or updated: two cases in
    src/domain/analysisView/webNetworkApiSurfaceView.test.ts — selected/omitted/
    explicit-false equivalence, per-record ordinals and case distinction across
    two groups, no value substring in the serialized result, and a malformed-member
    case (name: null, missing name, non-object member, cookies: null).
  • Base reproduction or other evidence: the new tests exercise the facet that does
    not exist on origin/main; there is no bug to reproduce before the change.
  • User-visible CLI/MCP output: not shown inline; the shape is asserted in the
    domain test above.
  • Remaining proof gaps: no real HAR file from a live browser capture was
    inspected; fixtures are synthetic HAR entries in the repository's existing
    analysis-view fixture.

For evidence-bearing changes:

  • Observed, derived, and inferred claims remain distinguishable.
  • Artifact identity, source provenance, and failed attempts remain preserved.
  • Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.

Validation performed

  • npx tsc --noEmit -p tsconfig.json — passed.
  • npm run check:fast (typecheck + lint) — 2 successful.
  • npx vitest run --project domain src/domain/analysisView/webNetworkApiSurfaceView.test.ts — 25 passed.
  • npx vitest run --project domain src/domain/analysisView/WebNetworkApiSurfaceAccumulator.test.ts — 14 passed.
  • npx vitest run --project services src/application/analysisView/AnalysisViewService.test.ts — 69 passed.
  • npm run build:cached — succeeded; dist/ present for the boundary suites.
  • npx vitest run --project cli-boundary tests/boundary/cli/analysisViewCli.test.ts — 31 passed.
  • npx vitest run --project mcp-boundary tests/boundary/mcp/analysisViewMcp.test.ts — 35 passed.
  • npm run docs:check — 9 passed.
  • Not run: full npm run check:pr, real-browser HAR capture, and any Windows or
    provider-backed workflow (none applies to retained-HAR projection).

Compatibility, safety, and release

  • Breaking changes or migration steps: none; the field is absent unless selected.
  • Real Hopper/Ghidra, browser, or OS coverage: none claimed.
  • Package or release metadata impact: none.
  • Security, privacy, process, or containment review: no header, cookie or
    set-cookie value, prefix, hash or substring is read or emitted, including for
    headers whose value capture-time redaction nulled. The existing
    sensitive_values_unretained refusal for parents inspected with sensitivity
    declarations still applies unchanged.

Author readiness

Human owner: @2SinceJuly

  • I reviewed the final diff and relevant existing code, understand the change and its edge cases, and take responsibility for the complete PR, including AI-assisted work.
  • I checked the evidence above, confirmed it supports the current change, and recorded meaningful validation gaps.
  • I declare this PR ready for initial maintainer review.

Before merge

  • This PR addresses a concrete problem or an agreed enhancement.
  • Material scope and authority questions are resolved, and any required maintainer scope decision covers the final diff.
  • The PR has one focused outcome and the title follows type(scope): outcome.
  • Related issue is linked, or the reason for not linking one is stated above.
  • Tests cover changed observable behavior and meaningful failure paths.
  • Owning docs, contracts, and generated metadata are updated where needed.
  • User-visible CLI/MCP changes include representative output.
  • I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

The api-surface view could only say that a group had query names. Analysts
asking which endpoints are signed or session-bearing had to page complete
capture records and compare header values by eye, which copies the values
into the agent's context.

Add an opt-in include_request_header_names facet that lists the request
header, cookie and set-cookie names recorded for each group, with the
records that presented them. Values are never read: the reader walks only
name members, tolerates malformed members without dropping the rest, and
reserves its scratch before collecting anything.

Omitting the option leaves group output byte-identical, and the existing
sensitive-parent refusal still applies.
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 8, 2026 •

Copy link
Copy Markdown
Codex Review Summary: ✅ Completed

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-12T02:43:01.378758Z b155bef New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@greptile-apps

greptile-apps Bot commented Nov 8, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerView in GreptileConfidence Score: 5/5 Tier: plus

[Medium impact] The PR appears safe to merge; both previous findings are fixed and no new blocking issue was found.

Summary

Adds an opt-in header-name inventory to the api-surface view.

  • API surface groups can show recorded header and cookie names.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Retained HAR Evidence] --> B[Admit parent and check sensitivity declarations]
  B --> C[Group request lines]
  C --> D{Header names selected?}
  D -->|No| E[Existing group output]
  D -->|Yes| F[Reserve space and read name members]
  F --> G[Merge names with source ordinals]
  G --> H[Sort names and return selected groups]
Loading

Reviews (4) · Last reviewed commit: "Merge remote-tracking branch 'origin/mai..." · Reviewed by Greptile

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7b849fdb45

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

readonly cookie: ReadonlySet<string>;
readonly setCookie: ReadonlySet<string>;
}
const recordedName = z.string().min(1);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve empty recorded header and cookie names

When a schema-valid HAR header or cookie has name: "", this validator discards it: the pinned HAR schema requires a string but not a nonempty one, and the new output schema likewise accepts any string. The selected facet therefore makes a recorded empty spelling indistinguishable from an omitted member despite documenting verbatim names; accept every string here and handle malformed nonstrings separately.

AGENTS.md reference: AGENTS.md:L19-L19

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 7ce84a8, same change as greptile's empty-name finding: every string name is accepted verbatim, including "", so a recorded empty spelling is no longer indistinguishable from an omitted member.

Comment on lines +97 to +98
const parsed = recordedName.safeParse(member);
if (parsed.success) selected.add(parsed.data);

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Surface malformed header-name members as unknown

When retained or imported capture Evidence contains a header/cookie member with a missing or non-string name (the new test exercises both), the failed parse is silently skipped and the view emits a complete-looking empty or partial name list with no unknown or limitation. Analysts can therefore read [] as observed absence even though a member was not interpretable; keep usable sibling names but propagate malformed-member coverage as an explicit unknown or limitation.

AGENTS.md reference: AGENTS.md:L37-L37

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Partly fixed in 7ce84a8. A member whose name is missing or non-string no longer silently disappears: readRecordedHeaderNames reports it as unreadableMember, and the accumulator surfaces "Some retained header, cookie or set-cookie members have no string name, so the selected name lists are incomplete." in unknowns while keeping the usable sibling names. Coverage is at the whole-view level rather than per group, because the unknown is a property of the parent capture and adding a per-group field would change the published schema; say if you want it per group and I will add the field.

Comment on lines +58 to +63
Array.isArray(value)
? value.map((member: unknown) =>
typeof member === "object" && member !== null && "name" in member
? (member as { readonly name: unknown }).name
: undefined,
)

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Reserve header-array scratch before mapping it

When a selected capture contains a large headers or cookies array that would exceed the projection allowance, value.map(...) allocates a full input-sized array before the temporary(bytes) check can reject it, so a request intended to return a bounded resource error has already consumed memory proportional to the rejected input. Count and reserve from the source array before materializing the name list.

AGENTS.md reference: AGENTS.md:L97-L97

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 7ce84a8: the arrays are charged by recordedNamesBytes before Array.map builds any name list, so an oversized headers or cookies array now returns the bounded resource error without first allocating a full input-sized array.

The branch's own change to `webNetworkRouteTemplates.ts` did not match oxfmt, which is what failed `Static checks` on this PR independently of the main-side formatting break. Merged current main and formatted the file.
Resolve the inspect_analysis_view description conflict by keeping both the
header-name opt-in added here and main's note that exported symbol bindings
are not module units.
# Conflicts:
#	src/contracts/analysisViewToolContracts.ts
readRecordedHeaderNames derived the request, cookie and set-cookie names
with Array.map before the temporary-byte reservation, so a capture whose
arrays exceed the projection allowance still allocated three input-sized
arrays. Charge each array's names from the borrowed members first, then
materialize the lists only once the reservation succeeds.

Keep a recorded empty name: the pinned HAR schema accepts any string
name, so dropping it made a recorded empty spelling indistinguishable
from an omitted member. Report a member whose name is not a string as an
explicit unknown instead.
@2SinceJuly

Copy link
Copy Markdown
Contributor Author

Addressed the two automated findings in 7ce84a8:

  • readRecordedHeaderNames now reserves each recorded array's names while the members are borrowed, before any name list is materialized.
  • A recorded empty name is kept verbatim; a member whose name is not a string now surfaces as an explicit unknown instead of being dropped silently.

Local: npx tsc --noEmit clean, prettier --check clean, npx vitest run src/domain/analysisView/webNetworkApiSurfaceView.test.ts 26/26 passing. The full suite was not run.

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 7ce84a8703

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

? (member as { readonly name: unknown }).name
: undefined,
)
: [];

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Mark malformed name collections as unknown

When an admitted/imported HAR record has request.headers, request.cookies, or response.cookies omitted or set to null/another non-array, this fallback converts the malformed collection to []; because unreadableMember only scans those returned arrays, the view emits a complete-looking empty name list with no header-name unknown. The malformed-member test masks this case because another malformed header sets the flag, so track a non-array collection as unreadable instead of treating it as observed absence.

AGENTS.md reference: AGENTS.md:L37-L37

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed in 523e9ef. memberNames still returns [] for a non-array collection, but readRecordedHeaderNames now tracks any present non-array request.headers / request.cookies / response.cookies as unreadable, so a record whose collections are all malformed reports the name list as incomplete instead of an observed-empty one. Omitted collections stay observed absence. Regression: treats a recorded non-array header collection as an unreadable member (headers array + non-array cookies, asserts the cookie name is absent and the unknown is present). Validation: npx vitest run src/domain/analysisView/webNetworkApiSurfaceView.test.ts 27/27, npx tsc --noEmit, npm run lint clean. Full suite not run.

…member

When a recorded request.headers, request.cookies or response.cookies member is
present but not an array, memberNames collapsed it to an empty list and only
the per-member scan could flag the record. A record whose collections are all
malformed therefore produced a complete-looking empty name list with no
unknown, while only an omitted collection is observed absence. Track a
non-array collection as unreadable so the name list is reported incomplete.
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Repo admins can enable using credits for code reviews in their settings.

Upstream main now commits the admission after add() returns, so the exclude and
append paths share one rollback checkpoint. Keep that structure and thread this
branch's request-header names through the merged #append call.
@2SinceJuly

Copy link
Copy Markdown
Contributor Author

Status re-checked against main (230db70fb): the branch is still MERGEABLE, and all 22 required checks pass — the 8 test shards, Portable tests, Source tests, Static checks, Native package baseline, browser / real-browser and CI required. It has not needed a rebase, so nothing here is stale except the merge itself.

The one thing worth your decision: this is Part A of #3592, and Part B (the corresponding response-side header names) is not in this branch. It is a self-contained 451-line change across 6 files — 232 of those lines are the new test file — so it should merge without touching the view's existing output.

Questions I cannot answer from the repo:

  • Is include_request_header_names the shape you want for this facet, or would you rather it share an option with include_query_parameter_names so callers enable both name inventories at once?
  • Should Part B wait on this landing, or do you want it prepared as a separate PR now?

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Report value-free request header and cookie name facets in HAR API surfaces

1 participant