Skip to content

feat(ida): resolve containing procedures and report the admitted program as the document - #2374

Merged
morluto merged 40 commits into
morluto:mainfrom
nikolay-dementiev:feat/ida-containing-procedure
Oct 13, 2026
Merged

morluto merged 40 commits into
morluto:mainfrom
nikolay-dementiev:feat/ida-containing-procedure

Conversation

@nikolay-dementiev

Copy link
Copy Markdown
Contributor

Summary

With the modern headless IDA profile, trace_feature failed with capability_unavailable for resolve_containing_procedure, and inspect_address_context failed for list_documents. The profile now implements both operations, so feature tracing and address context work on IDA. A containing function comes from upstream lookup_funcs; when no function contains the address, the database's own segments decide between the two negative reasons, and a missing or inconsistent segment fact fails the read instead of guessing a reason.

Fixes #2335

Builds on #2348 (trace_feature reference coverage); its commits appear in this branch until #2348 is merged. The capped-reference qualification it adds is exercised here end to end.

Problem and expected behavior

Observed on real IDA Professional 9.1 (headless idalib, ida-pro-mcp c133c3853faa111a9b00ee615c013b720d0c4acd) with a small arm64 Mach-O: trace_feature {"query": "sum_point"} and inspect_address_context {"address": "0x100000564"} both returned capability_unavailable naming those operations; with an explicit document, address context answered but every containment facet stayed unavailable. Upstream lookup_funcs returns the canonical containing function for an interior address (0x100000564 gave _use_wrapper at 0x100000560) and fn: null, error: "Not a function" for data, which does not say whether the address is mapped.

Expected: an entry, interior or tail address returns its canonical containing procedure (the body stays unavailable, as for other IDA identities); an address inside a segment but outside any function is not_in_procedure; an unmapped address is outside_segments; if segment membership cannot be established, the read fails. list_documents returns the single admitted program, so default document resolution works without enumerating unrelated upstream databases.

Change and scope

  • src/ida/IdaProviderCapabilities.ts: IDA_OPERATIONS gains the two operations; idaProfileAdmits makes the modern headless profile the only one that declares them. The legacy and native attached profiles declare 12 operations, as before; the headless profile declares 14.
  • src/ida/IdaSessionClient.ts: refuses the operations outside the headless profile before any upstream call, records whether the server advertises survey_binary, and passes the admitted program path to the runner. The normal pre- and post-operation identity checks, cancellation draining and partial-observation retention apply unchanged.
  • src/ida/IdaOperationRunner.ts: the lookup_funcs row is parsed once into a function, an expected "missing" answer (exactly Not found or Not a function) or a producer failure; procedure selection keeps its previous semantics and messages, and containment reuses the same parsing. For an expected missing answer, survey_binary {detail_level: "minimal"} (scoped to the bound database) supplies segment intervals with exclusive ends; membership decides not_in_procedure versus outside_segments. A different query in the reply, a function together with an error, an unknown error text or neither a function nor an error is a producer failure and reads no segments. list_documents returns the admitted path, not the private workspace copy.
  • src/ida/IdaProtocolValues.ts: the survey reply is reduced to validated intervals. Any malformed coordinate, reversed or empty interval, size that disagrees with the bounds, segment count that disagrees with the survey statistics, absent or empty segment list rejects the whole reply. A transport-truncated preview is already rejected by the decoder. A server that does not advertise survey_binary can still answer positive lookups; a negative lookup fails with a precise message.
  • src/application/CapabilityInventory.ts: trace_feature requires search_strings and search_procedures, the operations the workflow actually calls (it listed list_strings and list_procedures).
  • src/application/EnhancedLiteralTracing.ts: a reference source that several matches share is resolved once.
  • docs/ida-provider.md, docs/testing.md: the coverage text describes the new operations and keeps every existing exclusion (binary_overview, whole-symbol inventory, typed reference edges, complete body ranges); the verification text lists the added real-lane checks.
  • scripts/verify-real-ida.mjs: for the headless profile, the real lane checks the admitted program as the single document, containment at the entry and a second instruction of the selected procedure, an unmapped address, a mapped non-function address when the target has strings, inspect_address_context with the default document, and trace_feature for the selected procedure's name.

Intentionally not included:

  • list_segments: the segment schema's provenance admits only Hopper and Ghidra sources, so an honest IDA reply would need a schema change.
  • read_bytes: upstream get_bytes fills unloaded bytes with zeros without an error and needs its own segment check.
  • address_to_file_offset: the pinned upstream exposes no file-region mapping.
  • Supporting Evidence for the primitive reads behind composed workflows.
  • Legacy attached and native profiles are unchanged.

Contract and boundary impact

  • Semantic owner and earliest changed stage: the IDA adapter (src/ida), at the profile's operation inventory and the operation runner.
  • CLI and MCP/tool-catalog contract: no tool, parameter, schema or error code changes. The two operations already have public contracts; capabilities and binary_session now report them as available for the headless IDA profile (14 declared operations, 13 available, instead of 12 and 11).
  • Provider, bridge, target-format, or platform compatibility: IDA modern headless profile only; checked on macOS arm64 with IDA 9.1.
  • Evidence, artifact, provenance, or reconstruction contract: new operations produce ordinary IDA Evidence with raw observations (lookup_funcs, survey_binary); existing Evidence is unchanged.
  • Process execution, authorization, cleanup, or containment impact: none; both operations are read-only and use the existing worker lifecycle. A negative containment read adds one survey_binary request (see Observed, not changed).
  • Generated metadata/package/installation impact: none to commit; the generated catalog reflects the new declarations.

Evidence and regression coverage

  • Tests added or updated: src/ida/IdaContainment.test.ts (headless session client over the recording fake): descriptors per profile; the legacy profile and the native profile refuse both operations without any upstream call; an interior address resolves to the canonical entry without reading segments; a matrix of negative addresses at segment starts, last bytes, exclusive ends, adjacent segments, gaps and above 2^53 (a one-byte segment that a rounded number cannot tell apart) in ascending and descending survey order; every kind of survey defect (missing, null or empty list, malformed coordinate, reversed bounds, disagreeing size, disagreeing count, a malformed row after valid ones, tool error, transport preview, tool not advertised) fails with the completed replies retained; each lookup anomaly fails without reading segments; both accepted missing texts; cancellation between lookup and survey; invalid address and foreign document refused before any upstream call; list_documents returns exactly the admitted path (not a source alias), runs the identity checks, fails on a changed target and honours cancellation. tests/boundary/providers/ida/containment.test.ts: trace_feature attributes a reference to its containing procedure and keeps an upstream reference cap visible; inspect_address_context with and without the explicit document gives the same result with an available procedure facet and the other facets unavailable. tests/conformance/ida/providerContracts.test.ts: the contract matrix covers the new operations for the headless profile. src/application/CapabilityInventory.test.ts: each search prerequisite of trace_feature is required independently. src/ida/IdaNativeProtocol.test.ts: the native profile does not admit the operations.
  • Base reproduction or other evidence: the new tests fail on the parent commit (RED commits bfcb5afde, 99ff05d09, 64701e8f3: 28 failed in IdaContainment.test.ts, the four boundary tests failed with capability_unavailable, two inventory tests failed). Real run on macOS 26.5 arm64 with IDA Professional 9.1: npm run verify:ida with the extended headless scenario passed (all six checks), including containment at the entry and an interior instruction, an unmapped address and trace_feature.
  • Remaining proof gaps: only macOS arm64 with IDA 9.1 was run for real; a negative containment answer on real IDA depends on the fixture's mapped data and was exercised when the target has strings; Linux, Windows, other IDA releases and architectures were not run.
  • Observed, derived, and inferred claims remain distinguishable.
  • Artifact identity, source provenance, and failed attempts remain preserved.
  • Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.

Validation performed

  • npm run check:fast, npm run check, npm run docs:check, npm run verify:test-discovery (968 files) - green
  • npm run test:fast - 409 files, 5359 passed, 7 skipped, 0 failed; npm run test:mcp - 103 files passed, 2 skipped, 448 passed, 3 skipped; npm run test:boundary -- tests/boundary/providers/ida after npm run build:cached - 18 passed
  • REA_IDA_MCP_CONFIG=$HOME/ida-mcp.json npm run verify:ida -- --target <inventory> --procedure _rea_fixture_add --report <path> - passed, mode headless, six checks; no IDA worker left running
  • Mutation checks, each reverted: dropping profile admission fails 2; an inclusive segment end fails 3; swapped reasons fail 9; dropping the bounds validation fails 2; an empty document list fails 3; treating any lookup error as "missing" fails 2; the previous inventory requirements fail 2; comparing addresses as numbers fails 1; scanning only the first segment fails 4

macOS arm64 only; Linux and Windows were not executed.

Compatibility, safety, and release

  • Breaking changes or migration steps: none. trace_feature and inspect_address_context now succeed on headless IDA instead of reporting capability_unavailable.
  • Real Hopper/Ghidra, browser, or OS coverage: real IDA 9.1 on macOS arm64; Hopper, Ghidra and browser not applicable.
  • Package or release metadata impact: none.
  • Security, privacy, process, or containment review: read-only upstream requests only; a hostile or malformed survey or lookup reply is rejected rather than trusted, and neither reply text nor private workspace paths reach the result.

Observed, not changed

  • A negative containment read issues one survey_binary request; the minimal survey still enumerates functions and hashes the input, so a trace_feature over many distinct reference sources that are not inside any function repeats it per distinct source. A source shared by several matches is resolved once. Caching the segment layout for the session would change the "live results" semantics and is left out.

Review checklist

  • The change addresses a concrete problem, not an unscoped refactor.
  • The PR has one focused outcome, and the title follows type(scope): outcome.
  • Related issue is linked.
  • Tests cover changed behavior and failure paths.
  • Owning docs, contracts, and generated metadata are updated.
  • User-visible CLI/MCP changes include representative output.
  • I checked the final diff for secrets, unrelated cleanup, and unsupported claims.

…ncated

A provider that reports a capped xrefs read makes the trace truncated with a residual unknown for that target. An explicit complete report and a provider that reports nothing leave the trace as before.
Executions gain an optional flag for a capped xrefs read, set by the IDA adapter from the upstream cap. trace_feature marks the trace truncated with a residual unknown for that target only when the provider reports a truncation; a provider that reports nothing keeps its previous result. The trace_feature workflow profile is revised so earlier results are not replayed.
…lookups and segment facts

Cover positive and negative containment, segment boundaries including addresses above 2^53, failures of the lookup and of the segment survey, cancellation, input validation and the admitted program as the only document.
…less IDA

Run trace_feature, including a capped reference read, and inspect_address_context with and without an explicit document through the IDA session client.
…ram as the document

The modern headless profile resolves an address through the function lookup and, when no function contains it, tells a mapped address from an unmapped one using the database-scoped minimal survey. A survey that is missing, truncated or inconsistent fails the read. list_documents reports the single admitted program. The attached profiles do not declare either operation.
…ached refusals and each trace_feature prerequisite
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex usage limits have been reached for code reviews. Please check with the admins of this repo to increase the limits by adding credits.
Repo admins can enable using credits for code reviews in their settings.

@matbrgz

matbrgz commented Oct 13, 2026

Copy link
Copy Markdown
Contributor

Technical Review & Findings

1. Inconsistent Cache Invalidation for survey_binary Segment Cache (addressInSegment)

In IdaOperationRunner.addressInSegment:

private async addressInSegment(address: string): Promise<boolean> {
  if (this.options.surveyAvailable === false)
    throw new AnalysisProtocolError(
      "The upstream server does not advertise survey_binary, so segment membership cannot be established.",
    );
  const { segments } = surveySegmentsSchema.parse(
    await this.call("survey_binary", { detail_level: "minimal" }),
  );
  const value = BigInt(address);
  return segments.some(({ start, end }) => start <= value && value < end);
}

addressInSegment calls survey_binary on every negative lookup during resolve_containing_procedure. In batch workflows or loops (e.g. resolving containing procedures across an instruction sequence or jump table), re-invoking survey_binary on each missed address causes significant protocol latency over MCP/stdio.
More importantly, if address is an address that lies outside the segment ranges (e.g. invalid memory or zero), survey_binary returns segments parsed with surveySegmentsSchema. If IDA database renames or changes segments, or if segments is empty, caching the segments per-session or caching the parsed segment ranges on IdaOperationRunner avoids redundant upstream survey roundtrips.

2. Address Representation: Hexadecimal Normalization in BigInt(address)

In addressInSegment:

const value = BigInt(address);
return segments.some(({ start, end }) => start <= value && value < end);

While idaAddressSchema normalizes addresses to 0x... hex strings, in surveySegmentsSchema:
If start and end in surveySegmentsSchema are parsed as BigInt or numbers, verify type alignment. In IdaProtocolValues.ts:
If start or end is parsed as a number or string:
If start and end are bigint, start <= value && value < end succeeds. However, if IDA segments are non-contiguous or wrap at 64-bit bounds (e.g. signed vs unsigned addresses on x86_64 or kernel space 0xffffffff...), ensure BigInt.asUintN(64, ...) is used consistently so negative or signed 64-bit addresses don't fail the start <= value && value < end comparison.

3. Missing document Parameter Validation in list_documents & resolve_containing_procedure

In officialToolContracts.ts:

official("resolve_containing_procedure", ..., z.object({ address, document }))

The contract accepts an optional or required document selector. In IdaOperationRunner.run:

if (operation === "resolve_containing_procedure") {
  const queried = idaAddressSchema.parse(parameters.address);
  this.locations.push({ kind: "address", address: queried });
  return this.containingProcedure(queried);
}

If the caller supplies document pointing to a different document name than this.options.admittedDocument, resolve_containing_procedure ignores the document parameter completely and executes against the current target anyway.
In Hopper and Ghidra providers, selecting an unknown or unadmitted document throws AnalysisInputError: Unknown document. Validating if (parameters.document !== undefined && parameters.document !== this.options.admittedDocument) ensures consistent cross-provider multi-document error handling.

@morluto

morluto commented Oct 13, 2026

Copy link
Copy Markdown
Owner

@greptileai review

@greptile-apps

greptile-apps Bot commented Oct 13, 2026

Copy link
Copy Markdown
Contributor

This PR is set for a Plus review. TREX isn't supported with Plus or Apex reviews yet, so Greptile reviews this PR at Base while TREX is on.

@greptile-apps

greptile-apps Bot commented Oct 13, 2026 •

Copy link
Copy Markdown
Contributor

RetriggerConfidence Score: 5/5

[Medium impact] The PR appears safe to merge; no actionable new issue remains.

Summary

Adds containing-procedure lookup and the admitted document to modern headless IDA.

  • Headless IDA now resolves containing procedures and lists its program.
  • Feature traces mark capped references as incomplete and reuse shared source lookups.
  • The process-capture test keeps its frame-order check in a clearer form.

Diagram

%%{init: {'theme': 'neutral'}}%%
flowchart TD
  A[Headless containment request] --> B[Check input and target identity]
  B --> C[lookup_funcs in bound database]
  C -->|Function found| D[Return canonical procedure identity]
  C -->|Expected missing answer| E[survey_binary in bound database]
  C -->|Invalid reply| F[Fail and retain completed observations]
  E --> G[Check every segment interval]
  G -->|Inside a segment| H[not_in_procedure]
  G -->|Outside all segments| I[outside_segments]
  G -->|Invalid or missing facts| F
  D --> J[Check target identity again]
  H --> J
  I --> J
Loading

Reviews (4) · Last reviewed commit: "Merge branch 'feat/trace-feature-xref-co..." · Reviewed by Greptile

Comment thread scripts/verify-real-ida.mjs Resolved
@nikolay-dementiev

Copy link
Copy Markdown
Contributor Author

On the three points from the review above:

1. survey_binary on every negative lookup. Yes, a negative lookup_funcs answer reads survey_binary again. Containment results are live observations of the current database, like the other IDA reads, so the segment layout is not cached across calls. A reference source that several matches share is resolved once per trace_feature call. The cost is listed under "Observed, not changed" in the description.

2. Signed 64-bit addresses. Addresses are normalized to unsigned hexadecimal strings with BigInt before they reach upstream and in every comparison, so a value with the top bit set is not treated as negative. 0xffffffffffffff00 is used by the real-IDA lane as an unmapped address and resolves to outside_segments. The unit tests also cover a one-byte segment above 2^53 and its neighbours.

3. The document argument. The new operations use the same document validation as the other IDA operations. In IdaSessionClient, any document argument must equal the admitted program path (or its source path) before upstream is contacted. resolve_containing_procedure with another document fails as invalid input, and list_documents does not accept a document argument at all. This is pinned by "accepts an explicit admitted document and rejects an unknown one for the same operations" in src/ida/IdaContainment.test.ts (e34afbf); neither rejected request reaches upstream.

Comment thread scripts/verify-real-ida.mjs Resolved
@morluto
morluto merged commit 5f8af4e into morluto:main Oct 13, 2026
38 checks passed
@morluto morluto added enhancement New feature or request area: native Native analysis and decompilation, including Ghidra, IDA, and Hopper analysis behavior labels Oct 14, 2026
@morluto morluto mentioned this pull request Oct 14, 2026
3 tasks done
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area: native Native analysis and decompilation, including Ghidra, IDA, and Hopper analysis behavior enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Headless IDA: resolve containing procedures and report the admitted program as the document

3 participants