Repository navigation
fix(ida): wait out a transient busy worker health in headless sessions - #1778
Merged
morluto merged 8 commits intoOct 11, 2026
Merged
Conversation
…ssert clean release
…in its own module
… and cancellation
… honour a late abort
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
3 tasks done
10 tasks done
Owner
|
The transient busy health response was the source of those flaky IDA session opens; thanks for narrowing the retry to that state, @nikolay-dementiev. Waiting within the registration deadline turns a worker that stays busy into a retryable timeout, while unexpected health payloads keep their protocol-error diagnosis. |
Closed
2 tasks done
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
A freshly opened headless IDA worker can answer
server_healthwithstatus: "busy"before it reports its identity. REA parsed that first answer strictly, so a transient state surfaced as the non-retryableunreadable_output. The headless session now re-reads the health for a bounded time instead.Fixes #1766
Problem and expected behavior
Right after
idb_open,server_health(database)can return{"status":"busy","uptime_sec":1.041,"busy_tool":"config_json_get","busy_sec":0.0,"queued_calls":0}; a later call returns the full{"status":"ok", input_path, idb_path, imagebase, ...}.modernHealthSchemarequiresstatus: "ok"and every identity field, so the first answer was rejected as malformed producer data and reported asunreadable_outputwithretryable: false. On macOS arm64 with IDA 9.1 the realverify:idalane failed on this in 5 of 6 runs.Expected: a busy worker is waited out within the registration's request deadline, cancellation ends the wait at once, and a worker that stays busy past the deadline is reported as a retryable timeout that names the busy tool the worker reported. Any other unexpected health payload stays
unreadable_output.Change and scope
src/ida/IdaBusyHealth.ts(new):readSettledHealthreadsserver_healthand, only when the payload matches the narrow busy shape (status: "busy", optional stringbusy_tool, other fields passed through), waits 100 ms and reads again. The wait ends when the worker answers, the caller's signal aborts, ortimeoutMs(the registration's request deadline, default 300000) has elapsed since the first read. The deadline is checked again after every pause, so no further read is started once it has passed, and the signal is checked before every retry read.AnalysisTimeoutError("server_health: headless worker still busy in <busy_tool>", timeoutMs, { cause: <last busy payload> }), the existing provider-neutral retryable timeout (provider_timeout).busy_toolis repeated only when it is tool-name-shaped ([A-Za-z0-9_.:-], at most 64 characters); an omitted or other value is reported asunknown, so producer text is never echoed into an error.src/ida/IdaProtocolValues.ts: the narrowbusyHealthSchema.modernHealthSchemais unchanged, so every other non-okpayload still fails as before; a payload withstatus: "busy"but an invalid shape (for example a non-stringbusy_tool) is rejected by the busy schema, so itsunreadable_outputdetail names the busy-shape problem instead of listing the missingokfields.src/ida/IdaSessionClient.ts: the headless branch of#observecalls the new helper;#observenow receives the execution options so the caller's signal reaches the wait (private seam).docs/ida-provider.mdanddocs/testing.mddescribe the wait and record the macOS arm64 headless workflow with IDA Professional 9.1 at upstream ida-pro-mcpc133c3853faa111a9b00ee615c013b720d0c4acd, after the real lane passed. The Windows statements are kept; Linux and other IDA releases stay unverified.Intentionally not included:
provider_state: "busy"/retry_action: "wait"style field shared with the Hopper path) is out of scope here; the busy tool is carried in the existingoperationtext.unreadable_outputis a separate issue ([Bug] Headless IDA reports a missing procedure as unreadable_output #1769).timeoutMs(no per-call timeout is added to the connection), so a health read already in flight at the deadline can extend the total beyond onetimeoutMs; a validokthat arrives after the deadline is accepted.Contract and boundary impact
src/idaheadless session identity read (#observe), before any operation result is produced.provider_timeoutwith its existingdetails.operationanddetails.timeout_ms.close()still releases the owned worker and private workspace after a timeout or cancellation (tested).Evidence and regression coverage
Tests added or updated:
src/ida/IdaBusyHealth.test.tsdrives the wait through an internal, defaulted clock seam with a virtual clock (no real sleeps): exact read count and pauses, the remaining-budget clamp, no read after expiry, an abort after the pause has fulfilled, and the last busy tool being the one reported.src/ida/IdaSessionClient.test.tsat the session-client boundary with the recording fake connection (no module mocks): busy then ok after re-reads, a busy observation taken after an operation and its cancellation, an invalid busy payload keeping its own diagnostic; busy forever givesprovider_timeout, retryable, the exact operation naming the busy tool, the raw payload as cause, a wait of at least the deadline and a bounded number of reads; an omittedbusy_toolreported asunknown; free-form or over-longbusy_tooltext not echoed; abort during the wait ends it promptly asAnalysisCancelledError; unknown status, non-stringbusy_tool, a payload without status, and busy followed by a malformed payload stayunreadable_outputwithout further waiting; every case asserts a cleanclose().Base reproduction or other evidence: the new busy tests fail on the unfixed code (4 failed on the original base: busy then ok, busy forever, unknown tool, abort during the wait; the malformed-payload cases pass there by design). Real lane before the fix on macOS 26.5 arm64, IDA 9.1, fixture built from
tests/conformance/ida/inventory.cwithclang -O0 -arch arm64, procedure_rea_fixture_add: 1 of 6 runs passed; the failures were in different MCP phases and carryunreadable_output,retryable: false, "status: Invalid input: expected "ok"" with the identity fields undefined. After the fix: 6 of 6, 5 of 5 and then 2 of 2 on the final code, eachpassed: true, modeheadless, all five checks.User-visible CLI/MCP output: the busy-forever case, sanitized:
{ "code": "provider_timeout", "retryable": true, "details": { "operation": "server_health: headless worker still busy in config_json_get", "timeout_ms": 300000 } }Remaining proof gaps: only macOS arm64 with IDA 9.1 was run for real; Linux and other IDA releases are unverified. The busy window is intermittent, so a passing real run does not by itself prove the wait; the deterministic tests do.
Observed, derived, and inferred claims remain distinguishable.
Artifact identity, source provenance, and failed attempts remain preserved.
Unsupported, incomplete, unavailable, or uncertain outcomes remain visible.
Validation performed
npm run check:fast,npm run check,npm run docs:check,npm run verify:test-discovery(816 files) - greennpm run test:focusedover the sixsrc/idatest files - 49 passednpm run test:fast- 377 files, 4292 passed, 7 skipped, 0 failedREA_IDA_MCP_CONFIG=$HOME/ida-mcp.json npm run verify:ida -- --target <inventory> --procedure _rea_fixture_add --report <path>- before the fix 1 of 6 passed, after the fix 6 of 6, 5 of 5 and 2 of 2 on the final code passed; no IDA worker was left runningbusy_toolunfiltered fails 2; changing the pause 100 to 200 ms fails 3; removing the remaining-budget clamp fails 1; removing the post-pause deadline check fails 2; removing the pre-retry abort check fails 1; dropping the invalid-busy diagnostic fails 1macOS arm64 only; Linux and Windows were not executed for this change.
test:boundaryandtest:acceptancewere not run because no boundary or acceptance file is touched.Compatibility, safety, and release
okfirst behaves as before; the only changed outcome is busy-then-ok now succeeding and busy-until-deadline becoming a retryable timeout instead ofunreadable_output.busy_toolis repeated only when tool-name-shaped, the raw payload stays in the non-projected cause, the wait is bounded bytimeoutMs(no read starts after it) and by the caller's signal (checked before every retry read), and a timeout or cancellation leaves cleanup of the owned worker and private workspace to the existingclose()path.Review checklist
type(scope): outcome.