Repository navigation
fix(native): retain failed pwntools process and runtime cleanup - #1708
Conversation
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6aca51bec4
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
|
TL;DR: On Linux with real pwntools 4.15.0, the PR works as described. When a worker's stop is unconfirmed, head keeps its runtime folder, and base deletes it. One gap remains: the CLI can retry cleanup at exit, succeed, and still report failure. The pwntools version is 4.15.0 (PyPI license field: "Mostly MIT, some GPL/BSD"). I set it with REA_PWNTOOLS_PYTHON and installed nothing. Base is the merge-base 13e03e9 (#1694). Head is 6aca51b. Results
The one skip is a test that only runs on non-Linux hosts. The 34 Linux-gated tests the PR body lists as skipped did run here. The same tests against base code gave 9 failed and 55 passed on Node 24. The PR applies cleanly to main 2165045 (#1733), and the tests pass there on Node 24. What to know
Not run
Thanks, 1NormalGuy, for the clear PR body and for tests that kill real processes. They ran on Linux and passed. Commandsexport REA_PWNTOOLS_PYTHON=~/.local/opt/pwntools-4.15.0/bin/python
# normal runs, base and head, per ELF
TMPDIR=<fresh dir> node scripts/rea.mjs inspect-binary-layout <elf> --format json
# MCP over stdio: initialize, tools/call inspect_binary_layout, then close stdin and time the exit
# forced failure: REA_PWNTOOLS_PYTHON=<wrapper that starts "env -i sleep 600 &" then execs the real python>
# SIGSTOP run: a watcher sends SIGSTOP to the worker whose argv names my own TMPDIR, then SIGCONT and SIGKILL by PID
# the unverified child has no REA run token, so REA refuses to signal the group:
# that is the "unowned or PID-reused process" case. The worker itself still runs the real pwntools Python.
# changed tests, one Node at a time
npm run test:focused -- src/application/binaryDiagnostics/BinaryLayoutService.test.ts \
src/application/binaryDiagnostics/RecordedCrashService.test.ts \
tests/boundary/binaryDiagnostics/pwntoolsLayoutProvider.test.ts \
tests/boundary/mcp/binaryDiagnosticsMcp.test.ts tests/boundary/mcp/recordedCrashMcp.test.ts \
tests/boundary/native/PwndbgCoreContext.test.ts \
tests/process-global/binaryDiagnosticsCliCleanup.test.ts tests/process-global/pwntoolsCleanup.test.ts
# applies to main
git apply --check <merge-base..head diff> # in a copy of main 21650455aThe ELF files were /usr/bin/true from Fedora's coreutils 9.10 (GPL-3.0-or-later). The other two were a small C program built with GCC 16.2.1, once as a PIE file and once as a non-PIE file. Every stray child I started was killed by its PID. Every run used its own TMPDIR, so I could count leftover folders and processes. Test setup
🤖 Generated with Claude Code (Sonnet 5.5, verified by Opus 5.5) 🦊 |
dandgabr
left a comment
There was a problem hiding this comment.
Consolidated review
- P2 — Reconcile cleanup retries before returning the CLI result. In
src/cli/binaryDiagnosticsCommands.ts:64, an initialcleanup_incompleteresult remains projected even if the CLI's exit cleanup retry succeeds and removes the retained pwntools runtime. This reports failure after the cleanup has completed. Reconcile the final cleanup outcome before serializing the result. This consolidates the still-applicable inline finding; no additional actionable findings were confirmed.
Keep failed worker/debugger cleanup owners and their private files until verified process shutdown and successful directory removal. Retry cleanup before another inspection and expose shutdown through the shared services, CLI and MCP server. Serialize acquisition without delaying cancellation of queued callers, and preserve primary diagnostics when CLI cleanup fails. The original decoder reproduction left a live owned Node child with its runtime deleted and no retained cleanup owner; the regression now verifies retention, admission blocking, retries and eventual process/file cleanup. Validation: 10 lifecycle/public CLI/MCP tests, 20 service tests, test discovery 814/814, build:cached and full check including architecture guards. Actual POSIX process/filesystem boundaries used injected Node workers on macOS; real Linux pwntools/GDB provider parsing was not exercised.
6aca51b to
1b565a4
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 66a32a7348
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
# Conflicts: # src/server/createServer.ts
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 433a744d15
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| this.profile.operation, | ||
| ); | ||
| } | ||
| if (!outcome.ok) throw outcome.error; |
There was a problem hiding this comment.
Refresh diagnostics after retrying process cleanup
When a timeout or cancellation occurs before output closes and the first stop is incomplete, the retry at line 466 can collect additional stdout/stderr before it successfully stops and disposes the supervisor. This line nevertheless rethrows the original OwnedCommandFailure, whose snapshot was taken in collectOwnedCommand immediately after the first stop, so those final bytes are lost even though cleanup is confirmed; rebuild the failure from the retained supervisor's final snapshot before clearing ownership.
AGENTS.md reference: AGENTS.md:L13-L13
Useful? React with 👍 / 👎.
| [ | ||
| pending.process.launch.ownership?.runId ?? "owned-decoder", | ||
| ...(pending.root === undefined ? [] : [pending.root.path]), | ||
| ], |
There was a problem hiding this comment.
Include the retained process group in cleanup resources
When process cleanup remains incomplete, this array lists only the run ID and runtime directory even though the retained supervisor still has leaderPid and processGroupId. OwnedCommand's non-transfer cleanup error reports pid: and process-group: coordinates, and omitting them here means the projected cleanup result does not identify the potentially live group the caller must inspect; include both while this owner is retained.
AGENTS.md reference: AGENTS.md:L19-L19
Useful? React with 👍 / 👎.
|
@1NormalGuy, thanks for tackling the cleanup ownership edge cases across the pwntools and pwndbg flows. Retaining workers and runtime files until cleanup is confirmed, then reporting the final outcome while preserving validated crash evidence, makes failure handling much more trustworthy. Welcome aboard REA! |
Summary
Retain owned decoder/debugger processes and runtime files until cleanup is confirmed. Finalize one-shot CLI inspections before publishing their original outcome, so a successful cleanup retry returns the decoded report and a genuine decoder failure retains its original diagnostics. These defects were reproduced during repository review; there is no related issue.
Problem and expected behavior
Previously, failed cleanup could discard its supervisor and delete files still belonging to a worker. A later CLI cleanup retry could also succeed while the command still reported stale
cleanup_incompleteresources and hid its decoded report. Ownership must survive failed cleanup, and published outcomes must reflect the final cleanup attempt.Change and scope
close()to retry.partial_observationerror field when cleanup remains incomplete, including artifact identity and diagnostics. Failed debugger enrichment retains validated basic core evidence with explicitrequested: trueandstatus: unavailable.Contract and boundary impact
close()responsibility.Evidence and regression coverage
Actual owned Node workers and filesystem boundaries cover retained live children, retryable shutdown, admission blocking, queued cancellation, primary-decoder retirement before debugger acquisition, and current resource lists after partial cleanup. Public CLI regressions cover restored success, preserved command failure, recovered process cleanup, persistent runtime-removal failure with a retained report, and cancellation during cleanup. CLI/MCP service checks cover shared workflow projection and shutdown.
Representative final outcomes:
execution_failureafter cleanup recovers.cleanup_incompletewith current resources and a validatedpartial_observationwhen available.primary_errorandcleanup_error.Validation performed
On fix commit
a357aa41a, with Node 24.18.0 and npm 11.16.0:npm run check:ci: passed, including typecheck, lint, formatting, unused-code/metadata checks, duplication checks, module boundaries, and architecture guards.npm run docs:check: passed, including README consistency.npm run verify:test-discovery: all 949 test files discovered exactly once.git diff --checkand the pre-push check: passed.After merging current upstream
ad08510a1into433a744d1, the affected EVM/layout/recorded-crash MCP and retirement suite passed all 15 tests across four files, andnpm run check:cipassed again. The server conflict preserves upstream static EVM binding and this PR's shared diagnostic cleanup.The local Node-worker regressions establish actual POSIX ownership/filesystem behavior, with only the decoder availability gate simulated. They do not establish real pwntools/GDB behavior. Previous reviewed commit
66a32a734passed all CI, including real Linux pwntools and recorded-crash workflows. CI on final merge head433a744d1is pending; real-provider verification will be recorded separately when it completes.Review follow-up
Merged current upstream
maincleanly. Addressed the cleanup-retry/result-reconciliation thread at the owning process/provider boundary. Separate standards and specification reviews also found and fixed stale resource lists after partial cleanup, lost decoded observations on persistent cleanup failure, and missing failure-test teardown. Follow-up review also fixed both worker-file races: failure markers and debugger replies are consumed only after confirmed retirement. Eight real-worker regressions cover surviving descendants rewriting files, unresolved ownership in both stages, and debugger output overflow. Optional debugger validation retains truthful partial evidence. No remaining findings from those reviews.Review checklist