Repository navigation
Conversation
Dispatch a Claude Code run into a workstream — a child session dedicated to that one job — and mirror its top-level messages as items in that workstream's stream, so the originating request returns immediately and nobody has to open the harness transcript to find out what a run did. Production change is one option in one package. `claudeCodeAgent` and `createClaudeCodeAgentCapability` take `sessionState: false`, which suppresses three things together: the `sessionStateSchema` declaration, the session reads and writes that go with it, and the SDK `resume`. The third is the one that hides — suppressing the schema alone leaves provider resolution intact, so a custom `sessionProvider` returning a saved id for an empty key would silently resume a prior conversation while every schema-shaped assertion stayed green. The test for it asserts on the options handed to the SDK. Both halves are needed. A capability contributes its schema through a channel the task board's block walk cannot see, so a capability still declaring it would put the key back through the one door the refusal cannot check — and it would appear to work. Default is `true`; an unset caller is unaffected in every respect (BP-030), including that a mid-stream throw still throws. Settlement — how a finished, failed or lost run is recorded — is deliberately NOT here. It was revised seven times across seven different surfaces and none of them touched the mirror, so it moved to FIX-1182. This takes the board's default: no retry allowance, so `shouldRetryOnFail` is false and no agent ever re-runs over its own commits (the one genuine harm in this area). The cost is stated rather than hidden, in the spec and in the goal check: a LOST run is written off rather than staying recoverable. Verification: - goal check `goals/harness-workstream/mirrors-a-coding-run-as-workstream-items/` — a real Claude Code run dispatched into a real workstream through the real board, read back over the real HTTP routes on store-sqlite. Two consecutive PASSes. It names its adapter deliberately: the in-memory store ignores `include_items` by documented contract, so a readback assertion there proves nothing about the route. The check asserts 0 items WITHOUT the flag, so the flag is load-bearing rather than decorative. - 6 CI behaviours: 4 in `packages/claude-code/test/sdk/agent.spec.ts`, 1 in its capability spec, and board acceptance in `packages/integration-tests`. Acceptance lives there rather than in the vendor package (which would take an orchestration dev dep for one test) or only in the goal (which needs a real model and the optional SDK peer, so it does not run in CI). - `pnpm typecheck` across all 61 projects. Two manifest fixes the build needed, neither optional: - `goals/` had no dependency on `@flow-state-dev/claude-code`; there is no way to dispatch a coding run from a goal without it. - `packages/claude-code`'s optional peer is `"*"`, and the lockfile had pinned it to 0.1.77 while `goals` resolved 0.3.234 — two copies in one workspace, with which one you got depending on the importing file's location. The old copy's bundled CLI exits 1, surfacing as a bare "process exited with code 1" that reads like a broken dispatch. A root `pnpm.overrides` entry gives the workspace one copy. The published peer range is untouched.
Throwaway, on the spec branch, never merges. Settles the premise the spec rests on by running a real coding job rather than reading type declarations. Findings: file writes do arrive as Write/Edit tool_use blocks with file_path on the input; tool results are prose, not the declared structured output; and the todo surface is TaskCreate/TaskUpdate, not TodoWrite, which the shipped CLI does not offer at all. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MVcrTezeLMBQRVHRg2QX2f
…state Records what a coding run touched and what it thought its job was, as ordinary resource collections read over the route that already ships. The observed todo list lands where nothing will execute it, so seeing a fork stays separate from acting on one. Branched from fix/LAB-133, not main — nothing in this epic merges yet. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MVcrTezeLMBQRVHRg2QX2f
|
The latest updates on your projects. Learn more about Vercel for GitHub. 3 Skipped Deployments
|
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MVcrTezeLMBQRVHRg2QX2f
There was a problem hiding this comment.
Simplify review — spec(LAB-134)
Verdict: COMMENT (direction sound; a few simplification forks worth settling before implementation locks in)
~838 LOC across the spec + throwaway POC. This is a well-scoped spec PR: the POC did real work (TodoWrite → TaskCreate/TaskUpdate, non-positional ids), and the proposed shape — two TranslatedEvent kinds, a vendor-free recorder, two session-scoped resource collections, recordWork opt-in — fits the existing translate.ts / emit.ts split and tenet 2 better than inventing new framework machinery.
What looks right
- Resource collections over tasks for the plan half (decision 1) is the clean layering call — avoids fake execution envelopes and keeps
@flow-state-dev/claude-codeoff orchestration. - Translation-layer tool table + framework-vocabulary recorder mirrors how tool items already correlate via open-call state; the spec's own reviewer question (§7 sketch) lands on the right answer.
- POC placement under
spec-poc/per repo convention; probe is small and rerunnable. - §10 anti-game (empty list = FAIL, board row count, no transcript/git reads) directly addresses the epic's false-green history.
- Rejected alternatives (return-value-only, files-only) are argued honestly; I would not push a files-only v1 unless schedule forces it.
Where complexity may be higher than necessary
-
tool_use_resultnot measured (biggest fork). The POC and settled claims focus ontool_resultblock content being prose. The SDK'sSDKUserMessagealso carriestool_use_result— structured per-tool output (TaskCreateOutput.task.id,TaskUpdateOutput.taskId/statusChange,FileWriteOutput.type/filePathare all declared insdk-tools.d.ts).probe.mjsnever records that field, andSdkMessageLiketoday omits it. Before locking prose id-recovery and "authoritative record is tool input" in §7/§12, extend the POC one line: logm.tool_use_resulton eachusermessage. If structured output is present on the wire, the translation layer can emitplan_item/file_touchedwithout regex — and the settled-claims table needs a third row, not a blanket REFUTED on structured results. -
POC artifact drift.
observed-run.jsonschema (toolUse,todoWriteAvailable) does not match whatprobe.mjscurrently prints (fileMutations,planCalls,todoWriteOffered). A reviewer who diffs probe output against the checked-in capture will get a false signal. Either regenerate the JSON from the current probe or drop the file and point readers at probe stdout only. -
"Open questions: none" feels early while (1) is unsettled. I'd rename to one explicit open: structured result availability on
tool_use_resultvs prose-onlytool_result.content. -
Minor nits: §7 POC link still says
PR_NUMBER; probe tracksNotebookEdit/MultiEditbut the spec table only namesWrite/Edit— either scope the v1 table or say why notebook/multi are deferred.
Alternatives considered (and why I'd still stay on this spec's track)
| Alternative | Why not (for now) |
|---|---|
Post-hoc projection from existing tool_output items |
Loses mid-run readback; duplicates item stream semantics; resource route is the stated proof surface |
| Third-party file watcher / diff lib | Data is already on the tool stream; adding FS watchers would be strictly more complex |
| Files-only v1, plan later | Spec argues correctly that the plan half is the epic's kill-line test; only worth it under schedule pressure |
| Recorder owns tool-name table | Breaks vendor-free recorder + scripted vitest seam the spec depends on |
Performance
No hot-path concerns for a spec: per-tool-call upserts into bounded collections are fine; FIX-1180 note is sufficient. If tool_use_result removes prose parsing, that's a small win too.
Recommendation
Approve directionally after a short POC pass on tool_use_result and artifact sync. No need to re-architect — likely a spec §7/§12 wording change and simpler translation pseudocode, not a new subsystem.
Sent by Cursor Automation: Review For Best Approach
|
|
||
| - A file mutation's path is on the **tool input** (`file_path`). The result is a prose | ||
| sentence, so it supplies only success or failure. | ||
| - A to-do creation's wording is on the tool input; its **id is only in the result's prose**, |
There was a problem hiding this comment.
Before implementation commits to prose id recovery, extend the POC to log tool_use_result on each user message. The SDK types declare structured TaskCreateOutput (task.id) and TaskUpdateOutput (taskId, statusChange) separately from the prose string in tool_result.content. If the wire carries tool_use_result (likely — it's on SDKUserMessage in sdk.d.ts), translation can emit plan_item from typed fields and drop the fragile regex path. Worth settling here because it is the plan half's main complexity driver.
| - **Settled:** file mutations arrive as `Write` / `Edit` tool calls carrying `file_path` on the | ||
| input, at the seam the translation layer already sees — **CONFIRMED.** Measured twice. The | ||
| premise held; the design did not change. | ||
| - **Settled:** a tool result carries the structured output its type declares — **REFUTED.** |
There was a problem hiding this comment.
This settled claim may be too broad: you refuted structured output on tool_result.content, but the POC did not measure message-level tool_use_result. Suggest splitting into two rows — content block prose (REFUTED) vs tool_use_result structured (UNMEASURED / TBD) — so implementers do not treat prose parsing as the only path.
| this epic merges until the user says so. LAB-133 supplies the workstream a detached run lands | ||
| in, the item stream this sits beside, and the goal-check shape §10 follows. | ||
|
|
||
| **Open questions:** none. |
There was a problem hiding this comment.
Given the tool_use_result gap above, I'd keep one explicit open question here rather than "none". Otherwise §12 and §7 read as closed while the highest-risk parsing decision is still unverified.
| > vendor-free and unit-testable with scripted events — but it also means the pure layer now | ||
| > carries a little correlation state it did not before. | ||
|
|
||
| **POC — [`spec-poc/LAB-134-harness-graph-surface/`](https://github.com/fixpoint-labs/flow-state-dev/pull/1329/files)**, on this branch, never merges. |
There was a problem hiding this comment.
Placeholder PR_NUMBER — swap for #1329 (or drop the URL and keep the relative path only) so the spec link works on the branch.
| cliVersion = m.claude_code_version ?? null; | ||
| availableTools = m.tools ?? []; | ||
| } | ||
| if (m.type === "user") { |
There was a problem hiding this comment.
The probe only correlates tool_result block content. Consider also capturing m.tool_use_result on user messages (and printing whether it is structured for TaskCreate / Write). That one addition could simplify the whole §7 correlation story — or confirm prose parsing is truly required.
| "result": "Updated task #2 status" | ||
| } | ||
| ], | ||
| "toolResults": [], |
There was a problem hiding this comment.
Schema drift: this file uses todoWriteAvailable / toolUse, but the current probe.mjs emits todoWriteOffered / fileMutations / planCalls. Regenerate from the probe or remove the static capture so reviewers are not comparing against stale shape.
| > plan update → emit plan_item { id, status } ← id was on the input | ||
| > id not recoverable → emit nothing (decision 3) | ||
| > | ||
| > in the work recorder (framework vocabulary only, no tool names): |
There was a problem hiding this comment.
Perf follow-up (simplify review): §7's per-tool-result collection upserts inside the SDK message loop will stack read–modify–write + persist on top of existing per-message emitTranslatedEvent awaits (and FIX-1180 item persistence). For a busy coding run that's real wall-clock regression.
§10 only needs readback after a finished run — consider specifying the recorder as an in-memory accumulator with a single flush at run end (or debounced per-key coalescing). Same observable state for the goal check, far less hot-path I/O. If mid-run HTTP freshness is required, say so explicitly; otherwise flush-at-end is the simpler default.
Secondary: reuse or extend existing openTools correlation in emit.ts rather than parallel call-id maps in translate, to avoid drift.
| **Goal:** after a real coding run dispatched into a workstream, the files it changed and the | ||
| plan it kept are both readable from FSD state alone — no transcript, no working tree, no git. | ||
|
|
||
| `goals/harness-workstream/graphs-a-runs-files-and-plan-into-state/run.mts`, real model, real |
There was a problem hiding this comment.
Implementation shortcut from reuse review: clone goals/harness-workstream/mirrors-a-coding-run-as-workstream-items/run.mts (SQLite + HTTP + taskGoalToPrompt + sessionState: false) and add resource-route readback — don't stand up a second goal harness. Reuse normalizeResourcePath from @flow-state-dev/core/types for file keys; follow parse-output.ts style for any prose id recovery.
| > file_touched → upsert the file list, keyed by the normalized path, | ||
| > bumping the counter for its kind | ||
| > plan_item → upsert the plan list, keyed by the id, | ||
| > appending the status to a bounded transition log |
There was a problem hiding this comment.
Consider trimming v1 state shapes: goal check only needs files present + touch kinds + plan items with status that moved once. A bounded transitions[] log and per-file writes/edits counters add merge rules and tests beyond §10's proof — status + previousStatus + lastTouchedAt / lastKind may be enough for this slice.
| > anything that throws → swallow, surface as a status note, keep going (decision 3) | ||
| > ``` | ||
| > | ||
| > What this asks a reviewer: *is the translation layer the right home for the tool-name table, |
There was a problem hiding this comment.
Close the open reviewer question here — translation owns tool names + correlation, recorder stays vendor-free. Leaving it open invites two implementations.
| propagate (decision 3). *Depends on:* 1. | ||
| 3. **The block option and its two collection declarations**, off by default. *Test:* off is | ||
| byte-identical to today, including declaring no resources; on declares both. *Depends on:* 2. | ||
| 4. **Forward the option through the capability.** *Test:* a capability built with it declares |
There was a problem hiding this comment.
Worth confirming whether step 4 is a real seam: createClaudeCodeAgentCapability already forwards options to claudeCodeAgent, and resources bubble via declaredResources (unlike sessionStateSchema). If no dual-channel bug exists, collapse 3–4 into one step + smoke test.
second-look: spec review (automated, restraint lens)Per this PR's own review contract, one finding above the line: Finding — the plan record's transition log goes further than "how its status moved" requires. §7's sketch and §5's read-back example give each observed plan item a A single Worth noting: the sibling
Recommendation: fold this into §6 Decision 1 or add it as Decision as explicit scope — pick current-status-only (or Considered and deemed justified, not flagged:
Posted by an automated second-look pass (restraint lens) — for the implementing agent to weigh per the PR's §13 convention. Generated by Claude Code |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 1b2498d53d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| **What it touched.** Every file the run writes or edits becomes an entry in a list, keyed by | ||
| the file's path, recording how it was touched, when, and whether it worked. Not the contents | ||
| — just the fact of the change. Afterwards you can ask "what did this run change?" and get an | ||
| answer without opening a transcript or a diff. |
There was a problem hiding this comment.
Restrict or complete the touched-file claim
The spec promises that every file changed by a run appears here, but §7 only translates named file-mutation tools. With the default agent configuration, files changed through Bash—for example redirects, sed -i, rm, mv, formatters, generators, or git apply—produce no Write/Edit event, so downstream consumers can receive an empty or incomplete list after real mutations. Either reconcile against the repository/filesystem or describe this as observed file-tool operations rather than an authoritative changed-file index.
Useful? React with 👍 / 👎.
| - **`@flow-state-dev/core`** — untouched. Both collections are ordinary | ||
| `defineResourceCollection` values. | ||
| - **`@flow-state-dev/engine`** — untouched. `GET /sessions/:id/resources/:ref` already lists a | ||
| collection's instances with their state. |
There was a problem hiding this comment.
Expose collection state before relying on HTTP readback
The proposed readback cannot work with merely an ordinary collection declaration: handleListCollectionState in packages/engine/src/routes/resource-routes.ts returns 403 unless config.client?.state?.read === true, while defineResourceCollection defaults to no client visibility. Require both collections to declare intentional client-readable state and test that permission, otherwise the §5 examples and §10 real-HTTP goal fail.
Useful? React with 👍 / 👎.
| **Where the two collections live.** Session-scoped, so a detached run's records land in that | ||
| run's own workstream session and are read back there — the same place its items already are. |
There was a problem hiding this comment.
Separate records for reused workstream runs
When a second task uses the same board, worker, and topic, the task board deliberately reuses the existing workstream session (packages/orchestration/README.md:349-354), so these session-scoped collections retain the first run's rows. The second run will aggregate file counters by path and mix plan items with earlier work, meaning the route reports workstream-lifetime activity rather than answering the promised question about a particular run; namespace records by request/run identity or define and test an explicit latest-run reset.
Useful? React with 👍 / 👎.
| **Correlation, and where the data actually is.** Both events are emitted on the tool | ||
| **result**, not the call, because the call carries the subject and the result carries the | ||
| outcome. The translation layer already does exactly this for tool items via an open-call map; | ||
| these follow that pattern. |
There was a problem hiding this comment.
Persist mutation attempts before awaiting results
Emitting the file record only when a tool result arrives drops a mutation if cancellation, process loss, or an SDK throw occurs after the filesystem operation but before its tool_result is delivered. The file may already have changed while the audit state says it was never touched, which is especially damaging for the failed or lost detached runs this state should help inspect; record an attempted/unknown entry at tool_use and settle it on the result, or reconcile afterward.
Useful? React with 👍 / 👎.
| > on that call's result: | ||
| > file mutation → emit file_touched { path, kind, ok } | ||
| > plan create → recover the id from the result; emit plan_item { id, title, status } | ||
| > plan update → emit plan_item { id, status } ← id was on the input |
There was a problem hiding this comment.
Preserve failed plan-update outcomes
For a TaskUpdate whose result has is_error: true, this design still emits only the requested { id, status }, so the recorder can append completed even though the harness rejected that transition. The existing translator already exposes isError on tool results; require failed updates to be skipped or represented as failed attempts, and cover that error path in the scripted tests.
Useful? React with 👍 / 👎.
| | Case | Expected | | ||
| |---|---| | ||
| | Path that cannot be keyed (traversal segments, control characters) | Skip the entry, note it, continue — the key normalizer throws, and decision 3 says that must not reach the run | | ||
| | Relative and absolute forms of one path | Collide into one entry; leading slashes are stripped by the key normalizer. Acceptable; note it | |
There was a problem hiding this comment.
Canonicalize paths against the checkout root
The stated collision does not hold for actual absolute and relative forms of the same file: normalizeResourcePath only strips leading slashes, so /workspace/repo/src/a.ts becomes workspace/repo/src/a.ts, while src/a.ts remains distinct. If different harness messages report those two forms, the collection creates duplicate entries and splits the counters; normalize against the run's checkout/cwd before using the path as the key.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Grok simplify review (2nd parallel pass — this is the Grok version)
Verdict
Architecture: RIGHT. Fidelity lock: OVERSHOT. Decisions 1–3 are the right product calls. Two session-scoped resource collections behind one off-by-default option, vendor tool names only in @flow-state-dev/claude-code, core/engine/orchestration untouched — that is the clean shape. I would not redesign the storage model.
What went farther than necessary is locking the unstable plan half at full fidelity in the same bite: id-from-prose recovery + bounded transitions[] + a goal that hard-fails on empty plan and asserts transition-log movement. The POC correctly killed TodoWrite; it does not justify treating TaskCreate/TaskUpdate as a durable, history-bearing contract yet.
Disposition: COMMENT — directionally approve Decisions 1–3; settle the fidelity forks below before implement locks them. Not REQUEST_CHANGES (no rewrite needed). Not APPROVE (transition-log / goal coupling would bake in avoidable complexity).
What to keep
- Decision 1 — plan off the dispatching board. Real
taskSchemais an execution envelope (attempts,leaseUntil,claimedBy). Resource collection is correct; do not “simplify” onto Task. - Decision 2 — paths, never contents.
- Decision 3 — recorder never kills the run, paired with empty-list = FAIL.
- Both files and plan as the final epic answer — files-only forever defers the kill line. Thin/sequenced plan is the open fork, not dropping plan.
- Tool-name tables at the vendor edge — matches
SUBAGENT_TOOL_NAMESintranslate.ts. Correlation is not a smell. - No 3rd-party package win. Reuse
normalizeResourcePathfrom core. Directory shape (claude-codeonly) is clean.
Cleaner change available
- Drop/defer
transitions[](highest lever). Currentstatus(+ optionalupdatedAt) answers the epic claim. Live progression already rides the tool stream. You already flagged uncertainty; §10 hard-locks it. - Prefer not extending
TranslatedEvent. Existingtool_call/tool_resultcarry name, args, outcome. Emit’s switch is exhaustive — new kinds force no-ops. Recorder on existingtool_*events is likely smaller. Uncertain until both are sketched — don’t lock the sketch’s two new event kinds. - Plan thin under vendor churn (should-discuss, not certain). §3 correctly rejects files-only forever; it over-rejects files-hard + plan-thin. Same
recordWorkflag; files hard in the goal; plan as title+status best-effort until the harness.d.tsstops lying. - Decision surface gap. Expensive forks (status vs log; plan-goal hardness) are not numbered; Decision 3 is the cheap one. Surface those to the PO.
Skipped
Capability dual-declare may be cargo-culted from LAB-133 — verify bubble semantics. Counter shape fine. POC quality out of scope.
Bottom line
Keep the approach. Shrink plan to current status, don’t bake transitions[] into the goal, prefer consuming existing tool_* events. Still proves “readable from FSD state after a detached run.”
Sent by Cursor Automation: Grok Review For Best Approach
| **The simpler approach: record only the files, drop the to-do half.** Genuinely tempting — | ||
| the file half is the one with a stable surface and it carries most of the immediate value. | ||
| Insufficient because the to-do half is the only part that tests the epic's actual claim. Files | ||
| tell you what happened; the to-do list is the only place the agent says what it *thought it was | ||
| doing*, which is the thing a knowledge layer weighs. Dropping it defers the risk rather than | ||
| retiring it. |
There was a problem hiding this comment.
Grok simplify — should-discuss (not certain this must change)
Rejecting “files-only forever” is right for the epic kill line. What’s overshot is treating that as also rejecting files-hard + plan-thin sequencing.
The POC showed the plan surface already drifted (TodoWrite → TaskCreate/TaskUpdate, id-in-prose). That argues for less plan fidelity in v1, not locking dual hard-asserts in one goal. Cleaner middle: same recordWork option, durable touched-files as the load-bearing proof, plan as title + current status (best-effort) until the harness stops lying in its .d.ts.
Unsure whether this needs a follow-up issue vs a thinner §10 — flagging the fork, not demanding a rewrite.
| { "topic": "3", "state": { "title": "Create notes.txt", "status": "completed", | ||
| "transitions": [ { "status": "pending", "at": … }, | ||
| { "status": "in_progress", "at": … }, | ||
| { "status": "completed", "at": … } ] } }, |
There was a problem hiding this comment.
Grok simplify — highest-leverage cut
transitions[] is not entailed by “todo progression is readable as state.” Current status (and maybe updatedAt) already answers what the agent thought its job was after the run. Live progression is already on the tool stream.
This is the only field that grows with run length — you already flag uncertainty in the PR body. Prefer deferring history until LAB-135 names a grader that needs thrash. Example + sketch + §10 currently lock it as if Decision 1 required it; it doesn’t.
| > What this asks a reviewer: *is the translation layer the right home for the tool-name table, | ||
| > or should the recorder own it?* Keeping it in translation is what makes the recorder | ||
| > vendor-free and unit-testable with scripted events — but it also means the pure layer now | ||
| > carries a little correlation state it did not before. |
There was a problem hiding this comment.
Grok simplify — answer from existing code; soft on event kinds
Tool-name tables belong at the vendor edge — same pattern as SUBAGENT_TOOL_NAMES in packages/claude-code/src/sdk/translate.ts. Correlation on call→result already exists. Extending that is not a purity smell.
Softer fork: you may not need new file_touched / plan_item members on TranslatedEvent. tool_call already carries name + arguments; tool_result carries outcome. emitTranslatedEvent is an exhaustive switch — new union variants force emit no-ops. A recorder consuming existing tool_* events is likely the smaller change. Uncertain until sketched both ways — don’t lock the sketch’s two new event kinds as the contract.
| > file_touched → upsert the file list, keyed by the normalized path, | ||
| > bumping the counter for its kind | ||
| > plan_item → upsert the plan list, keyed by the id, | ||
| > appending the status to a bounded transition log |
There was a problem hiding this comment.
Grok simplify
“Appending to a bounded transition log” is the overshoot. Upsert by id with current title/status is enough for Decision 1’s fork-detection claim. If a bounded log survives anyway, mirror Task’s writeLogTruncated pattern — but prefer not shipping the log.
| **What it asserts:** | ||
| - the touched-files list names the held-out files, with the touch kinds the job implies | ||
| - the plan list carries at least the held-out number of distinct items, keyed by the harness's | ||
| own ids, and at least one row's transition log shows a status actually moving | ||
| - **the dispatching board's task collection still holds exactly the one row that was filed** — |
There was a problem hiding this comment.
Grok simplify — goal coupling hard-locks the overshoot
These asserts bake full plan fidelity into the proof: held-out plan count, harness ids, and transition log shows status moving.
Count + ids with current status only still prove Decision 1 (board still has exactly one row). The transition-log assert should drop with the log.
Also reconsider whether empty plan must FAIL the same way empty files must. Files surface is POC-confirmed; plan surface already moved. Symmetric hard-fail is the risky part. Uncertain on the exact softer criterion.
| **Why the plan is a resource collection and not a task collection.** Our `Task` carries an | ||
| execution contract — attempts, lease, claimant, retry budget. Nothing on our side ever claims, | ||
| leases, or settles an observed to-do, so writing one as a `Task` would populate that envelope | ||
| with defaults nobody set and nobody honours, and leave rows that look claimable and are not. | ||
| It would also invert the package layering: the vendor adapter depends on core alone, and the | ||
| task substrate is a layer above it. The observed shape is deliberately *smaller* than a task — | ||
| which is decision 1 expressed in the type. |
There was a problem hiding this comment.
Grok simplify — keep this; do not “simplify” onto Task
Checked against real packages/orchestration/src/tasks/schema/task.ts: execution envelope is real (attempts, leaseUntil, claimedBy, retry/abandonment counters, writeLog). Rationale holds. Off-board resource collection is the clean layering call, not accidental complexity.
| 1. **A run's to-do list is recorded where nothing will execute it.** The observed items go in | ||
| their own list, separate from the work queue that dispatched the run. *Alternative | ||
| rejected:* writing them straight onto that queue, which reads as "the agent's plan is now | ||
| our plan". *Locks in:* an agent that decides mid-run to do five more things cannot start | ||
| five more jobs by writing a to-do list — a human or a later step has to pick one up on | ||
| purpose. The cost is one extra step whenever we do want to adopt one. | ||
|
|
||
| 2. **We record what the run touched, never what it wrote.** Paths, how each was touched, when, | ||
| and whether it succeeded. Not file contents, not diffs. *Alternative rejected:* keeping the | ||
| written content, which the harness does hand us for free. *Locks in:* our record stays a | ||
| small index pointing at the working tree instead of becoming a second copy of the | ||
| customer's source in our store — with the storage growth and the disclosure question that | ||
| would bring. Anyone who needs the content reads the repo. | ||
|
|
||
| 3. **Watching the work must never break the work.** Anything the recorder cannot handle — a |
There was a problem hiding this comment.
Grok simplify — Decisions 1–3 are right; surface is incomplete
Keep all three. Soft note: Decision 3 is the cheap engineering call, while the expensive forks — status vs transition log, and how hard the plan goal is under vendor churn — are not numbered here. Those are the ones a PO should price before implement.
Re-draft: rewrites §2/§6/§7/§9/§10/§12 after review. The load-bearing correction: the POC concluded 'tool results are prose' having measured only the content block. Re-ran it logging tool_use_result — the full declared structured Output IS on the wire for file tools. Claim now scoped to what was measured, and the plan tools are marked unmeasured rather than assumed. Also: scoped the headline claim to observed file-tool operations (shell-driven edits are invisible and stay so); both collections must declare client state reads or the route 403s (verified in resource-routes.ts); records namespaced per run because a workstream session is reused; coalesced flush rather than per-event or end-only; rejected updates never record as applied; dropped the transition log and its cap for previousStatus; collapsed the capability step after verifying uses merges resources into block config; goal check extends the LAB-133 harness and grades the plan half at title+status with a shaped job. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01MVcrTezeLMBQRVHRg2QX2f
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8eaf7913d3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| { "items": [ | ||
| { "topic": "<runId>/repo/src/checkout.ts", | ||
| "state": { "lastKind": "edited", "lastTouchedAt": 1787021400123, "ok": true } } |
There was a problem hiding this comment.
Read collection rows from
clientData
Both readback examples place projected resource state under state, but handleListCollectionState in packages/engine/src/routes/resource-routes.ts returns each row as { topic, storageKey, clientData }, and the route tests assert that envelope. Following these examples in the planned HTTP goal or documentation therefore reads undefined after a valid 200 response; use clientData for both collections.
Useful? React with 👍 / 👎.
| *(Not a separate step for the capability: `uses` merges a capability's resources into the | ||
| consuming block's config, unlike `sessionStateSchema` — verified in `blocks/handler.ts` — | ||
| so forwarding the option is sufficient and a smoke test is the right coverage.)* |
There was a problem hiding this comment.
Declare the collections on the capability itself
When recordWork is enabled through createClaudeCodeAgentCapability, forwarding the option only places the resource-declaring agent handler in the capability's tools preset. resolveCapabilities merges resources declared by the capability itself, while generator remains a leaf and does not collect declarations from preset tools; consequently the consuming flow omits these collection refs and the later HTTP lookup returns 404. The capability must contribute the same resource definitions explicitly rather than relying only on option forwarding.
Useful? React with 👍 / 👎.
| - **Shape the job so planning is required** — multi-step work with a real dependency between the | ||
| steps, so the plan surface is exercised deterministically rather than incidentally. |
There was a problem hiding this comment.
Make plan-tool invocation deterministic
A dependent multi-step prompt does not make the plan surface deterministic: the POC job in probe.mjs already creates a file and then edits that same file, explicitly requesting a two-item plan, yet the measured runs reported that two of four executions used no plan tools. A correct recorder can therefore fail the mandatory non-empty-plan goal solely because the model completed the work without TaskCreate/TaskUpdate; use a deterministic invocation path or keep source non-invocation outside the implementation verdict.
Useful? React with 👍 / 👎.
| **State shapes are deliberately flat — no accumulating history.** A file entry carries | ||
| `lastKind`, `lastTouchedAt`, `ok`. A plan entry carries `title`, `status`, `previousStatus`, | ||
| `lastTouchedAt`. There is **no transition array and no cap policy**: the sibling file entry |
There was a problem hiding this comment.
Add a pending outcome for unsettled file calls
Fresh evidence since the earlier mutation-attempt comment is that the revised design now requires persisting an unsettled call-time attempt, but the specified row still has only boolean ok. That field can represent success or failure, not the unknown state required before a result arrives: writing false misreports a pending operation as failed, while waiting for settlement again loses killed-run attempts. Add an explicit pending outcome or make ok nullable and define its meaning.
Useful? React with 👍 / 👎.
| | A file changed through the shell (`sed -i`, redirect, `mv`, formatter, `git apply`) | Not recorded, by design. Decision 2; never closed by reading the filesystem | | ||
| | Path that cannot be keyed (traversal segments, control characters) | Skip the entry, note it, continue — the key normalizer throws and that must not reach the run | | ||
| | Same file as `/workspace/repo/src/a.ts` and `src/a.ts` | One entry. Canonicalize against the run's working directory before keying; the key normalizer alone only strips leading slashes | | ||
| | A `TaskUpdate` whose result is an error | Recorded as a failed attempt, **never as applied** — writing the requested status for a transition the harness refused is the worst available outcome | |
There was a problem hiding this comment.
Store rejected plan-update outcomes
Fresh evidence since the earlier rejected-update comment is that the revised state shape still provides only title, current/previous status, and a timestamp while this row now promises to record a failed update attempt without applying its requested status. Leaving the statuses unchanged records no evidence of the rejection, while changing either status falsifies the harness state. Add an attempt/outcome field or weaken this contract to explicitly ignore rejected updates.
Useful? React with 👍 / 👎.
Re-draft: rewrites §5/§7/§8/§10 after round 2.
The reversal: I concluded from blocks/handler.ts that a capability forwarding
the option inherits the block's resource declarations, and collapsed the
capability step on that basis. Wrong. mergeSurfaceInto merges surface.resources
and never reads surface.tools for resources; collectBlockResources gathers from
action blocks only; and defineFlow states outright that a generator 'bubbles
none of its tools' rails by design'. So the flow registers nothing and the route
404s at read time on a build that succeeded. Step 4 restored as a real
deliverable; size raised rather than absorbed.
Also: readback examples corrected to clientData (the route returns
{topic, storageKey, clientData}, so reading .state yields undefined after a
valid 200 — the seq/itemIndex failure shape); ok:boolean replaced with a
three-valued outcome so a recorded-at-attempt write can be pending and a
rejected plan update has somewhere to land without falsifying harness state;
and the goal check's INCONCLUSIVE arm reinstated, since the job-shaping meant
to make a hard fail safe was already measured failing.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MVcrTezeLMBQRVHRg2QX2f
|
Spec approved — direction settled, proceeding to implementation. Two review rounds spent, both closed. The direction was unchallenged in both: every reviewer endorsed the storage model (two session-scoped resource collections behind one off-by-default option, vendor tool names confined to the adapter, core/engine/orchestration untouched). What the rounds changed was fidelity and correctness, not shape. Three things settled here that are worth naming, because each was a claim someone held confidently and the code refuted:
Two product calls are recorded in §6 and are the place to overrule if either is wrong: the observed plan is recorded off the dispatching board (so an agent writing a to-do list cannot enqueue work nobody approved), and we record paths, never file contents (no second copy of source in our store, at the cost of not answering "what changed in this file" without going to the repo). This PR does not merge — Linear holds the durable copy. Implementation continues on Generated by Claude Code |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 17b5238803
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| | Path that cannot be keyed (traversal segments, control characters) | Skip the entry, note it, continue — the key normalizer throws and that must not reach the run | | ||
| | Same file as `/workspace/repo/src/a.ts` and `src/a.ts` | One entry. Canonicalize against the run's working directory before keying; the key normalizer alone only strips leading slashes | |
There was a problem hiding this comment.
Preserve goal files outside the run working directory
The required goal writes to an absolute OS-temp path (goals/harness-workstream/mirrors-a-coding-run-as-workstream-items/run.mts:135-140,275-279), while claudeCodeAgent exposes no cwd, so canonicalizing that path against the process working directory produces ../.... The adjacent rule then skips it because normalizeResourcePath rejects traversal, leaving the mandatory file list empty after a successful run. Encode outside-root absolute paths under a safe namespace or run the goal inside the canonical root.
Useful? React with 👍 / 👎.
| and each row's payload is on **`clientData`**, not `state`: | ||
|
|
||
| ``` | ||
| GET /sessions/<workstreamId>/resources/observed-file-ops |
There was a problem hiding this comment.
Scope and paginate readback to the requested run
Fresh evidence beyond the earlier reused-workstream finding is that rows are now namespaced, but this revised URL still requests an unfiltered first page. handleListCollectionState sorts and cursor-paginates collection rows, so after enough reused-workstream runs this response can mix prior runs or omit the requested run entirely. Add the run's full storage-key namespace as topicPrefix and require callers or the goal check to follow nextCursor; apply the same change to the plan URL.
Useful? React with 👍 / 👎.
| rows: a second run would merge its file entries by path with the first run's and mix its plan | ||
| items in, and the route would answer "what has this workstream ever done" instead of "what did | ||
| this run do". **Key every entry under the run's own identity** (the request id is the natural | ||
| handle) rather than resetting, so a workstream's history stays inspectable per run. |
There was a problem hiding this comment.
Make retained per-run collections lazy
When a workstream is reused, this rule intentionally retains a new namespace for every request, but the collection design never specifies prefetchMode: "lazy". Resource collections default to eager loading, so each later coding run bulk-loads and caches every historical file and plan row before updating only its own keys; execution cost therefore grows with the workstream's lifetime. Declare both collections lazy and test that a run does not issue a full-prefix load.
Useful? React with 👍 / 👎.
| | A `TaskUpdate` whose result is an error | Recorded as a failed attempt, **never as applied** — writing the requested status for a transition the harness refused is the worst available outcome | | ||
| | A to-do tool we do not recognise | Record nothing. The run is unaffected; §10 grades the empty result | | ||
| | `TaskCreate` with no recoverable id | Record nothing for that item; later updates naming it also record nothing | | ||
| | A file mutation that failed | Recorded with `ok: false` — absence would be indistinguishable from "never attempted" | |
There was a problem hiding this comment.
Use the three-valued outcome contract consistently
Fresh evidence since the earlier pending-outcome comment is that the revised design now explicitly defines outcome as pending | applied | failed, but this normative edge-case row still requires ok: false and the pseudocode at line 366 still writes ok. An implementer following these later instructions would recreate the pending-versus-failed ambiguity or produce schemas and tests using incompatible fields; replace both remnants with outcome: "failed".
Useful? React with 👍 / 👎.


The problem
When we hand a coding job to an agent we can now watch it talk, but we cannot see what it did. The two things that actually describe a run — which files it changed, and what it decided its job was — are discarded as they happen and survive only inside the harness's own transcript, which nothing but a person can read.
So "did this run touch something it shouldn't have?" and "did it quietly decide to do four things when we asked for one?" are questions only a human reading a transcript can answer today.
Why now. The mirror shipped, so a run's messages already land in our system. Everything above this layer — grading a run, filing what came back, deciding what happens next — reads state this layer produces. Built in the other order they read nothing.
What this does
Two things a run produces get recorded as ordinary state, beside the messages we already keep.
The file operations we saw it perform — every time it uses its file-writing or file-editing tool, an entry keyed by the file's path: how it was touched, when, and how it turned out. Not the contents, and not everything that changed — work done through the shell performs no tool call and never appears (see decision 2).
What it thought its job was — the run's own to-do list becomes readable progress, one row per item with its wording and how its status moved. These rows go in their own place, not onto the board that gave the run its work: we can see the agent decided to do five things and compare that with the one thing we asked for, without anything starting to execute those five.
It deliberately does not cover how a run gets settled when it finishes, fails, or is lost. That is FIX-1182, moved out of LAB-133 after seven distinct defects across seven surfaces; this takes the board's defaults and references it rather than working around it.
Nothing new is added to the framework. Both records are instances of a primitive that already exists, read over a route that already ships.
What's asked of you
Three calls, all numbered in the spec as §6 Decisions. Unchanged across both review rounds. Engineering calls are collapsed below.
A run's to-do list is recorded where nothing will execute it
We record the file operations we observed, not everything that changed — and never the contents
sed -i, a redirect,mv, a formatter,git apply) produces no tool call and never appears. Reconciling against the filesystem to close that gap is a different feature, deliberately out of scope.We keep where each to-do item is now and where it was, not how it got there
Parts worth reviewing closely
1. Decision 1 — where the observed to-dos live (§6, §7). The question: is a separate list the right call, or should the run's plan land on the board as real rows so a later step has less to do? Getting it wrong in one direction lets an agent enqueue work nobody approved; in the other it makes fork detection cost a manual step. Everything in §7 hangs off this.
2. §7 — the layer the tool-name table sits at. The tool names live in the pure translation layer, so the recorder underneath is vendor-free and unit-testable with scripted events. The cost is that the pure layer now carries correlation state it did not before. A wrong answer here is a rewrite, not an adjustment.
Where I'm unsure: nothing on direction — two rounds and it was never challenged. One thing is genuinely unmeasured and the spec says so rather than assuming: whether the harness's planning tools carry a typed id field or only prose. It changes implementation cost, not direction; the fallback is built either way.
Deliberately absent: settlement (FIX-1182), file contents and diffs (decision 2), and grading whether the run did a good job (that is LAB-135).
Links
Spec doc:
spec/LAB-134.md· Linear: LAB-134 · Epic: LAB-68 · Depends on: PR #1327 (LAB-133) — this branch is cut fromfix/LAB-133, notmain, and the PR targets it.This PR never merges. It closes unmerged once the direction is approved; Linear keeps the durable copy.
What the two review rounds changed (direction held; one deliverable came back)
Neither round challenged the approach. Both found claims that were wider than their evidence — the same failure twice, which is why scope every claim to exactly what was measured is now the first focus practice.
Round 1. The POC concluded "tool results are prose" having measured
tool_result.contentand never the siblingtool_use_result. Re-ran it: the full structured Output is on the wire for file tools. Also folded: a 403 blocker on client-invisible collections, the headline claim narrowed to observed file-tool operations, and per-run record namespacing.Round 2 — a reversal, and the work got bigger. I had concluded from
blocks/handler.tsthat a capability forwarding the option inherits the block's resource declarations, and collapsed a build step on it. A reviewer disagreed; rather than adjudicate between two assertions I read the three places that decide it:mergeSurfaceIntomergessurface.resourcesand never readssurface.toolsfor resourcescollectBlockResourcesgathersdeclaredResourcesfrom action blocks, and a tool block is not onedefineFlow's own comment: "A generator is a leaf that bubbles none of its tools' rails by design."The reviewer is right. A capability contributing only a resource-declaring block in its
toolspreset contributes no declarations at all — the flow registers nothing and the route 404s at read time on a build that succeeded. The capability must declare the collections itself; that is a restored sixth step with its own test, and size goes from ~5 production files / ~12 behaviours to ~6 / ~14 rather than being absorbed.The error's shape is worth naming:
blocks/handler.ts's sentence is true — it covers resources the capability declares itself, not resources declared by a block inside itstools. A true sentence carried past its scope, exactly like round 1's.POC on this branch — what a real run actually carries
Throwaway, never merges. Four real runs on
claude2.1.234 — the CLI version@anthropic-ai/claude-agent-sdk@0.3.234pins in its manifest, so this is the binary the SDK spawns.Write/Editarrive as ordinarytool_useblocks withfile_pathon the tool input. The design did not change — reporting this is the point, so a quiet POC isn't read as one that found nothing.tool_result.contentis prose;tool_use_result, a sibling on the same message, carries the full declared Output. An earlier version measured only the first and concluded about both. Not measured for the planning tools; marked unmeasured rather than assumed.TodoWrite— that tool is not offered at all. The issue names it; the shipped binary exposesTaskCreate/TaskUpdate, and its own.d.tsstill declares the old one. Opposite data models, so code written for either is silently inert against the other.#1/#2; run two#3/#4for the same list. The id can only be recovered from each create's own result.No captured-output file is checked in: an earlier one drifted against the probe and gave a false signal to anyone diffing the two. stdout is the artifact. Per the contract below, please don't review these files as code.
Engineering calls
capability.tsalready documents forsessionStateSchema; both directions of that seam now carry a note.clientData, notstate.handleListCollectionStatebuilds each row as{ topic, storageKey, clientData: applyClientData(...) }. Readingstateyieldsundefinedafter a valid 200 — the same shape as theseq/itemIndexclass of bug. And becauseclientDatais a projection, the fields the goal asserts on must be in the exposed set, so §10 tests the read rather than assuming it.falsewould misreport an in-flight write as failed while waiting for settlement would lose the killed-run attempts this exists to capture. The same field lets a rejected plan update land asfailedwithout moving statuses the harness never changed.defineResourceCollectionkeeps a collection invisible by default and the list route returns 403 unlessclient.state.read === true(verified inresource-routes.ts).tool_call/tool_resultwould have to match tool names itself, putting the vendor names in two places. The cost is two switch arms that emit nothing — andemit.tsalready carries exactly that pattern for itsresultkind.How to review this
This is a direction document, not an implementation. Reviewing it well means answering one question: is this the right approach?
In scope to challenge:
docs/philosophy.md?Out of scope — deliberately unsettled here, and owned by the implementing agent:
spec-poc). None of it ships. Read it to judge whether the shape holds; do not review it as code. Its quality is not a finding, and its findings are summarized in §7 — the summary is what a reviewer reacts to.One request, if you are an automated reviewer: the two lists above are the whole difference between a useful review of this document and a long one. Volume is not signal here — a single finding above the line is worth more than twenty below it.
Feedback in the second list is welcome and gets recorded verbatim in §13 for the implementer to weigh against real code. It will not be argued with, and it will not be folded into the design prose — that would pretend the spec can settle something it can't. Please don't re-raise it: one mention is enough for it to land in §13.
The one exception is the sketch and the POC. Line-level feedback on either is dropped, not recorded — there is nothing for an implementer to weigh about code that isn't shipping, and a §13 note would imply otherwise. Feedback about their direction is not line-level and is treated like any other above-the-bar finding.
The rest of the case — tradeoffs, focus practices
Alternative weighed: return it on the run's result instead of writing state. Rejected — a background job's result object goes nowhere, since conversation state is deliberately off for detached runs. The proof this issue owes is readable afterwards, from state, and a return value is not that.
The simpler approach: record only the files, drop the to-do half. Genuinely tempting; the file half has the stable surface and most of the immediate value. Insufficient because the to-do half is the only part that tests the epic's actual claim — files say what happened, the to-do list is the only place the agent says what it thought it was doing. Dropping it defers the risk rather than retiring it.
Where the complexity is: not the recording. It is that the harness's planning surface is unstable, optional, and partly undocumented — the tool the issue names does not exist on the version we run, and a run may decline to plan at all even when asked. See the POC block.
Focus practices: scope every claim to exactly what was measured · every assertion proved able to fail (break it, confirm red, revert) · BP-030 tolerate shapes we have never seen · BP-006 no tracking labels in code or tests · one convergence point for the vendor mapping.
Part II — technical design, sequence, edge cases, testing strategy, docs plan — is in the committed doc diff.