Skip to content

spec(LAB-134): Graph a coding agent's file writes and todos into FSD state - #1329

Closed
jhoffner wants to merge 7 commits into
fix/LAB-133from
spec/LAB-134
Closed

jhoffner wants to merge 7 commits into
fix/LAB-133from
spec/LAB-134

Conversation

@jhoffner

@jhoffner jhoffner commented Aug 18, 2026 •

Copy link
Copy Markdown
Collaborator

The problem

When we hand a coding job to an agent we can now watch it talk, but we cannot see what it did. The two things that actually describe a run — which files it changed, and what it decided its job was — are discarded as they happen and survive only inside the harness's own transcript, which nothing but a person can read.

So "did this run touch something it shouldn't have?" and "did it quietly decide to do four things when we asked for one?" are questions only a human reading a transcript can answer today.

Why now. The mirror shipped, so a run's messages already land in our system. Everything above this layer — grading a run, filing what came back, deciding what happens next — reads state this layer produces. Built in the other order they read nothing.

What this does

Two things a run produces get recorded as ordinary state, beside the messages we already keep.

The file operations we saw it perform — every time it uses its file-writing or file-editing tool, an entry keyed by the file's path: how it was touched, when, and how it turned out. Not the contents, and not everything that changed — work done through the shell performs no tool call and never appears (see decision 2).

What it thought its job was — the run's own to-do list becomes readable progress, one row per item with its wording and how its status moved. These rows go in their own place, not onto the board that gave the run its work: we can see the agent decided to do five things and compare that with the one thing we asked for, without anything starting to execute those five.

It deliberately does not cover how a run gets settled when it finishes, fails, or is lost. That is FIX-1182, moved out of LAB-133 after seven distinct defects across seven surfaces; this takes the board's defaults and references it rather than working around it.

flowchart LR
  SDK[harness message stream] --> T["translation layer<br/>the only place tool names live"]
  T -->|existing events| E[item emission]
  T -->|file op observed| G[work recorder]
  T -->|plan item observed| G
  G -->|coalesced flush| RC["two resource collections<br/>client state reads declared"]
  RC --> R["the resource route<br/>that already ships"]
  E --> S[item stream]
Loading

Nothing new is added to the framework. Both records are instances of a primitive that already exists, read over a route that already ships.

What's asked of you

Three calls, all numbered in the spec as §6 Decisions. Unchanged across both review rounds. Engineering calls are collapsed below.

A run's to-do list is recorded where nothing will execute it

  • The agent's observed plan lands in its own list, separate from the work queue that dispatched the run. Adopting an item onto the real queue is a deliberate, separate act.
  • If this is wrong: either an agent that decides mid-run to do five more things can start five more jobs by writing a to-do list, or — the other way — noticing a fork costs a manual step every single time and people stop doing it.

We record the file operations we observed, not everything that changed — and never the contents

  • Paths, how each was touched, when, how it turned out, for the agent's file tools only. A file changed through the shell (sed -i, a redirect, mv, a formatter, git apply) produces no tool call and never appears. Reconciling against the filesystem to close that gap is a different feature, deliberately out of scope.
  • If this is wrong: the record gets described as complete when it isn't, and someone later trusts "this run changed nothing" about a run that reformatted the tree through its shell. The other half: we either keep a second copy of the customer's source in our store, or we lack the detail to answer "what actually changed in this file" without going to the repo.

We keep where each to-do item is now and where it was, not how it got there

  • A plan row carries its wording, its current status and its previous one. No history log — and the goal check grades the plan half at that same depth rather than on a sequence of transitions.
  • If this is wrong: we can say "it planned this, and that item moved" but never "here is the order it changed its mind in" — so any later question about how an agent's plan evolved needs new work. Cheap to reverse: adding history is additive and nothing depends on its absence.

Parts worth reviewing closely

1. Decision 1 — where the observed to-dos live (§6, §7). The question: is a separate list the right call, or should the run's plan land on the board as real rows so a later step has less to do? Getting it wrong in one direction lets an agent enqueue work nobody approved; in the other it makes fork detection cost a manual step. Everything in §7 hangs off this.

2. §7 — the layer the tool-name table sits at. The tool names live in the pure translation layer, so the recorder underneath is vendor-free and unit-testable with scripted events. The cost is that the pure layer now carries correlation state it did not before. A wrong answer here is a rewrite, not an adjustment.

Where I'm unsure: nothing on direction — two rounds and it was never challenged. One thing is genuinely unmeasured and the spec says so rather than assuming: whether the harness's planning tools carry a typed id field or only prose. It changes implementation cost, not direction; the fallback is built either way.

Deliberately absent: settlement (FIX-1182), file contents and diffs (decision 2), and grading whether the run did a good job (that is LAB-135).

Links

Spec doc: spec/LAB-134.md · Linear: LAB-134 · Epic: LAB-68 · Depends on: PR #1327 (LAB-133) — this branch is cut from fix/LAB-133, not main, and the PR targets it.

This PR never merges. It closes unmerged once the direction is approved; Linear keeps the durable copy.

What the two review rounds changed (direction held; one deliverable came back)

Neither round challenged the approach. Both found claims that were wider than their evidence — the same failure twice, which is why scope every claim to exactly what was measured is now the first focus practice.

Round 1. The POC concluded "tool results are prose" having measured tool_result.content and never the sibling tool_use_result. Re-ran it: the full structured Output is on the wire for file tools. Also folded: a 403 blocker on client-invisible collections, the headline claim narrowed to observed file-tool operations, and per-run record namespacing.

Round 2 — a reversal, and the work got bigger. I had concluded from blocks/handler.ts that a capability forwarding the option inherits the block's resource declarations, and collapsed a build step on it. A reviewer disagreed; rather than adjudicate between two assertions I read the three places that decide it:

  • mergeSurfaceInto merges surface.resources and never reads surface.tools for resources
  • collectBlockResources gathers declaredResources from action blocks, and a tool block is not one
  • defineFlow's own comment: "A generator is a leaf that bubbles none of its tools' rails by design."

The reviewer is right. A capability contributing only a resource-declaring block in its tools preset contributes no declarations at all — the flow registers nothing and the route 404s at read time on a build that succeeded. The capability must declare the collections itself; that is a restored sixth step with its own test, and size goes from ~5 production files / ~12 behaviours to ~6 / ~14 rather than being absorbed.

The error's shape is worth naming: blocks/handler.ts's sentence is true — it covers resources the capability declares itself, not resources declared by a block inside its tools. A true sentence carried past its scope, exactly like round 1's.

POC on this branch — what a real run actually carries
node spec-poc/LAB-134-harness-graph-surface/probe.mjs

Throwaway, never merges. Four real runs on claude 2.1.234 — the CLI version @anthropic-ai/claude-agent-sdk@0.3.234 pins in its manifest, so this is the binary the SDK spawns.

  1. File writes: premise holds. Write / Edit arrive as ordinary tool_use blocks with file_path on the tool input. The design did not change — reporting this is the point, so a quiet POC isn't read as one that found nothing.
  2. The prose result and the structured result are two different fields. tool_result.content is prose; tool_use_result, a sibling on the same message, carries the full declared Output. An earlier version measured only the first and concluded about both. Not measured for the planning tools; marked unmeasured rather than assumed.
  3. The to-do surface is not TodoWrite — that tool is not offered at all. The issue names it; the shipped binary exposes TaskCreate / TaskUpdate, and its own .d.ts still declares the old one. Opposite data models, so code written for either is silently inert against the other.
  4. A run may keep no to-do list at all. Two of four runs created none — including one whose job explicitly requested a two-item plan with dependent steps (write a file, then edit that same file). Shaping the job harder is not a remedy; it was tried and it failed. This is why the goal check reports inconclusive for that case rather than passing or failing it.
  5. To-do ids are not positional. Run one allocated #1/#2; run two #3/#4 for the same list. The id can only be recovered from each create's own result.

No captured-output file is checked in: an earlier one drifted against the probe and gave a false signal to anyone diffing the two. stdout is the artifact. Per the contract below, please don't review these files as code.

Engineering calls
  • The capability must declare the collections itself — see the review-rounds block above. Forwarding the option is not sufficient and the failure is a 404 at read time, not a build error, so it needs a test that fails without the declaration. This is the mirror of the asymmetry capability.ts already documents for sessionStateSchema; both directions of that seam now carry a note.
  • Rows come back on clientData, not state. handleListCollectionState builds each row as { topic, storageKey, clientData: applyClientData(...) }. Reading state yields undefined after a valid 200 — the same shape as the seq/itemIndex class of bug. And because clientData is a projection, the fields the goal asserts on must be in the exposed set, so §10 tests the read rather than assuming it.
  • Outcome is three-valued, not a boolean. Recording a mutation when the call is seen makes "pending" a real state, so false would misreport an in-flight write as failed while waiting for settlement would lose the killed-run attempts this exists to capture. The same field lets a rejected plan update land as failed without moving statuses the harness never changed.
  • Both collections must declare client state reads. defineResourceCollection keeps a collection invisible by default and the list route returns 403 unless client.state.read === true (verified in resource-routes.ts).
  • Watching the work must never break the work. Anything the recorder cannot handle is skipped and left visible as a gap, never thrown — a run that has already written files and pushed commits must not be killed by our bookkeeping. It is a §9 rule rather than a §6 Decision because nobody would overrule it, but it is why an empty file record grades as a failure.
  • Records are namespaced per run. The board reuses a workstream session for the same board/worker/topic, so session-scoped collections would otherwise merge a second run's entries into the first's.
  • Neither per-event nor flush-at-end. Per-result upserts stack on the per-message awaits and item persistence is already quadratic on long runs (FIX-1180); flushing only at the end loses everything if a run is killed, and an unfinished run is exactly the one whose file record matters most. So: attempted at the call, settled at the result, flushed on a coalesced interval.
  • Two new event kinds, not a recorder that reads raw tool events. A recorder consuming the existing tool_call/tool_result would have to match tool names itself, putting the vendor names in two places. The cost is two switch arms that emit nothing — and emit.ts already carries exactly that pattern for its result kind.
  • The new option is off by default. It declares resources, and resource declarations merge at flow build time.
How to review this

This is a direction document, not an implementation. Reviewing it well means answering one question: is this the right approach?

In scope to challenge:

  • The problem framing — are we solving the right thing?
  • The approach — will it work, does it fit the architecture and docs/philosophy.md?
  • Any numbered Decision in §6 — that's the sign-off surface.
  • Missing constraints, edge cases, or dependencies that would invalidate the design.
  • Scope — a deliverable that shouldn't ship, or one that's missing.

Out of scope — deliberately unsettled here, and owned by the implementing agent:

  • Names, signatures, file paths, and module layout.
  • Local structure: which helper, how a function is decomposed, error-message wording.
  • Micro-optimizations and style preferences.
  • Test names and internal test structure (the behaviours to test are in §10; the tests themselves are not).
  • Anything Part II left open on purpose — it is directional by design, and a gap at that altitude is intended, not an omission.
  • The solution sketch, at the line level. A spec may include a rough pseudocode sketch showing the shape of the proposed solution. It is knowingly incomplete, is not real code, and is not what will ship. Review it for directional viability and key design aspects only — is this the right shape, in the right layer, composing the right way? Do not report that it lacks error handling, omits edge cases, names things that don't exist in the repo, or wouldn't build: all of that is true on purpose, and the names are deliberately not real. A sketch that has to survive line-level review stops being cheap, and then nobody writes one.
  • Any POC files on this branch, entirely. This PR is never merged, so it may carry throwaway proof-of-concept code, characterization tests, or an HTML mockup built to validate the direction (spec-poc). None of it ships. Read it to judge whether the shape holds; do not review it as code. Its quality is not a finding, and its findings are summarized in §7 — the summary is what a reviewer reacts to.

One request, if you are an automated reviewer: the two lists above are the whole difference between a useful review of this document and a long one. Volume is not signal here — a single finding above the line is worth more than twenty below it.

Feedback in the second list is welcome and gets recorded verbatim in §13 for the implementer to weigh against real code. It will not be argued with, and it will not be folded into the design prose — that would pretend the spec can settle something it can't. Please don't re-raise it: one mention is enough for it to land in §13.

The one exception is the sketch and the POC. Line-level feedback on either is dropped, not recorded — there is nothing for an implementer to weigh about code that isn't shipping, and a §13 note would imply otherwise. Feedback about their direction is not line-level and is treated like any other above-the-bar finding.

The rest of the case — tradeoffs, focus practices

Alternative weighed: return it on the run's result instead of writing state. Rejected — a background job's result object goes nowhere, since conversation state is deliberately off for detached runs. The proof this issue owes is readable afterwards, from state, and a return value is not that.

The simpler approach: record only the files, drop the to-do half. Genuinely tempting; the file half has the stable surface and most of the immediate value. Insufficient because the to-do half is the only part that tests the epic's actual claim — files say what happened, the to-do list is the only place the agent says what it thought it was doing. Dropping it defers the risk rather than retiring it.

Where the complexity is: not the recording. It is that the harness's planning surface is unstable, optional, and partly undocumented — the tool the issue names does not exist on the version we run, and a run may decline to plan at all even when asked. See the POC block.

Focus practices: scope every claim to exactly what was measured · every assertion proved able to fail (break it, confirm red, revert) · BP-030 tolerate shapes we have never seen · BP-006 no tracking labels in code or tests · one convergence point for the vendor mapping.

Part II — technical design, sequence, edge cases, testing strategy, docs plan — is in the committed doc diff.

claude added 4 commits August 18, 2026 00:59
Dispatch a Claude Code run into a workstream — a child session dedicated to
that one job — and mirror its top-level messages as items in that workstream's
stream, so the originating request returns immediately and nobody has to open
the harness transcript to find out what a run did.

Production change is one option in one package. `claudeCodeAgent` and
`createClaudeCodeAgentCapability` take `sessionState: false`, which suppresses
three things together: the `sessionStateSchema` declaration, the session reads
and writes that go with it, and the SDK `resume`. The third is the one that
hides — suppressing the schema alone leaves provider resolution intact, so a
custom `sessionProvider` returning a saved id for an empty key would silently
resume a prior conversation while every schema-shaped assertion stayed green.
The test for it asserts on the options handed to the SDK.

Both halves are needed. A capability contributes its schema through a channel
the task board's block walk cannot see, so a capability still declaring it
would put the key back through the one door the refusal cannot check — and it
would appear to work.

Default is `true`; an unset caller is unaffected in every respect (BP-030),
including that a mid-stream throw still throws.

Settlement — how a finished, failed or lost run is recorded — is deliberately
NOT here. It was revised seven times across seven different surfaces and none
of them touched the mirror, so it moved to FIX-1182. This takes the board's
default: no retry allowance, so `shouldRetryOnFail` is false and no agent ever
re-runs over its own commits (the one genuine harm in this area). The cost is
stated rather than hidden, in the spec and in the goal check: a LOST run is
written off rather than staying recoverable.

Verification:
- goal check `goals/harness-workstream/mirrors-a-coding-run-as-workstream-items/`
  — a real Claude Code run dispatched into a real workstream through the real
  board, read back over the real HTTP routes on store-sqlite. Two consecutive
  PASSes. It names its adapter deliberately: the in-memory store ignores
  `include_items` by documented contract, so a readback assertion there proves
  nothing about the route. The check asserts 0 items WITHOUT the flag, so the
  flag is load-bearing rather than decorative.
- 6 CI behaviours: 4 in `packages/claude-code/test/sdk/agent.spec.ts`, 1 in its
  capability spec, and board acceptance in `packages/integration-tests`.
  Acceptance lives there rather than in the vendor package (which would take an
  orchestration dev dep for one test) or only in the goal (which needs a real
  model and the optional SDK peer, so it does not run in CI).
- `pnpm typecheck` across all 61 projects.

Two manifest fixes the build needed, neither optional:
- `goals/` had no dependency on `@flow-state-dev/claude-code`; there is no way
  to dispatch a coding run from a goal without it.
- `packages/claude-code`'s optional peer is `"*"`, and the lockfile had pinned
  it to 0.1.77 while `goals` resolved 0.3.234 — two copies in one workspace,
  with which one you got depending on the importing file's location. The old
  copy's bundled CLI exits 1, surfacing as a bare "process exited with code 1"
  that reads like a broken dispatch. A root `pnpm.overrides` entry gives the
  workspace one copy. The published peer range is untouched.
Throwaway, on the spec branch, never merges. Settles the premise the spec
rests on by running a real coding job rather than reading type declarations.

Findings: file writes do arrive as Write/Edit tool_use blocks with file_path
on the input; tool results are prose, not the declared structured output; and
the todo surface is TaskCreate/TaskUpdate, not TodoWrite, which the shipped
CLI does not offer at all.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MVcrTezeLMBQRVHRg2QX2f
…state

Records what a coding run touched and what it thought its job was, as ordinary
resource collections read over the route that already ships. The observed todo
list lands where nothing will execute it, so seeing a fork stays separate from
acting on one.

Branched from fix/LAB-133, not main — nothing in this epic merges yet.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MVcrTezeLMBQRVHRg2QX2f
@linear-code

linear-code Bot commented Aug 18, 2026

Copy link
Copy Markdown

LAB-134

@vercel

vercel Bot commented Aug 18, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

3 Skipped Deployments
Project Deployment Actions Updated (UTC)
flow-state-dev-docs Ignored Ignored Aug 18, 2026 1:44am
flow-state-dev-kitchen-sink Ignored Ignored Aug 18, 2026 1:44am
flow-state-dev-ui Ignored Ignored Preview Aug 18, 2026 1:44am

@jhoffner jhoffner added the spec label Aug 18, 2026 — with Claude

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Simplify review — spec(LAB-134)

Verdict: COMMENT (direction sound; a few simplification forks worth settling before implementation locks in)

~838 LOC across the spec + throwaway POC. This is a well-scoped spec PR: the POC did real work (TodoWrite → TaskCreate/TaskUpdate, non-positional ids), and the proposed shape — two TranslatedEvent kinds, a vendor-free recorder, two session-scoped resource collections, recordWork opt-in — fits the existing translate.ts / emit.ts split and tenet 2 better than inventing new framework machinery.

What looks right

  • Resource collections over tasks for the plan half (decision 1) is the clean layering call — avoids fake execution envelopes and keeps @flow-state-dev/claude-code off orchestration.
  • Translation-layer tool table + framework-vocabulary recorder mirrors how tool items already correlate via open-call state; the spec's own reviewer question (§7 sketch) lands on the right answer.
  • POC placement under spec-poc/ per repo convention; probe is small and rerunnable.
  • §10 anti-game (empty list = FAIL, board row count, no transcript/git reads) directly addresses the epic's false-green history.
  • Rejected alternatives (return-value-only, files-only) are argued honestly; I would not push a files-only v1 unless schedule forces it.

Where complexity may be higher than necessary

  1. tool_use_result not measured (biggest fork). The POC and settled claims focus on tool_result block content being prose. The SDK's SDKUserMessage also carries tool_use_result — structured per-tool output (TaskCreateOutput.task.id, TaskUpdateOutput.taskId/statusChange, FileWriteOutput.type/filePath are all declared in sdk-tools.d.ts). probe.mjs never records that field, and SdkMessageLike today omits it. Before locking prose id-recovery and "authoritative record is tool input" in §7/§12, extend the POC one line: log m.tool_use_result on each user message. If structured output is present on the wire, the translation layer can emit plan_item/file_touched without regex — and the settled-claims table needs a third row, not a blanket REFUTED on structured results.

  2. POC artifact drift. observed-run.json schema (toolUse, todoWriteAvailable) does not match what probe.mjs currently prints (fileMutations, planCalls, todoWriteOffered). A reviewer who diffs probe output against the checked-in capture will get a false signal. Either regenerate the JSON from the current probe or drop the file and point readers at probe stdout only.

  3. "Open questions: none" feels early while (1) is unsettled. I'd rename to one explicit open: structured result availability on tool_use_result vs prose-only tool_result.content.

  4. Minor nits: §7 POC link still says PR_NUMBER; probe tracks NotebookEdit/MultiEdit but the spec table only names Write/Edit — either scope the v1 table or say why notebook/multi are deferred.

Alternatives considered (and why I'd still stay on this spec's track)

Alternative Why not (for now)
Post-hoc projection from existing tool_output items Loses mid-run readback; duplicates item stream semantics; resource route is the stated proof surface
Third-party file watcher / diff lib Data is already on the tool stream; adding FS watchers would be strictly more complex
Files-only v1, plan later Spec argues correctly that the plan half is the epic's kill-line test; only worth it under schedule pressure
Recorder owns tool-name table Breaks vendor-free recorder + scripted vitest seam the spec depends on

Performance

No hot-path concerns for a spec: per-tool-call upserts into bounded collections are fine; FIX-1180 note is sufficient. If tool_use_result removes prose parsing, that's a small win too.

Recommendation

Approve directionally after a short POC pass on tool_use_result and artifact sync. No need to re-architect — likely a spec §7/§12 wording change and simpler translation pseudocode, not a new subsystem.

Open in Web View Automation 

Sent by Cursor Automation: Review For Best Approach

Comment thread spec/LAB-134.md Outdated

- A file mutation's path is on the **tool input** (`file_path`). The result is a prose
sentence, so it supplies only success or failure.
- A to-do creation's wording is on the tool input; its **id is only in the result's prose**,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Before implementation commits to prose id recovery, extend the POC to log tool_use_result on each user message. The SDK types declare structured TaskCreateOutput (task.id) and TaskUpdateOutput (taskId, statusChange) separately from the prose string in tool_result.content. If the wire carries tool_use_result (likely — it's on SDKUserMessage in sdk.d.ts), translation can emit plan_item from typed fields and drop the fragile regex path. Worth settling here because it is the plan half's main complexity driver.

Comment thread spec/LAB-134.md Outdated
- **Settled:** file mutations arrive as `Write` / `Edit` tool calls carrying `file_path` on the
input, at the seam the translation layer already sees — **CONFIRMED.** Measured twice. The
premise held; the design did not change.
- **Settled:** a tool result carries the structured output its type declares — **REFUTED.**

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This settled claim may be too broad: you refuted structured output on tool_result.content, but the POC did not measure message-level tool_use_result. Suggest splitting into two rows — content block prose (REFUTED) vs tool_use_result structured (UNMEASURED / TBD) — so implementers do not treat prose parsing as the only path.

Comment thread spec/LAB-134.md Outdated
this epic merges until the user says so. LAB-133 supplies the workstream a detached run lands
in, the item stream this sits beside, and the goal-check shape §10 follows.

**Open questions:** none.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Given the tool_use_result gap above, I'd keep one explicit open question here rather than "none". Otherwise §12 and §7 read as closed while the highest-risk parsing decision is still unverified.

Comment thread spec/LAB-134.md Outdated
> vendor-free and unit-testable with scripted events — but it also means the pure layer now
> carries a little correlation state it did not before.

**POC — [`spec-poc/LAB-134-harness-graph-surface/`](https://github.com/fixpoint-labs/flow-state-dev/pull/1329/files)**, on this branch, never merges.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Placeholder PR_NUMBER — swap for #1329 (or drop the URL and keep the relative path only) so the spec link works on the branch.

cliVersion = m.claude_code_version ?? null;
availableTools = m.tools ?? [];
}
if (m.type === "user") {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The probe only correlates tool_result block content. Consider also capturing m.tool_use_result on user messages (and printing whether it is structured for TaskCreate / Write). That one addition could simplify the whole §7 correlation story — or confirm prose parsing is truly required.

"result": "Updated task #2 status"
}
],
"toolResults": [],

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Schema drift: this file uses todoWriteAvailable / toolUse, but the current probe.mjs emits todoWriteOffered / fileMutations / planCalls. Regenerate from the probe or remove the static capture so reviewers are not comparing against stale shape.

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Supplemental simplify note — hot-path persistence (see inline).

Open in Web View Automation 

Sent by Cursor Automation: Review For Best Approach

Comment thread spec/LAB-134.md
> plan update → emit plan_item { id, status } ← id was on the input
> id not recoverable → emit nothing (decision 3)
>
> in the work recorder (framework vocabulary only, no tool names):

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perf follow-up (simplify review): §7's per-tool-result collection upserts inside the SDK message loop will stack read–modify–write + persist on top of existing per-message emitTranslatedEvent awaits (and FIX-1180 item persistence). For a busy coding run that's real wall-clock regression.

§10 only needs readback after a finished run — consider specifying the recorder as an in-memory accumulator with a single flush at run end (or debounced per-key coalescing). Same observable state for the goal check, far less hot-path I/O. If mid-run HTTP freshness is required, say so explicitly; otherwise flush-at-end is the simpler default.

Secondary: reuse or extend existing openTools correlation in emit.ts rather than parallel call-id maps in translate, to avoid drift.

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Final simplify supplement — state-shape trim + implementation reuse (see inline).

Open in Web View Automation 

Sent by Cursor Automation: Review For Best Approach

Comment thread spec/LAB-134.md Outdated
**Goal:** after a real coding run dispatched into a workstream, the files it changed and the
plan it kept are both readable from FSD state alone — no transcript, no working tree, no git.

`goals/harness-workstream/graphs-a-runs-files-and-plan-into-state/run.mts`, real model, real

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Implementation shortcut from reuse review: clone goals/harness-workstream/mirrors-a-coding-run-as-workstream-items/run.mts (SQLite + HTTP + taskGoalToPrompt + sessionState: false) and add resource-route readback — don't stand up a second goal harness. Reuse normalizeResourcePath from @flow-state-dev/core/types for file keys; follow parse-output.ts style for any prose id recovery.

Comment thread spec/LAB-134.md Outdated
> file_touched → upsert the file list, keyed by the normalized path,
> bumping the counter for its kind
> plan_item → upsert the plan list, keyed by the id,
> appending the status to a bounded transition log

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Consider trimming v1 state shapes: goal check only needs files present + touch kinds + plan items with status that moved once. A bounded transitions[] log and per-file writes/edits counters add merge rules and tests beyond §10's proof — status + previousStatus + lastTouchedAt / lastKind may be enough for this slice.

Comment thread spec/LAB-134.md Outdated
> anything that throws → swallow, surface as a status note, keep going (decision 3)
> ```
>
> What this asks a reviewer: *is the translation layer the right home for the tool-name table,

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Close the open reviewer question here — translation owns tool names + correlation, recorder stays vendor-free. Leaving it open invites two implementations.

Comment thread spec/LAB-134.md Outdated
propagate (decision 3). *Depends on:* 1.
3. **The block option and its two collection declarations**, off by default. *Test:* off is
byte-identical to today, including declaring no resources; on declares both. *Depends on:* 2.
4. **Forward the option through the capability.** *Test:* a capability built with it declares

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Worth confirming whether step 4 is a real seam: createClaudeCodeAgentCapability already forwards options to claudeCodeAgent, and resources bubble via declaredResources (unlike sessionStateSchema). If no dual-channel bug exists, collapse 3–4 into one step + smoke test.

@claude

claude Bot commented Aug 18, 2026

Copy link
Copy Markdown
Contributor

second-look: spec review (automated, restraint lens)

Per this PR's own review contract, one finding above the line:

Finding — the plan record's transition log goes further than "how its status moved" requires.

§7's sketch and §5's read-back example give each observed plan item a transitions: [{status, at}, …] array with an eviction "bounded" cap (§9: "Collection at maxInstances… evict per policy" applies at the collection level, but the transition array itself needs its own bound). The PR body's own "Where I'm unsure" note already flags this exact tension: the issue's "todo progression is readable as state" phrasing argues for the log, but it's "the one field that grows with a run's length."

A single previousStatus (or statusChangedAt) field satisfies the same requirement — you can still see a task moved from pending to completed — without an accumulating, capped history array. That also deletes the entire "bounded" sub-problem this field currently creates: no cap policy to choose, no cap edge case in §9, and the §10 assertion ("at least one row's transition log shows a status actually moving") becomes a simpler equality check.

Worth noting: the sibling touched-files resource in the same spec already made this call — it keeps counters (writes/edits/failures), not a full per-touch event log. The transitions array is the one place the spec reaches for growing history; nothing elsewhere in the document does, and no existing resource-collection state schema in the repo carries this shape today (checked packages/core/src/types/resource-collection.ts and friends — no precedent).

  • Lens: YAGNI / 80-20
  • Est. Δlines (forward estimate, on the eventual build): −45 / +10 — drops the array type, the append-with-cap logic, the cap edge case, and shrinks the §5 example; adds one scalar field.
  • Severity: Medium · Confidence: Medium (the author's own spec already names this as the open question, which is most of my confidence)

Recommendation: fold this into §6 Decision 1 or add it as Decision as explicit scope — pick current-status-only (or previousStatus) before the implementing agent builds the array-with-cap version by default.


Considered and deemed justified, not flagged:

  • Two separate resource collections (touched-files keyed by path, observed-plan keyed by id) rather than one merged collection — the key spaces and state shapes are different enough that merging would need a discriminated union, which is more surface, not less.
  • writes/edits/failures as three counters on the touched-files entry — each is required by decision 2's own wording ("how it was touched… whether it worked"), not speculative.
  • Correlating file/plan tool calls via new entries in the translation layer's existing open-call-id state (TranslateState.openTools already does this for tool items) — reuses an established pattern (packages/claude-code/src/sdk/translate.ts) rather than introducing new architecture.
  • No architecture-refactor opportunity flagged elsewhere in the surrounding code: this PR touches only spec/LAB-134.md and the throwaway spec-poc/ directory (per the PR's own instructions, not reviewed as code) — no production files are in this diff to refactor.

Posted by an automated second-look pass (restraint lens) — for the implementing agent to weigh per the PR's §13 convention.


Generated by Claude Code

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 1b2498d53d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread spec/LAB-134.md Outdated
Comment on lines +28 to +31
**What it touched.** Every file the run writes or edits becomes an entry in a list, keyed by
the file's path, recording how it was touched, when, and whether it worked. Not the contents
— just the fact of the change. Afterwards you can ask "what did this run change?" and get an
answer without opening a transcript or a diff.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Restrict or complete the touched-file claim

The spec promises that every file changed by a run appears here, but §7 only translates named file-mutation tools. With the default agent configuration, files changed through Bash—for example redirects, sed -i, rm, mv, formatters, generators, or git apply—produce no Write/Edit event, so downstream consumers can receive an empty or incomplete list after real mutations. Either reconcile against the repository/filesystem or describe this as observed file-tool operations rather than an authoritative changed-file index.

Useful? React with 👍 / 👎.

Comment thread spec/LAB-134.md Outdated
Comment on lines +194 to +197
- **`@flow-state-dev/core`** — untouched. Both collections are ordinary
`defineResourceCollection` values.
- **`@flow-state-dev/engine`** — untouched. `GET /sessions/:id/resources/:ref` already lists a
collection's instances with their state.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Expose collection state before relying on HTTP readback

The proposed readback cannot work with merely an ordinary collection declaration: handleListCollectionState in packages/engine/src/routes/resource-routes.ts returns 403 unless config.client?.state?.read === true, while defineResourceCollection defaults to no client visibility. Require both collections to declare intentional client-readable state and test that permission, otherwise the §5 examples and §10 real-HTTP goal fail.

Useful? React with 👍 / 👎.

Comment thread spec/LAB-134.md Outdated
Comment on lines +219 to +220
**Where the two collections live.** Session-scoped, so a detached run's records land in that
run's own workstream session and are read back there — the same place its items already are.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Separate records for reused workstream runs

When a second task uses the same board, worker, and topic, the task board deliberately reuses the existing workstream session (packages/orchestration/README.md:349-354), so these session-scoped collections retain the first run's rows. The second run will aggregate file counters by path and mix plan items with earlier work, meaning the route reports workstream-lifetime activity rather than answering the promised question about a particular run; namespace records by request/run identity or define and test an explicit latest-run reset.

Useful? React with 👍 / 👎.

Comment thread spec/LAB-134.md Outdated
Comment on lines +208 to +211
**Correlation, and where the data actually is.** Both events are emitted on the tool
**result**, not the call, because the call carries the subject and the result carries the
outcome. The translation layer already does exactly this for tool items via an open-call map;
these follow that pattern.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Persist mutation attempts before awaiting results

Emitting the file record only when a tool result arrives drops a mutation if cancellation, process loss, or an SDK throw occurs after the filesystem operation but before its tool_result is delivered. The file may already have changed while the audit state says it was never touched, which is especially damaging for the failed or lost detached runs this state should help inspect; record an attempted/unknown entry at tool_use and settle it on the result, or reconcile afterward.

Useful? React with 👍 / 👎.

Comment thread spec/LAB-134.md Outdated
Comment on lines +233 to +236
> on that call's result:
> file mutation → emit file_touched { path, kind, ok }
> plan create → recover the id from the result; emit plan_item { id, title, status }
> plan update → emit plan_item { id, status } ← id was on the input

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Preserve failed plan-update outcomes

For a TaskUpdate whose result has is_error: true, this design still emits only the requested { id, status }, so the recorder can append completed even though the harness rejected that transition. The existing translator already exposes isError on tool results; require failed updates to be skipped or represented as failed attempts, and cover that error path in the scripted tests.

Useful? React with 👍 / 👎.

Comment thread spec/LAB-134.md Outdated
| Case | Expected |
|---|---|
| Path that cannot be keyed (traversal segments, control characters) | Skip the entry, note it, continue — the key normalizer throws, and decision 3 says that must not reach the run |
| Relative and absolute forms of one path | Collide into one entry; leading slashes are stripped by the key normalizer. Acceptable; note it |

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Canonicalize paths against the checkout root

The stated collision does not hold for actual absolute and relative forms of the same file: normalizeResourcePath only strips leading slashes, so /workspace/repo/src/a.ts becomes workspace/repo/src/a.ts, while src/a.ts remains distinct. If different harness messages report those two forms, the collection creates duplicate entries and splits the counters; normalize against the run's checkout/cwd before using the path as the key.

Useful? React with 👍 / 👎.

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Grok simplify review (2nd parallel pass — this is the Grok version)

Verdict

Architecture: RIGHT. Fidelity lock: OVERSHOT. Decisions 1–3 are the right product calls. Two session-scoped resource collections behind one off-by-default option, vendor tool names only in @flow-state-dev/claude-code, core/engine/orchestration untouched — that is the clean shape. I would not redesign the storage model.

What went farther than necessary is locking the unstable plan half at full fidelity in the same bite: id-from-prose recovery + bounded transitions[] + a goal that hard-fails on empty plan and asserts transition-log movement. The POC correctly killed TodoWrite; it does not justify treating TaskCreate/TaskUpdate as a durable, history-bearing contract yet.

Disposition: COMMENT — directionally approve Decisions 1–3; settle the fidelity forks below before implement locks them. Not REQUEST_CHANGES (no rewrite needed). Not APPROVE (transition-log / goal coupling would bake in avoidable complexity).

What to keep

  • Decision 1 — plan off the dispatching board. Real taskSchema is an execution envelope (attempts, leaseUntil, claimedBy). Resource collection is correct; do not “simplify” onto Task.
  • Decision 2 — paths, never contents.
  • Decision 3 — recorder never kills the run, paired with empty-list = FAIL.
  • Both files and plan as the final epic answer — files-only forever defers the kill line. Thin/sequenced plan is the open fork, not dropping plan.
  • Tool-name tables at the vendor edge — matches SUBAGENT_TOOL_NAMES in translate.ts. Correlation is not a smell.
  • No 3rd-party package win. Reuse normalizeResourcePath from core. Directory shape (claude-code only) is clean.

Cleaner change available

  1. Drop/defer transitions[] (highest lever). Current status (+ optional updatedAt) answers the epic claim. Live progression already rides the tool stream. You already flagged uncertainty; §10 hard-locks it.
  2. Prefer not extending TranslatedEvent. Existing tool_call/tool_result carry name, args, outcome. Emit’s switch is exhaustive — new kinds force no-ops. Recorder on existing tool_* events is likely smaller. Uncertain until both are sketched — don’t lock the sketch’s two new event kinds.
  3. Plan thin under vendor churn (should-discuss, not certain). §3 correctly rejects files-only forever; it over-rejects files-hard + plan-thin. Same recordWork flag; files hard in the goal; plan as title+status best-effort until the harness .d.ts stops lying.
  4. Decision surface gap. Expensive forks (status vs log; plan-goal hardness) are not numbered; Decision 3 is the cheap one. Surface those to the PO.

Skipped

Capability dual-declare may be cargo-culted from LAB-133 — verify bubble semantics. Counter shape fine. POC quality out of scope.

Bottom line

Keep the approach. Shrink plan to current status, don’t bake transitions[] into the goal, prefer consuming existing tool_* events. Still proves “readable from FSD state after a detached run.”

Open in Web View Automation 

Sent by Cursor Automation: Grok Review For Best Approach

Comment thread spec/LAB-134.md Outdated
Comment on lines +92 to +97
**The simpler approach: record only the files, drop the to-do half.** Genuinely tempting —
the file half is the one with a stable surface and it carries most of the immediate value.
Insufficient because the to-do half is the only part that tests the epic's actual claim. Files
tell you what happened; the to-do list is the only place the agent says what it *thought it was
doing*, which is the thing a knowledge layer weighs. Dropping it defers the risk rather than
retiring it.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Grok simplify — should-discuss (not certain this must change)

Rejecting “files-only forever” is right for the epic kill line. What’s overshot is treating that as also rejecting files-hard + plan-thin sequencing.

The POC showed the plan surface already drifted (TodoWrite → TaskCreate/TaskUpdate, id-in-prose). That argues for less plan fidelity in v1, not locking dual hard-asserts in one goal. Cleaner middle: same recordWork option, durable touched-files as the load-bearing proof, plan as title + current status (best-effort) until the harness stops lying in its .d.ts.

Unsure whether this needs a follow-up issue vs a thinner §10 — flagging the fork, not demanding a rewrite.

Comment thread spec/LAB-134.md Outdated
Comment on lines +153 to +156
{ "topic": "3", "state": { "title": "Create notes.txt", "status": "completed",
"transitions": [ { "status": "pending", "at": … },
{ "status": "in_progress", "at": … },
{ "status": "completed", "at": … } ] } },

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Grok simplify — highest-leverage cut

transitions[] is not entailed by “todo progression is readable as state.” Current status (and maybe updatedAt) already answers what the agent thought its job was after the run. Live progression is already on the tool stream.

This is the only field that grows with run length — you already flag uncertainty in the PR body. Prefer deferring history until LAB-135 names a grader that needs thrash. Example + sketch + §10 currently lock it as if Decision 1 required it; it doesn’t.

Comment thread spec/LAB-134.md Outdated
Comment on lines +248 to +251
> What this asks a reviewer: *is the translation layer the right home for the tool-name table,
> or should the recorder own it?* Keeping it in translation is what makes the recorder
> vendor-free and unit-testable with scripted events — but it also means the pure layer now
> carries a little correlation state it did not before.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Grok simplify — answer from existing code; soft on event kinds

Tool-name tables belong at the vendor edge — same pattern as SUBAGENT_TOOL_NAMES in packages/claude-code/src/sdk/translate.ts. Correlation on call→result already exists. Extending that is not a purity smell.

Softer fork: you may not need new file_touched / plan_item members on TranslatedEvent. tool_call already carries name + arguments; tool_result carries outcome. emitTranslatedEvent is an exhaustive switch — new union variants force emit no-ops. A recorder consuming existing tool_* events is likely the smaller change. Uncertain until sketched both ways — don’t lock the sketch’s two new event kinds as the contract.

Comment thread spec/LAB-134.md Outdated
> file_touched → upsert the file list, keyed by the normalized path,
> bumping the counter for its kind
> plan_item → upsert the plan list, keyed by the id,
> appending the status to a bounded transition log

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Grok simplify

“Appending to a bounded transition log” is the overshoot. Upsert by id with current title/status is enough for Decision 1’s fork-detection claim. If a bounded log survives anyway, mirror Task’s writeLogTruncated pattern — but prefer not shipping the log.

Comment thread spec/LAB-134.md
Comment on lines +318 to +322
**What it asserts:**
- the touched-files list names the held-out files, with the touch kinds the job implies
- the plan list carries at least the held-out number of distinct items, keyed by the harness's
own ids, and at least one row's transition log shows a status actually moving
- **the dispatching board's task collection still holds exactly the one row that was filed** —

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Grok simplify — goal coupling hard-locks the overshoot

These asserts bake full plan fidelity into the proof: held-out plan count, harness ids, and transition log shows status moving.

Count + ids with current status only still prove Decision 1 (board still has exactly one row). The transition-log assert should drop with the log.

Also reconsider whether empty plan must FAIL the same way empty files must. Files surface is POC-confirmed; plan surface already moved. Symmetric hard-fail is the risky part. Uncertain on the exact softer criterion.

Comment thread spec/LAB-134.md Outdated
Comment on lines +200 to +206
**Why the plan is a resource collection and not a task collection.** Our `Task` carries an
execution contract — attempts, lease, claimant, retry budget. Nothing on our side ever claims,
leases, or settles an observed to-do, so writing one as a `Task` would populate that envelope
with defaults nobody set and nobody honours, and leave rows that look claimable and are not.
It would also invert the package layering: the vendor adapter depends on core alone, and the
task substrate is a layer above it. The observed shape is deliberately *smaller* than a task —
which is decision 1 expressed in the type.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Grok simplify — keep this; do not “simplify” onto Task

Checked against real packages/orchestration/src/tasks/schema/task.ts: execution envelope is real (attempts, leaseUntil, claimedBy, retry/abandonment counters, writeLog). Rationale holds. Off-board resource collection is the clean layering call, not accidental complexity.

Comment thread spec/LAB-134.md Outdated
Comment on lines +56 to +70
1. **A run's to-do list is recorded where nothing will execute it.** The observed items go in
their own list, separate from the work queue that dispatched the run. *Alternative
rejected:* writing them straight onto that queue, which reads as "the agent's plan is now
our plan". *Locks in:* an agent that decides mid-run to do five more things cannot start
five more jobs by writing a to-do list — a human or a later step has to pick one up on
purpose. The cost is one extra step whenever we do want to adopt one.

2. **We record what the run touched, never what it wrote.** Paths, how each was touched, when,
and whether it succeeded. Not file contents, not diffs. *Alternative rejected:* keeping the
written content, which the harness does hand us for free. *Locks in:* our record stays a
small index pointing at the working tree instead of becoming a second copy of the
customer's source in our store — with the storage growth and the disclosure question that
would bring. Anyone who needs the content reads the repo.

3. **Watching the work must never break the work.** Anything the recorder cannot handle — a

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Grok simplify — Decisions 1–3 are right; surface is incomplete

Keep all three. Soft note: Decision 3 is the cheap engineering call, while the expensive forks — status vs transition log, and how hard the plan goal is under vendor churn — are not numbered here. Those are the ones a PO should price before implement.

Re-draft: rewrites §2/§6/§7/§9/§10/§12 after review.

The load-bearing correction: the POC concluded 'tool results are prose' having
measured only the content block. Re-ran it logging tool_use_result — the full
declared structured Output IS on the wire for file tools. Claim now scoped to
what was measured, and the plan tools are marked unmeasured rather than
assumed.

Also: scoped the headline claim to observed file-tool operations (shell-driven
edits are invisible and stay so); both collections must declare client state
reads or the route 403s (verified in resource-routes.ts); records namespaced
per run because a workstream session is reused; coalesced flush rather than
per-event or end-only; rejected updates never record as applied; dropped the
transition log and its cap for previousStatus; collapsed the capability step
after verifying uses merges resources into block config; goal check extends the
LAB-133 harness and grades the plan half at title+status with a shaped job.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MVcrTezeLMBQRVHRg2QX2f

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8eaf7913d3

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread spec/LAB-134.md Outdated
Comment on lines +154 to +156
{ "items": [
{ "topic": "<runId>/repo/src/checkout.ts",
"state": { "lastKind": "edited", "lastTouchedAt": 1787021400123, "ok": true } }

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Read collection rows from clientData

Both readback examples place projected resource state under state, but handleListCollectionState in packages/engine/src/routes/resource-routes.ts returns each row as { topic, storageKey, clientData }, and the route tests assert that envelope. Following these examples in the planned HTTP goal or documentation therefore reads undefined after a valid 200 response; use clientData for both collections.

Useful? React with 👍 / 👎.

Comment thread spec/LAB-134.md Outdated
Comment on lines +352 to +354
*(Not a separate step for the capability: `uses` merges a capability's resources into the
consuming block's config, unlike `sessionStateSchema` — verified in `blocks/handler.ts` —
so forwarding the option is sufficient and a smoke test is the right coverage.)*

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Declare the collections on the capability itself

When recordWork is enabled through createClaudeCodeAgentCapability, forwarding the option only places the resource-declaring agent handler in the capability's tools preset. resolveCapabilities merges resources declared by the capability itself, while generator remains a leaf and does not collect declarations from preset tools; consequently the consuming flow omits these collection refs and the later HTTP lookup returns 404. The capability must contribute the same resource definitions explicitly rather than relying only on option forwarding.

Useful? React with 👍 / 👎.

Comment thread spec/LAB-134.md Outdated
Comment on lines +427 to +428
- **Shape the job so planning is required** — multi-step work with a real dependency between the
steps, so the plan surface is exercised deterministically rather than incidentally.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Make plan-tool invocation deterministic

A dependent multi-step prompt does not make the plan surface deterministic: the POC job in probe.mjs already creates a file and then edits that same file, explicitly requesting a two-item plan, yet the measured runs reported that two of four executions used no plan tools. A correct recorder can therefore fail the mandatory non-empty-plan goal solely because the model completed the work without TaskCreate/TaskUpdate; use a deterministic invocation path or keep source non-invocation outside the implementation verdict.

Useful? React with 👍 / 👎.

Comment thread spec/LAB-134.md Outdated
Comment on lines +268 to +270
**State shapes are deliberately flat — no accumulating history.** A file entry carries
`lastKind`, `lastTouchedAt`, `ok`. A plan entry carries `title`, `status`, `previousStatus`,
`lastTouchedAt`. There is **no transition array and no cap policy**: the sibling file entry

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Add a pending outcome for unsettled file calls

Fresh evidence since the earlier mutation-attempt comment is that the revised design now requires persisting an unsettled call-time attempt, but the specified row still has only boolean ok. That field can represent success or failure, not the unknown state required before a result arrives: writing false misreports a pending operation as failed, while waiting for settlement again loses killed-run attempts. Add an explicit pending outcome or make ok nullable and define its meaning.

Useful? React with 👍 / 👎.

Comment thread spec/LAB-134.md
| A file changed through the shell (`sed -i`, redirect, `mv`, formatter, `git apply`) | Not recorded, by design. Decision 2; never closed by reading the filesystem |
| Path that cannot be keyed (traversal segments, control characters) | Skip the entry, note it, continue — the key normalizer throws and that must not reach the run |
| Same file as `/workspace/repo/src/a.ts` and `src/a.ts` | One entry. Canonicalize against the run's working directory before keying; the key normalizer alone only strips leading slashes |
| A `TaskUpdate` whose result is an error | Recorded as a failed attempt, **never as applied** — writing the requested status for a transition the harness refused is the worst available outcome |

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Store rejected plan-update outcomes

Fresh evidence since the earlier rejected-update comment is that the revised state shape still provides only title, current/previous status, and a timestamp while this row now promises to record a failed update attempt without applying its requested status. Leaving the statuses unchanged records no evidence of the rejection, while changing either status falsifies the harness state. Add an attempt/outcome field or weaken this contract to explicitly ignore rejected updates.

Useful? React with 👍 / 👎.

Re-draft: rewrites §5/§7/§8/§10 after round 2.

The reversal: I concluded from blocks/handler.ts that a capability forwarding
the option inherits the block's resource declarations, and collapsed the
capability step on that basis. Wrong. mergeSurfaceInto merges surface.resources
and never reads surface.tools for resources; collectBlockResources gathers from
action blocks only; and defineFlow states outright that a generator 'bubbles
none of its tools' rails by design'. So the flow registers nothing and the route
404s at read time on a build that succeeded. Step 4 restored as a real
deliverable; size raised rather than absorbed.

Also: readback examples corrected to clientData (the route returns
{topic, storageKey, clientData}, so reading .state yields undefined after a
valid 200 — the seq/itemIndex failure shape); ok:boolean replaced with a
three-valued outcome so a recorded-at-attempt write can be pending and a
rejected plan update has somewhere to land without falsifying harness state;
and the goal check's INCONCLUSIVE arm reinstated, since the job-shaping meant
to make a hard fail safe was already measured failing.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MVcrTezeLMBQRVHRg2QX2f

Copy link
Copy Markdown
Collaborator Author

Spec approved — direction settled, proceeding to implementation.

Two review rounds spent, both closed. The direction was unchallenged in both: every reviewer endorsed the storage model (two session-scoped resource collections behind one off-by-default option, vendor tool names confined to the adapter, core/engine/orchestration untouched). What the rounds changed was fidelity and correctness, not shape.

Three things settled here that are worth naming, because each was a claim someone held confidently and the code refuted:

  • The capability must declare the collections itself. Forwarding the option only places a resource-declaring block in the capability's tools preset, and a generator is a leaf that bubbles none of its tools' rails by design — so the flow registers nothing and the resource route 404s at read time on a build that succeeded. This reopened a deliverable rather than collapsing one; size moved to ~6 files / ~14 behaviours and §6 says so.
  • Readback rows come back as { topic, storageKey, clientData }. The examples read state, which returns undefined after a valid 200 — a check reading a field the payload never carries, surfacing as "found nothing" rather than an error. §10 now tests the read instead of assuming it, since clientData is a projection and the asserted fields have to be in the exposed set.
  • An empty plan is INCONCLUSIVE, not a failure. The mitigation for a hard fail — shape the job so planning is required — was already measured failing: the probe's own job requests a two-item plan over dependent steps, and 2 of 4 runs still invoked no plan tools. The third verdict must name which empty it is, never counts as a pass, and the verdict log records which arm each run took so drift toward never measuring the plan half stays visible.

Two product calls are recorded in §6 and are the place to overrule if either is wrong: the observed plan is recorded off the dispatching board (so an agent writing a to-do list cannot enqueue work nobody approved), and we record paths, never file contents (no second copy of source in our store, at the cost of not answering "what changed in this file" without going to the repo).

This PR does not merge — Linear holds the durable copy. Implementation continues on fix/LAB-134.


Generated by Claude Code

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 17b5238803

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread spec/LAB-134.md
Comment on lines +418 to +419
| Path that cannot be keyed (traversal segments, control characters) | Skip the entry, note it, continue — the key normalizer throws and that must not reach the run |
| Same file as `/workspace/repo/src/a.ts` and `src/a.ts` | One entry. Canonicalize against the run's working directory before keying; the key normalizer alone only strips leading slashes |

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Preserve goal files outside the run working directory

The required goal writes to an absolute OS-temp path (goals/harness-workstream/mirrors-a-coding-run-as-workstream-items/run.mts:135-140,275-279), while claudeCodeAgent exposes no cwd, so canonicalizing that path against the process working directory produces ../.... The adjacent rule then skips it because normalizeResourcePath rejects traversal, leaving the mandatory file list empty after a successful run. Encode outside-root absolute paths under a safe namespace or run the goal inside the canonical root.

Useful? React with 👍 / 👎.

Comment thread spec/LAB-134.md
and each row's payload is on **`clientData`**, not `state`:

```
GET /sessions/<workstreamId>/resources/observed-file-ops

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Scope and paginate readback to the requested run

Fresh evidence beyond the earlier reused-workstream finding is that rows are now namespaced, but this revised URL still requests an unfiltered first page. handleListCollectionState sorts and cursor-paginates collection rows, so after enough reused-workstream runs this response can mix prior runs or omit the requested run entirely. Add the run's full storage-key namespace as topicPrefix and require callers or the goal check to follow nextCursor; apply the same change to the plan URL.

Useful? React with 👍 / 👎.

Comment thread spec/LAB-134.md
rows: a second run would merge its file entries by path with the first run's and mix its plan
items in, and the route would answer "what has this workstream ever done" instead of "what did
this run do". **Key every entry under the run's own identity** (the request id is the natural
handle) rather than resetting, so a workstream's history stays inspectable per run.

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Make retained per-run collections lazy

When a workstream is reused, this rule intentionally retains a new namespace for every request, but the collection design never specifies prefetchMode: "lazy". Resource collections default to eager loading, so each later coding run bulk-loads and caches every historical file and plan row before updating only its own keys; execution cost therefore grows with the workstream's lifetime. Declare both collections lazy and test that a run does not issue a full-prefix load.

Useful? React with 👍 / 👎.

Comment thread spec/LAB-134.md
| A `TaskUpdate` whose result is an error | Recorded as a failed attempt, **never as applied** — writing the requested status for a transition the harness refused is the worst available outcome |
| A to-do tool we do not recognise | Record nothing. The run is unaffected; §10 grades the empty result |
| `TaskCreate` with no recoverable id | Record nothing for that item; later updates naming it also record nothing |
| A file mutation that failed | Recorded with `ok: false` — absence would be indistinguishable from "never attempted" |

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Use the three-valued outcome contract consistently

Fresh evidence since the earlier pending-outcome comment is that the revised design now explicitly defines outcome as pending | applied | failed, but this normative edge-case row still requires ok: false and the pseudocode at line 366 still writes ok. An implementer following these later instructions would recreate the pending-versus-failed ambiguity or produce schemas and tests using incompatible fields; replace both remnants with outcome: "failed".

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants