Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
23 commits
Select commit Hold shift + click to select a range
118b790
feat(claude-code): run the agent as a workstream (LAB-133)
claude Aug 18, 2026
feb4dc3
chore: name LAB-133 in the changeset body (process guard)
claude Aug 18, 2026
0a2dfcf
fix(goal): the ordering assertion was blind and passed anyway (LAB-133)
claude Aug 18, 2026
156eb7b
docs(goal): verdict rows for the corrected ordering assertion (LAB-133)
claude Aug 18, 2026
afe30a1
docs(claude-code): two claims the page was making that the code does …
claude Aug 18, 2026
76e0566
docs(claude-code): same two corrections in the README (LAB-133)
claude Aug 18, 2026
2f066e7
feat(claude-code): record a coding run's file operations and its plan…
claude Aug 18, 2026
5c07baa
docs(goal): cite the commit each verdict row was actually run on (LAB…
claude Aug 18, 2026
df6cfdb
fix(claude-code): three ways the record could misreport what happened…
claude Aug 18, 2026
4f2f63d
fix(claude-code): key the record by invocation, not by request (LAB-134)
claude Aug 18, 2026
ee55636
fix(goal): two assertions that passed the world they existed to rejec…
claude Aug 18, 2026
952bb09
fix(claude-code): make recorder shutdown structural, and settle the f…
claude Aug 18, 2026
0eba857
fix(claude-code): record what the harness confirms, not what the agen…
claude Aug 18, 2026
75dc695
fix(claude-code): never settle from call-time values wearing a confir…
claude Aug 18, 2026
877e25c
fix(claude-code): give observations call identity, and deliver them a…
claude Aug 18, 2026
8b01b39
fix(claude-code): confirm nothing that cannot be checked, and say whi…
claude Aug 18, 2026
281bed4
fix(claude-code): apply each rule on both sides of its axis (LAB-134)
claude Aug 18, 2026
f1b5c96
docs(goal): correct the round-10 before-count from 181 to 176 (LAB-134)
claude Aug 18, 2026
e6cc9b4
feat(claude-code): count applied file operations per path (LAB-134)
claude Aug 19, 2026
16a88ab
refactor(claude-code)!: rename `sessionState` to `detached`, invertin…
claude Aug 19, 2026
e326056
fix(changeset): drop the migration fragment — nothing shipped `sessio…
claude Aug 19, 2026
393cc3e
docs(claude-code): document `appliedCount` on both read surfaces (LAB…
claude Aug 19, 2026
aa8ade3
Merge origin/main into fix/LAB-134
claude Aug 20, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix(claude-code): three ways the record could misreport what happened…
… (LAB-134)

Review rounds 1 and 2 on #1332. All three defects are the same family — the
record saying something other than what the run did.

**The reporting path could fail the run.** `onSkipped` was called unguarded from
inside the serialized flush chain, so a throwing hook rejected `flush()`/`stop()`
and turned a successful coding run into a bookkeeping failure — precisely the
outcome this module exists to make impossible, hiding in the one place nobody
defends because it is the thing that reports failures. Every call now routes
through one guarded helper, so a future third call site cannot forget.

**A phantom unresolved mutation.** The attempt was keyed at call time and the
settlement keyed under the harness's resolved path, so a single write left a
permanent `pending` row beside an `applied` one whenever the two differed. The
key is fixed at call time by the recording cadence, so settling elsewhere can
only ever split one operation in two: settlement now uses the call-time path.
The harness's path still rides along and is compared AFTER canonicalization —
in the recorder, where `notes.txt` and `/work/notes.txt` are known to be one key
— and a real divergence becomes a gap row rather than a silent mis-keying.

**A successful re-wording was dropped.** `TaskUpdateInput.subject` is declared
on the pinned SDK and changes an item's wording, but only `taskId`/`status` were
kept — so `observed-plan` held the create-time title forever while claiming to
be the run's current plan. The new wording is now retained and applied on a
confirmed update only, the same rule the status already followed.

Also: the option's doc said two collections where three are declared, the
comment beside the prose id fallback now scopes what was and was not measured
(one probe run, `claude` 2.1.234), and the comment claiming pathless calls are
"skipped silently" said the opposite of the code — an invitation to simplify
away the gap emission that makes the next issue's assertion buildable.

Kept the coalescing flush: a turn-boundary flush survives a kill between turns
but not within one, and a long tool-heavy turn is exactly the coding-run shape.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01MVcrTezeLMBQRVHRg2QX2f
  • Loading branch information
claude committed Aug 18, 2026
commit df6cfdbedcf313bb2a0a2c766d42ec359b18583c
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,7 @@ were deleted for, and it is no less wrong when the difference is small.
| 2026-08-18 | working tree, pre-commit (plan tools **absent** from `allowedTools`) | claude-sonnet-5 (SDK default) | FAIL (plan arm INCONCLUSIVE) | The first real run of the LAB-134 half. File half green end to end. Plan arm INCONCLUSIVE — and the first suspect was this check's own configuration, not the harness: the job asked for a to-do list while the tools that keep one were not in `allowedTools`. Fixed, then re-measured below |
| 2026-08-18 | working tree, pre-commit (`maxTurns: 8`) | claude-sonnet-5 (SDK default) | FAIL (plan arm INCONCLUSIVE) | With the plan tools in `allowedTools`. 2 file rows in 2 run namespaces in one **reused** workstream, `created`/`applied`, each naming its own held-out file and no other's; **0** full-prefix loads of the recorded collections during the runs; board ledger holds 2 rows for 2 dispatches; 0 gap rows. Plan arm INCONCLUSIVE — 0 plan tool calls; tools used: `Bash`, `Write`. Rules out `allowedTools` |
| 2026-08-18 | `2f066e7c2` (behaviour-identical; only comment wording and this file changed after) | claude-sonnet-5 (SDK default) | FAIL (plan arm INCONCLUSIVE) | Same shape, with `maxTurns` raised 8 → 16. Rules out the turn budget too, which is what makes the arm a statement about the harness rather than about this file |
| 2026-08-18 | post-review (round 2 fixes) | — | FAIL *(deliberate)* | 7 further mutations, each restoring a defect review round 2 found, broken and observed red then reverted: the reporting hook called unguarded from the flush chain (both call sites), settlement keying under the harness's resolved path (the phantom `pending` row), a real path divergence swallowed, the divergence check comparing raw strings instead of canonical keys, a successful `TaskUpdate`'s new wording dropped, and a refused one's wording applied anyway. 7/7 caught |
| 2026-08-18 | `2f066e7c2` | — | FAIL *(deliberate)* | 13 mutations of the LAB-134 guards, each broken on purpose and observed red, then reverted: failed-mutation outcome, resolved-path preference, the two-results attribution guard, prose id recovery, a rejected update applying its status, a pathless file tool silently dropped, per-run keying, the durable gap row, a throwing write propagating, path relativisation reintroducing `..`, `client.state.read`, `prefetchMode: "lazy"`, and the capability forwarding the option instead of declaring the collections. 13/13 caught |
| 2026-08-18 | `0a2dfcfb6` | claude-sonnet-5 (SDK default) | PASS | 21 items with `include_items=true`, **0** without; 1 top-level message, 2 top-level tool_outputs; non-decreasing on `itemIndex` 0–20; names the held-out file; the originating request carried none of the run's mirrored items |
| 2026-08-18 | `0a2dfcfb6` | claude-sonnet-5 (SDK default) | PASS | second consecutive run — 22 items / 0 without, `itemIndex` 0–21, same shape |
Expand Down
22 changes: 15 additions & 7 deletions packages/claude-code/src/sdk/agent.ts
Original file line number Diff line number Diff line change
Expand Up @@ -131,7 +131,7 @@ export interface ClaudeCodeAgentOptions {
*
* **This is the canonical explanation; everywhere else links here.**
*
* Set `true` and the block declares two session-scoped resource collections
* Set `true` and the block declares three session-scoped resource collections
* and writes into them as the run goes:
*
* - `observed-file-ops` — one entry per path the run's file-writing and
Expand All @@ -145,9 +145,9 @@ export interface ClaudeCodeAgentOptions {
* could not record, so a missing row is never confused with a mutation that
* never happened.
*
* Both are readable over the resource route that already ships, keyed under
* the run's own request id so a workstream reused across runs answers "what
* did this run do" rather than "what has this workstream ever done".
* All three are readable over the resource route that already ships, keyed
* under the run's own request id so a workstream reused across runs answers
* "what did this run do" rather than "what has this workstream ever done".
*
* **What the file record is, precisely.** It is a log of the file operations
* the run's file TOOLS performed, not an index of everything that changed on
Expand All @@ -156,7 +156,9 @@ export interface ClaudeCodeAgentOptions {
* the file does not appear.
*
* Recording never fails the run: anything the recorder cannot handle is
* skipped and noted as a status item.
* skipped, written to `observed-gaps`, and noted as a status item. The gap
* ROW is the durable half — a status note dedupes on its own message and
* renders into a latest-wins slot, so two identical skips collapse to one.
*/
recordWork?: boolean;
/** Block name. Default `"claude-code-agent"`. */
Expand Down Expand Up @@ -190,15 +192,21 @@ export function forwardSignalToController(signal: AbortSignal | undefined): Abor
}

/**
* Open a {@link WorkRecorder} against the two collections the block declared,
* or return `null` with a status note when they are not on the context.
* Open a {@link WorkRecorder} against the three collections the block declared,
* or return `null` with a status note when the two records are not on the
* context.
*
* The absent case is real rather than defensive: the declaration and the run
* are wired at different layers, and a flow that reaches this block without the
* refs registered would otherwise throw here — killing a coding run over
* bookkeeping, which is the one thing this feature must never do. A note makes
* the gap visible; the run continues unrecorded.
*
* `observed-gaps` is the one that may be absent without stopping recording: a
* run whose file and plan records still land is worth having, and the skips
* degrade to status notes alone. The two records are not optional that way —
* without them there is nothing to record into.
*
* Every entry is keyed under the run's request id. That is the run's own
* identity, and it is what keeps a reused workstream's runs apart.
*/
Expand Down
66 changes: 54 additions & 12 deletions packages/claude-code/src/sdk/translate.ts
Original file line number Diff line number Diff line change
Expand Up @@ -86,10 +86,15 @@ function resultProse(content: unknown): string {

/**
* Recover a created item's id from the result prose ("Task #5 created
* successfully: …") when the structured Output is absent. The structured field
* IS present at the version measured, so this is the fallback rather than the
* path — but a run that yields no id at all must degrade to recording nothing
* (§9), and that only stays true if the cheaper source is tried first.
* successfully: …") when the structured Output is absent.
*
* **Scope of what was measured, because this fallback keeps being read as dead
* code.** On `claude` 2.1.234 — the CLI version `@anthropic-ai/claude-agent-sdk@0.3.234`
* pins — one probe run showed `tool_use_result.task.id` present on 2 of 2
* `TaskCreate` calls. That is a single run on a single pinned version, not a
* guarantee for every version or every call: the same probe is why we know this
* vendor renamed its whole to-do surface between versions while its shipped type
* declarations still described the old one. Keep the fallback.
*/
function recoverItemIdFromProse(content: unknown): string | null {
const match = /task\s*#\s*([A-Za-z0-9_.-]+)/i.exec(resultProse(content));
Expand Down Expand Up @@ -122,7 +127,10 @@ export interface TranslateState {
*/
readonly openPlanCreates: Map<string, { title: string | null }>;
/** Plan updates whose call was seen, awaiting confirmation or rejection. */
readonly openPlanUpdates: Map<string, { itemId: string; status: string | null }>;
readonly openPlanUpdates: Map<
string,
{ itemId: string; status: string | null; title: string | null }
>;
}
Comment thread
jhoffner marked this conversation as resolved.
Resolved

/** Options controlling how messages are interpreted across a run. */
Expand All @@ -139,7 +147,10 @@ export function createTranslateState(options: TranslateStateOptions = {}): Trans
partialMessages: options.partialMessages ?? true,
openFileOps: new Map<string, { path: string; op: ObservedFileOpKind }>(),
openPlanCreates: new Map<string, { title: string | null }>(),
openPlanUpdates: new Map<string, { itemId: string; status: string | null }>(),
openPlanUpdates: new Map<
string,
{ itemId: string; status: string | null; title: string | null }
>(),
};
}

Expand Down Expand Up @@ -258,8 +269,15 @@ function translateAssistant(
* the one whose record matters most, and a result that never arrives would
* otherwise erase it.
*
* A call whose input carries nothing to key on is skipped silently: the run is
* unaffected, and the empty result is what the goal check grades.
* A call whose input carries nothing to key on is **not** skipped silently: it
* emits `work_gap_observed`, which becomes a durable row. That row is the whole
* point — we recognised the tool and still recorded nothing, so without it a
* later comparison of the run's tool activity against the file record cannot
* tell "we could not key this" from "the record lost a write". Do not simplify
* the gap emission away on the grounds that the run is unaffected; the run is
* unaffected either way, and the reader is not.
*
* A tool we never claimed to record is different, and correctly stays silent.
*/
function observeToolCall(
block: { id: string; name: string; input?: unknown },
Expand Down Expand Up @@ -293,9 +311,16 @@ function observeToolCall(
});
return;
}
// An update can re-word the item, not just move it. Dropping the new
// wording leaves `observed-plan` holding the CREATE-time title forever
// while claiming to be the run's current plan — the collection's own claim,
// falsified by a stale field. Declared on `TaskUpdateInput` in the pinned
// SDK's types; only the status path has been observed on a live run, which
// is why both are read the same tolerant way and neither is required.
state.openPlanUpdates.set(block.id, {
itemId,
status: readString(block.input, "status"),
title: readString(block.input, "subject"),
});
Comment thread
jhoffner marked this conversation as resolved.
Resolved
// Attempted, not applied: the harness may still refuse the transition, and
// writing the requested status now would be recording a move that never
Expand Down Expand Up @@ -323,14 +348,27 @@ function observeToolResult(
const fileOp = state.openFileOps.get(callId);
if (fileOp !== undefined) {
state.openFileOps.delete(callId);
// Prefer the path the harness resolved (the structured Output's own
// `filePath`) over the one the model asked for; fall back to the input.
const path = readString(structured, "filePath") ?? fileOp.path;
// Settle under the CALL-TIME path, not the harness's resolved one.
//
// Recording at call time fixes the row's key at call time, so settling
// under a different path cannot update that row — it writes a SECOND one,
// leaving a permanent `pending` row beside an `applied` one for a single
// operation. A phantom unresolved mutation is worse than a slightly less
// canonical key: it is indistinguishable from the record having lost a
// write, which is the one confusion this whole feature exists to remove.
//
// The harness's own path still rides along so the recorder can compare the
// two AFTER canonicalization — which is where that comparison belongs,
// since `notes.txt` and `/work/notes.txt` are the same key and only the
// recorder knows it. A real divergence becomes a gap, never a silent
// mis-keying.
const resolved = readString(structured, "filePath");
events.push({
kind: "file_op_observed",
path,
path: fileOp.path,
op: fileOp.op,
outcome: isError ? "failed" : "applied",
...(resolved !== null && resolved !== fileOp.path ? { resolvedPath: resolved } : {}),
Comment thread
jhoffner marked this conversation as resolved.
Resolved
});
return;
}
Expand Down Expand Up @@ -368,9 +406,13 @@ function observeToolResult(
events.push({ kind: "plan_item_observed", itemId: update.itemId, outcome: "failed" });
return;
}
// Both the new wording and the new status land only HERE, on a confirmed
// update — the same rule for both, because a re-wording the harness refused
// is as wrong to record as a transition it refused.
events.push({
kind: "plan_item_observed",
itemId: update.itemId,
Comment thread
jhoffner marked this conversation as resolved.
Resolved
...(update.title !== null ? { title: update.title } : {}),
...(update.status !== null ? { status: update.status } : {}),
outcome: "applied",
});
Expand Down
13 changes: 12 additions & 1 deletion packages/claude-code/src/sdk/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -194,8 +194,19 @@ export type TranslatedEvent =
*/
| {
kind: "file_op_observed";
/** The path as the run addressed it; the recorder canonicalizes it. */
/**
* The path as the run addressed it AT CALL TIME; the recorder
* canonicalizes it into the row's key. Stable across the attempt and its
* settlement, which is what keeps one operation to one row.
*/
path: string;
/**
* The path the harness reported instead, present on a settlement only
* when it differs from {@link path} as a raw string. Not a second key —
* the recorder canonicalizes both and, if they still differ, records the
* divergence as a gap rather than silently keying under either.
*/
resolvedPath?: string;
op: ObservedFileOpKind;
outcome: ObservedOutcome;
}
Expand Down
4 changes: 3 additions & 1 deletion packages/claude-code/src/sdk/work-collections.ts
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,9 @@ export const observedFileOpsCollection = defineResourceCollection({
});

/**
* One row per mutation the recorder RECOGNISED and then could not record.
* One row per mutation the recorder RECOGNISED and then could not record
* faithfully — whether it recorded nothing, or recorded it under a key the
* harness disagrees with.
*
* Without this a skip is indistinguishable from a mutation that never happened,
* which is the exact blindness the rest of this feature exists to remove: a
Expand Down
37 changes: 33 additions & 4 deletions packages/claude-code/src/sdk/work-recorder.ts
Original file line number Diff line number Diff line change
Expand Up @@ -167,12 +167,27 @@ export function createWorkRecorder(options: WorkRecorderOptions): WorkRecorder {
* a path that cannot be a key — as a value, a control character is just a
* character.
*/
const gap = (reason: string, rawPath?: string): void => {
/**
* Call the caller's reporting hook, swallowing anything it throws.
*
* **A reporting hook that throws must not become the failure it reports.**
* Every call to `onSkipped` goes through here, and that is the point rather
* than tidiness: the reporting path is the one place nobody defends, because
* it is the thing that reports failures — and an unguarded call inside the
* serialized flush chain rejects `flush()`/`stop()`, turning a successful
* coding run into a bookkeeping failure. That is exactly the outcome this
* whole module exists to make impossible.
*/
const report = (reason: string): void => {
try {
onSkipped?.(reason);
} catch {
// A reporting hook that throws must not become the failure it reports.
// Deliberately empty: see above.
}
};

const gap = (reason: string, rawPath?: string): void => {
report(reason);
gapOrdinal += 1;
pendingGaps.set(`${runId}/${String(gapOrdinal).padStart(6, "0")}`, {
reason,
Expand Down Expand Up @@ -207,7 +222,21 @@ export function createWorkRecorder(options: WorkRecorderOptions): WorkRecorder {
};

const observeFileOp = (event: Extract<TranslatedEvent, { kind: "file_op_observed" }>): void => {
const key = keyFor("file operation", canonicalFilePathKey(event.path), event.path);
const canonical = canonicalFilePathKey(event.path);
// The harness named a different path, and canonicalization did not
// reconcile them — so the row about to be written is keyed under a path the
// harness says it did not touch. Recording the operation is still right (it
// happened), but a reader must be able to see that the key is contested,
// or a later comparison against the run's tool activity reads this as the
// record having lost a write.
if (event.resolvedPath !== undefined && canonicalFilePathKey(event.resolvedPath) !== canonical) {
gap(
`a file mutation was recorded under the path the run named, which is not the path ` +
`the harness reported ("${event.resolvedPath}")`,
event.path,
);
}
const key = keyFor("file operation", canonical, event.path);
if (key === null) return;
pendingFiles.set(key, {
lastKind: event.op,
Expand Down Expand Up @@ -260,7 +289,7 @@ export function createWorkRecorder(options: WorkRecorderOptions): WorkRecorder {
} catch (err) {
const reason = `a ${kind} entry for "${key}" could not be written: ${(err as Error).message}`;
if (recordFailureAsGap) gap(reason);
else onSkipped?.(reason);
else report(reason);
}
}
};
Expand Down
48 changes: 48 additions & 0 deletions packages/claude-code/test/sdk/agent.spec.ts
Original file line number Diff line number Diff line change
Expand Up @@ -745,6 +745,54 @@ describe("claudeCodeAgent — recordWork", () => {
});
});

it("leaves ONE row for one write, even when the harness resolves a different path", async () => {
// The defect this pins is only visible where translate and the recorder
// compose: the attempt is keyed at call time and the settlement arrived
// under the resolved path, so a single write produced a permanent `pending`
// row beside an `applied` one.
const divergent: SdkMessageLike[] = [
{
type: "assistant",
message: {
content: [
{
type: "tool_use",
id: "toolu_w",
name: "Write",
input: { file_path: "/work/notes.txt", content: "HELLO" },
},
],
},
},
{
type: "user",
tool_use_result: { type: "create", filePath: "/work/elsewhere/notes.txt" },
message: {
content: [{ type: "tool_result", tool_use_id: "toolu_w", content: "ok" }],
},
},
RESULT_OK,
];
const block = claudeCodeAgent({
resolveClaudeAgent: scriptedQuery(divergent),
recordWork: true,
includePartialMessages: false,
});
const runtime = await createTestContext({ declaredResources: block.declaredResources });
await block.config.execute?.({ prompt: "go" }, runtime.ctx as never);

const resources = runtime.ctx.resources as unknown as Record<
string,
{ list(): Promise<Array<{ path: string; state: Record<string, unknown> }>> }
>;
const fileRows = await resources["observed-file-ops"].list();
expect(fileRows).toHaveLength(1);
expect(fileRows[0].state).toMatchObject({ outcome: "applied" });
// …and the divergence is visible rather than swallowed.
const gapRows = await resources["observed-gaps"].list();
expect(gapRows).toHaveLength(1);
});

it("writes nothing when the option is off, on the same script", async () => {
// The contrast that makes the assertion above able to fail. Declaring the
// resources here would be the only way to read them back, and the point is
Expand Down
Loading
Loading