Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
47 changes: 47 additions & 0 deletions specs/issues/FIX-1185/BUSINESS-RULES.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,47 @@
# FIX-1185 · Business rules

[Spec](SPEC.md) · [Decisions](DECISIONS.md) · **Rules** · [Plan](PLAN.md) · [Docs](DOCS.md)

The cases, written as rules. Each says what a host or the run does and what happens. The
*proved by* column is the check the plan runs.

## What the run starts from

| # | When | Then | Proved by |
|---|---|---|---|
| BR-1 | A host gives no `systemPrompt` | The SDK is asked for Claude Code's own prompt (the `claude_code` preset), with nothing appended | CI, on the options the SDK receives · goal check |
| BR-2 | A host gives a string, including a long one | The string is forwarded as is and replaces Claude Code's prompt, as today | CI |
| BR-3 | A host gives `""` | Forwarded as `""`: the run starts blank, exactly as every run does on today's `main` | CI · the goal check's control |
| BR-4 | A host gives the preset with `append` | Forwarded as is: Claude Code's prompt plus the host's text | CI |
| BR-5 | A host gives an array of prompt blocks, or any other shape the SDK accepts | Forwarded as is. The SDK validates it, as it does `sandbox` | CI, one array case |
| BR-6 | The block is attached as a capability with any of the above | The same as the block built directly | CI |

## Every run, not only the first

| # | When | Then | Proved by |
|---|---|---|---|
| BR-7 | A detached run resumes an earlier session | The same prompt rule applies on the resumed run; its plan rows are its own | CI, on the resume path |
| BR-8 | A run is in-session (not detached) | The same prompt rule applies | CI |
| BR-9 | `recordWork` is off | The prompt rule is the same. Recording never changes what the agent is told | CI |

## The plan record

| # | When | Then | Proved by |
|---|---|---|---|
| BR-10 | A real run, no prompt given, works a multi-step job that names no tool | The plan record holds the agent's own to-do items, read back from state, and at least one reaches `completed` | Goal check, 3 of 3 runs |
| BR-11 | The same run with `""` | No plan rows | Goal check control, 2 of 2 runs |
| BR-12 | A run reports no plan at all | Nothing is invented. The record stays empty and the plan panel says the harness reported none ([ER-5](../../epics/FIX-1907/BUSINESS-RULES.md#what-a-team-gets-and-what-it-doesnt), FIX-1915's panel) | Existing recorder tests |

## Failure taxonomy

Nothing here is fatal. A prompt shape the SDK rejects fails the run the way any bad SDK option
does today, as a `ClaudeAgentRunError`. A run that plans nothing is not an error: the empty record
is the honest answer.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

BR-12 is “existing recorder tests” — not in PLAN’s V1–VG matrix and not owned by this issue. Consider moving to SPEC “What stays as it is” so acceptance criteria stay BR-10/11 only.


## Acceptance criteria this issue owns

[The goal](SPEC.md#the-goal-and-how-well-know-its-met): on a real model, a run with no prompt
given and a job that names no tool fills the plan record on 3 of 3 runs, and the same check fails
under `GOAL_CONTROL=empty-prompt`. This satisfies the epic's
[ER-7](../../epics/FIX-1907/BUSINESS-RULES.md#what-a-team-gets-and-what-it-doesnt) on the fix
branch, not the finding branch.
96 changes: 96 additions & 0 deletions specs/issues/FIX-1185/DECISIONS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,96 @@
# FIX-1185 · Decisions

[Spec](SPEC.md) · **Decisions** · [Rules](BUSINESS-RULES.md) · [Plan](PLAN.md) · [Docs](DOCS.md)

What was considered, what was chosen, and what each choice locks in. Two decisions are the
sign-off surface. They sit under the epic's [D2](../../epics/FIX-1907/DECISIONS.md#d2) (one
contract, harness differences live in the block's options) and
[D3](../../epics/FIX-1907/DECISIONS.md#d3) (each harness writes its own plan; FIX-1185 makes
Claude Code's fill).

## The tree

```mermaid
flowchart TD
I["FIX-1185"] --> D1["D1 · no prompt given means Claude Code's own prompt"]
D1 -.->|"rejected"| X1["keep the empty default · add a plan line when recording"]
D1 -.->|"rejected"| X2["change nothing · write the finding · each host adds the line"]
I --> D2["D2 · a caller's string still replaces · adding is explicit"]
D2 -.->|"rejected"| X3["every string becomes an addition"]
```

Solid edges are what you're signing. Dashed edges lost, and the label says why.

<a name="d1"></a>
## D1 · With no prompt given, a run starts from Claude Code's own prompt, not the SDK's empty one

| | |
|---|---|
| **Instead of** | Keeping the empty default and having `recordWork` add one line telling the agent to use its to-do tools · or changing nothing and documenting the finding, so each host adds the line itself |
| **Because** | The premise of a harness block is the same agent under our orchestration (tenet 1, the epic's goal). The CLI and the SDK run the same binary; the measured difference is that the CLI starts from Claude Code's prompt and our block starts from nothing ([POC](#settled)). A plan line fixes one surface the blank prompt loses and leaves the rest. A recorder that injects its own instruction would record a plan we asked for, then call it the agent's own. Documenting it leaves the gap in the package, which [ER-11](../../epics/FIX-1907/BUSINESS-RULES.md#what-no-child-may-do) forbids patching downstream |
| **Locks in** | Every host that omits the prompt changes behaviour on upgrade: Kitchen Sink, the coding-skill lab, the conductor lab, the workspace capability, and any app built on the block. Each run carries a larger prompt and follows Claude Code's own conventions. Our default now moves when Anthropic changes Claude Code's prompt |

![D1: what a run starts from with no prompt given. Chosen: Claude Code's own prompt, instead of an empty prompt plus a plan line when recording. Decides it: whose plan is recorded. Price: every host that omits the prompt changes. Locks in a default that moves with Claude Code's prompt](figures/d1-default-prompt.svg)

It comes down to whose plan it is: a line we inject records our instruction, not the agent's account.

**What would change my mind:** the goal check showing Claude Code's own prompt still yields no
plan rows. Then the blank prompt was not the whole cause, and the plan line is the fallback, said
in the docs as an instruction we add.

<a name="d2"></a>
## D2 · A caller's own string still replaces that prompt; adding to it is a separate, explicit form

| | |
|---|---|
| **Instead of** | Treating every string as an addition to Claude Code's prompt |
| **Because** | The option is documented as forwarded to the SDK, and in the SDK a string replaces. Turning it into an addition silently changes every host that wrote a full prompt to narrow the agent on purpose. The SDK already has the addition form (`preset` with `append`); forwarding it costs one type, not a new option (Step 3.5 question 7: a parameter on the existing entry) |
| **Locks in** | Two spellings to teach. A host that wrote one line of role text, and meant it as an addition, keeps an empty plan record until it moves to the added form. In this repo that is Shift Manager's harness and one devforce goal |

![D2: what a caller's prompt string means. Chosen: it replaces, as in the SDK, instead of becoming an addition. Decides it: a host that wrote a full prompt on purpose. Price: one-line role text keeps an empty plan until it moves. Locks in two spellings](figures/d2-string-replaces.svg)

It comes down to the host that wrote a full prompt on purpose: an addition would grow it unasked.

**What would change my mind:** finding that every known caller's string was one line of role
text meant to sit on top. Then replacement protects nobody and the addition is the better default.

## Decided, not asked

- **An empty string means today's blank start.** It is the escape hatch, and it is the goal
check's control. The SDK already reads `""` that way.
- **The option forwards the SDK's own prompt shape**, the way `sandbox` forwards its settings.
No FSD spelling of `preset`.
- **No warning when a string replaces the prompt with recording on.** A caller may want a plain
run; the docs say what it gives up, and the plan record's own emptiness is the signal.
- **The capability takes the same option.** It forwards the agent's options today.

## Considered and dropped

| Alternative | Why not |
|---|---|
| Drive one goal run through the CLI to exercise the plan half once (the issue's second option) | Proves the recorder on a path Kitchen Sink and every host don't use. The epic needs the SDK path to fill |
| Put the plan line in each host's job prompt | Shown to work, but every host copies a line that only exists to undo our default |
| Add `TaskCreate` and `TaskUpdate` to `allowedTools`, or raise `maxTurns` | Measured: no change on 8 runs |

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Keep task tools available on newer Claude models

On Opus 4.8, Sonnet 5, Fable 5, Mythos 5, and newer models, changing only the system prompt cannot satisfy BR-10: the locked Agent SDK 0.3.234 includes the 0.3.233 change that removes the task-tracking tools from the default surface unless they are named through tools, allowedTools, or the environment override. The eight-run result with a blank prompt does not rule out needing both the preset prompt and tool exposure, and the proposed documentation example also omits these tools from its allowlist. Revise the design to retain the task tools or explicitly limit the supported model set. Agent SDK 0.3.233 changelog.

Useful? React with 👍 / 👎.

| A `planGuidance: true` option | A new option for one line the default prompt already carries |

<a name="settled"></a>
## Settled

- **The two drivers differ at prompt assembly, before the model sees anything** — **CONFIRMED**,
model-free. Driving the real SDK `query()` with a stubbed agent process, the first message
it sends carries `systemPrompt: [""]` when none is given, the same as `""`. With the
`claude_code` preset it sends no custom prompt and only the appended text, so the agent
process keeps Claude Code's own. A planted fault (the omitted arm sent as the preset) turns

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Settled claims a planted fault turns the POC red; run.mts only sets refuted from omitted/preset arms. Either add an env-gated fault injection (nice for CI) or reword to “the assertions refute if omitted is not [""] or preset sends systemPrompt.”

the check red. Run: `pnpm tsx specs/issues/FIX-1185/poc/what-the-sdk-sends/run.mts`.
- **An instruction to use the tools restores the calls on the SDK path** — **CONFIRMED** once,
0 → 9 calls and 3 plan rows read back from state
([the Aug 19 measurement](https://linear.app/fixpoint-labs/issue/FIX-1185)). One run, not a
paired control; the goal check repeats it under the control.

## How it got here

- **Draft** — framed as a prompt-assembly gap, not a driver gap; the block's default moves to
Claude Code's own prompt and forwards the SDK's addition form; one PR in the package, proved by
a real-model goal check reading the plan record from state.

**Open: none.**
80 changes: 80 additions & 0 deletions specs/issues/FIX-1185/DOCS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,80 @@
# FIX-1185 · Documentation draft

[Spec](SPEC.md) · [Decisions](DECISIONS.md) · [Rules](BUSINESS-RULES.md) · [Plan](PLAN.md) · **Docs**

Two destinations, both extended, no new page. The epic's shared narrative (the harness picker,
the plan panel) belongs to FIX-1914 and FIX-1915 and is not drafted here. Voice rules most at
risk for this topic: no internal issue numbers in `apps/docs`, few em-dashes, and introduce
"system prompt" in plain terms on first use.

## UPDATE · `apps/docs/docs/tools/claude-code-sdk.md` · "As a sequencer step"

Replace the example's `systemPrompt` line so the first example a reader copies keeps the run's
to-do list:

```ts
const agent = claudeCodeAgent({
systemPrompt: {
type: "preset",
preset: "claude_code",
append: "You are a careful refactoring assistant.",
},
allowedTools: ["Read", "Grep", "Edit"],
});
```

## UPDATE · `apps/docs/docs/tools/claude-code-sdk.md` · "Configuring the run", new subsection before `settingSources`

### The prompt the run starts from

A system prompt is the standing instruction the agent reads before your request. Leave
`systemPrompt` out and the run starts from Claude Code's own, the same one the `claude` command
uses. That prompt is what tells the agent to keep a to-do list as it works, among much else, so
a run started this way behaves like the agent you know from the terminal.

To add your own instructions on top, append them:

```ts
claudeCodeAgent({
systemPrompt: {
type: "preset",
preset: "claude_code",
append: "Commit your work with git before you finish.",
},
});
```

A plain string replaces Claude Code's prompt entirely. The agent then knows only what your string
says. It will still have its tools, but nothing tells it to keep a to-do list, so with
`recordWork` on the plan record usually stays empty. Use a string when you want that narrow
agent; use `append` when you want to add a line.

`""` starts the run with no system prompt at all. The value is forwarded to the Agent SDK as
given, so any shape the SDK accepts works here, and the SDK rejects one it doesn't.

## UPDATE · `apps/docs/docs/tools/claude-code-sdk.md` · "Reading it back", after "The plan record reads the same way"

The plan record fills only when the agent keeps a to-do list. It does when the run starts from
Claude Code's own prompt, with or without `append`. A run given its own prompt string usually
writes its plan as prose instead, and the record stays empty. That is the honest answer, not a
recording failure, and nothing appears in `observed-gaps` for it.

## UPDATE · `packages/claude-code/README.md` · "Recording what a run did", after the code block

The plan half fills when the agent keeps a to-do list, which it does when the run starts from
Claude Code's own prompt. That is the default when `systemPrompt` is left out. To add
instructions, use `systemPrompt: { type: "preset", preset: "claude_code", append: "…" }`. A
plain string replaces Claude Code's prompt and usually leaves `observed-plan` empty; `""` starts
the run with no system prompt.

## UPDATE · `.changeset/*.md` · `@flow-state-dev/claude-code` minor

> `claudeCodeAgent` now starts a run from Claude Code's own system prompt when `systemPrompt` is
> left out, matching the `claude` CLI, and accepts the Agent SDK's preset form to add
> instructions with `append`. Runs that relied on starting with no system prompt pass
> `systemPrompt: ""`.

## Publication ownership

FIX-1185 publishes these four operations after V2 and the goal check pass. It does not edit
`coding-agents.md`: the harness contract is unchanged.
100 changes: 100 additions & 0 deletions specs/issues/FIX-1185/PLAN.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,100 @@
# FIX-1185 · Plan

[Spec](SPEC.md) · [Decisions](DECISIONS.md) · [Rules](BUSINESS-RULES.md) · **Plan** · [Docs](DOCS.md)

Written for the implementing agent. IDs cross-reference [BUSINESS-RULES.md](BUSINESS-RULES.md)
(BR-n) and [DECISIONS.md](DECISIONS.md) (D-n). `diagnose` for the cause (already settled, see
the POC line), then `tdd` for the default. One PR.

## Surfaces

| ID | Package · role | Change | Rules |
|---|---|---|---|
| S1 | `claude-code` · the agent options type and the SDK query-options type | `systemPrompt` takes the SDK's own union (string, string array, the `claude_code` preset with optional `append`), forwarded verbatim (D2) | BR-2 BR-4 BR-5 |
| S2 | `claude-code` · where the block builds the SDK query options | `undefined` becomes the preset with nothing appended (D1). Every other value, `""` included, passes through untouched | BR-1 BR-3 BR-7 BR-8 BR-9 |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

S1 widening to the full SDK union is the right long-term shape (D2 + sandbox precedent). For implementers: if V1 becomes a hand-copied union, it will drift — consider Parameters<typeof query>[0]['options']>['systemPrompt'] style alias, or document unknown on the internal query bag only with assignability smoke tests for the shapes you document publicly.

| S3 | `claude-code` · the option's doc comment, the README's options section | Say what omitting, a string, `""` and the preset each give, and that a string drops the to-do guidance | BR-2 BR-3 |
| S4 | `goals/claude-code-harness/it-keeps-its-own-plan/` | **New** goal: `goal.md`, `run.mts`, a held-out job fixture, `GOAL_CONTROL=empty-prompt` | BR-10 BR-11 |
| S5 | Docs | Publish [DOCS.md](DOCS.md) · one `minor` changeset for `@flow-state-dev/claude-code` (a default changes for every host) | — |

Nothing is removed.

## Sequence

```mermaid
flowchart TD
S1["S1 · widen the type"] --> S2["S2 · the default"]
S2 --> S3["S3 · doc comment and README"]
S2 --> S4["S4 · the goal check"]
S3 --> S5["S5 · docs and changeset"]
S4 --> S5
```

## Checks

| ID | Runs after | Passes when |
|---|---|---|
| V1 | S1 | Type test: each SDK prompt shape is accepted by the block and the capability; a non-prompt object is a type error |
| V2 | S2 | Agent spec: the options the stubbed SDK receives carry the preset when none is given (BR-1); the string, `""`, the preset and an array pass through byte-identical (BR-2 to BR-5). Red first: on today's `main` the BR-1 case fails |
| V3 | S2 | The second path (BP-035): the same assertions on a detached resumed run, an in-session run, `recordWork` off, and through the capability (BR-6 to BR-9) |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

V2 + V3 as separate checklist rows duplicate the same assertions across detached / in-session / recordWork off / capability. The fix is one builder — a single parameterized table in the agent spec (or one BP-035 row: “all entry paths”) would cut plan noise without weakening coverage.

| VG | S4 | [The goal](SPEC.md#the-goal-and-how-well-know-its-met): `pnpm tsx goals/claude-code-harness/it-keeps-its-own-plan/run.mts` PASSES 3 of 3 on a real model, after the same run FAILED 2 of 2 under `GOAL_CONTROL=empty-prompt` on *at least two plan rows* |
| V4 | S3 S5 | `pnpm --filter @flow-state-dev/claude-code test` and `typecheck` green; the docs build |

D1's check is V2's BR-1 case plus VG; D2's is V2's BR-2 case.

## Pinned names

| Where | Name | Why pinned |
|---|---|---|
| Goal folder | `goals/claude-code-harness/it-keeps-its-own-plan/` | The epic's closure cites it |
| Control | `GOAL_CONTROL=empty-prompt` | `SPEC.md` and the closure name it |

The option keeps its name, `systemPrompt`. Everything else is yours to name.

## Guardrails

| Rule | Because |
|---|---|
| The default is decided in one place, where the query options are built | The capability and every resolver path reach the SDK through it. A second place is how one path keeps the blank prompt (tenet 5) |
| Forward the SDK's shape; never translate or validate it | The package forwards `sandbox` the same way. A copy of the SDK's union drifts the day the SDK adds a field |
| The recorder is not touched | D1 rests on the plan being the agent's own. Any instruction the recorder adds would make the record ours |
| The goal check reads the plan record from state, through the resource route | The issue's verification asks for rows read back from state. A count of tool calls in the stream passes while the record stays empty |

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Scope goal reads to the current request

When the goal reuses a session to cover repeated or resumed runs, reading the session-scoped observed-plan collection without the current request's prefix also returns rows from earlier attempts. That can let runs 2 and 3 satisfy the row threshold using run 1's plan, corrupting the 3-of-3 result. Require a fresh session per attempt or query observed-plan/<escaped requestId>/ and assert only that namespace.

Useful? React with 👍 / 👎.

| The goal's job names no tool and does not mention planning | Otherwise the PASS proves the Aug 19 result again, not the default |
| `""` stays the blank start, and the control uses it | A host that wants today's behaviour needs one spelling, and the control must be exactly today's behaviour |

## Docs

Reconcile [DOCS.md](DOCS.md) against the shipped option and VG's verdict, then publish it with
S5. The README section (S3) and the docs page say the same thing in their own register.

## Sketch · pseudocode, illustrative

```
where the block builds the SDK's query options:
prompt ← the host's systemPrompt
if prompt is absent: ← the whole fix (D1)
prompt ← Claude Code's own (the preset), nothing appended
forward prompt unchanged (D2: a string still replaces; "" still blank)
```

**POC:** `poc/what-the-sdk-sends/` inside this spec. Model-free: it drives the real SDK `query()`
with a stubbed agent process and prints the first message the SDK sends. It showed the premise
holds: no prompt and `""` both send `systemPrompt: [""]`; the preset sends no custom prompt, so
the agent process keeps Claude Code's own. A planted fault turned it red. It does not show that
Claude Code's prompt makes the agent plan; VG does that on a real model.

## At implement time

- FIX-1915 may have moved the plan record's shape into `@flow-state-dev/core` first. Rebase on
it; neither PR changes the other's half ([epic seam](../../epics/FIX-1907/PLAN.md#coordination-seams-to-watch)).
- Check the SDK version the workspace resolves. If its default for an absent prompt is no longer
empty, S2 is a no-op: say so, keep S1, S3 and S4.
- The goal needs a host where Claude Code is signed in. A cloud session can't run it; hand it to
a local host and record where it ran.

## Follow-ups

- **Shift Manager's harness and the devforce goal pass a one-line string**, so under D2 their
plan record stays empty. Moving them to the added form is their owners' call. Flagged to the
epic coordinator; not filed here.
- **A run asserted that a sub-agent wrote a file it wrote itself** (same Aug 19 run). A separate
problem, noted on the issue, not filed.
Loading
Loading