Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
a2c8a4f
spec(FIX-2398): what each task and turn cost
claude Oct 31, 2026
9754783
spec(FIX-2398): what each task and turn cost
claude Oct 31, 2026
3383687
spec(FIX-2398): name the docs page that changes
claude Oct 31, 2026
ddf60c8
feat(harness-manager, orchestration, ui): what each task and turn cos…
claude Oct 31, 2026
20874f9
test(kitchen-sink): the turn record fixture names checks
claude Oct 31, 2026
03c0209
Merge FIX-2387's branch (with FIX-2384) into FIX-2398
claude Oct 31, 2026
9515565
fix(harness-manager): write a turn's spend inside the turn, before th…
claude Oct 31, 2026
7f47559
spec(FIX-2398): name the docs page that changes
claude Oct 31, 2026
2f4ac2c
spec(FIX-2398): round 1 — the turn's spend is written inside the turn
claude Oct 31, 2026
55472a3
Merge spec/FIX-2398 round 1 into the implementation
claude Oct 31, 2026
eeec25c
Merge remote-tracking branch 'origin/claude/project-thread-2ksmgs' in…
claude Oct 31, 2026
6143c18
FIX-2398: registry drift count is 40 with FIX-2387's run-checks copy
claude Oct 31, 2026
7b7e828
FIX-2398: Kitchen Sink installs FIX-2387's run-checks and FIX-2384's …
claude Oct 31, 2026
8c2e648
FIX-2398: a request's spend reads as a task's through orchestration's…
claude Oct 31, 2026
7dc5202
Merge claude/project-thread-2ksmgs (4710041) into FIX-2398
claude Oct 31, 2026
6855d10
FIX-2398: a continued request's recovered charge counts once
claude Nov 1, 2026
2fac919
FIX-2398: requestSpendView is spendAfterWrite's settled row, folded
claude Nov 1, 2026
d3c8b22
FIX-2398: a turn continued after an interruption counts every run
claude Nov 1, 2026
c28a080
FIX-2398: the changeset names its issue, and requestSpendView
claude Nov 1, 2026
13aa996
Merge remote-tracking branch 'origin/claude/project-thread-2ksmgs' in…
claude Nov 1, 2026
60ef27a
Merge claude/project-thread-2ksmgs (fe9f981) into FIX-2398; Fallow ig…
claude Nov 1, 2026
5378fad
FIX-2398: a continued turn keeps an earlier run that has no id
claude Nov 1, 2026
7d92efe
FIX-2398: the engine names a harness run reported without an id
claude Nov 1, 2026
7e9add2
ci: pin Kitchen-sink E2E to ubuntu-24.04 (ported from FIX-2438, #4431)
claude Nov 1, 2026
2802ef9
Merge remote-tracking branch 'origin/claude/project-thread-2ksmgs' in…
claude Nov 1, 2026
1cbfb5c
FIX-2398: no tracking id in the engine's settled-run comment
claude Nov 1, 2026
be2089d
Merge remote-tracking branch 'origin/claude/project-thread-2ksmgs' in…
claude Nov 1, 2026
7ef39e4
FIX-2398: a continued turn keeps the model calls its first run made
claude Nov 1, 2026
62a2816
Merge remote-tracking branch 'origin/claude/project-thread-2ksmgs' in…
claude Nov 1, 2026
12b1a3e
FIX-2398: a request carries its harness runs across endings; a turn r…
claude Nov 1, 2026
4675799
FIX-2398: a recovered report replaces its carried run; a live interru…
claude Nov 1, 2026
340c9d3
FIX-2398: a continued request keeps a dead run's later charge when on…
claude Nov 1, 2026
ed81590
Merge remote-tracking branch 'origin/claude/project-thread-2ksmgs' in…
claude Nov 1, 2026
036fbb4
Merge remote-tracking branch 'origin/claude/project-thread-2ksmgs' in…
claude Nov 1, 2026
8353940
Merge remote-tracking branch 'origin/claude/project-thread-2ksmgs' in…
claude Nov 1, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
spec(FIX-2398): what each task and turn cost
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01NczDiDfgRv9riWiVVdj8JG
  • Loading branch information
claude committed Oct 31, 2026
commit a2c8a4f1eae49248428464f748697bc04427bc3c
89 changes: 89 additions & 0 deletions specs/issues/FIX-2398/DECISIONS.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,89 @@
# FIX-2398 · Decisions

[Spec](SPEC.md) · **Decisions**

FIX-2288 put a task's spend on its row and every harness reports into it. These three say what
"usage for every harness" means when two vendors send no reading mid-run, where a Kitchen Sink
turn's spend comes from, and what a cost is.

## The tree

```mermaid
flowchart TD
I["FIX-2398"] --> D1["D1 · no onUsage for Codex and Cursor"]
D1 -.->|"rejected"| X1["an onUsage option on both · nothing on the wire to carry"]
I --> D2["D2 · a turn's spend is its request's ledger at turn end"]
D2 -.->|"rejected"| X2["a per-turn feed the harness calls · a second report path"]
D2 -.->|"rejected"| X3["sum the turn's items · not what the vendor billed"]
I --> D3["D3 · a cost is the vendor's charge or nothing"]
D3 -.->|"rejected"| X4["a price-table estimate · forbidden by FIX-2145 D2"]
```

Solid edges are what you're signing. Dashed edges lost, and the label says why.

<a name="d1"></a>
## D1 · Codex and Cursor get no `onUsage`; Teammates' gap is a reader and a dropped `onActivity`

| | |
|---|---|
| **Instead of** | An `onUsage` option on `codexAgent` and `cursorAgent`, as the issue asks |
| **Because** | `onUsage` carries a subscription window a vendor announces mid-run. Claude Code sends `rate_limit_event`; Codex's wire has `thread.*`, `turn.*`, `item.*` and `error` only, and Cursor's has `usage` (tokens) only (`packages/codex/src/types.ts`, `packages/cursor/src/types.ts`). Their tokens and cost already reach every task row through `reportHarnessSpend`, in all three apps. Codex's windows come from `readCodexUsage` as a `harnessUsageAction` reader. What Teammates actually drops is Cursor's `onActivity` (its host passes `cwd, resume, onSession, onSteerable` only) and any reader |
| **Locks in** | An app that wants Codex's windows registers a reader; nothing arrives by itself. If Codex or Cursor later stream a window event, `onUsage` is added in that harness package then, one hook per real event |

**What would change my mind:** a Codex or Cursor wire event carrying a limit reading. Then the
option is cheap and belongs in that package.

<a name="d2"></a>
## D2 · A turn's spend is its request's ledger at turn end, folded by `taskSpendView` and kept on the turn record

| | |
|---|---|
| **Instead of** | A feed callback on `HarnessSessionFeeds` the harness calls with its usage; or a sum over the turn's items |
| **Because** | Every harness already reports its run into the request's ledger, from its `finally`, however the run ends. The session flow's request `onFinished` runs after the harness block returns and before the ending hands the ledger on, so `readRequestSpend(ctx)` there is the whole turn: a failed or stopped turn included. Folding it with `taskSpendView` gives a turn the same view a task has, so one component draws both (tenet 5: one fold). A feed callback would be a second report path beside `reportHarnessSpend`, kept in step by every harness |
| **Locks in** | A turn's count is final, not live: it appears when the turn ends. The record's `spend` has the view's shape, so a change to the view changes both readers at once |

**What would change my mind:** a page that must show a turn's tokens rising while it runs. Then
the turn writes its live part on a timer, as a task's request does (FIX-2288 D2).

<a name="d3"></a>
## D3 · A cost is the vendor's billed charge, summed, or `null`

| | |
|---|---|
| **Instead of** | `$0.00` for a subscription run, or a price-table estimate for a run with tokens and no charge |
| **Because** | FIX-2145 D2 and the epic forbid estimates. A subscription run is not billed per run, so a dollar figure for it would be invented. `costUsd` is the sum of the sources that reported one; `null` when none did, and the line then shows tokens only |
| **Locks in** | A mixed task (an API-key retry after a subscription run) shows the API-key part's charge, with tokens covering both. Apps never compute a cost |

## Decided, not asked

- **The turn's duration is `endedAt − at`.** `at` is already the turn's start; `endedAt` is
written in the same turn-end write as `spend`. A task's elapsed time is FIX-2394's, on FIX-2168's
row fields; the component takes `durationMs` so both feed it.
- **The registry component takes a structural summary**, the view's fields plus optional
`attempts` and `durationMs`, and no row type: a turn and a task call it the same way. A
harness label is the app's (`harnessLabel`), so the registry names no harness.
- **The session flow carries the usage query** (`harnessUsage`, readers from `usageReaders`,
default none). The collection is not client-readable, so a write with no query on the flow
is invisible.
- **Kitchen Sink installs the byte copy in PR 1**: its drift check requires every registry file.
- **A turn-end write that fails is a note, never a failed turn**, like the pull-request check.

## Considered and dropped

| Alternative | Why it lost |
|---|---|
| Keep each lab's own spend line | Four copies of one line, which ER-3 says go to the package |
| Put `costUsd` on FIX-2379's row view only | The row view carries the spend view (ER-4); the cost belongs in the fold it carries |
| Write the turn spend from a request-end hook | It runs after the request is over, so a page that refreshes on the turn's end can read before it lands; `onFinished` runs after the harness reported and before the request ends |

## Analysis

2026-10-31, read from `main` 675f614. **Right approach.** Value: the four apps' "what did it cost"
line. Reuse: every Adds row composes an existing fold or writer ([Builds on / Adds](SPEC.md#whats-already-here-and-the-debt)).
Scope: D1 narrows the issue's Codex/Cursor ask to what their wires carry (recorded). Debt: net
down after PR 5.

## How it got here

- **Drafted (Oct 31)** from the issue and epic FIX-2369's seams, with the Codex and Cursor wire
types read to settle D1.
156 changes: 156 additions & 0 deletions specs/issues/FIX-2398/SPEC.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,156 @@
# FIX-2398 · What each task and turn cost

**Spec** · [Decisions](DECISIONS.md)

## Five people, before and after

| Someone who… | Today | After |
|---|---|---|
| **picks a harness for a coding task in Teammates** | Sees the harness and model the last run used, and nothing about what it cost. Choosing by cost is a guess | Sees tokens, the vendor's charge when there is one, and attempts on the card and in the popup, and a total for the Background list |
| **runs a Codex or Cursor task in any app** | Tokens reach the task row already, but no app but Claude Code-style draws them | The same line every harness gets: Codex and Cursor tokens and cost through the same view as Claude Code's |
| **works a Kitchen Sink harness session** | No turn records what it spent, and a Claude Code window warning heard mid-run is dropped | Each turn records its tokens, cost and duration; the window reading lands in the person's usage record, readable through the session flow's usage query |
| **opens a task in Shift Manager** | The inspector's *Harness · tokens · cost* section is a "not built yet" gap | The section shows the task's spend line from the row |
| **runs a subscription-billed harness** | n/a | Sees tokens and *no dollar figure*, never an estimate |

The row already carries a task's spend (FIX-2288) and every harness already reports it. What is
missing is one way to draw it, the cost in the view, and the same record for a Kitchen Sink turn,
which has no task row.

## The goal, and how we'll know it's met

**Every app that shows a coding task or a harness turn shows what it spent, in the same words:
model, tokens, the vendor's charge when it reported one, and for a turn how long it took.**

| Is it the right goal? | |
|---|---|
| **The real need** | Pick a harness by what it costs, and see a run's spend wherever the run shows ([FIX-2398](https://linear.app/fixpoint-labs/issue/FIX-2398), epic [FIX-2369](../../epics/FIX-2369/SPEC.md)'s *see what the work is doing*) |
| **Smaller, and rejected** | "Teammates draws the row's tokens." Hittable in one lab file, and leaves Kitchen Sink recording nothing and three more copies of the same line |
| **Bigger, and not this issue's** | A task's elapsed time and stall signals: FIX-2394, on the `startedAt`/`endedAt` row fields FIX-2168 adds. This ships the slot they fill. Subscription windows for Codex and Cursor: neither vendor sends one during a run ([D1](DECISIONS.md#d1)) |
| **Not done if** | The view has a cost but the line never shows it · a turn's count is read off a scripted harness only · a subscription run shows `$0.00` · a turn that failed or was stopped records nothing |

```mermaid
flowchart LR
I["a real harness turn · harnessSessionFlow · api-key sign-in"] --> T["the turn's record in session state"]
T -->|"tokens > 0 and the vendor's cost, model named"| P["PASS · goal met"]
I --> U["a stopped turn"]
U -->|"its tokens recorded too"| P
C["control · no-turn-spend"] -.-> T
T -.->|"under the control"| F["must FAIL · turn spend is null"]
```

The check reads the turn record the page reads, through the flow's own state, never the
harness's handle.

| How we verify | |
|---|---|
| **Goal check** | `goals/harness-manager/a-turn-records-what-it-spent/` · real Claude Code on an API key (so a charge is reported) · run by the implementer at completion where the host can run Claude Code, else handed to `fsd-qa` · verdict in the implementation PR |
| **Signal** | **t**: after a completed turn, its record's `spend` names the model, `inputTokens + outputTokens > 0`, `costUsd > 0`, and `endedAt ≥ at`. **s**: a stopped turn's record carries `spend` too |
| **Input** | A one-line prompt with a fresh token to echo. Any other prompt must pass |
| **Anti-game** | Read the session state the page reads. Don't assert on `HarnessRunSpend` or a mocked ledger: both pass while the turn record stays empty |
| **Control that must fail** | `GOAL_CONTROL=no-turn-spend` (the flow skips its turn-end write). **t** and **s** FAIL on `spend` being `null`. Today's `main` fails the same way |

## What changes

```mermaid
flowchart LR
H["claude-code · codex · cursor<br/>reportHarnessSpend (unchanged)"] --> L["the request's spend ledger"]
L -->|"task runs (FIX-2288)"| R["task row spend"]
L -->|"turn end, new"| TR["turn record spend + endedAt"]
R --> V["taskSpendView · + costUsd"]
TR --> V
V --> S["registry Spend · one line"]
S --> A["Teammates · Shift Manager · CC lab · Kitchen Sink"]
CC["Claude Code onUsage"] -->|"session feeds, new"| HU["harness-usage record · harnessUsage action"]
```

One fold (`taskSpendView`) for a task and a turn, one component for every app. A turn's spend is
the request's own ledger at its end ([D2](DECISIONS.md#d2)).

**A host, after:**

```diff
harnessSessionFlow({
kind: "claude-code-session",
workspace,
+ // The session flow's `harnessUsage` query reads what `onUsage` records. Optional.
+ usageReaders: { "claude-code": () => readClaudeUsage({ env }) },
agent: (feeds) => claudeCodeAgent({ ...feeds, permissionMode: "bypassPermissions" }),
});
```

```diff
- <Spend task={task} split harness /> // a lab's own copy
+ <Spend spend={row.spend} attempts={row.attempts} split harnessLabel={label} /> // the registry's
```

**Docs:** harness-manager README (the turn record's `spend` and `endedAt`, the `onUsage` feed,
`usageReaders`), orchestration README (`costUsd` on the spend view), ui README (the `spend`
item). No new docs-site page: the harness session and task-board pages gain one line each.

## The PR plan

| PR | What | Base |
|---|---|---|
| 1 · package slice | `taskSpendView.costUsd`; the session flow's turn spend, `onUsage` feed and `harnessUsage` action; the registry `spend` item and Kitchen Sink's byte copy | FIX-2387's implementation branch (the turn-end region), else `main` |
| 2 · Kitchen Sink | `harness-session.tsx` draws the latest turn's line | After 2410 → 2386 → 2384 (wave 2) |
| 3 · Shift Manager | the inspector's section and Tasks' COST column | After PR 1 and FIX-2418 |
| 4 · Teammates, stack 3 | card, popup and Background total; Cursor gets `onActivity` | After 2380 → 2382 (wave 4) |
| 5 · Claude Code-style | its `Spend.tsx` becomes the registry copy | After PR 1 |

## What stays as it is

- How a harness reports: `reportHarnessSpend` and the request's ledger, untouched. Codex and
Cursor already report tokens and cost through it.
- The task row's `spend` field and its writes (FIX-2288). Only the fold gains a field.
- `recordHarnessUsage`'s rules. The session flow is one more writer through it, as the manager
and the session door already are.
- FIX-2145 D2: a cost is only ever the vendor's billed charge.

## What's already here, and the debt

| Already here | What this does with it |
|---|---|
| `taskSpendView`, the fold of a row's spend (FIX-2288) | **Extends**: `costUsd`, the sum of reported charges |
| `requestSpendSources`, `readRequestSpend` (FIX-2145, FIX-2288) | **Builds on**: the turn's spend is the request's pending spend at its end |
| `recordHarnessUsage`, `harnessUsageCollection`, `harnessUsageAction` | **Reuses**: the session feeds call the one writer; the session flow carries the one query |
| Claude Code-style `Spend.tsx` | **Replaces** (PR 5): lifted into the registry, taking a view, not a lab row |
| The turn record (`harnessTurnRecordSchema`) | **Extends**: `spend`, `endedAt` |
| Anything else that draws a task's tokens | **None** but CC lab's: searched `tokenCount`, `spendLine`, `inputTokens` across `labs/`, `apps/`, `packages/*/src` |
| FIX-2288 D3, model and harness from the run's report | **Kept** |
| Epic ER-4, consume the owner's artifact | **Kept**: the row view is FIX-2379's; this adds to the spend view it carries, not a second view |

| Adds | Existing piece considered | Why it doesn't fit / how composed |
|---|---|---|
| `TaskSpendView.costUsd` | `TaskSpendSource.costUsd` | Composed: summed from the sources the view already folds |
| Turn `spend` and `endedAt` | `taskSpendView`; the task row | Composed: the turn stores `taskSpendView` of its own request's sources. A turn has no task row |
| `HarnessSessionFeeds.onUsage`, `usageReaders` | The manager's and the door's `onUsage` | Composed: the same one-line call into `recordHarnessUsage` |
| Registry `spend` item | CC lab's `Spend.tsx` | Lifted, so four apps stop copying it |

| The debt | |
|---|---|
| **Removes** | (PR 5) Claude Code-style's own `Spend.tsx` and `tokenCount` |
| **Adds** | None: every addition reuses an existing fold or writer |
| **Leaves** | Two spend shapes and two cost rules for one run (FIX-2330). The manager run row's own token count (FIX-2298) |

**Net: down** once PR 5 lands: one copy goes, nothing second is added.

## Sign off

**[The goal](#the-goal-and-how-well-know-its-met), at that size:** every app's task and turn shows
model, tokens, the vendor's charge and (for a turn) its duration, through one component. If wrong:
we ship one more lab copy, or hold this for task timing FIX-2394 already owns.

**[The debt](#whats-already-here-and-the-debt), net down:** a lab copy goes; no second fold. If
wrong: a fifth spend line someone keeps in step.

1. **[D1](DECISIONS.md#d1) · Codex and Cursor get no `onUsage`.** Their tokens already reach the
row; neither vendor sends a window reading mid-run. If wrong: a feed with nothing to carry, or
Teammates still missing Codex's windows, which a reader supplies.
2. **[D2](DECISIONS.md#d2) · A turn's spend is its request's ledger at turn end, folded by
`taskSpendView`.** If wrong: a second fold, or a harness callback beside `reportHarnessSpend`.
3. **[D3](DECISIONS.md#d3) · A cost is the vendor's charge or nothing.** If wrong: a subscription
run reads `$0.00`, or an estimate nobody billed.

**Open: none.** Number 1 is the one to weigh.

Feature · `orchestration` + `harness-manager` + `ui` registry, then four app PRs · small package slice · epic FIX-2369
Loading