Repository navigation
FIX-2398: a task's and a harness turn's spend, with the vendor's charge (package slice) - #4423
Conversation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NczDiDfgRv9riWiVVdj8JG
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NczDiDfgRv9riWiVVdj8JG
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NczDiDfgRv9riWiVVdj8JG
…t (FIX-2398) - taskSpendView gains costUsd: the sum of vendor-reported charges, null when none (a subscription run reports none; never an estimate). - harnessSessionFlow records each turn's spend (taskSpendView's shape) and endedAt at turn end, however the turn ended; hands harnesses an onUsage feed into the usage record; carries a harnessUsage action (usageReaders). - ui registry gains a spend item, lifted from Claude Code-style's Spend; Kitchen Sink installs its byte copy. - Goal check goals/harness-manager/a-turn-records-what-it-spent: PASS on real Claude Code; GOAL_CONTROL=no-turn-spend FAILs t and s. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NczDiDfgRv9riWiVVdj8JG
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NczDiDfgRv9riWiVVdj8JG
Turn spend joins FIX-2384's turn-end sequencer as one more step; the flow keeps one onFinished. Kitchen Sink installs run-changes.tsx so its registry drift check passes on the stacked branch. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NczDiDfgRv9riWiVVdj8JG
…e request publishes its end (FIX-2398) onFinished runs after runAction emits the terminal status, so a page that refreshes on the end could read the turn before its spend landed (Codex, spec PR #4398). The spend now lands in a tap after the harness step, or in recordFailure's write for a turn that threw. A test captures session state when request.completed is persisted. The goal's control is now main. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NczDiDfgRv9riWiVVdj8JG
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NczDiDfgRv9riWiVVdj8JG
Codex: onFinished runs after the terminal status is published, so D2's write moves inside the turn and the goal's control becomes main. Cursor's scope notes answered in DECISIONS. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NczDiDfgRv9riWiVVdj8JG
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NczDiDfgRv9riWiVVdj8JG
…to claude/project-thread-t857lk # Conflicts: # apps/kitchen-sink/test/harness-turn.test.ts
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NczDiDfgRv9riWiVVdj8JG
…updated run-changes The parent branch moved both registry files without updating the byte copies. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NczDiDfgRv9riWiVVdj8JG
|
The latest updates on your projects. Learn more about Vercel for GitHub. 4 Skipped Deployments
|
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
jhoffner
left a comment
There was a problem hiding this comment.
Second look + architecture pass. The direction is sound: writing spend inside the turn (not onFinished) is the right call, and reusing taskSpendView keeps turns and tasks reading alike. One refactor is worth taking now, while this PR is already touching the seam:
- Add
requestSpendView(spend, requestId)in orchestration and have bothtaskSpendViewandturnSpendOfuse it, instead of fabricating a fake completed Task with ten nulls in harness-manager (inline comment onturnSpendOf). - Remove the
ctx as nevercast inonUsage. - Make
turnSpendandrecordFailureidempotent onendedAt. - Check
costUsdconsumers use== null, since the field is optional on stored views.
I did not run the suites; this review is a read of the diff. Not flagged: the byte-copied spend.tsx (ui registry and Kitchen Sink) is the registry's intended drift-checked model, and the stacked run-changes/run-checks copies are justified by the red drift check.
Generated by Claude Code
There was a problem hiding this comment.
/simplify review (FIX-2398 package slice)
Verdict on approach: The spec’s direction looks like the right simplification for this epic: extend the existing taskSpendView fold (plus costUsd), store the same shape on a turn record via the request ledger at turn end (D2), and expose one registry Spend line for the four app PRs. That beats the rejected alternatives (per-harness callback spend, item sums, or another lab-local formatter). Reusing recordHarnessUsage / harnessUsageAction for session onUsage matches the manager and session door — no second usage writer.
Where complexity crept in anyway
-
Stacked work in one diff — This PR merges FIX-2387/FIX-2384 kitchen-sink registry copies (
run-changes,run-checks) plus turn spend,harnessUsage, and the goal script. The commits explain why (base branch + drift count), but the review surface is ~1.2k lines where the spend mechanism is a few hundred. Worth calling out in the PR description so reviewers don’t treat the run-review UI as FIX-2398 scope. -
turnSpendOfas an inline adapter — Building a synthetic one-requestTaskSpendinsidesession-flow.tsduplicates orchestration’s entry shape and leans on casts / “last harness run / last model key” heuristics. A small exported helper in orchestration (e.g. fold request spend for a single request id) would shrink harness-manager and give workforce/tests one place to anchor — optional for this PR, but it’s the main “could be cleaner” refactor I’d consider. -
SpendSummaryvsTaskSpendView— The registry component re-declares the fold type. ImportingTaskSpendView(and declaring orchestration as a registry dependency if needed) would avoid drift when the schema grows; the byte-identical kitchen-sink copy + drift test is the right enforcer either way. -
Error handling asymmetry on turn end —
turnSpendswallows state-write failures and emits status (“never fail the turn on spend write”), butrecordFailureapplies the sameturnEndOfpayload inside an unguardedchangeState. If that write throws, behavior diverges from the comment and may interfere with rethrowing the harness error. Worth aligning (shared helper or the same try/catch + status emit on both paths).
Performance: Nothing alarming for FIX-2398 — costUsd is O(sources) on once-per-turn paths. Tail latency is still dominated by harness runs and optional verify / git reads (pre-existing or FIX-2387), not the fold or Spend UI.
Third-party libs: Custom costAmount is appropriate (subscription runs must show no dollar figure; generic currency formatters don’t encode D3). No npm win here.
Follow-ups (spec already plans): CC lab Spend.tsx stays duplicated until PR 5; app PRs 2–4 should install registry Spend rather than formatting again.
Tests / goal: turn-spend.spec.ts and the goal script encode D2 ordering well (spend before terminal status). Nice anti-game choice.
I’m not approving yet mainly because of the recordFailure vs turnSpend write invariant; everything else reads as nits or follow-up extractions. Happy to approve after that path is consistent or you document why a throw path should propagate store errors.
Sent by Cursor Automation: Review For Best Approach
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7b7e8281a9
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
… requestSpendView Review on #4423: the session flow no longer hand-builds a task spend entry; orchestration owns the request-to-view step. The turn-end write happens once (a record that already ended keeps its end), and onUsage needs no cast. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NczDiDfgRv9riWiVVdj8JG
Turn spend stays a step inside the turn, before 2384's in-turn head write. Kitchen Sink takes the parent's run-changes and run-checks copies; the drift count is 40 with spend. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NczDiDfgRv9riWiVVdj8JG
There was a problem hiding this comment.
Code Snob
turnSpendOf builds a one-request task row by hand (last harness run, last model key, blank writer / admittedAt / recovered), then folds it with taskSpendView. spendAfterWrite on a blank row with part: "settle" is that row. The fold stays taskSpendView with status: "completed", so a turn still reads like a finished task. No new export.
Fold that subtraction in from #4425 (poc/shorter-turn-spend, draft, base claude/project-thread-2ksmgs). Diff vs 7b7e828 is packages/harness-manager/src/session-flow.ts +4/−16.
What still passes: pnpm --filter @flow-state-dev/harness-manager exec vitest run --root . test/turn-spend.spec.ts (6 passed). The hand-built row and this fold return the same view for an empty ledger, an API-key run, a subscription run, model calls only, a harness run mixed with model calls, and two api-key runs of one harness plus a subscription run.
Agent on this PR: fold the subtraction into this branch, then close #4425 unmerged after this PR lands. The POC does not merge.
Left alone: the registry Spend component and its Kitchen Sink byte copy, onUsage / harnessUsage, the two write sites (turnSpend notes a failed write; recordFailure rethrows), and the parent-branch run-checks / run-changes installs. Did not re-litigate D1–D3.
Sent by Cursor Automation: Code Snob
Codex on #4423: unrecovered() subtracted the recovered tokens and runs but kept the source's full costUsd, so the continuation's report of the logged run's charge counted twice. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NczDiDfgRv9riWiVVdj8JG
There was a problem hiding this comment.
Code Snob
requestSpendView builds a one-request task row by hand (sources, namesOf, blank writer / admittedAt / recovered), then folds it with taskSpendView. A blank row plus spendAfterWrite (part: "settle") is that row. The export stays. turnEndOnce stays.
The equality assert next to it compared the function to that spendAfterWrite call. Once the function is that call, the assert cannot fail, so it comes out. The view match and the empty-ledger null stay.
Fold that from #4425 (poc/shorter-turn-spend @ 78a103c73, draft, base claude/project-thread-2ksmgs). The commit is the subtraction: packages/orchestration/src/tasks/spend.ts and record-spend.test.ts, +6/−22 vs 7dc52024. The PR title still names turnSpendOf; read the commit.
What still passes: pnpm --filter @flow-state-dev/orchestration exec vitest run --root . test/collection/record-spend.test.ts (28 passed) and pnpm --filter @flow-state-dev/harness-manager exec vitest run --root . test/turn-spend.spec.ts (6 passed).
Agent on this PR: fold the subtraction into this branch, then close #4425 unmerged after this PR lands. The POC does not merge.
Left alone: turnEndOnce, onUsage / harnessUsage, the registry Spend byte copy, and the run-checks / run-changes installs. Did not re-litigate D1–D3.
Sent by Cursor Automation: Code Snob
Cursor on #4423: the hand-built entry duplicated spendAfterWrite's; a request's view is now the settle write and fold a task's row gets. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NczDiDfgRv9riWiVVdj8JG
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2fac9196b5
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Codex on #4423: the once-only guard kept the interrupted run's end and dropped the continuation's spend, and a continued request's ledger holds only its own runs. The turn record now keeps the runs it counts (spentRuns), each once by runId, and every write takes the latest end. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NczDiDfgRv9riWiVVdj8JG
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5378fad9cf
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Mechanism check (advisory, never blocks)Rows from
If an export is a new mechanism, add a row. CI filled the mechanical fields; - id: TODO
job: TODO
package: "@flow-state-dev/core"
anchor: readRequestTotalSpend
file: packages/core/src/types/request-end.ts
builtOn: [] # TODO
overlaps: [] # TODO
status: canonical
doc: TODO
- id: TODO
job: TODO
package: "@flow-state-dev/orchestration"
anchor: requestSpendView
file: packages/orchestration/src/tasks/spend.ts
builtOn: [] # TODO
overlaps: [] # TODO
status: canonical
doc: TODO |
A turn that wrote its spend and then lost its process before its step's trace was stored replays that write on continuation, while the ledger recovers the same id-less run from the harness step's trace. The turn record kept every id-less earlier run, so that run counted twice. The engine now gives an id-less report a settled_ id as it counts it, so the ledger, the trace and what a continuation recovers all name the run the same, and the turn dedupes it by id like any other run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NczDiDfgRv9riWiVVdj8JG
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NczDiDfgRv9riWiVVdj8JG
|
Kitchen-sink E2E (Playwright) failed on 7d92efe during setup, before any test ran. The error was Generated by Claude Code |
…to claude/project-thread-t857lk
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NczDiDfgRv9riWiVVdj8JG
…to claude/project-thread-t857lk
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: be2089d02a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
A disconnect ends the request's first run, whose ending hands its model calls on; the continuation replays the finished calls without making them again, so the pending slice no longer holds them and the turn's spend lost them. The turn now reads the request's whole model count (ctx.request.tokenUsage.byModel), which carries what earlier endings handed on and counts recovered calls once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NczDiDfgRv9riWiVVdj8JG
…to claude/project-thread-t857lk
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 62a28169fc
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…eads its total A run reported before a suspension (or an interruption) was settled by that ending and never seen again: the resumed ledger carried only model calls, and the turn's spentRuns only covered a write that the failure path made. The request record now carries spentHarnessRuns beside spentByModel, the ledger starts its total from them (never handing them on again), and core's readRequestTotalSpend reads the whole request's spend. The harness turn reads that total, so its spentRuns carry is gone. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NczDiDfgRv9riWiVVdj8JG
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 12b1a3e917
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…pted ending's logged runs are carried Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NczDiDfgRv9riWiVVdj8JG
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4675799a1f
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
…ly its charge changed Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NczDiDfgRv9riWiVVdj8JG
…to claude/project-thread-t857lk
…to claude/project-thread-t857lk
…to claude/project-thread-t857lk


Requested by Jake · project thread
Stacked on FIX-2387's branch (
claude/project-thread-2ksmgs, which carries FIX-2384). Retarget tomainwhen it merges. Spec: #4398 (not merged yet; its commits are merged into this branch).This is PR 1 of the spec's plan, the package slice. The four app PRs (Kitchen Sink, Shift Manager, Teammates, Claude Code-style) follow in their waves.
spend(model, tokens, vendor charge) andendedAt, however the turn ended. It is stored before the request says it is doneharnessUsageactioncostUsd: the vendor's charge, ornull. Never$0.00for a subscription runSpend.tsxfsdev ui add spend: one line for a task or a turnWhat changed
orchestration:taskSpendViewgainscostUsd, the sum of the non-null charges that sources reported (D3).requestSpendViewreads one request's spend the same way.harness-manager:harnessSessionFlowwrites the turn's spend andendedAtinside the turn (D2). The spend isrequestSpendViewof the request's total spend. The write happens:turn-spendstep after the harness step, or inrecordFailure's write when the turn threw or was stopped;onFinished, which runs after the terminal status is published (Codex, spec round 1).engine/core: a request keeps its whole spend across a suspension, an interruption or a crash. Changeset: engine and core, minor.spentHarnessRunsbesidespentByModel. A resumed ledger's total starts from them and never hands them on again.readRequestTotalSpend(ctx)reads that total.runIdgets asettled_…id, so a run recovered from the log keeps the same identity.onUsage, which callsrecordHarnessUsage. The flow carriesharnessUsage: harnessUsageAction({ readers: usageReaders ?? {} }).uiregistry: a newspenditem, lifted from Claude Code-style. It takes a structural summary plus optionalattemptsanddurationMs.run-changes.tsx(FIX-2384) andrun-checks.tsx(FIX-2387). Without them its drift check is red on the parent branch (count now 40).ubuntu-24.04, ported from FIX-2438 (fix(ci): pin Kitchen-sink E2E job to ubuntu-24.04 (FIX-2438) #4431).Goal verdict
goals/harness-manager/a-turn-records-what-it-spent/, real Claude Code (claude-haiku-4-5, API key), on this branch:main(675f614) in a worktree FAILS t (no spend, noendedAt) and s (no spend).Verification
harness-manager387,orchestration2203,engine4223 (8 skipped),workforce1727, integration-tests 152,ui258, Kitchen Sink 396 (2 skipped). CI is green at ed81590.onFinishedfails the "stored before the request publishes that it completed" test;onUsagecall fails the usage-query test;nullcost as 0 breaks the view test;null;$0.00for anullcost breaks the registry test, and so does dropping the attempts guard.Not here
onUsage: neither wire carries a window event (D1).🤖 Generated with Claude Code
https://claude.ai/code/session_01NczDiDfgRv9riWiVVdj8JG