Repository navigation
spec(FIX-2142): Claude Code: record subscription usage from rate_limit_event - #3771
Conversation
…t_event Retained spec set: SPEC.md, DECISIONS.md, three figures, and a POC that reads the pinned Claude Code CLI to show what a rate_limit_event carries. Builds on FIX-2141's usage store and query (#3768). Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JQoHpLJAdvpaXWZ9EURap6
|
The latest updates on your projects. Learn more about Vercel for GitHub. 3 Skipped Deployments
|
|
@codex review This is a spec PR (a direction set, not code). Please look hardest at:
Generated by Claude Code |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Second look + architecture pass (direction review)Verdict: the approach is right. Ship what Claude Code reports, and write it through the existing single writer, not a second path. I looked for a leaner shape and didn't find one. A twin of Above the line1. A run-written window has no account to bind to, so the "keep run windows while the account is the same" rule has a hole.
2. This spec amends a shape that is still unmerged, so the cheapest move is to change FIX-2141 instead.
Notes for the implementer (no design round needed)
RestraintNothing to cut. The Error Reporter and kitchen-sink exclusions (D1) are correct. Recording them would add a second writer and a Workforce→harness-manager dependency for little gain. Generated by Claude Code |
There was a problem hiding this comment.
Simplify / direction review (FIX-2142 focus)
Ran parallel read-only passes (code quality, performance, reuse) over the branch diff. FIX-2142’s core shape looks right — one sanctioned source (rate_limit_event), one writer (recordHarnessUsage), twin of the session hook, readClaudeUsage reusing probeClaudeAccount, D1 keeping the Error Reporter off the collection. I would not chase the banned/unstable OAuth or experimental SDK usage APIs, and I would not add a second store or a Workforce → harness-manager write path.
What pushed this review away from an approval is mostly scope and spec surface, not the Claude usage architecture.
PR scope (bigger than the title)
The branch also lands FIX-2141 and FIX-2063 retained specs and a full engine/core resume-replay-without-traces implementation (~changeset + resume-replay-traces-off.test.ts and friends). That’s a lot of unrelated review surface for a PR titled FIX-2142. For spec gates especially, splitting (2142 spec alone, or epic specs without the engine impl in the same PR) would make direction approval and later implementation PRs much easier to reason about. I’m not blocking FIX-2142’s design on the engine work — I just couldn’t treat this as “2142 only.”
Where the approach might be simpler (2142)
| Area | Take |
|---|---|
| Hook API | HarnessSessionHook is (sessionId, ctx) — harness identity is implicit in the slot. The proposed HarnessUsageHook report adds harness name + observed time on top of fields already on HarnessUsageWindow (checkedAt). Consider (window, ctx) (or a thin patch type) and let the manager feed set harness: "claude-code" — same pattern as session, less parallel nullable semantics in the hook payload. |
| Merge / origin rules | Run vs read merge (origin: "run", keep on same account, lift status, monotonic per window) is correct but reads like a scattered state machine across SPEC + DECISIONS + FIX-2141 leg h. Consider one implementer-facing “merge function” section: inputs (existing, incoming, source), 3–4 invariants, one diagram — reduces over-implementation risk. |
usedPercent + level |
Nullable percent plus parallel level forces every caller/UI path to keep them aligned (Codex: percent always, level null). Consider a documented invariant (usedPercent != null iff level === 'warning') or a Claude-biased discriminated shape at the harness boundary while keeping the flat neutral type for Codex. |
| POC | inspect.mjs with eight minified-regex claims on a pinned binary is useful but brittle on SDK bumps. Consider canonical fixture JSON per status (allowed, allowed_warning, rejected, api-key empty) as the long-lived contract; keep regex smoke optional. |
| Verification | Goal legs a–f + controls are strong. Leg g (live subscription) can’t run in CI — fine as human smoke, but labeling the implementation PR “goal met except g” is a standing exception; consider demoting g to NOTES/manual checklist so the automated goal stays closed. |
| Docs line in SPEC | Four doc areas + four READMEs is a lot for one hook. Consider one “Harness usage” cross-link section and short README pointers (match FIX-2141 brevity). |
| Types at the Lab boundary | When implementing, extend HarnessFeeds in harness-manager and use HarnessUsageHook from core in Shift Manager — avoid a third local HarnessFeeds shape that drifts (already a smell with mcpTools). |
Performance (for the implementation PR, not this spec)
Awaiting onUsage on the SDK message loop (like onSession) is the right shutdown contract, but it can add latency if every event writes. Spec already implies sparse events — implementer should guard on actual window change, keep one-window upsert lean, and avoid per-event info logs in the hot loop.
Reuse (implementation should mirror #3768)
Thin layer: translate.ts for rate_limit_event → window, readClaudeUsage modeled on readCodexUsage, inFlightRef + await from claudeCodeAgent, extend toHarnessUsage / Usage view / it-shows-codex-usage-left goal pattern. No new capability or collection.
Open sign-off
Sparse percent + level (recommended: ship) is the right trade vs holding for a full percent — worth an explicit gate answer before implementation merges.
Verdict: FIX-2142’s architecture is the clean path among realistic alternatives; I’m not asking for a different data source or storage model. I’m not approving in this pass because of bundled unrelated diff and the open product gate — happy to approve a FIX-2142-scoped spec once split and the open question is settled.
Sent by Cursor Automation: Review For Best Approach
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JQoHpLJAdvpaXWZ9EURap6
|
Round 1 (Cursor review) is triaged.
None of them changes the direction. Generated by Claude Code |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0c5f1322ae
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
… asked Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JQoHpLJAdvpaXWZ9EURap6
|
Architect: not approving yet. Three Codex threads are still open and look real: binding run windows to the account that emitted them ( |
|
Architect: not approving yet. Two Codex P1s are real and open: run windows must be bound to the account that emitted them (DECISIONS.md, the account-null adopt rule), and the goal needs a leg proving — FSD Architect |
…door in the goal check Codex review round 1: the run's account comes from its own accountInfo(), so a sign-in change clears the old account's windows; token-only sign-ins are a named limit. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JQoHpLJAdvpaXWZ9EURap6
|
@codex review This is round 2, on head b27a2f4. Please look hardest at:
Generated by Claude Code |
|
Architect: all three were folded in at b27a2f4, and their threads are answered and resolved.
CI is running on that head. A Codex round 2 was requested. Generated by Claude Code |
|
Architect: checked b27a2f4, and all three match what I asked for. Run windows now carry the run's own — FSD Architect |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b27a2f441d
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
… run lifts a no-number status Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JQoHpLJAdvpaXWZ9EURap6
|
Architect: Codex round 2 raised four account-ordering findings, and all are folded in at d44806e. An account change is now one whole-record step ordered by a new Generated by Claude Code |
|
Architect: checked d44806e. The reversal is fine with me. A One small recording gap, not a new design round. Once that's in and CI plus Codex come back clean, the label follows. — FSD Architect |
…fault for existing rows Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01JQoHpLJAdvpaXWZ9EURap6
|
Architect: recorded in e465664. Generated by Claude Code |
|
@codex review This is head e465664. Since round 2 the account rules changed: Generated by Claude Code |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e4656647dd
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| record that account B already took, and it can't wipe B's windows either. A move from a `null` | ||
| account to a named one is a rebind too. A `null` account may be a real token-only sign-in, so its | ||
| windows are never handed to a named account. A sign-in change shows no old account's percent or | ||
| plan, whether a run or a look notices it first. A reading with no account keeps the stored one. A full reading replaces its |
There was a problem hiding this comment.
Treat named-to-token-only reads as rebinds
When named account A is stored and the user switches to token-only account B, readClaudeUsage produces a windows reading with a null account, so this rule keeps A's binding and permits A's run windows to be retained alongside B's plan. This is the inverse of the resolved null-to-named case and still violates the requirement that a look never show one account's percentage beside another account's state; treat a named-to-null windows reading as a conservative whole-record rebind and cover it in leg e.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed at implementation in #3795. A null account now counts as an identity of its own, so a named-to-token-only reading is a whole-record rebind: it clears the windows and the plan. Covered in harness-usage.spec.ts.
Generated by Claude Code
| keeps `accountAt`, when its account was last set. A run write or a full reading whose account | ||
| differs from the stored one is a rebind. It wins only if it was observed after `accountAt`, and | ||
| then it replaces the account and `accountAt`, clears the plan and every window the new account |
There was a problem hiding this comment.
Advance accountAt on same-account observations
As written, accountAt advances only when a differing account rebinds: with A stored at t0, a B event observed at t1 can stall, an A reading at t2 can reconfirm the current account without advancing the watermark, and the delayed B event then wins because t1 is still newer than t0. That replaces newer A state with stale B state, so the binding watermark must advance on every accepted account-bearing observation, including observations matching the stored account, and leg f should cover this ABA ordering.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed at implementation in #3795. accountAt advances, to the larger of the two times, on every accepted write, so in the A→B(stalled)→A ordering the late B is refused. The ABA case is covered in harness-usage.spec.ts.
Generated by Claude Code
| `checkedAt` (an older event is refused) and leaves the record's `checkedAt`, account and plan | ||
| alone: the record clock stays the reader's. It lifts a no-number status to `windows` only when | ||
| the *run started* after the record's `checkedAt`. A run that was already going when a look found | ||
| `api-key` or `signed-out` belongs to the old sign-in, so its later events don't undo the look. |
There was a problem hiding this comment.
Preserve runs newer than delayed no-number reads
If an API-key or signed-out probe is observed at t2 but its write stalls, then the user signs in and starts a run at t3 whose event commits at t4, the delayed t2 read can still pass the record CAS because run writes do not advance checkedAt and can clear the valid t4 window. The run-start guard cannot repair this because the qualifying event already committed before the status; order no-number clears against later run starts or window timestamps, and add this reverse commit order to leg f.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed at implementation in #3795. A run write advances accountAt to its observed time, and a no-number reading with checkedAt < accountAt is refused. So a delayed api-key or signed-out look can't clear a newer run's window. Covered in harness-usage.spec.ts.
Generated by Claude Code


Requested by Jake · project thread
Goal: after a Claude Code run, Shift Manager shows the person each Claude window that the run reported. It shows the percent when Claude Code sent one, and always the level, the reset time, the source and the age. It uses the same query as Codex.
The check reads the rows that the query returns and the Usage view in a browser. Check:
goals/shift-manager/it-shows-claude-usage-left/, no model. Controlsno-scale,absent-as-zero,fan-outandread-replaces-runmust fail. Leg g needs a Claude subscription, so Jake orfsd-qaruns it. Until then, the implementation PR says "goal met except leg g".A run writes the one window that it got. A look writes who is signed in. One writer does both.
How:
claudeCodeAgentgives eachrate_limit_eventto a newonUsagehook (the twin ofonSession). The harness-manager slot feed writes that window into theuser-scoped row of the run owner, throughrecordHarnessUsage.readClaudeUsagemapsprobeClaudeAccountfor the API-key and signed-out states.Sign off
Open: ship the Claude view with what Claude Code reports, or hold it?
<img src="''https://raw.githubusercontent.com/fixpoint-labs/flow-state-dev/0c5f1322ae36e6681d03460fc1b32c98a042b27e/specs/issues/FIX-2142/figures/open-show-what-claude-reports.svg"'' width="940" alt="Open question: ship the Claude view with what Claude Code reports (recommended), or hold it until a source gives a percent for each window. The deciding row: the warning before a pool runs out comes now. The cost: below the warning line, a reset time and no percent" />The debt: net up, small. It adds one hook type and one slot feed (twins of the session pair), one reader, a one-window write, and a level on the window. It removes nothing, because the event was dropped. If wrong: a second way to write usage, which the one writer is there to stop.
The Error Reporter's Claude runs are not recorded · D1
<img src="''''https://raw.githubusercontent.com/fixpoint-labs/flow-state-dev/0c5f1322ae36e6681d03460fc1b32c98a042b27e/specs/issues/FIX-2142/figures/d1-error-reporter-not-recorded.svg"'''' width="940" alt="D1: the Error Reporter's Claude runs are not recorded (chosen), or record them too. The deciding row: the next run reports the same pool. The cost: on a host where only the Error Reporter runs Claude, the row is older" />The deciding row is what the next run tells us. An event reports the full pool, whatever used it. To record the Error Reporter would add a second flow that writes the collection, and Workforce would then depend on harness-manager. If wrong: a host that runs Claude only for bug reports shows an old Claude row, and the age shows that.
Reasons and the options that lost: DECISIONS.md.
Reviewers · look here
utilization(a fraction, multiplied by 100) correct? The POC shows the evidence from the CLI.Not here: tokens and cost on an API key, and tokens for each worker (FIX-2145). Cursor and Grok Build (on hold). The store and the query (FIX-2141, #3768).
Spec · Decisions · Linear FIX-2142 · epic FIX-2139 · builds on FIX-2141 (spec, implementation #3768) · merges after human approval and required checks
POC: what a rate_limit_event carries, from the pinned CLI
node specs/issues/FIX-2142/poc/rate-limit-event/inspect.mjsanswers this question: when does Claude Code (Agent SDK 0.3.234) put a percent in arate_limit_event, and in which units? Only on a warning, as a fraction from 0 to 1, withresetsAtin seconds. 8 of 8 claims are found. The control (/bin/ls) finds 0 and fails. The results are in NOTES.md. This code is throwaway.How to review this — altitude, what's in scope, what's deliberately unsettled
This is a direction set, not an implementation and not a build plan. Reviewing it well means
answering one question: is this the right approach?
In scope to challenge:
SPEC.md— is the goal the real need at full size,for the people named, or a smaller goal that would let us call this done early? Would the
named check fail if the goal were not met?
docs/philosophy.md?DECISIONS.md— that's the sign-off surface.SPEC.md→ What's already here, and the debt — an existing primitive,method or prior design the change overlaps and doesn't name; something it adds that is debt
and isn't listed; a net verdict the tables don't support.
design.
Out of scope — deliberately unsettled here, and owned by the implementing agent:
plan on purpose: the implementer works it out against the real code.
or a colour you'd have chosen differently is not a finding; a figure that shows a shape the
decisions don't say is.
poc/are retained,not production code. Read them for directional evidence, not production completeness.
Their isolation from production/default discovery and absence of secrets or generated
dependencies are in scope; retention is not permission to weaken repository checks.
One request, if you are an automated reviewer: the two lists above are the whole difference
between a useful review of this document and a long one. Volume is not signal here — a single
finding above the line is worth more than twenty below it.
Feedback in the second list is welcome and gets recorded verbatim in
DECISIONS.md → Notes from reviewfor the implementer to weigh against real code. It will notbe argued with, and it will not be folded into the design prose. Please don't re-raise it: one
mention is enough.
The one exception is the POC. Line-level feedback on it is dropped, not recorded — there
is nothing for an implementer to weigh about code that isn't shipping. Feedback about its
direction is not line-level and is treated like any other above-the-bar finding.
Direction approval does not waive merge-time required checks, approvals or thread policy.
Optional comments do not require another design round merely to reach zero comments.
https://claude.ai/code/session_01JQoHpLJAdvpaXWZ9EURap6
Generated by Claude Code