Repository navigation
FIX-2080: a task runs by the settings it was filed with, and nothing waits on work that will never finish - #3624
Conversation
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VaN4uTiYqiabCbsFLWM52A
…Task Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VaN4uTiYqiabCbsFLWM52A
|
The latest updates on your projects. Learn more about Vercel for GitHub. 3 Skipped Deployments
|
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: 6aea25e0-232b-4800-8f2e-1bfa694810f2) |
…l helpers Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VaN4uTiYqiabCbsFLWM52A
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: ff0ea72f-0391-46f7-bf0b-a4801c15c448) |
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: c7a12965-d0f6-4cfe-890c-b5fcbc0b0c63) |
jhoffner
left a comment
There was a problem hiding this comment.
Automated review, standing in for Bugbot
I reviewed the code at 7052203 against specs/issues/FIX-2080/ and the epic's DECISIONS, using the coherence, restraint, correctness and completeness lenses, and did a docs pass. Every defect below was reproduced with a scratch test that was not pushed, except finding 5, which comes from reading the replan code and the PR's own engineering call.
Verdict: the shape matches the spec. One status, a debt recorded in the ending's write, a cascade on every board, and one read map. Four defects are worth fixing before FIX-2078, FIX-2081 and FIX-2090 build on this. Two of them are the very failures the spec says this issue removes: a dependent stuck pending for good, and filed work cancelled when it shouldn't be.
| # | Severity | Where | What |
|---|---|---|---|
| 1 | must-fix | resource-backed.ts:694 (and :647) |
A dependency check made before create races a concurrent cancel of the dependency. The dependent is left pending for good, with no debt |
| 2 | must-fix | workforce/.../cascade.ts:84 |
A retry's debt paid late (crash window) cancels the next attempt's pieces, or the pieces of a task that then completed |
| 3 | must-fix | workforce/.../board.ts:671 |
Workforce addTasks files the earlier rows when a later row's settings are refused |
| 4 | must-fix | internal.ts:311 |
A stored blocked row with attempts > 0 reads as held, and release declines it as started. It is stuck (BP-030) |
| 5 | should-fix | plan-and-execute/index.ts:706 / apply-replan.ts |
A replan that names a failed task now fails the whole run instead of being skipped or sent back |
| 6 | should-fix | retired-shapes.ts:69 |
The migrated block reason goes into feedback, which is handed to the worker as failure feedback |
| 7 | optional | workforce/.../board.ts:278 |
A follow-up filed on the ref can run before its unstarted root, in the root's session. follow_up_of_unfinished now names a different rule |
| 8 | docs | task-substrate.md:106,129 |
Contradicts task-board.md on held and blocked-by-failures, states a history fact, and makes a batch claim that is false on Workforce |
Checked and fine
- The debt is written in the ending's own write on both backings:
applyTransitionandapplyAbandonmentSettlementwith the recorder, including the claim's abandon settlement. - Paying twice is harmless. Dependents are cancelled with
ifAllowed, and a filed cancel on a terminal piece leaves the debt for the next touch, which then settles it. settlematches ondue, so it never clears a newer debt.- The filing rules cover loops (within a batch and through a patch), self-dependencies, and ids within a batch. State-backed refusals are made inside the CAS.
heldis not claimable, andfailon a held row is refused. - BP-031:
session,resultTo,cascadeOwedandcausedByare top-level and written by the server. The "from a task session" test reads server state.FILED_MARKERis removed from caller metadata on both add and patch. I found no auth or routing decision taken from caller input. One coherence note: the spec says settings that drive routing stay out ofmetadata, while the filed marker (which decides where cancels go) lives there. It is guarded, but consider a top-level field when FIX-2083 touches this. - Both backings read every other retired shape correctly, and the read is a fixed point.
Engineering calls: I agree with all of them except the first, "Bad dependencies from a model now fail the run" (finding 5). That is right for the first plan and wrong for a replan. The resource-backed one-open-task race is admitted and acceptable for now. If two runs in one session would corrupt the session's task state, write that down as a follow-up.
Restraint notes, not blocking: every touch of a Workforce board now does payOwedCascades, which scans the whole board, and isStartable does an O(n) rows.some for each candidate. Cursor flagged the same thing on the spec. That is fine at today's board sizes; index it when it shows up in a profile.
Generated by Claude Code
There was a problem hiding this comment.
Simplify pass (FIX-2080 / PR #3624)
Parallel read-only review (code quality, performance, reuse). Verdict: the direction is the simpler architecture, not an over-built one. Deleting cascade-skip-dependents and paying cascades from the collection transition + cascadeOwed matches D2 and removes a second route patterns had to remember. filing.ts is the right convergence point for deps/settings on both backings. retired-shapes is heavier than a one-off blocked→held shim, but it matches FIX-2056 D3 and keeps BP-030 reads in one place; folding status-only entries into withMigratedStatus would be a smaller diff only if this issue were scoped to two shapes — here the registry pays off for the epic.
No must-fix simplification or correctness issues from this pass. Below: what I’d still tighten (mostly follow-up), and what I’d not chase.
What went further than necessary (actionable, mostly nits)
-
Filing convergence is almost complete.
TaskFilingRuleincludessetting_not_from_task_session, but only Workforceguarded.addTaskenforces it;assertFilingSettingsnever does. That contradicts thefiling.tsheader (“every add path… check a filing here”) and duplicates prose vsTaskFilingRefusedErrorelsewhere. Suggestion (optional before merge, fine as follow-up):assertFilingCallerContext(init, { fromTaskSession })infiling.ts, called from Workforce guarded add (and any future board that files from a task session). -
Resource-backed cascade owes two full-board scans on many endings:
owesCascadecallsopenDependentsOf(listAll(), …)andpayCascadelists again. At typical board sizes this is negligible; if boards grow, cache one snapshot per ending write or stamp debt wheneverendsDependents(ending)and let pay no-op. Not a reason to block. -
Near-duplicate
CascadeWritesadapters inresource-backed.tsandstate-backed.ts(~20 lines each). A tiny factory incascade.tswould keep the two backings aligned — nice cleanup PR, not FIX-2080 scope. -
internal.tsis very large (~1.3k lines) with FIX-2080 settings/cascade/held layered on transitions. Cohesive but hard to navigate; extractingsettingsWrite/ ending recording to sibling modules would be readability-only. -
Goal
runs-by-its-filing-settings/run.mts(~580 lines) is big but appropriate for end-to-end proof (kill leg, NDJSON proofs). Regex module patches are brittle by design for goals — I wouldn’t shrink this unless flakes appear.
What I would not simplify away
- Two cascade modules (
orchestration/.../cascade.tsvsworkforce/.../cascade.ts): substrate dependents + debt vscascadeFiledpayer — correct split, not duplication. - Custom DFS for dep loops in
filing.ts: board-local smalldeps[]; a graph library would add dependency weight without clarity. - Reviving a thin
cascade-skip-dependentswrapper: would reintroduce timing/crash semantics the spec rejected.
Third-party / structure smell check
- Directory layout under
tasks/collection/(filing,cascade, types) reads clean; workforce imports orchestration filing/errors, not the reverse — good boundary. blockedappears both inpersistedTaskStatusSchemapreprocess andRETIRED_SHAPES.blocked— intentional (parse vs read boundary); worth a one-line cross-reference in docs if confusion shows up in review, not a structural rewrite.
Performance (informational)
Hot-path costs are O(n) full-board lists on resource-backed cascade/claim paths. Acceptable for current scale; index or single-scan per write only if profiling says so.
Approval: Architecture matches the spec’s rejected alternatives; complexity tracks durability (D2), dual backings, and migration (D3). Remaining items are polish and follow-up dedupe, not a different approach.
Sent by Cursor Automation: Review For Best Approach
There was a problem hiding this comment.
Code Snob — already tight. No POC.
Read packages/orchestration/src/tasks/collection/cascade.ts, filing.ts, internal.ts (settingsWrite, recordCascadeOwed), both backings (resource-backed.ts, state-backed.ts), the bind in get-or-create.ts, packages/orchestration/src/tasks/schema/retired-shapes.ts, packages/workforce/src/conversation-board/board.ts and cascade.ts, and the hold path in task-tools-capability.ts.
A rewrite would not shrink this contract:
payCascadeis the one payer. Each backing only supplies its own reads and writes.endsDependentsandendsFiledWorkdiffer byretried. One function with a flag is longer at the two call sites.settingsWriteis the shared edit. The state backing's decline callback has to stay: a{ decline }returned from the patch is recorded as unchanged. The resource backing's extrasettingsDeclinebefore that call is two lines.assertDepsEditablecannot callassertDepsFileable. A patch whose deps are[own id, off-board]isdependency_loop; the batch check skips the row's own id and reportsdependency_not_on_board. Checking self first, then calling the batch check, flips that when the off-board id comes first.releaseandpatchSettingscopy the wake already onsetAssignee,dispatch, andunpark. A helper around those five is about as long as the tails it replaces. The post-row refusal and the optionaldispatchstay outside it.DeclaredFiledCascadeandTaskFiledCascadediffer byctxonpay. The bind is one line inget-or-create.ts.dependencyOrderis the insert order the guarded one-at-a-timeaddTaskneeds.ref.addTaskswould skip the chain, the assignee,markFiled, and the wake.- The six retired-shape entries are the map, including the four that are off.
owneris unread; deleting it does not shrink a read.
Left as recorded: D1, D2, the six-entry map, both cascades as one debt, the skipped label, and hold/release as the named entry points.
Sent by Cursor Automation: Code Snob
…edger lookup in Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VaN4uTiYqiabCbsFLWM52A
|
Architect: not approving yet. The branch conflicts with main, and Jake's four must-fix threads are open: the stuck dependent on a resource-backed board ( — FSD Architect |
|
@codex review |
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
Resolves the guarded add onto FIX-2076's reserveFiling: one set of filing checks (assertFilingCallerContext, assertFilingSettings, the worker's own step) runs for every row of a batch before its first add. withSteps moves to hold/release/patchSettings, refuses a step filed held, and checks a mixed batch whole before it files. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VaN4uTiYqiabCbsFLWM52A
…off feedback A release alone is never refused as started: a row stored blocked after a retry reads as held with attempts spent, and leaving held is how it starts again (review finding 4, BP-030). The block's reason moves to metadata.heldReason instead of feedback, which a worker reads as what to fix (finding 6). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VaN4uTiYqiabCbsFLWM52A
…cancels it On the resource backing the dependency check reads the board before the write and the ending's cascade reads it inside its own write, so a cancel committing between them left the dependent pending for good. Re-read once the dependent (or a deps edit) commits, and cancel it as its cascade would have (review finding 1). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VaN4uTiYqiabCbsFLWM52A
… own work The debt now records when its ending was written (cascadeOwed.endedAt, read through cascadeEndedAt with a fallback for debts without it), and the Workforce payer cancels only pieces filed by then. A retry's next attempt, or an attempt that went on to complete, keeps its pieces when an old debt is paid late (review finding 2). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VaN4uTiYqiabCbsFLWM52A
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0d5a235e90
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Covers review finding 3, fixed in the merge's guarded add: every row's checks (assertFilingCallerContext, assertFilingSettings, the worker's own step) and every chain reservation run before the batch's first add. The split harness can now file a turn's pieces as one addTasks batch. Red on the per-row loop it replaces: the first row was stored. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VaN4uTiYqiabCbsFLWM52A
…dency assertDepsFileable skipped the ended check for a dependency filed in the same batch, so a batch could file an errored or cancelled row with a pending dependent that could never start (Codex review, filing.ts). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VaN4uTiYqiabCbsFLWM52A
The dependency ordering kept only the first row for a repeated id, so the batch reported success with a row missing. assertUniqueIds refuses it as duplicate_task_id before anything is written (Codex review, board.ts). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VaN4uTiYqiabCbsFLWM52A
… own add The task's ending and its piece's add are separate writes, so an ending that lands between the filed marker and the add finds nothing to cancel, and the piece outlived its task. The add now reads the task after itself and cancels the piece when the task has failed for good or been cancelled (Codex review, board.ts). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VaN4uTiYqiabCbsFLWM52A
…s missed An ending records its debt from the rows its own process has seen, so on the resource backing a dependent filed by another process got no debt and stayed pending. Every touch now refreshes the mirror from the store and cancels unstarted rows whose dependency failed or was cancelled, leaving a dependency whose own debt isn't due yet to the ending's pass (Codex review, resource-backed.ts; review finding 1's sweep half). The post-create recheck stays for the same-process window. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VaN4uTiYqiabCbsFLWM52A
A release committed first could start the task under its old worker before the reassignment ran, and a refused reassignment (a follow-up keeps its worker) then reported failure for a task already released. The release now runs after every other field, so a refused reassignment leaves the task held (Codex review, task-tools-capability.ts). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VaN4uTiYqiabCbsFLWM52A
…s code A follow-up filed through the board's ref, naming a root that is held or still waiting, took the root's session and ran first. The guarded add now runs assertFollowUpHasRun, the rule the task tools already applied, and the code is follow_up_of_unrun, which says what the rule is (review finding 7). The orchestration README drops the retired busy-session code. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VaN4uTiYqiabCbsFLWM52A
…ing the run A replan comes after a failure, when a planner is most likely to name the failed task. applyReplan catches dependency_ended and files that task, and each batch task depending on it, cancelled and labelled skipped, the outcome such a dependent always had; the rest of the replan runs and the evaluator judges again. A dependency on no task, or a loop, still fails the run, and seeding the first plan stays strict (review finding 5). A row filed already ended no longer needs live dependencies. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VaN4uTiYqiabCbsFLWM52A
…ouch rules Task substrate: a held task holds the drain open and is reported with the failures, matching the task board page; the line about rows from an earlier version moves to the changeset; the batch rules name the ended batch row and duplicate_task_id; a touch catches dependents another process filed (review finding 8). Plan and execute: a replanned task on a failed one is skipped. The changeset records the reason's new home, the replan rule and the follow_up_of_unrun rename. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VaN4uTiYqiabCbsFLWM52A
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VaN4uTiYqiabCbsFLWM52A
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VaN4uTiYqiabCbsFLWM52A
FIX-2076 refuses a worker filing a task for itself, so the doer's pieces now go to eng.helper. The no-cascade control also removes the touch's dead-dependents sweep, so it still switches both cascades off. Re-run: PASS, and the control fails leg d only. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01VaN4uTiYqiabCbsFLWM52A
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: 4fb71475-14e0-4531-9e5f-cfd1e3665e11) |
|
Architect: not approving yet. The conflicts and Jake's must-fix threads are cleared, but four Codex findings on the durable path are still open and I read them as real: the dependency check and Fixing those (or a post-create recheck with compensating cancel for the first two), answering the threads, and green CI on the new head unblocks it. — FSD Architect |
|
@claude CI is failing |
|
There is one failing check on The failure comes from this PR. The test passes on main, and it fails only on branches that carry FIX-2080's changes, so the stacked #3623 and #3631 inherit it too. It passes when the file runs alone, which points to state leaking between tests that this change exposed. A worker is finding the root cause now. The test won't be skipped or loosened. The fix will come as its own commit, followed by a short review of the changes since the last one. Generated by Claude Code |
…ore its first claim
The drain's pay step resolved the board's collection a second time before
the first claim. On a resource-backed board that resolution reads the
store, so every board run started later. On a conversation board each next
filing then found the last row still pending and woke the board again
(11 wakes to main's 6 in task-screen's BR-17 Lab), and under CI load the
Lab's first read missed the screen's 1 s wait (task-screen.test.tsx:548).
Measured, the BR-17 group alone, 3 runs each, per test: before 0.75-1.3 s,
main 0.55-0.94 s, after 0.58-0.86 s. Removing only the touch's pay changed
nothing; removing only the drain's pay restored main's times.
The guarantee the early payment was there for still holds at the claim:
depsSatisfied (tasks/collection/internal.ts:842), read by isClaimable
(:959) on every claim, admits a row only when every dependency completed,
so a row behind an ended dependency is never claimed, whenever the payment
runs. The two-process race test now asserts that claim in the window before
any touch; with that guard weakened to treat an ended dependency as met, it
fails ("expected { ... } to be null"), and passes with it.
The touch's sweep also adopts only rows its mirror has never seen, instead
of reconciling the whole mirror: replacing a live ref with a listed one
could read back what the running request had already moved past.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VaN4uTiYqiabCbsFLWM52A
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: f8a5d9c0-f051-4b75-a3fb-537eea3511d1) |
|
Architect: approved at — FSD Architect |
jhoffner
left a comment
There was a problem hiding this comment.
Stand-in review: the fix round (7052203 → b194f86)
Scope: only what changed since the last stand-in review at 7052203. That covers FIX-2080's own commits (0d5a235..98db332, then d414662..b194f86) and the conflict resolutions in the merge with main (76a057a, read with --remerge-diff). It does not cover what FIX-2076 brought in. I applied the correctness, coherence and restraint lenses against specs/issues/FIX-2080/. Each defect below was reproduced with a scratch test, none of them pushed. The touched suites pass at b194f86: orchestration task-board, skills, filing-settings and retired-shapes (926 tests), Workforce filing-settings and filed-cascade, and patterns plan-and-execute.
Verdict: two findings block merge. The b194f86 move brings back a stuck-dependent path on onIdle: "complete" boards, and a mixed steps-plus-tasks batch still files part of itself when it is refused. Both are small fixes.
| # | Severity | Where | What |
|---|---|---|---|
| 1 | must-fix | task-board/index.ts:1356 |
With the payment after the workers, a crash-owed debt (or a dependent filed by another process) is never paid on an onIdle: "complete" drain (parallelTasks, eventActors). It spins to maxIterations. On the default mode, the final meta reports a pending row that the next step cancels |
| 2 | must-fix | workforce/.../board.ts:942 |
A withSteps batch refused at a step has already filed, and run, the delegates' tasks. Steps aren't put in dependency order, so a valid batch is refused too |
| 3 | should-fix | workforce/.../board.ts:769 |
assertFollowUpHasRun misses a root filed in the same batch. Finding 7 still reproduces through addTasks: the follow-up completes and the root stays held |
| 4 | should-fix | task-tools-capability.ts:1497 |
A release that runs last and declines does so after the assignee and other fields have landed, and the tool reports "Its assignee will not change" |
| 5 | nit | apply-replan.ts:264 |
A replan-skipped row has no causedBy or error, though the docs say "skipped the same way" |
| 6 | nit | workforce/.../cascade.ts:120 |
createdAt vs endedAt compares two boards' clocks. Note that attempt is the skew-free key |
Checked and fine
- b194f86's claim guard.
depsSatisfied(internal.ts:842) admits onlycompleteddependencies, andisClaimable(:959) reads it on every claim. A row behind an ended dependency is never claimed, whenever the payment runs. Finding 1 is about the exit check, not the claim. - The sweep's adopt-only change.
adoptUnknownRowsnever replaces or retires a held ref, and the two-handle test still passes. - The dependency races. The post-write re-check covers
addTask,addTasks(repeated, so a chain goes in one pass) andpatchSettings({ deps }), and skips rows filed terminal. The touch's sweep defers to a dependency whose own debt isn't due yet. - The late filed-cascade payment.
cascadeOwed.endedAtis written in the ending's own write.cascadeEndedAt'sdue - CASCADE_DUE_MSfallback matches how older debts were stamped (BP-030).payno-ops without a debt. The race between the piece's add and the parent's ending is closed bycancelIfParentEnded. - Workforce batch validation.
assertUniqueIds, every row'scheckFiling,assertDepsFileable(now treating a batch row filed ended asdependency_ended) and every chain reservation all run before the first add on the guarded board. Finding 2 is only thewithStepswrapper. - The retried
blockedrow. A release alone is exempt fromstartedon both backings, whiledepsandmaxAttemptsedits are still refused. The reason now goes tometadata.heldReason, andfeedbackstays clean. - Replan. It catches only
dependency_endedwith ataskId, skips that row's dependents transitively, terminates (each pass grows the skipped set or rethrows), and leavesdependency_not_on_boardanddependency_loopfatal. The first-plan seed is unchanged and strict. - The rename. Nothing references
follow_up_of_unfinishedexcept the goal's historical verdict row and the changeset's migration line, and both are correct to keep it. - Merge resolutions. The task tools keep both
TaskFilingRefusedErrorandTaskWriteRefusedError, and keep FIX-2080's one-argumentfollowUpTakenError, which was already the branch's shape before the merge. The task entry keeps both imports. The Workforce board folds the old inline caller-context check intoassertFilingCallerContext, keepsisStartable(added, ref.list()), and moveswithStepstohold/release/patchSettingswithstep_cannot_be_held.
Restraint: nothing to cut. cancelDeadDependents adds another O(n·deps) scan per touch, on top of the full-board scans already accepted. Fine at today's sizes.
Generated by Claude Code
| // the drain tests catch the unconditional one, and an `any`-typed insert on | ||
| // a board configuration those tests do not exercise is still silent. | ||
| .tap(boardExitsConnector, boardMetaCompleted) | ||
| .tap(payOwedCascades) |
There was a problem hiding this comment.
must-fix: with the payment after the workers, a drain on an onIdle: "complete" board never pays a debt a crash left. It spins until maxIterations.
The claim guard you cite does hold. depsSatisfied admits a row only when every dependency is completed, so a row behind an ended dependency is never claimed. But the early payment did a second job: it cleared that row before the exit check read it.
onIdle: "complete", used byparallelTasksandeventActors: a pending row with a dead dependency counts as in flight.classifyBoardanswerscontinue, and the workers idle-wait untilmaxIterations. Only then does this tap run and cancel the row. This is the "dependent stuck pending" failure D2 exists to remove, on the crash path the debt is for. The same goes for the new sweep (cancelDeadDependents), which is the only thing that catches a dependent filed by another process.onIdle: "complete-or-blocked"(the default): the drain exitsblocked-by-failures, andtask-board-metareportspending: 1. Then this tap cancels the row, so the final item contradicts the stored board.
Evidence: I ran a scratch drain test, not pushed. It seeds a request-backed board with a (errored, cascadeOwed past due) and b (pending, deps: ["a"]), using idlePollMs: 5 and maxIterations: 30.
- At b194f86:
completetakes 15.6 s and reportspending: 1.complete-or-blockedreportspending: 1.bendscancelledonly after the drain. - With
index.tsfrom b194f86^: both modes finish in about 70 ms total, reportingcancelled: 1, pending: 0.
The commit's timing problem was the second collectionFactory resolve before the first claim. Some ways to keep the payment ahead of the exit check without that cost:
- pay inside a step that already resolves the collection (
seedBlockorboardMetaActive); - pay only when a worker finds nothing claimable, just before
checkBoardclassifies.
Either way, add a drain-level test for the crash-owed path on both onIdle modes. Today only collection-level tests cover it, which is why this went through green.
Docs: task-substrate.md:135 says the drain pays the debt. That is only true once the drain pays before it decides to exit.
Generated by Claude Code
There was a problem hiding this comment.
Fixed in #3639 (FIX-2096), commit b234f0d. An idle worker now pays the owed cascade before checkBoard classifies, on the ref it already holds, and only when its rows show a debt or a dependent behind an ended row (owesATouch), so nothing reads the store before a start. The pay after the workers stays for rows only the store shows. New drain-level test drain-pays-owed-cascade.test.ts covers both onIdle modes. On main, complete took 15 648 ms and both reported pending: 1. Fixed, it's about 60 ms with pending: 0, cancelled: 1. The BR-17 group per test and the full shift-manager suite time the same as main; the numbers are in the PR.
Generated by Claude Code
| depsOnOneLedger(init, step); | ||
| if (step) checkStep(init); | ||
| } | ||
| const tasks = await board.addTasks(inits.filter((init) => !isStepFiling(init))); |
There was a problem hiding this comment.
must-fix: a mixed batch (steps plus delegate tasks) that is refused at a step still files the delegates' tasks.
The merge resolution says "a refused batch files nothing" and "checks a mixed batch whole". The loop above runs only depsOnOneLedger and checkStep. The plan ledger's own filing checks run later, one step at a time, inside fileStep → plan.addTask, and that happens after board.addTasks has already filed the delegate rows. Those checks are the step's deps, its settings and its chain reservation.
Evidence: I took worker-plans-its-steps.test.ts's stepper and changed its turn to file the plan with one ref.addTasks(...). The plan was [{ id: "w-draft", assignee: "writer" }, { id: "e-review", deps: ["e-outline"] }, { id: "e-outline" }]. The call threw TaskFilingRefusedError, and w-draft was filed and ran to completed. No step was filed.
The same run shows a second defect. Steps are not put in dependency order, so that batch is refused even though it is valid, because e-review is filed before e-outline. The board half uses dependencyOrder; the step half doesn't.
Fix: put the steps in dependencyOrder. Run assertDepsFileable over the steps against plan (batch ids count), plus the plan's settings check, and reserve every step's chain entry, all before board.addTasks. Then file. This is the same shape as the guarded addTasks.
Generated by Claude Code
There was a problem hiding this comment.
Fixed in #3639 (FIX-2096), commit b234f0d. withSteps.addTasks now runs, before the board's batch: the step rules, the plan ledger's settings check, assertDepsFileable over the steps against plan (the batch's steps count), and every step's chain reservation. Steps are added in dependencyOrder. Your repro is now two tests in worker-plans-its-steps.test.ts: the refused batch files nothing on either ledger, and the valid out-of-order batch files and runs.
Generated by Claude Code
| // with its lease. Then each row is added after the rows it depends on. | ||
| assertUniqueIds(inits); | ||
| const batch = dependencyOrder(inits.map((init) => ({ ...init, id: init.id ?? newTaskId() }))); | ||
| for (const init of batch) checkFiling(init); |
There was a problem hiding this comment.
should-fix: assertFollowUpHasRun misses a follow-up whose root is in the same batch, so the earlier finding 7 still reproduces through addTasks.
checkFiling looks the root up with ref.get. A root filed in the same batch isn't stored yet, so the lookup returns undefined, and the rule leaves it "to the caller's own lookup". By the time the follow-up is added, the root has already been added, so that lookup passes too.
Evidence: I ran the new follow_up_of_unrun test with h.controls.batch.add("s1") and an assignee on the follow-up. Nothing was refused. The root stayed held, and "Polish the brief" ran to completed in the root's session. Filed one at a time, the same rows are refused. The changeset says the rule holds "from the tools and from the board's ref alike".
Fix: give the batch check a lookup that sees the batch rows. A follow-up whose root is a row in the same batch has never run, so refuse it. For example, pass (id) => batchById.get(id) ?? ref.get(id) and let the run === undefined test refuse it.
Generated by Claude Code
There was a problem hiding this comment.
Fixed in #3639 (FIX-2096), commit b234f0d. The guarded batch passes its own rows to assertFollowUpHasRun (filedWith.get(id) ?? ref.get(id)), so a root filed in the same batch reads as never run and the follow-up is refused follow_up_of_unrun. The whole batch files nothing. New test in conversation-board-filing-settings.test.ts, which failed on main.
Generated by Claude Code
| if (patch.status === "pending") { | ||
| const outcome: TaskWriteOutcome | undefined = await c.release(input.taskId); | ||
| if (outcome != null && outcome.outcome === "declined") return outcome; |
There was a problem hiding this comment.
should-fix: a release declined here comes after the reassignment and the other fields have landed, and the tool then reports "Its assignee will not change."
Moving the release last fixes the Codex ordering bug. It breaks the invariant this mutator's header still states: "a decline means nothing has run and nothing was mutated". The release can decline. One case is started on a retried pending row: settingsDecline only exempts a row that is held.
Evidence: a scratch case using this test file's ctxWithTask("pending", { assignee: "researcher", attempts: 1, priority: 1 }), then updateTask({ status: "pending", assignee: "writer", priority: 9 }). The tool returned task_write_declined: … refused (started); the task is pending. Its assignee will not change. The row now has assignee: "writer" and priority: 9.
Fix: decide the release before any write. If patch.status === "pending" and the row isn't held, either decline up front, before the assignee write, or treat it as a no-op when the row is already pending. Then update the header comment and add the test.
Generated by Claude Code
There was a problem hiding this comment.
Fixed in #3639 (FIX-2096), commit b234f0d. Took the refuse-first option. Whether the release would be refused is decided from the row before any write (releaseDecline: terminal, or started and not held), so that refusal means nothing was written. Your scratch case is now a test: the row keeps assignee: "researcher", priority: 1. A claim landing between the read and the release can still produce a late refusal; that needs a race, and it's noted in the comment.
Generated by Claude Code
| if (!batch.some((row) => row.id !== undefined && skip.has(row.id) && row.status !== "cancelled")) return batch; | ||
| return batch.map((row) => | ||
| row.id !== undefined && skip.has(row.id) | ||
| ? { ...row, status: "cancelled" as const, labels: [...new Set([...(row.labels ?? []), SKIPPED_LABEL])] } |
There was a problem hiding this comment.
nit: a replan-skipped row is labelled skipped, but it carries no causedBy and no error naming the dead dependency.
plan-and-execute.md says such a task is "skipped the same way" as a cascade. The changeset says skipped rows come "with causedBy". A cascade-skipped row gets both fields; this one gets only the label, so the evaluator and the task screen can't say why it was skipped. If TaskInit can't carry causedBy, because it's server-written, then either soften the docs wording or file the row pending and let cancelIfDepEnded and the cascade cancel it.
Generated by Claude Code
There was a problem hiding this comment.
Fixed in #3639 (FIX-2096), commit b234f0d. A replan-skipped row now carries error and causedBy, the same as a cascade's skip (skippedFor, shared with cascadeCancelPatch), through two server-only TaskInit fields. Filing it pending wasn't an option, because filing refuses a pending row behind an ended dependency. The plan-and-execute test asserts dep s1 failed / dep after was cancelled.
Generated by Claude Code
| // finds the next attempt's pieces under the same parent too. | ||
| const endedAt = cascadeEndedAt(row.cascadeOwed); | ||
| for (const piece of board.list()) { | ||
| if (piece.metadata?.[PIECE_OF] !== row.id || isTerminalStatus(piece.status) || piece.createdAt > endedAt) continue; |
There was a problem hiding this comment.
nit: this compares a piece's createdAt against endedAt, and the two are stamped by different boards' clocks, possibly in different processes.
Within one process this is fine. Across processes, clock skew can push a next attempt's piece to createdAt <= endedAt, and a late payment would then cancel it. Or skew can push a stale piece past endedAt, and the payment would miss it. That needs both a crash and skew, so it's not worth more code now. Say so in the comment, so whoever picks up FIX-2083 knows attempt is the skew-free key if it ever matters.
Generated by Claude Code
There was a problem hiding this comment.
Fixed in #3639 (FIX-2096), commit b234f0d. Added the comment: the two clocks can be skewed across processes, it takes a crash and skew together, and the parent's attempt at filing is the skew-free key if it ever matters.
Generated by Claude Code


The problem
A delegated task can wait for ever. A task that depends on work on a different board waits until a person sees it. A task that depends on a failed task stays pending. A task whose parent failed keeps running pieces that nobody will read. When a follow-up goes to a session that is busy, it is refused, and the person must send it again later. Also, two flags say "do not start yet" (
blocked, and the "recorded" mark), and they act differently.What this does
A task now runs by the settings it was filed with:
held. A held task starts only when something releases it.heldreplacesblockedand the recorded mark. Rows stored in the old shapes read asheld.skipped, withcausedBy). When a task fails, is retried or is cancelled, the open pieces it filed are cancelled. The ending's own write records the cascade as owed, so a crash between the two does not lose it. The next touch of the board pays it.started).session: "new" | "ongoing"andresultTo: "filer" | "conversation".What's asked of you
Nothing new. This implements the approved direction in
specs/issues/FIX-2080/.One follow-up to know about. #3620 (FIX-2076) is now merged into this branch. A step filed held is refused here. Still open: if a request crashes mid-step in a conversation, its plan step stays
in_progress, and the next message waits on it until its lease runs out. The cascade in this PR acts on a task's ending, and a crashed request records none, so that case needs its own follow-up.Parts worth reviewing closely
packages/orchestration/src/tasks/collection/cascade.tsandrecordCascadeOwedincollection/internal.ts: the cascade debt is written in the same write as the ending, and paid after it. Check that both backings (state-backed.ts,resource-backed.ts) record it and pay it on the same paths.packages/orchestration/src/tasks/collection/filing.ts: the dependency rules and the caller-context, follow-up and unique-id checks. A batch's own ids count as on the board. A refusal files nothing, on a Workforce board too.resource-backed.tscancelIfDepEndedandcascade.tscancelDeadDependents: the resource backing re-reads dependencies after a filing commits, and every touch refreshes from the store and cancels unstarted rows behind a dead dependency, which covers an ending in another process.packages/workforce/src/conversation-board/board.ts:isStartable(one open task for each session), the guardedaddTask/addTasks, and theongoingsession key.packages/workforce/src/conversation-board/cascade.ts: the filed-work cascade cancels pieces directly on the filing task's partition. It does not go through the task session, which can be busy with a running turn.packages/orchestration/src/tasks/schema/retired-shapes.ts: the read-time map for stored rows in old shapes (BP-030).packages/devtool/src/react/lib/task-actions.ts: the DevTool found a board's tools by a fixed family that includedblockTask. Without this change, no board would match, and the DevTool would show no row actions.Spec:
specs/issues/FIX-2080/(spec PR #3595) · Linear: FIX-2080 · Epic: FIX-2056 · Fixes FIX-2080 · This PR merges.How to review this
This implements an approved spec. The approach and the numbered decisions in its
DECISIONS.mdare already signed off by a human, so please review the code against that direction, not the direction itself. The canonical set is retained atspecs/issues/FIX-2080/onmain; Linear links to it and the original merged spec PR, which keeps review history. Follow-up amendment PRs explain later direction changes. Current architecture docs remain the authority for shipped behavior (BP-037).Most valuable here: correctness on the second path (the legacy shape, the null boundary, the concurrent case, the cancel path), anything deriving an auth or routing decision from caller-controllable input, and a behaviour the tests assert around rather than on.
Already settled upstream: the approach and every decision in the spec's
DECISIONS.md(human-approved — if one is wrong, say so as a spec finding and it gets folded back; don't re-argue it inline), and the conventions indocs/contributing/best-practices.md.Engineering calls
applyReplanfiles that task, and each replanned task that depends on it,cancelledwith theskippedlabel, which is the outcome it had before, and the evaluator judges again. A dependency on no task, or a loop, still fails the run, and seeding the first plan stays strict. (Changed after review: the first version failed the run here too.)failon a held task is refused. A held task has not started, so there is nothing to fail. Cancel it.payOwedCascadesrefreshes the mirror before it sweeps, so each touch of a resource-backed board costs one listing. That is fine at today's board sizes; we'll index it when profiling shows it.started. Leavingheldis how a task starts again, including a storedblockedrow that had been retried. ChangingdepsormaxAttemptsafter the start is still refused.updateTaskreleases last. A release lands after the reassignment and the other fields, so a refused reassignment leaves the task held.updateTask'spatch.statusfor hold and release. FIX-2081 addscreateTask.minorfor orchestration, workforce and patterns (removed exports and a renamed status), andpatchfor react, shift-manager and devtool.Everything else: checks and goal evidence
Checks
pnpm typecheckpnpm --filter @flow-state-dev/orchestration testpnpm --filter @flow-state-dev/workforce testcoordinator-roundsandsplit-down-the-chain(BR-19) each failed once. Both pass alone, andsplit-down-the-chainpasses 4/4 under 4x parallel load on both this branch and mainci.ymlpnpm fallow:ci,fallow:new-dead-codeandfallow:audit --baseagainst maintest/command.test.tsfails here because the pages are not built (this sandbox only).test/task-screen.test.tsxfails a different test on each run withECONNRESET, and passes 23/23 alonetsc --noEmitThe goal check:
goals/task-board/runs-by-its-filing-settings/Model-free. A real host on SQLite, with the Workforce board on two flows, driven over HTTP. It includes a kill leg in a second process that dies between an ending and its cascade.
GOAL_CONTROL=no-cascade(no ending records the debt, and the touch's sweep is removed): FAIL on leg d only, for "the open filed task is cancelled" and "the dependent chain is cancelled", as the control requires.main's orchestration and workforcesrc: FAIL on legs a, b, d and e. Refused filings are stored, there is no settings edit, a follow-up of a running task is refused, there is no ongoing session, and retired rows do not read as held.The verdict log is in the goal's
goal.md.manager-queue-lab/it-stands-the-team-and-its-board-up-from-filesanddispatching-work/a-recorded-task-waits-for-dispatchwere updated forheldand run again: both PASS.Red state for the unit tests
On
main's source, the new orchestration tests fail (41) and the new workforce tests fail (6). Each review fix landed with its own test, seen failing first. The tests for the batch check and the follow-up check fail on the code they replace.Docs
Orchestration → Task substrate (
held, editing before the start, the dependency rules, the cascade), Task board (the built-in cascade replaces "Cascade-skipping dep-blocked tasks"), Workforce → Coordinators and Delegating work (held filing, follow-ups that wait, the session and result settings, the filed-work cascade), Patterns → Supervisor and Plan and execute (no cascade step), the glossary, and the orchestration, workforce, patterns and react READMEs.Generated by Claude Code
Note
High Risk
Changes core task lifecycle, dependency validation, and automatic cascades across orchestration and workforce, with breaking API removals (
blockTask,createCascadeSkipDependents) and migration-sensitive stored row shapes.Overview
FIX-2080 reshapes how tasks are filed, held, and torn down so boards do not leave work stuck forever.
heldreplacesblockedand the old “recorded but not started” mark. Filed tasks can stayhelduntil release;blockTaskis removed in favor ofupdateTask/hold/release/patchSettings. Status, change kinds, boardcounts.held, and UIs (kitchen-sink, DevTool, shift-manager goals) follow the rename; legacy storedblocked/undispatchedrows read asheldat the fold.Filing is validated up front.
addTask, batches, andpatchSettingsrefuse impossibledeps(missing on board, already ended, loops, duplicate ids) viaTaskFilingRefusedErrorand store nothing.held,deps, andmaxAttemptsare editable only until the task first starts.Failure and cancel propagate automatically. Every board cancels open dependents as
skippedwithcausedBy;createCascadeSkipDependentsand its export are removed—plan-and-execute/supervisor docs drop the extra tap. New cascade plumbing (cascade.ts,cascadeOwed,payOwedCascadeson drain) pays dependent and filed-work cascades after the ending write or on the next touch. Replan applies tasks that depend on dead work as skipped instead of failing the run.Workforce: conversation
addTaskfiles held; follow-ups into a busy session wait then run (follow_up_of_unrunreplaces unfinished refusal); optionalsession/resultToon board ref filing; open pieces cancel when the parent fails, retries, or is cancelled.Reviewed by Cursor Bugbot for commit b194f86. Configure here.
Generated by Claude Code