Skip to content

fix(FIX-2090): end harness-manager's refused run through the board's recorder (stacked on FIX-2080) - #3623

Merged
jhoffner merged 2 commits into
mainfrom
fix/FIX-2090
Oct 21, 2026
Merged

jhoffner merged 2 commits into
mainfrom
fix/FIX-2090

Conversation

@jhoffner

@jhoffner jhoffner commented Oct 21, 2026 •

Copy link
Copy Markdown
Collaborator

Stacked on FIX-2080's branch; retarget to main after it merges.

The problem

When a coding run is refused before it starts (its workspace won't take it, or the task names only half of what it should), the harness manager ended that task itself: it marked the row failed, or cancelled it on the fixed board. The epic's rule is that only the board ends a task: its claim gate for a dispatched task, its drain for an inline one. This was the last worker that wrote its own ending, and the closure check (the FIX-2087 surface sweep) flags it.

The issue also expected this path to skip cascades and the filer's notice. On FIX-2080's branch it doesn't. The collection's own fail and cancel already record those, and a probe showed a refused task's dependent was cancelled. So the user-visible behaviour stays the same. What changes is who writes the ending.

What this does

The manager no longer writes the ending. It throws an error that says "this failure is final", and the board's error recorder ends the row on that attempt, under the worker's claim:

  • Orchestration exports TaskEndsForGoodError. The error recorder ends the row errored with no retry, or cancelled when the error asks for that. Any other error re-queues as before.
  • Harness Manager: HarnessRunRefused extends it, carrying the ending its board needs. Outcomes are unchanged: errored on a board read from the claim, cancelled on the fixed board. The manager's own fail/cancel block is deleted.

What's asked of you

Nothing here is yours to decide. It's a bug fix, and this PR is its only gate.

Parts worth reviewing closely

  • The cancel arm in createRecordError (packages/orchestration/src/task-board/blocks/record-result.ts) calls collection.cancel directly, inside the same write-classify-release step as the fail arm. It is not wrapped in an advisory containment helper, because none exists for cancel. With ifAllowed the built-in backings return a decline rather than throw. A hand-written store that throws on a declined cancel would surface that throw, where a declined fail would have been contained.
  • advisoryFail now honours noRetry when it predicts the route (advisory-write-back.ts → resolveTarget). Without that, a contained throw on a noRetry fail would be judged against a predicted retry (pending) instead of errored.
  • Where the manager's error now ends up. The refused-run tests were green before this change and still are. Their evidence is the blast radius below: with the recorder ignoring the marker, exactly those tests go red.

Links: Linear FIX-2090 · epic FIX-2056 · rule ER-1 in specs/epics/FIX-2056/DECISIONS.md · found by FIX-2087's sweep (#3604) · merges after FIX-2080, retargeted to main

How to review this

How to review this. This is a bug fix on the direct route: there is no spec and no spec PR, because the hard part was the diagnosis and it happened in code. This PR is the only gate, so unlike a spec-backed change, the approach is in scope — nothing here was signed off upstream.

Most valuable here: the diagnosis chain — does the repro actually reproduce the reported failure, is the named cause the real one, does the fix address that cause rather than the symptom, and does the regression test fail without the fix? Then the second path (the legacy shape, the null boundary, the concurrent case, the cancel path).

In scope to challenge: where the guard was placed, any behaviour change a user could notice, and whether this should have been a spec'd feature instead of a fix.

Engineering calls
  • The fixed board keeps cancelled. Making every refused run errored would be one rule fewer, but packages/harness-manager/README.md documents the fixed board's cancel ("The row is cancelled on its first attempt"). Reversing a documented outcome isn't a bug fix's call, so the marker carries the ending.
  • The ending is chosen where the refusal is thrown (refusalEnding(ctx), read from the claim already on the manager's state), not set on the error later. A refusal can't reach the recorder without one.
  • The marker is general, not harness-specific. Any worker that knows a retry would end the same way can now say so without becoming an ending writer. One new export, documented in the orchestration README and the task-board guide.
  • HarnessRunRefused's constructor now requires the ending. It's exported, but only the manager constructs it. The changeset notes the signature.
Everything else: diagnosis, verification, files

Diagnosis.

  • Reported: the refused-run write bypasses endingOf, so cascades, ending debts and the filer's notice are skipped.
  • Checked: on FIX-2080's branch, both backings route fail and cancel through applyTransition → recordEnding → endingOf, and pay the cascade after the write. A probe (now kept as a test) seeded a refused row with a dependent. The row ended errored on attempt 1, and the dependent ended cancelled ("dep t1 failed").
  • Real cause: ER-1. A worker block wrote its own ending (manager.ts:2600: tasks.fail(..., { noRetry: true }) / tasks.cancel). It wrote outside the claim gate and the drain because the gate had no way to fail a row for good.

Red, then green (packages/orchestration/test/task-board/record-error-ends-for-good.test.ts, written before the recorder change):

× fails the row for good on this attempt, with the worker's reason   - "status": "errored"  + "status": "pending"
× cancels the row instead when the worker asks for a cancel         - "status": "cancelled" + "status": "pending"
✓ still re-queues an ordinary failure with attempts left

After the change, all 3 pass.

Blast radius for the harness-manager tests, which were green before the change. With the recorder's marker check disabled (forGood forced to undefined), exactly these went red and nothing else did (270 others passed):

  • claim-board-refused.spec.ts: claim board, half-feature payload; claim board, source refuses; fixed board, still cancelled; dependent cancelled (new)
  • workspace-host.spec.ts: "a run its workspace refuses › settles on its first attempt and is never retried" (refused, remote-not-allowed)

Closure sweep (specs/issues/FIX-2087/poc/surface-sweep/sweep.mjs from spec/FIX-2087, run against this branch):

  • Before: endings: tasks.fail(state.taskId at packages/harness-manager/src/manager.ts:2600
  • After: no harness-manager line. The ending lines left are task-tools-capability.ts:1389/1400 and split.ts:312/326, which are other issues'.
  • --control still reports its planted line.

Checks run:

  • pnpm typecheck: pass.
  • @flow-state-dev/orchestration: 1833 pass.
  • @flow-state-dev/harness-manager: 276 pass.
  • @flow-state-dev/workforce: 1421 pass and 3 fail on the full parallel run (two hit the 5 s timeout). The two files (coordinator-rounds, split-down-the-chain) pass 56/56 run alone, so the failures come from load, not this change.
  • No goal check: none is named for a direct-route bug. The issue's stated check is the sweep, above. No fsdev run either, because nothing model-facing changed. The harness-manager tests drive a real engine and store with a stub harness.

Files:

  • packages/orchestration/src/task-board/blocks/record-result.ts: the TaskEndsForGoodError class; createRecordError ends the row for good or cancels it.
  • packages/orchestration/src/task-board/index.ts: the export.
  • packages/orchestration/src/tasks/collection/advisory-write-back.ts: advisoryFail takes FailOptions, and resolveTarget honours noRetry.
  • packages/harness-manager/src/manager.ts: HarnessRunRefused extends the marker, refusalEnding, and the manager's own write is removed.
  • Tests: the new orchestration test; claim-board-refused.spec.ts gains the dependent case; claim-host.ts seed takes deps.
  • Docs: packages/orchestration/README.md, apps/docs/docs/orchestration/task-board.md.
  • .changeset/task-ends-for-good.md: orchestration minor (new export), harness-manager patch.

Not done here. The issue asks that FIX-2080's spec list this writer among the ending writers. That spec is merged, so it changes through an amendment PR from main, not on this stack. Once this lands, the writer is gone and FIX-2087's allow list needs no exemption for it, so there may be nothing left to list.


Generated by Claude Code


Note

Medium Risk
Changes core task-board error recording and harness refusal settlement; behavior is well-tested but touches claim gates, retry routing, and a breaking HarnessRunRefused constructor for external callers.

Overview
Introduces TaskEndsForGoodError so a worker can signal a non-retryable outcome without writing the task row itself: throw it with { taskId } (and optional { ending: "cancelled" }), and the task board's recordError path ends that row under the worker's claim—errored with noRetry by default, or cancelled when asked. A marker for a different taskId (e.g. rethrown from a nested board) still follows normal retry behavior.

Orchestration adds advisoryCancel, extends advisoryFail / releaseRow so noRetry failures are predicted as terminal errored, and documents the pattern in the task-board guide and README.

Harness Manager stops the unwind handler from calling tasks.fail / tasks.cancel on refused runs. HarnessRunRefused now extends the marker and takes { taskId, ending } via runRefused, preserving errored on claim boards and cancelled on the fixed board. Tests cover dependent cancellation on refused rows.

Reviewed by Cursor Bugbot for commit af0f1ce. Configure here.

@vercel

vercel Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

3 Skipped Deployments
Project Deployment Actions Updated
flow-state-dev-docs Ignored Ignored Oct 21, 2026 10:27pm UTC
flow-state-dev-kitchen-sink Ignored Ignored Oct 21, 2026 10:27pm UTC
flow-state-dev-ui Ignored Ignored Preview Oct 21, 2026 10:27pm UTC

Request Review

@linear-code

linear-code Bot commented Oct 21, 2026 •

Copy link
Copy Markdown

FIX-2090

FIX-2080

@cursor

cursor Bot commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: 3ed70378-e034-47ed-84bc-87f42947f4f5)

@cursor

cursor Bot commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: a0c44a99-a84b-4abd-a1e9-ad2335feb0d4)

@jhoffner jhoffner left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Review covering coherence, restraint, correctness and completeness. I'm standing in for Bugbot.

Verdict. The direction is right. Moving the refused run's ending out of the manager and into the claim gate / drain recorder is what ER-1 asks for, since blocks/record-result.ts is the gate's recorder in the epic's seam table. Two correctness gaps in the new recorder arms need fixing before merge, and the changeset bumps are swapped. Details are in the three line comments.

Checked and holding

  • Both boards end a refused run the same way as before. On the board read from the claim the row ends errored after 1 attempt; on the fixed board it ends cancelled. claim-board-refused.spec.ts passes 5/5 at head. refusalEnding(ctx) reads the same claim field the old handler read, and that field is patched (line ~1840) before any of the three throw sites. taskPayload's hard-coded "errored" is correct, because it only throws HarnessRunRefused on a board read from the claim.
  • The diagnosis holds. I ran this PR's new dependent test against FIX-2080's src (orchestration and harness-manager): it passes there too, and the dependent is cancelled. The symptom does not reproduce on FIX-2080's branch, and what remained was the ER-1 violation. A side effect is that this test cannot fail on this change. It characterises the ending; it is not a regression test for the fix.
  • The resolveTarget noRetry change is correct and needed. I checked it with a scratch test: a noRetry fail that commits errored and then throws on announcement. At head it is rethrown, which is right. With the old resolveTarget it is contained as declined: terminal, and FIX-963's signal is swallowed. No committed test covers this. Reverting the change leaves the suite green, so it is worth one test.

Optional, not line-commented

  • releaseRow (the undetermined-verdict recovery) calls advisoryFail without noRetry. On a store with no write provenance, a for-good ending that throws without committing gets re-queued by the release, which is the retry the marker exists to prevent. This is narrow, and a hand-written store may ignore noRetry anyway.
  • Restraint: ending: "cancelled" is a public option whose only purpose is to keep the fixed board's prior outcome, and it is the arm with the containment gap. If ending a refused run errored on the fixed board too is acceptable, the option, the cancel arm and the recorded: "cancelled" return all go away. That's a product call (it changes what the fixed board reports), so I'm not deciding it here.
  • On the shape of the export: a public class marker is consistent with DispatchRefusedError and the other classes the recorder branches on. It lets any worker override the filer's attempts, but not the board's maxTotalRetries budget, since noRetry spends no retry. That is acceptable for "the next attempt would be refused the same way", provided the marker is scoped to the thrower's own row (see the nested-board comment).

Generated by Claude Code

Comment thread .changeset/task-ends-for-good.md Resolved
@jhoffner
jhoffner marked this pull request as ready for review October 21, 2026 20:15
@cursor

cursor Bot commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: a4b008d1-aae7-4be6-8d39-c25857729939)

@jhoffner
jhoffner added this pull request to stack #3625 October 21, 2026 20:17

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Simplify / architecture read

Problem this PR solves: harness-manager was a second task-row ending writer on refused runs (best-effort runBoard + fail/cancel inside a swallowing catch), while the board recorder could still re-queue the same refusal. That fights ER-1 (one recorder owns endings) and duplicates FIX-951 advisory semantics.

Did the approach go too far? I considered the smaller deltas:

  1. Only fix noRetry in advisory-write-back and keep manager write-back — still two writers, still races with superseded attempts, still bypasses writeRethrowingIfNothingSaved / parked-row guards on the manager path.
  2. Have the worker call collection.fail(..., { noRetry }) directly — fewer types, but reintroduces a worker-as-writer exception and spreads board policy into harness-manager.
  3. Duck-typed flag on Error — less ceremony, weaker contract.

The TaskEndsForGoodError + recorder branch is the smallest change that deletes ~20 lines of manager settlement, routes refuses through the same path as every other worker failure (including dependency cascade), and wires noRetry into advisory attribution so containment matches the backings. Net complexity goes down; the new surface area is intentional API, not accidental abstraction.

Approval: Direction looks right for FIX-2090 / the stack. Tests and docs match the contract. I ran pnpm --filter @flow-state-dev/orchestration test -- record-error-ends-for-good (pass).

Optional follow-ups (not blocking)

  • Package boundary: TaskEndsForGoodError lives in blocks/record-result.ts but is subclassed from harness-manager — a tiny tasks/ (or task-board/errors.ts) home + re-export might read cleaner than inheriting from a block file.
  • Symmetry: terminal errored uses advisoryFail; cancelled calls collection.cancel directly (same as the old manager). Fine for parity with this PR; an advisoryCancel would only matter if you want identical throw-containment on cancel.
  • Micro-perf: readGrantedRetries may still list() on budgeted boards even when noRetry is already set — skip when options.noRetry if you care.
  • Naming: unrelated to filing’s endedForGood() — a one-line doc cross-link could save future grep confusion.

No changeset concern from me: minor orchestration + patch harness-manager with the documented HarnessRunRefused constructor break is appropriate.

Open in Web View Automation 

Sent by Cursor Automation: Review For Best Approach

Comment thread packages/harness-manager/src/manager.ts Resolved
Comment thread packages/harness-manager/src/manager.ts Resolved

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Snob. A shorter shape of the same contract is on #3626 (poc/shorter-ends-for-good @ 128e7e0cc). That PR does not merge.

Fold 128e7e0cc onto a3474adb6. Do not reset onto the POC. Close the POC unmerged after this PR lands.

Net −35 (12/47). Comments that restated the recorder, plus an unused cause on TaskEndsForGoodError (nothing passed it; super(message) is the only executable edit).

  • The class note is four lines: the next attempt would end the same way, the recorder ends the row under the worker's claim, ending defaults to errored. The field note that repeated the union is gone. The two-line note above forGood is gone.
  • resolveTarget's noRetry note is one clause: terminal in both backings, whatever attempts are left.
  • HarnessRunRefused keeps what a refusal is. refusalEnding is the one line that says why the boards differ: a cancel on a claim board reaches nobody, so that board ends errored; the fixed board stays cancelled. The unwind note is one line and points at the class.
  • The recorder test header is one line. The dependent-test comment that repeated the test name is gone.

record-error-ends-for-good.test.ts: 3 passed. Red: defaulting ending to cancelled failed the first test (status cancelled, expected errored). Restored. Harness files are comments only, so that suite was not re-run.

Left as recorded: the fixed board stays cancelled; the ending is chosen at the throw; the marker stays general; the constructor still requires the ending; the cancel arm, noRetry in resolveTarget, the docs, the changeset, and the assertions stay.

Tried and left: one recorded binding for the two return ternaries. It adds a line.

View PR

Open in Web View Automation 

Sent by Cursor Automation: Code Snob

jhoffner pushed a commit that referenced this pull request Oct 21, 2026
…el arm

Review fixes on #3623:
- TaskEndsForGoodError carries the taskId its thrower holds, and the error
  recorder acts on it only for that row. A board nested inside a worker
  rethrows its worker's error on onError "fail"; the inner row's verdict no
  longer ends the outer row on its first attempt.
- The cancel arm goes through a new advisoryCancel, so a store that throws on
  a declined transition is contained as it is on fail. The fixed board's
  cancelled ending is kept.
- releaseRow passes noRetry for a for-good row; observe skips the retry-budget
  scan on a noRetry fail; attribute takes FailOptions (no cast).
- Changeset bumps swapped: harness-manager minor, orchestration patch.
- Comments trimmed along the lines of POC #3626; the unused cause option goes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VaN4uTiYqiabCbsFLWM52A
@cursor

cursor Bot commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: 625efd2b-23e6-4ecc-b017-f712e6a4b666)

Copy link
Copy Markdown
Collaborator Author

Re review 5605762176 / POC #3626: the trims and the dropped cause are folded into 9f70177 by hand, on top of the taskId scope, which keeps one sentence of its own.


Generated by Claude Code

@cursor

cursor Bot commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: fa491477-5f60-40a5-8266-d59b779218c9)

@cursor

cursor Bot commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: 65c5faa6-1cac-4f56-b03e-35d43f94fd05)

@jhoffner jhoffner changed the title fix(FIX-2090): end harness-manager's refused run through the board's recorder fix(FIX-2090): end harness-manager's refused run through the board's recorder (stacked on FIX-2080) Oct 21, 2026
@cursor

cursor Bot commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: 8753b386-5db6-4277-8e75-984684c6e75e)

Base automatically changed from fix/FIX-2080 to main October 21, 2026 22:25
…recorder

A refused harness run failed (claim board) or cancelled (fixed board) its own
row from the manager's error handler, a worker writing an ending outside the
claim gate and the drain (ER-1). The FIX-2087 closure sweep flags it at
manager.ts:2600.

The worker now says the failure is final instead of writing it: orchestration
exports TaskEndsForGoodError, and the error recorder ends the row on that
attempt under the worker's claim (fail with noRetry, or cancel when asked).
HarnessRunRefused extends it with the ending its board needs, so outcomes are
unchanged: errored on a board read from the claim, cancelled on the fixed board.

Diagnosis: the issue's symptom (cascades, debts and notices skipped) does not
reproduce on FIX-2080's branch. The collection's own fail/cancel already
reached endingOf there, and a probe showed the refused row's dependent
cancelled. What remained was the ER-1 violation itself, which the sweep is the
check for.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VaN4uTiYqiabCbsFLWM52A
…el arm

Review fixes on #3623:
- TaskEndsForGoodError carries the taskId its thrower holds, and the error
  recorder acts on it only for that row. A board nested inside a worker
  rethrows its worker's error on onError "fail"; the inner row's verdict no
  longer ends the outer row on its first attempt.
- The cancel arm goes through a new advisoryCancel, so a store that throws on
  a declined transition is contained as it is on fail. The fixed board's
  cancelled ending is kept.
- releaseRow passes noRetry for a for-good row; observe skips the retry-budget
  scan on a noRetry fail; attribute takes FailOptions (no cast).
- Changeset bumps swapped: harness-manager minor, orchestration patch.
- Comments trimmed along the lines of POC #3626; the unused cause option goes.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VaN4uTiYqiabCbsFLWM52A
@cursor

cursor Bot commented Oct 21, 2026

Copy link
Copy Markdown
Contributor

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: 703861c9-63e6-4df1-8722-644c45b85e26)

@jhoffner
jhoffner merged commit b8328cc into main Oct 21, 2026
17 of 19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants