Skip to content

fix(workforce): the lead-fit check never refuses work that doesn't say what sort it is (FIX-2019) - #3460

Merged
jhoffner merged 6 commits into
mainfrom
claude/project-thread-x29km2
Oct 15, 2026
Merged

jhoffner merged 6 commits into
mainfrom
claude/project-thread-x29km2

Conversation

@jhoffner

@jhoffner jhoffner commented Oct 15, 2026 •

Copy link
Copy Markdown
Collaborator

Requested by Jake · project thread

Before: when a user asked their project coordinator for work without saying what sort it was, the lead-fit check sometimes refused the general lead, even though the general lead's description says it leads exactly that. The coordinator then opened nothing. FIX-1797's closure step b2 failed this way about 1 run in 10.

After: the check also says whether the workstream's title and objectives name a sort of work at all, and only work that does can be refused. A bare name passes; a clear misfit (research handed to the EM, software handed to the general lead) is still refused.

The check is a backstop for clear misfits, and work that names no sort can't be one. So the cause was the judge, not the routing: picking the general lead for such an ask is the call FIX-1987 settled.

How. lead-fit.ts: the judge answers { says, fits } and the verdict refuses only says && !fits. The b2 goal step also re-reads the store until it held still across the Workstreams screen read, since a lead editing its objectives in between failed b2 once in 24 runs.

Evidence (full numbers in /mnt/project-files/FIX-2019/measurements.md):

  • Main: a b2 run was refused on title: "orchid basalt 0e5b" with no objectives. Sampled directly, the old judge refused that input 5 of 30 times.
  • New judge, sampled 60 each: bare-name inputs 180/180 pass; research→EM refused 50/60 (old 52/60); feature→EM passes 59/60 (old 60/60); software→general lead refused 30/30.
  • Goal b2 on the final tree: 24/24 pass, 0 lead-fit refusals (first batch on the judge fix alone: 0 refusals in 24).
  • Controls on the final tree all fail at their own step: unpartitioned b4, no-follow-up b5, no-delegate-check b6, no-roster-check c4, org-scoped-workers c2.
  • Unit test for the new case fails without the verdict change and passes with it.

Not fixed here: a3 fails whenever a2 forks the Error Reporter (18 of 48 runs), filed as FIX-2020. split-down-the-chain.test.ts failed once in three local workforce runs, unrelated to this change.

Independent of #3450; based on main.

🤖 Generated with Claude Code

https://claude.ai/code/session_01BX2WA8i4ifQyPGr495Yy6A


Generated by Claude Code


Note

Medium Risk
Changes coordinator open refusal logic in lead-fit.ts, which can affect which workstreams get created; scope is narrow and covered by new tests, but it touches model-judged gating on opens.

Overview
Lead-fit now treats vague work differently from clear mismatches. The openWorkstreamTool judge returns says (whether title/objectives name a kind of work) and fits; lead-doesnt-fit is thrown only when says && !fits, so bare names and other unspecified work no longer get refused. The judge prompt and API docs/README describe this behavior.

Goal b2 retries Workstreams screen grading up to three times until workstream store entries stay unchanged across a read, reducing flakes when leads edit objectives during the check.

Tests and mock generators were updated for the { says, fits } shape, including a case where says: false still opens the workstream.

Reviewed by Cursor Bugbot for commit e67333f. Configure here.

RetriggerView in GreptileConfidence Score: 4/5

The lead-fit change appears sound, but the required changeset reference must be added before merging; handling exhausted screen retries is also recommended.

Fix All in CursorFindings

  1. P2 Release note lacks issue reference ▶
  2. P2 Unstable reads still get graded ▶

Summary

The lead-fit judge now returns says and fits. Only work that states its kind can be refused.

  • Lead-fit checks let work with no stated kind go ahead.
  • The shared-project check rereads workstreams around the screen read.

Reviews (1) · Last reviewed commit: "fix(goals): b2 grades the Workstreams sc..." · Reviewed by Greptile

…y what sort it is (FIX-2019)

The judge refused the general lead for a bare-name ask about 1 sample in 6,
though its description says it leads such work, so FIX-1797's step b2
opened no workstream about 1 run in 10. The judge now also says whether
the work says what sort it is, and only work that does can be refused:
no lead can be a clear misfit for work that names no sort.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BX2WA8i4ifQyPGr495Yy6A
…t, not that it is never refused (FIX-2019)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BX2WA8i4ifQyPGr495Yy6A
…d still across the read (FIX-2019)

The leads keep working after b2's asks, and one that added an objective
between the screen read and the store read failed b2's progress count.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BX2WA8i4ifQyPGr495Yy6A
@linear-code

linear-code Bot commented Oct 15, 2026

Copy link
Copy Markdown

FIX-2019

@vercel

vercel Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

3 Skipped Deployments
Project Deployment Actions Updated
flow-state-dev-docs Ignored Ignored Oct 15, 2026 1:40am UTC
flow-state-dev-kitchen-sink Ignored Ignored Oct 15, 2026 1:40am UTC
flow-state-dev-ui Ignored Ignored Preview Oct 15, 2026 1:40am UTC

Request Review

@jhoffner jhoffner self-assigned this Oct 15, 2026
@cursor

cursor Bot commented Oct 15, 2026

Copy link
Copy Markdown
Contributor

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: eb89de1b-d752-4099-96bd-e45cc784993e)

@jhoffner

Copy link
Copy Markdown
Collaborator Author

@codex review

@jhoffner jhoffner left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Second look + architecture pass on FIX-2019.

Approach: sound, and I wouldn't change it. Fixing the judge instead of the routing is the right layer. A backstop for clear misfits shouldn't be able to refuse work that names no sort, and the fix is small (one schema field, one &&). The evidence is good: sampled the judge directly, the clear-misfit cases barely moved, and the controls fail at their own steps.

Alternatives I considered and rejected:

  • A deterministic pre-filter for "bare names" (regex or length). It can't tell a codename from a terse real ask, so it would trade a flaky judge for a brittle heuristic.
  • Dropping the check for under-specified work. That turns a backstop off exactly where the coordinator is most likely to guess.

Architecture: no refactor is warranted by this PR. lead-fit.ts is already a single-purpose check (judge, then verdict) with one caller. Only the goal-leg retry is worth a small tidy, and it falls inside lines this PR already rewrites.

Findings (inline):

  1. legs.mts: the 3-try retry can fall through silently when the store never holds still, hiding the flake it targets. Worth fixing. Also resolve the repeated key === "alice" ? alice : bob once.
  2. lead-fit.ts (optional): says:false, fits:false is a meaningless state, and the record can't distinguish an "unsaid" pass from a real fit. Both are small and can be skipped.

Nit: the packages/workforce/README.md paragraph now has one very long line where the neighbours wrap at about 110 columns. Rewrap it.

Docs and changeset match the behaviour, and the unit test pins the new case.


Generated by Claude Code

…ary check (FIX-2019)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BX2WA8i4ifQyPGr495Yy6A
@cursor

cursor Bot commented Oct 15, 2026

Copy link
Copy Markdown
Contributor

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: e80445dd-dd7d-4ffc-a908-de2c521b7c36)

Comment thread .changeset/lead-fit-unsaid-work.md Resolved

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Summary

The workforce change is the right shape for this bug: keep a single judge call, make the product rule explicit in structured output (says + fits), and gate refusal on judged.says && !judged.fits. That is a small, testable delta on top of an already-LLM-based backstop, and the new integration test pins the garnet-style case well.

I considered simpler alternatives (below). None of them clearly beat this without trading one kind of complexity for another.

Simpler approaches considered

Approach Tradeoff
Prompt-only (“if work doesn’t name a kind, always answer fits: true”) No schema/verdict change, but you still refuse on !fits when the model disobeys — exactly the failure mode you’re fixing. Hard to test.
Deterministic pre-pass in TypeScript (e.g. empty objectives + title heuristics → skip refusal, no says) Avoids a second boolean and model reliance for “unsaid” work, but duplicates semantics the judge already owns and will drift from prompt examples like “the requested work”.
Skip leadFit entirely when under-specified (don’t call the model) Saves a call in some cases, but changes when the check runs and doesn’t help when title/objectives are ambiguous rather than empty.
Current: says in the same structured output One round-trip, explicit contract, verdict stays a one-liner. Modest complexity; fits BP-016-style strict outputs.

Recommendation: Ship the says/fits schema unless you want a deliberate split where code owns “too vague to be a misfit” and the model only answers fits when code says work is described. That’s cleaner philosophically but a wider behavior change than FIX-2019 needs.

/simplify notes (non-blocking)

  1. goals/.../legs.mts b2 retry — The stability loop is understandable (leads mutating objectives during Playwright reads), but it’s the heaviest part of the PR. Before growing a one-off “read UI until JSON.stringify(entries) matches” pattern, try await quiet(o.store) immediately before projectWorkstreams, same as later legs in this file. If that’s enough, you can drop most of the retry machinery. If not, a tiny local helper (readWorkstreamsWhenStoreStable) or a narrow fingerprint (ids/status, not full objectives) would read cleaner than inline drawn! + triple loop.

  2. Docs sentence (projects.md / README) — The new clause reads grammatically broken (“and work they don’t, such as a bare name, passes…”). Worth a quick editorial pass so outsiders aren’t parsing it twice.

  3. Tests — fit(fits, says = true) puts the primary knob first but disagrees with { says, fits } field order; easy to mis-call. Optional: fit({ says, fits }) or swap args to match the schema.

  4. Comment duplication — Module header + prompt + answerSchema comment all restate the unsaid-work rule; fine for a behavioral contract, but you could trim one layer later.

No blockers from my side; approving.

Open in Web View Automation 

Sent by Cursor Automation: Review For Best Approach

Comment thread apps/docs/docs/workforce/projects.md Resolved
…en read; changeset names FIX-2019

Also reads the person's store once and rewraps the README paragraph.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BX2WA8i4ifQyPGr495Yy6A

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Snob — already tight. No POC.

Read packages/workforce/src/projects/lead-fit.ts, the lead-fit cases in packages/workforce/test/project-coordinator.test.ts, the b2 loop in goals/workforce-privacy/two-users-share-a-project-and-nothing-else/legs.mts (plus quiet and entriesOf next to it), the lead-fit paragraphs in packages/workforce/README.md and apps/docs/docs/workforce/projects.md, and .changeset/lead-fit-unsaid-work.md.

A rewrite does not shrink this contract. The check is one shared { says, fits } schema, one prompt line, and says && !fits. answerSchema is that object written once for the generator and the judged union. The b2 loop is a screen read between two store reads, retried while they differ, at most three times. Hoisting key === "alice" ? alice : bob, or failing when the store never holds still, adds a line. quiet() waits until nothing is running, which is the b3 wait, a different contract. The header, the prompt, the README, the docs page, and the changeset each speak to a different reader. The test comment is the one line the vocabulary sweep already put on "sort".

Head 46ece5e3e67d19d9781b1075c95ae6ed93cdf0fb.

Open in Web View Automation 

Sent by Cursor Automation: Code Snob

@cursor

cursor Bot commented Oct 15, 2026

Copy link
Copy Markdown
Contributor

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: c2a2541e-6f4a-4665-bc33-e35ceee9a056)

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 8ccc93b472

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread .changeset/lead-fit-unsaid-work.md Resolved
Comment thread packages/workforce/README.md Resolved
@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-10-15T01:32:47.916116Z 8ccc93b Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@jhoffner

Copy link
Copy Markdown
Collaborator Author

Architect: not approving yet. Codex's P1 on the Shift Manager fixture is real: lead-fit.ts:52 now requires { says, fits } with no coercion, but packages/shift-manager/test/project-workstreams.test.tsx:90,93 still returns only { fits }, so the judge's output fails the schema, the rescue fails open, and the research-under-EM case at :336 won't see lead-doesnt-fit. Smaller, same thread of work: the openWorkstreamTool contract still says every mismatch is refused at workstream-writes.ts:260-263 and packages/workforce/README.md:2021.

To unblock: return says: true from both fixture branches, update those two descriptions to say only work that names its sort can be refused, and let CI go green. (The changeset and b2 retry findings from the same Codex pass are already fixed by c3f796e; resolve those threads.)

@jhoffner

Copy link
Copy Markdown
Collaborator Author

Architect: not approving yet. The judge's answer is now { says, fits }, but the Shift Manager fit fixture at packages/shift-manager/test/project-workstreams.test.tsx:90 and :93 still returns only { fits }. With repair: { coerce: false } that answer fails the schema, the check goes unanswered and lets the open through, so the test at :312 that expects lead-doesnt-fit (:336) should break. This is the open Codex P1 on lead-fit.ts:81.

To unblock: have the fixture return says: true alongside fits, and get CI green. Optional while you're there: the API row at packages/workforce/README.md:2021 still describes the refusal without the bare-name exemption.

@jhoffner

Copy link
Copy Markdown
Collaborator Author

Architect: not approving yet. Codex's P1 on the Shift Manager fixture is real: packages/shift-manager/test/project-workstreams.test.tsx:90,93 still answers only { fits }, so the new answerSchema (lead-fit.ts:52, says required) rejects it, the rescue records it as unanswered and lets the open through, and the refusal that test expects at line 336 can't happen anymore. Smaller: the API Surface row (packages/workforce/README.md:2019) and the openWorkstreamTool comment (workstream-writes.ts:259-261) still say any mismatch is refused.

What unblocks it: return { says: true, fits: ... } from that fixture, line up those two descriptions with the unsaid-work exemption, resolve the four Codex threads (two are already fixed by c3f796e), and let CI finish green.

@jhoffner

Copy link
Copy Markdown
Collaborator Author

Architect: not approving yet. The lead-fit schema now requires { says, fits }, but the Shift Manager fixture in packages/shift-manager/test/project-workstreams.test.tsx (~87–95) still returns only { fits }, so research-under-EM fail-opens instead of refusing lead-doesnt-fit. Unblock: add says on both fixture returns (true for described work).

@jhoffner

Copy link
Copy Markdown
Collaborator Author

Architect: not approving yet — this PR changes lead-fit to require { says, fits }, but packages/shift-manager/test/project-workstreams.test.tsx still returns only { fits } for the workstream-lead-fit fixture (~L87–95). That fails the new schema, the rescue path treats it as unanswered and fails open, and the research-under-EM case that expects lead-doesnt-fit (~L336) no longer refuses. Unblock by updating both fixture returns to include says (e.g. { says: true, fits: leadFits(...) } for the misfit cases that should still refuse).

@jhoffner

Copy link
Copy Markdown
Collaborator Author

Architect: not approving yet — packages/shift-manager/test/project-workstreams.test.tsx still returns only { fits } from the workstream-lead-fit fixture (~L87–95), but lead-fit.ts now requires { says, fits }. Missing says fail-opens via the unanswered rescue, so the research-under-EM case this suite asserts no longer refuses. Unblock: return { says: true, fits: leadFits(...) } from both generate and generateStep (and any other fixture that answers that block).

…notes say unsaid work passes (FIX-2019)

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01BX2WA8i4ifQyPGr495Yy6A
@cursor

cursor Bot commented Oct 15, 2026

Copy link
Copy Markdown
Contributor

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: 4fe7e68e-effc-4d3b-a7f9-ed76d3bf891c)

@jhoffner
jhoffner merged commit 046d8cc into main Oct 15, 2026
19 of 20 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants