Repository navigation
fix(workforce): the lead-fit check never refuses work that doesn't say what sort it is (FIX-2019) - #3460
Conversation
…y what sort it is (FIX-2019) The judge refused the general lead for a bare-name ask about 1 sample in 6, though its description says it leads such work, so FIX-1797's step b2 opened no workstream about 1 run in 10. The judge now also says whether the work says what sort it is, and only work that does can be refused: no lead can be a clear misfit for work that names no sort. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BX2WA8i4ifQyPGr495Yy6A
…t, not that it is never refused (FIX-2019) Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BX2WA8i4ifQyPGr495Yy6A
…d still across the read (FIX-2019) The leads keep working after b2's asks, and one that added an objective between the screen read and the store read failed b2's progress count. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BX2WA8i4ifQyPGr495Yy6A
|
The latest updates on your projects. Learn more about Vercel for GitHub. 3 Skipped Deployments
|
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: eb89de1b-d752-4099-96bd-e45cc784993e) |
|
@codex review |
jhoffner
left a comment
There was a problem hiding this comment.
Second look + architecture pass on FIX-2019.
Approach: sound, and I wouldn't change it. Fixing the judge instead of the routing is the right layer. A backstop for clear misfits shouldn't be able to refuse work that names no sort, and the fix is small (one schema field, one &&). The evidence is good: sampled the judge directly, the clear-misfit cases barely moved, and the controls fail at their own steps.
Alternatives I considered and rejected:
- A deterministic pre-filter for "bare names" (regex or length). It can't tell a codename from a terse real ask, so it would trade a flaky judge for a brittle heuristic.
- Dropping the check for under-specified work. That turns a backstop off exactly where the coordinator is most likely to guess.
Architecture: no refactor is warranted by this PR. lead-fit.ts is already a single-purpose check (judge, then verdict) with one caller. Only the goal-leg retry is worth a small tidy, and it falls inside lines this PR already rewrites.
Findings (inline):
legs.mts: the 3-try retry can fall through silently when the store never holds still, hiding the flake it targets. Worth fixing. Also resolve the repeatedkey === "alice" ? alice : bobonce.lead-fit.ts(optional):says:false, fits:falseis a meaningless state, and the record can't distinguish an "unsaid" pass from a real fit. Both are small and can be skipped.
Nit: the packages/workforce/README.md paragraph now has one very long line where the neighbours wrap at about 110 columns. Rewrap it.
Docs and changeset match the behaviour, and the unit test pins the new case.
Generated by Claude Code
…ary check (FIX-2019) Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BX2WA8i4ifQyPGr495Yy6A
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: e80445dd-dd7d-4ffc-a908-de2c521b7c36) |
There was a problem hiding this comment.
Summary
The workforce change is the right shape for this bug: keep a single judge call, make the product rule explicit in structured output (says + fits), and gate refusal on judged.says && !judged.fits. That is a small, testable delta on top of an already-LLM-based backstop, and the new integration test pins the garnet-style case well.
I considered simpler alternatives (below). None of them clearly beat this without trading one kind of complexity for another.
Simpler approaches considered
| Approach | Tradeoff |
|---|---|
Prompt-only (“if work doesn’t name a kind, always answer fits: true”) |
No schema/verdict change, but you still refuse on !fits when the model disobeys — exactly the failure mode you’re fixing. Hard to test. |
Deterministic pre-pass in TypeScript (e.g. empty objectives + title heuristics → skip refusal, no says) |
Avoids a second boolean and model reliance for “unsaid” work, but duplicates semantics the judge already owns and will drift from prompt examples like “the requested work”. |
Skip leadFit entirely when under-specified (don’t call the model) |
Saves a call in some cases, but changes when the check runs and doesn’t help when title/objectives are ambiguous rather than empty. |
Current: says in the same structured output |
One round-trip, explicit contract, verdict stays a one-liner. Modest complexity; fits BP-016-style strict outputs. |
Recommendation: Ship the says/fits schema unless you want a deliberate split where code owns “too vague to be a misfit” and the model only answers fits when code says work is described. That’s cleaner philosophically but a wider behavior change than FIX-2019 needs.
/simplify notes (non-blocking)
-
goals/.../legs.mtsb2 retry — The stability loop is understandable (leads mutating objectives during Playwright reads), but it’s the heaviest part of the PR. Before growing a one-off “read UI untilJSON.stringify(entries)matches” pattern, tryawait quiet(o.store)immediately beforeprojectWorkstreams, same as later legs in this file. If that’s enough, you can drop most of the retry machinery. If not, a tiny local helper (readWorkstreamsWhenStoreStable) or a narrow fingerprint (ids/status, not full objectives) would read cleaner than inlinedrawn!+ triple loop. -
Docs sentence (
projects.md/ README) — The new clause reads grammatically broken (“and work they don’t, such as a bare name, passes…”). Worth a quick editorial pass so outsiders aren’t parsing it twice. -
Tests —
fit(fits, says = true)puts the primary knob first but disagrees with{ says, fits }field order; easy to mis-call. Optional:fit({ says, fits })or swap args to match the schema. -
Comment duplication — Module header + prompt +
answerSchemacomment all restate the unsaid-work rule; fine for a behavioral contract, but you could trim one layer later.
No blockers from my side; approving.
Sent by Cursor Automation: Review For Best Approach
…en read; changeset names FIX-2019 Also reads the person's store once and rewraps the README paragraph. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BX2WA8i4ifQyPGr495Yy6A
There was a problem hiding this comment.
Code Snob — already tight. No POC.
Read packages/workforce/src/projects/lead-fit.ts, the lead-fit cases in packages/workforce/test/project-coordinator.test.ts, the b2 loop in goals/workforce-privacy/two-users-share-a-project-and-nothing-else/legs.mts (plus quiet and entriesOf next to it), the lead-fit paragraphs in packages/workforce/README.md and apps/docs/docs/workforce/projects.md, and .changeset/lead-fit-unsaid-work.md.
A rewrite does not shrink this contract. The check is one shared { says, fits } schema, one prompt line, and says && !fits. answerSchema is that object written once for the generator and the judged union. The b2 loop is a screen read between two store reads, retried while they differ, at most three times. Hoisting key === "alice" ? alice : bob, or failing when the store never holds still, adds a line. quiet() waits until nothing is running, which is the b3 wait, a different contract. The header, the prompt, the README, the docs page, and the changeset each speak to a different reader. The test comment is the one line the vocabulary sweep already put on "sort".
Head 46ece5e3e67d19d9781b1075c95ae6ed93cdf0fb.
Sent by Cursor Automation: Code Snob
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: c2a2541e-6f4a-4665-bc33-e35ceee9a056) |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 8ccc93b472
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
Architect: not approving yet. Codex's P1 on the Shift Manager fixture is real: To unblock: return |
|
Architect: not approving yet. The judge's answer is now To unblock: have the fixture return |
|
Architect: not approving yet. Codex's P1 on the Shift Manager fixture is real: What unblocks it: return |
|
Architect: not approving yet. The lead-fit schema now requires |
|
Architect: not approving yet — this PR changes |
|
Architect: not approving yet — |
…notes say unsaid work passes (FIX-2019) Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01BX2WA8i4ifQyPGr495Yy6A
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: 4fe7e68e-effc-4d3b-a7f9-ed76d3bf891c) |


Requested by Jake · project thread
Before: when a user asked their project coordinator for work without saying what sort it was, the lead-fit check sometimes refused the general lead, even though the general lead's description says it leads exactly that. The coordinator then opened nothing. FIX-1797's closure step b2 failed this way about 1 run in 10.
After: the check also says whether the workstream's title and objectives name a sort of work at all, and only work that does can be refused. A bare name passes; a clear misfit (research handed to the EM, software handed to the general lead) is still refused.
The check is a backstop for clear misfits, and work that names no sort can't be one. So the cause was the judge, not the routing: picking the general lead for such an ask is the call FIX-1987 settled.
How.
lead-fit.ts: the judge answers{ says, fits }and the verdict refuses onlysays && !fits. The b2 goal step also re-reads the store until it held still across the Workstreams screen read, since a lead editing its objectives in between failed b2 once in 24 runs.Evidence (full numbers in
/mnt/project-files/FIX-2019/measurements.md):title: "orchid basalt 0e5b"with no objectives. Sampled directly, the old judge refused that input 5 of 30 times.Not fixed here: a3 fails whenever a2 forks the Error Reporter (18 of 48 runs), filed as FIX-2020.
split-down-the-chain.test.tsfailed once in three local workforce runs, unrelated to this change.Independent of #3450; based on main.
🤖 Generated with Claude Code
https://claude.ai/code/session_01BX2WA8i4ifQyPGr495Yy6A
Generated by Claude Code
Note
Medium Risk
Changes coordinator open refusal logic in
lead-fit.ts, which can affect which workstreams get created; scope is narrow and covered by new tests, but it touches model-judged gating on opens.Overview
Lead-fit now treats vague work differently from clear mismatches. The
openWorkstreamTooljudge returnssays(whether title/objectives name a kind of work) andfits;lead-doesnt-fitis thrown only whensays && !fits, so bare names and other unspecified work no longer get refused. The judge prompt and API docs/README describe this behavior.Goal b2 retries Workstreams screen grading up to three times until workstream store entries stay unchanged across a read, reducing flakes when leads edit objectives during the check.
Tests and mock generators were updated for the
{ says, fits }shape, including a case wheresays: falsestill opens the workstream.Reviewed by Cursor Bugbot for commit e67333f. Configure here.
The lead-fit change appears sound, but the required changeset reference must be added before merging; handling exhausted screen retries is also recommended.
Summary
The lead-fit judge now returns
saysandfits. Only work that states its kind can be refused.Reviews (1) · Last reviewed commit: "fix(goals): b2 grades the Workstreams sc..." · Reviewed by Greptile