Skip to content

fix(workforce): one unaddressable hired row no longer bricks the reload (FIX-1536) - #2104

Merged
jhoffner merged 6 commits into
mainfrom
fix/fix-1536
Sep 23, 2026
Merged

jhoffner merged 6 commits into
mainfrom
fix/fix-1536

Conversation

@jhoffner

@jhoffner jhoffner commented Sep 23, 2026 •

Copy link
Copy Markdown
Collaborator

The problem. An app with no authentication runs every request as the reserved development org. When it hires a seat at runtime, the stored row sits under an org id that can't be part of a seat address. At the next boot, reloadHiredSeats threw on that one row and rejected the whole reload, so every other org's hired seats disappeared too, not just the bad row.

What this does. The reload now treats an unaddressable row the same way it already treats a gone kind or a row stamped for another org: it skips it and reports it in problems as organization "<org>", row "<key>" — <reason>. Every other row still reloads. The owning-org fence from #2091 runs first and is unchanged, so a row stamped for another org is still refused and never re-bound to a cell where it could be addressed. The same guard covers a stored seat id that starts with the user-owned ~ marker, which failed the same way.

What's asked of you. Nothing new to decide. The product call is already recorded on the issue: the development org stays unaddressable, and a user's private team stays in the org it was built in. This fix doesn't remap, promote or repair the row.

Parts worth reviewing closely (for the code reviewer):

  1. packages/workforce/src/roster/reload.ts: the manifest build moved inside a per-row try. The store read and the org-cap and timeout rejects sit outside the loop and still fail the boot, as they should.
  2. My weak spot: seatsSource in manifest-sources.ts calls hiredSeatManifest without a try. Under the development org, listing the seats manifest still throws for that org. It doesn't affect other orgs, because that path is scoped to one org per request, and the issue scopes the fix to reload. It's listed as a follow-up below rather than fixed here.
  3. Ported, not mine: commit 636542bc6 is db41f92 from docs(process): point corrections at issue-implement 10.6's sweeps #2094, cherry-picked because main doesn't compile @flow-state-dev/workforce (FIX-1529: pin hired seats to the org and user that hired them #2091 and Seat-hire catalog tools, and Discover lists the seats they mint #2079 collided: TS2339 .manifest). It touches manifest-sources.ts and seat-hire-capability.ts only, not the reload or rows code. It becomes a no-op once docs(process): point corrections at issue-implement 10.6's sweeps #2094 merges.

Fixes FIX-1536 · bug, direct route (no spec) · goal: pnpm tsx goals/hire-plane/reload-survives-an-unaddressable-row/run.mts → PASS; control (reload.ts reverted to ffe2b6e26) → FAIL on leg (c) only

How to review this

How to review this. This is a bug fix on the direct route: there is no spec and no
spec PR, because the hard part was the diagnosis and it happened in code. This PR is the
only gate
, so unlike a spec-backed change, the approach is in scope — nothing here was
signed off upstream.

Most valuable here: the diagnosis chain — does the repro actually reproduce the reported
failure, is the named cause the real one, does the fix address that cause rather than the
symptom, and does the regression test fail without the fix? Then the second path (the legacy
shape, the null boundary, the concurrent case, the cancel path).

In scope to challenge: where the guard was placed, any behaviour change a user could
notice, and whether this should have been a spec'd feature instead of a fix.

Diagnosis and engineering calls

Cause. hiredSeatManifest throws by contract when the org or seat id can't form an address (seatAddress → validateSegment). reloadHiredSeats only wrapped hireWorkforce in its per-row try, not the manifest build, so the throw escaped the loop and rejected the whole call.

Engineering calls (no approval needed):

  • Catch at the reload, not inside hiredSeatManifest. The other option was to make hiredSeatManifest return a RowProblem for a bad address. That would change a contract that the live hire paths (seat-hire-capability, kitchen-sink workforce-admin) depend on to throw on this deploy's mistake. It would also turn their record.manifest into undefined wherever a caller doesn't narrow. If this call is wrong, one more caller that walks stored rows needs its own try. That's the seatsSource follow-up.
  • One problem per row, not one per org. A per-org check up front would skip the read and couldn't name rows. Per-row also covers the ~ seat id in a legal org for free. The cost of a wrong call is only a noisier problems list for a dev org with many rows.
  • No user-docs change. apps/docs/docs/workforce/durable-hire.md and the workforce README already say problems holds one entry per row that didn't become a seat, and that only the org cap, the timeout and a failed store read reject. The fix makes the code match that contract. Only the internal file headers were touched.
  • No changeset. The contract is unchanged, so downstream consumers have nothing new to know.

Follow-up (not filed): seatsSource (manifest-sources.ts) still throws on an unaddressable row when listing a development-org session's seats manifest.

Verification

Red, before the fix (packages/workforce/test/roster.test.ts → "skips a row it cannot address, and still serves every other org"):

× V5 · a boot reload skips what it cannot use and serves the rest > skips a row it cannot address, and still serves every other org
  → Organization id "__fsd_default_org__" must be lowercase letters, digits, and single hyphens ...
 ❯ seatAddress src/roster/rows.ts:52:3
 ❯ hiredSeatManifest src/roster/rows.ts:237:11
 ❯ reloadHiredSeats src/roster/reload.ts:203:22

Green, after: both new tests pass; the full workforce suite is 48 files, 757 passed. Kitchen-sink workforce-admin.test.ts is 24/24. Turbo typecheck for workforce and its dependents, plus kitchen-sink, is 27/27 (with the ported fix). The goals package passes tsc --noEmit with 0 errors.

Blast radius for the fence test. The test "still refuses a row stamped for another org" was green before the fix, so I broke the property it claims instead: I disabled the owning-org mismatch check in rows.ts and bound owningOrgId to the cell. Exactly two tests went red: the new fence test and the existing hire-plane.test.ts → "E5 a row claiming another org is a named problem and is not registered". The other 755 stayed green. I restored rows.ts afterwards.

Goal (real path, HTTP router, in-memory store, no model).

  • PASS: the dev hire was stored under __fsd_default_org__. The reload over [__fsd_default_org__, acme] returned ["acme.support.ada"] and 2 problems. acme.support.ada answered an action after the restart. The dev row was named, and the row stamped for globex was refused (cannot be registered under "acme").
  • Control (reload.ts at ffe2b6e26): FAIL — [c] reload rejected, so no org's seats came back: Organization id "__fsd_default_org__" must be .... Only leg (c) failed. Legs (a) and (b) completed, which confirms the dev hire really does write under the default org through the HTTP path.

FSDEV_DEFAULT_MODEL was unset for local vitest runs. With it set, 73 unrelated tests fail with createModelResolver: FSDEV_DEFAULT_MODEL was set, but no intents are declared. That comes from this session's environment, not from this change.

🤖 Generated with Claude Code

https://claude.ai/code/session_018e4MkWPYX62KRow9TSn75D


Generated by Claude Code


Note

Medium Risk
Changes boot-time roster reload behavior for multi-org deployments; incorrect handling could still drop seats or admit bad rows, but scope is degrade-on-skip with existing owning-org fence and strong regression coverage.

Overview
Fixes FIX-1536: when reloadHiredSeats hit a roster row whose org or seat id cannot form a seat address (e.g. hires under DEFAULT_ORG_ID from an unauthenticated app, or a seat id starting with ~), hiredSeatManifest threw and aborted the entire reload, so every other org lost their seats at boot.

reloadHiredSeats now wraps manifest construction in a per-row try/catch and records those failures in problems like other bad rows, while valid rows still hire. The owning-org mismatch path is unchanged—rows stamped for another org still refuse with cannot be registered under.

Related hardening: manifest-sources skips RowProblem manifests when merging hired roster into discover; seat-hire throws if manifest build would be a problem. New vitest cases cover unaddressable default-org / tilde seat ids and cross-org stamps; a hire-plane goal exercises HTTP hire + reload + post-restart answer end-to-end.

Reviewed by Cursor Bugbot for commit 603dc88. Configure here.

…rejecting the boot (FIX-1536)

An app with no principal resolver runs as DEFAULT_ORG_ID, so a runtime
hire leaves a row in a cell whose id is not a legal address segment.
reloadHiredSeats built the seat address outside its per-row guard, so
seatAddress's throw escaped and rejected the whole reload -- every org's
team, not just the bad row. A seat id carrying the user-owned "~" marker
failed the same way.

Cause (the hypothesis that held): hiredSeatManifest throws for a bad org
or seat id by contract ("this deploy's mistake"), and reload.ts only
guarded hireWorkforce, not the manifest build. Now the manifest build is
guarded too and the throw becomes one named problem in the same
`organization "<org>", row "<key>" -- <reason>` shape the owning-org
mismatch already uses. The owning-org fence runs first inside
hiredSeatManifest and is unchanged: a row stamped for another org is
still refused, never re-bound.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018e4MkWPYX62KRow9TSn75D
…call sites

main has been red since ffe2b6e. #2091 (FIX-1529) changed hiredSeatManifest
to return `{ manifest } | RowProblem`; #2079 (seat-hire catalog tools),
merged minutes later, added two call sites that read `.manifest` without
narrowing, so @flow-state-dev/workforce no longer compiles.

Both now handle the problem case the way roster/reload.ts already does:
- manifest-sources: a row owned by another org is skipped when listing
  seats, alongside the unparseable-row skip on the line above.
- seat-hire-capability: the hire refuses with the row's reason.

Verified: the workforce build fails on main with exactly CI's four TS2339
errors and passes with this change; workforce 755/755; pnpm typecheck
70/70.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_014Zc4qjodAcrYK5kMfcrXS8
…oad with a goal

Real path, no model: an app with no resolver hires through its HTTP
action (the row lands under DEFAULT_ORG_ID), a signed-in org hires
through the same action, and a restarted app reloads both orgs. The
signed-in org's seat comes back, registers and answers; the dev row is
one named problem; a row stamped for another org is still refused.
Control: reverting reload.ts to ffe2b6e fails leg (c) only.

Also names the address case in reload.ts's header and where
hiredSeatManifest's throw is caught.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018e4MkWPYX62KRow9TSn75D
@vercel

vercel Bot commented Sep 23, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

3 Skipped Deployments
Project Deployment Actions Updated
flow-state-dev-docs Ignored Ignored Sep 23, 2026 11:37pm UTC
flow-state-dev-kitchen-sink Ignored Ignored Sep 23, 2026 11:37pm UTC
flow-state-dev-ui Ignored Ignored Preview Sep 23, 2026 11:37pm UTC

Request Review

@linear-code

linear-code Bot commented Sep 23, 2026

Copy link
Copy Markdown

FIX-1536

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 23, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-23T21:18:30.046984Z 603dc88 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Simplify / review (FIX-1536)

Verdict: approve. The reload change is the smallest fix that matches the documented contract (problems per skipped row; only cap/timeout/store read fail the boot). I do not think the PR over-built relative to the bug.

Did the approach go too far?

Mostly no. The production delta that matters is a per-row try around hiredSeatManifest plus two focused vitest cases. That is proportional to a dual-outcome API: org mismatch returns { problem }, bad address materialization throws from seatAddress (see rows.ts JSDoc and existing hire-plane.test.ts throw expectations).

The heavier surface is the new hire-plane goal (~272 lines). It overlaps roster.test.ts on reload semantics, but it is not redundant work: the goal adds HTTP hire → shared store → restart → register → answer, plus a source-revert control. That matches other goals/hire-plane/* runners (sibling goal is ~340 lines). I would not ask to delete the goal to “simplify” unless CI time becomes painful; trimming would mean extracting shared HTTP helpers in goals/lib, which is a cross-cutting refactor outside this bug.

The #2094 cherry-pick (manifest-sources / seat-hire-capability) adds scope noise but is explained in the PR body; the if ("problem" in record) continue in discover is a real guard (without it, record.manifest.id on a mismatch row is unsafe). It is not the reload fix.

Cleaner alternative (considered, not required here)

The structurally simpler long-term shape is one row-level outcome in rows.ts: map seatAddress failures to RowProblem (or export something like boundHiredSeatManifest that never throws), then every walker only checks "problem" in ….

Why staying with reload-local try/catch is defensible: interactive hire paths already call seatAddress first and treat throws as deploy-time mistakes; changing hiredSeatManifest to never throw is a public contract shift across kitchen-sink, tests, and hire tools. Your PR description already records that call. I agree with it for this bug route.

Follow-up (you already flagged): seatsSource still has no try around hiredSeatManifest, so an unaddressable row under the dev org can still break that org’s manifest listing even though reload is fixed. Worth a small ticket unless #2094 merge makes it moot.

Optional nit if more walkers appear: a tiny shared helper (parse + bind + normalize throws → reason) would dedupe reload and discover without pulling in hireWorkforce.

Performance

No concerns: per-row try/catch is negligible next to sequential org prefix reads and per-row hireWorkforce; goal polling is test-only.

Skipped (would need product / wider refactor)

  • Making runtime hire on DEFAULT_ORG_ID succeed (this PR correctly fixes boot, not unauthenticated hire semantics).
  • Replacing the goal with workforce integration tests only (loses the HTTP anti-game path).

Nice diagnosis chain and control in goal.md. Ship when CI is green.

Open in Web View Automation 

Sent by Cursor Automation: Review For Best Approach

Comment thread packages/workforce/src/roster/reload.ts Resolved

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Snob. The reload fix can be smaller.

reloadHiredSeats already catches a per-row hireWorkforce refusal and records `${where} — ${messageOf(error)}`. The new try around hiredSeatManifest does that same record-and-skip, and the comment above it repeats the file header: an org or seat id that cannot be an address is one named problem, and it must not reject the boot.

Put hiredSeatManifest in the existing try, ahead of the owning-org problem check. A throw is still one problem. A row stamped for another org still returns problem and is not hired. The second catch, the let, and that comment go away.

Shape sketch: #2110 (roster.test.ts 22 passed, including the unaddressable row and the cross-org stamp; hire-plane.test.ts 8 passed). That PR does not merge.

Agent on this PR: fold the subtraction into this branch, and close the POC unmerged after this PR lands.

Open in Web View Automation 

Sent by Cursor Automation: Code Snob

Comment thread packages/workforce/src/roster/reload.ts Resolved

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 603dc88216

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread packages/workforce/src/roster/reload.ts Resolved
…s skipping an unaddressable row (FIX-1536)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_018e4MkWPYX62KRow9TSn75D

Copy link
Copy Markdown
Collaborator Author

from: epic-lifecycle
session: epic/FIX-1528
kind: reply

Two lines in the description are now out of date:

The reload simplification from #2110 is folded in (ab86bb8), and #2110 can be closed once this merges.


Generated by Claude Code

@jhoffner
jhoffner merged commit 9041a2c into main Sep 23, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants