Repository navigation
fix(engine): scope check-interrupted sweep to the caller's tenant (FIX-1569) - #2205
Conversation
…X-1569) The user-addressed check-interrupted sweep admitted active-request entries by user and organization only. listStale spans every tenant, so a caller on one tenant who shares a user id with another tenant saw that tenant's request and session ids and forced its live requests to `interrupted`. Thread the request tenant into the handler and require `tenantMatches(entry.tenantId, ctx.tenantId)` in `sweepAdmits` before any other clause, on every path, so an entry from another tenant is never returned, marked interrupted, or deregistered. Same rule retry/continue already apply to a single record. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018e4MkWPYX62KRow9TSn75D
|
The latest updates on your projects. Learn more about Vercel for GitHub. 3 Skipped Deployments
|
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: serverGenReqId_fb98318d-9d66-47bb-8bbc-e2bd44910f12) |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 538cd1ab1a
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
- dispatched-work.md: the client-triggered sweep admits only entries whose tenant matches the request's (`tenantMatches`); a caller with no tenant header sweeps only untenanted entries; the startup and periodic sweeps pass no caller filter and still cover every tenant. - authentication.md: the `user` subject row says the guard compares no organization or tenant for user-addressed routes and each handler scopes its own rows. - apps/docs client API page and client README: the recovery sweep only touches the caller's tenant. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018e4MkWPYX62KRow9TSn75D
…X-1569) docs-editor: "tenant header" was used nowhere else in the docs; say the tenant id and its default header, matching the tenancy pages. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_018e4MkWPYX62KRow9TSn75D
The problem
This is a cross-tenant integrity fix. The recovery check that the DevTool runs on every session-list refresh (
POST /api/flows/users/:userId/check-interrupted) looks for in-flight requests that have stopped sending heartbeats, marks theminterrupted, and removes them from the active list. It picked which requests to touch by user and organization only. It never looked at the tenant.So in a multi-tenant deployment where one user id exists under two tenants, a caller on tenant A could:
interrupted, which stops B's work, not only exposes it.Retry and continue already refuse a request from another tenant. This route didn't.
What this does
The check now considers only requests in the caller's own tenant, before it reports or writes anything. A request in another tenant is left completely alone: it isn't returned, isn't marked
interrupted, and stays registered as running. A caller with no tenant header reaches only requests that have no tenant, which is the rule retry and continue already follow.What's asked of you
Nothing here is yours to decide. This is a bug on the direct route, so this PR is the review gate.
Parts worth reviewing closely
sweepAdmitsinpackages/engine/src/routes/recovery-routes.ts. The tenant check runs first on every path. That includes the framework-default path, which used to returnundefined(only the pathuserIdrestricted it). The function now always returns a predicate. Check that no path skips the tenant clause.packages/engine/src/routes/http-handlers.ts. One line:tenantIdgoes into the check-interrupted context, in the same place and shape fix(engine): list a seat's sessions and runs to the caller its own resolver accepts (FIX-1566) #2192 uses for the active-requests listing. fix(engine): list a seat's sessions and runs to the caller its own resolver accepts (FIX-1566) #2192 has since merged, andmainmerged into this branch without conflicts.detectInterruptedRequestsfilters the stale entries throughownedBybefore it reads any request record or writes anything, so an entry the predicate rejects is never touched. The tests assert this on the store (record stillin_progress, entry still registered), not only on the response.Links: Linear FIX-1569 · epic FIX-1528 · related #2192 (FIX-1566, the same class of leak on the listings) · merges after human review
How to review this — altitude, what's in scope, what's deliberately unsettled
Engineering calls — decided along the way, no product sign-off needed
sweepAdmits, not indetectInterruptedRequestsor route-auth. It sits with the other admission clauses, which scopes the fix to this one route. The periodic and startup sweeps are host-wide by design and should not be tenant-scoped. If this is wrong, a future caller ofdetectInterruptedRequestshas to remember to pass a tenant-awareownedBy.tenantIdis judged, not the request record's. The entry is what the sweep admits on, and the host writes both from the same dispatch envelope. If they could ever diverge, the record would need its own check. I found no path where they do.usersubject is unchanged. It sets noownerOrgId, so route-auth skips its org/tenant comparison on user-addressed routes, and each handler has to scope its own rows. That's a sharp edge for the next user-addressed route that writes. It isn't reworked here, to keep this minimal. It's worth a follow-up if anotheruserroute starts mutating.docs/architecture/dispatched-work.mdsays that the client-triggered sweep admits only entries in the caller's tenant (tenantMatches), that a caller with no tenant header sweeps only entries with no tenant, and that the startup and periodic sweeps still cover every tenant. Theuserrow indocs/architecture/authentication.mdsays the guard compares no organization or tenant for user-addressed routes, so each handler scopes its own rows. One sentence was added to the recovery example in both the client API docs page (apps/docs/docs/api/client.md) andpackages/client/README.md.Everything else — verification, files
Verification (BP-003). Run with
FSDEV_DEFAULT_MODELandFSDEV_INTENT_*unset.packages/engine/srcreverted toorigin/main, all three fail:expected [ 'acme_req', 'globex_req' ] to not include 'globex_req'expected [ 'acme_req', 'globex_req' ] to not include 'globex_req'expected [ 'untenanted_req', 'acme_req' ] to deeply equal [ 'untenanted_req' ]tenantMatchesline was commented out, with the tenant still threaded. Full engine suite:3 failed | 2725 passed | 4 skipped. The failures are exactly the three new tests.215 files passed; 2728 passed | 4 skipped. After mergingmain(with fix(engine): list a seat's sessions and runs to the caller its own resolver accepts (FIX-1566) #2192):216 files passed; 2740 passed | 4 skipped.pnpm --filter @flow-state-dev/engine typecheckexits 0.What the tests pin (
packages/engine/test/tenant-route-isolation.test.ts). Two tenants, the sameuserIdand org, and one stale in-flight request in each. A caller on tenant A runs check-interrupted:in_progressinterrupted, deregistered)This runs twice: once on the framework default resolver (no principal) and once with a host resolver that returns the same user and org for both tenants (the principal branch). A third case covers a caller with no tenant header: it sweeps the row with no tenant and leaves tenant A's row alone.
Files
packages/engine/src/routes/recovery-routes.ts: tenant clause first insweepAdmits, which now always returns a predicate.packages/engine/src/routes/http-handlers.ts: passestenantIdinto the check-interrupted context.packages/engine/test/tenant-route-isolation.test.ts: three regression tests..changeset/check-interrupted-tenant-scope.md: engine patch.docs/architecture/dispatched-work.md,docs/architecture/authentication.md,apps/docs/docs/api/client.md,packages/client/README.md: the tenant boundary.🤖 Generated with Claude Code
https://claude.ai/code/session_018e4MkWPYX62KRow9TSn75D