Repository navigation
feat(engine): a reply holds its conversation, so a task's notice waits for it (FIX-1834) - #2970
Conversation
A hold request runs at once and marks its key busy; a defer request waits until nothing on its key is running or waiting, then runs. Lets follow-up work for a conversation (a task's notice) land after the current reply without delaying the person's own next message. The defer wait has no budget: it ends when the holder settles (success, error or abort), or when a crashed process's place on a shared lease backend expires. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
The latest updates on your projects. Learn more about Vercel for GitHub. 3 Skipped Deployments
|
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: 261c5a5a-6a5a-42e9-996c-79c57a48e308) |
Caps waiting defer requests per key per process (32); the next is refused with ConcurrencyDeferLimitError, a ConcurrencyRejectedError, before anything is created. A defer yields to newer hold runs for 30s only, then lines up behind the runs present and waits for its turn. Tests show the policy is read only from the flow declaration, never caller body or metadata. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: 96168a92-351f-4282-a251-9cc47b61ddf8) |
…X-1834) On the in-memory default the arbiter admitted before the session's owner was checked, so a caller who did not own a session could briefly mark its key held (refusing the owner's reject) or fill its defer cap (refusing the owner's notices). hold and defer now admit after admitOwnership on every backend; a defer over its cap is refused through the handle. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: b5439aa6-b5d1-4f6d-859c-2d7f46986d31) |
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: 399722e7-c74d-449c-b60b-a0297cd548fa) |
There was a problem hiding this comment.
Problem this PR solves
Follow-up work (task notices, background completions) was able to start beside an in-flight reply on the same session. queue would serialize, but it also makes the user’s next message wait behind the reply and time out after 30s. FIX-1834 needs asymmetric rules: replies run immediately and mark the session busy; follow-ups wait until the conversation is quiet.
Verdict
Approve. The hold / defer split at the concurrency seam is the right layer for BP-031, cancelable long waits, shared-backend crash paths, and adapter-mapped refusals. I don’t see a materially simpler approach that keeps those properties without pushing the same state machine into every app.
The cost is real complexity in arbiter.ts (~130 lines + a second admission shape). That looks justified for the product contract (unbounded defer wait, 30s patience to newer holds, per-process cap of 32, serialized defers, post-ownership admit). Docs and tests match the behavior; pnpm --filter @flow-state-dev/engine test passes on this branch.
Where it went further than necessary (non-blocking)
-
deferredAdmissionparallel toadmissionFor— A hand-rolledConcurrencyAdmissionduplicatesrun/release/lostforwarding. Future fixes to renewal, give-back, orlostpropagation may need to land twice. Follow-up: fold defer-specific behavior intoadmissionForvia a small options object (claimMode,turnBudget) when someone touches this again — not worth a behavior-preserving mega-refactor in this PR. -
Policy aliasing encodes semantics — After an empty-key claim, defer returns
admissionFor("reject", …); after patience expires,admissionFor("defer", …, Infinity). That works today becauserejectskipswaitForTurn, but it’s fragile ifrejectanddeferdiverge. Follow-up: named internal mode (e.g.skipWait: true) instead of mis-labeling the policy. -
Infinityas unbounded wait —pollForTurntreatsbudgetMs === Infinityas “never time out.” Clear enough with comments; a named wait budget type would be easier to grep. -
Authoring footgun: external dispatch —
hold/defersilently becomeallowwhen handed to BullMQ/external dispatch (documented). Consider a dev-only log or metric when downgrade happens — separate observability issue. -
Per-process defer cap — Asymmetric with cluster-wide
queue/holdplaces on a shared lease backend. Docs already say “per process”; worth a one-line callout in architecture if operators assume a global cap.
Simpler approaches considered
| Approach | Why not enough |
|---|---|
| App-level session flag / handler gate | No transport parity (webhook/MCP), no SSE cancel/stub path, BP-031 and crash recovery reimplemented per app |
Single queue with lanes/priority |
Similar arbiter work, less obvious author API than hold vs defer |
Third-party mutex / p-queue |
Doesn’t model “run now but mark busy” or “wait until zero holders and zero waiters, then claim” |
| Document-only pattern | Doesn’t fix cross-request ordering with open streams |
Test / reuse nits (optional follow-ups)
- Extract shared
deferred()/tick()test helpers (duplicated across new test files and much ofpackages/engine/test/). - Direct
whenFree/recheckDelayMsunit tests inlease-backend.test.tsso defer-only paths can’t drift fromqueuebackoff.
No changes requested for merge — ship as-is; treat the items above as maintainability backlog.
Sent by Cursor Automation: Review For Best Approach
There was a problem hiding this comment.
Code Snob — a smaller shape of deferredAdmission exists. Same contract, about 26 fewer lines in arbiter.ts.
The admission builds a second AbortController whose only job is to repeat the signal of the place run claims. The host reads lost inside the run callback, after that claim, so the admission can expose claimed.lost (or NEVER_LOST before the claim). The shared-backend defer wait copies the queue wait's abortable timer; one sleep covers both. The withdrawal path and the inner finally both call leaveWaiting, which is idempotent, so one finally on the claim covers every exit. The comment above the function restates that loop.
In-memory take stays synchronous. I tried wrapping both takes in one helper; it adds a microtask on the free path, so that one lost.
POC, does not merge: #2978
Fold that subtraction into this branch. Close the POC unmerged after this lands.
Sent by Cursor Automation: Code Snob
…one lost (FIX-1834) A claimed defer no longer borrows the reject policy name: admissions take an explicit turn rule (has-turn, or waits with a budget), and the unbounded budget is the named UNBOUNDED_TURN_BUDGET. The defer admission exposes the claimed admission's lost signal through a getter instead of mirroring it, shares pollForTurn's abortable sleep, and leaves its waiting count at the run's start or once when the run settles. Tests cover a defer's release and lost signal. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: afc452e1-3440-4603-9ea7-e749c7781e7c) |
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Bugbot couldn't run - usage limit reachedBugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit. A user or team admin can review and increase usage limits in the Cursor dashboard. (requestId: 59037e76-049b-4d88-9e87-fa0217942160) |


The problem
A delegate can finish a task while the coordinator still replies in the same conversation. The task's notice then starts a second reply at once, and two replies stream in one conversation. FIX-1794's rule BR-27 says the notice runs after the reply. Today nothing holds a conversation while a reply runs.
What this does
The engine gets two new concurrency policies. Neither knows about notices or workers.
hold: the request runs at once, likeallow, and marks its key busy while it runs. It never waits and is never refused.defer: the request waits until nothing on its key runs or waits, then runs. Deferred requests run one at a time.A coordinator's reply declares
hold, and the notice entry declaresdefer. FIX-1794 P2 (#2935) adopts the primitive next: it sets those two policies on its BR-27 path and adds the goal leg that proves one reply runs at a time.allowby defaultWhat's asked of you
Nothing.
The rest of this is for whoever reviews the code.
Engineering calls: the design note (no separate spec, by the product owner's direction)
hold= take a place, skip the wait. It joins the key's line likequeueand starts at once likereject's claim. So aqueuewaiter lines up behind it and arejectis refused while it runs. That is coherent with the key being busy.defer=reject's claim, retried. It takes nothing at admission. When due, it claims the key only if no place is held or waiting (theifEmptytakerejectuses), else waits and retries. In memory it is woken when the key's line empties (a new privatewhenFreeon the in-memory backend). On a shared backend it polls with the queue's backoff (recheckDelayMs, factored out ofplanQueueWait). A claimeddeferholds the key, so defers serialize and aholdstill starts beside one.defer. A plain line lets a reply that started after the notice queued be overlapped by it. The claim-when-free form waits for every running reply, including later ones. The price: continuous replies can starve a notice until a gap. Documented.deferwait has no time budget, so two bounds keep it from being a resource-exhaustion path. (1) At most 32deferrequests wait on a key per process; the next is refused at admission withConcurrencyDeferLimitError, a subclass ofConcurrencyRejectedError, so every adapter already maps it (409 / skipped 200 / server-busy) and nothing is materialized. The count is left exactly once: when the run starts, when the wait is withdrawn or fails, or when the dispatch is released before running. (2) A defer yields to newerholdruns for 30 s (deferPatienceMs); then it takes an ordinary place behind the runs present and waits its turn with no budget, so back-to-back holds cannot starve it. Both are arbiter options for tests; the defaults are not public configuration.concurrency, elserequest.concurrency, found through the trustedsource. Body,metadataand headers cannot selectholdordefer; tests below forge all of them.queuewaiter: no, holds that start later join behind it. Makerejectrequests on the key refused while holds run: yes, which is whatrejectmeans. Run many holds at once: yes, exactly as many asallowlets it today, with no new cap.holdanddefernow take their place only after ownership passes, on every backend. Adeferover its cap is refused through the handle (accepted/finished), not by a synchronous throw. The key istenant:session, and only callers admitted to that session reach it, so two principals cannot share a key under the default"session"key. A custom key function remains the author's choice, as it is forqueueandreject.allowunder hold/defer; FIX-1836 adds it.c2dc0b7f1). The owed-notice marker is cleared only inside the conversation'sonTaskSettled(task-settled.ts,clearNoticeafterisNoticeOwed), so a notice whose request never runs keeps its marker. A cap refusal reachessendOwedNoticesas a not-started outcome (dispatch-operation.tsreports a rejectedacceptedasnotStarted), andtask-entry.ts'sonRefusedclears the marker only forsession-not-found/session-not-addressable. The next touch of the board replays it (board.ts→replayNotices). Tests on that branch:conversation-board-notices.test.ts"leaves the notice owed on the row, and the next touch of the board sends it, once" and "is refused when the conversation's worker was fired; the ending stands, still owed".allowstays the default, and a flow that declares neither policy sees no difference. No client or react change.Everything else: red to green, checks, files
Red, then green.
allow/allow): 4 of 8 fail. The notice starts mid-reply, and after an abortqueue): "every reply starts at once" fails, and the second reply waitsallowrun of the same scripthold-defer.test.ts, plus 2 abort/throw cases in the host testSecurity follow-up, red to green.
metadata.concurrencywin: all 3 "who chooses hold and defer" tests failConcurrencyDeferLimitErrorrejectaction threwConcurrencyRejectedErrorThe crash test has a control: a live process renewing its hold past three lease lengths keeps the notice waiting. Expiry frees a dead holder, not a slow one.
Checks run (this branch, head below):
engine: 274 files, 3610 passed, 6 skipped. One unrelated test (state-container-lock.test.ts, no arbiter use) failed once under full-suite load and passed 3 of 3 alone.core: 2316 passed, 4 failed. All 4 are inspec-folder-check.test.ts(the spec guard CLI run in a temp dir). This change does not touch that test orscripts/validate-spec-folder.mjs. It looks dependent on the environment (the temp path in this sandbox). Please confirm on CI.turbo run typecheck --concurrency=2: 74 of 74.Files.
core:types/concurrency.ts(the two names, validation), README line, validation test.engine:transports/concurrency/arbiter.ts(holdskips the wait,deferredAdmission),lease-backend.ts(recheckDelayMs, in-memorywhenFree),host/createInboundTransportHost.ts(defertakes the queued branch; external dispatch under either runs asallow), README.test/transports/host-concurrency-hold-defer.test.ts,test/transports/concurrency/hold-defer.test.ts.docs/architecture/inbound-transports.md(Concurrency arbitration),apps/docs/docs/advanced/concurrency-policies.md(new section, checked by the docs editor).coreandengine,minor.FIX-1834 · Linear · epic FIX-1786 · unblocks FIX-1794 P2 (#2935)
🤖 Generated with Claude Code
Note
Medium Risk
Changes shared dispatch arbitration and admission ordering (ownership before hold/defer), with new unbounded-wait semantics bounded by caps; incorrect arbiter behavior could affect session ordering or resource use under load.
Overview
Adds two concurrency policies so conversation-style flows can keep a user reply from overlapping background follow-ups without slowing the user’s next message like
queuewould.holdruns immediately (likeallow) but occupies the concurrency key for the whole run; it never waits or gets refused.deferdoes not take a place at admission—it polls until the key has no holders or waiters, then claims and runs (serialized with other defers). User turns can usehold; job notices or similar follow-ups can usedeferso they start after active replies instead of streaming beside them.The arbiter implements
holdas an immediate turn,deferviadeferredAdmission(in-memorywhenFreewakeups, shared-backend backoff viarecheckDelayMs), optional caps (maxDeferredPerKey32 →ConcurrencyDeferLimitError), and 30s patience before a waiting defer stops yielding to newerholdruns.createInboundTransportHosttreatsdeferlikequeuefor stub/heartbeat/cancel while waiting;hold/deferadmit only after session ownership; jobs sent to an external worker under either policy run asallow.@flow-state-dev/coreextendsConcurrencyPolicyName, validation, and README; docs add a hold/defer guide and architecture notes. Tests cover arbiter edge cases and fullhost.dispatchbehavior.Reviewed by Cursor Bugbot for commit 0f92c0f. Configure here.