Skip to content

feat(engine): a reply holds its conversation, so a task's notice waits for it (FIX-1834) - #2970

Merged
jhoffner merged 6 commits into
mainfrom
fix/FIX-1834-reply-hold
Oct 9, 2026
Merged

jhoffner merged 6 commits into
mainfrom
fix/FIX-1834-reply-hold

Conversation

@jhoffner

@jhoffner jhoffner commented Oct 9, 2026 •

Copy link
Copy Markdown
Collaborator

The problem

A delegate can finish a task while the coordinator still replies in the same conversation. The task's notice then starts a second reply at once, and two replies stream in one conversation. FIX-1794's rule BR-27 says the notice runs after the reply. Today nothing holds a conversation while a reply runs.

What this does

The engine gets two new concurrency policies. Neither knows about notices or workers.

  • hold: the request runs at once, like allow, and marks its key busy while it runs. It never waits and is never refused.
  • defer: the request waits until nothing on its key runs or waits, then runs. Deferred requests run one at a time.

A coordinator's reply declares hold, and the notice entry declares defer. FIX-1794 P2 (#2935) adopts the primitive next: it sets those two policies on its BR-27 path and adds the goal leg that proves one reply runs at a time.

Someone who… Today After
gets a task back while the coordinator replies Sees two replies stream at once Sees the notice's reply after the first reply ends
sends a second message mid-reply Reply starts at once Same: starts at once, never refused
sees a reply crash or get cancelled n/a The waiting notice runs right after
runs several server processes on a shared lease backend, and one dies mid-reply n/a The notice runs when the dead process's lease runs out
uses any flow that does not declare these policies allow by default No change

What's asked of you

Nothing.


The rest of this is for whoever reviews the code.

Engineering calls: the design note (no separate spec, by the product owner's direction)
  • Reuse, not a new mechanism. Both policies sit on the existing concurrency arbiter and its lease backend (FIX-837, FIX-1634). Admission, the queued stub record, heartbeat, cancel watch, renewals and lease expiry are all the existing machinery.
  • hold = take a place, skip the wait. It joins the key's line like queue and starts at once like reject's claim. So a queue waiter lines up behind it and a reject is refused while it runs. That is coherent with the key being busy.
  • defer = reject's claim, retried. It takes nothing at admission. When due, it claims the key only if no place is held or waiting (the ifEmpty take reject uses), else waits and retries. In memory it is woken when the key's line empties (a new private whenFree on the in-memory backend). On a shared backend it polls with the queue's backoff (recheckDelayMs, factored out of planQueueWait). A claimed defer holds the key, so defers serialize and a hold still starts beside one.
  • Why not FIFO for defer. A plain line lets a reply that started after the notice queued be overlapped by it. The claim-when-free form waits for every running reply, including later ones. The price: continuous replies can starve a notice until a gap. Documented.
  • Crash path, named. Abort or error: the reply's run settles and gives its place back, which wakes the notice (tests below). Process crash, in memory: every place dies with the process, so nothing in the engine stays stuck. Process crash, shared backend: the dead process stops renewing its place, the lease expires, and the deferred claim succeeds on its next check. No flag survives a crash, because nothing is written for a hold except a lease.
  • Bounds (security review follow-up). The defer wait has no time budget, so two bounds keep it from being a resource-exhaustion path. (1) At most 32 defer requests wait on a key per process; the next is refused at admission with ConcurrencyDeferLimitError, a subclass of ConcurrencyRejectedError, so every adapter already maps it (409 / skipped 200 / server-busy) and nothing is materialized. The count is left exactly once: when the run starts, when the wait is withdrawn or fails, or when the dispatch is released before running. (2) A defer yields to newer hold runs for 30 s (deferPatienceMs); then it takes an ordinary place behind the runs present and waits its turn with no budget, so back-to-back holds cannot starve it. Both are arbiter options for tests; the defaults are not public configuration.
  • Who picks the policy (BP-031). Only the flow's declaration: the entry's concurrency, else request.concurrency, found through the trusted source. Body, metadata and headers cannot select hold or defer; tests below forge all of them.
  • What a caller sending holds can do. Starve a defer: bounded by the patience. Delay a queue waiter: no, holds that start later join behind it. Make reject requests on the key refused while holds run: yes, which is what reject means. Run many holds at once: yes, exactly as many as allow lets it today, with no new cap.
  • Ownership before the key (second security review). Over the in-memory default the arbiter used to admit before the session's owner was checked, so another user could, for one store read, mark the owner's key held or fill its defer cap. hold and defer now take their place only after ownership passes, on every backend. A defer over its cap is refused through the handle (accepted / finished), not by a synchronous throw. The key is tenant:session, and only callers admitted to that session reach it, so two principals cannot share a key under the default "session" key. A custom key function remains the author's choice, as it is for queue and reject.
  • Decided: no time limit on a waiting notice, given the 32-per-conversation cap and the 30s patience (product owner, 2026-10-09).
  • Work handed to BullMQ workers runs as allow under hold/defer; FIX-1836 adds it.
  • A refused notice is late, not lost (checked on FIX-1794 P2: a conversation files tasks for its delegates and hears each one end #2935 at c2dc0b7f1). The owed-notice marker is cleared only inside the conversation's onTaskSettled (task-settled.ts, clearNotice after isNoticeOwed), so a notice whose request never runs keeps its marker. A cap refusal reaches sendOwedNotices as a not-started outcome (dispatch-operation.ts reports a rejected accepted as notStarted), and task-entry.ts's onRefused clears the marker only for session-not-found / session-not-addressable. The next touch of the board replays it (board.ts → replayNotices). Tests on that branch: conversation-board-notices.test.ts "leaves the notice owed on the row, and the next touch of the board sends it, once" and "is refused when the conversation's worker was fired; the ending stands, still owed".
  • The default is unchanged. allow stays the default, and a flow that declares neither policy sees no difference. No client or react change.
  • Rejected options (from FIX-1794 P2: a conversation files tasks for its delegates and hears each one end #2935): queue every reply (delays and refuses a person's message), a Workforce "replying" flag (stuck after a crash).
Everything else: red to green, checks, files

Red, then green.

Case Red Green
A notice that arrives mid-reply runs after it Same test with today's policies (allow/allow): 4 of 8 fail. The notice starts mid-reply, and after an abort 8 of 8
A person's message mid-reply behaves exactly as today Before the arbiter change (policy validated, but the arbiter treated it as queue): "every reply starts at once" fails, and the second reply waits Passes, and asserts the same observation sequence as an allow run of the same script
Queued notices still run after a crash or abort Abort and throw: red under today's policies (above). Crash on a shared backend: mutation that ends the deferred wait only on a give-back makes both crash-path tests time out 5 of 5 in hold-defer.test.ts, plus 2 abort/throw cases in the host test

Security follow-up, red to green.

Point Red Green
1. Cap on waiting defers Before the fix, the arbiter test "refuses a defer past the per-key cap" failed (the third defer was admitted). The host test "refuses the request past the cap at dispatch" fails under a mutation that removes the cap Both pass. The withdrawn-defer test shows a cancelled wait frees its slot
2. Caller cannot pick the policy No bug: the policy was already read only from the flow. Mutation that lets metadata.concurrency win: all 3 "who chooses hold and defer" tests fail 3 of 3 pass with forged body and metadata
3. Hold starvation Before the fix, "stops yielding to newer holds after its patience" failed: the defer ran only after an unbroken 1.15 s chain of holds Passes: it runs within 600 ms with a 100 ms patience
5. Another user fills the owner's defer cap "cannot use up the owner's defer cap" failed: the owner's notice was refused with ConcurrencyDeferLimitError Passes: the foreign defers fail ownership, and the owner's notice runs after the reply
6. Another user holds the owner's session "cannot hold the owner's session, even for a moment" failed: the owner's reject action threw ConcurrencyRejectedError Passes
4. No time limit only with the cap The cap of 32 is what bounds the wait

The crash test has a control: a live process renewing its hold past three lease lengths keeps the notice waiting. Expiry frees a dead holder, not a slow one.

Checks run (this branch, head below):

  • engine: 274 files, 3610 passed, 6 skipped. One unrelated test (state-container-lock.test.ts, no arbiter use) failed once under full-suite load and passed 3 of 3 alone.
  • core: 2316 passed, 4 failed. All 4 are in spec-folder-check.test.ts (the spec guard CLI run in a temp dir). This change does not touch that test or scripts/validate-spec-folder.mjs. It looks dependent on the environment (the temp path in this sandbox). Please confirm on CI.
  • turbo run typecheck --concurrency=2: 74 of 74.

Files.

  • core: types/concurrency.ts (the two names, validation), README line, validation test.
  • engine: transports/concurrency/arbiter.ts (hold skips the wait, deferredAdmission), lease-backend.ts (recheckDelayMs, in-memory whenFree), host/createInboundTransportHost.ts (defer takes the queued branch; external dispatch under either runs as allow), README.
  • Tests: test/transports/host-concurrency-hold-defer.test.ts, test/transports/concurrency/hold-defer.test.ts.
  • Docs: docs/architecture/inbound-transports.md (Concurrency arbitration), apps/docs/docs/advanced/concurrency-policies.md (new section, checked by the docs editor).
  • Changeset: core and engine, minor.

FIX-1834 · Linear · epic FIX-1786 · unblocks FIX-1794 P2 (#2935)

🤖 Generated with Claude Code


Note

Medium Risk
Changes shared dispatch arbitration and admission ordering (ownership before hold/defer), with new unbounded-wait semantics bounded by caps; incorrect arbiter behavior could affect session ordering or resource use under load.

Overview
Adds two concurrency policies so conversation-style flows can keep a user reply from overlapping background follow-ups without slowing the user’s next message like queue would.

hold runs immediately (like allow) but occupies the concurrency key for the whole run; it never waits or gets refused. defer does not take a place at admission—it polls until the key has no holders or waiters, then claims and runs (serialized with other defers). User turns can use hold; job notices or similar follow-ups can use defer so they start after active replies instead of streaming beside them.

The arbiter implements hold as an immediate turn, defer via deferredAdmission (in-memory whenFree wakeups, shared-backend backoff via recheckDelayMs), optional caps (maxDeferredPerKey 32 → ConcurrencyDeferLimitError), and 30s patience before a waiting defer stops yielding to newer hold runs. createInboundTransportHost treats defer like queue for stub/heartbeat/cancel while waiting; hold/defer admit only after session ownership; jobs sent to an external worker under either policy run as allow.

@flow-state-dev/core extends ConcurrencyPolicyName, validation, and README; docs add a hold/defer guide and architecture notes. Tests cover arbiter edge cases and full host.dispatch behavior.

Reviewed by Cursor Bugbot for commit 0f92c0f. Configure here.

jhoffner and others added 2 commits October 9, 2026 14:56
A hold request runs at once and marks its key busy; a defer request waits
until nothing on its key is running or waiting, then runs. Lets follow-up
work for a conversation (a task's notice) land after the current reply
without delaying the person's own next message. The defer wait has no
budget: it ends when the holder settles (success, error or abort), or when
a crashed process's place on a shared lease backend expires.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@vercel

vercel Bot commented Oct 9, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

3 Skipped Deployments
Project Deployment Actions Updated
flow-state-dev-docs Ignored Ignored Oct 9, 2026 7:50pm UTC
flow-state-dev-kitchen-sink Ignored Ignored Oct 9, 2026 7:50pm UTC
flow-state-dev-ui Ignored Ignored Preview Oct 9, 2026 7:50pm UTC

Request Review

@linear-code

linear-code Bot commented Oct 9, 2026

Copy link
Copy Markdown

FIX-1834

@cursor

cursor Bot commented Oct 9, 2026

Copy link
Copy Markdown
Contributor

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: 261c5a5a-6a5a-42e9-996c-79c57a48e308)

Caps waiting defer requests per key per process (32); the next is refused
with ConcurrencyDeferLimitError, a ConcurrencyRejectedError, before anything
is created. A defer yields to newer hold runs for 30s only, then lines up
behind the runs present and waits for its turn. Tests show the policy is
read only from the flow declaration, never caller body or metadata.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@cursor

cursor Bot commented Oct 9, 2026

Copy link
Copy Markdown
Contributor

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: 96168a92-351f-4282-a251-9cc47b61ddf8)

…X-1834)

On the in-memory default the arbiter admitted before the session's owner
was checked, so a caller who did not own a session could briefly mark its
key held (refusing the owner's reject) or fill its defer cap (refusing the
owner's notices). hold and defer now admit after admitOwnership on every
backend; a defer over its cap is refused through the handle.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@cursor

cursor Bot commented Oct 9, 2026

Copy link
Copy Markdown
Contributor

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: b5439aa6-b5d1-4f6d-859c-2d7f46986d31)

@jhoffner
jhoffner marked this pull request as ready for review October 9, 2026 19:21
@cursor

cursor Bot commented Oct 9, 2026

Copy link
Copy Markdown
Contributor

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: 399722e7-c74d-449c-b60b-a0297cd548fa)

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Problem this PR solves

Follow-up work (task notices, background completions) was able to start beside an in-flight reply on the same session. queue would serialize, but it also makes the user’s next message wait behind the reply and time out after 30s. FIX-1834 needs asymmetric rules: replies run immediately and mark the session busy; follow-ups wait until the conversation is quiet.

Verdict

Approve. The hold / defer split at the concurrency seam is the right layer for BP-031, cancelable long waits, shared-backend crash paths, and adapter-mapped refusals. I don’t see a materially simpler approach that keeps those properties without pushing the same state machine into every app.

The cost is real complexity in arbiter.ts (~130 lines + a second admission shape). That looks justified for the product contract (unbounded defer wait, 30s patience to newer holds, per-process cap of 32, serialized defers, post-ownership admit). Docs and tests match the behavior; pnpm --filter @flow-state-dev/engine test passes on this branch.

Where it went further than necessary (non-blocking)

  1. deferredAdmission parallel to admissionFor — A hand-rolled ConcurrencyAdmission duplicates run / release / lost forwarding. Future fixes to renewal, give-back, or lost propagation may need to land twice. Follow-up: fold defer-specific behavior into admissionFor via a small options object (claimMode, turnBudget) when someone touches this again — not worth a behavior-preserving mega-refactor in this PR.

  2. Policy aliasing encodes semantics — After an empty-key claim, defer returns admissionFor("reject", …); after patience expires, admissionFor("defer", …, Infinity). That works today because reject skips waitForTurn, but it’s fragile if reject and defer diverge. Follow-up: named internal mode (e.g. skipWait: true) instead of mis-labeling the policy.

  3. Infinity as unbounded wait — pollForTurn treats budgetMs === Infinity as “never time out.” Clear enough with comments; a named wait budget type would be easier to grep.

  4. Authoring footgun: external dispatch — hold / defer silently become allow when handed to BullMQ/external dispatch (documented). Consider a dev-only log or metric when downgrade happens — separate observability issue.

  5. Per-process defer cap — Asymmetric with cluster-wide queue/hold places on a shared lease backend. Docs already say “per process”; worth a one-line callout in architecture if operators assume a global cap.

Simpler approaches considered

Approach Why not enough
App-level session flag / handler gate No transport parity (webhook/MCP), no SSE cancel/stub path, BP-031 and crash recovery reimplemented per app
Single queue with lanes/priority Similar arbiter work, less obvious author API than hold vs defer
Third-party mutex / p-queue Doesn’t model “run now but mark busy” or “wait until zero holders and zero waiters, then claim”
Document-only pattern Doesn’t fix cross-request ordering with open streams

Test / reuse nits (optional follow-ups)

  • Extract shared deferred() / tick() test helpers (duplicated across new test files and much of packages/engine/test/).
  • Direct whenFree / recheckDelayMs unit tests in lease-backend.test.ts so defer-only paths can’t drift from queue backoff.

No changes requested for merge — ship as-is; treat the items above as maintainability backlog.

Open in Web View Automation 

Sent by Cursor Automation: Review For Best Approach

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Snob — a smaller shape of deferredAdmission exists. Same contract, about 26 fewer lines in arbiter.ts.

The admission builds a second AbortController whose only job is to repeat the signal of the place run claims. The host reads lost inside the run callback, after that claim, so the admission can expose claimed.lost (or NEVER_LOST before the claim). The shared-backend defer wait copies the queue wait's abortable timer; one sleep covers both. The withdrawal path and the inner finally both call leaveWaiting, which is idempotent, so one finally on the claim covers every exit. The comment above the function restates that loop.

In-memory take stays synchronous. I tried wrapping both takes in one helper; it adds a microtask on the free path, so that one lost.

POC, does not merge: #2978

Fold that subtraction into this branch. Close the POC unmerged after this lands.

Open in Web View Automation 

Sent by Cursor Automation: Code Snob

…one lost (FIX-1834)

A claimed defer no longer borrows the reject policy name: admissions take
an explicit turn rule (has-turn, or waits with a budget), and the unbounded
budget is the named UNBOUNDED_TURN_BUDGET. The defer admission exposes the
claimed admission's lost signal through a getter instead of mirroring it,
shares pollForTurn's abortable sleep, and leaves its waiting count at the
run's start or once when the run settles. Tests cover a defer's release and
lost signal.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@cursor

cursor Bot commented Oct 9, 2026

Copy link
Copy Markdown
Contributor

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: afc452e1-3440-4603-9ea7-e749c7781e7c)

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@cursor

cursor Bot commented Oct 9, 2026

Copy link
Copy Markdown
Contributor

Bugbot couldn't run - usage limit reached

Bugbot is counted against Cursor usage for this user or team, and this run hit a usage or spend limit.

A user or team admin can review and increase usage limits in the Cursor dashboard.

(requestId: 59037e76-049b-4d88-9e87-fa0217942160)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant