Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions .changeset/wait-reads-an-ended-task-once.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
---
"@flow-state-dev/orchestration": patch
---

A `getTask({ id, wait: true })` on a task that already ended returns it at once, and that answer is the filer's delivery: when the flow's `onTaskSettled` waits behind the turn that filed the task, as Workforce's turns make it, the ending's notice request still runs but no longer calls your block. Task collection refs gain an optional `dropNoticeReadByWait(id)`, which both built-in backings implement; a ref without it keeps the old behaviour (FIX-2112).
3 changes: 3 additions & 0 deletions apps/docs/docs/orchestration/task-board.md
Original file line number Diff line number Diff line change
Expand Up @@ -846,6 +846,9 @@ getTask({ id: "task_8f2", wait: true })
You can wait in the same turn or in a later one. Your turn pauses until the task ends, then
carries on with the task as `getTask` always returns it: its status, and its output once it
completed. Nothing holds a request open while it waits, so a server restart loses nothing.
A task that already ended is returned at once, and no `onTaskSettled` follows for it, as long
as your `onTaskSettled` waits behind the turn that filed the task. A Workforce conversation or
task turn works that way. The request still runs, but your block isn't called.

A wait has a deadline, five minutes unless you pass `deadline` in milliseconds. At the deadline
your turn carries on with the task as it stands, usually still `in_progress`. Nothing is
Expand Down
4 changes: 3 additions & 1 deletion goals/task-board/waits-on-a-task-it-filed/goal.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,7 +8,7 @@

**Signal:** each failure is tagged `<leg>:<assertion>`.

- `a` A conversation files with `createTask` and its request ends. A later request of that conversation waits with `getTask` and parks; once the task ends it gets the task `completed`, with its worker's token, in that request, and no `onTaskSettled` runs for it. A second conversation of the same user is refused (on a per-conversation board it can't reach the row: `task_not_found`), and a second request of the filing conversation is refused `wait_taken` while the first waits.
- `a` A conversation files with `createTask` and its request ends. A later request of that conversation waits with `getTask` and parks; once the task ends it gets the task `completed`, with its worker's token, in that request, and no `onTaskSettled` runs for it. A second conversation of the same user is refused (on a per-conversation board it can't reach the row: `task_not_found`), and a second request of the filing conversation is refused `wait_taken` while the first waits. A piece that ends between a task turn's `createTask` and its `getTask` wait, in one request, is returned by the wait, no `onTaskSettled` delivers it (a notice request that finds it no longer owed delivers nothing), and the waiting turn's answer is the task's result (FIX-2112).
- `b` Five nested waits, four through task requests (the conversation, then `doer.a` to `doer.c`) and one waited dispatch (`doer.d`'s), each resume with the result below: the top request's answer carries the probe's token. The probe's own waited dispatch, a sixth, is refused `wait_too_deep` and dispatches nothing (one probe request). A chain of five parked waits built below a task nobody waited on, then a late wait on that task, is refused `wait_too_deep` and marks nothing. A task session's turn files 101 steps for itself: 100 land on the plan ledger and the 101st is refused at the chain's cap.
- `c` A deadline shorter than the work resumes the request with the task `in_progress` on attempt 1, and a second wait gets it `completed`, still attempt 1. A task ending after the deadline while its request still runs reaches that request once, on its next `getTask`, and no `onTaskSettled` runs for it. A request that ends without waiting again hears the ending once, as `onTaskSettled`. An ending and a deadline due together resume the request once and deliver the ending once. A second process parks a wait and is killed with SIGKILL; reopened on the same store, the deadline resumes the wait within its due time plus 2 s. The task whose request waits is cancelled: that request makes no further model call and records no return.
- `d` A worker's step filed `session: "new"` runs in a session of its own while its plan's drain parks on it, waiting again at each of two deadlines (at least three parks) and starting no later step; once it ends, the next step's turn reads a token only that session's turn produced. A second plan whose step filed `session: "new"` is its last: once it ends, nothing is left open, and the worker still gets one answering turn, told that step's token, and its task completes with that answer (FIX-2110).
Expand Down Expand Up @@ -40,3 +40,5 @@ On `main` before FIX-2082 legs `a` and `b` fail: `getTask` takes no `wait`, a ta
| 2026-10-22 | fix/FIX-2082 fd85658ab (review round on #3653, main with FIX-2088 and FIX-2086 merged in) | n/a (model-free) | FAIL (control `drain-moves-on`, expected) | d only: the drain parked 1 time and moved on, and its request never finished |
| 2026-10-23 | fix/FIX-2110 on 14778344c, plan.ts fix set aside | n/a (model-free) | FAIL (expected, before the fix) | d: with the step in its own session last, 0 answering turns; its task landed the plan turn's earlier answer. The rest of d passes |
| 2026-10-23 | fix/FIX-2110 on 14778344c | n/a (model-free) | PASS | All legs. d: the drain parked 3 times; step 3 read step 2's token; with the step in its own session last, 1 answering turn, and its task landed the answer told that step's token. `drain-moves-on` still FAILs d (expected) |
| 2026-10-23 | fix/FIX-2112 da86f88b8 (on `main` 14778344c) | n/a (model-free) | PASS | All legs. a: a piece that ended before its task turn's wait was read by it; 1 `onTaskSettled` request in the task's session, admitting nothing (`owed: false`), 0 delivering it. b, c, d, e as before; the kill resumed 132 ms past due |
| 2026-10-23 | `main` 14778344c with this run's leg a (FIX-2112's fix stashed) | n/a (model-free) | FAIL (expected) | a: 1 `onTaskSettled` delivered the piece's ending after the wait read it, and its "heard" answer replaced the task's result |
51 changes: 51 additions & 0 deletions goals/task-board/waits-on-a-task-it-filed/run.mts
Original file line number Diff line number Diff line change
Expand Up @@ -471,6 +471,57 @@ await runGoal(async (failures, tell) => {
tell.over("a · onTaskSettled for a waited task", heard, { emptyIs: "the claim: the waiting request had the ending" });
if (heard.length !== 0) failures.push(`a:the ending reached its waiter, not onTaskSettled — ${heard.length} onTaskSettled request(s)`);
notes.push(`a: the later request read ${PRODUCED.get("a-work")}; another session: ${/"error":"([a-z_]+)/.exec(otherSaid)?.[1]}; the filing session's second request: wait_taken`);

// A task that ends between its filing and its filer's wait, in one request
// (a task's turn, which a notice waits behind): the wait returns it at
// once, and that is the filer's one hearing of it.
const quick = await h.http.open("desk", { workerId: "lead" });
const pieceGoal = word("a-quick-piece");
const topGoal = word("a-quick-top");
await h.send("desk", quick, "respond", {
message: scripted("a-quick-file", [
{
tool: "createTask",
args: {
goal: scripted(topGoal, [
{ tool: "createTask", args: { goal: scripted(pieceGoal, [{ answer: "a-quick-piece" }]), assignee: "doer.b" } },
{ hold: "a-quick" },
{ tool: "getTask", args: { id: "$0", wait: true } },
{ answer: "a-quick-waiter" }
]),
assignee: "doer.a"
}
},
{ answer: "a-quick-filer" }
])
});
const endedFirst = await h.until(async () => (await h.byGoal(pieceGoal))[0]?.status === "completed", 20_000);
gate("a-quick").open();
await h.until(async () => (await h.byGoal(topGoal))[0]?.status === "completed", 20_000);
await h.idle();
// Past the notice's due time, so a resend by the filer's pass has had its chance too.
await sleep(2_500);
await h.idle();
const [top] = await h.byGoal(topGoal);
const [piece] = await h.byGoal(pieceGoal);
const quickRead = top?.run?.requestId === undefined ? "" : (await h.toolOutputs(top.run.requestId, "getTask")).join(" ");
const notices = top?.run?.sessionId === undefined ? [] : (await h.requestsOf(top.run.sessionId)).filter((r) => r.actionName === "onTaskSettled");
// A notice request that finds its notice no longer owed delivers nothing (the entry's admit): count the ones it admitted.
const delivered = tell.over(
"a · onTaskSettled requests that delivered the piece's ending",
notices.filter((r) => r.result?.output?.owed === true && r.result.output.notice?.taskId === piece?.id),
{ emptyIs: "the claim: the wait's answer was the delivery" }
);
if (!endedFirst || !quickRead.includes('"status":"completed"') || !quickRead.includes(PRODUCED.get("a-quick-piece") ?? "<none>")) {
failures.push(`a:a task that ended before the wait is returned by it — ended first: ${endedFirst}; read ${quickRead.slice(0, 300)}`);
}
if (delivered.length !== 0) {
failures.push(`a:an ending read by the filer's wait starts no notice turn — ${delivered.length} onTaskSettled delivered it too`);
}
if (top?.status !== "completed" || !JSON.stringify(top.output ?? "").includes(PRODUCED.get("a-quick-waiter") ?? "<none>")) {
failures.push(`a:the waiting turn's answer is the task's result — task ${top?.status}, output ${JSON.stringify(top?.output ?? "").slice(0, 200)}`);
}
notes.push(`a: a piece that ended before its wait was read by it; ${notices.length} onTaskSettled request(s) in the task's session, ${delivered.length} delivering it`);
});

// ---- b. nested waits, and the depth ---------------------------------------------
Expand Down
7 changes: 5 additions & 2 deletions packages/orchestration/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -988,8 +988,11 @@ See [Per-generator binding](https://flow-state.dev/docs/skills/binding) for the
`getTask({ id, wait: true, deadline? })` parks the calling turn on a task its session filed,
until the task ends or the deadline passes, and returns the task as `getTask` always does. The
deadline is milliseconds from the call, five minutes by default, within the host's
`waitDeadlines` bounds. At the deadline the task runs on and nothing is cancelled. If it ends
before the waiting request does, that request's next `getTask` on it returns the ending. If
`waitDeadlines` bounds. A task that already ended is returned at once, and no `onTaskSettled`
follows for it when the flow's `onTaskSettled` waits behind the turn that filed the task, as
Workforce's `respond` and `work` turns make it: the request still runs, but your block isn't
called. At the deadline the task runs on and nothing is cancelled. If it ends before the
waiting request does, that request's next `getTask` on it returns the ending. If
the request ends without reading it, the ending goes to `onTaskSettled`, once. Waits nest: a
task's own turn may wait too. A wait is refused when it would make a chain of more than five
waits (`MAX_WAIT_DEPTH`), counting the waits above you and any already below the task you wait
Expand Down
15 changes: 14 additions & 1 deletion packages/orchestration/src/tasks/collection/internal.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ import {
} from "../schema/task-status";
import { extractTaskItems } from "../items/extract-window";
import { applyEndingRecorder, endingOf, type TaskEnding, type TaskEndingRecorder } from "./ending";
import { droppedHeldNotices, recordEnding as recordNoticeOwed, releasedHeldNotices } from "../notice/task-notice";
import { deliveredByWait, droppedHeldNotices, recordEnding as recordNoticeOwed, releasedHeldNotices } from "../notice/task-notice";
import { assertDepsEditable, TaskFilingRefusedError } from "./filing";
import { hasStarted, lockedFields, type TaskEditField } from "./field-rules";
import { steersForAttempt, untakenSteers } from "../steers";
Expand Down Expand Up @@ -279,6 +279,19 @@ export function releaseWaitWrite(
};
}

/**
* The one rule for a wait that found its row already ended
* (`TaskCollectionRef.dropNoticeReadByWait`, FIX-2112): the wait's answer
* delivered the ending, so the notice the ending owes the filer is dropped.
*
* @returns The patch to write, or `undefined` when the row owes no notice for its ending.
*/
export function noticeReadByWaitWrite(task: Task): { patch: Partial<Task> } | undefined {
const dropped = deliveredByWait(task);
if (Object.keys(dropped).length === 0) return undefined;
return { patch: { metadata: { ...(task.metadata ?? {}), ...dropped } } };
}

/**
* The fields a filing may change until the task first starts (FIX-2080,
* ER-4; the brief's fields and the assignee since FIX-2081). Each field's rule
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -135,6 +135,7 @@ import {
lapseWaitWrite,
markWaitWrite,
releaseWaitWrite,
noticeReadByWaitWrite,
settingsWrite,
settingsChangeKind,
type TaskSettingsPatch,
Expand Down Expand Up @@ -1386,6 +1387,10 @@ export async function createResourceBackedTaskCollection<TInput = unknown, TOutp
return patchRef(id, "wait_changed", (task) => releaseWaitWrite(task as Task, { ...release, dueAt })?.patch as Partial<Task<TInput, TOutput>> | undefined);
},

async dropNoticeReadByWait(id) {
return patchRef(id, "metadata_changed", (task) => noticeReadByWaitWrite(task as Task)?.patch as Partial<Task<TInput, TOutput>> | undefined);
},

async refresh(id) {
const taskRef = mirror.get(id);
if (taskRef !== undefined) await readCommitted(taskRef, () => undefined);
Expand Down
5 changes: 5 additions & 0 deletions packages/orchestration/src/tasks/collection/state-backed.ts
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,7 @@ import {
lapseWaitWrite,
markWaitWrite,
releaseWaitWrite,
noticeReadByWaitWrite,
settingsWrite,
settingsChangeKind,
type EndingRecording,
Expand Down Expand Up @@ -1069,6 +1070,10 @@ export function createStateBackedTaskCollection<TInput = unknown, TOutput = unkn
return patchOne(id, "wait_changed", (task) => releaseWaitWrite(task as Task, { ...release, dueAt })?.patch as Partial<Task<TInput, TOutput>> | undefined);
},

async dropNoticeReadByWait(id) {
return patchOne(id, "metadata_changed", (task) => noticeReadByWaitWrite(task as Task)?.patch as Partial<Task<TInput, TOutput>> | undefined);
},

async payOwedCascades() {
return payOwedCascades(cascadeWrites, now());
},
Expand Down
10 changes: 10 additions & 0 deletions packages/orchestration/src/tasks/collection/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -1041,6 +1041,16 @@ export interface TaskCollectionRef<TInput = unknown, TOutput = unknown> {
* the slot is not a lapsed one that request holds.
*/
releaseWait?(id: string, release: { readonly requestId: string; readonly read: boolean }): Promise<TaskWriteOutcome>;
/**
* The filer's wait found the row already ended and returned it (FIX-2112):
* drop the notice the ending owes the filer, which that answer delivered.
* `unchanged` when the row owes none for its ending. Server-owned, as the
* other wait writes are: a caller's `patchMetadata` can't clear a notice.
*
* Optional: both built-in backings implement it. On a ref without it the
* filer hears the ending again, as `onTaskSettled`.
*/
dropNoticeReadByWait?(id: string): Promise<TaskWriteOutcome>;

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Optional dropNoticeReadByWait means a custom TaskCollectionRef that implements waits but not this method keeps pre-2112 double-delivery on the fast path. The doc comment calls that out — worth a one-line note in package README’s “implementing a collection” area if such hosts exist, so it’s not only discoverable from the type.

/**
* A touch of the board: pay every cascade owed past its due time
* (`Task.cascadeOwed`, FIX-2080 D2), so one a crash left owed is paid.
Expand Down
28 changes: 25 additions & 3 deletions packages/orchestration/src/tasks/helpers/wait-on-task.ts
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,8 @@
* any already below the task it waits on, counted by the engine. A wait
* that waits again at its deadline counts again before each new park,
* except a drain on a row it handed over, whose wait already exists.
* A task already ended is returned at once.
* A task already ended is returned at once, and the notice its ending
* owes the filer is dropped: this answer delivered it (FIX-2112).
* 2. **Take the row's wait slot** in one conditional write (`markWait`): a
* second request waiting on the same task is refused.
* 3. **Park** on the one wait gate (`parkOnWait`), whose deadline the engine
Expand Down Expand Up @@ -66,7 +67,7 @@ import type { TaskCollectionRef } from "../collection/types";
import type { TaskClaimTicket } from "../claim-ticket";
import { resolveResourceCollection } from "../collection/resolve-resource-collection";
import { getOrCreateTaskCollection } from "../collection/get-or-create";
import { holdsAnEnding, LAPSED_SLOT_CHECK_CAP_MS, LAPSED_SLOT_CHECK_MS } from "../notice/task-notice";
import { deliveredByWait, holdsAnEnding, LAPSED_SLOT_CHECK_CAP_MS, LAPSED_SLOT_CHECK_MS } from "../notice/task-notice";
// The running request's own task, as the claim and the gate know it. Reached by
// path, not through the task-board barrel, which imports this module.
import { currentWorkerClaim } from "../../task-board/flow-policy-wiring";
Expand Down Expand Up @@ -481,14 +482,35 @@ function claimBoardOf(ctx: BlockContext, collectionId: string) {
}
}

/** The waited-on row ended: clear what it owes the resume, and return it. */
/**
* The waited-on row ended: clear what it owes the resume, and return it. An
* ending that came before the wait owes its filer a notice; when the filer is
* the one waiting, this answer delivers it, so it is dropped (FIX-2112).
*/
async function ended(ctx: BlockContext, collection: TaskCollectionRef, taskId: string): Promise<TaskWaitResult> {
await clearMarker(collection, taskId);
const task = collection.get(taskId)!;
await deliverHeldEnding(ctx, collection, task);
await deliverEndedBeforeWait(ctx, collection, collection.get(taskId) ?? task);
Comment thread
jhoffner marked this conversation as resolved.
Resolved

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small readability win: ended() calls collection.get(taskId) three times in a row. After deliverHeldEnding, one fresh read into a let row would be enough for both deliverEndedBeforeWait and the return value (metadata may change across the writes). Not a behavior issue.

return { ok: true, task: collection.get(taskId) ?? task, ended: true };
}

/**
* Drop the notice an ending owes its filer, when the filer's own wait read it
* (FIX-2112). Never fails the call: a drop that fails leaves the notice owed,
* and the filer hears the ending a second time.
*/
async function deliverEndedBeforeWait(ctx: BlockContext, collection: TaskCollectionRef, task: Task): Promise<void> {
Comment thread
jhoffner marked this conversation as resolved.
Resolved
const filer = collection.filerOf?.(task) ?? task.filer?.sessionId;
if (filer === undefined || filer !== ctx.session.identity.id) return;
if (Object.keys(deliveredByWait(task)).length === 0) return;
try {
await collection.dropNoticeReadByWait?.(task.id);
} catch (error) {
console.warn(`[orchestration] wait "${task.id}": its ending's notice stays owed: ${(error as Error).message}`);
}
}

/**
* Clear the row's resume-owed marker. Never fails the call: a clear that fails
* leaves the marker, and the next touch clears it, the gate already resolved.
Expand Down
17 changes: 16 additions & 1 deletion packages/orchestration/src/tasks/notice/task-notice.ts
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,9 @@
*
* A retry owes none: it is the row's start, not its ending. A waited-on row
* owes none either: the request parked on it is resumed instead
* (`Task.resumeOwed`), and no notice runs for it.
* (`Task.resumeOwed`), and no notice runs for it. A row that ended before its
* filer's wait on it was made owes one at its ending, and the wait that finds
* it ended drops it ({@link deliveredByWait}): the wait's answer delivered it.
*
* ## An ending held for the request that waited (FIX-2082)
*
Expand Down Expand Up @@ -218,6 +220,19 @@ export function droppedHeldNotices(row: Task): Record<string, null> {
return patch;
}

/**
* The metadata patch that drops the notice `row`'s ending owes, when a wait on
* the row found it already ended (FIX-2112): the wait's answer is the
* ending's delivery, as it is for a wait the ending resumes. Empty when the
* row hasn't ended or owes nothing for it. A marker held for a request whose
* wait lapsed is that request's to deliver ({@link droppedHeldNotices}).
*/
export function deliveredByWait(row: Task): Record<string, null> {
if (row.status !== "completed" && row.status !== "errored" && row.status !== "cancelled") return {};
const key = markerKey(row.attempts, row.status);
return isSendable(key, (row.metadata ?? {})[key]) ? { [key]: null } : {};
}

/**
* The metadata patch that releases every held marker on `row`, due at `dueAt`
* (FIX-2082): the request it was held for ended without reading the row, so
Expand Down
Loading
Loading