Repository navigation
spec(FIX-1654): Abort fence should compare the request's incarnation, not createdAt #2415
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
+794
−0
Merged
Changes from all commits
Commits
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,47 @@ | ||
| # FIX-1654 · Business rules | ||
|
|
||
| [Spec](SPEC.md) · [Decisions](DECISIONS.md) · **Rules** · [Plan](PLAN.md) · [Docs](DOCS.md) · [Evolution](EVOLUTION.md) | ||
|
|
||
| The cases, as rules. "Cancel" is `POST …/requests/:id/abort`; "the fence" is the check that its | ||
| write lands on the record its owner check read. "Incarnation" is FIX-1286's token, resolved | ||
| for a legacy record as `legacy_<createdAt>`. The *proved by* column is the check the plan runs. | ||
|
|
||
| ## The record changed between the check and the write | ||
|
|
||
| | # | When | Then | Proved by | | ||
| |---|---|---|---| | ||
| | BR-1 | Another tenant's or user's request takes the freed id, in the same millisecond | 404, nothing written; the other request runs on | Route case · conformance, equal `createdAt` | | ||
| | BR-2 | The same, in a later millisecond | As BR-1, as today | Existing route case (FIX-1021), kept | | ||
| | BR-3 | The caller's own retry hands the record off, rewriting `createdAt` | 202, the intent is on the caller's running request | Route case through `claimRequestRecord` · conformance, rewritten `createdAt` | | ||
| | BR-4 | Nothing changed | 202 or 204, as today | Existing route cases | | ||
| | BR-5 | The request finished in between | 409 with its terminal status, as today | Existing route and conformance cases | | ||
| | BR-12 | The write landed, then the request finished and a later request under the id started in this process before the route fires the local controller | The later request's controller is not fired; the route answers 202, the intent sits on the finished record | Route case with a tagged registry · integration scenario | | ||
| | BR-13 | The request is queued in this process, its controller registered before its record is claimed | A fenced cancel doesn't fire the untagged controller; it answers 202 and the run sees the intent when it starts. Callers that fire by id with no fence (the CLI, cascading child aborts) behave as today | Registry unit case | | ||
|
|
||
| ## Records from before incarnations | ||
|
|
||
| | # | When | Then | Proved by | | ||
| |---|---|---|---| | ||
| | BR-6 | A legacy record, untouched, is cancelled | Fenced on `legacy_<createdAt>`; applies | Conformance, every store | | ||
| | BR-7 | A legacy record was handed off, so it now stores `legacy_<old createdAt>` beside a new `createdAt` | Fenced on the stored value; applies | Conformance, every store | | ||
| | BR-8 | A legacy record's id is taken by a new, stamped request | `inc_…` never equals `legacy_…`; 404 | Conformance, every store | | ||
| | BR-9 | A stored incarnation is `null` rather than absent | Treated as absent, as `resolveRequestIncarnation` does | Conformance, one case | | ||
|
|
||
| ## Callers and stores | ||
|
|
||
| | # | When | Then | Proved by | | ||
| |---|---|---|---| | ||
| | BR-10 | A caller passes no fence | Status predicate only, as today | Existing conformance cases | | ||
| | BR-11 | An out-of-tree store still compares `createdAt` | Its build fails on the parameter's type; untyped, every fenced cancel misses and the conformance suite's fence case fails | Changeset note · conformance | | ||
|
|
||
| ## Failure taxonomy | ||
|
|
||
| Nothing new throws or refuses. A fenced miss is reported as an absent record, and the route's | ||
| 404 is the one it gives today. BR-3 turns a wrong 404 into a right 202. | ||
|
|
||
| ## Acceptance criteria this issue owns | ||
|
|
||
| [The goal](SPEC.md#the-goal-and-how-well-know-its-met): BR-1, BR-3 and BR-12 through the route on | ||
| the memory store and in the integration scenario, and BR-1, BR-3 and BR-6 to BR-9 in the | ||
| conformance suite on all four first-party stores; the route cases failed on `main` `70f777def` | ||
| before this change, and the PR names it. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,94 @@ | ||
| # FIX-1654 · Decisions | ||
|
|
||
| [Spec](SPEC.md) · **Decisions** · [Rules](BUSINESS-RULES.md) · [Plan](PLAN.md) · [Docs](DOCS.md) · [Evolution](EVOLUTION.md) | ||
|
|
||
| One decision is the sign-off. The rest follow from FIX-1286's | ||
| [D1](../FIX-1286/DECISIONS.md#d1), which already made the incarnation the request's identity; | ||
| this issue moves the one check still reading a timestamp onto it. | ||
|
|
||
| ## The tree | ||
|
|
||
| ```mermaid | ||
| flowchart TD | ||
| I["FIX-1654"] --> S["Settled · the createdAt fence fails both ways on main"] | ||
| I --> D1["D1 · the fence compares the incarnation, replacing createdAt"] | ||
| D1 -.->|"rejected · the owner's hand-off still misses"| X1["fence on both, incarnation beside createdAt"] | ||
| D1 -.->|"rejected · two fences with one meaning"| X2["accept either for a deprecation window"] | ||
| D1 -.->|"rejected · the leak stays"| X3["keep createdAt and document the window"] | ||
| ``` | ||
|
|
||
| Solid edges are what you're signing. Dashed edges lost, and the label says why. | ||
|
|
||
| <a name="d1"></a> | ||
| ## D1 · The fence names a request by its incarnation, replacing `createdAt` outright | ||
|
|
||
| | | | | ||
| |---|---| | ||
| | **Instead of** | Adding the incarnation as a second fence beside `createdAt` · accepting either for a deprecation window · keeping `createdAt` and documenting the same-millisecond window | | ||
| | **Because** | The [POC](PLAN.md#sketch-and-poc) shows `createdAt` failing in both directions: a same-millisecond reuse lets the cancel land on another tenant's request, and a same-owner hand-off, which rewrites `createdAt` and keeps the incarnation, makes the owner's own cancel miss. Any option that still compares `createdAt` keeps the second failure. FIX-1286 made the incarnation the identity; a second notion of "the same request" in the store contract is the incoherence tenet 1 warns against | | ||
| | **Locks in** | `RequestStore.setFieldsIfStatus`'s fifth argument is the caller's resolved incarnation, and every store compares it with the stored record's, deriving a legacy record's the one way `resolveRequestIncarnation` does. An out-of-tree store fails to compile until it follows; an untyped one fails closed (every fenced cancel answers 404) and the conformance suite names the case | | ||
|
|
||
|  | ||
|
|
||
| It comes down to the owner's hand-off: only dropping `createdAt` lets that cancel land. | ||
|
|
||
| **What would change my mind:** a known out-of-tree request store we must not break this release. | ||
| Then the incarnation goes in as an options field the old signature ignores, and the break moves to | ||
| the next minor. | ||
|
|
||
| **If wrong:** an adapter author meets a compile error for a window only a same-millisecond reuse | ||
| opens. The other way, cancel keeps two failure modes, one of them cross-tenant. | ||
|
|
||
| ## Decided, not asked | ||
|
|
||
| - **The caller passes the resolved value.** The route passes `resolveRequestIncarnation(record)`, | ||
| so a legacy record's fence is `legacy_<createdAt>` and behaves exactly as today's (BP-030). | ||
| - **Stores resolve the stored side the same way.** `resolveRequestIncarnation` is exported from | ||
| `@flow-state-dev/engine` for engine and out-of-tree TypeScript stores. SQLite restates the | ||
| one-line rule locally, because it may import only types from `engine`; Postgres states it in SQL. | ||
| The conformance suite's legacy cases hold all three equal. | ||
| - **The in-process fire is fenced too** (review round 1). After the store write applies, the route | ||
| fires a controller in this process only if it belongs to the incarnation it fenced on. Otherwise a | ||
| later request under the id, started here in between, could still be stopped, and "never" would | ||
| be false. Keeping it as a documented exception was the alternative; fencing costs one field on | ||
| each controller, so it isn't worth asking about. | ||
| - **A miss is reported as absent**, `{ applied: false, status: undefined }`, as today. The route's | ||
| 404 is unchanged. | ||
| - **No column, no index, no migration on Postgres.** The predicate reads the locked row's body by | ||
| primary key, as it already does for `createdAt`. | ||
| - **The fence stays optional.** Callers that pass no fence (the CLI's cancel, tests) are unchanged. | ||
| - **No backfill of legacy records.** Deriving on read is what FIX-1286 settled; writing on a cancel | ||
| would be a second writer of an identity field. | ||
| - **Minor changeset for `engine`**, naming the signature change for store authors; patch for the | ||
| SQLite and Postgres stores. | ||
|
|
||
| ## Considered and dropped | ||
|
jhoffner marked this conversation as resolved.
Resolved
|
||
|
|
||
| | Alternative | Why not | | ||
| |---|---| | ||
| | Fence on both | Still compares `createdAt`, so the owner's hand-off still misses. It only fixes the stranger half | | ||
| | Accept a number or a string for a window | Two fences under one name, and a store that honours only the old form looks correct | | ||
| | Keep `createdAt`, document the window | The window is cross-tenant: a cancel that stops someone else's run | | ||
| | Stop the hand-off rewriting `createdAt` | Fixes the owner half only, and `createdAt` still can't split a millisecond. It also changes a field request listings sort on, outside this issue | | ||
| | Put the incarnation in its own Postgres column | Buys an index this primary-key lookup doesn't use, at the cost of a migration | | ||
|
|
||
| ## Settled | ||
|
|
||
| - **`createdAt` lets a same-millisecond reuse take the cancel** — **CONFIRMED** on `main` | ||
| `70f777def`: 202, and the other tenant's request carried the intent. | ||
| ([POC](poc/abort-fence-createdat/README.md)) | ||
| - **`createdAt` makes a same-owner hand-off miss** — **CONFIRMED** on the same commit, through | ||
| the real `claimRequestRecord`: 404 while the request stayed `in_progress`, incarnation kept. | ||
| - **The Postgres fence reads the record body, not the `created_at` column** — code read: its | ||
| predicate is `(data->>'createdAt')::bigint`. So moving to the incarnation needs no schema change. | ||
|
|
||
| ## How it got here | ||
|
|
||
| - **Draft** — framed as FIX-1286's identity reaching the last `createdAt` check; a POC showed the | ||
| fence failing both ways on `main`; the fence argument is replaced across all four stores. One PR. | ||
| - **Review round 1** — the route's in-process fire joined the fence, because a Codex review showed | ||
| a fenced store write could still be followed by an unfenced fire on a later request under the id. | ||
| A `runAction` integration scenario joined the checks, and SQLite keeps the legacy rule local to | ||
| respect its type-only import boundary. | ||
|
|
||
| **Open: none.** | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,61 @@ | ||
| # FIX-1654 · Documentation draft | ||
|
|
||
| [Spec](SPEC.md) · [Decisions](DECISIONS.md) · [Rules](BUSINESS-RULES.md) · [Plan](PLAN.md) · **Docs** · [Evolution](EVOLUTION.md) | ||
|
|
||
| Two updates, no new page. The change is to a store contract, so its reader is someone writing a | ||
| request store, and that reader lives in the engine README. No page under `apps/docs` changes: the | ||
| cancel endpoint's answers keep their documented meaning (`apps/docs/docs/server/connection-resilience.md` | ||
| → "What the endpoint returns"), and the owner's hand-off case now gets the answer that page | ||
| already promises. | ||
|
|
||
| ## UPDATE · `packages/engine/README.md` · "Abort intent on `RequestStore` (adapter authors)" | ||
|
|
||
| Replace the signature block's `setFieldsIfStatus` with: | ||
|
|
||
| ```ts | ||
| setFieldsIfStatus( | ||
| id: string, | ||
| fields: ConditionalRequestFields, | ||
| allowedStatuses: readonly RequestStatus[], | ||
| updatedAt: number, | ||
| expectedIncarnation?: string | ||
| ): Promise<ConditionalWriteResult>; | ||
| ``` | ||
|
|
||
| After the paragraph that lists the three outcomes, add: | ||
|
|
||
| > **`expectedIncarnation` fences the write to one request.** A request id can be reused once | ||
| > retention deletes its record, so the record at `id` when you write may not be the one the caller | ||
| > checked. When `expectedIncarnation` is given, compare it with the stored record's incarnation, | ||
| > resolved with `resolveRequestIncarnation(record)` (exported from `@flow-state-dev/engine`), and | ||
| > when they differ report the record as absent, `{ applied: false, status: undefined }`, and write | ||
| > nothing. Do the comparison inside the same atomic step as the status check. | ||
| > | ||
| > Resolve the stored side with the helper rather than reading `record.incarnation` directly. | ||
| > Records written before incarnations existed have none, and the helper gives them a stable value | ||
| > derived from `createdAt`. If your store has to state the rule in its own query language, the | ||
| > derived form is `legacy_` followed by `createdAt` in decimal, used when `incarnation` is absent | ||
| > or `null`. | ||
| > | ||
| > Don't compare `createdAt` instead. Two requests under one id can be created in the same | ||
| > millisecond, and a retry by the same owner rewrites `createdAt` on a request that is still | ||
| > running. | ||
|
|
||
| The existing closing line, that the conformance suite covers all of this, stays and is now true of | ||
| the fence as well. | ||
|
|
||
| ## UPDATE · `docs/architecture/state-and-scopes.md` · "A request id names one request only while its record exists" | ||
|
|
||
| Replace the last sentence of the section ("Known exceptions to settle later: …") with: | ||
|
|
||
| > The cancel route's conditional write is fenced the same way: `setFieldsIfStatus` takes the | ||
| > incarnation the route's owner check read, and every store compares it with the stored record's, | ||
| > so a cancel never lands on a later request under the id and never misses its own request after a | ||
| > same-owner hand-off. The route's in-process fire checks the same incarnation: each registered | ||
| > abort controller carries the incarnation of the request it belongs to, and a fenced fire acts only | ||
| > on a match. | ||
|
|
||
| ## Publication ownership | ||
|
|
||
| FIX-1654 publishes both in its implementation PR, after the conformance cases pass on all four | ||
| stores. The `engine` changeset carries the upgrade note for store authors. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,10 @@ | ||
| # FIX-1654 · Evolution | ||
|
|
||
| [Spec](SPEC.md) · [Decisions](DECISIONS.md) · [Rules](BUSINESS-RULES.md) · [Plan](PLAN.md) · [Docs](DOCS.md) · **Evolution** | ||
|
|
||
| | Prior intent and precise source | Treatment | Why / evidence | Replacement | Compatibility | | ||
| |---|---|---|---|---| | ||
| | FIX-1021's abort fence (commit `e0a7f566b`, direct route, no retained spec): `setFieldsIfStatus(…, expectedCreatedAt)`; "a record at `id` with a different `createdAt` is a different request" | **Retained**: the fence, its place inside each store's atomic step, and a miss reported as absent. **Superseded**: `createdAt` as what it compares | [POC](poc/abort-fence-createdat/README.md): fails both ways on `main` | [D1](DECISIONS.md#d1), [PLAN S1 to S7](PLAN.md#surfaces) | Legacy records fence on `legacy_<createdAt>`, today's comparison ([BR-6](BUSINESS-RULES.md#records-from-before-incarnations)). Out-of-tree stores change signature ([BR-11](BUSINESS-RULES.md#callers-and-stores)) | | ||
| | The in-process abort registry (`packages/engine/src/execution/abort-registry.ts`; FIX-1026 made it the convergence point for cross-process delivery): one controller per request id | **Retained**: per-process, keyed by id, the single teardown point. **Amended**: each controller carries its request's incarnation, and the cancel route's fire checks it | Codex review round 1: a fenced store write followed by an unfenced fire by id can stop a later request | [PLAN S3b](PLAN.md#surfaces), [BR-12](BUSINESS-RULES.md#the-record-changed-between-the-check-and-the-write) | Callers that fire with no expected incarnation behave as today ([BR-13](BUSINESS-RULES.md#the-record-changed-between-the-check-and-the-write)) | | ||
| | FIX-1286 [D1](../FIX-1286/DECISIONS.md#d1) and its *Decided, not asked* on legacy records: the incarnation is the request's identity; a legacy record derives `legacy_<createdAt>` | **Retained** unchanged; this issue consumes it | — | — | — | | ||
| | `docs/architecture/state-and-scopes.md` → "A request id names one request only while its record exists": lists the abort fence as a known exception | **Amended**: the exception is removed | This issue closes it | [DOCS.md](DOCS.md) | — | |
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.