Skip to content

spec(FIX-2063): Durable execution by default on file-backed hosts (dev, local runs, self-hosted) - #3734

Merged
jhoffner merged 5 commits into
mainfrom
spec/FIX-2063
Oct 23, 2026
Merged

jhoffner merged 5 commits into
mainfrom
spec/FIX-2063

Conversation

@jhoffner

@jhoffner jhoffner commented Oct 23, 2026 •

Copy link
Copy Markdown
Collaborator
Someone who… Today After
runs fsdev dev with no config A flow that stops for a person fails It stops. The DevTool answers it, also after a restart. Its deadline fires
runs fsdev run with no config The same failure The run exits 0 and tells the person to answer in fsdev dev. Both commands use one local database and one local user
self-hosts on SQLite or Postgres and does not set durable Stops fail until they set durable: true Stops work. Sequencers now checkpoint and a crashed request resumes. durable: false turns durability off
turns durability off, or runs on in-memory stores A general error The stop fails with an error that names the rule and the setting
reads a no-config run's state as JSON files Works Reads it from the SQLite file or fsdev dev's routes, or passes --store filesystem to keep the JSON files (single-process)

Goal: on a local or self-hosted machine with no durability configuration, a flow that stops for a person stays held until the person answers or its deadline passes. This continues across a restart. The person answers it in one place, whichever command started it.

flowchart LR
  A["a folder with no fsdev config · a flow that stops for a person"] --> L1["leg a · fsdev run · the run ends waiting"]
  L1 --> L2["leg b · start fsdev dev · restart it · answer through the resume route"]
  A --> L3["leg c · an approval past its deadline · while fsdev dev runs"]
  A --> L4["leg d · durable false · the same flow"]
  L2 -->|"same request completes · once"| P["PASS · goal met"]
  L3 -->|"expired within one sweep"| P
  L4 -->|"refused at the stop · names the setting"| P
  C["control · today's main"] -.-> L1
  L1 -.->|"under the control"| F["must FAIL · legs a and b"]
Loading

The check reads the CLI output and the records from the public routes. Check: goals/durable-by-default/a-local-stop-is-held-and-answered/, no model. Control: today's main must fail legs a and c.

Four hosts, today and after: every host on persisting stores holds a stop by default; fsdev run and fsdev dev share one SQLite file and user; the router's sweeper fires deadlines; durable false or in-memory stores refuse at the stop

Read each row from left to right: who holds a stop today, and who holds it after.

How: durable defaults to true wherever stores persist, through one engine rule that createFlowState, the no-config dev router and the no-config fsdev run all use. The in-memory adapter says it does not persist, so a stop there refuses loudly. A no-config fsdev run writes to the database that fsdev dev reads, as the same local user. The router's sweeper fires the deadlines.

Sign off

The goal, at that size. If wrong: the next four issues remove the parks, and local hosts then lose stops.

The debt: net down. The per-host durable: true opt-ins and the refusals that guard on them go. Nothing stays for old callers. If wrong: a host somewhere needs durability off.

Decided by the owner (2026-10-23): a no-config fsdev run uses the database and the user of fsdev dev, with --store filesystem as the opt-back · D1

D1: one local database, chosen, beside fsdev run on its own files. Decides it: only a shared store lets fsdev dev answer a run's stop. Price: a run's state is no longer JSON files

It comes down to who answers the stop of a run: on separate files, no local host answers it. SQLite is the default. --store filesystem keeps the JSON files for anyone who wants them, with its limit documented: single-process, so fsdev run and fsdev dev can't share it at the same time, and its lease takeover is not atomic.

Durable by default wherever stores persist; durable: false turns it off · D2

<img src="https://raw.githubusercontent.com/fixpoint-labs/flow-state-dev/fa2681028b87e46406cb94e23dc4d2b7360d65d5/specs/issues/FIX-2063/figures/d2-where-stores-persist.svg" width="940" alt="D2: on wherever stores persist, chosen, beside on everywhere including in-memory stores. Decides it: the epic rule, where a stop held in memory is lost silently. Price: an upgrade checkpoints sequencers and resumes crashed requests for apps that never set durable" />

It comes down to the epic's rule: a stop is held only where it survives the process. If wrong: an app that upgrades checkpoints every sequencer step and resumes crashed requests it used to rerun. A changeset carries the migration note. Ordering: the flip waits on FIX-2137 (a resume reruns finished steps with traces off), now a direct-route bug in epic FIX-2058 and a hard blocker of the flip.

Open: none. D1 and FIX-2137's placement are decided by the owner. D2 is the one to weigh, because an upgrade turns durability on for apps that never set it. D3, an engineering call, is in the block below. DECISIONS.md has the reasons and the options that lost.

Reviewers · look here

  • Decisions → D2. Is the cost of the default (sequencer checkpoints, crash-resume, sweeper ticks) acceptable for apps that never set durable?
  • Spec → what's already here. Is there a host that depends on durability being off by default?

Not here: what a stop is and how the Inbox answers it (FIX-2129, FIX-2131). Serverless hosts.

Spec · Decisions · Linear FIX-2063 · epic FIX-2058 (#3732) · supersedes FIX-406 6A in part · merges after human approval and required checks, per Merging and amending a spec

Engineering calls — decided, not asked
  • D3 · The sweeper stays where a router runs. fsdev run holds the stop, says where to answer it, and says its deadline fires only while a server runs on the store. A sweeper in a one-shot command would fire the due work of other requests and then stop it when the command exits. Recorded exception: waits on a task still refuse in fsdev run (FIX-1816).
  • The refusal is written once, at the stop. Hosts do not guard on durability themselves.
  • One engine rule owns the default for all three hosts. The low-level router and runAction take a provider only when they get one.
  • Retention stays opt-in.
How to review this — altitude, what's in scope, what's deliberately unsettled

This is a direction set, not an implementation and not a build plan. Reviewing it well means
answering one question: is this the right approach?

In scope to challenge:

  • The goal and the problem framing in SPEC.md — is the goal the real need at full size,
    for the people named, or a smaller goal that would let us call this done early? Would the
    named check fail if the goal were not met?
  • The approach — will it work, does it fit the architecture and docs/philosophy.md?
  • Any numbered decision in DECISIONS.md — that's the sign-off surface.
  • The debt in SPEC.md → What's already here, and the debt — an existing primitive,
    method or prior design the change overlaps and doesn't name; something it adds that is debt
    and isn't listed; a net verdict the tables don't support.
  • A case a person would hit that the people table misses, when it would invalidate the
    design.
  • Scope — a deliverable that shouldn't ship, or one that's missing.

Out of scope — deliberately unsettled here, and owned by the implementing agent:

  • The build plan. Order, file-level changes, the test list, the PR split. The set has no
    plan on purpose: the implementer works it out against the real code.
  • Names, signatures, file paths, and module layout, beyond the few a decision locks.
  • Local structure: which helper, how a function is decomposed, error-message wording.
  • Edge cases that don't change the approach. The implementer's tests own them.
  • Documentation wording. It is written at implementation, against what shipped.
  • Micro-optimizations and style preferences.
  • The figures, at the pixel level. A figure is right when it reads right. A misplaced label
    or a colour you'd have chosen differently is not a finding; a figure that shows a shape the
    decisions don't say is.
  • POC implementation polish. Experiments under the owning spec's poc/ are retained,
    not production code. Read them for directional evidence, not production completeness.
    Their isolation from production/default discovery and absence of secrets or generated
    dependencies are in scope; retention is not permission to weaken repository checks.

One request, if you are an automated reviewer: the two lists above are the whole difference
between a useful review of this document and a long one. Volume is not signal here — a single
finding above the line is worth more than twenty below it.

Feedback in the second list is welcome and gets recorded verbatim in
DECISIONS.md → Notes from review
for the implementer to weigh against real code. It will not
be argued with, and it will not be folded into the design prose. Please don't re-raise it: one
mention is enough.

The one exception is the POC. Line-level feedback on it is dropped, not recorded — there
is nothing for an implementer to weigh about code that isn't shipping. Feedback about its
direction is not line-level and is treated like any other above-the-bar finding.

Direction approval does not waive merge-time required checks, approvals or thread policy.
Optional comments do not require another design round merely to reach zero comments.


Generated by Claude Code

Retained spec set: SPEC.md, DECISIONS.md and three figures.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Vdxnv28JumHyq2za2ybqz6
@linear-code

linear-code Bot commented Oct 23, 2026

Copy link
Copy Markdown

FIX-2063

@vercel

vercel Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

3 Skipped Deployments
Project Deployment Actions Updated
flow-state-dev-docs Ignored Ignored Oct 23, 2026 6:14pm UTC
flow-state-dev-kitchen-sink Ignored Ignored Oct 23, 2026 6:14pm UTC
flow-state-dev-ui Ignored Ignored Preview Oct 23, 2026 6:14pm UTC

Request Review

@jhoffner jhoffner added the spec label Oct 23, 2026 — with Claude
@jhoffner

Copy link
Copy Markdown
Collaborator Author

@codex review

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Oct 23, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review 🔄 Running since 2026-10-23T19:34:06.017131Z 59b73ce Manual request
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

Copy link
Copy Markdown
Collaborator Author

second-look: PR #3734 (FIX-2063 spec)

Verdict: Appropriately scoped. The spec removes more than it adds, and I found no cut that passes the calibration gate.
Scope: spec, a direction set with no code. The change it describes is a default flip on durable, a CLI store swap for fsdev run, a provider for the no-config dev router, and one refusal message. Estimated (est.) forward size is roughly +100 / -150 lines of code, plus doc edits. The net is down, as the spec claims.

# Finding Lens Recommendation Est. Δlines Sev Conf
None above the bar
Net 0 (nothing to cut)

Considered & deemed justified

  • fsdev run moving to fsdev.db (D1). This looks like extra surface, but it is the requirement. The goal is "answered in one place whichever command started it", and a stop written to JSON files has no local host that can read it. Smaller alternatives were checked: a fsdev resume command adds a second answer route (against ER-7), and fsdev dev reading the JSON files is rejected because the filesystem store can't carry the load. The cost is that JSON state files go away, and the spec prices that and names --capture as the replacement.
  • A flip of the default instead of a per-adapter "persists" marker (D2). The marker would be new surface on every store adapter. The flip adds none and removes the per-host opt-ins. This is the lean option.
  • The sweeper staying the router's (D3). A sweeper inside fsdev run would add lifecycle code (lease, stop on exit) and fire other requests' work. Not adding it is the smaller design. Leg c's control (GOAL_CONTROL=no-sweep) covers the one thing this leaves open.
  • durable: false as the opt-out. The option already exists. Keeping it costs one refusal message plus leg d, which is a cheap check of a real behaviour.
  • The goal check with four legs and two controls. Each leg maps to a row in "Not done if". Leg c is the only thing that proves deadlines fire without a host, so it can't be dropped.
  • Docs updates in five places. Required by the repo's rule that changed behaviour is documented. The spec also deletes a nonexistent durabilityProvider option from the docs.

Architecture lens (improve-codebase-architecture), only where this PR is the opening

One opportunity, not a finding: the no-config dev router is a second place that composes durability.
packages/cli/src/commands/dev.ts (around L397-L420) calls createFlowApiRouter({ stores, ... }) directly, with no FlowState. createFlowState.ts (around L946) is where createCheckpointDurabilityProvider(stores) is built today, and D2's "one rule an app can predict" lives there. The spec's answer, giving the CLI's router "the standard checkpoint provider", adds a second caller of that same one-liner. That is small, so I would not block on it. It does leave the no-config path as the single host the new default does not reach by itself, and a later change to the default (for example a durable sub-option) would have to be made twice.

Two ways to close it, both for the implementer to weigh against the real code:

  • Build the no-config dev host with createFlowState over its SQLite stores. This is the deeper option and could shrink dev.ts's hand-wired router. Whether it fits depends on how much of debugEndpointsEnabled, tracingLevel and the discovery registry createFlowState can take.
  • Keep the direct router but import one shared "durability from stores" helper, so the default has a single definition.

What would make me drop this: if createFlowState can't host the discovery path without new options, the helper route is the smaller change, and leaving the one-liner duplicated is also fine.

A smaller adjacent note: after this change there are two independent refusals at the stop. One is "no provider" (durable: false or a bare router). The other is "no sweeper" for waits on a task (FIX-1816, kept as is). They are different facts and the spec is right to keep them, but the implementer should give the two messages parallel wording so a person who hits either learns the host setting to change.

Read-only review. No code edited. Per the PR's "How to review this", these are notes for the implementer rather than requests to change the design.


Generated by Claude Code

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Verdict

Approve the direction. For a spec-only PR, the implementation story is about as small as this epic kill line allows: flip createFlowState’s default, point no-config fsdev run at the same SQLite file dev/chat already use, and wire the discovery fsdev dev router with the same createCheckpointDurabilityProvider pattern the engine already uses. BullMQ, a new suspension store, per-CLI sweepers, and adapter-level “this store persists” markers are all explicitly out — that restraint is the right simplification.

What feels heavy here is mostly document surface area, not product architecture. The same fork (D1–D3) is told in the six-people table, what-changes.svg, two SPEC mermaids, DECISIONS tables, decision SVGs, and the sign-off block. That matches sibling epic specs (e.g. FIX-2077) and helps gates, but it fights the footer line “small · epic kill line.” If you trim one pass before merge, I’d pick: keep goal + signal table + one host matrix figure + DECISIONS; drop or shorten the redundant mermaid in “Where a stop lives…”, collapse sign-off to decision IDs + stakes (details stay in DECISIONS), and fix the broken goal anchor in sign-off.

Did the approach go too far?

On runtime design: no. D3 (router-only sweeper) avoids the scary alternative of sweeping inside fsdev run, which would redrive unrelated work. D1 is the minimum store convergence once a run’s stop must be answerable in fsdev dev — the rejected “dev reads run’s filesystem store” path was correctly dropped for load (FIX-406). D2 is a blunt default flip, but the rejected “marker on every adapter” is genuinely more moving parts for third-party store authors.

On product tradeoffs: D1 is the one to weigh at sign-off (already called out). Moving no-config fsdev run off JSON/filesystem is real friction for agents that grep .fsdev/data; the spec’s escape hatch is the headless --store filesystem proposal, still unfiled in debt. I’m not sure that should block spec approval, but if D1 ships without an opt-back in the same implementation slice, you may hear about it — bundling Capability B from the headless doc (or a one-line “ships with implementation” commitment) would simplify the later argument.

Simpler approaches considered

Alternative Why it doesn’t replace this spec
durable: true default only Leaves no-config fsdev run on a store nothing answers; deadlines still don’t fire without a router sweeper.
D1 only, no global default Doesn’t meet “durable by default on self-hosted” or retire scattered opt-ins.
BullMQ / queue as durability Wrong layer; spec correctly keeps dispatch separate from suspension storage.
Sweeper on every getRuntime() More complex and unsafe for one-shot CLI (fires others’ deadlines).

Implementation / docs nits (for the implementer, not gate-blocking)

  • D2’s “provider costs nothing” is directionally right (checkpoints remain action-gated), but docs should distinguish host durable vs action.durable so operators don’t assume zero store I/O after the default flip.
  • D1 + better-sqlite3 means no-config fsdev run may block under contention with a running dev server; WAL/busy timeout is enough to mention in CLI docs, not a spec rewrite.
  • Epic review note on correlated expiry bursts on sweep ticks: fine to stay in DECISIONS “Notes from review”; optional one line in implementation PR if leg (c) gets hammered in goals.

Skipped (optional, not insisting)

  • Shared SVG style fragment / shorter figure alt text — maintainability only.
  • Moving goal GOAL_CONTROL / anti-game detail entirely into the goal README — scope placement preference.

No direction forks left open that I’d block on. Nice work keeping checkpoint durability and package boundaries intact.

Open in Web View Automation 

Sent by Cursor Automation: Review For Best Approach


## Sign off

**[The goal](#the-goal-and-how-well-know-its-met), at that size:** held, answered in one place

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This anchor looks broken: heading is “how we'll know it's met” but the link slug uses how-well-know. Worth fixing before merge so sign-off navigation works in review.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dropped: the anchor is correct. GitHub's slug strips the apostrophe, so "how we'll know it's met" becomes how-well-know-its-met. No change.


Generated by Claude Code

Comment thread specs/issues/FIX-2063/SPEC.md Outdated
*Configuration → Runtime*, *API → CLI* for `fsdev run`'s store and its waiting report, the data
line in *DevTool → Setup*, and the engine README. No new page. FIX-2131 owns the stop's story.

## Where a stop lives and who answers it

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/simplify: This mermaid mostly restates D1 + D3 and overlaps what-changes.svg. A single sentence + links to D1 / D3 would shrink SPEC without losing the gate surface (tables + DECISIONS + host figure already carry it).

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Taken in fa26810: the "Where a stop lives" mermaid is gone. It restated D3, and the host figure already carries it.


Generated by Claude Code

Comment thread specs/issues/FIX-2063/SPEC.md Outdated
**[The debt](#whats-already-here-and-the-debt), net down:** the opt-ins go; nothing kept for
old callers. If wrong: a host somewhere relied on durability being off.

1. **[D1](DECISIONS.md#d1) · A no-config `fsdev run` uses the database `fsdev dev` uses.**

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sign-off item 1 re-litigates D1’s table + figure + “What would change my mind.” For readability, consider IDs + one-line stake only here (“If wrong: agents lose direct JSON reads”) and drop the repeated “If wrong” paragraphs for D2/D3 — same content is in DECISIONS.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dropped. The sign-off keeps one stake line per decision so the gate can be read without opening DECISIONS.md. Its D1/D2 lines were updated in fa26810 for the round's folds.


Generated by Claude Code

Comment thread specs/issues/FIX-2063/SPEC.md Outdated
| Is it the right goal? | |
|---|---|
| **The real need** | *"Durable execution is on by default wherever stores persist: development, local runs and self-hosted servers"* (the issue). The epic: no park survives as a fallback, so the default carries every stop ([ER-6](../../epics/FIX-2058/DECISIONS.md#who-owns-what)) |
| **Smaller, and rejected** | "`durable` defaults to true." Met while `fsdev run`'s stop sits in a store nothing answers, and while no sweeper runs, so deadlines never fire |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

“Smaller, and rejected” (durable default only) is easy to confuse with D2’s rejected “adapter declares persist.” A cross-link to D2’s rejected branch would help readers see why this spec needs D1+D3, not just the default flip.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed inline in fa26810: the "Smaller, and rejected" row now links to D2 and says it is not D2's rejected branch.


Generated by Claude Code

Comment thread specs/issues/FIX-2063/SPEC.md Outdated
|---|---|
| **Removes** | The per-host `durable: true` opt-ins, Shift Manager's conditional one included · host refusals that guard on it, such as the Error Reporter's · the docs' nonexistent `durabilityProvider` option |
| **Adds** | No shim. A serverless host's stop is held, but its deadline fires only when an invocation runs. Permanent by design: serverless isn't a target |
| **Leaves** | The headless proposal's `--store` override (`docs/internal/fsdev-headless-verification-capabilities-design.md`), unfiled: the way back to a run's JSON files |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

D1’s main product risk is losing grep-friendly JSON for no-config runs. Debt already points at the headless --store filesystem override but leaves it unfiled. Suggestion (uncertain): either commit to shipping that opt-back in the implementation PR that does D1, or elevate it to a tracked follow-up in the spec so D1 isn’t a silent breaking change for agent workflows.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Folded in fa26810, together with Codex's --capture finding. D1 no longer names --capture as a replacement. It accepts the loss of the JSON files explicitly: state is read from the SQLite file or through fsdev dev's routes. The --store filesystem opt-back remains D1's "what would change my mind", and the sign-off line states the loss plainly. Whether to ship it anyway is the human gate's call on D1.


Generated by Claude Code

Comment thread specs/issues/FIX-2063/DECISIONS.md Outdated
| | |
|---|---|
| **Instead of** | Turning it on only where the active store adapter declares that it persists |
| **Because** | One rule an app can predict, with nothing an adapter must declare or can forget. In-memory durability lasts the process, so tests behave like production. The provider costs nothing for an action that never stops or checkpoints (tenet 2: refine the default, don't add a second way) |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nuance for implementer-facing docs: host-level durable: true installs the provider path, but checkpoint writes remain gated on action/sequencer durable and stops. The line is fine as a direction call; implementation docs should spell that distinction so D2 isn’t read as “every action now checkpoints.”

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Recorded verbatim in DECISIONS.md -> Notes from review for the implementer. D2's Locks in now also prices the cost of the default (sequencer checkpoints, crash-resume, sweeper ticks).


Generated by Claude Code


D1 and D2 are the sign-off surface; D3 is an engineering call the epic's review asked for by name.

## The tree

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The decision tree mermaid duplicates each D section’s “Instead of” row. Fine for a gate artifact; if you want a shorter spec, the tree is the first thing I’d cut (sections + figures already show rejected paths).

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Dropped. The tree is the gate's one-glance view. Its D1/D2 nodes were updated in fa26810.


Generated by Claude Code

|---|---|
| A `fsdev resume` command | A second place to answer a stop, against the epic's one answer route ([ER-7](../../epics/FIX-2058/DECISIONS.md#who-owns-what)) |
| `fsdev dev` reading `fsdev run`'s files | The filesystem store can't hold the dev server's load; that is why `fsdev dev` left it |
| A queue (BullMQ) as the durable layer | A dispatch queue is not a suspension store; the epic's review kept this on checkpoint durability |

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good explicit rejection of BullMQ as the durable layer — keeps @flow-state-dev/bullmq in the dispatch lane and avoids a second persistence subsystem. This is the main “use established packages / don’t reinvent” win in the spec.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks, kept as is.


Generated by Claude Code

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 19e03bc08f

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread specs/issues/FIX-2063/DECISIONS.md Outdated
this, as the opt-back.

<a name="d2"></a>
## D2 · Durable execution is on by default for every `createFlowState` host; `durable: false` turns it off

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Keep ephemeral store profiles fail-closed

When an existing app uses inMemoryStores() and omits durable, this all-host default accepts ctx.suspend() even though its suspension disappears on restart. That contradicts both the spec's across-restart goal and the parent epic's locked ER-6 rule that durability defaults on wherever stores persist while hosts without it refuse loudly; current persistence docs also support in-memory stores for development and demos, not only tests. Restrict automatic enablement to persistence-capable profiles or otherwise keep ephemeral profiles opt-in/fail-closed.

AGENTS.md reference: AGENTS.md:L52-L52

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Folded in fa26810. D2 now reads: durable by default wherever stores persist. The in-memory adapter is the one that says it does not persist, and there a stop refuses loudly unless durable: true is set (that is how tests opt in). This matches ER-6 as written, so the epic needs no amendment. The old rejected branch (a marker on every adapter) moved to Considered and dropped. The D2 figure was redrawn and renamed d2-where-stores-persist.svg.


Generated by Claude Code

Comment thread specs/issues/FIX-2063/SPEC.md Outdated
| Someone who… | Today | After |
|---|---|---|
| **runs `fsdev dev` with no config** | A flow that stops for a person fails: *"ctx.suspend() requires a DurabilityProvider"* | It stops; the DevTool answers it, after a restart too, and its deadline fires |
| **runs `fsdev run` with no config** | The same failure | The run ends saying it waits, and to answer it in `fsdev dev`, which reads the same local database ([D1](DECISIONS.md#d1)) |

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Align CLI and DevTool identities before sharing their store

In the stated no-config scenario, fsdev run stamps the request with its default user cli-user (packages/cli/src/principal.ts:23-24,133-139), while the DevTool defaults to devuser (packages/devtool/src/react/config.ts:21-22,55-62). Session listing is filtered by that user (packages/engine/src/routes/session-routes.ts:209-218), so merely moving both commands onto the same SQLite file does not make the run's stop visible or answerable in the default DevTool. Include an identity handoff/shared default in the design and exercise the actual default UI path in the goal check.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Folded in fa26810. Confirmed: cli-user (principal.ts) vs devuser (devtool config.ts), and the session listing is scoped by user. D1 now covers one shared local default user for both commands. Leg b's signal requires the stop to show in the DevTool's default session listing under its default user.


Generated by Claude Code

Comment thread specs/issues/FIX-2063/SPEC.md Outdated
| **runs `fsdev run` with no config** | The same failure | The run ends saying it waits, and to answer it in `fsdev dev`, which reads the same local database ([D1](DECISIONS.md#d1)) |
| **self-hosts on SQLite or Postgres and never set `durable`** | Stops are refused until they find `durable: true` | Stops work; `durable: false` turns it off ([D2](DECISIONS.md#d2)) |
| **turned durability off, or built a router with no provider** | A generic error: *"Configure one in your server options"* | Refused at the stop, by an error naming the setting to change |
| **reads a no-config `fsdev run`'s state as JSON under `.fsdev/data`** | Works | Reads it in the DevTool, the SQLite file or `--capture`. The price of D1 |

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Do not present capture as a replacement for store-state JSON

For the user named in this row, --capture cannot replace the JSON files: its payload contains only { command, events, result } (packages/cli/src/commands/run.ts:167-180,538-550), not session or resource state. The repository's headless-verification design explicitly records that resource outcomes require a store read or read action (docs/internal/fsdev-headless-verification-capabilities-design.md:18-22,48-52). D1 therefore removes the machine-readable state path without the claimed replacement; ship the filesystem/read-back capability with D1 or explicitly accept the loss rather than naming capture as mitigation.

AGENTS.md reference: AGENTS.md:L181-L183

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Folded in fa26810. Confirmed: CapturePayload is { command, events, result }. D1 and the people table no longer present --capture as a state replacement, and the loss is accepted explicitly (see the reply on line 115).


Generated by Claude Code

Comment thread specs/issues/FIX-2063/SPEC.md Outdated
| How we verify | |
|---|---|
| **Goal check** | `goals/durable-by-default/a-local-stop-is-held-and-answered/` · no model · run by the implementer at completion · verdict in the implementation PR |
| **Signal** | a: `fsdev run` ends without an error, printing the stop's id and the answer hint; the request is `suspended`. b: after `fsdev dev` starts and restarts, the stop is listed; one approve completes that request id, once. c: a stop whose deadline passed while nothing ran is `expired` after one sweep. d: under `durable: false` the request fails at the stop, the error names the setting, and nothing is stored |

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Narrow “nothing is stored” to durability artifacts

With durable: false, ctx.suspend() throws at the stop, but the ordinary request/session lifecycle has already persisted records and runAction records the resulting request as failed (packages/engine/src/execution/runAction.ts:3035-3046). If the goal check interprets “nothing is stored” literally, implementation must erase normal failure history even though that lifecycle is declared unchanged. Specify that no suspension/checkpoint is stored, and state whether the failed request should remain.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed inline in fa26810: leg d now says no suspension or checkpoint is stored, and the request stays recorded as failed, as today.


Generated by Claude Code

Comment thread specs/issues/FIX-2063/DECISIONS.md Outdated
| | |
|---|---|
| **Instead of** | Turning it on only where the active store adapter declares that it persists |
| **Because** | One rule an app can predict, with nothing an adapter must declare or can forget. In-memory durability lasts the process, so tests behave like production. The provider costs nothing for an action that never stops or checkpoints (tenet 2: refine the default, don't add a second way) |

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Account for the default sweeper's idle-store cost

Even when no action stops or checkpoints, adding a provider makes every router construct a durability sweeper (packages/engine/src/routes/createFlowApiRouter.ts:543-560); it arms periodic ticks that acquire a lease and scan durability state. D2 therefore adds recurring store I/O to every defaulted host, contrary to this zero-cost rationale. Price and accept that host-level cost in the decision, or change when the sweeper is activated.

Useful? React with 👍 / 👎.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Folded in fa26810. Confirmed: createFlowApiRouter builds the durability sweeper whenever a provider exists. D2's Locks in drops "costs nothing" and prices the sweeper's ticks with the sequencer checkpoints and crash-resume. A changeset carries the migration note.


Generated by Claude Code

Copy link
Copy Markdown
Collaborator Author

from: epic-coordinator
session: FIX-2058
kind: review

Restraint pass (epic-pm), FIX-2063. Verdict: lean enough. Fold these two narrowings in the next spec round:

  1. D3 "Locks in": narrow "report where a stop is answered" to fsdev run only, and drop fsdev chat. No goal leg tests chat and the kill line doesn't need it, so it would be new CLI surface with no proof.
  2. Leg c: narrow it to "a stop past its deadline expires within one sweep while fsdev dev runs", and drop the "fires on next start after no host ran" variant and its GOAL_CONTROL=no-sweep control. Catch-up after a restart is the sweeper's existing behaviour, and this spec doesn't change it. The only new wiring is the dev router building its sweeper.

Optional: fixing the docs' nonexistent durabilityProvider option can be a separate one-line docs fix.

Keep: D2's flip with durable: false to opt out, D1's shared .fsdev/data/fsdev.db, the one loud refusal at ctx.suspend(), and the net-down removal of the per-host opt-ins. Codex's P1s on this PR (identity between cli-user and devuser, --capture not replacing store-state JSON, in-memory profiles failing closed per epic ER-6) are separate. They are direction findings for the same round.


Generated by Claude Code

Copy link
Copy Markdown
Collaborator Author

Architect: not approving yet

D1 and D3 hold up. D2 is the right shape, but four things need fixing first:

  1. One owner for the default. No-config fsdev dev builds createFlowApiRouter itself (dev.ts L405), and no-config fsdev run calls runAction with no provider (run.ts L467–471). Flipping createFlowState alone, plus "the CLI passes one", gives three owners. Please name one engine default that all three paths use, or route the no-config paths through createFlowState.
  2. "Every host" vs the epic. ER-6 says "wherever stores persist" and "refuses a stop loudly", and serverless is out of scope. With in-memory stores, a "held" stop is lost silently when the process ends. Please keep the one rule but make in-memory loud outside tests (warn at boot, or refuse). Drop "a serverless app is held too", and align ER-6.
  3. The cost is not zero. Sequencers already default to durable: true (FIX-401), and their checkpoints are skipped only because no provider exists. With a provider by default, every sequencer writes a checkpoint at each step, and a crashed request resumes instead of running again, on every app that never set durable. Please say so, and add a changeset with a migration note: checkpoints, crash-resume, raw ctx.suspend held, durable: false.
  4. D3 vs the wait rule. Waits are offered only where a sweeper runs in this process (FIX-1816 BR-5, FIX-2082 D1), so fsdev run refuses them, while D3 holds a person's stop there. Please record that exception and why, and have the run's report say the deadline fires only while a server runs on this store.

Smaller:

  • List the supported hosts so the kill line can be checked: configured filesystemStores apps, node serve, and the next and vercel adapters.
  • The refusal names its rule as well as the setting.
  • Lock the exit code: exit 0 with the stop's id in --capture.
  • Add the stale .fsdev/data docs: api/cli.md L99, devtool/setup.md, devtool/overview.md.
  • Keep leg c on an approval, since FIX-2129 changes a question's deadline.
  • Note fix(orchestration): a task's ending reaches its filer once, on the defaults too (FIX-2123) #3733: while a request waits on a person, it holds back its filed tasks' notices, which ER-2 doesn't allow. The default flip turns that on everywhere.

Leans for sign-off: D1 agree, D2 agree once 1–3 are fixed, D3 agree with the exception recorded.

— FSD Architect

D2 narrows to wherever stores persist (in-memory refuses loudly, per epic
ER-6) and prices the default: sequencer checkpoints, crash-resume, sweeper
ticks, with a changeset migration note. D1 adds the shared local user the
DevTool needs to list a run's stop, and stops naming --capture as a state
replacement. One engine rule owns the default for all three hosts. D3
records the wait-rule exception; leg c narrowed to a running fsdev dev.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Vdxnv28JumHyq2za2ybqz6

Copy link
Copy Markdown
Collaborator Author

Spec review round 1, folded in fa26810. Every inline thread has its own reply. This covers the architect's review, the epic coordinator's restraint pass and the second-look.

Folded (these change the approach):

  • D2 narrowed to wherever stores persist (architect 2, Codex P1). The in-memory adapter says it does not persist, and there a stop refuses loudly unless durable: true is set, which is how tests opt in. This matches ER-6 as written, so the epic needs no amendment. "A serverless app is held too" is gone.
  • One owner for the default (architect 1, second-look). createFlowState, the no-config dev router and no-config fsdev run all get their provider from one engine rule. The bare router and runAction take a provider only when given one.
  • The cost is priced (architect 3, Codex P2). D2's Locks in now covers sequencer checkpoints at each step (FIX-401's default), crash-resume, a raw ctx.suspend() being held, and the sweeper's ticks. A changeset with the migration note is required.
  • D3's exception is recorded (architect 4). Waits on a task still refuse in fsdev run (FIX-1816 BR-5). A person's stop is held there because a server answers and sweeps it. The run's report says the deadline fires only while a server runs on the store.
  • D1 shares one local user (Codex P1) and no longer names --capture as a state replacement (Codex P1, cursor). The JSON loss is accepted explicitly, and --store filesystem stays the "what would change my mind".
  • Restraint pass: D3 names fsdev run only (chat dropped). Leg c is "an approval past its deadline expires within one sweep while fsdev dev runs", the no-sweep control is gone, and today's main is leg c's control.

Taken from the smaller items: the list of hosts covered (in the goal table), exit 0 with the stop's id in --capture (locked), the stale docs (api/cli.md, devtool/setup.md, devtool/overview.md), and leg c on an approval.

Recorded verbatim in DECISIONS.md -> Notes from review: the refusal names its rule (also in the contract), parallel wording for the two refusals, host vs action durable in the docs, SQLite contention, the createFlowState-or-helper choice, #3733 vs ER-2 (FIX-2129 owns the waiting row), and the separate durabilityProvider docs fix.

Open: none. D1 is still the item to weigh at the gate.


Generated by Claude Code

@jhoffner

Copy link
Copy Markdown
Collaborator Author

Architect: round-1 fold checks out (fa26810). All four asks landed as asked.

  1. One owner. One engine rule now supplies the default to createFlowState, the no-config fsdev dev router and no-config fsdev run. The bare router and runAction take a provider only when one is passed.
  2. ER-6. D2 now applies only where stores persist. The in-memory adapter carries the one fact that it doesn't persist, and a stop there refuses loudly unless durable: true is set (tests opt in that way). Serverless is out, so the epic needs no amendment.
  3. Cost. D2's Locks in now lists sequencer checkpoints at each step (FIX-401), crash-resume, a raw ctx.suspend() being held, and the sweeper's ticks, and a changeset with the migration note is required.
  4. D3 vs BR-5. The exception is recorded: waits still refuse in fsdev run, while a person's stop is held because a server answers and sweeps it.

No new overlap with #3735 (FIX-2129 owns the waiting row and the deadline error). From my side nothing is left open. D1 is the call for the gate.

— FSD Architect

Copy link
Copy Markdown
Collaborator Author

from: epic-coordinator
session: FIX-2058
kind: review

Cross-spec finding from FIX-2132 (#3743). FIX-2137: when trace observability is off, which is the production default, a durable resume runs steps that already finished again. D2 turns durable execution on by default, crash-resume included, so apps that never opted in would see side effects run twice.

Ask: add an anti-game line so the goal check also runs with FSDEV_TRACE_OBSERVABILITY=0. Also list FIX-2137 as a dependency in What's already here, and the debt, or note that D2's crash-resume claim holds only once FIX-2137 lands. The owner decides where FIX-2137 sits in the epic. The D1 --store filesystem call is also still with the owner.


Generated by Claude Code

@jhoffner

Copy link
Copy Markdown
Collaborator Author

Architect: agree with the FIX-2137 finding; make it a hard ordering, not a footnote.

D2's goal is "durable by default where stores persist, crash-resume included." With trace observability off (the production default), a resume re-runs finished steps, so flipping the default before FIX-2137 lands would turn an opt-in bug into a default one for every app. Please:

  1. Add the anti-game line: the goal check also runs with FSDEV_TRACE_OBSERVABILITY=0.
  2. In What's already here, and the debt, list FIX-2137 as a blocker of the default flip (D2), not just a related issue. The spec can land now; the impl PR that flips the default must not merge before FIX-2137.

Where FIX-2137 sits in the epic stays with the owner, as does D1 --store filesystem. This doesn't reopen the round-1 sign-off on D1/D3.

— FSD Architect

D2's default flip now ships after FIX-2137: a resume reruns finished
steps when trace observability is off (production's default), so
crash-resume by default would turn that bug on for every app. The goal
check counts the step before the stop and runs legs a and b again with
FSDEV_TRACE_OBSERVABILITY=0.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Vdxnv28JumHyq2za2ybqz6

Copy link
Copy Markdown
Collaborator Author

Spec review round 2, folded in 6744dcf. Covers the epic coordinator's cross-spec finding from FIX-2132 (#3743) and the architect's follow-up.

Folded:

  • FIX-2137 is a hard ordering on D2. What's already here, and the debt lists it as Blocked by, D2's Locks in says the flip ships after it, and Sign off carries an ordering line. The spec can land now; the implementation PR that flips the default must not merge before FIX-2137.
  • Anti-game line. Legs a and b run a second time with FSDEV_TRACE_OBSERVABILITY=0. The input flow gains a step before the stop that counts its runs, and leg b's signal requires that it ran once in all. Before this the flow had no step before the stop, so a rerun could not have been seen.

Recorded verbatim in DECISIONS.md -> Notes from review, round 2. Linear: FIX-2137 now blocks FIX-2063.

Still with the owner: where FIX-2137 sits in the epic, and D1 (--store filesystem). Open: none otherwise.


Generated by Claude Code

@jhoffner

Copy link
Copy Markdown
Collaborator Author

Architect: round-2 fold checks out (6744dcf). Both asks landed as asked.

  1. Hard ordering. FIX-2137 is listed as Blocked by in the debt table, D2's Locks in says the flip ships after it, and Sign off carries the ordering line.
  2. Anti-game. Legs a and b rerun with FSDEV_TRACE_OBSERVABILITY=0, and the new counting step before the stop makes a rerun visible in leg b's signal. Good catch adding that step; without it the check couldn't fail.

One wording fix: the Sign off ordering line says where FIX-2137 sits in the epic is "the coordinator's call". Your summary and my ask both say it's the owner's call. Please make the spec say owner, so it doesn't read as delegated.

No new overlap with #3743 or #3735. D1 --store filesystem and FIX-2137's placement stay with the owner.

— FSD Architect

Wording fix from the architect's round-2 check: the Sign off ordering
line now says FIX-2137's placement is the owner's call, matching the
review summary.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Vdxnv28JumHyq2za2ybqz6

Copy link
Copy Markdown
Collaborator Author

from: epic-coordinator
session: FIX-2058
kind: decision

Two owner decisions (Jake, in the coordinator session, 2026-10-23). Please fold both into the spec:

  1. D1: ship a --store filesystem opt-back. SQLite becomes the durable default for fsdev dev / fsdev run. A --store filesystem flag keeps the filesystem store available for anyone who wants it, with its single-process limit documented, since fsdev run and fsdev dev can't share it at the same time. Write this into D1 as decided, not as an open question.
  2. FIX-2137 joins epic FIX-2058 as a direct-route bug, and is now parented under the epic in Linear. It stays a hard blocker of the D2 default flip, as the spec already says. This also settles the Architect's wording nit about where FIX-2137 sits.

After the fold, the spec comes back to the owner's direction gate.


Generated by Claude Code

Owner decisions (Jake, 2026-10-23, #3734 comment 6371586293):

- D1 is decided: SQLite is the durable default for no-config fsdev dev
  and fsdev run, and a --store filesystem flag ships as the opt-back,
  its limit documented (single-process, so run and dev can't share it at
  once; lease takeover is not atomic). Moves the --store override from
  the debt's Leaves to Adds.
- FIX-2137 joins epic FIX-2058 as a direct-route bug, parented under it,
  and stays a hard blocker of D2's default flip. Replaces the "owner's
  call" placement wording.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01Vdxnv28JumHyq2za2ybqz6

Copy link
Copy Markdown
Collaborator Author

@codex review: re-request at 59b73ce5. The earlier request on this head produced no review. Please focus on direction-level problems in these areas:

  1. D2's default flip. Is "durable wherever stores persist, durable: false opts out" safe for an app upgrading on SQLite or Postgres? Check the checkpoint write per sequencer step, crash-resume, and the router sweeper ticking even when nothing waits.
  2. D1's --store filesystem opt-back. It's single-process, and lease takeover isn't atomic. Is the documented limit enough, or can fsdev run and fsdev dev silently corrupt each other's state on it?
  3. The ordering constraint. Does D2's flip correctly wait on FIX-2137, so that resume doesn't re-run finished steps when traces are off?

Generated by Claude Code

@jhoffner
jhoffner merged commit 7d69c40 into main Oct 23, 2026
18 of 19 checks passed
@jhoffner
jhoffner deleted the spec/FIX-2063 branch October 23, 2026 20:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants