Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .changeset/postgres-bus.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
"@flow-state-dev/store-postgres": minor
"@flow-state-dev/engine": patch
---

`postgresStores` now backs the `bus` slot, so on a host with more than one process over one Postgres a session view open on one process sees a change saved on another as soon as it is saved, and an idle view costs the database nothing. Each process with an open view holds one listening connection from `liveTailPool` and pings itself through it every 2 s; it reports `crossesProcesses: true` only while the ping comes back, so behind a transaction pooler (or after a silent drop) views keep the once-a-second read, and the process logs one warning naming the fix (a `liveTailPool` on a direct connection string). `createPostgresStores` returns the bus as `registry.bus`, and `close()` hands its connection back before ending the pools. Without a tail pool (`liveTailPool: null`, `{ pool }` without one, `{ executor }`) the bus stays in process. In `@flow-state-dev/engine`, `withChangeBus` now refuses stores that already publish to a different bus instead of publishing to both, a write's notice reaches subscribers one microtask sooner, and a notice that can't be assembled is debug-logged (FIX-2427).
8 changes: 5 additions & 3 deletions apps/docs/docs/persistence/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,9 +15,9 @@ The `bus` slot carries change notices. Every request and session write sends one
| Bus | What an open session view reads |
|---|---|
| Crosses processes | Only after a change, at most once a second. A view of a quiet session costs the store nothing |
| In memory (the default today) | After a change, and also about once a second, so a reply written by another process still arrives |
| In memory | After a change, and also about once a second, so a reply written by another process still arrives |

None of the adapters below backs `bus` yet, so every profile runs the in-memory bus for now. Views keep the once-a-second read, as they always have.
The Postgres adapter backs `bus` (see [Postgres](#postgres)). The in-memory, file and SQLite adapters don't, so their profiles run the in-memory bus, and views use the once-a-second read.

| Adapter | Persistence | When to use |
|---------|------------|-------------|
Expand Down Expand Up @@ -76,6 +76,8 @@ const flowstate = createFlowState({

On Vercel, use `vercelPostgresStores()` from `@flow-state-dev/vercel/store` instead — it bakes in the pool tuning and Neon client swap.

With more than one process on one database (a web process and BullMQ workers, or several web processes), a session view open on one process sees a change saved on another as soon as it is saved, and a view of a quiet session costs the database nothing. You don't configure anything for this: the Postgres adapter backs the `bus` slot, and each process with an open view holds one listening connection and pings itself through it every 2 seconds. When the ping stops coming back, because a transaction pooler doesn't deliver `LISTEN` notifications or the connection dropped, views on that process go back to the once-a-second read until it recovers, and nothing is missed. `vercelPostgresStores()` doesn't back `bus`, so views there use the once-a-second read. The [package README](https://github.com/fixpoint-labs/flow-state-dev/tree/main/packages/store-postgres#cross-process-notices) covers the connection, the pooler warning and `liveTailPool: null`.

Postgres provides the concurrency safety that compare-and-swap relies on. Compare-and-swap means a write carries the version it expects to find, and the store applies it only if that version is still current — so a write built on a stale read is refused instead of silently overwriting someone else's.

Not every scope-state write carries a version. An increment, an append, or a write to one key of a record field is handed to the store as the operation itself, and the store applies it to the value it currently holds. On the built-in stores, concurrent increments and appends both land; a custom store that doesn't implement those operations takes a version-checked whole-record write instead, and one of the two is refused. Concurrent writes to one field don't combine — the second one replaces the first, and neither call reports a conflict. [State Operations](../fundamentals/state-operations.md#cas-semantics) lists which calls go which way.
Expand Down Expand Up @@ -410,7 +412,7 @@ Pick by where you run — there's no single "recommended stack":
- **Local dev** — in-memory (default), or the filesystem store when you want data to survive restarts. Zero setup.
- **Single server** (Railway, Fly, a VPS) — SQLite or Postgres, for durable concurrency-safe writes. The filesystem store is fine for very low concurrency, but its per-request event log doesn't hold up under real load — reach for SQLite instead.
- **Multiple servers / cloud** — Postgres. You need a shared store with real concurrency semantics and cross-instance live tail (Postgres uses `LISTEN/NOTIFY`). `LISTEN/NOTIFY` is a wake-up signal with a real fan-out ceiling; at high event volume, put streaming on Redis or NATS behind the same `subscribeToEvents` interface.
- **Serverless / edge** — Postgres over an HTTP driver (e.g. a Neon/Supabase pooler). `LISTEN/NOTIFY` doesn't survive transaction poolers, so use an HTTP-based service (e.g. Upstash) for live updates.
- **Serverless / edge** — Postgres through a pooled endpoint (e.g. Neon or Supabase), with `vercelPostgresStores()` on Vercel. A transaction pooler doesn't deliver `LISTEN` notifications, so session views there use the once-a-second read. On a long-lived host behind a pooler, pass `liveTailPool` a direct connection string to get changes across processes as they are saved; see [Postgres](#postgres).
- **Special requirements** — implement a custom store against the `StoreAdapter` interface.

### Resource vs. app-owned tables
Expand Down
1 change: 1 addition & 0 deletions docs/mechanisms/mechanisms.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -299,6 +299,7 @@ mechanisms:
package: "@flow-state-dev/engine"
anchor: createInMemoryBus
file: packages/engine/src/bus/in-memory-bus.ts
adapters: ["@flow-state-dev/store-postgres"]
builtOn: [store]
overlaps:
- { id: streaming, relation: distinct-jobs, note: "streaming sends one request's item log and ends with it; a bus topic is a scope's and outlives every request." }
Expand Down
2 changes: 2 additions & 0 deletions goals/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -114,6 +114,8 @@ Every goal repeats the same scaffolding around the part that is actually its own
| `server` | `refuseIfAnswering`, `stopProcessGroup`, `onClosed` — for a goal that spawns its own server: refuse when anything already answers on the port, and stop the whole detached process group so no survivor is left for the next run to grade; `onClosed(child, done)` waits for a piped child's output to be read to the end (`close`, not `exit`), but at most 2s past `exit`, so a grandchild still holding the pipe can't hang a stop |
| `module-patch` | `modulePatchEnv`, `describePatches`, `assertPatched` — a named control as scratch patches to source modules, applied as the process under test loads them (`module-patch.mjs` through `NODE_OPTIONS`), never written to the checkout; the run prints the patch and fails when it never reached the code it served |
| `http-actions` | `actingAs(router, userId)` (`call`, `act`, `open`) — drive an in-process host's router as a verified caller and read an action's answer back from the session's ended request, as an app does; `untilIdle(stores)` — wait until no request is running. The host's `resolvePrincipal` reads the caller from `x-user` |
| `postgres` | `startPostgres({ slug, settings? })` — a real Postgres for a goal that grades what crosses between processes: `FSD_DB_URL` (with a scratch database on it when the role may make one), or a throwaway cluster from `initdb` and `pg_ctl` on a free port, run as the `postgres` user under root; `query` runs setup and `pg_terminate_backend`, `stop` removes it. PGlite can't stand in: it can't `LISTEN` across processes. **Not re-exported from `index.mts`** (it loads `pg`); import `../../lib/postgres.mts` directly |
| `peers` | `forkPeer(module, name, env)` and, in the forked module, `servePeer(commands)` — real processes (`child_process.fork`, under the same loader and control patches) driven over IPC: `call` a named command, hear what the peer `push`es. One process with two registries is not two processes. **Not re-exported from `index.mts`**; import `../../lib/peers.mts` directly |
| `playwright` | `launchChromium`, `preinstalledChromium` — Chromium for a goal that reads the shipped DevTool. **Not re-exported from `index.mts`**, so a goal that opens no browser does not load Playwright; import `../../lib/playwright.mts` directly |
| `shift-manager` | `buildShiftManagerCopy` (a patched scratch copy, built with Vite), `startShiftManager` (its command over a Lab, with `NODE_ENV` as it was before any in-process build; optional `root`, `tsx` and `timeoutMs` to serve another checkout, including an older one that has only `bin/start.mts`), `labApi` (the Lab's routes), `pendingSeatAsks` (the person's pending asks on the seats' sessions, picked as Shift Manager picks them), `workerOf` (the worker a listed session names) and `pickShift` (a theme picked with the sidebar's theme mark) — for the Shift Manager goals; and, for a goal that drives Shift Manager as a person in Playwright, `buildShiftManagerPages`, `labRoutes` (`labApi` read as one verified user) and `personPage` (a browser context as a given person). **Not re-exported from `index.mts`**; import `../../lib/shift-manager.mts` directly |

Expand Down
134 changes: 134 additions & 0 deletions goals/lib/peers.mts
Original file line number Diff line number Diff line change
@@ -0,0 +1,134 @@
/**
* Real processes for a goal that grades what crosses between them (FIX-2427,
* shared with the epic closure FIX-2368).
*
* One process with two registries is not two processes: it shares module
* state, timers and an event loop, and a bus that only reaches its own
* process passes there. So the goal forks each peer (`child_process.fork`,
* under the same `tsx` loader) and drives it over IPC: the runner calls a
* named command and awaits its answer, and the peer may push events back
* unasked (an SSE frame its view received, stamped where it was received).
*
* The runner side is {@link forkPeer}; the peer side, in the forked module,
* is {@link servePeer}. Not re-exported from `index.mts`; import directly:
*
* import { forkPeer } from "../../lib/peers.mts"; // the runner
* import { servePeer } from "../../lib/peers.mts"; // the peer module
*/
import { fork, type ChildProcess } from "node:child_process";

type Call = { id: number; command: string; args: unknown };
type Answer = { id: number; ok: true; value: unknown } | { id: number; ok: false; error: string };
type Pushed = { event: string; data: unknown };

/** A forked peer, as the runner holds it. */
export type Peer = {
readonly name: string;
/** Run `command` in the peer and await what it returns. Rejects with the peer's error. */
call<T = unknown>(command: string, args?: unknown): Promise<T>;
/** Hear every `event` the peer pushes. */
on(event: string, listener: (data: unknown) => void): void;
/** Ask the peer to stop (its `stop` command, when it has one), then end the process. */
stop(): Promise<void>;
};

/**
* Fork `modulePath` as a peer named `name`, with `env` added to this
* process's environment, and wait until it says it is ready.
*
* @throws When the peer exits or fails before it is ready.
*/
export async function forkPeer(modulePath: string, name: string, env: Record<string, string> = {}): Promise<Peer> {
const child: ChildProcess = fork(modulePath, [], {
env: { ...process.env, ...env, GOAL_PEER_NAME: name },
stdio: ["ignore", "inherit", "inherit", "ipc"]
});
const waiting = new Map<number, { resolve(value: unknown): void; reject(error: Error): void }>();
const listeners = new Map<string, Array<(data: unknown) => void>>();
let next = 0;
let exited = false;

const ready = new Promise<void>((resolve, reject) => {
child.on("message", (message: Answer | Pushed | { ready: true }) => {
if ("ready" in message) return resolve();
if ("event" in message) {
for (const listener of listeners.get(message.event) ?? []) listener(message.data);
return;
}
const pending = waiting.get(message.id);
waiting.delete(message.id);
if (message.ok) pending?.resolve(message.value);
else pending?.reject(new Error(`${name}: ${message.error}`));
});
child.on("exit", (code, signal) => {
exited = true;
const error = new Error(`${name} exited (${code ?? signal})`);
reject(error);
for (const pending of waiting.values()) pending.reject(error);
waiting.clear();
});
});
await ready;

return {
name,
call<T>(command: string, args?: unknown): Promise<T> {
if (exited) return Promise.reject(new Error(`${name} has exited`));
const id = (next += 1);
return new Promise<T>((resolve, reject) => {
waiting.set(id, { resolve: resolve as (value: unknown) => void, reject });
child.send({ id, command, args } satisfies Call);
});
},
on(event, listener) {
listeners.set(event, [...(listeners.get(event) ?? []), listener]);
},
async stop() {
if (exited) return;
const gone = new Promise<void>((resolve) => child.once("exit", () => resolve()));
try {
await Promise.race([
this.call("stop"),
new Promise((resolve) => setTimeout(resolve, 10_000))
]);
} catch {
// A peer with no `stop`, or one already failing, is ended below.
}
if (!exited) child.kill("SIGTERM");
await gone;
}
};
}

/**
* Serve `commands` to the runner that forked this process, then say ready.
* A `stop` command, when given, runs before the process exits on the
* runner's `stop()`.
*
* @returns `push`, to send the runner an event unasked.
*/
export function servePeer(commands: Record<string, (args: any) => unknown>): { push(event: string, data: unknown): void } {
if (process.send === undefined) throw new Error("servePeer: this module runs only as a forked peer (forkPeer)");
const send = process.send.bind(process);
process.on("message", (message: Call) => {
const handler = commands[message.command];
void (async () => {
try {
if (handler === undefined) throw new Error(`no command "${message.command}"`);
const value = await handler(message.args);
send({ id: message.id, ok: true, value } satisfies Answer);
if (message.command === "stop") setTimeout(() => process.exit(0), 10);
} catch (error) {
send({ id: message.id, ok: false, error: error instanceof Error ? (error.stack ?? error.message) : String(error) } satisfies Answer);
}
})();
});
// The runner owns this process: it lives while the IPC channel does.
process.on("disconnect", () => process.exit(0));
send({ ready: true });
return {
push(event, data) {
send({ event, data } satisfies Pushed);
}
};
}
Loading
Loading