Skip to content

spec(epic): keep the bus; D1/D6 flip in queries/s; FIX-2426/2428 review fixes (FIX-2367) - #4477

Merged
jhoffner merged 2 commits into
mainfrom
spec/FIX-2367-amend-1
Nov 1, 2026
Merged

jhoffner merged 2 commits into
mainfrom
spec/FIX-2367-amend-1

Conversation

@jhoffner

@jhoffner jhoffner commented Nov 1, 2026 •

Copy link
Copy Markdown
Collaborator

Amendment to the merged epic spec FIX-2367 (#4294) and its merged children FIX-2426 (#4445) and FIX-2428 (#4443). Rewritten in place; review history stays on the original PRs. #4476 (FIX-2427) and #4478 (FIX-1506) depend on this PR: it owns every FIX-2426 change, including the live crossesProcesses contract and the notice seams FIX-1506 raised.

The owner's decision: keep the bus

Epic D1's flip, as written ("widen if the queries fit with headroom at the labs' view counts"), fired: the labs and Shift Manager run one to five views, at 4.00 store queries a second each. The owner kept the bus anyway (comment). Widening would make polling the shipped way to deliver user and org changes, which FIX-1506 rules out, and FIX-380 would lose its seam.

D1 and D6 now flip in D1's own metric: store queries a second from session-stream connections, under 100 at 100 idle open views (widened to all three scopes for D1, the session stream alone for D6). Today that's about 1,200 widened and 400 for the session alone. 100 views is a small app, twenty times the labs. The bar is twelve times under the estimate, so only a redesigned read crosses it, not a tuned one.

D1: bus, Postgres, socket over widening; flips if, widened, 100 idle views cost under 100 queries a second

Evidence correction (Codex P2, comment). The 4.00 queries a second per view still holds. The pool shares and the 70–100-view crossover are gone, from the POC too: PGlite runs every query behind one lock, so its timings include waits.

What else changed

  • Epic D6 seam. The publish is one wrapper, applied in resolveStores and at each entry that takes a raw registry (runAction, BullMQ's job processor, fsdev run and chat). A board change has its own kind; the session stream wakes only on items / request / run. Set table ids swapped in.
  • FIX-2426. The open fork is settled and cites epic D1. Every write method is wrapped, patchField / incField / pushToArray / deleteField included. The wake reads at most once per intervalMs, and per-tick notice folding is allowed. Bus topic (tenantId required) and wire topic (topic.*, no user, org or tenant id) are split, and a resource-narrowed topic is its own channel. crossesProcesses may change: transitions arrive as an event (onCrossingChange), and a change to not crossing wakes waiting streams back onto the timed read. "Decided, not asked" is trimmed to what this issue decides.
  • FIX-2428. The D3 comment is fixed and D3 is tied to ER-5. useSession({ live }) stays the one way to follow a session; the headline example is now a flow read, and FIX-2429 moves liveBoardDriver. A refused topic gets the same answer as a missing session (hiddenSessionResponse), and the POST is added to the scoping table. Engine, contracts and core changesets, plus the client migration line. UserStreamEvent is removed; StreamEvent stays. pushSecret takes string | (() => string), and a failed ticket is logged. jose is allowed. { userStream, push } both stay.

Reviewers · look here

  • Is 100 queries a second at 100 views the right bar? The POC can't size a pool, so the number is a judgment, not a capacity figure.
  • The wake cap (picked over dropping in-progress publishes): latency is bounded by one interval during a burst.

Epic spec · FIX-2426 · FIX-2428

No changeset: specs only. The POC wasn't re-run (it only stops printing timings; statement counts are untouched).


Generated by Claude Code

…ew fixes (FIX-2367)

Epic D1/D6: the first flip fired at the labs' view counts and the owner kept
the bus (#4445). Both now flip under 100 store queries/s at 100 idle views.
D6's publish wraps raw-registry entries too. Set table ids swapped in.

FIX-2426: open fork settled; wrap every write method at every entry; wake
capped at one read per interval; bus vs wire topics, tenantId required;
crossesProcesses may change, delivered as a transition event (FIX-2427);
PGlite pool timings dropped as unestablished.

FIX-2428: D3 comment and ER-5 tie; one way to follow a session; refused
topic answers as missing; contracts/engine changesets; pushSecret shape.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UNkLEJU1iqxFYoPadTRfn
@vercel

vercel Bot commented Nov 1, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
flow-state-dev-build-site Ready Ready Preview Nov 1, 2026 8:53pm UTC
3 Skipped Deployments
Project Deployment Actions Updated
flow-state-dev-docs Ignored Ignored Nov 1, 2026 8:53pm UTC
flow-state-dev-kitchen-sink Ignored Ignored Nov 1, 2026 8:53pm UTC
flow-state-dev-ui Ignored Ignored Preview Nov 1, 2026 8:53pm UTC

Request Review

@linear-code

linear-code Bot commented Nov 1, 2026

Copy link
Copy Markdown

FIX-2367

@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Nov 1, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-11-01T20:55:12.504169Z 5504f93 New commits
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@jhoffner

jhoffner commented Nov 1, 2026

Copy link
Copy Markdown
Collaborator Author

@codex review

jhoffner commented Nov 1, 2026

Copy link
Copy Markdown
Collaborator Author

Second look + architecture pass (spec-only PR, so findings are about the approach, not code)

Verdict: the direction holds (keep the bus is the owner's call and the amendment records it coherently). Four points worth acting on before #4476 builds against this.

1. crossesProcesses + onCrossingChange is more API than the problem needs (FIX-2426 DECISIONS, "can change")

The event exists so a stream with its timer off can be woken back onto the timed read when the bus stops crossing. The spec then says "polling the value on each wait is an equivalent implementation", leaving two designs open on a locked public slot. Pick one. Leaner shape: keep a slow safety tick while crossing (e.g. 10-15s, ~0.07 q/s/view), have each tick re-read a plain crossesProcesses getter, and drop onCrossingChange from the adapter contract. That also removes the "stuck waiting stream" failure class instead of specifying a transition protocol for it. If the event stays, say why the safety tick was rejected and add it to the rejected-alternatives table.

2. "Wrap at every entry" is five call sites guarding one seam (FIX-2426 "One wrapper, applied at every entry")

resolveStores, runAction, createFlowJobProcessor, fsdev run, fsdev chat, plus the idempotency requirement, all exist because the wrapper is applied after a raw registry can already be handed around. That is the same "new writer forgets one" risk the rejected "publish at each write site" row cites, moved up one level. Consider decorating at registry construction/attachment (where bus is resolved against the profile) so a raw, un-wrapped registry can't be obtained, or at minimum add an engine test that enumerates the public entries taking a registry. Otherwise the next entry point (e.g. a future host adapter) silently misses views. Not a blocker; worth one line in the spec on why construction-time wrapping was or wasn't viable.

3. The D6 bar is 4x, not 12x, under today's number

PR text says the bar is "twelve times under the estimate". That's true for D1 (1,200 → 100). D6 is session-only: 400 → 100, so 4x. A tuned read (4 → 1 queries per tick) would land at 100, which is not "under 100" but is arguably tuning, not redesign. Either state D6's margin separately or restate "only a redesigned read crosses it" for D6. Also note neither flip considers a read coalesced across connections of the same session/user/org (one poll per topic rather than per view). That is the obvious redesign that would cross the bar, so the spec should say whether it's the thing the flip is waiting for or explicitly out of scope.

4. Stale "open fork" residue

specs/issues/FIX-2426/DECISIONS.md ~line 118 (analysis log) still reads "...which fired the flip and made it the open fork" while the same file now says Open: none. Reword to past tense ("settled by the owner"). Also confirm the FIX-2427/FIX-2429 specs reference onCrossingChange / the split bus/wire topic consistently (grep found no mentions there; #4476 owns the first, so check it lands in step with whichever option you pick in #1).

Smaller notes

  • 100 q/s at 100 views is a judgment, and the PR says so. Fine, but record in D1 what a team should do if they measure between the bar and today's number (i.e. who may re-flip it), or the metric is a one-time gate nobody can re-run.
  • Leading+trailing throttle at intervalMs means a streaming reply gets no better latency than today's poll (1s). The win is idle cost, not mid-reply latency. Worth one sentence so nobody expects better.
  • Pool-timing evidence correctly dropped; no remaining pool-share or 70-100 crossover references in the specs (grepped).

No changeset needed (specs only). No code to refactor here; the architecture opportunities are #1 and #2 above.


Generated by Claude Code

@chatgpt-codex-connector

Copy link
Copy Markdown

Codex Review: Didn't find any major issues. Already looking forward to the next diff.

Reviewed commit: 5af48a84a2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

…lineage limit

- ChangeNotice gains optional origin { requestId }; ER-1 widened to allow it (an id, not rows)
- A scope's subscriber hears notices narrowed under it; the session stream filters by kind
- Lineage-shared session resources publish the writer's topic only, as a stated limit
- Epic SPEC notes FIX-2410's shared state-only read

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UNkLEJU1iqxFYoPadTRfn
@jhoffner
jhoffner merged commit 4ca164c into main Nov 1, 2026
21 checks passed
@jhoffner
jhoffner deleted the spec/FIX-2367-amend-1 branch November 1, 2026 20:58
jhoffner pushed a commit that referenced this pull request Nov 1, 2026
… seams

- Resources a scope's client.derived reads now publish, and a resource
  notice reads state for them (Codex P2: a derived org count stayed stale).
- origin, narrowed notices, kind filtering and the lineage limit are cited
  as decided in FIX-2426 (#4477), not raised.
- D2: version-based dedupe recorded as dropped; safe failure is a
  duplicate read. Over-notifying bound stated. Routes share the seam's
  publish helper.
- Two structural suggestions recorded in Notes from review.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UNkLEJU1iqxFYoPadTRfn
jhoffner pushed a commit that referenced this pull request Nov 1, 2026
…4477)

- The parked-wait gap (Codex, jhoffner) is closed by FIX-2426's amended
  contract: crossesProcesses is a transition event that wakes waiting views.
  This issue now drives that event from its ping and makes no engine change.
- Pins the ping (2 s) and stale rule (3 s) FIX-2426 leaves here.
- The listener opens on the first subscriber, not when stores resolve.
- The LISTEN reconnect is shared with subscribeViaListen, not copied.
- Decided-not-asked inherits FIX-2426 instead of restating it.
- Goal signal (3) kills the backend while the view is parked, with no later
  write; signal (2) bounds the ping rate.
- Below-the-bar notes recorded verbatim in DECISIONS.md.

Re-draft: DECISIONS.md baseline 1210 words, now 1741 (verbatim review notes included).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UNkLEJU1iqxFYoPadTRfn
jhoffner added a commit that referenced this pull request Nov 1, 2026
…wers on another (#4476)

* spec(FIX-2427): the Postgres bus, a change on one process reaches viewers on another

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UNkLEJU1iqxFYoPadTRfn

* spec(FIX-2427): review round 1, built on FIX-2426's crossing event (#4477)

- The parked-wait gap (Codex, jhoffner) is closed by FIX-2426's amended
  contract: crossesProcesses is a transition event that wakes waiting views.
  This issue now drives that event from its ping and makes no engine change.
- Pins the ping (2 s) and stale rule (3 s) FIX-2426 leaves here.
- The listener opens on the first subscriber, not when stores resolve.
- The LISTEN reconnect is shared with subscribeViaListen, not copied.
- Decided-not-asked inherits FIX-2426 instead of restating it.
- Goal signal (3) kills the backend while the view is parked, with no later
  write; signal (2) bounds the ping rate.
- Below-the-bar notes recorded verbatim in DECISIONS.md.

Re-draft: DECISIONS.md baseline 1210 words, now 1741 (verbatim review notes included).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011UNkLEJU1iqxFYoPadTRfn

---------

Co-authored-by: Claude <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants