Skip to content

ci(guards): run the isolation-coordinate check, with a negative control - #1906

Merged
jhoffner merged 2 commits into
mainfrom
fix/wire-isolation-guard
Sep 19, 2026
Merged

jhoffner merged 2 commits into
mainfrom
fix/wire-isolation-guard

Conversation

@jhoffner

@jhoffner jhoffner commented Sep 18, 2026 •

Copy link
Copy Markdown
Collaborator

The problem

scripts/check-isolation-coordinate.mjs has been on main since FIX-1420. It is 168 carefully documented lines, it exits 0, and nothing runs it — not package.json, not any workflow, not a skill. Verified by grep across all five.

An unrun guard is indistinguishable from one nobody wrote. Three instruments were built across cycles 12 and 13 and none was retained: this one wired to nothing, and two throwaway checkers on #1886 never committed at all.

What this does

Wires it as a step in the Process guards job, beside validate-model-strings.mjs, whose rationale it shares verbatim — a doc teaching a removed rule is as wrong on main as on a PR.

Makes it testable, and tests it. The two scanners are exported pure functions over text, the corpus walk is private behind corpusFiles(), and main() runs only when the file is invoked directly. Fifteen negative controls land at packages/core/test/isolation-coordinate-check.test.ts, matching the sibling *-check.test.ts pattern.

The controls are aimed at the defect that produced the guard. FIX-1420's first scan was a phrase list generalised from the stale lines it had already found — it could only confirm its own list, and it reported green while two sites survived. So one control asserts the guard fires on a wording it was never shown; three assert silence on the current rule, on a route lookup, and on isolation used as a topic. Three more cover the aria-label drift the token scan cannot see at all.

Then review found three more ways the guard could report a false green, and closing them changed the scanners' behaviour in three specific places. Each is deliberate, and nothing else about the scan moved:

  1. A missing docs root scanned as zero files and exited 0. Renaming docs/atlas and running the pre-change script prints OK. The walk no longer swallows read errors, the roots are asserted before the scan, and main() exits 1 naming the missing root.
  2. Ordinary prose wrapping walked straight through the line scan, and all nine original controls used one-line fixtures, so the suite stayed green over the hole. The match window is now a sentence — wrapped lines rejoined, then split on sentence boundaries. scanLines is renamed scanClaims, since it no longer scans lines.
  3. The figure gate's hand-written vocabulary is derived from KEYING instead of restated, so the two scanners can't drift apart the way they did before.

What's asked of you

Nothing blocking. Two things on the record:

Two reviewers disagreed on where this belongs, and the disagreement is resolved here so it isn't re-litigated. Cursor recommended the typecheck rail. Cursor was wrong on the facts: root typecheck runs three validate-*.mjs scripts plus a differently-named goals guard, and all three are source-shape validators. Codex had it right — the five docs-corpus guards live in the Process guards job, which is where a documentation-claim check belongs. The half of Cursor's advice that survives is the one that mattered most: the negative control.

The window widened from a line to a sentence, and a wider window is how a guard starts crying wolf. The choice was measured rather than argued — see the table below. Two style suggestions were declined with reasons on their threads (the test's describe narrative, and the CI step comment), as was renaming check-* to validate-*.

Parts worth reviewing closely

  • Whether a sentence is the right window. It is wider than a line and narrower than a paragraph, and the measurement below is the whole argument for that spot. If the noise numbers don't convince you, this is the call to push on.
  • The three deliberate behaviour changes above. Everything else about what the scanners accept and reject is unchanged; main() is still guarded by the import.meta.url === pathToFileURL(process.argv[1]).href check the siblings use, so importing the module does not run the scan.
  • Whether the controls are the right ones. They encode why the guard is a concept co-occurrence match rather than a phrase list. If that reasoning is wrong, the tests are wrong with it.

Verification

Wired to nothing, before: grep over package.json, .github/, scripts/, docs/, AGENTS.md, CLAUDE.md, .agents/ — no hits outside the script itself.

Green on this tree: node scripts/check-isolation-coordinate.mjs → exit 0, ~70ms over 53 files.

Import does not run the scan: importing the module prints only missingRoots, scanClaims, scanFigureDrift.

Suite: 15 controls pass. pnpm --filter @flow-state-dev/core test → 130 files, 2070 tests pass. pnpm typecheck → 71 tasks pass.

Still fires, unwrapped: a planted namespaces its rows by flow kind line in docs/architecture/overview.md → exit 1, naming file and line. Tree restored.

Fires on a wrapped claim, which it previously could not. Planting a claim split the way prose actually wraps —

Each user-scoped resource is namespaced
by the flow kind, so two seats share a row.

— the pre-change script prints OK and exits 0; this one exits 1 naming overview.md:210.

A missing root is now loud. With docs/atlas renamed, the pre-change script prints OK and exits 0. This one exits 1: cannot scan the corpus — configured docs root(s) not found: docs/atlas.

The corpus wraps, measured not assumed: 2,986 mid-sentence line breaks across 40 of the 53 scanned files (1,013 multi-line prose paragraphs). The blind spot was live, not theoretical.

Why a sentence and not a paragraph. Candidate windows run against the current correct corpus, where every hit is by definition a false positive:

window false positives
physical line (before) 0 — but blind to wrapping
sentence (ships) 0
two-line sliding 3
paragraph 3

Every control proved red by mutation before its green was trusted:

mutation reds
narrow KIND back to a phrase list (the FIX-1420 defect) 5
drop the COORDINATE comparison in the drift scan 1
revert scanClaims to the physical-line window 2
widen the window to a paragraph 1 (the silence control)
restore the figure gate's old hand-written literal 1
stub missingRoots to return [] 1
rename docs/atlas on disk 1

Every row was re-run against the code that ships; the first was 3 before the window changed, which is the kind of stale number this block exists to prevent. The scanClaims row earned its keep twice over: it caught a bad fixture of mine, where the wrap split at a point that left all three concepts on one line, so the control passed under the very mutation it was meant to detect. The fixture now splits where wrapping actually hides a claim.

Rationale for why a per-claim guard should be retained at all is in #1904's cycle-13 entry rather than restated here. The broader policy question — deliverable or scratch file — is filed separately and is not decided by this PR.

No changeset — CI config and a test, no published-package consumer (BP-022).

🤖 Generated with Claude Code

https://claude.ai/code/session_01ULydJv3JEeMUgeZbSvm4Rt

The script has been on main since FIX-1420, 168 documented lines, exiting 0,
and referenced by nothing — not package.json, not any workflow, not a skill.
An unrun guard is indistinguishable from one nobody wrote.

Wired as a step in the Process guards job, beside validate-model-strings.mjs,
whose rationale it shares verbatim: a doc teaching a removed rule is as wrong
on main as on a PR. Not the typecheck rail — that carries three source-shape
validators; the docs-corpus guards live here.

Made importable so it can be tested: the two scanners are now exported pure
functions over text, the walk is behind corpusFiles(), and main() runs only
when the file is invoked directly. Behaviour is unchanged — it still exits 0 on
this tree and still exits 1 naming the line when one is planted.

Nine negative controls at packages/core/test/isolation-coordinate-check.test.ts,
matching the sibling *-check.test.ts pattern. They cover the defect that
produced the guard: FIX-1420's first scan was a phrase list generalised from
the lines it had already found, so it confirmed its own list and reported green
while two sites survived. One control asserts the guard fires on a wording it
was never shown; three assert silence on correct sentences, a route lookup, and
isolation used as a topic. Three more cover the aria-label drift the token scan
cannot see at all.

Both halves proved able to fail: narrowing KIND back to a phrase list reds 3
tests, dropping the drift comparison reds 1.

No changeset — CI config and a test, no published-package consumer (BP-022).

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ULydJv3JEeMUgeZbSvm4Rt
@vercel

vercel Bot commented Sep 18, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

3 Skipped Deployments
Project Deployment Actions Updated
flow-state-dev-docs Ignored Ignored Sep 19, 2026 1:20am UTC
flow-state-dev-kitchen-sink Ignored Ignored Sep 19, 2026 1:20am UTC
flow-state-dev-ui Ignored Ignored Preview Sep 19, 2026 1:20am UTC

Request Review

jhoffner pushed a commit that referenced this pull request Sep 18, 2026
Self-reported by the FIX-1377 implementation after cycle 13's collection
closed. Purely additive: no row, no class total, no reading-label rate, and
nothing above re-scored. Cursor has already passed the entry and its numbers
should not churn.

The fabricated citation is the one worth the space, and it is a class the entry
does not have. Every stale-restatement instance here is a claim that was true
and became false; #1888 was never true. Verified rather than relayed: #1888 is
a real PR — spec(FIX-1440), a different issue — so it was referentially valid
and semantically wrong, which is the one shape an existence check cannot see.
Both checkers this entry praises pass it, as does the figure agreement and the
surface sweep it just landed. It did not reach main, and the branch now carries
#1911 at all three sites. Proposed as `invented-reference` rather than minted,
following the precedent set for vacuous-assertion, and carried into claims as
something to collect deliberately instead of waiting for another self-report.

The control script that reverted uncommitted work is recorded as a hazard in
the plant-and-restore pattern, not against it — that pattern produced most of
this cycle's good evidence, including #1906's mutation runs. Commit before you
plant; `git checkout --` cannot tell a regression from your work.

The counter-evidence is stated as weak and reads that way. Two authors applying
the discipline unprompted on their own work is the first author-side catch in
thirteen cycles and cuts against the entry's own pessimistic reading, but n=2,
same day, same briefing. Scored in cycle 14 on authors with no such briefing.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ULydJv3JEeMUgeZbSvm4Rt
@jhoffner
jhoffner marked this pull request as ready for review September 19, 2026 01:04
@chatgpt-codex-connector

chatgpt-codex-connector Bot commented Sep 19, 2026 •

Copy link
Copy Markdown

Codex Review Summary

This comment shows the latest Codex review activity on this pull request.

Review Status Commit Review trigger
📝 Code Review ✅ Completed 2026-09-19T01:06:48.995578Z fd6dc71 Draft marked ready
ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review" or "@codex security review".

Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings.

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Simplify review (read-only)

Verdict: Approve. Wiring this guard into CI is the right fix (an unrun guard is indistinguishable from no guard), and the refactor matches how sibling guards work (validate-model-strings.mjs → packages/core/test/*-check.test.ts → unconditional node scripts/… in ci.yml). I ran node scripts/check-isolation-coordinate.mjs and pnpm --filter @flow-state-dev/core test isolation-coordinate-check locally; both pass.

Did this go too far?

Probably not for this repo, given the stated goal: a negative control, not just green CI. Exporting pure scanLines / scanFigureDrift and proving “fires on unseen wording / stays silent on correct corpus” is the same contract as model-strings-check.test.ts. A smaller change would have been “CI step only” on the pre-refactor script — fewer lines, but you could not demonstrate the guard fails without re-deriving FIX-1420’s blind spot in review. That tradeoff looks intentional and reasonable here.

Where I’d still trim without changing behavior:

Area Suggestion Confidence
corpusFiles export Only main() uses it; tests don’t. Drop the export or add a tiny scan-surface test (like isScannedPath in model-strings) if ROOTS/EXCLUDE are likely to move. Medium — API hygiene
Duplicated “keying” vocabulary scanFigureDrift reintroduces `/flowisolation isolat
Narrative length Script header is the right canonical story; test describe blocks largely repeat FIX-1420 / aria-label history. One-line framing per block + pointer to script header would match the spirit of model-strings tests without losing intent. Low — style
walk() empty catch Missing docs/atlas (etc.) yields silent empty subtree → false green. Assert roots exist in main() or rethrow non-ENOENT. Low unless paths are fragile
Naming Lone check-* among validate-* scripts; rename only if you care about directory grep ergonomics. Low

Alternatives considered (not clearly better)

  • Third-party markdown/HTML parser: Heavier dependency and different failure mode than existing regex guards; ~53 files / ~36ms today doesn’t justify it.
  • Tests co-located under scripts/: Clearer ownership, but fights the established packages/core/test/*-check.test.ts convention.
  • Merge figure + line into one scanner: Would obscure two failure classes and two error messages in main(); keep separate passes.

Performance / structure

No CI concern: refactor reduces double walk/double read vs the old script; full-corpus memory is ~1.75MB. Cross-import packages/core → scripts/ is the same smell as every other guard test — acceptable here.

Skipped (needs product call): Whether scanFigureDrift belongs in v1 of “wire CI” vs a follow-up; the measured aria-label blind spot makes a decent case for keeping it in this PR.

Open in Web View Automation 

Sent by Cursor Automation: Review For Best Approach

Comment thread scripts/check-isolation-coordinate.mjs Resolved
Comment thread scripts/check-isolation-coordinate.mjs Resolved
Comment thread scripts/check-isolation-coordinate.mjs Resolved
const lines = (text: string): LineHit[] => (scanLines as (t: string) => LineHit[])(text);
const drift = (src: string): DriftHit[] => (scanFigureDrift as (s: string) => DriftHit[])(src);

/**

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These blocks are valuable history, but they duplicate the script header almost verbatim. For simplify/readability, I’d lean toward short describe titles + “see scripts/check-isolation-coordinate.mjs header” — same pattern density as model-strings-check without losing the negative-control cases themselves (those are the important part).

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Declining this one, deliberately.

These blocks aren't history duplicated from the header — they encode why the guard is a concept co-occurrence match and not a phrase list, which is the reasoning that makes the controls reviewable. Replacing it with a pointer to a header in another directory is precisely how that reasoning gets lost: the next person editing the test sees nine assertions with no argument attached and has to go find one.

This round made the case concretely. Two of the six new controls only exist because the describe narrative stated what the suite was supposed to prove — which is what exposed that it wasn't proving it for wrapped prose.

The suggestion is right about model-strings-check being leaner. I'd rather that file gained the argument than this one shed it.


Generated by Claude Code

Comment thread .github/workflows/ci.yml
- name: No removed model-string syntax
run: node scripts/validate-model-strings.mjs

# Same shape and the same reason as the step above: a doc teaching the

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Aligned with the neighboring model-strings comment style. Optional trim: the script header already owns the FIX-1420 rationale; this step could be a single line (“unconditional on main — see script header”) if you want less triple-maintenance.

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Declining the trim, and it's your own first sentence that decides it: this is aligned with the neighbouring validate-model-strings step, which carries the same shape of comment for the same reason. Trimming this one to a pointer makes it the odd step out in a job where five guards all explain themselves in place. Conformance beats concision inside a file like this.

On triple-maintenance: the step comment doesn't restate the rule, it says why the step is unconditional and hands the rest to the script header. That division is the thing keeping it from drifting.

The step is unchanged in this push.


Generated by Claude Code

@cursor cursor Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Snob — already tight. No POC.

I read .github/workflows/ci.yml (the new Process-guards step), scripts/check-isolation-coordinate.mjs (the two scanners, corpusFiles(), the import.meta main gate), packages/core/test/isolation-coordinate-check.test.ts, and the sibling shape in scripts/validate-model-strings.mjs, packages/core/test/model-strings-check.test.ts, and packages/core/test/spec-folder-check.test.ts.

A rewrite would not shrink the contract. The new contract is one CI step plus two filesystem-free scanners plus nine controls that pin fire/silence. Folding the scanners into a sibling scanSources([{path,text}]) adds a path the tests never assert; scanLines(text) is already the smaller form. Dropping the nine controls or testing via spawn leaves the recorded negative-control half on the floor and grows the harness. Moving the step onto the typecheck rail is the call the body already closed. corpusFiles() is exported and unused by the tests — unexport it in a line if you want; it is not a shape.

Not re-litigating Process-guards-vs-typecheck, the concept-vs-phrase-list rationale, or the deliverable-vs-scratch policy the PR filed off.

Open in Web View Automation 

Sent by Cursor Automation: Code Snob

… green

Review found three variants of one defect: the guard reporting green for a
reason unrelated to the corpus being clean.

A missing docs root scanned as zero files and exited 0. Proved: renaming
docs/atlas and running the pre-change script prints OK. The walk no longer
swallows read errors, the roots are asserted before the scan, and main()
exits 1 with the missing root named.

Prose wrapping walked through the line scan. A claim split across two lines
matched nothing, and all nine controls used one-line fixtures, so the suite
stayed green over the hole. The corpus wraps — 2,986 mid-sentence breaks
across 40 of 53 scanned files — so this was live, not latent. Wrapped lines
are now rejoined and the window is a sentence. Deliberately not a paragraph:
measured on the current corpus, a paragraph window reports 3 false positives
and a two-line window 3, while the sentence window reports 0. scanLines is
renamed scanClaims, since it no longer scans lines.

The figure gate's hand-written vocabulary is derived from KEYING instead.
STORAGE_SUBJECT is deliberately not unioned in and the reason is recorded at
the site: its `row` and `cell` mean something else in a diagram label, and
unioning it fires on workforce.html:2039, a dispatch figure committing to no
storage coordinate.

corpusFiles is module-private again; nothing outside main() used it.

Six controls added, each proved red by mutation before its green was
trusted. Behaviour changes are limited to the three above.

Co-Authored-By: Claude <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ULydJv3JEeMUgeZbSvm4Rt

Copy link
Copy Markdown
Collaborator Author

Review round handled in 28643e51. Six findings, four fixed, three declined (one of them from the review summary rather than a thread).

Fixed — each with a control proved red by mutation before its green was trusted:

  • Empty catch on readdirSync. Not intentional. The pre-change script prints OK and exits 0 with docs/atlas renamed out of the way — a whole docs root gone and CI stays green. Now loud, with two controls.
  • Wrapped claims evaded the scan (Greptile P2, and the sharpest finding here). Measured before fixing: the corpus has 2,986 mid-sentence line breaks across 40 of 53 scanned files, so the hole was live. The window is now a sentence, not a physical line. Deliberately not a paragraph — measured on the correct corpus, a paragraph window reports 3 false positives and a two-line window 3, while the sentence window reports 0. Both halves pinned: it fires on the wrapped claim, and stays silent when concepts merely neighbour each other.
  • Figure gate's duplicated vocabulary. Derived from KEYING now. Checking it first changed the fix: unioning STORAGE_SUBJECT in fires on a real figure (workforce.html:2039), so the one genuine difference is recorded at the site instead of assumed away.
  • corpusFiles exported but unused. Module-private; verified nothing else imported it.

Declined, with reasons on their threads: the test's describe narrative (it carries the argument that makes the controls reviewable — and it is what exposed the wrapping hole), and the CI step comment (conforms to the sibling validate-model-strings step).

Also declined: renaming check-* to validate-*. It was raised in the review summary table rather than on a thread, so answering here. Grep ergonomics don't pay for moving a file that a CI step, a test import and the script's own header all reference. The cost is real and immediate; the benefit is a slightly tidier ls.

Two things worth flagging for a human reader rather than burying in a thread:

  1. scanLines is renamed scanClaims. It no longer scans lines, and a guard whose function name misdescribes what it does is the failure mode this whole PR is about.
  2. One of my own new controls was wrong on the first try. The wrapped fixture split at a point where the second line still carried all three concepts, so it passed under the very mutation meant to red it. Caught only because I ran the mutation. The PR body's verification block has the corrected numbers, re-derived against the code that ships rather than carried over.

Not merging, no labels applied.


Generated by Claude Code

@jhoffner
jhoffner merged commit e6ccb22 into main Sep 19, 2026
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants