Repository navigation
ci(guards): run the isolation-coordinate check, with a negative control - #1906
Conversation
The script has been on main since FIX-1420, 168 documented lines, exiting 0, and referenced by nothing — not package.json, not any workflow, not a skill. An unrun guard is indistinguishable from one nobody wrote. Wired as a step in the Process guards job, beside validate-model-strings.mjs, whose rationale it shares verbatim: a doc teaching a removed rule is as wrong on main as on a PR. Not the typecheck rail — that carries three source-shape validators; the docs-corpus guards live here. Made importable so it can be tested: the two scanners are now exported pure functions over text, the walk is behind corpusFiles(), and main() runs only when the file is invoked directly. Behaviour is unchanged — it still exits 0 on this tree and still exits 1 naming the line when one is planted. Nine negative controls at packages/core/test/isolation-coordinate-check.test.ts, matching the sibling *-check.test.ts pattern. They cover the defect that produced the guard: FIX-1420's first scan was a phrase list generalised from the lines it had already found, so it confirmed its own list and reported green while two sites survived. One control asserts the guard fires on a wording it was never shown; three assert silence on correct sentences, a route lookup, and isolation used as a topic. Three more cover the aria-label drift the token scan cannot see at all. Both halves proved able to fail: narrowing KIND back to a phrase list reds 3 tests, dropping the drift comparison reds 1. No changeset — CI config and a test, no published-package consumer (BP-022). Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ULydJv3JEeMUgeZbSvm4Rt
|
The latest updates on your projects. Learn more about Vercel for GitHub. 3 Skipped Deployments
|
Self-reported by the FIX-1377 implementation after cycle 13's collection closed. Purely additive: no row, no class total, no reading-label rate, and nothing above re-scored. Cursor has already passed the entry and its numbers should not churn. The fabricated citation is the one worth the space, and it is a class the entry does not have. Every stale-restatement instance here is a claim that was true and became false; #1888 was never true. Verified rather than relayed: #1888 is a real PR — spec(FIX-1440), a different issue — so it was referentially valid and semantically wrong, which is the one shape an existence check cannot see. Both checkers this entry praises pass it, as does the figure agreement and the surface sweep it just landed. It did not reach main, and the branch now carries #1911 at all three sites. Proposed as `invented-reference` rather than minted, following the precedent set for vacuous-assertion, and carried into claims as something to collect deliberately instead of waiting for another self-report. The control script that reverted uncommitted work is recorded as a hazard in the plant-and-restore pattern, not against it — that pattern produced most of this cycle's good evidence, including #1906's mutation runs. Commit before you plant; `git checkout --` cannot tell a regression from your work. The counter-evidence is stated as weak and reads that way. Two authors applying the discipline unprompted on their own work is the first author-side catch in thirteen cycles and cuts against the entry's own pessimistic reading, but n=2, same day, same briefing. Scored in cycle 14 on authors with no such briefing. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ULydJv3JEeMUgeZbSvm4Rt
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
Simplify review (read-only)
Verdict: Approve. Wiring this guard into CI is the right fix (an unrun guard is indistinguishable from no guard), and the refactor matches how sibling guards work (validate-model-strings.mjs → packages/core/test/*-check.test.ts → unconditional node scripts/… in ci.yml). I ran node scripts/check-isolation-coordinate.mjs and pnpm --filter @flow-state-dev/core test isolation-coordinate-check locally; both pass.
Did this go too far?
Probably not for this repo, given the stated goal: a negative control, not just green CI. Exporting pure scanLines / scanFigureDrift and proving “fires on unseen wording / stays silent on correct corpus” is the same contract as model-strings-check.test.ts. A smaller change would have been “CI step only” on the pre-refactor script — fewer lines, but you could not demonstrate the guard fails without re-deriving FIX-1420’s blind spot in review. That tradeoff looks intentional and reasonable here.
Where I’d still trim without changing behavior:
| Area | Suggestion | Confidence |
|---|---|---|
corpusFiles export |
Only main() uses it; tests don’t. Drop the export or add a tiny scan-surface test (like isScannedPath in model-strings) if ROOTS/EXCLUDE are likely to move. |
Medium — API hygiene |
| Duplicated “keying” vocabulary | scanFigureDrift reintroduces `/flowisolation |
isolat |
| Narrative length | Script header is the right canonical story; test describe blocks largely repeat FIX-1420 / aria-label history. One-line framing per block + pointer to script header would match the spirit of model-strings tests without losing intent. |
Low — style |
walk() empty catch |
Missing docs/atlas (etc.) yields silent empty subtree → false green. Assert roots exist in main() or rethrow non-ENOENT. |
Low unless paths are fragile |
| Naming | Lone check-* among validate-* scripts; rename only if you care about directory grep ergonomics. |
Low |
Alternatives considered (not clearly better)
- Third-party markdown/HTML parser: Heavier dependency and different failure mode than existing regex guards; ~53 files / ~36ms today doesn’t justify it.
- Tests co-located under
scripts/: Clearer ownership, but fights the establishedpackages/core/test/*-check.test.tsconvention. - Merge figure + line into one scanner: Would obscure two failure classes and two error messages in
main(); keep separate passes.
Performance / structure
No CI concern: refactor reduces double walk/double read vs the old script; full-corpus memory is ~1.75MB. Cross-import packages/core → scripts/ is the same smell as every other guard test — acceptable here.
Skipped (needs product call): Whether scanFigureDrift belongs in v1 of “wire CI” vs a follow-up; the measured aria-label blind spot makes a decent case for keeping it in this PR.
Sent by Cursor Automation: Review For Best Approach
| const lines = (text: string): LineHit[] => (scanLines as (t: string) => LineHit[])(text); | ||
| const drift = (src: string): DriftHit[] => (scanFigureDrift as (s: string) => DriftHit[])(src); | ||
|
|
||
| /** |
There was a problem hiding this comment.
These blocks are valuable history, but they duplicate the script header almost verbatim. For simplify/readability, I’d lean toward short describe titles + “see scripts/check-isolation-coordinate.mjs header” — same pattern density as model-strings-check without losing the negative-control cases themselves (those are the important part).
There was a problem hiding this comment.
Declining this one, deliberately.
These blocks aren't history duplicated from the header — they encode why the guard is a concept co-occurrence match and not a phrase list, which is the reasoning that makes the controls reviewable. Replacing it with a pointer to a header in another directory is precisely how that reasoning gets lost: the next person editing the test sees nine assertions with no argument attached and has to go find one.
This round made the case concretely. Two of the six new controls only exist because the describe narrative stated what the suite was supposed to prove — which is what exposed that it wasn't proving it for wrapped prose.
The suggestion is right about model-strings-check being leaner. I'd rather that file gained the argument than this one shed it.
Generated by Claude Code
| - name: No removed model-string syntax | ||
| run: node scripts/validate-model-strings.mjs | ||
|
|
||
| # Same shape and the same reason as the step above: a doc teaching the |
There was a problem hiding this comment.
Aligned with the neighboring model-strings comment style. Optional trim: the script header already owns the FIX-1420 rationale; this step could be a single line (“unconditional on main — see script header”) if you want less triple-maintenance.
There was a problem hiding this comment.
Declining the trim, and it's your own first sentence that decides it: this is aligned with the neighbouring validate-model-strings step, which carries the same shape of comment for the same reason. Trimming this one to a pointer makes it the odd step out in a job where five guards all explain themselves in place. Conformance beats concision inside a file like this.
On triple-maintenance: the step comment doesn't restate the rule, it says why the step is unconditional and hands the rest to the script header. That division is the thing keeping it from drifting.
The step is unchanged in this push.
Generated by Claude Code
There was a problem hiding this comment.
Code Snob — already tight. No POC.
I read .github/workflows/ci.yml (the new Process-guards step), scripts/check-isolation-coordinate.mjs (the two scanners, corpusFiles(), the import.meta main gate), packages/core/test/isolation-coordinate-check.test.ts, and the sibling shape in scripts/validate-model-strings.mjs, packages/core/test/model-strings-check.test.ts, and packages/core/test/spec-folder-check.test.ts.
A rewrite would not shrink the contract. The new contract is one CI step plus two filesystem-free scanners plus nine controls that pin fire/silence. Folding the scanners into a sibling scanSources([{path,text}]) adds a path the tests never assert; scanLines(text) is already the smaller form. Dropping the nine controls or testing via spawn leaves the recorded negative-control half on the floor and grows the harness. Moving the step onto the typecheck rail is the call the body already closed. corpusFiles() is exported and unused by the tests — unexport it in a line if you want; it is not a shape.
Not re-litigating Process-guards-vs-typecheck, the concept-vs-phrase-list rationale, or the deliverable-vs-scratch policy the PR filed off.
Sent by Cursor Automation: Code Snob
… green Review found three variants of one defect: the guard reporting green for a reason unrelated to the corpus being clean. A missing docs root scanned as zero files and exited 0. Proved: renaming docs/atlas and running the pre-change script prints OK. The walk no longer swallows read errors, the roots are asserted before the scan, and main() exits 1 with the missing root named. Prose wrapping walked through the line scan. A claim split across two lines matched nothing, and all nine controls used one-line fixtures, so the suite stayed green over the hole. The corpus wraps — 2,986 mid-sentence breaks across 40 of 53 scanned files — so this was live, not latent. Wrapped lines are now rejoined and the window is a sentence. Deliberately not a paragraph: measured on the current corpus, a paragraph window reports 3 false positives and a two-line window 3, while the sentence window reports 0. scanLines is renamed scanClaims, since it no longer scans lines. The figure gate's hand-written vocabulary is derived from KEYING instead. STORAGE_SUBJECT is deliberately not unioned in and the reason is recorded at the site: its `row` and `cell` mean something else in a diagram label, and unioning it fires on workforce.html:2039, a dispatch figure committing to no storage coordinate. corpusFiles is module-private again; nothing outside main() used it. Six controls added, each proved red by mutation before its green was trusted. Behaviour changes are limited to the three above. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ULydJv3JEeMUgeZbSvm4Rt
|
Review round handled in Fixed — each with a control proved red by mutation before its green was trusted:
Declined, with reasons on their threads: the test's Also declined: renaming Two things worth flagging for a human reader rather than burying in a thread:
Not merging, no labels applied. Generated by Claude Code |


The problem
scripts/check-isolation-coordinate.mjshas been onmainsince FIX-1420. It is 168 carefully documented lines, it exits 0, and nothing runs it — notpackage.json, not any workflow, not a skill. Verified by grep across all five.An unrun guard is indistinguishable from one nobody wrote. Three instruments were built across cycles 12 and 13 and none was retained: this one wired to nothing, and two throwaway checkers on #1886 never committed at all.
What this does
Wires it as a step in the
Process guardsjob, besidevalidate-model-strings.mjs, whose rationale it shares verbatim — a doc teaching a removed rule is as wrong onmainas on a PR.Makes it testable, and tests it. The two scanners are exported pure functions over text, the corpus walk is private behind
corpusFiles(), andmain()runs only when the file is invoked directly. Fifteen negative controls land atpackages/core/test/isolation-coordinate-check.test.ts, matching the sibling*-check.test.tspattern.The controls are aimed at the defect that produced the guard. FIX-1420's first scan was a phrase list generalised from the stale lines it had already found — it could only confirm its own list, and it reported green while two sites survived. So one control asserts the guard fires on a wording it was never shown; three assert silence on the current rule, on a route lookup, and on isolation used as a topic. Three more cover the
aria-labeldrift the token scan cannot see at all.Then review found three more ways the guard could report a false green, and closing them changed the scanners' behaviour in three specific places. Each is deliberate, and nothing else about the scan moved:
docs/atlasand running the pre-change script printsOK. The walk no longer swallows read errors, the roots are asserted before the scan, andmain()exits 1 naming the missing root.scanLinesis renamedscanClaims, since it no longer scans lines.KEYINGinstead of restated, so the two scanners can't drift apart the way they did before.What's asked of you
Nothing blocking. Two things on the record:
Two reviewers disagreed on where this belongs, and the disagreement is resolved here so it isn't re-litigated. Cursor recommended the
typecheckrail. Cursor was wrong on the facts: roottypecheckruns threevalidate-*.mjsscripts plus a differently-named goals guard, and all three are source-shape validators. Codex had it right — the five docs-corpus guards live in theProcess guardsjob, which is where a documentation-claim check belongs. The half of Cursor's advice that survives is the one that mattered most: the negative control.The window widened from a line to a sentence, and a wider window is how a guard starts crying wolf. The choice was measured rather than argued — see the table below. Two style suggestions were declined with reasons on their threads (the test's
describenarrative, and the CI step comment), as was renamingcheck-*tovalidate-*.Parts worth reviewing closely
main()is still guarded by theimport.meta.url === pathToFileURL(process.argv[1]).hrefcheck the siblings use, so importing the module does not run the scan.Verification
Wired to nothing, before: grep over
package.json,.github/,scripts/,docs/,AGENTS.md,CLAUDE.md,.agents/— no hits outside the script itself.Green on this tree:
node scripts/check-isolation-coordinate.mjs→ exit 0, ~70ms over 53 files.Import does not run the scan: importing the module prints only
missingRoots, scanClaims, scanFigureDrift.Suite: 15 controls pass.
pnpm --filter @flow-state-dev/core test→ 130 files, 2070 tests pass.pnpm typecheck→ 71 tasks pass.Still fires, unwrapped: a planted
namespaces its rows by flow kindline indocs/architecture/overview.md→ exit 1, naming file and line. Tree restored.Fires on a wrapped claim, which it previously could not. Planting a claim split the way prose actually wraps —
— the pre-change script prints
OKand exits 0; this one exits 1 namingoverview.md:210.A missing root is now loud. With
docs/atlasrenamed, the pre-change script printsOKand exits 0. This one exits 1:cannot scan the corpus — configured docs root(s) not found: docs/atlas.The corpus wraps, measured not assumed: 2,986 mid-sentence line breaks across 40 of the 53 scanned files (1,013 multi-line prose paragraphs). The blind spot was live, not theoretical.
Why a sentence and not a paragraph. Candidate windows run against the current correct corpus, where every hit is by definition a false positive:
Every control proved red by mutation before its green was trusted:
KINDback to a phrase list (the FIX-1420 defect)COORDINATEcomparison in the drift scanscanClaimsto the physical-line windowmissingRootsto return[]docs/atlason diskEvery row was re-run against the code that ships; the first was 3 before the window changed, which is the kind of stale number this block exists to prevent. The
scanClaimsrow earned its keep twice over: it caught a bad fixture of mine, where the wrap split at a point that left all three concepts on one line, so the control passed under the very mutation it was meant to detect. The fixture now splits where wrapping actually hides a claim.Rationale for why a per-claim guard should be retained at all is in #1904's cycle-13 entry rather than restated here. The broader policy question — deliverable or scratch file — is filed separately and is not decided by this PR.
No changeset — CI config and a test, no published-package consumer (BP-022).
🤖 Generated with Claude Code
https://claude.ai/code/session_01ULydJv3JEeMUgeZbSvm4Rt