Repository navigation
chore(lessons): cycle 13 — the sweep names a surface it cannot reach - #1904
Conversation
…eep has a hole Third collection on FIX-1351, covering the nine artifacts produced after cycle 12's. First cycle where the previous fix was carried by every commit it is scored against: #1857 (the BP-003 sharpening) is an ancestor of 22 of 22 commits on the seven carried artifacts, and #1848/#1849 are excluded because they carry none of it. The scoring result is that it did not bind. BP-003's third shape — a check whose stated scope is wider than its real one — was eighteen hours old, on the branch, and is the exact defect three bots filed against #1885, whose own subject is a detector. Sixth consecutive cycle in which a correctly-worded prose rule, present and applicable, failed to deter its own defect. That is why the fix here is not a seventh sentence in BP-003. Dominant class is stale-restatement, 12 of 38 non-nit findings, all of them a corrected claim and its stale copy inside one file. Most are reachable by 10.6's existing word sweep and are a compliance gap. The residue is a family 10.6 names and then abandons: surfaces carrying a claim as a label rather than a sentence — a status tag, an aria-label, a caption, a figure's visible text — where there is no sentence to grep. Four instances, each missed by a sweep that demonstrably ran; two escaped review entirely. Fix: two sentences inside 10.6's existing sweep paragraph, saying to enumerate such a surface instead of its words. A mechanical figure-vs-prose check was the hypothesis and is rejected in the entry with its reasons — prose agreement is not decidable, and the strict decidable version reaches one of three instances. No changeset — internal process artifacts, no published-package consumer (BP-022). Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ULydJv3JEeMUgeZbSvm4Rt
|
The latest updates on your projects. Learn more about Vercel for GitHub. 3 Skipped Deployments
|
Codex Review SummaryThis comment shows the latest Codex review activity on this pull request.
ℹ️ About Codex in GitHubYour team has set up Codex to review pull requests in this repo. Reviews are triggered when you
Codex reacts with 👀 while any review is running, comments if it has suggestions, and reacts with 👍 once all reviews finish with no findings. |
There was a problem hiding this comment.
Code Snob — outcome 3: already tight. No POC.
Read .agents/skills/issue-implement/SKILL.md (the 10.6 sweep paragraph), docs/internal/cycle-ledger.md (cycle 13 against cycle 12), and .agents/skills/distill-lessons/SKILL.md (what an entry is required to carry).
The landing is two sentences in an existing step. Those sentences are the method (enumerate the surface's instances) and the pass/fail (empty grep is not coverage). Folding them back into the older "states the claim more briefly / no string sweep sees either" clause keeps the same words and loses the instruction the #1886 agent did not have. Dropping the empty-grep sentence treats the #1886 failure mode as restatement; it isn't.
The +255 ledger is the house shape, not a helper: method, table, carried/not-carried split, population fence, residue evidence, recorded-not-fixed, rejected alternatives, claims to test. Cycle 12 is ~233 lines of the same sections. Thinning it to a cycle-1 stub would drop measurement distill-lessons asks for, not shrink the contract.
Not re-litigated: no seventh BP-003 sentence, no figure-vs-prose check, no wiring scripts/check-isolation-coordinate.mjs.
Sent by Cursor Automation: Code Snob
There was a problem hiding this comment.
Simplify review (/simplify)
Cycle 13 found that §10.6’s word sweep can report green while paraphrasing surfaces (figures, tagged tables, status columns) still teach the old claim. This PR’s landed fix is two sentences in issue-implement §10.6 plus the ledger entry — not another BP-003 sharpen, not a generic figure linter, not a new step. That restraint matches the evidence in the entry (and matches what actually worked on #1886: enumerate 22 tags; what works on main: concept co-occurrence in check-isolation-coordinate.mjs).
Did the approach go too far? On the intervention, no — this is the right altitude (distill ladder rung 4: one checklist line in a skill). On artifact size, the Cycle 13 block is long relative to the diff, but that’s the ledger’s job; the risk is repetition (thesis + blockquote duplicating §10.6), not over-engineering the fix.
Simpler alternatives considered (parallel read-only reviewers):
| Alternative | Verdict |
|---|---|
| Seventh BP-003 / tenet sentence | Correctly rejected in the entry |
| Mechanical figure co-edit / “prose near figure” guard | Correctly rejected — one of three instances, FP generator |
Wire scripts/check-isolation-coordinate.mjs into pnpm typecheck / CI in this PR |
Reasonable follow-up, not required to land the skill line; ledger already records “follow-up, not folded here.” Would close claim #3 faster than prose alone |
Add §10.6 bullet: “run committed per-claim guards when they exist” + link spec-figures.md / issue-spec Step 5 |
Would reuse existing patterns and shorten self-contained teaching; optional polish |
Split epic evidence into docs/internal/epic-wraps/ |
Navigation hygiene for future cycles; optional |
| Third-party packages for doc scan | No win — repo validate-*.mjs scripts stay zero-dep; internal wiring beats new deps |
Runtime / performance: No product code in the diff. The only cost angle is agent time to enumerate large tagged tables — intentional trade for coverage.
Recommendation before merge (small, same PR if you want): Align §10.6’s “two sweeps” / “report which of the two sweeps” wording with the new enumerate paraphrasing surfaces obligation (see inline). Without that, implementers may grep + brief-surface sweep and treat enumeration as optional.
Skipped (needs separate issue / epic): Linear mirror via epic-wake, Greptile concurrence process, wiring the orphan guard — all fairly called out as “recorded, not fixed.”
Bottom line: The cycle chose the smallest upstream fix that names the method the data already proved. Approve.
Sent by Cursor Automation: Review For Best Approach
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5189c985ba
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Three reviewers found three structurally distinct joints at which the enumeration rule could not bind. None overlaps, and each is folded: - hook (Cursor): "the two sweeps" was satisfiable by grepping and glancing. The sweeps are now named — word sweep, surface sweep — and the report must say whether enumeration ran and over what. - gate (Codex): the paragraph opens "before closing a round that changed behavior", and #1886 and #1898 changed no behavior. The rule for a stale aria-label sat behind a gate excluding every case it came from. Broadened to any corrected claim or documentation change. - unit (Greptile): enumerating a surface still resolves each label's opening subject, which is exactly how #1886's L1499 survived a 22-tag sweep. Every claim under a label must be resolved, and the label split where they differ. That is the entry's own thesis landing on the entry, observed rather than inferred, and it replaces the six-cycle count as cycle 13's result. The recommendation does not move — one checklist line is still the right altitude — but its claim is weaker and the entry says so: what landed is the line plus three rounds of adversarial review, and only the first half is reproducible. The measured failure mode is reachability, not wording, so claim 1 measures that first and claim 6 carries the reachability check as a candidate rather than minting it on one sitting's evidence. Also corrected, all found in review and all the same class: - the duplicate #1718 row (it already has one at cycle 10, and a row recording zero delta double-counts the epic); the observation moves to prose - "pnpm typecheck runs five sibling guards" — it runs three; the five docs-corpus guards are in CI, which is where the orphan guard belongs - "all three live inside one <svg>" two clauses before the same sentence says <figure> correctly; zero figcaptions sit inside an svg anywhere in docs/atlas - the ledger blockquote, relabelled as-landed and resynced to the folded text Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ULydJv3JEeMUgeZbSvm4Rt
Judged against the skill's own Step-3 gate and kept, as a candidate only — not landed. This cycle's fix is reviewed and done; a second one after three rounds of review is how a clean result becomes an unreviewed one. The objection that had to be answered is that it is the rejected figure check wearing a hat, since both ask whether a rule "covers" something. It isn't, and the distinction is the one the cycle turns on: the figure check had to decide semantic agreement between two statements over an unbounded corpus, which is not decidable, while this walks a bounded already-enumerated instance set and asks a concrete question of each. All three of this round's findings answer cleanly, and the L1499 one is measured rather than predicted — the 22-tag sweep ran and it still came back wrong. Recorded with the failure mode that would turn it into ceremony, and with what cycle 14 should score it on: whether it catches a reachability defect before review does, on a rule that would otherwise have shipped. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ULydJv3JEeMUgeZbSvm4Rt
Self-reported by the FIX-1377 implementation after cycle 13's collection closed. Purely additive: no row, no class total, no reading-label rate, and nothing above re-scored. Cursor has already passed the entry and its numbers should not churn. The fabricated citation is the one worth the space, and it is a class the entry does not have. Every stale-restatement instance here is a claim that was true and became false; #1888 was never true. Verified rather than relayed: #1888 is a real PR — spec(FIX-1440), a different issue — so it was referentially valid and semantically wrong, which is the one shape an existence check cannot see. Both checkers this entry praises pass it, as does the figure agreement and the surface sweep it just landed. It did not reach main, and the branch now carries #1911 at all three sites. Proposed as `invented-reference` rather than minted, following the precedent set for vacuous-assertion, and carried into claims as something to collect deliberately instead of waiting for another self-report. The control script that reverted uncommitted work is recorded as a hazard in the plant-and-restore pattern, not against it — that pattern produced most of this cycle's good evidence, including #1906's mutation runs. Commit before you plant; `git checkout --` cannot tell a regression from your work. The counter-evidence is stated as weak and reads that way. Two authors applying the discipline unprompted on their own work is the first author-side catch in thirteen cycles and cuts against the entry's own pessimistic reading, but n=2, same day, same briefing. Scored in cycle 14 on authors with no such briefing. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ULydJv3JEeMUgeZbSvm4Rt
…tion Claim 7 as written would have had the next cycle drop the proposal for want of a second instance. One arrived within the hour and it is a different subspecies, so it is recorded before the evidence is lost. The FIX-1416 agent attributed reasoning to the owner that the owner's quoted words do not carry, caught it itself, and corrected in cac995b — whose message is the clearest statement of the class anyone has written: the gloss was the agent's, presented as the owner's. That renames the proposal. It is not about citations: a wrong PR number and invented reasoning are two forms of one thing, content attributed to a source that never carried it. #1888 resolved to a real PR about a different issue; this resolved to a real person who did not say it. Neither is malformed, neither dangles, and every instrument in the section's table passes both. The attributed-reasoning form is the more dangerous and the reason is structural: a wrong citation is falsifiable by any reader who follows it, invented reasoning only by the person quoted. In a process where the owner rules in a sentence of chat and agents carry that ruling into DECISIONS.md, the invented half outlives the conversation that could have refuted it. Both instances were self-caught, neither by a reviewer or an instrument, so the survival condition changes: claim 7 no longer needs a second instance, it needs to know whether the class appears when nobody is looking for it. Both authors had just spent a session on adjacent classes — the same favourable-condition problem as the counter-evidence above it. Counts untouched: nine rows, 38 non-nit findings, same fence. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ULydJv3JEeMUgeZbSvm4Rt
7 sites false, 3 reported by two review automations, 4 found only by running 10.6's surface sweep over teamInstructions in packages/workforce. All four sweep-only sites verified present on the branch. The reported corpus total of 103 could not be reproduced — git grep finds 43 across 11 tracked files in that package, most likely because the total counted built dist/ output git does not track. Fenced rather than repeated; the load-bearing figure is 7/3/4 and the corpus size is context. Recording an unverified count inside the section about invented attribution would have been its own joke. Limits written in, because this is favourable-condition evidence for the third time: n=1 claim, n=1 package, an author told to enumerate by a coordinator who had spent the session on the class, and reviewers who were not trying to enumerate in the first place — "enumeration beats review at enumeration" is close to circular and the entry says so. What it does establish is the assumption the 10.6 line was landed on: the residue is real and larger than the review surface. The four missed sites were in files the diff touched. Also records the author's own note, which is the sharpest line written about this class today and cuts against its author: same PR, same sitting, they caught four vacuous assertions against an empty reader before review saw them, then shipped seven stale prose claims. The discipline is surface-specific, not general. Claim 8 gives the measurement a shape to repeat without the briefing. Claim 9 records, without acting on it, that two agents independently refused to rewrite a shipped record — a changelog entry and a ledger row — on the grounds that it was true when written. Counts untouched: nine rows, additive only. Co-Authored-By: Claude <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ULydJv3JEeMUgeZbSvm4Rt


The problem
We keep correcting a claim in one place and leaving its copy stale a few hundred lines down in the same file. Twelve findings this cycle, across four documentation PRs. One of them shipped a diagram whose
aria-labeltold a screen-reader user the opposite of what the prose said.We already have the rule for this.
issue-implement10.6 tells you to sweep every surface restating a claim, diagrams included. On PR #1886 the agent ran that sweep, published its grep terms, wrote down that the sweep cannot see a paraphrase — and shipped a wrong status tag anyway. The rule names the hole and attaches no method.What this does
Two things, both small.
A cycle-13 entry in the ledger. Nine artifacts from FIX-1351 after cycle 12's collection. The one result worth the read: cycle 12's fix was carried by 22 of 22 commits on the seven artifacts it is scored against, and it did not bind — BP-003's "stated scope wider than its real one" clause was eighteen hours old, on the branch, and is the exact defect three bots filed against #1885. That is the sixth consecutive cycle where a correctly-worded prose rule, present and applicable, failed to deter its own defect.
Two sentences added to
issue-implement10.6, closing the hole that paragraph opens:That is the method that already worked twice in this cycle's own data: #1886's eleven corrections came from enumerating 22 tags rather than grepping their words, and
scripts/check-isolation-coordinate.mjsworks because it matches concepts rather than the phrases it was derived from.No BP change and no tenet change. Sharpening BP-003 again was the obvious move and the entry rejects it with the measurement above.
What's asked of you
Nothing blocking — this is a draft and the fix is a skill line, not a standard. Two things worth your eye:
<text>/aria-label/<figcaption>) fires on 1 of the 3 figure instances, because in the other two the corrected prose sits outside the<figure>. If you think a check can express this, that changes the fix.scripts/check-isolation-coordinate.mjsis onmainand wired to nothing — notpackage.json, not CI, not a skill.pnpm typecheckalready runs five sibling guards. Recorded as a follow-up, not folded in here, because "should a per-claim guard be a deliverable or a scratch file" is a policy call rather than a lessons finding.Parts worth reviewing closely
Scoring cycle 12's fix). Asked per commit by ancestry, never by timestamp and never bybase.sha. spec(FIX-1426): DevForce Lab: first build slice — a filed board row wakes a WORKER.md seat into a supervised coding run #1848 and Fold pentest probe kind onto workforce/flows/workers (FIX-1427) #1849 are excluded from the fix sample entirely rather than scored as zeros.The population changed.vacuous-assertionfell from 15/37 to 6/34, and the entry declines to call that a win: cycle 12 was six code PRs, this cycle is four docs PRs plus a lab and a detector, and the two label sets were collected by different methods. Cycle 12's claim-to-test Kitchen-sink: Add inline editable artifacts andsaveArtifactaction #1 is answered "not yet measurable".The shape worth reading— this is the evidence the fix rests on. Two of the four escaped review entirely.Verification
4d7a5749frun for all nine artifacts; the 22-of-22 and the 0-of-3 (spec(FIX-1426): DevForce Lab: first build slice — a filed board row wakes a WORKER.md seat into a supervised coding run #1848) / 0-of-2 (Fold pentest probe kind onto workforce/flows/workers (FIX-1427) #1849) figures are from that run, not from branch heads.package.json,.github/,scripts/,docs/,AGENTS.md,CLAUDE.mdand.agents/. The guard was then run (exit 0) and proved able to fail — a plantednamespaces its rows by flow kindline indocs/architecture/overview.mdmade it exit 1 naming the line; the tree was restored.No changeset — internal process artifacts, no published-package consumer (BP-022).
🤖 Generated with Claude Code
https://claude.ai/code/session_01ULydJv3JEeMUgeZbSvm4Rt
Appears safe to merge, with non-blocking improvements to the sweep’s mixed-claim guidance and the ledger’s HTML example.
Summary
Adds a cycle-13 lessons entry and extends the implementation completion gate with explicit enumeration of paraphrasing surfaces.
Reviews (1) · Last reviewed commit: "chore(lessons): cycle 13 — stale-restate..."