Repository navigation
fix(examples): example 33 must teach the head budget build_head implements - #845
Merged
NandhaKishorM merged 2 commits intoOct 2, 2026
Merged
Conversation
The page that explains the option head taught three things laya/common.py does not do: * the cut waits for the options to overflow `head_max_len`. It waits for the spare to drop under 16, which happens while the options still fit: four options of 48 fill a 192 head exactly and are already cut to `max(4, (192-16)//4) = 44`. * `16` is a minimum reserved for the instruction prefix. The instruction is held to `head_ids[: max(8, opt_budget)]`, and the 77-option question this page runs leaves it with 8 tokens. `16` is the threshold the spare is judged against and the reserve the division keeps back. * the option block lives in `head_max_len`. Past roughly head_max_len/4 options every one sits at the 4-token floor and the head grows past the cap: measured 319 tokens against the English 192 cap, so the state has room for 192 of max_len=512, not the 320 the cap implies. `budget_report` now prints what it measures instead of leaving the reader to infer it: the head built and how far past `head_max_len` it went, the room left for the state, and how many options still have a token span of their own (67 of 77 at the English default -- 10 labels share a span).
…resholds and floors Two behavioural witnesses with the suite's own fake tokenizer -- the cut fires at 0 and at 15 tokens of spare and not at 16, and a 77-option question keeps 8 instruction tokens under a head that runs past the cap -- then four prose rules over the example's AST: no conditional clause may make overflow the trigger, no sentence may call 16 the instruction's reserve, no sentence may put the option block inside `head_max_len`, and every number in the page's two formulas must equal the literal `build_head` carries. The literals are read out of the function with `inspect` + `ast`, so a budget change in the library fails the page instead of agreeing with a stale copy of it. Each rule carries a non-vacuity witness: the suite asserts the page states formulas and conditional clauses at all, so a page rewritten into prose nothing can check does not pass.
4 tasks done
Owner
|
Merged for 0.3.24. Verified on the integrated branch rather than on the PR alone: the full shared test list plus the pytest lane green, Thank you. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
examples/33_high_cardinality_choice.pyis the page that teaches the option head budget. Its bannerstated three things
laya/common.py:build_headdoes not do; this corrects the prose, has the pageprint what it measures, and holds the page to the code in
tests/test_state_budget.py.The three claims, and what the code does instead:
if opt_budget < 16, i.e. it fires on the spare tokens, while the options still fit. With thesuite's one-id-per-word fake tokenizer, four options of 48 tokens fill a 192 head exactly (nothing
overflows) and are still cut to
max(4, (192-16)//4) = 44. The page's ownbudget_reporthad thisright all along (
if head_max_len - sum(full) < 16), so the banner contradicted the code beneath it.16is the reserved minimum for the instruction prefix)" — the instruction is held tohead_ids[: max(8, opt_budget)]. For the 77-option question this page runs, the instruction keeps8 tokens, from an instruction that is 12 tokens long.
16is the threshold the spare isjudged against, and the reserve the division keeps back before splitting between
Noptions.head_max_len" — measured on the English checkpoint's own tokenizer:77 options at the 4-token floor build a 319-token head against the 192 cap, so the state has
room for 192 tokens of
max_len=512, not the 320max_len - head_max_lenimplies, and only67 of 77 options keep a token span of their own.
budget_reportnow prints the head it built, how far pasthead_max_lenit went, the room left forthe state, and the distinct-option count — so the reader sees the cap being passed instead of being
told the block lives inside it.
Why
The suite
tests/test_state_budget.pyalready pins this relationship forREADME.md("past roughlyhead_max_len / 4options the head grows beyond the cap"), so the fix landed in the README and thetest and left behind the one example whose entire subject is this budget. A reader following the page
learns the wrong trigger, the wrong floor, and a size for their state that the renderer then discards.
The collapsed-span consequence is the one #543 measured (43 of 58 labels kept their own span there);
this page's default case is the same mechanism at 67 of 77, and now says so out loud.
How it was verified
Before and after, same command, in a worktree at
4aa6761(= currentmain):The four failures are the prose rules; the two behavioural tests pass on
maintoo, because thelibrary is unchanged — they are what the prose rules are measured against. The skip is the
checkpoint-tokenizer arm of
test_readme_quotes_the_measured_figures, which needs~/laya_modelsand is why every test added here is weight-free.
The gate is five rules plus two behavioural witnesses:
test_the_option_cut_waits_on_spare_tokens_not_on_overflow— spare 16 → not cut; spare 15 → cut;spare 0 with 192 of 192 used → cut. Built with the suite's existing
_FakeTok(one id per word),so it costs no weights and runs in CI.
test_the_instruction_floor_is_eight_not_the_reserved_sixteen— 77 options, an instruction longerthan 16 tokens: the retained prefix is exactly 8, and the head runs past the cap.
overflowthe trigger, nosentence may call 16 the instruction's reserve or minimum, no sentence may put the option block
inside
head_max_len(and one must say the cap is not hard). Sentences are split within eachstring constant, never over a join of them, so a true sentence cannot vouch for its neighbour.
test_page_arithmetic_is_build_head_arithmetic— the page'smax(4, (head_max_len - 16) // N)andhead_ids[: max(8, opt_budget)]are parsed and compared against the literals read out ofbuild_headwithinspect+ast. Nothing is typed in here: change a budget constant in thelibrary and the page fails instead of agreeing with a stale copy of itself.
Every rule carries a non-vacuity witness (the suite asserts the page states formulas and conditional
clauses at all), and all of it was mutation-checked — 15 mutants plus a control, each expected to
fail with named tests, run with
-B, file restored and byte-compared after every arm:The
L1–L4arms mutatelaya/common.pylocally and restore it — that file is not in this diff(open PRs #783 and #687 both touch it), so the fix is prose and prints only.
M1replaces the wholepage with
git show 4aa6761:examples/33_high_cardinality_choice.py, pinned to the SHA rather thanHEAD~nso it cannot silently stop being the historical text.The example runs end to end on MPS (torch 2.14.0), English checkpoint from the Hub bundle, and its
77-option default case now prints:
Those figures are the page's own measurement, taken before the edit as well: the
MISSonatm_supportand the confidence of 1.000 are unchanged, because only the prose and the reporting moved.
Checklist
mainFixes #
No issue: found by reading the example's banner against
build_head, then checking the page againstthe budget
tests/test_state_budget.pyalready pins for the README.