Skip to content

fix(examples): example 33 must teach the head budget build_head implements - #845

Merged
NandhaKishorM merged 2 commits into
NandhaKishorM:mainfrom
aashish254:example33-head-budget
Oct 2, 2026
Merged

NandhaKishorM merged 2 commits into
NandhaKishorM:mainfrom
aashish254:example33-head-budget

Conversation

@aashish254

Copy link
Copy Markdown
Contributor

What

examples/33_high_cardinality_choice.py is the page that teaches the option head budget. Its banner
stated three things laya/common.py:build_head does not do; this corrects the prose, has the page
print what it measures, and holds the page to the code in tests/test_state_budget.py.

The three claims, and what the code does instead:

  1. "When the full options overflow the head, the code grants each one …" — the cut is
    if opt_budget < 16, i.e. it fires on the spare tokens, while the options still fit. With the
    suite's one-id-per-word fake tokenizer, four options of 48 tokens fill a 192 head exactly (nothing
    overflows) and are still cut to max(4, (192-16)//4) = 44. The page's own budget_report had this
    right all along (if head_max_len - sum(full) < 16), so the banner contradicted the code beneath it.
  2. "(16 is the reserved minimum for the instruction prefix)" — the instruction is held to
    head_ids[: max(8, opt_budget)]. For the 77-option question this page runs, the instruction keeps
    8 tokens, from an instruction that is 12 tokens long. 16 is the threshold the spare is
    judged against, and the reserve the division keeps back before splitting between N options.
  3. "The option block lives in head_max_len" — measured on the English checkpoint's own tokenizer:
    77 options at the 4-token floor build a 319-token head against the 192 cap, so the state has
    room for 192 tokens of max_len=512, not the 320 max_len - head_max_len implies, and only
    67 of 77 options keep a token span of their own.

budget_report now prints the head it built, how far past head_max_len it went, the room left for
the state, and the distinct-option count — so the reader sees the cap being passed instead of being
told the block lives inside it.

Why

The suite tests/test_state_budget.py already pins this relationship for README.md ("past roughly
head_max_len / 4 options the head grows beyond the cap"), so the fix landed in the README and the
test and left behind the one example whose entire subject is this budget. A reader following the page
learns the wrong trigger, the wrong floor, and a size for their state that the renderer then discards.

The collapsed-span consequence is the one #543 measured (43 of 58 labels kept their own span there);
this page's default case is the same mechanism at 67 of 77, and now says so out loud.

How it was verified

python -B tests/test_state_budget.py
python examples/33_high_cardinality_choice.py
ruff check laya/ --select=E9,F63,F7,F82,F401,F811 --line-length=120
python -m compileall -q laya/ tests/

Before and after, same command, in a worktree at 4aa6761 (= current main):

main, untouched suite          7 passed, 0 failed, 1 skipped
main's page + the new gate    18 passed, 4 failed, 1 skipped
  FAIL test_page_teaches_the_spare_as_the_trigger_not_overflow: the page makes the options
       overflowing the head the trigger (['When the full options overflow the head'])
  FAIL test_page_does_not_reserve_sixteen_for_the_instruction: the page no longer shows the
       instruction clip (`head_ids[: max(8, opt_budget)]`)
  FAIL test_page_does_not_put_the_option_block_inside_the_cap: the page puts the option block
       inside `head_max_len` (['... The option block lives in `head_max_len`; ...'])
  FAIL test_page_arithmetic_is_build_head_arithmetic: the page no longer states the instruction
       clip as `head_ids[: max(8, opt_budget)]`
this branch                   26 passed, 0 failed, 1 skipped

The four failures are the prose rules; the two behavioural tests pass on main too, because the
library is unchanged — they are what the prose rules are measured against. The skip is the
checkpoint-tokenizer arm of test_readme_quotes_the_measured_figures, which needs ~/laya_models
and is why every test added here is weight-free.

The gate is five rules plus two behavioural witnesses:

  • test_the_option_cut_waits_on_spare_tokens_not_on_overflow — spare 16 → not cut; spare 15 → cut;
    spare 0 with 192 of 192 used → cut. Built with the suite's existing _FakeTok (one id per word),
    so it costs no weights and runs in CI.
  • test_the_instruction_floor_is_eight_not_the_reserved_sixteen — 77 options, an instruction longer
    than 16 tokens: the retained prefix is exactly 8, and the head runs past the cap.
  • three prose rules over the example's AST: no conditional clause may make overflow the trigger, no
    sentence may call 16 the instruction's reserve or minimum, no sentence may put the option block
    inside head_max_len (and one must say the cap is not hard). Sentences are split within each
    string constant
    , never over a join of them, so a true sentence cannot vouch for its neighbour.
  • test_page_arithmetic_is_build_head_arithmetic — the page's max(4, (head_max_len - 16) // N) and
    head_ids[: max(8, opt_budget)] are parsed and compared against the literals read out of
    build_head with inspect + ast. Nothing is typed in here: change a budget constant in the
    library and the page fails instead of agreeing with a stale copy of itself.

Every rule carries a non-vacuity witness (the suite asserts the page states formulas and conditional
clauses at all), and all of it was mutation-checked — 15 mutants plus a control, each expected to
fail with named tests, run with -B, file restored and byte-compared after every arm:

killed    control: pristine worktree         control green (26 passed, 1 skipped)
killed    M1 historical page (whole file)    ['test_page_arithmetic_is_build_head_arithmetic', 'test_page_does_not_put_the_option_block_inside_the_cap', 'test_page_does_not_reserve_sixteen_for_the_instruction', 'test_page_teaches_the_spare_as_the_trigger_not_overflow']
killed    M2 page: block lives in the cap    ['test_page_does_not_put_the_option_block_inside_the_cap']
killed    M3 page: 16 reserved for instruction ['test_page_does_not_reserve_sixteen_for_the_instruction']
killed    M4 page: overflow is the trigger   ['test_page_teaches_the_spare_as_the_trigger_not_overflow']
killed    M5 page: reserve 16 -> 24          ['reserve on the page']
killed    M6 page: per floor 4 -> 3          ['per-option floor on the page']
killed    M7 page: instruction floor 8 -> 16 ['instruction floor on the page']
killed    M8 page: no sentence says the cap can be passed ['test_page_does_not_put_the_option_block_inside_the_cap']
killed    M9 page: condition stops naming 16 ['test_page_teaches_the_spare_as_the_trigger_not_overflow']
killed    M10 page: instruction clip dropped ['test_page_arithmetic_is_build_head_arithmetic', 'test_page_does_not_reserve_sixteen_for_the_instruction']
killed    M11 page: worked instance result wrong ['worked instance max(4, (192-16)//77)']
killed    L1 code: trigger 16 -> 0           ['test_page_arithmetic_is_build_head_arithmetic', 'test_the_option_cut_waits_on_spare_tokens_not_on_overflow']
killed    L2 code: reserve 16 -> 24          ['cut size for 4 options at head_max_len=192', 'reserve on the page', 'test_page_arithmetic_is_build_head_arithmetic']
killed    L3 code: per floor 4 -> 3          ['77 options are cut to the per-option floor', 'per-option floor on the page']
killed    L4 code: instruction floor 8 -> 16 ['instruction floor on the page', 'instruction tokens retained', 'the retained prefix is below the 16 the page must not call its reserve']

restored: 26 passed, 0 failed, 1 skipped
16 arms, 0 not as expected

The L1–L4 arms mutate laya/common.py locally and restore it — that file is not in this diff
(open PRs #783 and #687 both touch it), so the fix is prose and prints only. M1 replaces the whole
page with git show 4aa6761:examples/33_high_cardinality_choice.py, pinned to the SHA rather than
HEAD~n so it cannot silently stop being the historical text.

The example runs end to end on MPS (torch 2.14.0), English checkpoint from the Hub bundle, and its
77-option default case now prints:

   sum(options)=1122 leaves -930 tokens of head spare, under the 16 the cut waits for, so per = max(4, (192-16)//77) = 4 tokens
   measured: 77/77 options present, 4..4 tokens each (avg 4.0), instruction prefix 8 tokens, sequence 342 of max_len=512
   the head built is 319 tokens, 127 past head_max_len=192, so the state has room for 192 tokens and 67 of 77 options still have a span of their own
   77 options, default      -> atm_support            p=1.000 conf=1.000  MISS   next: all others 0.000

Those figures are the page's own measurement, taken before the edit as well: the MISS on atm_support
and the confidence of 1.000 are unchanged, because only the prose and the reporting moved.

Checklist

  • Focused on one change (split unrelated work into another PR)
  • Rebased on the latest main
  • Tests pass locally
  • Docs or examples updated when the public API changed

Fixes #

No issue: found by reading the example's banner against build_head, then checking the page against
the budget tests/test_state_budget.py already pins for the README.

The page that explains the option head taught three things laya/common.py does not do:

* the cut waits for the options to overflow `head_max_len`. It waits for the spare to drop
  under 16, which happens while the options still fit: four options of 48 fill a 192 head
  exactly and are already cut to `max(4, (192-16)//4) = 44`.
* `16` is a minimum reserved for the instruction prefix. The instruction is held to
  `head_ids[: max(8, opt_budget)]`, and the 77-option question this page runs leaves it with
  8 tokens. `16` is the threshold the spare is judged against and the reserve the division
  keeps back.
* the option block lives in `head_max_len`. Past roughly head_max_len/4 options every one
  sits at the 4-token floor and the head grows past the cap: measured 319 tokens against the
  English 192 cap, so the state has room for 192 of max_len=512, not the 320 the cap implies.

`budget_report` now prints what it measures instead of leaving the reader to infer it: the head
built and how far past `head_max_len` it went, the room left for the state, and how many options
still have a token span of their own (67 of 77 at the English default -- 10 labels share a span).
…resholds and floors

Two behavioural witnesses with the suite's own fake tokenizer -- the cut fires at 0 and at 15
tokens of spare and not at 16, and a 77-option question keeps 8 instruction tokens under a head
that runs past the cap -- then four prose rules over the example's AST: no conditional clause may
make overflow the trigger, no sentence may call 16 the instruction's reserve, no sentence may put
the option block inside `head_max_len`, and every number in the page's two formulas must equal the
literal `build_head` carries. The literals are read out of the function with `inspect` + `ast`, so
a budget change in the library fails the page instead of agreeing with a stale copy of it.

Each rule carries a non-vacuity witness: the suite asserts the page states formulas and
conditional clauses at all, so a page rewritten into prose nothing can check does not pass.
@NandhaKishorM
NandhaKishorM merged commit 240852f into NandhaKishorM:main Oct 2, 2026
23 checks passed
@NandhaKishorM

Copy link
Copy Markdown
Owner

Merged for 0.3.24.

Verified on the integrated branch rather than on the PR alone: the full shared test list plus the pytest lane green, compileall clean, both TypeScript packages green (laya-ts 597 tests, the SDK 26), and a decision diff against 0.3.23 on the real english and multilingual checkpoints (74 answers over choice, score and noul; short and long states; budgeted calls and predict_long) showing zero decision flips and a max probability delta of 0.000000.

Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants