Repository navigation
docs(examples): the truncation pages must teach the report and the clamp they deny - #841
Conversation
…does not exist The example was written to show that a state longer than the window is cut down to its head, and its closing paragraph said "Truncation is silent in both directions: nothing warns you, and the answer looks as certain as any other". `predict_batch` has published `truncated`, `state_tokens`, `state_tokens_dropped` and `truncated_questions` since NandhaKishorM#174, so the page taught readers to estimate from the length of what they sent instead of reading the four keys sitting next to the token count it already printed. It now prints them off the same call, and prints a second call whose state fits so the report reads `truncated`=False with an empty `truncated_questions`. The sentence about the two answers staying indistinguishable is kept -- that is what the example is about -- but it names `truncated` as the thing that says so.
The page that draws the whole `usage` block then described it as "the four keys after them" and `options` as "a seventh key". tests/test_structured_docs.py holds the drawn block to the dict the agents build, so a fifth report key turns those two counts into a false statement while the gate stays green -- the count is not in the block it checks. Naming the keys does the teaching either way; the ordinals only rot.
Example 15 was free to describe the report however it liked: nothing compared its
printed keys with the `usage = {...}` literal in laya/agent.py, which is how the wrong
"nothing warns you" sentence survived NandhaKishorM#174's fix in the page a reader comes to for that
exact answer.
The key set is now read out of that literal by AST -- not repeated here, so a key added
to the dict has to be taught and a renamed one fails the example still pointing at it --
and the example is held to it in both directions: every report key read off a live call,
no key read that the agents do not publish, each one named in backticks so it is
greppable. A sentence may still say the cut is invisible in the answers, but only if it
names the key that reports it.
Example 03's drawn shape and `options` are already gated by tests/test_structured_docs.py,
so this only adds what that gate cannot see.
…ly picks The banner said `build_sequence` has a `truncate_left=True` switch that `predict`/`system_one` does not expose, and that anything past the window is invisible with nothing warning. Both are wrong in opposite directions: `predict` derives `truncate_left = isinstance(state, list)` on both agents, so the tail is reachable without touching the budget, and the `usage` report has named the cut since NandhaKishorM#174. `state_room` documents the type rule already. Measured on laya-multilingual: one 1452-token state handed over as a string and as the identical list drops 465 tokens either way, and the only difference is which end survives -- the closing sentence reaches the model in the second call and not in the first, while `noul` says yes to both.
|
Short answer: this PR got a second example in it, so I retitled it. It started as example 15 telling the reader "nothing warns you" when the report has warned since I folded it in rather than opening a separate PR because both live in |
|
Merged for 0.3.24. Verified on the integrated branch rather than on the PR alone: the full shared test list plus the pytest lane green, Thank you. |
What
Two truncation-teaching examples deny things the code does, and
tests/test_truncation.pygrows thegate that makes that unfalsifiable later:
examples/15_truncation_basics.pyprinted the batch token total and then told the reader"nothing warns you". It now prints the four report keys off the same call, plus a second call whose
state fits, so a reader sees
truncated=False with an emptytruncated_questionstoo.examples/37_long_document.pysaidbuild_sequencehas atruncate_left=Trueswitch "butpredict/system_onedoes not expose it", and that the tail is "only recoverable by setting thebudget". Both agents derive the clamp from the state's type (
truncate_left = isinstance(state, list)), so the tail is reachable without touching the budget. It now measures that: the same1452-token state handed over as a string and as the identical list, printing which end reached the
model.
usagekey set out oflaya/agent.pyby AST (not repeated in the test) andholds both examples to it, plus the clamp rule out of both agents' source.
examples/03loses two hand-counted ordinals ("the four keys after them") that no gate can see.4 files, +198/−17:
tests/test_truncation.py(+112/−1),examples/37_long_document.py(+63/−8),examples/15_truncation_basics.py(+20/−5),examples/03_reading_the_result.py(+3/−3).Why
examples/README.mdsends a reader to 15 for "what happens when the state does not fitmax_len,and how to see it in
usage", and to 37 for the window arithmetic. Each page asserted the oppositeof what ships:
The first was false when it was written:
predict_batchhas publishedtruncated,state_tokens,state_tokens_droppedandtruncated_questionssince #174. The second is false in the otherdirection --
laya/agent.py:1023andlaya/onnx_agent.py:606pick the clamp side from the state'stype, because a chronological list serializes newest-last, and
state_room's own docstring alreadydocuments it ("the start is kept, or the end for a conversation turn list"). A reader who believes 37
thinks a long document's tail is unrecoverable except by re-configuring the window; the actual answer
is to hand the state over as a list.
Neither claim could fail anything, because nothing compared an example's prose with the code that
builds the dict and the clamp. That is the gap the gate closes, in both directions: a key added to
the dict has to be taught by both examples, and a key renamed or dropped fails the example still
pointing at it.
Measured on
laya-multilingual, one 1452-token state, one question, default window:Same characters, same
state_tokens, samestate_tokens_dropped-- the type alone chose which endthe model read. Both answers say yes, which is the second half of the lesson and why the example says
so: the report, not the score, is what tells you the evidence was cut.
How it was verified
tests/test_truncation.py->57 passed, 0 failed(37 on main). The new 20: 6 pinning example 15, 10pinning example 37, 2 on the clamp derivation in each agent, 2 on the
usagedict itself being oneliteral and bigger than the two token totals.
tests/test_structured_docs.py->113 checks passed, 0 failed, unchanged.ruffwith CI's own selection ->All checks passed!(laya/is untouched bythis PR). Both examples ran offline against the cached checkpoints.
Every suite that reads
examples/re-ran green:test_compose_env237,test_server_example50,test_example_server_limits94,test_example_server_errors14,test_example_server_per_call_controls42,test_evals/test_hooks_api/test_mcp/test_packagingclean.
Mutation witness (15 arms, 15 behaved)
Each arm edits a real file, runs the gate, and is scored by which named checks failed. All originals
are held in memory and rewritten before every arm; nothing is restored with git.
laya/agent.pyclamps from neither side (truncate_left = False)clamp/laya/agent.pylaya/agent.pyclamps by the wrong type (isinstance(state, dict))clamp/laya/agent.pylaya/onnx_agent.pyclamps from neither sideclamp/laya/onnx_agent.pydoes not expose itbanner sentence37/a sentence about \truncate_left`,37/a claim that the cut is invisible`truncated37/a claim that the cut is invisible37/prints every key of the reportusage["truncated_q"]37/reads no key,37/prints every keystate_tokens_dropped37/names \state_tokens_dropped``truncate_leftanywhere37/teaches the clamp by its real namestate_tokens15/reads no key,37/reads no key(+1)truncatedtruncate_left=isinstance(state, list)in the call, no local)truncatedThree limits I want on the record rather than discovered in review:
printlines while keeping its "a state that fits the window" lineleaves the gate green, by design: the keys are still read from a live call and still named in
backticks. That arm therefore deletes both blocks. A rule making the labelled block itself
mandatory is stronger than I thought was true, and I did not write it.
truncate_leftsentence rule is deliberately per sentence. Paraphrasing "takes it fromisinstance(state, list)" to "takes it from the state's type" fails even though the page names theexpression two lines earlier -- the pairing of the flag with the type, in one sentence, is the
thing a reader takes away. Say the word if you would rather it were per string.
examples/, so the printed values are verified by local runs against the cachedcheckpoints plus the static gate. The gate adds only stdlib imports (
ast,re) to a file thatalready imports
laya.common, and runs offline in the job that already callspython tests/test_truncation.py(ci.yml:125).No overlap with open work: against every open PR's file list, nothing but this PR touches
examples/15_truncation_basics.py,examples/03_reading_the_result.pyortests/test_truncation.py, and nothing touchesexamples/37_long_document.py(#687 touchestests/test_truncation_direction.py, a different file, and rewrites the clamp line inline -- thederivation is read as a pattern for exactly that, and the control arm above proves it). The gate is
appended at EOF, after the existing
usage/source checks, and leaves them untouched.Checklist
usageand about theclamp, and the gate that holds them to the code. The
examples/03edit is three words ("thefour keys after them" -> "the keys after them", "a seventh key" -> "one more key") removed in
the same pass because they are hand-counted transcriptions of the dict this PR is about and no
gate can see them.
main(base4aa6761, which isupstream/mainat push time;git rev-list --count HEAD..upstream/main->0).57 passed,113 checks passed, every suite that readsexamples/green,ruff and
compileallclean, mutation 15/15).examples change, and the test is what keeps the pages from drifting back.
The
docs (strict build)job is the inherited red everymain-based PR shows until #794 lands; nofile in this PR is a docs page, so it adds no nav or link surface.
This covers the documentation half of #174: the report exists, and now the pages that teach it cannot
claim otherwise without failing a test.