Repository navigation
docs(structured): gate every documented usage shape against the code - #819
Conversation
1e9b962 to
4c26433
Compare
|
Short answer: the red check was not from this PR's code -- rebased onto current The only failing check was Root cause was a stale base: this branch was cut 15 commits behind Local gates on the rebased tree:
|
|
Merged for 0.3.24. Deriving the key sets from I checked the gate is load-bearing rather than decorative. Renaming one key in the documented sample: So it fails on exactly the drift it exists to catch, and it goes back to 113 passing when reverted. Fixing Verified: Thank you. |
What
Two pages tell a reader what an answered
usageblock contains. Both were wrong, and nothing couldsee it. This fixes the shapes and adds the gate, in the suite that already owns this page's claims:
docs/structured.md—result.usageshown as the six keys an answer builds, plus a paragraphsaying what each is and that
optionsis the conditional seventh.examples/03_reading_the_result.py— the same shape in the banner beginners are sent to, and theprose extended past the two keys it explained.
tests/test_structured_docs.py— 26 new checks (+113 total, from 87) that derive the key setsfrom
laya/agent.pyandlaya/onnx_agent.pywithastand hold each documented shape to thepath it illustrates.
Why
docs/structured.md:153showed{"input_tokens": 42, "output_tokens": 0}andexamples/03_reading_the_result.py:16showed the same two keys as the shape of everypredict()call. That is the shape of a call with no questions: the agents put six keys into every answered
result -- #174's truncation report (
state_tokens,state_tokens_dropped,truncated,truncated_questions) -- and a seventh,options, when the head budget collapsed a question'soptions (#538). Measured on this tree, cached
englishcheckpoint, CPU:docs/structured.mdoptionswhen a question's options collapseexamples/03bannerREADME.mdempty-questions sentencedocs/http-api.mdoptionsThe reader who cannot see
truncatedcannot see that the answer came from a state the model onlypartly read, which is the difference between trusting 0.95 and knowing it was 0.95 of a truncated
input. The example is where the docs send people for this shape, so the two-key version is the one
most likely to be copied into glue code.
Because the gate is per-path rather than blanket,
README.md's two-key sentence stays: it is aboutagent.predict(state, {}), whose result really is that shape. The gate also found a secondempty-
answersliteral --predict_longbuilds one, and its usage carrieswindows-- so thederivation is scoped to the function that holds the answered
usage = {...}literal. Asking thedocs for
windowswould have been a false claim:decidecannot return that shape.How it was verified
zensical build --strictwas not run:zensicalis not installed here. The change adds no link,no nav entry and no heading, so nothing the strict build resolves is touched -- but that is a
gap in this report, not a pass.
Then the new checks were mutation-checked. Eleven arms, control green, none survived:
optionsas an always keyoptionswithout bounding it as conditionaltruncateddecideforwards a subset instead of the blockArm 11 needed two tries, and the first one is worth recording: the initial helper identified the
no-questions literal by its key set (
{"input_tokens", "output_tokens"}), so an arm that added athird key made the helper raise instead of failing a named check -- a gate that aborts rather than
reports, and a control defined by the thing it is meant to check. It now finds that literal by
structure (the result dict whose
answersis{}) inside the answered path's own function, andarm 11 turns red on
README's empty-questions shape stays the empty one.Checklist
mainFixes # -- no open issue claims this, and no open PR touches
docs/structured.md,examples/03_reading_the_result.pyortests/test_structured_docs.py(checked against all 28 openPRs' file lists).
Two claims were deliberately left alone, because they cannot be checked from this repository:
README.md:607calls the payload's usage block{input_tokens, output_tokens}inside a sentenceabout TypeSafe's hosted Jev wire format -- comparing to a hosted API is out of scope here and the
sentence would need Jev's own schema to fix honestly.
docs/hooks/api.md:59documents the hookcontext's usage, which
laya/hooks.py:452-456 aggregate_usagereally does return as two summedkeys; that row is correct and stays.
Related: #816 (merged as
350df4b) pinned this contract fordocs/http-api.md's Response section;this covers the two other places it is written down, in the suite that already gates this page.
#817 is open on the
laya-tsside, filling in the sameusage.optionsfield there.