Repository navigation
feat(ts): report the options a head budget collapsed, as laya.common does - #817
Conversation
…does `buildQuestionPrefix` applied Python's option caps exactly -- 48 tokens each, then `max(4, (headMaxLen - 16) // n)` for all of them once they overflow the head -- and kept no count of what the cut cost. Two options sharing a prefix therefore collapse into one token span silently: the marker count still equals the option count, the guard in `_encodeItems` passes, and the question answers normally while no longer able to name those options apart. Python reports it as `usage["options"]` since NandhaKishorM#538; the SDK had the truncation half of the same budget story and not this half. `build_head`'s stats are mirrored whole (`total`, `distinct`, `tokens_per_option`), counted on the capped option ids before assembly so the last option cannot look distinguishable by running into the state, and filtered by a `collapsedOptions` that matches `laya.common.collapsed_options` including its `total`-is-the-question's rule. The agent adds the key only when something collapsed, so the 512-token request that loses nothing still reports nothing.
No weights and no network: the same character-level stub tokenizer the Python gate uses, so the expected numbers are Python's rather than assertions written to match this implementation. On `d8a2e59` the two agree token for token -- 186 tokens and four spans for the roomy head, 35 and one span at head_max_len 24, and 40 options behind 2 surviving markers at max_len 40 -- which is also the first time this path has been held to the Python side end to end. The Agent arms run a real Agent over a fake provider, so what is checked is the published `usage` block rather than the stats object, including its absence when nothing collapsed.
|
Merged for 0.3.24. I checked the port against the original rather than reading the description, and it is faithful. The detail worth calling out is that Verified on the merged tree: Note that Thank you. |
What
laya-tsnow reports the options its head budget collapsed, the waylaya/common.pydoes.laya-ts/src/common.ts: anOptionStatstype (total,distinct,tokens_per_option), carried onQuestionPrefix.optionStatsandSequenceStats.options, plus an exportedcollapsedOptions()that is the port of
laya.common.collapsed_options().laya-ts/src/agent.ts:SystemUsagegainsoptions?, and_systemOneManypublishes it only whensome question actually lost a span.
laya-ts/src/index.ts: exports the function and the type.laya-ts/tests/option-collapse.test.ts: 12 new tests;laya-ts/README.md: the budget section.Why
build_sequencecaps every option at 48 tokens and re-caps all of them tomax(4, (head_max_len - 16) // n)once they overflow the head (#538). Two options that share aprefix can come out of that cut as the same token span, and the marker count still equals the
option count, so nothing complains -- the question is answered from a head that can no longer name
those options apart. Core measures it and publishes it as
usage["options"], so a caller can see"58 labels, 42 spans". The TS port applies the identical caps -- same 48-token ceiling, same
re-cap formula -- but keeps no count of what they cost, so the same request over
laya-tsreturnsan answer whose confidence silently has a ceiling nothing in the response mentions.
Every expected number in the new test is what
laya/common.pyreturns for the same character-levelstub tokenizer, so a divergence here is a port bug rather than a style choice. Reproduced on this
tree:
build_sequence(..., return_stats=True)laya-tsafter this PR{options: 4, options_distinct: 4, tokens_per_option: None}, 186 tokens{total: 4, distinct: 4, tokens_per_option: null}, 186 ids{4, 1, 4}, 4 markers, 35 tokens{total: 4, distinct: 1, tokens_per_option: 4}, 4 markers, 35 ids{4, 4, 4}{total: 4, distinct: 4, tokens_per_option: 4}{40, 4, 9}, 2 markers in the sequence{total: 40, distinct: 4, tokens_per_option: 9},markers.length === 2The last row is the reason
totalcounts the question's options rather than the markers: a reportcounted from the markers would say "2 of 2" about a question with 40 of them. Before this change
usagehad six keys and none of them was this fact; a 40-label question at a 24-token headreported nothing.
Two facts stay separate, and the new tests hold them apart: a truncated state
(
state_tokens_dropped) and a collapsed head (options) are different losses, so the{max_len: 128, head_max_len: 24}arm assertstruncated: trueandoptions: {intent: {total: 4, distinct: 1, tokens_per_option: 4}}together, while the roomy armasserts
optionsis absent and the key set is exactly the six it was before.How it was verified
No Python file changed; the last three are the repo gates and are run anyway. Then the new gate
was mutation-checked -- nine faithful edits to the shipped code, each expected to turn it red
(
python -B /tmp/mut/run.py, restoring every file withgit checkout --between arms; eachresult is vitest's own line):
distinctalways equals the option counttotalcounted from the markers that reached the sequencetokens_per_option)SequenceStatssurvived: none. For the record: the second arm's first version (
total: markers.lengthinsidebuildQuestionPrefixUncached) survived green, and it was an equivalent mutant rather than a holein the gate -- pre-clamp
markers.length === opts.lengththere -- so it was replaced by thepost-clamp edit shown above, which the gate does catch.
Checklist
mainFixes # -- no open issue claims this, and no open PR touches
laya-ts/src/common.ts;tests/test_option_collapse.pyis the behavior this ports, so this is the TS half of #538'sreport. Related: #816 (merged as
350df4b) documentsusage.optionson the Python side and indocs/http-api.md; this makeslaya-tsactually fill that field, and the wording of both is thesame three keys.