Skip to content

docs(http-api): the option ceiling moves with the question's instructions too - #1151

Merged
NandhaKishorM merged 1 commit into
NandhaKishorM:mainfrom
aashish254:docs-option-ceiling-instructions
Oct 12, 2026
Merged

NandhaKishorM merged 1 commit into
NandhaKishorM:mainfrom
aashish254:docs-option-ceiling-instructions

Conversation

@aashish254

Copy link
Copy Markdown
Contributor

What

Reworks the option-ceiling paragraph in docs/http-api.md (the one #1149 added) so the four
ceilings it publishes are pinned to the question instructions they were measured with, and names
the mechanism that makes them move.

Before → after, same file, same section:

  • "is set by that same pair and by their text" → "is set by that same pair and by their text
    and by the question's own instructions", with the baseline cells now labelled as measured with
    the three-token instruction Which queue?.
  • Added the two other instruction lengths: a two-token instruction gives 169, 127, 339 and 255,
    an eight-token one gives 168, 126, 338 and 254, and a seventeen-token one gives the
    eight-token numbers exactly.
  • Added the mechanism: laya/common.py::build_head truncates the encoded
    choice question: <instructions> prefix to max(8, opt_budget) tokens (laya/common.py:259), so
    the first [MASK] lands at position 7, 8 or 10 for those three wordings and every marker after
    it shifts with it.
  • Added that the state does not move the ceiling: 338 answers and 339 refuses at max_len=1024 with
    the eight-token instruction whether the state is one word or the eleven-token sentence the section
    already uses.
  • "has to quote the budget it sent and the option text, not just N" → "…the budget it sent, the
    option text and the instruction, not just N".

The published 168 / 126 / 339 / 254 are kept, not replaced: they are what a three-token
instruction gives, which is the wording the earlier measurements used.

Why

Those four numbers were printed in #1149 without the instruction they were taken with, so the
paragraph reads as if any caller sending max_len=512/1024 at head_max_len=192 gets them. It
does not. The head is a fixed-length prefix plus one marker per option, and build_head truncates
that prefix to max(8, opt_budget) tokens, so a shorter instruction leaves more room for markers
and the ceiling rises; once the instruction reaches the 8-token cap the prefix is saturated and the
ceiling stops moving.

That is also what reconciles the follow-up correction in d4f5740c ("the 1024-budget ceiling for
the short label is 339, not 338"): 339 and 338 are both correct, for a three-token and an
eight-or-longer-token instruction respectively. Naming the instruction is what makes the section
reproducible instead of arguable — a caller comparing this endpoint against another server's stated
"maximum N options" needs the budget, the option text and the instruction, and N alone is not one of
the three.

How it was verified

Every cell in the paragraph was met through the public call on this exact tree (682ac671,
release 0.4.4), CPU and offline, answering at the ceiling and refusing one option above it. The
driver asserts laya.__file__ resolves inside the worktree before loading the checkpoint, so the
transcript below is from this branch's code and not an installed copy.

[instructions] token counts verified: [2, 3, 8, 17]

[marker] first marker index (build_head, head_max_len=192)
  ins= 2 tok  first_marker= 7  head_tokens=512  {"options": 168, "options_distinct": 168, "tokens_per_option": 4}
  ins= 3 tok  first_marker= 8  head_tokens=513  {"options": 168, ...}
  ins= 8 tok  first_marker=10  head_tokens=515  {"options": 168, ...}
  ins=17 tok  first_marker=10  head_tokens=515  {"options": 168, ...}

[ceiling] answers at N, refuses at N+1 (state=9 words, head_max_len=192)
  ins= 2 terse    max_len=512  n=169 ANSWERED input_tokens=512 options=absent | 170 REFUSED
  ins= 2 sentence max_len=512  n=127 ANSWERED total=127 distinct=1 tpo=4      | 128 REFUSED
  ins= 2 terse    max_len=1024 n=339 ANSWERED input_tokens=1024              | 340 REFUSED
  ins= 2 sentence max_len=1024 n=255 ANSWERED total=255 distinct=1 tpo=4      | 256 REFUSED
  ins= 3 terse    max_len=512  n=168 ANSWERED input_tokens=512 options=absent | 169 REFUSED
  ins= 3 sentence max_len=512  n=126 ANSWERED total=126 distinct=1 tpo=4      | 127 REFUSED
  ins= 3 terse    max_len=1024 n=339 ANSWERED input_tokens=1024              | 340 REFUSED
  ins= 3 sentence max_len=1024 n=254 ANSWERED total=254 distinct=1 tpo=4      | 255 REFUSED
  ins= 8 terse    max_len=512  n=168 ANSWERED                               | 169 REFUSED
  ins= 8 sentence max_len=512  n=126 ANSWERED                               | 127 REFUSED
  ins= 8 terse    max_len=1024 n=338 ANSWERED                               | 339 REFUSED
  ins= 8 sentence max_len=1024 n=254 ANSWERED                               | 255 REFUSED
  ins=17 terse    max_len=512  n=168 ANSWERED                               | 169 REFUSED
  ins=17 sentence max_len=512  n=126 ANSWERED                               | 127 REFUSED
  ins=17 terse    max_len=1024 n=338 ANSWERED                               | 339 REFUSED
  ins=17 sentence max_len=1024 n=254 ANSWERED                               | 255 REFUSED

[state] eight-token instruction, max_len=1024, n=338/339
  nine-word/eleven-token state (words=9 tokens=11): 338 ANSWERED input_tokens=1024 | 339 REFUSED
  one-word               state (words=1 tokens=2): 338 ANSWERED input_tokens=1024 | 339 REFUSED

[verdict] mismatches = 0 []

Each refusal is the budget error the section already quotes, e.g. question 'q': only 168 of its 169 option markers fit in max_len=512 with head_max_len=192 spent on the question; lower head_max_len, raise max_len, or use fewer options.

One wording error was caught while re-checking the paragraph against the probe: it said
"an eleven-word state" for a sentence that is nine words and eleven tokens. The paragraph now says
"the eleven-token sentence this section has been using", which matches the state_tokens=11 figure
in the paragraph above it.

Gates run on this tree:

python -B scripts/test_suites.py            -> rc=0, "all 87 suites passed", 499 passed, 1 skipped
python -B tests/test_example_server_limits.py -> rc=0, 155 passed, 0 failed
python -B tests/test_env_docs.py             -> rc=0, 63 passed, 0 failed
ruff check laya/ --select=E9,F63,F7,F82,F401,F811 --line-length=120 -> rc=0, "All checks passed!"
python -m compileall -q laya/ tests/         -> rc=0
zensical build --strict --clean              -> rc=0, "No issues found"

The rendered page was then parsed with tags stripped and whitespace collapsed, and each claim the
paragraph makes was required to appear exactly once: 169, 127, 339 and 255, an eight-token one gives 168, 126, 338 and 254, max(8, opt_budget), lands at position 7, 8 or 10, the same seventeen-token wording returns the eight-token numbers exactly, 338 options answer and 339 refuse whether the state is one word or the eleven-token sentence this section has been using, the budget it sent, the option text and the instruction, not just N. Both superseded strings — eleven-word
and the budget it sent and the option text, not just N — count zero, and the single remaining
338 and 254 is the qualified eight-token cell. [verdict] failures = 0.

Line lengths: the file's own p90 is 101 characters and its longest line is 341, so the five new
101–103 character lines match the surrounding convention; nothing else was reflowed.

Deconfliction

43 open PRs as of 2026-10-12T04:46Z. Only one of them touches docs/http-api.md — #687, whose
hunks in that file are at lines 128, 166 and 210, all above this change at 379–406. Disjoint.

This is not linked to an open issue; it is a correction to text that merged in #1149 and to the
ceiling disagreement in d4f5740c.

Checklist

  • Focused on one change (split unrelated work into another PR)
  • Rebased on the latest main (682ac671, release 0.4.4)
  • Tests pass locally
  • Documentation/examples updated as needed

…ions too

NandhaKishorM#1149 printed four ceilings without naming the instruction they were measured with, and the
paragraph reads as if any caller sending that budget gets those numbers. They do not: build_head
truncates the encoded `choice question: <instructions>` prefix to max(8, opt_budget) tokens, so the
first option marker sits at position 7, 8 or 10 depending on the wording and every marker after it
shifts. Measured through the public call on 682ac67, the four cells are 169/127/339/255 at two
tokens of instruction, 168/126/339/254 at three, 168/126/338/254 at eight -- and identical at
seventeen, because the prefix is saturated. The state does not move them.

Pins the published 168/126/339/254 to the three-token wording rather than replacing it, and adds the
instruction to what a caller has to quote when comparing "maximum N options" across servers.
@NandhaKishorM
NandhaKishorM merged commit 11a3590 into NandhaKishorM:main Oct 12, 2026
44 checks passed
@NandhaKishorM

Copy link
Copy Markdown
Owner

Merged in 0.4.5. Every number reproduces, which is worth saying plainly because this PR and #1149 appeared to contradict each other and neither was wrong.

Bisected on the laya checkpoint, CPU, offline, through the public call, with head_max_len=192:

ins_tokens=2   first [MASK] at 7    169  127  339  255
ins_tokens=3   first [MASK] at 8    168  126  339  254
ins_tokens=8   first [MASK] at 10   168  126  338  254
ins_tokens=15  first [MASK] at 10   168  126  338  254

Columns are T <i>/512, sentence/512, T <i>/1024, sentence/1024. The marker positions are exactly the 7, 8 and 10 you name, and the fourth row confirms the cap: once the instruction is at or past eight tokens the numbers stop moving, because head_ids[: max(8, opt_budget)] has already cut it to eight at these option counts.

So 338 and 339 were both measured correctly, with different instructions. I corrected #1149's 338 to 339 after bisecting it myself, and that correction stands for the three-token wording the published cells use. Keeping those cells and labelling them is the right call rather than replacing them.

The instruction is now the third thing a caller has to quote. That makes the paragraph less quotable and more true.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants