Skip to content

docs(http-api): report every key a decision response actually carries - #816

Merged
NandhaKishorM merged 2 commits into
NandhaKishorM:mainfrom
aashish254:fix-http-response-keys
Oct 1, 2026
Merged

NandhaKishorM merged 2 commits into
NandhaKishorM:mainfrom
aashish254:fix-http-response-keys

Conversation

@aashish254

Copy link
Copy Markdown
Contributor

What

docs/http-api.md's ### Response section described a smaller payload than the server sends, and
named one key no code path has ever set. This corrects the page and adds a gate that reads every key
list out of the source, so the page and both agents have to keep describing the same response.

Two files: docs/http-api.md (the sample, two new field tables, and the prose around them) and
tests/test_serve.py (one new test, plus the three helpers it reads the code with).

Why

What the page said, against what the code does, measured on 4185647 with the cached english
checkpoint on CPU:

field group   documented   actually built   where that comes from
usage         2 keys       6 always + 1     Agent.predict_batch's usage literal, and the same
                             conditional     seven in OnnxAgent._infer_batch's
routing       4 keys       5                the keywords every RouteDecision(...) in _route takes
detection     4 fields     8                laya.lang.analyse()'s return
lang_guess    1 key        0                the page called it routing evidence; it is a request
                                            control (BODY_CONTROLS) and no branch sets it

The truncation report is the part that matters. state_tokens, state_tokens_dropped, truncated
and truncated_questions are the only place a caller can learn that its state was cut before the
model read it -- the budget that decides that moves with max_len, head_max_len and every
question's own option prompt (#174), so nothing in the request tells it. options is the same story
one level down: when the head budget leaves the options without a token span each, the answer has a
ceiling the caller cannot otherwise see. A page that documents two of those keys leaves a caller
sizing states by character count with no hint that the four others exist.

lang_guess was worse than an omission -- the page listed it as evidence carried inside routing,
so code written against the page would find routing.get("lang_guess") is None on every response.
What a hint actually does is named in reason, and it is a request control.

And the sample could not be reproduced: it printed "input_tokens": 74 for the request the page
itself shows, while that request on this commit gives 83 with state_tokens: 12. It is now an
answer this server really gave (the printed request, cached english, CPU), which is also what lets
the gate check the sample against itself -- the flag is the worst-case drop being non-zero,
output_tokens is 0.

How it was verified

Nothing is transcribed. _decision_response_site() parses the two agents and returns the result
dict's keys, the keys its usage literal always carries, the keys it adds conditionally, and the
model constant it stamps; _route_decision_keys() collects the keywords of every
RouteDecision(...) in laya/router.py; analyse() is called over four states (English, German,
Japanese, empty) to show its key set does not move with the script -- which is what licenses one
table row to describe it. Each set is then held to the page in both directions, the way the
request-body and /health gates above already do, so a key added to a response and a key dropped
from the docs both fail here. The two agents are also held to each other, because the page describes
one payload for both.

  • python -m pytest tests/test_serve.py -q -> 185 passed (184 before this PR + the new one)
  • python tests/test_doc_tables.py -> 114 passed, 0 failed; python tests/test_env_docs.py -> 27
    passed, 0 failed (both parse docs pages, and both had to keep passing)
  • ruff check laya/ --select=E9,F63,F7,F82,F401,F811 --line-length=120 clean, compileall clean
  • the sample re-run from scratch through Router.predict(state, questions) before writing it, and
    every number in it checked field by field against that run

Fourteen mutation arms, each reverting after itself; the gate was green only with nothing changed:

arm verdict
the page exactly as it stands on main red -- "Right contains 4 more items, first extra: state_tokens"
agent.py stops building state_tokens red -- "no usage = {...} literal with a state_tokens key"
onnx_agent.py adds a key torch does not send red -- the two agents' usage sets compared
the usage table invents a key red
the usage table drops the options row red
the routing table drops workflow red
_route starts reporting lang_guess red
the detection row stops naming diacritic_rate red -- names the field and the row
the sample loses a detection field red
analyse() makes mixed_segment conditional red
the head renames itself in the result literal red
the sample claims truncated with nothing dropped red -- prints the usage block
the sample's routing block loses workflow red
the sample's usage block loses state_tokens red

One thing this deliberately does not touch: README.md says Laya's payload is
"schema-identical" to hosted Jev's and names a two-key usage block. The key set is wrong there the
same way, but the sentence is a parity claim about Jev's schema, which nothing in this repo can
check, so it is left for someone who can.

Related work: #814 edits the same page -- its 400 row and the "Not (yet) here" section, several
hundred lines below this change -- so the two will not conflict, but whichever lands second should
rebase. #716 and #667 touch tests/test_serve.py at other locations, and nothing open changes what
the response carries. No open issue proposes this; #390 tracks the docs site's structure, not field
lists.

Checklist

  • Focused on one change (split unrelated work into another PR)
  • Rebased on the latest main
  • Tests pass locally
  • Docs or examples updated when the public API changed

Fixes # (no open issue for this: the response field lists are only on docs/http-api.md, and no
issue proposes holding them to the code)

The Response section showed a usage block of two keys while both agents build
six always-present ones plus a conditional `options`, named three of the five
routing keys, listed four of the eight fields `laya.lang.analyse()` returns,
and described `lang_guess` as routing evidence when it is a request control
that leaves no key of its own. The sample itself predated NandhaKishorM#174's truncation
report, so the one thing a caller cannot see anywhere else in the payload --
that its state was cut before the model read it -- was invisible here too.

Replace the sample with an answer this server really gave (the request the page
prints, cached `english` checkpoint, CPU) and add the two field tables.
…uild

Every key list is read out of the source rather than transcribed: the result
and `usage` literals of both agents' batch inference, the keywords every
`RouteDecision(...)` in `_route` is built with, and `analyse()` at runtime over
four states to show its key set does not move with the script. Held to the page
in both directions, as the request-body and `/health` gates above already do, so
a key added to a response and a key dropped from the docs both fail here.

The sample is also checked against itself -- the flag is the worst-case drop
being non-zero, `output_tokens` is 0 -- because the page now presents it as one
real answer rather than as illustration.
@NandhaKishorM
NandhaKishorM merged commit 350df4b into NandhaKishorM:main Oct 1, 2026
25 checks passed
@NandhaKishorM

Copy link
Copy Markdown
Owner

Merged for 0.3.24.

Reading every key list out of the source rather than typing it into the test is the right shape, and it is the same approach as the /health field contract from #811, so the two now work the same way. A page that documents a smaller payload than the server sends is the failure mode where people write defensive code around fields they think are optional.

The lang_guess entry is the one I am glad you caught: the page named a routing key no code path has ever set, which is worse than omitting it.

Verified on the integrated branch: tests/test_serve.py 195 passed, test_doc_tables and test_env_docs green, full test lane green.

Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants