Skip to content

docs(structured): split decide_batch's keywords by who reads them - #1130

Merged
NandhaKishorM merged 1 commit into
NandhaKishorM:mainfrom
aashish254:docs-structured-batch-keywords
Oct 12, 2026
Merged

NandhaKishorM merged 1 commit into
NandhaKishorM:mainfrom
aashish254:docs-structured-batch-keywords

Conversation

@aashish254

Copy link
Copy Markdown
Contributor

What

The "Scoring many states" section of docs/structured.md says of decide_batch: "Keyword arguments
reach predict_batch, so batch_size=, model= and hooks work as they do for decide." That is
only true for some of the keywords, and model= -- the one named in the sentence -- is the one that
is not. Rewritten to state the split the code actually makes. Docs only; no code line changed.

Why

laya/structured.py::decide_batch handles its two kinds of keyword differently. Call-level keywords
are forwarded to predict_batch; the routing pins and the token budget
(laya/structured.py::_ROUTER_REQUEST_KEYS = model, task, lang, lang_guess, max_len,
head_max_len) are read off every request by a Router, whose predict_batch declares none of
them, so decide_batch copies them onto each request dict instead -- which is why model= works on
a Router but is not "a keyword that reaches predict_batch". On an Agent-like runner the
difference is something a caller meets: predict_batch declares batch_size=/lang=/hooks=, has
no model=, and swallows nothing, so passing model= there is a TypeError, not a silent extra
argument. Measured on c4a805b5 (first column is which class the signature belongs to):

Agent      declares: model=False lang=True batch_size=True hooks=True sort_by_length=True **kwargs=False
ONNXAgent  declares: model=False lang=True batch_size=True hooks=True sort_by_length=True **kwargs=False
Router     declares: model=False lang=False batch_size=True hooks=True sort_by_length=True **kwargs=False
_ROUTER_REQUEST_KEYS = ('model', 'task', 'lang', 'lang_guess', 'max_len', 'head_max_len')

# a runner shaped like Agent.predict_batch -- those keywords, no **kwargs, no route_batch:
model=           -> TypeError: Strict.predict_batch() got an unexpected keyword argument 'model'
batch_size=,lang= -> [{'d': 'billing'}] | reached predict_batch as {'batch_size': 4, 'lang': 'en', 'hooks': None}

The Router half is already pinned in the repo: tests/test_structured.py has
batch/router model= reaches a registered checkpoint (:442) and
batch/router decide_batch(model=) agrees with decide(model=) (:446). This PR's sibling for the
docstring half is #1125; this one changes only the page, so the two do not collide.

How it was verified

python -B tests/test_structured.py        # 156 passed, 0 failed -- unchanged from c4a805b5
python -B tests/test_structured_docs.py   # 116 checks passed, 0 failed
python -B scripts/test_suites.py          # all 87 suites passed
zensical build --strict --clean           # rc=0, "No issues found" in 1.37s
python -m compileall -q laya/ tests/      # rc=0
ruff check laya/ --select=E9,F63,F7,F82,F401,F811 --line-length=120

The two new sentences were grepped out of the built page (site/structured/index.html) rather than
assumed from the build's exit code.

Deconfliction and scanning:

Checklist

  • Focused on one change (split unrelated work into another PR)
  • Rebased on the latest main
  • Tests pass locally
  • Docs or examples updated when the public API changed (docs only; no API change)

Fixes #

@NandhaKishorM
NandhaKishorM merged commit fce2213 into NandhaKishorM:main Oct 12, 2026
40 checks passed
@NandhaKishorM

Copy link
Copy Markdown
Owner

Merged. The split by who reads a keyword is the thing that makes decide_batch's behaviour predictable instead of surprising. The call-level ones go to predict_batch as keywords; the routing pins and token budget are read off every request by a Router, whose predict_batch declares none of them, so decide_batch copies them onto each request instead. That is why model= pins a checkpoint on a Router and is a TypeError on an Agent-like runner, which looks like an inconsistency until you know which side reads it.

Naming laya/structured.py::_ROUTER_REQUEST_KEYS as the authoritative tuple is better than listing the keys in prose alone, since the list can grow.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants