Skip to content

test(structured): decide_batch does not forward model= to predict_batch, and now says what it does - #1125

Merged
NandhaKishorM merged 2 commits into
NandhaKishorM:mainfrom
aashish254:fix-decide-batch-forwarded-kwarg
Oct 12, 2026
Merged

NandhaKishorM merged 2 commits into
NandhaKishorM:mainfrom
aashish254:fix-decide-batch-forwarded-kwarg

Conversation

@aashish254

Copy link
Copy Markdown
Contributor

What

decide_batch told callers that model= is one of the keywords forwarded to runner.predict_batch. It
isn't: no predict_batch in the library declares it. The suite used to assert that forwarding anyway,
against a fake whose **kwargs absorbs anything. This corrects the prose and rebinds the claim to the
real signatures and to a runner shape that cannot absorb it.

  • laya/structured.py: 8 added / 3 removed lines of docstring, no code change.
  • tests/test_structured.py: +51 / -3.

Why

Router.predict_batch (laya/router.py:1590-1602), Agent.predict_batch (laya/agent.py:1409-1417) and
ONNXAgent.predict_batch (laya/onnx_agent.py:293-300) declare no model parameter and none of them
takes **kwargs, so a keyword call predict_batch(..., model=...) is a TypeError on all three. What
decide_batch actually does with model= depends on which convention the runner follows
(laya/structured.py:420-431):

  • A Router-like runner (one with route_batch) gets the routing pins and the token budget --
    model, task, lang, lang_guess, max_len, head_max_len (_ROUTER_REQUEST_KEYS,
    laya/structured.py:369) -- copied onto every request dict, because that is where
    Router.predict_batch reads them. So decide_batch(router, states, schema=..., model="...") works and
    pins a registered checkpoint. main already pins that path as
    batch/router model= reaches a registered checkpoint; this PR does not touch it.
  • An Agent-like runner gets **predict_kwargs passed straight through, so model= raises TypeError
    while lang= -- which Agent.predict_batch and ONNXAgent.predict_batch both declare
    (laya/agent.py:1410, laya/onnx_agent.py:294) -- does not.

The old test asserted the forwarding of model= against FakeBatchRunner.predict_batch(self, states, questions, **kwargs), i.e. against an object that accepts any keyword. The replacement forwards lang=,
which is meaningful under both conventions, and the model= claim is checked three ways: inspect. signature(...).bind(..., model=...) on the three real classes, _ROUTER_REQUEST_KEYS membership, and a
new StrictAgentRunner (named keywords, no **kwargs) that decide_batch meets the way a real Agent
would -- TypeError naming 'model', and batch_size=/lang= arriving as keywords. No checkpoint is
loaded by any of these; they are signature and dispatch checks.

How it was verified

python tests/test_structured.py
python scripts/test_suites.py
ruff check laya/ --select=E9,F63,F7,F82,F401,F811 --line-length=120
python -m compileall -q laya/ tests/
  • tests/test_structured.py on main at c4a805b5: 156 passed, 0 failed. On this branch:
    163 passed, 0 failed.
  • scripts/test_suites.py: 497 passed, 1 skipped -- all 87 suites passed, rc=0.
  • ruff: All checks passed!, rc=0. compileall: rc=0.

Mutation witnesses. Four mutants, each file restored between runs, everything committed first:

  1. Agent.predict_batch given a model: Optional[str] = None parameter: 159 passed, 1 failed,
    FAIL batch/Agent.predict_batch rejects model=: did not raise <class 'TypeError'>.
  2. "model" dropped from _ROUTER_REQUEST_KEYS: 155 passed, 5 failed -- my
    batch/model is one of the router's per-request keys plus four batch/router ... cases already on
    main, all failing with TypeError("Router.predict_batch() got an unexpected keyword argument 'model'").
  3. predict_kwargs.pop("model", None) inserted in decide_batch's Agent branch, i.e. the silent-drop
    version of the bug: 162 passed, 1 failed,
    FAIL batch/agent-like decide_batch rejects model=: got False, want True.
  4. **predict_kwargs removed from that same branch: 160 passed, 3 failed --
    batch/forwards predict kwargs (main's own, now keyed on lang=),
    batch/and the declared keywords reach predict_batch as keywords, and the model= rejection above.

The docstring is prose and no mutant can test it; the checks are what keep the claim honest if a runner
signature changes.

Deconfliction. All 48 open pull requests were scanned for the two files this PR touches: none has a
hunk in laya/structured.py or tests/test_structured.py. Rebased on c4a805b5 (0.4.3); neither file
changed between 68804629 and it. The diff was scanned for credentials and absolute local paths -- one
hit, the phrase "token budget" in the new docstring line -- no credential and no local path.

Deliberately not in this diff: docs/structured.md:145 repeats the same claim for the same function
("Keyword arguments reach predict_batch, so batch_size=, model= and hooks work as they do for
decide"). A docs/ change owes the strict build (zensical build --strict --clean) and that tool is not
installed in the environment I verified this in, so I kept it out rather than hand you an unverified docs
diff. Say the word and it comes as its own PR.

Checklist

  • Focused on one change (split unrelated work into another PR)
  • Rebased on the latest main
  • Tests pass locally
  • Docs or examples updated when the public API changed

Fixes: no open issue. Found while auditing the integration surfaces for controls that are documented but
not forwarded. The public API does not change -- this removes a promise the code never kept.

…onvention

The suite asserted that decide_batch forwards model= to predict_batch, against a fake
whose **kwargs absorbs anything. No predict_batch in the library declares model=, so
that call cannot work on a real runner. Bind the claim to the real signatures instead,
and add the user-facing half: an Agent-shaped runner (named keywords, no **kwargs)
raises TypeError naming 'model' while keeping batch_size= and lang= as keywords. The
Router half already had a check on main -- "batch/router model= reaches a registered
checkpoint" -- because decide_batch copies model= onto every request there rather than
passing it as a keyword, which is the asymmetry the docstring now names. tests/
test_structured.py goes 156 -> 163 checks.
…redict_batch keyword

The docstring listed model= among the keywords forwarded to runner.predict_batch. It is
not: Router.predict_batch declares none of the routing pins or budget keys, so
decide_batch copies them onto each request, and an Agent-like runner -- whose
predict_batch declares lang= but no model= -- raises TypeError on it.
@NandhaKishorM
NandhaKishorM merged commit af89848 into NandhaKishorM:main Oct 12, 2026
40 checks passed
@NandhaKishorM

Copy link
Copy Markdown
Owner

Merged. Replacing an assertion made against a fake whose **kwargs absorbs anything with one bound to the real signatures is the substantive change here, and it is the kind of test correction that is worth more than the docstring it fixes.

inspect.signature(cls.predict_batch).bind(..., model="english") raising TypeError for Agent, ONNXAgent and Router alike loads no checkpoint and cannot drift from the real parameter lists. The StrictAgentRunner with no **kwargs is the other half: it is what makes model= an error a caller actually meets rather than a silently swallowed extra argument. Keeping lang=/batch_size= in the forwarding assertion preserves the coverage the old test was aiming at.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants