Skip to content

fix(batch): refuse a batch_size the chunker cannot spend - #1458

Merged
NandhaKishorM merged 1 commit into
NandhaKishorM:mainfrom
aashish254:fix-batch-size-chunk
Nov 6, 2026
Merged

NandhaKishorM merged 1 commit into
NandhaKishorM:mainfrom
aashish254:fix-batch-size-chunk

Conversation

@aashish254

Copy link
Copy Markdown
Contributor

What

Adds laya.common.check_batch_size, a named validator for the per-call batch_size cap, and calls it at the entry of the three places that spend that value: Agent.predict_batch, ONNXAgent.predict_batch and window_batch_cap. A float, a str, a bool, a list, a dict, nan or inf is now refused with

ValueError: batch_size must be an integer count of states, got 2.5

instead of crashing later or quietly doing something else. None, 0 and negative integers keep meaning "no cap", and numpy.int64 and any other numbers.Integral are still spent as a cap.

Both predict_batch docstrings now name the refusal and the no-cap rule, and window_batch_cap's "an explicit batch_size is always honoured" says who decides what counts as explicit.

Why

batch_size is spent as a range() step, so a value range() cannot spend leaked the interpreter's own error from the middle of the batch loop. Measured at 0.4.22, on the weight-free fakes the two suites already build:

batch_size before after
2.5 TypeError: 'float' object cannot be interpreted as an integer ValueError, named, before any state is encoded
"2" TypeError: '>' not supported between instances of 'str' and 'int' ValueError, named
True [2, 2, 2, 2, 2, 2] on the six-state length fake — six one-state passes, and sort_by_length silently inert because its own gate is 1 < chunk ValueError, named
False read as "no cap" (one pass over every state), because the gate is batch_size > 0 ValueError, named
window_batch_cap(413, 23, 312, 2.5) returned 2.5, which predict_long then handed to predict_batch ValueError, named

laya/mcp/tools.py::_validate_batch_size and laya/serve.py::_validate_budget_param already refused both shapes at their own entry, so the raw TypeError reached only the Python route — the documented one, and the one with no schema in front of it. The MCP and HTTP surfaces keep their own validators and their own error codes untouched: their contract also refuses batch_size < 1, which is deliberately stricter than core, where 0 and -1 mean "no cap".

The shape is guarded, the sign is not. laya-java/fixtures/window_plan.json's batch_cap grid replays (None, 0, -1, 1, 8) through the Java, Go, C#, C++ and TS ports, whose typed int treats zero and negative as "unset", so every one of those rows must keep its recorded cap. Integral rather than int is the test because ece_score in the same module makes that argument for bins, and a caller who derives the cap from an array shape is the common case.

How it was verified

PYTHONPATH=. python -B tests/test_batch.py         # 89 -> 105 passed, 0 failed
PYTHONPATH=. python -B tests/test_predict_long.py  # 384 -> 391 passed, 0 failed
PYTHONPATH=. python -B tests/test_onnx_batch.py    # 67 -> 83 passed, 0 failed
ruff check laya/ --select=E9,F63,F7,F82,F401,F811 --line-length=120   # All checks passed!
python -m compileall -q laya/ tests/               # rc=0
PYTHONPATH=. python -B laya-java/scripts/gen_fixtures.py --check       # rc=1, see below
LAYA_TREE=$PWD python -B laya-bench/ci_gate.py     # all 93 suites passed, exit 0

The three suites are the ones carrying the new rows: test_batch.py refuses the ten bad shapes on Agent and pins the exact message, then asserts None/0/-1/10**400 still go out as one pass and numpy.int64(3) still chunks [6, 6]; test_predict_long.py adds the same refusals to window_batch_cap and pins its no-cap spellings at the measured un-capped pass width (30); test_onnx_batch.py does all of it over the stub session, so the ONNX route's guard line is covered by rows that assert the refusal runs no session at all.

gen_fixtures.py --check exits 1 on lang_detect.json is stale and python_json.json is stale. Both are stale at 462c264 with the same two lines and the same exit code — they need a checkpoint this run cannot reach, and neither family reads batch_size.

No committed golden moves. The generator skips the window_plan.json family without a checkpoint, so the rows were recomputed directly with the patched function: 810 committed batch_cap rows, 0 whose recorded cap changes, over the spellings [-1, 0, 1, 8, None] the grid actually contains.

Five mutation arms, each restored from a copy outside the tree, and the three suites re-run from disk afterwards:

  • delete the guard line in Agent.predict_batch — test_batch.py dies with the base TypeError: 'float' object cannot be interpreted as an integer at agent.py:1587, i.e. the leak itself.
  • delete the guard line in ONNXAgent.predict_batch — test_onnx_batch.py dies with the same TypeError at onnx_agent.py:832.
  • delete the guard call inside window_batch_cap — 4 named failures in test_predict_long.py.
  • swap Integral for int — both batch suites die on ValueError: ... got np.int64(3) / np.int64(2), so the numpy rows are load-bearing.
  • drop the bool clause — 2 named failures (chunk/batch_size True is refused by name, ... False ...).

Mergeability was checked against the current upstream head, 462c2644 (release 0.4.22): git merge-tree --write-tree returns rc=0 with no conflict. The nearest open PR, #687 (perf(agent)), edits the same two predict_batch bodies at overlapping context lines but never validates batch_size, so nothing here duplicates it; if it lands first, this rebases by hand.

Checklist

  • Focused on one change (split unrelated work into another PR)
  • Rebased on the latest main
  • Tests pass locally
  • Docs or examples updated when the public API changed

Fixes #

`predict_batch` spends `batch_size` as a `range()` step, so a value `range()`
cannot spend leaked the interpreter's own error from the middle of the batch
loop. Measured at 0.4.22: 2.5 raised TypeError ('float' object cannot be
interpreted as an integer) and "2" raised TypeError ('>' not supported between
instances of 'str' and 'int'), while True silently capped every pass at one
state and switched sort_by_length off. laya/mcp/tools.py and laya/serve.py
already refused those shapes at their own entry, so only the Python route met
the raw TypeError. common.check_batch_size refuses them by name at the entry of
Agent.predict_batch, ONNXAgent.predict_batch and window_batch_cap, following the
Integral precedent ece_score sets for `bins`.

The sign is untouched: None, 0 and -1 still mean "no cap", which is what
laya-java/fixtures/window_plan.json's batch_cap grid pins -- recomputing all 810
committed rows with the patched function moves none of them.

test_batch 89 -> 105, test_predict_long 384 -> 391, test_onnx_batch 67 -> 83;
the repo's full CI list (scripts/test_suites.py, 93 suites) passes.
@NandhaKishorM
NandhaKishorM merged commit 46277d0 into NandhaKishorM:main Nov 6, 2026
66 of 67 checks passed
NandhaKishorM added a commit that referenced this pull request Nov 6, 2026
… import

check_batch_size joined the laya.common import block in laya/agent.py, so every line
number below it moved by one and tests/test_doc_citations.py caught the one citation that
names a symbol beside it.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@NandhaKishorM

Copy link
Copy Markdown
Owner

Merged, thank you.

Verified here: on unpatched main tests/test_batch.py dies with the raw TypeError: 'float' object cannot be interpreted as an integer and tests/test_predict_long.py has the four named window_batch_cap failures; both pass on the merged tree, and tests/test_batch.py (105) also passes under a 3.12 interpreter with vegaml importable, which is the environment CI installs.

One side effect worth knowing for future PRs: adding check_batch_size to the import block in laya/agent.py moved every line below it by one, and tests/test_doc_citations.py caught the one laya/agent.py:NNNN citation that names a symbol beside it. I moved that citation in a follow-up commit; nothing for you to do.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants