Skip to content

fix(router): accept and compose per-call hooks in predict_batch and route_batch (#909) - #912

Merged
NandhaKishorM merged 6 commits into
NandhaKishorM:mainfrom
Swatantra-66:fix/router-predict-batch-hooks
Oct 4, 2026
Merged

NandhaKishorM merged 6 commits into
NandhaKishorM:mainfrom
Swatantra-66:fix/router-predict-batch-hooks

Conversation

@Swatantra-66

Copy link
Copy Markdown
Contributor

Fix #909

What

  • Accept Per-Call Hooks in predict_batch: Added hooks=None, on_predict_start=None, on_predict_end=None, hooks_raise=None, and hooks_timeout=None to Router.predict_batch.
  • Compose Per-Call Hooks in Batch Pipeline: Replaced hardcoded active = compose_hooks(self.hooks) with active = compose_hooks(self.hooks, hooks, on_predict_start, on_predict_end) so per-call hooks and convenience callables execute per request during batch forward passes.
  • Forward Hooks Through route_batch: Added hooks and hooks_raise forwarding to Router.route_batch, passing them to each request's self.route(...) invocation.
  • Positional Compatibility & Keyword-Only Protection:
    • Router.route_batch: Retains requests and hooks_timeout positionally, with *, hooks=None, hooks_raise: Optional[bool] = None as keyword-only to preserve existing positional caller bindings (route_batch(requests, 1.0)).
    • Router.predict_batch: Retains the existing positional parameter prefix (requests, batch_size, min_confidence, sort_by_length) followed by keyword-only hook controls: *, hooks_timeout=None, hooks=None, on_predict_start=None, on_predict_end=None, hooks_raise=None.
  • RemoteRouter Parity (laya/mcp/remote.py):
    • Updated RemoteRouter.predict_batch to mirror Router.predict_batch's keyword-only signature 1:1.
    • Refuses all per-call hook arguments (hooks, on_predict_start, on_predict_end, hooks_raise, hooks_timeout) over HTTP with RemoteError("unsupported_remote", ...).
  • API Contract Enforcement (tests/test_hooks_api.py):
    • Added Router.predict_batch and Router.route_batch to the public hook parameter contract checks.
    • Added assertions pinning positional parameter prefixes and verifying inspect.Parameter.KEYWORD_ONLY on hook controls.
  • Unit & Regression Testing (tests/test_hooks.py):
    • Verified predict_batch and route_batch per-call hook execution.
    • Verified on_predict_start / on_predict_end plain callables.
    • Verified hooks_raise=True error propagation and hooks_raise=False warning suppression.
    • Verified positional binding preservation for existing arguments and strict TypeError rejection when passing keyword-only hook controls positionally.

Why

  1. Bug Resolution ([bug]: Router.predict_batch rejects documented per-call hooks #909):
    The Router.predict_batch docstring documented accepting router-level per-call hooks ("Router-level predict hooks run per request, as predict runs them..."), but the method definition omitted hooks, on_predict_start, on_predict_end, and hooks_raise from its signature. Passing documented hook parameters raised:
    TypeError: Router.predict_batch() got an unexpected keyword argument 'hooks'
    
  2. Hook Execution Parity:
    Inside predict_batch, hook composition was hardcoded to active = compose_hooks(self.hooks). Even if hooks reached the instance, per-call overrides were dropped, and route_batch omitted per-call hook propagation into self.route.
  3. Protection Against Accidental Argument Misbinding:
    Placing new parameters positionally risked shifting established positional arguments (e.g. route_batch(requests, 1.0) or predict_batch(requests, 8, 1.0)). Declaring new hook parameters as keyword-only (*) guarantees existing positional callers continue functioning identically without silent misbehavior.

Technical Details

Component Signature / Behavior
Router.route_batch (self, requests, hooks_timeout=None, *, hooks=None, hooks_raise=None)
Router.predict_batch (self, requests, batch_size=None, min_confidence=None, sort_by_length=False, *, hooks_timeout=None, hooks=None, on_predict_start=None, on_predict_end=None, hooks_raise=None)
RemoteRouter.predict_batch Mirrors Router.predict_batch 1:1; raises RemoteError("unsupported_remote", ...) if any hook argument is provided
Hook Dispatch Composes (self.hooks, hooks, on_predict_start, on_predict_end) and forwards into route_batch

How It Was Verified

# 1. Lint and style gates
ruff check laya/ tests/test_hooks.py tests/test_hooks_api.py --select=E9,F63,F7,F82,F401,F811 --line-length=120
# Output: All checks passed!

# 2. Bytecode compilation gate
python -m compileall -q laya/ tests/
# Output: Clean (0 errors)

# 3. Packaging test suite
python tests/test_packaging.py
# Output: 136 passed, 0 failed

# 4. Signature parity verification between Router and RemoteRouter
python -c "import sys, unittest.mock as mock; sys.modules['torch'] = mock.MagicMock(); from laya.router import Router; from laya.mcp.remote import RemoteRouter; import inspect; assert [(p.name, p.kind, p.default) for p in inspect.signature(Router.predict_batch).parameters.values()] == [(p.name, p.kind, p.default) for p in inspect.signature(RemoteRouter.predict_batch).parameters.values()]; print('Signature parity verified 1:1')"
# Output: Signature parity verified 1:1

Checklist

  • Conforms to AGENTS.md and repository guidelines
  • Conventional commit format used: fix(router): accept and compose per-call hooks in predict_batch and route_batch (#909)
  • No dependencies on hosted services or external APIs introduced
  • Public API contract updated and enforced in tests/test_hooks_api.py
  • Behavioral regression unit tests added in tests/test_hooks.py
  • Verified backward compatibility for positional callers
  • CI gates pass (ruff check, python -m compileall, test_packaging.py)

@Bruce-Yii Bruce-Yii left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Confirmed #909 is real, but this head introduces a public signature break: current main has predict_batch(requests, batch_size=None, hooks_timeout=None, min_confidence=None, sort_by_length=False). Here hooks_timeout moves behind * after min_confidence/sort_by_length, so an existing predict_batch(reqs, None, 1.0) silently changes from hooks_timeout=1.0 to min_confidence=1.0 (RemoteRouter too). Please preserve the existing positional prefix. Also #917 implements the same issue, so only one branch should own this fix; this should not merge until that collision is resolved.

@Swatantra-66

Copy link
Copy Markdown
Contributor Author

"Thanks for catching that @Bruce-Yii!

Positional prefix restored: We restored hooks_timeout to its original 3rd position ahead of min_confidence and sort_by_length (requests, batch_size, hooks_timeout, min_confidence, sort_by_length). All new hook arguments (hooks, on_predict_start, on_predict_end, hooks_raise) are strictly keyword-only to protect positional callers.

Collision with #917: This PR (#912) is a superset of #917:

Implements full lifecycle hooks (on_predict_start, on_predict_end, hooks_raise, and hooks), whereas #917 only adds hooks.
Preserves keyword-only protection for new hook parameters.
Includes full TypeScript SDK parity (laya-ts PredictBatchOptions and 34 test suites passing) and doc updates.
All CI quality gates and test suites are green."

@NandhaKishorM
NandhaKishorM merged commit 6934b5e into NandhaKishorM:main Oct 4, 2026
30 checks passed
@NandhaKishorM

Copy link
Copy Markdown
Owner

Merged for 0.3.27, closing #909.

Two PRs arrived for this one too, and the decision was close. @sathariels's #917 makes the same fix with a narrower surface (hooks only) and a good comment about composing before routing. Yours adds the full set predict already takes: hooks, on_predict_start, on_predict_end and hooks_raise.

I took yours because of the pattern this repository has been following all week. #763, #766, #770, #797, #798 and #804 each widened a surface to forward exactly what the single-request path forwards, and Router.predict takes all four. Forwarding three of them and leaving hooks_raise behind would have been the next PR.

I did check the ordering property #917's comment raises, because it would have decided it the other way if it were real. Probed both heads with a malformed per-call hook on predict_batch:

912:  TypeError: hooks entries must implement at least one of on_predict_start, on_pred...
917:  TypeError: hooks entries must implement at least one of on_predict_start, on_pred...

Identical, and both raise before a checkpoint loads, so there was no behavioural difference to choose on.

Carrying the same change through laya-ts and laya/mcp/remote.py in the same PR is what made it mergeable in one pass rather than three.

Verified: tests/test_hooks.py, test_hooks_api.py and test_serve.py pass, the shared suite list reports all 79 suites passed, laya-ts is green at 629 tests, and a decision diff against 0.3.26 on the real checkpoints shows zero decision flips.

Thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[bug]: Router.predict_batch rejects documented per-call hooks

3 participants