Repository navigation
fix(router): accept and compose per-call hooks in predict_batch and route_batch (#909) - #912
Conversation
Bruce-Yii
left a comment
There was a problem hiding this comment.
Confirmed #909 is real, but this head introduces a public signature break: current main has predict_batch(requests, batch_size=None, hooks_timeout=None, min_confidence=None, sort_by_length=False). Here hooks_timeout moves behind * after min_confidence/sort_by_length, so an existing predict_batch(reqs, None, 1.0) silently changes from hooks_timeout=1.0 to min_confidence=1.0 (RemoteRouter too). Please preserve the existing positional prefix. Also #917 implements the same issue, so only one branch should own this fix; this should not merge until that collision is resolved.
|
"Thanks for catching that @Bruce-Yii! Positional prefix restored: We restored hooks_timeout to its original 3rd position ahead of min_confidence and sort_by_length (requests, batch_size, hooks_timeout, min_confidence, sort_by_length). All new hook arguments (hooks, on_predict_start, on_predict_end, hooks_raise) are strictly keyword-only to protect positional callers. Collision with #917: This PR (#912) is a superset of #917: Implements full lifecycle hooks (on_predict_start, on_predict_end, hooks_raise, and hooks), whereas #917 only adds hooks. |
|
Merged for 0.3.27, closing #909. Two PRs arrived for this one too, and the decision was close. @sathariels's #917 makes the same fix with a narrower surface ( I took yours because of the pattern this repository has been following all week. #763, #766, #770, #797, #798 and #804 each widened a surface to forward exactly what the single-request path forwards, and I did check the ordering property #917's comment raises, because it would have decided it the other way if it were real. Probed both heads with a malformed per-call hook on Identical, and both raise before a checkpoint loads, so there was no behavioural difference to choose on. Carrying the same change through Verified: Thank you. |
Fix #909
What
predict_batch: Addedhooks=None,on_predict_start=None,on_predict_end=None,hooks_raise=None, andhooks_timeout=NonetoRouter.predict_batch.active = compose_hooks(self.hooks)withactive = compose_hooks(self.hooks, hooks, on_predict_start, on_predict_end)so per-call hooks and convenience callables execute per request during batch forward passes.route_batch: Addedhooksandhooks_raiseforwarding toRouter.route_batch, passing them to each request'sself.route(...)invocation.Router.route_batch: Retainsrequestsandhooks_timeoutpositionally, with*, hooks=None, hooks_raise: Optional[bool] = Noneas keyword-only to preserve existing positional caller bindings (route_batch(requests, 1.0)).Router.predict_batch: Retains the existing positional parameter prefix (requests,batch_size,min_confidence,sort_by_length) followed by keyword-only hook controls:*, hooks_timeout=None, hooks=None, on_predict_start=None, on_predict_end=None, hooks_raise=None.laya/mcp/remote.py):RemoteRouter.predict_batchto mirrorRouter.predict_batch's keyword-only signature 1:1.hooks,on_predict_start,on_predict_end,hooks_raise,hooks_timeout) over HTTP withRemoteError("unsupported_remote", ...).tests/test_hooks_api.py):Router.predict_batchandRouter.route_batchto the public hook parameter contract checks.inspect.Parameter.KEYWORD_ONLYon hook controls.tests/test_hooks.py):predict_batchandroute_batchper-call hook execution.on_predict_start/on_predict_endplain callables.hooks_raise=Trueerror propagation andhooks_raise=Falsewarning suppression.TypeErrorrejection when passing keyword-only hook controls positionally.Why
The
Router.predict_batchdocstring documented accepting router-level per-call hooks ("Router-level predict hooks run per request, as predict runs them..."), but the method definition omittedhooks,on_predict_start,on_predict_end, andhooks_raisefrom its signature. Passing documented hook parameters raised:Inside
predict_batch, hook composition was hardcoded toactive = compose_hooks(self.hooks). Even if hooks reached the instance, per-call overrides were dropped, androute_batchomitted per-call hook propagation intoself.route.Placing new parameters positionally risked shifting established positional arguments (e.g.
route_batch(requests, 1.0)orpredict_batch(requests, 8, 1.0)). Declaring new hook parameters as keyword-only (*) guarantees existing positional callers continue functioning identically without silent misbehavior.Technical Details
Router.route_batch(self, requests, hooks_timeout=None, *, hooks=None, hooks_raise=None)Router.predict_batch(self, requests, batch_size=None, min_confidence=None, sort_by_length=False, *, hooks_timeout=None, hooks=None, on_predict_start=None, on_predict_end=None, hooks_raise=None)RemoteRouter.predict_batchRouter.predict_batch1:1; raisesRemoteError("unsupported_remote", ...)if any hook argument is provided(self.hooks, hooks, on_predict_start, on_predict_end)and forwards intoroute_batchHow It Was Verified
Checklist
AGENTS.mdand repository guidelinesfix(router): accept and compose per-call hooks in predict_batch and route_batch (#909)tests/test_hooks_api.pytests/test_hooks.pyruff check,python -m compileall,test_packaging.py)