Repository navigation
perf(router): bypass predict context and OS entropy when no hooks are active - #1386
Conversation
f602705 to
31e3bac
Compare
24 PRs. Vietnamese kept off the English checkpoint (#1392), deciding-window attribution preserved through end hooks (#1369), RemoteRouter identifying itself with a configurable User-Agent (#1396), ten more inputs refused where they enter (#1372, #1394, #1397, #1398, #1400, #1401, #1402, #1403, #1404, #1405, #1406), per-language temperatures and per-call budgets in .NET and Rust (#1370, #1371), .NET calibration files (#1384), the Go router lifecycle, SHA-256 pins and routed inference (#1374, #1379, #1395), a C++ foundation (#1399, #1407), and parity-lane sharding that replaces the budget headroom added for 0.4.16 (#1373, #1375). Verified before tagging: all 92 suites pass, 640 tests, up from 612. ruff clean on both lists. laya-ts 1251 tests across 44 files, tsc --noEmit clean. Every generated-artifact gate clean on CPython 3.12 / unicodedata 15.0.0: Java 20 families match laya (9 unverified), UnicodeTables matches CPython 15.0.0, LanguageTables matches laya/lang.py, three Go tables_gen.go match, both laya-cpp .inc files match. Version agrees across pyproject.toml, laya.__version__ and nix/package.nix. README carries 0.4.17 only. Security audit of the merge diff, 362766 added lines over 480 files: zero obfuscation, zero secrets, zero boundary hits, zero binaries, no mode or symlink changes, 20 of 20 actions SHA-pinned including the new cpp.yml, npm install hooks unchanged, and no third-party host in any added line (example.com 90, github.com 2, download.pytorch.org 1, docs.rs 1). The 12 dynamic-exec hits are the C++ test harness invoking its own checker through sys.executable, the Go sample generator reading laya's own example by AST at development time, and one comment in rust.yml. Held: #1364 (unchanged, it moves a cross-port contract), #1381 (a #[doc(hidden)] but pub Rust hook that would ship as public API able to replace the forward pass), #1386 (route_batch switched from self.route to self._route, silently bypassing subclass overrides the docstring promises), and nine stacked port PRs that conflict with each other and need linearizing (#1377, #1378, #1380, #1382, #1383, #1385, #1387, #1388, #1389, #1390). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
Holding this, and as with #1381 the performance work is sound and I would like it in. Driving the The blocker is one line. Nothing in the tree catches this. The fix I would suggest is cheap: resolve it once per call rather than per item. Then take Separately, and not a blocker: the request validation is now duplicated across the two branches, about fifteen lines. Once the override check above decides which path to take, that validation should be able to sit in one place ahead of the split. A test with a subclass that overrides Please also say in the PR description that |
31e3bac to
a6fbbbc
Compare
a6fbbbc to
79abd9c
Compare
|
updated! |
|
Merged, and I owe you an account of the review because I nearly rejected this on a mistake of my own. I first read the diff, then merged a local The cause looked damning: That was a stale checkout, not your code. My On your actual head the guard is already there: use_fast = type(self).route is Router.routewhich is the right test, and it is the thing I would have asked for. Full gate on the correct head: 785 tests across 93 suites, exit 0, and the MCP diagnostics suite green. What I verified beyond the suite, because the whole change rests on
And the behaviour the gate protects: a subclass overriding The unwrap loop rather than a single Shipping in 0.4.19. |
|
haha no worries at all those silent git fails happen to everyone... glad everything checked out with the hook edge cases and thanks for merging! |
What
in
laya/router.py:on_routebefore allocating aPredictContext.on_routehooks are active (standard path), skipsuuid.uuid4().hex(reading OS entropy via/dev/urandomsyscall) andtime.perf_counter(), directly returning the routing decision.route_batch.Why
in
Router.routeandroute_batch, every request previously instantiated a fullPredictContext. on Linux,uuid.uuid4()invokes thegetrandom(2)kernel syscall to read 16 bytes of entropy from/dev/urandom.when zero
on_routehooks are registered (the default for production inference), generating random UUIDs and timer timestamps is completely unused overhead.bypassing context creation when hooks are idle eliminates all
/dev/urandomsyscalls and speeds up batch routing by ~12% to 19% (and single pinned routes by 2.1x), with zero changes to routing decisions or hook lifecycle behavior.tested on
Router.route_batchbefore and after (no hooks registered):/dev/urandomSyscallsHow it was verified
all passed:
use_fast = type(self).route is Router.routeonce per call so subclasses overriding route retain their overrides duringroute_batch.route_batchinvokes subclass route overrides.