Repository navigation
feat(agent): let laya.load reach the compile option Agent already takes - #617
Merged
Merged
Conversation
laya.load names 14 of Agent.__init__'s 15 parameters and forwards each by hand; `compile` is the one it omits, so the torch.compile path is reachable only by constructing Agent directly -- while every guide shows load(). Forward it, and say what it does in both docstrings: Agent's own never mentioned it, which is why the default looks arbitrary. Measured on the laya checkpoint, CPU, 4 threads, 8 calls per workload, comparing decisions rather than floats: repetitive (one shape) 962.0 ms eager -> 907.1 ms compiled, 8/8 same labels varied (8 shapes) 1312.6 ms eager -> 15440.3 ms compiled, 8/8 same labels plus a 14.3 s first-call graph build. So the graphs pay off only when the request shape repeats, which is what the docstring now says. Gates: tests/test_download.py derives the parity from both signatures, so the next Agent option that is not forwarded fails there rather than silently; tests/test_hooks_api.py pins the name and default as the contract suite does.
Owner
|
Thanks @aashish254, merged. It ships in 0.3.21. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
laya.loadis a hand-written pass-through ofAgent.__init__. It names 14 of theconstructor's 15 parameters and forwards each one;
compileis the name it leaves out.So the
torch.compilepath exists on the class, is printed in the signature block indocs/hooks/api.md, and cannot be requested from the entry point that every guiderecommends (
docs/structured.md,docs/hooks/*):That diff is empty apart from this one name, so it is not a design decision about what a
loader should expose — the copy is just one parameter short of the thing it wraps.
Agent.__init__'s own docstring never describedcompileeither, so the reason itsdefault is
Falsewas only discoverable by reading the two comments at the use site.This adds those words to both docstrings.
Why it is worth having
Not because compiled inference is a win here — it usually is not, see below — but because
laya.load(path, fast=True)andlaya.load(path, subfolder=...)are one call, whilecompile=Trueforces a caller onto a different constructor for no reason. A caller whoread
docs/hooks/api.mdand typedlaya.load(..., compile=True)gets aTypeErrorwithno hint that
Agent(...)would have accepted it.This composes with #576, which changes what
compile=Truedoes (a recompile-freetorch.compilewrapper inlaya/_compile.py). Different half of the same option: that PRmakes the path cheaper, this one makes it reachable from
load(). The parameter name andposition are unchanged by either, so they do not collide — #576's hunks in
laya/agent.pyare at 11/233/387/710, and this one is at
load().Measured (CPU, real
layacheckpoint, torch 2.14, 4 threads, 8 calls per workload)Two workloads because the whole question is shape repetition. Same
Agentcode path, oneprocess per mode so dynamo state cannot leak, decisions compared as
choice label + noul score rounded to 0.01rather than raw floats:compile=Truelen=12x8)len3…120)Plus a one-off 14.33 s first-call graph build, and a second graph for the first new
shape (dynamo went from 1 graph to 2 across the varied run).
So: 16/16 identical decisions — the option does not change what the model answers — and
the timing is the reason
compileis off by default and has to stay opt-in. On a servicethat answers the same question set over and over it is worth ~6% once a caller has paid
14 s at startup; on traffic whose request shape moves, one recompile swallows the whole
run. That is what the added docstring says now, in place, instead of in a comment 100
lines away from the parameter.
Numbers are from one run each on a laptop CPU; the direction (compile pays off only under
repeated shapes) is what #576 measures on an RTX 4070 too, and its
benchmarks/bench_compile.pyis the right instrument for a firmer graph-count number.
Gate
tests/test_download.py(tiny local weights, no network, runs in the CI tests job):test_load_signature_names_every_agent_option— derives the claim frominspect.signatureof both functions rather than writing a list down, so it fails forthe next Agent option someone forgets to forward, not only for
compile.test_compile_true_wraps_the_model_loaded_by_load— the seam itself:load()returns anAgent whose
modelis anOptimizedModuleand whose encoder config carriesreference_compile, where the default returns neither. Nothing is actually compiled, sothe check stays cheap (
torch.compilewraps eagerly and specializes on the firstforward): the whole file goes from 6 to 7 tests in 0.5 s.
tests/test_hooks_api.pygetscheck_param("load", load, "compile", False)next to theother constructor parameters it already pins, per AGENTS.md.
Mutation checks, each one reverted after running:
compile=compilefrom the forwarding call, keep the keywordtest_compile_true_...fails:AssertionError: False is not truelaya.load cannot set: compile;load/compile: missing parameterin hooks_apiTrueload/compile default: got True, want Falseand the fixture's own state_dict test — a default flip is not silentCI gates run locally:
ruff check laya/ --select=E9,F63,F7,F82,F401,F811 --line-length=120clean,
python -m compileall -q laya/ tests/clean,test_download.py7 passed /8 subtests,
test_hooks_api.py132 passed / 0 failed.Overlap
compile=Truecosts. Complementary, no shared lines.calibrationparameter to the same signature, afterhooks_timeout. Whichever of the two lands second takes a two-line conflict in theparameter list; both names belong, so resolution is mechanical. This branch deliberately
puts
compilewhereAgentputs it (right afterfast) rather than at the end, so thetwo signatures keep the same order.
No behaviour changes for any existing call: the default stays
False. One thing to callout rather than gloss —
compilegoes whereAgentputs it, right afterfast, whichshifts every later parameter one place to the right for a caller that passes them
positionally (
revisionis now the 7th name, not the 6th). Every call site in this repo,the docs and the tests uses keywords, and matching the constructor's order seemed worth
more than preserving a positional order that
Agentitself does not use — but say so ifyou would rather it went at the end, after
hooks_timeout, and that is a one-line move.