Skip to content

feat(agent): let laya.load reach the compile option Agent already takes - #617

Merged
NandhaKishorM merged 1 commit into
NandhaKishorM:mainfrom
aashish254:feat-load-compile
Sep 27, 2026
Merged

NandhaKishorM merged 1 commit into
NandhaKishorM:mainfrom
aashish254:feat-load-compile

Conversation

@aashish254

Copy link
Copy Markdown
Contributor

What

laya.load is a hand-written pass-through of Agent.__init__. It names 14 of the
constructor's 15 parameters and forwards each one; compile is the name it leaves out.
So the torch.compile path exists on the class, is printed in the signature block in
docs/hooks/api.md, and cannot be requested from the entry point that every guide
recommends (docs/structured.md, docs/hooks/*):

Agent.__init__ params : 15
load() params         : 14
unreachable via load  : ['compile']
load-only             : []

>>> laya.load(path, compile=True)
TypeError: load() got an unexpected keyword argument 'compile'

That diff is empty apart from this one name, so it is not a design decision about what a
loader should expose — the copy is just one parameter short of the thing it wraps.

Agent.__init__'s own docstring never described compile either, so the reason its
default is False was only discoverable by reading the two comments at the use site.
This adds those words to both docstrings.

Why it is worth having

Not because compiled inference is a win here — it usually is not, see below — but because
laya.load(path, fast=True) and laya.load(path, subfolder=...) are one call, while
compile=True forces a caller onto a different constructor for no reason. A caller who
read docs/hooks/api.md and typed laya.load(..., compile=True) gets a TypeError with
no hint that Agent(...) would have accepted it.

This composes with #576, which changes what compile=True does (a recompile-free
torch.compile wrapper in laya/_compile.py). Different half of the same option: that PR
makes the path cheaper, this one makes it reachable from load(). The parameter name and
position are unchanged by either, so they do not collide — #576's hunks in laya/agent.py
are at 11/233/387/710, and this one is at load().

Measured (CPU, real laya checkpoint, torch 2.14, 4 threads, 8 calls per workload)

Two workloads because the whole question is shape repetition. Same Agent code path, one
process per mode so dynamo state cannot leak, decisions compared as
choice label + noul score rounded to 0.01 rather than raw floats:

workload shapes eager compile=True decisions changed
repetitive 1 (len=12 x8) 962.0 ms 907.1 ms (0.94x) 0/8
varied 8 (len 3…120) 1312.6 ms 15440.3 ms (11.8x) 0/8

Plus a one-off 14.33 s first-call graph build, and a second graph for the first new
shape (dynamo went from 1 graph to 2 across the varied run).

So: 16/16 identical decisions — the option does not change what the model answers — and
the timing is the reason compile is off by default and has to stay opt-in. On a service
that answers the same question set over and over it is worth ~6% once a caller has paid
14 s at startup; on traffic whose request shape moves, one recompile swallows the whole
run. That is what the added docstring says now, in place, instead of in a comment 100
lines away from the parameter.

Numbers are from one run each on a laptop CPU; the direction (compile pays off only under
repeated shapes) is what #576 measures on an RTX 4070 too, and its benchmarks/bench_compile.py
is the right instrument for a firmer graph-count number.

Gate

tests/test_download.py (tiny local weights, no network, runs in the CI tests job):

  • test_load_signature_names_every_agent_option — derives the claim from
    inspect.signature of both functions rather than writing a list down, so it fails for
    the next Agent option someone forgets to forward, not only for compile.
  • test_compile_true_wraps_the_model_loaded_by_load — the seam itself: load() returns an
    Agent whose model is an OptimizedModule and whose encoder config carries
    reference_compile, where the default returns neither. Nothing is actually compiled, so
    the check stays cheap (torch.compile wraps eagerly and specializes on the first
    forward): the whole file goes from 6 to 7 tests in 0.5 s.

tests/test_hooks_api.py gets check_param("load", load, "compile", False) next to the
other constructor parameters it already pins, per AGENTS.md.

Mutation checks, each one reverted after running:

mutation result
drop compile=compile from the forwarding call, keep the keyword test_compile_true_... fails: AssertionError: False is not true
remove the parameter and the forwarding (i.e. revert the fix) both new tests fail, the derived one naming laya.load cannot set: compile; load/compile: missing parameter in hooks_api
flip the default to True 3 failures, including load/compile default: got True, want False and the fixture's own state_dict test — a default flip is not silent

CI gates run locally: ruff check laya/ --select=E9,F63,F7,F82,F401,F811 --line-length=120
clean, python -m compileall -q laya/ tests/ clean, test_download.py 7 passed /
8 subtests, test_hooks_api.py 132 passed / 0 failed.

Overlap

No behaviour changes for any existing call: the default stays False. One thing to call
out rather than gloss — compile goes where Agent puts it, right after fast, which
shifts every later parameter one place to the right for a caller that passes them
positionally (revision is now the 7th name, not the 6th). Every call site in this repo,
the docs and the tests uses keywords, and matching the constructor's order seemed worth
more than preserving a positional order that Agent itself does not use — but say so if
you would rather it went at the end, after hooks_timeout, and that is a one-line move.

laya.load names 14 of Agent.__init__'s 15 parameters and forwards each by
hand; `compile` is the one it omits, so the torch.compile path is reachable
only by constructing Agent directly -- while every guide shows load().

Forward it, and say what it does in both docstrings: Agent's own never
mentioned it, which is why the default looks arbitrary.

Measured on the laya checkpoint, CPU, 4 threads, 8 calls per workload,
comparing decisions rather than floats:
  repetitive (one shape)  962.0 ms eager -> 907.1 ms compiled, 8/8 same labels
  varied     (8 shapes)  1312.6 ms eager -> 15440.3 ms compiled, 8/8 same labels
plus a 14.3 s first-call graph build. So the graphs pay off only when the
request shape repeats, which is what the docstring now says.

Gates: tests/test_download.py derives the parity from both signatures, so the
next Agent option that is not forwarded fails there rather than silently;
tests/test_hooks_api.py pins the name and default as the contract suite does.
@NandhaKishorM
NandhaKishorM merged commit 3616158 into NandhaKishorM:main Sep 27, 2026
22 checks passed
@NandhaKishorM

Copy link
Copy Markdown
Owner

Thanks @aashish254, merged. It ships in 0.3.21.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants