Skip to content

docs(tl_kernels): the caller that pads these kernels is laya.fast, not base.pad_batch - #1148

Merged
NandhaKishorM merged 2 commits into
NandhaKishorM:mainfrom
aashish254:fix-tl-kernels-padding-caller
Oct 12, 2026
Merged

NandhaKishorM merged 2 commits into
NandhaKishorM:mainfrom
aashish254:fix-tl-kernels-padding-caller

Conversation

@aashish254

Copy link
Copy Markdown
Contributor

What

laya/tl_kernels.py's module docstring told the reader that the caller of these kernels is
laya.backends.base.pad_batch:

caller in `laya.backends.base.pad_batch` pads the batch dim to the next power of two
(`bucket_rows`, so a single question stays one row) and the sequence dim to a 16-token bucket
up to `DYNAMIC_MAX_L` and a 64-token bucket beyond (`bucket_tokens`)

It is not. pad_batch has exactly one production call site, laya/backends/compile.py:111 -- the AOTInductor
route -- and laya/fast.py:10 is the only module in the package that imports tl_kernels. The two sets are
disjoint: the padding these kernels actually see happens in laya/fast.py:214-219,
FastLaya._forward_unlocked, which takes its row bucket from bucket_rows (through _bucket_n, imported at
laya/fast.py:11) but inlines the token bucketing against its own class attributes (laya/fast.py:115-116)
instead of calling bucket_tokens. The docstring now names that caller, says which half is shared and which
is duplicated, and keeps pad_batch in the sentence as the compile route's copy. 6 insertions and 3 deletions.

Why

The wrong name has a working consequence. bucket_rows is shared, but the token policy exists twice --
laya/backends/base.py:21-22 and laya/fast.py:115-116, both DYNAMIC_MAX_L = 256, LONG_BUCKET = 64. A
maintainer who retunes TileLang sequence padding by editing pad_batch, bucket_tokens, or base.DYNAMIC_MAX_L
changes nothing on the TileLang route, and nothing in the tree notices: tests/test_confidence.py's existing
tl-kernels gate checks the docstring's wording ("power of two", "16-token bucket", bucket_rows,
bucket_tokens) and the row bucketing, never who calls what. The duplicated constants are also why the
docstring could be wrong for so long while still describing the right numbers -- it does describe the same
policy on both routes, it just attributes it to a function on the other one.

No behaviour changes here, and none was claimed: tilelang is not installed on this machine and the fast path
needs CUDA, so every check below is AST- or arithmetic-level and runs CPU-only.

How it was verified

Call graph read out of the AST, no tilelang import (/tmp/probe_tl_caller.py):

tree: /Users/aashish/github contribution /GIT/laya-tl-caller
production callers of `pad_batch`   -> {'laya/backends/compile.py': [111]}
production importers of `tl_kernels`-> {'laya/fast.py': [10]}
FastLaya class constants -> {'DYNAMIC_MAX_L': 256, 'LONG_BUCKET': 64}
_forward_unlocked calls bucket_tokens?  False
_forward_unlocked pads rows via _bucket_n? True
_forward_unlocked token line: ['g = 16 if L0 <= self.DYNAMIC_MAX_L else self.LONG_BUCKET', 'L = min(self.max_len, (L0 + g - 1) // g * g)']
laya/backends/base.py DYNAMIC_MAX_L, LONG_BUCKET -> 256 64
all equal: True
bucket_rows(1..5) -> [1, 2, 4, 4, 8]

The all equal: True line is base.bucket_tokens(L0, 1024) compared against the inlined expression for
L0 in (1, 3, 16, 17, 255, 256, 257, 300, 1000, 1025) -- every row printed as (L0, inline, base, equal)
and all ten matched, so the docstring is describing one policy that happens to live in two places, which is
what makes the duplication worth a tripwire rather than a correction.

Three tests in tests/test_confidence.py, added to the script runner alongside the existing tl-kernels gates:

  • test_tl_kernels_names_a_caller_that_is_on_the_kernels_route -- derives the importer set and the caller set
    from laya/** by AST and requires them to be exactly {laya/fast.py} and {laya/backends/compile.py} and
    disjoint; then bans the "caller … pad_batch" phrasing and requires the docstring to name laya.fast and
    _forward_unlocked. The two set equalities fail with a message that tells the reader to re-derive rather
    than to edit the test.
  • test_the_fast_path_still_inlines_the_token_bucket_the_docstring_says_it_repeats -- ast.unparses
    _forward_unlocked and requires self.DYNAMIC_MAX_L/self.LONG_BUCKET to be used there, bucket_tokens(
    to be absent, and the row pad to come from bucket_rows (through _bucket_n or directly).
  • test_the_two_bucket_policies_the_docstring_describes_are_still_the_same_numbers -- reads
    FastLaya.DYNAMIC_MAX_L/LONG_BUCKET as class literals (no import, because this machine cannot import
    laya.fast) and requires them to equal laya.backends.base's, then requires the docstring to print the
    bucket numbers the code uses (16-token bucket, and %d-token bucket from LONG_BUCKET).

Seven mutants, each applied to the committed tree and reverted before the next (git checkout -- per file):

mutant result
docstring names TileLangBackend as the caller rc=1 KILLED -- 222 passed, 1 failed
docstring reverts to "caller in laya.backends.base.pad_batch" rc=1 KILLED
docstring's large bucket changed 64 -> 128 rc=1 KILLED
FastLaya.DYNAMIC_MAX_L = 512, docstring untouched rc=1 KILLED
_forward_unlocked drops the class constants (g = 16) rc=1 KILLED
laya/fast.py gains a pad_batch call rc=1 KILLED
compile.py stops calling pad_batch rc=1 KILLED

The last two are the ones that make the first test more than a string match: a future PR that puts pad_batch
back on the kernels' route fails here instead of silently making the docstring true again by accident.

Gates, in this worktree:

$ PYTHONDONTWRITEBYTECODE=1 ../laya/.venv/bin/python -B scripts/test_suites.py
FAIL: 0            (x4, no non-zero FAIL line)
497 passed, 1 skipped, 5 warnings in 15.89s
all 87 suites passed
$ ../laya/.venv/bin/python -B tests/test_confidence.py
223 passed, 0 failed
all confidence tests passed
$ ruff check laya/ --select=E9,F63,F7,F82,F401,F811 --line-length=120
All checks passed!
$ ../laya/.venv/bin/python -B -m compileall -q laya/ tests/
(exit 0, no output)

tests/test_confidence.py is a script suite: pytest collects nothing from it (SystemExit: 0 at import, the
same on upstream/main), so the 220 -> 223 count comes from running it directly. ruff check tests/ is outside
the CI lint path and was not run.

Diff scan for secrets: the broad pattern (api[_-]?key|secret|token|password|bearer|sk-…|ghp_|AKIA|hf_…) gives
17 hits, and every one is the word token inside bucket_tokens, token bucket or token dim. The
credential-shaped pattern ((api_key|secret|password|bearer|ghp_|AKIA|hf_)[A-Za-z0-9_-]* [:=] <value>) gives
0 hits. No value, no key material.

Deconfliction: laya/tl_kernels.py, laya/backends/base.py and laya/backends/compile.py are touched by no
open PR. tests/test_confidence.py is touched only by #1119, whose single hunk in that file is
@@ -281,6 +281,26 @@; this change's hunk there is @@ -2041,9 +2041,149 @@, so no shared line. The three
files the gate reads but does not edit are laya/fast.py, laya/backends/base.py and
laya/backends/compile.py; of those only #921 touches laya/fast.py, and its one hunk there is
@@ -30,6 +30,8 @@ (the FastLaya.__init__ signature area, well clear of the constants at :115-116 and the
padding at :214-219) -- gh pr diff 921 matches none of DYNAMIC_MAX_L, LONG_BUCKET, bucket_rows,
bucket_tokens, _bucket_n or pad_batch.

Branch has two commits: bb973e02 (the docstring and the gate) and c3b0d312 (one wording tightening, plus
letting the row check accept a direct bucket_rows call as well as _bucket_n, so a faithful refactor is not
punished for naming the helper it inlined).

Checklist

  • Focused on one change (split unrelated work into another PR)
  • Rebased on the latest main
  • Tests pass locally
  • Docs or examples updated when the public API changed

…t base.pad_batch

pad_batch has one production call site (laya/backends/compile.py:111, the compile route);
laya/fast.py is the only module that imports tl_kernels, and FastLaya._forward_unlocked does
its own bucketing -- bucket_rows for rows, an inlined copy of bucket_tokens' arithmetic against
FastLaya.DYNAMIC_MAX_L/LONG_BUCKET for tokens. Retune pad_batch for TileLang and nothing on the
TileLang route changes, so name the real caller and gate the call graph, the inlining and the
duplicated constants off the AST (no tilelang import, so this runs CPU-only).
@NandhaKishorM
NandhaKishorM merged commit 0fe4065 into NandhaKishorM:main Oct 12, 2026
40 checks passed
@NandhaKishorM

Copy link
Copy Markdown
Owner

Merged. The attribution was false in a way nothing could catch, and the AST checks are the right way to keep it true.

Confirmed both sets: laya/fast.py is the only module in the package that imports tl_kernels, and pad_batch has one production call site in laya/backends/compile.py. Those being disjoint is exactly what made the sentence wrong, so a maintainer retuning pad_batch or bucket_tokens for TileLang padding would have changed nothing on the TileLang route.

The second check is the better one. _forward_unlocked takes its row bucket from bucket_rows through _bucket_n but inlines the token bucketing against its own FastLaya.DYNAMIC_MAX_L/LONG_BUCKET, so there are two copies of one policy, and the third check asserts the constants still agree, which is the only condition under which one sentence can describe both routes. Reading all of it out of the AST means it runs on a CPU-only box where the fast path cannot execute, which is where most of this repo's contributors are.

Pinning the ban as a regex against the docstring is slightly unusual and justified here: it catches the specific wrong sentence coming back, and the error text explains the call graph rather than just failing.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants