Skip to content

Skip redundant initialization when loading checkpoints - #195

Merged
NandhaKishorM merged 1 commit into
NandhaKishorM:mainfrom
SharkyRawr:fix/skip-checkpoint-initialization
Sep 23, 2026
Merged

NandhaKishorM merged 1 commit into
NandhaKishorM:mainfrom
SharkyRawr:fix/skip-checkpoint-initialization

Conversation

@SharkyRawr

@SharkyRawr SharkyRawr commented Sep 23, 2026 •

Copy link
Copy Markdown
Contributor

Loading a Laya checkpoint currently initializes random encoder and decision-head weights immediately before overwriting them with checkpoint weights. If the checkpoint has no bundled encoder config, it also loads base-model weights that are subsequently discarded.

Wrap inference model construction in Transformers' no_init_weights() and construct the encoder from its config when loading a full Laya checkpoint. Keep build_model()'s existing pretrained default for other callers and retain strict checkpoint validation and loading. Support both Transformers 4.x and the newer initialization-module import path.

Before / after profiling

Compared the parent commit (c752770) with this fix (5170536) using the same cached convaiinnovations/laya checkpoint. Measurements are medians of three fresh Python processes per version, alternating before/after runs. CPU, four Torch threads, Python 3.14.7, PyTorch 2.14.0, Transformers 5.17.0; network access disabled. Timed with time.perf_counter().

Phase Before After
Model construction (build_model) 29.56 s 0.070 s
Checkpoint weight copying (load_state_dict) 0.60 s 0.110 s
Complete laya.load() call 32.34 s 2.08 s

That is approximately 30 seconds saved per load, or 15.5× faster loading in this benchmark. Total loading times ranged from 30.51–33.45 s before and 1.73–2.45 s after. The total includes lazy imports performed inside laya.load(), but excludes initial imports of Torch and Laya and excludes inference. Phase medians need not sum to the total; weight-copy timings also varied between runs. The change targets checkpoint loading, not inference speed.

Validation

All 5 offline checkpoint/download tests pass (python tests/test_download.py). The new regression covers bundled and external encoder configs, checks that loading does not consume Torch RNG state or load base-model weights, verifies exact checkpoint tensor equality, and runs inference.

@NandhaKishorM

Copy link
Copy Markdown
Owner

Thank you, this is a great find, and a very small diff for what it buys.

I checked it against the real English checkpoint before merging: 396 labelled decisions (emotion, tweet topic, prompt injection), and every answer and every probability is bit-identical to main, while loading drops from 22.0 s to 1.5 s on CPU. It also skips exactly the weight-init pass that segfaults on Windows with Python 3.14 (#123), so it very likely fixes that too. Merging, thanks again.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants