Skip to content

fix(agent): don't rewrite tokenizer_config through the HuggingFace cache symlink - #107

Merged
NandhaKishorM merged 3 commits into
NandhaKishorM:mainfrom
emiliano-go:fix/hub-cache-symlink
Sep 23, 2026
Merged

NandhaKishorM merged 3 commits into
NandhaKishorM:mainfrom
emiliano-go:fix/hub-cache-symlink

Conversation

@emiliano-go

@emiliano-go emiliano-go commented Sep 21, 2026 •

Copy link
Copy Markdown
Contributor

What

_fix_tokenizer_config patches tokenizer/tokenizer_config.json in place to keep the
checkpoint loadable across transformers versions. huggingface_hub.snapshot_download
lays snapshots out as symlinks into a shared blobs/ store, so open(path, "w")
truncates the shared blob rather than a local file.

Why it matters

Writing through the link mutates a file shared with other revisions and other processes,
races the first concurrent load (e.g. multiple server workers all calling Agent(...) at
once), and leaves the hub cache metadata (refs/etag) out of sync with the file on disk.

Reproduction

import os, tempfile
from laya.agent import _fix_tokenizer_config

root = tempfile.mkdtemp()
blob = os.path.join(root, "models--x", "blobs", "deadbeef")
os.makedirs(os.path.dirname(blob))
open(blob, "w").write('{"tokenizer_class": "TokenizersBackend", "backend": "x", "is_local": true}')
snap = os.path.join(root, "models--x", "snapshots", "rev1", "tokenizer")
os.makedirs(snap)
os.symlink(blob, os.path.join(snap, "tokenizer_config.json"))

_fix_tokenizer_config(os.path.dirname(snap))
print(open(blob).read())
# before this fix: {"tokenizer_class": "PreTrainedTokenizerFast"}  <- shared blob destroyed
# after:            {"tokenizer_class": "TokenizersBackend", "backend": "x", "is_local": true}

Change

if changed:
    # snapshots are symlinks into a shared blob store; detach the local file first
    if os.path.islink(cfg_file):
        os.unlink(cfg_file)
    with open(cfg_file, "w") as f:
        json.dump(tcfg, f, indent=2)

Also: the function wrapped everything in except Exception: pass. If the patch fails (for
example an unreadable or malformed config), that silently skips it and AutoTokenizer then
fails later with no hint that the config was the cause. It now emits a RuntimeWarning
naming the file and the reason, and leaves the file untouched.

Verification

  • python tests/test_tokenizer_cache.py (new): asserts the blob is byte-identical, the
    symlink is replaced by a regular file, the local file carries the patched value, and a
    malformed config warns instead of raising.
  • python tests/test_router.py (150), python tests/test_criteria.py (34).
  • ruff check laya/ --select=E9,F63,F7,F82,F401,F811 --line-length=120, compileall.

@NandhaKishorM

Copy link
Copy Markdown
Owner

Thank you, this is a real bug: writing through the snapshot symlink mutates the shared blob that other revisions and processes read. Today's main moved a lot (35 PRs merged), so this now conflicts. Could you rebase onto the latest main? agent.py changed today (#195, #183, #144). I'd like to land this.

HuggingFace snapshots are symlinks into a shared blobs/ store, so
open(path, "w") truncated the shared blob: it mutated a file shared with
other revisions/processes, raced the first concurrent load, and desynced the
hub cache metadata. Detach the local file before writing.

Adds tests/test_tokenizer_cache.py and runs it in CI.
The broad 'except Exception: pass' hid failures: if the patch did not apply,
AutoTokenizer would fail later with a confusing error and no hint the config was
the cause. Emit a RuntimeWarning with the path and reason. Covered by a new
malformed-JSON case in tests/test_tokenizer_cache.py.
@NandhaKishorM

Copy link
Copy Markdown
Owner

Thank you for rebasing. Writing through the snapshot symlink truncates a blob shared with other revisions and processes, so detaching the local file first is the right fix, and a warning instead of a silent pass will save someone a confusing tokenizer error. Merging. A small follow-up if you have time: between the unlink and the write, a concurrent load can see no file for a moment, and writing to a temporary file then os.replace would close that window.

@NandhaKishorM
NandhaKishorM merged commit 8e63136 into NandhaKishorM:main Sep 23, 2026
14 checks passed
@emiliano-go

Copy link
Copy Markdown
Contributor Author

thanks! good call on the gap between unlink and write, i'll do the temp file + os.replace follow-up in a small PR.

baninaveen pushed a commit to baninaveen/laya that referenced this pull request Sep 24, 2026
…che symlink (NandhaKishorM#107)

* fix(agent): don't rewrite tokenizer_config through the hub-cache symlink

HuggingFace snapshots are symlinks into a shared blobs/ store, so
open(path, "w") truncated the shared blob: it mutated a file shared with
other revisions/processes, raced the first concurrent load, and desynced the
hub cache metadata. Detach the local file before writing.

Adds tests/test_tokenizer_cache.py and runs it in CI.

* fix(agent): warn when tokenizer_config cannot be patched

The broad 'except Exception: pass' hid failures: if the patch did not apply,
AutoTokenizer would fail later with a confusing error and no hint the config was
the cause. Emit a RuntimeWarning with the path and reason. Covered by a new
malformed-JSON case in tests/test_tokenizer_cache.py.

---------

Co-authored-by: NandhaKishorM <nandakishor@convaiinnovations.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants