Repository navigation
fix(agent): write tokenizer_config atomically via a temp file + os.replace - #267
Merged
NandhaKishorM merged 1 commit intoSep 23, 2026
Conversation
…place _fix_tokenizer_config detached the snapshot symlink and then wrote the file in place, so a concurrent load could see no config for a moment. Write to a temp file in the same directory and os.replace it, which swaps the snapshot entry atomically and never exposes a missing or half-written file. The temp file is removed if the swap fails, and the original file mode is preserved.
Owner
|
Thank you for the quick follow-up to #107. Writing to a temp file and |
baninaveen
pushed a commit
to baninaveen/laya
that referenced
this pull request
Sep 24, 2026
…place (NandhaKishorM#267) _fix_tokenizer_config detached the snapshot symlink and then wrote the file in place, so a concurrent load could see no config for a moment. Write to a temp file in the same directory and os.replace it, which swaps the snapshot entry atomically and never exposes a missing or half-written file. The temp file is removed if the swap fails, and the original file mode is preserved.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
Follow-up to #107, as NandhaKishorM suggested.
_fix_tokenizer_configdetached the snapshot symlink and then wrote the file in place, so between the unlink and the write a concurrent load could see no config at all.Now it writes to a temp file in the same directory and
os.replaces it onto the snapshot entry. The replace is atomic, so a reader only ever sees the old file or the complete new one, and the entry becomes a regular file so the shared blob is still untouched. If the swap fails, the temp file is removed and the original is left as it was. The file mode is preserved too (mkstemp creates 0600, the cache file is usually 0644).Verification
tests/test_tokenizer_cache.py(14 checks, no weights downloaded):.tokenizer_config.*.tmpleft behindos.replacefailure warns, cleans up the temp file, and leaves the symlink and blob intactAlso ran
test_lazy_import,test_load_errors,test_download,test_criteria,test_router,test_packaging, plus ruff and compileall.