Repository navigation
[HfFileSystem] Fix bucket prefix collisions - #4630
Merged
hanouticelina merged 5 commits intoAug 7, 2026
Merged
Conversation
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
lewtun
marked this pull request as ready for review
August 4, 2026 21:55
Member
Author
|
The failing tests seem unrelated to this PR AFAICT |
hanouticelina
approved these changes
Aug 6, 2026
hanouticelina
left a comment
Collaborator
There was a problem hiding this comment.
The fix looks good to me, thanks!
hanouticelina
deleted the
agent/fix-hf-filesystem-bucket-prefix-collision
branch
August 7, 2026 09:05
Contributor
|
This PR has been shipped as part of the v1.27.0 release. |
Sanjays2402
added a commit
to Sanjays2402/huggingface_hub
that referenced
this pull request
Aug 27, 2026
The server matches prefix lexically, so a prefix of 'logs' also returns 'logs.json', 'logs_backup/...', and 'logsx/...'. Every other consumer in the library (HfFileSystem, _buckets._list_remote_files, hf_api.copy, cli/_skills) compensates with a path-component filter; the destructive 'remove --recursive' branch was the missing one, causing silent over-deletion. Add the same filter already used by HfFileSystem (db2c685, huggingface#4630) so that only the prefix itself and files under prefix/ are removed. The test fixture now includes lexical siblings ('logs.json', 'logs_backup/a.log', 'logsx/c.log') so the suite structurally catches the over-deletion bug. Closes huggingface#4749.
Sanjays2402
added a commit
to Sanjays2402/huggingface_hub
that referenced
this pull request
Aug 27, 2026
The server matches prefix lexically, so a prefix of 'logs' also returns 'logs.json', 'logs_backup/...', and 'logsx/...'. Every other consumer in the library (HfFileSystem, _buckets._list_remote_files, hf_api.copy, cli/_skills) compensates with a path-component filter; the destructive 'remove --recursive' branch was the missing one, causing silent over-deletion. Add the same filter already used by HfFileSystem (db2c685, huggingface#4630) so that only the prefix itself and files under prefix/ are removed. The test fixture now includes lexical siblings ('logs.json', 'logs_backup/a.log', 'logsx/c.log') so the suite structurally catches the over-deletion bug. Closes huggingface#4749.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Issue
The Buckets API applies
prefixlexically, butHfFileSystemneeds path-component semantics. If a bucket containslogs_existing/result.txtbut nologs/, listinglogsreturns the sibling.HfFileSystemthen caches it under its real parent, andexists(".../logs/new-result.txt")raisesKeyErrorinstead of returningFalse;ls(".../logs")can also return the unrelated sibling.Reproducer
Fix
Filter bucket tree results in
_list_bucket_tree_with_folders()so entries are accepted only when the requested prefix is empty, matches exactly, or is followed by/. This keepsHfApi.list_bucket_tree()'s documented lexical behavior unchanged while restoring normal filesystem behavior:exists()returnsFalseglob()returns no matchesls()raisesFileNotFoundErrorIncludes regression coverage using the existing
data/fixture and colliding missing prefixdat.Validation
1 passed38 passed, 2 skipped(2 transient staging 504 reruns)make style: passedmake quality: passedNote
Low Risk
Small, localized filter in bucket listing with a regression test; no auth or API contract changes.
Overview
Fixes bucket path prefix collisions in
HfFileSystemwhenlist_bucket_treereturns lexically similar paths (e.g. listingdatalso returnsdata/...).In
_list_bucket_tree_with_folders, results are now filtered so only entries that equal the prefix or sit underprefix/are kept—restoring normal filesystem behavior forexists,glob, andlswithout changing the API’s lexical prefix behavior.Adds
test_prefix_collisionagainst the shareddata/fixture and the missing prefixdat.Reviewed by Cursor Bugbot for commit fd16996. Bugbot is set up for automated code reviews on this repo. Configure here.