Skip to content

[HfFileSystem] Make refresh=True clear cached not-found results - #5106

Merged
hanouticelina merged 1 commit into
mainfrom
hffs-refresh-cached-misses
Oct 7, 2026
Merged

hanouticelina merged 1 commit into
mainfrom
hffs-refresh-cached-misses

Conversation

@hanouticelina

@hanouticelina hanouticelina commented Oct 7, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

HfFileSystem caches "not found" for repos, revisions and buckets, and exists(path, refresh=True) / invalidate_cache(path) couldn't clear it: invalidate_cache resolves the path first, which re-raises the cached miss. After create_repo, create_branch or create_bucket in the same process, the target stayed missing until a full invalidate_cache() (buckets: not even that).

  • invalidate_cache(path) now drops cached misses when the path doesn't resolve, so refresh=True works after creation like it already does after deletion (Fix HfFileSystem.exists() for deleted repos and update documentation #2643).
  • invalidate_cache() with no path now also clears the bucket cache.
  • Misses stay cached otherwise: a missing repo still costs one repo_info per instance.
fs.exists(repo_id)                  # False
api.create_repo(repo_id)
fs.exists(repo_id, refresh=True)    # main: False, this PR: True

Same for repo_id@dev after create_branch and buckets/... after create_bucket (checked on hub-ci).

Alternative to #5089, which stops caching missing repos. That fixes exists() without refresh=True, but it leaves branches and buckets stale and sends a new repo_info on every call for a missing repo. Writes made through HfApi still need refresh=True, the same as files uploaded with upload_file.

Fixes #1945.


Note

Low Risk
Targeted cache-layer fix in HfFileSystem with a regression test; behavior change is limited to refresh/invalidate paths that previously failed silently.

Overview
HfFileSystem cache invalidation now works when a path was previously cached as missing. invalidate_cache(path) used to call resolve_path first, which re-threw the cached not-found and blocked exists(..., refresh=True) after in-process create_repo, create_branch, or create_bucket.

On FileNotFoundError during path invalidation, the implementation drops cached misses from _repo_and_revision_exists_cache and _bucket_exists_cache so the next lookup hits the Hub. A full invalidate_cache() with no path also clears _bucket_exists_cache.

A regression test covers exists(repo_id, refresh=True) flipping to true after the repo is created (#1945).

Reviewed by Cursor Bugbot for commit 62aab27. Bugbot is set up for automated code reviews on this repo. Configure here.

@bot-ci-comment

bot-ci-comment Bot commented Oct 7, 2026

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@hanouticelina hanouticelina left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✔️

@hanouticelina
hanouticelina marked this pull request as ready for review October 7, 2026 12:30
@hanouticelina
hanouticelina merged commit cf992a9 into main Oct 7, 2026
31 checks passed
@hanouticelina
hanouticelina deleted the hffs-refresh-cached-misses branch October 7, 2026 12:31
@huggingface-hub-bot

Copy link
Copy Markdown
Contributor

This PR has been shipped as part of the v2.2.0 release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Cannot get a repo info which just created in a same thread.

1 participant