Skip to content

[Repos] Warn when duplicated repo files are still being copied - #5041

Merged
Wauplin merged 3 commits into
mainfrom
duplicate-status
Oct 1, 2026
Merged

Wauplin merged 3 commits into
mainfrom
duplicate-status

Conversation

@Wauplin

@Wauplin Wauplin commented Sep 29, 2026 •

Copy link
Copy Markdown
Collaborator

Follow-up to moon-landing#19664: the Hub now copies LFS/Xet files of a duplicated repo in a background job and returns filesCopyPending. Until the copy is done, downloading these files from the duplicate fails.

Summary

  • duplicate_repo returns a DuplicatedRepoUrl (subclass of RepoUrl, no breaking change) with a files_copy_pending: bool attribute.
  • hf repos duplicate prints a warning on stderr when the copy is still pending.
  • The GET …/duplicate/status endpoint is not exposed on purpose (too low-level for now).

Example

>>> hf repos duplicate __DUMMY_HUB_USER__/dup-src-simple __DUMMY_HUB_USER__/dup-copy-simple --type dataset
✓ Repo duplicated
  from_id: __DUMMY_HUB_USER__/dup-src-simple
  to_id: __DUMMY_HUB_USER__/dup-copy-simple
  url: https://hub-ci.huggingface.co/datasets/__DUMMY_HUB_USER__/dup-copy-simple
Warning: LFS/Xet files are still being copied in the background. Downloading them from '__DUMMY_HUB_USER__/dup-copy-simple' may fail until the copy is complete.

🤖 Generated with Claude Code


Note

Low Risk
Backward-compatible return type subclassing RepoUrl; adds visibility for async LFS/Xet copy behavior without changing duplication semantics.

Overview
Repo duplication now surfaces when LFS/Xet blobs are still copying on the Hub after the duplicate call finishes.

duplicate_repo / HfApi.duplicate_repo return a DuplicatedRepoUrl (subclass of RepoUrl) with files_copy_pending, populated from the API’s filesCopyPending field. hf repos duplicate prints a stderr warning when that flag is true so users know downloads from the new repo may fail until the background copy completes.

Docs cover the new type (hf_api reference), CLI hf repos duplicate section, and the files_copy_pending check in the repository guide. DuplicatedRepoUrl is exported from the public package API.

Reviewed by Cursor Bugbot for commit b98c8d6. Bugbot is set up for automated code reviews on this repo. Configure here.

@bot-ci-comment

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@cursor cursor Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 1dde237. Configure here.

Comment thread src/huggingface_hub/hf_api.py Resolved
Wauplin and others added 2 commits September 30, 2026 16:51
The Hub now copies LFS/Xet files of a duplicated repo in a background job.
`duplicate_repo` returns a `DuplicatedRepoUrl` with `files_copy_pending`,
`get_duplication_status` returns the copy progress, and the CLI warns when the
copy is still pending and adds `hf repos duplication-status`.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
Keep only DuplicatedRepoUrl.files_copy_pending and the CLI warning.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

@Wauplin Wauplin left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✔️

@Wauplin Wauplin changed the title [Repos] Report background file copy of duplicated repos [Repos] Warn when duplicated repo files are still being copied Sep 30, 2026
@Wauplin
Wauplin marked this pull request as ready for review September 30, 2026 15:04

@hanouticelina hanouticelina left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good!

# Conflicts:
#	docs/source/en/guides/repository.md
@Wauplin
Wauplin merged commit 4c88ea6 into main Oct 1, 2026
22 of 25 checks passed
@Wauplin
Wauplin deleted the duplicate-status branch October 1, 2026 09:48
@huggingface-hub-bot

Copy link
Copy Markdown
Contributor

This PR has been shipped as part of the v2.1.0 release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants