Repository navigation
[httpx migration] Expose httpx as a huggingface_hub submodule - #4803
Merged
Merged
Conversation
16 of 18 tasks
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
Collaborator
Author
|
failing CI is unrelated |
Contributor
|
This PR has been shipped as part of the v1.31.0 release. |
This was referenced Sep 10, 2026
g0rdonL
added a commit
to g0rdonL/huggingface_hub
that referenced
this pull request
Oct 8, 2026
* Post-release: bump version to 1.30.0.dev0 (#4763)
Co-authored-by: huggingface-hub-bot[bot] <huggingface-hub-bot[bot]@users.noreply.github.com>
* [CI] Enable enforcement in close-unscoped-community-prs workflow (#4766)
* Align `hf jobs scheduled ls` filtering with `hf jobs ls` (`--status`/`--label`/`--name`) (#4773)
* Align `hf jobs scheduled ls` filters with `hf jobs ls`
Requested by @Wauplin (Lucain Pouget) via Moon Bot.
Co-authored-by: Wauplin <Wauplin@users.noreply.github.com>
Slack thread: https://huggingface.slack.com/archives/C02V5EA0A95/p1787841077235539?thread_ts=1787841077.235539&cid=C02V5EA0A95
* Remove tests from PR
Requested by @Wauplin (Lucain Pouget) via Moon Bot.
Co-authored-by: Wauplin <Wauplin@users.noreply.github.com>
Slack thread: https://huggingface.slack.com/archives/C02V5EA0A95/p1787841077235539?thread_ts=1787841077.235539&cid=C02V5EA0A95
* Regenerate CLI reference docs
Requested by @Wauplin (Lucain Pouget) via Moon Bot.
Co-authored-by: Wauplin <Wauplin@users.noreply.github.com>
Slack thread: https://huggingface.slack.com/archives/C02V5EA0A95/p1787841077235539?thread_ts=1787841077.235539&cid=C02V5EA0A95
* Fix mypy: rename label loop variable to avoid shadowing
Requested by @Wauplin (Lucain Pouget) via Moon Bot.
Co-authored-by: Wauplin <Wauplin@users.noreply.github.com>
Slack thread: https://huggingface.slack.com/archives/C02V5EA0A95/p1787841077235539?thread_ts=1787841077.235539&cid=C02V5EA0A95
---------
Co-authored-by: moonbot[bot] <moonbot[bot]@users.noreply.github.com>
Co-authored-by: Wauplin <Wauplin@users.noreply.github.com>
* [Download] Follow redirects between Hub hosts when resolving files (#4739)
* [Download] Follow redirects between Hub hosts when resolving files
* [Download] Keep auth header when download location stays on a Hub host
* [Download] Drop redirect cap, keep the original unbounded loop
* [Download] Bound redirect following at httpx's default of 20
* [Download] Name the redirect bound
* less verbose please
---------
Co-authored-by: Lucain <lucain@huggingface.co>
* [Download] A `ResolvedRevision` only pins the repo it was resolved for (#4767)
* [Download] A `ResolvedRevision` only pins the repo it was resolved for
A commit hash means nothing outside of the repo it comes from, but a
`ResolvedRevision` used to be trusted by any repo: `resolve_revision`
returned it untouched, and the download helpers used its commit hash
whatever repo they were fetching from. A library resolving a revision for
one repo and reusing it for another (a base model and its adapter, a
pipeline component in its own repo, ...) would then request a commit that
does not exist there.
Instances now remember the repo (id + type) they were resolved against,
and only pin that one. Anywhere else, the revision initially requested is
used instead, and resolved again if needed.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* [Download] Scope the fix to resolve_revision: drop _commit_hash_for
Download helpers trust the passed `ResolvedRevision` again (`revision.resolved`
as before). The only behavior change is in `HfApi.resolve_revision`: a revision
resolved for another repo is resolved again for the requested one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* [CI] Fix torch_1.11 case-arm label to match renamed matrix entry (#4776)
The torch~=1.11 matrix entry's test_name was renamed from
"Python 3.9, torch_1.11" to "Python 3.10, torch_1.11", but the
install-step case arm further down was not updated to match. The
esac has no `*)` default arm, so the case silently falls through,
torch is never installed on this lane, and its tests skip with
"Test requires torch" while the job still reports green.
* [Sandbox] Document experimental status (#4783)
* [Inference Providers] Do not require the conversational tag for chat-completion on hf-inference (#4784)
* Do not require the conversational tag for chat-completion on hf-inference
The Hub no longer requires the "conversational" tag to serve
text-generation / image-text-to-text models through the chat-completion
API for inference providers.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Clarify comment: raw text-generation is hf-inference specific
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Remove comment
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* Update task-check test expectations
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* [CI] Disable auto close (#4800)
* [Release] Consolidate the HF CLI skill sync into the release workflow (#4765)
* [Release] Drop duplicate HF CLI skill sync job
Every release opened two PRs on huggingface/skills: one from this job and
one from sync-hf-cli-skill.yml, which the release tag push also triggers.
Only the standalone workflow bumps the plugin version and regenerates the
skills repo artifacts, so its PR is the one that passes validate.
* [Release] Keep the skill sync in the release workflow instead
Reverses the previous commit's direction: rather than deleting the job and
relying on the tag-triggered workflow, port the working implementation (uv,
plugin version bump, publish.sh, SKILL.md-scoped diff) into the release job
and delete sync-hf-cli-skill.yml. One sync, inside the workflow that already
reports to the release Slack thread and the final status check.
The PR body now points at the release tag rather than github.sha, which on a
workflow_dispatch run is main's HEAD, not the commit that was synced.
* Post-release: bump version to 1.31.0.dev0 (#4806)
* Bump the actions group across 1 directory with 5 updates (#4798)
Bumps the actions group with 5 updates in the / directory:
| Package | From | To |
| --- | --- | --- |
| [huggingface/doc-builder/.github/workflows/build_main_documentation.yml](https://github.com/huggingface/doc-builder) | `af49b8dc3859c3d80c7220c8c0ba146431132e49` | `c2d27f6f231df3d1b07f3a7ea54fe3d0f8bc3e27` |
| [huggingface/doc-builder/.github/workflows/build_pr_documentation.yml](https://github.com/huggingface/doc-builder) | `af49b8dc3859c3d80c7220c8c0ba146431132e49` | `c2d27f6f231df3d1b07f3a7ea54fe3d0f8bc3e27` |
| [astral-sh/setup-uv](https://github.com/astral-sh/setup-uv) | `10.0.0` | `10.0.1` |
| [trufflesecurity/trufflehog](https://github.com/trufflesecurity/trufflehog) | `3.96.0` | `3.97.1` |
| [huggingface/doc-builder/.github/workflows/upload_pr_documentation.yml](https://github.com/huggingface/doc-builder) | `af49b8dc3859c3d80c7220c8c0ba146431132e49` | `c2d27f6f231df3d1b07f3a7ea54fe3d0f8bc3e27` |
Updates `huggingface/doc-builder/.github/workflows/build_main_documentation.yml` from af49b8dc3859c3d80c7220c8c0ba146431132e49 to c2d27f6f231df3d1b07f3a7ea54fe3d0f8bc3e27
- [Release notes](https://github.com/huggingface/doc-builder/releases)
- [Changelog](https://github.com/huggingface/doc-builder/blob/main/RELEASE.md)
- [Commits](https://github.com/huggingface/doc-builder/compare/af49b8dc3859c3d80c7220c8c0ba146431132e49...c2d27f6f231df3d1b07f3a7ea54fe3d0f8bc3e27)
Updates `huggingface/doc-builder/.github/workflows/build_pr_documentation.yml` from af49b8dc3859c3d80c7220c8c0ba146431132e49 to c2d27f6f231df3d1b07f3a7ea54fe3d0f8bc3e27
- [Release notes](https://github.com/huggingface/doc-builder/releases)
- [Changelog](https://github.com/huggingface/doc-builder/blob/main/RELEASE.md)
- [Commits](https://github.com/huggingface/doc-builder/compare/af49b8dc3859c3d80c7220c8c0ba146431132e49...c2d27f6f231df3d1b07f3a7ea54fe3d0f8bc3e27)
Updates `astral-sh/setup-uv` from 10.0.0 to 10.0.1
- [Release notes](https://github.com/astral-sh/setup-uv/releases)
- [Commits](https://github.com/astral-sh/setup-uv/compare/ae62891fec2bb8e7d6c99fc78c9fec3a63790f8d...20cfd1bf945f4377ade1205e4dbc17946fc9a30d)
Updates `trufflesecurity/trufflehog` from 3.96.0 to 3.97.1
- [Release notes](https://github.com/trufflesecurity/trufflehog/releases)
- [Commits](https://github.com/trufflesecurity/trufflehog/compare/6f3c981e7b77f235fd2702dd74af25fc4b72bf11...20652fbbdefffcdaa493a5bf57ab2ac6b1db715b)
Updates `huggingface/doc-builder/.github/workflows/upload_pr_documentation.yml` from af49b8dc3859c3d80c7220c8c0ba146431132e49 to c2d27f6f231df3d1b07f3a7ea54fe3d0f8bc3e27
- [Release notes](https://github.com/huggingface/doc-builder/releases)
- [Changelog](https://github.com/huggingface/doc-builder/blob/main/RELEASE.md)
- [Commits](https://github.com/huggingface/doc-builder/compare/af49b8dc3859c3d80c7220c8c0ba146431132e49...c2d27f6f231df3d1b07f3a7ea54fe3d0f8bc3e27)
---
updated-dependencies:
- dependency-name: astral-sh/setup-uv
dependency-version: 10.0.1
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: actions
- dependency-name: huggingface/doc-builder/.github/workflows/build_main_documentation.yml
dependency-version: c2d27f6f231df3d1b07f3a7ea54fe3d0f8bc3e27
dependency-type: direct:production
dependency-group: actions
- dependency-name: huggingface/doc-builder/.github/workflows/build_pr_documentation.yml
dependency-version: c2d27f6f231df3d1b07f3a7ea54fe3d0f8bc3e27
dependency-type: direct:production
dependency-group: actions
- dependency-name: huggingface/doc-builder/.github/workflows/upload_pr_documentation.yml
dependency-version: c2d27f6f231df3d1b07f3a7ea54fe3d0f8bc3e27
dependency-type: direct:production
dependency-group: actions
- dependency-name: trufflesecurity/trufflehog
dependency-version: 3.97.1
dependency-type: direct:production
update-type: version-update:semver-minor
dependency-group: actions
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* [Docs] Clarify Jobs argument forwarding and align examples (#4809)
* [Docs] Clarify Jobs argument forwarding and align examples
* [Docs] Distinguish option ordering from argument forwarding
* [Docs] Focus Jobs guidance on the explicit argument separator
* [Docs] Explain where script help arguments are forwarded
* [httpx migration] Expose `httpx` as a `huggingface_hub` submodule (#4803)
* Expose httpx module
* add note in docs
* [hf_api] Make CommitInfo copy-able and pickle-able (#4822)
`CommitInfo.__new__` took `commit_url` as a required keyword-only argument, so
the reconstruction path used by `copy`/`pickle` (`cls.__new__(cls,
*self.__getnewargs__())`, i.e. positional string value only) raised a
TypeError. `commit_url` is now a regular parameter and a `__reduce__` is added
so all fields survive the round-trip, mirroring what `ResolvedRevision`
already does.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* [Download] Prevent cached file copies during local-dir dry runs (#4817)
* [Download] Prevent cached file copies during local-dir dry runs
* Apply suggestion from @Wauplin
* Apply suggestion from @Wauplin
---------
Co-authored-by: Lucain <lucainp@gmail.com>
* Fix: keep single-quoted values verbatim in load_dotenv (#4827)
* Keep single-quoted values verbatim in load_dotenv
* Apply style fixes
---------
Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
* [CLI] Fix 'hf buckets rm --recursive' deleting lexical siblings of the prefix (#4804)
* [CLI] Fix 'hf buckets rm --recursive' deleting lexical siblings of the prefix
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* [CLI] Apply the same path-boundary filter to 'hf buckets ls'
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* [Sandbox] Support custom Job labels (#4824)
* [Sandbox] Support custom labels in Sandbox.create
* [CLI] Add labels to sandbox create
* [CLI] Don't let `hf update` self-upgrade a pip install on Windows (#4823)
* [CLI] Don't let `hf update` self-upgrade a pip install on Windows
Windows refuses to replace the `hf.exe` launcher of a running process,
so `pip install -U huggingface_hub` spawned by `hf update` fails with
WinError 32 after having uninstalled the current version, leaving a dead
launcher and no `huggingface_hub` module. Print the command for the user
to run instead, like pip does for its own self-upgrade.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* [CLI] Use out.error + Exit(1) instead of CLIError
Avoids the "set HF_DEBUG=1 for full traceback" hint: this is an
instruction to the user, not an internal failure.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* [HfFileSystem] Reject unsafe paths before local downloads (#4828)
* [HfFileSystem] Reject unsafe paths before local downloads
* Apply suggestion from @Wauplin
* Apply suggestion from @Wauplin
* [Download] Share retry handling for stream entry and body failures (#4826)
* [Download] Share retry handling for stream entry and body failures
* [Download] Remove added retry unit tests
* [Download] Tolerate missing HEAD Content-Length (#4805)
* [Download] Tolerate missing HEAD Content-Length
* [Download] Allow missing HEAD size when Xet is unavailable
---------
Co-authored-by: Celina Hanouti <hanouticelina@gmail.com>
* [Download] Write cache ref file atomically in `snapshot_download` (#4829)
`snapshot_download` rewrote `refs/<revision>` with a plain `open(path, "w")` on every
call, even when the commit hash was unchanged. `open(..., "w")` truncates the file
before rewriting it, so a concurrent reader can observe an empty revision and conclude
that a fully cached file is missing.
Use the existing `_cache_commit_hash_for_specific_revision` helper instead: it skips the
write when the ref is already up to date and otherwise writes to a temporary file and
`os.replace`s it into place. `hf_hub_download` and `HfApi.resolve_revision` already go
through it.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* [CLI] Raise explicit error for shell-script extensions on Windows (#4846)
* [CLI] Raise explicit error for shell-script extensions on Windows
* [CLI] Surface install errors from extension auto-install
* [CLI] Drop Windows-specific extension install tests
* [Jobs] Add network groups to hf jobs run (#4833)
* [Jobs] Add network groups to hf jobs run
* Trim network groups guide sections
---------
Co-authored-by: Lucain <lucain@huggingface.co>
* Fix dotenv parser truncating unquoted values containing '#' (#4842)
* Fix dotenv parser truncating unquoted values containing '#'
* fix empty values swallowing inline comments and quoted values absoprtion bug
* bare keys with comments dropped bug fixed
* bare key alternative fix
* (LFS)Fix SliceFileObj.__iter__ yielding only the first 4MB chunk (#4844)
Co-authored-by: Lucain <lucain@huggingface.co>
* [CLI] Fix truncated command descriptions in the CLI reference (#4849)
* Post-release: bump version to 1.32.0.dev0 (#4856)
Co-authored-by: huggingface-hub-bot[bot] <huggingface-hub-bot[bot]@users.noreply.github.com>
* docs(jobs): scope network groups to the namespace and resource group (#4855)
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* [Community] Fix docstring for DiscussionComment.last_edited_by (#4864)
* [Community] Fix docstring for DiscussionComment.last_edited_by
* Update tests/test_cli_discussions.py
* Update tests/test_cli_discussions.py
---------
Co-authored-by: célina <hanouticelina@gmail.com>
* fix(sandbox): preserve terminal command results (#4851)
* [CLI] Read a UV script's `[tool.hf-jobs]` launch config (opus-generated) (#4598)
* [CLI] Read the launch config a UV script carries in `[tool.hf-jobs]`
Implements the `[tool.hf-jobs]` PEP 723 table discussed in #4558: a UV script
can carry the runtime it needs (image, flavor, python, env, secrets, timeout,
name, namespace, labels, volumes) and `hf jobs uv run` reads it client-side.
- CLI flags always win; env/secrets/labels/volumes merge entry by entry
- secrets list names only, values read from the caller's environment (error if
unset, warning when sent)
- unknown keys and non-string values are errors, not silently dropped intents
- URL scripts are downloaded once at submit time and shipped like local scripts
- every run echoes the resolved config (secrets redacted); `--dry-run` prints it
without submitting
- default Job name now hashes the resolved config, not just the command line
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* [CLI] Address review: utf-8 script read, consistent auto-naming, API/CLI note
- read scripts as UTF-8 rather than with the locale encoding (review comment)
- `hf jobs run` and `hf jobs uv run` now hash the same resolved values into the
default Job name (env values included, secret names only - never their values)
- document in `run_uv_job` / `create_scheduled_uv_job` and in the Jobs guide that
the Python API ignores `[tool.hf-jobs]`, unlike `hf jobs uv run`
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* [CLI] Simplify UV script loading: context-managed download, lazy environ
- `load_uv_script` is now a context manager built on `SoftTemporaryDirectory`:
the downloaded script's lifetime is the `with` block, instead of a
`TemporaryDirectory` kept alive by a `tmp_dir` field on `UvScript` plus a
`source` field on `_UvJobConfig` whose only job was to keep *that* alive.
- `_resolve_uv_job_config` yields the config from inside the block, so the
Job is always submitted while the download still exists.
- `parse_env_map` no longer copies `os.environ` + reads the token file when
there are no env args and no env file to resolve (twice per `jobs run`).
- `_resolve_script_secrets` skips the same work when the script requests no
secrets.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* [CLI] Address review: request timeout on script download, dry-run secrets
- `_download_script` passes `constants.DEFAULT_REQUEST_TIMEOUT`, so an
unreachable URL fails the CLI instead of hanging it.
- The secrets warning says "would be sent" under `--dry-run`, where nothing
is submitted.
- A secret the script requests but the caller has not set is shown as
`NAME=<not set>` in a dry run instead of erroring, so the full config of a
not-yet-provisioned script stays visible. A real run still errors.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* [CLI] Avoid script metadata in the UV header module docstring
* Update docs/source/en/guides/cli.md
Co-authored-by: Daniel van Strien <davanstrien@users.noreply.github.com>
* [CLI] Docs: scope Job auto-naming to the CLI, add TOML examples for [tool.hf-jobs] keys
Follow-up on review feedback:
- cli.md/jobs.md naming paragraphs now say the hash covers the resolved
runtime settings (CLI only; the Python API still hashes the command).
- show timeout/env/labels/volumes syntax in the header example and spell
out that env/labels values must be TOML strings.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Co-authored-by: Daniel van Strien <davanstrien@users.noreply.github.com>
Co-authored-by: célina <hanouticelina@gmail.com>
* Add `--container-registry-{username,password}` (#4854)
* Add `--container-registry-{username,password}`
Custom containers might live in private registries e.g., a Hugging Face
Space or a private AWS ECR, and a `username` and `password` might be
required. Given that the underlying API already supports those *only
with custom containers*, then this commit adds both
`--container-registry-username` and `--container-registry-password` in
such cases. More information about the API spec at
https://api.endpoints.huggingface.cloud/#post-/v2/endpoint/-namespace-
Signed-off-by: Alvaro Bartolome <36760800+alvarobartt@users.noreply.github.com>
* Fix `mypy`
Signed-off-by: Alvaro Bartolome <36760800+alvarobartt@users.noreply.github.com>
---------
Signed-off-by: Alvaro Bartolome <36760800+alvarobartt@users.noreply.github.com>
* fix(serialization): save_torch_state_dict cleanup regex silently matches/deletes unrelated files (#4857)
* fix(serialization): escape+anchor cleanup regex in save_torch_state_dict
The pre-save cleanup step in save_torch_state_dict() built its
"existing files" regex via filename_pattern.format(suffix=...) and
matched with re.match(). Two bugs compound:
1. Literal characters in the pattern (e.g. the "." in the default
"model{suffix}.safetensors") were left unescaped, so "." acted as
a regex wildcard.
2. re.match() only anchors at the start, so any file merely starting
with the shard filename also matched.
Together this means files like model.safetensors.backup,
model.safetensors.dvc (DVC sidecar), or modelXsafetensors were
silently deleted on every save (failures are only logged as
warnings, never raised).
Fix: split the pattern on the {suffix} placeholder, re.escape() both
literal halves, and match with fullmatch() instead of match(). Added
a regression test with near-miss fixture files.
AI-assisted: used an AI coding agent to help locate/verify/write this
fix; I read, tested, and understand the change and take responsibility
for it. All existing + new serialization tests pass locally
(pytest tests/test_serialization.py -k save_torch_state_dict, 14 passed);
ruff format/check clean.
* Update tests/test_serialization.py
* Update src/huggingface_hub/serialization/_torch.py
---------
Co-authored-by: célina <hanouticelina@gmail.com>
* Bump the actions group with 4 updates (#4848)
Bumps the actions group with 4 updates: [huggingface/doc-builder/.github/workflows/build_main_documentation.yml](https://github.com/huggingface/doc-builder), [huggingface/doc-builder/.github/workflows/build_pr_documentation.yml](https://github.com/huggingface/doc-builder), [trufflesecurity/trufflehog](https://github.com/trufflesecurity/trufflehog) and [huggingface/doc-builder/.github/workflows/upload_pr_documentation.yml](https://github.com/huggingface/doc-builder).
Updates `huggingface/doc-builder/.github/workflows/build_main_documentation.yml` from c2d27f6f231df3d1b07f3a7ea54fe3d0f8bc3e27 to cf20b09f274383f0d91a7055c5e43f0a2ab3d1a3
- [Release notes](https://github.com/huggingface/doc-builder/releases)
- [Changelog](https://github.com/huggingface/doc-builder/blob/main/RELEASE.md)
- [Commits](https://github.com/huggingface/doc-builder/compare/c2d27f6f231df3d1b07f3a7ea54fe3d0f8bc3e27...cf20b09f274383f0d91a7055c5e43f0a2ab3d1a3)
Updates `huggingface/doc-builder/.github/workflows/build_pr_documentation.yml` from c2d27f6f231df3d1b07f3a7ea54fe3d0f8bc3e27 to cf20b09f274383f0d91a7055c5e43f0a2ab3d1a3
- [Release notes](https://github.com/huggingface/doc-builder/releases)
- [Changelog](https://github.com/huggingface/doc-builder/blob/main/RELEASE.md)
- [Commits](https://github.com/huggingface/doc-builder/compare/c2d27f6f231df3d1b07f3a7ea54fe3d0f8bc3e27...cf20b09f274383f0d91a7055c5e43f0a2ab3d1a3)
Updates `trufflesecurity/trufflehog` from 3.97.1 to 3.97.2
- [Release notes](https://github.com/trufflesecurity/trufflehog/releases)
- [Commits](https://github.com/trufflesecurity/trufflehog/compare/20652fbbdefffcdaa493a5bf57ab2ac6b1db715b...cc1fe982afc515d2991365ce8d4d0dd07170fcad)
Updates `huggingface/doc-builder/.github/workflows/upload_pr_documentation.yml` from c2d27f6f231df3d1b07f3a7ea54fe3d0f8bc3e27 to cf20b09f274383f0d91a7055c5e43f0a2ab3d1a3
- [Release notes](https://github.com/huggingface/doc-builder/releases)
- [Changelog](https://github.com/huggingface/doc-builder/blob/main/RELEASE.md)
- [Commits](https://github.com/huggingface/doc-builder/compare/c2d27f6f231df3d1b07f3a7ea54fe3d0f8bc3e27...cf20b09f274383f0d91a7055c5e43f0a2ab3d1a3)
---
updated-dependencies:
- dependency-name: huggingface/doc-builder/.github/workflows/build_main_documentation.yml
dependency-version: cf20b09f274383f0d91a7055c5e43f0a2ab3d1a3
dependency-type: direct:production
dependency-group: actions
- dependency-name: huggingface/doc-builder/.github/workflows/build_pr_documentation.yml
dependency-version: cf20b09f274383f0d91a7055c5e43f0a2ab3d1a3
dependency-type: direct:production
dependency-group: actions
- dependency-name: trufflesecurity/trufflehog
dependency-version: 3.97.2
dependency-type: direct:production
update-type: version-update:semver-patch
dependency-group: actions
- dependency-name: huggingface/doc-builder/.github/workflows/upload_pr_documentation.yml
dependency-version: cf20b09f274383f0d91a7055c5e43f0a2ab3d1a3
dependency-type: direct:production
dependency-group: actions
...
Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* Key the doc build concurrency group on the pull request number (#4876)
* [Buckets] Fix file metadata size on redirects (#4874)
* [RepoCard] Fix AttributeError when model-index has source: null (#4870)
* [Spaces] Fix AttributeError in SpaceRuntime when hardware is null (#4873)
* [CI] Make CI less flaky and stop 6h hung jobs (#4830)
* [CI] Make CI less flaky and stop 6h hung jobs
Follow-up to an investigation of the recent CI flakiness. All the failures
themselves come from a hub-ci incident (plain-LFS uploads return 403
AccessDenied on s3:PutObject), which is being handled by the infra team. This
commit fixes how our CI *reacts* to such an incident.
- `_upload_large_folder`: cap consecutive errors instead of retrying forever.
A permanent error made the worker loop spin until GitHub's 6h job timeout
(136k retries, 60MB of logs, thousands of requests/min at the Hub).
- Add `timeout-minutes: 30` and a `concurrency` group to `python-tests.yml`,
and stop triggering the whole suite on the `assigned` PR event.
- Drop the `lfs` matrix entry: `TestHfLargefiles` is `@pytest.mark.skip`ped, so
the job selected 1 test and skipped it (green but empty).
- Speed up `TestValidateUploadLimits`: 100k `MagicMock`s -> plain objects
(85s -> 1s).
- De-flake `test_commit_to_repo_in_background` (wall-clock assertion) and
`test_list_private_models` (shared `__DUMMY_HUB_USER__` recency page).
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Apply suggestion from @Wauplin
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* [Serialization] Harden torch checkpoint deserialization (#4877)
* fix(security): case-insensitive safetensors extension check
`_is_safetensors` was case-sensitive, so `model.SAFETENSORS` fell through to
`torch.load(weights_only=False)`. On Windows/macOS, where globbing is
case-insensitive, this made the pickle path reachable automatically.
Refs V1 of the torch serialization security analysis.
Co-Authored-By: Claude Code <noreply@anthropic.com>
* fix(security): thread `safe` through the single-file loading path
`load_torch_model`'s single-file branch never passed `safe` to
`load_state_dict_from_file`, so `load_torch_model(model, "model.bin", safe=True)`
still executed the pickle on every OS. `load_state_dict_from_file` now accepts a
keyword-only `safe` parameter and routes to the safetensors loader when it is True.
Refs V3 of the torch serialization security analysis.
Co-Authored-By: Claude Code <noreply@anthropic.com>
* fix(security): default `load_state_dict_from_file` to `safe=True`
Router selection was derived from the filename and defaulted to pickle, i.e. a
denylist by omission: anything not spelled exactly `.safetensors` reached
`torch.load`. `safe=True` is now the default and means "always use the
safetensors loader", whatever the file is called; the filename becomes a hint
used only in the (already unsafe) `safe=False` branch.
BREAKING CHANGE: `load_state_dict_from_file("model.bin")` now raises. Pass
`safe=False` to load pickle checkpoints.
Refs V2 of the torch serialization security analysis.
Co-Authored-By: Claude Code <noreply@anthropic.com>
* fix(security): default `weights_only=True` for pickle checkpoints
Since torch 2.6, `torch.load` defaults to `weights_only=True`. Passing an
explicit `weights_only=False` downgraded that protection and made
huggingface_hub strictly less safe than calling `torch.load` directly. The
pickle path now only uses the unrestricted unpickler when the caller explicitly
passes `weights_only=False`.
BREAKING CHANGE: pickle checkpoints holding non-tensor globals (optimizer
states, full models) now need `weights_only=False` to load.
Refs V4 of the torch serialization security analysis.
Co-Authored-By: Claude Code <noreply@anthropic.com>
* fix(security): reject pickle `filename_pattern` with `safe=True`
`filename_pattern` used to silently override `safe`, so
`load_torch_model(model, dir, safe=True, filename_pattern=PYTORCH_WEIGHTS_FILE_PATTERN)`
loaded pickle shards with no warning. The conflict is now an explicit
`ValueError`, and the docstring no longer advertises the old precedence.
Refs V5 of the torch serialization security analysis.
Co-Authored-By: Claude Code <noreply@anthropic.com>
* fix(security): pass `safe` down to sharded checkpoint loading
`_load_sharded_checkpoint` picked the loader per shard from a second,
independent string check. It now takes an explicit `safe` parameter and forwards
it to `load_state_dict_from_file`, so shards obey the same guarantee as the
single-file path. The index extension guard stays a strict, case-sensitive
allowlist, documented against the case-insensitive routing hint.
Refs V6 of the torch serialization security analysis.
Co-Authored-By: Claude Code <noreply@anthropic.com>
* fix(security): report loaded keys from the shards, not from the index
`strict` validation and the returned `_IncompatibleKeys` were computed from
`index["weight_map"]`, an attacker-controlled file that does not necessarily
match the shard contents. A caller gating on `missing_keys`/`unexpected_keys`
could be told whatever the repo author wanted, independently of what was
actually loaded. Keys are now collected from the shards while loading, and
divergence from the index is warned about.
Refs V7 of the torch serialization security analysis.
Co-Authored-By: Claude Code <noreply@anthropic.com>
* fix(security): make `strict=True` work for multi-shard checkpoints
A single shard never holds all the model's keys, so passing `strict` to the
per-shard `model.load_state_dict` raised `RuntimeError: Missing key(s)` on the
first shard of every legitimate multi-shard checkpoint. `strict=True` could
therefore never succeed, which pushed callers to turn it off — exactly the
opposite of what a careful caller should do when loading untrusted weights.
Shards are now loaded with `strict=False` and strictness is enforced after the
loop against the keys the shards actually contained (V7), keeping the existing
pre-validation against the index as a fail-fast check.
Co-Authored-By: Claude Code <noreply@anthropic.com>
* fix(security): load safetensors first in a directory when `safe=False`
The docstring promises that `safe=False` tries safetensors "if they are
available" and only then falls back to pickle files, but the single-file
lookup globbed `*.bin` outright under `safe=False`: a directory holding only
`model.safetensors` raised `Directory ... does not contain a valid checkpoint`,
and a directory holding a lone `pytorch_model.bin` was handed to the safetensors
loader (`safe` was not forwarded), so it could not be loaded either.
The glob now looks for safetensors first and only falls back to `.bin` when
nothing was found and `safe=False`, and forwards `safe` to the loader so the
selected file is actually parsed with the matching format.
Co-Authored-By: Claude Code <noreply@anthropic.com>
* fix(security): validate the checkpoint index before using it
`json.load` on the index file was followed directly by `index["weight_map"]`,
so a crafted index surfaced as a raw traceback from attacker-controlled input:
a missing `weight_map` raised `KeyError`, a non-dict raised `AttributeError`,
non-string values raised `TypeError` in `PureWindowsPath(...)`/`.endswith`, and
unhashable ones raised `TypeError` in `set(...)`. There was also no size cap, so
a huge `index.json` was parsed into memory before anything else happened.
The index is now rejected with an actionable `ValueError` unless it is a JSON
object whose `weight_map` is an object of strings, and files above a few MB are
refused before being read.
Co-Authored-By: Claude Code <noreply@anthropic.com>
* fix(security): forward `mmap` in the single-file branch of `load_torch_model`
`load_torch_model(model, "…/model.bin", mmap=True)` was a silent no-op: the
sharded and in-directory branches forwarded `mmap`, the single-file one did not.
Co-Authored-By: Claude Code <noreply@anthropic.com>
* test(security): regression tests for unsafe checkpoint loading
Extend the hackerone#141 regression class (renamed `TestCheckpointLoadingSecurity`) with high-level tests:
- a pickle named `model.safetensors` / `.SAFETENSORS` / `.SafeTensors` is never deserialized as pickle, asserted
with a payload that touches a marker file when unpickled, plus a control proving the payload is live;
- `weights_only` default refuses non-tensor globals;
- neither an explicit `.bin` path nor a directory picks up a pickle under `safe=True`;
- `strict=True` on a legitimate multi-shard checkpoint succeeds;
- reported keys come from the shard contents, not from the index declaring them;
- `safe=False` still loads a safetensors file;
- a malformed index raises a clear `ValueError`.
All of them fail against the previous implementation.
Co-Authored-By: Claude Code <noreply@anthropic.com>
* [Serialization] Address review feedback on checkpoint loading hardening
- Fall back to `.bin` whenever the safetensors glob did not yield exactly one
file, not only when it yielded zero. A directory holding several safetensors
files next to a single `pytorch_model.bin` regressed under `safe=False`.
- Drop the pre-load `strict` validation against the index. It made the
attacker-controlled index authoritative again, in the fail-closed direction:
a stale or crafted `weight_map` entry failed `strict=True` on a checkpoint
whose shards matched the model perfectly. The post-load check against the
real loaded keys is the only one left.
- Only warn about `torch.load` when `weights_only=False`. Under the new
`weights_only=True` default torch uses its restricted unpickler, so the
warning was both untrue and emitted once per shard.
- Document the `safe=True` / `weights_only=True` defaults in the serialization
guide, mention the new `ValueError` in the `Raises:` section, and note that
`strict` is checked after loading (the model may be partially updated).
- Tests: cover the `.bin` fallback with several safetensors files, `mmap`
forwarding in the single-file branch, and `strict=True` against a lying
index. Drop an unused `mocker` fixture.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* [Serialization] Use `> [!WARNING]` callout syntax in the loading docs
The `<Tip>` MDX components are gone from `docs/source/en/`; the blockquote
alert syntax is the only one used now.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* [Serialization] Warn when `weights_only=True` cannot be honored (torch < 1.13)
The pickle warning was keyed off the requested `weights_only`, but that flag is
only forwarded to `torch.load` on torch >= 1.13. On older versions (CI still
covers `torch~=1.11`) the restricted unpickler never runs, so the new
`weights_only=True` default silently unpickled without restriction and the
warning was skipped — a fail-open on the very default this PR introduces.
Key the warning off what is actually passed to `torch.load` instead, so it
fires both when the caller opted out and when torch is too old to honor the
opt-in. Docstrings now say `weights_only` has no effect below 1.13 rather than
"only supported in PyTorch >= 1.13", which read as if the argument were merely
unavailable.
Reported by Cursor Bugbot.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Code <noreply@anthropic.com>
* feat(jobs): expose the network group on JobInfo (#4862)
The Hub returns `network: {group, aliases}` on a job, but `JobInfo` dropped it, so neither `inspect_job` nor `hf jobs inspect` could tell which group a job joined. Adds a `JobNetwork` dataclass and a `network` field on `JobInfo`, `None` for jobs started without a group.
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
* [Http] Fix race in get_session() creating multiple clients (#4868)
* [CLI] Make --format human override agent auto-detection (#4878)
* [CLI] Make --format human override agent auto-detection
* Update _output.py
Co-authored-by: Lucain <lucain@huggingface.co>
---------
Co-authored-by: Lucain <lucain@huggingface.co>
* [Serialization] Sync `_get_dtype_size` with safetensors (#4902)
Adds uint16/uint32/uint64, float8_e4m3fnuz, float8_e5m2fnuz,
float8_e8m0fnu, float4_e2m1fn_x2 and complex64 to the dtype size table.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* [Safetensors] Recognize proposed U3 metadata (#4898)
* [Safetensors] Recognize proposed U3 metadata
* Delete tests/test_safetensors_u3.py
---------
Co-authored-by: Lucain <lucainp@gmail.com>
* [Download] Fix silently disabled tree cache on long Windows paths (#4896)
* [Download] Support Windows long paths for the tree cache file
`_tree_cache_path()` joined `<folder>/trees/<commit_hash>.json` with no
Windows extended-length handling, unlike every sibling path in
`_local_folder.py`, `_snapshot_download.py` and `file_download.py`. On a deep
`local_dir` or `cache_dir` (path over 255 characters with the default
LongPathsEnabled=0) the write failed with WinError 206, `write_tree_cache`
swallowed it, and the next read missed, so the tree cache was silently
disabled and every download re-fetched the tree listing.
Apply the extended-length prefix in the one function both the reader and the
writer go through, so they agree on the on-disk path and the in-memory cache
key. Drive paths become `\\?\C:\...`; UNC paths become
`\\?\UNC\server\share\...`, because prefixing a UNC path verbatim produces
`\\?\\\server\...`, which Windows rejects.
Add two Windows-only tests: a disk round-trip parametrized at 260 (only the
JSON file exceeds the limit) and 300 (the `trees/` folder does too), and a
pure-function check of the drive, UNC, already-prefixed and short-path
spellings.
* [Download] Share a single extended-length path helper
The `\\?\` prefixing was inlined 6 times across the download stack (`_local_folder.py`,
`file_download.py`, `_snapshot_download.py`), so the tree cache fix added a 7th copy.
Move it to `utils/_paths.as_extended_path()` and reuse it everywhere, UNC handling included.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Lucain Pouget <lucainp@gmail.com>
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* docs: make the sandbox security contract match the implementation (#4831)
The sandbox docs promise a few isolation properties we do not currently
deliver, and a user reading them could reasonably pick a pool for a
workload that needs a dedicated VM. This corrects the claims and adds an
explicit "Known limitations" section so the trade-off is visible at the
point of the decision.
Corrections in `concepts/sandbox.md`:
- The token is per *job*, not per sandbox. In a pool, one nonce is minted
per host and every sandbox on it shares that token, which also gates the
host's management routes -- so a leak is host-wide, not per-sandbox. Added
a "Token scope" table and dropped "a leaked sandbox token compromises
that one sandbox only".
- "Only the HF token used to spin-up the Job can access the sandbox" was
imprecise in both directions: any holder of that token can derive it
(that is the point of stateless reconnect), and the token is derived for
whichever job carries the matching nonce label.
- The uid+Landlock guarantee list describes what code *inside* a sandbox
can do directly. It says nothing about the root control plane, which
acts on some names the sandbox controls. Said so, next to the list.
- The file API's `..` guarantee is lexical. Symlinks are a separate
question and currently not handled; said so instead of implying the
path is confined.
- "The cache is safe precisely because it is never trusted as truth" --
it is trusted for the URL credentials get sent to, which is the part
that matters. Replaced with what it is and isn't trusted for, plus the
practical advice ($HF_HOME is sensitive; the key is not bound to
endpoint/principal/namespace).
- "No secret is ever stored on a host" -- true for disk and job metadata,
not for the host server's memory, which holds a pooled sandbox's env for
its lifetime. Stated precisely.
- Dedicated mode isolates the job from everything outside it; inside the
VM the workload and sbx-server share root. Added, since "right choice
for untrusted code" invites reusing one sandbox across trust boundaries.
New "Known limitations" section covering the control-plane confused
deputies, label-based host discovery, Landlock's fail-open path, the
residual shared channels (loopback, UDP, /proc, /dev, IPC), missing
quotas, and the lifecycle gaps (`kill()` addressing, the idle watchdog vs
foreground commands, per-process `max_hosts`, `close()` cancelling
discovered hosts, silent teardown failures, unpinned binary). Plus a
per-actor threat-model table for both modes.
Docstrings updated to match: `_derive_sandbox_token`, `SandboxFiles`,
`SandboxProcess.kill`, `SandboxPool`, `_discover_hosts`, `close`,
`idle_timeout` and `max_hosts`. `_discover_hosts` claimed to match hosts
on image/flavor; it matches on labels only.
This is documentation only -- no behaviour change. Each subsequent
hardening PR should delete the corresponding entry from Known
limitations.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(sandbox): use each pooled sandbox's own capability token (#4832)
The pool host's token authorized every sandbox on the host plus its
management routes, and it was the only credential the client had. So every
per-sandbox call -- and `proxy_headers`, which is handed to browsers and
WebSocket clients -- offered authority over the whole host.
sbx-server 0.6.0 mints a capability token per sandbox. This uses it:
- `_SandboxServer.request`/`stream` take an optional `sandbox_token` that
overrides `X-Sandbox-Token` for one request; the client-level header keeps
carrying the host credential for management calls.
- `Sandbox` holds its own token and sends it on every scoped call, including
the `DELETE` that kills a pooled sandbox.
- `SandboxPool.create()` keeps the token from the create response.
- `Sandbox.connect("<host>.<id>")` recovers it via
`GET /v1/sandboxes/{id}/token`, so reattaching stays stateless.
- `proxy_headers` hands out the sandbox's token, not the host's.
Hosts running an older server return no token and have no recovery route;
those sandboxes fall back to the host credential, so a pool booted before
the server upgrade keeps working. Requires the server change to be deployed
first.
Also makes the test fake enforce the same scoping as the real server --
host token on management routes, the sandbox's own token on scoped ones --
rather than accepting one value everywhere. That laxness is exactly how a
client/server protocol mismatch survives a green suite, and it caught two
of this change's own call sites while writing it.
Validation:
- 44 tests pass, including new ones asserting per-sandbox operations present
the scoped token, `proxy_headers` does not leak the host one, and the
older-server fallback still works.
- Driven end-to-end against a real sbx-server 0.6.0 in a container: A's
token cannot exec, read, write, delete or proxy into B, cannot create,
list or delete sandboxes, and cannot recover any token; the host token
recovers A's token and matches.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(sandbox): validate a pool host before sending it a credential (#4834)
Pool hosts were found by filtering Jobs on labels, and adopted on that
basis alone. Labels are set by whoever creates the Job, and the nonce the
host token is derived from is itself a public label -- so any namespace
member who can create a Job could publish one carrying another user's pool
labels and that host's nonce, and the victim's client would adopt it and
send it the token derived for the *real* host. The Jobs proxy strips
`Authorization` but forwards `X-Sandbox-Token`, so the impostor receives
exactly the capability the real host accepts.
Nothing in the old path bound a Job to its creator, image, flavor, command
or URL. `_discover_hosts` even claimed in its docstring to match on
image/flavor; it did not.
Adoption now requires everything the *backend* asserts about a Job to line
up, not just the labels its creator chose:
- `initiator` -- the load-bearing check, and the one field `run_job` gives no
way to set, so it cannot be forged client-side;
- owner namespace, image (normalized for registry prefixes and case), flavor,
and that the command is the sandbox bootstrap;
- exactly one exposed URL, over HTTPS, whose hostname is derived from *this*
job's id -- so a Job cannot name where the credentials should go.
New `adopt_hosts` argument on `SandboxPool` and `SandboxPool.connect`:
`"own"` (default) adopts only this principal's hosts, `"namespace"` restores
cross-user sharing for namespaces whose members trust each other, `"never"`
disables adoption. A refusal names the opt-in, so someone legitimately
sharing hosts is not left guessing.
`SandboxPool.connect`'s cold path gets the same treatment: it rebuilds the
pool's entire configuration from a discovered host's spec, so an unvalidated
match would let a Job in the namespace choose the image the pool's next
hosts boot. `_connect_host` addresses a job the caller named explicitly, so
ownership there is the caller's assertion, but the URL and bootstrap checks
still apply.
Separately, the HF bearer is no longer frozen when the transport is built.
A host lives up to 24h, and an OIDC/OAuth credential can expire inside that
window, after which every request failed even though the handle was fine. An
`httpx.Auth` now reads it per request. The `X-Sandbox-Token` is deliberately
*not* re-derived: the server holds the value derived from the bearer that
created the job, so re-deriving from a rotated bearer would produce a token
it has never seen. The two credentials answer different gates and only the
bearer needs refreshing. (A host created under one HF token therefore stays
reachable only by holders of that token -- inherent to deriving the
credential, and now stated in the docs.)
The discovery tests were part of the problem: their Job fixtures set only
labels, so they asserted that a label match is sufficient. They now describe
a Job the way the Jobs API really does, and a new suite covers the hostile
cases -- an impostor's initiator, a mismatched image/flavor/command/namespace,
a URL belonging to another job, a plain-HTTP URL, extra exposed ports, and a
missing initiator (refused rather than assumed).
Validation: 55 tests pass, up from 44. Writing them caught two real bugs in
this change -- `urlparse` lowercases hostnames, which broke the URL check on
a mixed-case job id, and a fixture whose host had no capacity.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(sandbox): bind the pool cache to the context that wrote it (#4838)
The local pool cache is not passive data. `SandboxPool.connect()`
rebuilds a pool from `$HF_HOME/sandbox/pools/<pool-id>.json` with no
HTTP at all, `_seed_hosts_from_cache` turns each entry into a
`_SandboxServer` from the cached `base_url` and `nonce`, and the first
thing that happens to it is an authenticated `POST /v1/sandboxes`
carrying the HF bearer and the token derived for that nonce. Two
problems followed from that.
The file was keyed by pool id alone, and its `namespace` field was read
as configuration rather than compared against what the caller asked
for -- `namespace=cache.namespace if namespace is None else namespace`.
So a pool cached for org-A and then reached with
`connect(pool, namespace="org-B")` routed the workload, and its env, to
org-A's hosts. That needs no attacker: one credential and two orgs is
an ordinary state of affairs, and nothing in the flow ever contradicts
the file. Nor was the cache bound to the endpoint or the credential, so
two users sharing a machine shared each other's entries.
And the URL was taken on trust. A cache file (or anything that can
write `$HF_HOME`) could name `http://127.0.0.1:1234` and receive both
credentials on the first create -- with `follow_redirects=True` on the
client, a legitimate-looking HTTPS URL could bounce them elsewhere too.
`verified=False` only means "drop the host if it fails"; by then the
credentials have been sent.
What changed:
- The cache lives at `pools/<context>/<pool-id>.json`, where `<context>`
is a digest of endpoint, credential fingerprint and namespace. The
same digest is stored in the payload and re-checked on read, so an
entry moved or copied into a matching directory is still a miss. The
credential appears only as a `sha256` fingerprint; the token is never
written. Fingerprinting is local on purpose: resolving the credential
to a user id would cost a `whoami` and defeat the point of the cache.
- `connect()` therefore honours `namespace=` instead of inheriting the
cached one, and a mismatch on any of the three is a plain cache miss
(debug log, cold path, never an exception).
- A cached `base_url` is only used if it is the HTTPS jobs-proxy URL of
the job that same entry names, reusing the reasoning already applied
to a discovered host's URL -- both now go through
`_server_host_rejection`. An entry naming anything else is dropped
from the file.
- Every field is type- and range-checked on read. `dataclass` enforces
nothing, so a well-shaped file with `"capacity": "50"` used to parse
happily and blow up much later on `host.capacity - host.live`, as an
uncaught `TypeError` inside `create()`. It is now the documented miss.
- `follow_redirects=False` on the credentialed client. There is no
redirect in this protocol; the proxy serves the in-job server.
- Freshness carve-out: an entry written by this principal within 15
minutes is still credited with no round-trip (the `pool create` ->
`create --pool` case the cache exists for). Past that, `inspect_job`
has to confirm the job is running, labelled for this pool, carrying
that nonce, adoptable, and on that URL before the transport is built.
Local hygiene in the same pass, all of it in the same "the file is not
authenticated, so don't hand it authority" spirit:
- cache dirs `0700` (tightened even if they predate this), and the
atomic write goes through `tempfile.mkstemp` -- `0600` regardless of
umask, unguessable name, removed on any failure -- instead of a
predictable `<name>.<pid>.tmp` opened at `0666 & ~umask`.
- `files.download()` writes to an `O_EXCL|O_NOFOLLOW`, `0600` temp file
next to the destination and `os.replace`s it into place. Before, it
wrote in place: an interrupted download left a truncated file at the
final name, and `open(path, "wb")` followed a symlink planted there.
`rename` does not follow symlinks, so such a destination is now
replaced rather than written through.
- `files.upload()` opens the file once and both `fstat`s and streams
that descriptor, instead of `stat`ing the path and reopening it.
Validation (`PYTHONPATH=src python -m pytest tests/test_sandbox.py -q`):
83 passed, including new cases for cross-principal / cross-namespace /
cross-endpoint misses, moved and renamed files, mistyped fields, URL
admission, the freshness carve-out, redirect handling and the two
transfer paths. Two of them assert against a recording server that a
cache naming `http://127.0.0.1:<port>`, and a cross-namespace
`connect()`, result in zero requests reaching it. `ruff check` and
`ruff format --check` are clean on the three files.
Latency, modelling a 100 ms round-trip and timing
`connect(pool) + create()` (median of 5, cross-process CLI shape):
fresh cache hit 141 ms before / 141 ms after, cold path 560 ms before /
557 ms after, and an entry past the TTL 249 ms (one added round-trip).
The common case is unchanged; only stale entries pay.
Behaviour changes: existing cache files are all misses (new key, new
layout, `version` 2), so the first run after upgrading takes the cold
path once. A pool cached under an explicit `namespace=` is only found
again when the same `namespace=` is passed. A rotated credential reads
as a different principal and gets a miss. Anyone relying on a partial
file being left behind by a failed download no longer gets one.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(sandbox): bound what a transfer or a command's output costs the client (#4839)
Four unbounded allocations in the caller's process, each proportional to
something the sandbox controls.
**Output was accumulated even when a callback was consuming it.** `run()`
appended every chunk to `stdout_parts`/`stderr_parts` regardless, so a
streaming consumer still paid full memory for the command's entire output,
and a runaway command (a loop printing to stdout) was an unbounded
allocation. New `capture_output=False` hands output to the callbacks and
drops it; the default now raises above a ceiling instead of being
OOM-killed with no explanation, and the error names the flag to use.
An explicit flag rather than "stop capturing when a callback is present":
silently changing what `result.stdout` contains based on another argument is
a footgun, and code that passes a callback *and* reads `result.stdout` today
keeps working.
**A parallel read doubled peak memory.** `_read_ranges` collected every
chunk into a list and the caller then joined it, so a 2 GB file peaked at
about 4 GB. It is a generator now, yielding `(offset, bytes)` as ranges
land: `read()` places each chunk into one preallocated buffer, `download()`
seeks and writes it out, and both drop it immediately. `read()` also refuses
above a ceiling and points at `download()`, which streams to disk.
**A parallel upload read the whole file into memory.** `upload()` did
`f.read()` above the threshold -- 10 GB resident for a 10 GB file. Each
worker now `pread`s its own range off the shared descriptor (`pread`, not
`seek`+`read`, because the workers share the descriptor and a seeking read
would race with its siblings).
**And it left a stale tail.** A ranged write does not truncate, so
overwriting a larger file with a smaller one left the old bytes past the new
end. Each upload's final chunk now sends the intended total size, which the
server truncates to.
Also follows the server's new listing pagination, so `files.list()` still
returns one complete list however the directory is chunked on the wire --
and the fake server now serves `/files/list` at all, which it never did, so
that path had no coverage.
Validation: 88 tests pass, up from 83. New coverage for opting out of
capture, the runaway-output ceiling and its escape hatch, the read ceiling,
and cursor-following in `list()`.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(sandbox): decide host teardown per host, and report it honestly (#4840)
Three ways a pool's lifecycle could destroy or leak something without
saying so.
**`close()` cancelled hosts it had merely discovered.** Ownership was
decided once per *handle* (`_owns_hosts = not _connect_mode`), while
`_discover_hosts` appended hosts found via labels -- possibly started by
another process, or another member of the namespace, and possibly serving
their sandboxes -- into the same list. So a plain `with SandboxPool(...)`
block that happened to adopt a colleague's warm host terminated it, and
their sandboxes with it, on exit.
Ownership now belongs to the host: `_SandboxServer.owned` is set only by
`_boot_host`, and `close()` cancels only those. A discovered host has its
local HTTP client released and is otherwise left alone.
**A `create()` in flight could outlive `close()`.** `close()` snapshotted
the host list and set `_closed`, but a concurrent `create()` that had
already passed its `_closed` check could finish booting afterwards and
append to a list nobody would ever tear down. `create()` now registers
itself under the same lock as the closed check, and `close()` waits (with a
timeout, so a wedged create cannot hang process exit) for those to drain.
**A failed cancellation was reported as success.** `close()` logged a
warning, deleted the cache entry, and returned normally -- leaving a host
running and billing with no local record of it. It now raises, naming the
jobs and the command to cancel them, and *keeps* their cache entries so they
stay discoverable. Reached through `__exit__` with an exception already in
flight, the failure is logged instead, so it cannot mask the original error.
Also: `max_hosts` was compared against `len(self._hosts)`, an in-process
count, so two processes using one pool could each stay under the cap while
together exceeding it -- for a documented cost ceiling, with real money
attached. It now counts every host running for the pool, found via labels.
That narrows the window considerably but does not close it: two processes can
still list simultaneously and both decide there is room. A hard cap has to be
enforced where Jobs are created, and the docs now say best-effort rather than
implying a guarantee.
The test stubs were part of the problem: `_boot_host` is monkeypatched in
several tests, and a stub that does not mark the host as owned makes
`close()` correctly refuse to cancel it. They go through a `_booted_server`
helper now, which is the contract `_boot_host` guarantees.
Validation: 94 tests pass, up from 88. New coverage for a booted host being
cancelled, a discovered host being released but not cancelled, a failed
cancellation raising while keeping the cache entry, a teardown failure not
masking the original exception, `close()` waiting for an in-flight create and
tearing down what it booted, and `max_hosts` counting another process' hosts.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(sandbox): address background processes by their server-assigned id (#4836)
`SandboxProcess.kill()` sent the OS pid. The server matches on the opaque id
it assigned (`p-3`) and answered 200 regardless, so the call stopped
nothing, reported success, and left the process running -- which also kept
the sandbox non-idle, so idle eviction never fired and the job kept billing.
`SandboxProcess` now carries `id` as its identifier and `kill()` uses it.
`pid` stays, documented as observational: it is useful for correlating with
`ps` inside the sandbox, but the OS may reuse it. `kill()` returns whether
this call is what stopped the process, so a caller can tell "stopped it"
from "it had already exited" -- both are successes, but they are different
answers.
A process from a host running a server that predates opaque ids has
`id=None`; `kill()` raises with an explanation rather than sending a pid the
new server will reject.
The test fake is the reason this survived a green suite: it deleted by pid,
so it implemented a protocol nobody serves. It now deletes by opaque id only
and 400s on a pid, exactly like the real server -- and there is a test that
asserts the pid path *fails*, so the fake going lax again would be caught
rather than silently re-enabling the bug.
Also corrects the `processes()` docstring: the server keeps a bounded number
of finished processes, not all of them until the sandbox is deleted.
Requires the server change to be deployed first. Validation: 60 tests pass,
up from 55.
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* [sandbox audit] Pin the sandbox server binary by digest and verify it before running it (#4837)
* fix(sandbox): pin the server binary by digest and verify it before running it
Every sandbox job ran a bootstrap that fetched a *mutable* public object,
`chmod +x`ed it and `exec`ed it -- as root, as PID 1, holding the derived
sandbox token:
d=/tmp/.sbx-server
if command -v wget ...; then wget -q -O "$d" "$SBX_SERVER_URL"
...
chmod +x "$d"
exec "$d"
`$SBX_SERVER_URL` was `/buckets/huggingface/sbx-server/resolve/sbx-server`,
the alias the publish workflow overwrites on every release. Nothing in the
chain checked a version, a digest or a signature, so the client had no way
to know which bytes it had just made PID 1. A single write to that bucket
path reaches every sandbox of every user on their next cold start. Nothing
suggests that happened -- the point is that the design offered no guarantee
about the next fetch, even though the immutable artefact it needed was
already being published alongside the alias.
Related: `wait_ready` polled `/health` and read nothing out of it. A pool
host keeps the binary it downloaded at boot for up to 24h, so a client and a
server from different releases meet routinely, and the per-sandbox
capability token added recently is exactly the kind of change that fails
quietly -- a 403 on an unrelated route, several calls later.
What changed:
- `SANDBOX_SERVER_SHA256` and `SANDBOX_SERVER_VERSION` pin one exact server
build. `SBX_SERVER_URL` now names the digest-addressed bucket object
(`sbx-server-<sha256>`), not the alias: a fetch that names the content it
wants cannot be answered with different content.
- The bootstrap verifies the download against that digest and refuses to run
it on a mismatch. The check happens **before** `chmod +x` -- an unverified
file that is already executable is one slip away from being executed.
- `sha256sum` is not guaranteed in an arbitrary image, so `openssl dgst
-sha256` is the second try. With neither, the job **refuses** rather than
running unverified code. `SBX_ALLOW_UNVERIFIED_SERVER=1` (passed through
`env=`) is the only way past it, and it can only skip a *missing tool*,
never a failed comparison.
- The bucket-mount fallback (for images with no wget or curl) reads the
digest-named file and goes through the same check. It previously read the
alias, so leaving it alone would have left the hole open on exactly the
images least likely to notice.
- `wait_ready` reads `protocol` from `/health` and raises a `SandboxError`
naming host recycling when the server is older than this client needs.
Permissive in one direction only: a newer server that still reports a
protocol this client knows is accepted; an older one, or one that reports
none, is refused.
On failing closed: a survey of the plausible base images found nothing that
lacks both hash tools. `python:3.12` and `node:22` have sha256sum and
openssl; `python:3.12-slim` has both but neither wget nor curl;
`ubuntu:26.04`, `alpine:3.22` and `busybox` have sha256sum. So the default
is expected to cost nothing in practice, and the escape hatch is there for
the image we have not thought of.
Validation:
`PYTHONPATH=src pytest tests/test_sandbox.py -q` -> 64 passed (+9).
`ruff check` / `ruff format --check` clean; `ty check src` reports the same
4 pre-existing diagnostics as on the base.
The bootstrap was also run by hand -- the real musl binary, real images, a
local `python3 -m http.server`, never the real bucket -- across 14 shapes.
Every good case exec'd (`sbx-server 0.6.0 listening on 0.0.0.0:49912`);
every corrupted case exited 1 without exec'ing:
1 python:3.12 wget, good download exit=0 EXEC'D
2 python:3.12 wget, CORRUPTED download exit=1 did NOT exec
3 python:3.12 curl only, good exit=0 EXEC'D
4 python:3.12 curl only, CORRUPTED exit=1 did NOT exec
5 python:3.12 no wget/curl, mount good exit=0 EXEC'D
6 python:3.12 no wget/curl, mount CORRUPT exit=1 did NOT exec
7 python:3.12 no sha256sum/openssl exit=1 did NOT exec
8 python:3.12 no hash tool + opt-out exit=0 EXEC'D
9 python:3.12 openssl only, good exit=0 EXEC'D
10 python:3.12 openssl only, CORRUPTED exit=1 did NOT exec
11 alpine:3.22 busybox, good exit=0 EXEC'D
12 alpine:3.22 busybox, CORRUPTED exit=1 did NOT exec
13 ubuntu:26.04 no dl tool, mount good exit=0 EXEC'D
14 ubuntu:26.04 no dl tool, mount CORRUPT exit=1 did NOT exec
A mismatch is diagnosed rather than silent, which matters because
`_tail_job_logs` is what surfaces it in the `SandboxError`:
sbx: sandbox server digest mismatch: got de7f7486...,
sbx: expected 501290ea...
sbx: refusing to run it. Upgrade huggingface_hub if this client is
sbx: pinned to a digest that is no longer published.
Not validated: no live HF Jobs run. The digest constant is also a local
musl build of the companion server branch, so it has to be replaced…
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
First step of #4802 (comment).
In practice, downstream libraries will have to change from
to
Note
Low Risk
Adds a documented re-export and documentation-only callout formatting; no changes to HTTP client behavior or request paths.
Overview
This PR is the first step toward the httpx → httpx2 migration (issue #4802). It re-exports the HTTP stack as
huggingface_hub.utils.httpxby importinghttpxinutils/__init__.py, so downstream libraries can catchhttpxexceptions and use types without pinning to a separatehttpxinstall.The utilities docs add a “The
httpxmodule” section with import/exception-handling examples, guidance to prefer [get_session] for Hub requests, and notes on v1.x (httpx) vs planned v2.x (httpx2). Doc-builder callouts were updated from<Tip>to> [!TIP]/> [!WARNING]markdown.Reviewed by Cursor Bugbot for commit 1fb8d49. Bugbot is set up for automated code reviews on this repo. Configure here.