Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
160 commits
Select commit Hold shift + click to select a range
5ec8e27
Post-release: bump version to 1.30.0.dev0 (#4763)
huggingface-hub-bot[bot] Aug 27, 2026
ae20a50
[CI] Enable enforcement in close-unscoped-community-prs workflow (#4766)
hanouticelina Aug 27, 2026
97655e8
Align `hf jobs scheduled ls` filtering with `hf jobs ls` (`--status`/…
moon-bot-app[bot] Aug 28, 2026
1de4817
[Download] Follow redirects between Hub hosts when resolving files (#…
hanouticelina Aug 28, 2026
4cc16bc
[Download] A `ResolvedRevision` only pins the repo it was resolved fo…
Wauplin Aug 28, 2026
94cb25d
[CI] Fix torch_1.11 case-arm label to match renamed matrix entry (#4776)
MohammedAlkindi Aug 31, 2026
6a2eb09
[Sandbox] Document experimental status (#4783)
Wauplin Sep 1, 2026
f0a5c6d
[Inference Providers] Do not require the conversational tag for chat-…
hanouticelina Sep 1, 2026
176d4f3
[CI] Disable auto close (#4800)
Wauplin Sep 2, 2026
b84090b
[Release] Consolidate the HF CLI skill sync into the release workflow…
hanouticelina Sep 3, 2026
c804cdd
Post-release: bump version to 1.31.0.dev0 (#4806)
huggingface-hub-bot[bot] Sep 3, 2026
15bb844
Bump the actions group across 1 directory with 5 updates (#4798)
dependabot[bot] Sep 4, 2026
deb9777
[Docs] Clarify Jobs argument forwarding and align examples (#4809)
davanstrien Sep 4, 2026
634ad9b
[httpx migration] Expose `httpx` as a `huggingface_hub` submodule (#4…
Wauplin Sep 7, 2026
f1bb85f
[hf_api] Make CommitInfo copy-able and pickle-able (#4822)
Wauplin Sep 7, 2026
21e687e
[Download] Prevent cached file copies during local-dir dry runs (#4817)
wakamex Sep 7, 2026
6c8c56b
Fix: keep single-quoted values verbatim in load_dotenv (#4827)
AnishPatel526 Sep 8, 2026
a9b9ac6
[CLI] Fix 'hf buckets rm --recursive' deleting lexical siblings of th…
Wauplin Sep 8, 2026
01fe4bc
[Sandbox] Support custom Job labels (#4824)
Wauplin Sep 8, 2026
204b717
[CLI] Don't let `hf update` self-upgrade a pip install on Windows (#4…
Wauplin Sep 8, 2026
aa3c2fd
[HfFileSystem] Reject unsafe paths before local downloads (#4828)
Wauplin Sep 8, 2026
9d7bbfe
[Download] Share retry handling for stream entry and body failures (#…
Wauplin Sep 8, 2026
8934ec0
[Download] Tolerate missing HEAD Content-Length (#4805)
Wauplin Sep 8, 2026
b9c14d9
[Download] Write cache ref file atomically in `snapshot_download` (#4…
Wauplin Sep 8, 2026
063b37b
[CLI] Raise explicit error for shell-script extensions on Windows (#4…
hanouticelina Sep 9, 2026
6cae778
[Jobs] Add network groups to hf jobs run (#4833)
co42 Sep 9, 2026
22fe960
Fix dotenv parser truncating unquoted values containing '#' (#4842)
BenYang12 Sep 9, 2026
c4f076c
(LFS)Fix SliceFileObj.__iter__ yielding only the first 4MB chunk (#4844)
BenYang12 Sep 9, 2026
f148666
[CLI] Fix truncated command descriptions in the CLI reference (#4849)
hanouticelina Sep 9, 2026
ff4292b
Post-release: bump version to 1.32.0.dev0 (#4856)
huggingface-hub-bot[bot] Sep 10, 2026
a2af2e0
docs(jobs): scope network groups to the namespace and resource group …
co42 Sep 10, 2026
0790f7b
[Community] Fix docstring for DiscussionComment.last_edited_by (#4864)
Saniyagupte Sep 10, 2026
bf85073
fix(sandbox): preserve terminal command results (#4851)
evalstate Sep 10, 2026
819cd77
[CLI] Read a UV script's `[tool.hf-jobs]` launch config (opus-generat…
Wauplin Sep 10, 2026
b853ec6
Add `--container-registry-{username,password}` (#4854)
alvarobartt Sep 10, 2026
e203a7f
fix(serialization): save_torch_state_dict cleanup regex silently matc…
g0rdonL Sep 10, 2026
3493edb
Bump the actions group with 4 updates (#4848)
dependabot[bot] Sep 10, 2026
4110cac
Key the doc build concurrency group on the pull request number (#4876)
albertvillanova Sep 11, 2026
edb2c83
[Buckets] Fix file metadata size on redirects (#4874)
davanstrien Sep 11, 2026
5694eda
[RepoCard] Fix AttributeError when model-index has source: null (#4870)
Saniyagupte Sep 11, 2026
b2678ec
[Spaces] Fix AttributeError in SpaceRuntime when hardware is null (#4…
Saniyagupte Sep 11, 2026
8a408cd
[CI] Make CI less flaky and stop 6h hung jobs (#4830)
Wauplin Sep 11, 2026
dd6919c
[Serialization] Harden torch checkpoint deserialization (#4877)
Wauplin Sep 11, 2026
9588035
feat(jobs): expose the network group on JobInfo (#4862)
co42 Sep 11, 2026
129bbb5
[Http] Fix race in get_session() creating multiple clients (#4868)
hanouticelina Sep 11, 2026
98e2db8
[CLI] Make --format human override agent auto-detection (#4878)
hanouticelina Sep 14, 2026
c5ff040
[Serialization] Sync `_get_dtype_size` with safetensors (#4902)
Wauplin Sep 14, 2026
f1e70a2
[Safetensors] Recognize proposed U3 metadata (#4898)
7dsolv Sep 14, 2026
fb21643
[Download] Fix silently disabled tree cache on long Windows paths (#4…
askalf Sep 14, 2026
490f06e
docs: make the sandbox security contract match the implementation (#4…
Wauplin Sep 14, 2026
1fb5b3b
feat(sandbox): use each pooled sandbox's own capability token (#4832)
Wauplin Sep 14, 2026
ef2662f
fix(sandbox): validate a pool host before sending it a credential (#4…
Wauplin Sep 14, 2026
40efc5e
fix(sandbox): bind the pool cache to the context that wrote it (#4838)
Wauplin Sep 14, 2026
0613ead
fix(sandbox): bound what a transfer or a command's output costs the c…
Wauplin Sep 14, 2026
c7e60d8
fix(sandbox): decide host teardown per host, and report it honestly (…
Wauplin Sep 14, 2026
ec61993
fix(sandbox): address background processes by their server-assigned i…
Wauplin Sep 14, 2026
0219ad5
[sandbox audit] Pin the sandbox server binary by digest and verify it…
Wauplin Sep 14, 2026
1173050
[sandbox audit] Keep secret values out of argv in the CLI and fix the…
Wauplin Sep 14, 2026
48cdc03
Fix doubled colon in snapshot download transfer bar description (#4890)
sandeepb2013 Sep 14, 2026
a3b62f6
[CLI] Add `hf --skills` as an alias for `hf skills preview` (#4906)
Wauplin Sep 14, 2026
bd9ece0
[CLI] Support file URIs in hf cache rm (#4847)
hanouticelina Sep 14, 2026
0a57df9
[Core] Restrict repo_id validation regex to ASCII word characters (#4…
bodapatisaikrishna Sep 15, 2026
af6b41e
[Core] Reject path traversal via embedded ".." segments in path_in_re…
bodapatisaikrishna Sep 15, 2026
2075fd6
[Cache] Support kernel repos in `hf cache` commands (#4905)
hanouticelina Sep 15, 2026
5a9cdda
[Core] Speed up package imports (#4914)
Wauplin Sep 15, 2026
6e3d21f
[Cache] Add cross-repo shared blob store (#4498)
hanouticelina Sep 17, 2026
f94ea55
[Buckets] Send mtime when copying files (#4920)
Wauplin Sep 17, 2026
d7b62c7
[CLI] Fix card data serialization (#4882)
davanstrien Sep 17, 2026
79dd41e
[Docs] Fix hf discussions info options that do not exist (#4886)
rishabhsai Sep 17, 2026
b064a14
[Download] Send X-HF-Download-Counter header on download calls (#4613)
Wauplin Sep 17, 2026
e9c55ce
Post-release: bump version to 1.33.0.dev0 (#4926)
huggingface-hub-bot[bot] Sep 17, 2026
618250b
[Download] Publish symlink support only after probing (#4923)
Wauplin Sep 17, 2026
77b6b2b
[Core] Skip invalid eval results entries instead of raising (#4924)
davanstrien Sep 17, 2026
34ae660
[Cache] Use posix paths for scan_cache_dir ref names (#4925)
gokay-ai Sep 18, 2026
0ba990a
[Download] Measure http_get progress from the initial file position (…
Wauplin Sep 18, 2026
cea607a
[Serialization] Fix device type when map_location is None (#4938)
Wauplin Sep 18, 2026
83ba61a
[CLI] Honor --format / --json / -q on jobs run commands (#4936)
Wauplin Sep 18, 2026
8c9546f
[Quality] Replace `re.match` with `re.fullmatch` / `re.search` and ba…
Wauplin Sep 18, 2026
22b5e87
[Docs] Sync provider task table with the PROVIDERS registry (#4930)
rishabhsai Sep 18, 2026
8efc0a9
[HfFileSystem] Make lpath optional in get_file (fsspec>=2026.9.0) (#4…
assafvayner Sep 21, 2026
4c80e7e
Fix `format_timesince` showing "60 seconds ago" / "60 minutes ago" / …
Sreekant13 Sep 21, 2026
55668e2
Fix Windows conftest erroring test subsets on lazy-loaded file_downlo…
dltsum Sep 21, 2026
7ce68b9
[Webhooks] Allow `bucket` as a watched item type (#4958)
davanstrien Sep 21, 2026
7025391
[MCP] Follow nextCursor when listing tools in MCPClient (#4960)
Kayvan-Zahiri Sep 22, 2026
80aef09
Align the `hf` package's python_requires with huggingface_hub (#4959)
sharziki Sep 22, 2026
94cf392
Remove dots from allowed characters in job labels. (#4962)
evalstate Sep 22, 2026
8d4fba6
[CLI] Honor `CLAUDE_CONFIG_DIR` in `hf skills add -g --claude` (#4957)
hanouticelina Sep 22, 2026
aa8742c
[CLI] Always install skills for Claude Code, deprecate --claude (#4967)
hanouticelina Sep 22, 2026
ce69fdb
[Buckets] Raise when a batch reports failed operations (#4954)
hanouticelina Sep 22, 2026
0f88f32
[Download] Extend tmp .incomplete path on Windows when too long (#4978)
Wauplin Sep 23, 2026
7532272
Fix HfFileSystem.get_file with a bare local filename (#4981)
davanstrien Sep 23, 2026
da5e244
Scope GITHUB_TOKEN permissions per job (#4982)
hf-security-analysis[bot] Sep 23, 2026
1e0c305
Send PrivateLink config under privateService in create_inference_endp…
co42 Sep 23, 2026
b904b94
Apply the repo_id length limit to the namespace as well (#4988)
anishmehta24 Sep 24, 2026
ac3ed2f
[Xet] Show Validating progress during shard finalization (#4478)
seanses Sep 24, 2026
fa26f34
[CI] Support major (pre-)releases in release workflow (#4991)
Wauplin Sep 24, 2026
c2e6e0a
Post-release: bump version to 1.34.0.dev0 (#4993)
huggingface-hub-bot[bot] Sep 24, 2026
9f51867
[Release] Prepare huggingface_hub 2.0 (#4994)
Wauplin Sep 24, 2026
89ae51f
unbump
Wauplin Sep 24, 2026
f5b624c
Post-release: bump version to 2.1.0.dev0 (#4999)
huggingface-hub-bot[bot] Sep 24, 2026
32456ab
[Buckets] Make include/exclude patterns case-sensitive on all platfor…
Wauplin Sep 25, 2026
c749dfa
Support secrets on Job-triggered webhooks (#5009)
moon-bot-app[bot] Sep 25, 2026
ee4f802
[HTTP] Fix repo_id in errors for query strings and download URLs (#5008)
Wauplin Sep 25, 2026
79874bc
[Buckets] Fix copy_files treating lexical siblings as an existing des…
AnishPatel526 Sep 25, 2026
ff0ccab
[CI] Bump doc-builder workflows pin to 63ec6f1 (#5010)
Wauplin Sep 25, 2026
34e8baa
[docs] xet caching and concurrency updates (#4883)
cbensimon Sep 25, 2026
62a1f2f
[HfFileSystem] Download Xet files with hf_xet in get_file (#4990)
hanouticelina Sep 25, 2026
e155ad3
[docs] Fix hf.cos typo in repository guide link (#5016)
pratikgx Sep 28, 2026
a076863
[datasets sql] Run the DuckDB CLI subprocess with UTF-8 encoding (#5020)
825pranav Sep 28, 2026
b15915b
[Auth] Store token names as UTF-8 instead of locale encoding (#5021)
Evolian-o Sep 28, 2026
41222c5
Fix scalar tags splitting into single-character list in card front ma…
zeenat28-ui Sep 28, 2026
14ee250
[CLI] Hint to use --global after local 'hf skills add/update' (#5024)
Wauplin Sep 28, 2026
f151320
[Sandbox] Require scoped credentials for pooled sandboxes (#4782)
Wauplin Sep 28, 2026
f64c26e
[Dataclasses] Accept bare typing.List/Dict/Set in @strict validation …
anishmehta24 Sep 29, 2026
7caeebb
Suggest `hf auth login` when `hf auth whoami` fails (#5033)
moon-bot-app[bot] Sep 29, 2026
5583746
[CLI] Align 'hf auth whoami' not-logged-in message with 'hf auth toke…
Wauplin Sep 29, 2026
862f23d
[Sandbox] Pin sbx-server 0.7.0 (protocol 3) (#5028)
Wauplin Sep 29, 2026
1b8b2fa
[CLI] Escape control characters in listings and table output (#5029)
Wauplin Sep 29, 2026
548f7e3
[Jobs] Allow rescheduling scheduled jobs (#5026)
Wauplin Sep 29, 2026
3fd18f2
[Jobs] Manage public and private exposed ports (#5027)
Wauplin Sep 29, 2026
231a2eb
[Jobs] Add rerun and retry attempts (#5025)
Wauplin Sep 29, 2026
d889712
[Tests] Reduce redundant Hub calls in repo settings and move tests (#…
Wauplin Sep 29, 2026
cc2f368
[Utils] Retry the first pagination page (#5035)
Wauplin Sep 29, 2026
1092497
[Spaces] Update docs for cpu-basic gating and dev mode disabling (#5037)
Wauplin Sep 29, 2026
013afa7
[CLI] Raise if a bare --secrets NAME is not set locally in hf jobs (#…
Wauplin Sep 30, 2026
83a714e
[CLI] Update hf upload help text after upload_large_folder removal (#…
davanstrien Sep 30, 2026
881a6c4
[Docs] Fix links in i18n issue template (#5051)
pratikgx Oct 1, 2026
60fdb08
[Docs] Fix duplicated words in docstring and comments (#5052)
Ad1th Oct 1, 2026
b5e3a12
[HfApi] Support resource_group_id in duplicate_repo (#5038)
Wauplin Oct 1, 2026
3bb08e2
[Endpoints] Migrate catalog to the new /api/v1 API (#4928)
Wauplin Oct 1, 2026
4c88ea6
[Repos] Warn when duplicated repo files are still being copied (#5041)
Wauplin Oct 1, 2026
604252e
[HfApi] Add visibility to create_bucket (#5053)
hanouticelina Oct 1, 2026
0b26b50
[Spaces] Add get_zero_gpu_quota and hf spaces zero-gpu-quota (#5040)
Wauplin Oct 1, 2026
ff61343
Post-release: bump version to 2.2.0.dev0 (#5054)
huggingface-hub-bot[bot] Oct 1, 2026
b3fa3a4
[CLI] Rename 'hf spaces zero-gpu-quota' to 'hf spaces zero-gpu quota'…
Wauplin Oct 1, 2026
7da19ff
[CI] Split tests into api / transfer / inference suites (#5042)
Wauplin Oct 1, 2026
7a54109
[Tests] Always clean up staging repos and buckets created by tests (#…
Wauplin Oct 1, 2026
1a76ad2
[CI] Reduce flaky tests: isolate jedi cache, sleep before read-after-…
Wauplin Oct 1, 2026
c31df86
[Bugfix] Fix wrong exception type and unbound variable guard (#5063)
vector15-05 Oct 2, 2026
37ce057
[CLI] Upload existing paths with glob characters as-is in hf upload (…
Ganesh2325 Oct 2, 2026
0ce4427
[Download] Warn when allow/ignore patterns match no files in snapshot…
Ad1th Oct 2, 2026
a625317
[Repocard] Respect configured endpoint when validating cards (#5069)
Wauplin Oct 2, 2026
d711944
[CLI] Read env/secrets files as UTF-8 and strip the BOM (#5066)
hanouticelina Oct 2, 2026
42f9085
[Setup] Install hf-xet on Windows ARM64 (#5074)
angt Oct 5, 2026
d699913
[Docs] Fix HTTP migration and async session references (#5079)
Yuilona Oct 5, 2026
bc9b691
Bump doc-builder workflow pin (#5087)
paulinebm Oct 5, 2026
3b05c63
[Jobs] Ignore 406 from metrics endpoint for finished jobs (#5091)
hanouticelina Oct 6, 2026
e817417
Bump doc-builder pin to 9fc8a41 in build_documentation.yaml (#5093)
paulinebm Oct 6, 2026
9071b17
[Jobs] Fix no-follow logs hanging while the job keeps printing (#5092)
hanouticelina Oct 6, 2026
68becf4
[CommitScheduler] Fix race that skips the first scheduled push (#5086)
hanouticelina Oct 6, 2026
ff4c989
[CLI] Print [] from hf jobs stats --json when no jobs are running (#5…
hanouticelina Oct 6, 2026
83ce8b0
[Download] Consider port when following /resolve redirects (#4942)
assafvayner Oct 6, 2026
a5277cf
[CLI] Snapshot by default in hf jobs stats, add -f/--follow (#5090)
hanouticelina Oct 6, 2026
14a48e7
[HfFileSystem] fix `ls()` on new empty bucket (#5097)
lhoestq Oct 7, 2026
38e0b3c
[HfFileSystem] Replace cached listings instead of appending to them (…
ChihebBENCHEIKH1 Oct 7, 2026
af74167
[Tests] Wait for bucket listings after writes (#5095)
hanouticelina Oct 7, 2026
cf992a9
[HfFileSystem] Make refresh=True clear cached not-found results (#5106)
hanouticelina Oct 7, 2026
bb3a875
[Docs] Fix from_pretrained integration example (#5105)
Ayushdevo Oct 7, 2026
8abbfe5
Fix race in `_get_version` that makes concurrent downloads skip hf_xe…
BramVanroy Oct 7, 2026
3cfa593
Guard upload_pr_documentation workflow to upstream repo only
g0rdonL Oct 8, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Add --container-registry-{username,password} (huggingface#4854)
* Add `--container-registry-{username,password}`

Custom containers might live in private registries e.g., a Hugging Face
Space or a private AWS ECR, and a `username` and `password` might be
required. Given that the underlying API already supports those *only
with custom containers*, then this commit adds both
`--container-registry-username` and `--container-registry-password` in
such cases. More information about the API spec at
https://api.endpoints.huggingface.cloud/#post-/v2/endpoint/-namespace-

Signed-off-by: Alvaro Bartolome <36760800+alvarobartt@users.noreply.github.com>

* Fix `mypy`

Signed-off-by: Alvaro Bartolome <36760800+alvarobartt@users.noreply.github.com>

---------

Signed-off-by: Alvaro Bartolome <36760800+alvarobartt@users.noreply.github.com>
  • Loading branch information
alvarobartt authored Sep 10, 2026
commit b853ec66c6daa2f705d8a8d84f5a3bf55cb84e9c
18 changes: 18 additions & 0 deletions docs/source/en/guides/inference_endpoints.md
Original file line number Diff line number Diff line change
Expand Up @@ -127,6 +127,24 @@ By default the Inference Endpoint is built from a docker image provided by Huggi

The value to pass as `custom_image` is a dictionary containing a url to the docker container and configuration to run it. For more details about it, checkout the [Swagger documentation](https://api.endpoints.huggingface.cloud/#/v2%3A%3Aendpoint/create_endpoint).

For an image hosted in a private container registry, pass `container_registry_username` and, when required by the registry, `container_registry_password`. The credentials are only supported for a plain custom container:

```python
>>> endpoint = create_inference_endpoint(
... # Other endpoint and hardware arguments omitted for brevity.
... custom_image={"url": "private.registry.example/my-image:latest", "port": 8080},
... container_registry_username="user",
... container_registry_password="password",
... )
```

The equivalent CLI options are `--container-registry-username` and `--container-registry-password`:

```bash
hf endpoints deploy my-endpoint ... --custom-image private.registry.example/my-image:latest \
--container-registry-username user --container-registry-password password
```

`custom_image` also accepts the engine-specific container types supported by the API, by keying the dictionary with the engine name (`vLLM`, `vLLMNeuron`, `sGLang`, `tgi`, `tgiNeuron`, `tei`, `llamacpp`, `hfServe`, ...) instead of leaving it flat. Each engine takes the usual container fields (`url`, `port`, `healthRoute`) plus its own tuning options. Any dict without a top-level `url` is forwarded to the API untouched, so engines added to the API later work without upgrading `huggingface_hub`:

```python
Expand Down
2 changes: 2 additions & 0 deletions docs/source/en/package_reference/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -1803,6 +1803,8 @@ $ hf endpoints deploy [OPTIONS] NAME
* `--engine [custom|hf-serve|llamacpp|sglang|tei|tgi|tgi-neuron|vllm|vllm-neuron]`: Managed engine image to run --custom-image with (e.g. 'vllm'). Defaults to an arbitrary container.
* `--health-route TEXT`: Health check route exposed by the container (e.g. '/health'). Requires --custom-image.
* `--port INTEGER`: Port the container listens on (e.g. 30000). Requires --custom-image.
* `--container-registry-username TEXT`: Username used to authenticate with the registry hosting --custom-image.
* `--container-registry-password TEXT`: Optional password used to authenticate with the registry hosting --custom-image.
* `--tensor-parallel-size INTEGER`: Number of accelerators to shard a single model copy across (vLLM and SGLang engines only).
* `--data-parallel-size INTEGER`: Number of model copies to run, one per accelerator (vLLM engine only).
* `--container-command TEXT`: Override the container entrypoint, as a quoted string split into tokens (e.g. "python -m sglang.launch_server").
Expand Down
25 changes: 21 additions & 4 deletions src/huggingface_hub/_inference_endpoints.py
Original file line number Diff line number Diff line change
Expand Up @@ -18,17 +18,34 @@
logger = logging.get_logger(__name__)


def _build_endpoint_image_payload(custom_image: dict) -> dict:
def _build_endpoint_image_payload(
custom_image: dict,
*,
container_registry_username: str | None = None,
container_registry_password: str | None = None,
) -> dict:
"""Build the `model.image` payload of an Inference Endpoint from a user-provided image dict.

`model.image` is a union keyed by variant (`{"vLLM": {...}}`, `{"custom": {...}}`, ...). Only a flat
container dict has a top-level `url` (required server-side, and no variant is named `url`), so dicts with
one are wrapped in `{"custom": ...}`. Everything else is forwarded as-is, so variants added to the API
later work without a release.
"""
if "url" in custom_image:
return {"custom": custom_image}
return custom_image
image = {"custom": custom_image} if "url" in custom_image else custom_image

if container_registry_password is not None and container_registry_username is None:
raise ValueError("`container_registry_password` requires `container_registry_username`.")
if container_registry_username is None:
return image

custom_container = image.get("custom")
if not isinstance(custom_container, dict):
raise ValueError("Container registry credentials can only be set for a custom container image.")

credentials = {"username": container_registry_username}
if container_registry_password is not None:
credentials["password"] = container_registry_password
return {**image, "custom": {**custom_container, "credentials": credentials}}


# Image variants that declare `tensorParallelSize` / `dataParallelSize`. The API ignores a field a variant doesn't
Expand Down
27 changes: 27 additions & 0 deletions src/huggingface_hub/cli/inference_endpoints.py
Original file line number Diff line number Diff line change
Expand Up @@ -361,6 +361,18 @@ def deploy(
engine: EngineOpt = None,
health_route: HealthRouteOpt = None,
port: PortOpt = None,
container_registry_username: Annotated[
str | None,
Option(
help="Username used to authenticate with the registry hosting --custom-image.",
),
] = None,
container_registry_password: Annotated[
str | None,
Option(
help="Optional password used to authenticate with the registry hosting --custom-image.",
),
] = None,
tensor_parallel_size: TensorParallelSizeOpt = None,
data_parallel_size: DataParallelSizeOpt = None,
container_command: Annotated[
Expand Down Expand Up @@ -406,6 +418,8 @@ def deploy(
engine=engine,
health_route=health_route,
port=port,
container_registry_username=container_registry_username,
container_registry_password=container_registry_password,
tensor_parallel_size=tensor_parallel_size,
data_parallel_size=data_parallel_size,
)
Expand All @@ -419,6 +433,10 @@ def deploy(
params["type"] = endpoint_type
if custom_image_dict is not None:
params["custom_image"] = custom_image_dict
if container_registry_username is not None:
params["container_registry_username"] = container_registry_username
if container_registry_password is not None:
params["container_registry_password"] = container_registry_password
if container_command:
params["container_command"] = shlex.split(container_command)
if container_args:
Expand Down Expand Up @@ -782,6 +800,8 @@ def _build_custom_image(
engine: str | None = None,
health_route: str | None = None,
port: int | None = None,
container_registry_username: str | None = None,
container_registry_password: str | None = None,
tensor_parallel_size: int | None = None,
data_parallel_size: int | None = None,
) -> dict | None:
Expand All @@ -798,13 +818,20 @@ def _build_custom_image(
"--engine": engine,
"--health-route": health_route,
"--port": port,
"--container-registry-username": container_registry_username,
"--container-registry-password": container_registry_password,
"--tensor-parallel-size": tensor_parallel_size,
"--data-parallel-size": data_parallel_size,
}
if used := [flag for flag, value in image_flags.items() if value is not None]:
raise CLIError(f"--custom-image is required when using {', '.join(used)}.")
return None

if container_registry_password is not None and container_registry_username is None:
raise CLIError("--container-registry-password requires --container-registry-username.")
if engine is not None and container_registry_username is not None:
raise CLIError("Container registry credentials can only be set for a custom container without --engine.")

if engine is None and (tensor_parallel_size is not None or data_parallel_size is not None):
# Without an engine the config is a plain custom container, whose parallelism fields the API ignores
# rather than rejects. Fail here instead of deploying something that quietly runs on one accelerator.
Expand Down
21 changes: 20 additions & 1 deletion src/huggingface_hub/hf_api.py
Original file line number Diff line number Diff line change
Expand Up @@ -9385,6 +9385,8 @@ def create_inference_endpoint(
revision: str | None = None,
task: str | None = None,
custom_image: dict | None = None,
container_registry_username: str | None = None,
container_registry_password: str | None = None,
container_command: list[str] | None = None,
container_args: list[str] | None = None,
env: dict[str, str] | None = None,
Expand Down Expand Up @@ -9445,6 +9447,11 @@ def create_inference_endpoint(
`llamacpp`, `hfServe`, ...), which is forwarded as-is, or a flat dict describing a custom
container (e.g. `{"url": ..., "port": ...}`), which is sent as `{"custom": ...}` (see examples).
Defaults to the Hugging Face managed image.
container_registry_username (`str`, *optional*):
Username used to authenticate with the registry hosting a custom container image.
container_registry_password (`str`, *optional*):
Password used to authenticate with the registry hosting a custom container image. Requires
`container_registry_username`. Omitted from the API payload when not provided.
container_command (`list[str]`, *optional*):
Override the container entrypoint command (maps to `model.command` in the API payload). Works with
both managed engine images (e.g. vLLM, SGLang) and custom images.
Expand Down Expand Up @@ -9566,7 +9573,19 @@ def create_inference_endpoint(
FutureWarning,
)

image = _build_endpoint_image_payload(custom_image) if custom_image is not None else {"huggingface": {}}
image: dict[str, Any]
if custom_image is None:
if container_registry_password is not None and container_registry_username is None:
raise ValueError("`container_registry_password` requires `container_registry_username`.")
if container_registry_username is not None:
raise ValueError("`custom_image` is required when setting container registry credentials.")
image = {"huggingface": {}}
else:
image = _build_endpoint_image_payload(
custom_image,
container_registry_username=container_registry_username,
container_registry_password=container_registry_password,
)

payload: dict = {
"accountId": account_id,
Expand Down
5 changes: 3 additions & 2 deletions src/huggingface_hub/utils/_xet.py
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@
from collections.abc import Mapping
from dataclasses import dataclass
from enum import Enum
from typing import Any

import httpx

Expand Down Expand Up @@ -258,9 +259,9 @@ class XetSessionHolder:
or ``sigint_abort()`` without the GIL serialising them.
"""

def __init__(self):
def __init__(self) -> None:
self._lock = threading.Lock()
self._session = None
self._session: Any = None
self._session_pid: int | None = None

def get(self):
Expand Down
25 changes: 24 additions & 1 deletion tests/test_cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -2683,6 +2683,10 @@ def test_deploy_custom_image(self, runner: CliRunner) -> None:
"/health",
"--port",
"30000",
"--container-registry-username",
"user",
"--container-registry-password",
"secret",
"--container-args",
"--tp 8 --reasoning-parser qwen3",
"--env",
Expand All @@ -2699,6 +2703,8 @@ def test_deploy_custom_image(self, runner: CliRunner) -> None:
"port": 30000,
}
assert kwargs["container_args"] == ["--tp", "8", "--reasoning-parser", "qwen3"]
assert kwargs["container_registry_username"] == "user"
assert kwargs["container_registry_password"] == "secret"
assert "container_command" not in kwargs
assert kwargs["env"] == {"MODEL_ID": "/repository"}
assert kwargs["type"] == "authenticated"
Expand Down Expand Up @@ -3030,11 +3036,28 @@ def test_build_custom_image_warns_for_engine_without_the_field(engine: str, size
"custom_image, extra, match",
[
(None, {"engine": "vllm", "tensor_parallel_size": 8}, "--custom-image is required"),
(None, {"container_registry_username": "user"}, "--custom-image is required"),
(
IMAGE_URL,
{"container_registry_password": "secret"},
"--container-registry-password requires --container-registry-username",
),
(
IMAGE_URL,
{"engine": "vllm", "container_registry_username": "user"},
"only be set for a custom container without --engine",
),
# Without an engine key the API ignores the parallelism fields rather than rejecting them, which would
# deploy an endpoint quietly running on a single accelerator.
(IMAGE_URL, {"tensor_parallel_size": 8}, "require --engine"),
],
ids=["flags_without_image", "sizes_without_engine"],
ids=[
"flags_without_image",
"registry_credentials_without_image",
"registry_password_without_username",
"registry_credentials_with_engine",
"sizes_without_engine",
],
)
def test_build_custom_image_rejects(custom_image: str | None, extra: dict, match: str) -> None:
with pytest.raises(CLIError, match=match):
Expand Down
67 changes: 63 additions & 4 deletions tests/test_hf_api.py
Original file line number Diff line number Diff line change
Expand Up @@ -4711,16 +4711,37 @@ def test_build_endpoint_image_payload(custom_image: dict, expected_image_payload


@pytest.mark.parametrize(
"custom_image, expected_image_payload",
"custom_image, registry_credentials, expected_image_payload",
[
(None, {"huggingface": {}}),
({"vLLM": {"url": "vllm/vllm-openai:v0.23.0"}}, {"vLLM": {"url": "vllm/vllm-openai:v0.23.0"}}),
(None, {}, {"huggingface": {}}),
(
{"vLLM": {"url": "vllm/vllm-openai:v0.23.0"}},
{},
{"vLLM": {"url": "vllm/vllm-openai:v0.23.0"}},
),
(
{"url": "private.registry/image:latest", "port": 8080},
{"container_registry_username": "user", "container_registry_password": "secret"},
{
"custom": {
"url": "private.registry/image:latest",
"port": 8080,
"credentials": {"username": "user", "password": "secret"},
}
},
),
(
{"url": "private.registry/image:latest"},
{"container_registry_username": "user"},
{"custom": {"url": "private.registry/image:latest", "credentials": {"username": "user"}}},
),
],
ids=["no_custom_image", "custom_image"],
ids=["no_custom_image", "custom_image", "registry_credentials", "registry_username_only"],
)
def test_create_inference_endpoint_custom_image_payload(
mocker,
custom_image: Optional[dict],
registry_credentials: dict,
expected_image_payload: dict,
):
"""`custom_image` reaches `model.image`, and defaults to the Hugging Face managed image."""
Expand Down Expand Up @@ -4759,12 +4780,50 @@ def test_create_inference_endpoint_custom_image_payload(
task="text-generation",
namespace="Wauplin",
custom_image=custom_image,
**registry_credentials,
)

payload = mock_session.post.call_args[1]["json"]
assert payload["model"]["image"] == expected_image_payload


@pytest.mark.parametrize(
"custom_image, registry_credentials, match",
[
(None, {"container_registry_username": "user"}, "`custom_image` is required"),
(
{"url": "private.registry/image:latest"},
{"container_registry_password": "secret"},
"`container_registry_password` requires `container_registry_username`",
),
(
{"vLLM": {"url": "private.registry/image:latest"}},
{"container_registry_username": "user"},
"only be set for a custom container",
),
],
ids=["credentials_without_image", "password_without_username", "credentials_for_engine"],
)
def test_create_inference_endpoint_rejects_invalid_registry_credentials(
custom_image: dict | None, registry_credentials: dict, match: str
):
api = HfApi(endpoint=ENDPOINT_STAGING, token=TOKEN)
with pytest.raises(ValueError, match=match):
api.create_inference_endpoint(
name="test-endpoint-custom-img",
repository="meta-llama/Llama-2-7b-chat-hf",
framework="custom",
accelerator="gpu",
instance_size="medium",
instance_type="nvidia-a10g",
region="us-east-1",
vendor="aws",
namespace="Wauplin",
custom_image=custom_image,
**registry_credentials,
)


def test_create_inference_endpoint_container_command_and_args_payload(mocker):
mock_post = mocker.patch("huggingface_hub.hf_api.get_session")
mock_session = mock_post.return_value
Expand Down