Skip to content

Make hf jobs logs agent-friendly with --follow/-f and --tail/-n flags - #3783

Merged
hanouticelina merged 6 commits into
huggingface:mainfrom
davanstrien:jobs-logs-follow
Feb 9, 2026
Merged

hanouticelina merged 6 commits into
huggingface:mainfrom
davanstrien:jobs-logs-follow

Conversation

@davanstrien

Copy link
Copy Markdown
Member

Opening as a draft PR to start.

Currently, hf jobs logs <id> streams logs indefinitely via SSE until the job completes, with no way to opt out. This makes it harder to use from agents (e.g. Claude Code's Bash tool times out at 2 min).

I've seen Claude, etc., trying to pass --follow --no-follow to escape this behaviour so it might be better to make the pattern easier for agents?

The PR aligns hf jobs logs with Docker's docker logs convention i.e. print existing logs and exit by default, opt into streaming with -f:

  • hf jobs logs <id> — prints available logs and exits (non-blocking)
  • hf jobs logs -f <id> — streams until the job completes (follow mode)
  • hf jobs logs --tail N <id> — shows only the last N lines
Docker HF (this PR)
Print & exit docker logs <id> (default) hf jobs logs <id> (default)
Stream/follow docker logs -f <id> hf jobs logs -f <id>
Last N lines docker logs --tail N <id> hf jobs logs --tail N <id>

Discussion: CLI default behavior

Python API: No breaking change — fetch_job_logs() gains a follow parameter that defaults to True, preserving existing behavior.

CLI: This PR changes the default from streaming (blocking) to snapshot (non-blocking), matching Docker's convention. However, this is a behaviour change for existing CLI users who expect hf jobs logs <id> to stream until completion.

An alternative would be to keep the current default and require --no-follow to opt out, though this diverges from Docker's convention and keeps the agent-unfriendly default. I've seen agents try this too.

Note: hf jobs run (without --detach) is unaffected and still streams logs after launching.

cc @lhoestq (I couldn't find a previous discussion of this, but my Slack searching skills may have failed me).

Test plan

  • Unit tests added (5 new tests in TestJobsCommand)
  • make quality passes
  • Smoke tested with real jobs on HF infrastructure

`hf jobs logs` currently streams logs indefinitely via SSE until a job
completes, blocking the caller. This makes it harder to use for agents,
CI scripts, and any programmatic consumer that needs to check logs
without hanging.

This follows Docker's convention: `docker logs` prints existing logs and
exits by default, while `docker logs -f` opts into streaming. Similarly:

- `hf jobs logs <id>` now prints available logs and exits (non-blocking)
- `hf jobs logs -f <id>` streams until the job completes (blocking)
- `hf jobs logs --tail N <id>` shows only the last N lines

The Python API adds a `follow` parameter to `fetch_job_logs()` (default
`True` for backward compatibility). The CLI defaults to `follow=False`.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@davanstrien

Copy link
Copy Markdown
Member Author

cc @Wauplin for viz - happy to open issue/discuss first but thought draft PR might make it easier to discuss if it makes sense

@bot-ci-comment

bot-ci-comment Bot commented Feb 6, 2026

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@codecov

codecov Bot commented Feb 6, 2026 •

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 64.28571% with 5 lines in your changes missing coverage. Please review.
✅ Project coverage is 76.64%. Comparing base (1daa48b) to head (89f6e1e).
⚠️ Report is 73 commits behind head on main.

Files with missing lines Patch % Lines
src/huggingface_hub/hf_api.py 0.00% 4 Missing ⚠️
src/huggingface_hub/cli/jobs.py 90.00% 1 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3783      +/-   ##
==========================================
+ Coverage   75.00%   76.64%   +1.64%     
==========================================
  Files         145      153       +8     
  Lines       13978    15111    +1133     
==========================================
+ Hits        10484    11582    +1098     
- Misses       3494     3529      +35     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@Wauplin

Wauplin commented Feb 6, 2026

Copy link
Copy Markdown
Collaborator

Python API: No breaking change — fetch_job_logs() gains a follow parameter that defaults to True, preserving existing behavior.

CLI: This PR changes the default from streaming (blocking) to snapshot (non-blocking), matching Docker's convention. However, this is a behavior change for existing CLI users who expect hf jobs logs to stream until completion.

I'd be in favor of breaking fetch_job_logs default behavior as well. I don't think many people is using it directly and I'd rather have a consistent API/CLI than non breaking the API.

Per maintainer feedback, align the Python API default with the CLI:
both now default to non-blocking (follow=False). The two internal call
sites that need streaming (jobs_run, jobs_uv_run without --detach)
already pass follow=True explicitly.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
@davanstrien
davanstrien marked this pull request as ready for review February 6, 2026 14:06
@davanstrien

davanstrien commented Feb 6, 2026 •

Copy link
Copy Markdown
Member Author

Marked as ready for review, but happy to iterate on the best approach to this first. Updated to make CLI and Python consistent.

Currently

  • hf jobs run without --detach: streams logs after launching a job
  • hf jobs uv run without --detach: also streams

i.e. both call follow=True. No strong opinion but I think this might be okay? i.e. it follows the docker run convention?

Comment thread src/huggingface_hub/hf_api.py Outdated

# Short timeout: the server replays historical logs quickly, then pauses.
# 5 seconds is enough to receive all buffered logs.
read_timeout = 5

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This may not be a good assumption by Claude

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel that it's fine (smaller might miss some logs on slow environments, higher would degrade UX)

@davanstrien
davanstrien requested review from Wauplin, hanouticelina and lhoestq and removed request for lhoestq February 9, 2026 09:55

@Wauplin Wauplin left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks good! Left a few comments otherwise I'd be interested in @lhoestq 's comments regarding DX

Comment thread src/huggingface_hub/cli/jobs.py Resolved
if tail is not None:
from collections import deque

logs = deque(logs, maxlen=tail)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe raise a valueerror if both--follow and --tail are used? (they won't work together, right?)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

added this + a test. test might be overkill but maybe good to catch changes to follow etc defaults later.

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

but maybe good to catch changes to follow etc defaults later.

Yep 👍 Thanks for adding it

Comment thread src/huggingface_hub/hf_api.py Outdated
log = event["data"]
yield log

def _fetch_job_logs_no_follow(

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

instead of having a new helper, could we define add a follow parameter in _fetch_running_job_sse ?

(if not possible that's fine)

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude attempted a refactor of this. Don't think it looks too gnarly but let me know if you think the branching in _fetch_running_job_sse looks too much now.

Comment thread src/huggingface_hub/hf_api.py Outdated

# Short timeout: the server replays historical logs quickly, then pauses.
# 5 seconds is enough to receive all buffered logs.
read_timeout = 5

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I feel that it's fine (smaller might miss some logs on slow environments, higher would degrade UX)

davanstrien and others added 3 commits February 9, 2026 10:45
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Instead of a separate `_fetch_job_logs_no_follow` helper, integrate the
follow/no-follow behavior directly into `_fetch_running_job_sse`. When
`follow=False`, disables retries and breaks on timeout after collecting
buffered events.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
*,
job_id: str,
namespace: Optional[str] = None,
follow: bool = False,

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a breaking change no? we should probably set the default value to True and only the CLI should default to False

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

didn't see @Wauplin's comment. ok to break this then 👍

@Wauplin Wauplin left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

@hanouticelina hanouticelina left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

looks good! thank you

@hanouticelina
hanouticelina merged commit e009079 into huggingface:main Feb 9, 2026
28 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants