Repository navigation
Make hf jobs logs agent-friendly with --follow/-f and --tail/-n flags - #3783
Conversation
`hf jobs logs` currently streams logs indefinitely via SSE until a job completes, blocking the caller. This makes it harder to use for agents, CI scripts, and any programmatic consumer that needs to check logs without hanging. This follows Docker's convention: `docker logs` prints existing logs and exits by default, while `docker logs -f` opts into streaming. Similarly: - `hf jobs logs <id>` now prints available logs and exits (non-blocking) - `hf jobs logs -f <id>` streams until the job completes (blocking) - `hf jobs logs --tail N <id>` shows only the last N lines The Python API adds a `follow` parameter to `fetch_job_logs()` (default `True` for backward compatibility). The CLI defaults to `follow=False`. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
cc @Wauplin for viz - happy to open issue/discuss first but thought draft PR might make it easier to discuss if it makes sense |
|
The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update. |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #3783 +/- ##
==========================================
+ Coverage 75.00% 76.64% +1.64%
==========================================
Files 145 153 +8
Lines 13978 15111 +1133
==========================================
+ Hits 10484 11582 +1098
- Misses 3494 3529 +35 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
I'd be in favor of breaking |
Per maintainer feedback, align the Python API default with the CLI: both now default to non-blocking (follow=False). The two internal call sites that need streaming (jobs_run, jobs_uv_run without --detach) already pass follow=True explicitly. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
|
Marked as ready for review, but happy to iterate on the best approach to this first. Updated to make CLI and Python consistent. Currently
i.e. both call |
|
|
||
| # Short timeout: the server replays historical logs quickly, then pauses. | ||
| # 5 seconds is enough to receive all buffered logs. | ||
| read_timeout = 5 |
There was a problem hiding this comment.
This may not be a good assumption by Claude
There was a problem hiding this comment.
I feel that it's fine (smaller might miss some logs on slow environments, higher would degrade UX)
| if tail is not None: | ||
| from collections import deque | ||
|
|
||
| logs = deque(logs, maxlen=tail) |
There was a problem hiding this comment.
maybe raise a valueerror if both--follow and --tail are used? (they won't work together, right?)
There was a problem hiding this comment.
added this + a test. test might be overkill but maybe good to catch changes to follow etc defaults later.
There was a problem hiding this comment.
but maybe good to catch changes to follow etc defaults later.
Yep 👍 Thanks for adding it
| log = event["data"] | ||
| yield log | ||
|
|
||
| def _fetch_job_logs_no_follow( |
There was a problem hiding this comment.
instead of having a new helper, could we define add a follow parameter in _fetch_running_job_sse ?
(if not possible that's fine)
There was a problem hiding this comment.
Claude attempted a refactor of this. Don't think it looks too gnarly but let me know if you think the branching in _fetch_running_job_sse looks too much now.
|
|
||
| # Short timeout: the server replays historical logs quickly, then pauses. | ||
| # 5 seconds is enough to receive all buffered logs. | ||
| read_timeout = 5 |
There was a problem hiding this comment.
I feel that it's fine (smaller might miss some logs on slow environments, higher would degrade UX)
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Instead of a separate `_fetch_job_logs_no_follow` helper, integrate the follow/no-follow behavior directly into `_fetch_running_job_sse`. When `follow=False`, disables retries and breaks on timeout after collecting buffered events. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
| *, | ||
| job_id: str, | ||
| namespace: Optional[str] = None, | ||
| follow: bool = False, |
There was a problem hiding this comment.
this is a breaking change no? we should probably set the default value to True and only the CLI should default to False
hanouticelina
left a comment
There was a problem hiding this comment.
looks good! thank you
Opening as a draft PR to start.
Currently,
hf jobs logs <id>streams logs indefinitely via SSE until the job completes, with no way to opt out. This makes it harder to use from agents (e.g. Claude Code's Bash tool times out at 2 min).I've seen Claude, etc., trying to pass --follow --no-follow to escape this behaviour so it might be better to make the pattern easier for agents?
The PR aligns
hf jobs logswith Docker'sdocker logsconvention i.e. print existing logs and exit by default, opt into streaming with-f:hf jobs logs <id>— prints available logs and exits (non-blocking)hf jobs logs -f <id>— streams until the job completes (follow mode)hf jobs logs --tail N <id>— shows only the last N linesdocker logs <id>(default)hf jobs logs <id>(default)docker logs -f <id>hf jobs logs -f <id>docker logs --tail N <id>hf jobs logs --tail N <id>Discussion: CLI default behavior
Python API: No breaking change —
fetch_job_logs()gains afollowparameter that defaults toTrue, preserving existing behavior.CLI: This PR changes the default from streaming (blocking) to snapshot (non-blocking), matching Docker's convention. However, this is a behaviour change for existing CLI users who expect
hf jobs logs <id>to stream until completion.An alternative would be to keep the current default and require
--no-followto opt out, though this diverges from Docker's convention and keeps the agent-unfriendly default. I've seen agents try this too.Note:
hf jobs run(without--detach) is unaffected and still streams logs after launching.cc @lhoestq (I couldn't find a previous discussion of this, but my Slack searching skills may have failed me).
Test plan
TestJobsCommand)make qualitypasses