Skip to content

[CI] Reduce flaky tests (jedi cache, read-after-write, fewer reruns) - #5058

Merged
Wauplin merged 1 commit into
mainfrom
ci-reduce-flaky-tests
Oct 1, 2026
Merged

Wauplin merged 1 commit into
mainfrom
ci-reduce-flaky-tests

Conversation

@Wauplin

@Wauplin Wauplin commented Oct 1, 2026 •

Copy link
Copy Markdown
Collaborator

Summary

This PR fixes the flaky tests seen in CI over the last ~12h (2026-10-01):

  • jedi cache race: test_utils_strict_dataclass.py::test_autocompletion_* failed with EOFError: Ran out of input. parso writes its pickle cache in ~/.cache/jedi non-atomically, so one xdist worker could read a half-written file from another worker. Each test now gets its own cache directory in tmp_path.
  • Read-after-write lag on hub-ci: added a time.sleep(1) after writes, only in the tests that actually failed:
    • test_list_and_delete_lfs_files: the listing was stale after upload and after delete.
    • test_access_requests_normal_usage: the accepted list was empty after grant and not updated after cancel.
    • test_copy_files_bucket_to_different_bucket_folder: "File not found in Xet storage" right after the upload.
  • Reruns 8 → 3: in 258 jobs, only 3 tests were rerun and all passed on the second try. We don't need 8 reruns anymore.

Not addressed here: the Windows + hf_xet commit 400 "LFS pointer pointed to a file that does not exist". It hasn't happened since 12:16 UTC and is most likely server-side.

🤖 Generated with Claude Code


Note

Low Risk
Test and CI workflow changes only; no production library behavior changes.

Overview
Reduces CI flake by tightening pytest reruns from 8 to 3 on Ubuntu and Windows, since recent runs rarely needed more than one retry.

Jedi/parso cache races under pytest-xdist are addressed with an isolated_jedi_cache fixture that points jedi.settings.cache_directory at per-test tmp_path, applied to the test_autocompletion_* tests in test_utils_strict_dataclass.py.

Hub staging read-after-write lag is worked around with time.sleep(1) after writes in the tests that failed in CI: bucket copy after batch_bucket_files, LFS list/delete in test_list_and_delete_lfs_files, and grant/cancel/reject/accept steps in test_access_requests_normal_usage.

Reviewed by Cursor Bugbot for commit ca7cf63. Bugbot is set up for automated code reviews on this repo. Configure here.

…write, reruns 8 -> 3

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

@Wauplin Wauplin left a comment

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✔️ (will merge if CI gets green)

@Wauplin
Wauplin merged commit 1a76ad2 into main Oct 1, 2026
31 checks passed
@Wauplin
Wauplin deleted the ci-reduce-flaky-tests branch October 1, 2026 14:26
@bot-ci-comment

bot-ci-comment Bot commented Oct 1, 2026

Copy link
Copy Markdown

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@huggingface-hub-bot

Copy link
Copy Markdown
Contributor

This PR has been shipped as part of the v2.2.0 release.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant