[data] fix: tolerate shared filesystem mkdir races

[janbernloehr]

What does this PR do ?

Tolerates transient shared-filesystem mkdir races in SFT dataset cache paths when multiple distributed ranks create the same directories concurrently.

Changelog

• Catch FileExistsError and FileNotFoundError around mkdir(parents=True, exist_ok=True) in get_dataset_root.
• Apply the same guard to the default packed parquet cache directory in FinetuningDatasetBuilder.default_pack_path.
• Add unit tests covering both leaked exception types at both mkdir call sites.

GitHub Actions CI

See the CI section in the Contributing doc for how to trigger the CI. A Nvidia developer will need to approve and trigger the CI for external contributors.

Local validation:

• uv run --no-sync pre-commit run --all-files
• uv run --no-sync ruff check src/megatron/bridge/data/datasets/sft.py src/megatron/bridge/data/builders/finetuning_dataset.py tests/unit_tests/data/datasets/test_sft.py tests/unit_tests/data/builders/test_finetuning_dataset.py
• uv run --no-sync ruff format --check src/megatron/bridge/data/datasets/sft.py src/megatron/bridge/data/builders/finetuning_dataset.py tests/unit_tests/data/datasets/test_sft.py tests/unit_tests/data/builders/test_finetuning_dataset.py
• Syntax check for changed Python files via uv run --no-sync python -c ...

Not run locally:

• uv run python -m pytest tests/unit_tests/data/datasets/test_sft.py tests/unit_tests/data/builders/test_finetuning_dataset.py because dependency sync is blocked on this macOS ARM host by nvidia-resiliency-ext==0.6.0, which only publishes Linux wheels; --no-sync lacks pytest in the generated venv.

Before your PR is "Ready for review"

Pre checks:

• Make sure you read and followed Contributor guidelines
• Did you write any new necessary tests?
• Did you add or update any necessary documentation? Not needed for this bug fix.
• Does the PR affect components that are optional to install? No.
  • Reviewer: Does the PR have correct import guards for all optional libraries?

If you haven't finished some of the above items you can still open "Draft" PR.

Additional Information

• Fixes [bug] data-layer mkdir leaks FileExistsError / FileNotFoundError on shared FS with concurrent ranks #3886

[copy-pr-bot]

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.