Skip to content

build(docker): add ARM64 and DGX Spark container builds - #263

Merged
NandhaKishorM merged 3 commits into
NandhaKishorM:mainfrom
maxfield-allison:docker-arm64-spark
Sep 23, 2026
Merged

NandhaKishorM merged 3 commits into
NandhaKishorM:mainfrom
maxfield-allison:docker-arm64-spark

Conversation

@maxfield-allison

Copy link
Copy Markdown
Contributor

This is the ARM64 and DGX Spark follow-up I mentioned on #45. It adds native Linux ARM64 builds of the quickstart image, keeps CPU as the default on both architectures, and adds compose.spark.yaml to select ARM64 with CUDA 13.0 for a Spark.

docker compose -f compose.yaml -f compose.spark.yaml run --build --rm laya

PyTorch's ARM64 CUDA 13.0 wheels depend on cuSPARSELt 0.8.0 (torch 2.11) or 0.8.1 (torch 2.14), and NVIDIA's aarch64 wheels for both declare manylinux2014_sbsa inside their WHEEL file, so pip check fails. 0.9.0 fixes the tag but no torch release pins it yet. docker/check_torch.py checks the library is ELF64 AArch64 and loads it, then corrects only that tag and its RECORD hash. If another version shows up with the same problem, the build fails instead of patching it. @TheIrritainer found this first in FastLaya.

Other changes:

  • TORCH_VERSION (pinned at 2.14.0, the current release) and PYTHON_IMAGE are build args. Torch wasn't pinned, so a new release could've changed the image under an unrelated PR. The same check fails the build if the installed wheel doesn't match the requested index (cpu, cu128, cu130).
  • The Docker workflow runs the existing smoke job on ubuntu-latest and ubuntu-24.04-arm, and a spark-build job builds the cu130 image on ARM64 and checks the CUDA libraries load. There's no GPU in CI, so that stops at installation.
  • docs/docker-platforms.md has the platform notes and a command for anyone with a Spark to report real GPU results. It also says why Apple Silicon gets CPU in the container: Docker on a Mac runs Linux, so there's no MPS.

Validation:

  • tests/test_docker_torch.py covers the repair for both versions and makes sure anything else is left alone. It runs inside the image in CI.
  • docker compose config passes for every combination the workflow checks, including the Spark override.
  • The 0.8.0 and 0.8.1 aarch64 wheels are both ELF64 AArch64, with the same library path and the same tag defect.
  • Native ARM64 CPU build on a Raspberry Pi 5, capped at 2 CPUs and 4 GB: the router, criteria, entrypoint and torch-check tests pass, pip check is clean, and the quickstart's SDK request runs online and again offline. laya-serve answers /health, returns 401 without a token or with the wrong one, and 200 with all three answers for the sample request.
  • Native ARM64 cu130 build on the same machine: the build corrects the cuSPARSELt 0.8.1 tag, 2.14.0+cu130 imports, cuSPARSELt loads (801), and pip check is clean. No GPU on that machine, so this is the same installation check as the CI job.

Spark GPU inference isn't tested yet. It needs someone with the hardware, and the docs say so.

Since #234 made laya-serve its own service, compose.spark.yaml repeats the Spark settings for it the same way compose.cuda.yaml does, and it builds with the pinned torch too.

docker compose -f compose.yaml -f compose.http.yaml -f compose.spark.yaml up --build laya-serve

maxfield-allison and others added 3 commits September 23, 2026 14:42
The quickstart image builds natively on ARM64, and a DGX Spark override selects
ARM64 with CUDA 13.0. PyTorch and the Python base image become build arguments,
with torch pinned at 2.14.0, so a new torch release cannot change the image
under an unrelated PR.

PyTorch's ARM64 CUDA 13.0 wheels depend on cuSPARSELt 0.8.0 (torch 2.11) or
0.8.1 (torch 2.14), whose wheels declare an SBSA tag that pip check rejects.
docker/check_torch.py checks the library is ELF64 AArch64 and loads it before
correcting only that tag and its RECORD hash. Any other version that carries
the same tag fails the build.

CI runs the smoke job on native AMD64 and ARM64 runners and builds the Spark
image without a GPU to check its CUDA libraries load.

Co-authored-by: TheIrritainer <theirritainer@gmail.com>
NandhaKishorM#234 added laya-serve as its own service, so the Spark override repeats the
ARM64, CUDA 13.0 and GPU settings for it, and its build takes the pinned torch
version. The workflow validates the HTTP plus Spark combination, and the Spark
additions sit apart from the other jobs so later workflow changes merge cleanly.

Co-authored-by: TheIrritainer <theirritainer@gmail.com>
@NandhaKishorM

Copy link
Copy Markdown
Owner

Thank you, native ARM64 builds and a DGX Spark compose file are a great fit for running Laya locally on more hardware, and working around the aarch64 cuSPARSELt wheel tag while keeping pip check in place is careful work. Merging.

@NandhaKishorM
NandhaKishorM merged commit 21d16ba into NandhaKishorM:main Sep 23, 2026
16 checks passed
baninaveen pushed a commit to baninaveen/laya that referenced this pull request Sep 24, 2026
…M#263)

* build(docker): add ARM64 and DGX Spark container builds

The quickstart image builds natively on ARM64, and a DGX Spark override selects
ARM64 with CUDA 13.0. PyTorch and the Python base image become build arguments,
with torch pinned at 2.14.0, so a new torch release cannot change the image
under an unrelated PR.

PyTorch's ARM64 CUDA 13.0 wheels depend on cuSPARSELt 0.8.0 (torch 2.11) or
0.8.1 (torch 2.14), whose wheels declare an SBSA tag that pip check rejects.
docker/check_torch.py checks the library is ELF64 AArch64 and loads it before
correcting only that tag and its RECORD hash. Any other version that carries
the same tag fails the build.

CI runs the smoke job on native AMD64 and ARM64 runners and builds the Spark
image without a GPU to check its CUDA libraries load.

Co-authored-by: TheIrritainer <theirritainer@gmail.com>

* docs(docker): credit the cuSPARSELt finding by handle

* build(docker): run laya-serve on DGX Spark

NandhaKishorM#234 added laya-serve as its own service, so the Spark override repeats the
ARM64, CUDA 13.0 and GPU settings for it, and its build takes the pinned torch
version. The workflow validates the HTTP plus Spark combination, and the Spark
additions sit apart from the other jobs so later workflow changes merge cleanly.

Co-authored-by: TheIrritainer <theirritainer@gmail.com>

---------

Co-authored-by: TheIrritainer <theirritainer@gmail.com>
@maxfield-allison
maxfield-allison deleted the docker-arm64-spark branch September 24, 2026 12:32
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants