Skip to content

feat: vllm container - CUDA 13 #4763

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
dmitry-tokarev-nv wants to merge 8 commits into
base: main
Choose a base branch
from
+65 −51
Draft

feat: vllm container - CUDA 13 #4763

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
d0e46c7
vllm container - CUDA 13
dmitry-tokarev-nv Dec 5, 2025
ba2afb9
Merge branch 'main' into dtokarev/vllm-cuda-13
dmitry-tokarev-nv Dec 5, 2025
f12a85f
bump to 25.11
dmitry-tokarev-nv Dec 5, 2025
9549ebb
Support building vllm from source on arm. Fix nixl and lmcache instal…
dmitry-tokarev-nv Dec 8, 2025
863e593
switch to upstream vllm + cherry-pick nvshmem fix
dmitry-tokarev-nv Dec 8, 2025
ef1066f
updated github workflow for vllm to use cuda 13
dmitry-tokarev-nv Dec 9, 2025
44661bd
uv pip install --no-cache setuptools_scm to build vllm from source on…
dmitry-tokarev-nv Dec 9, 2025
923500f
Disable LMcache for CUDA 13
dmitry-tokarev-nv Dec 11, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
bump to 25.11 
Signed-off-by: Dmitry Tokarev <[email protected]>
  • Loading branch information
@dmitry-tokarev-nv
dmitry-tokarev-nv committed Dec 5, 2025
commit f12a85fbcbfe0f0c8ed7b9e7b63dac3d6c9a41a8
2 changes: 1 addition & 1 deletion container/Dockerfile.vllm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ ARG PYTHON_VERSION
ARG ENABLE_KVBM

ARG RUNTIME_IMAGE="nvcr.io/nvidia/cuda-dl-base"
ARG RUNTIME_IMAGE_TAG="25.10-cuda13.0-runtime-ubuntu24.04"
ARG RUNTIME_IMAGE_TAG="25.11-cuda13.0-runtime-ubuntu24.04"
ARG CUDA_VERSION="13.0"

# Make sure to update the dependency version in pyproject.toml when updating this
Expand Down
8 changes: 4 additions & 4 deletions container/build.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -106,7 +106,7 @@ VLLM_BASE_IMAGE="nvcr.io/nvidia/cuda-dl-base"
# Please check https://github.com/ai-dynamo/dynamo/pull/1065
# for details and reproducer to manually test if the image
# can be updated to later versions.
VLLM_BASE_IMAGE_TAG="25.10-cuda13.0-devel-ubuntu24.04"
VLLM_BASE_IMAGE_TAG="25.11-cuda13.0-devel-ubuntu24.04"

NONE_BASE_IMAGE="nvcr.io/nvidia/cuda-dl-base"
NONE_BASE_IMAGE_TAG="25.01-cuda12.8-devel-ubuntu24.04"
Expand Down Expand Up @@ -511,14 +511,14 @@ BUILD_ARGS+=" --build-arg DYNAMO_COMMIT_SHA=$DYNAMO_COMMIT_SHA "
if [[ $FRAMEWORK == "VLLM" ]] && [[ "$PLATFORM" == *"linux/arm64"* ]]; then
# Set base image tag to CUDA 12.9 if using the default value (user didn't override)
if [ "$BASE_IMAGE_TAG" == "$VLLM_BASE_IMAGE_TAG" ]; then
BASE_IMAGE_TAG="25.10-cuda13.0-devel-ubuntu24.04"
BASE_IMAGE_TAG="25.11-cuda13.0-devel-ubuntu24.04"
echo "INFO: Automatically setting base-image-tag to $BASE_IMAGE_TAG for vLLM ARM64"
fi

# Add required build args if not already present
if [[ "$BUILD_ARGS" != *"RUNTIME_IMAGE_TAG"* ]]; then
BUILD_ARGS+=" --build-arg RUNTIME_IMAGE_TAG=25.10-cuda13.0-runtime-ubuntu24.04 "
echo "INFO: Automatically setting RUNTIME_IMAGE_TAG=25.10-cuda13.0-runtime-ubuntu24.04 for vLLM ARM64"
BUILD_ARGS+=" --build-arg RUNTIME_IMAGE_TAG=25.11-cuda13.0-runtime-ubuntu24.04 "
echo "INFO: Automatically setting RUNTIME_IMAGE_TAG=25.11-cuda13.0-runtime-ubuntu24.04 for vLLM ARM64"
fi

if [[ "$BUILD_ARGS" != *"CUDA_VERSION"* ]]; then
Expand Down
Loading