diff --git a/.devcontainer/README.md b/.devcontainer/README.md index d1d7fb4610..1f501c1ced 100644 --- a/.devcontainer/README.md +++ b/.devcontainer/README.md @@ -225,7 +225,9 @@ git config --local gpg.program gpg1 ### Custom devcontainer.json Configuration -You can create a custom devcontainer configuration by copying the main configuration to another directory inside the `.devcontainer` directory. Below is an example where the custom name is `jensen_dev`, but feel free to name the directory whatever you want: +You can create custom devcontainer configurations by copying the main configuration to another directory inside the `.devcontainer` directory. + +#### General Custom Configuration ```bash # By convention, Dev Container will look at the project's .devcontainer//devcontainer.json file. @@ -236,29 +238,63 @@ cp .devcontainer/devcontainer.json .devcontainer/jensen_dev/devcontainer.json Common customizations include additional mounts, environment variables, IDE extensions, and build arguments. When you open a new Dev Container, you can pick from any of the `.devcontainer//devcontainer.json` files available. -### SGLANG Custom devcontainer.json Configuration (EXPERIMENTAL) +### Framework-Specific DevContainer Configuration (EXPERIMENTAL) -This is experimental. Please update/fix if you encounter problems. For sglang Dev Container, you first need to build `dynamo:latest-sglang-local-dev` image like this (wait about half an hour): +> **Warning**: This feature is experimental and may have issues. Only VLLM has been thoroughly tested. Other frameworks are still being developed and may not work as expected. -```bash -./container/build.sh --framework SGLANG --target local-dev -``` +The `copy_devcontainer.sh` script automatically generates framework-specific devcontainer configurations for different AI frameworks. -Then, make a copy of the `devcontainer.json file` to a directory of your choice. For this example, we'll just call it `sglang`: +#### Available Frameworks +- **VLLM**: VLLM framework (✅ **Verified and stable** - tested by many users) +- **TRTLLM**: TensorRT-LLM framework (🔄 **In development** - being worked on) +- **SGLANG**: SGLang framework (🔄 **In development** - being worked on) +- **NONE**: No specific framework (🔄 **In development** - being worked on) + +#### Usage ```bash -mkdir .devcontainer/sglang/ -cp -a .devcontainer/devcontainer.json .devcontainer/sglang/ +# Generate framework-specific configurations +./copy_devcontainer.sh + +# Preview changes without applying them +./copy_devcontainer.sh --dryrun + +# Force overwrite existing configurations +./copy_devcontainer.sh --force ``` -Afterwards, edit your `.devcontainer/sglang/devcontainer.json` so that the name and image correspond to SGLANG. Example: -```json - "name": "[sglang] This is my amazing custom Dev Container Development", - ... - "image": "dynamo:latest-sglang-local-dev", +#### Generated Structure ``` +.devcontainer/ +├── devcontainer.json # Original VLLM configuration +├── copy_devcontainer.sh # Original generation script +├── vllm/ +│ └── devcontainer.json # auto-generated VLLM-specific config +├── sglang/ +│ └── devcontainer.json # auto-generated SGLang-specific config +├── trtllm/ +│ └── devcontainer.json # auto-generated TRTLLM-specific config +└── none/ + └── devcontainer.json # auto-generated no-framework config +``` + +#### Building Framework Images +Before using framework-specific configurations, build the corresponding Docker images: + +```bash +# Build VLLM image (already built by default) +./container/build.sh --framework VLLM --target local-dev -Now, go to **Dev Containers: Open Folder in Container** and select `[sglang] This is my amazing custom Dev Container Development`. The post-create.sh script should be running. +# Build SGLang image +./container/build.sh --framework SGLANG --target local-dev + +# Build TRTLLM image +./container/build.sh --framework TRTLLM --target local-dev + +# Build no-framework image +./container/build.sh --framework NONE --target local-dev +``` +When you open a Dev Container, your IDE will show all available configurations from `.devcontainer/*/devcontainer.json` files. ### SSH Keys for Git Operations diff --git a/.devcontainer/copy_devcontainer.sh b/.devcontainer/copy_devcontainer.sh new file mode 100755 index 0000000000..823fcabde4 --- /dev/null +++ b/.devcontainer/copy_devcontainer.sh @@ -0,0 +1,186 @@ +#!/bin/bash +# SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +# SPDX-License-Identifier: Apache-2.0 + +# copy_devcontainer.sh - Framework-Specific DevContainer Distribution Script +# +# PURPOSE: Distributes devcontainer.json to framework-specific directories +# +# WHAT IT DOES: +# - Creates .devcontainer/{vllm,sglang,trtllm,none}/ directories +# - Copies and customizes devcontainer.json for each framework +# - Substitutes: vllm->$framework, VLLM->$framework_upper +# +# USAGE: ./copy_devcontainer.sh [--dry-run] [--force] [--silent] +# +# DIRECTORY STRUCTURE: +# +# BEFORE running the script: +# .devcontainer/ +# ├── devcontainer.json +# └── copy_devcontainer.sh +# +# AFTER running the script: +# .devcontainer/ +# ├── devcontainer.json +# ├── copy_devcontainer.sh +# ├── vllm/ +# │ └── devcontainer.json +# ├── sglang/ +# │ └── devcontainer.json +# ├── trtllm/ +# │ └── devcontainer.json +# └── none/ +# └── devcontainer.json +# +# ============================================================================== + +set -eu + +# Define base directory and source file +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +SOURCE_FILE="${SCRIPT_DIR}/devcontainer.json" +DEVCONTAINER_DIR="${SCRIPT_DIR}" + +# Define frameworks (lowercase for directory names) +FRAMEWORKS=("vllm" "sglang" "trtllm" "none") + +# Check for flags +DRYRUN=false +FORCE=false +SILENT=false +while [ $# -gt 0 ]; do + case $1 in + --dryrun|--dry-run) + DRYRUN=true + ;; + --force) + FORCE=true + ;; + --silent) + SILENT=true + ;; + --help|-h) + echo "Usage: $0 [OPTIONS]" + echo "" + echo "Options:" + echo " --dry-run, --dryrun Preview changes without making them" + echo " --force Force sync even if files already exist" + echo " --silent Suppress all output (for cron jobs)" + echo " --help, -h Show this help message" + echo "" + echo "This script copies devcontainer.json from bin/ to framework-specific" + echo "directories under .devcontainer/, customizing the Docker image for each." + exit 0 + ;; + *) + echo "Unknown option: $1" + echo "Use --help for usage information" + exit 1 + ;; + esac + shift +done + +# Function to handle dry run output +dry_run_echo() { + if [ "$SILENT" = true ]; then + return + fi + + if [ "$DRYRUN" = true ]; then + echo "[DRYRUN] $*" + else + echo "$*" + fi +} + +# Command wrapper that shows commands using set -x format and respects dry-run mode +cmd() { + if [ "$DRYRUN" = true ]; then + # Dry run mode: show command but don't execute + if [ "$SILENT" != true ]; then + echo "[DRYRUN] $*" + fi + # Return success in dryrun mode + return 0 + else + # Not dry run: execute the command + if [ "$SILENT" != true ]; then + # Show and execute + ( set -x; "$@" ) + else + # Execute silently + "$@" + fi + fi +} + +# Check if source file exists +if [ ! -f "$SOURCE_FILE" ]; then + dry_run_echo "ERROR: Source file not found at $SOURCE_FILE" + exit 1 +fi + +dry_run_echo "INFO: Distributing devcontainer.json to framework-specific directories..." +dry_run_echo "INFO: Detected frameworks: ${FRAMEWORKS[*]}" + +# Process each framework +SYNC_COUNT=0 +TEMP_OUTPUT_FILE=$(mktemp) + +for framework in "${FRAMEWORKS[@]}"; do + FRAMEWORK_DIR="${DEVCONTAINER_DIR}/${framework}" + DEST_FILE="${FRAMEWORK_DIR}/devcontainer.json" + + # Check if destination already exists (unless force flag is set) + if [ -f "$DEST_FILE" ] && [ "$FORCE" = false ] && [ "$DRYRUN" = false ]; then + dry_run_echo "INFO: Skipping ${framework} - file already exists (use --force to overwrite)" + continue + fi + + # Create framework directory if it doesn't exist + if [ ! -d "$FRAMEWORK_DIR" ]; then + cmd mkdir -p "${FRAMEWORK_DIR}" + fi + + # Apply customizations to JSON file for this framework + # Substitute: name, container name, vllm->$framework, VLLM->$framework_upper + framework_upper="${framework^^}" # Convert to uppercase for display name + repo_basename=$(basename "$(dirname "${SCRIPT_DIR}")") # Get repo basename + + sed "s|\"name\": \"|\"name\": \"[${repo_basename}] |g" "${SOURCE_FILE}" | \ + sed "s|\"--name\", \"dynamo-|\"--name\", \"${repo_basename}-|g" | \ + sed "s|vllm|${framework}|g" | \ + sed "s|VLLM|${framework_upper}|g" > "${TEMP_OUTPUT_FILE}" + + # Copy the modified file to the destination + if ! cmd cp "${TEMP_OUTPUT_FILE}" "${DEST_FILE}"; then + dry_run_echo "ERROR: Failed to copy devcontainer.json to ${DEST_FILE}" + fi + + SYNC_COUNT=$((SYNC_COUNT + 1)) +done + +# Clean up temporary file +rm -f "${TEMP_OUTPUT_FILE}" 2>/dev/null + +dry_run_echo "INFO: Distribution complete. Processed $SYNC_COUNT framework configurations." + +# Directory structure AFTER running the script: +# .devcontainer/ +# ├── devcontainer.json +# ├── copy_devcontainer.sh +# ├── vllm/ +# │ └── devcontainer.json +# ├── sglang/ +# │ └── devcontainer.json +# ├── trtllm/ +# │ └── devcontainer.json +# └── none/ +# └── devcontainer.json + +dry_run_echo "Framework-specific devcontainer.json files created in:" +for framework in "${FRAMEWORKS[@]}"; do + dry_run_echo " - ${DEVCONTAINER_DIR}/${framework}/devcontainer.json" +done diff --git a/.devcontainer/devcontainer.json b/.devcontainer/devcontainer.json index 2f6974c1fe..643ab104ea 100644 --- a/.devcontainer/devcontainer.json +++ b/.devcontainer/devcontainer.json @@ -4,11 +4,12 @@ "SPDX-FileCopyrightText: Copyright (c) 2025 NVIDIA CORPORATION & AFFILIATES. All rights reserved.", "SPDX-License-Identifier: Apache-2.0" ], - "name": "NVIDIA Dynamo Dev Container Development", + "name": "NVIDIA Dynamo VLLM Dev Container", "remoteUser": "ubuntu", // Matches our container user "updateRemoteUserUID": true, // Updates the UID of the remote user to match the host user, avoids permission errors "image": "dynamo:latest-vllm-local-dev", // Use the latest VLLM local dev image "runArgs": [ + "--name", "dynamo-vllm-devcontainer", "--gpus=all", "--network=host", "--ipc=host", diff --git a/.gitignore b/.gitignore index e3ae3c9d0b..2d02ec8d04 100644 --- a/.gitignore +++ b/.gitignore @@ -90,4 +90,8 @@ generated-values.yaml # Local build artifacts for devcontainer .build/ **/.devcontainer/.env + +# Auto-generated custom devcontainer.json directories +.devcontainer/*/ + TensorRT-LLM diff --git a/container/Dockerfile b/container/Dockerfile index 7a7b051c2f..01e0b01c4e 100644 --- a/container/Dockerfile +++ b/container/Dockerfile @@ -358,4 +358,102 @@ RUN --mount=type=bind,source=./container/launch_message.txt,target=/opt/dynamo/l echo "cat ~/.launch_screen" >> ~/.bashrc ENTRYPOINT ["/opt/nvidia/nvidia_entrypoint.sh"] -CMD [] \ No newline at end of file +CMD [] + + +################################################## +########## Dev Container Local Development ####### +################################################## +# +# PURPOSE: Local development for .devcontainer +# +# This stage adds development tools, utilities, and dependencies specifically +# needed for: +# - Local development and debugging +# - vscode/cursor development +# +# Use this stage when you need a full development environment with additional +# tooling beyond the base runtime image. + +FROM base AS local-dev + +# https://code.visualstudio.com/remote/advancedcontainers/add-nonroot-user +# Will use the default ubuntu user, but give sudo access +# Needed so files permissions aren't set to root ownership when writing from inside container + +# Don't want ubuntu to be editable, just change uid and gid. User ubuntu is hardcoded in .devcontainer +ENV USERNAME=ubuntu +ARG USER_UID=1000 +ARG USER_GID=1000 + +# Install developer utilities +RUN apt-get update -y && \ + apt-get install -y --no-install-recommends \ + # Install utilities + nvtop \ + wget \ + tmux \ + vim \ + git \ + openssh-client \ + iproute2 \ + rsync \ + zip \ + unzip \ + htop \ + # Build Dependencies + autoconf \ + automake \ + cmake \ + libtool \ + meson \ + net-tools \ + pybind11-dev \ + # Rust build dependencies + clang \ + libclang-dev \ + protobuf-compiler && \ + rm -rf /var/lib/apt/lists/* + +# Rust environment setup +ENV RUSTUP_HOME=/usr/local/rustup \ + CARGO_HOME=/usr/local/cargo \ + CARGO_TARGET_DIR=/opt/dynamo/target \ + PATH=/usr/local/cargo/bin:$PATH + +COPY --from=base $RUSTUP_HOME $RUSTUP_HOME +COPY --from=base $CARGO_HOME $CARGO_HOME + +RUN apt-get update && apt-get install -y sudo gnupg2 gnupg1 \ + && echo "$USERNAME ALL=(root) NOPASSWD:ALL" > /etc/sudoers.d/$USERNAME \ + && chmod 0440 /etc/sudoers.d/$USERNAME \ + && mkdir -p /home/$USERNAME \ + && chown -R $USERNAME:$USERNAME /home/$USERNAME \ + && rm -rf /var/lib/apt/lists/* \ + && chsh -s /bin/bash $USERNAME + +# This is a slow operation (~40s on my cpu) +# Much better than chown -R $USERNAME:$USERNAME /opt/dynamo/venv (~10min on my cpu) +COPY --from=base --chown=$USER_UID:$USER_GID ${VIRTUAL_ENV} ${VIRTUAL_ENV} +RUN chown $USERNAME:$USERNAME ${VIRTUAL_ENV} +COPY --from=base --chown=$USERNAME:$USERNAME /usr/local/bin /usr/local/bin + +# so we can use maturin develop +RUN uv pip install maturin[patchelf] + +USER $USERNAME +ENV HOME=/home/$USERNAME +ENV PYTHONPATH=$PYTHONPATH:$HOME/dynamo/components/planner/src +ENV CARGO_TARGET_DIR=$HOME/dynamo/.build/target +WORKDIR $HOME + +# https://code.visualstudio.com/remote/advancedcontainers/persist-bash-history +RUN SNIPPET="export PROMPT_COMMAND='history -a' && export HISTFILE=$HOME/.commandhistory/.bash_history" \ + && mkdir -p $HOME/.commandhistory \ + && touch $HOME/.commandhistory/.bash_history \ + && echo "$SNIPPET" >> "$HOME/.bashrc" + +RUN mkdir -p /home/$USERNAME/.cache/ + +ENTRYPOINT ["/opt/nvidia/nvidia_entrypoint.sh"] +CMD [] diff --git a/container/Dockerfile.sglang b/container/Dockerfile.sglang index b40f156c42..d3102cf68f 100644 --- a/container/Dockerfile.sglang +++ b/container/Dockerfile.sglang @@ -245,9 +245,19 @@ RUN ARCH=$(dpkg --print-architecture) && \ chmod +x /usr/local/bin/prometheus && \ rm -rf /tmp/prometheus-${PROM_VERSION}.${PLATFORM} -####################################### -########## Local Development ########## -####################################### +################################################## +########## Dev Container Local Development ####### +################################################## +# +# PURPOSE: Local development for .devcontainer +# +# This stage adds development tools, utilities, and dependencies specifically +# needed for: +# - Local development and debugging +# - vscode/cursor development +# +# Use this stage when you need a full development environment with additional +# tooling beyond the base runtime image. FROM base AS local-dev @@ -260,6 +270,44 @@ ENV USERNAME=ubuntu ARG USER_UID=1000 ARG USER_GID=1000 +# Install developer utilities +RUN apt-get update -y && \ + apt-get install -y --no-install-recommends \ + # Install utilities + nvtop \ + wget \ + tmux \ + vim \ + git \ + openssh-client \ + iproute2 \ + rsync \ + zip \ + unzip \ + htop \ + # Build Dependencies + autoconf \ + automake \ + cmake \ + libtool \ + meson \ + net-tools \ + pybind11-dev \ + # Rust build dependencies + clang \ + libclang-dev \ + protobuf-compiler && \ + rm -rf /var/lib/apt/lists/* + +# Rust environment setup +ENV RUSTUP_HOME=/usr/local/rustup \ +CARGO_HOME=/usr/local/cargo \ +CARGO_TARGET_DIR=/opt/dynamo/target \ +PATH=/usr/local/cargo/bin:$PATH + +COPY --from=base $RUSTUP_HOME $RUSTUP_HOME +COPY --from=base $CARGO_HOME $CARGO_HOME + RUN apt-get update && apt-get install -y sudo gnupg2 gnupg1 \ && echo "$USERNAME ALL=(root) NOPASSWD:ALL" > /etc/sudoers.d/$USERNAME \ && chmod 0440 /etc/sudoers.d/$USERNAME \ @@ -274,6 +322,9 @@ COPY --from=base --chown=$USER_UID:$USER_GID /opt/dynamo/venv/ /opt/dynamo/venv/ RUN chown $USERNAME:$USERNAME /opt/dynamo/venv COPY --from=base --chown=$USERNAME:$USERNAME /usr/local/bin /usr/local/bin +# so we can use maturin develop +RUN uv pip install maturin[patchelf] + USER $USERNAME ENV HOME=/home/$USERNAME ENV PYTHONPATH=/workspace/dynamo/components/planner/src:/workspace/examples/sglang:$PYTHONPATH diff --git a/container/Dockerfile.trtllm b/container/Dockerfile.trtllm index 9d846adfca..9cceefd5e1 100644 --- a/container/Dockerfile.trtllm +++ b/container/Dockerfile.trtllm @@ -509,3 +509,102 @@ RUN --mount=type=bind,source=./container/launch_message.txt,target=/workspace/la ENTRYPOINT ["/opt/nvidia/nvidia_entrypoint.sh"] CMD [] + + +################################################## +########## Dev Container Local Development ####### +################################################## +# +# PURPOSE: Local development for .devcontainer +# +# This stage adds development tools, utilities, and dependencies specifically +# needed for: +# - Local development and debugging +# - vscode/cursor development +# +# Use this stage when you need a full development environment with additional +# tooling beyond the base runtime image. + +FROM build AS local-dev + +# https://code.visualstudio.com/remote/advancedcontainers/add-nonroot-user +# Will use the default ubuntu user, but give sudo access +# Needed so files permissions aren't set to root ownership when writing from inside container + +# Don't want ubuntu to be editable, just change uid and gid. User ubuntu is hardcoded in .devcontainer +ENV USERNAME=ubuntu +ARG USER_UID=1000 +ARG USER_GID=1000 + +# Install developer utilities +RUN apt-get update -y && \ + apt-get install -y --no-install-recommends \ + # Install utilities + nvtop \ + wget \ + tmux \ + vim \ + git \ + openssh-client \ + iproute2 \ + rsync \ + zip \ + unzip \ + htop \ + # Build Dependencies + autoconf \ + automake \ + cmake \ + libtool \ + meson \ + net-tools \ + pybind11-dev \ + # Rust build dependencies + clang \ + libclang-dev \ + protobuf-compiler && \ + rm -rf /var/lib/apt/lists/* + +# Rust environment setup +ENV RUSTUP_HOME=/usr/local/rustup \ + CARGO_HOME=/usr/local/cargo \ + CARGO_TARGET_DIR=/opt/dynamo/target \ + PATH=/usr/local/cargo/bin:$PATH + +COPY --from=build $RUSTUP_HOME $RUSTUP_HOME +COPY --from=build $CARGO_HOME $CARGO_HOME + +RUN apt-get update && apt-get install -y sudo gnupg2 gnupg1 \ + && echo "$USERNAME ALL=(root) NOPASSWD:ALL" > /etc/sudoers.d/$USERNAME \ + && chmod 0440 /etc/sudoers.d/$USERNAME \ + && mkdir -p /home/$USERNAME \ + && chown -R $USERNAME:$USERNAME /home/$USERNAME \ + && rm -rf /var/lib/apt/lists/* \ + && chsh -s /bin/bash $USERNAME + +# This is a slow operation (~40s on my cpu) +# Much better than chown -R $USERNAME:$USERNAME /opt/dynamo/venv (~10min on my cpu) +COPY --from=build --chown=$USER_UID:$USER_GID ${VIRTUAL_ENV} ${VIRTUAL_ENV} +RUN chown $USERNAME:$USERNAME ${VIRTUAL_ENV} +COPY --from=build --chown=$USERNAME:$USERNAME /usr/local/bin /usr/local/bin + +# so we can use maturin develop +RUN uv pip install maturin[patchelf] + +USER $USERNAME +ENV HOME=/home/$USERNAME +ENV PATH=$VIRTUAL_ENV/bin:$PATH +ENV PYTHONPATH=$PYTHONPATH:$HOME/dynamo/components/planner/src +ENV CARGO_TARGET_DIR=$HOME/dynamo/.build/target +WORKDIR $HOME + +# https://code.visualstudio.com/remote/advancedcontainers/persist-bash-history +RUN SNIPPET="export PROMPT_COMMAND='history -a' && export HISTFILE=$HOME/.commandhistory/.bash_history" \ + && mkdir -p $HOME/.commandhistory \ + && touch $HOME/.commandhistory/.bash_history \ + && echo "$SNIPPET" >> "$HOME/.bashrc" + +RUN mkdir -p /home/$USERNAME/.cache/ + +ENTRYPOINT ["/opt/nvidia/nvidia_entrypoint.sh"] +CMD [] diff --git a/container/Dockerfile.vllm b/container/Dockerfile.vllm index f6967c4be2..0a690a6c9e 100644 --- a/container/Dockerfile.vllm +++ b/container/Dockerfile.vllm @@ -280,11 +280,11 @@ RUN --mount=type=bind,source=./container/launch_message.txt,target=/workspace/la ENTRYPOINT ["/opt/nvidia/nvidia_entrypoint.sh"] CMD [] -####################################### -########## Local Development ####### -####################################### +################################################## +########## Dev Container Local Development ####### +################################################## # -# PURPOSE: Local development +# PURPOSE: Local development for .devcontainer # # This stage adds development tools, utilities, and dependencies specifically # needed for: @@ -296,7 +296,16 @@ CMD [] FROM runtime AS local-dev -# Install utilities +# https://code.visualstudio.com/remote/advancedcontainers/add-nonroot-user +# Will use the default ubuntu user, but give sudo access +# Needed so files permissions aren't set to root ownership when writing from inside container + +# Don't want ubuntu to be editable, just change uid and gid. User ubuntu is hardcoded in .devcontainer +ENV USERNAME=ubuntu +ARG USER_UID=1000 +ARG USER_GID=1000 + +# Install developer utilities RUN apt-get update -y && \ apt-get install -y --no-install-recommends \ # Install utilities @@ -334,15 +343,6 @@ ENV RUSTUP_HOME=/usr/local/rustup \ COPY --from=dynamo_base $RUSTUP_HOME $RUSTUP_HOME COPY --from=dynamo_base $CARGO_HOME $CARGO_HOME -# https://code.visualstudio.com/remote/advancedcontainers/add-nonroot-user -# Will use the default ubuntu user, but give sudo access -# Needed so files permissions aren't set to root ownership when writing from inside container - -# Don't want ubuntu to be editable, just change uid and gid. User ubuntu is hardcoded in .devcontainer -ENV USERNAME=ubuntu -ARG USER_UID=1000 -ARG USER_GID=1000 - RUN apt-get update && apt-get install -y sudo gnupg2 gnupg1 \ && echo "$USERNAME ALL=(root) NOPASSWD:ALL" > /etc/sudoers.d/$USERNAME \ && chmod 0440 /etc/sudoers.d/$USERNAME \