Skip to content

feat: add runtime image for trtllm container build #1796

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
nv-anants merged 12 commits into from
Jul 8, 2025
Merged

feat: add runtime image for trtllm container build #1796

Changes from 1 commit
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
67eec7e
init runtime image for trtllm
nv-anants Jul 2, 2025
48ea1ec
copy pt from base img
nv-anants Jul 3, 2025
1d1ddaf
fix stuff
nv-anants Jul 4, 2025
9f25c85
use runtime container
nv-anants Jul 7, 2025
605a169
fix mpi
nv-anants Jul 7, 2025
213b62a
fix copy
nv-anants Jul 7, 2025
e3bb46b
cleanup
nv-anants Jul 7, 2025
c696e9d
cleanup 2
nv-anants Jul 7, 2025
f3680ab
fix dev
nv-anants Jul 7, 2025
0091cf6
minor
nv-anants Jul 7, 2025
a878ccf
use arch for lib copy
nv-anants Jul 8, 2025
bcc70f5
comments
nv-anants Jul 8, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix dev
  • Loading branch information
@nv-anants
nv-anants committed Jul 7, 2025
commit f3680abaee4ccadf676ce1031279bff3ebfaa2cb
1 change: 0 additions & 1 deletion container/Dockerfile.tensorrt_llm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -221,7 +221,6 @@ RUN mkdir /opt/dynamo && \

# Activate virtual environment
ENV VIRTUAL_ENV=/opt/dynamo/venv
ENV PATH="${VIRTUAL_ENV}/bin:${PATH}"

# Install NIXL Python module
RUN cd /opt/nixl && uv build . --out-dir /workspace/wheels/nixl
Copy link
Contributor

@nv-kmcgill53 nv-kmcgill53 Jul 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this build able to be cached and optionally downloaded? If we already have a build with the commit this was going to use, then we can skip the build step and download and install.

Copy link
Contributor Author

@nv-anants nv-anants Jul 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Build is cached, as we force a commit sha. Download is a bit of bigger change. Ideally I would want to use the wheel built from nixl repo CI and not build anything in dynamo

Expand Down
Loading