Skip to content

feat: add runtime image for trtllm container build #1796

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
nv-anants merged 12 commits into from
Jul 8, 2025
Merged

feat: add runtime image for trtllm container build #1796

Changes from 1 commit
Commits
Show all changes
12 commits
Select commit Hold shift + click to select a range
67eec7e
init runtime image for trtllm
nv-anants Jul 2, 2025
48ea1ec
copy pt from base img
nv-anants Jul 3, 2025
1d1ddaf
fix stuff
nv-anants Jul 4, 2025
9f25c85
use runtime container
nv-anants Jul 7, 2025
605a169
fix mpi
nv-anants Jul 7, 2025
213b62a
fix copy
nv-anants Jul 7, 2025
e3bb46b
cleanup
nv-anants Jul 7, 2025
c696e9d
cleanup 2
nv-anants Jul 7, 2025
f3680ab
fix dev
nv-anants Jul 7, 2025
0091cf6
minor
nv-anants Jul 7, 2025
a878ccf
use arch for lib copy
nv-anants Jul 8, 2025
bcc70f5
comments
nv-anants Jul 8, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
cleanup 2
  • Loading branch information
@nv-anants
nv-anants committed Jul 7, 2025
commit c696e9d23adf2ea8846ffe292a4b2adf91d2993d
2 changes: 0 additions & 2 deletions container/Dockerfile.tensorrt_llm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -460,8 +460,6 @@ COPY --from=build /usr/local/lib/python3.12/dist-packages/packaging-${PACKAGING_
COPY --from=build /usr/local/lib/python3.12/dist-packages/flash_attn /usr/local/lib/python3.12/dist-packages/flash_attn
COPY --from=build /usr/local/lib/python3.12/dist-packages/flash_attn-${FLASH_ATTN_VER}.dist-info /usr/local/lib/python3.12/dist-packages/flash_attn-${FLASH_ATTN_VER}.dist-info
COPY --from=build /usr/local/lib/python3.12/dist-packages/flash_attn_2_cuda.cpython-312-*-linux-gnu.so /usr/local/lib/python3.12/dist-packages/
# COPY --from=build /usr/local/lib/python3.12/dist-packages/mpmath /usr/local/lib/python3.12/dist-packages/mpmath
# COPY --from=build /usr/local/lib/python3.12/dist-packages/mpmath-${MPMATH_VER}.dist-info /usr/local/lib/python3.12/dist-packages/mpmath-${MPMATH_VER}.dist-info

# Setup environment variables
ARG ARCH_ALT
Expand Down
Loading