Skip to content

feat: add vLLM Dynamo image to Earthly #700

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
hhzhang16 wants to merge 24 commits into from
Closed

feat: add vLLM Dynamo image to Earthly #700

Changes from 1 commit
Commits
Show all changes
24 commits
Select commit Hold shift + click to select a range
b7fbfe3
feat: no parallelism so do some reverting of earthfile
hhzhang16 Apr 11, 2025
e5598d4
feat: copy files to sdk/cli/bin
hhzhang16 Apr 12, 2025
44bd0ac
feat: split cargo build and copying of files so that they can cache
hhzhang16 Apr 12, 2025
3a64d20
feat: fix earthfile!
hhzhang16 Apr 14, 2025
c3fe46b
Merge branch 'main' of github.com:ai-dynamo/dynamo into hannahz/earth…
hhzhang16 Apr 14, 2025
0d4c52b
Merge branch 'main' into hannahz/earthfile-improvements
hhzhang16 Apr 14, 2025
fb937ff
ci: use wheels instead of .
hhzhang16 Apr 14, 2025
42f4251
Merge branch 'hannahz/earthfile-improvements' of github.com:ai-dynamo…
hhzhang16 Apr 14, 2025
d12a06c
ci: earthfile improvements
hhzhang16 Apr 14, 2025
18eafed
ci: replace cp with symlinks
hhzhang16 Apr 15, 2025
42807cd
feat: remove README from Cargo.toml
hhzhang16 Apr 15, 2025
3f5b1ea
ci: cache build steps, slim down final image to 585MB
hhzhang16 Apr 15, 2025
95e913d
Merge branch 'main' into hannahz/earthfile-improvements
hhzhang16 Apr 15, 2025
babf871
ci: add cuda to dynamo base docker
hhzhang16 Apr 15, 2025
34d7162
ci: add Earthfile for vLLM (patched) and add dynamo-base-docker-llm E…
hhzhang16 Apr 16, 2025
6309c92
Merge branch 'main' of github.com:ai-dynamo/dynamo into hannahz/earth…
hhzhang16 Apr 16, 2025
530d886
ci: install cuda in dynamo base docker llm
hhzhang16 Apr 16, 2025
5b8eede
[wip] feat: add nixl
hhzhang16 Apr 16, 2025
0c0a46d
feat: move cuda setup to an Earthly function
hhzhang16 Apr 16, 2025
527dad7
ci: working container/Earthfile for NIXL
hhzhang16 Apr 16, 2025
36bff6e
feat: cut vLLM docker iamge down to 13GB
hhzhang16 Apr 17, 2025
5ef2587
Merge branch 'main' of github.com:ai-dynamo/dynamo into hannahz/earth…
hhzhang16 Apr 18, 2025
225d51c
feat: fix all-docker
hhzhang16 Apr 18, 2025
5b0e15f
feat: update docker server and image tag args
hhzhang16 Apr 21, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
feat: copy files to sdk/cli/bin
  • Loading branch information
@hhzhang16
hhzhang16 committed Apr 12, 2025
commit e5598d4038a6fbbfc49d095337900fd80a01e62f
16 changes: 11 additions & 5 deletions Earthfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -103,11 +103,17 @@ dynamo-base-docker:

RUN cargo build --release --locked --features mistralrs,sglang,vllm,python && \
cargo doc --no-deps && \
cp target/release/dynamo-run /usr/local/bin && \
cp target/release/http /usr/local/bin && \
cp target/release/llmctl /usr/local/bin && \
cp target/release/metrics /usr/local/bin && \
cp target/release/mock_worker /usr/local/bin
# Create directory structure for Python package
mkdir -p /workspace/deploy/dynamo/sdk/src/dynamo/sdk/cli/bin/ && \
# Copy binaries to both locations
cp target/release/dynamo-run /usr/local/bin/ && \
cp target/release/http /usr/local/bin/ && \
cp target/release/llmctl /usr/local/bin/ && \
cp target/release/metrics /usr/local/bin/ && \
cp target/release/mock_worker /usr/local/bin/ && \
cp target/release/dynamo-run /workspace/deploy/dynamo/sdk/src/dynamo/sdk/cli/bin/ && \
cp target/release/http /workspace/deploy/dynamo/sdk/src/dynamo/sdk/cli/bin/ && \
cp target/release/llmctl /workspace/deploy/dynamo/sdk/src/dynamo/sdk/cli/bin/

RUN uv build --wheel --out-dir /workspace/dist && \
uv pip install /workspace/dist/ai_dynamo*any.whl
Expand Down