docs: Update EKS guide for Dynamo v0.6.0 #4448

Wenhan-Tan · 2025-11-18T22:48:44Z

Overview:

Update EKS guide for Dynamo 0.6.0

Summary by CodeRabbit

Documentation
- Added comprehensive deployment guide for EKS environments, covering cluster creation, storage configuration, platform installation, and inference graph deployment with testing procedures.

copy-pr-bot · 2025-11-18T22:48:48Z

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

github-actions · 2025-11-18T22:48:54Z

👋 Hi Wenhan-Tan! Thank you for contributing to ai-dynamo/dynamo.

Just a reminder: The NVIDIA Test Github Validation CI runs an essential subset of the testing framework to quickly catch errors.Your PR reviewers may elect to test the changes comprehensively before approving your changes.

🚀

coderabbitai · 2025-11-18T22:50:55Z

Walkthrough

Adds comprehensive EKS deployment documentation for Dynamo, covering cluster creation, EFS configuration, platform installation, inference graph deployment, and endpoint testing procedures.

Changes

Cohort / File(s)	Summary
EKS Deployment Documentation `examples/deployments/EKS/README.md`	New guide detailing deployment of Dynamo on EKS clusters, including prerequisites, cluster creation, EFS storage setup, Kubernetes platform installation, inference graph deployment, and endpoint verification

Estimated code review effort

🎯 1 (Trivial) | ⏱️ ~5 minutes

Documentation-only addition with no code or API changes
Single file addition requiring straightforward content verification

Poem

🐰 A guide through the clouds, so clear and bright,
EKS paths unfold, with storage right,
Dynamo's deployed with every step,
From cluster to inference, what perfect prep!
Documentation hops, making deployments light! 🚀

Pre-merge checks

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Description check	⚠️ Warning	The pull request description is largely incomplete, providing only an Overview section with minimal detail. Required sections like Details, Where should the reviewer start, and Related Issues are missing.	Complete the pull request description by adding Details explaining specific changes made, identifying key files for review, and including any related GitHub issue references using action keywords.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	No functions found in the changed files to evaluate docstring coverage. Skipping docstring coverage check.
Title check	✅ Passed	The title 'docs: Update EKS guide for Dynamo v0.6.0' directly and accurately summarizes the main change: updating documentation for the EKS deployment guide to reflect version 0.6.0.

Tip

📝 Customizable high-level summaries are now available in beta!

You can now customize how CodeRabbit generates the high-level summary in your pull requests — including its content, structure, tone, and formatting.

Provide your own instructions using the high_level_summary_instructions setting.
Format the summary however you like (bullet lists, tables, multi-section layouts, contributor stats, etc.).
Use high_level_summary_in_walkthrough to move the summary from the description to the walkthrough section.

Example instruction:

"Divide the high-level summary into five sections:

📝 Description — Summarize the main change in 50–60 words, explaining what was done.

📓 References — List relevant issues, discussions, documentation, or related PRs.

📦 Dependencies & Requirements — Mention any new/updated dependencies, environment variable changes, or configuration updates.

📊 Contributor Summary — Include a Markdown table showing contributions:
| Contributor | Lines Added | Lines Removed | Files Changed |

✔️ Additional Notes — Add any extra reviewer context.
Keep each section concise (under 200 words) and use bullet or numbered lists for clarity."

Note: This feature is currently in beta for Pro-tier users, and pricing will be announced later.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

coderabbitai

Actionable comments posted: 0

🧹 Nitpick comments (3)

examples/deployments/EKS/README.md (3)
1-11: Fix heading hierarchy to follow Markdown standards.

The document starts with # Dynamo Deployment on EKS (h1) but jumps directly to ### Step 1. Create EKS cluster (h3), skipping the h2 level. Heading levels should increment by only one level at a time.

Apply this diff to fix the heading hierarchy:
-# Dynamo Deployment on EKS
+# Dynamo Deployment on EKS
 
 This guide covers steps of creating an Amazon EKS cluster...
 
-[Step 1. Create EKS cluster](#step-1-create-eks-cluster)
+## Steps
 
-[Step 2. Install Dynamo Kubernetes Platform](#step-2-install-dynamo-kubernetes-platform)
+[Step 1. Create EKS cluster](#step-1-create-eks-cluster)
 
-[Step 3. Deploy Dynamo Inference Graph](#step-3-deploy-dynamo-inference-graph)
+[Step 2. Install Dynamo Kubernetes Platform](#step-2-install-dynamo-kubernetes-platform)
 
-### Step 1. Create EKS cluster
+[Step 3. Deploy Dynamo Inference Graph](#step-3-deploy-dynamo-inference-graph)
+
+## Step 1. Create EKS cluster
94-94: Fix bare URL and improve phrasing.

Line 94 contains a bare URL that should be wrapped in Markdown link syntax. Additionally, the phrasing "able to access" is more clearly expressed as "can access".

Apply this diff:
-Follow the steps to create an EFS file system: https://github.com/kubernetes-sigs/aws-efs-csi-driver/blob/master/docs/efs-create-filesystem.md. Make sure you mount subnets in the last step correctly. This will affect whether your nodes are able to access the created EFS file system.
+Follow the [steps to create an EFS file system](https://github.com/kubernetes-sigs/aws-efs-csi-driver/blob/master/docs/efs-create-filesystem.md). Make sure you mount subnets correctly in the final step. This will affect whether your nodes can access the created EFS file system.
Also applies to: 94-94

17-84: Add language identifiers to all fenced code blocks.

All code blocks lack language specifications, which impacts syntax highlighting and readability in documentation viewers. Based on context, add appropriate language identifiers: yaml for Kubernetes manifests and bash for shell commands.

Examples of the fixes needed:
-```
+```yaml
 apiVersion: eksctl.io/v1alpha5
 kind: ClusterConfig
-```
+```bash
 eksctl create cluster -f <FILENAME>.yaml
-```
+```bash
 # Create Image Pull Secret
 export DOCKER_SERVER=<ECR_REGISTRY>
Apply similar fixes to all remaining code blocks. Use yaml for Kubernetes manifests and YAML configs, and bash for shell commands and command examples.

Also applies to: 88-90, 100-114, 118-120, 126-142, 145-153, 157-159, 163-169, 177-187, 191-199, 205-396, 400-402, 406-408, 412-433, 437-455, 459-461

📜 Review details

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between c65f1ac and a649b96.

📒 Files selected for processing (1)

examples/deployments/EKS/README.md (1 hunks)

🧰 Additional context used

🪛 LanguageTool

examples/deployments/EKS/README.md

[style] ~94-~94: As a shorter alternative for ‘able to’, consider using “can”.
Context: ...ly. This will affect whether your nodes are able to access the created EFS file system. ##...

(BE_ABLE_TO)

🪛 markdownlint-cli2 (0.18.1)

examples/deployments/EKS/README.md

11-11: Heading levels should only increment by one level at a time
Expected: h2; Actual: h3

(MD001, heading-increment)

17-17: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

88-88: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

94-94: Bare URL used

(MD034, no-bare-urls)

100-100: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

118-118: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

126-126: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

145-145: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

157-157: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

163-163: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

177-177: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

191-191: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

205-205: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

400-400: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

406-406: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

412-412: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

437-437: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

459-459: Fenced code blocks should have a language specified

(MD040, fenced-code-language)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)

GitHub Check: Build and Test - dynamo

kshitizgupta21 · 2025-11-19T00:55:56Z

examples/deployments/EKS/README.md

+
+#### c) Create Dynamo Inference Graph
+
+For this example, we'll deploy `Qwen/Qwen3-32B` in disaggregated mode. We'll create 12 prefill workers with TP1 and 1 decode worker with TP4 for a total of 16 x H200 GPUs (p5en.48xlarge). Please change `<DYNAMO_TRTLLM_IMAGE>`.


let us note, this is disagg with kv router

kshitizgupta21 · 2025-11-19T01:03:22Z

examples/deployments/EKS/README.md

+        fsx: true
+
+  - name: p5en-ng
+    instanceType: p5en.48xlarge


add note saying that you can switch instance type to g6e or any other instance if p5en is not available

kshitizgupta21 · 2025-11-19T01:04:03Z

examples/deployments/EKS/README.md

+
+### Step 3. Deploy Dynamo Inference Graph
+
+#### a) Build Dynamo TRTLLM runtime image


let's also show prebuilt NGC container way, show both ways - NGC container and build image

rmccorm4 · 2025-11-19T01:05:12Z

examples/deployments/EKS/README.md

+#### b) Run below to install Dynamo Kubernetes Platform
+```
+# Install CRDs
+helm fetch https://helm.ngc.nvidia.com/nvidia/ai-dynamo/charts/dynamo-crds-0.6.0.tgz


Just curious - v0.6.1 is already out, with v0.7.0 coming soon, should this just be updated in preparation for v0.7.0 instead?

Sure we can wait for 0.7.0 since it'll be out this week

Hi Wenhan, Kshitiz - v0.7.0 is out, so this can be updated: #4448 (comment)

Can we close out this PR if updates are still needed? It'll get progressively harder to merge the longer it stays open.

biswapanda · 2025-11-20T17:38:37Z

examples/deployments/EKS/README.md

+      sharedMemory:
+        size: 64Gi
+      envs:
+        - name: DYN_LOG


Are these debug log levels intentional or we'd comment it out and leave a note to enable for diagnosis?

I already left a note for all these env vars. I can definitely comment them out

julienmancuso · 2025-11-20T17:46:38Z

examples/deployments/EKS/README.md

+  -n dynamo-system
+```
+
+#### b) Run below to install Dynamo Kubernetes Platform


I'm wondering if we shouldn't just point to the production install doc here

That works too. Probably even better

Ubuntu added 11 commits November 18, 2025 21:31

Update EKS example

e40e7b3

Update EKS example

82a7172

Update EKS example

39dace3

Update EKS example

89d8543

Update EKS example

ea6e667

Update EKS example

30c6faf

Update EKS example

241ac10

Update EKS example

c38b83d

Update EKS example

cc99a09

Update EKS example

ad06cff

Update EKS example

a649b96

Wenhan-Tan requested review from a team as code owners November 18, 2025 22:48

pull-request-size bot added the size/L label Nov 18, 2025

github-actions bot added the external-contribution Pull request is from an external contributor label Nov 18, 2025

coderabbitai bot reviewed Nov 18, 2025

View reviewed changes

Update EKS example

defa190

pull-request-size bot added size/XL and removed size/L labels Nov 18, 2025

kshitizgupta21 reviewed Nov 19, 2025

View reviewed changes

rmccorm4 changed the title ~~Wenhant update EKS guide~~ docs: Update EKS guide for Dynamo v0.6.0 Nov 19, 2025

github-actions bot added the docs label Nov 19, 2025

rmccorm4 reviewed Nov 19, 2025

View reviewed changes

Update EKS example

57e6369

biswapanda reviewed Nov 20, 2025

View reviewed changes

julienmancuso reviewed Nov 20, 2025

View reviewed changes

Update EKS example

594680e


		#### c) Create Dynamo Inference Graph

		For this example, we'll deploy `Qwen/Qwen3-32B` in disaggregated mode. We'll create 12 prefill workers with TP1 and 1 decode worker with TP4 for a total of 16 x H200 GPUs (p5en.48xlarge). Please change `<DYNAMO_TRTLLM_IMAGE>`.


		### Step 3. Deploy Dynamo Inference Graph

		#### a) Build Dynamo TRTLLM runtime image

docs: Update EKS guide for Dynamo v0.6.0 #4448

Are you sure you want to change the base?

docs: Update EKS guide for Dynamo v0.6.0 #4448

Uh oh!

Conversation

Wenhan-Tan commented Nov 18, 2025 • edited by coderabbitai bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview:

Summary by CodeRabbit

Uh oh!

copy-pr-bot bot commented Nov 18, 2025

Uh oh!

github-actions bot commented Nov 18, 2025

Uh oh!

coderabbitai bot commented Nov 18, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Estimated code review effort

Poem

Pre-merge checks

Uh oh!

coderabbitai bot left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

Wenhan-Tan commented Nov 18, 2025 •

edited by coderabbitai bot

Loading

coderabbitai bot commented Nov 18, 2025 •

edited

Loading