Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix all readmes + added in quickstart commands
  • Loading branch information
athreesh committed Aug 5, 2025
commit 11146a89ac1f295b8f153c19151755c1f404b310
10 changes: 5 additions & 5 deletions components/backends/sglang/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,11 +43,11 @@ git checkout $(git describe --tags $(git rev-list --tags --max-count=1))

### Large Scale P/D and WideEP Features

| Feature | SGLang | Notes |
|--------------------|--------|-----------------------------------------------------------------------|
| **WideEP** | ✅/🚧 | Full support on H100s/GB200 WIP [PR](https://github.com/sgl-project/sglang/pull/7556) |
| **DP Rank Routing**| 🚧 | Direct routing supported. Process per DP rank is not supported |
| **GB200 Support** | 🚧 | WIP [PR](https://github.com/sgl-project/sglang/pull/7556) |
| Feature | SGLang | Notes |
|---------------------|--------|--------------------------------------------------------------|
| **WideEP** | ✅ | Full support on H100s/GB200 |
| **DP Rank Routing** | 🚧 | Direct routing supported. Dynamo KV router does not router to DP worker |
| **GB200 Support** | ✅ | |


## Quick Start
Expand Down
2 changes: 1 addition & 1 deletion components/backends/vllm/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ git checkout $(git describe --tags $(git rev-list --tags --max-count=1))
| Feature | vLLM | Notes |
|--------------------|------|-----------------------------------------------------------------------|
| **WideEP** | ✅ | Support for PPLX / DeepEP not verified |
| **DP Rank Routing**| ✅ | Supported via external control of DP ranks |
| **Attention DP** | ✅ | Supported via external control of DP ranks |
| **GB200 Support** | 🚧 | Container functional on main |

## Quick Start
Expand Down
2 changes: 1 addition & 1 deletion docs/architecture/dynamo_flow.md
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ Coordination and messaging support:

## Technical Implementation Details

### NIXL (NVIDIA Interchange Library):
### NIXL (NVIDIA Inference Xfer Library):
- Enables high-speed GPU-to-GPU data transfers using NVLink/PCIe
- Decode Worker publishes GPU metadata to ETCD for coordination
- PrefillWorker loads metadata to establish direct communication channels
Expand Down
50 changes: 49 additions & 1 deletion docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,55 @@ The NVIDIA Dynamo Platform is a high-performance, low-latency inference framewor

Quick Start
-----------------
Follow the :doc:`Quick Guide to install Dynamo Platform <guides/dynamo_deploy/quickstart>`.

Local Deployment
~~~~~~~~~~~~~~~~

Get started with Dynamo locally in just a few commands:

**1. Install Dynamo**

.. code-block:: bash

# Install uv (recommended Python package manager)
curl -LsSf https://astral.sh/uv/install.sh | sh

# Create virtual environment and install Dynamo
uv venv venv
source venv/bin/activate
uv pip install "ai-dynamo[sglang]" # or [vllm], [trtllm]

**2. Start etcd/NATS**

.. code-block:: bash

# Start etcd and NATS using Docker Compose
docker compose -f deploy/docker-compose.yml up -d

**3. Run Dynamo**

.. code-block:: bash

# Start the OpenAI compatible frontend
python -m dynamo.frontend

# In another terminal, start an SGLang worker
python -m dynamo.sglang.worker deepseek-ai/DeepSeek-R1-Distill-Llama-8B

**4. Test your deployment**

.. code-block:: bash

curl localhost:8080/v1/chat/completions \
-H "Content-Type: application/json" \
-d '{"model": "deepseek-ai/DeepSeek-R1-Distill-Llama-8B",
"messages": [{"role": "user", "content": "Hello!"}],
"max_tokens": 50}'

Kubernetes Deployment
~~~~~~~~~~~~~~~~~~~~~

For deployments on Kubernetes, follow the :doc:`Dynamo Platform Quickstart Guide <guides/dynamo_deploy/quickstart>`.


Dive in: Examples
Expand Down