Skip to content
Merged
Changes from 1 commit
Commits
Show all changes
28 commits
Select commit Hold shift + click to select a range
558482d
feat: initial benchmarking wrapper in-cluster work
hhzhang16 Sep 8, 2025
7cc6edb
Merge branch 'main' of github.com:ai-dynamo/dynamo into hannahz/dyn-9…
hhzhang16 Sep 18, 2025
5a09233
feat: update benchmark job for in-cluster benchmarking following late…
hhzhang16 Sep 18, 2025
8f19a4d
feat: update in-cluster benchmark job and yaml
hhzhang16 Sep 19, 2025
3ff6675
feat: enhance GPT OSS frontend with improved harmony tool calling par…
zhongdaor-nv Sep 18, 2025
9482320
feat(operator): mechanism for disabling imagePullSecrets discovery (#…
tmonty12 Sep 18, 2025
f7cc9e9
refactor: simplify Dockerfile.vllm, enable local-dev for all framewor…
keivenchang Sep 19, 2025
d5f0495
feat: Request Cancellation unary request support (#3004)
kthui Sep 19, 2025
1648836
build: update trtllm to v1.1.0rc5 to enable trtllm + KVBM integration…
richardhuo-nv Sep 19, 2025
91181f6
build: OPS-597, OPS-861 restructure TRT-LLM to follow container strat…
nv-tusharma Sep 19, 2025
89e074c
feat: Sglang canary health check (#3103)
tzulingk Sep 19, 2025
271ef47
feat: Convert message[content] from list to string. (#3067)
KrishnanPrash Sep 19, 2025
f79e57b
feat: KVBM connector : enabling vectorized copy from pinned memory to…
oandreeva-nv Sep 19, 2025
8ee077f
feat: update READMe commands
hhzhang16 Sep 19, 2025
4ac8147
feat: update READMe commands
hhzhang16 Sep 19, 2025
e7ed272
Merge branch 'main' of github.com:ai-dynamo/dynamo into hannahz/dyn-9…
hhzhang16 Sep 19, 2025
534ba19
docs: move in-cluster benchmarking doc to the overall benchmarking do…
hhzhang16 Sep 19, 2025
0235ece
feat: minor adjustments based on self look-through and coderabbit com…
hhzhang16 Sep 19, 2025
b392205
Merge branch 'main' of github.com:ai-dynamo/dynamo into hannahz/dyn-9…
hhzhang16 Sep 22, 2025
ef92388
docs: add benchmarking cross-namespace
hhzhang16 Sep 22, 2025
69bcfa8
docs: have user modify benchmark job instead of using envsubst
hhzhang16 Sep 22, 2025
e83590b
docs: add tldr
hhzhang16 Sep 22, 2025
efd16d6
docs: minor doc updates
hhzhang16 Sep 22, 2025
ae9e70e
Merge branch 'main' of github.com:ai-dynamo/dynamo into hannahz/dyn-9…
hhzhang16 Sep 22, 2025
5131348
docs: update k8s-related stuff in benchmarking.md
hhzhang16 Sep 23, 2025
38955ef
Merge branch 'main' into hannahz/dyn-973-allow-in-cluster-perf-benchm…
hhzhang16 Sep 23, 2025
a5e5b18
docs: updating client-side prereqs
hhzhang16 Sep 23, 2025
de853cf
Merge branch 'main' of github.com:ai-dynamo/dynamo into hannahz/dyn-9…
hhzhang16 Sep 23, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
docs: minor doc updates
Signed-off-by: Hannah Zhang <[email protected]>
  • Loading branch information
hhzhang16 committed Sep 22, 2025
commit efd16d64e4129ec24c5c22db32e5d5cfc93109ad
7 changes: 3 additions & 4 deletions docs/benchmarks/benchmarking.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,6 @@ Need high performance/load testing? Server-side.
Just quick testing/comparison? Client-side.

### Use Client-Side Benchmarking When:
- You already have an HTTP endpoint ready to go
- You want to quickly test deployments
- You want immediate access to results on your local machine
- You're comparing external services or deployments (not necessarily just Dynamo deployments)
Expand Down Expand Up @@ -305,7 +304,7 @@ The server-side benchmarking solution:

## Prerequisites

1. **Kubernetes cluster** with NVIDIA GPUs and Dynamo namespace setup (see [Dynamo Cloud/Platform docs](../../guides/dynamo_deploy/README.md))
1. **Kubernetes cluster** with NVIDIA GPUs and Dynamo namespace setup (see [Dynamo Cloud/Platform docs](../guides/dynamo_deploy/README.md))
2. **Storage and service account** PersistentVolumeClaim and service account configured with appropriate permissions (see [deploy/utils README](../../deploy/utils/README.md))
3. **Docker image** containing the Dynamo benchmarking tools

Expand All @@ -327,7 +326,7 @@ kubectl apply -f benchmarks/incluster/benchmark_job.yaml -n $NAMESPACE
kubectl logs -f job/dynamo-benchmark -n $NAMESPACE

# Check job status
kubectl get jobs -n $NAMESPACE
kubectl get job dynamo-benchmark -n $NAMESPACE
```

#### Customize the job configuration
Expand Down Expand Up @@ -365,7 +364,7 @@ SERVICE_URL=vllm-agg-frontend:8000
SERVICE_URL=vllm-agg-frontend.production.svc.cluster.local:8000
```

**DNS Format**: `service-name.namespace.svc.cluster.local:port`
**DNS Format**: `<service-name>.<namespace>.svc.cluster.local:port`

This allows you to:
- Benchmark multiple services across different namespaces in a single job
Expand Down