Skip to content

feat: set model specific prompt templates in the multimodal config files, add documentation for multimodal example deployment #1366

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
hhzhang16 merged 11 commits into from
Jun 4, 2025
Merged

feat: set model specific prompt templates in the multimodal config files, add documentation for multimodal example deployment #1366

Changes from 1 commit
Commits
Show all changes
11 commits
Select commit Hold shift + click to select a range
6f5926e
feat: use prompt-template instead of hardcoding prompt
hhzhang16 May 30, 2025
1d1176a
Merge branch 'main' of github.com:ai-dynamo/dynamo into hannahz/dep-1…
hhzhang16 Jun 2, 2025
8bcab41
docs: add deployment docs to multimodal example
hhzhang16 Jun 2, 2025
356ab7f
Merge branch 'main' of github.com:ai-dynamo/dynamo into hannahz/dep-1…
hhzhang16 Jun 3, 2025
93f2f2c
feat: add prompt template to engine args, fix gpu
hhzhang16 Jun 3, 2025
accf06a
feat: load decode worker vision model just like encode worker's
hhzhang16 Jun 3, 2025
69831ad
docs: have agg be the default in multimodal example
hhzhang16 Jun 4, 2025
cd02158
Merge branch 'main' of github.com:ai-dynamo/dynamo into hannahz/dep-1…
hhzhang16 Jun 4, 2025
14135c3
docs: minor doc change
hhzhang16 Jun 4, 2025
3f82cd2
docs: minor doc change
hhzhang16 Jun 4, 2025
555d447
docs: minor doc change
hhzhang16 Jun 4, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
docs: minor doc change
  • Loading branch information
@hhzhang16
hhzhang16 committed Jun 4, 2025
commit 14135c3e219bd68a98ce919c90054ad893f7650f
4 changes: 2 additions & 2 deletions examples/multimodal/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -195,10 +195,10 @@ DYNAMO_TAG=$(dynamo build graphs.agg:Frontend | grep "Successfully built" | awk
# Deploy to Kubernetes
export DEPLOYMENT_NAME=multimodal-agg
# For aggregated serving:
dynamo deployment create $DYNAMO_TAG -n $DEPLOYMENT_NAME -f ./configs/agg.yaml
dynamo deploy $DYNAMO_TAG -n $DEPLOYMENT_NAME -f ./configs/agg.yaml
# For disaggregated serving:
# export DEPLOYMENT_NAME=multimodal-disagg
# dynamo deployment create $DYNAMO_TAG -n $DEPLOYMENT_NAME -f ./configs/disagg.yaml
# dynamo deploy $DYNAMO_TAG -n $DEPLOYMENT_NAME -f ./configs/disagg.yaml
```

**Note**: To avoid rate limiting from unauthenticated requests to HuggingFace (HF), you can provide your `HF_TOKEN` as a secret in your deployment. See the [operator deployment guide](../../docs/guides/dynamo_deploy/operator_deployment.md#referencing-secrets-in-your-deployment) for instructions on referencing secrets like `HF_TOKEN` in your deployment configuration.
Expand Down
Loading