Skip to content

fix(vllm): warn that stream interval is not respected for now #4650

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
alec-flowers wants to merge 3 commits into
base: main
Choose a base branch
from
Open

fix(vllm): warn that stream interval is not respected for now #4650

Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
b6bdbf1
vllm - warn that stream interval is not respected for now
alec-flowers Nov 26, 2025
6c9e135
fix random pre-commit error that popped up
alec-flowers Nov 26, 2025
ae0a768
Merge branch 'main' into aflowers/stream-warning-vllm
alec-flowers Nov 26, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
vllm - warn that stream interval is not respected for now 
Signed-off-by: alec-flowers <[email protected]>
  • Loading branch information
@alec-flowers
alec-flowers committed Nov 26, 2025
commit b6bdbf1f0a96a52c8824e346e97b15c533f45dbb
7 changes: 7 additions & 0 deletions components/src/dynamo/vllm/args.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -198,6 +198,13 @@ def parse_args() -> Config:
args = parser.parse_args()
engine_args = AsyncEngineArgs.from_cli_args(args)

if hasattr(engine_args, "stream_interval") and engine_args.stream_interval != 1:
logger.warning(
"--stream-interval is currently not respected in Dynamo. "
"Dynamo uses its own post-processing implementation on the frontend, "
"bypassing vLLM's OutputProcessor buffering. "
)

if engine_args.enable_prefix_caching is None:
logger.debug(
"--enable-prefix-caching or --no-enable-prefix-caching not specified. Defaulting to True (vLLM v1 default behavior)"
Expand Down
Loading