Skip to content

fix(vllm): warn that stream interval is not respected for now #4650

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
alec-flowers wants to merge 3 commits into
base: main
Choose a base branch
from
Open

fix(vllm): warn that stream interval is not respected for now #4650

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
b6bdbf1
vllm - warn that stream interval is not respected for now
alec-flowers Nov 26, 2025
6c9e135
fix random pre-commit error that popped up
alec-flowers Nov 26, 2025
ae0a768
Merge branch 'main' into aflowers/stream-warning-vllm
alec-flowers Nov 26, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions components/src/dynamo/vllm/args.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -198,6 +198,13 @@ def parse_args() -> Config:
args = parser.parse_args()
engine_args = AsyncEngineArgs.from_cli_args(args)

if hasattr(engine_args, "stream_interval") and engine_args.stream_interval != 1:
logger.warning(
"--stream-interval is currently not respected in Dynamo. "
"Dynamo uses its own post-processing implementation on the frontend, "
"bypassing vLLM's OutputProcessor buffering. "
)

if engine_args.enable_prefix_caching is None:
logger.debug(
"--enable-prefix-caching or --no-enable-prefix-caching not specified. Defaulting to True (vLLM v1 default behavior)"
Expand Down
3 changes: 2 additions & 1 deletion ..._tolerance/hardware/fault-injection-service/agents/gpu-fault-injector/gpu_xid_injector.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -110,7 +110,8 @@
57: "Clocks Event: Power limit exceeded",
# Common Graphics XIDs (often seen in test environments)
13: "Graphics Engine Exception",
31: "GPU stopped responding", # Can be both MMU or timeout context
# 31: "GPU stopped responding",
# Note: XID 31 is already defined above as "MMU Error" (can also be GPU timeout)
45: "Preemptive Cleanup, due to previous errors",
69: "Graphics Exception: Class Error",
}
Expand Down
Loading