[BUG]: Interactive frontend cannot stop the output

[LyleLuo]

Describe the Bug

Interactive frontend cannot stop the output

Steps to Reproduce

1. uv pip install ai-dynamo[vllm]==0.4.1
2. python -m dynamo.vllm --model deepseek-ai/DeepSeek-R1-Distill-Llama-8B
3. python -m dynamo.frontend --interactive
4. input anything

Expected Behavior

The output should stop and wait for my second input.

Actual Behavior

(dynamo-vllm) root@H20-GPU-19:~# python -m dynamo.frontend --interactive
2025-09-06T09:24:28.874789Z  INFO dynamo_llm::entrypoint::input::common: Waiting for remote model..
2025-09-06T09:24:30.050880Z  INFO dynamo_llm::discovery::watcher: added model model_name="deepseek-ai/DeepSeek-R1-Distill-Llama-8B"
2025-09-06T09:24:30.050930Z  INFO dynamo_llm::entrypoint::input::common: Connected to deepseek-ai/DeepSeek-R1-Distill-Llama-8B
2025-09-06T09:24:30.050958Z  INFO dynamo_llm::entrypoint::input::text: Ctrl-c to exit
✔ User · hi
Alright, the user just said "hi." That's pretty straightforward. I should respond in a friendly and welcoming manner.

I want to make sure they feel comfortable asking for help, so I'll keep it open-ended.

Maybe I can offer assistance without being too pushy.

I should probably mention the different ways they can ask for help, like with questions or specific topics.

Let me keep it simple and conversational.

Yeah, something like, "Hello! How can I assist you today?" should work nicely.

Hello! How can I assist you today? Whether you have a question, need advice, or just want to chat, I'm here to help. Let me know what's on your mind! 😊

Hello! How can I assist you today? Whether you have a question, need advice, or just want to chat, I'm here to help. Let me know what's on your mind! 😊

(Or, if you prefer a different approach, feel free to ask!) 😊

If you have any specific questions or need information on a particular topic, feel free to ask, and I'll do my best to help you out! 😊

Example of how to continue the conversation:

You: "hi"

Me: "Hello! How can I assist you today?"

You: "I need help with something."

Me: "Of course! What would you like to know or discuss?"

You: "I'm not sure where to start."

Me: "That's okay! Let me know what you're thinking about, and I'll do my best to guide you."

You: "Okay, I have a question about..."

Me: "Sure, I'd be happy to help! What would you like to know about [topic]?"

And so on... 😊

I'm here to help! Just let me know what you need. 😊

(If you'd like a more structured or step-by-step approach, let me know, and I can adjust accordingly! 😊)

So, go ahead and share what you need help with, and I'll be here to assist you! 😊

……
……
……
cannot stop

Environment

I used a new uv venv and installed dynamo by

uv pip install ai-dynamo[vllm]==0.4.1

My driver version is 550.127.08.
GPU is H20-3e.

Additional Context

No response

Screenshots

No response

[grahamking]

@LyleLuo Is this specific to the interactive client, or do you see it in HTTP too?

[LyleLuo]

@LyleLuo Is this specific to the interactive client, or do you see it in HTTP too?

It only happens on the interactive client. Both SGLang and vllm have this bug. I also tried DeepSeek but it still won't terminate the output. HTTP is OK.

[grahamking]

Very interesting, thank you! I will try to take a look.

[grahamking]

The fix is likely to delete this: https://github.com/ai-dynamo/dynamo/blob/main/lib/llm/src/entrypoint/input/text.rs#L110-L113

[grahamking]

I originally put that in to make the raw echo engine work, which is helpful for debugging templates - it echos the chat completion in it's template. But that's not an important engine and might be removed.

[LyleLuo]

Thanks for your processing!