Describe the bug
A clear and concise description of what the bug is.
To Reproduce
Steps to reproduce the behavior:
- Use vLLM to launch an OpenAI API-compatible LLM Server
uv run python -m vllm.entrypoints.openai.api_server \
--model google/gemma-4-26b-a4b-it \
--dtype bfloat16 \
--max-model-len 32768 \
--gpu-memory-utilization 0.90 \
--port 8080 \
--host 0.0.0.0 \
--enable-auto-tool-choice \
--tool-call-parser gemma4
-
Set up an OpenAI provider with Goose
- API Key:
- API Host: http://(IP address of vLLM Server):8080
- Base Path(OPENAI_BASE_PATH): v1/chat/completions
-
Select the model on vLLM Server in Goose (google/gemma-4-26b-a4b-it)
-
When you throw a request, the request is processed and the result is output to Goose.
- However, the following errors will be logged..
Expected behavior
No error is displayed.
Screenshots
- "POST /v1/chat/completions HTTP/1.1" 200 OK is output in the vLLM log, but "gpt-4o-mini does not exist." There is also an error.
(APIServer pid=39819) ERROR 04-16 14:34:18 [serving.py:193] Error with model error=ErrorInfo(message='The model `gpt-4o-mini` does not exist.', type='NotFoundError', param='model', code=404)
The entire error log:
(APIServer pid=39819) ERROR 04-16 14:34:10 [serving.py:193] Error with model error=ErrorInfo(message='The model `gpt-4o-mini` does not exist.', type='NotFoundError', param='model', code=404)
(APIServer pid=39819) INFO: 172.16.0.101:58385 - "POST /v1/chat/completions HTTP/1.1" 404 Not Found
(APIServer pid=39819) INFO: 172.16.0.101:58386 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=39819) ERROR 04-16 14:34:11 [serving.py:193] Error with model error=ErrorInfo(message='The model `gpt-4o-mini` does not exist.', type='NotFoundError', param='model', code=404)
(APIServer pid=39819) INFO: 172.16.0.101:58385 - "POST /v1/chat/completions HTTP/1.1" 404 Not Found
(APIServer pid=39819) ERROR 04-16 14:34:13 [serving.py:193] Error with model error=ErrorInfo(message='The model `gpt-4o-mini` does not exist.', type='NotFoundError', param='model', code=404)
(APIServer pid=39819) INFO: 172.16.0.101:58385 - "POST /v1/chat/completions HTTP/1.1" 404 Not Found
(APIServer pid=39819) INFO 04-16 14:34:14 [loggers.py:259] Engine 000: Avg prompt throughput: 335.7 tokens/s, Avg generation throughput: 35.0 tokens/s, Running: 1 reqs, Waiting: 0 reqs, GPU KV cache usage: 1.3%, Prefix cache hit rate: 0.0%
(APIServer pid=39819) ERROR 04-16 14:34:18 [serving.py:193] Error with model error=ErrorInfo(message='The model `gpt-4o-mini` does not exist.', type='NotFoundError', param='model', code=404)
(APIServer pid=39819) INFO: 172.16.0.101:58386 - "POST /v1/chat/completions HTTP/1.1" 404 Not Found
(APIServer pid=39819) INFO: 172.16.0.101:58386 - "POST /v1/chat/completions HTTP/1.1" 200 OK
(APIServer pid=39819) INFO 04-16 14:34:24 [loggers.py:259] Engine 000: Avg prompt throughput: 15.0 tokens/s, Avg generation throughput: 5.2 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 0.0%
(APIServer pid=39819) INFO 04-16 14:34:34 [loggers.py:259] Engine 000: Avg prompt throughput: 0.0 tokens/s, Avg generation throughput: 0.0 tokens/s, Running: 0 reqs, Waiting: 0 reqs, GPU KV cache usage: 0.0%, Prefix cache hit rate: 0.0%
(APIServer pid=39819) INFO: 172.16.0.101:58395 - "GET /v1/models HTTP/1.1" 200 OK
Please provide the following information
- vLLM Server
- OS: Red Hat Enterprise Linux 10.1
- vLLM v0.19.0
- Goose
- OS: macOS Tahoe v26.4.1
- Interface: UI
- Version: v1.29.1
Additional context
vLLM Server is located in the company's environment and is accessed over VPN.
But it's working well.
Describe the bug
A clear and concise description of what the bug is.
To Reproduce
Steps to reproduce the behavior:
uv run python -m vllm.entrypoints.openai.api_server \ --model google/gemma-4-26b-a4b-it \ --dtype bfloat16 \ --max-model-len 32768 \ --gpu-memory-utilization 0.90 \ --port 8080 \ --host 0.0.0.0 \ --enable-auto-tool-choice \ --tool-call-parser gemma4Set up an OpenAI provider with Goose
Select the model on vLLM Server in Goose (google/gemma-4-26b-a4b-it)
When you throw a request, the request is processed and the result is output to Goose.
Expected behavior
No error is displayed.
Screenshots
The entire error log:
Please provide the following information
Additional context
vLLM Server is located in the company's environment and is accessed over VPN.
But it's working well.