无法正常使用deepseek-vl-chat-pytorch-1_3b.

[WindLWQ]

System Info / 系統信息

正常下载导入deepseek-vl-chat-pytorch-1_3b.在chatbox中与模型对话时，发生如下报错

2025-02-13 11:04:48,128 xinference.api.restful_api 3829897 ERROR    Handling request http://localhost:9997/deepseek-vl-chat/run/predict failed: Unable to generate pydantic-core schema for <class 'starlette.requests.Request'>. Set `arbitrary_types_allowed=True` in the model_config to ignore this error or implement `__get_pydantic_core_schema__` on your type to fully support it.

If you got this error by calling handler(<some type>) within `__get_pydantic_core_schema__` then you likely need to call `handler.generate_schema(<some type>)` since we do not call `__get_pydantic_core_schema__` on `<some type>` otherwise to avoid infinite recursion.

For further information visit https://errors.pydantic.dev/2.10/u/schema-for-unknown-type
Traceback (most recent call last):
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/pydantic/type_adapter.py", line 271, in _init_core_attrs
   self.core_schema = _getattr_no_parents(self._type, '__pydantic_core_schema__')
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/pydantic/type_adapter.py", line 55, in _getattr_no_parents
   raise AttributeError(attribute)
AttributeError: __pydantic_core_schema__

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/starlette/middleware/errors.py", line 165, in __call__
   await self.app(scope, receive, _send)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/aioprometheus/asgi/middleware.py", line 184, in __call__
   await self.asgi_callable(scope, receive, wrapped_send)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/starlette/middleware/cors.py", line 93, in __call__
   await self.simple_response(scope, receive, send, request_headers=headers)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/starlette/middleware/cors.py", line 144, in simple_response
   await self.app(scope, receive, send)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
   await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
   raise exc
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
   await app(scope, receive, sender)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/starlette/routing.py", line 715, in __call__
   await self.middleware_stack(scope, receive, send)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/starlette/routing.py", line 735, in app
   await route.handle(scope, receive, send)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/starlette/routing.py", line 460, in handle
   await self.app(scope, receive, send)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/fastapi/applications.py", line 1054, in __call__
   await super().__call__(scope, receive, send)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/starlette/applications.py", line 112, in __call__
   await self.middleware_stack(scope, receive, send)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/starlette/middleware/errors.py", line 187, in __call__
   raise exc
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/starlette/middleware/errors.py", line 165, in __call__
   await self.app(scope, receive, _send)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/gradio/route_utils.py", line 695, in __call__
   await self.simple_response(scope, receive, send, request_headers=headers)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/gradio/route_utils.py", line 711, in simple_response
   await self.app(scope, receive, send)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
   await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
   raise exc
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
   await app(scope, receive, sender)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/starlette/routing.py", line 715, in __call__
   await self.middleware_stack(scope, receive, send)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/starlette/routing.py", line 735, in app
   await route.handle(scope, receive, send)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/starlette/routing.py", line 288, in handle
   await self.app(scope, receive, send)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/starlette/routing.py", line 76, in app
   await wrap_app_handling_exceptions(app, request)(scope, receive, send)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
   raise exc
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
   await app(scope, receive, sender)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/starlette/routing.py", line 73, in app
   response = await f(request)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/fastapi/routing.py", line 291, in app
   solved_result = await solve_dependencies(
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/fastapi/dependencies/utils.py", line 666, in solve_dependencies
   ) = await request_body_to_args(  # body_params checked above
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/fastapi/dependencies/utils.py", line 891, in request_body_to_args
   fields_to_extract = get_cached_model_fields(first_field.type_)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/fastapi/_compat.py", line 659, in get_cached_model_fields
   return get_model_fields(model)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/fastapi/_compat.py", line 285, in get_model_fields
   return [
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/fastapi/_compat.py", line 286, in <listcomp>
   ModelField(field_info=field_info, name=name)
 File "<string>", line 6, in __init__
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/fastapi/_compat.py", line 111, in __post_init__
   self._type_adapter: TypeAdapter[Any] = TypeAdapter(
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/pydantic/type_adapter.py", line 228, in __init__
   self._init_core_attrs(
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/pydantic/type_adapter.py", line 290, in _init_core_attrs
   core_schema = schema_generator.generate_schema(self._type)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 610, in generate_schema
   schema = self._generate_schema_inner(obj)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 863, in _generate_schema_inner
   return self._annotated_schema(obj)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 1977, in _annotated_schema
   schema = self._apply_annotations(source_type, annotations)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 2056, in _apply_annotations
   schema = get_inner_schema(source_type)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/pydantic/_internal/_schema_generation_shared.py", line 84, in __call__
   schema = self._handler(source_type)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 2131, in new_handler
   schema = metadata_get_schema(source, get_inner_schema)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 2127, in <lambda>
   lambda source, handler: handler(source)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/pydantic/_internal/_schema_generation_shared.py", line 84, in __call__
   schema = self._handler(source_type)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 2037, in inner_handler
   schema = self._generate_schema_inner(obj)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 884, in _generate_schema_inner
   return self.match_type(obj)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 986, in match_type
   return self._match_generic_type(obj, origin)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 1014, in _match_generic_type
   return self._union_schema(obj)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 1325, in _union_schema
   choices.append(self.generate_schema(arg))
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 610, in generate_schema
   schema = self._generate_schema_inner(obj)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 884, in _generate_schema_inner
   return self.match_type(obj)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 995, in match_type
   return self._unknown_type_schema(obj)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 513, in _unknown_type_schema
   raise PydanticSchemaGenerationError(
pydantic.errors.PydanticSchemaGenerationError: Unable to generate pydantic-core schema for <class 'starlette.requests.Request'>. Set `arbitrary_types_allowed=True` in the model_config to ignore this error or implement `__get_pydantic_core_schema__` on your type to fully support it.

If you got this error by calling handler(<some type>) within `__get_pydantic_core_schema__` then you likely need to call `handler.generate_schema(<some type>)` since we do not call `__get_pydantic_core_schema__` on `<some type>` otherwise to avoid infinite recursion.

For further information visit https://errors.pydantic.dev/2.10/u/schema-for-unknown-type
2025-02-13 11:04:49,153 xinference.api.restful_api 3829897 ERROR    Handling request http://localhost:9997/deepseek-vl-chat/queue/join failed: Unable to generate pydantic-core schema for <class 'starlette.requests.Request'>. Set `arbitrary_types_allowed=True` in the model_config to ignore this error or implement `__get_pydantic_core_schema__` on your type to fully support it.

If you got this error by calling handler(<some type>) within `__get_pydantic_core_schema__` then you likely need to call `handler.generate_schema(<some type>)` since we do not call `__get_pydantic_core_schema__` on `<some type>` otherwise to avoid infinite recursion.

For further information visit https://errors.pydantic.dev/2.10/u/schema-for-unknown-type
Traceback (most recent call last):
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/pydantic/type_adapter.py", line 271, in _init_core_attrs
   self.core_schema = _getattr_no_parents(self._type, '__pydantic_core_schema__')
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/pydantic/type_adapter.py", line 55, in _getattr_no_parents
   raise AttributeError(attribute)
AttributeError: __pydantic_core_schema__

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/starlette/middleware/errors.py", line 165, in __call__
   await self.app(scope, receive, _send)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/aioprometheus/asgi/middleware.py", line 184, in __call__
   await self.asgi_callable(scope, receive, wrapped_send)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/starlette/middleware/cors.py", line 93, in __call__
   await self.simple_response(scope, receive, send, request_headers=headers)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/starlette/middleware/cors.py", line 144, in simple_response
   await self.app(scope, receive, send)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
   await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
   raise exc
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
   await app(scope, receive, sender)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/starlette/routing.py", line 715, in __call__
   await self.middleware_stack(scope, receive, send)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/starlette/routing.py", line 735, in app
   await route.handle(scope, receive, send)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/starlette/routing.py", line 460, in handle
   await self.app(scope, receive, send)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/fastapi/applications.py", line 1054, in __call__
   await super().__call__(scope, receive, send)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/starlette/applications.py", line 112, in __call__
   await self.middleware_stack(scope, receive, send)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/starlette/middleware/errors.py", line 187, in __call__
   raise exc
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/starlette/middleware/errors.py", line 165, in __call__
   await self.app(scope, receive, _send)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/gradio/route_utils.py", line 695, in __call__
   await self.simple_response(scope, receive, send, request_headers=headers)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/gradio/route_utils.py", line 711, in simple_response
   await self.app(scope, receive, send)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/starlette/middleware/exceptions.py", line 62, in __call__
   await wrap_app_handling_exceptions(self.app, conn)(scope, receive, send)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
   raise exc
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
   await app(scope, receive, sender)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/starlette/routing.py", line 715, in __call__
   await self.middleware_stack(scope, receive, send)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/starlette/routing.py", line 735, in app
   await route.handle(scope, receive, send)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/starlette/routing.py", line 288, in handle
   await self.app(scope, receive, send)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/starlette/routing.py", line 76, in app
   await wrap_app_handling_exceptions(app, request)(scope, receive, send)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/starlette/_exception_handler.py", line 53, in wrapped_app
   raise exc
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/starlette/_exception_handler.py", line 42, in wrapped_app
   await app(scope, receive, sender)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/starlette/routing.py", line 73, in app
   response = await f(request)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/fastapi/routing.py", line 291, in app
   solved_result = await solve_dependencies(
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/fastapi/dependencies/utils.py", line 666, in solve_dependencies
   ) = await request_body_to_args(  # body_params checked above
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/fastapi/dependencies/utils.py", line 891, in request_body_to_args
   fields_to_extract = get_cached_model_fields(first_field.type_)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/fastapi/_compat.py", line 659, in get_cached_model_fields
   return get_model_fields(model)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/fastapi/_compat.py", line 285, in get_model_fields
   return [
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/fastapi/_compat.py", line 286, in <listcomp>
   ModelField(field_info=field_info, name=name)
 File "<string>", line 6, in __init__
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/fastapi/_compat.py", line 111, in __post_init__
   self._type_adapter: TypeAdapter[Any] = TypeAdapter(
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/pydantic/type_adapter.py", line 228, in __init__
   self._init_core_attrs(
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/pydantic/type_adapter.py", line 290, in _init_core_attrs
   core_schema = schema_generator.generate_schema(self._type)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 610, in generate_schema
   schema = self._generate_schema_inner(obj)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 863, in _generate_schema_inner
   return self._annotated_schema(obj)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 1977, in _annotated_schema
   schema = self._apply_annotations(source_type, annotations)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 2056, in _apply_annotations
   schema = get_inner_schema(source_type)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/pydantic/_internal/_schema_generation_shared.py", line 84, in __call__
   schema = self._handler(source_type)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 2131, in new_handler
   schema = metadata_get_schema(source, get_inner_schema)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 2127, in <lambda>
   lambda source, handler: handler(source)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/pydantic/_internal/_schema_generation_shared.py", line 84, in __call__
   schema = self._handler(source_type)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 2037, in inner_handler
   schema = self._generate_schema_inner(obj)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 884, in _generate_schema_inner
   return self.match_type(obj)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 986, in match_type
   return self._match_generic_type(obj, origin)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 1014, in _match_generic_type
   return self._union_schema(obj)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 1325, in _union_schema
   choices.append(self.generate_schema(arg))
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 610, in generate_schema
   schema = self._generate_schema_inner(obj)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 884, in _generate_schema_inner
   return self.match_type(obj)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 995, in match_type
   return self._unknown_type_schema(obj)
 File "/root/miniconda3/envs/xinference/lib/python3.9/site-packages/pydantic/_internal/_generate_schema.py", line 513, in _unknown_type_schema
   raise PydanticSchemaGenerationError(
pydantic.errors.PydanticSchemaGenerationError: Unable to generate pydantic-core schema for <class 'starlette.requests.Request'>. Set `arbitrary_types_allowed=True` in the model_config to ignore this error or implement `__get_pydantic_core_schema__` on your type to fully support it.

If you got this error by calling handler(<some type>) within `__get_pydantic_core_schema__` then you likely need to call `handler.generate_schema(<some type>)` since we do not call `__get_pydantic_core_schema__` on `<some type>` otherwise to avoid infinite recursion.

For further information visit https://errors.pydantic.dev/2.10/u/schema-for-unknown-type

Running Xinference with Docker? / 是否使用 Docker 运行 Xinfernece？

• docker / docker
• pip install / 通过 pip install 安装
• installation from source / 从源码安装

Version info / 版本信息

transformers 4.48.3
python 3.9.21
pytorch 2.5.1
xinference 1.2.2

The command used to start Xinference / 用以启动 xinference 的命令

xinference-local --host 0.0.0.0 --port 9997

Reproduction / 复现过程

1. launch model页面检索deepseek-vl-chat
2. 下载deepseek-vl-chat
3. 进入running models
4. 使用chatbox对话

Expected behavior / 期待表现

如何配置多模态模型？

[qinxuye]

pip show pydantic

看下 pydantic 版本。

[WindLWQ]

Name: pydantic
Version: 2.10.6
Summary: Data validation using Python type hints

[qinxuye]

Version: 2.8.2

我的是这个版本，你试下降级版本能不能解决？

[WindLWQ]

还是不行呢

[qinxuye]

pip install fastapi==0.111.0

降级 fastapi 看下

[WindLWQ]

目前是可以用了，但是这个模型的回复很奇怪。请问如何查看详细的推理日志呢？

[qinxuye]

用哪个引擎跑的？

[WindLWQ]

transformers

[qinxuye]

确实奇怪，不过 deepseek-vl2 也出了，我们也计划适配。可以一起看下。

[WindLWQ]

这个模型使用的时候， 有的对话是可以正常回复的，有的就不行。请问如何查看推理日志定位问题呢？

[qinxuye]

https://github.com/xorbitsai/inference/blob/main/xinference/model/llm/transformers/deepseek_vl.py

这个是主要实现文件。可能要调试看下。