Skip to content

fix specify local path issue use model from www.modelscope.cn #2934

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merrymercy merged 2 commits into from
Jan 18, 2024
+9 −2
Merged

fix specify local path issue use model from www.modelscope.cn #2934

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
a4c8c06
fix specify local path issue
Jan 18, 2024
78bfebc
fix format issue
Jan 18, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 7 additions & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -247,7 +247,13 @@ python3 -m fastchat.serve.gradio_web_server_multi
```
- The default model worker based on huggingface/transformers has great compatibility but can be slow. If you want high-throughput batched serving, you can try [vLLM integration](docs/vllm_integration.md).
- If you want to host it on your own UI or third party UI, see [Third Party UI](docs/third_party_ui.md).


## Use models from modelscope
For Chinese users, you can use models from www.modelscope.cn via specify the following environment variables.
```bash
export FASTCHAT_USE_MODELSCOPE=True
```

## API
### OpenAI-Compatible RESTful APIs & SDK
FastChat provides OpenAI-compatible APIs for its supported models, so you can use FastChat as a local drop-in replacement for OpenAI APIs.
Expand Down
3 changes: 2 additions & 1 deletion fastchat/model/model_adapter.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -337,7 +337,8 @@ def load_model(
try:
from modelscope.hub.snapshot_download import snapshot_download

model_path = snapshot_download(model_id=model_path, revision=revision)
if not os.path.exists(model_path):
model_path = snapshot_download(model_id=model_path, revision=revision)
except ImportError as e:
warnings.warn(
"Use model from www.modelscope.cn need pip install modelscope"
Expand Down