Skip to content
This repository was archived by the owner on Mar 17, 2025. It is now read-only.
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
46 commits
Select commit Hold shift + click to select a range
2264580
Remove hardcode flash-attn disable setting (#2342)
Trangle Sep 1, 2023
24a8755
Document turning off proxy_buffering when api is streaming (#2337)
nathanstitt Sep 1, 2023
b039a66
Simplify huggingface api example (#2355)
merrymercy Sep 4, 2023
ea045e6
Update sponsor logos (#2367)
merrymercy Sep 5, 2023
85bec47
if LOGDIR is empty, then don't try output log to local file (#2357)
leiwen83 Sep 5, 2023
f99663c
add best_of and use_beam_search for completions interface (#2348)
leiwen83 Sep 6, 2023
3cf04c2
Extract upvote/downvote from log files (#2369)
merrymercy Sep 6, 2023
94f4dd6
Revert "add best_of and use_beam_search for completions interface" (#…
merrymercy Sep 6, 2023
dc3dd12
Improve doc (#2371)
merrymercy Sep 6, 2023
a5e6abf
add best_of and use_beam_search for completions interface (#2372)
leiwen83 Sep 7, 2023
1d703b2
update monkey patch for llama2 (#2379)
merrymercy Sep 7, 2023
56744d1
Make E5 adapter more restrict to reduce mismatch (#2381)
merrymercy Sep 7, 2023
6af0a7c
Update UI and sponsers (#2387)
merrymercy Sep 8, 2023
9b3147e
Use fsdp api for save save (#2390)
merrymercy Sep 10, 2023
a6167db
Release v0.2.27
merrymercy Sep 10, 2023
7dcdafe
Spicyboros + airoboros 2.2 template update. (#2392)
jondurbin Sep 11, 2023
b921f16
bugfix of openai_api_server for fastchat.serve.vllm_worker (#2398)
Rayrtfr Sep 11, 2023
13f40b3
Revert "bugfix of openai_api_server for fastchat.serve.vllm_worker" (…
merrymercy Sep 11, 2023
77aa4df
Revert "add best_of and use_beam_search for completions interface" (#…
merrymercy Sep 11, 2023
11b05bb
Release a v0.2.28 with bug fixes and more test cases
merrymercy Sep 11, 2023
a8088ba
Fix model_worker error (#2404)
wangxiyuan Sep 12, 2023
b49d789
Added google/flan models and fixed AutoModelForSeq2SeqLM when loading…
wangzhen263 Sep 12, 2023
7dfcf1a
Rename twitter to X (#2406)
karshPrime Sep 12, 2023
aa153d5
Update huggingface_api.py (#2409)
merrymercy Sep 12, 2023
3149253
Add support for baichuan2 models (#2408)
Sep 13, 2023
2e0e60b
Fixed character overlap issue when api streaming output (#2431)
Somezak1 Sep 18, 2023
c7e3e67
Support custom conversation template in multi_model_worker (#2434)
hi-jin Sep 18, 2023
c685951
Add Ascend NPU support (#2422)
zhangsibo1129 Sep 18, 2023
54a8353
Add raw conversation template (#2417) (#2418)
tobiabir Sep 18, 2023
1119c51
Improve docs & UI (#2436)
merrymercy Sep 18, 2023
658736f
Fix Salesforce xgen inference (#2350)
jaywonchung Sep 18, 2023
d26d9e7
Add support for Phind-CodeLlama models (#2415) (#2416)
tobiabir Sep 18, 2023
0a5f503
Add falcon 180B chat conversation template (#2384)
Btlmd Sep 18, 2023
318d070
Improve docs (#2438)
merrymercy Sep 18, 2023
9cf3c8b
add dtype and seed (#2430)
Ying1123 Sep 18, 2023
24acac1
Data cleaning scripts for dataset release (#2440)
merrymercy Sep 18, 2023
30a6ffc
merge google/flan based adapters: T5Adapter, CodeT5pAdapter, FlanAdap…
wangzhen263 Sep 18, 2023
16be5cf
Fix docs
merrymercy Sep 18, 2023
e4758da
Update UI (#2446)
merrymercy Sep 18, 2023
68f1fac
Add Optional SSL Support to controller.py (#2448)
brandonbiggs Sep 19, 2023
db8e271
Format & Improve docs
merrymercy Sep 19, 2023
c4c195c
Release v0.2.29 (#2450)
merrymercy Sep 20, 2023
a040cdc
Show terms of use as an JS alert (#2461)
merrymercy Sep 22, 2023
bcb8076
vllm worker awq quantization update (#2463)
dongxiaolong Sep 22, 2023
2855bf9
Fix falcon chat template (#2464)
merrymercy Sep 22, 2023
20cfb32
Merge commit '2855bf9' into merge_0922
renning22 Sep 23, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Improve docs (lm-sys#2438)
  • Loading branch information
merrymercy authored Sep 18, 2023
commit 318d07071081176e8f21472cf337e0503117e820
4 changes: 3 additions & 1 deletion docs/model_support.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,13 +31,15 @@
- [openaccess-ai-collective/manticore-13b-chat-pyg](https://huggingface.co/openaccess-ai-collective/manticore-13b-chat-pyg)
- [OpenAssistant/oasst-sft-4-pythia-12b-epoch-3.5](https://huggingface.co/OpenAssistant/oasst-sft-4-pythia-12b-epoch-3.5)
- [VMware/open-llama-7b-v2-open-instruct](https://huggingface.co/VMware/open-llama-7b-v2-open-instruct)
- [Phind/Phind-CodeLlama-34B-v2](https://huggingface.co/Phind/Phind-CodeLlama-34B-v2)
- [project-baize/baize-v2-7b](https://huggingface.co/project-baize/baize-v2-7b)
- [Qwen/Qwen-7B-Chat](https://huggingface.co/Qwen/Qwen-7B-Chat)
- [Salesforce/codet5p-6b](https://huggingface.co/Salesforce/codet5p-6b)
- [StabilityAI/stablelm-tuned-alpha-7b](https://huggingface.co/stabilityai/stablelm-tuned-alpha-7b)
- [THUDM/chatglm-6b](https://huggingface.co/THUDM/chatglm-6b)
- [THUDM/chatglm2-6b](https://huggingface.co/THUDM/chatglm2-6b)
- [tiiuae/falcon-40b](https://huggingface.co/tiiuae/falcon-40b)
- [tiiuae/falcon-180B-chat](https://huggingface.co/tiiuae/falcon-180B-chat)
- [timdettmers/guanaco-33b-merged](https://huggingface.co/timdettmers/guanaco-33b-merged)
- [togethercomputer/RedPajama-INCITE-7B-Chat](https://huggingface.co/togethercomputer/RedPajama-INCITE-7B-Chat)
- [WizardLM/WizardLM-13B-V1.0](https://huggingface.co/WizardLM/WizardLM-13B-V1.0)
Expand Down Expand Up @@ -71,7 +73,7 @@ You can add `--debug` to see the actual prompt sent to the model.

FastChat uses the `Conversation` class to handle prompt templates and `BaseModelAdapter` class to handle model loading.

1. Implement a conversation template for the new model at [fastchat/conversation.py](https://github.com/lm-sys/FastChat/blob/main/fastchat/conversation.py). You can follow existing examples and use `register_conv_template` to add a new one.
1. Implement a conversation template for the new model at [fastchat/conversation.py](https://github.com/lm-sys/FastChat/blob/main/fastchat/conversation.py). You can follow existing examples and use `register_conv_template` to add a new one. Please also add a link to the official reference code if possible.
2. Implement a model adapter for the new model at [fastchat/model/model_adapter.py](https://github.com/lm-sys/FastChat/blob/main/fastchat/model/model_adapter.py). You can follow existing examples and use `register_model_adapter` to add a new one.
3. (Optional) add the model name to the "Supported models" [section](#supported-models) above and add more information in [fastchat/model/model_registry.py](https://github.com/lm-sys/FastChat/blob/main/fastchat/model/model_registry.py).

Expand Down
2 changes: 2 additions & 0 deletions fastchat/conversation.py
Original file line number Diff line number Diff line change
Expand Up @@ -953,6 +953,7 @@ def get_conv_template(name: str) -> Conversation:
)

# Falcon 180B chat template
# source: https://huggingface.co/spaces/tiiuae/falcon-180b-demo/blob/d1590ee7fae9b6ce331ba7808e61a29dcce9239f/app.py#L28-L37
register_conv_template(
Conversation(
name="falcon-chat",
Expand All @@ -966,6 +967,7 @@ def get_conv_template(name: str) -> Conversation:
)

# Phind template
# source: https://huggingface.co/Phind/Phind-CodeLlama-34B-v2
register_conv_template(
Conversation(
name="phind",
Expand Down
24 changes: 12 additions & 12 deletions fastchat/model/model_adapter.py
Original file line number Diff line number Diff line change
Expand Up @@ -665,7 +665,7 @@ def match(self, model_path: str):


class KoalaAdapter(BaseModelAdapter):
"""The model adapter for koala"""
"""The model adapter for Koala"""

use_fast_tokenizer = False

Expand All @@ -677,7 +677,7 @@ def get_default_conv_template(self, model_path: str) -> Conversation:


class AlpacaAdapter(BaseModelAdapter):
"""The model adapter for alpaca"""
"""The model adapter for Alpaca"""

use_fast_tokenizer = False

Expand Down Expand Up @@ -1267,7 +1267,7 @@ def get_default_conv_template(self, model_path: str) -> Conversation:


class Llama2Adapter(BaseModelAdapter):
"""The model adapter for llama-2"""
"""The model adapter for Llama-2 (e.g., meta-llama/Llama-2-7b-hf)"""

def match(self, model_path: str):
return "llama-2" in model_path.lower()
Expand All @@ -1283,7 +1283,7 @@ def get_default_conv_template(self, model_path: str) -> Conversation:


class CuteGPTAdapter(BaseModelAdapter):
"""The model adapter for llama-2"""
"""The model adapter for CuteGPT"""

def match(self, model_path: str):
return "cutegpt" in model_path.lower()
Expand Down Expand Up @@ -1327,7 +1327,7 @@ def get_default_conv_template(self, model_path: str) -> Conversation:


class WizardCoderAdapter(BaseModelAdapter):
"""The model adapter for WizardCoder"""
"""The model adapter for WizardCoder (e.g., WizardLM/WizardCoder-Python-34B-V1.0)"""

use_fast_tokenizer = False

Expand Down Expand Up @@ -1401,7 +1401,7 @@ def get_default_conv_template(self, model_path: str) -> Conversation:


class BGEAdapter(BaseModelAdapter):
"""The model adapter for BGE"""
"""The model adapter for BGE (e.g., BAAI/bge-large-en-v1.5)"""

use_fast_tokenizer = False

Expand Down Expand Up @@ -1430,7 +1430,7 @@ def get_default_conv_template(self, model_path: str) -> Conversation:


class E5Adapter(BaseModelAdapter):
"""The model adapter for E5"""
"""The model adapter for E5 (e.g., intfloat/e5-large-v2)"""

use_fast_tokenizer = False

Expand Down Expand Up @@ -1508,7 +1508,7 @@ def get_default_conv_template(self, model_path: str) -> Conversation:


class VigogneInstructAdapter(BaseModelAdapter):
"""The model adapter for Vigogne-Instruct"""
"""The model adapter for Vigogne-Instruct (e.g., bofenghuang/vigogne-2-7b-instruct)"""

use_fast_tokenizer = False

Expand Down Expand Up @@ -1536,7 +1536,7 @@ def get_default_conv_template(self, model_path: str) -> Conversation:


class VigogneChatAdapter(BaseModelAdapter):
"""The model adapter for Vigogne-Chat"""
"""The model adapter for Vigogne-Chat (e.g., bofenghuang/vigogne-7b-chat)"""

use_fast_tokenizer = False

Expand Down Expand Up @@ -1564,7 +1564,7 @@ def get_default_conv_template(self, model_path: str) -> Conversation:


class OpenLLaMaOpenInstructAdapter(BaseModelAdapter):
"""The model adapter for OpenLLaMa-Open-Instruct"""
"""The model adapter for OpenLLaMa-Open-Instruct (e.g., VMware/open-llama-7b-open-instruct)"""

use_fast_tokenizer = False

Expand Down Expand Up @@ -1594,7 +1594,7 @@ def get_default_conv_template(self, model_path: str) -> Conversation:


class CodeLlamaAdapter(BaseModelAdapter):
"""The model adapter for Code Llama"""
"""The model adapter for CodeLlama (e.g., codellama/CodeLlama-34b-hf)"""

def match(self, model_path: str):
return "codellama" in model_path.lower()
Expand All @@ -1610,7 +1610,7 @@ def get_default_conv_template(self, model_path: str) -> Conversation:


class PhindCodeLlamaAdapter(CodeLlamaAdapter):
"""The model adapter for Phind Code Llama"""
"""The model adapter for Phind-CodeLlama (e.g., Phind/Phind-CodeLlama-34B-v2)"""

def match(self, model_path: str):
return "phind-codellama-" in model_path.lower()
Expand Down
11 changes: 9 additions & 2 deletions fastchat/model/model_registry.py
Original file line number Diff line number Diff line change
Expand Up @@ -252,9 +252,16 @@ def get_model_info(name: str) -> ModelInfo:
"A chatbot fine-tuned from RedPajama-INCITE-7B-Base by Together",
)
register_model_info(
["falcon-7b", "falcon-7b-instruct", "falcon-40b", "falcon-40b-instruct"],
[
"falcon-7b",
"falcon-7b-instruct",
"falcon-40b",
"falcon-40b-instruct",
"falcon-180b",
"falcon-180b-chat",
],
"Falcon",
"https://huggingface.co/tiiuae/falcon-40b",
"https://huggingface.co/tiiuae/falcon-180B",
"TII's flagship series of large language models",
)
register_model_info(
Expand Down