Add Hermes 2.5 to OpenOrcaAdapter

[teknium1]

They use the same prompt format

Why are these changes needed?

This adds support for the OpenHermes 2.5 model, I added it under OpenOrca's because they use the same format, though you may prefer it have it's own. Let me know.

[infwinston]

could you resolve conflict?

[teknium1]

Updated

[infwinston]

looks like some special token <|end_of_turn|> is not correctly filtered out. Have you tested it on your end with
python3 -m fastchat.serve.cli --model-path teknium/OpenHermes-2.5-Mistral-7B --debug?

[teknium1]

Okay, it seems the actual setting of the template moved places after my original commit, which wasnt obvious when having to resolve the conflict at first, and it had to be reimplemented in the final call of that method.

[teknium1]

well I tried it now but found that it has a preset system prompt:
{'conv_template': 'mistral-7b-openorca', 'prompt': '<|im_start|>system\nYou are MistralOrca, a large language model trained by Alignment Lab AI. Write out your reasoning step-by-step to be sure you get the right answers!<|im_end|>

Not ideal for my model so I will redo a new one :D

[teknium1]

Okay, it should be good to go now

Hermes2Adapter :)

[teknium1]

Reversed my changes to the OpenOrca mistral adapter as well

[teknium1]

Am now getting [INST] format - not seeing exactly why..

[infwinston]

@teknium1 any luck getting it work?

[teknium1]

No not yet I had to focus on other things, will return to it on mon or tues

[infwinston]

@teknium1 any update? we'd love to host your model on Arena!

[teknium1]

I am just not sure what it needs to work - adding the Hermes adapter did not result in it using the format, instead defaulted to llama-2-chat format. The only issue is I can use orca mistral 7b format but it has a very bad system prompt that I dont want to use

[merrymercy]

@teknium1

1. You need to register your adapter, similar to this line

   FastChat/fastchat/model/model_adapter.py

   Line 1934 in 99d19ac

   register_model_adapter(MicrosoftOrcaAdapter)

2. You can start debugging by adding some print here

   FastChat/fastchat/model/model_adapter.py

   Lines 337 to 340 in 99d19ac

   	def get_conversation_template(model_path: str) -> Conversation:
   	"""Get the default conversation template."""
   	adapter = get_model_adapter(model_path)
   	return adapter.get_default_conv_template(model_path)

[teknium1]

We rebased and restarted, new PR is here: #2725

Closing this one