Skip to content

Conversation

@spacegoing
Copy link
Contributor

As discussed here: #1089

sglang generate_sequences() doesn't use validate flag as vllm did:

is_validate = prompts.meta_info.get('validate', False)
if not do_sample:
kwargs = {
'best_of': 1,
'top_p': 1.0,
'top_k': -1,
'min_p': 0.0,
'temperature': 0,
'n': 1 # if greedy, only 1 response
}
elif is_validate:
# TODO: try **
kwargs = {
'top_k': self.config.val_kwargs.top_k,
'top_p': self.config.val_kwargs.top_p,
'temperature': self.config.val_kwargs.temperature,
'n': 1, # if validate, already repeat in ray_trainer
}

copied vllm's code in this commit

@CLAassistant
Copy link

CLAassistant commented Apr 15, 2025

CLA assistant check
All committers have signed the CLA.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants