Skip to content

NOT MEANT FOR REVIEW YET [TRTLLM-6121] TRTLLM Sampler PP support #6415

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
netanel-haber wants to merge 29 commits into from
Closed

NOT MEANT FOR REVIEW YET [TRTLLM-6121] TRTLLM Sampler PP support #6415

Changes from 1 commit
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
0a8caed
Make TRTLLM Sampler the default.
dcampora Jul 21, 2025
2877c7c
Fix default.
dcampora Jul 21, 2025
a336a00
Fix test_return_logits.
dcampora Jul 22, 2025
77d3cfe
Wider support of executor_config.speculative_config.
dcampora Jul 22, 2025
d2dda76
Set max batch size to 64 in test_llm_api_pytorch qwen
dcampora Jul 22, 2025
46322e1
Fix run of tests against spec decoding)
dcampora Jul 22, 2025
829339b
Fix max in max_attn_window values.
dcampora Jul 23, 2025
e54b466
Formatting.
dcampora Jul 23, 2025
47578b9
Remove enable_trtllm_sampler option.
dcampora Jul 23, 2025
08099fa
Unwaive qwen test, adapt sampler output.
dcampora Jul 24, 2025
225c84e
Adapt e2e test format.
dcampora Jul 24, 2025
76de001
Use Qwen3/Qwen3-0.6B-Base instead of TinyLlama for its longer max_pos…
dcampora Jul 28, 2025
3de720f
Remove unnecessary enable_torch_sampler in test.
dcampora Jul 28, 2025
30899b6
Removed unnecessary setting of Torch sampler to false.
dcampora Jul 28, 2025
78b7cf8
Change enable_torch_sampler to use_torch_sampler.
dcampora Jul 28, 2025
957fa95
Fix remaining single gpu issues.
dcampora Jul 29, 2025
d200cd8
Fix remaining failing tests.
dcampora Jul 29, 2025
29e5aba
Apply test correction.
dcampora Jul 30, 2025
b07a850
test_disaggregated_torch_sampler
dcampora Aug 4, 2025
f462ce4
Use torch sampler for tests that use large models.
dcampora Aug 5, 2025
761b09e
refactor: Rename maxBatchSize to maxNumSequences in decoder and gptDe…
Funatiq Jun 18, 2025
0a3472b
finalize_events: dict[str, CudaEvent]** | None = None**
netanel-haber Jul 28, 2025
0817776
wording
netanel-haber Jul 28, 2025
179e3a0
Adapt to use_torch_sampler.
dcampora Aug 5, 2025
35f0c17
Fix test_openai_chat_multimodal.
dcampora Aug 5, 2025
dacf557
setup is_attention_dp_dummy=True gen requests
netanel-haber Aug 5, 2025
b3411bb
fix dc846955207768a34a59486aad398dad995e96ae pkl5.Intracomm.Request h…
netanel-haber Aug 5, 2025
68a69bf
minimal disagg fix
netanel-haber Aug 6, 2025
2834cb1
Merge branch 'main' into user/nhaber/fix/TRTLLM-6121-trtllm-sampler-p…
netanel-haber Aug 6, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
Fix test_openai_chat_multimodal. 
Signed-off-by: Daniel Campora <961215+dcampora@users.noreply.github.com>
  • Loading branch information
@dcampora
dcampora committed Aug 5, 2025
commit 35f0c17dba989dce2b32df8af59215738c88fbd2
2 changes: 1 addition & 1 deletion tests/unittest/llmapi/apps/_test_openai_chat_multimodal.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -48,7 +48,7 @@ def server(model_name: str, temp_extra_llm_api_options_file: str):
model_path = get_model_path(model_name)
args = [
"--extra_llm_api_options", temp_extra_llm_api_options_file,
"--use_torch_sampler"
"--max_batch_size", "64"
]
with RemoteOpenAIServer(model_path, args) as remote_server:
yield remote_server
Expand Down