test: conditional disagg and cache aware balancing for deepseek v3 #4522
Conversation
|
/bot run |
|
PR_Github #5969 [ run ] triggered by Bot |
|
PR_Github #5969 [ run ] completed with state |
|
/bot run --only-multi-gpu-test |
|
PR_Github #5994 [ run ] triggered by Bot |
|
PR_Github #5994 [ run ] completed with state |
|
/bot run --only-multi-gpu-test |
|
PR_Github #6084 [ run ] triggered by Bot |
|
PR_Github #6084 [ run ] completed with state |
|
/bot run |
|
PR_Github #6122 [ run ] triggered by Bot |
|
PR_Github #6122 [ run ] completed with state |
zhengd-nv
force-pushed
the
cond-disagg-deepseek
branch
from
2707da4 to
93e2fff
Compare
|
/bot run |
2 similar comments
|
/bot run |
|
/bot run |
|
PR_Github #6456 [ run ] triggered by Bot |
|
PR_Github #6456 [ run ] completed with state |
|
/bot run |
|
PR_Github #6517 [ run ] triggered by Bot |
|
PR_Github #6517 [ run ] completed with state |
zhengd-nv
force-pushed
the
cond-disagg-deepseek
branch
from
c42187e to
6c6cd7b
Compare
|
/bot run |
|
PR_Github #6601 [ run ] triggered by Bot |
|
PR_Github #6601 [ run ] completed with state |
zhengd-nv
force-pushed
the
cond-disagg-deepseek
branch
from
40f2b8b to
4410671
Compare
|
/bot run |
|
PR_Github #7483 [ run ] completed with state |
|
/bot run |
|
PR_Github #7597 [ run ] triggered by Bot |
|
PR_Github #7597 [ run ] completed with state |
|
/bot run |
|
PR_Github #7638 [ run ] triggered by Bot |
|
PR_Github #7638 [ run ] completed with state |
|
/bot run --disable-fail-fast |
|
PR_Github #7680 [ run ] triggered by Bot |
|
PR_Github #7680 [ run ] completed with state |
|
/bot run |
|
PR_Github #7768 [ run ] triggered by Bot |
|
PR_Github #7768 [ run ] completed with state |
|
/bot run |
|
PR_Github #7806 [ run ] triggered by Bot |
|
PR_Github #7806 [ run ] completed with state |
|
/bot run |
|
PR_Github #7929 [ run ] triggered by Bot |
|
PR_Github #7929 [ run ] completed with state |
Signed-off-by: Zheng Duan <[email protected]>
zhengd-nv
force-pushed
the
cond-disagg-deepseek
branch
from
20298e2 to
5b5cee2
Compare
|
/bot run |
|
PR_Github #8068 [ run ] triggered by Bot |
|
PR_Github #8068 [ run ] completed with state |
|
/bot run |
|
PR_Github #8191 [ run ] triggered by Bot |
|
PR_Github #8191 [ run ] completed with state |
4 participants
Description
Add conditional disaggregation and kv cache aware router tests for DeepSeek V3 Lite model.
Since FP8 kv cache reuse is not supported for now (https://github.com/NVIDIA/TensorRT-LLM/blob/main/examples/models/core/deepseek_v3/README.md#kv-cache-reuse), bf16 one is used in tests.Test Coverage
disaggregated/test_disaggregated.py::test_disaggregated_deepseek_v3_lite_bf16_cache_aware_balance[DeepSeek-V3-Lite-bf16]disaggregated/test_disaggregated.py::test_disaggregated_deepseek_v3_lite_bf16_conditional[DeepSeek-V3-Lite-bf16]disaggregated/test_workers.py::test_workers_conditional_disaggregation_deepseek_v3_lite_bf16[DeepSeek-V3-Lite-bf16]disaggregated/test_workers.py::test_workers_kv_cache_aware_router_deepseek_v3_lite_bf16[DeepSeek-V3-Lite-bf16]GitHub Bot Help
/bot [-h] ['run', 'kill', 'skip', 'reuse-pipeline'] ...Provide a user friendly way for developers to interact with a Jenkins server.
Run
/bot [-h|--help]to print this help message.See details below for each supported subcommand.
Details
run [--disable-fail-fast --skip-test --stage-list "A10-1, xxx" --gpu-type "A30, H100_PCIe" --add-multi-gpu-test --only-multi-gpu-test --disable-multi-gpu-test --post-merge --extra-stage "H100_PCIe-[Post-Merge]-1, xxx"]Launch build/test pipelines. All previously running jobs will be killed.
--disable-fail-fast(OPTIONAL) : Disable fail fast on build/tests/infra failures.--skip-test(OPTIONAL) : Skip all test stages, but still run build stages, package stages and sanity check stages. Note: Does NOT update GitHub check status.--stage-list "A10-1, xxx"(OPTIONAL) : Only run the specified test stages. Examples: "A10-1, xxx". Note: Does NOT update GitHub check status.--gpu-type "A30, H100_PCIe"(OPTIONAL) : Only run the test stages on the specified GPU types. Examples: "A30, H100_PCIe". Note: Does NOT update GitHub check status.--only-multi-gpu-test(OPTIONAL) : Only run the multi-GPU tests. Note: Does NOT update GitHub check status.--disable-multi-gpu-test(OPTIONAL) : Disable the multi-GPU tests. Note: Does NOT update GitHub check status.--add-multi-gpu-test(OPTIONAL) : Force run the multi-GPU tests. Will also run L0 pre-merge pipeline.--post-merge(OPTIONAL) : Run the L0 post-merge pipeline instead of the ordinary L0 pre-merge pipeline.--extra-stage "H100_PCIe-[Post-Merge]-1, xxx"(OPTIONAL) : Run the ordinary L0 pre-merge pipeline and specified test stages. Examples: --extra-stage "H100_PCIe-[Post-Merge]-1, xxx".kill
killKill all running builds associated with pull request.
skip
skip --comment COMMENTSkip testing for latest commit on pull request.
--comment "Reason for skipping build/test"is required. IMPORTANT NOTE: This is dangerous since lack of user care and validation can cause top of tree to break.reuse-pipeline
reuse-pipelineReuse a previous pipeline to validate current commit. This action will also kill all currently running builds associated with the pull request. IMPORTANT NOTE: This is dangerous since lack of user care and validation can cause top of tree to break.