Skip to content

[ci] feat: add profiling tests to vLLM ci #5215

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
Gary-cjy wants to merge 4 commits into
base: main
Choose a base branch
from
+27 −1
Draft

[ci] feat: add profiling tests to vLLM ci #5215

Changes from 1 commit
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
b7a34f2
add ci test shell
Gary-cjy Feb 5, 2026
c2cd3ee
fix ci test shell
Gary-cjy Feb 5, 2026
f808664
fix a ci test shell
Gary-cjy Feb 6, 2026
41a96bc
update ci test shell
Gary-cjy Feb 6, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
add ci test shell
  • Loading branch information
@Gary-cjy
Gary-cjy committed Feb 5, 2026
commit b7a34f21e19607dbd8a08f0b1a39783410e9a5a3
29 changes: 28 additions & 1 deletion tests/special_npu/run_qwen2_5_05b_grpo.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,14 @@ set -x
MODEL_ID=${MODEL_ID:-Qwen/Qwen2.5-0.5B-Instruct}
MODEL_PATH=${MODEL_PATH:-${HOME}/.cache/models/${MODEL_ID}}

SAVE_PATH="your_path"
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot Feb 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

critical

The SAVE_PATH is set to a placeholder "your_path". This is not a valid directory and will cause the script to fail, especially in a CI environment where such a path is unlikely to exist. Please use a valid, descriptive path for the output data.

Suggested change
SAVE_PATH="your_path"
SAVE_PATH="outputs/profile_qwen2_5_05b_grpo"

LEVEL="level1"
CONTENTS=['npu', 'cpu']
ANALYSIS=True
PROFILE_STEPS="[1]"
PROFILE_RANKS_ALL=True
DISCRETE=False

python3 -m verl.trainer.main_ppo \
algorithm.adv_estimator=grpo \
data.train_files=$HOME/data/gsm8k/train.parquet \
Expand Down Expand Up @@ -44,4 +52,23 @@ python3 -m verl.trainer.main_ppo \
trainer.save_freq=-1 \
trainer.test_freq=-1 \
trainer.total_epochs=1 \
trainer.total_training_steps=1 $@
trainer.total_training_steps=1 \
actor_rollout_ref.actor.profiler.enable=True \
actor_rollout_ref.actor.profiler.all_ranks=$PROFILE_RANKS_ALL \
actor_rollout_ref.actor.profiler.all_tool_config.npu.discrete=$DISCRETE \
actor_rollout_ref.actor.profiler.all_tool_config.npu.contents=$CONTENTS \
actor_rollout_ref.actor.profiler.all_tool_config.npu.level=$LEVEL \
actor_rollout_ref.actor.profiler.all_tool_config.npu.analysis=$ANALYSIS \
actor_rollout_ref.ref.profiler.enable=True \
actor_rollout_ref.ref.profiler.all_ranks=$PROFILE_RANKS_ALL \
actor_rollout_ref.ref.profiler.all_tool_config.npu.discrete=$DISCRETE \
actor_rollout_ref.ref.profiler.all_tool_config.npu.contents=$CONTENTS \
actor_rollout_ref.ref.profiler.all_tool_config.npu.level=$LEVEL \
actor_rollout_ref.ref.profiler.all_tool_config.npu.analysis=$ANALYSIS \
global_profiler.tool=npu \
global_profiler.steps=$PROFILE_STEPS \
global_profiler.save_path=$SAVE_PATH \
trainer.device=npu $@
Copy link
Collaborator

@tardis-key tardis-key Feb 6, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

trainer.device is not required, device type can be obtained automatically