[vllm] fix: verl + vllm-ascend(version 0.9.1) running failed issue

[leo-pony]

What does this PR do?

Handle the use case of verl + vllm + vllm-ascend(v0.9.1), detail information see #2564
vllm-ascend v0.9.1 is the next upcoming commercial release branch, with the previous commercial release branch is vllm-ascend v0.7.3.

Checklist Before Starting

• Search for similar PRs. Paste at least one query link here: ...
• Format the PR title as [{modules}] {type}: {description} (This will be checked by the CI)
  • {modules} include fsdp, megatron, sglang, vllm, rollout, trainer, ci, training_utils, recipe, hardware, deployment, ray, worker, single_controller, misc, perf, model, algo, env, tool, ckpt, doc, data
  • If this PR involves multiple modules, separate them with , like [megatron, fsdp, doc]
  • {type} is in feat, fix, refactor, chore, test
  • If this PR breaks any API (CLI arguments, config, function signature, etc.), add [BREAKING] to the beginning of the title.
  • Example: [BREAKING][fsdp, megatron] feat: dynamic batching

Test

Test on ascend npu: GRPO, FSDP backend, Qwen2.5-0.5B mode.

API and Usage Example

No changes

Design & Code Changes

vllm+vllm-ascend(v0.9.1) normal use case:

In vllm 0.7.3 uses pytorch2.5.1, and the type hint for infer_schema of mode is List[int]. vllm 0.9.1 uses pytorch 2.7, and to keep consistence with pytorch2.7, and vllm changed hint type to list[int] to infer_schema of mode. Type hint List[int] and list[int] is not compatible.

vllm-ascend version 0.9.1 needs to be used in conjunction with vllm 0.9.1. But vllm-ascend 0.9.1 max supportted pytorch version is 2.5.1. As pytorch 2.5.1 using List[int] as type hint, vllm-ascend needs add patch to infer_schema of mode to successfully running in pytorch 2.5.1 environment. The patch workflow as following graph display.
As vllm hardware limits, vllm-ascend currently patch type hint list[int] during vllm LLM instance creating process. This is okay for most vllm-ascend applications.

vllm+vllm-ascend(v0.9.1) currently workflow in verl:

For verl+vllm-ascend+pytorch2.5.1 there has a problem, as following:
verl currently import vllm modes before create vllm LLM instance operation. So error take place: pass list[int] to infer_schema which needs List[int], and then running failed.

workflow in this PR in verl to handle hint type mismatch issue:

Actully patch_vllm_moe_model_weight_loader is after the operation of LLM create. "import vllm modes" action just been prematurely executed during _build_rollout processing, just file becasue vllm_utils.py in which there has another functions that needs by _build_rollout, by the way those functions been imported, vllm mode been imported. Lets vllm-hardware plugins makes it's patchs take effect, then this problem fixed: move patch_vllm_moe_model_weight_loader funciton to independent file, and import patch_vllm_moe_model_weight_loader just before wight loader patching operation.

Checklist Before Submitting

Important

Please check all the following items before requesting a review, otherwise the reviewer might deprioritize this PR for review.

• Read the Contribute Guide.
• Apply pre-commit checks: pre-commit install && pre-commit run --all-files --show-diff-on-failure --color=always
• Add / Update the documentation.
• Add unit or end-to-end test(s) to the CI workflow to cover all the code. If not feasible, explain why: ...
• Once your PR is ready for CI, send a message in the ci-request channel in the verl Slack workspace. (If not accessible, please try the Feishu group (飞书群).)

[gemini-code-assist]

Code Review

This pull request refactors the vLLM patching logic to resolve a type hint incompatibility issue when using verl with vllm-ascend. By moving the patch_vllm_moe_model_weight_loader function to a new vllm_patch.py module and deferring its import until it's needed, the change ensures that hardware-specific patches from vllm-ascend are applied before any vLLM models are imported by verl, which fixes the runtime error. The overall approach is sound. I've identified a critical issue in the new vllm_patch.py module that could lead to runtime errors and have provided a detailed comment with a suggested fix.

[FightingZhen]

LGTM!

[leo-pony]

@vermouth1992
I'd like to ask for help. The code in this pull request doesn't appear to modify the GPU logic, but I'm confused as to why three GPU-related test cases in CI are failing.
Additional information: The three test cases of yesterday's PR code passed. The difference between today's code and last night's is that vllm_utils.py and vllm_patch.py were moved to the newly created vllm directory.
Do you have any idea?