Skip to content

feat: Add vllm multimodal qwen aggregated support #2694

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
krishung5 merged 14 commits into from
Aug 26, 2025
Merged

feat: Add vllm multimodal qwen aggregated support #2694

krishung5 merged 14 commits into from
Aug 26, 2025

Conversation

@krishung5
Copy link
Contributor

@krishung5 krishung5 commented Aug 25, 2025 •
edited by coderabbitai bot
Loading

Overview:

Add support for vllm multimodal qwen agg.

Closes DIS-504.

Details:

Where should the reviewer start?

Related Issues: (use one of the action keywords Closes / Fixes / Resolves / Relates to)

  • closes GitHub issue: #xxx

Summary by CodeRabbit

  • New Features
    • Adds multimodal support for Qwen2.5-VL models alongside LLaVA, including correct image feature handling.
  • Documentation
    • Updated multimodal README with Qwen serving commands and client usage; clarified future support note.
  • Tests
    • Expanded test matrix to run separate multimodal flows for LLaVA and Qwen.
  • Refactor
    • Streamlined multimodal data handling to dynamically adapt to the selected model.
  • Chores
    • Simplified vision model loading to reduce setup complexity.

@copy-pr-bot
Copy link

copy-pr-bot bot commented Aug 25, 2025

This pull request requires additional validation before any workflows can run on NVIDIA's runners.

Pull request vetters can view their responsibilities here.

Contributors can view more details about this message here.

@github-actions github-actions bot added the feat label Aug 25, 2025
@krishung5 krishung5 marked this pull request as ready for review August 25, 2025 18:40
indrajit96
indrajit96 reviewed Aug 25, 2025
View reviewed changes
@krishung5 krishung5 requested a review from indrajit96 August 26, 2025 01:58
@krishung5
Copy link
Contributor Author

krishung5 commented Aug 26, 2025

/ok to test 1b1916f

@krishung5
Copy link
Contributor Author

krishung5 commented Aug 26, 2025

/ok to test 73c69d6

indrajit96
indrajit96 approved these changes Aug 26, 2025
View reviewed changes
Copy link
Contributor

@indrajit96 indrajit96 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!!

@krishung5
Copy link
Contributor Author

krishung5 commented Aug 26, 2025

/ok to test 6496ade

@krishung5
Copy link
Contributor Author

krishung5 commented Aug 26, 2025

Rebased to fix the GPG sign.

@krishung5
Copy link
Contributor Author

krishung5 commented Aug 26, 2025

@coderabbitai resolve

@coderabbitai
Copy link
Contributor

coderabbitai bot commented Aug 26, 2025

✅ Actions performed

Comments resolved.

@krishung5 krishung5 merged commit 50cd81f into main Aug 26, 2025
11 checks passed
@krishung5 krishung5 deleted the krish/qwen-agg branch August 26, 2025 21:18
@coderabbitai coderabbitai bot mentioned this pull request Aug 27, 2025
Merged
hhzhang16 pushed a commit that referenced this pull request Aug 27, 2025
@krishung5 @hhzhang16
feat: Add vllm multimodal qwen aggregated support (#2694)
6c10e43 
Signed-off-by: Hannah Zhang <[email protected]>
ayushag-nv pushed a commit that referenced this pull request Aug 27, 2025
@krishung5 @ayushag-nv
feat: Add vllm multimodal qwen aggregated support (#2694)
2bfeed3 
Signed-off-by: ayushag <[email protected]>
@coderabbitai coderabbitai bot mentioned this pull request Aug 28, 2025
Merged
jasonqinzhou pushed a commit that referenced this pull request Aug 30, 2025
@krishung5
feat: Add vllm multimodal qwen aggregated support (#2694)
b9029f7 
Signed-off-by: Jason Zhou <[email protected]>
KrishnanPrash pushed a commit that referenced this pull request Sep 2, 2025
@krishung5 @KrishnanPrash
feat: Add vllm multimodal qwen aggregated support (#2694)
712dc5b 
Signed-off-by: Krishnan Prashanth <[email protected]>
nnshah1 pushed a commit that referenced this pull request Sep 8, 2025
@krishung5 @nnshah1
feat: Add vllm multimodal qwen aggregated support (#2694)
84c81bb 
Signed-off-by: nnshah1 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@coderabbitai coderabbitai[bot] coderabbitai[bot] left review comments

@indrajit96 indrajit96 indrajit96 approved these changes

@nnshah1 nnshah1 nnshah1 approved these changes

@whoisj whoisj Awaiting requested review from whoisj

@hhzhang16 hhzhang16 Awaiting requested review from hhzhang16

@tanmayv25 tanmayv25 Awaiting requested review from tanmayv25

@piotrm-nvidia piotrm-nvidia Awaiting requested review from piotrm-nvidia

@ptarasiewiczNV ptarasiewiczNV Awaiting requested review from ptarasiewiczNV

@ryanolson ryanolson Awaiting requested review from ryanolson

@grahamking grahamking Awaiting requested review from grahamking

@paulhendricks paulhendricks Awaiting requested review from paulhendricks

@biswapanda biswapanda Awaiting requested review from biswapanda

@tmonty12 tmonty12 Awaiting requested review from tmonty12

@GuanLuo GuanLuo Awaiting requested review from GuanLuo

@rmccorm4 rmccorm4 Awaiting requested review from rmccorm4

@tedzhouhk tedzhouhk Awaiting requested review from tedzhouhk

@alec-flowers alec-flowers Awaiting requested review from alec-flowers

@kkranen kkranen Awaiting requested review from kkranen

@jthomson04 jthomson04 Awaiting requested review from jthomson04

@ishandhanani ishandhanani Awaiting requested review from ishandhanani

@PeaBrane PeaBrane Awaiting requested review from PeaBrane

Assignees

No one assigned

Labels

feat size/L

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@krishung5 @indrajit96 @nnshah1