Skip to content

UPSTREAM PR #17907: [WIP]gml-hexagon: Q4_0 mm opt #512

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
loci-dev wants to merge 25 commits into
base: main
Choose a base branch
from
Open

UPSTREAM PR #17907: [WIP]gml-hexagon: Q4_0 mm opt #512

loci-dev wants to merge 25 commits into from
+170 −100

Conversation

@loci-dev
Copy link

@loci-dev loci-dev commented Dec 10, 2025

Mirrored from ggml-org/llama.cpp#17907

Make sure to read the contributing guidelines before submitting a PR

chraac added 25 commits November 27, 2025 12:54
@chraac
fix test failure
407b408
@chraac
fix: correct scaling calculations in rope_cache_init
4ddb8a4
@chraac
wip
cfca78b
@chraac
wip
e9a02fd
@chraac
fix: optimize element copying in rope_hex_f32 using memcpy
e324bb0
@chraac
fix: optimize loop boundaries in rope_hex_f32 for better performance
0121291
@chraac
rename
010039a
@chraac
wip
a6ef41f
@chraac
Merge branch 'master' into dev-fix-rope
0376146
@chraac
Merge tag 'b7207' into dev-fix-rope
8abecfa
@chraac
feat: add profiling macros for performance measurement in operations
b567413
@chraac
refactor: replace manual timing with profiling macros in matmul opera…
7c8f101 
…tions
@chraac
Merge branch 'master' into dev-fix-rope
3a70465
@chraac
Revert "refactor: replace manual timing with profiling macros in matm…
3b0cef4 
…ul operations"

This reverts commit 7c8f101.
@chraac
Revert "feat: add profiling macros for performance measurement in ope…
121e656 
…rations"

This reverts commit b567413.
@chraac
refactor: optimize vector operations in vec_dot_q4x4x2_q8x4x2_rx2 fun…
401fd3e 
…ction
@chraac
wip
cf491f2
@chraac
feat: enhance vec_dot_q4x4x2_q8x4x2_rx2 function with optimized data …
3a01d82 
…handling and processing
@chraac
Merge branch 'master' into dev-mulmat-opt
87ad8b2
@chraac
feat: add hvx_vec_load_d_and_mpy function for optimized data loading …
421d031 
…and multiplication
@chraac
wip
bd43860
@chraac
feat: add hvx_vec_load_d_and_mpy_r2x2 function for optimized vector l…
b197464 
…oading and multiplication
@chraac
feat: optimize vec_dot functions with improved data handling and loading
309d782
@chraac
wip
dbe9309
@chraac
feat: add build information and update vector loading functions for o…
00d5fb3 
…ptimization
@loci-agentic-ai
Copy link

loci-agentic-ai bot commented Dec 10, 2025

Explore the complete analysis inside the Version Insights

Performance Analysis Summary

Project: llama.cpp (auroralabs-loci)
PR #512: UPSTREAM PR #17907: [WIP]gml-hexagon: Q4_0 mm opt
Versions Compared: 571689ba-c827-4914-a21b-e7f9319ce9fa vs babe7961-3459-4634-9db9-ac5ad6523347

Summary

This PR introduces Qualcomm Hexagon HTP backend optimizations for Q4_0 quantized matrix multiplication through loop unrolling, helper function extraction, and RoPE implementation refactoring. Analysis reveals no measurable performance impact across all binaries and functions. Changes affect 2 files (matmul-ops.c, rope-ops.c) with code quality improvements including const-correctness, constant hoisting, and memcpy optimization for bulk copies. The modifications maintain performance parity with power consumption changes below 0.001% across all binaries.

@loci-dev loci-dev force-pushed the main branch 3 times, most recently from 0e7b989 to 24b5a2d Compare December 10, 2025 19:08
@loci-dev loci-dev force-pushed the main branch 7 times, most recently from 78ff3d3 to 117bfc3 Compare December 11, 2025 18:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

3 participants

@loci-dev @chraac