IQ3_S: a much better alternative to Q3_K#5676

Merged

ikawrakow merged 27 commits intomasterfrom

Feb 24, 2024

Commits on Feb 23, 2024

iq4_nl: squash commits for easier rebase
Iwan Kawrakow
committed
Resurrecting iq3_xs
Iwan Kawrakow
committed
Minor PPL improvement via a block scale fudge factor
Iwan Kawrakow
committed
Minor improvement via 3 neighbours
Iwan Kawrakow
committed
iq3_xs: working scalar and AVX2 dot products
Iwan Kawrakow
committed
iq3_xs: ARM_NEON dot product - works but extremely slow (10 t/s)
Iwan Kawrakow
committed
iq3_xs: working Metal implementation
Iwan Kawrakow
committed
Adding IQ3_M - IQ3_XS mix with mostly Q4_K
Iwan Kawrakow
committed
iiq3_xs: a 3.4375 bpw variant
Iwan Kawrakow
committed
iq3_xs: make CUDA work for new version
Iwan Kawrakow
committed
iq3_xs: make scalar and AVX2 work for new version
Iwan Kawrakow
committed
iq3_s: make ARM_NEON work with new version
Iwan Kawrakow
committed
iq3_xs: make new version work on metal
Iwan Kawrakow
committed
iq3_xs: tiny Metal speed improvement
Iwan Kawrakow
committed
iq3_xs: tiny Metal speed improvement
Iwan Kawrakow
committed
Fix stupid warning
Iwan Kawrakow
committed
Q3_K_XS now uses a mix of IQ3_XS and IQ3_XXS
Iwan Kawrakow
committed
iq3_xs: rename to iq3_s
Iwan Kawrakow
committed
iq3_s: make tests pass
Iwan Kawrakow
committed
Move Q3_K_XS mix to 3.25 bpw
Iwan Kawrakow
committed
Attempt to fix failing tests
Iwan Kawrakow
committed
Another attempt to fix the Windows builds
Iwan Kawrakow
committed
Attempt to fix ROCm
Iwan Kawrakow
committed
ROCm again
Iwan Kawrakow
committed
iq3_s: partial fix for QK_K = 64
Iwan Kawrakow
committed
iq3_s: make it work on metal for QK_K = 64
Iwan Kawrakow
committed
Will this fix ROCm?
Iwan Kawrakow
committed