Skip to content

support quant_lm_head arg in all WOQ configs #1881

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 15 commits into from
Jun 27, 2024
Merged

support quant_lm_head arg in all WOQ configs #1881

Changes from 1 commit
Commits
Show all changes
15 commits
Select commit Hold shift + click to select a range
131f2cb
start from RTNConfig
xin3he Jun 19, 2024
b461291
fix bug
xin3he Jun 19, 2024
0ae3c7b
add UTs
xin3he Jun 19, 2024
39f649c
update UT
xin3he Jun 20, 2024
26b643d
enhance gptq forward as awq
xin3he Jun 20, 2024
b208ae3
update hqq
xin3he Jun 20, 2024
7073400
fix bug
xin3he Jun 20, 2024
98ae916
update doc and UTs for set_local
xin3he Jun 21, 2024
b8429a5
fix UTs
xin3he Jun 21, 2024
1104f44
fix bug
xin3he Jun 21, 2024
33bb948
add document for quant_lm_head
xin3he Jun 21, 2024
c7808e4
fix bug
xin3he Jun 21, 2024
ff99765
remove useless __all__
xin3he Jun 25, 2024
2351abb
update per review
xin3he Jun 25, 2024
39a6866
Merge branch 'master' into xinhe/quant_lm_head
xin3he Jun 25, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
fix bug 
Signed-off-by: xin3he <[email protected]>
  • Loading branch information
@xin3he
xin3he committed Jun 21, 2024
commit c7808e44d5ccc2390503a1517e1d8b01e6f4db14
2 changes: 1 addition & 1 deletion test/3x/torch/quantization/weight_only/test_rtn.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -241,7 +241,7 @@ def test_double_quant_params(self, dtype, double_quant_bits, double_quant_group_
out = model(self.example_inputs)[0]
atol_true = (out - self.q_label).amax()
# compare atol, this case is an ideal case.
if not (dtype, double_quant_bits, double_quant_group_size) == (256, 6, "nf4"):
if not (dtype, double_quant_bits, double_quant_group_size) == ("nf4", 6, 256):
assert (
atol_false < atol_true
), "asym for double quant should have smaller atol because scales is bigger than zero, please double check."
Expand Down