Skip to content

fix bf16 symbolic_trace bug #1892

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Jul 9, 2024
Merged

fix bf16 symbolic_trace bug #1892

Changes from 1 commit
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
9d3bd12
fix bf16 symbolic_trace bug
xin3he Jul 1, 2024
c97b6d8
patch 1
xin3he Jul 1, 2024
f2673f4
update mix_precision usage
xin3he Jul 3, 2024
8201841
add 1x back
xin3he Jul 4, 2024
3c8d881
fix bug
xin3he Jul 8, 2024
f52a34e
Merge branch 'master' into xinhe/fix-1
XuehaoSun Jul 9, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Prev Previous commit
Next Next commit
patch 1 
Signed-off-by: xin3he <[email protected]>
  • Loading branch information
@xin3he
xin3he committed Jul 1, 2024
commit c97b6d83cfe2750ca39b923657566182ea759485
18 changes: 10 additions & 8 deletions neural_compressor/adaptor/pytorch.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3579,6 +3579,16 @@ def quantize(self, tune_cfg, model, dataloader, q_func=None):
return q_model

self.tune_cfg["fx_sub_module_list"] = self.sub_module_list

# BF16 fallback
if (
len(self.tune_cfg["bf16_ops_list"]) > 0
and self.version.release >= Version("1.11.0").release
and self.use_bf16
and (CpuInfo().bf16 or os.getenv("FORCE_BF16") == "1")
): # pragma: no cover
q_model._model = torch_utils.bf16_convert.Convert(q_model._model, self.tune_cfg)

if self.approach == "quant_aware_training":
q_model._model.train()
if self.sub_module_list is None:
Expand Down Expand Up @@ -3665,14 +3675,6 @@ def quantize(self, tune_cfg, model, dataloader, q_func=None):
self.sub_module_list, q_model._model, prefix="", custom_config=self.prepare_custom_config_dict
)

if (
len(self.tune_cfg["bf16_ops_list"]) > 0
and self.version.release >= Version("1.11.0").release
and self.use_bf16
and (CpuInfo().bf16 or os.getenv("FORCE_BF16") == "1")
): # pragma: no cover
q_model._model = torch_utils.bf16_convert.Convert(q_model._model, self.tune_cfg)

self.fused_dict = self.get_fused_list(q_model.model)
q_model.is_quantized = True
q_model.q_config = copy.deepcopy(self.tune_cfg)
Expand Down