Skip to content
Merged
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next Next commit
add gptq g_idx faq
Signed-off-by: Kaihui-intel <[email protected]>
  • Loading branch information
Kaihui-intel committed Mar 4, 2025
commit b246f9bed75b7f96d5be8cf3f6328b0912136f0e
11 changes: 10 additions & 1 deletion docs/source/faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,9 +26,18 @@ Intel TBB is required, try:
$ conda/pip install tbb
```
**Solution:** It's actually already installed by `requirements_pt.txt`, so just need to set up with `export LD_LIBRARY_PATH=/usr/local/lib/:$LD_LIBRARY_PATH`.
#### Issue 6:
#### Issue 7:
torch._C._LinAlgError: linalg.cholesky: The factorization could not be completed because the input is not positive-definite.
**Solution:** This is a known issue. For more details, refer to
[AutoGPTQ/AutoGPTQ#196](https://github.com/AutoGPTQ/AutoGPTQ/issues/196).
Try increasing `percdamp` (percent of the average Hessian diagonal to use for dampening),
or increasing `nsamples` (the number of calibration samples).
#### Issue 6:
If you run GPTQ quantization with transformers-like API on xpu device, then you may encounter the following error:
```shell
[ERROR][modeling_auto.py:128] index 133 is out of bounds for dimension 0 with size 128
[ERROR][modeling_auto.py:129] Saved low bit model loading failed, please check your model.
HINT:
XPU device does not support `g_idx` for GPTQ quantization now. Please stay tuned.
You can set desc_act=False.
```