已关闭
[Bug-Report|缺陷反馈]: Qwen2.5-7b-Instruct量化w8a8出现异常报错 #331
Zctoylm创建于  2025年11月5日关闭于  2025年11月17日
Zctoylm
Zctoylm
2025年11月5日 创建

Thanks for sending an issue! Please fill in the following template to help quickly solve your problem.

Describe the current behavior / 问题描述 (Mandatory / 必填)

RT,在310p上量化Qwen2.5-7b-Instruct时出现报错:
TypeError: fake_quantize_save() got an unexpected keyword argument 'max_bound'

File "/usr/local/lib/python3.11/site-packages/msmodelslim/pytorch/llm_ptq/llm_ptq_tools/quant_tools.py", line 566, in save
self._save(output_path, safetensors_name, json_name, save_type, part_file_size)
File "/usr/local/lib/python3.11/site-packages/msmodelslim/pytorch/llm_ptq/llm_ptq_tools/quant_tools.py", line 930, in _save
self._save_weights_of_model(quantifier, saver)
File "/usr/local/lib/python3.11/site-packages/msmodelslim/pytorch/llm_ptq/llm_ptq_tools/quant_tools.py", line 937, in _save_weights_of_model
for name, meta, tensor in weight_collector:
File "/usr/local/lib/python3.11/site-packages/msmodelslim/pytorch/llm_ptq/llm_ptq_tools/quant_tools.py", line 577, in generate_weight_of_model
yield from weight_of_module_generator(name, module)
File "/usr/local/lib/python3.11/site-packages/msmodelslim/pytorch/llm_ptq/llm_ptq_tools/save/complex_quantifier.py", line 223, in generate_weight_of_module
yield from self.generate_weight_of_linear_module(name, module, model_quant_type)
File "/usr/local/lib/python3.11/site-packages/msmodelslim/pytorch/llm_ptq/llm_ptq_tools/save/complex_quantifier.py", line 301, in generate_weight_of_linear_module
quant_weight, fp_weight, weight_scale, weight_offset = self.get_param_from_quantizer(module)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib64/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/msmodelslim/pytorch/llm_ptq/llm_ptq_tools/save/complex_quantifier.py", line 454, in get_param_from_quantizer
quant_weight, _ = fake_quantize_save(fp_weight, weight_scale, weight_offset, bit=module.cfg.w_bit,
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "quant_funcs.py", line 157, in quant_funcs.fake_quantize_save
TypeError: fake_quantize_save() got an unexpected keyword argument 'max_bound'

Environment / 环境信息 (Mandatory / 必填)

310P 在mindie容器内操作
CANN 8.0.0,MindIE 1.0.0 和 ATB-Models 1.0.0

Steps to reproduce the issue / 重现步骤 (Mandatory / 必填)

bash examples/models/qwen/convert_quant_weight.sh -src /models/Qwen2.5-7B-Instruct -dst /models/Qwen2.5-7B-W8A8 -type qwen_w8a8

Describe the expected behavior / 预期结果 (Mandatory / 必填)

正确量化

image.png

Special notes for this issue/备注 (Optional / 选填)

likedislike
anreywmh成员
2025年11月6日 评论:

您好,这个问题可能是因为您CANN包版本比较旧导致,请替换最新版本的CANN

likedislike
Zctoylm
Zctoylm
2025年11月6日 评论:

您好,这个问题可能是因为您CANN包版本比较旧导致,请替换最新版本的CANN

8.2Rc1可以吗,需要更新驱动吗?

likedislike
anreywmh成员
2025年11月6日 评论:

可以的,重新source一下
qwen3系列问题请先自检cann包版本
1、请根据(内部客户)https://cmc-szv.clouddragon.huawei.com/cmcversion/index/releaseView?deltaId=12969651207537920&isSelect=Software这个链接 下载最新版本的cann
(外部客户T18 MindIE镜像或社区镜像/CANN包 https://www.hiascend.com/developer/download/community/result?module=cann)
2、source set_env.sh
3、cd msit/msmodelslim 重新 bash install.sh

likedislike
Zctoylm
Zctoylm
2025年11月10日 评论:

可以的,重新source一下
qwen3系列问题请先自检cann包版本
1、请根据(内部客户)https://cmc-szv.clouddragon.huawei.com/cmcversion/index/releaseView?deltaId=12969651207537920&isSelect=Software这个链接 下载最新版本的cann
(外部客户T18 MindIE镜像或社区镜像/CANN包 https://www.hiascend.com/developer/download/community/result?module=cann)
2、source set_env.sh
3、cd msit/msmodelslim 重新 bash install.sh

目前更新成 CANN 8.2RC1,MindIE 2.1RC1,Decode速度从11.5token/s到16.7token/s。这个量化的期望加速应该是2倍以上?

likedislike
Zctoylm
Zctoylm
2025年11月10日 评论:

以及更新后原本的print_log信息都消失了,并且大量警告,应该如何才能让log信息输出?

The old environment variable ATB_LOG_TO_STDOUT will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_TO_STDOUT as soon as possible.
The old environment variable ATB_LOG_LEVEL will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_LEVEL as soon as possible.
The old environment variable MINDIE_LLM_LOG_LEVEL will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_LEVEL as soon as possible.
The old environment variable MINDIE_LLM_PYTHON_LOG_TO_STDOUT will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_TO_STDOUT as soon as possible.
The old environment variable MINDIE_LLM_LOG_TO_STDOUT will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_TO_STDOUT as soon as possible.
The old environment variable MINDIE_LLM_PYTHON_LOG_PATH will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_PATH as soon as possible.
The old environment variable ATB_LOG_TO_FILE will be deprecated on 2025/12/31. Please use the new environment variable MINDIE_LOG_TO_FILE as soon as possible

likedislike
anreywmh成员
2025年11月12日 评论:

您好,这个日志打印应该是MINDIE的打印吧,建议在昇腾社区咨询Mindie服务或者查看他们的指导文档,开启日志打屏开关

likedislike
anreywmh成员
2025年11月17日 评论:

尊敬的用户,您好!由于工单长时间未收到您的响应,现暂为关闭。感谢您对 msModelSlim 开源生态的贡献!后续如有疑问,欢迎随时重新提问。

likedislike
Aanreywmh成员
2025年11月17日 issue状态由 TODO 改变为 DONE
Aanreywmh成员
2025年11月17日 关闭了 issue