👉 遇到问题先看这里

🌟 第一次使用工具遇到问题？

按🚀 快速入门

❓ 搜索历史issue，查看冷门的同类问题

在🔖 Issue中搜索历史类似问题

操作系统及版本

Ubuntu 22.04.3

python版本

3.12

MsModelSlim 工具版本

master

是否必现

是

MsModelSlim 执行命令

msmodelslim quant \ --model_path ${model_path} \ --save_path ${save_path} \ --model_type DeepSeek-V4-Flash \ --quant_type w8a8 \ --trust_remote_code True

预期行为

量化该模型：https://www.modelscope.cn/models/Chunjiang-Intelligence/DeepSeek-v4-Fable，要量化的是这个模型 ，量化成功

实际行为

^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/python3.12.13/lib/python3.12/site-packages/msmodelslim/core/runner/layer_wise_runner.py", line 75, in run
model = self.adapter.init_model(device=DeviceType.CPU)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/usr/local/python3.12.13/lib/python3.12/site-packages/msmodelslim/utils/logging.py", line 314, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^

File "/usr/local/python3.12.13/lib/python3.12/site-packages/msmodelslim/model/deepseek_v4/model_adapter.py", line 87, in init_model
model.load_state_dict(state_dict)
File "/usr/local/python3.12.13/lib/python3.12/site-packages/torch/nn/modules/module.py", line 2635, in load_state_dict
raise RuntimeError(
RuntimeError: Error(s) in loading state_dict for Transformer:
size mismatch for layers.0.ffn.experts.0.w1.weight: copying a param with shape torch.Size([2048, 2048]) from checkpoint, the shape in current model is torch.Size([2048, 4096])
size mismatch for layers.0.ffn.experts.0.w2.weight: copying a param with shape torch.Size([4096, 1024]) from checkpoint, the shape in current model is torch.Size([4096, 2048])

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/python3.12.13/bin/msmodelslim", line 6, in
sys.exit(main())
^^^^^^
File "/usr/local/python3.12.13/lib/python3.12/site-packages/msmodelslim/cli/main.py", line 281, in main
quant_main(args)
File "/usr/local/python3.12.13/lib/python3.12/site-packages/msmodelslim/cli/naive_quantization/main.py", line 91, in main
app.quant(
File "/usr/local/python3.12.13/lib/python3.12/site-packages/msmodelslim/utils/logging.py", line 314, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/python3.12.13/lib/python3.12/site-packages/msmodelslim/utils/exception_decorator.py", line 247, in wrapper
raise unexpected_error from e
msmodelslim.utils.exception.UnexpectedError: Code: 500, Message: Unexpected error.
TIP: Please report this issue to the msModelSlim developers. Repository: https://gitcode.com/Ascend/msmodelslim Q&A: https://gitcode.com/Ascend/msmodelslim/blob/master/docs/zh/support/faq.md
[ERROR] 2026-07-09-06:51:01 (PID:808, Device:-1, RankID:-1) ERR99999 UNKNOWN applicaiton exception

欢迎加入社区，感谢您对社区的贡献 🎉!