[Bug] Qwen3.6-35B-A3B w8a8量化报错ImportError: cannot import name 'Qwen3_5MoeForConditionalGeneration' from 'transformers'-MindStudio-ModelSlim-AtomGit

已关闭

[Bug] Qwen3.6-35B-A3B w8a8量化报错ImportError: cannot import name 'Qwen3_5MoeForConditionalGeneration' from 'transformers' #185

DonaldTom创建于 4月28日关闭于 4月28日

DonaldTom

4月28日创建

👉 遇到问题先看这里

🌟 第一次使用工具遇到问题？

按🚀 快速入门

❓ 搜索历史issue，查看冷门的同类问题

在🔖 Issue中搜索历史类似问题

操作系统及版本

rhel8.10 + quay.io/ascend/vllm-ascend:v0.18.0rc1-openeuler镜像

python版本

3.11

MsModelSlim 工具版本

commit ddbbac6ceb98375b2d732f65500c84deeec15c5e

是否必现

是

MsModelSlim 执行命令

msmodelslim quant --model_path ./Qwen3.6-35B-A3B --save_path ./Qwen3.6-35B-A3B-w8a8 --model_type Qwen3.5-35B-A3B --quant_type w8a8

预期行为

正常量化成功

实际行为

[root@rhel8 data]# pip list|grep torch
torch                                    2.9.0+cpu
torch_npu                                2.9.0.post1+gitee7ba04
torchaudio                               2.9.0
torchvision                              0.24.0
[root@rhel8 data]# pip list|grep msmodelslim
msmodelslim                              26.0.0a1
[root@rhel8 data]# pip list|grep transformers
transformers                             4.57.6
[root@rhel8 data]# pip list|grep vllm
vllm                                     0.18.0+empty           /vllm-workspace/vllm
vllm_ascend                              0.18.0rc1              /vllm-workspace/vllm-ascend

[root@rhel8 data]# msmodelslim quant  --model_path ./Qwen3.6-35B-A3B  --save_path ./Qwen3.6-35B-A3B-w8a8  --model_type Qwen3.5-35B-A3B  --quant_type w8a8
/usr/local/python3.11.14/lib/python3.11/site-packages/torch_npu/contrib/transfer_to_npu.py:347: ImportWarning:
    *************************************************************************************************************
    The torch.Tensor.cuda and torch.nn.Module.cuda are replaced with torch.Tensor.npu and torch.nn.Module.npu now..
    The torch.cuda.DoubleTensor is replaced with torch.npu.FloatTensor cause the double type is not supported now..
    The backend in torch.distributed.init_process_group set to hccl now..
    The torch.cuda.* and torch.cuda.amp.* are replaced with torch.npu.* and torch.npu.amp.* now..
    The device parameters have been replaced with npu in the function below:
    torch.logspace, torch.randint, torch.hann_window, torch.rand, torch.full_like, torch.ones_like, torch.rand_like, torch.randperm, torch.arange, torch.frombuffer, torch.normal, torch._empty_per_c                                   hannel_affine_quantized, torch.empty_strided, torch.empty_like, torch.scalar_tensor, torch.tril_indices, torch.bartlett_window, torch.ones, torch.sparse_coo_tensor, torch.randn, torch.kaiser_window                                   , torch.tensor, torch.triu_indices, torch.as_tensor, torch.zeros, torch.randint_like, torch.full, torch.eye, torch._sparse_csr_tensor_unsafe, torch.empty, torch._sparse_coo_tensor_unsafe, torch.bla                                   ckman_window, torch.zeros_like, torch.range, torch.sparse_csr_tensor, torch.randn_like, torch.from_file, torch._cudnn_init_dropout_state, torch._empty_affine_quantized, torch.linspace, torch.hammin                                   g_window, torch.empty_quantized, torch._pin_memory, torch.load, torch.set_default_device, torch.get_device_module, torch.sparse_compressed_tensor, torch.Tensor.new_empty, torch.Tensor.new_empty_str                                   ided, torch.Tensor.new_full, torch.Tensor.new_ones, torch.Tensor.new_tensor, torch.Tensor.new_zeros, torch.Tensor.to, torch.Tensor.pin_memory, torch.nn.Module.to, torch.nn.Module.to_empty
    *************************************************************************************************************

  warnings.warn(msg, ImportWarning)
/usr/local/python3.11.14/lib/python3.11/site-packages/torch_npu/contrib/transfer_to_npu.py:276: RuntimeWarning: torch.jit.script and torch.jit.script_method will be disabled by transfer_to_npu, whi                                   ch currently does not support them, if you need to enable them, please do not use transfer_to_npu.
  warnings.warn(msg, RuntimeWarning)
2026-04-28 01:08:09,839 - msmodelslim.app.naive_quantization - WARNING - write directory not exists, creating directory './Qwen3.6-35B-A3B-w8a8'
2026-04-28 01:08:09,840 - msmodelslim.app.naive_quantization - INFO - quantization with following parameters:
2026-04-28 01:08:09,840 - msmodelslim.app.naive_quantization - INFO - model_type: Qwen3.5-35B-A3B
2026-04-28 01:08:09,840 - msmodelslim.app.naive_quantization - INFO - model_path: /data/Qwen3.6-35B-A3B
2026-04-28 01:08:09,840 - msmodelslim.app.naive_quantization - INFO - save_path: /data/Qwen3.6-35B-A3B-w8a8
2026-04-28 01:08:09,840 - msmodelslim.app.naive_quantization - INFO - device_type: DeviceType.NPU
2026-04-28 01:08:09,841 - msmodelslim.app.naive_quantization - INFO - using single device (default): npu
2026-04-28 01:08:09,841 - msmodelslim.app.naive_quantization - INFO - quant_type: QuantType.W8A8
2026-04-28 01:08:09,841 - msmodelslim.app.naive_quantization - INFO - trust_remote_code: False
2026-04-28 01:08:09,841 - msmodelslim.app.naive_quantization - INFO - ===========ANALYSE MODEL===========
2026-04-28 01:08:09,862 - msmodelslim.app.naive_quantization - INFO - Found 63 model adapters: ['DeepSeek-R1', 'DeepSeek-R1-0528', 'DeepSeek-R1-Distill-Qwen-1.5B', 'DeepSeek-R1-Distill-Qwen-7B', 'D                                   eepSeek-V3', 'DeepSeek-V3-0324', 'DeepSeek-V3.1', 'DeepSeek-V3.2', 'DeepSeek-V3.2-Exp', 'DeepSeek-V4-Flash', 'FLUX.1-dev', 'GLM-4.5', 'GLM-4.6', 'GLM-4.6V', 'GLM-4.7', 'GLM-5', 'HunyuanVideo', 'Kim                                   i-K2-Instruct-0905', 'Kimi-K2-Thinking', 'Kimi-K2.5', 'QwQ-32B', 'Qwen-Image-Edit-2509', 'Qwen-QwQ-32B', 'Qwen1.5-110B', 'Qwen2-72B', 'Qwen2-7B', 'Qwen2-7B-Instruct', 'Qwen2.5-32B-Instruct', 'Qwen2                                   .5-72B-Instruct', 'Qwen2.5-7B-Instruct', 'Qwen2.5-Coder-7B-Instruct', 'Qwen2.5-Omni-7B', 'Qwen2.5-VL-32B-Instruct', 'Qwen2.5-VL-72B-Instruct', 'Qwen2.5-VL-7B-Instruct', 'Qwen3-14B', 'Qwen3-235B', '                                   Qwen3-30B', 'Qwen3-32B', 'Qwen3-8B', 'Qwen3-Coder-480B-A35B', 'Qwen3-Next-80B-A3B-Instruct', 'Qwen3-Omni-30B-A3B-Instruct', 'Qwen3-Omni-30B-A3B-Thinking', 'Qwen3-VL-235B-A22B', 'Qwen3-VL-30B-A3B',                                    'Qwen3-VL-32B-Instruct', 'Qwen3-VL-4B-Instruct', 'Qwen3.5-122B-A10B', 'Qwen3.5-27B', 'Qwen3.5-35B-A3B', 'Qwen3.5-397B-A17B', 'Wan2.1', 'Wan2.2', 'Wan2.2-I2V-A14B', 'Wan2.2-T2V-A14B', 'Wan2.2-TI2V-5                                   B', 'Wan2_1', 'Wan2_2', 'default', 'hunyuan-video', 'hunyuan_video', 'hunyuanvideo']
2026-04-28 01:08:09,891 - msmodelslim.app.naive_quantization - ERROR - [UnexpectedError] Code: 500, Message: Unexpected error., TIP: Please report this issue to the msModelSlim developers. Reposito                                   ry: https://gitcode.com/Ascend/msmodelslim Q&A: https://gitcode.com/Ascend/msmodelslim/blob/master/docs/zh/appendix/faq.md
2026-04-28 01:08:09,891 - msmodelslim.app.naive_quantization - ERROR - Original exception: ImportError("cannot import name 'Qwen3_5MoeForConditionalGeneration' from 'transformers' (/usr/local/pytho                                   n3.11.14/lib/python3.11/site-packages/transformers/__init__.py)")
2026-04-28 01:08:09,894 - msmodelslim.app.naive_quantization - ERROR - Traceback: Traceback (most recent call last):   File "/usr/local/python3.11.14/lib/python3.11/site-packages/msmodelslim/utils/                                   exception_decorator.py", line 178, in wrapper     return func(*args, **kwargs)            ^^^^^^^^^^^^^^^^^^^^^   File "/usr/local/python3.11.14/lib/python3.11/site-packages/msmodelslim/app/naive_q                                   uantization/application.py", line 420, in quant     self._quant(   File "/usr/local/python3.11.14/lib/python3.11/site-packages/msmodelslim/utils/logging.py", line 307, in wrapper     return func(*a                                   rgs, **kwargs)            ^^^^^^^^^^^^^^^^^^^^^   File "/usr/local/python3.11.14/lib/python3.11/site-packages/msmodelslim/app/naive_quantization/application.py", line 438, in _quant     model_adapt                                   er = self.model_factory.create(                     ^^^^^^^^^^^^^^^^^^^^^^^^^^   File "/usr/local/python3.11.14/lib/python3.11/site-packages/msmodelslim/model/plugin_factory.py", line 84, in create                                        adapter_class = model_map[model_type].load()                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^   File "/usr/local/python3.11.14/lib/python3.11/site-packages/importlib_metadata/__init__.py", lin                                   e 221, in load     module = import_module(self.module)              ^^^^^^^^^^^^^^^^^^^^^^^^^^   File "/usr/local/python3.11.14/lib/python3.11/importlib/__init__.py", line 126, in import_module                                        return _bootstrap._gcd_import(name[level:], package, level)            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^   File "<frozen importlib._bootstrap>", line 1204, in _gcd_import   File                                    "<frozen importlib._bootstrap>", line 1176, in _find_and_load   File "<frozen importlib._bootstrap>", line 1126, in _find_and_load_unlocked   File "<frozen importlib._bootstrap>", line 241, in _ca                                   ll_with_frames_removed   File "<frozen importlib._bootstrap>", line 1204, in _gcd_import   File "<frozen importlib._bootstrap>", line 1176, in _find_and_load   File "<frozen importlib._bootstrap>",                                    line 1147, in _find_and_load_unlocked   File "<frozen importlib._bootstrap>", line 690, in _load_unlocked   File "<frozen importlib._bootstrap_external>", line 940, in exec_module   File "<frozen                                    importlib._bootstrap>", line 241, in _call_with_frames_removed   File "/usr/local/python3.11.14/lib/python3.11/site-packages/msmodelslim/model/qwen3_5_moe/__init__.py", line 39, in <module>     fro                                   m .model_adapter import Qwen3_5ModelAdapter   File "/usr/local/python3.11.14/lib/python3.11/site-packages/msmodelslim/model/qwen3_5_moe/model_adapter.py", line 37, in <module>     from transformers                                    import Qwen3_5MoeForConditionalGeneration, Qwen3_5ForConditionalGeneration ImportError: cannot import name 'Qwen3_5MoeForConditionalGeneration' from 'transformers' (/usr/local/python3.11.14/lib/py                                   thon3.11/site-packages/transformers/__init__.py)
Traceback (most recent call last):
  File "/usr/local/python3.11.14/lib/python3.11/site-packages/msmodelslim/utils/exception_decorator.py", line 178, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/python3.11.14/lib/python3.11/site-packages/msmodelslim/app/naive_quantization/application.py", line 420, in quant
    self._quant(
  File "/usr/local/python3.11.14/lib/python3.11/site-packages/msmodelslim/utils/logging.py", line 307, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/python3.11.14/lib/python3.11/site-packages/msmodelslim/app/naive_quantization/application.py", line 438, in _quant
    model_adapter = self.model_factory.create(
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/python3.11.14/lib/python3.11/site-packages/msmodelslim/model/plugin_factory.py", line 84, in create
    adapter_class = model_map[model_type].load()
                    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/python3.11.14/lib/python3.11/site-packages/importlib_metadata/__init__.py", line 221, in load
    module = import_module(self.module)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/python3.11.14/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1126, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1204, in _gcd_import
  File "<frozen importlib._bootstrap>", line 1176, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1147, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 690, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 940, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "/usr/local/python3.11.14/lib/python3.11/site-packages/msmodelslim/model/qwen3_5_moe/__init__.py", line 39, in <module>
    from .model_adapter import Qwen3_5ModelAdapter
  File "/usr/local/python3.11.14/lib/python3.11/site-packages/msmodelslim/model/qwen3_5_moe/model_adapter.py", line 37, in <module>
    from transformers import Qwen3_5MoeForConditionalGeneration, Qwen3_5ForConditionalGeneration
ImportError: cannot import name 'Qwen3_5MoeForConditionalGeneration' from 'transformers' (/usr/local/python3.11.14/lib/python3.11/site-packages/transformers/__init__.py)

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/local/python3.11.14/bin/msmodelslim", line 6, in <module>
    sys.exit(main())
             ^^^^^^
  File "/usr/local/python3.11.14/lib/python3.11/site-packages/msmodelslim/cli/__main__.py", line 128, in main
    quant_main(args)
  File "/usr/local/python3.11.14/lib/python3.11/site-packages/msmodelslim/cli/naive_quantization/__main__.py", line 91, in main
    app.quant(
  File "/usr/local/python3.11.14/lib/python3.11/site-packages/msmodelslim/utils/logging.py", line 307, in wrapper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/python3.11.14/lib/python3.11/site-packages/msmodelslim/utils/exception_decorator.py", line 189, in wrapper
    raise unexpected_error from e
msmodelslim.utils.exception.UnexpectedError: Code: 500, Message: Unexpected error.
TIP: Please report this issue to the msModelSlim developers. Repository: https://gitcode.com/Ascend/msmodelslim Q&A: https://gitcode.com/Ascend/msmodelslim/blob/master/docs/zh/appendix/faq.md
[ERROR] 2026-04-28-01:08:09 (PID:47250, Device:-1, RankID:-1) ERR99999 UNKNOWN applicaiton exception

欢迎加入社区，感谢您对社区的贡献 🎉!

anreywmh成员

4月28日评论：

👋 您好,欢迎向 MindStudio msModelSlim 提交 Issue！
我们已收到您的反馈，感谢你对开源社区的支持。🎉

📅处理时效：维护团队将在24小时内 查看并回复您的问题（工作日）。
🔍自助查询：在等待期间，建议您先查阅以下资料，可能已有解决方案：

📖 MindStudio msModelSlim 官方文档
📝 贡献者指南

请确保 Issue 描述清晰，包含复现步骤和日志，这将帮助我们更快定位问题。谢谢！

A anreywmh成员

4月28日添加了label：triaged

ascend-robot成员

4月28日添加了label：bug

libowen成员

4月28日评论：

你好，qwen3.5系列需要transformers版本号5.2.0，请执行pip install transformers==5.2.0再做量化

Z zhangz200102成员

4月28日修改标题为 “[Bug] Qwen3.6-35B-A3B w8a8量化”，原标题为“[Bug] ”

Z zhangz200102成员

4月28日修改标题为 “[Bug] Qwen3.6-35B-A3B w8a8量化报错ImportError: cannot import name 'Qwen3_5MoeForConditionalGeneration' from 'transformers'”，原标题为“[Bug] Qwen3.6-35B-A3B w8a8量化”