[Usage]: 使用Qwen3-32B 稀疏量化该模型的量化已经集成至一键量化。报错-MindStudio-ModelSlim-AtomGit

已关闭

[Usage]: 使用Qwen3-32B 稀疏量化该模型的量化已经集成至一键量化。报错 #2

laosun88888888创建于 1月13日关闭于 1月27日

1月13日创建

在提交新问题之前，请确保您已经在社区中搜索过相关问题，并使用了社区中提供的资源/工具后，仍未找到满意的解决方式。

环境信息

例如：
- 操作系统
- 昇腾硬件信息
- CANN软件版本
- 安装的对应软件版本

麒麟v10服务器版操作系统
华为昇腾 Atlas 300I Duo 推理卡 *4
使用的MindIE镜像是swr.cn-south-1.myhuaweicloud.com/ascendhub/mindie:2.2.RC1-300I-Duo-py311-openeuler24.03-lts
MindIE版本 2.2.RC1
CANN版本 8.3.RC2
Ascend Extension for PyTorch 7.2.0

我是在docker容器内部运行命令msmodelslim quant --model_path /mindie/models/Qwen3-32B --save_path /mindie/models/Qwen3-32B-lim/w8a8s --device npu --model_type Qwen3-32B --quant_type w8a8s --trust_remote_code True 报错

使用场景及问题

msmodelslim quant --model_path /mindie/models/Qwen3-32B --save_path /mindie/models/Qwen3-32B-lim/w8a8s --device npu --model_type Qwen3-32B --quant_type w8a8s --trust_remote_code True
/usr/local/lib64/python3.11/site-packages/torch_npu/contrib/transfer_to_npu.py:311: ImportWarning:
*************************************************************************************************************
The torch.Tensor.cuda and torch.nn.Module.cuda are replaced with torch.Tensor.npu and torch.nn.Module.npu now..
The torch.cuda.DoubleTensor is replaced with torch.npu.FloatTensor cause the double type is not supported now..
The backend in torch.distributed.init_process_group set to hccl now..
The torch.cuda.* and torch.cuda.amp.* are replaced with torch.npu.* and torch.npu.amp.* now..
The device parameters have been replaced with npu in the function below:
torch.logspace, torch.randint, torch.hann_window, torch.rand, torch.full_like, torch.ones_like, torch.rand_like, torch.randperm, torch.arange, torch.frombuffer, torch.normal, torch._empty_per_channel_affine_quantized, torch.empty_strided, torch.empty_like, torch.scalar_tensor, torch.tril_indices, torch.bartlett_window, torch.ones, torch.sparse_coo_tensor, torch.randn, torch.kaiser_window, torch.tensor, torch.triu_indices, torch.as_tensor, torch.zeros, torch.randint_like, torch.full, torch.eye, torch._sparse_csr_tensor_unsafe, torch.empty, torch._sparse_coo_tensor_unsafe, torch.blackman_window, torch.zeros_like, torch.range, torch.sparse_csr_tensor, torch.randn_like, torch.from_file, torch._cudnn_init_dropout_state, torch._empty_affine_quantized, torch.linspace, torch.hamming_window, torch.empty_quantized, torch._pin_memory, torch.load, torch.set_default_device, torch.Tensor.new_empty, torch.Tensor.new_empty_strided, torch.Tensor.new_full, torch.Tensor.new_ones, torch.Tensor.new_tensor, torch.Tensor.new_zeros, torch.Tensor.to, torch.Tensor.pin_memory, torch.nn.Module.to, torch.nn.Module.to_empty
*************************************************************************************************************

The above exception was the direct cause of the following exception:

欢迎加入社区，感谢您对社区的贡献 🎉!

A anreywmh成员

1月14日添加了label：usage

A anreywmh成员

1月14日关联了看板：msit

anreywmh成员

1月14日评论：

感谢您提出的问题，请将config.json中的dtype改为fp16

laosun88888888

1月15日评论：

没有找到dtype，我将config.json修改为 "torch_dtype": "float16",
[root@localhost Qwen3-32B]# vi config.json
{
"architectures": [
"Qwen3ForCausalLM"
],
"attention_bias": false,
"attention_dropout": 0.0,
"bos_token_id": 151643,
"eos_token_id": 151645,
"head_dim": 128,
"hidden_act": "silu",
"hidden_size": 5120,
"initializer_range": 0.02,
"intermediate_size": 25600,
"max_position_embeddings": 40960,
"max_window_layers": 64,
"model_type": "qwen3",
"num_attention_heads": 64,
"num_hidden_layers": 64,
"num_key_value_heads": 8,
"rms_norm_eps": 1e-06,
"rope_scaling": null,
"rope_theta": 1000000,
"sliding_window": null,
"tie_word_embeddings": false,
"torch_dtype": "float16",
"transformers_version": "4.51.0",
"use_cache": true,
"use_sliding_window": false,
"vocab_size": 151936
}

报错信息
msmodelslim quant --model_path /mindie/models/Qwen3-32B --save_path /mindie/models/Qwen3-32B-lim/w8a8s --device npu --model_type Qwen3-32B --quant_type w8a8s --trust_remote_code True
/usr/local/lib64/python3.11/site-packages/torch_npu/contrib/transfer_to_npu.py:311: ImportWarning:
*************************************************************************************************************
The torch.Tensor.cuda and torch.nn.Module.cuda are replaced with torch.Tensor.npu and torch.nn.Module.npu now..
The torch.cuda.DoubleTensor is replaced with torch.npu.FloatTensor cause the double type is not supported now..
The backend in torch.distributed.init_process_group set to hccl now..
The torch.cuda.* and torch.cuda.amp.* are replaced with torch.npu.* and torch.npu.amp.* now..
The device parameters have been replaced with npu in the function below:
torch.logspace, torch.randint, torch.hann_window, torch.rand, torch.full_like, torch.ones_like, torch.rand_like, torch.randperm, torch.arange, torch.frombuffer, torch.normal, torch._empty_per_channel_affine_quantized, torch.empty_strided, torch.empty_like, torch.scalar_tensor, torch.tril_indices, torch.bartlett_window, torch.ones, torch.sparse_coo_tensor, torch.randn, torch.kaiser_window, torch.tensor, torch.triu_indices, torch.as_tensor, torch.zeros, torch.randint_like, torch.full, torch.eye, torch._sparse_csr_tensor_unsafe, torch.empty, torch._sparse_coo_tensor_unsafe, torch.blackman_window, torch.zeros_like, torch.range, torch.sparse_csr_tensor, torch.randn_like, torch.from_file, torch._cudnn_init_dropout_state, torch._empty_affine_quantized, torch.linspace, torch.hamming_window, torch.empty_quantized, torch._pin_memory, torch.load, torch.set_default_device, torch.Tensor.new_empty, torch.Tensor.new_empty_strided, torch.Tensor.new_full, torch.Tensor.new_ones, torch.Tensor.new_tensor, torch.Tensor.new_zeros, torch.Tensor.to, torch.Tensor.pin_memory, torch.nn.Module.to, torch.nn.Module.to_empty
*************************************************************************************************************

warnings.warn(msg, ImportWarning)
/usr/local/lib64/python3.11/site-packages/torch_npu/contrib/transfer_to_npu.py:260: RuntimeWarning: torch.jit.script and torch.jit.script_method will be disabled by transfer_to_npu, which currently does not support them, if you need to enable them, please do not use transfer_to_npu.
warnings.warn(msg, RuntimeWarning)
2026-01-15 09:02:51,682 - msmodelslim.app.naive_quantization - INFO - write directory exists, write file to directory '/mindie/models/Qwen3-32B-lim/w8a8s'
2026-01-15 09:02:51,682 - msmodelslim.app.naive_quantization - INFO - quantization with following parameters:
2026-01-15 09:02:51,682 - msmodelslim.app.naive_quantization - INFO - model_type: Qwen3-32B
2026-01-15 09:02:51,683 - msmodelslim.app.naive_quantization - INFO - model_path: /mindie/models/Qwen3-32B
2026-01-15 09:02:51,683 - msmodelslim.app.naive_quantization - INFO - save_path: /mindie/models/Qwen3-32B-lim/w8a8s
2026-01-15 09:02:51,683 - msmodelslim.app.naive_quantization - INFO - device_type: DeviceType.NPU
2026-01-15 09:02:51,683 - msmodelslim.app.naive_quantization - INFO - using single device (default): npu
2026-01-15 09:02:51,683 - msmodelslim.app.naive_quantization - INFO - quant_type: QuantType.W8A8S
2026-01-15 09:02:51,683 - msmodelslim.app.naive_quantization - INFO - trust_remote_code: True
2026-01-15 09:02:51,683 - msmodelslim.app.naive_quantization - INFO - ===========ANALYSE MODEL===========
2026-01-15 09:02:51,698 - msmodelslim.app.naive_quantization - INFO - Found 38 model adapters: ['DeepSeek-R1', 'DeepSeek-R1-0528', 'DeepSeek-R1-Distill-Qwen-1.5B', 'DeepSeek-R1-Distill-Qwen-7B', 'DeepSeek-V3', 'DeepSeek-V3-0324', 'DeepSeek-V3.1', 'DeepSeek-V3.2', 'DeepSeek-V3.2-Exp', 'HunyuanVideo', 'Kimi-K2-Instruct-0905', 'Kimi-K2-Thinking', 'QwQ-32B', 'Qwen-QwQ-32B', 'Qwen1.5-110B', 'Qwen2-72B', 'Qwen2-7B', 'Qwen2.5-32B-Instruct', 'Qwen2.5-72B-Instruct', 'Qwen2.5-7B-Instruct', 'Qwen2.5-Coder-7B-Instruct', 'Qwen3-14B', 'Qwen3-235B', 'Qwen3-30B', 'Qwen3-32B', 'Qwen3-8B', 'Qwen3-Next-80B-A3B-Instruct', 'Qwen3-VL-235B-A22B', 'Qwen3-VL-30B-A3B', 'Qwen3-VL-32B-Instruct', 'Wan2.1', 'Wan2.2', 'Wan2_1', 'Wan2_2', 'default', 'hunyuan-video', 'hunyuan_video', 'hunyuanvideo']
WARNING:root:Overwriting existing implementation for <class 'msmodelslim.core.quantizer.base.AutoWeightQuantizer'> with dispatch key: (QScheme(scope=per_channel, dtype=int4, symmetric=True), 'minmax')
2026-01-15 09:02:52,185 - msmodelslim.app.naive_quantization - INFO - Using model adapter Qwen3ModelAdapter.
2026-01-15 09:02:52,185 - msmodelslim.app.naive_quantization - INFO - ===========GET BEST PRACTICE===========
2026-01-15 09:02:52,220 - msmodelslim.app.naive_quantization - INFO - Naive Quant apply config_id: qwen3-32b-w8a8s
2026-01-15 09:02:52,221 - msmodelslim.app.naive_quantization - INFO - Get best practice qwen3-32b-w8a8s success.
2026-01-15 09:02:52,221 - msmodelslim.app.naive_quantization - INFO - ===========QUANTIZE MODEL===========
2026-01-15 09:02:52,293 - msmodelslim.core.quant_service.proxy - INFO - Load quant service plugin modelslim_v0 success!
2026-01-15 09:02:52,293 - msmodelslim.core.quant_service.proxy - INFO - Load quant service plugin modelslim_v1 success!
2026-01-15 09:02:52,294 - msmodelslim.core.quant_service.proxy - INFO - Load quant service plugin multimodal_sd_modelslim_v1 success!
2026-01-15 09:02:52,299 - msmodelslim.core.quant_service.proxy - INFO - Load quant service plugin multimodal_vlm_modelslim_v1 success!
2026-01-15 09:02:52,299 - msmodelslim.core.quant_service.modelslim_v0 - INFO - ==========QUANTIZATION: PREPARE DATASET==========
2026-01-15 09:02:52,299 - msmodelslim.core.quant_service.modelslim_v0 - INFO - prepare calib_data from cn_en.jsonl
2026-01-15 09:02:58,511 - msmodelslim.core.quant_service.modelslim_v0 - INFO - prepare calib_data from cn_en.jsonl success
2026-01-15 09:02:58,512 - msmodelslim.core.quant_service.modelslim_v0 - INFO - ==========QUANTIZATION: LOAD MODEL==========
Loading checkpoint shards: 100%|██████████████████████████████████████████████████████████████████████████████████████████████| 17/17 [00:10<00:00, 1.62it/s]
2026-01-15 09:03:13,252 - msmodelslim.core.quant_service.modelslim_v0 - INFO - load model from /mindie/models/Qwen3-32B success
2026-01-15 09:03:13,252 - msmodelslim.core.quant_service.modelslim_v0 - INFO - ==========QUANTIZATION: CALIBRATION==========
2026-01-15 09:03:13,253 - msmodelslim - WARNING - No dev_id of npu device is configured, default device id 0 is set instead.
2026-01-15 09:03:13,268 - msmodelslim - INFO - Automatically disabling the last linear layer: lm_head based on the disable_last_linear parameter setting.
feature process: 0%| | 0/5 [00:00<?, ?it/s][W compiler_depend.ts:116] Warning: Driver Version: ж▒ū▒ is invalid or not supported yet. (function operator())
feature process: 0%| | 0/5 [00:00<?, ?it/s]
2026-01-15 09:03:13,405 - msmodelslim.app.naive_quantization - ERROR - [UnexpectedError] Code: 500, Message: Unexpected error., TIP: Please report this issue to the msModelSlim developers. Repository: https://gitcode.com/Ascend/msmodelslim Q&A: https://gitcode.com/Ascend/msmodelslim/blob/master/docs/zh/faq.md
2026-01-15 09:03:13,405 - msmodelslim.app.naive_quantization - ERROR - Original exception: Exception('Please check the model and calibration data, ensure that your model can run with model(*(calib_data[0])).', RuntimeError("copy_d2d:build/CMakeFiles/torch_npu.dir/compiler_depend.ts:285 NPU function error: c10_npu::acl::AclrtSynchronizeStreamWithTimeout(copy_stream), error code is 507013\n[ERROR] 2026-01-15-09:03:13 (PID:6212, Device:0, RankID:-1) ERR00100 PTA call acl api failed\n[Error]: System Direct Memory Access (DMA) hardware execution error. \n Rectify the fault based on the error information in the ascend log.\nEI9999: Inner Error!\n The error from device(0), serial number is 7. there is a sdma error, sdma channel is 0, the channel exist the following problems: The SMMU returns a Terminate error during page table translation.. the value of CQE status is 2. the description of CQE status: When the SQE translates a page table, the SMMU returns a Terminate error.it's config include: setting1=0xc000000880e0000, setting2=0xff009000ff004c, setting3=0, sq base addr=0x800d00001004c000[FUNC:ProcessSdmaErrorInfo][FILE:device_error_proc.cc][LINE:780]\nEI9999[PID: 6212] 2026-01-15-09:03:13.395.274 (EI9999): Memory async copy failed, device_id=0, stream_id=2, task_id=7, flip_num=0, copy_type=2, memcpy_type=0, copy_data_type=0, length=3164160[FUNC:GetError][FILE:stream.cc][LINE:1191]\n TraceBack (most recent call last):\n rtStreamSynchronizeWithTimeout execute failed, reason=[sdma copy error][FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:53]\n synchronize stream with timeout failed, runtime result = 507013[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:162]\n\n\nDEVICE[0] PID[6212]: \nEXCEPTION STREAM:\n Exception info:TGID=1792054, model id=65535, stream id=2, stream phase=SCHEDULE\n Message info[0]:RTS_HWTS: hwts sdma error, slot_id=21, stream_id=2\n Other info[0]:time=2026-01-15-09:03:12.102.414, function=int_process_hwts_sdma_error, line=1381, error code=0x20b"))
2026-01-15 09:03:13,412 - msmodelslim.app.naive_quantization - ERROR - Traceback: Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/msmodelslim/pytorch/llm_ptq/llm_ptq_tools/quant_tools.py", line 477, in rollback_names_process self.act_states = get_features(model, calib_data[:5], "features.npy", enable_tensor_dump) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "quant_funcs.py", line 1423, in quant_funcs.get_features File "quant_funcs.py", line 1424, in quant_funcs.get_features File "/usr/local/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/accelerate/hooks.py", line 170, in new_forward output = module._old_forward(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/transformers/utils/generic.py", line 965, in wrapper output = func(self, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/transformers/models/qwen3/modeling_qwen3.py", line 850, in forward outputs: BaseModelOutputWithPast = self.model( ^^^^^^^^^^^ File "/usr/local/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/transformers/utils/generic.py", line 965, in wrapper output = func(self, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/transformers/models/qwen3/modeling_qwen3.py", line 553, in forward position_embeddings = self.rotary_emb(hidden_states, position_ids) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl return self._call_impl(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl return forward_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/accelerate/hooks.py", line 165, in new_forward args, kwargs = module._hf_hook.pre_forward(module, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/msmodelslim/pytorch/llm_ptq/accelerate_adapter/hook_adapter.py", line 241, in pre_forward args, kwargs = self.old_hook.pre_forward(module, *args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/accelerate/hooks.py", line 364, in pre_forward return send_to_device(args, self.execution_device), send_to_device( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/accelerate/utils/operations.py", line 175, in send_to_device return honor_type( ^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/accelerate/utils/operations.py", line 82, in honor_type return type(obj)(generator) ^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/accelerate/utils/operations.py", line 176, in tensor, (send_to_device(t, device, non_blocking=non_blocking, skip_keys=skip_keys) for t in tensor) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/accelerate/utils/operations.py", line 156, in send_to_device return tensor.to(device, non_blocking=non_blocking) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib64/python3.11/site-packages/torch_npu/contrib/transfer_to_npu.py", line 165, in decorated return fn(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib64/python3.11/site-packages/torch_npu/utils/_to_int.py", line 34, in decorated return fn(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^ RuntimeError: copy_d2d:build/CMakeFiles/torch_npu.dir/compiler_depend.ts:285 NPU function error: c10_npu::acl::AclrtSynchronizeStreamWithTimeout(copy_stream), error code is 507013 [ERROR] 2026-01-15-09:03:13 (PID:6212, Device:0, RankID:-1) ERR00100 PTA call acl api failed [Error]: System Direct Memory Access (DMA) hardware execution error. Rectify the fault based on the error information in the ascend log. EI9999: Inner Error! The error from device(0), serial number is 7. there is a sdma error, sdma channel is 0, the channel exist the following problems: The SMMU returns a Terminate error during page table translation.. the value of CQE status is 2. the description of CQE status: When the SQE translates a page table, the SMMU returns a Terminate error.it's config include: setting1=0xc000000880e0000, setting2=0xff009000ff004c, setting3=0, sq base addr=0x800d00001004c000[FUNC:ProcessSdmaErrorInfo][FILE:device_error_proc.cc][LINE:780] EI9999[PID: 6212] 2026-01-15-09:03:13.395.274 (EI9999): Memory async copy failed, device_id=0, stream_id=2, task_id=7, flip_num=0, copy_type=2, memcpy_type=0, copy_data_type=0, length=3164160[FUNC:GetError][FILE:stream.cc][LINE:1191] TraceBack (most recent call last): rtStreamSynchronizeWithTimeout execute failed, reason=[sdma copy error][FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:53] synchronize stream with timeout failed, runtime result = 507013[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:162] DEVICE[0] PID[6212]: EXCEPTION STREAM: Exception info:TGID=1792054, model id=65535, stream id=2, stream phase=SCHEDULE Message info[0]:RTS_HWTS: hwts sdma error, slot_id=21, stream_id=2 Other info[0]:time=2026-01-15-09:03:12.102.414, function=int_process_hwts_sdma_error, line=1381, error code=0x20b The above exception was the direct cause of the following exception: Traceback (most recent call last): File "/usr/local/lib/python3.11/site-packages/msmodelslim/utils/exception_decorator.py", line 178, in wrapper return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/msmodelslim/app/naive_quantization/application.py", line 269, in quant self._quant( File "/usr/local/lib/python3.11/site-packages/msmodelslim/utils/logging.py", line 307, in wrapper return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/msmodelslim/app/naive_quantization/application.py", line 300, in _quant self.quant_service.quantize( File "/usr/local/lib/python3.11/site-packages/msmodelslim/utils/logging.py", line 307, in wrapper return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/msmodelslim/core/quant_service/proxy.py", line 59, in quantize self.quant_service.quantize( File "/usr/local/lib/python3.11/site-packages/msmodelslim/utils/logging.py", line 307, in wrapper return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/msmodelslim/core/quant_service/modelslim_v0/quant_service.py", line 95, in quantize return self.quant_process(ModelslimV0QuantConfig.from_base(quant_config), model_adapter, save_path, device) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/msmodelslim/utils/logging.py", line 307, in wrapper return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/msmodelslim/core/quant_service/modelslim_v0/quant_service.py", line 157, in quant_process calibrator = Calibrator( ^^^^^^^^^^^ File "/usr/local/lib64/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.11/site-packages/msmodelslim/pytorch/llm_ptq/llm_ptq_tools/quant_tools.py", line 184, in init self.rollback_names_process(model) File "/usr/local/lib/python3.11/site-packages/msmodelslim/pytorch/llm_ptq/llm_ptq_tools/quant_tools.py", line 479, in rollback_names_process raise Exception("Please check the model and calibration data, " Exception: ('Please check the model and calibration data, ensure that your model can run with model(*(calib_data[0])).', RuntimeError("copy_d2d:build/CMakeFiles/torch_npu.dir/compiler_depend.ts:285 NPU function error: c10_npu::acl::AclrtSynchronizeStreamWithTimeout(copy_stream), error code is 507013\n[ERROR] 2026-01-15-09:03:13 (PID:6212, Device:0, RankID:-1) ERR00100 PTA call acl api failed\n[Error]: System Direct Memory Access (DMA) hardware execution error. \n Rectify the fault based on the error information in the ascend log.\nEI9999: Inner Error!\n The error from device(0), serial number is 7. there is a sdma error, sdma channel is 0, the channel exist the following problems: The SMMU returns a Terminate error during page table translation.. the value of CQE status is 2. the description of CQE status: When the SQE translates a page table, the SMMU returns a Terminate error.it's config include: setting1=0xc000000880e0000, setting2=0xff009000ff004c, setting3=0, sq base addr=0x800d00001004c000[FUNC:ProcessSdmaErrorInfo][FILE:device_error_proc.cc][LINE:780]\nEI9999[PID: 6212] 2026-01-15-09:03:13.395.274 (EI9999): Memory async copy failed, device_id=0, stream_id=2, task_id=7, flip_num=0, copy_type=2, memcpy_type=0, copy_data_type=0, length=3164160[FUNC:GetError][FILE:stream.cc][LINE:1191]\n TraceBack (most recent call last):\n rtStreamSynchronizeWithTimeout execute failed, reason=[sdma copy error][FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:53]\n synchronize stream with timeout failed, runtime result = 507013[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:162]\n\n\nDEVICE[0] PID[6212]: \nEXCEPTION STREAM:\n Exception info:TGID=1792054, model id=65535, stream id=2, stream phase=SCHEDULE\n Message info[0]:RTS_HWTS: hwts sdma error, slot_id=21, stream_id=2\n Other info[0]:time=2026-01-15-09:03:12.102.414, function=int_process_hwts_sdma_error, line=1381, error code=0x20b"))
Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/msmodelslim/pytorch/llm_ptq/llm_ptq_tools/quant_tools.py", line 477, in rollback_names_process
self.act_states = get_features(model, calib_data[:5], "features.npy", enable_tensor_dump)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "quant_funcs.py", line 1423, in quant_funcs.get_features
File "quant_funcs.py", line 1424, in quant_funcs.get_features
File "/usr/local/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/accelerate/hooks.py", line 170, in new_forward
output = module._old_forward(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/transformers/utils/generic.py", line 965, in wrapper
output = func(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/transformers/utils/deprecation.py", line 172, in wrapped_func
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/transformers/models/qwen3/modeling_qwen3.py", line 850, in forward
outputs: BaseModelOutputWithPast = self.model(
^^^^^^^^^^^
File "/usr/local/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/transformers/utils/generic.py", line 965, in wrapper
output = func(self, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/transformers/models/qwen3/modeling_qwen3.py", line 553, in forward
position_embeddings = self.rotary_emb(hidden_states, position_ids)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 1518, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib64/python3.11/site-packages/torch/nn/modules/module.py", line 1527, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/accelerate/hooks.py", line 165, in new_forward
args, kwargs = module._hf_hook.pre_forward(module, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/msmodelslim/pytorch/llm_ptq/accelerate_adapter/hook_adapter.py", line 241, in pre_forward
args, kwargs = self.old_hook.pre_forward(module, *args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/accelerate/hooks.py", line 364, in pre_forward
return send_to_device(args, self.execution_device), send_to_device(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/accelerate/utils/operations.py", line 175, in send_to_device
return honor_type(
^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/accelerate/utils/operations.py", line 82, in honor_type
return type(obj)(generator)
^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/accelerate/utils/operations.py", line 176, in
tensor, (send_to_device(t, device, non_blocking=non_blocking, skip_keys=skip_keys) for t in tensor)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/accelerate/utils/operations.py", line 156, in send_to_device
return tensor.to(device, non_blocking=non_blocking)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib64/python3.11/site-packages/torch_npu/contrib/transfer_to_npu.py", line 165, in decorated
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib64/python3.11/site-packages/torch_npu/utils/_to_int.py", line 34, in decorated
return fn(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^
RuntimeError: copy_d2d:build/CMakeFiles/torch_npu.dir/compiler_depend.ts:285 NPU function error: c10_npu::acl::AclrtSynchronizeStreamWithTimeout(copy_stream), error code is 507013
[ERROR] 2026-01-15-09:03:13 (PID:6212, Device:0, RankID:-1) ERR00100 PTA call acl api failed
[Error]: System Direct Memory Access (DMA) hardware execution error.
Rectify the fault based on the error information in the ascend log.
EI9999: Inner Error!
The error from device(0), serial number is 7. there is a sdma error, sdma channel is 0, the channel exist the following problems: The SMMU returns a Terminate error during page table translation.. the value of CQE status is 2. the description of CQE status: When the SQE translates a page table, the SMMU returns a Terminate error.it's config include: setting1=0xc000000880e0000, setting2=0xff009000ff004c, setting3=0, sq base addr=0x800d00001004c000[FUNC:ProcessSdmaErrorInfo][FILE:device_error_proc.cc][LINE:780]
EI9999[PID: 6212] 2026-01-15-09:03:13.395.274 (EI9999): Memory async copy failed, device_id=0, stream_id=2, task_id=7, flip_num=0, copy_type=2, memcpy_type=0, copy_data_type=0, length=3164160[FUNC:GetError][FILE:stream.cc][LINE:1191]
TraceBack (most recent call last):
rtStreamSynchronizeWithTimeout execute failed, reason=[sdma copy error][FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:53]
synchronize stream with timeout failed, runtime result = 507013[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:162]

DEVICE[0] PID[6212]:
EXCEPTION STREAM:
Exception info:TGID=1792054, model id=65535, stream id=2, stream phase=SCHEDULE
Message info[0]:RTS_HWTS: hwts sdma error, slot_id=21, stream_id=2
Other info[0]:time=2026-01-15-09:03:12.102.414, function=int_process_hwts_sdma_error, line=1381, error code=0x20b

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/lib/python3.11/site-packages/msmodelslim/utils/exception_decorator.py", line 178, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/msmodelslim/app/naive_quantization/application.py", line 269, in quant
self._quant(
File "/usr/local/lib/python3.11/site-packages/msmodelslim/utils/logging.py", line 307, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/msmodelslim/app/naive_quantization/application.py", line 300, in _quant
self.quant_service.quantize(
File "/usr/local/lib/python3.11/site-packages/msmodelslim/utils/logging.py", line 307, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/msmodelslim/core/quant_service/proxy.py", line 59, in quantize
self.quant_service.quantize(
File "/usr/local/lib/python3.11/site-packages/msmodelslim/utils/logging.py", line 307, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/msmodelslim/core/quant_service/modelslim_v0/quant_service.py", line 95, in quantize
return self.quant_process(ModelslimV0QuantConfig.from_base(quant_config), model_adapter, save_path, device)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/msmodelslim/utils/logging.py", line 307, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/msmodelslim/core/quant_service/modelslim_v0/quant_service.py", line 157, in quant_process
calibrator = Calibrator(
^^^^^^^^^^^
File "/usr/local/lib64/python3.11/site-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/msmodelslim/pytorch/llm_ptq/llm_ptq_tools/quant_tools.py", line 184, in init
self.rollback_names_process(model)
File "/usr/local/lib/python3.11/site-packages/msmodelslim/pytorch/llm_ptq/llm_ptq_tools/quant_tools.py", line 479, in rollback_names_process
raise Exception("Please check the model and calibration data, "
Exception: ('Please check the model and calibration data, ensure that your model can run with model(*(calib_data[0])).', RuntimeError("copy_d2d:build/CMakeFiles/torch_npu.dir/compiler_depend.ts:285 NPU function error: c10_npu::acl::AclrtSynchronizeStreamWithTimeout(copy_stream), error code is 507013\n[ERROR] 2026-01-15-09:03:13 (PID:6212, Device:0, RankID:-1) ERR00100 PTA call acl api failed\n[Error]: System Direct Memory Access (DMA) hardware execution error. \n Rectify the fault based on the error information in the ascend log.\nEI9999: Inner Error!\n The error from device(0), serial number is 7. there is a sdma error, sdma channel is 0, the channel exist the following problems: The SMMU returns a Terminate error during page table translation.. the value of CQE status is 2. the description of CQE status: When the SQE translates a page table, the SMMU returns a Terminate error.it's config include: setting1=0xc000000880e0000, setting2=0xff009000ff004c, setting3=0, sq base addr=0x800d00001004c000[FUNC:ProcessSdmaErrorInfo][FILE:device_error_proc.cc][LINE:780]\nEI9999[PID: 6212] 2026-01-15-09:03:13.395.274 (EI9999): Memory async copy failed, device_id=0, stream_id=2, task_id=7, flip_num=0, copy_type=2, memcpy_type=0, copy_data_type=0, length=3164160[FUNC:GetError][FILE:stream.cc][LINE:1191]\n TraceBack (most recent call last):\n rtStreamSynchronizeWithTimeout execute failed, reason=[sdma copy error][FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:53]\n synchronize stream with timeout failed, runtime result = 507013[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:162]\n\n\nDEVICE[0] PID[6212]: \nEXCEPTION STREAM:\n Exception info:TGID=1792054, model id=65535, stream id=2, stream phase=SCHEDULE\n Message info[0]:RTS_HWTS: hwts sdma error, slot_id=21, stream_id=2\n Other info[0]:time=2026-01-15-09:03:12.102.414, function=int_process_hwts_sdma_error, line=1381, error code=0x20b"))

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/usr/local/bin/msmodelslim", line 8, in
sys.exit(main())
^^^^^^
File "/usr/local/lib/python3.11/site-packages/msmodelslim/cli/main.py", line 118, in main
quant_main(args)
File "/usr/local/lib/python3.11/site-packages/msmodelslim/cli/naive_quantization/main.py", line 69, in main
app.quant(
File "/usr/local/lib/python3.11/site-packages/msmodelslim/utils/logging.py", line 307, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.11/site-packages/msmodelslim/utils/exception_decorator.py", line 189, in wrapper
raise unexpected_error from e
msmodelslim.utils.exception.UnexpectedError: Code: 500, Message: Unexpected error.
TIP: Please report this issue to the msModelSlim developers. Repository: https://gitcode.com/Ascend/msmodelslim Q&A: https://gitcode.com/Ascend/msmodelslim/blob/master/docs/zh/faq.md
[W compiler_depend.ts:545] Warning: NPU warning, error code is 507013[Error]:
[Error]: System Direct Memory Access (DMA) hardware execution error.
Rectify the fault based on the error information in the ascend log.
EH9999: Inner Error!
rtDeviceSynchronizeWithTimeout execute failed, reason=[sdma copy error][FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:53]
EH9999[PID: 6212] 2026-01-15-09:03:13.424.574 (EH9999): wait for compute device to finish failed, runtime result = 507013.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:162]
TraceBack (most recent call last):
(function npuSynchronizeUsedDevices)
[W compiler_depend.ts:527] Warning: NPU warning, error code is 507013[Error]:
[Error]: System Direct Memory Access (DMA) hardware execution error.
Rectify the fault based on the error information in the ascend log.
EH9999: Inner Error!
rtDeviceSynchronizeWithTimeout execute failed, reason=[sdma copy error][FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:53]
EH9999[PID: 6212] 2026-01-15-09:03:13.426.415 (EH9999): wait for compute device to finish failed, runtime result = 507013.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:162]
TraceBack (most recent call last):
(function npuSynchronizeDevice)
[ERROR] 2026-01-15-09:03:14 (PID:6212, Device:0, RankID:-1) ERR99999 UNKNOWN application exception

anreywmh成员

1月22日评论：

您好，看错误日志的话，300I Duo不支持多卡跨卡量化，可以尝试设置单卡量化环境export PYTORCH_NPU_ALLOC_CONF=expandable_segments:Trueexport ASCEND_RT_VISIBLE_DEVICES=0 #指定使用的NPU卡

若依旧有报错，尝试设置export ASCEND_RT_VISIBLE_DEVICES=0,1，一卡双芯使用

A anreywmh成员

1月22日将 anreywmh 设为负责人

A anreywmh成员

1月22日添加了label：triaged

A anreywmh成员

1月24日添加了label：resolved；删除了label：triagedusage

A anreywmh成员

1月27日 issue状态由 TODO 改变为 DONE

A anreywmh成员

1月27日关闭了 issue