您好，看错误日志的话，300I Duo不支持多卡跨卡量化，可以尝试设置单卡量化环境
export PYTORCH_NPU_ALLOC_CONF=expandable_segments:True
export ASCEND_RT_VISIBLE_DEVICES=0 #指定使用的NPU卡
加上了模型加载失败，请问下可能是什么原因呢，不加这个设置还是原来的错误
bbd55077a38545ef87e59bae7bd6a03c.log
[root@AI-001 /]# npu-smi info
+--------------------------------------------------------------------------------------------------------+
| npu-smi 25.2.0 Version: 25.2.0 |
+-------------------------------+-----------------+------------------------------------------------------+
| NPU Name | Health | Power(W) Temp(C) Hugepages-Usage(page) |
| Chip Device | Bus-Id | AICore(%) Memory-Usage(MB) |
+===============================+=================+======================================================+
| 0 310P3 | OK | NA 64 976 / 976 |
| 0 0 | 0000:01:00.0 | 0 3752 / 44280 |
+-------------------------------+-----------------+------------------------------------------------------+
| 0 310P3 | OK | NA 64 10 / 10 |
| 1 1 | 0000:01:00.0 | 0 1274 / 43693 |
+===============================+=================+======================================================+
| 32 310P3 | OK | NA 59 18784 / 18784 |
| 0 2 | 0000:02:00.0 | 0 38938/ 44280 |
+-------------------------------+-----------------+------------------------------------------------------+
| 32 310P3 | OK | NA 60 10 / 10 |
| 1 3 | 0000:02:00.0 | 0 1711 / 43693 |
+===============================+=================+======================================================+
+-------------------------------+-----------------+------------------------------------------------------+
| NPU Chip | Process id | Process name | Process memory(MB) |
+===============================+=================+======================================================+
| 0 0 | 3712039 | | 2021 |
| 0 0 | 3711517 | | 107 |
| 0 1 | 3711552 | | 107 |
+===============================+=================+======================================================+
| 32 0 | 3711553 | | 106 |
| 32 0 | 3055226 | | 37641 |
| 32 1 | 3711554 | | 107 |
+===============================+=================+======================================================+
1、请确认模型权重的config.json文件在/home/SSTOR/weights/Qwen3-8B下
2、请问 您的Transformer版本是什么呢？pip install transformers==4.51.0试试？
config.json 存在的，transformers 也是 4.51.0 [捂脸]
[root@AI-001 Qwen3-8B]# ls -al
total 16013212
drwxr-xr-x. 2 root root 4096 Oct 24 11:23 .
drwxr-xr-x. 16 root root 4096 Oct 23 17:04 ..
-rw-r--r--. 1 root root 36 Sep 23 15:49 .mdl
-rw-------. 1 root root 1150 Sep 23 16:07 .msc
-rw-r--r--. 1 root root 36 Sep 23 16:07 .mv
-rw-r--r--. 1 root root 11343 Sep 23 15:49 LICENSE
-rw-r--r--. 1 root root 16660 Sep 23 15:49 README.md
-rw-r--r--. 1 root root 728 Sep 23 17:01 config.json
-rw-r--r--. 1 root root 728 Sep 23 17:01 config.json.bk
-rw-r--r--. 1 root root 73 Sep 23 15:49 configuration.json
-rw-r--r--. 1 root root 4 Oct 24 11:23 fusion_result.json
-rw-r--r--. 1 root root 239 Sep 23 15:49 generation_config.json
-rw-r--r--. 1 root root 1671853 Sep 23 15:49 merges.txt
-rw-r--r--. 1 root root 3996250744 Sep 23 16:06 model-00001-of-00005.safetensors
-rw-r--r--. 1 root root 3993160032 Sep 23 16:07 model-00002-of-00005.safetensors
-rw-r--r--. 1 root root 3959604768 Sep 23 16:07 model-00003-of-00005.safetensors
-rw-r--r--. 1 root root 3187841392 Sep 23 16:02 model-00004-of-00005.safetensors
-rw-r--r--. 1 root root 1244659840 Sep 23 15:55 model-00005-of-00005.safetensors
-rw-r--r--. 1 root root 32878 Sep 23 15:49 model.safetensors.index.json
-rw-r--r--. 1 root root 11422654 Sep 23 15:49 tokenizer.json
-rw-r--r--. 1 root root 9732 Sep 23 15:49 tokenizer_config.json
-rw-r--r--. 1 root root 2776833 Sep 23 15:49 vocab.json
[root@AI-001 Qwen3-8B]# pip list | grep trans
transformers 4.51.0
请帮忙再看看，目npu量化还是失败的
2.1.RC1-300I-Duo-py311-openeuler24.03-lts 环境，量化过程中存在报错，期间尝试过使用cpu量化流程可以跑起来但是太慢了，跑到某个部分程序也会挂掉，使用npu一键量化按照下面的命令会报错
export PYTORCH_NPU_ALLOC_CONF=expandable_segments:True
export ASCEND_RT_VISIBLE_DEVICES=1 #指定使用的NPU卡
msmodelslim quant --model_path /topdraw/SSTOR/weights/Qwen3-8B-Instruct --save_path /topdraw/SSTOR/weights/Qwen3-8B-W8A8S --device npu --model_type Qwen3-8B --quant_type w8a8s --trust_remote_code True
量化报错日志
a4474a03426e4c04bb0d1c3942d27dca.log
修改 config.json dtype 为 float16 报错日志
c31ce00202674405bedb7f12cf529d63.log
你好，可以尝试设置export ASCEND_RT_VISIBLE_DEVICES=0,1，切换成两芯跑
Thanks for sending an issue! Please fill in the following template to help quickly solve your problem.
Describe the current behavior / 问题描述 (Mandatory / 必填)
300I DOU Qwen3-8B 一键量化失败，报NPU错误
Environment / 环境信息 (Mandatory / 必填)
300I DOU
npu-smi info
+--------------------------------------------------------------------------------------------------------+
| npu-smi 25.2.0 Version: 25.2.0 |
+-------------------------------+-----------------+------------------------------------------------------+
| NPU Name | Health | Power(W) Temp(C) Hugepages-Usage(page) |
| Chip Device | Bus-Id | AICore(%) Memory-Usage(MB) |
+===============================+=================+======================================================+
| 0 310P3 | OK | NA 60 0 / 0 |
| 0 0 | 0000:01:00.0 | 0 1730 / 44280 |
+-------------------------------+-----------------+------------------------------------------------------+
| 0 310P3 | OK | NA 59 0 / 0 |
| 1 1 | 0000:01:00.0 | 0 1221 / 43693 |
+===============================+=================+======================================================+
| 32 310P3 | OK | NA 56 18774 / 18774 |
| 0 2 | 0000:02:00.0 | 0 38886/ 44280 |
+-------------------------------+-----------------+------------------------------------------------------+
| 32 310P3 | OK | NA 57 0 / 0 |
| 1 3 | 0000:02:00.0 | 0 1656 / 43693 |
+===============================+=================+======================================================+
+-------------------------------+-----------------+------------------------------------------------------+
| NPU Chip | Process id | Process name | Process memory(MB) |
+===============================+=================+======================================================+
| No running processes found in NPU 0 |
+===============================+=================+======================================================+
| 32 0 | 3055226 | | 37641 |
+===============================+=================+======================================================+
cat /usr/local/Ascend/ascend-toolkit/latest/version.cfg
version: 1.0
runtime_running_version=[8.2.0.0.201:8.2.RC1]
compiler_running_version=[8.2.0.0.201:8.2.RC1]
hccl_running_version=[8.2.0.0.201:8.2.RC1]
opp_running_version=[8.2.0.0.201:8.2.RC1]
toolkit_running_version=[8.2.0.0.201:8.2.RC1]
aoe_running_version=[8.2.0.0.201:8.2.RC1]
ncs_running_version=[8.2.0.0.201:8.2.RC1]
opp_kernel_running_version=[8.2.0.0.201:8.2.RC1]
runtime_upgrade_version=[8.2.0.0.201:8.2.RC1]
compiler_upgrade_version=[8.2.0.0.201:8.2.RC1]
hccl_upgrade_version=[8.2.0.0.201:8.2.RC1]
opp_upgrade_version=[8.2.0.0.201:8.2.RC1]
toolkit_upgrade_version=[8.2.0.0.201:8.2.RC1]
aoe_upgrade_version=[8.2.0.0.201:8.2.RC1]
ncs_upgrade_version=[8.2.0.0.201:8.2.RC1]
opp_kernel_upgrade_version=[8.2.0.0.201:8.2.RC1]
runtime_installed_version=[8.2.0.0.201:8.2.RC1]
compiler_installed_version=[8.2.0.0.201:8.2.RC1]
hccl_installed_version=[8.2.0.0.201:8.2.RC1]
opp_installed_version=[8.2.0.0.201:8.2.RC1]
toolkit_installed_version=[8.2.0.0.201:8.2.RC1]
aoe_installed_version=[8.2.0.0.201:8.2.RC1]
ncs_installed_version=[8.2.0.0.201:8.2.RC1]
opp_kernel_installed_version=[8.2.0.0.201:8.2.RC1]
Steps to reproduce the issue / 重现步骤 (Mandatory / 必填)
msmodelslim quant --model_path /home/SSTOR/weights/Qwen3-8B --save_path /home/SSTOR/weights/Qwen3-8B-W8A8S --device npu --model_type Qwen3-8B --quant_type w8a8s --trust_remote_code True
Describe the expected behavior / 预期结果 (Mandatory / 必填)
不报错，成功量化
Related log / screenshot / 日志 / 截图 (Mandatory / 必填)
daecacd053be457f9aa4041924814f4e.log
Special notes for this issue/备注 (Optional / 选填)