👉 遇到问题先看这里

🌟 第一次使用工具遇到问题？

按🚀 快速入门

❓ 搜索历史issue，查看冷门的同类问题

在🔖 Issue中搜索历史类似问题

操作系统及版本

openEuler 24.03 (LTS-SP4)

python版本

3.11

MsModelSlim 工具版本

ae02743bb763c552431906de60bf834564d75de2

是否必现

是

MsModelSlim 执行命令

msmodelslim eval --device npu --device_id 0 1 2 3 --max_new_tokens 3 --model_path /home/autotest/AutoTestProperty/modelslim/llm_ptq/model/DeepSeek_V31_Terminus_W8A8C8 --model_type DeepSeek-V3.1-Terminus --prompt_file /home/autotest/AutoTestProperty/modelslim/llm_ptq/lab_calib/deepseek_v31_terminus_eval_prompt_file.json

预期行为

1、执行伪量化推理成功，对话正常

实际行为

1、伪量化推理失败，报越界错误：
2026-09-15 20:06:11,364 - msmodelslim.app.eval.application - INFO - Fake-quant inference start: model_type=DeepSeek-V3.1-Terminus, device=DeviceType.NPU, devices=[4, 5, 6, 7], mode=multi-card-DP, samples=1, max_new_tokens=3
2026-09-15 20:06:11,965 - msmodelslim.app.eval.application - ERROR - [UnexpectedError] Code: 500, Message: Unexpected error., TIP: Please report this issue to the msModelSlim developers. Repository: https://gitcode.com/Ascend/msmodelslim Q&A: https://gitcode.com/Ascend/msmodelslim/blob/master/docs/zh/support/faq.md
2026-09-15 20:06:11,967 - msmodelslim.app.eval.application - ERROR - Original exception: IndexError('list index out of range')
2026-09-15 20:06:11,973 - msmodelslim.app.eval.application - ERROR - Traceback: Traceback (most recent call last):
File "/home/anaconda3/envs/vlm_ptq_py311_pt271/lib/python3.11/site-packages/msmodelslim/utils/exception_decorator.py", line 236, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/anaconda3/envs/vlm_ptq_py311_pt271/lib/python3.11/site-packages/msmodelslim/app/eval/application.py", line 147, in run
infer_result = engine.run(
^^^^^^^^^^^
File "/home/anaconda3/envs/vlm_ptq_py311_pt271/lib/python3.11/site-packages/msmodelslim/utils/logging.py", line 314, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/anaconda3/envs/vlm_ptq_py311_pt271/lib/python3.11/site-packages/msmodelslim/core/infer_engine/dp_fake_quant/dp_engine.py", line 83, in run
return self._run_dp(adapter, format_loader, inputs, inference_config, device, device_indices)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/anaconda3/envs/vlm_ptq_py311_pt271/lib/python3.11/site-packages/msmodelslim/utils/logging.py", line 314, in wrapper
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/anaconda3/envs/vlm_ptq_py311_pt271/lib/python3.11/site-packages/msmodelslim/core/infer_engine/dp_fake_quant/dp_engine.py", line 109, in _run_dp
shards = [shard_samples_by_rank(inputs, rank, world_size) for rank in range(world_size)]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/anaconda3/envs/vlm_ptq_py311_pt271/lib/python3.11/site-packages/msmodelslim/core/infer_engine/dp_fake_quant/dp_engine.py", line 109, in
shards = [shard_samples_by_rank(inputs, rank, world_size) for rank in range(world_size)]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/anaconda3/envs/vlm_ptq_py311_pt271/lib/python3.11/site-packages/msmodelslim/core/infer_engine/dp_fake_quant/dp_sharding.py", line 55, in shard_samples_by_rank
return [padded[i] for i in range(rank, total_size, world_size)]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/anaconda3/envs/vlm_ptq_py311_pt271/lib/python3.11/site-packages/msmodelslim/core/infer_engine/dp_fake_quant/dp_sharding.py", line 55, in
return [padded[i] for i in range(rank, total_size, world_size)]
~~~~~~^^^
IndexError: list index out of range

欢迎加入社区，感谢您对社区的贡献 🎉!