在提交新问题之前，请确保您已经在社区中搜索过相关问题，并使用了社区中提供的资源/工具后，仍未找到满意的解决方式。

环境信息

MindIE 3.0.0 (300I Duo 镜像) 在 Atlas 300I Pro 上加载失败导致 NPU 设备故障

环境信息

操作系统

硬件

NPU 驱动 (HDK)

CANN (Toolkit)

MindIE 及其他

问题描述

使用 MindIE 3.0.0 的 300I Duo 镜像在 Atlas 300I Pro（单卡 310P3）上启动服务化推理时，导致 NPU 设备进入不可恢复的故障状态（HDC 断连），后续所有操作均失败。

同一硬件上 vllm-ascend v0.17.0 可正常运行 Qwen3-4B（~9 tok/s），排除硬件故障。

复现步骤

1. 启动容器

docker run -it -d --net=host --shm-size=1g \
   --name mindie \
   -w /home \
   --device=/dev/davinci0:rwm \
   --device=/dev/davinci_manager:rwm \
   --device=/dev/hisi_hdc:rwm \
   --device=/dev/devmm_svm:rwm \
   -v /usr/local/Ascend/driver:/usr/local/Ascend/driver:ro \
   -v /usr/local/Ascend/firmware:/usr/local/Ascend/firmware:ro \
   -v /usr/local/dcmi:/usr/local/dcmi:ro \
   -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi:ro \
   -v /usr/local/sbin:/usr/local/sbin:ro \
   -v /data/weight:/home/weight:ro \
   -v /dev/hugepages:/dev/hugepages \
   swr.cn-south-1.myhuaweicloud.com/ascendhub/mindie:3.0.0-300I-Duo-py311-openeuler24.03-lts bash

2. 安装依赖

cd /usr/local/Ascend/atb-models/requirements/models
pip3 install -r requirements_qwen3.txt

3. 修改 config.json

{
    "ServerConfig": { "httpsEnabled": false },
    "BackendConfig": {
        "npuDeviceIds": [[0]],
        "ModelDeployConfig": {
            "ModelConfig": [{
                "modelName": "qwen3-4b",
                "modelWeightPath": "/home/weight/Qwen3-4B",
                "worldSize": 1,
                "backendType": "atb",
                "trustRemoteCode": false
            }]
        }
    }
}

4. 设置权限并启动服务

cd /usr/local/Ascend/mindie/latest
chmod 750 mindie-service
chmod -R 550 mindie-service/bin
chmod 550 mindie-service/lib
chmod 440 mindie-service/lib/*
chmod -R 550 mindie-service/include
chmod -R 550 mindie-service/scripts
chmod 750 mindie-service/logs
chmod 750 mindie-service/conf
chmod 640 mindie-service/conf/config.json
chmod 700 mindie-service/security
chmod -R 700 mindie-service/security/*

source /usr/local/Ascend/ascend-toolkit/set_env.sh
source /usr/local/Ascend/nnal/atb/set_env.sh
source /usr/local/Ascend/atb-models/set_env.sh
source /usr/local/Ascend/mindie/latest/mindie-llm/set_env.sh
source /usr/local/Ascend/mindie/latest/mindie-service/set_env.sh
export MINDIE_LOG_TO_STDOUT=1

cd /usr/local/Ascend/mindie/latest/mindie-service
./bin/mindieservice_daemon

错误日志（关键部分）

[E527 21:07:44.513698882 compiler_depend.ts:454] operator():build/CMakeFiles/torch_npu.dir/compiler_depend.ts:3490 NPU function error: call aclnnTril failed, error code is 107000
[ERROR] 2026-05-27-21:07:44 (PID:357, Device:0, RankID:-1) ERR00100 PTA call acl api failed
[Error]: Parameter verification failed.
        Check whether the input parameters of the interface are correct.
[PID: 357] 2026-05-27-21:07:44.554.108 Invalid_Argument(EE1001): The argument is invalid.
    Reason: BinaryGetFunction failed, kernel_name=Tril_9cada6f06376fa1b999483e5970057e7_high_performance__kernel0, retCode=0x7020010
    Solution: 1.Check the input parameter range of the function. 2.Check the function invocation relationship.
    TraceBack (most recent call last):
    [drv api] halMemAlloc failed:size=14240(bytes), type=2, moduleId=255,
        drvFlag=18374686480754820096, drvRetCode=42, device_id=0, ErrCode=507899,
        desc=[driver error:internal error], InnerCode=0x7020010
        [FUNC:DevMemAllocManaged][FILE:npu_driver.cc][LINE:1657]
    The argument is invalid.
        Reason: BinaryGetFunction failed, kernel_name=Tril_9cada6f06376fa1b999483e5970057e7_high_performance__kernel0, retCode=0x7020010
    The argument is invalid.
        Reason: rtsFuncGetByName execution failed.
    rtBinaryGetFunction failed, runtime result = 107000.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:148]
    aclrtBinaryGetFunction failed, kernel name: Tril_9cada6f06376fa1b999483e5970057e7_high_performance__kernel0
    Init function handle failed.
    Kernel Run failed. opType: 5, Tril
    launch failed for Tril, errno:361001.

后续级联错误：

[drv api] halMemAlloc failed:size=2097152(bytes), type=64, moduleId=7,
    drvFlag=504403158467347456, drvRetCode=42, device_id=0, ErrCode=507899,
    desc=[driver error:internal error], InnerCode=0x7020010
    [FUNC:DevMemAllocHugePageManaged][FILE:npu_driver.cc][LINE:1534]

RuntimeError: npuSynchronizeDevice: NPU function error:
    AclrtSynchronizeDeviceWithTimeout, error code is 507901
[Error]: The internal host device communication (HDC) session is disconnected.

使用场景及问题

欢迎加入社区，感谢您对社区的贡献 🎉!