Star48
159
代码
代码
Issues208
Pull Requests194
项目讨论
Wiki
分析
Star48
159
已合并
[bugfix]accelerator-type自动获取bugfix & docker only文档优化 #481
ganglv创建于 7月11日
[bugfix]accelerator-type自动获取bugfix & docker only文档优化 #481
已合并
ganglv创建于 7月11日
从ganglv/MindIE-PyMotor_0701docker_only合入到Ascend/MindIE-Motormaster
共 8 个文件变更+397-223
Mdocs/zh/developer_guide/build_motor_image_from_vllm_ascend.md+154-124
@@ -1,6 +1,66 @@
1-# 镜像制作1+# 基于vllm-ascend/sglang镜像安装Motor
2 2 
3-## 获取vLLM-Ascend发布的镜像版本3+## 依赖下载（可选）
4+ 
5+>[!NOTE]说明
6+>如果制作镜像的机器不能联网，先下载依赖。
7+ 
8+在有网的环境执行如下步骤：
9+ 
10+### 1. 下载pciutils
11+ 
12+```sh
13+mkdir -p /mnt/pciutils-offline
14+cd /mnt/pciutils-offline
15+ 
16+apt-get install -y apt-rdepends
17+apt-get download $(apt-rdepends pciutils | grep -v "^ ")
18+ 
19+cd /mnt/
20+tar -czvf pciutils-offline.tar.gz pciutils-offline
21+```
22+ 
23+将`/mnt/pciutils-offline.tar.gz`拷贝到制作镜像机器的`/mnt/`路径下
24+ 
25+### 2. 下载whl依赖
26+ 
27+下载Motor代码到`/mnt/`路径下
28+ 
29+```bash
30+cd /mnt/
31+git clone <motor的git链接>
32+ 
33+mkdir -p /mnt/packages-offline
34+ 
35+# 镜像已自带 transformers，下载前删除该依赖，避免版本冲突
36+sed -i '/^transformers/d' /mnt/MindIE-PyMotor/requirements.txt
37+ 
38+pip download -r /mnt/MindIE-PyMotor/requirements.txt -d /mnt/packages-offline -i https://pypi.tuna.tsinghua.edu.cn/simple
39+ 
40+cd /mnt/
41+tar -czvf packages-offline.tar.gz packages-offline
42+```
43+ 
44+将`/mnt/packages-offline.tar.gz`拷贝到制作镜像机器的`/mnt/`路径下
45+ 
46+### 3. 构建motor的whl包
47+ 
48+```bash
49+cd /mnt/MindIE-PyMotor
50+ 
51+# 构建好的whl包在/mnt/MindIE-PyMotor/dist/路径下
52+bash build.sh
53+ 
54+cd /mnt/
55+tar -czvf MindIE-PyMotor.tar.gz MindIE-PyMotor
56+```
57+ 
58+将`/mnt/MindIE-PyMotor.tar.gz`拷贝到制作镜像机器的`/mnt/`路径下
59+ 
60+## 获取基础镜像，以vLLM-Ascend为例
61+ 
62+>[!NOTE]说明
63+>为提高下载速度，可将`quay.io`替换为`quay.nju.edu.cn`。
4 64 
5获取方法：打开[RED HAT](https://quay.io/repository/ascend/vllm-ascend?tab=tags)，点击需要下载的版本。65获取方法：打开[RED HAT](https://quay.io/repository/ascend/vllm-ascend?tab=tags)，点击需要下载的版本。
6以v0.13.0版本为例，下载命令为：66以v0.13.0版本为例，下载命令为：
@@ -9,145 +69,115 @@
9docker pull quay.io/ascend/vllm-ascend:v0.13.069docker pull quay.io/ascend/vllm-ascend:v0.13.0
10```70```
11 71 
12->[!NOTE]说明72+## 安装PyMotor
13->为提高下载速度，可将`quay.io`替换为`quay.nju.edu.cn`。
14 73 
15-## 在镜像中安装PyMotor74+### 1. 查看镜像
16 75 
17->[!NOTE]说明76+```bash
18->如果制作镜像的机器不能联网，需要在步骤4中下载依赖。77+docker images
78+```
19 79 
20-1. 准备好目标motor代码，执行以下命令，git命令根据需要下载的分支或tag进行修改。80+### 2. 创建容器，并挂载mnt目录
81+ 
82+```bash
83+docker run -d --name docker-vllm-ascend -v /mnt/:/mnt/ <镜像名称>
84+```
85+ 
86+### 3. 启动容器
87+ 
88+```bash
89+docker start docker-vllm-ascend
90+```
91+ 
92+### 4. 进入容器
93+ 
94+```bash
95+docker exec -it docker-vllm-ascend bash
96+```
97+ 
98+### 5. 安装motor及其依赖
99+ 
100+#### 5.1 安装 pciutils
101+ 
102+- 在线安装：
103+ 
104+```bash
105+apt-get update && apt-get install pciutils -y
106+```
107+ 
108+- 离线安装：
109+ 
110+```sh
111+cd /mnt/
112+tar -xzvf pciutils-offline.tar.gz
113+cd pciutils-offline
114+ 
115+dpkg -i *.deb
116+```
117+ 
118+#### 5.2 安装whl依赖
119+ 
120+- 在线安装：
21 121 
22 ```bash122 ```bash
123+ # 下载motor代码，执行以下命令，git命令根据需要下载的分支或tag进行修改
23 cd /mnt/124 cd /mnt/
24 git clone <motor的git链接>125 git clone <motor的git链接>
126+ 
127+ cd /mnt/MindIE-PyMotor
128+ 
129+ # 镜像已自带 transformers，安装前删除该依赖，避免版本冲突
130+ sed -i '/^transformers/d' requirements.txt
131+ 
132+ pip install -r requirements.txt
133+ 
134+ bash build.sh
135+ pip install --force-reinstall ./dist/motor-0.1.0-py3-none-any.whl
136+ 
137+ mkdir -p /tmp/motor/
138+ cp -r ./examples/ /tmp/motor/
139+ 
140+ # 退出容器
141+ exit
25 ```142 ```
26 143 
27-2. 执行以下命令查看第一步下载下来的镜像。144+- 离线安装
28 145 
29 ```bash146 ```bash
30- docker images147+ # 安装whl依赖
148+ cd /mnt/
149+ tar -xzvf packages-offline.tar.gz
150+ pip install /mnt/packages-offline/*.whl --force-reinstall --no-index -v
151+ 
152+ # 安装motor
153+ pip install --force-reinstall /mnt/MindIE-PyMotor/dist/motor-0.1.0-py3-none-any.whl --force-reinstall --no-index -v
154+ 
155+ # 拷贝examples
156+ mkdir -p /tmp/motor/
157+ cp -r /mnt/MindIE-PyMotor/examples/ /tmp/motor/
158+ 
159+ # 退出容器
160+ exit
31 ```161 ```
32 162 
33-3. 执行以下命令运行容器并挂载mnt目录。163+### 6. 保存镜像
34 164 
35- ```bash165+```bash
36- docker run -d --name docker-vllm-ascend -v /mnt/:/mnt/ <镜像名称>166+docker commit -m "add motor" docker-vllm-ascend mindie-motor-vllm:dev-800I-A3-py311-lts-aarch64
37- ```167+```
38 168 
39-4. 执行以下命令启动容器。169+保存后，`mindie-motor-vllm:dev-800I-A3-py311-lts-aarch64`镜像就是制作好之后带motor的镜像。
40 170 
41- ```bash171+### 7. 打包镜像
42- docker start docker-vllm-ascend
43- ```
44 172 
45-5. 依赖下载，**如果制作镜像的机器能联网，可在线安装，忽略此步骤**173+```bash
174+docker save -o /mnt/motor-vllm-ascend.tar mindie-motor-vllm:dev-800I-A3-py311-lts-aarch64
175+```
46 176 
47- 1. 下载`pciutils`177+### 8. 导入带有motor的镜像
48 178 
49- ```sh179+在非制作镜像的节点导入镜像
50- mkdir -p /mnt/pciutils-offline
51- cd /mnt/pciutils-offline
52 180 
53- apt-get install -y apt-rdepends181+```bash
54- apt-get download $(apt-rdepends pciutils | grep -v "^ ")182+docker load -i /mnt/motor-vllm-ascend.tar
55- 183+```
56- cd /mnt/
57- tar -czvf pciutils-offline.tar.gz pciutils-offline
58- ```
59- 
60- 将`/mnt/pciutils-offline.tar.gz`拷贝到制作镜像机器的`/mnt/`路径下
61- 
62- 2. 下载`whl`依赖
63- 
64- ```sh
65- mkdir -p /mnt/packages-offline
66- pip download -r MindIE-PyMotor/requirements.txt -d /mnt/packages-offline -i https://pypi.tuna.tsinghua.edu.cn/simple
67- 
68- cd /mnt/
69- tar -czvf packages-offline.tar.gz packages-offline
70- ```
71- 
72- 将`/mnt/packages-offline.tar.gz`拷贝到制作镜像机器的`/mnt/`路径下
73- 
74-6. 执行以下命令制作镜像。
75- 
76- **进入容器**
77- 
78- ```bash
79- docker exec -it docker-vllm-ascend bash
80- ```
81- 
82- **安装 `pciutils`**
83- 
84- - 在线安装：
85- 
86- ```bash
87- apt-get update && apt-get install pciutils -y
88- ```
89- 
90- - 离线安装：
91- 
92- ```sh
93- cd /mnt/
94- tar -xzvf pciutils-offline.tar.gz
95- cd pciutils-offline
96- 
97- dpkg -i *.deb
98- ```
99- 
100- **安装whl依赖**
101- 
102- - 在线安装：
103- 
104- ```bash
105- cd /mnt/MindIE-PyMotor
106- pip install -r requirements.txt
107- bash build.sh
108- pip install --force-reinstall ./dist/motor-0.1.0-py3-none-any.whl
109- 
110- mkdir -p /tmp/motor/
111- cp -r ./examples/ /tmp/motor/
112- 
113- exit
114- ```
115- 
116- - 离线安装
117- 
118- ```bash
119- cd /mnt/
120- tar -xzvf packages-offline.tar.gz
121- 
122- cd /mnt/MindIE-PyMotor
123- pip install --no-index --find-links=/mnt/packages-offline -r requirements.txt
124- bash build.sh
125- pip install --force-reinstall ./dist/motor-0.1.0-py3-none-any.whl
126- 
127- mkdir -p /tmp/motor/
128- cp -r ./examples/ /tmp/motor/
129- 
130- exit
131- ```
132- 
133-7. 执行以下命令保存镜像。
134- 
135- ```bash
136- docker commit -m "add motor" docker-vllm-ascend mindie-motor-vllm:dev-800I-A3-py311-lts-aarch64
137- ```
138- 
139- 保存后，`mindie-motor-vllm:dev-800I-A3-py311-lts-aarch64`镜像就是制作好之后带motor的镜像。
140- 
141-8. 打包镜像
142- 
143- ```bash
144- docker save -o /mnt/motor-vllm-ascend.tar mindie-motor-vllm:dev-800I-A3-py311-lts-aarch64
145- ```
146- 
147-9. 导入带有motor的镜像
148- 
149- 在非制作镜像的节点导入镜像
150- 
151- ```bash
152- docker load -i /mnt/motor-vllm-ascend.tar
153- ```
Mdocs/zh/user_guide/deployment/docker/single_container.md+80-44
@@ -6,6 +6,36 @@
6 6 
7## 部署流程7## 部署流程
8 8 
9+以`/mnt/motor`作为根路径，目录结构如下：
10+ 
11+```text
12+/mnt/motor/
13+├── prepare.sh
14+├── start_motor.sh
15+├── start_docker.sh
16+├── user_config.json
17+├── env.json
18+├── examples/
19+└── configmap/ # 该目录下的文件都是自动生成的
20+ ├── boot.sh
21+ ├── common.sh
22+ ├── hccl_tools.py
23+ ├── mooncake_config.py
24+ ├── all_combine_in_single_container.sh
25+ ├── controller.sh
26+ ├── coordinator.sh
27+ ├── engine.sh
28+ ├── kv_conductor.sh
29+ ├── kv_pool.sh
30+ ├── mf_store.sh
31+ ├── user_config.json
32+ └── env.json
33+```
34+ 
35+### 准备examples
36+ 
37+`examples` 获取方式见[quick_start](../../quick_start.md#服务部署)
38+ 
9### 准备user_config.json和env.json配置文件39### 准备user_config.json和env.json配置文件
10 40 
11可从如下路径获取[user_config.json](https://gitcode.com/Ascend/MindIE-PyMotor/blob/master/examples/infer_engines/vllm/user_config.json)和[env.json](https://gitcode.com/Ascend/MindIE-PyMotor/blob/master/examples/infer_engines/vllm/env.json)模板，本文主要介绍docker-only部署方式相关适配点，其他特性请参考[quick_start](../../quick_start.md)。41可从如下路径获取[user_config.json](https://gitcode.com/Ascend/MindIE-PyMotor/blob/master/examples/infer_engines/vllm/user_config.json)和[env.json](https://gitcode.com/Ascend/MindIE-PyMotor/blob/master/examples/infer_engines/vllm/env.json)模板，本文主要介绍docker-only部署方式相关适配点，其他特性请参考[quick_start](../../quick_start.md)。
@@ -61,17 +91,20 @@
61}91}
62```92```
63 93 
64-### 准备CONFIGMAP_PATH94+### 准备configmap
65 95 
66准备阶段需将配置文件、启动脚本拷贝到环境变量**CONFIGMAP_PATH**对应目录下，并通过set_env_docker.py加载环境变量。准备阶段脚本**prepare.sh**示例(**EXAMPLES_PATH**、**CONFIGMAP_PATH**、**USER_CONFIG_PATH**、**ENV_PATH**需修改为实际路径)：96准备阶段需将配置文件、启动脚本拷贝到环境变量**CONFIGMAP_PATH**对应目录下，并通过set_env_docker.py加载环境变量。准备阶段脚本**prepare.sh**示例(**EXAMPLES_PATH**、**CONFIGMAP_PATH**、**USER_CONFIG_PATH**、**ENV_PATH**需修改为实际路径)：
67 97 
98+以下以`/mnt/motor`作为根路径为例
99+ 
68```shell100```shell
69-EXAMPLES_PATH="xxx" # 主机examples部署脚本路径101+EXAMPLES_PATH="/mnt/motor/examples/"
70-CONFIGMAP_PATH="xxx" # 服务启动脚本路径，需挂载到容器内102+CONFIGMAP_PATH="/mnt/motor/configmap/"
71-USER_CONFIG_PATH="xxx" # user_config.json路径103+USER_CONFIG_PATH="/mnt/motor/user_config.json"
72-ENV_PATH="xxx" # env.json路径104+ENV_PATH="/mnt/motor/env.json"
73 105 
74mkdir -p $CONFIGMAP_PATH106mkdir -p $CONFIGMAP_PATH
107+ 
75# 容器启动脚本boot.sh，其运行时会调用startup目录下其他脚本，需要将其统一拷贝到$CONFIGMAP_PATH目录下。108# 容器启动脚本boot.sh，其运行时会调用startup目录下其他脚本，需要将其统一拷贝到$CONFIGMAP_PATH目录下。
76cp -f $EXAMPLES_PATH/deployer/startup/boot.sh $CONFIGMAP_PATH/boot.sh109cp -f $EXAMPLES_PATH/deployer/startup/boot.sh $CONFIGMAP_PATH/boot.sh
77cp -f $EXAMPLES_PATH/deployer/startup/common.sh $CONFIGMAP_PATH/common.sh110cp -f $EXAMPLES_PATH/deployer/startup/common.sh $CONFIGMAP_PATH/common.sh
@@ -109,38 +142,52 @@ python $EXAMPLES_PATH/deployer/startup/set_env_docker.py --configmap_path $CONFI
109sh prepare.sh142sh prepare.sh
110```143```
111 144 
112-### Docker启动服务145+执行完成后，在`/mnt/motor/configmap/`目录下会生成一些脚本
113 146 
114-准备启动脚本start_docker.sh，脚本示例(**CONFIGMAP_PATH**需修改为实际路径，**IMAGE_NAME**需修改为实际镜像名)：147+### 准备Motor启动脚本
148+ 
149+准备`start_motor.sh`脚本
150+ 
151+```sh
152+CONFIGMAP_PATH="/mnt/motor/configmap" # CONFIGMAP_PATH需与prepare.sh保持一致，且必须使用绝对路径
153+CONFIG_PATH=/usr/local/Ascend/pyMotor/conf
154+ 
155+ROLE=SINGLE_CONTAINER
156+ 
157+# mooncake池化配置
158+# 若开启池化，KVS_MASTER_SERVICE设置为任意非空字符串,如kvp_master，不开启池化设置为空。
159+KVS_MASTER_SERVICE=""
160+KV_STORE_PORT=50088
161+KV_STORE_EVICTION_HIGH_WATERMARK_RATIO=0.9
162+KV_STORE_EVICTION_RATIO=0.1
163+DEFAULT_KV_LEASE_TTL=11000
164+ 
165+source $CONFIGMAP_PATH/boot.sh
166+```
167+ 
168+环境变量说明：
169+ 
170+| 变量名 | 含义 | 取值 |
171+| :--- | :--- | :--- |
172+| KVS_MASTER_SERVICE | mooncake_master部署域名 | 若开启kv_pool，设置为任意非空字符串，如kvp_master，boot.sh会自动适配为容器ip；若不开启则设置为空 |
173+| KV_STORE_PORT | mooncake_master部署端口 | 若开启kv_pool，设置任意有效端口，如50088；若不开启则设置为空 |
174+| KV_STORE_EVICTION_HIGH_WATERMARK_RATIO | mooncake_master进程高水位比例 | 若开启kv_pool，取值0~1；若不开启则设置为空 |
175+| KV_STORE_EVICTION_RATIO | mooncake_master进程逐出比例 | 若开启kv_pool，取值0~1；若不开启则设置为空 |
176+| DEFAULT_KV_LEASE_TTL | 控制 KV 对象的默认租约 TTL（毫秒） | 配置值需大于env.json中vllm实例的环境变量`ASCEND_CONNECT_TIMEOUT`和`ASCEND_TRANSFER_TIMEOUT`。默认值11000；若不开启kv_pool则设置为空 |
177+ 
178+### 准备Docker启动脚本
179+ 
180+准备启动脚本`start_docker.sh`，脚本示例(**CONFIGMAP_PATH**需修改为实际路径，**IMAGE_NAME**需修改为实际镜像名)：
115 181 
116```shell182```shell
117# 默认不开启特权容器，如需开启，将--privileged=false改为--privileged=true183# 默认不开启特权容器，如需开启，将--privileged=false改为--privileged=true
118-CONFIGMAP_PATH="xxx" # CONFIGMAP_PATH需与prepare.sh保持一致，且必须使用绝对路径184+CONFIGMAP_PATH="/mnt/motor/configmap" # CONFIGMAP_PATH需与prepare.sh保持一致，且必须使用绝对路径
119IMAGE_NAME="xxx" # 镜像名185IMAGE_NAME="xxx" # 镜像名
120 186 
121-# 从环境变量读取可见卡，默认自动检测主机昇腾卡，用逗号拼接，如"0,1,2,3"
122-if [ -z "$ASCEND_VISIBLE_DEVICES" ]; then
123- ASCEND_VISIBLE_DEVICES=$(ls /dev/davinci[0-9]* 2>/dev/null | sed 's/[^0-9]//g' | paste -sd "," -)
124-fi
125ASCEND_DEVICES="--device=/dev/davinci_manager --device=/dev/devmm_svm --device=/dev/hisi_hdc"187ASCEND_DEVICES="--device=/dev/davinci_manager --device=/dev/devmm_svm --device=/dev/hisi_hdc"
atomgit-bot
atomgit-botatomgit-bot7月11日

🟠 High Priority

变更行：第 188 行，start_docker.sh 示例中 ASCEND_DEVICES 仅保留了管理设备（davinci_manager、devmm_svm、hisi_hdc），删除了旧版中通过 ASCEND_VISIBLE_DEVICES 循环挂载各个 /dev/davinci$i 计算设备的逻辑。

影响：昇腾 NPU 架构中，davinci_manager 仅为管理设备，实际 AI 计算需要各个 /dev/davinci0、/dev/davinci1 等计算设备。容器内缺少这些设备将导致服务无法执行推理任务，启动即失败。

证据：同一文档 A5 环境章节（第 235 行）仍保留了 # 按 ASCEND_VISIBLE_DEVICES 循环追加 --device=/dev/davinci$i 的注释，说明该循环逻辑是必需的，此处删除是遗漏。

建议：恢复 ASCEND_VISIBLE_DEVICES 的检测与循环挂载逻辑，将计算设备 /dev/davinci$i 加入 ASCEND_DEVICES 变量中

likedislike
不准确？
126-# 循环挂载ASCEND_VISIBLE_DEVICES指定卡
127-IFS=',' read -ra ADDR <<< "$ASCEND_VISIBLE_DEVICES"
128-for i in "${ADDR[@]}"; do
129- ASCEND_DEVICES="$ASCEND_DEVICES --device=/dev/davinci$i"
130-done
131 188 
132docker run -u root --rm --name single_container \189docker run -u root --rm --name single_container \
133-e ASCEND_RUNTIME_OPTIONS=NODRV --privileged=false \190-e ASCEND_RUNTIME_OPTIONS=NODRV --privileged=false \
134--e CONFIGMAP_PATH=$CONFIGMAP_PATH \
135--e CONFIG_PATH=/usr/local/Ascend/pyMotor/conf \
136--e ROLE=SINGLE_CONTAINER \
137--e KVS_MASTER_SERVICE=$KVS_MASTER_SERVICE \
138--e KV_STORE_PORT=$KV_STORE_PORT \
139--e KV_STORE_EVICTION_HIGH_WATERMARK_RATIO=$KV_STORE_EVICTION_HIGH_WATERMARK_RATIO \
140--e KV_STORE_EVICTION_RATIO=$KV_STORE_EVICTION_RATIO \
141--e DEFAULT_KV_LEASE_TTL=$DEFAULT_KV_LEASE_TTL \
142--p $ENDPOINT_PORT_RANGE:$ENDPOINT_PORT_RANGE \
143--p $KV_PORT_RANGE:$KV_PORT_RANGE \
144$ASCEND_DEVICES \191$ASCEND_DEVICES \
145-v /usr/local/Ascend/driver:/usr/local/Ascend/driver \192-v /usr/local/Ascend/driver:/usr/local/Ascend/driver \
146-v /usr/local/Ascend/add-ons/:/usr/local/Ascend/add-ons/ \193-v /usr/local/Ascend/add-ons/:/usr/local/Ascend/add-ons/ \
@@ -148,30 +195,20 @@ $ASCEND_DEVICES \
148-v /usr/local/sbin:/usr/local/sbin \195-v /usr/local/sbin:/usr/local/sbin \
149-v /var/log/npu/:/usr/slog \196-v /var/log/npu/:/usr/slog \
150-v /mnt:/mnt \197-v /mnt:/mnt \
198+-p 31015:1025 \
199+-p 31017:1027 \
151$IMAGE_NAME \200$IMAGE_NAME \
152-bash -c "export POD_IP=\$(grep \$(hostname) /etc/hosts | cut -f1) && source \$CONFIGMAP_PATH/boot.sh"201+bash -c "export POD_IP=\$(grep \$(hostname) /etc/hosts | cut -f1) && source /mnt/motor/start_motor.sh"
atomgit-bot
atomgit-botatomgit-bot7月11日

🟠 High Priority

变更行：第 190–200 行，start_docker.sh 的 docker run 命令删除了旧版中的端口映射参数：

旧版有： -p ENDPOINTPORTRANGE:ENDPOINT_PORT_RANGE:ENDPOINT_PORT_RANGE
-p KVPORTRANGE:KV_PORT_RANGE:KV_PORT_RANGE \

新版完全移除了这些端口映射，同时 ENDPOINT_PORT_RANGE 和 KV_PORT_RANGE 变量及其环境变量说明表也被一并删除。

影响：新版 docker run 未使用 --network host（仅 A5 章节使用），且没有端口映射。容器内 Motor 的推理服务端口（如 coordinator 1025/1026/1027、controller 2026 等）将无法从宿主机或其他节点访问，导致服务不可达。用户按文档操作后将无法成功调用推理服务。

建议：恢复 docker run 中的端口映射参数（-p），或添加说明引导用户根据 user_config.json 配置自行添加端口映射

likedislike
不准确？
G
ganglv
7月13日 评论：
153```202```
154 203 
155-环境变量说明：204+**注意：挂载路径要包含/mnt**
156 205 
157-| 变量名 | 含义 | 取值 |206+### 启动Docker
158-| :--- | :--- | :--- |
159-| CONFIGMAP_PATH | 启动脚本路径 | 与2.2小节保持一致，需挂载到容器中 |
160-| IMAGE_NAME | 镜像名 | 版本镜像，确保docker images能查询到 |
161-| ASCEND_VISIBLE_DEVICES | 可见卡 | 指定挂载卡，如"0,1,2,3"，默认自动检测主机昇腾卡 |
162-| ENDPOINT_PORT_RANGE | endpoint端口映射区间 | 非host网络部署设置endpoint端口映射，起始端口默认值10000，先P后D，每dp端口偏移2，分别对应推理端口和管理端口 |
163-| KV_PORT_RANGE | kv_port映射端口区间 | 非host网络部署设置kv_port映射端口，起始端口user-config.json中motor_engine_prefill_config下kv_port值，先P后D，每实例端口偏移1 |
164-| KVS_MASTER_SERVICE | mooncake_master部署域名 | 若开启kv_pool，设置为任意非空字符串，如kvp_master，boot.sh会自动适配为容器ip；若不开启则设置为空 |
165-| KV_STORE_PORT | mooncake_master部署端口 | 若开启kv_pool，设置任意有效端口，如50088；若不开启则设置为空 |
166-| KV_STORE_EVICTION_HIGH_WATERMARK_RATIO | mooncake_master进程高水位比例 | 若开启kv_pool，取值0~1；若不开启则设置为空 |
167-| KV_STORE_EVICTION_RATIO | mooncake_master进程逐出比例 | 若开启kv_pool，取值0~1；若不开启则设置为空 |
168-| DEFAULT_KV_LEASE_TTL | 控制 KV 对象的默认租约 TTL（毫秒） | 配置值需大于env.json中vllm实例的环境变量`ASCEND_CONNECT_TIMEOUT`和`ASCEND_TRANSFER_TIMEOUT`。默认值11000；若不开启kv_pool则设置为空 |
169 207 
170启动服务示例（1P1D）：208启动服务示例（1P1D）：
171 209 
172```shell210```shell
173-# 若开启池化，KVS_MASTER_SERVICE设置为任意非空字符串,如kvp_master，不开启池化设置为空。211+sh start_docker.sh
174-ASCEND_VISIBLE_DEVICES=0,1 KVS_MASTER_SERVICE="" KV_STORE_PORT=50088 KV_STORE_EVICTION_HIGH_WATERMARK_RATIO=0.9 KV_STORE_EVICTION_RATIO=0.1 DEFAULT_KV_LEASE_TTL=11000 sh start_docker.sh
175```212```
176 213 
177### A5 环境额外修改内容214### A5 环境额外修改内容
@@ -196,7 +233,6 @@ A5 启动示例片段（基于上述实例基础修改）：
196 233 
197```shell234```shell
198ASCEND_DEVICES="--device=/dev/davinci_manager --device=/dev/hisi_hdc"235ASCEND_DEVICES="--device=/dev/davinci_manager --device=/dev/hisi_hdc"
199-# 按 ASCEND_VISIBLE_DEVICES 循环追加 --device=/dev/davinci$i
200 236 
201docker run -u root --rm --name single_container \237docker run -u root --rm --name single_container \
202 --network host \238 --network host \
Mexamples/deployer/lib/config_validator.py+2-2
@@ -182,12 +182,12 @@ def _get_hardware_node_labels(hardware_type):
182 if hardware_type in C.HARDWARE_TYPE_A2 or hardware_type in C.HARDWARE_TYPE_A3:182 if hardware_type in C.HARDWARE_TYPE_A2 or hardware_type in C.HARDWARE_TYPE_A3:
183 return {183 return {
184 C.ACCELERATOR: C.ACCELERATOR_910,184 C.ACCELERATOR: C.ACCELERATOR_910,
185- C.ACCELERATOR_TYPE: get_accelerator_type_from_cluster(),185+ C.ACCELERATOR_TYPE: get_accelerator_type_from_cluster(hardware_type),
186 }186 }
187 if hardware_type in C.HARDWARE_TYPE_950I_A5:187 if hardware_type in C.HARDWARE_TYPE_950I_A5:
188 return {188 return {
189 C.ACCELERATOR: C.ACCELERATOR_A5,189 C.ACCELERATOR: C.ACCELERATOR_A5,
190- C.ACCELERATOR_TYPE: get_accelerator_type_from_cluster(),190+ C.ACCELERATOR_TYPE: get_accelerator_type_from_cluster(hardware_type),
191 }191 }
192 known = [*sorted(C.HARDWARE_TYPE_A2), *sorted(C.HARDWARE_TYPE_A3), *C.HARDWARE_TYPE_950I_A5]192 known = [*sorted(C.HARDWARE_TYPE_A2), *sorted(C.HARDWARE_TYPE_A3), *C.HARDWARE_TYPE_950I_A5]
193 raise ValueError(f"Unknown hardware_type '{hardware_type}'. Supported values: {known}")193 raise ValueError(f"Unknown hardware_type '{hardware_type}'. Supported values: {known}")
Mexamples/deployer/lib/generator/engine.py+3-1
@@ -197,9 +197,11 @@ def set_engine_npu(container, deploy_config, node_type):
197 197 
198 198 
199def apply_node_selector_by_hardware(pod_spec, hardware_type):199def apply_node_selector_by_hardware(pod_spec, hardware_type):
200- pod_spec[C.NODE_SELECTOR][C.ACCELERATOR_TYPE] = k8s_utils.get_accelerator_type_from_cluster()200+ if hardware_type in C.HARDWARE_TYPE_A2 or hardware_type in C.HARDWARE_TYPE_A3:
201+ pod_spec[C.NODE_SELECTOR][C.ACCELERATOR] = C.ACCELERATOR_910
201 if hardware_type in C.HARDWARE_TYPE_950I_A5:202 if hardware_type in C.HARDWARE_TYPE_950I_A5:
202 pod_spec[C.NODE_SELECTOR][C.ACCELERATOR] = C.ACCELERATOR_A5203 pod_spec[C.NODE_SELECTOR][C.ACCELERATOR] = C.ACCELERATOR_A5
204+ pod_spec[C.NODE_SELECTOR][C.ACCELERATOR_TYPE] = k8s_utils.get_accelerator_type_from_cluster(hardware_type)
203 205 
204 206 
205def apply_pd_heterogeneous_node_selector(pod_spec, deploy_config, node_type):207def apply_pd_heterogeneous_node_selector(pod_spec, deploy_config, node_type):
Mexamples/deployer/lib/generator/k8s_utils.py+66-16
@@ -220,29 +220,79 @@ def _get_kubectl_path():
220 return kubectl220 return kubectl
221 221 
222 222 
223-def get_accelerator_type_from_cluster():223+def _get_cluster_nodes(label_selector):
224- """Resolve accelerator-type from the first cluster node that has the label via kubectl."""
225- if C.ACCELERATOR_TYPE in _g_accelerator_type_cache:
226- return _g_accelerator_type_cache[C.ACCELERATOR_TYPE]
227- 
228 kubectl = _get_kubectl_path()224 kubectl = _get_kubectl_path()
229- out = run_cmd_get_output([kubectl, "get", "nodes", "-o", "json"])225+ out = run_cmd_get_output([kubectl, "get", "nodes", "-l", label_selector, "-o", "json"])
230- nodes = json.loads(out).get("items", [])226+ return json.loads(out).get("items", [])
231 227 
228+ 
229+def _collect_node_accelerator_types(nodes):
230+ accelerator_types = set()
232 for node in nodes:231 for node in nodes:
233 labels = node.get("metadata", {}).get("labels", {})232 labels = node.get("metadata", {}).get("labels", {})
234 accelerator_type = labels.get(C.ACCELERATOR_TYPE)233 accelerator_type = labels.get(C.ACCELERATOR_TYPE)
235 if accelerator_type:234 if accelerator_type:
236- logger.info(235+ accelerator_types.add(accelerator_type)
237- "Resolved %s=%s from node %s",236+ return accelerator_types
238- C.ACCELERATOR_TYPE,
239- accelerator_type,
240- node.get("metadata", {}).get("name", "<unknown>"),
241- )
242- _g_accelerator_type_cache[C.ACCELERATOR_TYPE] = accelerator_type
243- return accelerator_type
244 237 
245- raise RuntimeError(f"No node in cluster has label {C.ACCELERATOR_TYPE}")238+ 
239+def _matches_hardware_generation(accelerator_type, hardware_type):
240+ if hardware_type in C.HARDWARE_TYPE_A2:
241+ return "910b" in accelerator_type.lower()
242+ if hardware_type in C.HARDWARE_TYPE_A3:
243+ return "a3" in accelerator_type.lower()
244+ return True
245+ 
246+ 
247+def _resolve_accelerator_type_from_nodes(nodes, hardware_type):
248+ accelerator_types = _collect_node_accelerator_types(nodes)
249+ if not accelerator_types:
250+ raise RuntimeError(f"Matched nodes for hardware_type={hardware_type} do not have label {C.ACCELERATOR_TYPE}")
251+ 
252+ matched_types = {value for value in accelerator_types if _matches_hardware_generation(value, hardware_type)}
253+ if not matched_types:
254+ raise RuntimeError(
255+ f"No {C.ACCELERATOR_TYPE} on cluster matches hardware_type={hardware_type}. "
256+ f"Found values: {sorted(accelerator_types)}"
257+ )
258+ if len(matched_types) == 1:
259+ return next(iter(matched_types))
260+ 
261+ raise RuntimeError(
262+ f"Multiple {C.ACCELERATOR_TYPE} values match hardware_type={hardware_type}: {sorted(matched_types)}"
263+ )
264+ 
265+ 
266+def get_accelerator_type_from_cluster(hardware_type):
267+ """Resolve accelerator-type node label value from cluster nodes via kubectl."""
268+ if hardware_type in _g_accelerator_type_cache:
269+ return _g_accelerator_type_cache[hardware_type]
270+ 
271+ if hardware_type in C.HARDWARE_TYPE_950I_A5:
272+ label_selector = f"{C.ACCELERATOR}={C.ACCELERATOR_A5},{C.ACCELERATOR_TYPE}={hardware_type}"
273+ nodes = _get_cluster_nodes(label_selector)
274+ if not nodes:
275+ raise RuntimeError(f"No node in cluster matches {label_selector} for hardware_type={hardware_type}")
276+ accelerator_type = hardware_type
277+ elif hardware_type in C.HARDWARE_TYPE_A2 or hardware_type in C.HARDWARE_TYPE_A3:
278+ nodes = _get_cluster_nodes(f"{C.ACCELERATOR}={C.ACCELERATOR_910}")
279+ if not nodes:
280+ raise RuntimeError(
281+ f"No node in cluster with label {C.ACCELERATOR}={C.ACCELERATOR_910} for hardware_type={hardware_type}"
282+ )
283+ accelerator_type = _resolve_accelerator_type_from_nodes(nodes, hardware_type)
284+ else:
285+ known = [*sorted(C.HARDWARE_TYPE_A2), *sorted(C.HARDWARE_TYPE_A3), *C.HARDWARE_TYPE_950I_A5]
286+ raise ValueError(f"Unknown hardware_type '{hardware_type}'. Supported values: {known}")
287+ 
288+ logger.info(
289+ "Resolved %s=%s from cluster for hardware_type=%s",
290+ C.ACCELERATOR_TYPE,
291+ accelerator_type,
292+ hardware_type,
293+ )
294+ _g_accelerator_type_cache[hardware_type] = accelerator_type
295+ return accelerator_type
246 296 
247 297 
248def get_baseline_config_from_configmap(job_id):298def get_baseline_config_from_configmap(job_id):
Mtests/examples/deployer/conftest.py+11-1
@@ -1,3 +1,13 @@
1+# Copyright (c) Huawei Technologies Co., Ltd. 2026. All rights reserved.
2+# MindIE is licensed under Mulan PSL v2.
3+# You can use this software according to the terms and conditions of the Mulan PSL v2.
4+# You may obtain a copy of Mulan PSL v2 at:
5+# http://license.coscl.org.cn/MulanPSL2
6+# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND,
7+# EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT,
8+# MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE.
9+# See the Mulan PSL v2 for more details.
10+ 
1import sys11import sys
2import types12import types
3from pathlib import Path13from pathlib import Path
@@ -37,7 +47,7 @@ def mock_accelerator_type_from_cluster(request, monkeypatch):
37 monkeypatch.setattr(47 monkeypatch.setattr(
38 k8s_utils,48 k8s_utils,
39 "get_accelerator_type_from_cluster",49 "get_accelerator_type_from_cluster",
40- lambda: C.ACCELERATOR_TYPE_A3,50+ lambda _hardware_type: C.ACCELERATOR_TYPE_A3,
41 )51 )
42 yield52 yield
43 k8s_utils._g_accelerator_type_cache.clear()53 k8s_utils._g_accelerator_type_cache.clear()
Mtests/examples/deployer/test_accelerator_type.py+80-34
@@ -1,3 +1,13 @@
1+# Copyright (c) Huawei Technologies Co., Ltd. 2026. All rights reserved.
2+# MindIE is licensed under Mulan PSL v2.
3+# You can use this software according to the terms and conditions of the Mulan PSL v2.
4+# You may obtain a copy of Mulan PSL v2 at:
5+# http://license.coscl.org.cn/MulanPSL2
6+# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND,
7+# EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT,
8+# MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE.
9+# See the Mulan PSL v2 for more details.
10+ 
1import json11import json
2 12 
3import pytest13import pytest
@@ -11,55 +21,67 @@ def mock_kubectl_path(monkeypatch):
11 monkeypatch.setattr(k8s_utils, "_get_kubectl_path", lambda: "kubectl")21 monkeypatch.setattr(k8s_utils, "_get_kubectl_path", lambda: "kubectl")
12 22 
13 23 
14-def test_get_accelerator_type_from_cluster_returns_first_node_label(monkeypatch):24+def _nodes_json(*label_maps):
15- k8s_utils._g_accelerator_type_cache.clear()25+ return {
16- nodes_json = {
17 "items": [26 "items": [
18- {"metadata": {"name": "node-0", "labels": {"host-arch": "huawei-arm"}}},
19 {27 {
20 "metadata": {28 "metadata": {
21- "name": "node-1",29+ "name": f"node-{index}",
22- "labels": {C.ACCELERATOR_TYPE: C.ACCELERATOR_TYPE_A3},30+ "labels": labels,
23 }31 }
24- },32+ }
25- {33+ for index, labels in enumerate(label_maps)
26- "metadata": {
27- "name": "node-2",
28- "labels": {C.ACCELERATOR_TYPE: C.ACCELERATOR_TYPE_910B},
29- }
30- },
31 ]34 ]
32 }35 }
33- monkeypatch.setattr(k8s_utils, "run_cmd_get_output", lambda _args: json.dumps(nodes_json))
34- 
35- assert k8s_utils.get_accelerator_type_from_cluster() == C.ACCELERATOR_TYPE_A3
36 36 
37 37 
38-def test_get_accelerator_type_from_cluster_uses_accelerator_type_cache_key(monkeypatch):38+def test_get_accelerator_type_from_cluster_returns_matching_a3_label(monkeypatch):
39+ k8s_utils._g_accelerator_type_cache.clear()
40+ monkeypatch.setattr(
41+ k8s_utils,
42+ "run_cmd_get_output",
43+ lambda _args: json.dumps(
44+ _nodes_json(
45+ {"host-arch": "huawei-arm"},
46+ {C.ACCELERATOR_TYPE: C.ACCELERATOR_TYPE_A3},
47+ {C.ACCELERATOR_TYPE: C.ACCELERATOR_TYPE_910B},
48+ )
49+ ),
50+ )
51+ 
52+ assert k8s_utils.get_accelerator_type_from_cluster(C.HARDWARE_TYPE_800I_A3) == C.ACCELERATOR_TYPE_A3
53+ 
54+ 
55+def test_get_accelerator_type_from_cluster_returns_matching_a2_label(monkeypatch):
56+ k8s_utils._g_accelerator_type_cache.clear()
57+ monkeypatch.setattr(
58+ k8s_utils,
59+ "run_cmd_get_output",
60+ lambda _args: json.dumps(
61+ _nodes_json(
62+ {C.ACCELERATOR_TYPE: C.ACCELERATOR_TYPE_A3},
63+ {C.ACCELERATOR_TYPE: C.ACCELERATOR_TYPE_910B},
64+ )
65+ ),
66+ )
67+ 
68+ assert k8s_utils.get_accelerator_type_from_cluster(C.HARDWARE_TYPE_800I_A2) == C.ACCELERATOR_TYPE_910B
69+ 
70+ 
71+def test_get_accelerator_type_from_cluster_uses_hardware_type_cache_key(monkeypatch):
39 k8s_utils._g_accelerator_type_cache.clear()72 k8s_utils._g_accelerator_type_cache.clear()
40 call_count = {"n": 0}73 call_count = {"n": 0}
41 74 
42 def fake_run(_args):75 def fake_run(_args):
43 call_count["n"] += 176 call_count["n"] += 1
44- return json.dumps(77+ return json.dumps(_nodes_json({C.ACCELERATOR_TYPE: C.ACCELERATOR_TYPE_910B}))
45- {
46- "items": [
47- {
48- "metadata": {
49- "name": "node-0",
50- "labels": {C.ACCELERATOR_TYPE: C.ACCELERATOR_TYPE_910B},
51- }
52- }
53- ]
54- }
55- )
56 78 
57 monkeypatch.setattr(k8s_utils, "run_cmd_get_output", fake_run)79 monkeypatch.setattr(k8s_utils, "run_cmd_get_output", fake_run)
58 80 
59- assert k8s_utils.get_accelerator_type_from_cluster() == C.ACCELERATOR_TYPE_910B81+ assert k8s_utils.get_accelerator_type_from_cluster(C.HARDWARE_TYPE_800I_A2) == C.ACCELERATOR_TYPE_910B
60- assert k8s_utils.get_accelerator_type_from_cluster() == C.ACCELERATOR_TYPE_910B82+ assert k8s_utils.get_accelerator_type_from_cluster(C.HARDWARE_TYPE_800I_A2) == C.ACCELERATOR_TYPE_910B
61 assert call_count["n"] == 183 assert call_count["n"] == 1
62- assert k8s_utils._g_accelerator_type_cache[C.ACCELERATOR_TYPE] == C.ACCELERATOR_TYPE_910B84+ assert k8s_utils._g_accelerator_type_cache[C.HARDWARE_TYPE_800I_A2] == C.ACCELERATOR_TYPE_910B
63 85 
64 86 
65def test_get_accelerator_type_from_cluster_raises_when_label_missing(monkeypatch):87def test_get_accelerator_type_from_cluster_raises_when_label_missing(monkeypatch):
@@ -67,8 +89,32 @@ def test_get_accelerator_type_from_cluster_raises_when_label_missing(monkeypatch
67 monkeypatch.setattr(89 monkeypatch.setattr(
68 k8s_utils,90 k8s_utils,
69 "run_cmd_get_output",91 "run_cmd_get_output",
70- lambda _args: json.dumps({"items": [{"metadata": {"name": "node-0", "labels": {}}}]}),92+ lambda _args: json.dumps(_nodes_json({})),
71 )93 )
72 94 
73 with pytest.raises(RuntimeError, match=C.ACCELERATOR_TYPE):95 with pytest.raises(RuntimeError, match=C.ACCELERATOR_TYPE):
74- k8s_utils.get_accelerator_type_from_cluster()96+ k8s_utils.get_accelerator_type_from_cluster(C.HARDWARE_TYPE_800I_A3)
97+ 
98+ 
99+def test_get_accelerator_type_from_cluster_raises_when_no_matching_generation(monkeypatch):
100+ k8s_utils._g_accelerator_type_cache.clear()
101+ monkeypatch.setattr(
102+ k8s_utils,
103+ "run_cmd_get_output",
104+ lambda _args: json.dumps(_nodes_json({C.ACCELERATOR_TYPE: C.ACCELERATOR_TYPE_910B})),
105+ )
106+ 
107+ with pytest.raises(RuntimeError, match=C.HARDWARE_TYPE_800I_A3):
108+ k8s_utils.get_accelerator_type_from_cluster(C.HARDWARE_TYPE_800I_A3)
109+ 
110+ 
111+def test_get_accelerator_type_from_cluster_a5_uses_hardware_type(monkeypatch):
112+ k8s_utils._g_accelerator_type_cache.clear()
113+ a5_type = C.HARDWARE_TYPE_950I_A5[0]
114+ monkeypatch.setattr(
115+ k8s_utils,
116+ "run_cmd_get_output",
117+ lambda _args: json.dumps(_nodes_json({C.ACCELERATOR_TYPE: a5_type})),
118+ )
119+ 
120+ assert k8s_utils.get_accelerator_type_from_cluster(a5_type) == a5_type
Mtests/examples/deployer/test_pd_hybrid.py+1-1
@@ -134,7 +134,7 @@ def test_generate_yaml_engine_creates_hybrid_workload(tmp_path, monkeypatch):
134 monkeypatch.setattr(134 monkeypatch.setattr(
135 k8s_utils,135 k8s_utils,
136 "get_accelerator_type_from_cluster",136 "get_accelerator_type_from_cluster",
137- lambda: cluster_accelerator_type,137+ lambda _hardware_type: cluster_accelerator_type,
138 )138 )
139 139 
140 user_config = make_pd_hybrid_user_config()140 user_config = make_pd_hybrid_user_config()