已合并
chore: add VS Code devcontainer configuration. #2911
chore: add VS Code devcontainer configuration. #2911
已合并
xubin创建于 7月31日
10 个文件变更+380-158
@@ -0,0 +1,65 @@
1+# MindSpeed-MM Dev Container
2+ 
3+在 VS Code 中一键启动 MindSpeed-MM 开发环境。
yaoyaoxu
yaoyaoxuyaoyaoxu8月1日

建议补充说明这个功能的背景与利弊

likedislike
xubin
8月1日 评论:
4+ 
5+## 工作模式
6+ 
7+[devcontainer.json](./devcontainer.json) 默认基于预先构建好的镜像(`image` 字段),拉取即用,无需本地构建。
8+ 
9+> 配置文件中填写的 `image` 仅为示例,请根据实际需求选择所需镜像。镜像获取方式与更多信息请查看 [docker/OVERVIEW.zh.md](../docker/OVERVIEW.zh.md)。
10+ 
11+如需自行修改构建参数,请在 `devcontainer.json` 中:
12+ 
13+1. 注释掉 `image`
14+2. 取消注释 `build` 块与 `initializeCommand`
15+ 
16+即可切换为从 [../docker/Dockerfile](../docker/Dockerfile) 本地构建的模式。
17+ 
18+## 前置要求
19+ 
20+- [VS Code](https://code.visualstudio.com/) + [Dev Containers](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-containers) 扩展
21+- 本地安装 Docker
22+- 宿主机已安装昇腾 NPU 驱动,用于容器内访问 NPU
23+ 
24+## 使用方式
25+ 
26+1. 用 VS Code 打开本项目根目录
yaoyaoxu
yaoyaoxuyaoyaoxu8月1日

是不是需要先ssh到服务器?

likedislike
xubin
8月1日 评论:
27+2.`F1` 打开命令面板,执行 **Dev Containers: Reopen in Container**
28+3. 首次启动会拉取预构建镜像(默认模式)或本地构建镜像(自建模式,约 20-40 分钟,取决于网络),完成后进入开发环境
29+ 
30+> 自建模式下,`initializeCommand` 会调用 [prepare-context.sh](./prepare-context.sh),自动下载 Miniconda 并准备 Dockerfile 所需的构建上下文文件,无需手动操作。
31+ 
32+## 构建参数调整(自建模式)
33+ 
34+编辑 [devcontainer.json](./devcontainer.json) 的 `build.args` 即可调整构建参数:
35+如果参数只是一个示例,请根据实际需求填写。
36+ 
37+| 参数 | 默认值 | 说明 |
38+| --- | --- | --- |
39+| `NPU_TYPE` | `a3` | NPU 型号:`910b``a3` |
40+| `OS` | `openeuler24.03` | 操作系统:`openeuler24.03``ubuntu22.04` |
41+| `BASE_IMAGE_VERSION` | `9.0.0` | CANN 基础镜像版本 |
42+| `TORCH_VERSION` | `2.7.1` | PyTorch 版本(在线安装) |
43+| `TORCH_NPU_VERSION` | `2.7.1.post8` | torch_npu 版本(在线安装) |
44+| `MINDSPEED_MM_BRANCH` | `26.1.0` | MindSpeed-MM 分支 |
45+ 
46+## NPU 设备配置
47+ 
48+默认挂载 `/dev/davinci0``/dev/davinci1`。如需使用其他卡号,修改 `devcontainer.json``runArgs``--device=/dev/davinci2` 索引,或新增 `--device=/dev/davinci2` 等。
49+ 
50+宿主机无 NPU 时,注释掉 `runArgs``mounts` 即可作为纯代码编辑环境使用。
51+ 
52+## 工作区说明
53+ 
54+- 本地源码通过 bind mount 挂载到容器 `/workspace/MindSpeed-MM`,编辑实时生效
55+- 容器创建后自动执行 `pip install -e . --no-deps` 以可编辑模式安装项目,并配置 pre-commit
56+- 默认 Python 解释器:`/opt/conda/bin/python`(conda base 环境)
57+ 
58+## 常见问题排查
59+ 
60+- **Miniconda 下载超时(自建模式)**
61+ 手动下载 [Miniconda3-py311_26.1.1-1-Linux-${ARCH}.sh](https://repo.anaconda.com/miniconda/) 放到 `docker/downloads/` 目录后重新构建
62+ 
63+- **decord 依赖下载失败(仅 ARM 架构需要 & 自建模式)**
64+ decord 编译依赖较多且常因网络问题下载失败,可用浏览器手动下载后放入 `docker/decord_deps/` 目录,再重新执行 **Reopen in Container**。下载链接参考 [docker/common/download_decord_deps.sh](../docker/common/download_decord_deps.sh)。
65+ 完成后 `docker/decord_deps/` 目录结构应包含上述全部文件与目录。x86_64 架构无需下载这些依赖(decord 在镜像构建阶段一已通过 pip 安装)。
@@ -0,0 +1,85 @@
1+{
2+ "name": "MindSpeed-MM Dev",
3+ "image": "swr.cn-south-1.myhuaweicloud.com/ascendhub/mindspeed-mm:26.0.0-a3-openeuler24.03-py3.11-aarch64",
4+ // // ----- build 模式(从 Dockerfile 构建时取消注释,注释上方 image 行) -----
5+ // "build": {
6+ // "dockerfile": "../docker/Dockerfile",
7+ // "context": "../docker",
8+ // // 构建时使用 host 网络,使构建容器可通过 127.0.0.1:6268 访问宿主机代理
9+ // "options": ["--network=host"],
10+ // "args": {
11+ // // PyTorch / torch_npu 版本(在线安装)
12+ // "TORCH_VERSION": "2.7.1",
13+ // "TORCH_NPU_VERSION": "2.7.1.post8",
14+ // // MindSpeed-MM 分支(dev 镜像会克隆该分支;工作区挂载会覆盖克隆内容)
15+ // "MINDSPEED_MM_BRANCH": "26.1.0",
16+ // // 构建开发镜像(非 CI 多版本镜像)
17+ // "BUILD_CI": "false",
18+ // // CANN 基础镜像版本
19+ // "BASE_IMAGE": "swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:9.0.0-a3-openeuler24.03-py3.11",
20+ // // 以下参数对应 prepare-context.sh 暂存到上下文中的文件/目录名
21+ // "MINICONDA_SH": "miniconda.sh",
22+ // "DECORD_SCRIPT": "install_decord_on_arm.sh",
23+ // "DECORD_DEPS_DIR": "decord_deps",
24+ // "TORCH_WHL_DIR": "torch_wheels"
25+ // }
26+ // },
27+ // // build 模式专用:构建前在宿主机执行,下载 Miniconda、暂存辅助脚本
28+ // "initializeCommand": "bash .devcontainer/prepare-context.sh",
29+ // 将本地源码挂载到镜像默认工作目录,覆盖构建时克隆的仓库,实现实时编辑
30+ "workspaceMount": "source=${localWorkspaceFolder},target=/workspace/MindSpeed-MM,type=bind,consistency=cached",
31+ "workspaceFolder": "/workspace/MindSpeed-MM",
32+ "remoteUser": "root",
33+ // 昇腾 NPU 设备访问。请按宿主机实际卡号调整 /dev/davinci0 索引。
34+ // 若宿主机无 NPU,注释掉 runArgs 与 mounts 即可作为纯代码编辑环境使用。
35+ "runArgs": [
36+ "--network=host",
37+ "--device=/dev/davinci0",
38+ "--device=/dev/davinci1",
39+ "--device=/dev/davinci_manager",
40+ "--device=/dev/devmm_svm",
41+ "--device=/dev/hisi_hdc",
42+ "--ipc=host"
43+ ],
44+ "mounts": [
45+ "source=/usr/local/dcmi,target=/usr/local/dcmi,type=bind",
46+ "source=/usr/local/bin/npu-smi,target=/usr/local/bin/npu-smi,type=bind",
47+ "source=/usr/local/Ascend/driver/lib64,target=/usr/local/Ascend/driver/lib64,type=bind",
48+ "source=/usr/local/Ascend/driver/version.info,target=/usr/local/Ascend/driver/version.info,type=bind",
49+ "source=/etc/ascend_install.info,target=/etc/ascend_install.info,type=bind"
50+ ],
51+ // 容器创建后:以可编辑模式安装项目(依赖已在镜像中预装),并配置 pre-commit
52+ "postCreateCommand": "pip install -e . --no-deps && pip install pre-commit && pre-commit install",
53+ "customizations": {
54+ "vscode": {
55+ "extensions": [
56+ // Python 核心
57+ "ms-python.python",
58+ "ms-python.vscode-pylance",
59+ "ms-python.debugpy",
60+ // Ruff(linting + formatting)
61+ "charliermarsh.ruff",
62+ // 通用
63+ "eamodio.gitlens",
64+ "yzhang.markdown-all-in-one",
65+ "tamasfe.even-better-toml",
66+ "redhat.vscode-yaml"
67+ ],
68+ "settings": {
69+ "python.defaultInterpreterPath": "/opt/conda/bin/python",
70+ "python.terminal.activateEnvironment": true,
71+ "python.analysis.typeCheckingMode": "basic",
72+ "python.analysis.autoImportCompletions": true,
73+ "editor.formatOnSave": true,
74+ "editor.rulers": [120],
75+ "files.trimTrailingWhitespace": true,
76+ "files.insertFinalNewline": true,
77+ "yaml.schemas": {},
78+ "search.exclude": {
79+ "**/kernel_meta_*": true,
80+ "**/.cache": true
81+ }
82+ }
83+ }
84+ }
85+}
@@ -0,0 +1,88 @@
1+#!/bin/bash
2+set -e
3+ 
4+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
5+PROJECT_ROOT="$(cd "${SCRIPT_DIR}/.." && pwd)"
6+DOCKER_DIR="${PROJECT_ROOT}/docker"
7+COMMON_DIR="${DOCKER_DIR}/common"
8+MODEL_INSTALL_DIR="${DOCKER_DIR}/scripts/model_install"
9+ 
10+# 可通过环境变量覆盖默认值
11+OS="${OS:-openeuler24.03}"
12+ 
13+# 根据 OS 推导 OS_FAMILY 与软件源配置脚本
14+case "$OS" in
15+ openeuler*) OS_FAMILY="openeuler"; REPO_SCRIPT="configure_yum_repo.sh" ;;
16+ ubuntu*) OS_FAMILY="ubuntu"; REPO_SCRIPT="configure_apt_repo.sh" ;;
17+ *) echo "ERROR: 不支持的操作系统: $OS(可选: openeuler24.03 / ubuntu22.04)"; exit 1 ;;
18+esac
19+ 
20+# 检测 CPU 架构
21+ARCH="$(uname -m)"
22+if [ "$ARCH" != "x86_64" ] && [ "$ARCH" != "aarch64" ]; then
23+ echo "ERROR: 不支持的 CPU 架构: $ARCH(仅支持 x86_64 / aarch64)"
24+ exit 1
25+fi
26+ 
27+echo "=========================================="
28+echo "准备 devcontainer 构建上下文"
29+echo " OS: ${OS}"
30+echo " OS_FAMILY: ${OS_FAMILY}"
31+echo " 架构: ${ARCH}"
32+echo " 上下文目录: ${DOCKER_DIR}"
33+echo "=========================================="
34+ 
35+cd "${DOCKER_DIR}"
36+ 
37+# 1. 下载并暂存 Miniconda 安装器(统一命名为 miniconda.sh,便于构建参数固定)
38+MINICONDA_FILE="Miniconda3-py311_26.1.1-1-Linux-${ARCH}.sh"
39+if [ ! -f "miniconda.sh" ]; then
40+ if [ -f "downloads/${MINICONDA_FILE}" ]; then
41+ echo ">>> 复用已下载的 Miniconda: downloads/${MINICONDA_FILE}"
42+ else
43+ echo ">>> 下载 Miniconda (${ARCH})..."
44+ bash "${COMMON_DIR}/download_miniconda.sh" "${DOCKER_DIR}/downloads" "${ARCH}"
45+ fi
46+ cp "downloads/${MINICONDA_FILE}" "miniconda.sh"
47+else
48+ echo ">>> Miniconda 已存在: miniconda.sh"
49+fi
50+ 
51+# 2. 暂存 common_functions.sh
52+cp "${COMMON_DIR}/common_functions.sh" "common_functions.sh"
53+ 
54+# 3. 暂存软件源配置脚本为 configure_repo.sh(Dockerfile 固定 COPY 该名称)
55+cp "${COMMON_DIR}/${REPO_SCRIPT}" "configure_repo.sh"
56+ 
57+# 4. 暂存模型环境安装脚本到 install_scripts/
58+mkdir -p "install_scripts"
59+for script in "${MODEL_INSTALL_DIR}"/install_*.sh; do
60+ [ -f "$script" ] && cp "$script" "install_scripts/"
61+done
62+ 
63+# 5. 暂存 decord 安装脚本(ARM 构建需要;x86 构建会被跳过,但文件必须存在)
64+cp "${COMMON_DIR}/install_decord_on_arm.sh" "install_decord_on_arm.sh"
65+ 
66+# 6. 创建 torch_wheels 占位目录(空目录 = 在线安装 PyTorch)
67+mkdir -p "torch_wheels"
68+touch "torch_wheels/.placeholder"
69+ 
70+# 7. 创建 decord_deps 目录
71+# - x86_64: decord 已在阶段一通过 pip 安装,仅创建占位目录满足 COPY 路径
72+# - aarch64: 下载 decord 源码编译依赖
73+if [ "$ARCH" = "aarch64" ]; then
74+ if [ ! -d "decord_deps" ] || [ -z "$(ls -A decord_deps 2>/dev/null)" ]; then
75+ echo ">>> 下载 decord 编译依赖(ARM)..."
76+ rm -rf "decord_deps"
77+ bash "${COMMON_DIR}/download_decord_deps.sh" "${DOCKER_DIR}/decord_deps"
78+ else
79+ echo ">>> decord 依赖已存在: decord_deps"
80+ fi
81+else
82+ mkdir -p "decord_deps"
83+ touch "decord_deps/.placeholder"
84+fi
85+ 
86+echo "=========================================="
87+echo "构建上下文准备完成"
88+echo "=========================================="
@@ -158,4 +158,15 @@ mindspeed
158/tests/st/run_logs/158/tests/st/run_logs/
159 159 
160fusion_result.json160fusion_result.json
161-cache_dir/161+cache_dir/
162+ 
163+# devcontainer: Build context artifacts staged to docker/ via .devcontainer/prepare-context.sh
164+/docker/miniconda.sh
165+/docker/Miniconda3-*.sh
166+/docker/common_functions.sh
167+/docker/configure_repo.sh
168+/docker/install_decord_on_arm.sh
169+/docker/install_scripts/
170+/docker/torch_wheels/
171+/docker/decord_deps/
172+/docker/downloads/
@@ -19,6 +19,7 @@ repos:
19 - id: check-merge-conflict19 - id: check-merge-conflict
20 - id: detect-private-key20 - id: detect-private-key
21 - id: check-json21 - id: check-json
22+ exclude: \.devcontainer/devcontainer\.json$
22 23 
23 # -------------------------- Python 核心检查 --------------------------24 # -------------------------- Python 核心检查 --------------------------
24 # codespell25 # codespell
@@ -17,20 +17,15 @@
17# then produce the final image.17# then produce the final image.
18# ============================================18# ============================================
19 19 
20-# --- Base image arguments ---20+# BASE_IMAGE: Full base image URL; when empty, falls back to the hardcoded default below.
21-# OS: Operating system type, e.g. openeuler24.03 or ubuntu22.0421+# NOTE: Do not use nested ${VAR} inside ${BASE_IMAGE:-default} — Docker's :- parser
22-ARG OS=openeuler24.0322+# stops at the first '}' and produces a malformed image name.
23-# BASE_IMAGE_VERSION: CANN base image version tag
24-ARG BASE_IMAGE_VERSION=9.0.0-beta.2
25-# NPU_TYPE: Ascend NPU model (910b / a3 / etc.)
26-ARG NPU_TYPE=910b
27-# BASE_IMAGE: Full base image URL; the default splicing rule is used when empty
28ARG BASE_IMAGE=""23ARG BASE_IMAGE=""
29 24 
30# Stage 1 (base): Build the base runtime environment.25# Stage 1 (base): Build the base runtime environment.
31# Pulls the CANN base image from the Huawei Cloud Ascend registry by default;26# Pulls the CANN base image from the Huawei Cloud Ascend registry by default;
32# comes with CANN and Python 3.11 pre-installed.27# comes with CANN and Python 3.11 pre-installed.
33-FROM ${BASE_IMAGE:-swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:${BASE_IMAGE_VERSION}-${NPU_TYPE}-${OS}-py3.11} AS base28+FROM ${BASE_IMAGE:-swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:9.0.0-a3-openeuler24.03-py3.11} AS base
34 29 
35# Switch to root (the CANN base image defaults to a non-root user)30# Switch to root (the CANN base image defaults to a non-root user)
36USER root31USER root
@@ -40,11 +35,9 @@ USER root
40ARG MINICONDA_SH35ARG MINICONDA_SH
41# TORCH_VERSION / TORCH_NPU_VERSION: PyTorch and torch_npu versions36# TORCH_VERSION / TORCH_NPU_VERSION: PyTorch and torch_npu versions
42ARG TORCH_VERSION=2.7.137ARG TORCH_VERSION=2.7.1
43-ARG TORCH_NPU_VERSION=2.7.138+ARG TORCH_NPU_VERSION=2.7.1.post8
44# TORCH_WHL_DIR: Directory containing local PyTorch .whl files (optional, for offline install)39# TORCH_WHL_DIR: Directory containing local PyTorch .whl files (optional, for offline install)
45ARG TORCH_WHL_DIR=torch_wheels40ARG TORCH_WHL_DIR=torch_wheels
46-# OS_FAMILY: Operating system family (openeuler / ubuntu), used to select the package manager
47-ARG OS_FAMILY
48 41 
49# Detect CPU architecture and print info (x86_64 / aarch64)42# Detect CPU architecture and print info (x86_64 / aarch64)
50RUN ARCH=$(uname -m) && \43RUN ARCH=$(uname -m) && \
@@ -60,18 +53,6 @@ RUN ARCH=$(uname -m) && \
60 fi && \53 fi && \
61 echo "=========================================="54 echo "=========================================="
62 55 
63-# Configure DNS resolution (ensures the container can access the public internet).
64-# Uses: domestic DNS (114) -> Google DNS (8.8.8.8) -> Alibaba DNS (223.5.5.5)
65-RUN if [ -f /etc/resolv.conf ]; then \
66- echo "Existing resolv.conf:" && cat /etc/resolv.conf; \
67- else \
68- echo "No resolv.conf found, creating one..."; \
69- fi && \
70- echo "nameserver 114.114.114.114" > /etc/resolv.conf && \
71- echo "nameserver 8.8.8.8" >> /etc/resolv.conf && \
72- echo "nameserver 223.5.5.5" >> /etc/resolv.conf && \
73- echo "Updated resolv.conf:" && cat /etc/resolv.conf
74- 
75# Configure system package repositories (via external script for yum/apt sources)56# Configure system package repositories (via external script for yum/apt sources)
76COPY configure_repo.sh /tmp/configure_repo.sh57COPY configure_repo.sh /tmp/configure_repo.sh
77RUN chmod +x /tmp/configure_repo.sh && \58RUN chmod +x /tmp/configure_repo.sh && \
@@ -82,8 +63,16 @@ RUN chmod +x /tmp/configure_repo.sh && \
82# - Build toolchain: gcc/g++, make, cmake (for compiling C++ extensions like decord)63# - Build toolchain: gcc/g++, make, cmake (for compiling C++ extensions like decord)
83# - Network tools: git, wget, curl64# - Network tools: git, wget, curl
84# - Media processing: ffmpeg and related dev libraries (for video decoding)65# - Media processing: ffmpeg and related dev libraries (for video decoding)
85-RUN echo "Installing system dependencies..." && \66+# OS family is detected from /etc/os-release (ID field), so no build arg is needed.
86- if [ "$OS_FAMILY" = "openeuler" ]; then \67+RUN . /etc/os-release && \
68+ DETECTED_OS_FAMILY="" && \
69+ ID_LOWER=$(echo "$ID" | tr '[:upper:]' '[:lower:]') && \
70+ case "$ID_LOWER" in \
71+ *openeuler*) DETECTED_OS_FAMILY="openeuler" ;; \
72+ *ubuntu*) DETECTED_OS_FAMILY="ubuntu" ;; \
73+ esac && \
74+ echo "Installing system dependencies (ID=$ID, OS_FAMILY=$DETECTED_OS_FAMILY)..." && \
75+ if [ "$DETECTED_OS_FAMILY" = "openeuler" ]; then \
87 yum install -y \76 yum install -y \
88 git \77 git \
89 iproute \78 iproute \
@@ -96,7 +85,7 @@ RUN echo "Installing system dependencies..." && \
96 cmake \85 cmake \
97 ffmpeg-devel \86 ffmpeg-devel \
98 && yum clean all; \87 && yum clean all; \
99- elif [ "$OS_FAMILY" = "ubuntu" ]; then \88+ elif [ "$DETECTED_OS_FAMILY" = "ubuntu" ]; then \
100 apt-get install -y \89 apt-get install -y \
101 git \90 git \
102 iproute2 \91 iproute2 \
@@ -114,8 +103,7 @@ RUN echo "Installing system dependencies..." && \
114 libswscale-dev \103 libswscale-dev \
115 && apt-get clean; \104 && apt-get clean; \
116 else \105 else \
117- echo "ERROR: Unsupported OS: $OS_FAMILY"; \106+ echo "ERROR: Unsupported OS ID: $ID" && exit 1; \
118- exit 1; \
119 fi107 fi
120 108 
121# Install Miniconda (for creating and managing Python virtual environments).109# Install Miniconda (for creating and managing Python virtual environments).
@@ -66,7 +66,7 @@ Taking `v26.1.0-cann9.0.0-torch_npu2.7.1-a3-openeuler24.03-py3.11-aarch64` as an
66| Python version | `py3.11` | Python 3.11 |66| Python version | `py3.11` | Python 3.11 |
67| architecture | `aarch64` | ARM64 architecture |67| architecture | `aarch64` | ARM64 architecture |
68 68 
69-> The CANN version and TorchNPU version are derived from the base image and the `--torch-npu-version` parameter at build time.69+> The CANN version, NPU type, OS, and Python version are all derived from the base image tag. The TorchNPU version comes from the `--torch-npu-version` parameter at build time.
70 70 
71Dockerfile naming follows the template: `Dockerfile[.{chip info}.{OS}.{other fields}]`71Dockerfile naming follows the template: `Dockerfile[.{chip info}.{OS}.{other fields}]`
72 72 
@@ -181,12 +181,11 @@ The image includes the following pre-configured environments:
181 181 
182### Build Script Parameter Description182### Build Script Parameter Description
183 183 
184-The build script `build.sh` supports multiple parameter configurations. The following default values are examples only; please adjust according to actual needs. The default value `9.0.0` for `--base-image-version` is an example only. **It is strongly recommended to explicitly specify the base image matching your target environment via `--base-image` or `--base-image-version`.**184+The build script `build.sh` supports multiple parameter configurations. The CANN version, NPU type, OS, and Python version are **only** obtained by parsing the `--base-image` tag — they cannot be specified manually.
185 185 
186-| Parameter | Description | Default (Example) |186+| Parameter | Description | Default |
187| ------ | ------ | ------------ |187| ------ | ------ | ------------ |
188-| `-t, --npu-type` | NPU type: A3 or 910B (auto-detected from `--base-image` if not specified) | None (required) |188+| `--base-image` | **Required.** Full base image name. CANN version, NPU type, OS, and Python version are auto-detected from the image tag | None (required) |
189-| `-o, --os` | Operating system: openeuler24.03 or ubuntu22.04 (auto-detected from `--base-image` if not specified) | openeuler24.03 |
190| `-v, --version` | MindSpeed MM version identifier, also used as Git branch name and script directory selection basis | 26.1.0 |189| `-v, --version` | MindSpeed MM version identifier, also used as Git branch name and script directory selection basis | 26.1.0 |
191| `-m, --miniconda` | Miniconda installer path (auto-downloaded if not specified) | None (auto-download) |190| `-m, --miniconda` | Miniconda installer path (auto-downloaded if not specified) | None (auto-download) |
192| `-d, --decord-deps` | decord dependencies directory path (auto-downloaded for ARM if not specified) | None (auto-download) |191| `-d, --decord-deps` | decord dependencies directory path (auto-downloaded for ARM if not specified) | None (auto-download) |
@@ -200,8 +199,6 @@ The build script `build.sh` supports multiple parameter configurations. The foll
200| `--torch-npu-whl` | torch-npu.whl file path (offline installation) | None |199| `--torch-npu-whl` | torch-npu.whl file path (offline installation) | None |
201| `--torchvision-whl` | torchvision.whl file path (optional, offline installation) | None |200| `--torchvision-whl` | torchvision.whl file path (optional, offline installation) | None |
202| `--torchaudio-whl` | torchaudio.whl file path (optional, offline installation) | None |201| `--torchaudio-whl` | torchaudio.whl file path (optional, offline installation) | None |
203-| `--base-image-version` | Base image CANN version (example value; recommended to specify explicitly) | 9.0.0 |
204-| `--base-image` | Complete base image name (higher priority than `--base-image-version`; recommended) | None |
205| `--build-ci` | Build CI image with multi-version conda environments (skip verl + MindSpeed-MM clone) | None |202| `--build-ci` | Build CI image with multi-version conda environments (skip verl + MindSpeed-MM clone) | None |
206| `--cleanup-on-fail` | Clean up dangling images/containers on build failure | None |203| `--cleanup-on-fail` | Clean up dangling images/containers on build failure | None |
207 204 
@@ -210,28 +207,22 @@ The build script `build.sh` supports multiple parameter configurations. The foll
210```bash207```bash
211cd docker208cd docker
212 209 
213-# Build A3 + openEuler image (default)210+# Build A3 + openEuler image
214-bash build.sh -t A3211+bash build.sh --base-image swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:9.0.0-a3-openeuler24.03-py3.11
215- 
216-# Build A2 + openEuler image
217-bash build.sh -t 910B
218 212 
219# Build A3 + Ubuntu image213# Build A3 + Ubuntu image
220-bash build.sh -t A3 -o ubuntu22.04214+bash build.sh --base-image swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:9.0.0-a3-ubuntu22.04-py3.11
221 215 
222# Build with specified PyTorch version216# Build with specified PyTorch version
223-bash build.sh -t A3 --torch-version 2.7.1 --torch-npu-version 2.7.1217+bash build.sh --base-image swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:9.0.0-a3-openeuler24.03-py3.11 --torch-version 2.7.1 --torch-npu-version 2.7.1.post4
224 218 
225# Build with offline .whl files219# Build with offline .whl files
226-bash build.sh -t A3 \220+bash build.sh --base-image swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:9.0.0-a3-openeuler24.03-py3.11 \
227 --torch-whl /path/to/torch-2.7.1+cpu-cp311-cp311-linux_x86_64.whl \221 --torch-whl /path/to/torch-2.7.1+cpu-cp311-cp311-linux_x86_64.whl \
228 --torch-npu-whl /path/to/torch_npu-2.7.1-cp311-cp311-linux_x86_64.whl222 --torch-npu-whl /path/to/torch_npu-2.7.1-cp311-cp311-linux_x86_64.whl
229 223 
230-# Build with specified base image version
231-bash build.sh -t A3 --base-image-version 9.0.0
232- 
233# Build with specified version224# Build with specified version
234-bash build.sh -t A3 -v 26.1.0225+bash build.sh --base-image swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:9.0.0-a3-openeuler24.03-py3.11 -v 26.1.0
235```226```
236 227 
237### Automatic Download Function Description228### Automatic Download Function Description
@@ -246,12 +237,13 @@ The build script supports automatic download of the following resources. Please
246 237 
247### Automatic Base Image Recognition238### Automatic Base Image Recognition
248 239 
249-The build script automatically recognizes key information from the base image name:240+The build script automatically recognizes key information from the base image tag (the `--base-image` parameter is required):
250 241 
251-1. **NPU type recognition**: Recognizes the NPU type (e.g., `a3`) from the image tag242+1. **CANN version recognition**: Extracts the CANN version number (e.g., `9.0.0`) from the image tag
252-2. **Operating system recognition**: Recognizes `openeuler24.03` or `ubuntu22.04` from the image tag243+2. **NPU type recognition**: Recognizes the NPU type (e.g., `a3`) from the image tag
253-3. **CANN version recognition**: Extracts the CANN version number (e.g., `9.0.0`) from the image tag244+3. **Operating system recognition**: Recognizes `openeuler24.03` or `ubuntu22.04` from the image tag
254-4. **Automatic image tag generation**: Automatically generates image tags conforming to naming rules based on recognized information245+4. **Python version recognition**: Extracts the Python version (e.g., `3.11`) from the `py<x.y>` field in the image tag
246+5. **Automatic image tag generation**: Automatically generates image tags conforming to naming rules based on recognized information
255 247 
256### Best Practice Example248### Best Practice Example
257 249 
@@ -262,7 +254,7 @@ The following example shows how to build a MindSpeed MM image using a custom bas
262SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"254SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
263 255 
264# Custom configuration256# Custom configuration
265-BASE_IMAGE="swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:9.0.0-910b-openeuler24.03-py3.11"257+BASE_IMAGE="swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:9.0.0-a3-openeuler24.03-py3.11"
266TORCH_VERSION="2.7.1"258TORCH_VERSION="2.7.1"
267TORCH_NPU_VERSION="2.7.1.post6"259TORCH_NPU_VERSION="2.7.1.post6"
268MINDSPEED_MM_VERSION="26.0.0"260MINDSPEED_MM_VERSION="26.0.0"
@@ -278,7 +270,7 @@ bash "${SCRIPT_DIR}/build.sh" \
278 270 
279**Key Feature Description:**271**Key Feature Description:**
280 272 
281-1. **Automatic recognition**: The script automatically recognizes the NPU type, operating system (openeuler24.03), and CANN version (9.0.0) from `BASE_IMAGE`. If `BASE_IMAGE` doesn't exist in the system, it will be automatically pulled.273+1. **Automatic recognition**: The script automatically recognizes the CANN version (9.0.0), NPU type (a3), operating system (openeuler24.03), and Python version (3.11) from `BASE_IMAGE`. If `BASE_IMAGE` doesn't exist in the system, it will be automatically pulled.
2822. **Automatic tag generation**: Automatically generates image tags conforming to the naming rules based on recognition results.2742. **Automatic tag generation**: Automatically generates image tags conforming to the naming rules based on recognition results.
2833. **Automatic download**: If the Miniconda installer or decord dependencies are not available locally, the script will automatically download them2753. **Automatic download**: If the Miniconda installer or decord dependencies are not available locally, the script will automatically download them
2844. **Failure cleanup**: The `--cleanup-on-fail` parameter ensures cleanup of dangling resources if the build fails2764. **Failure cleanup**: The `--cleanup-on-fail` parameter ensures cleanup of dangling resources if the build fails
@@ -375,7 +367,7 @@ Build an image containing the new model environment using the following command:
375 367 
376```bash368```bash
377# Use the -v parameter to specify the version number, and the build script will automatically find the corresponding installation script369# Use the -v parameter to specify the version number, and the build script will automatically find the corresponding installation script
378-bash build.sh -t A3 -v 26.1.0370+bash build.sh --base-image swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:9.0.0-a3-openeuler24.03-py3.11 -v 26.1.0
379```371```
380 372 
381#### 5. Update Documentation373#### 5. Update Documentation
@@ -66,7 +66,7 @@ MindSpeed MM 镜像基于 Ubuntu 22.04 和 openEuler 24.03 两种操作系统,
66| Python版本 | `py3.11` | Python 3.11 |66| Python版本 | `py3.11` | Python 3.11 |
67| 架构类型 | `aarch64` | ARM64 架构 |67| 架构类型 | `aarch64` | ARM64 架构 |
68 68 
69-> CANN 版本 TorchNPU 版本来源于构建时基础镜像 `--torch-npu-version` 参数。69+> CANN 版本、NPU 类型、操作系统和 Python 版本来源于构建时基础镜像 tag,TorchNPU 版本来源于 `--torch-npu-version` 参数。
70 70 
71Dockerfile 命名遵循模板:`Dockerfile[.{芯片信息}.{操作系统}.{其他字段}]`71Dockerfile 命名遵循模板:`Dockerfile[.{芯片信息}.{操作系统}.{其他字段}]`
72 72 
@@ -181,12 +181,11 @@ docker run -it --rm \
181 181 
182### 构建脚本参数说明182### 构建脚本参数说明
183 183 
184-构建脚本 `build.sh` 支持多种参数配置,以下默认值仅为示例,请根据实际需求调整。其中 `--base-image-version` 的默认值 `9.0.0` 仅为示例,**强烈建议通过 `--base-image` `--base-image-version` 显式指定与目标环境匹配的基础镜像**184+构建脚本 `build.sh` 支持多种参数配置。其中 CANN 版本、NPU 类型、操作系统和 Python 版本**只能**通过解析 `--base-image` tag 获取,不支持手动指定。
185 185 
186-| 参数 | 说明 | 默认值(示例) |186+| 参数 | 说明 | 默认值 |
187| ------ | ------ | ------------ |187| ------ | ------ | ------------ |
188-| `-t, --npu-type` | NPU 类型:A3 910B(未指定时可`--base-image` 自动识别 | 无(必需) |188+| `--base-image` | **必选。** 完整基础镜像名称,CANN 版本、NPU 类型、操作系统和 Python 版本均镜像 tag 自动识别 | 无(必需) |
189-| `-o, --os` | 操作系统:openeuler24.03 或 ubuntu22.04(未指定时可从 `--base-image` 自动识别) | openeuler24.03 |
190| `-v, --version` | MindSpeed MM 版本标识,同时作为 Git 分支名称和脚本目录选择依据 | 26.1.0 |189| `-v, --version` | MindSpeed MM 版本标识,同时作为 Git 分支名称和脚本目录选择依据 | 26.1.0 |
191| `-m, --miniconda` | Miniconda 安装器路径(未指定时自动下载) | 无(自动下载) |190| `-m, --miniconda` | Miniconda 安装器路径(未指定时自动下载) | 无(自动下载) |
192| `-d, --decord-deps` | decord 依赖目录路径(ARM 架构下未指定时自动下载) | 无(自动下载) |191| `-d, --decord-deps` | decord 依赖目录路径(ARM 架构下未指定时自动下载) | 无(自动下载) |
@@ -200,8 +199,6 @@ docker run -it --rm \
200| `--torch-npu-whl` | torch-npu.whl 文件路径(离线安装) | 无 |199| `--torch-npu-whl` | torch-npu.whl 文件路径(离线安装) | 无 |
201| `--torchvision-whl` | torchvision.whl 文件路径(可选,离线安装) | 无 |200| `--torchvision-whl` | torchvision.whl 文件路径(可选,离线安装) | 无 |
202| `--torchaudio-whl` | torchaudio.whl 文件路径(可选,离线安装) | 无 |201| `--torchaudio-whl` | torchaudio.whl 文件路径(可选,离线安装) | 无 |
203-| `--base-image-version` | 基础镜像 CANN 版本(示例值,建议显式指定) | 9.0.0 |
204-| `--base-image` | 完整基础镜像名称(优先级高于 `--base-image-version`,推荐使用) | 无 |
205| `--build-ci` | 构建 CI 镜像,包含多版本 conda 环境(跳过 verl + MindSpeed-MM 克隆) | 无 |202| `--build-ci` | 构建 CI 镜像,包含多版本 conda 环境(跳过 verl + MindSpeed-MM 克隆) | 无 |
206| `--cleanup-on-fail` | 构建失败时清理悬空镜像/容器 | 无 |203| `--cleanup-on-fail` | 构建失败时清理悬空镜像/容器 | 无 |
207 204 
@@ -210,28 +207,22 @@ docker run -it --rm \
210```bash207```bash
211cd docker208cd docker
212 209 
213-# 构建 A3 + openEuler 镜像(默认)210+# 构建 A3 + openEuler 镜像
214-bash build.sh -t A3211+bash build.sh --base-image swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:9.0.0-a3-openeuler24.03-py3.11
215- 
216-# 构建 A2 + openEuler 镜像
217-bash build.sh -t 910B
218 212 
219# 构建 A3 + Ubuntu 镜像213# 构建 A3 + Ubuntu 镜像
220-bash build.sh -t A3 -o ubuntu22.04214+bash build.sh --base-image swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:9.0.0-a3-ubuntu22.04-py3.11
221 215 
222# 指定 PyTorch 版本构建216# 指定 PyTorch 版本构建
223-bash build.sh -t A3 --torch-version 2.7.1 --torch-npu-version 2.7.1217+bash build.sh --base-image swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:9.0.0-a3-openeuler24.03-py3.11 --torch-version 2.7.1 --torch-npu-version 2.7.1.post4
224 218 
225# 使用离线 .whl 文件构建219# 使用离线 .whl 文件构建
226-bash build.sh -t A3 \220+bash build.sh --base-image swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:9.0.0-a3-openeuler24.03-py3.11 \
227 --torch-whl /path/to/torch-2.7.1+cpu-cp311-cp311-linux_x86_64.whl \221 --torch-whl /path/to/torch-2.7.1+cpu-cp311-cp311-linux_x86_64.whl \
228 --torch-npu-whl /path/to/torch_npu-2.7.1-cp311-cp311-linux_x86_64.whl222 --torch-npu-whl /path/to/torch_npu-2.7.1-cp311-cp311-linux_x86_64.whl
229 223 
230-# 指定基础镜像版本构建
231-bash build.sh -t A3 --base-image-version 9.0.0
232- 
233# 指定版本构建224# 指定版本构建
234-bash build.sh -t A3 -v 26.1.0225+bash build.sh --base-image swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:9.0.0-a3-openeuler24.03-py3.11 -v 26.1.0
235```226```
236 227 
237### 自动下载功能说明228### 自动下载功能说明
@@ -246,12 +237,13 @@ bash build.sh -t A3 -v 26.1.0
246 237 
247### 自动识别基础镜像238### 自动识别基础镜像
248 239 
249-构建脚本会自动识别基础镜像名称关键信息:240+构建脚本会自动基础镜像 tag 识别关键信息(`--base-image` 参数为必选)
250 241 
251-1. **NPU 类型识别**:从镜像 tag 中识别NPU类型(如`a3`)242+1. **CANN 版本识别**:从镜像 tag 中提取 CANN 版本号( `9.0.0`
252-2. **操作系统识别**:从镜像 tag 中识别 `openeuler24.03` `ubuntu22.04`243+2. **NPU 类型识别**:从镜像 tag 中识别 NPU 类型(如 `a3`
253-3. **CANN 版本识别**:从镜像 tag 中提取 CANN 版本号(如 `9.0.0`244+3. **操作系统识别**:从镜像 tag 中识别 `openeuler24.03` `ubuntu22.04`
254-4. **自动生成镜像 tag**:基于识别到的信息自动生成符合命名规则的镜像 tag245+4. **Python 版本识别**:镜像 tag 中的 `py<x.y>` 字段提取 Python 版本(如 `3.11`)
246+5. **自动生成镜像 tag**:基于识别到的信息自动生成符合命名规则的镜像 tag
255 247 
256### 最佳实践示例248### 最佳实践示例
257 249 
@@ -262,7 +254,7 @@ bash build.sh -t A3 -v 26.1.0
262SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"254SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
263 255 
264# 自定义配置256# 自定义配置
265-BASE_IMAGE="swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:9.0.0-910b-openeuler24.03-py3.11"257+BASE_IMAGE="swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:9.0.0-a3-openeuler24.03-py3.11"
266TORCH_VERSION="2.7.1"258TORCH_VERSION="2.7.1"
267TORCH_NPU_VERSION="2.7.1.post6"259TORCH_NPU_VERSION="2.7.1.post6"
268MINDSPEED_MM_VERSION="26.0.0"260MINDSPEED_MM_VERSION="26.0.0"
@@ -278,7 +270,7 @@ bash "${SCRIPT_DIR}/build.sh" \
278 270 
279**关键特性说明:**271**关键特性说明:**
280 272 
281-1. **自动识别**:脚本会自动从 `BASE_IMAGE` 中识别 NPU 类型、操作系统(openeuler24.03)和 CANN 版本(9.0.0)。如果`BASE_IMAGE`在系统中不存在,会自动拉取。273+1. **自动识别**:脚本会自动从 `BASE_IMAGE` 中识别 CANN 版本(9.0.0)、NPU 类型(a3)、操作系统(openeuler24.03)和 Python 版本(3.11)。如果`BASE_IMAGE`在系统中不存在,会自动拉取。
2822. **自动生成 tag**:基于识别结果自动生成符合命名规则的镜像 tag。2742. **自动生成 tag**:基于识别结果自动生成符合命名规则的镜像 tag。
2833. **自动下载**:如果本地没有 Miniconda 安装器或 decord 依赖,脚本会自动下载2753. **自动下载**:如果本地没有 Miniconda 安装器或 decord 依赖,脚本会自动下载
2844. **失败清理**`--cleanup-on-fail` 参数确保构建失败时清理悬空资源2764. **失败清理**`--cleanup-on-fail` 参数确保构建失败时清理悬空资源
@@ -375,7 +367,7 @@ echo "=========================================="
375 367 
376```bash368```bash
377# 使用 -v 参数指定版本号,构建脚本会自动找到对应的安装脚本369# 使用 -v 参数指定版本号,构建脚本会自动找到对应的安装脚本
378-bash build.sh -t A3 -v 26.1.0370+bash build.sh --base-image swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:9.0.0-a3-openeuler24.03-py3.11 -v 26.1.0
379```371```
380 372 
381#### 5. 更新文档373#### 5. 更新文档
@@ -26,7 +26,9 @@ cleanup_dangling() {
26# Dockerfile naming: Dockerfile (unified, supports all NPU types and OS)26# Dockerfile naming: Dockerfile (unified, supports all NPU types and OS)
27# Image tag naming: {version}-cann{cann_version}-torch_npu{torch_npu_version}-{chip}-{os}-py{python_version}-{architecture}27# Image tag naming: {version}-cann{cann_version}-torch_npu{torch_npu_version}-{chip}-{os}-py{python_version}-{architecture}
28# Usage:28# Usage:
29-# bash build.sh -t A3 [-m /path/to/miniconda.sh] [-o ubuntu22.04]29+# bash build.sh --base-image <base-image> [-m /path/to/miniconda.sh]
30+# CANN version, NPU type, OS and Python version are auto-detected from the
31+# base image tag and cannot be specified manually.
30# ============================================32# ============================================
31 33 
32set -e34set -e
@@ -42,7 +44,8 @@ Usage: $0 [OPTIONS]
42Build MindSpeed MM Docker Image44Build MindSpeed MM Docker Image
43 45 
44Required:46Required:
45- -t, --npu-type TYPE NPU type: A3 or 910B (auto-detected from --base-image if not specified)47+ --base-image IMAGE Full base image name (e.g. swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:9.0.0-910b-openeuler24.03-py3.11)
48+ CANN version, NPU type, OS and Python version are auto-detected from the image tag.
46 49 
47Optional:50Optional:
48 -m, --miniconda PATH Miniconda installer path (auto-download if not specified)51 -m, --miniconda PATH Miniconda installer path (auto-download if not specified)
@@ -51,7 +54,6 @@ Optional:
51 -i, --image-name NAME Image name (default: mindspeed-mm:v{version}-cann{cann_ver}-torch_npu{torch_npu_ver}-{chip}-{os}-py{py_ver}-{arch})54 -i, --image-name NAME Image name (default: mindspeed-mm:v{version}-cann{cann_ver}-torch_npu{torch_npu_ver}-{chip}-{os}-py{py_ver}-{arch})
52 --tag TAG Custom image tag (overrides the default tag; keeps repo name 'mindspeed-mm')55 --tag TAG Custom image tag (overrides the default tag; keeps repo name 'mindspeed-mm')
53 -n, --no-cache Build without cache56 -n, --no-cache Build without cache
54- -o, --os OS OS: openeuler24.03 or ubuntu22.04 (auto-detected from --base-image if not specified)
55 -v, --version VERSION MindSpeed MM version (default: 26.1.0, determines model install scripts)57 -v, --version VERSION MindSpeed MM version (default: 26.1.0, determines model install scripts)
56 --torch-version VER PyTorch version (default: 2.7.1, for online install)58 --torch-version VER PyTorch version (default: 2.7.1, for online install)
57 --torch-npu-version VER torch-npu version (default: 2.7.1, for online install)59 --torch-npu-version VER torch-npu version (default: 2.7.1, for online install)
@@ -59,14 +61,11 @@ Optional:
59 --torch-npu-whl PATH torch-npu .whl file path (offline install)61 --torch-npu-whl PATH torch-npu .whl file path (offline install)
60 --torchvision-whl PATH torchvision .whl file path (optional, offline install)62 --torchvision-whl PATH torchvision .whl file path (optional, offline install)
61 --torchaudio-whl PATH torchaudio .whl file path (optional, offline install)63 --torchaudio-whl PATH torchaudio .whl file path (optional, offline install)
62- --base-image-version VER Base image CANN version (default: 9.0.0, example only; recommended to specify via --base-image or --base-image-version)
63- --base-image IMAGE Full base image name (higher priority than --base-image-version; recommended)
64 --build-ci Build CI image with multi-version conda environments (skip verl + MindSpeed-MM clone)64 --build-ci Build CI image with multi-version conda environments (skip verl + MindSpeed-MM clone)
65 --cleanup-on-fail Clean up dangling images/containers if build fails65 --cleanup-on-fail Clean up dangling images/containers if build fails
66 -h, --help Show help66 -h, --help Show help
67 67 
68Dockerfile naming convention: Dockerfile (unified, supports all NPU types and OS)68Dockerfile naming convention: Dockerfile (unified, supports all NPU types and OS)
69- NPU type and OS are passed as build arguments
70 69 
71Image tag naming convention: {version}-cann{cann_version}-torch_npu{torch_npu_version}-{chip}-{os}-py{python_version}-{architecture}[-ci]70Image tag naming convention: {version}-cann{cann_version}-torch_npu{torch_npu_version}-{chip}-{os}-py{python_version}-{architecture}[-ci]
72 e.g. v26.0.0-cann9.0.0-torch_npu2.7.1-a3-openeuler24.03-py3.11-x86_6471 e.g. v26.0.0-cann9.0.0-torch_npu2.7.1-a3-openeuler24.03-py3.11-x86_64
@@ -74,51 +73,39 @@ Image tag naming convention: {version}-cann{cann_version}-torch_npu{torch_npu_ve
74 v26.1.0-cann9.0.0-torch_npu2.7.1-910b-openeuler24.03-py3.11-aarch64-ci (CI build via --build-ci)73 v26.1.0-cann9.0.0-torch_npu2.7.1-910b-openeuler24.03-py3.11-aarch64-ci (CI build via --build-ci)
75 74 
76Examples:75Examples:
77- bash $0 -t A376+ bash $0 --base-image swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:9.0.0-a3-openeuler24.03-py3.11
78- bash $0 -t A3 -o ubuntu22.0477+ bash $0 --base-image swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:9.0.0-910b-openeuler24.03-py3.11 --torch-version 2.7.1 --torch-npu-version 2.7.1
79- bash $0 --base-image swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:9.0.0-beta.2-910b-openeuler24.03-py3.1178+ bash $0 --base-image swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:9.0.0-a3-openeuler24.03-py3.11 -i myproject/mindspeed-mm:v1.0
80- bash $0 -t 910B --torch-version 2.7.1 --torch-npu-version 2.7.179+ bash $0 --base-image swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:9.0.0-a3-openeuler24.03-py3.11 --torch-whl /path/to/torch.whl --torch-npu-whl /path/to/torch_npu.whl
81- bash $0 -t A3 --base-image-version 9.0.080+ bash $0 --base-image swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:9.0.0-910b-openeuler24.03-py3.11 --build-ci # Build CI image (tag auto-appends '-ci')
82- bash $0 -t A3 -i myproject/mindspeed-mm:v1.0
83- bash $0 -t A3 --torch-whl /path/to/torch.whl --torch-npu-whl /path/to/torch_npu.whl
84- bash $0 -t 910B --build-ci # Build CI image (tag auto-appends '-ci')
85EOF81EOF
86}82}
87 83 
88-NPU_TYPE=""
89MINICONDA_PATH=""84MINICONDA_PATH=""
90DECORD_DEPS_PATH=""85DECORD_DEPS_PATH=""
91DECORD_SCRIPT_PATH=""86DECORD_SCRIPT_PATH=""
92IMAGE_NAME=""87IMAGE_NAME=""
93IMAGE_TAG=""88IMAGE_TAG=""
94NO_CACHE=""89NO_CACHE=""
95-OS="openeuler24.03"
96TORCH_VERSION="2.7.1"90TORCH_VERSION="2.7.1"
97TORCH_NPU_VERSION="2.7.1"91TORCH_NPU_VERSION="2.7.1"
98TORCH_WHL_PATH=""92TORCH_WHL_PATH=""
99TORCH_NPU_WHL_PATH=""93TORCH_NPU_WHL_PATH=""
100TORCHVISION_WHL_PATH=""94TORCHVISION_WHL_PATH=""
101TORCHAUDIO_WHL_PATH=""95TORCHAUDIO_WHL_PATH=""
102-# Default value is an example only. It is recommended to explicitly specify the96+BASE_IMAGE=""
103-# base image via --base-image or --base-image-version to match your target environment.
104-BASE_IMAGE_VERSION="9.0.0"
105MINDSPEED_MM_VERSION="26.1.0"97MINDSPEED_MM_VERSION="26.1.0"
106-PYTHON_VERSION="3.11"
107-NPU_TYPE_EXPLICIT=false
108-OS_EXPLICIT=false
109CLEANUP_ON_FAIL=false98CLEANUP_ON_FAIL=false
110BUILD_CI=false99BUILD_CI=false
111 100 
112while [[ $# -gt 0 ]]; do101while [[ $# -gt 0 ]]; do
113 case $1 in102 case $1 in
114- -t|--npu-type) NPU_TYPE="$2"; NPU_TYPE_EXPLICIT=true; shift 2 ;;
115 -m|--miniconda) MINICONDA_PATH="$2"; shift 2 ;;103 -m|--miniconda) MINICONDA_PATH="$2"; shift 2 ;;
116 -d|--decord-deps) DECORD_DEPS_PATH="$2"; shift 2 ;;104 -d|--decord-deps) DECORD_DEPS_PATH="$2"; shift 2 ;;
117 -s|--decord-script) DECORD_SCRIPT_PATH="$2"; shift 2 ;;105 -s|--decord-script) DECORD_SCRIPT_PATH="$2"; shift 2 ;;
118 -i|--image-name) IMAGE_NAME="$2"; shift 2 ;;106 -i|--image-name) IMAGE_NAME="$2"; shift 2 ;;
119 --tag) IMAGE_TAG="$2"; shift 2 ;;107 --tag) IMAGE_TAG="$2"; shift 2 ;;
120 -n|--no-cache) NO_CACHE="--no-cache"; shift ;;108 -n|--no-cache) NO_CACHE="--no-cache"; shift ;;
121- -o|--os) OS="$2"; OS_EXPLICIT=true; shift 2 ;;
122 -v|--version) MINDSPEED_MM_VERSION="$2"; shift 2 ;;109 -v|--version) MINDSPEED_MM_VERSION="$2"; shift 2 ;;
123 --torch-version) TORCH_VERSION="$2"; shift 2 ;;110 --torch-version) TORCH_VERSION="$2"; shift 2 ;;
124 --torch-npu-version) TORCH_NPU_VERSION="$2"; shift 2 ;;111 --torch-npu-version) TORCH_NPU_VERSION="$2"; shift 2 ;;
@@ -126,7 +113,6 @@ while [[ $# -gt 0 ]]; do
126 --torch-npu-whl) TORCH_NPU_WHL_PATH="$2"; shift 2 ;;113 --torch-npu-whl) TORCH_NPU_WHL_PATH="$2"; shift 2 ;;
127 --torchvision-whl) TORCHVISION_WHL_PATH="$2"; shift 2 ;;114 --torchvision-whl) TORCHVISION_WHL_PATH="$2"; shift 2 ;;
128 --torchaudio-whl) TORCHAUDIO_WHL_PATH="$2"; shift 2 ;;115 --torchaudio-whl) TORCHAUDIO_WHL_PATH="$2"; shift 2 ;;
129- --base-image-version) BASE_IMAGE_VERSION="$2"; shift 2 ;;
130 --base-image) BASE_IMAGE="$2"; shift 2 ;;116 --base-image) BASE_IMAGE="$2"; shift 2 ;;
131 --build-ci) BUILD_CI=true; shift ;;117 --build-ci) BUILD_CI=true; shift ;;
132 --cleanup-on-fail) CLEANUP_ON_FAIL=true; shift ;;118 --cleanup-on-fail) CLEANUP_ON_FAIL=true; shift ;;
@@ -172,6 +158,12 @@ parse_base_image_tag() {
172 detected_cann="${tag_lower%%-*}"158 detected_cann="${tag_lower%%-*}"
173 fi159 fi
174 160 
161+ # Python version: match 'py<x.y>' field in the tag (e.g. py3.11 -> 3.11)
162+ local detected_python=""
163+ if [[ "$tag_lower" =~ py([0-9]+\.[0-9]+) ]]; then
164+ detected_python="${BASH_REMATCH[1]}"
165+ fi
166+ 
175 if [ -n "$detected_npu" ]; then167 if [ -n "$detected_npu" ]; then
176 DETECTED_NPU_TYPE="$detected_npu"168 DETECTED_NPU_TYPE="$detected_npu"
177 echo ">>> Auto-detected NPU type from base image: $detected_npu"169 echo ">>> Auto-detected NPU type from base image: $detected_npu"
@@ -187,34 +179,57 @@ parse_base_image_tag() {
187 echo ">>> Auto-detected OS from base image: $detected_os"179 echo ">>> Auto-detected OS from base image: $detected_os"
188 fi180 fi
189 181 
182+ if [ -n "$detected_python" ]; then
183+ DETECTED_PYTHON_VERSION="$detected_python"
184+ echo ">>> Auto-detected Python version from base image: $detected_python"
185+ fi
186+ 
190 return 0187 return 0
191}188}
192 189 
193DETECTED_NPU_TYPE=""190DETECTED_NPU_TYPE=""
194DETECTED_OS=""191DETECTED_OS=""
195DETECTED_CANN_VERSION=""192DETECTED_CANN_VERSION=""
193+DETECTED_PYTHON_VERSION=""
196 194 
197-if [ -n "$BASE_IMAGE" ]; then195+if [ -z "$BASE_IMAGE" ]; then
198- echo ">>> Auto-detecting NPU type, OS, and CANN version from base image..."196+ echo "Error: --base-image is required. CANN version, NPU type, OS and Python version are auto-detected from the base image tag."
199- parse_base_image_tag "$BASE_IMAGE"
200- 
201- if [ "$NPU_TYPE_EXPLICIT" = false ] && [ -n "$DETECTED_NPU_TYPE" ]; then
202- NPU_TYPE="$DETECTED_NPU_TYPE"
203- echo ">>> Using auto-detected NPU type: $NPU_TYPE"
204- fi
205- 
206- if [ "$OS_EXPLICIT" = false ] && [ -n "$DETECTED_OS" ]; then
207- OS="$DETECTED_OS"
208- echo ">>> Using auto-detected OS: $OS"
209- fi
210-fi
211- 
212-if [ -z "$NPU_TYPE" ]; then
213- echo "Error: NPU type is required (-t or --npu-type) or provide --base-image for auto-detection"
214 show_help197 show_help
215 exit 1198 exit 1
216fi199fi
217 200 
201+echo ">>> Auto-detecting NPU type, OS, CANN version and Python version from base image..."
202+parse_base_image_tag "$BASE_IMAGE"
203+ 
204+if [ -z "$DETECTED_NPU_TYPE" ]; then
205+ echo "Error: Failed to detect NPU type from base image tag (expected '910b' or 'a3')."
206+ exit 1
207+fi
208+NPU_TYPE="$DETECTED_NPU_TYPE"
209+ 
210+if [ -z "$DETECTED_OS" ]; then
211+ echo "Error: Failed to detect OS from base image tag (expected 'openeuler24.03' or 'ubuntu22.04')."
212+ exit 1
213+fi
214+OS="$DETECTED_OS"
215+ 
216+if [ -z "$DETECTED_CANN_VERSION" ]; then
217+ echo "Error: Failed to detect CANN version from base image tag."
218+ exit 1
219+fi
220+CANN_VERSION="$DETECTED_CANN_VERSION"
221+ 
222+if [ -z "$DETECTED_PYTHON_VERSION" ]; then
223+ echo "Error: Failed to detect Python version from base image tag (expected 'py<x.y>' field, e.g. py3.11)."
224+ exit 1
225+fi
226+PYTHON_VERSION="$DETECTED_PYTHON_VERSION"
227+ 
228+echo ">>> Using detected NPU type: $NPU_TYPE"
229+echo ">>> Using detected OS: $OS"
230+echo ">>> Using detected CANN version: $CANN_VERSION"
231+echo ">>> Using detected Python: $PYTHON_VERSION"
232+ 
218NPU_TYPE=$(echo "$NPU_TYPE" | tr '[:lower:]' '[:upper:]')233NPU_TYPE=$(echo "$NPU_TYPE" | tr '[:lower:]' '[:upper:]')
219NPU_TYPE_LOWER=$(echo "$NPU_TYPE" | tr '[:upper:]' '[:lower:]')234NPU_TYPE_LOWER=$(echo "$NPU_TYPE" | tr '[:upper:]' '[:lower:]')
220OS=$(echo "$OS" | tr '[:upper:]' '[:lower:]')235OS=$(echo "$OS" | tr '[:upper:]' '[:lower:]')
@@ -319,15 +334,6 @@ if [[ "$MINICONDA_NAME" == *"aarch64"* ]]; then
319 ARCH_NAME="aarch64"334 ARCH_NAME="aarch64"
320fi335fi
321 336 
322-# Determine CANN version for the image tag:
323-# - From the base image tag (auto-detected) when --base-image is provided
324-# - From --base-image-version otherwise
325-if [ -n "$BASE_IMAGE" ] && [ -n "$DETECTED_CANN_VERSION" ]; then
326- CANN_VERSION="$DETECTED_CANN_VERSION"
327-else
328- CANN_VERSION="$BASE_IMAGE_VERSION"
329-fi
330- 
331if [ -n "$IMAGE_NAME" ] && [ -n "$IMAGE_TAG" ]; then337if [ -n "$IMAGE_NAME" ] && [ -n "$IMAGE_TAG" ]; then
332 echo "Warning: both --image-name and --tag provided; --image-name takes precedence (--tag ignored)"338 echo "Warning: both --image-name and --tag provided; --image-name takes precedence (--tag ignored)"
333fi339fi
@@ -404,14 +410,15 @@ fi
404echo "=========================================="410echo "=========================================="
405echo "Build Configuration"411echo "Build Configuration"
406echo "=========================================="412echo "=========================================="
413+echo "Base Image: ${BASE_IMAGE}"
407echo "NPU Type: ${NPU_TYPE}"414echo "NPU Type: ${NPU_TYPE}"
408echo "OS: ${OS}"415echo "OS: ${OS}"
416+echo "CANN Version: ${CANN_VERSION}"
417+echo "Python Version: ${PYTHON_VERSION}"
409echo "CPU Architecture: ${ARCH_NAME}"418echo "CPU Architecture: ${ARCH_NAME}"
410echo "Dockerfile: ${DOCKERFILE}"419echo "Dockerfile: ${DOCKERFILE}"
411echo "Image Name: ${IMAGE_NAME}"420echo "Image Name: ${IMAGE_NAME}"
412echo "Image Tag: ${IMAGE_TAG:-(auto)}"421echo "Image Tag: ${IMAGE_TAG:-(auto)}"
413-echo "Base Image Version: ${BASE_IMAGE_VERSION}"
414-echo "CANN Version: ${CANN_VERSION}"
415echo "PyTorch Version: ${TORCH_VERSION}"422echo "PyTorch Version: ${TORCH_VERSION}"
416echo "torch-npu Version: ${TORCH_NPU_VERSION}"423echo "torch-npu Version: ${TORCH_NPU_VERSION}"
417echo "MindSpeed MM Ver: ${MINDSPEED_MM_VERSION}"424echo "MindSpeed MM Ver: ${MINDSPEED_MM_VERSION}"
@@ -434,21 +441,19 @@ echo "Build CI: ${BUILD_CI}"
434echo "No Cache: ${NO_CACHE:-No}"441echo "No Cache: ${NO_CACHE:-No}"
435echo "=========================================="442echo "=========================================="
436 443 
437-if [ -n "$BASE_IMAGE" ]; then444+echo ""
438- echo ""445+echo ">>> Checking if base image exists..."
439- echo ">>> Checking if base image exists..."446+if ! docker image inspect "$BASE_IMAGE" > /dev/null 2>&1; then
440- if ! docker image inspect "$BASE_IMAGE" > /dev/null 2>&1; then447+ echo ">>> Base image not found, pulling: ${BASE_IMAGE}"
441- echo ">>> Base image not found, pulling: ${BASE_IMAGE}"448+ docker pull "$BASE_IMAGE"
442- docker pull "$BASE_IMAGE"449+ if [ $? -ne 0 ]; then
443- if [ $? -ne 0 ]; then450+ echo "Error: Failed to pull base image"
444- echo "Error: Failed to pull base image"451+ exit 1
445- exit 1
446- fi
447- else
448- echo ">>> Base image already exists: ${BASE_IMAGE}"
449 fi452 fi
450- echo ""453+else
454+ echo ">>> Base image already exists: ${BASE_IMAGE}"
451fi455fi
456+echo ""
452 457 
453cd "$SCRIPT_DIR"458cd "$SCRIPT_DIR"
454 459 
@@ -506,9 +511,7 @@ fi
506 511 
507BUILD_ARGS="--build-arg MINICONDA_SH=${MINICONDA_NAME}"512BUILD_ARGS="--build-arg MINICONDA_SH=${MINICONDA_NAME}"
508BUILD_ARGS="$BUILD_ARGS --build-arg DECORD_SCRIPT=${DECORD_SCRIPT_NAME}"513BUILD_ARGS="$BUILD_ARGS --build-arg DECORD_SCRIPT=${DECORD_SCRIPT_NAME}"
509-BUILD_ARGS="$BUILD_ARGS --build-arg OS=${OS}"
510BUILD_ARGS="$BUILD_ARGS --build-arg OS_FAMILY=${OS_FAMILY}"514BUILD_ARGS="$BUILD_ARGS --build-arg OS_FAMILY=${OS_FAMILY}"
511-BUILD_ARGS="$BUILD_ARGS --build-arg NPU_TYPE=${NPU_TYPE_LOWER}"
512BUILD_ARGS="$BUILD_ARGS --build-arg TORCH_VERSION=${TORCH_VERSION}"515BUILD_ARGS="$BUILD_ARGS --build-arg TORCH_VERSION=${TORCH_VERSION}"
513BUILD_ARGS="$BUILD_ARGS --build-arg TORCH_NPU_VERSION=${TORCH_NPU_VERSION}"516BUILD_ARGS="$BUILD_ARGS --build-arg TORCH_NPU_VERSION=${TORCH_NPU_VERSION}"
514BUILD_ARGS="$BUILD_ARGS --build-arg TORCH_WHL_DIR=torch_wheels"517BUILD_ARGS="$BUILD_ARGS --build-arg TORCH_WHL_DIR=torch_wheels"
@@ -519,11 +522,8 @@ if [ "$BUILD_CI" = true ]; then
519 BUILD_ARGS="$BUILD_ARGS --build-arg FINAL_WORKDIR=/workspace"522 BUILD_ARGS="$BUILD_ARGS --build-arg FINAL_WORKDIR=/workspace"
520fi523fi
521 524 
522-if [ -n "$BASE_IMAGE" ]; then525+# Always pass the full BASE_IMAGE to the Dockerfile (required, auto-detected values come from its tag).
523- BUILD_ARGS="$BUILD_ARGS --build-arg BASE_IMAGE=${BASE_IMAGE}"526+BUILD_ARGS="$BUILD_ARGS --build-arg BASE_IMAGE=${BASE_IMAGE}"
524-else
525- BUILD_ARGS="$BUILD_ARGS --build-arg BASE_IMAGE_VERSION=${BASE_IMAGE_VERSION}"
526-fi
527 527 
528BUILD_ARGS="$BUILD_ARGS --build-arg DECORD_DEPS_DIR=${DECORD_DEPS_NAME}"528BUILD_ARGS="$BUILD_ARGS --build-arg DECORD_DEPS_DIR=${DECORD_DEPS_NAME}"
529 529 
@@ -89,4 +89,4 @@ echo ""
89echo "Next steps:"89echo "Next steps:"
90echo " 1. Verify the file: bash ${MINICONDA_FILE} --help"90echo " 1. Verify the file: bash ${MINICONDA_FILE} --help"
91echo " 2. Use it to build Docker image:"91echo " 2. Use it to build Docker image:"
92-echo " bash build.sh -t A3 -m ${OUTPUT_DIR}/${MINICONDA_FILE}"92+echo " bash build.sh --base-image swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:9.0.0-a3-openeuler24.03-py3.11 -m ${OUTPUT_DIR}/${MINICONDA_FILE}"