已合并
基于 devcontainer 的 MindStudio 统一开发环境方案 #392
Zhang-Yu001创建于 20 天前
基于 devcontainer 的 MindStudio 统一开发环境方案 #392
已合并
Zhang-Yu001创建于 20 天前
13 个文件变更+613-15
A.clangd+2-0
@@ -0,0 +1,2 @@
1+CompileFlags:
2+ CompilationDatabase: build/
A.devcontainer/devcontainer.json+57-0
@@ -0,0 +1,57 @@
1+{
2+ "name": "msprof-devcontainer",
3+ "image": "swr.cn-north-4.myhuaweicloud.com/mindstudio-image/mindstudio-build:26.1.0-0701",
4+ "workspaceFolder": "/workspace",
5+ "workspaceMount": "source=${localWorkspaceFolder},target=/workspace,type=bind",
6+ "remoteUser": "mindstudio",
7+ "updateRemoteUserUID": true,
8+ "_privileged_comment": "privileged and seccomp=unconfined are required for Ascend NPU hardware access, only for internal dev environments",
9+ "runArgs": [
10+ "--network=host",
11+ "--privileged",
Mrtutu
MrtutuMrtutu19 天前

严重程度: 建议

问题: runArgs 中同时启用了 --privileged--security-opt=seccomp=unconfined,容器拥有宿主机全部特权能力。

原因: 这是 devcontainer 配置中权限提升最大的组合,一旦容器被攻破或镜像被篡改,等于宿主机失守;对多人共享的服务器尤其危险。文档与配置中未说明为何必须使用 privileged(若是为 NPU/设备访问,通常可以收窄)。

怎么改: 在 devcontainer.json 中注释说明 privileged 的必要性;如仅为设备访问,可改用具体的 --device 映射与所需 capabilities(如 --cap-add=SYS_ADMIN),避免无差别特权提升。

likedislike
12+ "--ipc=host",
13+ "--ulimit=nproc=65535:65535",
14+ "--security-opt=seccomp=unconfined"
atomgit-bot
atomgit-botatomgit-bot20 天前

🟡 Medium Priority

changed line: .devcontainer/devcontainer.json L10、L13(新增 "--privileged""--security-opt=seccomp=unconfined")→ affected behavior: 容器以特权模式运行,seccomp 过滤被禁用,结合已有的 --network=host(L9)和 --ipc=host(L11),容器与宿主机之间几乎无安全边界→ failure mode: 容器内任何进程(包括构建脚本、第三方依赖编译产物)均拥有完整 root 能力和系统调用权限,可访问宿主机网络、进程间通信、所有设备。若构建过程中执行的第三方代码或下载的依赖包含恶意行为,可直接危害宿主机。→ suggested fix: 评估是否确实需要 --privileged(通常用于需要访问硬件设备或特定内核模块的场景)。如仅为性能分析工具(msprof)需要 perf 等能力,可考虑使用更细粒度的 --cap-add(如 --cap-add=SYS_PTRACE--cap-add=PERFMON)替代 --privilegedseccomp=unconfined 也应在确认必要性后再开启,或使用自定义 seccomp profile。

建议:评估是否可用细粒度 --cap-add=SYS_PTRACE 等替代 --privileged--security-opt=seccomp=unconfined 如非必要应移除,或使用自定义 seccomp profile。

likedislike
15+ ],
16+ "mounts": [
17+ "source=/usr/local/sbin,target=/usr/local/sbin,type=bind,ro"
18+ ],
19+ "containerEnv": {
20+ "GCC11_NO_RPATH": "1",
21+ "NPM_CONFIG_PREFIX": "/home/mindstudio/.local"
22+ },
23+ "remoteEnv": {
24+ "PATH": "/home/mindstudio/.local/bin:${containerEnv:PATH}"
25+ },
26+ "postCreateCommand": "bash /workspace/.devcontainer/post-create.sh",
27+ "customizations": {
28+ "vscode": {
29+ "extensions": [
30+ "ms-vscode.cpptools",
31+ "ms-vscode.cmake-tools",
32+ "ms-python.python",
33+ "ms-python.vscode-pylance",
34+ "charliermarsh.ruff",
35+ "llvm-vs-code-extensions.vscode-clangd",
36+ "llvm-vs-code-extensions.vscode-clang-format",
37+ "bierner.markdown-preview-github-styles"
38+ ],
39+ "settings": {
40+ "terminal.integrated.profiles.linux": {
41+ "bash": {
42+ "path": "bash",
43+ "args": ["-l"]
44+ }
45+ },
46+ "terminal.integrated.defaultProfile.linux": "bash",
47+ "C_Cpp.intelliSenseEngine": "disabled",
48+ "clangd.serverArgs": [
49+ "--query-driver=*",
50+ "--header-insertion=iwyu",
51+ "--completion-style=detailed"
52+ ],
53+ "clangd.pathToCompdb": "build"
54+ }
55+ }
56+ }
57+}
A.devcontainer/post-create.sh+303-0
@@ -0,0 +1,303 @@
1+#!/bin/bash
2+# -*- coding: utf-8 -*-
3+# -------------------------------------------------------------------------
4+# This file is part of the MindStudio project.
5+# Copyright (c) 2026 Huawei Technologies Co.,Ltd.
6+#
7+# MindStudio is licensed under Mulan PSL v2.
8+# You can use this software according to the terms and conditions of the Mulan PSL v2.
9+# You may obtain a copy of Mulan PSL v2 at:
10+#
11+# http://license.coscl.org.cn/MulanPSL2
12+#
13+# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND,
14+# EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT,
15+# MERCHANTABILITY OR FIT FOR A PARTICULAR PURPOSE.
16+# See the Mulan PSL v2 for more details.
17+# -------------------------------------------------------------------------
18+# -------------------------------------------------------------------------
19+# 容器首次创建后的幂等初始化脚本 (msprof)
20+# 所有动作必须幂等,失败不阻塞容器创建
21+# 不使用 set -e,每个步骤独立处理错误
22+# -------------------------------------------------------------------------
23+ 
24+log() { echo "[post-create] $*"; }
25+warn() { echo "[post-create] WARN: $*"; }
26+ 
27+# ──────────────────────────────────────────────────────────────────────────────
28+# 1. 用户级命令目录
29+# ──────────────────────────────────────────────────────────────────────────────
30+configure_user_bin() {
31+ log "Configuring user bin directory..."
32+ mkdir -p "$HOME/.local/bin"
33+ npm config set prefix "$HOME/.local" 2>/dev/null || warn "npm config set prefix failed"
34+ 
35+ local marker="# msprof-devcontainer-user-bin"
36+ for rcfile in "$HOME/.bashrc" "$HOME/.bash_profile"; do
37+ if [ -f "$rcfile" ] && ! grep -qF "$marker" "$rcfile"; then
38+ cat >> "$rcfile" <<EOF
39+$marker
40+export PATH="\$HOME/.local/bin:\$PATH"
41+EOF
42+ log "Appended PATH to $rcfile"
43+ fi
44+ done
45+}
46+ 
47+# ──────────────────────────────────────────────────────────────────────────────
48+# 2. Python 3 (最低要求 3.8)
49+# ──────────────────────────────────────────────────────────────────────────────
50+configure_python3() {
51+ log "Configuring Python 3..."
52+ if [ -f "/etc/profile.d/pyenv.sh" ]; then
53+ source /etc/profile.d/pyenv.sh 2>/dev/null || warn "Failed to source pyenv.sh"
54+ log "Loaded pyenv profile"
55+ fi
56+ 
57+ # 如果存在 pyenv 管理的 Python,将其优先于系统 python3
58+ for candidate in /opt/python/cp*-cp*; do
59+ if [ -d "$candidate/bin" ] && [ -x "$candidate/bin/python3" ]; then
60+ local pyenv_python="$candidate/bin"
61+ log "Python (pyenv) found at $pyenv_python"
62+ 
63+ local marker="# msprof-pyenv-python"
64+ for rcfile in "$HOME/.bashrc" "$HOME/.bash_profile"; do
65+ if [ -f "$rcfile" ] && ! grep -qF "$marker" "$rcfile"; then
66+ cat >> "$rcfile" <<EOF
67+$marker
68+export PATH="$pyenv_python:\$PATH"
69+EOF
70+ log "Prepended pyenv Python to PATH in $rcfile"
71+ fi
72+ done
73+ export PATH="$pyenv_python:$PATH"
74+ break
75+ fi
76+ done
77+ 
78+ if command -v python3 &>/dev/null; then
79+ log "Python 3 found: $(python3 --version 2>&1)"
80+ else
81+ warn "python3 not found in PATH"
82+ fi
83+}
84+ 
85+# ──────────────────────────────────────────────────────────────────────────────
86+# 3. 安装编译和测试依赖 (幂等)
87+# ──────────────────────────────────────────────────────────────────────────────
88+install_build_deps() {
89+ log "Installing system build dependencies..."
90+ 
91+ # --- 系统包 (dnf) ---
92+ if command -v dnf &>/dev/null; then
93+ local sys_pkgs=(
94+ python3-devel
95+ )
96+ for pkg in "${sys_pkgs[@]}"; do
97+ if ! rpm -q "$pkg" &>/dev/null; then
98+ log " Installing: $pkg"
99+ sudo dnf install -y "$pkg" 2>/dev/null || warn "dnf install failed: $pkg"
100+ else
101+ log " System pkg OK: $pkg"
102+ fi
103+ done
104+ elif command -v apt-get &>/dev/null; then
105+ local sys_pkgs=(
106+ python3-dev
107+ )
108+ for pkg in "${sys_pkgs[@]}"; do
109+ if ! dpkg -s "$pkg" &>/dev/null; then
110+ log " Installing: $pkg"
111+ sudo apt-get update -qq && sudo apt-get install -y "$pkg" 2>/dev/null || warn "apt-get install failed: $pkg"
112+ else
113+ log " System pkg OK: $pkg"
114+ fi
115+ done
116+ fi
117+ 
118+ # gitleaks 用于 pre-commit 密钥扫描,根据架构自动选择下载
119+ if ! command -v gitleaks &>/dev/null; then
120+ log " Installing gitleaks..."
121+ GITLEAKS_VER="8.18.4"
122+ case "$(uname -m)" in
123+ x86_64) GITLEAKS_ARCH="amd64";;
124+ aarch64) GITLEAKS_ARCH="arm64";;
125+ *) warn "Unsupported architecture $(uname -m), skipping gitleaks"; return;;
126+ esac
127+ GITLEAKS_INSTALLED=false
128+ for MIRROR in \
129+ "https://ghproxy.com/https://github.com/gitleaks/gitleaks/releases/download/v${GITLEAKS_VER}/gitleaks_${GITLEAKS_VER}_linux_${GITLEAKS_ARCH}.tar.gz" \
130+ "https://mirror.ghproxy.com/https://github.com/gitleaks/gitleaks/releases/download/v${GITLEAKS_VER}/gitleaks_${GITLEAKS_VER}_linux_${GITLEAKS_ARCH}.tar.gz" \
131+ "https://github.com/gitleaks/gitleaks/releases/download/v${GITLEAKS_VER}/gitleaks_${GITLEAKS_VER}_linux_${GITLEAKS_ARCH}.tar.gz"; do
132+ log " Trying: ${MIRROR}"
133+ curl -fsSL "${MIRROR}" -o /tmp/gitleaks.tar.gz --connect-timeout 10 2>/dev/null && \
134+ sudo tar -xzf /tmp/gitleaks.tar.gz -C /usr/local/bin gitleaks 2>/dev/null && \
135+ GITLEAKS_INSTALLED=true && break
136+ rm -f /tmp/gitleaks.tar.gz
137+ done
138+ if ${GITLEAKS_INSTALLED}; then
139+ sudo chmod +x /usr/local/bin/gitleaks
140+ log " gitleaks ${GITLEAKS_VER} (${GITLEAKS_ARCH}) installed"
141+ else
142+ warn "gitleaks install failed (all mirrors unreachable)"
143+ fi
144+ else
145+ log " System pkg OK: gitleaks"
146+ fi
147+ 
148+ # --- pip 包 ---
149+ local pip_pkgs=(
150+ packaging
151+ wheel
152+ pytest
153+ coverage
154+ pre-commit
155+ "bandit[toml]"
156+ )
157+ 
158+ for PY in $(command -v python3 2>/dev/null) $(command -v python 2>/dev/null); do
159+ log "Installing pip packages for: $($PY --version 2>&1)"
160+ "$PY" -m pip install --quiet --upgrade pip setuptools >/dev/null 2>&1 || warn "pip/setuptools upgrade failed for $PY"
161+ 
162+ for pkg in "${pip_pkgs[@]}"; do
163+ # pip show 不解析 extras(如 bandit[toml]),用裸包名检查
164+ local pkg_name="${pkg%%[*}"
165+ if ! "$PY" -m pip show "$pkg_name" &>/dev/null; then
166+ log " Installing for $PY: $pkg"
167+ "$PY" -m pip install "$pkg" || warn "pip install failed for $PY: $pkg"
168+ else
169+ log " Pip pkg OK ($PY): $pkg"
170+ fi
171+ done
172+ done
173+ 
174+ log "Build dependencies check complete"
175+}
176+ 
177+# ──────────────────────────────────────────────────────────────────────────────
178+# 4. Git 身份同步
179+# ──────────────────────────────────────────────────────────────────────────────
180+sync_git_identity() {
181+ log "Syncing Git identity..."
182+ local gitconfig="$HOME/.devcontainer-host-gitconfig"
183+ if [ -f "$gitconfig" ] && [ -s "$gitconfig" ]; then
184+ local name email
185+ name=$(git config --file "$gitconfig" --get user.name 2>/dev/null) || true
186+ email=$(git config --file "$gitconfig" --get user.email 2>/dev/null) || true
187+ [ -n "$name" ] && git config --global user.name "$name"
188+ [ -n "$email" ] && git config --global user.email "$email"
189+ log "Git identity synced from host"
190+ else
191+ warn "No host Git config found, skipping identity sync"
192+ fi
193+}
194+ 
195+# ──────────────────────────────────────────────────────────────────────────────
196+# 5. 开发命令提示
197+# ──────────────────────────────────────────────────────────────────────────────
198+append_dev_hint_once() {
199+ local marker="# msprof-dev-hint"
200+ local rcfile="$HOME/.bashrc"
201+ if grep -qF "$marker" "$rcfile" 2>/dev/null; then return; fi
202+ 
203+ cat >> "$rcfile" <<EOF
204+$marker
205+# msprof development commands:
206+# python3 build.py Build Release (full)
207+# python3 build.py local Build Release (skip deps)
208+# python3 build.py test Build and run unit tests
209+# python3 build.py test local Run unit tests (skip deps)
210+EOF
211+ log "Appended dev hints to $rcfile"
212+}
213+ 
214+# ──────────────────────────────────────────────────────────────────────────────
215+# 6. pre-commit 自动安装
216+# ──────────────────────────────────────────────────────────────────────────────
217+install_pre_commit_hook() {
218+ log "Installing pre-commit hook..."
219+ export PATH="$HOME/.local/bin:$PATH"
220+ 
221+ local pre_commit_cmd=""
222+ if command -v pre-commit &>/dev/null; then
223+ pre_commit_cmd="pre-commit"
224+ elif python3 -m pre_commit --version &>/dev/null 2>&1; then
225+ pre_commit_cmd="python3 -m pre_commit"
226+ elif python -m pre_commit --version &>/dev/null 2>&1; then
227+ pre_commit_cmd="python -m pre_commit"
228+ else
229+ warn "pre-commit not found, skipping hook installation"
230+ return
231+ fi
232+ 
233+ if ! git rev-parse --git-dir &>/dev/null; then
234+ warn "Not a Git repository, skipping pre-commit hook"
235+ return
236+ fi
237+ 
238+ $pre_commit_cmd install || warn "pre-commit install failed"
239+ log "pre-commit hook installed via: $pre_commit_cmd"
240+}
241+ 
242+# ──────────────────────────────────────────────────────────────────────────────
243+# 7. clangd
244+# ──────────────────────────────────────────────────────────────────────────────
245+setup_clangd() {
246+ log "Setting up clangd..."
247+ if command -v clangd &>/dev/null; then
248+ log "clangd found: $(clangd --version 2>&1 | head -1)"
249+ return
250+ fi
251+ warn "clangd not found, attempting to install..."
252+ if command -v dnf &>/dev/null; then
253+ sudo dnf install -y clangd 2>/dev/null || warn "Failed to install clangd via dnf"
254+ elif command -v apt-get &>/dev/null; then
255+ sudo apt-get update -qq && sudo apt-get install -y clangd 2>/dev/null || warn "Failed to install clangd via apt-get"
256+ else
257+ warn "Cannot install clangd automatically"
258+ fi
259+}
260+ 
261+# ──────────────────────────────────────────────────────────────────────────────
262+# 8. 忽略本地可修改的文件
263+# ──────────────────────────────────────────────────────────────────────────────
264+ignore_local_changes() {
265+ log "Setting up skip-worktree for local-modifiable files..."
266+ if [ -f ".vscode/settings.json" ]; then
267+ git update-index --skip-worktree .vscode/settings.json 2>/dev/null || true
268+ log " skip-worktree: .vscode/settings.json"
269+ fi
270+ if [ -f "version.info" ]; then
271+ git update-index --skip-worktree version.info 2>/dev/null || true
272+ log " skip-worktree: version.info"
273+ fi
274+}
275+ 
276+# ──────────────────────────────────────────────────────────────────────────────
277+# 9. compile_commands.json 提示
278+# ──────────────────────────────────────────────────────────────────────────────
279+check_compile_commands() {
280+ if [ ! -f "build/compile_commands.json" ]; then
281+ warn "build/compile_commands.json not found (expected on cold start)"
282+ warn "Run 'python3 build.py' to generate it for clangd support"
283+ else
284+ log "compile_commands.json found, clangd ready"
285+ fi
286+}
287+ 
288+# ──────────────────────────────────────────────────────────────────────────────
289+# 主流程
290+# ──────────────────────────────────────────────────────────────────────────────
291+log "Starting container initialization (msprof)..."
292+ 
293+configure_user_bin
294+configure_python3
295+install_build_deps
296+sync_git_identity
297+append_dev_hint_once
298+install_pre_commit_hook
299+setup_clangd
300+ignore_local_changes
301+check_compile_commands
302+ 
303+log "Container initialization complete!"
M.gitignore+13-1
@@ -22,7 +22,7 @@ inc_coverage_result.html
22*.profdata22*.profdata
23 23 
24# IDE and local tooling24# IDE and local tooling
25-.vscode/25+.vscode/settings.json
26.vs/26.vs/
27.idea/27.idea/
28.codex28.codex
@@ -72,3 +72,15 @@ inc_coverage_result.html
72db_assembler/72db_assembler/
73PROF/73PROF/
74*.slice*74*.slice*
75+ 
76+# Build artifacts
77+/artifacts/
78+/dist/
79+.venv/
80+ 
81+# Cache directories
82+.cache/
83+.ruff_cache/
84+ 
85+# Devcontainer generated
86+.devcontainer/.host-gitconfig
M.pre-commit-config.yaml+2-1
@@ -61,6 +61,7 @@ repos:
61 - id: bandit61 - id: bandit
62 name: bandit (Python 安全漏洞检查)62 name: bandit (Python 安全漏洞检查)
63 types: [ python ]63 types: [ python ]
64+ additional_dependencies: [".[toml]"]
64 args: [65 args: [
65 "--config=pre-commit/pyproject.toml",66 "--config=pre-commit/pyproject.toml",
66 "--quiet",67 "--quiet",
@@ -102,7 +103,7 @@ repos:
102 hooks:103 hooks:
103 - id: gitleaks-offline-scan104 - id: gitleaks-offline-scan
104 name: Gitleaks Secret Scan(Local Binary)105 name: Gitleaks Secret Scan(Local Binary)
105- entry: ./gitleaks106+ entry: gitleaks
Mrtutu
MrtutuMrtutu19 天前

严重程度: 建议

问题: gitleaks 的 entry 从本地二进制 ./gitleaks 改为系统命令 gitleaks,这是一处影响所有贡献者/CI 的共享配置变更,而不仅是 devcontainer 场景。

原因: 非 devcontainer 用户(此前依赖仓库内/本地下载的 ./gitleaks 二进制)以及未在 PATH 中安装 gitleaks 的 CI 环境,pre-commit 会直接因找不到命令而失败。结合 post-create.sh 目前仅 arm64 会安装 gitleaks 的问题,x86_64 容器内同样会失败。

怎么改: 保持向后兼容,例如 entry 改为 gitleaks 并让 pre-commit 环境安装 gitleaks,或在文档/CI 中明确要求安装 gitleaks 到 PATH,并保证 post-create.sh 能对 x86_64 正确安装。

likedislike
106 language: system107 language: system
107 pass_filenames: true108 pass_filenames: true
108 exclude: ^\.pre-commit-config\.yaml$109 exclude: ^\.pre-commit-config\.yaml$
A.vscode/launch.json+30-0
@@ -0,0 +1,30 @@
1+{
2+ "version": "0.2.0",
3+ "configurations": [
4+ {
5+ "name": "Python: Debug Active File",
6+ "type": "debugpy",
7+ "request": "launch",
8+ "program": "${file}",
9+ "console": "integratedTerminal",
10+ "justMyCode": false,
11+ "env": {
12+ "PYTHONPATH": "${workspaceFolder}/build/analysis:${workspaceFolder}:${env:PYTHONPATH}",
13+ "LD_LIBRARY_PATH": "${workspaceFolder}/build/analysis/lib64:${workspaceFolder}/prefix:${env:LD_LIBRARY_PATH}"
14+ }
15+ },
16+ {
17+ "name": "Python: Pytest Case Debugging",
18+ "type": "debugpy",
19+ "request": "launch",
20+ "module": "pytest",
21+ "args": ["${file}", "-v"],
22+ "console": "integratedTerminal",
23+ "justMyCode": false,
24+ "env": {
25+ "PYTHONPATH": "${workspaceFolder}/build/analysis:${workspaceFolder}:${env:PYTHONPATH}",
26+ "LD_LIBRARY_PATH": "${workspaceFolder}/build/analysis/lib64:${workspaceFolder}/prefix:${env:LD_LIBRARY_PATH}"
27+ }
28+ }
29+ ]
30+}
A.vscode/tasks.json+59-0
@@ -0,0 +1,59 @@
1+{
2+ "version": "2.0.0",
3+ "tasks": [
4+ {
5+ "label": "Build: Release Mode",
6+ "type": "shell",
7+ "command": "python3 build.py local",
8+ "group": {
9+ "kind": "build",
10+ "isDefault": true
11+ },
12+ "presentation": {
13+ "reveal": "always",
14+ "panel": "dedicated"
15+ },
16+ "problemMatcher": []
17+ },
18+ {
19+ "label": "Build: Debug Mode",
20+ "type": "shell",
21+ "command": "python3 build.py -e only_down_deps=true && cmake -S . -B build -DCMAKE_BUILD_TYPE=Debug && make -C build -j$(nproc)",
22+ "group": {
23+ "kind": "build"
24+ },
25+ "presentation": {
26+ "reveal": "always",
27+ "panel": "dedicated"
28+ },
29+ "problemMatcher": []
30+ },
31+ {
32+ "label": "Test: Run Unit Tests",
33+ "type": "shell",
34+ "command": "python3 build.py test local",
35+ "group": {
36+ "kind": "test",
37+ "isDefault": true
38+ },
39+ "presentation": {
40+ "reveal": "always",
41+ "panel": "dedicated"
42+ },
43+ "problemMatcher": []
44+ },
45+ {
46+ "label": "Clean: All Workspace",
47+ "type": "shell",
48+ "command": "find build -depth -type d -name CMakeFiles -exec rm -rf {} + ; find build -depth -type d -name Testing -exec rm -rf {} + ; find build -depth -type d -name CTestTestfile.cmake -exec rm -rf {} + ; find build -depth -type f -name cmake_install.cmake -delete ; find build -depth -type d -name analysis-prefix -exec rm -rf {} + ; rm -rf prefix output artifacts test/build_llt test/output test/opensource opensource platform",
49+ "options": {
50+ "cwd": "${workspaceFolder}"
51+ },
52+ "presentation": {
53+ "reveal": "always",
54+ "panel": "dedicated"
55+ },
56+ "problemMatcher": []
57+ }
58+ ]
59+}
MCMakeLists.txt+1-0
@@ -2,6 +2,7 @@ cmake_minimum_required(VERSION 3.14.1)
2project(msprof)2project(msprof)
3 3 
4set(CMAKE_SKIP_RPATH TRUE)4set(CMAKE_SKIP_RPATH TRUE)
5+set(CMAKE_EXPORT_COMPILE_COMMANDS ON)
Mrtutu
MrtutuMrtutu19 天前

严重程度: 建议

问题: set(CMAKE_EXPORT_COMPILE_COMMANDS ON) 加在了根 CMakeLists.txt,但该文件并未被真实构建流程使用:build/build.shcmake -S cmake/superbuild -B build/analysis(superbuild 通过 ExternalProject_Add 构建 analysis/csrc),UT 走 test/build_llt,都不会配置根 CMakeLists。只有 Debug 任务的 cmake -S . -B build 会用到它,而根工程没有任何 target,生成的 compile_commands.json 为空数组。

原因: .clangd(CompilationDatabase: build/)与 devcontainer.json 的 clangd.pathToCompdb: build 都期望 build/compile_commands.json,但 release 构建不会生成该文件,clangd 的代码跳转(文档宣称的核心能力)实际拿不到编译数据库。另外 build/compile_commands.json 是仓库中已 tracked 的陈旧文件(内容指向 collector 构建),一旦通过 cmake -S . -B build 重新生成会被覆盖为空数组并弄脏 git status。

怎么改: 将开关放到实际被构建的工程中(如 analysis/csrc/CMakeLists.txttest/CMakeLists.txt),或直接在 build.sh 的 cmake 命令中传 -DCMAKE_EXPORT_COMPILE_COMMANDS=ON;并考虑把陈旧的 tracked build/compile_commands.json 从 git 中移除。

likedislike
5set(TOP_DIR ${CMAKE_CURRENT_LIST_DIR})6set(TOP_DIR ${CMAKE_CURRENT_LIST_DIR})
6 7 
7set(CMAKE_MODULE_PATH8set(CMAKE_MODULE_PATH
Mbuild.py+15-6
@@ -8,7 +8,7 @@
8# You can use this software according to the terms and conditions of the Mulan PSL v2.8# You can use this software according to the terms and conditions of the Mulan PSL v2.
9# You may obtain a copy of Mulan PSL v2 at:9# You may obtain a copy of Mulan PSL v2 at:
10#10#
11-# http://license.coscl.org.cn/MulanPSL211+# http://license.coscl.org.cn/MulanPSL2
12#12#
13# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND,13# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND,
14# EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT,14# EITHER EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT,
@@ -95,20 +95,29 @@ class BuildManager:
95 95 
96 if 'test' in self.args.command:96 if 'test' in self.args.command:
97 # -------------------- 单元测试 --------------------97 # -------------------- 单元测试 --------------------
98- # 在非 local 场景下按需更新依赖;在 local 场景下仅使用本地已有代码,不更新依赖。
99 if 'local' not in self.args.command:98 if 'local' not in self.args.command:
100 self._execute_command(["bash", "scripts/download_thirdparty.sh", "ut"])99 self._execute_command(["bash", "scripts/download_thirdparty.sh", "ut"])
100+ else:
101+ # -------------------- 产品构建 --------------------
102+ if 'local' not in self.args.command:
103+ self._execute_command(["bash", "scripts/download_thirdparty.sh"])
101 104 
105+ # only_down_deps 在依赖下载后、编译/测试前统一检查,test 和产品构建分支均生效
106+ extra_options = {}
107+ for opt in self.args.extra:
108+ key, _, val = opt.partition('=')
109+ extra_options[key] = val
110+ if extra_options.get('only_down_deps') == 'true':
111+ logging.info("only_down_deps=true, exiting after dependency download.")
112+ return
113+ 
114+ if 'test' in self.args.command:
102 self._execute_command(["bash", "scripts/execute_cpp_test_case.sh"])115 self._execute_command(["bash", "scripts/execute_cpp_test_case.sh"])
103 self._execute_command(["bash", "scripts/execute_py_test_case.sh"])116 self._execute_command(["bash", "scripts/execute_py_test_case.sh"])
104 # self._execute_command(["bash", "scripts/generate_coverage_py.sh"])117 # self._execute_command(["bash", "scripts/generate_coverage_py.sh"])
105 # self._execute_command(["bash", "scripts/generate_coverage_cpp.sh"])118 # self._execute_command(["bash", "scripts/generate_coverage_cpp.sh"])
106 else:119 else:
107 # -------------------- 产品构建 --------------------120 # -------------------- 产品构建 --------------------
108- # 在非 local 场景下按需更新依赖;在 local 场景下仅使用本地已有代码,不更新依赖。
109- if 'local' not in self.args.command:
110- self._execute_command(["bash", "scripts/download_thirdparty.sh"])
111- 
112 logging.info("--version: %s", self.args.version)121 logging.info("--version: %s", self.args.version)
113 for opt in self.args.extra:122 for opt in self.args.extra:
114 key, _, val = opt.partition('=')123 key, _, val = opt.partition('=')
Mdocs/en/developer_guide/development_guide.md+51-2
@@ -43,7 +43,56 @@ Before starting development, identify the specific layer that your changes will
43 43 
44## 3. Development Environment Settings44## 3. Development Environment Settings
45 45 
46-### 3.1 Foundational Software46+### 3.1 Option 1: One-Click devcontainer Setup (Recommended)
47+ 
48+msprof includes a pre-configured [devcontainer](https://containers.dev/) environment. Open the repository in VS Code and enter a fully standardized container with a single click — no manual dependency installation required. The container automatically handles:
49+ 
50+- Python 3 environment and build toolchain (GCC 11.2.0, CMake 3.14+)
51+- System dependencies (python3-devel, pip packages including pytest/coverage)
52+- Third-party dependency pre-download (googletest, mockcpp, boost, protobuf, json, rapidjson, securec)
53+- pre-commit auto-installation (including gitleaks secret scanning)
54+- clangd C++ language server
55+- Git identity sync
56+ 
57+**Prerequisites:**
58+ 
59+| Environment | Requirement |
60+|------|------|
61+| PC | VS Code with Dev Containers and Remote-SSH extensions installed |
62+| Linux Server | Docker service running |
63+ 
64+**Steps:**
65+ 
66+1. Clone the msprof repository to a Linux server
67+2. Connect VS Code to the server via Remote-SSH and open the repository folder
68+3. VS Code will auto-detect the `.devcontainer` configuration; click **"Reopen in Container"** in the bottom-left corner
69+4. The container starts and runs `post-create.sh` automatically (~1-2 minutes)
70+5. Press `Ctrl+Shift+P``Tasks: Run Task` to select a build task
71+ 
72+**Built-in VS Code Tasks:**
73+ 
74+| Task | Shortcut | Description |
75+|------|--------|------|
76+| `Build: Release Mode` | `Ctrl+Shift+B` | One-click Release build (equivalent to `bash build/build.sh`) |
77+| `Build: Debug Mode` | — | Debug build (with debug symbols) |
78+| `Test: Run Unit Tests` | — | Run C++ and Python unit tests |
79+| `Clean: All Workspace` | — | Clean all build artifacts (build/CMakeFiles, prefix, output, test/build_llt, etc.) |
80+ 
81+**Code Navigation:**
82+ 
83+- C++: Cross-file jump-to-definition (F12), find references (Shift+F12), and code completion via clangd after building
84+- Python: Semantic navigation and type inference via Pylance
85+ 
86+**Graphical Debugging:**
87+ 
88+- Open a Python source file, press `F5`, select `Python: Debug Active File` to start debugpy debugging
89+- Breakpoints, variable inspection, and call stacks work as expected
90+ 
91+### 3.2 Option 2: Manual Environment Setup
92+ 
93+If devcontainer is not available, follow the manual setup steps below.
94+ 
95+#### 3.2.1 Foundational Software
47 96 
48| Software| Version Requirement| Purpose|97| Software| Version Requirement| Purpose|
49| --- | --- | --- |98| --- | --- | --- |
@@ -52,7 +101,7 @@ Before starting development, identify the specific layer that your changes will
52| SQLite3 | Building dependency| Parsing-related capabilities|101| SQLite3 | Building dependency| Parsing-related capabilities|
53| Bash | Recommended for Linux environments| Build and script execution|102| Bash | Recommended for Linux environments| Build and script execution|
54 103 
55-### 3.2 Prerequisites104+#### 3.2.2 Prerequisites
56 105 
571. A compatible version of the CANN environment has been installed.1061. A compatible version of the CANN environment has been installed.
582. The `cann` installation directory is available.1072. The `cann` installation directory is available.
Mdocs/zh/development_guide/development_guide.md+49-2
@@ -1,4 +1,4 @@
1-# msProf 开发指南1+# msProf 开发指南
2 2 
3本文面向 msProf 的开发和维护人员,介绍源码目录、构建方式、采集与解析链路、功能改动后的验证方法,以及资料联动更新要求。本文重点结合 msProf 当前仓库和现有文档内容编写,适用于新增命令参数、扩展解析能力、增加交付件或维护 run 包安装方式等场景。3本文面向 msProf 的开发和维护人员,介绍源码目录、构建方式、采集与解析链路、功能改动后的验证方法,以及资料联动更新要求。本文重点结合 msProf 当前仓库和现有文档内容编写,适用于新增命令参数、扩展解析能力、增加交付件或维护 run 包安装方式等场景。
4 4 
@@ -43,7 +43,54 @@ msProf 提供 AI 任务运行性能数据和昇腾 AI 处理器系统数据的
43 43 
44## 3. 开发环境配置44## 3. 开发环境配置
45 45 
46-按照《[msProf 安装指南 源码安装](../install_guide/msprof_install_guide.md#231-环境准备)》章节完成编译和测试环境的搭建。46+### 3.1 方式一:devcontainer 一键开发环境(推荐)
47+ 
48+msprof 已内置 [devcontainer](https://containers.dev/) 开发环境配置,开发者通过 VS Code 打开仓库后可一键进入标准化容器,无需手工安装任何依赖。容器自动完成以下准备:
49+ 
50+- Python 3 环境与编译工具链(GCC 11.2.0、CMake 3.14+)
51+- 系统依赖安装(python3-devel、pip 包含 pytest/coverage)
52+- 三方依赖预下载(googletest、mockcpp、boost、protobuf、json、rapidjson、securec)
atomgit-bot
atomgit-botatomgit-bot20 天前

🟡 Medium Priority

文档第 52 行声明容器初始化后"三方依赖预下载(googletest、mockcpp、boost、protobuf、json、rapidjson、securec)"会自动完成。但实际 post-create.sh 脚本(共 9 个初始化步骤)中没有任何步骤执行 download_thirdparty.sh 来下载这些第三方依赖。

更严重的是:VS Code 的 Release 构建任务(Build: Release Mode)实际执行的是 python3 build.py local,而 build.pylocal 参数会明确跳过依赖下载(见 build.py 第 118 行 if 'local' not in self.args.command)。这意味着开发者按文档进入容器后直接执行默认构建任务(Ctrl+Shift+B),会因为缺少第三方依赖而导致构建失败。

开发者必须先执行 Debug 构建任务(该任务会触发 only_down_deps=true 下载依赖)或手动运行 bash scripts/download_thirdparty.sh,但文档完全没有提及这一前置步骤。

建议:修正三方依赖预下载的描述,说明实际行为:依赖不是容器初始化时预下载的,而是由构建任务按需触发;同时提醒开发者 Release 构建前需确保依赖已下载。

likedislike
53+- pre-commit 自动启用(含 gitleaks 密钥扫描)
54+- clangd C++ 语言服务就绪
55+- Git 身份同步
56+ 
57+**前置条件:**
58+ 
59+| 环境 | 要求 |
60+|------|------|
61+| PC | VS Code,安装 Dev Containers 插件 和 Remote-SSH 插件 |
62+| Linux 服务器 | Docker 服务运行中 |
63+ 
64+**使用步骤:**
65+ 
66+1. 将 msprof 仓库 clone 到 Linux 服务器
67+2. VS Code 通过 Remote-SSH 连接服务器,打开仓库目录
68+3. VS Code 自动检测到 `.devcontainer` 配置,点击左下角 **"Reopen in Container"**
69+4. 容器启动后自动执行 `post-create.sh` 完成初始化(约 1-2 分钟)
70+5.`Ctrl+Shift+P``Tasks: Run Task` 选择构建任务
71+ 
72+**VS Code 内置任务:**
73+ 
74+| 任务 | 快捷键 | 说明 |
75+|------|--------|------|
76+| `Build: Release Mode` | `Ctrl+Shift+B` | 一键 Release 构建(等同 `bash build/build.sh`) |
atomgit-bot
atomgit-botatomgit-bot20 天前

🟡 Medium Priority

文档第 76 行将 Build: Release Mode 任务描述为"等同 bash build/build.sh"。但实际 .vscode/tasks.json 中该任务执行的是 python3 build.py local(见 tasks.json 第 7 行)。

两者行为存在关键差异:

  • bash build/build.sh:直接调用 shell 构建脚本,需手动传 --version 等参数,且不会自动下载依赖。
  • python3 build.py local:跳过第三方依赖下载(local 语义),自动传入 --version--whl_version 参数,构建完成后还会将产物归档到 artifacts/ 目录。

如果开发者按文档说明手动执行 bash build/build.sh(而非使用 VS Code 任务),会得到不同的构建行为——特别是版本号参数缺失可能导致产物命名不一致。同时"等同"的说法会误导开发者认为两者的前置条件相同,但实际上 python3 build.py local 要求第三方依赖已提前下载就绪。

建议:将描述改为任务实际执行的命令和语义,避免"等同"的模糊表述,明确说明该任务封装了版本号传参和产物归档,但跳过依赖下载。

likedislike
77+| `Build: Debug Mode` | — | Debug 编译(带调试符号) |
78+| `Test: Run Unit Tests` | — | 运行 C++ 和 Python 单元测试 |
79+| `Clean: All Workspace` | — | 清理所有构建产物(build/CMakeFiles、prefix/、output/、test/build_llt/ 等) |
80+ 
81+**代码智能跳转:**
82+ 
83+- C++:构建后通过 clangd 实现跨文件跳转(F12)、查找引用(Shift+F12)、代码补全
84+- Python:Pylance 提供语义跳转和类型推导
85+ 
86+**图形化调试:**
87+ 
88+- 打开 Python 源文件,按 `F5` 选择 `Python: Debug Active File` 进入 debugpy 调试
89+- 断点、变量查看、调用栈等功能均可正常使用
90+ 
91+### 3.2 方式二:手动环境配置
92+ 
93+如果不能使用 devcontainer,请按按照《[msProf 安装指南 — 源码安装](../install_guide/msprof_install_guide.md#231-环境准备)》章节完成编译和测试环境的搭建。
47 94 
48> **说明:** 环境镜像的构建方法及配套软件版本由 MindStudio 统一镜像制作指南维护,本仓库不重复定义。95> **说明:** 环境镜像的构建方法及配套软件版本由 MindStudio 统一镜像制作指南维护,本仓库不重复定义。
49 96 
Mscripts/download_thirdparty.sh+11-1
@@ -67,7 +67,17 @@ mkdir -p ${OPENSOURCE_DIR} && cd ${OPENSOURCE_DIR}
67 67 
68if [ "${isUT}" == "1" ]; then68if [ "${isUT}" == "1" ]; then
69 mkdir -p ${LLT_DIR} && cd ${LLT_DIR}69 mkdir -p ${LLT_DIR} && cd ${LLT_DIR}
70- [ ! -d "googletest" ] && git clone https://gitcode.com/GitHub_Trending/go/googletest.git googletest -b release-1.12.170+ if [ ! -d "googletest" ]; then
Mrtutu
MrtutuMrtutu19 天前

严重程度: 建议

问题: googletest 降级(1.12.1 → 1.10.0)和移除 -Werror 的 sed 修补都放在 if [ ! -d "googletest" ]; then 内,只对全新 clone 生效。已有 test/opensource/googletest 目录(CI 缓存、存量开发机)会继续停留在 1.12.1 且带 -Werror,编译失败问题依旧存在。

原因: 该脚本同时被 CI 与所有本地构建复用,降级与修补不幂等,导致同一仓库在不同环境下 googletest 版本不一致。同时这是一次全项目共享的三方依赖版本变更(超出 devcontainer 目标范围),应确认不影响 CI 及其它测试,并核对测试代码是否依赖 gtest 1.11/1.12 新增 API。

怎么改: 对已存在的 googletest 目录同样执行版本校验与修补(或检测版本不符时强制重新 clone),并把 -Werror 移除逻辑从 if 块中提出,保证对存量目录也生效;建议同步评估并记录全项目降级带来的兼容性影响。

likedislike
71+ git clone https://gitcode.com/GitHub_Trending/go/googletest.git googletest -b release-1.10.0
72+ fi
73+ # 对已存在的 googletest 目录同样移除 -Werror,保证 CI 缓存和存量环境兼容。
74+ # 使用 \b 边界匹配移除所有 -Werror 出现(同时处理不同格式),并校验结果。
75+ if [ -f "googletest/googletest/cmake/internal_utils.cmake" ]; then
76+ sed -i 's/-Werror\b//g' googletest/googletest/cmake/internal_utils.cmake
77+ if grep -q -- '-Werror\b' googletest/googletest/cmake/internal_utils.cmake 2>/dev/null; then
78+ echo "[WARN] -Werror removal from googletest may be incomplete"
79+ fi
80+ fi
atomgit-bot
atomgit-botatomgit-bot20 天前

🟡 Medium Priority

changed line: scripts/download_thirdparty.sh L70-77(修改 googletest clone 分支和新增 sed 补丁)→ affected behavior: 旧代码使用 release-1.12.1,新代码改为 release-1.10.0,这是一个显著的版本回退(跨越 2 个小版本);同时新增两个 sed 命令移除 -Werror→ failure mode: (1) release-1.10.0 可能缺少 release-1.12.1 引入的 bug 修复或功能,导致测试行为差异或编译问题;(2) sed 命令依赖 internal_utils.cmake 中精确的字符串格式,如果 googletest 版本更新导致格式变化(如空格、引号风格差异),sed 静默无匹配,-Werror 仍然保留,编译仍可能因警告而失败。→ suggested fix: 优先考虑保持 release-1.12.1 版本并通过 CMake 变量(如 -DCMAKE_CXX_FLAGS=-Wno-error)或 patch 文件方式移除 -Werror;如确需降级,请在注释中说明原因并使用更健壮的补丁方式(如 patch 文件)。

建议:考虑保持 release-1.12.1,通过 cmake 参数 -Dgtest_disable_pthreads=OFF 等或 -DCMAKE_CXX_FLAGS=-Wno-error 处理 -Werror;如必须降级请在注释中说明原因。

likedislike
71 [ ! -d "mockcpp" ] && git clone https://gitcode.com/Ascend/mockcpp.git mockcpp -b ascend_mindstudio_mockcpp_master81 [ ! -d "mockcpp" ] && git clone https://gitcode.com/Ascend/mockcpp.git mockcpp -b ascend_mindstudio_mockcpp_master
72 download_boost82 download_boost
73fi83fi
Mscripts/execute_cpp_test_case.sh+20-2
@@ -40,8 +40,26 @@ if [[ -n "$1" && "$1" == "analysis" ]]; then
40elif [[ -n "$1" && "$1" == "all" ]]; then40elif [[ -n "$1" && "$1" == "all" ]]; then
41 cmake ../ -DPACKAGE=ut -DMODE=all41 cmake ../ -DPACKAGE=ut -DMODE=all
42else42else
43- change_file_to_unix_format # change file from dos to unix format, so that gcov exclude comment can be added43+ # gcov 覆盖率统计需要给日志宏加 LCOV_EXCL_LINE 注释,且需要 Unix 换行。
44- add_gcov_excl_line # add gcov exclude comment for macro definition code lines to raise branch coverage44+ # 为避免污染源文件,先备份 analysis/csrc,修改后编译测试,最后恢复。
45+ BACKUP_DIR=$(mktemp -d)
46+ cp -a ${TOP_DIR}/analysis/csrc ${BACKUP_DIR}/csrc_bak || { echo "[execute_cpp_test_case] WARN: failed to backup csrc, aborting"; exit 1; }
47+ 
48+ # trap 必须在 csrc 被修改之前注册,包含 EXIT/INT/TERM 信号
49+ cleanup_and_restore() {
50+ echo "[execute_cpp_test_case] Restoring analysis/csrc from backup..."
51+ rm -rf ${TOP_DIR}/analysis/csrc
Mrtutu
MrtutuMrtutu19 天前

严重程度: 建议

问题: cleanup_and_restore 恢复流程先 rm -rf ${TOP_DIR}/analysis/csrc 再从备份 cp -a 回源;若恢复时 cp 失败(如磁盘满、权限问题),源目录已被删除,且函数末尾还会 rm -rf ${BACKUP_DIR} 把备份一并清掉,导致源码不可逆丢失。

原因: 脚本开启了 set -e,正常情况下 trap EXIT 会兜底恢复,但恢复动作本身没有失败校验,属于数据安全风险点。此外若进程被 SIGKILL(如 OOM)终止,EXIT trap 不会执行,analysis/csrc 会残留 LCOV_EXCL_LINE 注释与换行符改动。

怎么改: 恢复时先校验 cp 成功再删除备份,或直接采用 mv 方式恢复;并补充 INT/TERM 的 trap。例如:

cleanup_and_restore() {
    rm -rf ${TOP_DIR}/analysis/csrc
    if cp -a ${BACKUP_DIR}/csrc_bak ${TOP_DIR}/analysis/csrc; then
        rm -rf ${BACKUP_DIR}
    else
        echo "[execute_cpp_test_case] ERROR: restore failed, backup kept at ${BACKUP_DIR}"
    fi
}
trap cleanup_and_restore EXIT INT TERM
likedislike
52+ if cp -a ${BACKUP_DIR}/csrc_bak ${TOP_DIR}/analysis/csrc; then
53+ rm -rf ${BACKUP_DIR}
54+ else
55+ echo "[execute_cpp_test_case] ERROR: restore failed, backup kept at ${BACKUP_DIR}"
56+ fi
57+ }
58+ trap cleanup_and_restore EXIT INT TERM
59+ 
60+ change_file_to_unix_format
61+ add_gcov_excl_line
62+ 
45 cmake ../ -DPACKAGE=ut -DMODE=all63 cmake ../ -DPACKAGE=ut -DMODE=all
46fi64fi
47make -j$(nproc)65make -j$(nproc)