已合并
同步 CheckerL2 最新代码到 hccl_vm #3274
c15029001705创建于 7月1日
同步 CheckerL2 最新代码到 hccl_vm #3274
已合并
c15029001705创建于 7月1日
175 个文件变更+10697-6590
Mtest/hccl_vm/CMakeLists.txt+2-0
@@ -93,6 +93,7 @@ include_directories(
93 93 
94function(config_hcclvm_install)94function(config_hcclvm_install)
95 install(DIRECTORY DESTINATION "data")95 install(DIRECTORY DESTINATION "data")
96+ install(DIRECTORY DESTINATION "archive")
96 install(DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}/config/"97 install(DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}/config/"
97 DESTINATION "config")98 DESTINATION "config")
98 install(FILES99 install(FILES
@@ -100,6 +101,7 @@ function(config_hcclvm_install)
100 "${CMAKE_CURRENT_SOURCE_DIR}/asset/generate_cluster_topo.sh"101 "${CMAKE_CURRENT_SOURCE_DIR}/asset/generate_cluster_topo.sh"
101 "${CMAKE_CURRENT_SOURCE_DIR}/asset/generate_server_topo.sh"102 "${CMAKE_CURRENT_SOURCE_DIR}/asset/generate_server_topo.sh"
102 "${CMAKE_CURRENT_SOURCE_DIR}/asset/hccl_config.sh"103 "${CMAKE_CURRENT_SOURCE_DIR}/asset/hccl_config.sh"
104+ "${CMAKE_CURRENT_SOURCE_DIR}/asset/archive.sh"
103 DESTINATION "script"105 DESTINATION "script"
104 PERMISSIONS106 PERMISSIONS
105 OWNER_READ OWNER_WRITE OWNER_EXECUTE107 OWNER_READ OWNER_WRITE OWNER_EXECUTE
Mtest/hccl_vm/README.md+29-27
@@ -31,7 +31,7 @@ chmod +x Ascend-cann-950-ops_9.1.0_linux-x86_64.run
31 31 
32hccl_test是昇腾官方提供的HCCL性能测试工具,详见[HCCL性能测试工具](https://www.hiascend.com/document/detail/zh/CANNCommunityEdition/910beta1/devaids/hccltool/HCCLpertest_16_0001.html),HCCL-VM支持在虚拟环境中运行hccl_test用例。请先参照[hccl_test用例构建](#42-hccl-test用例构建)章节进行用例二进制程序的编译。32hccl_test是昇腾官方提供的HCCL性能测试工具,详见[HCCL性能测试工具](https://www.hiascend.com/document/detail/zh/CANNCommunityEdition/910beta1/devaids/hccltool/HCCLpertest_16_0001.html),HCCL-VM支持在虚拟环境中运行hccl_test用例。请先参照[hccl_test用例构建](#42-hccl-test用例构建)章节进行用例二进制程序的编译。
33 33 
34-备注:可选,未来支持Pytorch用例34+备注:可选,未来支持Pytorch用例
35 35 
36---36---
37 37 
@@ -64,11 +64,11 @@ bash ./build.sh --full
64 64 
65#### 3.2.1 环境配置65#### 3.2.1 环境配置
66 66 
67-请参照[hccl_rootinfo文件内容](#47-hccl_rootinfojson文件),创建并配置hccl_rootinfo.json文件67+请参照[hccl_rootinfo文件内容](#47-hccl_rootinfojson文件),创建并配置hccl_rootinfo.json文件
68 68 
69#### 3.2.2 CCU模式69#### 3.2.2 CCU模式
70 70 
71-1. 环境变量配置71+1. 环境变量配置
72 72 
73```bash73```bash
74# 进入工具安装目录74# 进入工具安装目录
@@ -103,22 +103,24 @@ cd /home/workspace/hcomm/test/hccl_vm/hccl_vm_install/bin
103(hvm)$> exit103(hvm)$> exit
104```104```
105 105 
106-3. 验证hccl_test用例运行结果 [Runner结果查看](#491-runner插件结果) [Checker结果查看](#492-checker插件结果)106+3. 验证hccl_test用例运行结果
107+[Runner结果查看](#491-runner插件结果)
108+[Checker结果查看](#492-checker插件结果)
107 109 
108#### 3.2.3 AICPU模式110#### 3.2.3 AICPU模式
109 111 
110-AICPU展开模式需要将算法展开步骤放到设备侧执行,因此hccl-vm工具需要将HCCL的设备侧的符号编译并模拟执行。由于设备侧符号是ARM架构的,因此在X86环境上编译时需要借助交叉编译器,运行时需要借助QEMU实现AICPU模式的模拟运行112+AICPU展开模式需要将算法展开步骤放到设备侧执行,因此hccl-vm工具需要将HCCL的设备侧的符号编译并模拟执行。由于设备侧符号是ARM架构的,因此在X86环境上编译时需要借助交叉编译器,运行时需要借助QEMU实现AICPU模式的模拟运行
111 113 
112-设备侧符号使用hccl和hcomm的源码编译,为了保证Host与Device通信协议正确,需要同时编译Host侧的安装包并进行替换安装114+设备侧符号使用hccl和hcomm的源码编译,为了保证Host与Device通信协议正确,需要同时编译Host侧的安装包并进行替换安装
113 115 
114-1. HCCL设备侧符号编译、安装、拷贝等116+1. HCCL设备侧符号编译、安装、拷贝等
115 117 
116```bash118```bash
117cd /home/workspace/hcomm/test/hccl_vm/119cd /home/workspace/hcomm/test/hccl_vm/
118bash ./build_pkg.sh120bash ./build_pkg.sh
119```121```
120 122 
121-2. 环境变量配置123+2. 环境变量配置
122 124 
123```bash125```bash
124# 进入工具安装目录126# 进入工具安装目录
@@ -156,9 +158,9 @@ cd /home/workspace/hcomm/test/hccl_vm/hccl_vm_install/bin
156 158 
157#### 3.2.4 AIV模式159#### 3.2.4 AIV模式
158 160 
159-**当前仅支持Runner插件**161+**当前仅支持Runner插件**
160 162 
161-1. 环境变量配置163+1. 环境变量配置
162 164 
163```bash165```bash
164# 进入工具安装目录166# 进入工具安装目录
@@ -193,7 +195,7 @@ cd /home/workspace/hcomm/test/hccl_vm/hccl_vm_install/bin
193 195 
194### 3.3 Pytorch用例示例196### 3.3 Pytorch用例示例
195 197 
196-暂不支持198+暂不支持
197 199 
198### 3.4 hccl代码修改验证示例200### 3.4 hccl代码修改验证示例
199 201 
@@ -208,8 +210,8 @@ export HCCL_CODE_HOME=/home/workspace/hccl
208export HCOMM_CODE_HOME=/home/workspace/hcomm210export HCOMM_CODE_HOME=/home/workspace/hcomm
209```211```
210 212 
211-1. 若您修改了CANN hccl仓代码,请执行bash build_pkg.sh --install hccl213+1. 若您修改了CANN hccl仓代码,请执行bash build_pkg.sh --install hccl
212-2. 若您修改了CANN hcomm仓代码,请执行bash build_pkg.sh --install hcomm214+2. 若您修改了CANN hcomm仓代码,请执行bash build_pkg.sh --install hcomm
2133. 参考[使用示例](#32-使用示例)步骤,重新运行用例。2153. 参考[使用示例](#32-使用示例)步骤,重新运行用例。
214 216 
215---217---
@@ -262,7 +264,7 @@ MPI_HOME=/usr/lib/x86_64-linux-gnu/openmpi make ASCEND_DIR=${ASCEND_HOME_PATH}
262 264 
263#### 4.2.2 MPICH环境编译265#### 4.2.2 MPICH环境编译
264 266 
265-假设mpich的路径为: `/usr/lib/mpich`267+假设mpich的路径为: `/usr/lib/mpich`
266 268 
267```bash269```bash
268# 进入hccl_test用例源码目录270# 进入hccl_test用例源码目录
@@ -414,7 +416,7 @@ topology:
414`topo.json``ranktable.json` 文件不需要手动创建,工具会根据以下信息自动生成:416`topo.json``ranktable.json` 文件不需要手动创建,工具会根据以下信息自动生成:
415 417 
416- **拓扑配置编号**:用户在启动时指定的编号(如 112、113 等)418- **拓扑配置编号**:用户在启动时指定的编号(如 112、113 等)
417-- **芯片类型**:根据运行环境自动识别的芯片类型419+- **芯片类型**:根据运行环境自动识别的芯片类型
418 420 
419虽然配置文件由工具自动生成,但了解其结构有助于理解拓扑配置。421虽然配置文件由工具自动生成,但了解其结构有助于理解拓扑配置。
420 422 
@@ -478,11 +480,11 @@ topology:
478 480 
479**字段说明**481**字段说明**
480 482 
481-- `server.device_count`:设备总数483+- `server.device_count`:设备总数
482-- `server.groups`:设备分组信息484+- `server.groups`:设备分组信息
483-- `ports`:端口配置485+- `ports`:端口配置
484 - `usage`:端口用途(`peer2peer` 表示设备间连接,`peer2net` 表示与外部连接)486 - `usage`:端口用途(`peer2peer` 表示设备间连接,`peer2net` 表示与外部连接)
485-- `links`:链路配置487+- `links`:链路配置
486 - `link_type`:链路类型(`PEER2PEER``PEER2NET`488 - `link_type`:链路类型(`PEER2PEER``PEER2NET`
487 - `topo_type`:拓扑类型(`1DMESH``CLOS` 等)489 - `topo_type`:拓扑类型(`1DMESH``CLOS` 等)
488 490 
@@ -508,11 +510,11 @@ topology:
508 510 
509**字段说明**511**字段说明**
510 512 
511-- `server_count`:服务器数量513+- `server_count`:服务器数量
512-- `device_count`:设备总数514+- `device_count`:设备总数
513-- `server_list`:服务器和设备列表515+- `server_list`:服务器和设备列表
514- - `device_ip`:设备 IP 地址516+ - `device_ip`:设备 IP 地址
515- - `port`:设备端口号517+ - `port`:设备端口号
516 518 
517### 4.4 hccl\_config.sh文件说明519### 4.4 hccl\_config.sh文件说明
518 520 
@@ -756,7 +758,7 @@ data_size(Bytes): | aveg_time(us): | alg_bandwidth(GB/s): | check_result:
756 758 
757### 开源第三方软件依赖759### 开源第三方软件依赖
758 760 
759-编译本项目时,依赖的第三方开源软件列表如下,离线编译场景可下载并重命名软件包后放置在本项目内的third_party目录下761+编译本项目时,依赖的第三方开源软件列表如下,离线编译场景可下载并重命名软件包后放置在本项目内的third_party目录下
760 762 
761| 开源软件 | 版本 |下载地址 |763| 开源软件 | 版本 |下载地址 |
762| ------------ | ------------- | -------------------------------------------------------------------------------------------------------------------------------------------- |764| ------------ | ------------- | -------------------------------------------------------------------------------------------------------------------------------------------- |
@@ -781,5 +783,5 @@ data_size(Bytes): | aveg_time(us): | alg_bandwidth(GB/s): | check_result:
781 783 
782---784---
783 785 
784-**文档版本**:v1.0786+**文档版本**:v1.1。
785-**最后更新**:2026-06-26787+**最后更新**:2026-06-30。
Atest/hccl_vm/asset/archive.sh+101-0
@@ -0,0 +1,101 @@
1+#!/bin/bash
2+# Copyright (c) 2026 Huawei Technologies Co., Ltd.
3+# This program is free software, you can redistribute it and/or modify it under the terms and conditions of
4+# CANN Open Software License Agreement Version 2.0 (the "License").
5+# Please refer to the License for details. You may not use this file except in compliance with the License.
6+# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED,
7+# INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE.
8+# See LICENSE in the root of the software repository for the full text of the License.
9+ 
10+set -e
11+ 
12+MAX_ARCHIVE_COUNT=10
13+ 
14+RED='\033[0;31m'
15+GREEN='\033[0;32m'
16+YELLOW='\033[1;33m'
17+BLUE='\033[0;34m'
18+NC='\033[0m'
19+ 
20+error_exit() {
21+ echo -e "${RED}[ERROR] $1${NC}" >&2
22+ exit 1
23+}
24+ 
25+success_msg() {
26+ echo -e "${GREEN}[SUCCESS] $1${NC}"
27+}
28+ 
29+info_msg() {
30+ echo -e "${BLUE}[INFO] $1${NC}"
31+}
32+ 
33+warn_msg() {
34+ echo -e "${YELLOW}[WARN] $1${NC}"
35+}
36+ 
37+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
38+case "$(basename "${SCRIPT_DIR}")" in
39+ bin|script)
40+ INSTALL_DIR="$(dirname "${SCRIPT_DIR}")"
41+ ;;
42+ *)
43+ INSTALL_DIR="${SCRIPT_DIR}"
44+ ;;
45+esac
46+ 
47+ARCHIVE_DIR="${INSTALL_DIR}/archive"
48+LOGS_DIR="${INSTALL_DIR}/logs"
49+DATA_DIR="${INSTALL_DIR}/data"
50+ 
51+mkdir -p "${ARCHIVE_DIR}"
52+ 
53+TIMESTAMP=$(date +"%Y%m%d_%H%M%S")
54+ARCHIVE_SUBDIR="${ARCHIVE_DIR}/${TIMESTAMP}"
55+mkdir -p "${ARCHIVE_SUBDIR}/logs"
56+mkdir -p "${ARCHIVE_SUBDIR}/data"
57+ 
58+info_msg "Archive directory created: ${ARCHIVE_SUBDIR}"
59+ 
60+ARCHIVE_DIRS=($(ls -1dt "${ARCHIVE_DIR}"/*/ 2>/dev/null))
61+ARCHIVE_COUNT=${#ARCHIVE_DIRS[@]}
62+ 
63+if [[ ${ARCHIVE_COUNT} -gt ${MAX_ARCHIVE_COUNT} ]]; then
64+ DELETE_COUNT=$((ARCHIVE_COUNT - MAX_ARCHIVE_COUNT))
65+ info_msg "Archive count (${ARCHIVE_COUNT}) exceeds limit (${MAX_ARCHIVE_COUNT}), removing ${DELETE_COUNT} oldest"
66+ for ((i=0; i<DELETE_COUNT; i++)); do
67+ OLDEST="${ARCHIVE_DIRS[$((ARCHIVE_COUNT - 1 - i))]}"
68+ warn_msg "Removing old archive: ${OLDEST}"
69+ rm -rf "${OLDEST}"
70+ done
71+fi
72+ 
73+if [[ -d "${LOGS_DIR}" ]]; then
74+ shopt -s dotglob nullglob
75+ LOGS_CONTENTS=("${LOGS_DIR}"/*)
76+ if [[ ${#LOGS_CONTENTS[@]} -gt 0 ]]; then
77+ mv "${LOGS_DIR}"/* "${ARCHIVE_SUBDIR}/logs/"
78+ info_msg "Moved logs contents to ${ARCHIVE_SUBDIR}/logs/"
79+ else
80+ info_msg "Logs directory is empty, nothing to move"
81+ fi
82+ shopt -u dotglob nullglob
83+else
84+ warn_msg "Logs directory not found: ${LOGS_DIR}"
85+fi
86+ 
87+if [[ -d "${DATA_DIR}" ]]; then
88+ shopt -s dotglob nullglob
89+ DATA_CONTENTS=("${DATA_DIR}"/*)
90+ if [[ ${#DATA_CONTENTS[@]} -gt 0 ]]; then
91+ mv "${DATA_DIR}"/* "${ARCHIVE_SUBDIR}/data/"
92+ info_msg "Moved data contents to ${ARCHIVE_SUBDIR}/data/"
93+ else
94+ info_msg "Data directory is empty, nothing to move"
95+ fi
96+ shopt -u dotglob nullglob
97+else
98+ warn_msg "Data directory not found: ${DATA_DIR}"
99+fi
100+ 
101+success_msg "Archive completed: ${ARCHIVE_SUBDIR}"
Atest/hccl_vm/docs/faq/faq.md+1229-0
@@ -0,0 +1,1229 @@
1+# HCCL-VM FAQ 测试文档
2+ 
3+> 本文档用于测试FAQ HTML生成框架。
4+ 
5+---
6+ 
7+## 模块:HCCL-VM
8+ 
9+### 子模块:命令行
10+ 
11+---
12+ 
13+#### FAQ-E001
14+ 
15+**标题:** 未配置通信域
16+ 
17+**错误码:** `NA` (4)
18+ 
19+**错误函数:** `db_sim_runner_common.cc::GetDeviceByRankId()`
20+ 
21+**关键日志:**
22+```
23+[error][PID:173579][TID:173579][db_sim_runner_common.cc][GetDeviceByRankId] cannot find rank by rank id 0
24+[error][PID:173579][TID:173579][aclrt_device_stub.cc][aclrtSetDevice] [DEVICE_STUB]device not found by rankId:0
25+acl interface return err ./common/src/hccl_test_common.cc:861, retcode: 100000.
26+This is an error in device_init.
27+```
28+ 
29+**问题现象:** 执行业务用例,报找不到rank id为0的设备。
30+ 
31+**可能原因:**
32+ 在执行业务用例前,需要用户自行判断本次算子所使用的通信域规模,并通过hccl-vm mock-comm aa命令配置通信域。其中aa.yaml文件所在路径为$HCCL_VM_INSTALL_DIR/config/topo_meta/aa.yaml。
33+ 
34+---
35+ 
36+#### FAQ-E002
37+ 
38+**标题:** RANK_TABLE_FILE 未设置
39+ 
40+**错误码:** `HCCL_SIM_E_PARA` (1)
41+ 
42+**错误函数:** `hccl_comm_stub.cc::HcclCommInitRootInfo()`
43+ 
44+**关键日志:**
45+```
46+RANK_TABLE_FILE env not set, please check your config.
47+```
48+ 
49+**问题现象:** 通信域初始化时找不到 rank table 配置文件。
50+ 
51+**可能原因:**
52+1. 环境变量未设置
53+2. 文件路径错误
54+ 
55+**解决方案:**
56+```bash
57+export RANK_TABLE_FILE=/path/to/rank_table.json
58+```
59+ 
60+---
61+ 
62+#### FAQ-E003
63+ 
64+**标题:** HCCL_VM_INSTALL_DIR 未设置
65+ 
66+**错误码:** `HCCL_SIM_E_INTERNAL` (4)
67+ 
68+**错误函数:** `hccl_op_stub.cc::VirtualExecuteAivKernel()`
69+ 
70+**关键日志:**
71+```
72+[virtual-aiv] env HCCL_VM_INSTALL_DIR is not set, can not locate <path> for kernel <name>
73+```
74+ 
75+**问题现象:** AIV kernel 虚拟执行失败,找不到对应的 .so 文件。
76+ 
77+**解决方案:**
78+```bash
79+export HCCL_VM_INSTALL_DIR=/path/to/hccl_vm/install/dir
80+```
81+ 
82+---
83+ 
84+#### FAQ-E004
85+ 
86+**标题:** 在子shell中重复执行start命令
87+ 
88+**错误码:** `NA` (无错误码,仅WARNING)
89+ 
90+**错误函数:** `subcmd_start.cc::StartCommand::Execute()`
91+ 
92+**关键日志:**
93+```
94+[warning][PID:<PID>][TID:<TID>][subcmd_start.cc][Execute] hccl-vm has already started. Please do not start it again in a sub-bash.
95+```
96+ 
97+**问题现象:** 在hvm子shell环境中再次执行`hccl-vm start`命令,系统提示已启动并忽略本次操作。
98+ 
99+**可能原因:** `hccl-vm start`会fork一个子bash进程,用户在该子bash(提示符为`(hvm)$>`)内再次输入`hccl-vm start`时,系统拒绝重复启动。
100+ 
101+**解决方法:**
102+ 不要在子shell内重复执行`hccl-vm start`。如需重新启动仿真环境,先退出当前子shell(输入`exit`),再重新执行`hccl-vm start`
103+ 
104+---
105+ 
106+#### FAQ-E005
107+ 
108+**标题:** fork子进程失败
109+ 
110+**错误码:** `HCCL_SIM_HOST_ERROR_CMD` (无标准错误码)
111+ 
112+**错误函数:** `cmd_base_utils.cc::StartHvmCmd()`
113+ 
114+**关键日志:**
115+```
116+fork failed: Resource temporarily unavailable
117+```
118+ 
119+**问题现象:** 执行`hccl-vm start`命令后,系统无法创建子shell进程,仿真环境启动失败。
120+ 
121+**可能原因:**
122+1. 系统用户进程数已达上限(ulimit -u)
123+2. 系统内存不足,无法为新进程分配资源
124+3. PID资源耗尽(/proc/sys/kernel/pid_max)
125+ 
126+**排查步骤:**
127+```bash
128+ulimit -u
129+cat /proc/sys/kernel/pid_max
130+free -m
131+ps -eLf | wc -l
132+```
133+ 
134+**解决方法:**
135+1. 增大用户进程数限制:`ulimit -u <更大的值>`
136+2. 清理系统中残留的僵尸进程
137+3. 检查是否有其他程序占用过多系统资源
138+ 
139+---
140+ 
141+#### FAQ-E006
142+ 
143+**标题:** 插件名称格式错误
144+ 
145+**错误码:** `NA` (CLI参数校验)
146+ 
147+**错误函数:** `subcmd_plugin.cc::PluginCommand::Setup()`
148+ 
149+**关键日志:**
150+```
151+[HVM] [ERROR] Install plugin : Invalid format! Plugin name must start with '@' (e.g., @myplugin).
152+[HVM] [ERROR] Uninstall plugin : Invalid format! Plugin name must start with '@' (e.g., @myplugin).
153+[HVM] [ERROR] Run plugin : Invalid format! Plugin name must start with '@' (e.g., @myplugin).
154+```
155+ 
156+**问题现象:** 执行`hccl-vm plugin install/run/uninstall`命令时,CLI参数校验失败,拒绝执行。
157+ 
158+**可能原因:** 插件名称未以`@`符号开头。例如输入`hccl-vm plugin install runner`而非`hccl-vm plugin install @runner`
159+ 
160+**解决方法:**
161+ 确保插件名称以`@`开头,例如:
162+```bash
163+hccl-vm plugin install @runner
164+hccl-vm plugin install @checker
165+hccl-vm plugin uninstall @runner
166+```
167+ 
168+---
169+ 
170+#### FAQ-E007
171+ 
172+**标题:** 拓扑配置文件不存在
173+ 
174+**错误码:** `NA` (CLI参数校验)
175+ 
176+**错误函数:** `cmd_base_utils.cc::FileInModelDir()`
177+ 
178+**关键日志:**
179+```
180+[HVM] model File not found: <install_path>/config/topo_meta/<name>.yaml
181+```
182+ 
183+**问题现象:** 执行`hccl-vm mock-comm <name>`命令时,指定的拓扑yaml配置文件不存在,CLI参数校验直接拒绝。通信域配置文件是用于描述算子执行通信域的规模(如通信域包含几个超节点,几个server,以及每个server内取哪几张卡,具体详见文件描述)。
184+ 
185+**可能原因:**
186+1. 指定的拓扑名称拼写错误
187+2. 对应的yaml文件未放置在`$HCCL_VM_INSTALL_DIR/config/topo_meta/`目录下
188+3. 文件扩展名错误(应为`.yaml`
189+ 
190+**排查步骤:**
191+```bash
192+ls $HCCL_VM_INSTALL_DIR/config/topo_meta/
193+```
194+ 
195+**解决方法:**
196+ 确认拓扑yaml文件已放置在正确目录中,且文件名与命令参数一致。例如执行`hccl-vm mock-comm 121`需要`config/topo_meta/121.yaml`文件存在。
197+ 
198+---
199+ 
200+#### FAQ-E008
201+ 
202+**标题:** YAML拓扑文件格式解析异常
203+ 
204+**错误码:** `NA` (运行时解析错误)
205+ 
206+**错误函数:** `cmd_cluster_model_utils.cc::ParseYamlTopoImpl()`
207+ 
208+**关键日志:**
209+```
210+[error][PID:<PID>][TID:<TID>][cmd_cluster_model_utils.cc][ParseYamlTopoImpl] Exception when parsing YAML: <detail>
211+```
212+ 
213+**问题现象:** 执行`hccl-vm mock-comm <name>`命令时,YAML拓扑配置文件解析失败,通信域初始化中断。
214+ 
215+**可能原因:**
216+1. YAML文件存在语法错误(如缩进不正确、冒号后缺少空格、非法字符等)
217+2. YAML文件中包含不支持的字段类型或格式
218+3. YAML文件编码非UTF-8
219+ 
220+**排查步骤:**
221+```bash
222+# 使用python验证yaml格式
223+python3 -c "import yaml; yaml.safe_load(open('$HCCL_VM_INSTALL_DIR/config/topo_meta/<name>.yaml'))"
224+```
225+ 
226+**解决方法:**
227+ 根据日志中的`<detail>`信息修正YAML文件的语法错误。常见问题包括:
228+1. 缩进必须使用空格,不能使用Tab
229+2. 键值对的冒号后需要有空格
230+3. 列表项(`-`)的缩进需要与所在层级一致
231+ 
232+---
233+ 
234+### 子模块:内存管理
235+ 
236+---
237+ 
238+#### FAQ-M001
239+ 
240+**标题:** 设备内存分配超限
241+ 
242+**错误码:** `HCCL_SIM_E_MEMORY` (3)
243+ 
244+**错误函数:** `store_sim_device_memory_manager.cc::AllocPhyMem()`
245+ 
246+**关键日志:**
247+```
248+dev:<N> alloc phy mem:<ADDR> size:<SIZE> exceeds pool ceiling:<CEILING>, reject
249+```
250+ 
251+**问题现象:** 设备内存分配请求超出模拟的内存池上限。
252+ 
253+**图示说明:**
254+```mermaid
255+graph LR
256+ A[内存分配请求] --> B{检查池上限}
257+ B -->|未超限| C[分配成功]
258+ B -->|超限| D[拒绝分配]
259+ D --> E[报错:exceeds pool ceiling]
260+```
261+ 
262+---
263+ 
264+#### FAQ-M002
265+ 
266+**标题:** 共享内存创建失败
267+ 
268+**错误码:** `HCCL_SIM_E_SYSCALL` (8)
269+ 
270+**错误函数:** `store_sim_shm_ops.cc::ShmCreate()`
271+ 
272+**关键日志:**
273+```
274+[SHM_OPS] create: shm_open failed, name: <name>
275+[SHM_OPS] create: ftruncate failed, name: <name>
276+[SHM_OPS] create: mmap failed, name: <name>
277+```
278+ 
279+**问题现象:** 无法创建共享内存段。
280+ 
281+**可能原因:**
282+1. `/dev/shm` 空间不足
283+2. 权限不足
284+3. 同名共享内存已存在且冲突
285+ 
286+**排查步骤:**
287+```bash
288+df -h /dev/shm
289+ls /dev/shm/ | grep hccl
290+```
291+ 
292+---
293+ 
294+#### FAQ-M003
295+ 
296+**标题:** 通信内存分配失败
297+ 
298+**错误码:** `HCCL_SIM_E_NOT_FOUND` (6)
299+ 
300+**错误函数:** `store_sim_comm_memory_manager.cc`
301+ 
302+**关键日志:**
303+```
304+[COMM_MEM] alloc failed, name: <name>
305+[COMM_MEM] acquire failed, name: <name>
306+[COMM_MEM] write size too large, size: <N>, max: <MAX>
307+```
308+ 
309+**问题现象:** 跨进程通信内存操作失败。
310+ 
311+---
312+ 
313+### 子模块:桩代理(proxy)
314+ 
315+---
316+ 
317+#### FAQ-PX001
318+ 
319+**标题:** AIV Kernel虚拟执行失败
320+ 
321+**错误码:** `HCCL_SIM_E_INTERNAL` (4)
322+ 
323+**错误函数:** `hccl_op_stub.cc::VirtualExecuteAivKernel()`
324+ 
325+**关键日志:**
326+```
327+[virtual-aiv] env HCCL_VM_INSTALL_DIR is not set
328+[virtual-aiv] missing aiv stub shared library, kernel=<name>
329+[virtual-aiv] dlopen <so> failed, err = <error>
330+[virtual-aiv] dlsym <symbol> from <so> failed, err = <error>
331+```
332+ 
333+**问题现象:** AIV kernel在虚拟环境中执行失败。
334+ 
335+**排查步骤:**
336+```bash
337+echo $HCCL_VM_INSTALL_DIR
338+ls -la $HCCL_VM_INSTALL_DIR/lib/aiv/
339+nm -D $HCCL_VM_INSTALL_DIR/lib/aiv/<kernel>.so | grep <symbol>
340+```
341+ 
342+---
343+ 
344+#### FAQ-PX002
345+ 
346+**标题:** 算子数据库记录失败
347+ 
348+**错误码:** `HCCL_SIM_E_INTERNAL` (4)
349+ 
350+**错误函数:** `hccl_op_stub.cc::RecordOpDbInfo()`
351+ 
352+**关键日志:**
353+```
354+[RecordOpDbInfo] insert op detail+mem failed
355+[HcclAllReduce] record op db info failed
356+```
357+ 
358+**问题现象:** HCCL集合通信算子的参数无法写入仿真数据库。
359+ 
360+**影响的算子:** AlltoAll, AlltoAllV, AllGather, Broadcast, AllReduce, Scatter, Reduce, ReduceScatter
361+ 
362+---
363+ 
364+#### FAQ-PX003
365+ 
366+**标题:** QP未找到或状态错误
367+ 
368+**错误码:** `HCCL_SIM_E_NOT_FOUND` (6)
369+ 
370+**错误函数:** `hccp_stub.cc::RaSendWr()`
371+ 
372+**关键日志:**
373+```
374+[HCCP] RaSendWr: QP <N> not found
375+[HCCP] RaSendWr: QP <N> not in RTS state, current state:<N>
376+```
377+ 
378+**问题现象:** RDMA QP操作失败——QP不存在或未达到RTS状态。
379+ 
380+**图示说明:**
381+```mermaid
382+stateDiagram-v2
383+ [*] --> INIT
384+ INIT --> RTR: RaQpConnect
385+ RTR --> RTS: RaTypicalQpModify
386+ RTS --> [*]: 可以发送数据
387+ RTS --> ERROR: 状态异常
388+ INIT --> ERROR: 未正确初始化
389+```
390+ 
391+---
392+ 
393+#### FAQ-PX004
394+ 
395+**标题:** EndPoint查找失败
396+ 
397+**错误码:** `HCCL_SIM_E_NOT_FOUND` (6)
398+ 
399+**错误函数:** `hccp_stub.cc::RaCtxQpImport()`
400+ 
401+**关键日志:**
402+```
403+[HCCP] cannot find endpoint addr:<IP>
404+Get remote endpoint failed. ip:<IP>, eid:<EID>
405+```
406+ 
407+**问题现象:** 网络端点查找失败。
408+ 
409+**可能原因:** IP地址不在rank table配置的端点列表中。
410+ 
411+---
412+ 
413+#### FAQ-PX005
414+ 
415+**标题:** CCU微码加载失败
416+ 
417+**错误码:** `HCCL_SIM_E_INTERNAL` (4)
418+ 
419+**错误函数:** `hccp_ccu_stub.cc::LoadMicrocodeInstruction()`
420+ 
421+**关键日志:**
422+```
423+[LoadMicrocodeInstruction] get device by logic id <N> failed.
424+[LoadMicrocodeInstruction] get ccu from device by die id <N> failed.
425+[LoadMicrocodeInstruction] insert instr failed
426+```
427+ 
428+**问题现象:** CCU微码指令加载到模拟器失败。
429+ 
430+---
431+ 
432+#### FAQ-PX006
433+ 
434+**标题:** 无法获取当前Context
435+ 
436+**错误码:** `HCCL_SIM_E_NOT_FOUND` (6)
437+ 
438+**错误函数:** `hccp_stub.cc::RaRdevInit()`
439+ 
440+**关键日志:**
441+```
442+[error][PID:<PID>][TID:<TID>][hccp_stub.cc][RaRdevInit] can not get CurrContext: <N>
443+```
444+ 
445+**问题现象:** 在RDMA设备初始化时,无法通过当前Runner获取活跃的Context,导致RDMA设备创建失败。
446+ 
447+**可能原因:**
448+1. 应用层未调用`aclrtSetDevice`/`aclrtCreateContext`初始化设备和上下文
449+2. Context已被提前销毁
450+3. Runner的TLS(线程局部存储)中current_ctx_id无效
451+4. 应用层在调用`aclrtSetDevice`初始化设备上下文前,调用了其他runtime接口获取了上下文
452+ 
453+**排查步骤:**
454+```bash
455+# 检查Context表
456+hccl-vm table show Context
457+# 检查Runner表中的current_ctx_id
458+hccl-vm table show Runner
459+```
460+ 
461+**解决方法:**
462+ 确认应用层在调用RDMA操作前已正确调用`aclrtSetDevice``aclrtCreateContext`,且Context未被提前销毁。
463+ 
464+---
465+ 
466+#### FAQ-PX007
467+ 
468+**标题:** 找不到AICPU二进制文件
469+ 
470+**错误码:** `ACL_ERROR_RT_FEATURE_NOT_SUPPORT`
471+ 
472+**错误函数:** `aclrt_kernel_stub.cc::aclrtDestroyBinary()`
473+ 
474+**关键日志:**
475+```
476+[error][PID:<PID>][TID:<TID>][aclrt_kernel_stub.cc][aclrtDestroyBinary] can not find this binary
477+```
478+ 
479+**问题现象:** 销毁AICPU二进制对象时,在全局kernel binary注册表中找不到对应的二进制句柄。
480+ 
481+**可能原因:**
482+1. 该二进制文件未被正确加载(`aclrtLoadBinary`未执行或失败)
483+2. 二进制句柄已被重复销毁(double-free)
484+3. 二进制对象在多线程环境下被并发操作导致状态不一致
485+ 
486+**排查步骤:**
487+```bash
488+# 检查是否存在重复的destroy调用
489+# 确认aclrtLoadBinary的返回值
490+```
491+ 
492+**解决方法:**
493+ 确保`aclrtLoadBinary`成功返回后再调用`aclrtDestroyBinary`,且不要对同一二进制对象重复销毁。
494+ 
495+---
496+ 
497+#### FAQ-PX008
498+ 
499+**标题:** AICPU设备进程异常退出
500+ 
501+**错误码:** `NA` (进程级错误)
502+ 
503+**错误函数:** `aclrt_kernel_stub.cc::WaitAicpuProcess()`
504+ 
505+**关键日志:**
506+```
507+[error][PID:<PID>][TID:<TID>][aclrt_kernel_stub.cc][WaitAicpuProcess] device process[<PID>] exited with status <N>
508+[error][PID:<PID>][TID:<TID>][aclrt_kernel_stub.cc][WaitAicpuProcess] device process[<PID>] killed by signal <N>
509+```
510+ 
511+**问题现象:** AICPU设备子进程异常退出或被信号杀死,导致主进程随后也退出(`exit(EXIT_FAILURE)`)。
512+ 
513+**可能原因:**
514+1. AICPU进程内部发生未捕获异常或段错误
515+2. 系统资源不足(内存、文件描述符等)导致子进程被OOM killer杀死
516+3. AICPU二进制文件本身存在bug
517+4. 子进程依赖的共享库缺失
518+ 
519+**排查步骤:**
520+```bash
521+# 检查系统日志是否有OOM记录
522+dmesg | grep -i "oom\|killed"
523+# 确认AICPU二进制文件是否完整
524+ls -la $HCCL_VM_INSTALL_DIR/bin/
525+# 检查系统资源
526+ulimit -a
527+free -m
528+```
529+ 
530+**解决方法:**
531+1. 检查AICPU二进制文件是否正确编译和部署
532+2. 确认系统资源充足(内存、文件描述符限制等)
533+3. 如为信号杀死,根据信号编号(如11=SIGSEGV, 9=SIGKILL)进一步定位原因
534+ 
535+---
536+ 
537+#### FAQ-PX009
538+ 
539+**标题:** CCU加载微码时找不到任何rank
540+ 
541+**错误码:** `HCCL_SIM_E_NOT_FOUND` (6)
542+ 
543+**错误函数:** `hccp_ccu_stub.cc::LoadMicrocodeInstruction()`
544+ 
545+**关键日志:**
546+```
547+[error][PID:<PID>][TID:<TID>][hccp_ccu_stub.cc][LoadMicrocodeInstruction] can not find any rank
548+```
549+ 
550+**问题现象:** CCU微码指令加载过程中,无法在当前设备对应的Rank表中找到任何rank记录。
551+ 
552+**可能原因:**
553+1. 通信域未通过`mock-comm`命令初始化,Rank表为空
554+2. 当前设备ID在通信域配置中不存在
555+ 
556+**排查步骤:**
557+```bash
558+# 检查Rank表是否有数据
559+hccl-vm table show Rank
560+# 检查设备表
561+hccl-vm table show Device
562+```
563+ 
564+**解决方法:**
565+ 确保在执行CCU相关操作前,已通过`hccl-vm mock-comm`命令正确初始化通信域,且通信域配置覆盖当前设备。
566+ 
567+---
568+ 
569+#### FAQ-PX010
570+ 
571+**标题:** 按rankId查找设备失败
572+ 
573+**错误码:** `HCCL_E_NOT_FOUND`
574+ 
575+**错误函数:** `aclrt_device_stub.cc::hrtSetDevice()`
576+ 
577+**关键日志:**
578+```
579+[error][PID:<PID>][TID:<TID>][aclrt_device_stub.cc][hrtSetDevice] device not found by rankId:<N>
580+```
581+ 
582+**问题现象:** 调用`aclrtSetDevice`设置当前设备时,根据rankId查找对应的设备失败。
583+ 
584+**可能原因:**
585+1. rankId超出了通信域中实际存在的rank范围 —— 如通信域配置4个NPU,但实际mpirun起了6个NPU进程,导致rankid 4, 5都报找不到device。
586+2. 通信域未初始化(未执行`mock-comm`命令) —— 【大概率】初始化通信域后,工具内部才会初始化Rank表
587+3. ranktable配置与实际使用的rank数量不匹配 —— 可能是`RANK_TABLE_FILE`配置了错误的文件路径
588+ 
589+**排查步骤:**
590+```bash
591+# 检查rankId是否在有效范围内
592+hccl-vm table show Rank
593+```
594+ 
595+**解决方法:**
596+ 确认rankId在通信域配置的合法范围内(0 到 rank_count-1),且`RANK_TABLE_FILE`环境变量指向正确的ranktable.json文件。
597+ 
598+---
599+ 
600+#### FAQ-PX011
601+ 
602+**标题:** 桩接口暂未实现
603+ 
604+**错误码:** `HCCL_SIM_E_INTERNAL` (4) 或 `NA`
605+ 
606+**错误函数:** 多个桩函数文件(`hccp_stub.cc``ascend_hal_stub.cc``aclrt_kernel_stub.cc`等)
607+ 
608+**关键日志:**
609+```
610+[warning][PID:<PID>][TID:<TID>][ascend_hal_stub.cc][*] [STUB] is empty
611+[warning][PID:<PID>][TID:<TID>][hccp_stub.cc][*] [STUB] is empty
612+[error][PID:<PID>][TID:<TID>][hccp_stub.cc][RaCtxGetAuxInfo] Not support yet
613+[error][PID:<PID>][TID:<TID>][hccp_stub.cc][RaCtxGetCrErrInfoList] Not support yet
614+```
615+ 
616+**问题现象:** 应用层调用了仿真器尚未实现的底层驱动或运行时接口,日志中出现`[STUB] is empty``Not support yet`告警/错误。此类桩函数直接返回默认值(通常为0或成功),不进行任何实际操作。
617+ 
618+**可能原因:**
619+ 仿真器当前版本仅实现了HCCL集合通信所需的核心接口子集。部分底层驱动接口(如drvGetDeviceCapability、RaCtxGetAuxInfo、drvMemPrefetch等)不在HCCL通信的核心路径上,因此桩函数体为空或标记为不支持。
620+ 一般情况下,对于HCCL-VM工具支持的流程不会调用这些接口,因此不会出现此类告警。若用户调用错误应用层接口,或进入了错误的HCCL业务流程,可能会出现此类告警。
621+ 
622+**涉及的主要接口类型:**
623+1. **驱动层接口**`ascend_hal_stub.cc`):drvGetDeviceCapability、drvMemPrefetch、drvStreamQuery等约315个接口
624+2. **RDMA接口**`hccp_stub.cc`):RaRestoreSnapshot、RaRdevInitWithBackup、RaCtxGetAuxInfo等约44个接口
625+3. **Runtime适配层**`adapter_rts_stub.cc`):部分aclrt扩展接口
626+4. **TSD客户端**`tsd_client_stub.cc`):TSD相关接口
627+ 
628+**解决方法:**
629+1. 此类告警通常不影响HCCL算子的正确性仿真,可安全忽略
630+2. 如果该告警伴随功能异常,说明应用依赖了未实现的接口,请反馈给仿真器开发团队
631+3. 如需特定接口的桩实现,可联系开发团队优先适配
632+ 
633+---
634+ 
635+### 子模块:组网
636+ 
637+---
638+ 
639+#### FAQ-N001
640+ 
641+**标题:** Ranktable环境变量配置错误
642+ 
643+**错误码:** `NA` (1)
644+ 
645+**错误函数:** `param_check_v2.cc::RanktableRealPath`
646+ 
647+**关键日志:**
648+```
649+[error][PID:172019][TID:172019][log_stub.cc][DlogPrintStub] [HCCL_LOG][param_check_v2.cc:457][172019]RanktableRealPath: /home/teamserver/workspace/CheckerL2_2128/hccl_vm_install/ranktable.json is not a valid real path
650+ 
651+[info][PID:172021][TID:172021][log_stub.cc][DlogPrintStub] [HCCL_LOG][adapter_rts.cc:234] [172021][hrtGetDeviceRefresh]deviceLogicId[3]
652+[error][PID:172020][TID:172020][log_stub.cc][DlogPrintStub] [HCCL_LOG][param_check_v2.cc:457][172020]RanktableRealPath: /home/teamserver/workspace/CheckerL2_2128/hccl_vm_install/ranktable.json is not a valid real path
653+ 
654+[info][PID:172018][TID:172018][log_stub.cc][DlogPrintStub] [HCCL_LOG][adapter_rts.cc:234] [172018][hrtGetDeviceRefresh]deviceLogicId[0]
655+[error][PID:172019][TID:172019][log_stub.cc][DlogPrintStub] [HCCL_LOG][op_base_v2.cc:294][172019][HcclCommInitClusterInfoV2]call trace: hcclRet -> 1
656+ 
657+[error][PID:172019][TID:172019][log_stub.cc][DlogPrintStub] [HCCL_LOG][op_base.cc:811] [172019][operator()]call trace: hcclRet -> 1
658+```
659+ 
660+**问题现象:** 运行用例,初始化通信域失败。
661+ 
662+**可能原因:** ranktable.json文件路径配置错误,请查看RANK_TABLE_FILE环境变量的配置。ranktable.json由工具生成,其路径为$HCCL_VM_INSTALL_DIR/data/ranktable.json。
663+ 
664+**排查步骤:**
665+ 
666+```bash
667+echo $RANK_TABLE_FILE
668+```
669+ 
670+**解决方法:**
671+ 确认RANK_TABLE_FILE环境变量配置正确,指向ranktable.json文件的路径。
672+ 
673+---
674+ 
675+#### FAQ-N002
676+ 
677+**标题:** topo.json路径配置错误
678+ 
679+**错误码:** `NA` (1)
680+ 
681+**错误函数:** `communicator_impl.cc::GetTopoFilePath`
682+ 
683+**关键日志:**
684+```
685+[error][PID:172635][TID:172635][log_stub.cc][DlogPrintStub] [HCCL_LOG][communicator_impl.cc:1339][172635][GetTopoFilePath] topo_file_path[/home/teamserver/workspace/CheckerL2_2128/hccl_vm_install/topo.json] is not a valid real path
686+```
687+ 
688+**问题现象:** 运行用例,初始化通信域失败。
689+ 
690+**可能原因:** topo.json文件路径在/etc/hccl_rootinfo.json文件中配置错误,请查看topo_file_path字段。topo.json由工具生成,其路径为$HCCL_VM_INSTALL_DIR/data/topo.json。
691+ 
692+**排查步骤:**
693+ 
694+```bash
695+echo $TOPO_FILE_PATH
696+```
697+ 
698+**解决方法:**
699+ 确认TOPO_FILE_PATH环境变量配置正确,指向topo.json文件的路径。
700+ 
701+---
702+ 
703+#### FAQ-N003
704+ 
705+**标题:** mock-comm命令报错
706+ 
707+**错误码:** `NA` (6)
708+ 
709+**错误函数:** `db_sim_runner_ops.cc::GetServerKeyById`
710+ 
711+**关键日志:**
712+```
713+(hvm)$> hccl-vm mock-comm 144
714+[error][PID:172799][TID:172875][db_sim_runner_ops.cc][GetServerKeyById] can not find server by id: 0, 2
715+[error][PID:172799][TID:172875][topo_ascend_cluster_parser.cc][InitDynamicModelData] cannot find device by physical id 0
716+[error][PID:172799][TID:172875][cmd_base_utils.cc][InitHvmCommEnv] [HVM] InitHvmCommEnv failed
717+[error][PID:172799][TID:172875][subcmd_mock_comm.cc][Execute] [HVM] Failed to initialize mock communication environment. Cleaning up environment.
718+```
719+ 
720+**问题现象:** 运行用例前,通过mock-comm命令配置通信域失败。
721+ 
722+**可能原因:** mock-comm命令配置的通信域144配置,超过了工具启动的集群配置。比如工具启动的集群,一个超节点只有2个server,但通信域144表示该超节点下有4个server。
723+ 
724+**解决方法:**
725+ 检查工具启动的集群配置,确认每个超节点下的server数量。若确实需要配置144通信域,则确保工具启动时选择一个更大的集群组网配置。
726+ 
727+**排查步骤:**
728+ 确认工具启动时的集群配置文件和mock-comm命令配置的通信域配置文件。
729+ 
730+**解决方法:**
731+ 确保mock-comm命令配置的通信域,不超过工具启动的集群配置。
732+ 
733+---
734+ 
735+#### FAQ-N004
736+ 
737+**标题:** EndPoint IP 查找失败
738+ 
739+**错误码:** `HCCL_SIM_E_NOT_FOUND` (6)
740+ 
741+**错误函数:** `topo_ascend_cluster_parser.cc::AddLinkInfo()`
742+ 
743+**关键日志:**
744+```
745+cannot find endPoint by ip <IP_ADDR>
746+```
747+ 
748+**问题现象:** 网络链路配置中引用的 IP 地址在拓扑中不存在。
749+ 
750+---
751+ 
752+### 子模块:数据库
753+ 
754+---
755+ 
756+#### FAQ-DB001
757+ 
758+**标题:** SQLite 数据库连接失败
759+ 
760+**错误码:** `HCCL_SIM_E_OPEN_FILE_FAILURE` (10)
761+ 
762+**错误函数:** `db_hccl_db_sqlite.cc::Connect()`
763+ 
764+**关键日志:**
765+```
766+[dbInit] Connect database failed
767+Connect database:<path> failed
768+```
769+ 
770+**问题现象:** 无法连接到 SQLite 数据库文件。
771+ 
772+**可能原因:**
773+1. 数据库文件不存在
774+2. 文件权限不足
775+3. 文件被其他进程锁定
776+ 
777+---
778+ 
779+#### FAQ-DB002
780+ 
781+**标题:** 数据库备份文件未找到
782+ 
783+**错误码:** `HCCL_SIM_E_OPEN_FILE_FAILURE` (10)
784+ 
785+**错误函数:** `sim_loader.cc::BackupDatabase()`
786+ 
787+**关键日志:**
788+```
789+[Loader] Backup database file not found: <dbPath>
790+```
791+ 
792+**问题现象:** Loader 无法找到仿真数据库文件。
793+ 
794+**可能原因:**
795+1. 仿真数据文件路径配置错误
796+2. 仿真数据尚未生成
797+3. 文件权限不足
798+ 
799+**排查步骤:**
800+```bash
801+ls -la <dbPath>
802+```
803+ 
804+---
805+ 
806+#### FAQ-DB003
807+ 
808+**标题:** SQLite 查询失败
809+ 
810+**错误码:** `HCCL_SIM_E_INTERNAL` (4)
811+ 
812+**错误函数:** `db_hccl_db_sqlite.cc`
813+ 
814+**关键日志:**
815+```
816+Prepare failed: <error> sql:<SQL>
817+Step failed: <error>, sql:<SQL>
818+```
819+ 
820+**问题现象:** SQL 查询执行失败。
821+ 
822+**可能原因:**
823+1. 数据库表结构不匹配(版本不兼容)
824+2. 数据库文件损坏
825+3. 磁盘空间不足
826+ 
827+---
828+ 
829+## 模块:插件
830+ 
831+### 子模块:checker
832+ 
833+---
834+ 
835+##### HCCL_SIM_E_INTERNAL (4)
836+ 
837+---
838+ 
839+#### FAQ-C001
840+ 
841+**标题:** 内存切片溢出
842+ 
843+**错误函数:** `task_graph_single_task_check_v3.cc::CheckMemorySlice()`
844+ 
845+**关键日志:**
846+```
847+[TaskGraphSingleTaskCheckV3] Memory slice overflow while accumulating coverage, <detail>
848+[TaskGraphSingleTaskCheckV3] Memory slice overflow, node=<node>, slice=<slice>
849+```
850+ 
851+**问题现象:** 单个任务的内存切片覆盖范围超出了目标buffer的总大小。
852+ 
853+**可能原因:**
854+1. HCCL算法层计算的内存偏移量有误(HCCL业务问题)
855+2. 仿真数据中内存布局信息与任务参数不匹配(工具数据问题)
856+ 
857+**图示说明:**
858+```mermaid
859+graph LR
860+ A[任务内存切片] --> B[目标Buffer]
861+ A -.->|超出边界| C[溢出报错]
862+ style C fill:#f96,stroke:#333
863+```
864+ 
865+---
866+ 
867+#### FAQ-C002
868+ 
869+**标题:** 缓冲区语义不完整
870+ 
871+**错误函数:** `task_graph_semantic_check_v3.cc::CheckBufferContinuity()`
872+ 
873+**关键日志:**
874+```
875+[TaskGraphSemanticCheckV3] Head gap, expect start=0x<ADDR>, actual start=0x<ADDR>
876+[TaskGraphSemanticCheckV3] Middle gap, prev end=0x<ADDR>, cur start=0x<ADDR>
877+[TaskGraphSemanticCheckV3] Tail gap, expect end=0x<ADDR>, actual end=0x<ADDR>
878+```
879+ 
880+**问题现象:** 目标buffer的数据语义覆盖存在空洞。
881+ 
882+**可能原因:**
883+1. HCCL算法遗漏了部分数据区域
884+2. Checker未能正确追踪传输路径
885+ 
886+**图示说明:**
887+```mermaid
888+graph TB
889+ subgraph 目标Buffer
890+ A[地址0x0] --> B[地址0x100]
891+ B -.->|空洞| C[地址0x200]
892+ C --> D[地址0x300]
893+ end
894+ style B fill:#ff9,stroke:#333
895+ style C fill:#ff9,stroke:#333
896+```
897+ 
898+---
899+ 
900+#### FAQ-C003
901+ 
902+**标题:** Reduce语义错误
903+ 
904+**错误函数:** `task_graph_semantic_check_v3.cc::CheckReduceSemantics()`
905+ 
906+**关键日志:**
907+```
908+[TaskGraphSemanticCheckV3] Reduce type mismatch, pair=<pair>
909+[TaskGraphSemanticCheckV3] Duplicate reduce source, pair=<pair>, srcOffset=0x<ADDR>
910+[TaskGraphSemanticCheckV3] Destination reduce semantic incomplete, pair=<pair>
911+```
912+ 
913+**问题现象:** Reduce操作的数据语义校验失败。
914+ 
915+**可能原因:**
916+1. Reduce操作的数据类型不匹配
917+2. 存在重复的reduce源
918+3. 目标buffer未被所有reduce源完全覆盖
919+ 
920+---
921+ 
922+##### HCCL_SIM_E_PARA (1)
923+ 
924+---
925+ 
926+#### FAQ-C004
927+ 
928+**标题:** rankSize 为零
929+ 
930+**错误函数:** `task_graph_semantic_check_v3.cc`
931+ 
932+**关键日志:**
933+```
934+[TaskGraphSemanticCheckV3] rankSize is zero
935+```
936+ 
937+**问题现象:** 语义检查时参与通信的rank数量为0。
938+ 
939+**可能原因:** 通信域未正确初始化,或rank table解析失败。
940+ 
941+---
942+ 
943+#### FAQ-C005
944+ 
945+**标题:** Batch Trans 配对大小不匹配
946+ 
947+**错误函数:** `task_graph_single_task_check_v3.cc::CheckBatchTrans()`
948+ 
949+**关键日志:**
950+```
951+[TaskGraphSingleTaskCheckV3] Batch trans slice length mismatch, node=<node>, label=<label>, index=<N>
952+[TaskGraphSingleTaskCheckV3] Batch trans pair size mismatch, node=<node>, label=<label>
953+```
954+ 
955+**问题现象:** 批量传输操作中,切片长度或配对数量不匹配。
956+ 
957+**可能原因:** AlltoAll等算子的rank间数据分布不均等。
958+ 
959+---
960+ 
961+##### HCCL_SIM_E_NOT_SUPPORT (5)
962+ 
963+---
964+ 
965+#### FAQ-C006
966+ 
967+**标题:** 不支持的内存类型
968+ 
969+**错误函数:** `task_graph_single_task_check_v3.cc`
970+ 
971+**关键日志:**
972+```
973+[TaskGraphSingleTaskCheckV3] Unsupported memory type, node=<node>, slice=<slice>
974+[TaskGraphSingleTaskCheckV3] Invalid memory slice, node=<node>, slice=<slice>
975+```
976+ 
977+**问题现象:** 内存切片的类型不在Checker支持的范围内。
978+ 
979+---
980+ 
981+##### HCCL_SIM_E_OPEN_FILE_FAILURE (10)
982+ 
983+---
984+ 
985+#### FAQ-C007
986+ 
987+**标题:** Dump文件写入失败
988+ 
989+**错误函数:** `dump_manager.cc`, `dump_v3_manager.cc`
990+ 
991+**关键日志:**
992+```
993+[DumpManager::WriteMsgpackFile] failed to open file: <path>
994+[DumpManager::WriteJsonFile] json serialize failed: <error>, file: <path>
995+[DumpV3Manager::WriteMsgpack] failed to open file: <path>
996+```
997+ 
998+**问题现象:** Checker的中间结果dump文件写入失败。
999+ 
1000+**可能原因:**
1001+1. 磁盘空间不足
1002+2. 目录不存在或无写权限
1003+ 
1004+---
1005+ 
1006+#### FAQ-C008
1007+ 
1008+**标题:** 二进制文件magic number不匹配
1009+ 
1010+**错误函数:** `binary_data_operator.cc::FileHeaderRead()`
1011+ 
1012+**关键日志:**
1013+```
1014+[FileHeaderRead] Unmatched magic number:0x<N>≠0x<M>
1015+```
1016+ 
1017+**问题现象:** 读取仿真数据文件时,文件头中的magic number不匹配。
1018+ 
1019+**可能原因:**
1020+1. 数据文件版本与工具版本不兼容
1021+2. 文件损坏
1022+ 
1023+---
1024+ 
1025+### 子模块:runner
1026+ 
1027+---
1028+ 
1029+#### FAQ-R001
1030+ 
1031+**标题:** CCU版本不支持
1032+ 
1033+**错误码:** `HCCL_SIM_E_NOT_SUPPORT` (5)
1034+ 
1035+**错误函数:** `ccu_resource_manager.cc::Init()`
1036+ 
1037+**关键日志:**
1038+```
1039+[CcuResourceManager][Init] ccu version <N> not supported
1040+Invalid ccu version:<version>
1041+```
1042+ 
1043+**问题现象:** CCU执行器不支持当前设备的CCU版本。
1044+ 
1045+**可能原因:** 仿真数据中的CCU版本与工具支持的版本不匹配。
1046+ 
1047+**图示说明:**
1048+```mermaid
1049+graph TD
1050+ A[加载CCU微码] --> B{版本检查}
1051+ B -->|CCU_V1| C[执行V1指令集]
1052+ B -->|CCU_V2| D[执行V2指令集]
1053+ B -->|其他版本| E[报错:版本不支持]
1054+```
1055+ 
1056+---
1057+ 
1058+#### FAQ-R002
1059+ 
1060+**标题:** 数据传输大小为0
1061+ 
1062+**错误码:** `HCCL_SIM_E_PARA` (1)
1063+ 
1064+**错误函数:** `trans_loc_ms_to_loc_mem_executor.cc::Run()`
1065+ 
1066+**关键日志:**
1067+```
1068+[TransLocMSToLocMemExecutor] The size of data transfer is 0.
1069+```
1070+ 
1071+**问题现象:** 数据传输操作的数据量为0。
1072+ 
1073+**可能原因:**
1074+1. 源地址和目的地址相同
1075+2. CCU指令参数计算错误
1076+ 
1077+---
1078+ 
1079+#### FAQ-R003
1080+ 
1081+**标题:** AIV初始化失败
1082+ 
1083+**错误码:** `HCCL_SIM_E_INTERNAL` (4)
1084+ 
1085+**错误函数:** `hccl_task_thread.cc`
1086+ 
1087+**关键日志:**
1088+```
1089+Failed to init AivGraphExecutor, rankId=<N>, streamId=<N>, launchIndex=<N>
1090+AivGraphExecutor failed, rankId=<N>, streamId=<N>, launchIndex=<N>, ret=<N>
1091+```
1092+ 
1093+**问题现象:** AIV图执行器初始化或执行失败。
1094+ 
1095+---
1096+ 
1097+#### FAQ-R004
1098+ 
1099+**标题:** CCU指令执行失败
1100+ 
1101+**错误码:** `HCCL_SIM_E_INTERNAL` (4)
1102+ 
1103+**错误函数:** `ccu_simulator.cc::ExecuteInstr()`
1104+ 
1105+**关键日志:**
1106+```
1107+[CcuSimulator][ExecuteInstr] locCcu[<rank>:<die>], Invalid curInstrId_[<N>], endInstrId[<M>]
1108+```
1109+ 
1110+**问题现象:** CCU微码指令执行时,当前指令ID无效。
1111+ 
1112+---
1113+ 
1114+#### FAQ-R005
1115+ 
1116+**标题:** CCU资源地址越界
1117+ 
1118+**错误码:** `HCCL_SIM_E_PARA` (1)
1119+ 
1120+**错误函数:** `ccu_resource_manager.cc`
1121+ 
1122+**关键日志:**
1123+```
1124+dieId is out of range, dieId=[<N>]
1125+msAddr is out of range, msAddr=[<N>],addr=[<N>]
1126+```
1127+ 
1128+**问题现象:** CCU资源访问时地址或ID越界。
1129+ 
1130+---
1131+ 
1132+#### FAQ-R006
1133+ 
1134+**标题:** 内存搬运地址无效
1135+ 
1136+**错误码:** `HCCL_SIM_E_MEMORY` (3)
1137+ 
1138+**错误函数:** `ccu_resource_manager.cc`
1139+ 
1140+**关键日志:**
1141+```
1142+[TransMemToMem] 无法获取srcBuf的设备地址(addr= 0x<ADDR>)!
1143+[TransMemToMem] 无法获取dstBuf的设备地址(addr= 0x<ADDR>)!
1144+```
1145+ 
1146+**问题现象:** CCU执行内存搬运时,源或目标地址无法在设备内存表中找到。
1147+ 
1148+---
1149+ 
1150+### 子模块:insight
1151+ 
1152+---
1153+ 
1154+#### FAQ-I001
1155+ 
1156+**标题:** Insight前端页面打不开
1157+ 
1158+**关键日志:**
1159+```
1160+[Insight] Failed to start web server on port <N>
1161+[Insight] Address already in use
1162+```
1163+ 
1164+**问题现象:** Insight可视化界面无法访问。
1165+ 
1166+**可能原因:**
1167+1. 端口被占用
1168+2. 网络配置问题
1169+3. Python依赖未安装
1170+ 
1171+**排查步骤:**
1172+```bash
1173+netstat -tlnp | grep <port>
1174+```
1175+ 
1176+---
1177+ 
1178+#### FAQ-I002
1179+ 
1180+**标题:** Insight数据加载异常
1181+ 
1182+**关键日志:**
1183+```
1184+[Insight] Failed to load data from SQLite
1185+[Insight] JSON parse error: <error>
1186+```
1187+ 
1188+**问题现象:** Insight面板中数据无法正常显示。
1189+ 
1190+**可能原因:**
1191+1. 数据库连接失败
1192+2. 数据文件损坏
1193+3. JSON格式错误
1194+ 
1195+---
1196+ 
1197+#### FAQ-I003
1198+ 
1199+**标题:** DAG图渲染问题
1200+ 
1201+**关键日志:**
1202+```
1203+[Insight] DAG render failed: <error>
1204+[Insight] Node count exceeds limit: <N>
1205+```
1206+ 
1207+**问题现象:** 任务图无法正常渲染或显示不全。
1208+ 
1209+**可能原因:**
1210+1. 任务节点数量过多
1211+2. 图布局算法超时
1212+3. 浏览器内存不足
1213+ 
1214+---
1215+ 
1216+## 附录:错误码速查表
1217+ 
1218+| 错误码 | 枚举值 | 含义 |
1219+|--------|--------|------|
1220+| 0 | HCCL_SIM_SUCCESS | 成功 |
1221+| 1 | HCCL_SIM_E_PARA | 参数错误 |
1222+| 2 | HCCL_SIM_E_PTR | 空指针 |
1223+| 3 | HCCL_SIM_E_MEMORY | 内存错误 |
1224+| 4 | HCCL_SIM_E_INTERNAL | 内部错误 |
1225+| 5 | HCCL_SIM_E_NOT_SUPPORT | 不支持的特性 |
1226+| 6 | HCCL_SIM_E_NOT_FOUND | 资源未找到 |
1227+| 8 | HCCL_SIM_E_SYSCALL | 系统调用错误 |
1228+| 9 | HCCL_SIM_E_TIMEOUT | 超时 |
1229+| 10 | HCCL_SIM_E_OPEN_FILE_FAILURE | 文件打开失败 |
Atest/hccl_vm/docs/faq/generate_faq_html.py+605-0
@@ -0,0 +1,605 @@
1+#!/usr/bin/env python3
2+"""
3+FAQ Markdown 转 HTML 转换脚本
4+ 
5+功能:
6+1. 解析 Markdown FAQ 文档,提取结构化数据(支持模块→子模块→错误码分组的分层结构)
7+2. 生成 JSON 数据
8+3. 将 JSON 嵌入到 HTML 模板中,生成最终的 index.html
9+ 
10+用法:
11+ python3 scripts/generate_faq_html.py docs/faq/test_faq.md docs/faq/index.html
12+"""
13+ 
14+import re
15+import sys
16+import json
17+from pathlib import Path
18+from typing import List, Dict
19+ 
20+ 
21+def parse_faq_markdown(md_content: str) -> List[Dict]:
22+ """解析Markdown FAQ文档,提取结构化数据"""
23+ faq_items = []
24+ 
25+ lines = md_content.split('\n')
26+ current_module = ''
27+ current_submodule = ''
28+ current_error_code_group = ''
29+ 
30+ faq_blocks = re.split(r'\n#### FAQ-', md_content)
31+ 
32+ for i, block in enumerate(faq_blocks[1:]):
33+ faq_id_match = re.match(r'([A-Z]+\d+)', block)
34+ if not faq_id_match:
35+ continue
36+ 
37+ faq_id = faq_id_match.group(1)
38+ 
39+ original_block = '\n#### FAQ-' + block
40+ current_module, current_submodule, current_error_code_group = find_context(md_content, original_block)
41+ 
42+ faq_item = parse_faq_block(faq_id, block)
43+ if faq_item:
44+ faq_item['module'] = current_module
45+ faq_item['submodule'] = current_submodule
46+ faq_item['errorCodeGroup'] = current_error_code_group
47+ faq_items.append(faq_item)
48+ 
49+ return faq_items
50+ 
51+ 
52+def find_context(md_content: str, block: str) -> tuple:
53+ """查找FAQ块在文档中的上下文(模块、子模块、错误码分组)"""
54+ block_start = md_content.find(block)
55+ if block_start == -1:
56+ return '', '', ''
57+ 
58+ context_text = md_content[:block_start]
59+ lines = context_text.split('\n')
60+ 
61+ module = ''
62+ submodule = ''
63+ error_code_group = ''
64+ 
65+ for line in reversed(lines):
66+ line = line.strip()
67+ if line.startswith('##### ') and not error_code_group and not submodule:
68+ match = re.match(r'#####\s*([A-Z_]+)\s*\(\d+\)', line)
69+ if match:
70+ error_code_group = match.group(1)
71+ elif line.startswith('### 子模块:') and not submodule:
72+ submodule = line.replace('### 子模块:', '').strip()
73+ elif line.startswith('## 模块:') and not module:
74+ module = line.replace('## 模块:', '').strip()
75+ break
76+ 
77+ return module, submodule, error_code_group
78+ 
79+ 
80+def parse_faq_block(faq_id: str, block: str) -> Dict:
81+ """解析单个FAQ块"""
82+ faq_item = {
83+ 'id': f'FAQ-{faq_id}',
84+ 'module': '',
85+ 'submodule': '',
86+ 'errorCodeGroup': '',
87+ 'title': extract_field(block, '标题'),
88+ 'errorCode': extract_field(block, '错误码'),
89+ 'errorFunctions': extract_list_field(block, '错误函数'),
90+ 'keywords': [],
91+ 'logPatterns': extract_code_block(block, '关键日志'),
92+ 'symptom': extract_field(block, '问题现象'),
93+ 'reasons': extract_numbered_list(block, '可能原因'),
94+ 'steps': extract_code_block(block, '排查步骤'),
95+ 'solution': extract_code_block(block, '解决方案'),
96+ 'diagram': '',
97+ 'diagramType': ''
98+ }
99+ 
100+ if faq_item['errorCode']:
101+ code_match = re.search(r'HCCL_SIM_\w+', faq_item['errorCode'])
102+ if code_match:
103+ faq_item['keywords'].append(code_match.group())
104+ 
105+ if faq_item['logPatterns']:
106+ keywords = extract_keywords_from_log(faq_item['logPatterns'][0])
107+ faq_item['keywords'].extend(keywords)
108+ 
109+ diagram_info = extract_diagram(block)
110+ faq_item['diagram'] = diagram_info['content']
111+ faq_item['diagramType'] = diagram_info['type']
112+ 
113+ return faq_item
114+ 
115+ 
116+def extract_field(block: str, field_name: str) -> str:
117+ """提取字段值"""
118+ pattern = rf'\*\*{field_name}:\*\*\s*(.+)'
119+ match = re.search(pattern, block)
120+ return match.group(1).strip() if match else ''
121+ 
122+ 
123+def extract_list_field(block: str, field_name: str) -> List[str]:
124+ """提取逗号分隔的列表字段"""
125+ value = extract_field(block, field_name)
126+ if not value:
127+ return []
128+ return [v.strip() for v in value.split(',') if v.strip()]
129+ 
130+ 
131+def extract_code_block(block: str, field_name: str) -> List[str]:
132+ """提取代码块"""
133+ pattern = rf'\*\*{field_name}:\*\*\s*\n```.*?\n(.*?)```'
134+ matches = re.findall(pattern, block, re.DOTALL)
135+ return [m.strip() for m in matches] if matches else []
136+ 
137+ 
138+def extract_numbered_list(block: str, field_name: str) -> List[str]:
139+ """提取编号列表"""
140+ pattern = rf'\*\*{field_name}:\*\*\s*\n((?:\d+\..+\n)+)'
141+ match = re.search(pattern, block)
142+ if not match:
143+ return []
144+ 
145+ list_text = match.group(1)
146+ items = re.findall(r'\d+\.\s*(.+)', list_text)
147+ return [item.strip() for item in items]
148+ 
149+ 
150+def extract_keywords_from_log(log_content: str) -> List[str]:
151+ """从日志中提取关键词"""
152+ keywords = []
153+ bracket_keywords = re.findall(r'\[([^\]]+)\]', log_content)
154+ keywords.extend(bracket_keywords[:3])
155+ 
156+ env_vars = re.findall(r'\b[A-Z_]{5,}\b', log_content)
157+ keywords.extend([v for v in env_vars if 'HCCL' in v or 'RANK' in v])
158+ 
159+ return keywords[:5]
160+ 
161+ 
162+def extract_diagram(block: str) -> Dict:
163+ """提取图示信息"""
164+ mermaid_pattern = r'\*\*图示说明:\*\*\s*\n```mermaid\s*\n(.*?)```'
165+ mermaid_match = re.search(mermaid_pattern, block, re.DOTALL)
166+ if mermaid_match:
167+ return {'content': mermaid_match.group(1).strip(), 'type': 'mermaid'}
168+ 
169+ image_pattern = r'\*\*图示说明:\*\*\s*\n!\[.*?\]\((.*?)\)'
170+ image_match = re.search(image_pattern, block)
171+ if image_match:
172+ return {'content': image_match.group(1), 'type': 'image'}
173+ 
174+ return {'content': '', 'type': ''}
175+ 
176+ 
177+def generate_html(faq_data: List[Dict], modules: List[str], submodules: Dict[str, List[str]], error_codes: List[str]) -> str:
178+ """生成完整的HTML文件内容"""
179+ html_template = get_html_template()
180+ 
181+ json_data = json.dumps(faq_data, ensure_ascii=False, indent=2)
182+ modules_json = json.dumps(modules, ensure_ascii=False)
183+ submodules_json = json.dumps(submodules, ensure_ascii=False)
184+ codes_json = json.dumps(error_codes, ensure_ascii=False)
185+ 
186+ html_content = html_template.replace('{{FAQ_DATA}}', json_data)
187+ html_content = html_content.replace('{{MODULES}}', modules_json)
188+ html_content = html_content.replace('{{SUBMODULES}}', submodules_json)
189+ html_content = html_content.replace('{{ERROR_CODES}}', codes_json)
190+ 
191+ return html_content
192+ 
193+ 
194+def get_html_template() -> str:
195+ """获取HTML模板"""
196+ return '''<!DOCTYPE html>
197+<html lang="zh-CN">
198+<head>
199+ <meta charset="UTF-8">
200+ <meta name="viewport" content="width=device-width, initial-scale=1.0">
201+ <title>HCCL-VM FAQ 查询系统</title>
202+ <script src="https://cdn.jsdelivr.net/npm/mermaid@10/dist/mermaid.min.js"></script>
203+ <style>
204+ * { margin: 0; padding: 0; box-sizing: border-box; }
205+ body {
206+ font-family: -apple-system, BlinkMacSystemFont, "Segoe UI", Roboto, sans-serif;
207+ background: linear-gradient(135deg, #667eea 0%, #764ba2 100%);
208+ min-height: 100vh; padding: 20px;
209+ }
210+ .container {
211+ max-width: 1200px; margin: 0 auto;
212+ background: #fff; border-radius: 12px;
213+ box-shadow: 0 20px 60px rgba(0,0,0,0.3); overflow: hidden;
214+ }
215+ .header {
216+ background: linear-gradient(90deg, #2c3e50, #3498db);
217+ color: white; padding: 30px; text-align: center;
218+ }
219+ .header h1 { font-size: 28px; margin-bottom: 10px; }
220+ .header p { opacity: 0.9; font-size: 14px; }
221+ .search-panel { background: #f8f9fa; padding: 25px; border-bottom: 1px solid #dee2e6; }
222+ .search-row { display: flex; gap: 15px; margin-bottom: 15px; flex-wrap: wrap; }
223+ .search-group { flex: 1; min-width: 180px; }
224+ .search-group label { display: block; font-weight: 600; margin-bottom: 8px; color: #2c3e50; }
225+ .search-group select, .search-group input {
226+ width: 100%; padding: 10px 15px; border: 2px solid #e0e0e0;
227+ border-radius: 8px; font-size: 14px; transition: all 0.3s;
228+ }
229+ .search-group select:focus, .search-group input:focus {
230+ border-color: #3498db; outline: none;
231+ box-shadow: 0 0 0 3px rgba(52, 152, 219, 0.1);
232+ }
233+ .btn-row { display: flex; gap: 10px; justify-content: center; }
234+ .btn { padding: 12px 30px; border: none; border-radius: 8px;
235+ font-size: 14px; font-weight: 600; cursor: pointer; transition: all 0.3s; }
236+ .btn-primary { background: #3498db; color: white; }
237+ .btn-primary:hover { background: #2980b9; transform: translateY(-2px); }
238+ .btn-secondary { background: #95a5a6; color: white; }
239+ .btn-secondary:hover { background: #7f8c8d; }
240+ .stats-bar { background: #ecf0f1; padding: 15px 25px;
241+ display: flex; justify-content: space-between; align-items: center;
242+ font-size: 13px; color: #7f8c8d; }
243+ .results-container { padding: 25px; min-height: 400px; }
244+ .no-results { text-align: center; padding: 60px; color: #95a5a6; }
245+ .no-results h3 { font-size: 20px; margin-bottom: 10px; }
246+ .faq-item { background: #fff; border: 1px solid #e0e0e0; border-radius: 10px;
247+ margin-bottom: 20px; overflow: hidden; transition: all 0.3s; }
248+ .faq-item:hover { box-shadow: 0 5px 15px rgba(0,0,0,0.1); border-color: #3498db; }
249+ .faq-header { background: linear-gradient(90deg, #f8f9fa, #e9ecef); padding: 15px 20px;
250+ cursor: pointer; display: flex; justify-content: space-between; align-items: center; flex-wrap: wrap; gap: 8px; }
251+ .faq-header:hover { background: linear-gradient(90deg, #ecf0f1, #d5dbdb); }
252+ .faq-id { font-weight: 700; color: #3498db; font-size: 14px; }
253+ .faq-title { flex: 1; margin-left: 15px; font-weight: 600; color: #2c3e50; min-width: 200px; }
254+ .faq-tags { display: flex; gap: 8px; flex-wrap: wrap; }
255+ .tag { padding: 4px 10px; border-radius: 12px; font-size: 12px; font-weight: 500; }
256+ .tag-module { background: #e8f5e9; color: #2e7d32; }
257+ .tag-submodule { background: #e3f2fd; color: #1565c0; }
258+ .tag-code { background: #fff3e0; color: #e65100; }
259+ .faq-content { padding: 20px; display: none; }
260+ .faq-content.active { display: block; }
261+ .faq-section { margin-bottom: 15px; }
262+ .faq-section h4 { font-size: 14px; color: #34495e; margin-bottom: 8px; font-weight: 600; }
263+ .faq-section p, .faq-section ul { font-size: 13px; color: #555; line-height: 1.6; }
264+ .faq-section ul { padding-left: 20px; }
265+ .faq-section li { margin-bottom: 5px; }
266+ .code-block { background: #2d3436; color: #dfe6e9; padding: 15px; border-radius: 6px;
267+ font-family: "Consolas", "Monaco", monospace; font-size: 13px;
268+ overflow-x: auto; white-space: pre-wrap; }
269+ .diagram-container { background: #f8f9fa; padding: 15px; border-radius: 6px;
270+ margin-top: 10px; text-align: center; }
271+ .diagram-container img { max-width: 100%; border-radius: 6px; }
272+ .expand-icon { font-size: 18px; color: #95a5a6; transition: transform 0.3s; }
273+ .expand-icon.rotated { transform: rotate(180deg); }
274+ .highlight { background: #fff176; padding: 2px 4px; border-radius: 3px; }
275+ .module-group-header {
276+ background: linear-gradient(90deg, #2c3e50, #3498db); color: white;
277+ padding: 14px 20px; font-size: 17px; font-weight: 700;
278+ margin: 25px 0 12px 0; border-radius: 8px;
279+ display: flex; align-items: center; gap: 10px;
280+ }
281+ .module-group-header:first-child { margin-top: 0; }
282+ .module-count {
283+ background: rgba(255,255,255,0.25); padding: 2px 10px;
284+ border-radius: 10px; font-size: 12px; font-weight: 500;
285+ }
286+ .submodule-group-header {
287+ background: #f0f4f8; border-left: 4px solid #3498db;
288+ padding: 10px 16px; margin: 16px 0 8px 8px; border-radius: 0 6px 6px 0;
289+ cursor: pointer; display: flex; align-items: center;
290+ gap: 8px; transition: background 0.2s; user-select: none;
291+ }
292+ .submodule-group-header:hover { background: #e2e8f0; }
293+ .submodule-group-header .sub-title { font-size: 15px; font-weight: 600; color: #2c3e50; }
294+ .submodule-group-header .sub-count {
295+ background: #3498db; color: white; padding: 2px 8px;
296+ border-radius: 10px; font-size: 11px; font-weight: 500;
297+ }
298+ .submodule-group-header .sub-toggle { color: #7f8c8d; font-size: 12px; margin-left: auto; }
299+ .submodule-group.collapsed .faq-item { display: none; }
300+ .submodule-group.collapsed .submodule-group-header { opacity: 0.65; }
301+ </style>
302+</head>
303+<body>
304+ <div class="container">
305+ <div class="header">
306+ <h1>HCCL-VM 常见问题查询系统</h1>
307+ <p>按模块、子模块、错误码、关键词快速定位问题 | 支持模糊匹配</p>
308+ </div>
309+ <div class="search-panel">
310+ <div class="search-row">
311+ <div class="search-group">
312+ <label>按模块筛选</label>
313+ <select id="moduleFilter" onchange="onModuleChange()"><option value="">全部模块</option></select>
314+ </div>
315+ <div class="search-group">
316+ <label>按子模块筛选</label>
317+ <select id="submoduleFilter"><option value="">全部子模块</option></select>
318+ </div>
319+ <div class="search-group">
320+ <label>按错误码筛选</label>
321+ <select id="errorCodeFilter"><option value="">全部错误码</option></select>
322+ </div>
323+ </div>
324+ <div class="search-row">
325+ <div class="search-group" style="flex: 2;">
326+ <label>关键词搜索</label>
327+ <input type="text" id="keywordInput"
328+ placeholder="输入关键词:函数名、日志内容、问题描述...">
329+ </div>
330+ </div>
331+ <div class="btn-row">
332+ <button class="btn btn-primary" onclick="searchFAQ()">🔍 搜索</button>
333+ <button class="btn btn-secondary" onclick="clearFilters()">🔄 清空</button>
334+ </div>
335+ </div>
336+ <div class="stats-bar">
337+ <span id="statsInfo">共 <strong>0</strong> 条FAQ</span>
338+ <span id="matchInfo"></span>
339+ </div>
340+ <div class="results-container" id="resultsContainer">
341+ <div class="no-results">
342+ <h3>请输入搜索条件</h3>
343+ <p>选择模块、子模块或错误码,或输入关键词进行搜索</p>
344+ </div>
345+ </div>
346+ </div>
347+ 
348+ <script>
349+ const faqData = {{FAQ_DATA}};
350+ const modules = {{MODULES}};
351+ const submodules = {{SUBMODULES}};
352+ const errorCodes = {{ERROR_CODES}};
353+ let currentKeyword = '';
354+ 
355+ mermaid.initialize({ startOnLoad: false, theme: 'neutral' });
356+ 
357+ function initModuleFilter() {
358+ const select = document.getElementById('moduleFilter');
359+ modules.forEach(m => {
360+ const option = document.createElement('option');
361+ option.value = m; option.textContent = m;
362+ select.appendChild(option);
363+ });
364+ }
365+ 
366+ function initSubmoduleFilter(moduleName) {
367+ const select = document.getElementById('submoduleFilter');
368+ select.innerHTML = '<option value="">全部子模块</option>';
369+ if (!moduleName || !submodules[moduleName]) return;
370+ submodules[moduleName].forEach(sm => {
371+ const option = document.createElement('option');
372+ option.value = sm; option.textContent = sm;
373+ select.appendChild(option);
374+ });
375+ }
376+ 
377+ function initErrorCodeFilter() {
378+ const select = document.getElementById('errorCodeFilter');
379+ errorCodes.forEach(c => {
380+ const option = document.createElement('option');
381+ option.value = c; option.textContent = c;
382+ select.appendChild(option);
383+ });
384+ }
385+ 
386+ function onModuleChange() {
387+ const moduleName = document.getElementById('moduleFilter').value;
388+ initSubmoduleFilter(moduleName);
389+ document.getElementById('submoduleFilter').value = '';
390+ searchFAQ();
391+ }
392+ 
393+ function searchFAQ() {
394+ const moduleFilter = document.getElementById('moduleFilter').value;
395+ const submoduleFilter = document.getElementById('submoduleFilter').value;
396+ const errorCodeFilter = document.getElementById('errorCodeFilter').value;
397+ const keyword = document.getElementById('keywordInput').value.trim().toLowerCase();
398+ currentKeyword = keyword;
399+ 
400+ let results = faqData.filter(item => {
401+ if (moduleFilter && item.module !== moduleFilter) return false;
402+ if (submoduleFilter && item.submodule !== submoduleFilter) return false;
403+ if (errorCodeFilter) {
404+ if (item.errorCode && item.errorCode.includes(errorCodeFilter)) return true;
405+ if (item.errorCodeGroup && item.errorCodeGroup === errorCodeFilter) return true;
406+ return false;
407+ }
408+ if (keyword) {
409+ const searchFields = [
410+ item.title, item.errorCode, item.errorCodeGroup, ...item.errorFunctions,
411+ ...item.keywords, ...item.logPatterns, item.symptom,
412+ ...item.reasons, item.solution, item.id, item.module, item.submodule
413+ ].join(' ').toLowerCase();
414+ return searchFields.includes(keyword);
415+ }
416+ return true;
417+ });
418+ displayResults(results);
419+ }
420+ 
421+ function displayResults(results) {
422+ const container = document.getElementById('resultsContainer');
423+ document.getElementById('statsInfo').innerHTML = `共 <strong>${faqData.length}</strong> 条FAQ`;
424+ document.getElementById('matchInfo').innerHTML = results.length > 0 ?
425+ `匹配到 <strong>${results.length}</strong> 条结果` : '';
426+ 
427+ if (results.length === 0) {
428+ container.innerHTML = '<div class="no-results"><h3>未找到匹配结果</h3><p>请调整筛选条件</p></div>';
429+ return;
430+ }
431+ 
432+ // 按模块 → 子模块分组
433+ const grouped = {};
434+ results.forEach(item => {
435+ const mod = item.module || '未知模块';
436+ const sub = item.submodule || '其他';
437+ if (!grouped[mod]) grouped[mod] = {};
438+ if (!grouped[mod][sub]) grouped[mod][sub] = [];
439+ grouped[mod][sub].push(item);
440+ });
441+ 
442+ let html = '';
443+ const moduleOrder = modules.filter(m => grouped[m]);
444+ Object.keys(grouped).filter(m => !modules.includes(m)).forEach(m => moduleOrder.push(m));
445+ 
446+ moduleOrder.forEach(mod => {
447+ const moduleCount = Object.values(grouped[mod]).reduce((sum, arr) => sum + arr.length, 0);
448+ html += `<div class="module-group-header">
449+ <span>${mod}</span>
450+ <span class="module-count">${moduleCount} 条</span>
451+ </div>`;
452+ 
453+ const submoduleOrder = (submodules[mod] || []).filter(sm => grouped[mod][sm]);
454+ Object.keys(grouped[mod]).filter(sm => !(submodules[mod] || []).includes(sm)).forEach(sm => submoduleOrder.push(sm));
455+ 
456+ submoduleOrder.forEach(sub => {
457+ const items = grouped[mod][sub];
458+ const groupId = `group-${mod}-${sub}`.replace(/[\\s()()]/g, '_');
459+ html += `<div class="submodule-group" id="${groupId}">
460+ <div class="submodule-group-header" onclick="toggleSubmoduleGroup('${groupId}')">
461+ <span class="sub-title">${sub}</span>
462+ <span class="sub-count">${items.length} 条</span>
463+ <span class="sub-toggle">收起/展开</span>
464+ </div>`;
465+ 
466+ items.forEach(item => {
467+ html += `<div class="faq-item">
468+ <div class="faq-header" onclick="toggleFAQ('${item.id}')">
469+ <span class="faq-id">${item.id}</span>
470+ <span class="faq-title">${highlightText(item.title)}</span>
471+ <div class="faq-tags">
472+ ${item.errorCode ? `<span class="tag tag-code">${extractErrorCode(item.errorCode)}</span>` : (item.errorCodeGroup ? `<span class="tag tag-code">${item.errorCodeGroup}</span>` : '')}
473+ </div>
474+ <span class="expand-icon" id="icon-${item.id}">▼</span>
475+ </div>
476+ <div class="faq-content" id="content-${item.id}">${renderFAQContent(item)}</div>
477+ </div>`;
478+ });
479+ html += '</div>';
480+ });
481+ });
482+ container.innerHTML = html;
483+ 
484+ setTimeout(() => {
485+ document.querySelectorAll('.mermaid').forEach(el => mermaid.run({ nodes: [el] }));
486+ }, 100);
487+ }
488+ 
489+ function toggleSubmoduleGroup(groupId) {
490+ const group = document.getElementById(groupId);
491+ if (group) group.classList.toggle('collapsed');
492+ }
493+ 
494+ function renderFAQContent(item) {
495+ let html = '';
496+ if (item.module) html += `<div class="faq-section"><h4>所属模块</h4><p><strong>${item.module}</strong>${item.submodule ? ' → ' + item.submodule : ''}</p></div>`;
497+ if (item.errorCode) html += `<div class="faq-section"><h4>错误码</h4><p>${item.errorCode}</p></div>`;
498+ if (item.errorFunctions && item.errorFunctions.length > 0)
499+ html += `<div class="faq-section"><h4>错误函数</h4><ul>${item.errorFunctions.map(f => `<li>${f}</li>`).join('')}</ul></div>`;
500+ if (item.logPatterns && item.logPatterns.length > 0)
501+ html += `<div class="faq-section"><h4>关键日志</h4><div class="code-block">${highlightText(item.logPatterns[0])}</div></div>`;
502+ if (item.symptom) html += `<div class="faq-section"><h4>问题现象</h4><p>${highlightText(item.symptom)}</p></div>`;
503+ if (item.reasons && item.reasons.length > 0)
504+ html += `<div class="faq-section"><h4>可能原因</h4><ul>${item.reasons.map(r => `<li>${highlightText(r)}</li>`).join('')}</ul></div>`;
505+ if (item.steps) html += `<div class="faq-section"><h4>排查步骤</h4><div class="code-block">${item.steps}</div></div>`;
506+ if (item.solution) html += `<div class="faq-section"><h4>解决方案</h4><div class="code-block">${item.solution}</div></div>`;
507+ if (item.diagram)
508+ html += `<div class="faq-section"><h4>图示说明</h4><div class="diagram-container">${item.diagramType === 'mermaid' ? `<pre class="mermaid">${item.diagram}</pre>` : `<img src="${item.diagram}" alt="示意图">`}</div></div>`;
509+ return html;
510+ }
511+ 
512+ function toggleFAQ(id) {
513+ const content = document.getElementById('content-' + id);
514+ const icon = document.getElementById('icon-' + id);
515+ content.classList.toggle('active');
516+ icon.classList.toggle('rotated');
517+ }
518+ 
519+ function highlightText(text) {
520+ if (!currentKeyword || !text) return text;
521+ const regex = new RegExp(currentKeyword, 'gi');
522+ return text.replace(regex, match => `<span class="highlight">${match}</span>`);
523+ }
524+ 
525+ function extractErrorCode(codeStr) {
526+ const match = codeStr.match(/HCCL_SIM_\w+/);
527+ return match ? match[0] : codeStr;
528+ }
529+ 
530+ function clearFilters() {
531+ document.getElementById('moduleFilter').value = '';
532+ document.getElementById('submoduleFilter').innerHTML = '<option value="">全部子模块</option>';
533+ document.getElementById('errorCodeFilter').value = '';
534+ document.getElementById('keywordInput').value = '';
535+ currentKeyword = '';
536+ displayResults(faqData);
537+ }
538+ 
539+ window.onload = function() {
540+ initModuleFilter(); initErrorCodeFilter(); displayResults(faqData);
541+ };
542+ 
543+ document.getElementById('keywordInput').addEventListener('keypress', function(e) {
544+ if (e.key === 'Enter') searchFAQ();
545+ });
546+ </script>
547+</body>
548+</html>'''
549+ 
550+ 
551+def main():
552+ if len(sys.argv) < 3:
553+ print("用法: python3 scripts/generate_faq_html.py <input_md> <output_html>")
554+ print("示例: python3 scripts/generate_faq_html.py docs/faq/test_faq.md docs/faq/index.html")
555+ sys.exit(1)
556+ 
557+ input_md = sys.argv[1]
558+ output_html = sys.argv[2]
559+ 
560+ if not Path(input_md).exists():
561+ print(f"❌ 输入文件不存在: {input_md}")
562+ sys.exit(1)
563+ 
564+ output_dir = Path(output_html).parent
565+ output_dir.mkdir(parents=True, exist_ok=True)
566+ 
567+ print(f"📖 解析: {input_md}")
568+ md_content = Path(input_md).read_text(encoding='utf-8')
569+ 
570+ faq_items = parse_faq_markdown(md_content)
571+ 
572+ print(f" 提取到 {len(faq_items)} 条FAQ")
573+ 
574+ modules = sorted(set(item['module'] for item in faq_items if item['module']))
575+ 
576+ submodules = {}
577+ for item in faq_items:
578+ if item['module'] and item['submodule']:
579+ if item['module'] not in submodules:
580+ submodules[item['module']] = []
581+ if item['submodule'] not in submodules[item['module']]:
582+ submodules[item['module']].append(item['submodule'])
583+ for key in submodules:
584+ submodules[key] = sorted(submodules[key])
585+ 
586+ error_codes = []
587+ for item in faq_items:
588+ if item['errorCode']:
589+ match = re.search(r'HCCL_SIM_\w+', item['errorCode'])
590+ if match and match.group() not in error_codes:
591+ error_codes.append(match.group())
592+ if item['errorCodeGroup'] and item['errorCodeGroup'] not in error_codes:
593+ error_codes.append(item['errorCodeGroup'])
594+ error_codes = sorted(error_codes)
595+ 
596+ print(f"🔧 生成HTML...")
597+ html_content = generate_html(faq_items, modules, submodules, error_codes)
598+ Path(output_html).write_text(html_content, encoding='utf-8')
599+ 
600+ print(f"✅ 完成: {output_html}")
601+ print(f"\n请在浏览器中打开查看效果")
602+ 
603+ 
604+if __name__ == '__main__':
605+ main()
Atest/hccl_vm/docs/faq/index.html+1600-0
Mtest/hccl_vm/docs/hccl_simulator.md+780-804文件内容审核中,请稍后刷新重试
Mtest/hccl_vm/docs/hccl_vm_binary_file_format.md+21-21
@@ -21,7 +21,7 @@
21| 2 | `{dataId}_hcclvm_instr_data.bin` | `0x434D4349` | 微码指令数据 |21| 2 | `{dataId}_hcclvm_instr_data.bin` | `0x434D4349` | 微码指令数据 |
22| 3 | `{dataId}_hcclvm_task_data.bin` | `0x48565444` | 任务元数据 |22| 3 | `{dataId}_hcclvm_task_data.bin` | `0x48565444` | 任务元数据 |
23 23 
24-> **说明**:`{dataId}` 为时间戳格式的唯一标识,例如 `20250115_143052_3847`24+> **说明**:`{dataId}` 为时间戳格式的唯一标识,例如 `20250115_143052_3847`
25 25 
26---26---
27 27 
@@ -79,7 +79,7 @@ struct FileHeader {
79 79 
80#### 3.2.1 ModelInfoCommInner80#### 3.2.1 ModelInfoCommInner
81 81 
82-**大小**:36 字节82+**大小**:36 字节
83 83 
84| 偏移量 | 字节数 | 类型 | 字段名 | 说明 |84| 偏移量 | 字节数 | 类型 | 字段名 | 说明 |
85|:------:|:------:|---------|-------------------|------------------|85|:------:|:------:|---------|-------------------|------------------|
@@ -98,9 +98,9 @@ struct FileHeader {
98 98 
99#### 3.2.2 VDataDesTag99#### 3.2.2 VDataDesTag
100 100 
101-**用途**:ReduceScatterV / AllGatherV 操作101+**用途**:ReduceScatterV / AllGatherV 操作
102 102 
103-**大小**:变长103+**大小**:变长
104 104 
105| 偏移量 | 字节数 | 类型 | 字段名 | 说明 |105| 偏移量 | 字节数 | 类型 | 字段名 | 说明 |
106|:-------:|:----------:|----------|---------|--------------------|106|:-------:|:----------:|----------|---------|--------------------|
@@ -111,9 +111,9 @@ struct FileHeader {
111 111 
112#### 3.2.3 All2AllDataDesTag112#### 3.2.3 All2AllDataDesTag
113 113 
114-**用途**:All2All / All2AllV 操作114+**用途**:All2All / All2AllV 操作
115 115 
116-**大小**:变长116+**大小**:变长
117 117 
118| 偏移量 | 字节数 | 类型 | 字段名 | 说明 |118| 偏移量 | 字节数 | 类型 | 字段名 | 说明 |
119|:------:|:----------:|----------|-----------------|------------------------|119|:------:|:----------:|----------|-----------------|------------------------|
@@ -137,7 +137,7 @@ struct FileHeader {
137 137 
138#### 3.3.2 ChannelData138#### 3.3.2 ChannelData
139 139 
140-**大小**:152 字节140+**大小**:152 字节
141 141 
142| 偏移量 | 字节数 | 类型 | 字段名 | 说明 |142| 偏移量 | 字节数 | 类型 | 字段名 | 说明 |
143|:------:|:------:|------------|-----------|---------------|143|:------:|:------:|------------|-----------|---------------|
@@ -165,7 +165,7 @@ struct FileHeader {
165 165 
166#### 3.4.2 JettyData166#### 3.4.2 JettyData
167 167 
168-**大小**:48 字节168+**大小**:48 字节
169 169 
170| 偏移量 | 字节数 | 类型 | 字段名 | 说明 |170| 偏移量 | 字节数 | 类型 | 字段名 | 说明 |
171|:------:|:------:|-----------|----------|--------------|171|:------:|:------:|-----------|----------|--------------|
@@ -189,7 +189,7 @@ struct FileHeader {
189 189 
190#### 3.5.2 MemLayoutData190#### 3.5.2 MemLayoutData
191 191 
192-**大小**:32 字节192+**大小**:32 字节
193 193 
194| 偏移量 | 字节数 | 类型 | 字段名 | 说明 |194| 偏移量 | 字节数 | 类型 | 字段名 | 说明 |
195|:------:|:------:|--------|--------------|--------------------|195|:------:|:------:|--------|--------------|--------------------|
@@ -213,7 +213,7 @@ struct FileHeader {
213 213 
214### 4.2 MicrocodeInstrDesc214### 4.2 MicrocodeInstrDesc
215 215 
216-**大小**:8 字节216+**大小**:8 字节
217 217 
218| 偏移量 | 字节数 | 类型 | 字段名 | 说明 |218| 偏移量 | 字节数 | 类型 | 字段名 | 说明 |
219|:------:|:------:|--------|----------|----------|219|:------:|:------:|--------|----------|----------|
@@ -259,9 +259,9 @@ struct FileHeader {
259 259 
260#### 5.3.1 TransMemTask260#### 5.3.1 TransMemTask
261 261 
262-**适用任务类型**:`MEM_CPY (3)`262+**适用任务类型**:`MEM_CPY (3)`
263 263 
264-**大小**:33 字节264+**大小**:33 字节
265 265 
266| 偏移量 | 字节数 | 类型 | 字段名 | 说明 |266| 偏移量 | 字节数 | 类型 | 字段名 | 说明 |
267|:------:|:------:|--------|-----------|--------------|267|:------:|:------:|--------|-----------|--------------|
@@ -274,9 +274,9 @@ struct FileHeader {
274 274 
275#### 5.3.2 ReduceTask275#### 5.3.2 ReduceTask
276 276 
277-**适用任务类型**:`REDUCE (2)`277+**适用任务类型**:`REDUCE (2)`
278 278 
279-**大小**:35 字节279+**大小**:35 字节
280 280 
281| 偏移量 | 字节数 | 类型 | 字段名 | 说明 |281| 偏移量 | 字节数 | 类型 | 字段名 | 说明 |
282|:------:|:------:|--------|-----------|--------------|282|:------:|:------:|--------|-----------|--------------|
@@ -291,9 +291,9 @@ struct FileHeader {
291 291 
292#### 5.3.3 NotifyTask292#### 5.3.3 NotifyTask
293 293 
294-**适用任务类型**:`NOTIFY_WAIT (0)` / `NOTIFY_RECORD (1)`294+**适用任务类型**:`NOTIFY_WAIT (0)` / `NOTIFY_RECORD (1)`
295 295 
296-**大小**:18 字节296+**大小**:18 字节
297 297 
298| 偏移量 | 字节数 | 类型 | 字段名 | 说明 |298| 偏移量 | 字节数 | 类型 | 字段名 | 说明 |
299|:------:|:------:|--------|-------------|------------|299|:------:|:------:|--------|-------------|------------|
@@ -722,18 +722,18 @@ bool ReadTaskMetaData(const char* filename) {
722 722 
723## 附录 B:文件读取流程723## 附录 B:文件读取流程
724 724 
725-**Step 1**:打开二进制文件725+**Step 1**:打开二进制文件
726 726 
727**Step 2**:读取 FileHeader (20 Bytes)727**Step 2**:读取 FileHeader (20 Bytes)
728 728 
729**Step 3**:验证魔数 (magic)729**Step 3**:验证魔数 (magic)
730 730 
731-- 魔数不匹配 → 返回错误,关闭文件731+- 魔数不匹配 → 返回错误,关闭文件
732-- 魔数匹配 → 继续732+- 魔数匹配 → 继续
733 733 
734-**Step 4**:根据 count 读取数据内容734+**Step 4**:根据 count 读取数据内容
735 735 
736-**Step 5**:关闭文件736+**Step 5**:关闭文件
737 737 
738---738---
739 739 
Mtest/hccl_vm/docs/insightV3_guide.md+4-4
@@ -306,11 +306,11 @@ DAGView 画布支持以下操作:
306 306 
307## 13. 常见问题307## 13. 常见问题
308 308 
309-### 页面没有数据309+### 13.1 页面没有数据
310 310 
311请确认 Insight 服务地址是否正确,并确认 Checker 已生成 Insight 数据。311请确认 Insight 服务地址是否正确,并确认 Checker 已生成 Insight 数据。
312 312 
313-### 进入关联页后 DAG 为空313+### 13.2 进入关联页后 DAG 为空
314 314 
315请确认数据集中存在以下文件:315请确认数据集中存在以下文件:
316 316 
@@ -319,10 +319,10 @@ graph/graph.msgpack
319graph/layout.msgpack319graph/layout.msgpack
320```320```
321 321 
322-### DAG 图过于密集322+### 13.3 DAG 图过于密集
323 323 
324建议先减少左侧 Rank 勾选数量,只查看部分 Rank,再放大局部区域进行分析。324建议先减少左侧 Rank 勾选数量,只查看部分 Rank,再放大局部区域进行分析。
325 325 
326-### 搜索不到节点326+### 13.4 搜索不到节点
327 327 
328请确认搜索关键词完整,并确认目标节点所在 Rank 已被勾选。328请确认搜索关键词完整,并确认目标节点所在 Rank 已被勾选。
Mtest/hccl_vm/docs/insight_guide.md+67-67
@@ -1,6 +1,6 @@
1# HVRM Insight 使用指南1# HVRM Insight 使用指南
2 2 
3-## 概述3+## 1. 概述
4 4 
5HVRM Insight 是一个分布式算子可视化分析工具,用于对 HCCL 集合通信算子执行过程中的图结构、内存快照和校验报错信息进行统一展示和联动分析。工具包含三个页面:5HVRM Insight 是一个分布式算子可视化分析工具,用于对 HCCL 集合通信算子执行过程中的图结构、内存快照和校验报错信息进行统一展示和联动分析。工具包含三个页面:
6 6 
@@ -12,13 +12,13 @@ HVRM Insight 是一个分布式算子可视化分析工具,用于对 HCCL 集
12 12 
13---13---
14 14 
15-## 安装与启动15+## 2. 安装与启动
16 16 
17-### 运行前准备17+### 2.1 运行前准备
18 18 
19HVRM Insight 依赖 Checker 插件输出的数据文件。使用前,请确保已完整运行过 Checker,并在 Checker 的配置文件中开启以下两个数据输出开关:19HVRM Insight 依赖 Checker 插件输出的数据文件。使用前,请确保已完整运行过 Checker,并在 Checker 的配置文件中开启以下两个数据输出开关:
20 20 
21-Checker配置文件默认位于`/path/to/hccl_vm_install/plugin/checker/manifest.json`21+Checker配置文件默认位于`/path/to/hccl_vm_install/plugin/checker/manifest.json`
22 22 
23```json23```json
24"setting": {24"setting": {
@@ -27,12 +27,12 @@ Checker配置文件默认位于`/path/to/hccl_vm_install/plugin/checker/manifest
27}27}
28```28```
29 29 
30-- **`enable_insight_dump`**:开启后,Checker 运行时会输出 Insight 所需的图结构和校验结果文件30+- **`enable_insight_dump`**:开启后,Checker 运行时会输出 Insight 所需的图结构和校验结果文件
31-- **`enable_memory_snapshot_dump`**:开启后,Checker 运行时会输出各 Rank 的内存快照文件31+- **`enable_memory_snapshot_dump`**:开启后,Checker 运行时会输出各 Rank 的内存快照文件
32 32 
33这两个开关默认为 `false`。如未开启,Insight 工具将无法读取到完整的分析数据。33这两个开关默认为 `false`。如未开启,Insight 工具将无法读取到完整的分析数据。
34 34 
35-### 数据目录结构35+### 2.2 数据目录结构
36 36 
37工具从 `data/` 目录读取数据集,每个数据集是一个独立的子目录:37工具从 `data/` 目录读取数据集,每个数据集是一个独立的子目录:
38 38 
@@ -48,7 +48,7 @@ data/
48 48 
49缺少可选目录时,对应页面会显示空状态。49缺少可选目录时,对应页面会显示空状态。
50 50 
51-### 方式一:通过 hccl-vm 插件(推荐)51+### 2.3 方式一:通过 hccl-vm 插件(推荐)
52 52 
53```bash53```bash
54# 安装并启动后端服务54# 安装并启动后端服务
@@ -62,7 +62,7 @@ hccl-vm plugin run @insight
62hccl-vm plugin uninstall @insight62hccl-vm plugin uninstall @insight
63```63```
64 64 
65-### 方式二:本地直接启动65+### 2.4 方式二:本地直接启动
66 66 
67```bash67```bash
68# 进入 insight 插件目录68# 进入 insight 插件目录
@@ -71,27 +71,27 @@ python3 serve.py
71 71 
72后端默认监听 `http://localhost:8000`,同时提供前端页面和数据 API。72后端默认监听 `http://localhost:8000`,同时提供前端页面和数据 API。
73 73 
74-### 验证启动成功74+### 2.5 验证启动成功
75 75 
76浏览器打开地址后,如果能看到 Dashboard 页面的数据集列表,说明启动成功。如果页面空白,请检查 `data/` 目录是否存在且包含 `manifest.json` 文件。76浏览器打开地址后,如果能看到 Dashboard 页面的数据集列表,说明启动成功。如果页面空白,请检查 `data/` 目录是否存在且包含 `manifest.json` 文件。
77 77 
78---78---
79 79 
80-## 场景演示80+## 3. 场景演示
81 81 
82-### 场景一:选择数据集并查看概览(Dashboard)82+### 3.1 场景一:选择数据集并查看概览(Dashboard)
83 83 
84浏览所有数据集,点击选择目标数据集,查看算子信息和统计数据,然后进入关联分析页面。84浏览所有数据集,点击选择目标数据集,查看算子信息和统计数据,然后进入关联分析页面。
85 85 
86![Dashboard 操作演示](insight_image/dashboard.gif)86![Dashboard 操作演示](insight_image/dashboard.gif)
87 87 
88-### 场景二:联动分析内存与任务图(MemView)88+### 3.2 场景二:联动分析内存与任务图(MemView)
89 89 
90通过内存时间线和 DAG 任务图的联动,定位某个 Step 的内存操作、查看节点详情、追溯数据来源。90通过内存时间线和 DAG 任务图的联动,定位某个 Step 的内存操作、查看节点详情、追溯数据来源。
91 91 
92![MemView 操作演示](insight_image/memView.gif)92![MemView 操作演示](insight_image/memView.gif)
93 93 
94-### 场景三:查看报错并跳转到问题现场(Analytic)94+### 3.3 场景三:查看报错并跳转到问题现场(Analytic)
95 95 
96在 Analytic 页面或 MemView 左侧报错列表中查看错误详情,然后一键跳转到关联的 DAG 节点或内存上下文。96在 Analytic 页面或 MemView 左侧报错列表中查看错误详情,然后一键跳转到关联的 DAG 节点或内存上下文。
97 97 
@@ -99,15 +99,15 @@ python3 serve.py
99 99 
100---100---
101 101 
102-## 页面详解102+## 4.页面详解
103 103 
104-### Dashboard 页面104+### 4.1 Dashboard 页面
105 105 
106Dashboard 是使用工具的起点,页面从左到右分为三个区域:106Dashboard 是使用工具的起点,页面从左到右分为三个区域:
107 107 
108-- **左侧**:算子摘要和 Rank 树。Rank 树支持勾选 / 取消勾选,选择结果会同步到所有页面108+- **左侧**:算子摘要和 Rank 树。Rank 树支持勾选 / 取消勾选,选择结果会同步到所有页面
109-- **中间**:数据集列表。点击某一行选中数据集,左侧和右侧同步刷新109+- **中间**:数据集列表。点击某一行选中数据集,左侧和右侧同步刷新
110-- **右侧**:选中数据集的详情面板,展示统计信息和跳转按钮110+- **右侧**:选中数据集的详情面板,展示统计信息和跳转按钮
111 111 
112![Dashboard 总览](insight_image/dashboard_overview.png)112![Dashboard 总览](insight_image/dashboard_overview.png)
113 113 
@@ -117,10 +117,10 @@ Dashboard 是使用工具的起点,页面从左到右分为三个区域:
117 117 
118详情面板底部提供两个跳转入口:118详情面板底部提供两个跳转入口:
119 119 
120-- **`进入关联视图`**:跳转到 MemView,带入当前数据集和 Rank 选择120+- **`进入关联视图`**:跳转到 MemView,带入当前数据集和 Rank 选择
121-- **`查看诊断`**:跳转到 Analytic,查看该数据集的所有校验错误121+- **`查看诊断`**:跳转到 Analytic,查看该数据集的所有校验错误
122 122 
123-### MemView 页面123+### 4.2 MemView 页面
124 124 
125MemView 将内存时间线和 DAG 任务图放在同一视图中联动展示,是排查问题的核心页面。125MemView 将内存时间线和 DAG 任务图放在同一视图中联动展示,是排查问题的核心页面。
126 126 
@@ -128,92 +128,92 @@ MemView 将内存时间线和 DAG 任务图放在同一视图中联动展示,
128 128 
129**页面布局:**129**页面布局:**
130 130 
131-- **左侧**:Rank 选择树、报错列表(Issues)、搜索面板131+- **左侧**:Rank 选择树、报错列表(Issues)、搜索面板
132-- **上半区**:内存时间线 —— 按 Step 展示各 Rank 的 buffer 状态演化132+- **上半区**:内存时间线 —— 按 Step 展示各 Rank 的 buffer 状态演化
133-- **下半区**:DAG 任务图 —— 按 Rank / Stream 泳道展示任务节点和依赖关系133+- **下半区**:DAG 任务图 —— 按 Rank / Stream 泳道展示任务节点和依赖关系
134-- **右侧**:详情栏 —— 展示当前选中 Step 或节点的详细信息134+- **右侧**:详情栏 —— 展示当前选中 Step 或节点的详细信息
135 135 
136![DAG 节点视图](insight_image/memview_dag_node.png)136![DAG 节点视图](insight_image/memview_dag_node.png)
137 137 
138-#### 核心联动机制:Step、Rank、Task 三者联动138+#### 4.3 核心联动机制:Step、Rank、Task 三者联动
139 139 
140-- 在时间线上选中一个 Step → DAG 同步高亮对应节点140+- 在时间线上选中一个 Step → DAG 同步高亮对应节点
141-- 在 DAG 中点击一个节点 → 右侧切到节点详情141+- 在 DAG 中点击一个节点 → 右侧切到节点详情
142-- 从右侧点击跳转按钮 → 时间线同步定位到对应 Step142+- 从右侧点击跳转按钮 → 时间线同步定位到对应 Step
143 143 
144**右侧详情栏**有两种模式,根据是否选中 DAG 节点自动切换:144**右侧详情栏**有两种模式,根据是否选中 DAG 节点自动切换:
145 145 
146-- **Step 详情模式**:展示当前 Step 的内存操作列表(Task Memory Ops)和 Buffer 切片详情(Buffer Layout)。Buffer Layout 中的每个切片可展开查看数据来源语义,支持点击跳转146+- **Step 详情模式**:展示当前 Step 的内存操作列表(Task Memory Ops)和 Buffer 切片详情(Buffer Layout)。Buffer Layout 中的每个切片可展开查看数据来源语义,支持点击跳转
147-- **节点详情模式**:展示节点属性、父/子节点关系、跨阶段节点映射、以及该节点关联的内存操作147+- **节点详情模式**:展示节点属性、父/子节点关系、跨阶段节点映射、以及该节点关联的内存操作
148 148 
149**其他功能:**149**其他功能:**
150 150 
151-- **底部播放器**:支持按 Step 顺序浏览,可拖拽滑块快速跳转,播放速度为 1 秒 / Step151+- **底部播放器**:支持按 Step 顺序浏览,可拖拽滑块快速跳转,播放速度为 1 秒 / Step
152-- **Focus 视图**:选中 Step 后 DAG 自动进入局部视图,只显示当前 Step 相关节点及其上下文,适合节点密集时缩小范围152+- **Focus 视图**:选中 Step 后 DAG 自动进入局部视图,只显示当前 Step 相关节点及其上下文,适合节点密集时缩小范围
153-- **子图浏览**:节点包含子图时,可在右侧点击 `打开子图` 进入查看更细粒度的任务结构153+- **子图浏览**:节点包含子图时,可在右侧点击 `打开子图` 进入查看更细粒度的任务结构
154- **Stage 切换**:通过顶部下拉框切换图阶段(如 `input_graph` / `input_task_queues`154- **Stage 切换**:通过顶部下拉框切换图阶段(如 `input_graph` / `input_task_queues`
155-- **搜索**:支持按 taskId / taskType / notifyId 搜索 DAG 节点,结果点击即定位155+- **搜索**:支持按 taskId / taskType / notifyId 搜索 DAG 节点,结果点击即定位
156 156 
157-### Analytic 页面157+### 4.4 Analytic 页面
158 158 
159Analytic 集中展示校验过程中发现的所有错误,页面分为三个区域:159Analytic 集中展示校验过程中发现的所有错误,页面分为三个区域:
160 160 
161![Analytic 报错详情](insight_image/analytic_issue_detail.png)161![Analytic 报错详情](insight_image/analytic_issue_detail.png)
162 162 
163-- **左侧**:数据集与 Rank 选择163+- **左侧**:数据集与 Rank 选择
164-- **中间**:报错列表,每条错误展示标题、严重程度标签(颜色区分)、关键字段摘要和原始错误码164+- **中间**:报错列表,每条错误展示标题、严重程度标签(颜色区分)、关键字段摘要和原始错误码
165-- **右侧**:选中错误的详情面板,包括错误基本信息、涉及的 Rank、关联节点信息、Slice 详情(精确定位数据不一致的内存位置)、补充信息和原始 JSON165+- **右侧**:选中错误的详情面板,包括错误基本信息、涉及的 Rank、关联节点信息、Slice 详情(精确定位数据不一致的内存位置)、补充信息和原始 JSON
166 166 
167详情面板顶部提供跳转按钮:167详情面板顶部提供跳转按钮:
168 168 
169-- **`查看 memView`**:跳转到 MemView,定位到错误关联的内存 Step169+- **`查看 memView`**:跳转到 MemView,定位到错误关联的内存 Step
170-- **`查看 dagView`**:跳转到 MemView,直接定位到错误关联的 DAG 节点170+- **`查看 dagView`**:跳转到 MemView,直接定位到错误关联的 DAG 节点
171 171 
172跳转时自动带入数据集、Rank 和节点位置信息,到达 MemView 后无需重新选择即可开始排查。172跳转时自动带入数据集、Rank 和节点位置信息,到达 MemView 后无需重新选择即可开始排查。
173 173 
174---174---
175 175 
176-## 常见使用流程176+## 5. 常见使用流程
177 177 
178-### 从 Dashboard 开始分析178+### 5.1 从 Dashboard 开始分析
179 179 
180-1. 在数据集列表中点击目标数据集180+1. 在数据集列表中点击目标数据集
181-2. 在左侧 Rank 树中勾选需要查看的 Rank181+2. 在左侧 Rank 树中勾选需要查看的 Rank
182-3. 在右侧确认数据集的文件数量和错误统计182+3. 在右侧确认数据集的文件数量和错误统计
183-4. 点击 `进入关联视图` 进入 MemView183+4. 点击 `进入关联视图` 进入 MemView
184 184 
185-### 在 MemView 中定位内存问题185+### 5.2 在 MemView 中定位内存问题
186 186 
1871. 在时间线上点击目标 Step(或使用底部播放器跳转)1871. 在时间线上点击目标 Step(或使用底部播放器跳转)
188-2. 在右侧 `Task Memory Ops` 查看该 Step 的内存操作188+2. 在右侧 `Task Memory Ops` 查看该 Step 的内存操作
189-3. 在 `Buffer Layout` 中选择 Rank 和 Buffer Type,展开 slice 查看数据来源189+3. 在 `Buffer Layout` 中选择 Rank 和 Buffer Type,展开 slice 查看数据来源
190-4. 点击 `定位 Task` 或语义卡片跳转到对应 DAG 节点190+4. 点击 `定位 Task` 或语义卡片跳转到对应 DAG 节点
191 191 
192-### 从 DAG 节点反查内存192+### 5.3 从 DAG 节点反查内存
193 193 
194-1. 在 DAG 中点击目标节点194+1. 在 DAG 中点击目标节点
195-2. 在右侧查看 `节点 Memory Ops`195+2. 在右侧查看 `节点 Memory Ops`
196-3. 点击 `跳转到 Step` 回到时间线,或点击父子节点继续沿依赖链排查196+3. 点击 `跳转到 Step` 回到时间线,或点击父子节点继续沿依赖链排查
197 197 
198-### 从报错快速定位198+### 5.4 从报错快速定位
199 199 
200-1. 在 Analytic 页面或 MemView 左侧报错列表中点击一条错误200+1. 在 Analytic 页面或 MemView 左侧报错列表中点击一条错误
201-2. 查看详情中的关联节点和 Slice 信息201+2. 查看详情中的关联节点和 Slice 信息
202-3. 点击 `查看 memView` 或 `查看 dagView` 跳转到问题现场202+3. 点击 `查看 memView` 或 `查看 dagView` 跳转到问题现场
203 203 
204---204---
205 205 
206-## 使用技巧206+## 6. 使用技巧
207 207 
208-- 先在 Dashboard 确认数据集和 Rank 范围,再进入 MemView 分析208+- 先在 Dashboard 确认数据集和 Rank 范围,再进入 MemView 分析
209-- 看内存问题时,优先用 `Task Memory Ops` 和 `Buffer Layout` 配合排查209+- 看内存问题时,优先用 `Task Memory Ops` 和 `Buffer Layout` 配合排查
210-- 看节点依赖关系时,优先在 DAG 中选中节点,再用右侧父子关系和节点映射跳转210+- 看节点依赖关系时,优先在 DAG 中选中节点,再用右侧父子关系和节点映射跳转
211-- DAG 节点太密集时,先用 Focus 视图缩小范围,再逐步展开211+- DAG 节点太密集时,先用 Focus 视图缩小范围,再逐步展开
212-- MemView 左侧的报错列表可以在不离开当前页面的情况下快速浏览并定位错误节点212+- MemView 左侧的报错列表可以在不离开当前页面的情况下快速浏览并定位错误节点
213 213 
214---214---
215 215 
216-## 术语说明216+## 7. 术语说明
217 217 
218| 术语 | 说明 |218| 术语 | 说明 |
219|------|------|219|------|------|
Mtest/hccl_vm/docs/simulator_runner_data_model.md+142-142
@@ -1,8 +1,8 @@
1# HCCL 模拟运行器数据模型设计1# HCCL 模拟运行器数据模型设计
2 2 
3-## 软硬件资源交互关系建模3+## 1. 软硬件资源交互关系建模
4 4 
5-数据流交互示意图5+数据流交互示意图
6 6 
7```mermaid7```mermaid
8sequenceDiagram8sequenceDiagram
@@ -35,9 +35,9 @@ sequenceDiagram
35 35 
36只要是塞进 Stream 里的东西,都是由 Device 硬件执行的。36只要是塞进 Stream 里的东西,都是由 Device 硬件执行的。
37 37 
38-## Device、Context、Stream 等硬件资源关系建模38+## 2. Device、Context、Stream 等硬件资源关系建模
39 39 
40-Device、Context、Stream 与用户主机线程之间的关系40+Device、Context、Stream 与用户主机线程之间的关系
41 41 
42```mermaid42```mermaid
43graph TD43graph TD
@@ -116,9 +116,9 @@ graph TD
116 116
117```117```
118 118 
119-### 基础设备关系建模119+### 2.1 基础设备关系建模
120 120 
121-#### 层次结构总览121+#### 2.1.1 层次结构总览
122 122 
123```mermaid123```mermaid
124erDiagram124erDiagram
@@ -234,7 +234,7 @@ erDiagram
234 Device ||--o{ DeviceConnection : "peer access"234 Device ||--o{ DeviceConnection : "peer access"
235```235```
236 236 
237-#### 网络通信资源237+#### 2.1.2 网络通信资源
238 238 
239```mermaid239```mermaid
240erDiagram240erDiagram
@@ -298,7 +298,7 @@ erDiagram
298 CcuChannel }o..|| Link : "based-on"298 CcuChannel }o..|| Link : "based-on"
299```299```
300 300 
301-#### 关键关系说明301+#### 2.1.3 关键关系说明
302 302 
303##### 第一层:物理拓扑层303##### 第一层:物理拓扑层
304 304 
@@ -336,7 +336,7 @@ erDiagram
336| Link → EndPoint | 物理连接关联端点 | 由 topo.json 定义,描述物理拓扑 |336| Link → EndPoint | 物理连接关联端点 | 由 topo.json 定义,描述物理拓扑 |
337| Link → Link-Protocol-Mapping | 连接支持的协议 | 一个Link可支持多种协议(UB_CTP/UB_MEM等) |337| Link → Link-Protocol-Mapping | 连接支持的协议 | 一个Link可支持多种协议(UB_CTP/UB_MEM等) |
338 338 
339-#### 配套接口映射表339+#### 2.1.4 配套接口映射表
340 340 
341##### 基础与设备层 (Device / Server)341##### 基础与设备层 (Device / Server)
342 342 
@@ -359,7 +359,7 @@ erDiagram
359| Stream.activated | `rtStreamStop` |359| Stream.activated | `rtStreamStop` |
360| Stream.failure-mode | `rtSetStreamAttribute`,`rtGetStreamAttribute` |360| Stream.failure-mode | `rtSetStreamAttribute`,`rtGetStreamAttribute` |
361 361 
362-### 基础内存管理关系建模362+### 2.2 基础内存管理关系建模
363 363 
364```mermaid364```mermaid
365erDiagram365erDiagram
@@ -420,9 +420,9 @@ erDiagram
420 %%}420 %%}
421```421```
422 422 
423-#### 关键关系说明(基础内存管理关系建模)423+#### 2.2.1 关键关系说明(基础内存管理关系建模)
424 424 
425-1. **物理内存核心地位**425+1. **物理内存核心地位**
426 `PhyMemBlock`作为基础实体,通过`phy_mem_id`与所有其他实体关联,体现华为昇腾"物理内存池化"的设计理念[3]。426 `PhyMemBlock`作为基础实体,通过`phy_mem_id`与所有其他实体关联,体现华为昇腾"物理内存池化"的设计理念[3]。
4272. **三层映射体系**4272. **三层映射体系**
428 428 
@@ -434,7 +434,7 @@ erDiagram
4344. **特殊映射类型**4344. **特殊映射类型**
435 `MemMapRecord`记录双虚拟地址映射场景(如`aclrtMapMem`产生的映射),支持华为NPU的零拷贝数据传输[3]。435 `MemMapRecord`记录双虚拟地址映射场景(如`aclrtMapMem`产生的映射),支持华为NPU的零拷贝数据传输[3]。
436 436 
437-#### 配套接口映射表(基础内存管理关系建模)437+#### 2.1.2 配套接口映射表(基础内存管理关系建模)
438 438 
439| 实体 | 关键管理接口 |439| 实体 | 关键管理接口 |
440| ------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------- |440| ------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------- |
@@ -448,9 +448,9 @@ erDiagram
448| IpcMemWhiteList.pid | `rtIpcMemSetImportPid` |448| IpcMemWhiteList.pid | `rtIpcMemSetImportPid` |
449| MemMapRecord | `rtHostRegister`, `rtHostUnRegister` |449| MemMapRecord | `rtHostRegister`, `rtHostUnRegister` |
450 450 
451-## 在基础模型上扩展数据任务模型451+## 3. 在基础模型上扩展数据任务模型
452 452 
453-### 数据 / 任务流建模453+### 3.1 数据 / 任务流建模
454 454 
455```mermaid455```mermaid
456erDiagram456erDiagram
@@ -498,7 +498,7 @@ erDiagram
498 498 
499```499```
500 500 
501-### ccu资源建模501+### 3.2 ccu资源建模
502 502 
503一个NPU device包含2个CCU,分别为die0和die1。503一个NPU device包含2个CCU,分别为die0和die1。
504 504 
@@ -534,7 +534,7 @@ graph RL
534 534 
535```535```
536 536 
537-#### 配套接口映射表(ccu资源建模)537+#### 3.2.1 配套接口映射表(ccu资源建模)
538 538 
539| 实体 | 关键管理接口 |539| 实体 | 关键管理接口 |
540| ------------------ | -------------------------------------------------------- |540| ------------------ | -------------------------------------------------------- |
@@ -545,24 +545,24 @@ graph RL
545| Local/Rmt-Addr | `rtGetDeviceLocalAddr`, `rtGetDeviceRemoteAddr` |545| Local/Rmt-Addr | `rtGetDeviceLocalAddr`, `rtGetDeviceRemoteAddr` |
546| CCU资源查询 | `rtGetCcudieInfo`, `rtGetCcudieNum` |546| CCU资源查询 | `rtGetCcudieInfo`, `rtGetCcudieNum` |
547 547 
548-#### CCU资源生命周期说明548+#### 3.2.2 CCU资源生命周期说明
549 549 
550**CCU初始化流程**550**CCU初始化流程**
551 551 
552-1. Device启动时,两个CCU(die0/die1)自动初始化552+1. Device启动时,两个CCU(die0/die1)自动初始化
553-2. 每个CCU分配独立的CcuBuf、Variable、Notify资源池553+2. 每个CCU分配独立的CcuBuf、Variable、Notify资源池
554-3. CompletedEvent用于通知任务完成状态554+3. CompletedEvent用于通知任务完成状态
555 555 
556**资源约束**556**资源约束**
557 557 
558- 每个CCU的CcuBuf数量有限(与Device.soc-version相关)558- 每个CCU的CcuBuf数量有限(与Device.soc-version相关)
559-- Variable用于存储通信过程中的共享变量559+- Variable用于存储通信过程中的共享变量
560-- Notify用于跨CCU的同步通知机制560+- Notify用于跨CCU的同步通知机制
561-- Local/Rmt-Addr用于跨die通信时的地址转换561+- Local/Rmt-Addr用于跨die通信时的地址转换
562 562 
563-### 异步 / 同步执行建模563+### 3.3 异步 / 同步执行建模
564 564 
565-#### [Notify资源管理](https://www.hiascend.com/document/detail/zh/CANNCommunityEdition/850alpha001/appdevg/acldevg/aclcppdevg_000524.html)565+#### 3.3.1 [Notify资源管理](https://www.hiascend.com/document/detail/zh/CANNCommunityEdition/850alpha001/appdevg/acldevg/aclcppdevg_000524.html)
566 566 
567```mermaid567```mermaid
568erDiagram568erDiagram
@@ -607,20 +607,20 @@ erDiagram
607 607 
608- 每个Device的Notify数量上限为`max-notify-cnt`(如A3芯片为8192)608- 每个Device的Notify数量上限为`max-notify-cnt`(如A3芯片为8192)
609- `device-notify-seq`是Notify在Device内的物理序号(0~8191)609- `device-notify-seq`是Notify在Device内的物理序号(0~8191)
610-- Notify创建时必须指定所属Context,Context绑定到特定Device610+- Notify创建时必须指定所属Context,Context绑定到特定Device
611 611 
612**Notify的IPC共享机制**612**Notify的IPC共享机制**
613 613 
614-- `IpcNotify`允许跨进程共享Notify实例614+- `IpcNotify`允许跨进程共享Notify实例
615-- `name-or-key`是共享标识,通过`rtNotifyGetExportKey`获取615+- `name-or-key`是共享标识,通过`rtNotifyGetExportKey`获取
616-- 其他进程通过`rtNotifyImportByKey`导入并使用616+- 其他进程通过`rtNotifyImportByKey`导入并使用
617-- `IpcNotifyVistorList`记录有权访问该Notify的进程PID617+- `IpcNotifyVistorList`记录有权访问该Notify的进程PID
618 618 
619**Notify状态管理**619**Notify状态管理**
620 620 
621-- `value`字段映射到硬件寄存器,用于读写状态621+- `value`字段映射到硬件寄存器,用于读写状态
622-- `rtWaitAndResetNotify`等待Notify变为Ready状态并重置622+- `rtWaitAndResetNotify`等待Notify变为Ready状态并重置
623-- Notify用于Stream间同步、跨进程同步场景623+- Notify用于Stream间同步、跨进程同步场景
624 624 
625##### 配套接口映射表([Notify资源管理](https://www.hiascend.com/document/detail/zh/CANNCommunityEdition/850alpha001/appdevg/acldevg/aclcppdevg_000524.html))625##### 配套接口映射表([Notify资源管理](https://www.hiascend.com/document/detail/zh/CANNCommunityEdition/850alpha001/appdevg/acldevg/aclcppdevg_000524.html))
626 626 
@@ -631,7 +631,7 @@ erDiagram
631| IpcNotify.name-or-key | `rtNotifyGetExportKey`,`rtNotifyImportByKey` |631| IpcNotify.name-or-key | `rtNotifyGetExportKey`,`rtNotifyImportByKey` |
632| IpcNotifyVistorList.ipc-id | `rtNotifySetImportPid` |632| IpcNotifyVistorList.ipc-id | `rtNotifySetImportPid` |
633 633 
634-#### Notify同步控制634+#### 3.3.2 Notify同步控制
635 635 
636```mermaid636```mermaid
637erDiagram637erDiagram
@@ -682,14 +682,14 @@ erDiagram
682 682 
683**Notify任务类型**683**Notify任务类型**
684 684 
685-- `NotifyRecordTask`:将Notify状态设置为Ready,表示某个事件已完成685+- `NotifyRecordTask`:将Notify状态设置为Ready,表示某个事件已完成
686-- `NotifyWaitTask`:等待Notify状态变为Ready,实现Stream间的同步686+- `NotifyWaitTask`:等待Notify状态变为Ready,实现Stream间的同步
687 687 
688**任务执行顺序**688**任务执行顺序**
689 689 
690-- NotifyRecordTask在StreamA执行,设置Notify为Ready690+- NotifyRecordTask在StreamA执行,设置Notify为Ready
691-- NotifyWaitTask在StreamB执行,等待同一个Notify691+- NotifyWaitTask在StreamB执行,等待同一个Notify
692-- 当Notify变为Ready后,StreamB后续任务才能继续执行692+- 当Notify变为Ready后,StreamB后续任务才能继续执行
693 693 
694**跨Stream同步示例**694**跨Stream同步示例**
695 695 
@@ -706,7 +706,7 @@ StreamB: NotifyWaitTask(notify-id=1) -> Task3
706| NotifyRecordTask | `rtRecordNotify` |706| NotifyRecordTask | `rtRecordNotify` |
707| NotifyWaitTask | `lrtWaitAndResetNotify` |707| NotifyWaitTask | `lrtWaitAndResetNotify` |
708 708 
709-#### Event资源管理709+#### 3.3.3 Event资源管理
710 710 
711```mermaid711```mermaid
712erDiagram712erDiagram
@@ -735,19 +735,19 @@ erDiagram
735 735 
736- 每个Device的Event数量上限为`max-event-cnt`(如A3芯片为65536)736- 每个Device的Event数量上限为`max-event-cnt`(如A3芯片为65536)
737- `device-res-seq`是Event在Device内的物理序号(0~65535)737- `device-res-seq`是Event在Device内的物理序号(0~65535)
738-- Event创建时必须指定所属Context,Context绑定到特定Device738+- Event创建时必须指定所属Context,Context绑定到特定Device
739 739 
740**Event与Context的关系**740**Event与Context的关系**
741 741 
742-- `created-ctx-id`记录Event的创建Context742+- `created-ctx-id`记录Event的创建Context
743-- Event可在多个Stream间共享,但必须属于同一Context743+- Event可在多个Stream间共享,但必须属于同一Context
744- 跨Context的Event共享需要通过IPC机制(类似Notify)744- 跨Context的Event共享需要通过IPC机制(类似Notify)
745 745 
746**Event状态管理**746**Event状态管理**
747 747 
748-- `status`字段表示Event当前状态:NotRecorded/Recorded/Completed748+- `status`字段表示Event当前状态:NotRecorded/Recorded/Completed
749- `event-flag`用于控制Event的行为(如是否自动重置)749- `event-flag`用于控制Event的行为(如是否自动重置)
750-- `created-time`用于性能统计750+- `created-time`用于性能统计
751 751 
752##### 配套接口映射表(Event资源管理)752##### 配套接口映射表(Event资源管理)
753 753 
@@ -756,7 +756,7 @@ erDiagram
756| Event.event-id | `rtCreateEvent`, `rtCreateEventWithFlag`,`rtDestroyEvent`,`rtGetEventId` |756| Event.event-id | `rtCreateEvent`, `rtCreateEventWithFlag`,`rtDestroyEvent`,`rtGetEventId` |
757| Event.status | `rtRecordEvent`,`rtQueryEventStatus` |757| Event.status | `rtRecordEvent`,`rtQueryEventStatus` |
758 758 
759-#### Event 流程控制759+#### 3.3.4 Event 流程控制
760 760 
761```mermaid761```mermaid
762erDiagram762erDiagram
@@ -834,19 +834,19 @@ erDiagram
834 834 
835**Event任务类型分类**835**Event任务类型分类**
836 836 
837-- `EventRecordTask`:将Event状态设置为Recorded/Completed837+- `EventRecordTask`:将Event状态设置为Recorded/Completed
838-- `EventWaitTask`:等待Event状态变为Completed838+- `EventWaitTask`:等待Event状态变为Completed
839- `EventSyncTask`:同步等待Event完成(阻塞调用)839- `EventSyncTask`:同步等待Event完成(阻塞调用)
840-- `EventRICaptureTask`:RI Capture模式下的特殊记录任务840+- `EventRICaptureTask`:RI Capture模式下的特殊记录任务
841-- `EventTimeTask`:记录时间戳相关任务841+- `EventTimeTask`:记录时间戳相关任务
842-- `EventTraceTask`:用于性能追踪的任务记录842+- `EventTraceTask`:用于性能追踪的任务记录
843 843 
844**Event任务继承关系**844**Event任务继承关系**
845 845 
846-- `EventTask`是基类,包含task-id、event-id、execute-time、finish-time846+- `EventTask`是基类,包含task-id、event-id、execute-time、finish-time
847-- `EventSyncTask`继承EventTask,增加op-timeout-s超时参数847+- `EventSyncTask`继承EventTask,增加op-timeout-s超时参数
848-- `EventRecordTask`和`EventWaitTask`继承EventSyncTask848+- `EventRecordTask`和`EventWaitTask`继承EventSyncTask
849-- `EventRICaptureTask`、`EventTimeTask`、`EventTraceTask`直接继承EventTask849+- `EventRICaptureTask`、`EventTimeTask`、`EventTraceTask`直接继承EventTask
850 850 
851**Event执行流程**851**Event执行流程**
852 852 
@@ -858,9 +858,9 @@ StreamB: EventWaitTask(event-id=1) -> KernelTask2
858 858 
859**任务追踪关系**859**任务追踪关系**
860 860 
861-- `first-capture-task-id`记录首次Capture的Task ID861+- `first-capture-task-id`记录首次Capture的Task ID
862-- `EventTraceTask.start-task-id`关联追踪的开始任务862+- `EventTraceTask.start-task-id`关联追踪的开始任务
863-- EventRecordTask通过EventWaitTask映射实现跨Stream同步863+- EventRecordTask通过EventWaitTask映射实现跨Stream同步
864 864 
865##### 配套接口映射表(Event 流程控制)865##### 配套接口映射表(Event 流程控制)
866 866 
@@ -872,16 +872,16 @@ StreamB: EventWaitTask(event-id=1) -> KernelTask2
872| EventTimeTask | `rtResetEvent`, `rtRecordEvent` |872| EventTimeTask | `rtResetEvent`, `rtRecordEvent` |
873| EventTraceTask | `rtResetEvent`, `rtRecordEvent` |873| EventTraceTask | `rtResetEvent`, `rtRecordEvent` |
874 874 
875-## 通信域建模875+## 4. 通信域建模
876 876 
877跨机通信涉及到多通信域混合任务编排。877跨机通信涉及到多通信域混合任务编排。
878通信域的本质是HCCL在框架层通过Rdma_Agent提供的建链能力,维护的一张多卡的网络拓扑,保存在host进程中。878通信域的本质是HCCL在框架层通过Rdma_Agent提供的建链能力,维护的一张多卡的网络拓扑,保存在host进程中。
879 879 
880-### 通信域核心概念880+### 4.1 通信域核心概念
881 881 
882通信域(Communicator)是HCCL集合通信的基本抽象,每个通信域定义了一组参与通信的Rank及其拓扑关系。882通信域(Communicator)是HCCL集合通信的基本抽象,每个通信域定义了一组参与通信的Rank及其拓扑关系。
883 883 
884-#### 通信域基础定义884+#### 4.1.1 通信域基础定义
885 885 
886```mermaid886```mermaid
887erDiagram887erDiagram
@@ -907,7 +907,7 @@ erDiagram
907 Communicator ||--o{ Communicator : "派生(MPI_Comm_split)"907 Communicator ||--o{ Communicator : "派生(MPI_Comm_split)"
908```908```
909 909 
910-#### 控制面:Socket通信910+#### 4.1.2 控制面:Socket通信
911 911 
912```mermaid912```mermaid
913erDiagram913erDiagram
@@ -949,7 +949,7 @@ erDiagram
949 RaSocket ||--o{ RaEpoll : "被监控"949 RaSocket ||--o{ RaEpoll : "被监控"
950```950```
951 951 
952-#### 数据面:RDMA通信952+#### 4.1.3 数据面:RDMA通信
953 953 
954```mermaid954```mermaid
955erDiagram955erDiagram
@@ -1046,7 +1046,7 @@ erDiagram
1046 RaNdaQP ||--|| RaNdaCQ : "使用"1046 RaNdaQP ||--|| RaNdaCQ : "使用"
1047```1047```
1048 1048 
1049-#### 数据面:UB统一总线1049+#### 4.1.4 数据面:UB统一总线
1050 1050 
1051```mermaid1051```mermaid
1052erDiagram1052erDiagram
@@ -1160,7 +1160,7 @@ erDiagram
1160 }1160 }
1161```1161```
1162 1162 
1163-#### 异步请求管理1163+#### 4.1.5 异步请求管理
1164 1164 
1165```mermaid1165```mermaid
1166erDiagram1166erDiagram
@@ -1173,7 +1173,7 @@ erDiagram
1173 }1173 }
1174```1174```
1175 1175 
1176-#### 通信域架构总结1176+#### 4.1.6 通信域架构总结
1177 1177 
1178```text1178```text
1179┌─────────────────────────────────────────────────────────────────┐1179┌─────────────────────────────────────────────────────────────────┐
@@ -1204,7 +1204,7 @@ erDiagram
1204└─────────────────────────────────────────────────────────────────┘1204└─────────────────────────────────────────────────────────────────┘
1205```1205```
1206 1206 
1207-#### 关键实体对照表1207+#### 4.1.7 关键实体对照表
1208 1208 
1209| 概念 | RDMA模式 | UB模式 | 说明 |1209| 概念 | RDMA模式 | UB模式 | 说明 |
1210| --- | --- | --- | --- |1210| --- | --- | --- | --- |
@@ -1217,7 +1217,7 @@ erDiagram
1217| 传输路径 | - | RaTp | 物理路径管理 |1217| 传输路径 | - | RaTp | 物理路径管理 |
1218| 安全令牌 | - | RaTokenId | 访问控制 |1218| 安全令牌 | - | RaTokenId | 访问控制 |
1219 1219 
1220-#### HCCP接口分类1220+#### 4.1.8 HCCP接口分类
1221 1221 
1222```text1222```text
1223HCCP Network API1223HCCP Network API
@@ -1421,63 +1421,63 @@ HCCP Network API
1421 1421 
1422**通信域层次结构**1422**通信域层次结构**
1423 1423 
1424-- `Communicator`是HCCL集合通信的核心抽象,定义了一组参与通信的Rank1424+- `Communicator`是HCCL集合通信的核心抽象,定义了一组参与通信的Rank
1425-- `Rank`是通信域中的参与节点,每个Rank绑定到具体的Device1425+- `Rank`是通信域中的参与节点,每个Rank绑定到具体的Device
1426- 父通信域派生子通信域(如`color`属性实现分组)1426- 父通信域派生子通信域(如`color`属性实现分组)
1427 1427 
1428**控制平面与数据平面分离**1428**控制平面与数据平面分离**
1429 1429 
1430-- **控制平面(Socket)**:用于建链、交换QP信息、控制信令,基于TCP协议1430+- **控制平面(Socket)**:用于建链、交换QP信息、控制信令,基于TCP协议
1431-- **数据平面(RDMA/UB)**:用于高性能数据传输,基于RDMA Verbs或UB协议1431+- **数据平面(RDMA/UB)**:用于高性能数据传输,基于RDMA Verbs或UB协议
1432 1432 
1433**RDMA资源层次**1433**RDMA资源层次**
1434 1434 
1435-- `RaDevice`是虚拟网卡抽象,一个Device可创建多个RaDevice1435+- `RaDevice`是虚拟网卡抽象,一个Device可创建多个RaDevice
1436-- `RaQP`是队列对,包含Send Queue和Receive Queue1436+- `RaQP`是队列对,包含Send Queue和Receive Queue
1437-- `RaCQ`是完成队列,用于轮询WR完成状态1437+- `RaCQ`是完成队列,用于轮询WR完成状态
1438-- `RaMR`是内存注册,将虚拟内存映射为RDMA可访问的物理内存1438+- `RaMR`是内存注册,将虚拟内存映射为RDMA可访问的物理内存
1439-- `RaSRQ`是共享接收队列,多个QP可共享同一SRQ以提高资源利用率1439+- `RaSRQ`是共享接收队列,多个QP可共享同一SRQ以提高资源利用率
1440 1440 
1441**UB资源层次**1441**UB资源层次**
1442 1442 
1443-- `RaContext`是UB统一上下文,替代RaDevice的设备抽象1443+- `RaContext`是UB统一上下文,替代RaDevice的设备抽象
1444- `RaJetty`是QP等价物,支持多种模式(URMA_NORMAL/CCU等)1444- `RaJetty`是QP等价物,支持多种模式(URMA_NORMAL/CCU等)
1445-- `RaJfc`是CQ等价物,用于完成请求管理1445+- `RaJfc`是CQ等价物,用于完成请求管理
1446-- `RaLmem/RaRmem`是本地/远端内存管理,替代RaMR1446+- `RaLmem/RaRmem`是本地/远端内存管理,替代RaMR
1447-- `RaTp`是传输路径管理,支持RTP/CTP/UTP三种类型1447+- `RaTp`是传输路径管理,支持RTP/CTP/UTP三种类型
1448-- `RaTokenId`是安全通信令牌,用于跨进程内存访问控制1448+- `RaTokenId`是安全通信令牌,用于跨进程内存访问控制
1449 1449 
1450**实体关联要点**1450**实体关联要点**
1451 1451 
1452-1. `RaSocketPair`需要两个`RaSocket`(client端和server端)才能建立连接1452+1. `RaSocketPair`需要两个`RaSocket`(client端和server端)才能建立连接
1453-2. `RaQP.state`必须经历RESET→INIT→RTR→RTS的状态转换才能正常通信1453+2. `RaQP.state`必须经历RESET→INIT→RTR→RTS的状态转换才能正常通信
1454-3. `RaMR.lkey`用于本地访问,`RaMR.rkey`用于远端RDMA访问1454+3. `RaMR.lkey`用于本地访问,`RaMR.rkey`用于远端RDMA访问
1455-4. `RaJetty`通过`Bind`操作与对端Jetty建立逻辑连接1455+4. `RaJetty`通过`Bind`操作与对端Jetty建立逻辑连接
1456-5. `RaLmem`必须注册后,`RaRmem`才能从对端导入并访问1456+5. `RaLmem`必须注册后,`RaRmem`才能从对端导入并访问
1457 1457 
1458**NDA(Network Direct Access)直接访问机制**1458**NDA(Network Direct Access)直接访问机制**
1459 1459 
1460-- `RaNdaQP`和`RaNdaCQ`是网络直接访问的QP/CQ变体1460+- `RaNdaQP`和`RaNdaCQ`是网络直接访问的QP/CQ变体
1461-- NDA模式允许绕过部分协议栈,降低延迟1461+- NDA模式允许绕过部分协议栈,降低延迟
1462-- `RaNdaGetDirectFlag`检查设备是否支持NDA模式1462+- `RaNdaGetDirectFlag`检查设备是否支持NDA模式
1463 1463 
1464**异步请求管理**1464**异步请求管理**
1465 1465 
1466-- `AsyncRequest`统一管理所有异步操作的请求句柄1466+- `AsyncRequest`统一管理所有异步操作的请求句柄
1467-- 异步操作包括:连接、监听、QP创建/销毁、内存注册等1467+- 异步操作包括:连接、监听、QP创建/销毁、内存注册等
1468-- `RaGetAsyncReqResult`轮询异步操作结果1468+- `RaGetAsyncReqResult`轮询异步操作结果
1469 1469 
1470**Socket事件机制**1470**Socket事件机制**
1471 1471 
1472-- `RaSocketEvent`是事件等待机制的句柄1472+- `RaSocketEvent`是事件等待机制的句柄
1473-- `RaEpoll`实现类似Linux Epoll的事件监控机制1473+- `RaEpoll`实现类似Linux Epoll的事件监控机制
1474-- 支持添加、修改、删除监控的Socket事件1474+- 支持添加、修改、删除监控的Socket事件
1475 1475 
1476**网络接口与配置**1476**网络接口与配置**
1477 1477 
1478-- `InterfaceInfo`描述网络接口的IP/MAC/MTU等属性1478+- `InterfaceInfo`描述网络接口的IP/MAC/MTU等属性
1479-- `HccnConfig`存储HCCN网络配置键值对1479+- `HccnConfig`存储HCCN网络配置键值对
1480-- `Snapshot`支持设备状态的保存和恢复1480+- `Snapshot`支持设备状态的保存和恢复
1481 1481 
1482##### 控制平面 vs 数据平面1482##### 控制平面 vs 数据平面
1483 1483 
@@ -1501,11 +1501,11 @@ HCCP Network API
1501 1501 
1502##### 关键点说明1502##### 关键点说明
1503 1503 
1504-1. **RaContext/RaDevice**: 统一上下文实体,支持 RDMA 和 UB 双模式1504+1. **RaContext/RaDevice**: 统一上下文实体,支持 RDMA 和 UB 双模式
1505-2. **RaJetty/RaJfc/RaQP/RaCQ**: UB 模式下的 QP/CQ 等价物1505+2. **RaJetty/RaJfc/RaQP/RaCQ**: UB 模式下的 QP/CQ 等价物
1506-3. **RaLmem/RaRmem/RaMR**: UB 模式下的本地/远端内存管理1506+3. **RaLmem/RaRmem/RaMR**: UB 模式下的本地/远端内存管理
1507-4. **RaTp**: UB 传输路径管理1507+4. **RaTp**: UB 传输路径管理
1508-5. **RaTokenId**: 安全通信令牌1508+5. **RaTokenId**: 安全通信令牌
1509 1509 
1510##### 关键属性补充1510##### 关键属性补充
1511 1511 
@@ -1576,7 +1576,7 @@ HCCP Network API
1576| RaSocketEvent.events | `RaEpollCtlAdd`, `RaEpollCtlMod`, `RaEpollCtlDel` 管理 |1576| RaSocketEvent.events | `RaEpollCtlAdd`, `RaEpollCtlMod`, `RaEpollCtlDel` 管理 |
1577| RaDevice.direct-flag | `RaNdaGetDirectFlag` 检查NDA支持 |1577| RaDevice.direct-flag | `RaNdaGetDirectFlag` 检查NDA支持 |
1578 1578 
1579-## 回调与报告关系建模1579+## 5. 回调与报告关系建模
1580 1580 
1581```mermaid1581```mermaid
1582erDiagram1582erDiagram
@@ -1628,11 +1628,11 @@ erDiagram
1628 1628 
1629```1629```
1630 1630 
1631-### 关键关系说明(回调与报告关系建模)1631+### 5.1 关键关系说明(回调与报告关系建模)
1632 1632 
1633-==**设备**==执行到 CallbackTask 时,触发 Host Runner 线程执行回调1633+==**设备**==执行到 CallbackTask 时,触发 Host Runner 线程执行回调
1634 1634 
1635-#### 配套接口映射表(回调与报告关系建模)1635+#### 5.1.1 配套接口映射表(回调与报告关系建模)
1636 1636 
1637| 实体 | 关键管理接口 |1637| 实体 | 关键管理接口 |
1638| ------------- | --------------------------------------------------------------------- |1638| ------------- | --------------------------------------------------------------------- |
@@ -1640,7 +1640,7 @@ erDiagram
1640| ReportChannel | `rtSubscribeReport`, `rtUnSubscribeReport` |1640| ReportChannel | `rtSubscribeReport`, `rtUnSubscribeReport` |
1641| Runner | `rtProcessReport` |1641| Runner | `rtProcessReport` |
1642 1642 
1643-## 基础`设备`模型细粒度底层扩展1643+## 6. 基础`设备`模型细粒度底层扩展
1644 1644 
1645```mermaid1645```mermaid
1646erDiagram1646erDiagram
@@ -1701,52 +1701,52 @@ erDiagram
1701 }1701 }
1702```1702```
1703 1703 
1704-### 关键关系说明(基础`设备`模型细粒度底层扩展)1704+### 6.1 关键关系说明(基础`设备`模型细粒度底层扩展)
1705 1705 
1706**设备调度器层次结构**1706**设备调度器层次结构**
1707 1707 
1708-- `TaskSchedulerDevice`是设备调度器的抽象,一个Device可包含多个调度器1708+- `TaskSchedulerDevice`是设备调度器的抽象,一个Device可包含多个调度器
1709- 调度器类型包括:`Scalar`(标量处理器)、`CCU`(集合通信单元)、`CPU`(AI CPU)1709- 调度器类型包括:`Scalar`(标量处理器)、`CCU`(集合通信单元)、`CPU`(AI CPU)
1710-- 不同类型调度器负责不同计算任务类型1710+- 不同类型调度器负责不同计算任务类型
1711 1711 
1712**ComputeDie计算单元**1712**ComputeDie计算单元**
1713 1713 
1714-- `ComputeDie`是计算单元的抽象,继承关系表示计算单元类型1714+- `ComputeDie`是计算单元的抽象,继承关系表示计算单元类型
1715-- `Vector`:向量计算单元,处理向量运算1715+- `Vector`:向量计算单元,处理向量运算
1716-- `Cube`:立方计算单元,处理矩阵运算1716+- `Cube`:立方计算单元,处理矩阵运算
1717-- `HybridComputeDie`:混合计算单元,同时支持Vector和Cube1717+- `HybridComputeDie`:混合计算单元,同时支持Vector和Cube
1718 1718 
1719**CCU内部结构**1719**CCU内部结构**
1720 1720 
1721- `xnNum`:XN节点数量(跨节点通信)1721- `xnNum`:XN节点数量(跨节点通信)
1722- `ckeNum`:CKE引擎数量(Checksum引擎)1722- `ckeNum`:CKE引擎数量(Checksum引擎)
1723- `msNum`:MS模块数量(Memory Scheduler)1723- `msNum`:MS模块数量(Memory Scheduler)
1724-- `channelNum`:通信通道数量1724+- `channelNum`:通信通道数量
1725- `version`:CCU版本(v1/v2,决定功能差异)1725- `version`:CCU版本(v1/v2,决定功能差异)
1726 1726 
1727**DeviceStatus状态管理**1727**DeviceStatus状态管理**
1728 1728 
1729-- `overflow-status`:溢出状态1729+- `overflow-status`:溢出状态
1730-- `synchronize-strategy`:同步策略配置1730+- `synchronize-strategy`:同步策略配置
1731-- `synchronize-timeout`:同步超时设置1731+- `synchronize-timeout`:同步超时设置
1732-- `capability-mask`:设备能力掩码1732+- `capability-mask`:设备能力掩码
1733-- `run-by-host`:是否运行在Host模式1733+- `run-by-host`:是否运行在Host模式
1734-- `ts-core`:调度器核心数量1734+- `ts-core`:调度器核心数量
1735-- `online-status`:设备在线状态1735+- `online-status`:设备在线状态
1736 1736 
1737**Scalar调度关系**1737**Scalar调度关系**
1738 1738 
1739-- `Scalar`调度器管理`ComputeDie`的执行1739+- `Scalar`调度器管理`ComputeDie`的执行
1740-- 不同ComputeDie类型对应不同的计算负载1740+- 不同ComputeDie类型对应不同的计算负载
1741 1741 
1742-#### 配套接口映射表(基础设备模型细粒度底层扩展)1742+#### 6.1.1 配套接口映射表(基础设备模型细粒度底层扩展)
1743 1743 
1744| 实体 | 关键管理接口 |1744| 实体 | 关键管理接口 |
1745| ------------------- | ---------------------------------- |1745| ------------------- | ---------------------------------- |
1746| TaskSchedulerDevice | `rtGetDeviceInfo`, `rtSetTsDevice` |1746| TaskSchedulerDevice | `rtGetDeviceInfo`, `rtSetTsDevice` |
1747| DeviceStatus | `rtGetRunMode` |1747| DeviceStatus | `rtGetRunMode` |
1748 1748 
1749-## Kernel 运行时关系建模1749+## 7. Kernel 运行时关系建模
1750 1750 
1751```mermaid1751```mermaid
1752erDiagram1752erDiagram
@@ -1791,21 +1791,21 @@ erDiagram
1791 }1791 }
1792```1792```
1793 1793 
1794-### 关键关系说明(Kernel 运行时关系建模)1794+### 7.1 关键关系说明(Kernel 运行时关系建模)
1795 1795 
1796**KernelBinary加载流程**1796**KernelBinary加载流程**
1797 1797 
1798-1. `KernelBinary`存储.o/.so等二进制文件路径和创建进程PID1798+1. `KernelBinary`存储.o/.so等二进制文件路径和创建进程PID
1799-2. `rtBinaryLoadFromFile`或`rtBinaryLoadFromData`将二进制加载到设备内存1799+2. `rtBinaryLoadFromFile`或`rtBinaryLoadFromData`将二进制加载到设备内存
1800-3. 加载后生成`KernelBinaryHandle`,包含handle-id和kernel-id1800+3. 加载后生成`KernelBinaryHandle`,包含handle-id和kernel-id
1801-4. `KernelFuncHandle`表示二进制中的具体函数,包含func-name、kernel-name、地址信息1801+4. `KernelFuncHandle`表示二进制中的具体函数,包含func-name、kernel-name、地址信息
1802 1802 
1803**Kernel函数调用关系**1803**Kernel函数调用关系**
1804 1804 
1805-- `aic-addr`是AI Core函数地址1805+- `aic-addr`是AI Core函数地址
1806-- `aiv-addr`是AI Vector函数地址1806+- `aiv-addr`是AI Vector函数地址
1807-- `KernelFuncArgsHandle`存储函数参数信息1807+- `KernelFuncArgsHandle`存储函数参数信息
1808-- `KernelFuncArgsParamHandle`记录参数大小和是否占位符1808+- `KernelFuncArgsParamHandle`记录参数大小和是否占位符
1809- `KernelLaunchCfg`配置Kernel启动参数(block/grid等)1809- `KernelLaunchCfg`配置Kernel启动参数(block/grid等)
1810 1810 
1811**Kernel生命周期**1811**Kernel生命周期**
@@ -1818,11 +1818,11 @@ rtCreateBinary -> rtBinaryLoad -> rtBinaryGetFunction
1818 1818 
1819**参数管理机制**1819**参数管理机制**
1820 1820 
1821-- args-handle支持device和host两种类型1821+- args-handle支持device和host两种类型
1822- is-place-holder标识参数是否为占位符(延迟绑定)1822- is-place-holder标识参数是否为占位符(延迟绑定)
1823-- param-size记录单个参数大小1823+- param-size记录单个参数大小
1824 1824 
1825-#### 配套接口映射表(Kernel 运行时关系建模)1825+#### 7.1.1 配套接口映射表(Kernel 运行时关系建模)
1826 1826 
1827| 实体 | 关键管理接口 |1827| 实体 | 关键管理接口 |
1828| ------------------ | --------------------------------------------------------------------------------------------------------------- |1828| ------------------ | --------------------------------------------------------------------------------------------------------------- |
@@ -1830,4 +1830,4 @@ rtCreateBinary -> rtBinaryLoad -> rtBinaryGetFunction
1830| KernelBinaryHandle | `rtBinaryLoad`, `rtBinaryUnLoad`,`rtBinaryLoadFromFile`,`rtBinaryLoadFromData` |1830| KernelBinaryHandle | `rtBinaryLoad`, `rtBinaryUnLoad`,`rtBinaryLoadFromFile`,`rtBinaryLoadFromData` |
1831| KernelFuncHandle | `rtBinaryGetFunction`, `rtBinaryGetFunctionByEntry`,`rtGetFunctionAddr`,`rtGetFunctionName`,`rtRegisterCpuFunc` |1831| KernelFuncHandle | `rtBinaryGetFunction`, `rtBinaryGetFunctionByEntry`,`rtGetFunctionAddr`,`rtGetFunctionName`,`rtRegisterCpuFunc` |
1832 1832 
1833-## 模型加载关系建模1833+## 8. 模型加载关系建模
Mtest/hccl_vm/include/runnerdb/db_sim_sqlite_db.h+1-1
@@ -235,7 +235,7 @@ public:
235 return *static_cast<sim::SqliteTable<T>*>(it->second);235 return *static_cast<sim::SqliteTable<T>*>(it->second);
236 }236 }
237 std::string name = std::string("Unregistered_") + typeid(T).name();237 std::string name = std::string("Unregistered_") + typeid(T).name();
238- fprintf(stderr, "[SimRunnerSqliteDB::GetTable] Lazily registering unregistered type: %s\n", name.c_str());238+ HCCL_VM_WARN("[SimRunnerSqliteDB::GetTable] Lazily registering unregistered type: {}", name.c_str());
239 auto table = std::make_unique<sim::SqliteTable<T>>(m_db.GetDb(), name);239 auto table = std::make_unique<sim::SqliteTable<T>>(m_db.GetDb(), name);
240 auto* rawPtr = table.get();240 auto* rawPtr = table.get();
241 m_lazyTables.push_back(std::move(table));241 m_lazyTables.push_back(std::move(table));
Mtest/hccl_vm/include/runnerdb/sim_sqlite_table.h+16-14
@@ -27,6 +27,8 @@
27#include <unordered_map>27#include <unordered_map>
28#include <vector>28#include <vector>
29 29 
30+#include "sim_log.h"
31+ 
30namespace sim {32namespace sim {
31inline std::shared_mutex& GetConnectionMutex() {33inline std::shared_mutex& GetConnectionMutex() {
32 static std::shared_mutex s_connMtx;34 static std::shared_mutex s_connMtx;
@@ -74,7 +76,7 @@ public:
74 sqlite3_stmt* rawStmt;76 sqlite3_stmt* rawStmt;
75 int rc = sqlite3_prepare_v2(m_db, sql.c_str(), -1, &rawStmt, nullptr);77 int rc = sqlite3_prepare_v2(m_db, sql.c_str(), -1, &rawStmt, nullptr);
76 if (rc != SQLITE_OK) {78 if (rc != SQLITE_OK) {
77- fprintf(stderr, "[SqliteTable::Add] SQL prepare failed [%s]: %s (rc=%d)\n",79+ HCCL_VM_ERROR("[SqliteTable::Add] SQL prepare failed [{}]: {} (rc={})",
78 m_tableName.c_str(), sqlite3_errmsg(m_db), rc);80 m_tableName.c_str(), sqlite3_errmsg(m_db), rc);
79 return 0;81 return 0;
80 }82 }
@@ -98,7 +100,7 @@ public:
98 connLock.unlock();100 connLock.unlock();
99 101 
100 if (!isRetryable || attempt == kMaxRetries) {102 if (!isRetryable || attempt == kMaxRetries) {
101- fprintf(stderr, "[SqliteTable::Add] SQL insert failed [%s]: rc=%d, errc=%d, ext=%d, msg=%s\n",103+ HCCL_VM_ERROR("[SqliteTable::Add] SQL insert failed [{}]: rc={}, errc={}, ext={}, msg={}",
102 m_tableName.c_str(), stepRc, errc, extErrc, sqlite3_errmsg(m_db));104 m_tableName.c_str(), stepRc, errc, extErrc, sqlite3_errmsg(m_db));
103 return 0;105 return 0;
104 }106 }
@@ -131,7 +133,7 @@ public:
131 sqlite3_stmt* rawStmt;133 sqlite3_stmt* rawStmt;
132 if (sqlite3_prepare_v2(m_db, sql.c_str(), -1, &rawStmt, nullptr) != SQLITE_OK) {134 if (sqlite3_prepare_v2(m_db, sql.c_str(), -1, &rawStmt, nullptr) != SQLITE_OK) {
133 sqlite3_exec(m_db, "ROLLBACK", nullptr, nullptr, nullptr);135 sqlite3_exec(m_db, "ROLLBACK", nullptr, nullptr, nullptr);
134- fprintf(stderr, "[SqliteTable::Update] SQL prepare failed [%s]: %s\n",136+ HCCL_VM_ERROR("[SqliteTable::Update] SQL prepare failed [{}]: {}",
135 m_tableName.c_str(), sqlite3_errmsg(m_db));137 m_tableName.c_str(), sqlite3_errmsg(m_db));
136 return false;138 return false;
137 }139 }
@@ -153,7 +155,7 @@ public:
153 connLock.unlock();155 connLock.unlock();
154 156 
155 if (!isRetryable || attempt == kMaxRetries) {157 if (!isRetryable || attempt == kMaxRetries) {
156- fprintf(stderr, "[SqliteTable::Update] SQL step failed [%s]: rc=%d, errc=%d, msg=%s\n",158+ HCCL_VM_ERROR("[SqliteTable::Update] SQL step failed [{}]: rc={}, errc={}, msg={}",
157 m_tableName.c_str(), result, errc, sqlite3_errmsg(m_db));159 m_tableName.c_str(), result, errc, sqlite3_errmsg(m_db));
158 return false;160 return false;
159 }161 }
@@ -176,7 +178,7 @@ public:
176 178 
177 sqlite3_stmt* rawStmt;179 sqlite3_stmt* rawStmt;
178 if (sqlite3_prepare_v2(m_db, sql.c_str(), -1, &rawStmt, nullptr) != SQLITE_OK) {180 if (sqlite3_prepare_v2(m_db, sql.c_str(), -1, &rawStmt, nullptr) != SQLITE_OK) {
179- fprintf(stderr, "[SqliteTable::Delete] SQL prepare failed [%s]: %s\n",181+ HCCL_VM_ERROR("[SqliteTable::Delete] SQL prepare failed [{}]: {}",
180 m_tableName.c_str(), sqlite3_errmsg(m_db));182 m_tableName.c_str(), sqlite3_errmsg(m_db));
181 return false;183 return false;
182 }184 }
@@ -190,7 +192,7 @@ public:
190 192 
191 int errc = sqlite3_errcode(m_db);193 int errc = sqlite3_errcode(m_db);
192 if (errc != SQLITE_BUSY && errc != SQLITE_LOCKED) {194 if (errc != SQLITE_BUSY && errc != SQLITE_LOCKED) {
193- fprintf(stderr, "[SqliteTable::Delete] SQL step failed [%s]: rc=%d, errc=%d\n",195+ HCCL_VM_ERROR("[SqliteTable::Delete] SQL step failed [{}]: rc={}, errc={}",
194 m_tableName.c_str(), result, errc);196 m_tableName.c_str(), result, errc);
195 return false;197 return false;
196 }198 }
@@ -208,7 +210,7 @@ public:
208 210 
209 std::string sql = "DELETE FROM " + m_tableName;211 std::string sql = "DELETE FROM " + m_tableName;
210 if (sqlite3_exec(m_db, sql.c_str(), nullptr, nullptr, nullptr) != SQLITE_OK) {212 if (sqlite3_exec(m_db, sql.c_str(), nullptr, nullptr, nullptr) != SQLITE_OK) {
211- fprintf(stderr, "[SqliteTable::DeleteAll] SQL delete all failed [%s]: %s\n",213+ HCCL_VM_ERROR("[SqliteTable::DeleteAll] SQL delete all failed [{}]: {}",
212 m_tableName.c_str(), sqlite3_errmsg(m_db));214 m_tableName.c_str(), sqlite3_errmsg(m_db));
213 return false;215 return false;
214 }216 }
@@ -246,7 +248,7 @@ public:
246 248 
247 sqlite3_stmt* rawStmt;249 sqlite3_stmt* rawStmt;
248 if (sqlite3_prepare_v2(m_db, sql.c_str(), -1, &rawStmt, nullptr) != SQLITE_OK) {250 if (sqlite3_prepare_v2(m_db, sql.c_str(), -1, &rawStmt, nullptr) != SQLITE_OK) {
249- fprintf(stderr, "[SqliteTable::QueryList] SQL prepare failed [%s]: %s\n",251+ HCCL_VM_ERROR("[SqliteTable::QueryList] SQL prepare failed [{}]: {}",
250 m_tableName.c_str(), sqlite3_errmsg(m_db));252 m_tableName.c_str(), sqlite3_errmsg(m_db));
251 return result;253 return result;
252 }254 }
@@ -283,7 +285,7 @@ public:
283 connLock.lock();285 connLock.lock();
284 }286 }
285 287 
286- fprintf(stderr, "[SqliteTable::QueryList] SQLITE_BUSY after %d retries [%s]\n",288+ HCCL_VM_ERROR("[SqliteTable::QueryList] SQLITE_BUSY after {} retries [{}]",
287 kMaxRetries, m_tableName.c_str());289 kMaxRetries, m_tableName.c_str());
288 return result;290 return result;
289 }291 }
@@ -295,7 +297,7 @@ public:
295 for (int attempt = 0; attempt <= kMaxRetries; ++attempt) {297 for (int attempt = 0; attempt <= kMaxRetries; ++attempt) {
296 sqlite3_stmt* rawStmt;298 sqlite3_stmt* rawStmt;
297 if (sqlite3_prepare_v2(m_db, sql.c_str(), -1, &rawStmt, nullptr) != SQLITE_OK) {299 if (sqlite3_prepare_v2(m_db, sql.c_str(), -1, &rawStmt, nullptr) != SQLITE_OK) {
298- fprintf(stderr, "[SqliteTable::Query] SQL prepare failed [%s]: %s\n",300+ HCCL_VM_ERROR("[SqliteTable::Query] SQL prepare failed [{}]: {}",
299 m_tableName.c_str(), sqlite3_errmsg(m_db));301 m_tableName.c_str(), sqlite3_errmsg(m_db));
300 return {ValueType{}, false};302 return {ValueType{}, false};
301 }303 }
@@ -333,7 +335,7 @@ public:
333 connLock.lock();335 connLock.lock();
334 }336 }
335 337 
336- fprintf(stderr, "[SqliteTable::Query] SQLITE_BUSY after %d retries [%s]\n",338+ HCCL_VM_ERROR("[SqliteTable::Query] SQLITE_BUSY after {} retries [{}]",
337 kMaxRetries, m_tableName.c_str());339 kMaxRetries, m_tableName.c_str());
338 return {ValueType{}, false};340 return {ValueType{}, false};
339 }341 }
@@ -349,7 +351,7 @@ private:
349 " (id INTEGER PRIMARY KEY AUTOINCREMENT, data BLOB NOT NULL)";351 " (id INTEGER PRIMARY KEY AUTOINCREMENT, data BLOB NOT NULL)";
350 352 
351 if (sqlite3_exec(m_db, sql.c_str(), nullptr, nullptr, nullptr) != SQLITE_OK) {353 if (sqlite3_exec(m_db, sql.c_str(), nullptr, nullptr, nullptr) != SQLITE_OK) {
352- fprintf(stderr, "[SqliteTable::CreateTableIfNotExists] SQL create table failed [%s]: %s\n",354+ HCCL_VM_ERROR("[SqliteTable::CreateTableIfNotExists] SQL create table failed [{}]: {}",
353 m_tableName.c_str(), sqlite3_errmsg(m_db));355 m_tableName.c_str(), sqlite3_errmsg(m_db));
354 }356 }
355 }357 }
@@ -359,7 +361,7 @@ private:
359 361 
360 sqlite3_stmt* rawStmt;362 sqlite3_stmt* rawStmt;
361 if (sqlite3_prepare_v2(m_db, sql.c_str(), -1, &rawStmt, nullptr) != SQLITE_OK) {363 if (sqlite3_prepare_v2(m_db, sql.c_str(), -1, &rawStmt, nullptr) != SQLITE_OK) {
362- fprintf(stderr, "[SqliteTable::FindInternal] SQL prepare failed [%s]: %s\n",364+ HCCL_VM_ERROR("[SqliteTable::FindInternal] SQL prepare failed [{}]: {}",
363 m_tableName.c_str(), sqlite3_errmsg(m_db));365 m_tableName.c_str(), sqlite3_errmsg(m_db));
364 return false;366 return false;
365 }367 }
@@ -467,7 +469,7 @@ public:
467 if (sqlite3_open_v2(actualPath.c_str(), &m_db,469 if (sqlite3_open_v2(actualPath.c_str(), &m_db,
468 SQLITE_OPEN_READWRITE | SQLITE_OPEN_CREATE | SQLITE_OPEN_URI,470 SQLITE_OPEN_READWRITE | SQLITE_OPEN_CREATE | SQLITE_OPEN_URI,
469 nullptr) != SQLITE_OK) {471 nullptr) != SQLITE_OK) {
470- fprintf(stderr, "[SqliteDatabase] CRITICAL: Failed to open SQLite database: %s\n",472+ HCCL_VM_ERROR("[SqliteDatabase] CRITICAL: Failed to open SQLite database: {}",
471 sqlite3_errmsg(m_db));473 sqlite3_errmsg(m_db));
472 } else {474 } else {
473 sqlite3_exec(m_db, "PRAGMA journal_mode = WAL", nullptr, nullptr, nullptr);475 sqlite3_exec(m_db, "PRAGMA journal_mode = WAL", nullptr, nullptr, nullptr);
Mtest/hccl_vm/include/sim_log.h+12-6
@@ -22,45 +22,51 @@
22 22 
23extern spdlog::logger* g_logger;23extern spdlog::logger* g_logger;
24 24 
25+#ifdef HCCL_VM_MODULE
26+#define HCCL_VM_TAG "[HCCL-VM-" HCCL_VM_MODULE "]"
27+#else
28+#define HCCL_VM_TAG "[HCCL-VM]"
29+#endif
30+ 
25#define HCCL_VM_TRACE(...) \31#define HCCL_VM_TRACE(...) \
26 do { \32 do { \
27 if (g_logger != nullptr) { \33 if (g_logger != nullptr) { \
28- SPDLOG_LOGGER_TRACE(g_logger, __VA_ARGS__); \34+ SPDLOG_LOGGER_TRACE(g_logger, HCCL_VM_TAG __VA_ARGS__); \
29 } \35 } \
30 } while(0)36 } while(0)
31 37 
32#define HCCL_VM_DEBUG(...) \38#define HCCL_VM_DEBUG(...) \
33 do { \39 do { \
34 if (g_logger != nullptr) { \40 if (g_logger != nullptr) { \
35- SPDLOG_LOGGER_DEBUG(g_logger, __VA_ARGS__); \41+ SPDLOG_LOGGER_DEBUG(g_logger, HCCL_VM_TAG __VA_ARGS__); \
36 } \42 } \
37 } while(0)43 } while(0)
38 44 
39#define HCCL_VM_INFO(...) \45#define HCCL_VM_INFO(...) \
40 do { \46 do { \
41 if (g_logger != nullptr) { \47 if (g_logger != nullptr) { \
42- SPDLOG_LOGGER_INFO(g_logger, __VA_ARGS__); \48+ SPDLOG_LOGGER_INFO(g_logger, HCCL_VM_TAG __VA_ARGS__); \
43 } \49 } \
44 } while(0)50 } while(0)
45 51 
46#define HCCL_VM_WARN(...) \52#define HCCL_VM_WARN(...) \
47 do { \53 do { \
48 if (g_logger != nullptr) { \54 if (g_logger != nullptr) { \
49- SPDLOG_LOGGER_WARN(g_logger, __VA_ARGS__); \55+ SPDLOG_LOGGER_WARN(g_logger, HCCL_VM_TAG __VA_ARGS__); \
50 } \56 } \
51 } while(0)57 } while(0)
52 58 
53#define HCCL_VM_ERROR(...) \59#define HCCL_VM_ERROR(...) \
54 do { \60 do { \
55 if (g_logger != nullptr) { \61 if (g_logger != nullptr) { \
56- SPDLOG_LOGGER_ERROR(g_logger, __VA_ARGS__); \62+ SPDLOG_LOGGER_ERROR(g_logger, HCCL_VM_TAG __VA_ARGS__); \
57 } \63 } \
58 } while(0)64 } while(0)
59 65 
60#define HCCL_VM_CRITICAL(...) \66#define HCCL_VM_CRITICAL(...) \
61 do { \67 do { \
62 if (g_logger != nullptr) { \68 if (g_logger != nullptr) { \
63- SPDLOG_LOGGER_CRITICAL(g_logger, __VA_ARGS__); \69+ SPDLOG_LOGGER_CRITICAL(g_logger, HCCL_VM_TAG __VA_ARGS__); \
64 } \70 } \
65 } while(0)71 } while(0)
66 72 
Mtest/hccl_vm/include/sim_process_syncer.h+7-5
@@ -21,6 +21,8 @@
21#include <thread>21#include <thread>
22#include <unistd.h>22#include <unistd.h>
23 23 
24+#include "sim_log.h"
25+ 
24namespace sim {26namespace sim {
25// Sentinel value written to proxy_ready to signal runner to exit.27// Sentinel value written to proxy_ready to signal runner to exit.
26// Normal rounds use values in [1, UINT32_MAX-1].28// Normal rounds use values in [1, UINT32_MAX-1].
@@ -135,23 +137,23 @@ private:
135 137 
136 int fd = ::open(tmpPath.c_str(), O_WRONLY | O_CREAT | O_TRUNC, 0644);138 int fd = ::open(tmpPath.c_str(), O_WRONLY | O_CREAT | O_TRUNC, 0644);
137 if (fd < 0) {139 if (fd < 0) {
138- std::fprintf(stderr, "[ProcessSyncer] open tmp failed: %s, error: %s\n",140+ HCCL_VM_ERROR("[ProcessSyncer] open tmp failed: {}, error: {}",
139- tmpPath.c_str(), std::strerror(errno));141+ tmpPath.c_str(), std::strerror(errno));
140 return;142 return;
141 }143 }
142 144 
143 ssize_t written = ::write(fd, buf, n);145 ssize_t written = ::write(fd, buf, n);
144 if (written != n) {146 if (written != n) {
145- std::fprintf(stderr, "[ProcessSyncer] write failed: %s\n", std::strerror(errno));147+ HCCL_VM_ERROR("[ProcessSyncer] write failed: {}", std::strerror(errno));
146 }148 }
147 149 
148 if (::fsync(fd) != 0) {150 if (::fsync(fd) != 0) {
149- std::fprintf(stderr, "[ProcessSyncer] fsync failed: %s\n", std::strerror(errno));151+ HCCL_VM_ERROR("[ProcessSyncer] fsync failed: {}", std::strerror(errno));
150 }152 }
151 ::close(fd);153 ::close(fd);
152 154 
153 if (::rename(tmpPath.c_str(), filepath.c_str()) != 0) {155 if (::rename(tmpPath.c_str(), filepath.c_str()) != 0) {
154- std::fprintf(stderr, "[ProcessSyncer] rename failed: %s\n", std::strerror(errno));156+ HCCL_VM_ERROR("[ProcessSyncer] rename failed: {}", std::strerror(errno));
155 }157 }
156 }158 }
157 159 
Mtest/hccl_vm/src/README.md+5-5
@@ -2,11 +2,11 @@
2 2 
3[toc]3[toc]
4 4 
5-## 核心功能介绍5+## 1 核心功能介绍
6 6 
7-### 1.启动模拟环境7+### 1.1启动模拟环境
8 8 
9-#### 查看当前所有表名称9+#### 1.1.1 查看当前所有表名称
10 10 
11```bash11```bash
12(hvm)$> hccl-vm table show all12(hvm)$> hccl-vm table show all
@@ -19,7 +19,7 @@ Device
19(hvm)$>19(hvm)$>
20```20```
21 21 
22-#### 查看指定表内容22+#### 1.1.2 查看指定表内容
23 23 
24```bash24```bash
25(hvm)$> hccl-vm table show Device25(hvm)$> hccl-vm table show Device
@@ -32,7 +32,7 @@ Device
32(hvm)$>32(hvm)$>
33```33```
34 34 
35-#### 更新指定表指定行指定内容(目前只支持devicesoc_version修改)35+#### 1.1.3 更新指定表指定行指定内容(目前只支持devicesoc_version修改)
36 36 
37```bash37```bash
38(hvm)$> hccl-vm table show Device38(hvm)$> hccl-vm table show Device
Mtest/hccl_vm/src/cmd/CMakeLists.txt+3-0
@@ -62,3 +62,6 @@ target_link_libraries(cmd PRIVATE
62if(TARGET third_party_yaml_cpp)62if(TARGET third_party_yaml_cpp)
63 add_dependencies(cmd third_party_yaml_cpp)63 add_dependencies(cmd third_party_yaml_cpp)
64endif()64endif()
65+ 
66+# 日志染色: 模块 tag
67+target_compile_definitions(cmd PRIVATE HCCL_VM_MODULE="CMD")
Mtest/hccl_vm/src/cmd/cmd_base_utils.cc+72-72
@@ -51,6 +51,7 @@
51#include "store_sim_device_memory_manager.h"51#include "store_sim_device_memory_manager.h"
52#include "sim_process_syncer.h"52#include "sim_process_syncer.h"
53#include "db_sim_runner_common.h"53#include "db_sim_runner_common.h"
54+#include "sim_common_api.h"
54#include "db_sim_runner_db.h"55#include "db_sim_runner_db.h"
55#include "yaml-cpp/yaml.h"56#include "yaml-cpp/yaml.h"
56 57 
@@ -93,7 +94,7 @@ static void RemoveMessageQueue(const char *name)
93{ 94{
94 const std::string mqName = ToPosixMqName(name); 95 const std::string mqName = ToPosixMqName(name);
95 if (mq_unlink(mqName.c_str()) == -1 && errno != ENOENT) { 96 if (mq_unlink(mqName.c_str()) == -1 && errno != ENOENT) {
96- HCCL_VM_WARN("[HVM] {}", MakeMqErrorMessage("remove", mqName, errno)); 97+ HCCL_VM_WARN("{}", MakeMqErrorMessage("remove", mqName, errno));
97 } 98 }
98} 99}
99 100 
@@ -282,27 +283,23 @@ bool TryParseAivRankIdFromTaskFileName(const std::string &fileName, uint32_t &ra
282 283 
283HcclVmResult ValidateAivTaskJsonByRank(const std::vector<sim::Rank> &allRank) 284HcclVmResult ValidateAivTaskJsonByRank(const std::vector<sim::Rank> &allRank)
284{ 285{
285- const char *installDirEnv = std::getenv("HCCL_VM_INSTALL_DIR"); 286+ const std::string &installDir = InstallPath::GetHcclVmInstallAbsPath();
286- if (installDirEnv == nullptr || installDirEnv[0] == '\0') {
287- HCCL_VM_ERROR("[HVM] env HCCL_VM_INSTALL_DIR is not set in AIV mode.");
288- return HcclVmResult::HCCL_SIM_HOST_ERROR_CMD;
289- }
290 287 
291 if (allRank.empty()) { 288 if (allRank.empty()) {
292 return HcclVmResult::HCCL_SIM_HOST_SUCCESS_CMD; 289 return HcclVmResult::HCCL_SIM_HOST_SUCCESS_CMD;
293 } 290 }
294 291 
295- const fs::path dataDir = fs::path(installDirEnv) / "data"; 292+ const fs::path dataDir = fs::path(InstallPath::ResolveToInstallRoot("data"));
296 std::error_code ec; 293 std::error_code ec;
297 if (!fs::exists(dataDir, ec) || ec || !fs::is_directory(dataDir, ec)) { 294 if (!fs::exists(dataDir, ec) || ec || !fs::is_directory(dataDir, ec)) {
298- HCCL_VM_ERROR("[HVM] AIV data directory is missing: {}", dataDir.string()); 295+ HCCL_VM_ERROR("AIV data directory is missing: {}", dataDir.string());
299 return HcclVmResult::HCCL_SIM_HOST_ERROR_CMD; 296 return HcclVmResult::HCCL_SIM_HOST_ERROR_CMD;
300 } 297 }
301 298 
302 std::unordered_set<uint32_t> existingRanks; 299 std::unordered_set<uint32_t> existingRanks;
303 for (const auto &entry : fs::directory_iterator(dataDir, ec)) { 300 for (const auto &entry : fs::directory_iterator(dataDir, ec)) {
304 if (ec) { 301 if (ec) {
305- HCCL_VM_ERROR("[HVM] failed to iterate AIV data directory: {}", dataDir.string()); 302+ HCCL_VM_ERROR("failed to iterate AIV data directory: {}", dataDir.string());
306 return HcclVmResult::HCCL_SIM_HOST_ERROR_CMD; 303 return HcclVmResult::HCCL_SIM_HOST_ERROR_CMD;
307 } 304 }
308 if (!entry.is_regular_file()) { 305 if (!entry.is_regular_file()) {
@@ -317,7 +314,7 @@ HcclVmResult ValidateAivTaskJsonByRank(const std::vector<sim::Rank> &allRank)
317 314 
318 for (const auto &rank : allRank) { 315 for (const auto &rank : allRank) {
319 if (existingRanks.find(rank.rank_id) == existingRanks.end()) { 316 if (existingRanks.find(rank.rank_id) == existingRanks.end()) {
320- HCCL_VM_ERROR("[HVM] missing AIV task json file for rank {}", rank.rank_id); 317+ HCCL_VM_ERROR("missing AIV task json file for rank {}", rank.rank_id);
321 return HcclVmResult::HCCL_SIM_HOST_ERROR_CMD; 318 return HcclVmResult::HCCL_SIM_HOST_ERROR_CMD;
322 } 319 }
323 } 320 }
@@ -356,7 +353,7 @@ std::string ArgvToString(int argc, char *argv[]) {
356} 353}
357 354 
358void RemoveFromLDPreload(const std::string& targetValue) { 355void RemoveFromLDPreload(const std::string& targetValue) {
359- HCCL_VM_DEBUG("[HVM] clean LD_PRELOAD env: {}", targetValue); 356+ HCCL_VM_DEBUG("clean LD_PRELOAD env: {}", targetValue);
360 const char* curVal = std::getenv("LD_PRELOAD"); 357 const char* curVal = std::getenv("LD_PRELOAD");
361 358 
362 if (curVal == nullptr) { 359 if (curVal == nullptr) {
@@ -397,14 +394,14 @@ void RemoveFromLDPreload(const std::string& targetValue) {
397 394 
398std::string FileInModelDir(const std::string& fileName) { 395std::string FileInModelDir(const std::string& fileName) {
399 if (fileName == "ranktable") { 396 if (fileName == "ranktable") {
400- std::string clusterInfo = GetBinLocation() + "/" + fileName + ".json"; 397+ std::string clusterInfo = InstallPath::ResolveToInstallRoot(fileName + ".json");
401 std::ifstream f(clusterInfo.c_str()); 398 std::ifstream f(clusterInfo.c_str());
402 if (!f.good()) { 399 if (!f.good()) {
403 return "config ranktable, but rank table file not found: " + clusterInfo; 400 return "config ranktable, but rank table file not found: " + clusterInfo;
404 } 401 }
405 return ""; 402 return "";
406 } 403 }
407- std::string filePath = GetBinLocation() + "/config/topo_meta/" + fileName + ".yaml"; 404+ std::string filePath = InstallPath::ResolveToInstallRoot("config/topo_meta/" + fileName + ".yaml");
408 auto fileExistd = [&]()->bool { 405 auto fileExistd = [&]()->bool {
409 std::ifstream f(filePath.c_str()); 406 std::ifstream f(filePath.c_str());
410 return f.good(); 407 return f.good();
@@ -423,33 +420,33 @@ std::string GenerateClusterTopo(const std::string& topoFileName) {
423 topoName.erase(topoName.find(".yaml"), 5); 420 topoName.erase(topoName.find(".yaml"), 5);
424 } 421 }
425 422 
426- std::string generateShellPath = GetBinLocation() + "/script/generate_cluster_topo.sh"; 423+ std::string generateShellPath = InstallPath::ResolveToInstallRoot("script/generate_cluster_topo.sh");
427 if (!fs::exists(generateShellPath)) { 424 if (!fs::exists(generateShellPath)) {
428- HCCL_VM_ERROR("[HVM] generate_cluster_topo.sh not found: {}", generateShellPath); 425+ HCCL_VM_ERROR("generate_cluster_topo.sh not found: {}", generateShellPath);
429 return "[HVM] generate_cluster_topo.sh not found: " + generateShellPath; 426 return "[HVM] generate_cluster_topo.sh not found: " + generateShellPath;
430 } 427 }
431- std::string clusterConfigFilePath = GetBinLocation() + "/config/cluster/" + topoName + ".yaml"; 428+ std::string clusterConfigFilePath = InstallPath::ResolveToInstallRoot("config/cluster/" + topoName + ".yaml");
432 if (!fs::exists(clusterConfigFilePath)) { 429 if (!fs::exists(clusterConfigFilePath)) {
433- HCCL_VM_ERROR("[HVM] cluster config file not found: {}", clusterConfigFilePath); 430+ HCCL_VM_ERROR("cluster config file not found: {}", clusterConfigFilePath);
434 return "[HVM] cluster config file not found: " + clusterConfigFilePath; 431 return "[HVM] cluster config file not found: " + clusterConfigFilePath;
435 } 432 }
436 // 执行generate_cluster_topo.sh脚本(默认路径),生成集群组网 433 // 执行generate_cluster_topo.sh脚本(默认路径),生成集群组网
437 std::string cmd = "bash " + generateShellPath + " " + clusterConfigFilePath; 434 std::string cmd = "bash " + generateShellPath + " " + clusterConfigFilePath;
438 if (std::system(cmd.c_str()) != 0) { 435 if (std::system(cmd.c_str()) != 0) {
439- HCCL_VM_ERROR("[HVM] generate_cluster_topo.sh failed: {}", cmd); 436+ HCCL_VM_ERROR("generate_cluster_topo.sh failed: {}", cmd);
440 return "[HVM] generate_cluster_topo.sh failed: " + cmd; 437 return "[HVM] generate_cluster_topo.sh failed: " + cmd;
441 } 438 }
442 return ""; 439 return "";
443} 440}
444 441 
445void ShowModel() { 442void ShowModel() {
446- std::string modelPath = GetBinLocation() + "/cluster_model"; 443+ std::string modelPath = InstallPath::ResolveToInstallRoot("cluster_model");
447 if (!fs::exists(modelPath)) { 444 if (!fs::exists(modelPath)) {
448- HCCL_VM_ERROR("[HVM] path not exist -> {}", modelPath); 445+ HCCL_VM_ERROR("path not exist -> {}", modelPath);
449 return; 446 return;
450 } 447 }
451 if (!fs::is_directory(modelPath)) { 448 if (!fs::is_directory(modelPath)) {
452- HCCL_VM_ERROR("[HVM] path not a dict -> {}", modelPath); 449+ HCCL_VM_ERROR("path not a dict -> {}", modelPath);
453 return; 450 return;
454 } 451 }
455 bool hasFiles = false; 452 bool hasFiles = false;
@@ -468,7 +465,7 @@ void ShowModel() {
468 } 465 }
469 } 466 }
470 if (!hasFiles) { 467 if (!hasFiles) {
471- HCCL_VM_WARN("[HVM] there is no model"); 468+ HCCL_VM_WARN("there is no model");
472 } 469 }
473 return; 470 return;
474} 471}
@@ -483,13 +480,13 @@ HcclVmResult InitHvmEnv(const std::string& configClusterDir, uint32_t level, boo
483 void* commPool = sim::MemoryManager::GetInstance().AllocMemByName(480 void* commPool = sim::MemoryManager::GetInstance().AllocMemByName(
484 sim::CommPoolPolicy::kPoolName, sim::CommPoolPolicy::kPoolSize);481 sim::CommPoolPolicy::kPoolName, sim::CommPoolPolicy::kPoolSize);
485 if (commPool == nullptr) {482 if (commPool == nullptr) {
486- HCCL_VM_ERROR("[HVM] create HcclCommPool fail");483+ HCCL_VM_ERROR("create HcclCommPool fail");
487 return HcclVmResult::HCCL_SIM_HOST_ERROR_CMD;484 return HcclVmResult::HCCL_SIM_HOST_ERROR_CMD;
488 }485 }
489 }486 }
490 void *shmptr =sim::MemoryManager::GetInstance().AllocMemByName("HcclAicpuData", sizeof(HcclAicpuData));487 void *shmptr =sim::MemoryManager::GetInstance().AllocMemByName("HcclAicpuData", sizeof(HcclAicpuData));
491 if (shmptr == nullptr) { 488 if (shmptr == nullptr) {
492- HCCL_VM_ERROR("[HVM] Alloc Shared Memory fail "); 489+ HCCL_VM_ERROR("Alloc Shared Memory fail ");
493 return HcclVmResult::HCCL_SIM_HOST_ERROR_CMD; 490 return HcclVmResult::HCCL_SIM_HOST_ERROR_CMD;
494 } 491 }
495 492 
@@ -500,7 +497,7 @@ HcclVmResult InitHvmEnv(const std::string& configClusterDir, uint32_t level, boo
500 std::string checkerTag = "@checker"; 497 std::string checkerTag = "@checker";
501 auto chkInstallRet = InstallUserPlugin(checkerTag); 498 auto chkInstallRet = InstallUserPlugin(checkerTag);
502 if (chkInstallRet != HCCL_SIM_HOST_SUCCESS_CMD) { 499 if (chkInstallRet != HCCL_SIM_HOST_SUCCESS_CMD) {
503- HCCL_VM_ERROR("[HVM] default plugin install fail, please check your plugin path"); 500+ HCCL_VM_ERROR("default plugin install fail, please check your plugin path");
504 } 501 }
505 502 
506 HCCL_VM_INFO("===================================="); 503 HCCL_VM_INFO("====================================");
@@ -525,7 +522,7 @@ HcclVmResult InitHvmCommEnv(const TopoMeta& topoMeta, const std::string& configF
525 ret = AscendClusterTopoParser::GetInstance().InitCommunicationDomain(topoMeta, false); 522 ret = AscendClusterTopoParser::GetInstance().InitCommunicationDomain(topoMeta, false);
526 } else { 523 } else {
527 HCCL_VM_INFO("mock-comm cmd config rank table file, config by ranktable.json"); 524 HCCL_VM_INFO("mock-comm cmd config rank table file, config by ranktable.json");
528- std::string clusterInfo = GetBinLocation() + "/data/ranktable.json"; 525+ std::string clusterInfo = InstallPath::ResolveToInstallRoot("data/ranktable.json");
529 std::ifstream f(clusterInfo.c_str()); 526 std::ifstream f(clusterInfo.c_str());
530 if (!f.good()) { 527 if (!f.good()) {
531 HCCL_VM_ERROR("ranktable.json file not found: {}", clusterInfo); 528 HCCL_VM_ERROR("ranktable.json file not found: {}", clusterInfo);
@@ -535,9 +532,10 @@ HcclVmResult InitHvmCommEnv(const TopoMeta& topoMeta, const std::string& configF
535 ret = AscendClusterTopoParser::GetInstance().ParseRanktableAndInitCommDomain(clusterInfo); 532 ret = AscendClusterTopoParser::GetInstance().ParseRanktableAndInitCommDomain(clusterInfo);
536 } 533 }
537 if (ret != HcclVmResult::HCCL_SIM_SUCCESS) { 534 if (ret != HcclVmResult::HCCL_SIM_SUCCESS) {
538- HCCL_VM_ERROR("[HVM] InitHvmCommEnv failed"); 535+ HCCL_VM_ERROR("InitHvmCommEnv failed");
539 return ret; 536 return ret;
540 } 537 }
538+ setenv("RANK_TABLE_FILE", InstallPath::ResolveToInstallRoot("data/ranktable.json").c_str(), 1);
541 539 
542 return HcclVmResult::HCCL_SIM_HOST_SUCCESS_CMD; 540 return HcclVmResult::HCCL_SIM_HOST_SUCCESS_CMD;
543} 541}
@@ -553,7 +551,7 @@ HcclVmResult HcclVmExit()
553 HcclPluginManager &pluginManager = HcclPluginManager::GetInstance(); 551 HcclPluginManager &pluginManager = HcclPluginManager::GetInstance();
554 auto ret = pluginManager.StopAllPlugins(); 552 auto ret = pluginManager.StopAllPlugins();
555 553 
556- const fs::path backupDir = fs::path(g_binDir) / "data" / MakeDataBackupTimestamp();554+ const fs::path backupDir = fs::path(InstallPath::ResolveToInstallRoot("data/" + MakeDataBackupTimestamp()));
557 BackupAivTaskFiles(backupDir);555 BackupAivTaskFiles(backupDir);
558 556 
559 HCCL_VM_INFO("start Destroy ALL Resources.");557 HCCL_VM_INFO("start Destroy ALL Resources.");
@@ -565,6 +563,7 @@ HcclVmResult HcclVmExit()
565 }563 }
566 int ret1 = system("sudo rm -fr /dev/shm/* 2>/dev/null");564 int ret1 = system("sudo rm -fr /dev/shm/* 2>/dev/null");
567 ret1 |= system("sudo rm -fr /tmp/hccl_sim.db* 2>/dev/null"); 565 ret1 |= system("sudo rm -fr /tmp/hccl_sim.db* 2>/dev/null");
566+ 
568 return ret; 567 return ret;
569} 568}
570 569 
@@ -572,7 +571,7 @@ HcclVmResult InstallUserPlugin(std::string argStr) {
572 // 处理插件tag和路径571 // 处理插件tag和路径
573 HcclVmResult ret {HcclVmResult::HCCL_SIM_HOST_ERROR_CMD}; 572 HcclVmResult ret {HcclVmResult::HCCL_SIM_HOST_ERROR_CMD};
574 if (argStr.empty() || argStr[0] != '@') { 573 if (argStr.empty() || argStr[0] != '@') {
575- HCCL_VM_ERROR("[HVM] plugin tag should start with '@', invalid tag: {}", argStr); 574+ HCCL_VM_ERROR("plugin tag should start with '@', invalid tag: {}", argStr);
576 return ret; 575 return ret;
577 } 576 }
578 argStr.erase(argStr.begin()); 577 argStr.erase(argStr.begin());
@@ -581,13 +580,13 @@ HcclVmResult InstallUserPlugin(std::string argStr) {
581 HcclPluginManager &pluginManager = HcclPluginManager::GetInstance(); 580 HcclPluginManager &pluginManager = HcclPluginManager::GetInstance();
582 ret = pluginManager.RegisterPlugin(argStr);581 ret = pluginManager.RegisterPlugin(argStr);
583 if (ret != HcclVmResult::HCCL_SIM_SUCCESS) {582 if (ret != HcclVmResult::HCCL_SIM_SUCCESS) {
584- HCCL_VM_ERROR("[HVM] Install plugin [{}] failed", argStr);583+ HCCL_VM_ERROR("Install plugin [{}] failed", argStr);
585 return ret;584 return ret;
586 }585 }
587 586 
588 // 装 Runner 时若仅校验模式开着,复用池仍在、大块仍会引流,可能覆盖 Runner 真实数据,告警提示。587 // 装 Runner 时若仅校验模式开着,复用池仍在、大块仍会引流,可能覆盖 Runner 真实数据,告警提示。
589 if (argStr == "runner" && sim::IsCheckOnlyMode()) {588 if (argStr == "runner" && sim::IsCheckOnlyMode()) {
590- HCCL_VM_WARN("[HVM] check-only mode on while runner installed; big-block contents are not guaranteed");589+ HCCL_VM_WARN("check-only mode on while runner installed; big-block contents are not guaranteed");
591 }590 }
592 591 
593 return HcclVmResult::HCCL_SIM_HOST_SUCCESS_CMD;592 return HcclVmResult::HCCL_SIM_HOST_SUCCESS_CMD;
@@ -631,7 +630,7 @@ HcclVmResult UninstallUserPlugin(std::string argStr) {
631 630 
632 for (int i = 0; i < pluginTags.size(); ++i) {631 for (int i = 0; i < pluginTags.size(); ++i) {
633 if (rets[i] != HcclVmResult::HCCL_SIM_SUCCESS) {632 if (rets[i] != HcclVmResult::HCCL_SIM_SUCCESS) {
634- HCCL_VM_ERROR("[HVM] plugin Uninstall fail : {}", pluginTags[i]);633+ HCCL_VM_ERROR("plugin Uninstall fail : {}", pluginTags[i]);
635 return rets[i];634 return rets[i];
636 }635 }
637 }636 }
@@ -645,7 +644,7 @@ void ShowUserPlugin() {
645 listPlugins = pluginManager.GetPluginStatus(); 644 listPlugins = pluginManager.GetPluginStatus();
646 645 
647 if (listPlugins.empty()) { 646 if (listPlugins.empty()) {
648- HCCL_VM_INFO("[HVM] no plugin installed in hccl_vm"); 647+ HCCL_VM_INFO("no plugin installed in hccl_vm");
649 } else { 648 } else {
650 for (auto &plugin : listPlugins) { 649 for (auto &plugin : listPlugins) {
651 HCCL_VM_INFO("{}", plugin);650 HCCL_VM_INFO("{}", plugin);
@@ -656,18 +655,18 @@ void ShowUserPlugin() {
656 655 
657HcclVmResult SetConsoleLogLevel(int level) { 656HcclVmResult SetConsoleLogLevel(int level) {
658 (void)level;657 (void)level;
659- HCCL_VM_WARN("[HVM] set console log level is disabled because ProxyConfig shared memory is removed");658+ HCCL_VM_WARN("set console log level is disabled because ProxyConfig shared memory is removed");
660 return HcclVmResult::HCCL_SIM_HOST_ERROR_CMD;659 return HcclVmResult::HCCL_SIM_HOST_ERROR_CMD;
661} 660}
662 661 
663HcclVmResult SetFileLogLevel(int level) { 662HcclVmResult SetFileLogLevel(int level) {
664 (void)level;663 (void)level;
665- HCCL_VM_WARN("[HVM] set file log level is disabled because ProxyConfig shared memory is removed");664+ HCCL_VM_WARN("set file log level is disabled because ProxyConfig shared memory is removed");
666 return HcclVmResult::HCCL_SIM_HOST_ERROR_CMD;665 return HcclVmResult::HCCL_SIM_HOST_ERROR_CMD;
667} 666}
668 667 
669HcclVmResult ShowCurrentLogLevel() { 668HcclVmResult ShowCurrentLogLevel() {
670- HCCL_VM_WARN("[HVM] show log level is disabled because ProxyConfig shared memory is removed");669+ HCCL_VM_WARN("show log level is disabled because ProxyConfig shared memory is removed");
671 return HcclVmResult::HCCL_SIM_HOST_ERROR_CMD;670 return HcclVmResult::HCCL_SIM_HOST_ERROR_CMD;
672} 671}
673 672 
@@ -683,10 +682,10 @@ HcclVmResult StartHvmCmd() {
683 682 
684 // Child Process(Bash) 683 // Child Process(Bash)
685 // 劫持库存在性判断 684 // 劫持库存在性判断
686- std::string hcclVmbin = g_binDir + "/bin/hccl-vm";685+ std::string hcclVmbin = InstallPath::ResolveToInstallRoot("bin/hccl-vm");
687- std::string proxyPath = g_binDir + "/lib/x86_64/libhccl_proxy_level" + std::to_string(g_hcclVmLevel) + ".so";686+ std::string proxyPath = InstallPath::ResolveToInstallRoot("lib/x86_64/libhccl_proxy_level" + std::to_string(g_hcclVmLevel) + ".so");
688 if (!fs::exists(proxyPath)) { 687 if (!fs::exists(proxyPath)) {
689- HCCL_VM_ERROR("[HVM] [ERROR] proxy hacking .so not found {}, please check your proxy hacking .so:" 688+ HCCL_VM_ERROR("proxy hacking .so not found {}, please check your proxy hacking .so:"
690 "1. Whether the hook library has been successfully built and installed. 2. Whether the simulation level matches the proxy hook library version. Current simulation level: {}, Default simulation level: 2" 689 "1. Whether the hook library has been successfully built and installed. 2. Whether the simulation level matches the proxy hook library version. Current simulation level: {}, Default simulation level: 2"
691 , proxyPath, g_hcclVmLevel); 690 , proxyPath, g_hcclVmLevel);
692 return HCCL_SIM_HOST_ERROR_CMD; 691 return HCCL_SIM_HOST_ERROR_CMD;
@@ -695,7 +694,7 @@ HcclVmResult StartHvmCmd() {
695 // 管道处理的用途是隔绝进程终端在std::cout中的残留 694 // 管道处理的用途是隔绝进程终端在std::cout中的残留
696 int pipefds[2] = {-1, -1}; 695 int pipefds[2] = {-1, -1};
697 if (pipe(pipefds) == -1) { 696 if (pipe(pipefds) == -1) {
698- HCCL_VM_ERROR("[HVM] pipe failed"); 697+ HCCL_VM_ERROR("pipe failed");
699 return HCCL_SIM_HOST_ERROR_CMD; 698 return HCCL_SIM_HOST_ERROR_CMD;
700 } 699 }
701 700 
@@ -712,7 +711,7 @@ HcclVmResult StartHvmCmd() {
712 711 
713 ssize_t written = write(pipefds[1], bashrcHack.c_str(), bashrcHack.size()); 712 ssize_t written = write(pipefds[1], bashrcHack.c_str(), bashrcHack.size());
714 if (written != static_cast<ssize_t>(bashrcHack.size())) { 713 if (written != static_cast<ssize_t>(bashrcHack.size())) {
715- HCCL_VM_ERROR("[HVM] Failed to write full script to pipe"); 714+ HCCL_VM_ERROR("Failed to write full script to pipe");
716 close(pipefds[0]); 715 close(pipefds[0]);
717 close(pipefds[1]); 716 close(pipefds[1]);
718 return HCCL_SIM_HOST_ERROR_CMD; 717 return HCCL_SIM_HOST_ERROR_CMD;
@@ -733,14 +732,15 @@ HcclVmResult StartHvmCmd() {
733 // Fork Bash 732 // Fork Bash
734 pid_t pid = fork(); 733 pid_t pid = fork();
735 if (pid == -1) { 734 if (pid == -1) {
736- auto ret = HcclVmExit(); 735+ auto ret = HcclVmExit();
737- perror("fork failed"); 736+ HCCL_VM_ERROR("fork failed: {}", std::strerror(errno));
738 close(pipefds[0]); 737 close(pipefds[0]);
739 return (ret == HcclVmResult::HCCL_SIM_HOST_SUCCESS_CMD) ? HcclVmResult::HCCL_SIM_HOST_ERROR_CMD : ret; 738 return (ret == HcclVmResult::HCCL_SIM_HOST_SUCCESS_CMD) ? HcclVmResult::HCCL_SIM_HOST_ERROR_CMD : ret;
740 } else if (pid == 0) { 739 } else if (pid == 0) {
741 setenv(HVM_BASH_ENV_KEY.c_str(), g_binDir.c_str(), 1); 740 setenv(HVM_BASH_ENV_KEY.c_str(), g_binDir.c_str(), 1);
742 setenv("LD_PRELOAD", proxyPath.c_str(), 1); 741 setenv("LD_PRELOAD", proxyPath.c_str(), 1);
743- setenv("HCCL_VM_INSTALL_ROOT", g_binDir.c_str(), 1); 742+ setenv("HCCL_VM_INSTALL_ROOT", g_binDir.c_str(), 1);
743+ setenv("RANK_TABLE_FILE", InstallPath::ResolveToInstallRoot("data/ranktable.json").c_str(), 1);
744 execv("/bin/bash", bashArgv); 744 execv("/bin/bash", bashArgv);
745 exit(1); 745 exit(1);
746 } else { 746 } else {
@@ -749,7 +749,7 @@ HcclVmResult StartHvmCmd() {
749 int status; 749 int status;
750 waitpid(pid, &status, 0);750 waitpid(pid, &status, 0);
751 auto ret = HcclVmExit(); 751 auto ret = HcclVmExit();
752- HCCL_VM_INFO("[HVM] Shell exited. Host shutting down."); 752+ HCCL_VM_INFO("Shell exited. Host shutting down.");
753 } 753 }
754 return HcclVmResult::HCCL_SIM_HOST_SUCCESS_CMD; 754 return HcclVmResult::HCCL_SIM_HOST_SUCCESS_CMD;
755} 755}
@@ -772,9 +772,9 @@ void StartHostClient(int argc, char *argv[]) {
772 unsigned int priority = 0; 772 unsigned int priority = 0;
773 bool hasMessage = mqResp.TimedReceive(&buffer, MAX_MSG_SIZE, &priority, deadline, recvdSize); 773 bool hasMessage = mqResp.TimedReceive(&buffer, MAX_MSG_SIZE, &priority, deadline, recvdSize);
774 if (hasMessage) { 774 if (hasMessage) {
775- HCCL_VM_DEBUG("[HVM] Client : Command parsing response received: {}", std::string(buffer, recvdSize)); 775+ HCCL_VM_DEBUG("Client : Command parsing response received: {}", std::string(buffer, recvdSize));
776 } else { 776 } else {
777- HCCL_VM_WARN("[HVM] Client : Command parsing out of time, Client quit"); 777+ HCCL_VM_WARN("Client : Command parsing out of time, Client quit");
778 } 778 }
779} 779}
780 780 
@@ -783,7 +783,7 @@ void ServerListen() {
783 mq_attr attr = MakeMessageQueueAttr(); 783 mq_attr attr = MakeMessageQueueAttr();
784 PosixMessageQueue mqReq(MQ_REQ_NAME, O_CREAT | O_EXCL | O_RDWR, &attr); 784 PosixMessageQueue mqReq(MQ_REQ_NAME, O_CREAT | O_EXCL | O_RDWR, &attr);
785 PosixMessageQueue mqResp(MQ_RESP_NAME, O_CREAT | O_EXCL | O_RDWR, &attr); 785 PosixMessageQueue mqResp(MQ_RESP_NAME, O_CREAT | O_EXCL | O_RDWR, &attr);
786- HCCL_VM_INFO("[HVM] HOST Server listening..."); 786+ HCCL_VM_INFO("HOST Server listening...");
787 787 
788 while(true) { 788 while(true) {
789 char buffer[MAX_MSG_SIZE]; 789 char buffer[MAX_MSG_SIZE];
@@ -792,7 +792,7 @@ void ServerListen() {
792 ssize_t recvdSize = mqReq.Receive(&buffer, MAX_MSG_SIZE, &priority); 792 ssize_t recvdSize = mqReq.Receive(&buffer, MAX_MSG_SIZE, &priority);
793 793 
794 std::string cmd(buffer, recvdSize); 794 std::string cmd(buffer, recvdSize);
795- HCCL_VM_DEBUG("[HVM] HOST : Command parsing request received: {}", cmd); 795+ HCCL_VM_DEBUG("HOST : Command parsing request received: {}", cmd);
796 796 
797 // 执行接收到的命令逻辑 797 // 执行接收到的命令逻辑
798 ParseCommand(cmd); 798 ParseCommand(cmd);
@@ -804,7 +804,7 @@ void ServerListen() {
804 804 
805// 监听proxy和runner进程,中转进程状态 805// 监听proxy和runner进程,中转进程状态
806void RunnerListen() { 806void RunnerListen() {
807- HCCL_VM_INFO("[HVM] Runner listening..."); 807+ HCCL_VM_INFO("Runner listening...");
808 808 
809 sim::ProcessSyncer syncer; 809 sim::ProcessSyncer syncer;
810 syncer.Init(); 810 syncer.Init();
@@ -835,7 +835,7 @@ void RunnerListen() {
835 } 835 }
836 } else if(syncer.getCurrentRound() >= 1){ 836 } else if(syncer.getCurrentRound() >= 1){
837 HCCL_VM_INFO("skip dumping data for round {:d}", syncer.getCurrentRound()); 837 HCCL_VM_INFO("skip dumping data for round {:d}", syncer.getCurrentRound());
838- std::string dataDir = g_binDir + "/data/"; 838+ std::string dataDir = InstallPath::ResolveToInstallRoot("data") + "/";
839 std::remove((dataDir + "runner_hcclvm_instr_data.bin").c_str()); 839 std::remove((dataDir + "runner_hcclvm_instr_data.bin").c_str());
840 std::remove((dataDir + "runner_hcclvm_syn_data.bin").c_str()); 840 std::remove((dataDir + "runner_hcclvm_syn_data.bin").c_str());
841 std::remove((dataDir + "runner_hcclvm_task_data.bin").c_str()); 841 std::remove((dataDir + "runner_hcclvm_task_data.bin").c_str());
@@ -932,11 +932,11 @@ void ParseCommand(std::string& cmd) {
932static void BackupDatabase(const std::string& srcPath) 932static void BackupDatabase(const std::string& srcPath)
933{ 933{
934 if (!fs::exists(srcPath)) { 934 if (!fs::exists(srcPath)) {
935- HCCL_VM_ERROR("[HVM] Source database {} does not exist, skipping backup.", srcPath); 935+ HCCL_VM_INFO("Source database {} does not exist, skipping backup.", srcPath);
936 return; 936 return;
937 } 937 }
938 938 
939- fs::path dataDir = fs::path(g_binDir) / "data"; 939+ fs::path dataDir = fs::path(InstallPath::ResolveToInstallRoot("data"));
940 fs::create_directories(dataDir); 940 fs::create_directories(dataDir);
941 941 
942 auto now = std::chrono::system_clock::now(); 942 auto now = std::chrono::system_clock::now();
@@ -950,9 +950,9 @@ static void BackupDatabase(const std::string& srcPath)
950 950 
951 auto ret = HcclSim::DB::OpDbOps::Instance().Backup(destPath); 951 auto ret = HcclSim::DB::OpDbOps::Instance().Backup(destPath);
952 if (ret != HcclSim::HCCL_SIM_SUCCESS) { 952 if (ret != HcclSim::HCCL_SIM_SUCCESS) {
953- HCCL_VM_ERROR("[HVM] Failed to backup database to {}", destPath); 953+ HCCL_VM_ERROR("Failed to backup database to {}", destPath);
954 } else { 954 } else {
955- HCCL_VM_INFO("[HVM] Database backed up to: {}", destPath); 955+ HCCL_VM_INFO("Database backed up to: {}", destPath);
956 } 956 }
957} 957}
958 958 
@@ -972,7 +972,7 @@ static bool EnsureBackupDir(const fs::path &backupDir)
972 std::error_code ec;972 std::error_code ec;
973 fs::create_directories(backupDir, ec);973 fs::create_directories(backupDir, ec);
974 if (ec) {974 if (ec) {
975- HCCL_VM_WARN("[HVM] failed to create backup directory {}: {}", backupDir.string(), ec.message());975+ HCCL_VM_WARN("failed to create backup directory {}: {}", backupDir.string(), ec.message());
976 return false;976 return false;
977 }977 }
978 return true;978 return true;
@@ -980,7 +980,7 @@ static bool EnsureBackupDir(const fs::path &backupDir)
980 980 
981static void BackupBinFiles(const fs::path &backupDir)981static void BackupBinFiles(const fs::path &backupDir)
982{982{
983- fs::path dataDir = fs::path(g_binDir) / "data";983+ fs::path dataDir = fs::path(InstallPath::ResolveToInstallRoot("data"));
984 std::vector<fs::path> binFiles = {984 std::vector<fs::path> binFiles = {
985 dataDir / "runner_hcclvm_instr_data.bin",985 dataDir / "runner_hcclvm_instr_data.bin",
986 dataDir / "runner_hcclvm_syn_data.bin",986 dataDir / "runner_hcclvm_syn_data.bin",
@@ -1002,18 +1002,18 @@ static void BackupBinFiles(const fs::path &backupDir)
1002 } 1002 }
1003 fs::rename(f, backupDir / f.filename(), ec); 1003 fs::rename(f, backupDir / f.filename(), ec);
1004 if (ec) { 1004 if (ec) {
1005- HCCL_VM_WARN("[HVM] failed to move {} to {}: {}", f.string(),(backupDir / f.filename()).string(), ec.message()); 1005+ HCCL_VM_WARN("failed to move {} to {}: {}", f.string(),(backupDir / f.filename()).string(), ec.message());
1006 } 1006 }
1007 } 1007 }
1008} 1008}
1009 1009 
1010static void BackupAivTaskFiles(const fs::path &backupDir)1010static void BackupAivTaskFiles(const fs::path &backupDir)
1011{1011{
1012- fs::path dataDir = fs::path(g_binDir) / "data";1012+ fs::path dataDir = fs::path(InstallPath::ResolveToInstallRoot("data"));
1013 std::error_code ec;1013 std::error_code ec;
1014 if (!fs::exists(dataDir, ec) || ec || !fs::is_directory(dataDir, ec)) {1014 if (!fs::exists(dataDir, ec) || ec || !fs::is_directory(dataDir, ec)) {
1015 if (ec) {1015 if (ec) {
1016- HCCL_VM_WARN("[HVM] failed to access data directory {}: {}", dataDir.string(), ec.message());1016+ HCCL_VM_WARN("failed to access data directory {}: {}", dataDir.string(), ec.message());
1017 }1017 }
1018 return;1018 return;
1019 }1019 }
@@ -1022,12 +1022,12 @@ static void BackupAivTaskFiles(const fs::path &backupDir)
1022 static const std::string aivFilePrefix = "hcclvm_aiv_";1022 static const std::string aivFilePrefix = "hcclvm_aiv_";
1023 fs::directory_iterator iter(dataDir, ec);1023 fs::directory_iterator iter(dataDir, ec);
1024 if (ec) {1024 if (ec) {
1025- HCCL_VM_WARN("[HVM] failed to iterate data directory {}: {}", dataDir.string(), ec.message());1025+ HCCL_VM_WARN("failed to iterate data directory {}: {}", dataDir.string(), ec.message());
1026 return;1026 return;
1027 }1027 }
1028 for (const fs::directory_iterator end; iter != end; iter.increment(ec)) {1028 for (const fs::directory_iterator end; iter != end; iter.increment(ec)) {
1029 if (ec) {1029 if (ec) {
1030- HCCL_VM_WARN("[HVM] failed to iterate data directory {}: {}", dataDir.string(), ec.message());1030+ HCCL_VM_WARN("failed to iterate data directory {}: {}", dataDir.string(), ec.message());
1031 return;1031 return;
1032 }1032 }
1033 1033 
@@ -1055,7 +1055,7 @@ static void BackupAivTaskFiles(const fs::path &backupDir)
1055 ec.clear();1055 ec.clear();
1056 fs::rename(f, backupDir / f.filename(), ec);1056 fs::rename(f, backupDir / f.filename(), ec);
1057 if (ec) {1057 if (ec) {
1058- HCCL_VM_WARN("[HVM] failed to move {} to {}: {}", f.string(), (backupDir / f.filename()).string(),1058+ HCCL_VM_WARN("failed to move {} to {}: {}", f.string(), (backupDir / f.filename()).string(),
1059 ec.message());1059 ec.message());
1060 }1060 }
1061 }1061 }
@@ -1067,7 +1067,7 @@ static void CleanShmFilesByPrefix(const std::vector<std::string> &prefixes)
1067 std::error_code ec;1067 std::error_code ec;
1068 if (!fs::exists(shmDir, ec) || ec || !fs::is_directory(shmDir, ec)) {1068 if (!fs::exists(shmDir, ec) || ec || !fs::is_directory(shmDir, ec)) {
1069 if (ec) {1069 if (ec) {
1070- HCCL_VM_WARN("[HVM] failed to access shared memory directory {}: {}", shmDir.string(), ec.message());1070+ HCCL_VM_WARN("failed to access shared memory directory {}: {}", shmDir.string(), ec.message());
1071 }1071 }
1072 return;1072 return;
1073 }1073 }
@@ -1076,12 +1076,12 @@ static void CleanShmFilesByPrefix(const std::vector<std::string> &prefixes)
1076 uint32_t failedCount = 0;1076 uint32_t failedCount = 0;
1077 fs::directory_iterator iter(shmDir, ec);1077 fs::directory_iterator iter(shmDir, ec);
1078 if (ec) {1078 if (ec) {
1079- HCCL_VM_WARN("[HVM] failed to iterate shared memory directory {}: {}", shmDir.string(), ec.message());1079+ HCCL_VM_WARN("failed to iterate shared memory directory {}: {}", shmDir.string(), ec.message());
1080 return;1080 return;
1081 }1081 }
1082 for (const fs::directory_iterator end; iter != end; iter.increment(ec)) {1082 for (const fs::directory_iterator end; iter != end; iter.increment(ec)) {
1083 if (ec) {1083 if (ec) {
1084- HCCL_VM_WARN("[HVM] failed to iterate shared memory directory {}: {}", shmDir.string(), ec.message());1084+ HCCL_VM_WARN("failed to iterate shared memory directory {}: {}", shmDir.string(), ec.message());
1085 return;1085 return;
1086 }1086 }
1087 1087 
@@ -1103,21 +1103,21 @@ static void CleanShmFilesByPrefix(const std::vector<std::string> &prefixes)
1103 std::error_code removeEc;1103 std::error_code removeEc;
1104 if (!fs::remove(filePath, removeEc) || removeEc) {1104 if (!fs::remove(filePath, removeEc) || removeEc) {
1105 ++failedCount;1105 ++failedCount;
1106- HCCL_VM_WARN("[HVM] failed to remove shared memory file {}: {}", filePath.string(),1106+ HCCL_VM_WARN("failed to remove shared memory file {}: {}", filePath.string(),
1107 removeEc.message());1107 removeEc.message());
1108 continue;1108 continue;
1109 }1109 }
1110 ++removedCount;1110 ++removedCount;
1111 }1111 }
1112- HCCL_VM_INFO("[HVM] cleaned {} shared memory files in {}, failed {}", removedCount, shmDir.string(), failedCount);1112+ HCCL_VM_INFO("cleaned {} shared memory files in {}, failed {}", removedCount, shmDir.string(), failedCount);
1113}1113}
1114 1114 
1115HcclVmResult ClearDbTables() 1115HcclVmResult ClearDbTables()
1116{ 1116{
1117- const fs::path backupDir = fs::path(g_binDir) / "data" / MakeDataBackupTimestamp();1117+ const fs::path backupDir = fs::path(InstallPath::ResolveToInstallRoot("data/" + MakeDataBackupTimestamp()));
1118 BackupBinFiles(backupDir);1118 BackupBinFiles(backupDir);
1119 BackupAivTaskFiles(backupDir);1119 BackupAivTaskFiles(backupDir);
1120- BackupDatabase(g_binDir + "/data/hccl_vm_data.db"); 1120+ BackupDatabase(InstallPath::ResolveToInstallRoot("data/hccl_vm_data.db"));
1121 1121 
1122 // 1. 清空 OpDbOps (SQLite) 中的静态表数据 1122 // 1. 清空 OpDbOps (SQLite) 中的静态表数据
1123 std::vector<std::string> staticTables = { 1123 std::vector<std::string> staticTables = {
@@ -1191,8 +1191,8 @@ HcclVmResult HcclVmResetCommDomain()
1191HcclVmResult CopyFile(const std::string& clusterDir) {1191HcclVmResult CopyFile(const std::string& clusterDir) {
1192 // 1. 拼接源文件和目标文件路径1192 // 1. 拼接源文件和目标文件路径
1193 auto srcPath = clusterDir + "/superpod0/server0/topo.json";1193 auto srcPath = clusterDir + "/superpod0/server0/topo.json";
1194- fs::create_directories(fs::path(GetBinLocation() + "/data"));1194+ fs::create_directories(fs::path(InstallPath::ResolveToInstallRoot("data")));
1195- auto destPath = GetBinLocation() + "/data/topo.json";1195+ auto destPath = InstallPath::ResolveToInstallRoot("data/topo.json");
1196 1196 
1197 // 显式提示:文件已存在,将覆盖1197 // 显式提示:文件已存在,将覆盖
1198 if (fs::exists(destPath)) {1198 if (fs::exists(destPath)) {
Mtest/hccl_vm/src/cmd/cmd_cluster_model_utils.cc+6-7
@@ -14,11 +14,10 @@
14#include <cstring>14#include <cstring>
15#include <string>15#include <string>
16 16 
17+#include "sim_common_api.h"
17#include "sim_log.h"18#include "sim_log.h"
18#include "yaml-cpp/yaml.h"19#include "yaml-cpp/yaml.h"
19 20 
20-std::string GetBinLocation();
21- 
22namespace {21namespace {
23constexpr const char* HVM_MODEL_ENV_KEY = "HCCL_VM_MODEL";22constexpr const char* HVM_MODEL_ENV_KEY = "HCCL_VM_MODEL";
24constexpr const char* HVM_POD_NUM_ENV_KEY = "HCCL_VM_POD_NUM";23constexpr const char* HVM_POD_NUM_ENV_KEY = "HCCL_VM_POD_NUM";
@@ -31,18 +30,18 @@ bool SetEnvValue(const char* key, const std::string& value)
31 if (setenv(key, value.c_str(), 1) == 0) {30 if (setenv(key, value.c_str(), 1) == 0) {
32 return true;31 return true;
33 }32 }
34- HCCL_VM_ERROR("[HVM] setenv failed: {}={}, errno={}, err={}", key, value, errno, std::strerror(errno));33+ HCCL_VM_ERROR("setenv failed: {}={}, errno={}, err={}", key, value, errno, std::strerror(errno));
35 return false;34 return false;
36}35}
37 36 
38bool ParseYamlTopoImpl(const std::string& fileName, TopoMeta& topo)37bool ParseYamlTopoImpl(const std::string& fileName, TopoMeta& topo)
39{38{
40 try {39 try {
41- std::string filePath = GetBinLocation() + "/config/topo_meta/" + fileName + ".yaml";40+ std::string filePath = InstallPath::ResolveToInstallRoot("config/topo_meta/" + fileName + ".yaml");
42 YAML::Node root = YAML::LoadFile(filePath);41 YAML::Node root = YAML::LoadFile(filePath);
43 42 
44 if (!root["meta"]) {43 if (!root["meta"]) {
45- HCCL_VM_ERROR("[HVM] YAML : 'meta' node not found.");44+ HCCL_VM_ERROR("YAML : 'meta' node not found.");
46 return false;45 return false;
47 }46 }
48 uint32_t podNum = root["meta"]["podNum"].as<uint32_t>();47 uint32_t podNum = root["meta"]["podNum"].as<uint32_t>();
@@ -51,7 +50,7 @@ bool ParseYamlTopoImpl(const std::string& fileName, TopoMeta& topo)
51 HCCL_VM_DEBUG("PodNum: {}, SerNum: {}, RankNum: {}", podNum, serNum, rankNum);50 HCCL_VM_DEBUG("PodNum: {}, SerNum: {}, RankNum: {}", podNum, serNum, rankNum);
52 topo.reserve(podNum);51 topo.reserve(podNum);
53 if (podNum <= 0 || podNum >1024 || serNum <= 0 || serNum >1024 || rankNum <= 0 || rankNum >1024) {52 if (podNum <= 0 || podNum >1024 || serNum <= 0 || serNum >1024 || rankNum <= 0 || rankNum >1024) {
54- HCCL_VM_ERROR("[HVM] YAML : 'meta' number not surport, please check your config.yaml.");53+ HCCL_VM_ERROR("YAML : 'meta' number not surport, please check your config.yaml.");
55 return false;54 return false;
56 }55 }
57 if (root["topology"] && root["topology"].IsSequence()) {56 if (root["topology"] && root["topology"].IsSequence()) {
@@ -73,7 +72,7 @@ bool ParseYamlTopoImpl(const std::string& fileName, TopoMeta& topo)
73 }72 }
74 return true;73 return true;
75 } catch (const YAML::Exception& e) {74 } catch (const YAML::Exception& e) {
76- HCCL_VM_ERROR("[HVM] Exception when parsing YAML: {}", e.what());75+ HCCL_VM_ERROR("Exception when parsing YAML: {}", e.what());
77 return false;76 return false;
78 }77 }
79}78}
Mtest/hccl_vm/src/cmd/subcmd/subcmd_mock_comm.cc+4-4
@@ -28,19 +28,19 @@ void MockCommCommand::Setup(CLI::App& app) {
28}28}
29 29 
30void MockCommCommand::Execute() {30void MockCommCommand::Execute() {
31- HCCL_VM_INFO("[HVM] Initializing mock communication domain: Config={}, Level={}", configFileName, g_hcclVmLevel);31+ HCCL_VM_INFO("Initializing mock communication domain: Config={}, Level={}", configFileName, g_hcclVmLevel);
32 32 
33 // 首先清理动态数据库表项33 // 首先清理动态数据库表项
34 auto ret = ClearDbTables();34 auto ret = ClearDbTables();
35 if (ret != HcclVmResult::HCCL_SIM_HOST_SUCCESS_CMD) {35 if (ret != HcclVmResult::HCCL_SIM_HOST_SUCCESS_CMD) {
36- HCCL_VM_ERROR("[HVM] Failed to clear database tables.");36+ HCCL_VM_ERROR("Failed to clear database tables.");
37 return;37 return;
38 }38 }
39 39 
40 // 重置通信域40 // 重置通信域
41 ret = HcclVmResetCommDomain();41 ret = HcclVmResetCommDomain();
42 if (ret != HcclVmResult::HCCL_SIM_HOST_SUCCESS_CMD) {42 if (ret != HcclVmResult::HCCL_SIM_HOST_SUCCESS_CMD) {
43- HCCL_VM_ERROR("[HVM] Failed to reset mock communication domain.");43+ HCCL_VM_ERROR("Failed to reset mock communication domain.");
44 return;44 return;
45 }45 }
46 46
@@ -52,7 +52,7 @@ void MockCommCommand::Execute() {
52 52 
53 ret = InitHvmCommEnv(topoMeta, configFileName, g_hcclVmLevel);53 ret = InitHvmCommEnv(topoMeta, configFileName, g_hcclVmLevel);
54 if (ret != HcclVmResult::HCCL_SIM_HOST_SUCCESS_CMD) {54 if (ret != HcclVmResult::HCCL_SIM_HOST_SUCCESS_CMD) {
55- HCCL_VM_ERROR("[HVM] Failed to initialize mock communication environment. Cleaning up environment.");55+ HCCL_VM_ERROR("Failed to initialize mock communication environment. Cleaning up environment.");
56 return;56 return;
57 }57 }
58 return;58 return;
Mtest/hccl_vm/src/cmd/subcmd/subcmd_reset.cc+3-3
@@ -23,13 +23,13 @@ void ResetCommand::Setup(CLI::App& app) {
23}23}
24 24 
25void ResetCommand::Execute() {25void ResetCommand::Execute() {
26- HCCL_VM_INFO("[HVM] Resetting: Clearing database tables...");26+ HCCL_VM_INFO("Resetting: Clearing database tables...");
27 auto ret = ClearDbTables();27 auto ret = ClearDbTables();
28 if (ret != HcclVmResult::HCCL_SIM_HOST_SUCCESS_CMD) {28 if (ret != HcclVmResult::HCCL_SIM_HOST_SUCCESS_CMD) {
29- HCCL_VM_ERROR("[HVM] Failed to clear database tables.");29+ HCCL_VM_ERROR("Failed to clear database tables.");
30 return;30 return;
31 }31 }
32- HCCL_VM_INFO("[HVM] Database tables cleared successfully.");32+ HCCL_VM_INFO("Database tables cleared successfully.");
33}33}
34 34 
35static inline CommandAutoRegister<ResetCommand> g_reset_cmd_reg{};35static inline CommandAutoRegister<ResetCommand> g_reset_cmd_reg{};
Mtest/hccl_vm/src/cmd/subcmd/subcmd_run.cc+11-10
@@ -12,6 +12,7 @@
12 12 
13#include "cmd_base_utils.h"13#include "cmd_base_utils.h"
14#include "db_sim_runner_db.h"14#include "db_sim_runner_db.h"
15+#include "sim_common_api.h"
15#include "sim_common_defs.h"16#include "sim_common_defs.h"
16#include "sim_log.h"17#include "sim_log.h"
17#include "sim_models.h"18#include "sim_models.h"
@@ -32,40 +33,40 @@ void RunCommand::Setup(CLI::App& app) {
32 33 
33void RunCommand::Execute(CLI::App& app) {34void RunCommand::Execute(CLI::App& app) {
34 if (g_hcclVmBashFlag) {35 if (g_hcclVmBashFlag) {
35- HCCL_VM_WARN("[HVM] hccl-vm is already running. Please do not run examples in one_shot mode within a sub-bash. Exit the sub-bash and try again.");36+ HCCL_VM_WARN("hccl-vm is already running. Please do not run examples in one_shot mode within a sub-bash. Exit the sub-bash and try again.");
36 return;37 return;
37 }38 }
38 CLI::App* tmp_cmd = app.get_subcommand("run");39 CLI::App* tmp_cmd = app.get_subcommand("run");
39 std::vector<std::string> leftargvs = tmp_cmd->remaining();40 std::vector<std::string> leftargvs = tmp_cmd->remaining();
40 if (leftargvs.empty()) {41 if (leftargvs.empty()) {
41- HCCL_VM_ERROR("[HVM] In one_shot mode, an example execution command must be provided.");42+ HCCL_VM_ERROR("In one_shot mode, an example execution command must be provided.");
42 return;43 return;
43 }44 }
44- HCCL_VM_INFO("[HVM] Initializing: Model={}, Level={}", configClusterName, g_hcclVmLevel);45+ HCCL_VM_INFO("Initializing: Model={}, Level={}", configClusterName, g_hcclVmLevel);
45- auto clusterDir = GetBinLocation() + "/config/network/cluster/" + configClusterName;46+ auto clusterDir = InstallPath::ResolveToInstallRoot("config/network/cluster/" + configClusterName);
46 47 
47 sim::RunModeConfig runMode{};48 sim::RunModeConfig runMode{};
48 runMode.mode = checkOnlyMode ? 1 : 0;49 runMode.mode = checkOnlyMode ? 1 : 0;
49 RunnerDB::DeleteAll<sim::RunModeConfig>();50 RunnerDB::DeleteAll<sim::RunModeConfig>();
50 RunnerDB::Add<sim::RunModeConfig>(runMode);51 RunnerDB::Add<sim::RunModeConfig>(runMode);
51- HCCL_VM_INFO("[HVM] run mode: {}", checkOnlyMode ? "check-only" : "normal");52+ HCCL_VM_INFO("run mode: {}", checkOnlyMode ? "check-only" : "normal");
52 53 
53 auto ret = InitHvmEnv(clusterDir, g_hcclVmLevel, checkOnlyMode);54 auto ret = InitHvmEnv(clusterDir, g_hcclVmLevel, checkOnlyMode);
54 if (ret != HcclVmResult::HCCL_SIM_HOST_SUCCESS_CMD) {55 if (ret != HcclVmResult::HCCL_SIM_HOST_SUCCESS_CMD) {
55- HCCL_VM_ERROR("[HVM] Failed to initialize simulation environment. Cleaning up environment.");56+ HCCL_VM_ERROR("Failed to initialize simulation environment. Cleaning up environment.");
56 auto cleanRet = HcclVmExit();57 auto cleanRet = HcclVmExit();
57 if (cleanRet != HcclVmResult::HCCL_SIM_HOST_SUCCESS_CMD) {58 if (cleanRet != HcclVmResult::HCCL_SIM_HOST_SUCCESS_CMD) {
58- HCCL_VM_ERROR("[HVM] Failed to clean up environment. Please check for residual environment artifacts.");59+ HCCL_VM_ERROR("Failed to clean up environment. Please check for residual environment artifacts.");
59 }60 }
60 return;61 return;
61 }62 }
62 CstyleCmd syscmd(leftargvs);63 CstyleCmd syscmd(leftargvs);
63- HCCL_VM_INFO("[HVM] one_shot mode, executing: {}", syscmd.cmd());64+ HCCL_VM_INFO("one_shot mode, executing: {}", syscmd.cmd());
64- std::string proxyPath = GetBinLocation() + "/lib/" + GetArchStr() + "/libhccl_proxy_level2.so";65+ std::string proxyPath = InstallPath::ResolveToInstallRoot("lib/" + GetArchStr() + "/libhccl_proxy_level2.so");
65 setenv("LD_PRELOAD", proxyPath.c_str(), 1);66 setenv("LD_PRELOAD", proxyPath.c_str(), 1);
66 int sysRet = std::system(syscmd.cmd().c_str()); // system() 会阻塞当前进程直到子命令结束67 int sysRet = std::system(syscmd.cmd().c_str()); // system() 会阻塞当前进程直到子命令结束
67 if (sysRet != 0) {68 if (sysRet != 0) {
68- HCCL_VM_ERROR("[HVM] Example execution failed: {}", sysRet);69+ HCCL_VM_ERROR("Example execution failed: {}", sysRet);
69 }70 }
70 RemoveFromLDPreload(proxyPath);71 RemoveFromLDPreload(proxyPath);
71 auto cleanRet = HcclVmExit();72 auto cleanRet = HcclVmExit();
Mtest/hccl_vm/src/cmd/subcmd/subcmd_start.cc+9-8
@@ -13,6 +13,7 @@
13 13 
14#include "cmd_base_utils.h"14#include "cmd_base_utils.h"
15#include "db_sim_runner_db.h"15#include "db_sim_runner_db.h"
16+#include "sim_common_api.h"
16#include "sim_common_defs.h"17#include "sim_common_defs.h"
17#include "sim_log.h"18#include "sim_log.h"
18#include "sim_models.h"19#include "sim_models.h"
@@ -32,15 +33,15 @@ void StartCommand::Setup(CLI::App& app) {
32 33 
33void StartCommand::Execute() {34void StartCommand::Execute() {
34 if (g_hcclVmBashFlag) {35 if (g_hcclVmBashFlag) {
35- HCCL_VM_WARN("[HVM] hccl-vm has already started. Please do not start it again in a sub-bash.");36+ HCCL_VM_WARN("hccl-vm has already started. Please do not start it again in a sub-bash.");
36 return;37 return;
37 }38 }
38 if (configClusterName.find(".yaml") != std::string::npos) {39 if (configClusterName.find(".yaml") != std::string::npos) {
39 configClusterName.erase(configClusterName.find(".yaml"), 5);40 configClusterName.erase(configClusterName.find(".yaml"), 5);
40 }41 }
41 42 
42- HCCL_VM_INFO("[HVM] Initializing: Model={}, Level={}", configClusterName, g_hcclVmLevel);43+ HCCL_VM_INFO("Initializing: Model={}, Level={}", configClusterName, g_hcclVmLevel);
43- auto clusterDir = GetBinLocation() + "/config/network/cluster/" + configClusterName;44+ auto clusterDir = InstallPath::ResolveToInstallRoot("config/network/cluster/" + configClusterName);
44 45 
45 // 拷贝topo.json文件到执行目录46 // 拷贝topo.json文件到执行目录
46 auto ret = CopyFile(clusterDir);47 auto ret = CopyFile(clusterDir);
@@ -53,24 +54,24 @@ void StartCommand::Execute() {
53 runMode.mode = checkOnlyMode ? 1 : 0;54 runMode.mode = checkOnlyMode ? 1 : 0;
54 RunnerDB::DeleteAll<sim::RunModeConfig>();55 RunnerDB::DeleteAll<sim::RunModeConfig>();
55 RunnerDB::Add<sim::RunModeConfig>(runMode);56 RunnerDB::Add<sim::RunModeConfig>(runMode);
56- HCCL_VM_INFO("[HVM] run mode: {}", checkOnlyMode ? "check-only" : "normal");57+ HCCL_VM_INFO("run mode: {}", checkOnlyMode ? "check-only" : "normal");
57 58 
58 ret = InitHvmEnv(clusterDir, g_hcclVmLevel, checkOnlyMode);59 ret = InitHvmEnv(clusterDir, g_hcclVmLevel, checkOnlyMode);
59 if (ret != HcclVmResult::HCCL_SIM_HOST_SUCCESS_CMD) {60 if (ret != HcclVmResult::HCCL_SIM_HOST_SUCCESS_CMD) {
60- HCCL_VM_ERROR("[HVM] Failed to initialize simulation environment. Cleaning up environment.");61+ HCCL_VM_ERROR("Failed to initialize simulation environment. Cleaning up environment.");
61 auto cleanRet = HcclVmExit();62 auto cleanRet = HcclVmExit();
62 if (cleanRet != HcclVmResult::HCCL_SIM_HOST_SUCCESS_CMD) {63 if (cleanRet != HcclVmResult::HCCL_SIM_HOST_SUCCESS_CMD) {
63- HCCL_VM_ERROR("[HVM] Failed to clean up environment. Please check for residual environment artifacts.");64+ HCCL_VM_ERROR("Failed to clean up environment. Please check for residual environment artifacts.");
64 }65 }
65 return;66 return;
66 }67 }
67 68 
68 ret = StartHvmCmd();69 ret = StartHvmCmd();
69 if (ret != HcclVmResult::HCCL_SIM_HOST_SUCCESS_CMD) {70 if (ret != HcclVmResult::HCCL_SIM_HOST_SUCCESS_CMD) {
70- HCCL_VM_ERROR("[HVM] Failed to start hvm command.");71+ HCCL_VM_ERROR("Failed to start hvm command.");
71 auto cleanRet = HcclVmExit();72 auto cleanRet = HcclVmExit();
72 if (cleanRet != HcclVmResult::HCCL_SIM_HOST_SUCCESS_CMD) {73 if (cleanRet != HcclVmResult::HCCL_SIM_HOST_SUCCESS_CMD) {
73- HCCL_VM_ERROR("[HVM] Failed to clean up environment. Please check for residual environment artifacts.");74+ HCCL_VM_ERROR("Failed to clean up environment. Please check for residual environment artifacts.");
74 }75 }
75 return;76 return;
76 }77 }
Mtest/hccl_vm/src/device_arm/CMakeLists.txt+3-0
@@ -138,3 +138,6 @@ install(TARGETS device
138)138)
139 139 
140 140 
141+ 
142+# 日志染色: 模块 tag
143+target_compile_definitions(device PRIVATE HCCL_VM_MODULE="DEVICE")
Mtest/hccl_vm/src/device_arm/hccl_device_pub.cc+17-16
@@ -74,7 +74,7 @@ void UpdateSqTail(uint32_t sqId, uint32_t newTail)
74 sqTailMap[sqId] = newTail;74 sqTailMap[sqId] = newTail;
75}75}
76 76 
77-void *GetRealPtrByDevPtr(void *devPtr)77+void *GetRealPtrByDevPtrImpl(void *devPtr, const char *file, int line)
78{78{
79 uint64_t devAddr = reinterpret_cast<uint64_t>(devPtr);79 uint64_t devAddr = reinterpret_cast<uint64_t>(devPtr);
80 auto virMemRes = RunnerDB::GetOneByPred<sim::VirtualMemBlock>(80 auto virMemRes = RunnerDB::GetOneByPred<sim::VirtualMemBlock>(
@@ -83,20 +83,20 @@ void *GetRealPtrByDevPtr(void *devPtr)
83 (devAddr < (virMem.start_ptr + virMem.size)) &&83 (devAddr < (virMem.start_ptr + virMem.size)) &&
84 (virMem.src_type == (uint8_t)sim::VIR_MEM_TYPE_DEV)); });84 (virMem.src_type == (uint8_t)sim::VIR_MEM_TYPE_DEV)); });
85 if (!virMemRes.second) {85 if (!virMemRes.second) {
86- HCCL_VM_ERROR("[GetRealPtrByDevPtr] cannot find virMemRes by devAddr[{}]", devAddr);86+ HCCL_VM_ERROR("cannot find virMemRes by devAddr[{}], called from {}:{}]", devAddr, file, line);
87 return nullptr;87 return nullptr;
88 }88 }
89 89 
90 auto phyMemId = virMemRes.first.phy_mem_id;90 auto phyMemId = virMemRes.first.phy_mem_id;
91 auto phyMemRes = RunnerDB::GetById<sim::PhyMemBlock>(phyMemId);91 auto phyMemRes = RunnerDB::GetById<sim::PhyMemBlock>(phyMemId);
92 if (!phyMemRes.has_value()) {92 if (!phyMemRes.has_value()) {
93- HCCL_VM_ERROR("[GetRealPtrByDevPtr] cannot find phyMemRes by phyMemId[{}]", phyMemId);93+ HCCL_VM_ERROR("cannot find phyMemRes by phyMemId[{}], called from {}:{}]", phyMemId, file, line);
94 return nullptr;94 return nullptr;
95 };95 };
96 96 
97 std::string memName(phyMemRes->name);97 std::string memName(phyMemRes->name);
98 void *realPtr = sim::MemoryManager::GetInstance().AcquireMemByName(memName.c_str());98 void *realPtr = sim::MemoryManager::GetInstance().AcquireMemByName(memName.c_str());
99- HCCL_VM_INFO("[GetRealPtrByDevPtr] devAddr[{}] realPtr[{}] memName[{}]", devAddr, reinterpret_cast<uint64_t>(realPtr), memName);99+ HCCL_VM_INFO("devAddr[{}] realPtr[{}] memName[{}]", devAddr, reinterpret_cast<uint64_t>(realPtr), memName);
100 100 
101 return realPtr;101 return realPtr;
102}102}
@@ -109,21 +109,21 @@ uint32_t GetRankIdByDevAddr(uint64_t devAddr)
109 (devAddr < (virMem.start_ptr + virMem.size)) &&109 (devAddr < (virMem.start_ptr + virMem.size)) &&
110 (virMem.src_type == (uint8_t)sim::VIR_MEM_TYPE_DEV)); });110 (virMem.src_type == (uint8_t)sim::VIR_MEM_TYPE_DEV)); });
111 if (!virMemRes.second) {111 if (!virMemRes.second) {
112- HCCL_VM_ERROR("[GetRankIdByDevAddr] cannot find virMemRes by devAddr[{}]", devAddr);112+ HCCL_VM_ERROR("cannot find virMemRes by devAddr[{}]", devAddr);
113 return 0;113 return 0;
114 }114 }
115 115 
116 auto phyMemId = virMemRes.first.phy_mem_id;116 auto phyMemId = virMemRes.first.phy_mem_id;
117 auto phyMemRes = RunnerDB::GetById<sim::PhyMemBlock>(phyMemId);117 auto phyMemRes = RunnerDB::GetById<sim::PhyMemBlock>(phyMemId);
118 if (!phyMemRes.has_value()) {118 if (!phyMemRes.has_value()) {
119- HCCL_VM_ERROR("[GetRankIdByDevAddr] cannot find phyMemRes by phyMemId[{}]", phyMemId);119+ HCCL_VM_ERROR("cannot find phyMemRes by phyMemId[{}]", phyMemId);
120 return 0;120 return 0;
121 };121 };
122 122 
123 uint32_t device_id = phyMemRes->device_id;123 uint32_t device_id = phyMemRes->device_id;
124 auto rank = RunnerDB::GetOneByPred<sim::Rank>([device_id](const sim::Rank &rank) { return rank.device_id == device_id; });124 auto rank = RunnerDB::GetOneByPred<sim::Rank>([device_id](const sim::Rank &rank) { return rank.device_id == device_id; });
125 if (!rank.second) {125 if (!rank.second) {
126- HCCL_VM_ERROR("[GetRankIdByDevAddr] cannot find rank by device_id[{}]", device_id);126+ HCCL_VM_ERROR("cannot find rank by device_id[{}]", device_id);
127 return 0;127 return 0;
128 }128 }
129 129 
@@ -134,7 +134,7 @@ HcclAicpuData *GetHcclAicpuDataShmPtr()
134{134{
135 void *shmptr = sim::MemoryManager::GetInstance().AcquireMemByName("HcclAicpuData");135 void *shmptr = sim::MemoryManager::GetInstance().AcquireMemByName("HcclAicpuData");
136 if (shmptr == nullptr) {136 if (shmptr == nullptr) {
137- HCCL_VM_ERROR("[GetHcclAicpuDataShmPtr] acquire HcclAicpuData shm failed.");137+ HCCL_VM_ERROR("acquire HcclAicpuData shm failed.");
138 return nullptr;138 return nullptr;
139 }139 }
140 140 
@@ -145,14 +145,14 @@ uint32_t GetRankIdByIpAddr(std::string ipAddr)
145{145{
146 auto ret = RunnerDB::GetOneByPred<sim::EndPoint>([ipAddr](const sim::EndPoint &p) { return strcmp(p.ip_addr, ipAddr.c_str()) == 0; });146 auto ret = RunnerDB::GetOneByPred<sim::EndPoint>([ipAddr](const sim::EndPoint &p) { return strcmp(p.ip_addr, ipAddr.c_str()) == 0; });
147 if (!ret.second) {147 if (!ret.second) {
148- HCCL_VM_ERROR("[GetRankIdByDevAddr] cannot find Port by ipAddr[{}]", ipAddr);148+ HCCL_VM_ERROR("cannot find Port by ipAddr[{}]", ipAddr);
149 return 0;149 return 0;
150 }150 }
151 151 
152 uint32_t device_id = ret.first.device_id;152 uint32_t device_id = ret.first.device_id;
153 auto rank = RunnerDB::GetOneByPred<sim::Rank>([device_id](const sim::Rank &rank) { return rank.device_id == device_id; });153 auto rank = RunnerDB::GetOneByPred<sim::Rank>([device_id](const sim::Rank &rank) { return rank.device_id == device_id; });
154 if (!rank.second) {154 if (!rank.second) {
155- HCCL_VM_ERROR("[GetRankIdByDevAddr] cannot find Rank by device_id[{}]", device_id);155+ HCCL_VM_ERROR("cannot find Rank by device_id[{}]", device_id);
156 return 0;156 return 0;
157 }157 }
158 158 
@@ -162,7 +162,7 @@ uint32_t GetRankIdByIpAddr(std::string ipAddr)
162void UpdataKfcStatus(uint64_t d2hAddr)162void UpdataKfcStatus(uint64_t d2hAddr)
163{163{
164 if (d2hAddr == 0) {164 if (d2hAddr == 0) {
165- HCCL_VM_ERROR("[UpdataKfcStatus] d2hAddr is 0, cannot update kfc status.");165+ HCCL_VM_ERROR("d2hAddr is 0, cannot update kfc status.");
166 return;166 return;
167 }167 }
168 168 
@@ -173,7 +173,7 @@ void UpdataKfcStatus(uint64_t d2hAddr)
173 *reinterpret_cast<uint8_t *>(d2hAddr) = devDoneStatus; // 更新设备状态173 *reinterpret_cast<uint8_t *>(d2hAddr) = devDoneStatus; // 更新设备状态
174 *reinterpret_cast<uint32_t *>(d2hAddr + headCntOffset) = headTailCnt; // 更新headCnt174 *reinterpret_cast<uint32_t *>(d2hAddr + headCntOffset) = headTailCnt; // 更新headCnt
175 *reinterpret_cast<uint32_t *>(d2hAddr + tailCntOffset) = headTailCnt; // 更新tailCnt175 *reinterpret_cast<uint32_t *>(d2hAddr + tailCntOffset) = headTailCnt; // 更新tailCnt
176- HCCL_VM_INFO("[UpdataKfcStatus] update kfc status success.");176+ HCCL_VM_INFO("update kfc status success.");
177}177}
178 178 
179// 信号处理函数179// 信号处理函数
@@ -186,7 +186,8 @@ void SignalHandler(int signum)
186 int count = backtrace(stack_pointers, MAX_STACK_FRAMES);186 int count = backtrace(stack_pointers, MAX_STACK_FRAMES);
187 187
188 // 2. 打印基本错误信息188 // 2. 打印基本错误信息
189- HCCL_VM_INFO("\n========================================");189+ HCCL_VM_ERROR("device process crash captured.");
190+ HCCL_VM_INFO("========================================");
190 HCCL_VM_INFO(" ERROR CRASH DETECTED! Signal: {}", signum);191 HCCL_VM_INFO(" ERROR CRASH DETECTED! Signal: {}", signum);
191 HCCL_VM_INFO("========================================");192 HCCL_VM_INFO("========================================");
192 193
@@ -199,13 +200,13 @@ void SignalHandler(int signum)
199 if (info.dli_sname != nullptr) {200 if (info.dli_sname != nullptr) {
200 ptrdiff_t symOff = reinterpret_cast<char*>(stack_pointers[i])201 ptrdiff_t symOff = reinterpret_cast<char*>(stack_pointers[i])
201 - reinterpret_cast<char*>(info.dli_saddr);202 - reinterpret_cast<char*>(info.dli_saddr);
202- printf(" [%02d] %s(%s+%#tx) [addr: %#tx]\n", i,203+ HCCL_VM_INFO(" [{:02d}] {}({}+{:#x}) [addr: {:#x}]", i,
203 info.dli_fname, info.dli_sname, symOff, elfOffset);204 info.dli_fname, info.dli_sname, symOff, elfOffset);
204 } else {205 } else {
205- printf(" [%02d] %s(?""?) [addr: %#tx]\n", i, info.dli_fname, elfOffset);206+ HCCL_VM_INFO(" [{:02d}] {}(?""?) [addr: {:#x}]", i, info.dli_fname, elfOffset);
206 }207 }
207 } else {208 } else {
208- printf(" [%02d] %p (dladdr failed)\n", i, stack_pointers[i]);209+ HCCL_VM_INFO(" [{:02d}] {} (dladdr failed)", i, stack_pointers[i]);
209 }210 }
210 }211 }
211 212
Mtest/hccl_vm/src/device_arm/hccl_device_pub.h+3-1
@@ -31,7 +31,9 @@ HcclSim::HcclVmResult GetPiValByJettyId(uint32_t jettyId, uint32_t *piValue);
31 31 
32HcclSim::HcclVmResult UpdatePiValByJettyId(uint32_t jettyId, uint32_t piValue);32HcclSim::HcclVmResult UpdatePiValByJettyId(uint32_t jettyId, uint32_t piValue);
33 33 
34-void *GetRealPtrByDevPtr(void *devPtr);34+void *GetRealPtrByDevPtrImpl(void *devPtr, const char *file, int line);
35+ 
36+#define GetRealPtrByDevPtr(devPtr) GetRealPtrByDevPtrImpl(devPtr, __FILE__, __LINE__)
35 37 
36uint32_t GetRankIdByDevAddr(uint64_t devAddr);38uint32_t GetRankIdByDevAddr(uint64_t devAddr);
37 39 
Mtest/hccl_vm/src/device_arm/hccl_kernel_executor.cc+13-13
@@ -17,6 +17,7 @@
17#include "aicpu_args_stub.h"17#include "aicpu_args_stub.h"
18#include "alg_param.h"18#include "alg_param.h"
19#include "hccl_device_pub.h"19#include "hccl_device_pub.h"
20+#include "sim_common_api.h"
20#include "sim_log.h"21#include "sim_log.h"
21#include "store_sim_memory_manager.h"22#include "store_sim_memory_manager.h"
22 23 
@@ -38,9 +39,9 @@ uint64_t d2hAddr = 0;
38 39 
39void ExecuteAicpuKernel(uint32_t rankId, const std::string &kernelName, uint64_t args)40void ExecuteAicpuKernel(uint32_t rankId, const std::string &kernelName, uint64_t args)
40{41{
41- HCCL_VM_INFO("[device] rankId[{}] kernel[{}] start run...", rankId, kernelName);42+ HCCL_VM_INFO("rankId[{}] kernel[{}] start run...", rankId, kernelName);
42 if (!InitKernelFuncHandle()) {43 if (!InitKernelFuncHandle()) {
43- HCCL_VM_ERROR("[device] rankId[{}] init func handle failed.", rankId);44+ HCCL_VM_ERROR("rankId[{}] init func handle failed.", rankId);
44 return;45 return;
45 }46 }
46 47 
@@ -72,7 +73,7 @@ void ExecuteAicpuKernel(uint32_t rankId, const std::string &kernelName, uint64_t
72 uint32_t listNum = res->listNum;73 uint32_t listNum = res->listNum;
73 uint32_t totalListSize = res->uniqueIdSize;74 uint32_t totalListSize = res->uniqueIdSize;
74 uint32_t listSize = totalListSize / listNum;75 uint32_t listSize = totalListSize / listNum;
75- HCCL_VM_INFO("[device] start packing, listNum:{}, totalListSize:{}, listSize:{}", listNum, totalListSize, listSize);76+ HCCL_VM_INFO("start packing, listNum:{}, totalListSize:{}, listSize:{}", listNum, totalListSize, listSize);
76 res->channelList = GetRealPtrByDevPtr(res->channelList);77 res->channelList = GetRealPtrByDevPtr(res->channelList);
77 res->channelSizeAddr = GetRealPtrByDevPtr(res->channelSizeAddr);78 res->channelSizeAddr = GetRealPtrByDevPtr(res->channelSizeAddr);
78 res->uniqueIdAddr = GetRealPtrByDevPtr(res->uniqueIdAddr);79 res->uniqueIdAddr = GetRealPtrByDevPtr(res->uniqueIdAddr);
@@ -82,7 +83,7 @@ void ExecuteAicpuKernel(uint32_t rankId, const std::string &kernelName, uint64_t
82 for (auto i = 0; i < listNum; i++) {83 for (auto i = 0; i < listNum; i++) {
83 uint32_t startOffset = i * listSize;84 uint32_t startOffset = i * listSize;
84 UniqueIdV2Header *header = reinterpret_cast<UniqueIdV2Header *>(startAddr + startOffset + UNIQUEID_HEADER_OFFSET);85 UniqueIdV2Header *header = reinterpret_cast<UniqueIdV2Header *>(startAddr + startOffset + UNIQUEID_HEADER_OFFSET);
85- HCCL_VM_INFO("[device] uniqueId header type:{} notifyNum:{} bufferNum:{} rmtBufferNum:{} connNum:{}.", header->type, header->notifyNum, header->bufferNum, header->rmtBufferNum, header->connNum);86+ HCCL_VM_INFO("uniqueId header type:{} notifyNum:{} bufferNum:{} rmtBufferNum:{} connNum:{}.", header->type, header->notifyNum, header->bufferNum, header->rmtBufferNum, header->connNum);
86 // 根据UniqueIdV2Header计算偏移87 // 根据UniqueIdV2Header计算偏移
87 uint32_t offset = COMMON_DATA_SIZE + header->notifyNum * NOTIFY_ID_SIZE +88 uint32_t offset = COMMON_DATA_SIZE + header->notifyNum * NOTIFY_ID_SIZE +
88 COMMON_DATA_SIZE + header->notifyNum * NOTIFY_BUFFER_SIZE +89 COMMON_DATA_SIZE + header->notifyNum * NOTIFY_BUFFER_SIZE +
@@ -101,10 +102,10 @@ void ExecuteAicpuKernel(uint32_t rankId, const std::string &kernelName, uint64_t
101 hcclLaunchAicpuKernelPtr(opParam);102 hcclLaunchAicpuKernelPtr(opParam);
102 UpdataKfcStatus(d2hAddr);103 UpdataKfcStatus(d2hAddr);
103 } else {104 } else {
104- HCCL_VM_ERROR("[device] rankId[{}] kernel[{}] not support.", rankId, kernelName);105+ HCCL_VM_ERROR("rankId[{}] kernel[{}] not support.", rankId, kernelName);
105 }106 }
106 107 
107- HCCL_VM_INFO("[device] rankId[{}] kernel[{}] finish run...", rankId, kernelName);108+ HCCL_VM_INFO("rankId[{}] kernel[{}] finish run...", rankId, kernelName);
108}109}
109 110 
110void* LoadLibrary(const std::string &libDir, const std::string &libName)111void* LoadLibrary(const std::string &libDir, const std::string &libName)
@@ -113,11 +114,11 @@ void* LoadLibrary(const std::string &libDir, const std::string &libName)
113 std::string libPath = libDir + libName;114 std::string libPath = libDir + libName;
114 handle = dlopen(libPath.c_str(), RTLD_NOW | RTLD_GLOBAL);115 handle = dlopen(libPath.c_str(), RTLD_NOW | RTLD_GLOBAL);
115 if (handle != nullptr) {116 if (handle != nullptr) {
116- HCCL_VM_INFO("[LoadLibrary] Load so {} from {}", libName, libPath);117+ HCCL_VM_INFO("Load so {} from {}", libName, libPath);
117 return handle;118 return handle;
118 }119 }
119 120
120- HCCL_VM_ERROR("[LoadLibrary] Failed to load so {} from {}", libName, dlerror());121+ HCCL_VM_ERROR("Failed to load so {} from {}", libName, dlerror());
121 return nullptr;122 return nullptr;
122}123}
123 124 
@@ -126,14 +127,13 @@ bool InitKernelFuncHandle()
126 if (gLibsLoaded) {127 if (gLibsLoaded) {
127 return true;128 return true;
128 }129 }
129- const char* installDir = getenv("HCCL_VM_INSTALL_DIR");130+ std::string libDir = InstallPath::ResolveToInstallRoot("lib/" + GetArchStr()) + "/";
130- std::string libDir = installDir ? std::string(installDir) + "/lib/" + GetArchStr() + "/" : "./lib/" + GetArchStr() + "/";
131 gSlogHandle = LoadLibrary(libDir, "libslog.so");131 gSlogHandle = LoadLibrary(libDir, "libslog.so");
132 gCsecHandle = LoadLibrary(libDir, "libc_sec.so");132 gCsecHandle = LoadLibrary(libDir, "libc_sec.so");
133 gHcclKerHandle = LoadLibrary(libDir, "libscatter_aicpu_kernel.so");133 gHcclKerHandle = LoadLibrary(libDir, "libscatter_aicpu_kernel.so");
134 gHcommHandle = LoadLibrary(libDir, "libccl_kernel.so");134 gHcommHandle = LoadLibrary(libDir, "libccl_kernel.so");
135 if (gHcclKerHandle == nullptr || gHcommHandle == nullptr || gSlogHandle == nullptr || gCsecHandle == nullptr) {135 if (gHcclKerHandle == nullptr || gHcommHandle == nullptr || gSlogHandle == nullptr || gCsecHandle == nullptr) {
136- HCCL_VM_ERROR("[InitKernelFuncHandle] Failed to load kernel libs.");136+ HCCL_VM_ERROR("Failed to load kernel libs.");
137 return false;137 return false;
138 }138 }
139 runAicpuIndOpCommInitPtr = reinterpret_cast<uint32_t (*)(void *)>(dlsym(gHcommHandle, "RunAicpuIndOpCommInit"));139 runAicpuIndOpCommInitPtr = reinterpret_cast<uint32_t (*)(void *)>(dlsym(gHcommHandle, "RunAicpuIndOpCommInit"));
@@ -143,10 +143,10 @@ bool InitKernelFuncHandle()
143 hcclLaunchAicpuKernelPtr = reinterpret_cast<unsigned int (*)(OpParam *)>(dlsym(gHcclKerHandle, "HcclLaunchAicpuKernel"));143 hcclLaunchAicpuKernelPtr = reinterpret_cast<unsigned int (*)(OpParam *)>(dlsym(gHcclKerHandle, "HcclLaunchAicpuKernel"));
144 if (runAicpuIndOpCommInitPtr == nullptr || runAicpuIndOpThreadInitPtr == nullptr || runAicpuIndOpChannelInitV2Ptr == nullptr ||144 if (runAicpuIndOpCommInitPtr == nullptr || runAicpuIndOpThreadInitPtr == nullptr || runAicpuIndOpChannelInitV2Ptr == nullptr ||
145 runAicpuDfxOpInfoInitV2Ptr == nullptr || hcclLaunchAicpuKernelPtr == nullptr) {145 runAicpuDfxOpInfoInitV2Ptr == nullptr || hcclLaunchAicpuKernelPtr == nullptr) {
146- HCCL_VM_ERROR("[InitKernelFuncHandle] Failed to get kernel func handle.");146+ HCCL_VM_ERROR("Failed to get kernel func handle.");
147 return false;147 return false;
148 }148 }
149- HCCL_VM_INFO("[InitKernelFuncHandle] Load kernel libs and get func handles successfully.");149+ HCCL_VM_INFO("Load kernel libs and get func handles successfully.");
150 gLibsLoaded = true;150 gLibsLoaded = true;
151 return true;151 return true;
152}152}
Mtest/hccl_vm/src/device_arm/hccl_task_collection.cc+2-2
@@ -21,7 +21,7 @@ void InsertTaskToCollectionDev(HcclTaskMetaData *task)
21 21 
22 // 1. 入参合法性检查(避免空指针访问)22 // 1. 入参合法性检查(避免空指针访问)
23 if (task == nullptr) {23 if (task == nullptr) {
24- HCCL_VM_ERROR("[InsertTaskToCollection] 错误:输入任务指针 task 不能为空!");24+ HCCL_VM_ERROR("错误:输入任务指针 task 不能为空!");
25 return;25 return;
26 }26 }
27 27 
@@ -38,7 +38,7 @@ void InsertTaskToCollectionDev(HcclTaskMetaData *task)
38 38
39 auto ret = sim::InsertOpTask(opTaskInfo, true);39 auto ret = sim::InsertOpTask(opTaskInfo, true);
40 if (ret != 0) {40 if (ret != 0) {
41- HCCL_VM_ERROR("[InsertTaskToCollection] 错误:插入任务到数据库失败 - {}", ret);41+ HCCL_VM_ERROR("错误:插入任务到数据库失败 - {}", ret);
42 return;42 return;
43 }43 }
44 return;44 return;
Mtest/hccl_vm/src/device_arm/main.cc+3-3
@@ -31,11 +31,11 @@ int main(int argc, char *argv[])
31 31 
32 RegisterSignalHandler();32 RegisterSignalHandler();
33 setvbuf(stdout, nullptr, _IOLBF, 0);33 setvbuf(stdout, nullptr, _IOLBF, 0);
34- HCCL_VM_INFO("[device] main process start.");34+ HCCL_VM_INFO("main process start.");
35 35
36 uint32_t rankId = static_cast<uint32_t>(std::stoi(argv[1]));36 uint32_t rankId = static_cast<uint32_t>(std::stoi(argv[1]));
37 SetCurRankId(rankId);37 SetCurRankId(rankId);
38- HCCL_VM_INFO("[device] rankId[{}] process start...", rankId);38+ HCCL_VM_INFO("rankId[{}] process start...", rankId);
39 39 
40 HcclAicpuData *aicpuData = GetHcclAicpuDataShmPtr();40 HcclAicpuData *aicpuData = GetHcclAicpuDataShmPtr();
41 if (aicpuData == nullptr) {41 if (aicpuData == nullptr) {
@@ -56,7 +56,7 @@ int main(int argc, char *argv[])
56 uint32_t opDetailId = 0;56 uint32_t opDetailId = 0;
57 int ret = sim::QueryNewestOpDeatailIdByPid(getppid(), opDetailId);57 int ret = sim::QueryNewestOpDeatailIdByPid(getppid(), opDetailId);
58 if (ret != 0) {58 if (ret != 0) {
59- HCCL_VM_ERROR("[device] rankId[{}] QueryNewestOpDeatailIdByPid failed.", rankId);59+ HCCL_VM_ERROR("rankId[{}] QueryNewestOpDeatailIdByPid failed.", rankId);
60 }60 }
61 sim::g_currOpDetailId = opDetailId;61 sim::g_currOpDetailId = opDetailId;
62 }62 }
Mtest/hccl_vm/src/device_arm/proxy/CMakeLists.txt+1-0
@@ -12,6 +12,7 @@ target_sources(hccl_device_proxy PRIVATE
12 aicpu_stub.cc12 aicpu_stub.cc
13 ascend_hal_stub.cc13 ascend_hal_stub.cc
14 device_sqe_parse_stub.cc14 device_sqe_parse_stub.cc
15+ aprofiling_stub.cc
15 ${HCCL_VM_SRC}/utils/sim_log.cc16 ${HCCL_VM_SRC}/utils/sim_log.cc
16 ${HCCL_VM_SRC}/utils/sim_yaml_config.cc17 ${HCCL_VM_SRC}/utils/sim_yaml_config.cc
17 ${HCCL_VM_SRC}/utils/sim_common_api.cc18 ${HCCL_VM_SRC}/utils/sim_common_api.cc
Atest/hccl_vm/src/device_arm/proxy/aprofiling_stub.cc+121-0
@@ -0,0 +1,121 @@
1+/**
2+ * Copyright (c) 2026 Huawei Technologies Co., Ltd.
3+ * This program is free software, you can redistribute it and/or modify it under the terms and conditions of
4+ * CANN Open Software License Agreement Version 2.0 (the "License").
5+ * Please refer to the License for details. You may not use this file except in compliance with the License.
6+ * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED,
7+ * INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE.
8+ * See LICENSE in the root of the software repository for the full text of the License.
9+ */
10+ 
11+#include <cstdint>
12+ 
13+#include "aprof_pub.h"
14+#include "sim_log.h"
15+ 
16+#ifdef __cplusplus
17+extern "C" {
18+#endif // __cplusplus
19+ 
20+uint64_t MsprofStr2Id(const char *hashInfo, size_t length)
21+{
22+ (void) hashInfo;
23+ (void) length;
24+ HCCL_VM_INFO("[APROF] stub");
25+ return 1;
26+}
27+ 
28+int32_t MsprofRegTypeInfo(uint16_t level, uint32_t typeId, const char *typeName)
29+{
30+ (void) level;
31+ (void) typeId;
32+ (void) typeName;
33+ HCCL_VM_INFO("[APROF] stub");
34+ return 0;
35+}
36+ 
37+uint64_t MsprofSysCycleTime()
38+{
39+ HCCL_VM_INFO("[APROF] stub");
40+ return 1;
41+}
42+ 
43+int32_t MsprofRegisterCallback(uint32_t moduleId, ProfCommandHandle handle)
44+{
45+ (void) moduleId;
46+ (void) handle;
47+ HCCL_VM_INFO("[APROF] stub");
48+ return 0;
49+}
50+ 
51+int32_t MsprofReportApi(uint32_t agingFlag, const MsprofApi *api)
52+{
53+ (void) agingFlag;
54+ (void) api;
55+ HCCL_VM_INFO("[APROF] stub");
56+ return 0;
57+}
58+ 
59+int32_t MsprofReportAdditionalInfo(uint32_t agingFlag, const VOID_PTR data, uint32_t length)
60+{
61+ (void) agingFlag;
62+ (void) data;
63+ (void) length;
64+ HCCL_VM_INFO("[APROF] stub");
65+ return 0;
66+}
67+ 
68+int32_t MsprofReportCompactInfo(uint32_t agingFlag, const VOID_PTR data, uint32_t length)
69+{
70+ (void) agingFlag;
71+ (void) data;
72+ (void) length;
73+ HCCL_VM_INFO("[APROF] stub");
74+ return 0;
75+}
76+ 
77+int32_t MsprofReportBatchAdditionalInfo(uint32_t nonPersistantFlag, const VOID_PTR data, uint32_t length)
78+{
79+ (void) nonPersistantFlag;
80+ (void) data;
81+ (void) length;
82+ HCCL_VM_INFO("[APROF] stub");
83+ return 0;
84+}
85+ 
86+int32_t AdprofReportAdditionalInfo(uint32_t agingFlag, const void *data, uint32_t length)
87+{
88+ (void) agingFlag;
89+ (void) data;
90+ (void) length;
91+ HCCL_VM_INFO("[APROF] stub");
92+ return 0;
93+}
94+ 
95+int32_t AdprofReportBatchAdditionalInfo(uint32_t nonPersistantFlag, const void *data, uint32_t length)
96+{
97+ (void) nonPersistantFlag;
98+ (void) data;
99+ (void) length;
100+ HCCL_VM_INFO("[APROF] stub");
101+ return 0;
102+}
103+ 
104+uint64_t AdprofGetHashId(const char *hashInfo, size_t length)
105+{
106+ (void) hashInfo;
107+ (void) length;
108+ HCCL_VM_INFO("[APROF] stub");
109+ return 1;
110+}
111+ 
112+int32_t AdprofCheckFeatureIsOn(uint64_t feature)
113+{
114+ (void) feature;
115+ HCCL_VM_INFO("[APROF] stub");
116+ return 0;
117+}
118+ 
119+#ifdef __cplusplus
120+}
121+#endif // __cplusplus
Mtest/hccl_vm/src/device_arm/proxy/ascend_hal_stub.cc+34-30
@@ -8,6 +8,9 @@
8 * See LICENSE in the root of the software repository for the full text of the License.8 * See LICENSE in the root of the software repository for the full text of the License.
9 */9 */
10 10 
11+// 日志染色: 模块 tag (须在 include sim_log.h 之前)
12+#define HCCL_VM_MODULE "STUB"
13+ 
11#include <cstdint>14#include <cstdint>
12#include <cstdio>15#include <cstdio>
13#include <cstring>16#include <cstring>
@@ -21,6 +24,7 @@
21#include "db_sim_runner_common.h"24#include "db_sim_runner_common.h"
22#include "sim_log.h"25#include "sim_log.h"
23 26 
27+ 
24using namespace HcclSim;28using namespace HcclSim;
25 29 
26#ifdef __cplusplus30#ifdef __cplusplus
@@ -30,14 +34,14 @@ extern "C" {
30drvError_t halEschedAttachDevice(unsigned int devId)34drvError_t halEschedAttachDevice(unsigned int devId)
31{35{
32 (void) devId;36 (void) devId;
33- HCCL_VM_INFO("[STUB]AttachDevice is empty.");37+ HCCL_VM_INFO("AttachDevice is empty.");
34 return DRV_ERROR_NONE;38 return DRV_ERROR_NONE;
35}39}
36 40 
37drvError_t halEschedDettachDevice(unsigned int devId) 41drvError_t halEschedDettachDevice(unsigned int devId)
38{42{
39 (void) devId;43 (void) devId;
40- HCCL_VM_INFO("[STUB]DettachDevice is empty.");44+ HCCL_VM_INFO("DettachDevice is empty.");
41 return DRV_ERROR_NONE;45 return DRV_ERROR_NONE;
42}46}
43 47 
@@ -46,7 +50,7 @@ drvError_t halEschedCreateGrpEx(uint32_t devId, struct esched_grp_para *grpPara,
46 (void) devId;50 (void) devId;
47 (void) grpPara;51 (void) grpPara;
48 (void) grpId;52 (void) grpId;
49- HCCL_VM_INFO("[STUB]CreateGrpEx is empty.");53+ HCCL_VM_INFO("CreateGrpEx is empty.");
50 return DRV_ERROR_NONE;54 return DRV_ERROR_NONE;
51}55}
52 56 
@@ -57,7 +61,7 @@ drvError_t halEschedSubscribeEvent(unsigned int devId, unsigned int grpId,
57 (void) grpId;61 (void) grpId;
58 (void) threadId;62 (void) threadId;
59 (void) eventBitmap;63 (void) eventBitmap;
60- HCCL_VM_INFO("[STUB]SubscribeEvent is empty.");64+ HCCL_VM_INFO("SubscribeEvent is empty.");
61 return DRV_ERROR_NONE;65 return DRV_ERROR_NONE;
62}66}
63 67 
@@ -68,7 +72,7 @@ drvError_t halEschedQueryInfo(unsigned int devId, ESCHED_QUERY_TYPE type, struct
68 (void) type;72 (void) type;
69 (void) inPut;73 (void) inPut;
70 (void) outPut;74 (void) outPut;
71- HCCL_VM_INFO("[STUB]QueryInfo is empty.");75+ HCCL_VM_INFO("QueryInfo is empty.");
72 return DRV_ERROR_NONE;76 return DRV_ERROR_NONE;
73}77}
74 78 
@@ -80,7 +84,7 @@ drvError_t halEschedWaitEvent(unsigned int devId, unsigned int grpId,
80 (void) threadId;84 (void) threadId;
81 (void) timeout;85 (void) timeout;
82 (void) event;86 (void) event;
83- HCCL_VM_INFO("[STUB]WaitEvent is empty.");87+ HCCL_VM_INFO("WaitEvent is empty.");
84 return DRV_ERROR_NONE;88 return DRV_ERROR_NONE;
85}89}
86 90 
@@ -91,14 +95,14 @@ drvError_t halEschedRegisterAckFunc(unsigned int grpId, EVENT_ID eventId,
91 (void) grpId;95 (void) grpId;
92 (void) eventId;96 (void) eventId;
93 (void) ackFunc;97 (void) ackFunc;
94- HCCL_VM_INFO("[STUB]RegisterAckFunc is empty.");98+ HCCL_VM_INFO("RegisterAckFunc is empty.");
95 return DRV_ERROR_NONE;99 return DRV_ERROR_NONE;
96}100}
97 101 
98drvError_t drvGetDevNum(uint32_t *num_dev)102drvError_t drvGetDevNum(uint32_t *num_dev)
99{103{
100 *num_dev = 1;104 *num_dev = 1;
101- HCCL_VM_INFO("[STUB]drvGetDevNum:{}", *num_dev);105+ HCCL_VM_INFO("drvGetDevNum:{}", *num_dev);
102 return DRV_ERROR_NONE;106 return DRV_ERROR_NONE;
103}107}
104 108 
@@ -106,11 +110,11 @@ drvError_t halGetChipInfo(unsigned int devId, halChipInfo *chipInfo)
106{110{
107 sim::Device locDevice{};111 sim::Device locDevice{};
108 if (sim::GetDeviceByPhysicalId(devId, locDevice) != ACL_SUCCESS) {112 if (sim::GetDeviceByPhysicalId(devId, locDevice) != ACL_SUCCESS) {
109- HCCL_VM_ERROR("[STUB]halGetChipInfo get device by physic id {} failed.", devId);113+ HCCL_VM_ERROR("halGetChipInfo get device by physic id {} failed.", devId);
110 return DRV_ERROR_INNER_ERR;114 return DRV_ERROR_INNER_ERR;
111 }115 }
112 strncpy((char *)chipInfo->name, locDevice.soc_version, MAX_CHIP_NAME - 1);116 strncpy((char *)chipInfo->name, locDevice.soc_version, MAX_CHIP_NAME - 1);
113- HCCL_VM_INFO("[STUB]GetChipInfo is empty.");117+ HCCL_VM_INFO("GetChipInfo is empty.");
114 return DRV_ERROR_NONE;118 return DRV_ERROR_NONE;
115}119}
116 120 
@@ -121,7 +125,7 @@ drvError_t halHostRegister(void *src_ptr, UINT64 size, UINT32 flag, UINT32 devid
121 (void) flag;125 (void) flag;
122 (void) devid;126 (void) devid;
123 (void) dst_ptr;127 (void) dst_ptr;
124- HCCL_VM_INFO("[STUB]HostRegister is empty.");128+ HCCL_VM_INFO("HostRegister is empty.");
125 return DRV_ERROR_NONE;129 return DRV_ERROR_NONE;
126}130}
127 131 
@@ -129,7 +133,7 @@ drvError_t halHostUnregister(void *src_ptr, UINT32 devid)
129{133{
130 (void) src_ptr;134 (void) src_ptr;
131 (void) devid;135 (void) devid;
132- HCCL_VM_INFO("[STUB]HostUnregister is empty.");136+ HCCL_VM_INFO("HostUnregister is empty.");
133 return DRV_ERROR_NONE;137 return DRV_ERROR_NONE;
134}138}
135 139 
@@ -141,14 +145,14 @@ drvError_t halMemCtl(int type, void *param_value, size_t param_value_size, void
141 (void) param_value_size;145 (void) param_value_size;
142 (void) out_value;146 (void) out_value;
143 (void) out_size_ret;147 (void) out_size_ret;
144- HCCL_VM_INFO("[STUB]MemCtl is empty.");148+ HCCL_VM_INFO("MemCtl is empty.");
145 return DRV_ERROR_NONE;149 return DRV_ERROR_NONE;
146}150}
147 151 
148drvError_t drvGetPlatformInfo(uint32_t *info)152drvError_t drvGetPlatformInfo(uint32_t *info)
149{153{
150 *info = 1;154 *info = 1;
151- HCCL_VM_INFO("[STUB]drvGetPlatformInfo:{}", *info);155+ HCCL_VM_INFO("drvGetPlatformInfo:{}", *info);
152 return DRV_ERROR_NONE;156 return DRV_ERROR_NONE;
153}157}
154 158 
@@ -160,7 +164,7 @@ drvError_t drvQueryProcessHostPid(int pid, unsigned int *chip_id, unsigned int *
160 (void) vfid;164 (void) vfid;
161 (void) host_pid;165 (void) host_pid;
162 (void) cp_type;166 (void) cp_type;
163- HCCL_VM_INFO("[STUB]QueryProcessHostPid is empty.");167+ HCCL_VM_INFO("QueryProcessHostPid is empty.");
164 return DRV_ERROR_NONE;168 return DRV_ERROR_NONE;
165}169}
166 170 
@@ -182,21 +186,21 @@ drvError_t halEschedSubmitEvent(unsigned int devId, struct event_summary *event)
182{186{
183 (void) devId;187 (void) devId;
184 (void) event;188 (void) event;
185- HCCL_VM_INFO("[STUB]SubmitEvent is empty.");189+ HCCL_VM_INFO("SubmitEvent is empty.");
186 return DRV_ERROR_NONE;190 return DRV_ERROR_NONE;
187}191}
188 192 
189drvError_t halBindCgroup(BIND_CGROUP_TYPE bindType)193drvError_t halBindCgroup(BIND_CGROUP_TYPE bindType)
190{194{
191 (void) bindType;195 (void) bindType;
192- HCCL_VM_INFO("[STUB]BindCgroup is empty.");196+ HCCL_VM_INFO("BindCgroup is empty.");
193 return DRV_ERROR_NONE;197 return DRV_ERROR_NONE;
194}198}
195 199 
196drvError_t halGetAPIVersion(int *halAPIVersion)200drvError_t halGetAPIVersion(int *halAPIVersion)
197{201{
198 (void) halAPIVersion;202 (void) halAPIVersion;
199- HCCL_VM_INFO("[STUB]GetAPIVersion is empty.");203+ HCCL_VM_INFO("GetAPIVersion is empty.");
200 return DRV_ERROR_NONE;204 return DRV_ERROR_NONE;
201}205}
202 206 
@@ -205,7 +209,7 @@ drvError_t halSensorNodeRegister(uint32_t devId, struct halSensorNodeCfg *cfg, u
205 (void) devId;209 (void) devId;
206 (void) cfg;210 (void) cfg;
207 (void) handle;211 (void) handle;
208- HCCL_VM_INFO("[STUB]RegisterSensorNode is empty.");212+ HCCL_VM_INFO("RegisterSensorNode is empty.");
209 return DRV_ERROR_NONE;213 return DRV_ERROR_NONE;
210}214}
211 215 
@@ -213,7 +217,7 @@ drvError_t halSensorNodeUnregister(uint32_t devId, uint64_t handle)
213{217{
214 (void) devId;218 (void) devId;
215 (void) handle;219 (void) handle;
216- HCCL_VM_INFO("[STUB]UnregisterSensorNode is empty.");220+ HCCL_VM_INFO("UnregisterSensorNode is empty.");
217 return DRV_ERROR_NONE;221 return DRV_ERROR_NONE;
218}222}
219 223 
@@ -223,7 +227,7 @@ drvError_t halSensorNodeUpdateState(uint32_t devId, uint64_t handle, int val, ha
223 (void) handle;227 (void) handle;
224 (void) val;228 (void) val;
225 (void) assertion;229 (void) assertion;
226- HCCL_VM_INFO("[STUB]UpdateSensorNodeState is empty.");230+ HCCL_VM_INFO("UpdateSensorNodeState is empty.");
227 return DRV_ERROR_NONE;231 return DRV_ERROR_NONE;
228}232}
229 233 
@@ -251,28 +255,28 @@ int halGrpQuery(GroupQueryCmdType cmd, void *inBuff, unsigned int inLen, void *o
251 (void) inLen;255 (void) inLen;
252 (void) outBuff;256 (void) outBuff;
253 (void) outLen;257 (void) outLen;
254- HCCL_VM_INFO("[STUB]GrpQuery is empty.");258+ HCCL_VM_INFO("GrpQuery is empty.");
255 return 0;259 return 0;
256}260}
257 261 
258pid_t drvDeviceGetBareTgid(void)262pid_t drvDeviceGetBareTgid(void)
259{263{
260 pid_t pid = getpid();264 pid_t pid = getpid();
261- HCCL_VM_INFO("[STUB]drvDeviceGetBareTgid] pid:%u", pid);265+ HCCL_VM_INFO("drvDeviceGetBareTgid] pid:{}", pid);
262 return pid;266 return pid;
263}267}
264 268 
265drvError_t halResourceIdCheck(struct drvResIdKey *info)269drvError_t halResourceIdCheck(struct drvResIdKey *info)
266{270{
267 (void) info;271 (void) info;
268- HCCL_VM_INFO("[STUB]ResourceIdCheck is empty.");272+ HCCL_VM_INFO("ResourceIdCheck is empty.");
269 return DRV_ERROR_NONE;273 return DRV_ERROR_NONE;
270}274}
271 275 
272drvError_t halSqCqQuery(uint32_t devId, struct halSqCqQueryInfo *info)276drvError_t halSqCqQuery(uint32_t devId, struct halSqCqQueryInfo *info)
273{277{
274 (void) devId;278 (void) devId;
275- HCCL_VM_INFO("[STUB]SqCqQuery is empty.");279+ HCCL_VM_INFO("SqCqQuery is empty.");
276 if (info == nullptr) {280 if (info == nullptr) {
277 return DRV_ERROR_INNER_ERR;281 return DRV_ERROR_INNER_ERR;
278 }282 }
@@ -304,7 +308,7 @@ drvError_t halSqCqQuery(uint32_t devId, struct halSqCqQueryInfo *info)
304 308 
305drvError_t halSqCqConfig(uint32_t devId, struct halSqCqConfigInfo *info)309drvError_t halSqCqConfig(uint32_t devId, struct halSqCqConfigInfo *info)
306{310{
307- HCCL_VM_INFO("[STUB] devId[{}] is empty.", devId);311+ HCCL_VM_INFO("devId[{}] is empty.", devId);
308 if (info->prop == DRV_SQCQ_PROP_SQ_TAIL) {312 if (info->prop == DRV_SQCQ_PROP_SQ_TAIL) {
309 ParseA5SqeFromSqBuffer(devId, info);313 ParseA5SqeFromSqBuffer(devId, info);
310 }314 }
@@ -315,7 +319,7 @@ drvError_t halShrIdInfoGet(const char *name, struct shrIdGetInfo *info)
315{319{
316 (void) name;320 (void) name;
317 (void) info;321 (void) info;
318- HCCL_VM_INFO("[STUB] is empty.");322+ HCCL_VM_INFO("is empty.");
319 return DRV_ERROR_NONE;323 return DRV_ERROR_NONE;
320}324}
321 325 
@@ -327,21 +331,21 @@ drvError_t halTsdrvCtl(uint32_t devId, int cmd, void *param, size_t paramSize, v
327 (void) paramSize;331 (void) paramSize;
328 (void) out;332 (void) out;
329 (void) outSize;333 (void) outSize;
330- HCCL_VM_INFO("[STUB] is empty.");334+ HCCL_VM_INFO("is empty.");
331 return DRV_ERROR_NONE;335 return DRV_ERROR_NONE;
332}336}
333 337 
334drvError_t halResourceIdRestore(struct drvResIdKey *info)338drvError_t halResourceIdRestore(struct drvResIdKey *info)
335{339{
336 (void) info;340 (void) info;
337- HCCL_VM_INFO("[STUB] is empty.");341+ HCCL_VM_INFO("is empty.");
338 return DRV_ERROR_NONE;342 return DRV_ERROR_NONE;
339}343}
340 344 
341drvError_t drvGetLocalDevIDByHostDevID(uint32_t host_dev_id, uint32_t *local_dev_id)345drvError_t drvGetLocalDevIDByHostDevID(uint32_t host_dev_id, uint32_t *local_dev_id)
342{346{
343 *local_dev_id = host_dev_id;347 *local_dev_id = host_dev_id;
344- HCCL_VM_INFO("[STUB] host_dev_id:{}, local_dev_id:{}", host_dev_id, *local_dev_id);348+ HCCL_VM_INFO("host_dev_id:{}, local_dev_id:{}", host_dev_id, *local_dev_id);
345 return DRV_ERROR_NONE;349 return DRV_ERROR_NONE;
346}350}
347 351 
Mtest/hccl_vm/src/device_arm/proxy/device_sqe_parse_stub.cc+19-15
@@ -8,6 +8,9 @@
8 * See LICENSE in the root of the software repository for the full text of the License.8 * See LICENSE in the root of the software repository for the full text of the License.
9 */9 */
10 10 
11+// 日志染色: 模块 tag (须在 include sim_log.h 之前)
12+#define HCCL_VM_MODULE "DEVICE"
13+ 
11#include "device_sqe_parse_stub.h"14#include "device_sqe_parse_stub.h"
12 15 
13#include <cstdint>16#include <cstdint>
@@ -21,6 +24,7 @@
21#include "sqe_v82_stub.h"24#include "sqe_v82_stub.h"
22#include "udma_data_struct_stub.h"25#include "udma_data_struct_stub.h"
23 26 
27+ 
24constexpr int SHIFT_BIT32 = 32;28constexpr int SHIFT_BIT32 = 32;
25uint32_t curRankId = 0;29uint32_t curRankId = 0;
26 30 
@@ -72,7 +76,7 @@ void ParseA5SqeFromSqBuffer(uint32_t devId, struct halSqCqConfigInfo *info)
72 break;76 break;
73 }77 }
74 default: {78 default: {
75- HCCL_VM_ERROR("[ParseA5SqeFromSqBuffer] not support sqe type[{}].", static_cast<uint32_t>(header->type));79+ HCCL_VM_ERROR("not support sqe type[{}].", static_cast<uint32_t>(header->type));
76 break;80 break;
77 }81 }
78 }82 }
@@ -170,7 +174,7 @@ uint32_t CalculateCiValue(uint64_t wqeBuffer, uint32_t piValue)
170 ubCommon = reinterpret_cast<UdmaSqeCommon *>(wqeAddr);174 ubCommon = reinterpret_cast<UdmaSqeCommon *>(wqeAddr);
171 }175 }
172 176
173- HCCL_VM_ERROR("[CalculateCiValue] calculate ciValue failed.");177+ HCCL_VM_ERROR("calculate ciValue failed.");
174 return 0;178 return 0;
175}179}
176 180 
@@ -181,24 +185,24 @@ void ParseDavidUDMASqe(uint32_t streamId, void *sqeBuf)
181 185 
182 HcclAicpuData *aicpuData = GetHcclAicpuDataShmPtr();186 HcclAicpuData *aicpuData = GetHcclAicpuDataShmPtr();
183 if (aicpuData == nullptr) {187 if (aicpuData == nullptr) {
184- HCCL_VM_ERROR("[ParseDavidUDMASqe] aicpuData is nullptr.");188+ HCCL_VM_ERROR("aicpuData is nullptr.");
185 return;189 return;
186 }190 }
187 uint64_t wqeAddrDev = aicpuData->common.jettyId2WqeBufMap[jettyId];191 uint64_t wqeAddrDev = aicpuData->common.jettyId2WqeBufMap[jettyId];
188 uint64_t wqeBuffer = reinterpret_cast<uint64_t>(GetRealPtrByDevPtr(reinterpret_cast<void *>(wqeAddrDev)));192 uint64_t wqeBuffer = reinterpret_cast<uint64_t>(GetRealPtrByDevPtr(reinterpret_cast<void *>(wqeAddrDev)));
189 if (wqeBuffer == 0) {193 if (wqeBuffer == 0) {
190- HCCL_VM_ERROR("[ParseDavidUDMASqe] wqeBuffer is nullptr, wqeAddrDev[{}].", wqeAddrDev);194+ HCCL_VM_ERROR("wqeBuffer is nullptr, wqeAddrDev[{}].", wqeAddrDev);
191 return;195 return;
192 }196 }
193 197 
194 uint32_t piValue = ubSqe->piValue1;198 uint32_t piValue = ubSqe->piValue1;
195 uint32_t ciValue = CalculateCiValue(wqeBuffer, ubSqe->piValue1);199 uint32_t ciValue = CalculateCiValue(wqeBuffer, ubSqe->piValue1);
196 if (ciValue >= piValue) {200 if (ciValue >= piValue) {
197- HCCL_VM_ERROR("[ParseDavidUDMASqe] jettyId[{}] ciVal[{}] piValue[{}] streamId[{}].", jettyId, ciValue, piValue, streamId);201+ HCCL_VM_ERROR("jettyId[{}] ciVal[{}] piValue[{}] streamId[{}].", jettyId, ciValue, piValue, streamId);
198 return;202 return;
199 }203 }
200 204 
201- HCCL_VM_INFO("[ParseDavidUDMASqe] jettyId[{}] ciVal[{}] piValue[{}] streamId[{}].", jettyId, ciValue, piValue, streamId);205+ HCCL_VM_INFO("jettyId[{}] ciVal[{}] piValue[{}] streamId[{}].", jettyId, ciValue, piValue, streamId);
202 for (auto index = ciValue; index < piValue; index++) {206 for (auto index = ciValue; index < piValue; index++) {
203 uint64_t wqeAddr = wqeBuffer + index * HCCL_WQE_SIZE;207 uint64_t wqeAddr = wqeBuffer + index * HCCL_WQE_SIZE;
204 UdmaSqeCommon *ubCommon = reinterpret_cast<UdmaSqeCommon *>(wqeAddr);208 UdmaSqeCommon *ubCommon = reinterpret_cast<UdmaSqeCommon *>(wqeAddr);
@@ -217,7 +221,7 @@ void ParseDavidUDMASqe(uint32_t streamId, void *sqeBuf)
217 break;221 break;
218 }222 }
219 default: {223 default: {
220- HCCL_VM_ERROR("[ParseDavidUDMASqe] not support opcode[{}].", static_cast<uint32_t>(ubCommon->opcode));224+ HCCL_VM_ERROR("not support opcode[{}].", static_cast<uint32_t>(ubCommon->opcode));
221 return;225 return;
222 }226 }
223 }227 }
@@ -344,7 +348,7 @@ HcclReduceOp ParseReduceTypeDavid(uint8_t result)
344 return DavidReduceOpMap[reduceType];348 return DavidReduceOpMap[reduceType];
345 }349 }
346 350 
347- HCCL_VM_ERROR("[ParseReduceTypeDavid] not support reduceType[{}].", reduceType);351+ HCCL_VM_ERROR("not support reduceType[{}].", reduceType);
348 return HcclReduceOp::HCCL_REDUCE_RESERVED;352 return HcclReduceOp::HCCL_REDUCE_RESERVED;
349}353}
350 354 
@@ -367,7 +371,7 @@ HcclDataType ParseDataTypeDavid(uint8_t result)
367 return DavidDataTypeMap[dataType];371 return DavidDataTypeMap[dataType];
368 }372 }
369 373 
370- HCCL_VM_ERROR("[ParseDataTypeDavid] not support dataType[{}].", dataType);374+ HCCL_VM_ERROR("not support dataType[{}].", dataType);
371 return HcclDataType::HCCL_DATA_TYPE_RESERVED;375 return HcclDataType::HCCL_DATA_TYPE_RESERVED;
372}376}
373 377 
@@ -383,7 +387,7 @@ HcclReduceOp ParseUbReduceTypeDavid(uint32_t type)
383 return DavidUbReduceOpMap[type];387 return DavidUbReduceOpMap[type];
384 }388 }
385 389 
386- HCCL_VM_ERROR("[ParseUbReduceTypeDavid] not support type[{}].", type);390+ HCCL_VM_ERROR("not support type[{}].", type);
387 return HcclReduceOp::HCCL_REDUCE_RESERVED;391 return HcclReduceOp::HCCL_REDUCE_RESERVED;
388}392}
389 393 
@@ -405,7 +409,7 @@ HcclDataType ParseUbDataTypeDavid(uint32_t type)
405 return DavidUbDataTypeMap[type];409 return DavidUbDataTypeMap[type];
406 }410 }
407 411 
408- HCCL_VM_ERROR("[ParseUbDataTypeDavid] not support type[{}].", type);412+ HCCL_VM_ERROR("not support type[{}].", type);
409 return HcclDataType::HCCL_DATA_TYPE_RESERVED;413 return HcclDataType::HCCL_DATA_TYPE_RESERVED;
410}414}
411 415 
@@ -414,21 +418,21 @@ void PrintTaskMetaData(const HcclTaskMetaData &taskMeta)
414 pid_t pid = getpid();418 pid_t pid = getpid();
415 switch (taskMeta.taskType) {419 switch (taskMeta.taskType) {
416 case HccLTaskMetaType::MEM_CPY:420 case HccLTaskMetaType::MEM_CPY:
417- HCCL_VM_INFO("pid{}: rankId[{}] streamId[{}] taskType[MEM_CPY], srcOffset[{}] dstOffset[{}], len[{}], srcRankId[{}], dstRankId[{}]\n",421+ HCCL_VM_INFO("pid{}: rankId[{}] streamId[{}] taskType[MEM_CPY], srcOffset[{}] dstOffset[{}], len[{}], srcRankId[{}], dstRankId[{}]",
418 pid, taskMeta.rankId, taskMeta.streamId, taskMeta.taskData.transMem.srcOffset, taskMeta.taskData.transMem.dstOffset, taskMeta.taskData.transMem.len,422 pid, taskMeta.rankId, taskMeta.streamId, taskMeta.taskData.transMem.srcOffset, taskMeta.taskData.transMem.dstOffset, taskMeta.taskData.transMem.len,
419 taskMeta.taskData.transMem.srcRankId, taskMeta.taskData.transMem.dstRankId);423 taskMeta.taskData.transMem.srcRankId, taskMeta.taskData.transMem.dstRankId);
420 break;424 break;
421 case HccLTaskMetaType::REDUCE: 425 case HccLTaskMetaType::REDUCE:
422- HCCL_VM_INFO("pid{}: rankId[{}] streamId[{}] taskType[REDUCE], srcOffset[{}] dstOffset[{}], len[{}], srcRankId[{}], dstRankId[{}], reduceOp[%{}], dataType[%{}]\n",426+ HCCL_VM_INFO("pid{}: rankId[{}] streamId[{}] taskType[REDUCE], srcOffset[{}] dstOffset[{}], len[{}], srcRankId[{}], dstRankId[{}], reduceOp[%{}], dataType[%{}]",
423 pid, taskMeta.rankId, taskMeta.streamId, taskMeta.taskData.reduce.srcOffset, taskMeta.taskData.reduce.dstOffset, taskMeta.taskData.reduce.dataCount,427 pid, taskMeta.rankId, taskMeta.streamId, taskMeta.taskData.reduce.srcOffset, taskMeta.taskData.reduce.dstOffset, taskMeta.taskData.reduce.dataCount,
424 taskMeta.taskData.reduce.srcRankId, taskMeta.taskData.reduce.dstRankId, taskMeta.taskData.reduce.reduceOp, taskMeta.taskData.reduce.dataType);428 taskMeta.taskData.reduce.srcRankId, taskMeta.taskData.reduce.dstRankId, taskMeta.taskData.reduce.reduceOp, taskMeta.taskData.reduce.dataType);
425 break;429 break;
426 case HccLTaskMetaType::NOTIFY_WAIT:430 case HccLTaskMetaType::NOTIFY_WAIT:
427- HCCL_VM_INFO("pid{}: rankId[{}] streamId[{}] taskType[NOTIFY_WAIT], notifyId[{}], srcRankId[{}], dstRankId[{}]\n",431+ HCCL_VM_INFO("pid{}: rankId[{}] streamId[{}] taskType[NOTIFY_WAIT], notifyId[{}], srcRankId[{}], dstRankId[{}]",
428 pid, taskMeta.rankId, taskMeta.streamId, taskMeta.taskData.notify.notifyId, taskMeta.taskData.notify.srcRankId, taskMeta.taskData.notify.dstRankId);432 pid, taskMeta.rankId, taskMeta.streamId, taskMeta.taskData.notify.notifyId, taskMeta.taskData.notify.srcRankId, taskMeta.taskData.notify.dstRankId);
429 break;433 break;
430 case HccLTaskMetaType::NOTIFY_RECORD:434 case HccLTaskMetaType::NOTIFY_RECORD:
431- HCCL_VM_INFO("pid{}: rankId[{}] streamId[{}] taskType[NOTIFY_RECORD], notifyId[{}], srcRankId[{}], dstRankId[{}]\n",435+ HCCL_VM_INFO("pid{}: rankId[{}] streamId[{}] taskType[NOTIFY_RECORD], notifyId[{}], srcRankId[{}], dstRankId[{}]",
432 pid, taskMeta.rankId, taskMeta.streamId, taskMeta.taskData.notify.notifyId, taskMeta.taskData.notify.srcRankId, taskMeta.taskData.notify.dstRankId);436 pid, taskMeta.rankId, taskMeta.streamId, taskMeta.taskData.notify.notifyId, taskMeta.taskData.notify.srcRankId, taskMeta.taskData.notify.dstRankId);
433 break;437 break;
434 default:438 default:
Mtest/hccl_vm/src/main.cc+33-1
@@ -14,13 +14,44 @@
14 14 
15#include "cmd_base.h"15#include "cmd_base.h"
16#include "cmd_base_utils.h"16#include "cmd_base_utils.h"
17+#include "sim_common_api.h"
17#include "sim_log.h"18#include "sim_log.h"
18#include "db_sim_op_db_ops.h"19#include "db_sim_op_db_ops.h"
19 20 
21+void ArchiveLogsAndData()
22+{
23+ const std::string& installRoot = InstallPath::GetHcclVmInstallAbsPath();
24+ if (installRoot.empty())
25+ return;
26+ 
27+ std::string archiveScript = installRoot + "/script/archive.sh";
28+ if (access(archiveScript.c_str(), F_OK) != 0)
29+ return;
30+ 
31+ pid_t pid1 = fork();
32+ if (pid1 == 0) {
33+ pid_t pid2 = fork();
34+ if (pid2 == 0) {
35+ setsid();
36+ int devNull = open("/dev/null", O_RDWR);
37+ dup2(devNull, STDIN_FILENO);
38+ dup2(devNull, STDOUT_FILENO);
39+ dup2(devNull, STDERR_FILENO);
40+ if (devNull > STDERR_FILENO) {
41+ close(devNull);
42+ }
43+ sleep(1);
44+ execlp("bash", "bash", archiveScript.c_str(), nullptr);
45+ _exit(1);
46+ }
47+ _exit(0);
48+ }
49+}
50+ 
20void envInit()51void envInit()
21{52{
22 setenv("HCCL_VM_INSTALL_ROOT", GetBinLocation().c_str(), 1);53 setenv("HCCL_VM_INSTALL_ROOT", GetBinLocation().c_str(), 1);
23- std::string dbPrefix = GetBinLocation() + "/data/hccl_vm_data.db";54+ std::string dbPrefix = InstallPath::ResolveToInstallRoot("data/hccl_vm_data.db");
24 int ret3 = system(("sudo rm -fr " + dbPrefix + " " + dbPrefix + "-wal " + dbPrefix + "-shm 2>/dev/null").c_str());55 int ret3 = system(("sudo rm -fr " + dbPrefix + " " + dbPrefix + "-wal " + dbPrefix + "-shm 2>/dev/null").c_str());
25 int ret1 = system("sudo rm -fr /dev/shm/* 2>/dev/null");56 int ret1 = system("sudo rm -fr /dev/shm/* 2>/dev/null");
26 int ret2 = system("sudo rm -fr /tmp/hccl_sim.db* 2>/dev/null");57 int ret2 = system("sudo rm -fr /tmp/hccl_sim.db* 2>/dev/null");
@@ -41,6 +72,7 @@ int main(int argc, char *argv[])
41 LogConfig config = LoadLogConfig("hccl_vm");72 LogConfig config = LoadLogConfig("hccl_vm");
42 InitLogger(config);73 InitLogger(config);
43 envInit();74 envInit();
75+ std::atexit(ArchiveLogsAndData);
44 std::string cmd = ArgvToString(argc, argv);76 std::string cmd = ArgvToString(argc, argv);
45 if (argc == 1) {77 if (argc == 1) {
46 cmd += " --help";78 cmd += " --help";
Mtest/hccl_vm/src/plugin/checker/CMakeLists.txt+3-0
@@ -99,3 +99,6 @@ install(FILES "manifest.json"
99 GROUP_READ GROUP_EXECUTE99 GROUP_READ GROUP_EXECUTE
100 WORLD_READ WORLD_EXECUTE100 WORLD_READ WORLD_EXECUTE
101)101)
102+ 
103+# 日志染色: 模块 tag
104+target_compile_definitions(checker PRIVATE HCCL_VM_MODULE="CHECKER")
Mtest/hccl_vm/src/plugin/checker/main.cc+108-66
@@ -31,6 +31,8 @@
31#include "mem_conflict_check_utils.h"31#include "mem_conflict_check_utils.h"
32#include "sim_loader.h"32#include "sim_loader.h"
33#include "sim_common_defs.h"33#include "sim_common_defs.h"
34+#include "utils/check_utils.h"
35+#include "utils/error_codes.h"
34 36 
35using json = nlohmann::json;37using json = nlohmann::json;
36loader::Loader g_loader;38loader::Loader g_loader;
@@ -132,32 +134,52 @@ static const char* HcclReduceOpToString(HcclReduceOp t) {
132 }134 }
133}135}
134 136 
137+static void AppendApplicableRoleFields(std::ostringstream &os, const HcclSim::CheckerParam &param)
138+{
139+ switch (param.cmdType) {
140+ case HCCL_CMD_SEND:
141+ case HCCL_CMD_RECEIVE:
142+ os << ", sourceRank=" << param.srcRank << ", targetRank=" << param.dstRank;
143+ break;
144+ case HCCL_CMD_BROADCAST:
145+ case HCCL_CMD_REDUCE:
146+ case HCCL_CMD_SCATTER:
147+ os << ", rootRank=" << param.root;
148+ break;
149+ default:
150+ break;
151+ }
152+}
153+ 
135static HcclResult LoadCheckerDataBase(154static HcclResult LoadCheckerDataBase(
136 std::vector<sim::CcuChannelTab>& channels,155 std::vector<sim::CcuChannelTab>& channels,
137 std::vector<sim::CcuInstrResTab>& instrRes,156 std::vector<sim::CcuInstrResTab>& instrRes,
138 std::vector<sim::SyncRecordTab>& syncRecords,157 std::vector<sim::SyncRecordTab>& syncRecords,
139 uint32_t& syncIterMaxNum)158 uint32_t& syncIterMaxNum)
140{159{
141- HcclResult ret = g_loader.GetCcuChannelInfo(channels);160+ HcclResult ret = g_loader.GetCcuChannelInfo(channels);
142 if (ret != HcclResult::HCCL_SUCCESS) {161 if (ret != HcclResult::HCCL_SUCCESS) {
143- HCCL_VM_ERROR("[LoadCheckerDataBase] GetCcuChannelInfo failed, ret[{}]", static_cast<u32>(ret));162+ HCCL_VM_ERROR("{} Failed to load CCU channel information",
163+ HcclSim::MakeErrorCodeText(HcclSim::ErrorCode::CHECKER_RUNTIME_ERROR));
144 return ret;164 return ret;
145 }165 }
146 166 
147 ret = g_loader.GetInstrResInfo(instrRes);167 ret = g_loader.GetInstrResInfo(instrRes);
148 if (ret != HcclResult::HCCL_SUCCESS) {168 if (ret != HcclResult::HCCL_SUCCESS) {
149- HCCL_VM_ERROR("[LoadCheckerDataBase] GetInstrResInfo failed, ret[{}]", static_cast<u32>(ret));169+ HCCL_VM_ERROR("{} Failed to load CCU instruction resource information",
170+ HcclSim::MakeErrorCodeText(HcclSim::ErrorCode::CHECKER_RUNTIME_ERROR));
150 return ret;171 return ret;
151 }172 }
152 173 
153 ret = g_loader.GetSyncInfo(syncRecords);174 ret = g_loader.GetSyncInfo(syncRecords);
154 if (ret != HcclResult::HCCL_SUCCESS) {175 if (ret != HcclResult::HCCL_SUCCESS) {
155- HCCL_VM_ERROR("[LoadCheckerDataBase] GetSyncInfo failed, ret[{}]", static_cast<u32>(ret));176+ HCCL_VM_ERROR("{} Failed to load sync records",
177+ HcclSim::MakeErrorCodeText(HcclSim::ErrorCode::CHECKER_RUNTIME_ERROR));
156 return ret;178 return ret;
157 }179 }
158 180 
159 if (syncRecords.empty()) {181 if (syncRecords.empty()) {
160- HCCL_VM_WARN("[LoadCheckerDataBase] No sync records found");182+ HCCL_VM_WARN("No sync records were found");
161 return HcclResult::HCCL_E_PARA;183 return HcclResult::HCCL_E_PARA;
162 }184 }
163 185 
@@ -166,7 +188,8 @@ static HcclResult LoadCheckerDataBase(
166 });188 });
167 189 
168 syncIterMaxNum = syncRecords.back().syncIter;190 syncIterMaxNum = syncRecords.back().syncIter;
169- HCCL_VM_INFO("[LoadCheckerDataBase] channels={}, instrRes={}, syncRecords={}, syncIterMaxNum={}",191+ HCCL_VM_INFO("Checker database loaded, channelCount={}, instrResCount={}, syncRecordCount={}, "
192+ "syncIterMaxNum={}",
170 channels.size(), instrRes.size(), syncRecords.size(), syncIterMaxNum);193 channels.size(), instrRes.size(), syncRecords.size(), syncIterMaxNum);
171 return HcclResult::HCCL_SUCCESS;194 return HcclResult::HCCL_SUCCESS;
172}195}
@@ -209,11 +232,12 @@ static HcclResult DispatchCheckByCmdType(
209 return CheckAll2AllVC(taskQueues, param.rankSize,232 return CheckAll2AllVC(taskQueues, param.rankSize,
210 static_cast<HcclDataType>(param.all2AllDataDes.sendType), param.all2AllDataDes.sendCountMatrix);233 static_cast<HcclDataType>(param.all2AllDataDes.sendType), param.all2AllDataDes.sendCountMatrix);
211 case HcclCMDType::HCCL_CMD_ALLTOALLV:234 case HcclCMDType::HCCL_CMD_ALLTOALLV:
212- HCCL_VM_WARN("CheckOpSemantics does not support HCCL_CMD_ALLTOALLV, using HCCL_CMD_ALLTOALLVV instead");235+ HCCL_VM_WARN("Checker does not support AllToAllV and will use the AllToAllVC validation path");
213 return CheckAll2AllVC(taskQueues, param.rankSize,236 return CheckAll2AllVC(taskQueues, param.rankSize,
214 static_cast<HcclDataType>(param.all2AllDataDes.sendType), param.all2AllDataDes.sendCountMatrix);237 static_cast<HcclDataType>(param.all2AllDataDes.sendType), param.all2AllDataDes.sendCountMatrix);
215 default:238 default:
216- HCCL_VM_ERROR("Unknown HcclCMDType {}", static_cast<u32>(cmdType));239+ HCCL_VM_ERROR("{} Unsupported collective type, collectiveTypeCode={}",
240+ HcclSim::MakeErrorCodeText(HcclSim::ErrorCode::CHECKER_RUNTIME_ERROR), static_cast<u32>(cmdType));
217 return HcclResult::HCCL_E_NOT_SUPPORT;241 return HcclResult::HCCL_E_NOT_SUPPORT;
218 }242 }
219}243}
@@ -227,20 +251,21 @@ static HcclResult LoadOpDataForOneRank(
227{251{
228 HcclResult ret = storage.LoadHcclVmSynthesisData(rankId, op.memInfo, channels);252 HcclResult ret = storage.LoadHcclVmSynthesisData(rankId, op.memInfo, channels);
229 if (ret != HcclResult::HCCL_SUCCESS) {253 if (ret != HcclResult::HCCL_SUCCESS) {
230- HCCL_VM_ERROR("[LoadOpDataForOneRank] LoadHcclVmSynthesisData rankId={} failed, ret={}",254+ HCCL_VM_ERROR("{} Failed to load synthesized memory information for this rank, rankId={}",
231- rankId, static_cast<u32>(ret));255+ HcclSim::MakeErrorCodeText(HcclSim::ErrorCode::CHECKER_RUNTIME_ERROR), rankId);
232 return ret;256 return ret;
233 }257 }
234 258 
235 ret = storage.LoadHcclVmInstrData(instrRes);259 ret = storage.LoadHcclVmInstrData(instrRes);
236 if (ret != HcclResult::HCCL_SUCCESS) {260 if (ret != HcclResult::HCCL_SUCCESS) {
237- HCCL_VM_ERROR("[LoadOpDataForOneRank] LoadHcclVmInstrData rankId={} failed, ret={}",261+ HCCL_VM_ERROR("{} Failed to load instruction data for this rank, rankId={}",
238- rankId, static_cast<u32>(ret));262+ HcclSim::MakeErrorCodeText(HcclSim::ErrorCode::CHECKER_RUNTIME_ERROR), rankId);
239 return ret;263 return ret;
240 }264 }
241 265 
242 if (op.detail.opDetail.size() < sizeof(::OpDetails)) {266 if (op.detail.opDetail.size() < sizeof(::OpDetails)) {
243- HCCL_VM_ERROR("[LoadOpDataForOneRank] opDetail BLOB too small, rankId={}", rankId);267+ HCCL_VM_ERROR("{} Op detail payload is too small to parse, rankId={}",
268+ HcclSim::MakeErrorCodeText(HcclSim::ErrorCode::CHECKER_RUNTIME_ERROR), rankId);
244 return HcclResult::HCCL_E_PARA;269 return HcclResult::HCCL_E_PARA;
245 }270 }
246 271 
@@ -248,8 +273,8 @@ static HcclResult LoadOpDataForOneRank(
248 std::memcpy(&opDetails, op.detail.opDetail.data(), sizeof(::OpDetails));273 std::memcpy(&opDetails, op.detail.opDetail.data(), sizeof(::OpDetails));
249 ret = storage.Trans2CheckerParam(op.detail, opDetails);274 ret = storage.Trans2CheckerParam(op.detail, opDetails);
250 if (ret != HcclResult::HCCL_SUCCESS) {275 if (ret != HcclResult::HCCL_SUCCESS) {
251- HCCL_VM_ERROR("[LoadOpDataForOneRank] Trans2CheckerParam rankId={} failed, ret={}",276+ HCCL_VM_ERROR("{} Failed to convert this rank into checker input parameters, rankId={}",
252- rankId, static_cast<u32>(ret));277+ HcclSim::MakeErrorCodeText(HcclSim::ErrorCode::CHECKER_RUNTIME_ERROR), rankId);
253 return ret;278 return ret;
254 }279 }
255 return HcclResult::HCCL_SUCCESS;280 return HcclResult::HCCL_SUCCESS;
@@ -272,17 +297,19 @@ static HcclResult ProcessOneOpGroup(
272 bool enableOldChecker = settingManager.IsOldCheckerEnabled();297 bool enableOldChecker = settingManager.IsOldCheckerEnabled();
273 bool isAivOp = false;298 bool isAivOp = false;
274 299 
275- HCCL_VM_INFO("[RunChecker] opgroup size {}", opGroup.size());300+ HCCL_VM_INFO("Start checking one op group, opGroupSize={}", opGroup.size());
276 std::vector<std::vector<sim::OpTaskTab>> allTasks;301 std::vector<std::vector<sim::OpTaskTab>> allTasks;
277 for (auto& entry : opGroup) {302 for (auto& entry : opGroup) {
278 uint32_t rankId = entry.first;303 uint32_t rankId = entry.first;
279- HCCL_VM_INFO("[ProcessOneOpGroup] rankId={}", rankId);304+ HCCL_VM_INFO("Load one rank from this op group, rankId={}", rankId);
280 sim::CompositeOpDetail& op = entry.second;305 sim::CompositeOpDetail& op = entry.second;
281 isAivOp = isAivOp || IsAivOpExpansionMode(op.detail.opExpansionMode);306 isAivOp = isAivOp || IsAivOpExpansionMode(op.detail.opExpansionMode);
282 allTasks.push_back(op.tasks);307 allTasks.push_back(op.tasks);
283- HcclResult r = LoadOpDataForOneRank(storage, channels, instrRes, rankId, op);308+ HcclResult ret = LoadOpDataForOneRank(storage, channels, instrRes, rankId, op);
284- if (r != HcclResult::HCCL_SUCCESS) {309+ if (ret != HcclResult::HCCL_SUCCESS) {
285- return r;310+ HCCL_VM_ERROR("{} Failed to load one rank from this op group, opIndex={}, rankId={}",
311+ HcclSim::MakeErrorCodeText(HcclSim::ErrorCode::CHECKER_RUNTIME_ERROR), opIdx, rankId);
312+ return ret;
286 }313 }
287 if (dumpManager.IsEnabled()) {314 if (dumpManager.IsEnabled()) {
288 HcclSim::DumpRunManifest::GetInstance().SetOpParam(BuildOpParamSummaryJson(storage.GetCheckerParam()));315 HcclSim::DumpRunManifest::GetInstance().SetOpParam(BuildOpParamSummaryJson(storage.GetCheckerParam()));
@@ -290,7 +317,10 @@ static HcclResult ProcessOneOpGroup(
290 }317 }
291 318 
292 if (!enableNewChecker && !enableOldChecker) {319 if (!enableNewChecker && !enableOldChecker) {
293- HCCL_VM_WARN("[ProcessOneOpGroup] both new checker and old checker are disabled, skip op[{}].", opIdx);320+ HCCL_VM_ERROR("{} This op is skipped because both the new checker and the old checker are disabled, "
321+ "opIndex={}, newCheckerEnabled={}, oldCheckerEnabled={}",
322+ HcclSim::MakeErrorCodeText(HcclSim::ErrorCode::SETTING_WARNING), opIdx, enableNewChecker,
323+ enableOldChecker);
294 return HcclResult::HCCL_SUCCESS;324 return HcclResult::HCCL_SUCCESS;
295 }325 }
296 326 
@@ -299,18 +329,21 @@ static HcclResult ProcessOneOpGroup(
299 329 
300 auto ret = storage.LoadHcclVmTaskMetaData(allTasks);330 auto ret = storage.LoadHcclVmTaskMetaData(allTasks);
301 if (ret != HcclResult::HCCL_SUCCESS) {331 if (ret != HcclResult::HCCL_SUCCESS) {
302- HCCL_VM_ERROR("[ProcessOneOpGroup] LoadHcclVmTaskMetaData failed, ret={}",332+ HCCL_VM_ERROR("{} Failed to load V3 task metadata for this op group",
303- static_cast<u32>(ret));333+ HcclSim::MakeErrorCodeText(HcclSim::ErrorCode::CHECKER_RUNTIME_ERROR));
304 return ret;334 return ret;
305 }335 }
306 isAivOp = isAivOp || HasAivGraphTask(allTasks);336 isAivOp = isAivOp || HasAivGraphTask(allTasks);
307 if (isAivOp) {337 if (isAivOp) {
308 if (!enableNewChecker) {338 if (!enableNewChecker) {
309- HCCL_VM_ERROR("[ProcessOneOpGroup] AIV checker requires V3 checker, but new checker is disabled.");339+ HCCL_VM_ERROR("{} This AIV op requires the V3 checker, but V3 is disabled by configuration, "
340+ "opIndex={}, action=abort, newCheckerEnabled={}",
341+ HcclSim::MakeErrorCodeText(HcclSim::ErrorCode::SETTING_WARNING), opIdx, enableNewChecker);
310 return HcclResult::HCCL_E_NOT_SUPPORT;342 return HcclResult::HCCL_E_NOT_SUPPORT;
311 }343 }
312 if (enableOldChecker) {344 if (enableOldChecker) {
313- HCCL_VM_INFO("[ProcessOneOpGroup] AIV op detected, skip legacy checker and use V3-only.");345+ HCCL_VM_WARN("AIV op detected, the old checker is skipped and only CheckerV3 will run, "
346+ "opIndex={}, oldCheckerEnabled={}", opIdx, enableOldChecker);
314 enableOldChecker = false;347 enableOldChecker = false;
315 }348 }
316 enableNewChecker = true;349 enableNewChecker = true;
@@ -318,41 +351,47 @@ static HcclResult ProcessOneOpGroup(
318 351 
319 {352 {
320 auto checkerParamBrief = storage.GetCheckerParam();353 auto checkerParamBrief = storage.GetCheckerParam();
321- HCCL_VM_INFO("[Checker][OpInfo] op[{}] op = {}, rankSize = {}, dataType = {}, dataCount = {}, reduceType = {}, "354+ std::ostringstream summary;
322- "srcRank = {}, dstRank = {}, root = {}, opGroupSize={}, isAivOp={}",355+ summary << "[Main] Op summary, opIndex=" << opIdx
323- opIdx, HcclCmdTypeToString(checkerParamBrief.cmdType), checkerParamBrief.rankSize,356+ << ", collectiveType=" << HcclSim::HcclCmdTypeToString(checkerParamBrief.cmdType)
324- HcclDataTypeToString(checkerParamBrief.dataType), checkerParamBrief.dataCount,357+ << ", rankCount=" << checkerParamBrief.rankSize
325- HcclReduceOpToString(checkerParamBrief.reduceType),358+ << ", dataType=" << HcclSim::HcclDataTypeToString(checkerParamBrief.dataType)
326- checkerParamBrief.srcRank, checkerParamBrief.dstRank, checkerParamBrief.root,359+ << ", elementCount=" << checkerParamBrief.dataCount
327- opGroup.size(), isAivOp);360+ << ", reduceType=" << HcclReduceOpToString(checkerParamBrief.reduceType);
361+ AppendApplicableRoleFields(summary, checkerParamBrief);
362+ summary << ", opGroupSize=" << opGroup.size()
363+ << ", containsAivTask=" << isAivOp;
364+ HCCL_VM_INFO("{}", summary.str());
328 }365 }
329 366 
330 HcclResult newCheckerRet = HcclResult::HCCL_SUCCESS;367 HcclResult newCheckerRet = HcclResult::HCCL_SUCCESS;
331 if (enableNewChecker) {368 if (enableNewChecker) {
369+ HCCL_VM_INFO("----------[Start CheckerV3]----------");
332 newCheckerRet = HcclSim::GenAndCheckGraphV3();370 newCheckerRet = HcclSim::GenAndCheckGraphV3();
333- HCCL_VM_WARN("----------[V3 Checker Finished]----------");371+ HCCL_VM_INFO("----------[CheckerV3 Finished]----------");
334- HCCL_VM_INFO("[ProcessOneOpGroup] new checker finished, ret={}", static_cast<u32>(newCheckerRet));372+ HCCL_VM_INFO("CheckerV3 finished for this op, opIndex={}", opIdx);
335 } else {373 } else {
336- HCCL_VM_INFO("[ProcessOneOpGroup] new checker disabled by manifest setting.");374+ HCCL_VM_INFO("CheckerV3 is disabled by configuration");
337 }375 }
338 376 
339 HcclResult oldCheckerRet = HcclResult::HCCL_SUCCESS;377 HcclResult oldCheckerRet = HcclResult::HCCL_SUCCESS;
340 if (enableOldChecker) {378 if (enableOldChecker) {
341- HCCL_VM_WARN("----------[Start Old Checker]----------");379+ HCCL_VM_INFO("----------[Start Old Checker]----------");
380+ HCCL_VM_INFO("Start running the old checker, opIndex={}, dataId={}", opIdx,
381+ storage.GetDataId());
342 HcclSim::AllRankTaskQueues taskQueues;382 HcclSim::AllRankTaskQueues taskQueues;
343 HcclSim::ConvertTaskQueue(taskQueues);383 HcclSim::ConvertTaskQueue(taskQueues);
344 auto checkerParam = storage.GetCheckerParam();384 auto checkerParam = storage.GetCheckerParam();
345- HCCL_VM_INFO("[ProcessOneOpGroup] dispatch legacy checker, rankCount={}, cmdType={}",
346- taskQueues.size(), static_cast<u32>(checkerParam.cmdType));
347 oldCheckerRet = DispatchCheckByCmdType(taskQueues, checkerParam);385 oldCheckerRet = DispatchCheckByCmdType(taskQueues, checkerParam);
348- HCCL_VM_INFO("[ProcessOneOpGroup] old checker finished, ret={}", static_cast<u32>(oldCheckerRet));386+ HCCL_VM_INFO("----------[Old Checker Finished]----------");
387+ HCCL_VM_INFO("Old checker finished for this op, opIndex={}", opIdx);
349 } else {388 } else {
350- HCCL_VM_INFO("[ProcessOneOpGroup] old checker disabled by manifest setting.");389+ HCCL_VM_INFO("Old checker is disabled by configuration");
351 }390 }
352 391 
353 if (!isAivOp && enableNewChecker && enableOldChecker && newCheckerRet != oldCheckerRet) {392 if (!isAivOp && enableNewChecker && enableOldChecker && newCheckerRet != oldCheckerRet) {
354- HCCL_VM_WARN("[ProcessOneOpGroup] new checker and old checker results differ, newCheckerRet={}, oldCheckerRet={}",393+ HCCL_VM_WARN("CheckerV3 result differs from old checker result, opIndex={}, checkerV3Ret={}, "
355- static_cast<u32>(newCheckerRet), static_cast<u32>(oldCheckerRet));394+ "oldCheckerRet={}", opIdx, static_cast<u32>(newCheckerRet), static_cast<u32>(oldCheckerRet));
356 }395 }
357 if (enableNewChecker) {396 if (enableNewChecker) {
358 return newCheckerRet;397 return newCheckerRet;
@@ -420,21 +459,22 @@ void RunChecker(const std::string& data_id) {
420 storage.SetDataId(data_id);459 storage.SetDataId(data_id);
421 const HcclResult settingRefreshRet = HcclSim::SettingManager::GetInstance().Refresh();460 const HcclResult settingRefreshRet = HcclSim::SettingManager::GetInstance().Refresh();
422 if (settingRefreshRet != HcclResult::HCCL_SUCCESS) {461 if (settingRefreshRet != HcclResult::HCCL_SUCCESS) {
423- HCCL_VM_WARN("[RunChecker] failed to refresh manifest settings, keep previous settings, ret = {}",462+ HCCL_VM_WARN("Failed to refresh manifest settings, the previous checker settings will be kept");
424- static_cast<u32>(settingRefreshRet));
425 }463 }
426 HcclSim::DumpManager &dumpManager = HcclSim::DumpManager::GetInstance();464 HcclSim::DumpManager &dumpManager = HcclSim::DumpManager::GetInstance();
427 dumpManager.Reset();465 dumpManager.Reset();
428 HcclResult dumpInitRet = dumpManager.Initialize(data_id);466 HcclResult dumpInitRet = dumpManager.Initialize(data_id);
429 if (dumpInitRet != HcclResult::HCCL_SUCCESS) {467 if (dumpInitRet != HcclResult::HCCL_SUCCESS) {
430- HCCL_VM_ERROR("[RunChecker] DumpManager initialize failed, ret = {}", static_cast<u32>(dumpInitRet));468+ HCCL_VM_ERROR("{} Failed to initialize the old checker dump manager, old checker output files "
469+ "cannot be written, dataId={}", HcclSim::MakeErrorCodeText(HcclSim::ErrorCode::DUMP_FAILED), data_id);
431 return;470 return;
432 }471 }
433 HcclSim::DumpV3Manager &dumpV3Manager = HcclSim::DumpV3Manager::GetInstance();472 HcclSim::DumpV3Manager &dumpV3Manager = HcclSim::DumpV3Manager::GetInstance();
434 dumpV3Manager.Reset();473 dumpV3Manager.Reset();
435 dumpInitRet = dumpV3Manager.Initialize(data_id);474 dumpInitRet = dumpV3Manager.Initialize(data_id);
436 if (dumpInitRet != HcclResult::HCCL_SUCCESS) {475 if (dumpInitRet != HcclResult::HCCL_SUCCESS) {
437- HCCL_VM_ERROR("[RunChecker] DumpV3Manager initialize failed, ret = {}", static_cast<u32>(dumpInitRet));476+ HCCL_VM_ERROR("{} Failed to initialize the V3 dump manager, checker output files cannot be written, "
477+ "dataId={}", HcclSim::MakeErrorCodeText(HcclSim::ErrorCode::DUMP_FAILED), data_id);
438 return;478 return;
439 }479 }
440 HcclSim::DumpRunManifest::GetInstance().Reset(data_id);480 HcclSim::DumpRunManifest::GetInstance().Reset(data_id);
@@ -446,7 +486,7 @@ void RunChecker(const std::string& data_id) {
446 486 
447 HcclResult loadRet = g_loader.LoadOpTaskFile();487 HcclResult loadRet = g_loader.LoadOpTaskFile();
448 if (loadRet != HcclResult::HCCL_SUCCESS) {488 if (loadRet != HcclResult::HCCL_SUCCESS) {
449- HCCL_VM_ERROR("[RunChecker] LoadOpTaskFile failed, ret = {}", static_cast<u32>(loadRet));489+ HCCL_VM_ERROR("Failed to load the op task file");
450 return;490 return;
451 }491 }
452 492 
@@ -459,35 +499,36 @@ void RunChecker(const std::string& data_id) {
459 return;499 return;
460 }500 }
461 501 
462- HCCL_VM_INFO("[RunChecker] start check semantic: syncRecords size = {}", syncRecords.size());502+ HCCL_VM_INFO("Start checker run, syncRecordCount={}", syncRecords.size());
463 uint32_t opIdx = 0;503 uint32_t opIdx = 0;
464 do {504 do {
465 for (uint32_t syncIter = 0; syncIter <= syncIterMaxNum; syncIter++) {505 for (uint32_t syncIter = 0; syncIter <= syncIterMaxNum; syncIter++) {
466 std::map<uint32_t, std::vector<sim::CompositeOpDetail>> compositeDataMap;506 std::map<uint32_t, std::vector<sim::CompositeOpDetail>> compositeDataMap;
467 g_loader.LoadCompositeOpDetailBySyncIter(syncIter, compositeDataMap);507 g_loader.LoadCompositeOpDetailBySyncIter(syncIter, compositeDataMap);
468 auto opGroups = TransposeCompositeOpMap(compositeDataMap);508 auto opGroups = TransposeCompositeOpMap(compositeDataMap);
469- HCCL_VM_INFO("[RunChecker] syncIter={}, opGroups={}", syncIter, opGroups.size());509+ HCCL_VM_INFO("Start one sync iteration, syncIter={}, opGroupCount={}", syncIter, opGroups.size());
470 for (auto& opGroup : opGroups) {510 for (auto& opGroup : opGroups) {
471- HCCL_VM_INFO("[RunChecker] opgroup size {}", opGroup.size());511+ HCCL_VM_INFO("Check one op group in this sync iteration, opGroupSize={}", opGroup.size());
472 ret = ProcessOneOpGroup(storage, channels, instrRes, opIdx, opGroup);512 ret = ProcessOneOpGroup(storage, channels, instrRes, opIdx, opGroup);
473 if (dumpManager.IsEnabled()) {513 if (dumpManager.IsEnabled()) {
474 HcclSim::DumpRunManifest::GetInstance().SetCheckResult(ret);514 HcclSim::DumpRunManifest::GetInstance().SetCheckResult(ret);
475 const HcclResult flushRet = HcclSim::ValidationIssueRecorder::GetInstance().Flush();515 const HcclResult flushRet = HcclSim::ValidationIssueRecorder::GetInstance().Flush();
476 if (flushRet != HcclResult::HCCL_SUCCESS) {516 if (flushRet != HcclResult::HCCL_SUCCESS) {
477- HCCL_VM_WARN("[RunChecker] failed to flush validation issue dump for op[{}], ret={}",517+ HCCL_VM_WARN("{} Failed to flush the validation issue dump, dataId={}, opIndex={}, "
478- opIdx, static_cast<u32>(flushRet));518+ "dumpType=validation_issues", HcclSim::MakeErrorCodeText(HcclSim::ErrorCode::DUMP_FAILED),
519+ data_id, opIdx);
479 }520 }
480 const HcclResult manifestRet = HcclSim::DumpRunManifest::GetInstance().Flush();521 const HcclResult manifestRet = HcclSim::DumpRunManifest::GetInstance().Flush();
481 if (manifestRet != HcclResult::HCCL_SUCCESS) {522 if (manifestRet != HcclResult::HCCL_SUCCESS) {
482- HCCL_VM_WARN("[RunChecker] failed to flush dump manifest for op[{}], ret={}",523+ HCCL_VM_WARN("{} Failed to flush the dump manifest, dataId={}, opIndex={}",
483- opIdx, static_cast<u32>(manifestRet));524+ HcclSim::MakeErrorCodeText(HcclSim::ErrorCode::DUMP_FAILED), data_id, opIdx);
484 }525 }
485 }526 }
486 if (ret != HcclResult::HCCL_SUCCESS) {527 if (ret != HcclResult::HCCL_SUCCESS) {
487- HCCL_VM_ERROR("[RunChecker] op[{}] failed with error code {}", opIdx, static_cast<u32>(ret));528+ HCCL_VM_ERROR("op[{}] Checker failed", opIdx);
488 break;529 break;
489 }530 }
490- HCCL_VM_INFO("[RunChecker] op[{}] Checker Success.", opIdx);531+ HCCL_VM_INFO("op[{}] Checker Success", opIdx);
491 opIdx++;532 opIdx++;
492 }533 }
493 }534 }
@@ -518,30 +559,31 @@ void JoinCheckerWorker()
518void ProcessCommand(const std::string& line) {559void ProcessCommand(const std::string& line) {
519 try {560 try {
520 auto j = json::parse(line);561 auto j = json::parse(line);
521- const HcclResult settingRefreshRet = HcclSim::SettingManager::GetInstance().Refresh();
522- if (settingRefreshRet != HcclResult::HCCL_SUCCESS) {
523- HCCL_VM_WARN("[Checker] failed to refresh manifest settings, keep previous settings, ret = {}",
524- static_cast<u32>(settingRefreshRet));
525- }
526 std::string action = j.value("action", "");562 std::string action = j.value("action", "");
527 auto payload = j.value("payload", json::object());563 auto payload = j.value("payload", json::object());
528 564 
529 if (action == "status") {565 if (action == "status") {
530 std::string status = payload.value("status", "");566 std::string status = payload.value("status", "");
531- HCCL_VM_INFO("[Checker] Status signal received. Status: {}", status);567+ HCCL_VM_INFO("Received checker status signal, status={}", status);
532 if (status != "finish") {568 if (status != "finish") {
533 return;569 return;
534 }570 }
571+ // 运行前刷新设置,确保最新的配置生效
572+ const HcclResult settingRefreshRet = HcclSim::SettingManager::GetInstance().Refresh();
573+ if (settingRefreshRet != HcclResult::HCCL_SUCCESS) {
574+ HCCL_VM_WARN("Failed to refresh manifest settings, use the previous settings");
575+ }
576+ 
535 std::string data_id = payload.value("data_id", "");577 std::string data_id = payload.value("data_id", "");
536 StartCheckerWorker(data_id);578 StartCheckerWorker(data_id);
537 } 579 }
538 else if (action == "stop") {580 else if (action == "stop") {
539- HCCL_VM_INFO("[Checker] Stop signal received. Initiating shutdown...");581+ HCCL_VM_INFO("Received checker stop signal, shutdown...");
540 g_keep_running.store(false); // 仅仅修改标志位582 g_keep_running.store(false); // 仅仅修改标志位
541 }583 }
542 // 其他 action...584 // 其他 action...
543 } catch (const std::exception& e) {585 } catch (const std::exception& e) {
544- std::cerr << "[Error] Command processing failed: " << e.what() << std::endl;586+ HCCL_VM_ERROR("Command processing failed: {}", e.what());
545 }587 }
546}588}
547 589 
@@ -549,7 +591,7 @@ int main() {
549 LogConfig config = LoadLogConfig("checker");591 LogConfig config = LoadLogConfig("checker");
550 InitLogger(config);592 InitLogger(config);
551 593 
552- HCCL_VM_INFO("[Checker] Plugin process active. Listening for commands...");594+ HCCL_VM_INFO("Plugin process active. Listening for commands...");
553 595 
554 // 主循环检查标志位596 // 主循环检查标志位
555 std::string line;597 std::string line;
@@ -562,6 +604,6 @@ int main() {
562 }604 }
563 605 
564 JoinCheckerWorker();606 JoinCheckerWorker();
565- HCCL_VM_INFO("[Checker] shutdown.");607+ HCCL_VM_INFO("shutdown.");
566 return 0; // 整个进程唯一的正常出口608 return 0; // 整个进程唯一的正常出口
567}609}
Mtest/hccl_vm/src/plugin/checker/src/checker/checker.cc+51-834
@@ -33,6 +33,7 @@
33#include "dump/dump_graph.h"33#include "dump/dump_graph.h"
34#include "dump/validation_issue_recorder.h"34#include "dump/validation_issue_recorder.h"
35#include "dump_v3/dump_v3_dag.h"35#include "dump_v3/dump_v3_dag.h"
36+#include "dump_v3/dump_v3_manager.h"
36#include "framework/task_graph_generator_v3/task_graph_generator_v3.h"37#include "framework/task_graph_generator_v3/task_graph_generator_v3.h"
37#include "framework/task_graph_generator_v3/task_graph_mem_conflict_v3.h"38#include "framework/task_graph_generator_v3/task_graph_mem_conflict_v3.h"
38#include "framework/task_graph_generator_v3/task_graph_semantic_check_v3.h"39#include "framework/task_graph_generator_v3/task_graph_semantic_check_v3.h"
@@ -51,16 +52,15 @@
51#include "task_graph_revamp_bilateral_ccu.h"52#include "task_graph_revamp_bilateral_ccu.h"
52#include "task_graph_revamp_parallel.h"53#include "task_graph_revamp_parallel.h"
53#include "task_utils.h"54#include "task_utils.h"
55+#include "utils/error_codes.h"
54 56 
55using namespace std;57using namespace std;
56 58 
57namespace HcclSim {59namespace HcclSim {
58namespace {60namespace {
59using V3TaskNode = TaskGraphGeneratorV3::TaskNode;61using V3TaskNode = TaskGraphGeneratorV3::TaskNode;
60-using OldNodeStreamPosMap = std::map<TaskNodePtr, uint32_t>;
61using V3NodeStreamPosMap = std::map<const V3TaskNode *, uint32_t>;62using V3NodeStreamPosMap = std::map<const V3TaskNode *, uint32_t>;
62using V3StageClock = std::chrono::steady_clock;63using V3StageClock = std::chrono::steady_clock;
63-constexpr size_t MAX_GRAPH_COMPARE_DETAIL_LOG_COUNT = 20;
64constexpr uint32_t INVALID_GRAPH_STREAM_POS = std::numeric_limits<uint32_t>::max();64constexpr uint32_t INVALID_GRAPH_STREAM_POS = std::numeric_limits<uint32_t>::max();
65constexpr uint64_t V3_NS_PER_MS = 1000000ULL;65constexpr uint64_t V3_NS_PER_MS = 1000000ULL;
66 66 
@@ -80,11 +80,11 @@ public:
80 {80 {
81 const uint64_t elapsedNs = ElapsedV3Ns(start_, V3StageClock::now());81 const uint64_t elapsedNs = ElapsedV3Ns(start_, V3StageClock::now());
82 if (extraInfo_.empty()) {82 if (extraInfo_.empty()) {
83- HCCL_VM_WARN("[TaskGraphGeneratorV3][StageTime] {} {}, costMs={}", stage_, status_,83+ HCCL_VM_INFO("CheckerV3 stage finished, stage={}, status={}, costMs={}", stage_, status_,
84 elapsedNs / V3_NS_PER_MS);84 elapsedNs / V3_NS_PER_MS);
85 return;85 return;
86 }86 }
87- HCCL_VM_WARN("[TaskGraphGeneratorV3][StageTime] {} {}, costMs={}, {}", stage_, status_,87+ HCCL_VM_INFO("CheckerV3 stage finished, stage={}, status={}, costMs={}, {}", stage_, status_,
88 elapsedNs / V3_NS_PER_MS, extraInfo_);88 elapsedNs / V3_NS_PER_MS, extraInfo_);
89 }89 }
90 90 
@@ -118,15 +118,6 @@ struct V3StreamKey {
118 }118 }
119};119};
120 120 
121-bool IsOldGraphCompareTransparent(TaskNodePtr node)
122-{
123- if (node == nullptr || node->task == nullptr) {
124- return false;
125- }
126- const TaskTypeStub taskType = node->task->GetType();
127- return taskType == TaskTypeStub::GRAPH_SEPARATE || taskType == TaskTypeStub::SUB_GRAPH_END;
128-}
129- 
130bool IsV3Boundary(const V3TaskNode *node, TaskGraphGeneratorV3::BoundaryType boundaryType)121bool IsV3Boundary(const V3TaskNode *node, TaskGraphGeneratorV3::BoundaryType boundaryType)
131{122{
132 if (node == nullptr) {123 if (node == nullptr) {
@@ -143,7 +134,7 @@ bool IsV3Boundary(const V3TaskNode *node, TaskGraphGeneratorV3::BoundaryType bou
143 return false;134 return false;
144}135}
145 136 
146-bool IsV3GraphCompareTransparent(const V3TaskNode *node)137+bool IsV3StreamPosHiddenNode(const V3TaskNode *node)
147{138{
148 if (node == nullptr) {139 if (node == nullptr) {
149 return false;140 return false;
@@ -203,32 +194,6 @@ bool IsAivDagLogNode(const V3TaskNode *node)
203 IsV3Boundary(node, TaskGraphGeneratorV3::BoundaryType::AIV_SUB_GRAPH);194 IsV3Boundary(node, TaskGraphGeneratorV3::BoundaryType::AIV_SUB_GRAPH);
204}195}
205 196 
206-std::vector<TaskNodePtr> BuildOldGraphCompareNodes(const std::vector<TaskNodePtr> &oldBfsNodes)
207-{
208- std::vector<TaskNodePtr> result;
209- result.reserve(oldBfsNodes.size());
210- for (TaskNodePtr node : oldBfsNodes) {
211- if (IsOldGraphCompareTransparent(node)) {
212- continue;
213- }
214- result.push_back(node);
215- }
216- return result;
217-}
218- 
219-std::vector<const V3TaskNode *> BuildV3GraphCompareNodes(const std::vector<const V3TaskNode *> &newBfsNodes)
220-{
221- std::vector<const V3TaskNode *> result;
222- result.reserve(newBfsNodes.size());
223- for (const V3TaskNode *node : newBfsNodes) {
224- if (IsV3GraphCompareTransparent(node)) {
225- continue;
226- }
227- result.push_back(node);
228- }
229- return result;
230-}
231- 
232std::string EscapeGraphvizLabel(const std::string &label)197std::string EscapeGraphvizLabel(const std::string &label)
233{198{
234 std::string escaped;199 std::string escaped;
@@ -263,7 +228,7 @@ std::string FormatGraphvizLabelText(const std::string &text)
263 228 
264std::unique_ptr<TaskGraphGeneratorV3::TaskGraphGeneratorV3> GenGraphV3FromTaskMeta()229std::unique_ptr<TaskGraphGeneratorV3::TaskGraphGeneratorV3> GenGraphV3FromTaskMeta()
265{230{
266- V3StageTimer timer("GenGraphV3FromTaskMeta");231+ V3StageTimer timer("GenGraph");
267 StorageManager &storage = StorageManager::GetInstance();232 StorageManager &storage = StorageManager::GetInstance();
268 TaskGraphGeneratorV3::TaskMetaTranslatorV3 taskMetaTranslatorV3;233 TaskGraphGeneratorV3::TaskMetaTranslatorV3 taskMetaTranslatorV3;
269 // V3 主流程入口:234 // V3 主流程入口:
@@ -272,8 +237,8 @@ std::unique_ptr<TaskGraphGeneratorV3::TaskGraphGeneratorV3> GenGraphV3FromTaskMe
272 // 3. 再生成 V3 图并在日志里记录成图/CCU 展开耗时。237 // 3. 再生成 V3 图并在日志里记录成图/CCU 展开耗时。
273 HcclResult ret = taskMetaTranslatorV3.Translate(storage);238 HcclResult ret = taskMetaTranslatorV3.Translate(storage);
274 if (ret != HCCL_SUCCESS) {239 if (ret != HCCL_SUCCESS) {
275- HCCL_VM_WARN("[TaskGraphGeneratorV3] Translate task meta failed, skip V3 graph generation, ret={}",240+ HCCL_VM_WARN("Failed to translate one task into the V3 internal format, V3 graph generation is "
276- static_cast<uint32_t>(ret));241+ "stopped, ret={}", static_cast<uint32_t>(ret));
277 return nullptr;242 return nullptr;
278 }243 }
279 244 
@@ -281,21 +246,21 @@ std::unique_ptr<TaskGraphGeneratorV3::TaskGraphGeneratorV3> GenGraphV3FromTaskMe
281 auto translatedTaskQueues = taskMetaTranslatorV3.TakeTaskQueues();246 auto translatedTaskQueues = taskMetaTranslatorV3.TakeTaskQueues();
282 ret = TaskGraphGeneratorV3::CheckSlaveTaskQueue(translatedNodes, translatedTaskQueues);247 ret = TaskGraphGeneratorV3::CheckSlaveTaskQueue(translatedNodes, translatedTaskQueues);
283 if (ret != HCCL_SUCCESS) {248 if (ret != HCCL_SUCCESS) {
284- HCCL_VM_WARN("[TaskGraphGeneratorV3] Check slave task queue failed, skip V3 graph generation, ret={}",249+ HCCL_VM_WARN("Slave-stream check failed, V3 graph generation is stopped, ret={}",
285 static_cast<uint32_t>(ret));250 static_cast<uint32_t>(ret));
286 return nullptr;251 return nullptr;
287 }252 }
288 std::unique_ptr<TaskGraphGeneratorV3::TaskGraphGeneratorV3> graphGeneratorV3(253 std::unique_ptr<TaskGraphGeneratorV3::TaskGraphGeneratorV3> graphGeneratorV3(
289 new (std::nothrow) TaskGraphGeneratorV3::TaskGraphGeneratorV3);254 new (std::nothrow) TaskGraphGeneratorV3::TaskGraphGeneratorV3);
290 if (graphGeneratorV3 == nullptr) {255 if (graphGeneratorV3 == nullptr) {
291- HCCL_VM_WARN("[TaskGraphGeneratorV3] Create graph generator failed.");256+ HCCL_VM_WARN("Failed to create the V3 graph generator object");
292 return nullptr;257 return nullptr;
293 }258 }
294 259 
295 ret = graphGeneratorV3->GenGraph(std::move(translatedNodes), std::move(translatedTaskQueues));260 ret = graphGeneratorV3->GenGraph(std::move(translatedNodes), std::move(translatedTaskQueues));
296 if (ret != HCCL_SUCCESS) {261 if (ret != HCCL_SUCCESS) {
297- HCCL_VM_WARN("[TaskGraphGeneratorV3] Generate graph from translated task nodes failed, ret={}",262+ HCCL_VM_WARN("Failed to build the graph from translated tasks, ret={}, rankCount={}, nodeCount={}",
298- static_cast<uint32_t>(ret));263+ static_cast<uint32_t>(ret), graphGeneratorV3->GetTaskQueues().size(), graphGeneratorV3->GetNodes().size());
299 return nullptr;264 return nullptr;
300 }265 }
301 const auto &ccuExpandStats = graphGeneratorV3->GetCcuExpandStats();266 const auto &ccuExpandStats = graphGeneratorV3->GetCcuExpandStats();
@@ -312,35 +277,6 @@ std::unique_ptr<TaskGraphGeneratorV3::TaskGraphGeneratorV3> GenGraphV3FromTaskMe
312 return graphGeneratorV3;277 return graphGeneratorV3;
313}278}
314 279 
315-std::vector<TaskNodePtr> BfsOldGraph(TaskNodePtr start)
316-{
317- std::vector<TaskNodePtr> result;
318- if (start == nullptr) {
319- return result;
320- }
321- 
322- std::queue<TaskNodePtr> nodeQue;
323- std::set<TaskNodePtr> visited;
324- nodeQue.push(start);
325- visited.insert(start);
326- 
327- while (!nodeQue.empty()) {
328- TaskNodePtr currNode = nodeQue.front();
329- nodeQue.pop();
330- result.push_back(currNode);
331- 
332- for (TaskNodePtr child : currNode->children) {
333- if (child == nullptr || visited.count(child) != 0) {
334- continue;
335- }
336- visited.insert(child);
337- nodeQue.push(child);
338- }
339- }
340- 
341- return result;
342-}
343- 
344std::vector<const V3TaskNode *> BfsGraphV3(const V3TaskNode *start)280std::vector<const V3TaskNode *> BfsGraphV3(const V3TaskNode *start)
345{281{
346 std::vector<const V3TaskNode *> result;282 std::vector<const V3TaskNode *> result;
@@ -370,48 +306,12 @@ std::vector<const V3TaskNode *> BfsGraphV3(const V3TaskNode *start)
370 return result;306 return result;
371}307}
372 308 
373-OldNodeStreamPosMap BuildOldStreamPosMap(const std::vector<TaskNodePtr> &bfsNodes)
374-{
375- std::map<V3StreamKey, std::vector<TaskNodePtr>> nodesByStream;
376- for (TaskNodePtr node : bfsNodes) {
377- if (node == nullptr || node->task == nullptr || IsOldGraphCompareTransparent(node)) {
378- continue;
379- }
380- V3StreamKey key;
381- key.rankId = node->rankIdx;
382- key.streamId = node->queIdx;
383- nodesByStream[key].push_back(node);
384- }
385- 
386- OldNodeStreamPosMap streamPosByNode;
387- for (auto &streamItem : nodesByStream) {
388- auto &streamNodes = streamItem.second;
389- std::sort(streamNodes.begin(), streamNodes.end(), [](TaskNodePtr lhs, TaskNodePtr rhs) {
390- if (lhs == rhs) {
391- return false;
392- }
393- if (lhs == nullptr || rhs == nullptr) {
394- return lhs < rhs;
395- }
396- if (lhs->pos != rhs->pos) {
397- return lhs->pos < rhs->pos;
398- }
399- return lhs < rhs;
400- });
401- 
402- for (size_t pos = 0; pos < streamNodes.size(); ++pos) {
403- streamPosByNode[streamNodes[pos]] = static_cast<uint32_t>(pos);
404- }
405- }
406- return streamPosByNode;
407-}
408- 
409V3NodeStreamPosMap BuildV3StreamPosMap(const std::vector<const V3TaskNode *> &bfsNodes)309V3NodeStreamPosMap BuildV3StreamPosMap(const std::vector<const V3TaskNode *> &bfsNodes)
410{310{
411 std::map<V3StreamKey, std::vector<const V3TaskNode *>> nodesByStream;311 std::map<V3StreamKey, std::vector<const V3TaskNode *>> nodesByStream;
412 for (const V3TaskNode *node : bfsNodes) {312 for (const V3TaskNode *node : bfsNodes) {
413 if (node == nullptr || node->GetType() == TaskGraphGeneratorV3::TaskType::START ||313 if (node == nullptr || node->GetType() == TaskGraphGeneratorV3::TaskType::START ||
414- IsV3GraphCompareTransparent(node)) {314+ IsV3StreamPosHiddenNode(node)) {
415 continue;315 continue;
416 }316 }
417 const auto &loc = node->GetPosition();317 const auto &loc = node->GetPosition();
@@ -481,67 +381,6 @@ const char *TaskTypeKeyName(TaskGraphGeneratorV3::TaskType type)
481 }381 }
482}382}
483 383 
484-TaskGraphGeneratorV3::MemType ConvertOldMemTypeToV3(BufferType bufferType)
485-{
486- switch (bufferType) {
487- case ::INPUT:
488- return TaskGraphGeneratorV3::MemType::INPUT;
489- case ::OUTPUT:
490- return TaskGraphGeneratorV3::MemType::OUTPUT;
491- case ::CCL:
492- case ::SCRATCH:
493- return TaskGraphGeneratorV3::MemType::CCL;
494- case ::MS:
495- return TaskGraphGeneratorV3::MemType::MS_CCU;
496- case ::INPUT_AIV:
497- case ::OUTPUT_AIV:
498- case ::AIV_COMMINFO:
499- case ::USERBUF_AIV:
500- return TaskGraphGeneratorV3::MemType::UB_AIV;
501- default:
502- return TaskGraphGeneratorV3::MemType::INVALID;
503- }
504-}
505- 
506-TaskGraphGeneratorV3::ProtocolType ConvertOldProtocolToV3(LinkProtoStub protocol)
507-{
508- if (protocol == LinkProtoStub::RDMA) {
509- return TaskGraphGeneratorV3::ProtocolType::RDMA;
510- }
511- if (protocol == LinkProtoStub::SDMA) {
512- return TaskGraphGeneratorV3::ProtocolType::SDMA;
513- }
514- if (protocol == LinkProtoStub::CCU) {
515- return TaskGraphGeneratorV3::ProtocolType::CCU;
516- }
517- return TaskGraphGeneratorV3::ProtocolType::INVALID;
518-}
519- 
520-TaskGraphGeneratorV3::TaskType ConvertOldTaskTypeToV3(TaskTypeStub oldType)
521-{
522- switch (oldType) {
523- case TaskTypeStub::LOCAL_COPY:
524- case TaskTypeStub::READ:
525- case TaskTypeStub::WRITE:
526- return TaskGraphGeneratorV3::TaskType::TRANS_MEM;
527- case TaskTypeStub::LOCAL_REDUCE:
528- case TaskTypeStub::LOCAL_BATCH_REDUCE:
529- case TaskTypeStub::READ_REDUCE:
530- case TaskTypeStub::WRITE_REDUCE:
531- return TaskGraphGeneratorV3::TaskType::REDUCE;
532- case TaskTypeStub::LOCAL_POST_TO:
533- case TaskTypeStub::POST:
534- return TaskGraphGeneratorV3::TaskType::RECORD;
535- case TaskTypeStub::LOCAL_WAIT_FROM:
536- case TaskTypeStub::WAIT:
537- return TaskGraphGeneratorV3::TaskType::WAIT;
538- case TaskTypeStub::CCU_GRAPH:
539- return TaskGraphGeneratorV3::TaskType::CCU_GRAPH;
540- default:
541- return TaskGraphGeneratorV3::TaskType::INVALID;
542- }
543-}
544- 
545std::string MakePositionKey(uint32_t rankId, uint32_t streamId, uint32_t streamPos)384std::string MakePositionKey(uint32_t rankId, uint32_t streamId, uint32_t streamPos)
546{385{
547 std::ostringstream os;386 std::ostringstream os;
@@ -564,11 +403,6 @@ std::string MakeMemSliceKey(RankId rankId, TaskGraphGeneratorV3::MemType memType
564 return os.str();403 return os.str();
565}404}
566 405 
567-std::string MakeMemSliceKey(RankId rankId, const DataSlice &slice)
568-{
569- return MakeMemSliceKey(rankId, ConvertOldMemTypeToV3(slice.GetType()), slice.GetOffset(), slice.GetSize());
570-}
571- 
572std::string MakeMemSliceKey(const TaskGraphGeneratorV3::MemSlice &slice)406std::string MakeMemSliceKey(const TaskGraphGeneratorV3::MemSlice &slice)
573{407{
574 return MakeMemSliceKey(slice.rankId, slice.memType, slice.offset, slice.len);408 return MakeMemSliceKey(slice.rankId, slice.memType, slice.offset, slice.len);
@@ -591,162 +425,6 @@ std::string MakeNotifyKey(RankId recordRankId, RankId waitRankId, uint64_t notif
591 return MakeNotifyKey(notify);425 return MakeNotifyKey(notify);
592}426}
593 427 
594-bool MakeOldNodeSemanticKey(TaskNodePtr oldNode, const OldNodeStreamPosMap *streamPosByNode, std::string &key)
595-{
596- if (oldNode == nullptr) {
597- return false;
598- }
599- 
600- if (oldNode->task == nullptr) {
601- key = "type=START|boundary=MAIN_GRAPH";
602- return true;
603- }
604- 
605- const TaskGraphGeneratorV3::TaskType newType = ConvertOldTaskTypeToV3(oldNode->task->GetType());
606- if (newType == TaskGraphGeneratorV3::TaskType::INVALID) {
607- return false;
608- }
609- 
610- uint32_t streamPos = oldNode->pos;
611- if (streamPosByNode != nullptr) {
612- const auto posIter = streamPosByNode->find(oldNode);
613- streamPos = (posIter == streamPosByNode->end()) ? INVALID_GRAPH_STREAM_POS : posIter->second;
614- }
615- 
616- std::ostringstream os;
617- os << "type=" << TaskTypeKeyName(newType) << "|" << MakePositionKey(oldNode->rankIdx, oldNode->queIdx,
618- streamPos);
619- 
620- switch (oldNode->task->GetType()) {
621- case TaskTypeStub::LOCAL_COPY: {
622- auto *copy = dynamic_cast<TaskStubLocalCopy *>(oldNode->task);
623- if (copy == nullptr) {
624- return false;
625- }
626- os << "|src=" << MakeMemSliceKey(oldNode->rankIdx, copy->GetSrcSlice())
627- << "|dst=" << MakeMemSliceKey(oldNode->rankIdx, copy->GetDstSlice())
628- << "|" << MakeProtocolKey(TaskGraphGeneratorV3::ProtocolType::SDMA);
629- break;
630- }
631- case TaskTypeStub::READ: {
632- auto *read = dynamic_cast<TaskStubRead *>(oldNode->task);
633- if (read == nullptr) {
634- return false;
635- }
636- os << "|src=" << MakeMemSliceKey(read->GetRemoteRank(), read->GetRemoteSlice())
637- << "|dst=" << MakeMemSliceKey(oldNode->rankIdx, read->GetLocalSlice())
638- << "|" << MakeProtocolKey(ConvertOldProtocolToV3(read->GetLinkType()));
639- break;
640- }
641- case TaskTypeStub::WRITE: {
642- auto *write = dynamic_cast<TaskStubWrite *>(oldNode->task);
643- if (write == nullptr) {
644- return false;
645- }
646- os << "|src=" << MakeMemSliceKey(oldNode->rankIdx, write->GetLocalSlice())
647- << "|dst=" << MakeMemSliceKey(write->GetRemoteRank(), write->GetRemoteSlice())
648- << "|" << MakeProtocolKey(ConvertOldProtocolToV3(write->GetLinkType()));
649- break;
650- }
651- case TaskTypeStub::LOCAL_REDUCE: {
652- auto *reduce = dynamic_cast<TaskStubLocalReduce *>(oldNode->task);
653- if (reduce == nullptr) {
654- return false;
655- }
656- os << "|src=" << MakeMemSliceKey(oldNode->rankIdx, reduce->GetSrcSlice())
657- << "|dst=" << MakeMemSliceKey(oldNode->rankIdx, reduce->GetDstSlice())
658- << "|dataType=" << static_cast<uint32_t>(reduce->GetDataType())
659- << "|reduceOp=" << static_cast<uint32_t>(reduce->GetReduceOp())
660- << "|" << MakeProtocolKey(TaskGraphGeneratorV3::ProtocolType::SDMA);
661- break;
662- }
663- case TaskTypeStub::LOCAL_BATCH_REDUCE: {
664- auto *reduce = dynamic_cast<TaskStubLocalBatchReduce *>(oldNode->task);
665- if (reduce == nullptr || reduce->GetSrcSlices().empty()) {
666- return false;
667- }
668- os << "|src=" << MakeMemSliceKey(oldNode->rankIdx, reduce->GetSrcSlice(0))
669- << "|dst=" << MakeMemSliceKey(oldNode->rankIdx, reduce->GetDstSlice())
670- << "|dataType=" << static_cast<uint32_t>(reduce->GetDataType())
671- << "|reduceOp=" << static_cast<uint32_t>(reduce->GetReduceOp())
672- << "|" << MakeProtocolKey(TaskGraphGeneratorV3::ProtocolType::SDMA);
673- break;
674- }
675- case TaskTypeStub::READ_REDUCE: {
676- auto *reduce = dynamic_cast<TaskStubReadReduce *>(oldNode->task);
677- if (reduce == nullptr) {
678- return false;
679- }
680- os << "|src=" << MakeMemSliceKey(reduce->GetRemoteRank(), reduce->GetRemoteSlice())
681- << "|dst=" << MakeMemSliceKey(oldNode->rankIdx, reduce->GetLocalSlice())
682- << "|dataType=" << static_cast<uint32_t>(reduce->GetDataType())
683- << "|reduceOp=" << static_cast<uint32_t>(reduce->GetReduceOp())
684- << "|" << MakeProtocolKey(ConvertOldProtocolToV3(reduce->GetLinkType()));
685- break;
686- }
687- case TaskTypeStub::WRITE_REDUCE: {
688- auto *reduce = dynamic_cast<TaskStubWriteReduce *>(oldNode->task);
689- if (reduce == nullptr) {
690- return false;
691- }
692- os << "|src=" << MakeMemSliceKey(oldNode->rankIdx, reduce->GetLocalSlice())
693- << "|dst=" << MakeMemSliceKey(reduce->GetRemoteRank(), reduce->GetRemoteSlice())
694- << "|dataType=" << static_cast<uint32_t>(reduce->GetDataType())
695- << "|reduceOp=" << static_cast<uint32_t>(reduce->GetReduceOp())
696- << "|" << MakeProtocolKey(ConvertOldProtocolToV3(reduce->GetLinkType()));
697- break;
698- }
699- case TaskTypeStub::LOCAL_POST_TO: {
700- auto *post = dynamic_cast<TaskStubLocalPostTo *>(oldNode->task);
701- if (post == nullptr) {
702- return false;
703- }
704- os << "|notify=" << MakeNotifyKey(oldNode->rankIdx, oldNode->rankIdx, post->GetNotifyId())
705- << "|" << MakeProtocolKey(TaskGraphGeneratorV3::ProtocolType::INVALID);
706- break;
707- }
708- case TaskTypeStub::LOCAL_WAIT_FROM: {
709- auto *wait = dynamic_cast<TaskStubLocalWaitFrom *>(oldNode->task);
710- if (wait == nullptr) {
711- return false;
712- }
713- os << "|notify=" << MakeNotifyKey(oldNode->rankIdx, oldNode->rankIdx, wait->GetNotifyId())
714- << "|" << MakeProtocolKey(TaskGraphGeneratorV3::ProtocolType::INVALID);
715- break;
716- }
717- case TaskTypeStub::POST: {
718- auto *post = dynamic_cast<TaskStubPost *>(oldNode->task);
719- if (post == nullptr) {
720- return false;
721- }
722- os << "|notify=" << MakeNotifyKey(oldNode->rankIdx, post->GetRemoteRank(), post->GetNotifyId())
723- << "|" << MakeProtocolKey(ConvertOldProtocolToV3(post->GetLinkType()));
724- break;
725- }
726- case TaskTypeStub::WAIT: {
727- auto *wait = dynamic_cast<TaskStubWait *>(oldNode->task);
728- if (wait == nullptr) {
729- return false;
730- }
731- os << "|notify=" << MakeNotifyKey(wait->GetRemoteRank(), oldNode->rankIdx, wait->GetNotifyId())
732- << "|" << MakeProtocolKey(ConvertOldProtocolToV3(wait->GetLinkType()));
733- break;
734- }
735- case TaskTypeStub::CCU_GRAPH:
736- break;
737- default:
738- return false;
739- }
740- 
741- key = os.str();
742- return true;
743-}
744- 
745-bool MakeOldNodeSemanticKey(TaskNodePtr oldNode, std::string &key)
746-{
747- return MakeOldNodeSemanticKey(oldNode, nullptr, key);
748-}
749- 
750bool MakeV3NodeSemanticKey(const V3TaskNode *newNode, const V3NodeStreamPosMap &streamPosByNode,428bool MakeV3NodeSemanticKey(const V3TaskNode *newNode, const V3NodeStreamPosMap &streamPosByNode,
751 std::string &key)429 std::string &key)
752{430{
@@ -884,27 +562,6 @@ bool MakeV3NodeSemanticKey(const V3TaskNode *newNode, const V3NodeStreamPosMap &
884 return true;562 return true;
885}563}
886 564 
887-std::string MakeOldGraphvizNodeLabel(TaskNodePtr oldNode, size_t bfsIndex)
888-{
889- std::ostringstream os;
890- os << "old_" << bfsIndex << "\nbfs=" << bfsIndex;
891- if (oldNode == nullptr) {
892- os << "\n<null old node>";
893- return os.str();
894- }
895- 
896- os << "\nrank=" << oldNode->rankIdx << ", stream=" << oldNode->queIdx << ", index=" << oldNode->pos;
897- std::string semanticKey;
898- if (MakeOldNodeSemanticKey(oldNode, semanticKey)) {
899- os << "\n" << FormatGraphvizLabelText(semanticKey);
900- } else if (oldNode->task != nullptr) {
901- os << "\n" << FormatGraphvizLabelText(oldNode->task->Describe());
902- } else {
903- os << "\n<invalid old node>";
904- }
905- return os.str();
906-}
907- 
908std::string MakeV3GraphvizNodeLabel(const V3TaskNode *node, const V3NodeStreamPosMap &streamPosByNode,565std::string MakeV3GraphvizNodeLabel(const V3TaskNode *node, const V3NodeStreamPosMap &streamPosByNode,
909 size_t bfsIndex)566 size_t bfsIndex)
910{567{
@@ -932,47 +589,6 @@ std::string MakeV3GraphvizNodeLabel(const V3TaskNode *node, const V3NodeStreamPo
932 return os.str();589 return os.str();
933}590}
934 591 
935-std::string BuildOldGraphvizDot(const std::vector<TaskNodePtr> &oldBfsNodes)
936-{
937- std::map<TaskNodePtr, size_t> nodeIndexByPtr;
938- for (size_t i = 0; i < oldBfsNodes.size(); ++i) {
939- nodeIndexByPtr[oldBfsNodes[i]] = i;
940- }
941- 
942- std::set<std::pair<size_t, size_t>> edgeIndexes;
943- for (TaskNodePtr oldNode : oldBfsNodes) {
944- const auto parentIter = nodeIndexByPtr.find(oldNode);
945- if (parentIter == nodeIndexByPtr.end() || oldNode == nullptr) {
946- continue;
947- }
948- 
949- for (TaskNodePtr child : oldNode->children) {
950- const auto childIter = nodeIndexByPtr.find(child);
951- if (childIter == nodeIndexByPtr.end()) {
952- continue;
953- }
954- edgeIndexes.insert(std::make_pair(parentIter->second, childIter->second));
955- }
956- }
957- 
958- std::ostringstream os;
959- os << "digraph old_task_graph {\n";
960- os << " graph [rankdir=LR, label=\"old graph BFS DAG\", labelloc=t, fontsize=20];\n";
961- os << " node [shape=box, style=\"rounded,filled\", fillcolor=\"#fff7e6\", fontname=\"Courier\"];\n";
962- os << " edge [fontname=\"Courier\"];\n\n";
963- for (size_t i = 0; i < oldBfsNodes.size(); ++i) {
964- os << " old_" << i << " [label=\"" << EscapeGraphvizLabel(MakeOldGraphvizNodeLabel(oldBfsNodes[i], i))
965- << "\"];\n";
966- }
967- 
968- os << "\n";
969- for (const auto &edge : edgeIndexes) {
970- os << " old_" << edge.first << " -> old_" << edge.second << ";\n";
971- }
972- os << "}\n";
973- return os.str();
974-}
975- 
976std::string BuildV3GraphvizDot(const std::vector<const V3TaskNode *> &newBfsNodes)592std::string BuildV3GraphvizDot(const std::vector<const V3TaskNode *> &newBfsNodes)
977{593{
978 const V3NodeStreamPosMap streamPosByNode = BuildV3StreamPosMap(newBfsNodes);594 const V3NodeStreamPosMap streamPosByNode = BuildV3StreamPosMap(newBfsNodes);
@@ -1018,8 +634,8 @@ std::string BuildV3GraphvizDot(const std::vector<const V3TaskNode *> &newBfsNode
1018 634 
1019void LogGraphvizDot(const std::string &graphName, const std::string &dot)635void LogGraphvizDot(const std::string &graphName, const std::string &dot)
1020{636{
1021- HCCL_VM_INFO("[TaskGraphGeneratorV3][GraphvizDot][{}] GRAPHVIZ_DOT_BEGIN\n{}", graphName, dot);637+ HCCL_VM_INFO("Graphviz DOT dump begin, graphName={}{}", graphName, dot);
1022- HCCL_VM_INFO("[TaskGraphGeneratorV3][GraphvizDot][{}] GRAPHVIZ_DOT_END", graphName);638+ HCCL_VM_INFO("Graphviz DOT dump end, graphName={}", graphName);
1023}639}
1024 640 
1025std::string FormatV3NodeIdList(const std::vector<V3TaskNode *> &nodes)641std::string FormatV3NodeIdList(const std::vector<V3TaskNode *> &nodes)
@@ -1044,19 +660,19 @@ std::string FormatV3NodeIdList(const std::vector<V3TaskNode *> &nodes)
1044std::string MakeAivDagTaskLogLine(const V3TaskNode *node, size_t bfsIndex)660std::string MakeAivDagTaskLogLine(const V3TaskNode *node, size_t bfsIndex)
1045{661{
1046 if (node == nullptr) {662 if (node == nullptr) {
1047- return "<null v3 node>";663+ return "node=null";
1048 }664 }
1049 const auto &loc = node->GetPosition();665 const auto &loc = node->GetPosition();
1050 std::ostringstream os;666 std::ostringstream os;
1051- os << "bfs=" << bfsIndex667+ os << "bfsIndex=" << bfsIndex
1052 << ", nodeId=" << node->GetNodeId()668 << ", nodeId=" << node->GetNodeId()
1053- << ", type=" << TaskTypeKeyName(node->GetType())669+ << ", taskType=" << TaskTypeKeyName(node->GetType())
1054- << ", rank=" << loc.rankId670+ << ", rankId=" << loc.rankId
1055- << ", stream=" << loc.streamId671+ << ", streamId=" << loc.streamId
1056- << ", queue=" << loc.queueId672+ << ", queueId=" << loc.queueId
1057- << ", parents=" << FormatV3NodeIdList(node->GetParents())673+ << ", parentNodeIds=" << FormatV3NodeIdList(node->GetParents())
1058- << ", children=" << FormatV3NodeIdList(node->GetChildren())674+ << ", childNodeIds=" << FormatV3NodeIdList(node->GetChildren())
1059- << ", desc=" << node->Describe();675+ << ", detail=" << node->Describe();
1060 return os.str();676 return os.str();
1061}677}
1062 678 
@@ -1068,10 +684,10 @@ size_t LogAivExpandedDagTasks(const std::vector<const V3TaskNode *> &v3BfsNodes)
1068 if (!IsAivDagLogNode(node)) {684 if (!IsAivDagLogNode(node)) {
1069 continue;685 continue;
1070 }686 }
1071- HCCL_VM_DEBUG("[TaskGraphGeneratorV3][AivDagTask] {}", MakeAivDagTaskLogLine(node, index));687+ HCCL_VM_DEBUG("Expanded AIV DAG node detail, {}", MakeAivDagTaskLogLine(node, index));
1072 ++aivTaskCount;688 ++aivTaskCount;
1073 }689 }
1074- HCCL_VM_DEBUG("[TaskGraphGeneratorV3][AivDagTask] Dump AIV expanded DAG task detail, taskCount={}",690+ HCCL_VM_DEBUG("Finished dumping expanded AIV DAG node details, taskCount={}",
1075 aivTaskCount);691 aivTaskCount);
1076 return aivTaskCount;692 return aivTaskCount;
1077}693}
@@ -1079,12 +695,12 @@ size_t LogAivExpandedDagTasks(const std::vector<const V3TaskNode *> &v3BfsNodes)
1079void LogGraphV3Dag(const V3TaskNode *start)695void LogGraphV3Dag(const V3TaskNode *start)
1080{696{
1081 if (start == nullptr) {697 if (start == nullptr) {
1082- HCCL_VM_WARN("[TaskGraphGeneratorV3][GraphvizDot][v3] Skip DAG dump, start is null.");698+ HCCL_VM_WARN("Skip DAG dump because the graph start node is null");
1083 return;699 return;
1084 }700 }
1085 701 
1086 const auto v3BfsNodes = BfsGraphV3(start);702 const auto v3BfsNodes = BfsGraphV3(start);
1087- HCCL_VM_INFO("[TaskGraphGeneratorV3][GraphvizDot][v3] Dump V3 DAG, nodeCount={}", v3BfsNodes.size());703+ HCCL_VM_INFO("Dumping the V3 DAG as Graphviz DOT text, nodeCount={}", v3BfsNodes.size());
1088 LogGraphvizDot("v3", BuildV3GraphvizDot(v3BfsNodes));704 LogGraphvizDot("v3", BuildV3GraphvizDot(v3BfsNodes));
1089 LogAivExpandedDagTasks(v3BfsNodes);705 LogAivExpandedDagTasks(v3BfsNodes);
1090}706}
@@ -1092,427 +708,26 @@ void LogGraphV3Dag(const V3TaskNode *start)
1092void DumpGraphV3DotToFile(const V3TaskNode *start, const std::string &path)708void DumpGraphV3DotToFile(const V3TaskNode *start, const std::string &path)
1093{709{
1094 if (start == nullptr) {710 if (start == nullptr) {
1095- HCCL_VM_WARN("[TaskGraphGeneratorV3][GraphvizDot][v3] Skip file dump, start is null.");711+ HCCL_VM_WARN("Skip file dump because the graph start node is null, path={}", path);
1096 return;712 return;
1097 }713 }
1098 const auto v3BfsNodes = BfsGraphV3(start);714 const auto v3BfsNodes = BfsGraphV3(start);
1099 std::ofstream out(path, std::ios::out | std::ios::trunc);715 std::ofstream out(path, std::ios::out | std::ios::trunc);
1100 if (!out.is_open()) {716 if (!out.is_open()) {
1101- HCCL_VM_WARN("[TaskGraphGeneratorV3][GraphvizDot][v3] Failed to open graph file: {}", path);717+ HCCL_VM_WARN("Failed to open the graph output file, path={}", path);
1102 return;718 return;
1103 }719 }
1104 out << BuildV3GraphvizDot(v3BfsNodes);720 out << BuildV3GraphvizDot(v3BfsNodes);
1105 out.close();721 out.close();
1106- HCCL_VM_WARN("[TaskGraphGeneratorV3][GraphvizDot][v3] Graph dumped to {}, nodeCount={}",722+ HCCL_VM_INFO("Wrote V3 DAG DOT file, path={}, nodeCount={}",
1107 path, v3BfsNodes.size());723 path, v3BfsNodes.size());
1108}724}
1109 725 
1110-struct SemanticGraphSnapshot {
1111- std::map<std::string, uint32_t> nodeCounts;
1112- std::set<std::string> edgeKeys;
1113- size_t invalidNodeCount{0};
1114- size_t duplicateNodeCount{0};
1115-};
1116- 
1117-void CollectOldVisibleChildren(TaskNodePtr node, std::set<TaskNodePtr> &visibleChildren)
1118-{
1119- if (node == nullptr) {
1120- return;
1121- }
1122- 
1123- std::vector<TaskNodePtr> nodeStack(node->children.begin(), node->children.end());
1124- std::set<TaskNodePtr> visited;
1125- while (!nodeStack.empty()) {
1126- TaskNodePtr currNode = nodeStack.back();
1127- nodeStack.pop_back();
1128- if (currNode == nullptr || visited.count(currNode) != 0) {
1129- continue;
1130- }
1131- visited.insert(currNode);
1132- if (!IsOldGraphCompareTransparent(currNode)) {
1133- visibleChildren.insert(currNode);
1134- continue;
1135- }
1136- for (TaskNodePtr childNode : currNode->children) {
1137- nodeStack.push_back(childNode);
1138- }
1139- }
1140-}
1141- 
1142-void CollectV3VisibleChildren(const V3TaskNode *node, std::set<const V3TaskNode *> &visibleChildren)
1143-{
1144- if (node == nullptr) {
1145- return;
1146- }
1147- 
1148- std::vector<const V3TaskNode *> nodeStack(node->GetChildren().begin(), node->GetChildren().end());
1149- std::set<const V3TaskNode *> visited;
1150- while (!nodeStack.empty()) {
1151- const V3TaskNode *currNode = nodeStack.back();
1152- nodeStack.pop_back();
1153- if (currNode == nullptr || visited.count(currNode) != 0) {
1154- continue;
1155- }
1156- visited.insert(currNode);
1157- if (!IsV3GraphCompareTransparent(currNode)) {
1158- visibleChildren.insert(currNode);
1159- continue;
1160- }
1161- for (const V3TaskNode *childNode : currNode->GetChildren()) {
1162- nodeStack.push_back(childNode);
1163- }
1164- }
1165-}
1166- 
1167-void AddNodeKey(SemanticGraphSnapshot &snapshot, const std::string &nodeKey)
1168-{
1169- uint32_t &count = snapshot.nodeCounts[nodeKey];
1170- ++count;
1171- if (count > 1) {
1172- ++snapshot.duplicateNodeCount;
1173- }
1174-}
1175- 
1176-SemanticGraphSnapshot BuildOldSemanticGraph(const std::vector<TaskNodePtr> &oldBfsNodes)
1177-{
1178- SemanticGraphSnapshot snapshot;
1179- const OldNodeStreamPosMap streamPosByNode = BuildOldStreamPosMap(oldBfsNodes);
1180- std::map<TaskNodePtr, std::string> nodeKeyByPtr;
1181- 
1182- for (TaskNodePtr oldNode : oldBfsNodes) {
1183- if (IsOldGraphCompareTransparent(oldNode)) {
1184- continue;
1185- }
1186- std::string nodeKey;
1187- if (!MakeOldNodeSemanticKey(oldNode, &streamPosByNode, nodeKey)) {
1188- ++snapshot.invalidNodeCount;
1189- continue;
1190- }
1191- AddNodeKey(snapshot, nodeKey);
1192- nodeKeyByPtr[oldNode] = nodeKey;
1193- }
1194- 
1195- for (TaskNodePtr oldNode : oldBfsNodes) {
1196- const auto parentIter = nodeKeyByPtr.find(oldNode);
1197- if (parentIter == nodeKeyByPtr.end()) {
1198- continue;
1199- }
1200- 
1201- std::set<TaskNodePtr> visibleChildren;
1202- CollectOldVisibleChildren(oldNode, visibleChildren);
1203- for (TaskNodePtr child : visibleChildren) {
1204- const auto childIter = nodeKeyByPtr.find(child);
1205- if (childIter == nodeKeyByPtr.end()) {
1206- continue;
1207- }
1208- snapshot.edgeKeys.insert(parentIter->second + " -> " + childIter->second);
1209- }
1210- }
1211- 
1212- return snapshot;
1213-}
1214- 
1215-SemanticGraphSnapshot BuildV3SemanticGraph(const std::vector<const V3TaskNode *> &newBfsNodes)
1216-{
1217- SemanticGraphSnapshot snapshot;
1218- const V3NodeStreamPosMap streamPosByNode = BuildV3StreamPosMap(newBfsNodes);
1219- std::map<const V3TaskNode *, std::string> nodeKeyByPtr;
1220- 
1221- for (const V3TaskNode *node : newBfsNodes) {
1222- if (IsV3GraphCompareTransparent(node)) {
1223- continue;
1224- }
1225- std::string nodeKey;
1226- if (!MakeV3NodeSemanticKey(node, streamPosByNode, nodeKey)) {
1227- ++snapshot.invalidNodeCount;
1228- continue;
1229- }
1230- AddNodeKey(snapshot, nodeKey);
1231- nodeKeyByPtr[node] = nodeKey;
1232- }
1233- 
1234- for (const V3TaskNode *node : newBfsNodes) {
1235- const auto parentIter = nodeKeyByPtr.find(node);
1236- if (parentIter == nodeKeyByPtr.end() || node == nullptr) {
1237- continue;
1238- }
1239- 
1240- std::set<const V3TaskNode *> visibleChildren;
1241- CollectV3VisibleChildren(node, visibleChildren);
1242- for (const V3TaskNode *childNode : visibleChildren) {
1243- const auto childIter = nodeKeyByPtr.find(childNode);
1244- if (childIter == nodeKeyByPtr.end()) {
1245- continue;
1246- }
1247- snapshot.edgeKeys.insert(parentIter->second + " -> " + childIter->second);
1248- }
1249- }
1250- 
1251- return snapshot;
1252-}
1253- 
1254-size_t LogBfsOrderDiff(const std::vector<TaskNodePtr> &oldBfsNodes,
1255- const std::vector<const V3TaskNode *> &newBfsNodes)
1256-{
1257- size_t mismatchCount = 0;
1258- size_t detailCount = 0;
1259- const OldNodeStreamPosMap oldStreamPosByNode = BuildOldStreamPosMap(oldBfsNodes);
1260- const V3NodeStreamPosMap streamPosByNode = BuildV3StreamPosMap(newBfsNodes);
1261- const size_t commonCount = std::min(oldBfsNodes.size(), newBfsNodes.size());
1262- 
1263- for (size_t i = 0; i < commonCount; ++i) {
1264- std::string oldKey;
1265- if (!MakeOldNodeSemanticKey(oldBfsNodes[i], &oldStreamPosByNode, oldKey)) {
1266- oldKey = "<invalid old bfs node>";
1267- }
1268- 
1269- std::string newKey;
1270- if (!MakeV3NodeSemanticKey(newBfsNodes[i], streamPosByNode, newKey)) {
1271- newKey = "<invalid v3 bfs node>";
1272- }
1273- 
1274- if (oldKey == newKey) {
1275- continue;
1276- }
1277- 
1278- ++mismatchCount;
1279- if (detailCount < MAX_GRAPH_COMPARE_DETAIL_LOG_COUNT) {
1280- HCCL_VM_WARN("[TaskGraphGeneratorV3] Graph BFS order mismatch, bfsIndex={}, oldNode={}, v3Node={}",
1281- i, oldKey, newKey);
1282- ++detailCount;
1283- }
1284- }
1285- 
1286- for (size_t i = commonCount; i < oldBfsNodes.size(); ++i) {
1287- ++mismatchCount;
1288- if (detailCount < MAX_GRAPH_COMPARE_DETAIL_LOG_COUNT) {
1289- std::string oldKey;
1290- if (!MakeOldNodeSemanticKey(oldBfsNodes[i], &oldStreamPosByNode, oldKey)) {
1291- oldKey = "<invalid old bfs node>";
1292- }
1293- HCCL_VM_WARN("[TaskGraphGeneratorV3] Graph BFS order mismatch, bfsIndex={}, oldNode={}, v3Node={}",
1294- i, oldKey, "<missing v3 bfs node>");
1295- ++detailCount;
1296- }
1297- }
1298- 
1299- for (size_t i = commonCount; i < newBfsNodes.size(); ++i) {
1300- ++mismatchCount;
1301- if (detailCount < MAX_GRAPH_COMPARE_DETAIL_LOG_COUNT) {
1302- std::string newKey;
1303- if (!MakeV3NodeSemanticKey(newBfsNodes[i], streamPosByNode, newKey)) {
1304- newKey = "<invalid v3 bfs node>";
1305- }
1306- HCCL_VM_WARN("[TaskGraphGeneratorV3] Graph BFS order mismatch, bfsIndex={}, oldNode={}, v3Node={}",
1307- i, "<missing old bfs node>", newKey);
1308- ++detailCount;
1309- }
1310- }
1311- 
1312- if (detailCount >= MAX_GRAPH_COMPARE_DETAIL_LOG_COUNT) {
1313- HCCL_VM_WARN("[TaskGraphGeneratorV3] Too many BFS order mismatches, stop detail logging.");
1314- }
1315- 
1316- if (mismatchCount == 0) {
1317- HCCL_VM_INFO("[TaskGraphGeneratorV3] Graph BFS order matched, count={}", oldBfsNodes.size());
1318- } else {
1319- HCCL_VM_WARN("[TaskGraphGeneratorV3] Graph BFS order mismatch, mismatchCount={}, oldCount={}, v3Count={}",
1320- mismatchCount, oldBfsNodes.size(), newBfsNodes.size());
1321- }
1322- return mismatchCount;
1323-}
1324- 
1325-size_t LogNodeCountDiff(const SemanticGraphSnapshot &oldSnapshot, const SemanticGraphSnapshot &newSnapshot)
1326-{
1327- size_t mismatchCount = 0;
1328- size_t detailCount = 0;
1329- 
1330- for (const auto &oldEntry : oldSnapshot.nodeCounts) {
1331- const auto newIter = newSnapshot.nodeCounts.find(oldEntry.first);
1332- const uint32_t newCount = (newIter == newSnapshot.nodeCounts.end()) ? 0 : newIter->second;
1333- if (newCount >= oldEntry.second) {
1334- continue;
1335- }
1336- 
1337- const uint32_t missingCount = oldEntry.second - newCount;
1338- mismatchCount += missingCount;
1339- if (detailCount < MAX_GRAPH_COMPARE_DETAIL_LOG_COUNT) {
1340- HCCL_VM_WARN("[TaskGraphGeneratorV3] V3 graph missing semantic node, count={}, key={}",
1341- missingCount, oldEntry.first);
1342- ++detailCount;
1343- }
1344- }
1345- 
1346- for (const auto &newEntry : newSnapshot.nodeCounts) {
1347- const auto oldIter = oldSnapshot.nodeCounts.find(newEntry.first);
1348- const uint32_t oldCount = (oldIter == oldSnapshot.nodeCounts.end()) ? 0 : oldIter->second;
1349- if (oldCount >= newEntry.second) {
1350- continue;
1351- }
1352- 
1353- const uint32_t extraCount = newEntry.second - oldCount;
1354- mismatchCount += extraCount;
1355- if (detailCount < MAX_GRAPH_COMPARE_DETAIL_LOG_COUNT) {
1356- HCCL_VM_WARN("[TaskGraphGeneratorV3] V3 graph extra semantic node, count={}, key={}",
1357- extraCount, newEntry.first);
1358- ++detailCount;
1359- }
1360- }
1361- 
1362- if (detailCount >= MAX_GRAPH_COMPARE_DETAIL_LOG_COUNT) {
1363- HCCL_VM_WARN("[TaskGraphGeneratorV3] Too many semantic node mismatches, stop detail logging.");
1364- }
1365- return mismatchCount;
1366-}
1367- 
1368-bool IsCcuGraphSemanticKey(const std::string &key)
1369-{
1370- return key.find("type=CCU_GRAPH") != std::string::npos;
1371-}
1372- 
1373-bool HasCcuGraphNode(const SemanticGraphSnapshot &snapshot)
1374-{
1375- for (const auto &nodeEntry : snapshot.nodeCounts) {
1376- if (IsCcuGraphSemanticKey(nodeEntry.first)) {
1377- return true;
1378- }
1379- }
1380- return false;
1381-}
1382- 
1383-bool IsCcuBoundaryEdge(const std::string &edgeKey)
1384-{
1385- return IsCcuGraphSemanticKey(edgeKey);
1386-}
1387- 
1388-size_t LogEdgeDiff(const std::set<std::string> &oldEdgeKeys, const std::set<std::string> &newEdgeKeys,
1389- bool ignoreCcuBoundaryEdges = false)
1390-{
1391- size_t mismatchCount = 0;
1392- size_t detailCount = 0;
1393- 
1394- for (const auto &edgeKey : oldEdgeKeys) {
1395- if (ignoreCcuBoundaryEdges && IsCcuBoundaryEdge(edgeKey)) {
1396- continue;
1397- }
1398- if (newEdgeKeys.count(edgeKey) != 0) {
1399- continue;
1400- }
1401- 
1402- ++mismatchCount;
1403- if (detailCount < MAX_GRAPH_COMPARE_DETAIL_LOG_COUNT) {
1404- HCCL_VM_WARN("[TaskGraphGeneratorV3] V3 graph missing edge, edge={}", edgeKey);
1405- ++detailCount;
1406- }
1407- }
1408- 
1409- for (const auto &edgeKey : newEdgeKeys) {
1410- if (ignoreCcuBoundaryEdges && IsCcuBoundaryEdge(edgeKey)) {
1411- continue;
1412- }
1413- if (oldEdgeKeys.count(edgeKey) != 0) {
1414- continue;
1415- }
1416- 
1417- ++mismatchCount;
1418- if (detailCount < MAX_GRAPH_COMPARE_DETAIL_LOG_COUNT) {
1419- HCCL_VM_WARN("[TaskGraphGeneratorV3] V3 graph extra edge, edge={}", edgeKey);
1420- ++detailCount;
1421- }
1422- }
1423- 
1424- if (detailCount >= MAX_GRAPH_COMPARE_DETAIL_LOG_COUNT) {
1425- HCCL_VM_WARN("[TaskGraphGeneratorV3] Too many semantic edge mismatches, stop detail logging.");
1426- }
1427- return mismatchCount;
1428-}
1429- 
1430-void CompareGraphV3BfsOrder(const std::vector<TaskNodePtr> &oldBfsNodes,
1431- const std::vector<const V3TaskNode *> &newBfsNodes)
1432-{
1433- if (oldBfsNodes.size() != newBfsNodes.size()) {
1434- HCCL_VM_WARN("[TaskGraphGeneratorV3] Graph BFS node count mismatch, old={}, v3={}",
1435- oldBfsNodes.size(), newBfsNodes.size());
1436- } else {
1437- HCCL_VM_INFO("[TaskGraphGeneratorV3] Graph BFS node count matched, count={}", oldBfsNodes.size());
1438- }
1439- 
1440- const size_t bfsOrderMismatchCount = LogBfsOrderDiff(oldBfsNodes, newBfsNodes);
1441- if (bfsOrderMismatchCount == 0 && oldBfsNodes.size() == newBfsNodes.size()) {
1442- HCCL_VM_INFO("[TaskGraphGeneratorV3] Graph BFS compare matched, orderMatched=true, nodeCount={}",
1443- oldBfsNodes.size());
1444- } else {
1445- HCCL_VM_WARN("[TaskGraphGeneratorV3] Graph BFS compare mismatch, orderMismatch={}, oldNodeCount={}, "
1446- "v3NodeCount={}", bfsOrderMismatchCount, oldBfsNodes.size(), newBfsNodes.size());
1447- }
1448-}
1449- 
1450-void CompareGraphV3Semantic(const std::vector<TaskNodePtr> &oldBfsNodes,
1451- const std::vector<const V3TaskNode *> &newBfsNodes)
1452-{
1453- const SemanticGraphSnapshot oldSnapshot = BuildOldSemanticGraph(oldBfsNodes);
1454- const SemanticGraphSnapshot newSnapshot = BuildV3SemanticGraph(newBfsNodes);
1455- const bool hasCcuGraph = HasCcuGraphNode(oldSnapshot) || HasCcuGraphNode(newSnapshot);
1456- const size_t nodeMismatchCount = LogNodeCountDiff(oldSnapshot, newSnapshot);
1457- const size_t edgeMismatchCount = LogEdgeDiff(oldSnapshot.edgeKeys, newSnapshot.edgeKeys, hasCcuGraph);
1458- if (hasCcuGraph) {
1459- HCCL_VM_INFO("[TaskGraphGeneratorV3] Graph semantic compare uses CCU-compatible view: "
1460- "V3 CCU expanded nodes and old CCU graph separators are ignored; CCU boundary edges are skipped.");
1461- }
1462- 
1463- if (oldSnapshot.invalidNodeCount != 0 || newSnapshot.invalidNodeCount != 0 ||
1464- oldSnapshot.duplicateNodeCount != 0 || newSnapshot.duplicateNodeCount != 0) {
1465- HCCL_VM_WARN("[TaskGraphGeneratorV3] Graph semantic compare snapshot abnormal, oldInvalid={}, "
1466- "v3Invalid={}, oldDuplicate={}, v3Duplicate={}", oldSnapshot.invalidNodeCount,
1467- newSnapshot.invalidNodeCount, oldSnapshot.duplicateNodeCount, newSnapshot.duplicateNodeCount);
1468- }
1469- 
1470- if (nodeMismatchCount == 0 && edgeMismatchCount == 0 && oldSnapshot.invalidNodeCount == 0 &&
1471- newSnapshot.invalidNodeCount == 0) {
1472- HCCL_VM_INFO("[TaskGraphGeneratorV3] Graph semantic compare matched, nodeKinds={}, edgeCount={}",
1473- oldSnapshot.nodeCounts.size(), oldSnapshot.edgeKeys.size());
1474- } else {
1475- HCCL_VM_WARN("[TaskGraphGeneratorV3] Graph semantic compare mismatch, nodeMismatch={}, edgeMismatch={}, "
1476- "oldInvalid={}, v3Invalid={}, oldNodeKinds={}, v3NodeKinds={}, oldEdges={}, v3Edges={}",
1477- nodeMismatchCount, edgeMismatchCount, oldSnapshot.invalidNodeCount, newSnapshot.invalidNodeCount,
1478- oldSnapshot.nodeCounts.size(), newSnapshot.nodeCounts.size(), oldSnapshot.edgeKeys.size(),
1479- newSnapshot.edgeKeys.size());
1480- }
1481-}
1482- 
1483-void CompareGraphV3WithOldGraph(TaskNodePtr oldStart, const V3TaskNode *newStart)
1484-{
1485- if (oldStart == nullptr || newStart == nullptr) {
1486- HCCL_VM_WARN("[TaskGraphGeneratorV3] Skip graph compare, oldStart or newStart is null.");
1487- return;
1488- }
1489- 
1490- const auto oldBfsNodes = BfsOldGraph(oldStart);
1491- const auto newBfsNodes = BfsGraphV3(newStart);
1492- const auto oldCompareNodes = BuildOldGraphCompareNodes(oldBfsNodes);
1493- const auto newCompareNodes = BuildV3GraphCompareNodes(newBfsNodes);
1494- // compare 统一使用 BFS 收集节点,随后再过滤掉透明节点(如队列占位节点、CCU 边界节点),
1495- // 这样既能保持视图稳定,也能让 old/v3 两侧在同一语义层面上比较。
1496- const bool useTransparentCompareView = oldCompareNodes.size() != oldBfsNodes.size() ||
1497- newCompareNodes.size() != newBfsNodes.size();
1498- if (useTransparentCompareView) {
1499- HCCL_VM_INFO("[TaskGraphGeneratorV3] Graph compare view filtered nodes, oldRaw={}, oldVisible={}, "
1500- "v3Raw={}, v3Visible={}", oldBfsNodes.size(), oldCompareNodes.size(), newBfsNodes.size(),
1501- newCompareNodes.size());
1502- }
1503- if (useTransparentCompareView) {
1504- HCCL_VM_INFO("[TaskGraphGeneratorV3] Skip BFS order compare in transparent graph compare view.");
1505- } else {
1506- CompareGraphV3BfsOrder(oldCompareNodes, newCompareNodes);
1507- }
1508- CompareGraphV3Semantic(oldCompareNodes, newCompareNodes);
1509-}
1510- 
1511HcclResult CheckGraphV3TaskMem(const V3TaskNode *start)726HcclResult CheckGraphV3TaskMem(const V3TaskNode *start)
1512{727{
1513- V3StageTimer timer("CheckGraphV3TaskMem");728+ V3StageTimer timer("SingleTaskCheck");
1514 if (start == nullptr) {729 if (start == nullptr) {
1515- HCCL_VM_WARN("[TaskGraphSingleTaskCheckV3] Skip V3 task memory check, start is null.");730+ HCCL_VM_WARN("Skip single-task memory check because the graph start node is null");
1516 timer.SetStatus("skipped");731 timer.SetStatus("skipped");
1517 return HCCL_SUCCESS;732 return HCCL_SUCCESS;
1518 }733 }
@@ -1535,9 +750,9 @@ HcclResult CheckGraphV3TaskMem(const V3TaskNode *start)
1535 750 
1536HcclResult CheckGraphV3MemConflict(const V3TaskNode *start)751HcclResult CheckGraphV3MemConflict(const V3TaskNode *start)
1537{752{
1538- V3StageTimer timer("CheckGraphV3MemConflict");753+ V3StageTimer timer("MemConflict");
1539 if (start == nullptr) {754 if (start == nullptr) {
1540- HCCL_VM_WARN("[TaskGraphMemConflictV3] Skip V3 memory conflict check, start is null.");755+ HCCL_VM_WARN("Skip memory-conflict check because the graph start node is null");
1541 timer.SetStatus("skipped");756 timer.SetStatus("skipped");
1542 return HCCL_SUCCESS;757 return HCCL_SUCCESS;
1543 }758 }
@@ -1561,9 +776,9 @@ HcclResult CheckGraphV3MemConflict(const V3TaskNode *start)
1561 776 
1562HcclResult CheckGraphV3Semantic(const V3TaskNode *start)777HcclResult CheckGraphV3Semantic(const V3TaskNode *start)
1563{778{
1564- V3StageTimer timer("CheckGraphV3Semantic");779+ V3StageTimer timer("SemanticCheck");
1565 if (start == nullptr) {780 if (start == nullptr) {
1566- HCCL_VM_WARN("[TaskGraphSemanticCheckV3] Skip V3 semantic check, start is null.");781+ HCCL_VM_WARN("Skip semantic check because the graph start node is null");
1567 timer.SetStatus("skipped");782 timer.SetStatus("skipped");
1568 return HCCL_SUCCESS;783 return HCCL_SUCCESS;
1569 }784 }
@@ -1613,7 +828,8 @@ HcclResult GenAndCheckGraphV3()
1613 }828 }
1614 const HcclResult dumpRet = DumpV3Dag(*graphGeneratorV3);829 const HcclResult dumpRet = DumpV3Dag(*graphGeneratorV3);
1615 if (dumpRet != HCCL_SUCCESS) {830 if (dumpRet != HCCL_SUCCESS) {
1616- HCCL_VM_WARN("[GenAndCheckGraphV3] failed to dump V3 dag, ret={}", static_cast<uint32_t>(dumpRet));831+ HCCL_VM_WARN("Failed to dump the V3 DAG, ret={}",
832+ static_cast<uint32_t>(dumpRet));
1617 }833 }
1618 const V3TaskNode *dummyStartV3 = graphGeneratorV3->GetMainStartNode();834 const V3TaskNode *dummyStartV3 = graphGeneratorV3->GetMainStartNode();
1619 // 打印dag task任务835 // 打印dag task任务
@@ -1654,7 +870,8 @@ HcclResult Checker::GenAndCheckGraph(AllRankTaskQueues &allRankTaskQueues, TaskC
1654 HCCL_VM_INFO("-------------------------------------------------------");870 HCCL_VM_INFO("-------------------------------------------------------");
1655 for (auto& task : taskQueue[i]) {871 for (auto& task : taskQueue[i]) {
1656 std::string tempStr = task->Describe();872 std::string tempStr = task->Describe();
1657- HCCL_VM_INFO("rankIdx:{:d}, taskIdx:{:d}, {:s}, pointer: {:p}", rankIdx, taskIdx, tempStr, static_cast<void*>(task.get()));873+ HCCL_VM_INFO("rankIdx:{:d}, taskIdx:{:d}, {:s}, pointer: {:p}", rankIdx, taskIdx, tempStr,
874+ static_cast<void*>(task.get()));
1658 taskIdx++;875 taskIdx++;
1659 }876 }
1660 }877 }
@@ -1662,18 +879,18 @@ HcclResult Checker::GenAndCheckGraph(AllRankTaskQueues &allRankTaskQueues, TaskC
1662 {879 {
1663 const HcclResult dumpRet = DumpInputTaskQueues(allRankTaskQueues);880 const HcclResult dumpRet = DumpInputTaskQueues(allRankTaskQueues);
1664 if (dumpRet != HcclResult::HCCL_SUCCESS) {881 if (dumpRet != HcclResult::HCCL_SUCCESS) {
1665- HCCL_VM_WARN("[Checker::GenAndCheckGraph] failed to dump input task queues, ret[%u].",882+ HCCL_VM_WARN("Failed to dump the old checker input task queues, ret={}",
1666 static_cast<u32>(dumpRet));883 static_cast<u32>(dumpRet));
1667 }884 }
1668 }885 }
1669 886 
1670 // 1. 检查从流887 // 1. 检查从流
1671- HCCL_INFO("1. 检查从流");888+ HCCL_VM_INFO("1. 检查从流");
1672 SingleTaskCheck taskChecker;889 SingleTaskCheck taskChecker;
1673 CHK_RET(taskChecker.CheckSlaveTaskQueue(allRankTaskQueues));890 CHK_RET(taskChecker.CheckSlaveTaskQueue(allRankTaskQueues));
1674 891 
1675 // 2. 成图892 // 2. 成图
1676- HCCL_INFO("2. 成图");893+ HCCL_VM_INFO("2. 成图");
1677 TaskNode dummyStart = TaskNode(nullptr, -1, 0, 0);894 TaskNode dummyStart = TaskNode(nullptr, -1, 0, 0);
1678 TaskNode dummyStartCopy = TaskNode(nullptr, -1, 0, 0);895 TaskNode dummyStartCopy = TaskNode(nullptr, -1, 0, 0);
1679 TaskGraphGenerator graphGenerator;896 TaskGraphGenerator graphGenerator;
@@ -1681,13 +898,13 @@ HcclResult Checker::GenAndCheckGraph(AllRankTaskQueues &allRankTaskQueues, TaskC
1681 898 
1682 // 3. Task内存校验899 // 3. Task内存校验
1683 // 是否可以复用 taskChecker900 // 是否可以复用 taskChecker
1684- HCCL_INFO("3. Task内存校验");901+ HCCL_VM_INFO("3. Task内存校验");
1685 CHK_RET(taskChecker.CheckTaskMem(&dummyStart));902 CHK_RET(taskChecker.CheckTaskMem(&dummyStart));
1686- HCCL_INFO("3. Task内存校验 END: %d", closeRankMemCheck_);903+ HCCL_VM_INFO("3. Task内存校验 END: {}", closeRankMemCheck_);
1687 904 
1688 if (!closeRankMemCheck_) {905 if (!closeRankMemCheck_) {
1689 // 4. 图复制906 // 4. 图复制
1690- HCCL_INFO("4. 图复制");907+ HCCL_VM_INFO("4. 图复制");
1691 if (dummyStart.hasCcuTask) {908 if (dummyStart.hasCcuTask) {
1692 CHK_RET(CopyCcuTaskGraph(&dummyStart, &dummyStartCopy, rankNum));909 CHK_RET(CopyCcuTaskGraph(&dummyStart, &dummyStartCopy, rankNum));
1693 dummyStartCopy.hasCcuTask = true;910 dummyStartCopy.hasCcuTask = true;
@@ -1696,7 +913,7 @@ HcclResult Checker::GenAndCheckGraph(AllRankTaskQueues &allRankTaskQueues, TaskC
1696 }913 }
1697 914
1698 // 5. 图改造915 // 5. 图改造
1699- HCCL_INFO("5. 图改造");916+ HCCL_VM_INFO("5. 图改造");
1700 GraphRevampBilateralSemantics graphRevamp;917 GraphRevampBilateralSemantics graphRevamp;
1701 918 
1702 // ccu图改造919 // ccu图改造
@@ -1720,17 +937,17 @@ HcclResult Checker::GenAndCheckGraph(AllRankTaskQueues &allRankTaskQueues, TaskC
1720 CHK_RET(graphRevamp.Revamp(&dummyStartCopy));937 CHK_RET(graphRevamp.Revamp(&dummyStartCopy));
1721 938 
1722 // 6. Rank内存校验939 // 6. Rank内存校验
1723- HCCL_INFO("6. Rank内存校验");940+ HCCL_VM_INFO("6. Rank内存校验");
1724 CheckRankMem checkRankmem(&dummyStartCopy);941 CheckRankMem checkRankmem(&dummyStartCopy);
1725 CHK_RET(checkRankmem.Execute());942 CHK_RET(checkRankmem.Execute());
1726 }943 }
1727 944 
1728 // 7. 语义校验945 // 7. 语义校验
1729- HCCL_INFO("7. 语义校验");946+ HCCL_VM_INFO("7. 语义校验");
1730 // PrintCcuGraph(&dummyStart);947 // PrintCcuGraph(&dummyStart);
1731 opSemanticsChecker.SetGraphHead(&dummyStart);948 opSemanticsChecker.SetGraphHead(&dummyStart);
1732 CHK_RET(opSemanticsChecker.Execute());949 CHK_RET(opSemanticsChecker.Execute());
1733- HCCL_INFO("8. 语义校验 END");950+ HCCL_VM_INFO("8. 语义校验 END");
1734 951 
1735 // 成图及校验成功952 // 成图及校验成功
1736 return HCCL_SUCCESS;953 return HCCL_SUCCESS;
Mtest/hccl_vm/src/plugin/checker/src/dfx/ccu_error_handler.cc+1-1
@@ -73,7 +73,7 @@ void CcuErrorHandler::GetCcuErrorMsg(int32_t deviceId, const ParaCcu &ccuTaskPar
73 const uint16_t missionStatus = missionContext.GetStatus();73 const uint16_t missionStatus = missionContext.GetStatus();
74 if (missionStatus == 0) {74 if (missionStatus == 0) {
75 HCCL_INFO("[CcuErrorHandler] no err found, mission status is 0, deviceId[%d], dieId[%u], execMissionId[%u]",75 HCCL_INFO("[CcuErrorHandler] no err found, mission status is 0, deviceId[%d], dieId[%u], execMissionId[%u]",
76- __func__, deviceId, static_cast<u32>(ccuTaskParam.dieId), static_cast<u32>(ccuTaskParam.execMissionId));76+ deviceId, static_cast<u32>(ccuTaskParam.dieId), static_cast<u32>(ccuTaskParam.execMissionId));
77 return;77 return;
78 }78 }
79 79 
Mtest/hccl_vm/src/plugin/checker/src/framework/mem_conflict_check/check_rank_mem.cc+4-4
@@ -495,7 +495,7 @@ HcclResult CheckRankMem::CompareSliceMemoryInfo(FragQueueMemStatus &left, FragQu
495 if (right.count(type) != 0) {495 if (right.count(type) != 0) {
496 auto ret = CompareBufferTypeMemoryInfo(iter->second, right[type]);496 auto ret = CompareBufferTypeMemoryInfo(iter->second, right[type]);
497 if (ret != HcclResult::HCCL_SUCCESS) {497 if (ret != HcclResult::HCCL_SUCCESS) {
498- HCCL_ERROR("failed to check memory %", type);498+ HCCL_ERROR("failed to check memory type[%d]", static_cast<int>(type));
499 return ret;499 return ret;
500 }500 }
501 }501 }
@@ -901,7 +901,7 @@ HcclResult CheckRankMem::CcuGraphMemCheck()
901 }901 }
902 numCcuGraph++;902 numCcuGraph++;
903 }903 }
904- HCCL_VM_INFO("[CcuGraphMemCheck] %u CcuGraphMemCheck proc success. ccu graph num: %d", rankId, numCcuGraph);904+ HCCL_VM_INFO("{} CcuGraphMemCheck proc success. ccu graph num: {}", rankId, numCcuGraph);
905 }905 }
906 ClearCcuData();906 ClearCcuData();
907 907 
@@ -1034,7 +1034,7 @@ void CheckRankMem::GenFragQueueInOneQueueOnly(TaskNode *head)
1034 auto type = child->task->GetType();1034 auto type = child->task->GetType();
1035 if (type == TaskTypeStub::LOCAL_WAIT_FROM ||1035 if (type == TaskTypeStub::LOCAL_WAIT_FROM ||
1036 type == TaskTypeStub::LOCAL_WAIT_FROM_SHADOW) {1036 type == TaskTypeStub::LOCAL_WAIT_FROM_SHADOW) {
1037- HCCL_VM_DEBUG("[GenFragQueueInOneQueueOnly] ccu multi samequeue node found, node: {} "1037+ HCCL_VM_DEBUG("ccu multi samequeue node found, node: {} "
1038 "ignore child: {} addr: {}",1038 "ignore child: {} addr: {}",
1039 curNode->task->Describe(),1039 curNode->task->Describe(),
1040 child->task->Describe(),1040 child->task->Describe(),
@@ -1126,7 +1126,7 @@ HcclResult CheckRankMem::CcuGraphMemCheckProc(RankId rankId, uint32_t queueId, T
1126 CHK_RET(GenFragQueConcurrencyMatrixAndCompare(rankId));1126 CHK_RET(GenFragQueConcurrencyMatrixAndCompare(rankId));
1127 ClearCcuData();1127 ClearCcuData();
1128 1128 
1129- HCCL_VM_DEBUG("[CcuGraphMemCheckProc] ccuHeadChildNum= {}", ccuHeadChildNum);1129+ HCCL_VM_DEBUG("ccuHeadChildNum= {}", ccuHeadChildNum);
1130 return HcclResult::HCCL_SUCCESS;1130 return HcclResult::HCCL_SUCCESS;
1131}1131}
1132 1132 
Mtest/hccl_vm/src/plugin/checker/src/framework/semantics_check/all2all_semantics_checker.cc+46-19
@@ -17,7 +17,9 @@
17#include <vector>17#include <vector>
18 18 
19#include "checker_def.h"19#include "checker_def.h"
20-#include "log.h"20+#include "sim_log.h"
21+#include "utils/dump/dump_json_utils.h"
22+#include "utils/error_codes.h"
21 23 
22namespace HcclSim {24namespace HcclSim {
23void GenSendOffsetForOneRank(All2AllDataDesTagInner &all2AllDataDes, u32 rankSize, RankId targetRank, std::vector<u64> &sendOffsets)25void GenSendOffsetForOneRank(All2AllDataDesTagInner &all2AllDataDes, u32 rankSize, RankId targetRank, std::vector<u64> &sendOffsets)
@@ -42,7 +44,9 @@ HcclResult TaskCheckAll2AllSemantics(std::map<RankId, RankMemorySemantics> &allR
42 for (RankId rankId = 0; rankId < rankSize; rankId++) {44 for (RankId rankId = 0; rankId < rankSize; rankId++) {
43 // 对应的rank不存在需要报错45 // 对应的rank不存在需要报错
44 if (allRankMemSemantics.count(rankId) == 0) {46 if (allRankMemSemantics.count(rankId) == 0) {
45- HCCL_ERROR("Missing rank %d mem semantics", rankId);47+ HCCL_VM_ERROR("{} AllToAll produced no result data for rank {}, but this rank is "
48+ "expected to receive data from peer ranks, expectedSourceRankCount={}",
49+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_MISSING), rankId, rankSize);
46 return HcclResult::HCCL_E_PARA;50 return HcclResult::HCCL_E_PARA;
47 }51 }
48 52 
@@ -59,39 +63,59 @@ HcclResult TaskCheckAll2AllSemantics(std::map<RankId, RankMemorySemantics> &allR
59 }63 }
60 64
61 for (auto &ele : allRankMemSemantics[rankId][BufferType::OUTPUT]) {65 for (auto &ele : allRankMemSemantics[rankId][BufferType::OUTPUT]) {
66+ const u64 rangeEnd = ele.startAddr + ele.size;
62 if (ele.startAddr != totalSize) {67 if (ele.startAddr != totalSize) {
63- HCCL_ERROR("[rankId:%u]Missing buffer semantic: "68+ HCCL_VM_ERROR("{} AllToAll result data does not start from the expected address, "
64- "exepected startAddr is %llu, while cur buffer semantic startAddr is %llu, cur buffer semantic is %s",69+ "rankId={}, expectedStartAddr=0x{:x}, actualStartAddr=0x{:x}, actualBufferRange=[0x{:x},0x{:x})"
65- rankId, totalSize, ele.startAddr, ele.Describe().c_str());70+ "\nCurrent result range detail:\n{}",
71+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_MISSING), rankId, totalSize, ele.startAddr,
72+ ele.startAddr, rangeEnd, ele.Describe());
66 return HcclResult::HCCL_E_PARA;73 return HcclResult::HCCL_E_PARA;
67 }74 }
68 75 
69 if (ele.srcBufs.size() != 1) {76 if (ele.srcBufs.size() != 1) {
70- HCCL_ERROR("[rankId:%u]Cur buffer semantic should not be reduce, which mean srcBufs size should be 1, "77+ HCCL_VM_ERROR("{} This AllToAll result range combines multiple sources, but this "
71- "while cur buffer semantic is %s", rankId, ele.Describe().c_str());78+ "operator expects exactly one source, rankId={}, actualSourceCount={}, expectedSourceCount=1, "
79+ "outputRange=[0x{:x},0x{:x})\nCurrent result range detail:\n{}",
80+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_REDUCE_ERROR), rankId, ele.srcBufs.size(),
81+ ele.startAddr, rangeEnd, ele.Describe());
72 return HcclResult::HCCL_E_PARA;82 return HcclResult::HCCL_E_PARA;
73 }83 }
74 84 
75 if (curRankId >= rankSize) {85 if (curRankId >= rankSize) {
76- HCCL_ERROR("[rankId:%u]Extra buffer semantic found", rankId);86+ HCCL_VM_ERROR("{} Extra output data exists after the expected AllToAll result "
87+ "range, rankId={}, extraBufferRange=[0x{:x},0x{:x})\nCurrent result range detail:\n{}",
88+ MakeErrorCodeText(ErrorCode::CHECKER_RUNTIME_ERROR), rankId, ele.startAddr, rangeEnd,
89+ ele.Describe());
77 return HcclResult::HCCL_E_PARA;90 return HcclResult::HCCL_E_PARA;
78 }91 }
79 92
80 if (ele.srcBufs.begin()->rankId != curRankId) {93 if (ele.srcBufs.begin()->rankId != curRankId) {
81- HCCL_ERROR("[rankId:%u]Cur buffer semantic should come from rank %u, while it come from rank %u, "94+ HCCL_VM_ERROR("{} This AllToAll result range comes from the wrong source rank, "
82- "cur buffer semantic is %s", rankId, curRankId, ele.srcBufs.begin()->rankId, ele.Describe().c_str());95+ "rankId={}, actualSourceRank={}, expectedSourceRank={}, actualBufferRange=[0x{:x},0x{:x})"
96+ "\nCurrent result range detail:\n{}",
97+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_SRC_ERROR), rankId,
98+ ele.srcBufs.begin()->rankId, curRankId, ele.startAddr, rangeEnd, ele.Describe());
83 return HcclResult::HCCL_E_PARA;99 return HcclResult::HCCL_E_PARA;
84 }100 }
85 101 
86 if (ele.srcBufs.begin()->bufType != BufferType::INPUT) {102 if (ele.srcBufs.begin()->bufType != BufferType::INPUT) {
87- HCCL_ERROR("[rankId:%u]Cur buffer semantic srcBufs bufType is not INPUT, cur buffer semantic is %s",103+ HCCL_VM_ERROR("{} This AllToAll result range comes from a non-INPUT buffer, but it "
88- rankId, ele.Describe().c_str());104+ "should come from INPUT, rankId={}, actualSourceRank={}, actualSourceBufferType={}, "
105+ "actualBufferRange=[0x{:x},0x{:x})\nCurrent result range detail:\n{}",
106+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_SRC_ERROR), rankId,
107+ ele.srcBufs.begin()->rankId, BufferTypeToString(ele.srcBufs.begin()->bufType),
108+ ele.startAddr, rangeEnd, ele.Describe());
89 return HcclResult::HCCL_E_PARA;109 return HcclResult::HCCL_E_PARA;
90 }110 }
91 111 
92 if (ele.srcBufs.begin()->srcAddr != sendOffsets[curRankId] + curDataSize) {112 if (ele.srcBufs.begin()->srcAddr != sendOffsets[curRankId] + curDataSize) {
93- HCCL_ERROR("[rankId:%u]Cur buffer semantic srcBufs srcAddr should be %llu, while it is %llu, cur buffer semantic is %s",113+ HCCL_VM_ERROR("{} Source address for this AllToAll result range does not match the "
94- rankId, sendOffsets[curRankId] + curDataSize, ele.srcBufs.begin()->srcAddr, ele.Describe().c_str());114+ "expected input address, rankId={}, sourceRank={}, expectedAddr=0x{:x}, actualAddr=0x{:x}, "
115+ "actualBufferRange=[0x{:x},0x{:x})\nCurrent result range detail:\n{}",
116+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_SRC_ERROR), rankId,
117+ ele.srcBufs.begin()->rankId, sendOffsets[curRankId] + curDataSize,
118+ ele.srcBufs.begin()->srcAddr, ele.startAddr, rangeEnd, ele.Describe());
95 return HcclResult::HCCL_E_PARA;119 return HcclResult::HCCL_E_PARA;
96 }120 }
97 121 
@@ -106,17 +130,20 @@ HcclResult TaskCheckAll2AllSemantics(std::map<RankId, RankMemorySemantics> &allR
106 curRankId++;130 curRankId++;
107 }131 }
108 } else if (curDataSize > recvDataSizeFromCurRank) {132 } else if (curDataSize > recvDataSizeFromCurRank) {
109- HCCL_ERROR("[rankId:%u]Accumulated semantic size from rank %u is %llu, greater than expected %llu",133+ HCCL_VM_ERROR("{} Data taken from one source rank is larger than expected in "
110- rankId, curRankId, curDataSize, recvDataSizeFromCurRank);134+ "AllToAll, rankId={}, sourceRank={}, actualSize=0x{:x}, expectedSize=0x{:x}, "
135+ "actualBufferRange=[0x{:x},0x{:x})\nCurrent result range detail:\n{}",
136+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_SIZE_ERROR), rankId, curRankId, curDataSize,
137+ recvDataSizeFromCurRank, ele.startAddr, rangeEnd, ele.Describe());
111 return HcclResult::HCCL_E_PARA;138 return HcclResult::HCCL_E_PARA;
112 }139 }
113 totalSize += ele.size;140 totalSize += ele.size;
114 }141 }
115 // 如果curRankId等于rankSize,表示已经接受到其他所有rank的数据142 // 如果curRankId等于rankSize,表示已经接受到其他所有rank的数据
116 if (curRankId != rankSize) {143 if (curRankId != rankSize) {
117- HCCL_ERROR("[rankId:%u]Missing buffer semantics in tail: already checked total size is %llu, "144+ HCCL_VM_ERROR("{} AllToAll result data ends before the expected total size is reached, "
118- "accumulated semantic size from rank %u is %llu, while rankSize is %u",145+ "rankId={}, checkedSize=0x{:x}, remainingSourceRank={}, remainingSizeFromThatRank=0x{:x}",
119- rankId, totalSize, curRankId, curDataSize, rankSize);146+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_MISSING), rankId, totalSize, curRankId, curDataSize);
120 return HcclResult::HCCL_E_PARA;147 return HcclResult::HCCL_E_PARA;
121 }148 }
122 }149 }
Mtest/hccl_vm/src/plugin/checker/src/framework/semantics_check/allgather_semantics_checker.cc+42-13
@@ -14,6 +14,9 @@
14 14 
15#include "check_utils.h"15#include "check_utils.h"
16#include "sim_common.h"16#include "sim_common.h"
17+#include "sim_log.h"
18+#include "utils/dump/dump_json_utils.h"
19+#include "utils/error_codes.h"
17 20 
18namespace HcclSim {21namespace HcclSim {
19HcclResult TaskCheckAllGatherSemantics(std::map<RankId, RankMemorySemantics> &allRankMemSemantics, u64 dataSize)22HcclResult TaskCheckAllGatherSemantics(std::map<RankId, RankMemorySemantics> &allRankMemSemantics, u64 dataSize)
@@ -23,7 +26,9 @@ HcclResult TaskCheckAllGatherSemantics(std::map<RankId, RankMemorySemantics> &al
23 for (RankId rankId = 0; rankId < rankSize; rankId++) {26 for (RankId rankId = 0; rankId < rankSize; rankId++) {
24 // 对应的rank不存在需要报错27 // 对应的rank不存在需要报错
25 if (allRankMemSemantics.count(rankId) == 0) {28 if (allRankMemSemantics.count(rankId) == 0) {
26- HCCL_ERROR("Missing rank %d mem semantics", rankId);29+ HCCL_VM_ERROR("{} AllGather produced no result data for rank {}, but this rank is "
30+ "expected to receive data from all ranks, expectedSourceRankCount={}, expectedResultSize=0x{:x}",
31+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_MISSING), rankId, rankSize, dataSize * rankSize);
27 return HcclResult::HCCL_E_PARA;32 return HcclResult::HCCL_E_PARA;
28 }33 }
29 34 
@@ -31,31 +36,51 @@ HcclResult TaskCheckAllGatherSemantics(std::map<RankId, RankMemorySemantics> &al
31 RankId curRankId = 0;36 RankId curRankId = 0;
32 u64 curDataSize = 0;37 u64 curDataSize = 0;
33 for (auto &ele : allRankMemSemantics[rankId][BufferType::OUTPUT]) {38 for (auto &ele : allRankMemSemantics[rankId][BufferType::OUTPUT]) {
39+ const u64 rangeEnd = ele.startAddr + ele.size;
34 if (ele.startAddr != totalSize) {40 if (ele.startAddr != totalSize) {
35- HCCL_ERROR("[rankId:%u]Missing buffer semantic: expected startAddr is %llu, while cur buffer semantic startAddr is %llu, cur buffer semantic is %s",41+ HCCL_VM_ERROR("{} AllGather result data does not start from the expected address, "
36- rankId, totalSize, ele.startAddr, ele.Describe().c_str());42+ "rankId={}, expectedStartAddr=0x{:x}, actualStartAddr=0x{:x}, actualBufferRange=[0x{:x},0x{:x})"
43+ "\nCurrent result range detail:\n{}",
44+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_MISSING), rankId, totalSize, ele.startAddr,
45+ ele.startAddr, rangeEnd, ele.Describe());
37 return HcclResult::HCCL_E_PARA;46 return HcclResult::HCCL_E_PARA;
38 }47 }
39 48 
40 if (ele.srcBufs.size() != 1) {49 if (ele.srcBufs.size() != 1) {
41- HCCL_ERROR("[rankId:%u]Cur buffer semantic should not be reduce, which mean srcBufs size should be 1, while cur buffer semantic is %s", rankId, ele.Describe().c_str());50+ HCCL_VM_ERROR("{} This AllGather result range combines multiple sources, but this "
51+ "operator expects exactly one source, rankId={}, actualSourceCount={}, expectedSourceCount=1, "
52+ "outputRange=[0x{:x},0x{:x})\nCurrent result range detail:\n{}",
53+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_REDUCE_ERROR), rankId, ele.srcBufs.size(),
54+ ele.startAddr, rangeEnd, ele.Describe());
42 return HcclResult::HCCL_E_PARA;55 return HcclResult::HCCL_E_PARA;
43 }56 }
44 57 
45 if (ele.srcBufs.begin()->rankId != curRankId) {58 if (ele.srcBufs.begin()->rankId != curRankId) {
46- HCCL_ERROR("[rankId:%u]Cur buffer semantic should come from rank %u, while it come from rank %u, cur buffer semantic is %s", rankId, curRankId, ele.srcBufs.begin()->rankId, ele.Describe().c_str());59+ HCCL_VM_ERROR("{} This AllGather result range comes from the wrong source rank, "
60+ "rankId={}, actualSourceRank={}, expectedSourceRank={}, actualBufferRange=[0x{:x},0x{:x})"
61+ "\nCurrent result range detail:\n{}",
62+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_SRC_ERROR), rankId,
63+ ele.srcBufs.begin()->rankId, curRankId, ele.startAddr, rangeEnd, ele.Describe());
47 return HcclResult::HCCL_E_PARA;64 return HcclResult::HCCL_E_PARA;
48 }65 }
49 66 
50 if (ele.srcBufs.begin()->bufType != BufferType::INPUT) {67 if (ele.srcBufs.begin()->bufType != BufferType::INPUT) {
51- HCCL_ERROR("[rankId:%u]Cur buffer semantic srcBufs bufType is not INPUT, cur buffer semantic is %s",68+ HCCL_VM_ERROR("{} This AllGather result range comes from a non-INPUT buffer, but it "
52- rankId, ele.Describe().c_str());69+ "should come from INPUT, rankId={}, actualSourceRank={}, actualSourceBufferType={}, "
70+ "actualBufferRange=[0x{:x},0x{:x})\nCurrent result range detail:\n{}",
71+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_SRC_ERROR), rankId,
72+ ele.srcBufs.begin()->rankId, BufferTypeToString(ele.srcBufs.begin()->bufType),
73+ ele.startAddr, rangeEnd, ele.Describe());
53 return HcclResult::HCCL_E_PARA;74 return HcclResult::HCCL_E_PARA;
54 }75 }
55 76 
56 if (ele.srcBufs.begin()->srcAddr != curDataSize) {77 if (ele.srcBufs.begin()->srcAddr != curDataSize) {
57- HCCL_ERROR("[rankId:%u]Cur buffer semantic srcBufs srcAddr should be %llu, while it is %llu, cur buffer semantic is %s",78+ HCCL_VM_ERROR("{} Source address for this AllGather result range does not match the "
58- rankId, curDataSize, ele.srcBufs.begin()->srcAddr, ele.Describe().c_str());79+ "expected input address, rankId={}, sourceRank={}, expectedAddr=0x{:x}, actualAddr=0x{:x}, "
80+ "actualBufferRange=[0x{:x},0x{:x})\nCurrent result range detail:\n{}",
81+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_SRC_ERROR), rankId,
82+ ele.srcBufs.begin()->rankId, curDataSize, ele.srcBufs.begin()->srcAddr, ele.startAddr,
83+ rangeEnd, ele.Describe());
59 return HcclResult::HCCL_E_PARA;84 return HcclResult::HCCL_E_PARA;
60 }85 }
61 86 
@@ -64,15 +89,19 @@ HcclResult TaskCheckAllGatherSemantics(std::map<RankId, RankMemorySemantics> &al
64 curDataSize = 0;89 curDataSize = 0;
65 curRankId++;90 curRankId++;
66 } else if (curDataSize > dataSize) {91 } else if (curDataSize > dataSize) {
67- HCCL_ERROR("[rankId:%u]Accumulated semantic size from rank %u is %llu, greater than expected %llu",92+ HCCL_VM_ERROR("{} Data taken from one source rank is larger than expected in "
68- rankId, curRankId, curDataSize, dataSize);93+ "AllGather, rankId={}, sourceRank={}, actualSize=0x{:x}, expectedSize=0x{:x}, "
94+ "actualBufferRange=[0x{:x},0x{:x})\nCurrent result range detail:\n{}",
95+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_SIZE_ERROR), rankId, curRankId, curDataSize,
96+ dataSize, ele.startAddr, rangeEnd, ele.Describe());
69 return HcclResult::HCCL_E_PARA;97 return HcclResult::HCCL_E_PARA;
70 }98 }
71 totalSize += ele.size;99 totalSize += ele.size;
72 }100 }
73 if (totalSize != dataSize * rankSize) {101 if (totalSize != dataSize * rankSize) {
74- HCCL_ERROR("[rankId:%u]Missing buffer semantics in tail: already checked total size is %llu, accumulated semantic size from rank %u is %llu, while expected total size is %llu",102+ HCCL_VM_ERROR("{} AllGather result data ends before the expected total size is "
75- rankId, totalSize, curRankId, curDataSize, dataSize * rankSize);103+ "reached, rankId={}, checkedSize=0x{:x}, expectedSize=0x{:x}",
104+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_MISSING), rankId, totalSize, dataSize * rankSize);
76 return HcclResult::HCCL_E_PARA;105 return HcclResult::HCCL_E_PARA;
77 }106 }
78 }107 }
Mtest/hccl_vm/src/plugin/checker/src/framework/semantics_check/allgather_v_semantics_checker.cc+42-18
@@ -14,7 +14,9 @@
14#include <map>14#include <map>
15 15 
16#include "checker_def.h"16#include "checker_def.h"
17-#include "log.h"17+#include "sim_log.h"
18+#include "utils/dump/dump_json_utils.h"
19+#include "utils/error_codes.h"
18 20 
19namespace HcclSim {21namespace HcclSim {
20HcclResult TaskCheckAllGatherVSemantics(std::map<RankId, RankMemorySemantics> &allRankMemSemantics, VDataDesTagInner &vDataDes)22HcclResult TaskCheckAllGatherVSemantics(std::map<RankId, RankMemorySemantics> &allRankMemSemantics, VDataDesTagInner &vDataDes)
@@ -32,7 +34,9 @@ HcclResult TaskCheckAllGatherVSemantics(std::map<RankId, RankMemorySemantics> &a
32 for (RankId rankId = 0; rankId < rankSize; rankId++) {34 for (RankId rankId = 0; rankId < rankSize; rankId++) {
33 // 对应的rank不存在需要报错35 // 对应的rank不存在需要报错
34 if (allRankMemSemantics.count(rankId) == 0) {36 if (allRankMemSemantics.count(rankId) == 0) {
35- HCCL_ERROR("Missing rank %d mem semantics", rankId);37+ HCCL_VM_ERROR("{} AllGatherV produced no result data for rank {}, but this rank is "
38+ "expected to receive data from all ranks, expectedSourceRankCount={}, expectedResultSize=0x{:x}",
39+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_MISSING), rankId, rankSize, outputSize);
36 return HcclResult::HCCL_E_PARA;40 return HcclResult::HCCL_E_PARA;
37 }41 }
38 42 
@@ -45,35 +49,52 @@ HcclResult TaskCheckAllGatherVSemantics(std::map<RankId, RankMemorySemantics> &a
45 curRankId++;49 curRankId++;
46 inputSize = vDataDes.counts[curRankId] * CHECK_SIZE_TABLE[vDataDes.dataType];50 inputSize = vDataDes.counts[curRankId] * CHECK_SIZE_TABLE[vDataDes.dataType];
47 }51 }
52+ const u64 rangeEnd = ele.startAddr + ele.size;
48 53 
49 if (ele.startAddr != totalSize) {54 if (ele.startAddr != totalSize) {
50- HCCL_ERROR("[rankId:%u]Missing buffer semantic: "55+ HCCL_VM_ERROR("{} AllGatherV result data does not start from the expected address, "
51- "exepected startAddr is %llu, while cur buffer semantic startAddr is %llu, cur buffer semantic is %s",56+ "rankId={}, expectedStartAddr=0x{:x}, actualStartAddr=0x{:x}, actualBufferRange=[0x{:x},0x{:x})"
52- rankId, totalSize, ele.startAddr, ele.Describe().c_str());57+ "\nCurrent result range detail:\n{}",
58+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_MISSING), rankId, totalSize, ele.startAddr,
59+ ele.startAddr, rangeEnd, ele.Describe());
53 return HcclResult::HCCL_E_PARA;60 return HcclResult::HCCL_E_PARA;
54 }61 }
55 62 
56 if (ele.srcBufs.size() != 1) {63 if (ele.srcBufs.size() != 1) {
57- HCCL_ERROR("[rankId:%u]Cur buffer semantic should not be reduce, which mean srcBufs size should be 1, "64+ HCCL_VM_ERROR("{} This AllGatherV result range combines multiple sources, but this "
58- "while cur buffer semantic is %s", rankId, ele.Describe().c_str());65+ "operator expects exactly one source, rankId={}, actualSourceCount={}, expectedSourceCount=1, "
66+ "outputRange=[0x{:x},0x{:x})\nCurrent result range detail:\n{}",
67+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_REDUCE_ERROR), rankId, ele.srcBufs.size(),
68+ ele.startAddr, rangeEnd, ele.Describe());
59 return HcclResult::HCCL_E_PARA;69 return HcclResult::HCCL_E_PARA;
60 }70 }
61 71 
62 if (ele.srcBufs.begin()->rankId != curRankId) {72 if (ele.srcBufs.begin()->rankId != curRankId) {
63- HCCL_ERROR("[rankId:%u]Cur buffer semantic should come from rank %u, while it come from rank %u, "73+ HCCL_VM_ERROR("{} This AllGatherV result range comes from the wrong source rank, "
64- "cur buffer semantic is %s", rankId, curRankId, ele.srcBufs.begin()->rankId, ele.Describe().c_str());74+ "rankId={}, actualSourceRank={}, expectedSourceRank={}, actualBufferRange=[0x{:x},0x{:x})"
75+ "\nCurrent result range detail:\n{}",
76+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_SRC_ERROR), rankId,
77+ ele.srcBufs.begin()->rankId, curRankId, ele.startAddr, rangeEnd, ele.Describe());
65 return HcclResult::HCCL_E_PARA;78 return HcclResult::HCCL_E_PARA;
66 }79 }
67 80 
68 if (ele.srcBufs.begin()->bufType != BufferType::INPUT) {81 if (ele.srcBufs.begin()->bufType != BufferType::INPUT) {
69- HCCL_ERROR("[rankId:%u]Cur buffer semantic srcBufs bufType is not INPUT, cur buffer semantic is %s",82+ HCCL_VM_ERROR("{} This AllGatherV result range comes from a non-INPUT buffer, but "
70- rankId, ele.Describe().c_str());83+ "it should come from INPUT, rankId={}, actualSourceRank={}, actualSourceBufferType={}, "
84+ "actualBufferRange=[0x{:x},0x{:x})\nCurrent result range detail:\n{}",
85+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_SRC_ERROR), rankId,
86+ ele.srcBufs.begin()->rankId, BufferTypeToString(ele.srcBufs.begin()->bufType),
87+ ele.startAddr, rangeEnd, ele.Describe());
71 return HcclResult::HCCL_E_PARA;88 return HcclResult::HCCL_E_PARA;
72 }89 }
73 90 
74 if (ele.srcBufs.begin()->srcAddr != curDataSize) {91 if (ele.srcBufs.begin()->srcAddr != curDataSize) {
75- HCCL_ERROR("[rankId:%u]Cur buffer semantic srcBufs srcAddr should be %llu, while it is %llu, cur buffer semantic is %s",92+ HCCL_VM_ERROR("{} Source address for this AllGatherV result range does not match "
76- rankId, curDataSize, ele.srcBufs.begin()->srcAddr, ele.Describe().c_str());93+ "the expected input address, rankId={}, sourceRank={}, expectedAddr=0x{:x}, actualAddr=0x{:x}, "
94+ "actualBufferRange=[0x{:x},0x{:x})\nCurrent result range detail:\n{}",
95+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_SRC_ERROR), rankId,
96+ ele.srcBufs.begin()->rankId, curDataSize, ele.srcBufs.begin()->srcAddr, ele.startAddr,
97+ rangeEnd, ele.Describe());
77 return HcclResult::HCCL_E_PARA;98 return HcclResult::HCCL_E_PARA;
78 }99 }
79 100 
@@ -83,16 +104,19 @@ HcclResult TaskCheckAllGatherVSemantics(std::map<RankId, RankMemorySemantics> &a
83 curDataSize = 0;104 curDataSize = 0;
84 curRankId++;105 curRankId++;
85 } else if (curDataSize > inputSize) {106 } else if (curDataSize > inputSize) {
86- HCCL_ERROR("[rankId:%u]Accumulated semantic size from rank %u is %llu, greater than expected %llu",107+ HCCL_VM_ERROR("{} Data taken from one source rank is larger than expected in "
87- rankId, curRankId, curDataSize, inputSize);108+ "AllGatherV, rankId={}, sourceRank={}, actualSize=0x{:x}, expectedSize=0x{:x}, "
109+ "actualBufferRange=[0x{:x},0x{:x})\nCurrent result range detail:\n{}",
110+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_SIZE_ERROR), rankId, curRankId, curDataSize,
111+ inputSize, ele.startAddr, rangeEnd, ele.Describe());
88 return HcclResult::HCCL_E_PARA;112 return HcclResult::HCCL_E_PARA;
89 }113 }
90 totalSize += ele.size;114 totalSize += ele.size;
91 }115 }
92 if (totalSize != outputSize) {116 if (totalSize != outputSize) {
93- HCCL_ERROR("[rankId:%u]Missing buffer semantics in tail: already checked total size is %llu, "117+ HCCL_VM_ERROR("{} AllGatherV result data ends before the expected total size is "
94- "while outputSize is %llu, rankSize is %u",118+ "reached, rankId={}, checkedSize=0x{:x}, expectedSize=0x{:x}",
95- rankId, totalSize, outputSize, rankSize);119+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_MISSING), rankId, totalSize, outputSize);
96 return HcclResult::HCCL_E_PARA;120 return HcclResult::HCCL_E_PARA;
97 }121 }
98 }122 }
Mtest/hccl_vm/src/plugin/checker/src/framework/semantics_check/allreduce_semantics_checker.cc+43-15
@@ -14,6 +14,9 @@
14 14 
15#include "base.h"15#include "base.h"
16#include "check_utils.h"16#include "check_utils.h"
17+#include "sim_log.h"
18+#include "utils/dump/dump_json_utils.h"
19+#include "utils/error_codes.h"
17 20 
18namespace HcclSim {21namespace HcclSim {
19HcclResult TaskCheckAllReduceSemantics(std::map<RankId, RankMemorySemantics> &allRankMemSemantics, u64 dataSize,22HcclResult TaskCheckAllReduceSemantics(std::map<RankId, RankMemorySemantics> &allRankMemSemantics, u64 dataSize,
@@ -24,54 +27,79 @@ HcclResult TaskCheckAllReduceSemantics(std::map<RankId, RankMemorySemantics> &al
24 for (RankId rankId = 0; rankId < rankSize; rankId++) {27 for (RankId rankId = 0; rankId < rankSize; rankId++) {
25 // 对应的rank不存在需要报错28 // 对应的rank不存在需要报错
26 if (allRankMemSemantics.count(rankId) == 0) {29 if (allRankMemSemantics.count(rankId) == 0) {
27- HCCL_ERROR("Missing rank %d mem semantics", rankId);30+ HCCL_VM_ERROR("{} AllReduce produced no result data for rank {}, but this rank is "
31+ "expected to have a full reduced result, expectedSourceRankCount={}, expectedResultSize=0x{:x}",
32+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_MISSING), rankId, rankSize, dataSize);
28 return HcclResult::HCCL_E_PARA;33 return HcclResult::HCCL_E_PARA;
29 }34 }
30 35 
31 u64 totalSize = 0;36 u64 totalSize = 0;
32 for (auto &ele : allRankMemSemantics[rankId][BufferType::OUTPUT]) {37 for (auto &ele : allRankMemSemantics[rankId][BufferType::OUTPUT]) {
38+ const u64 rangeEnd = ele.startAddr + ele.size;
33 if (ele.startAddr != totalSize) {39 if (ele.startAddr != totalSize) {
34- HCCL_ERROR("[rankId:%u]Missing buffer semantic: expected startAddr is %llu, while cur buffer semantic startAddr is %llu, cur buffer semantic is %s",40+ HCCL_VM_ERROR("{} AllReduce result data does not start from the expected address, "
35- rankId, totalSize, ele.startAddr, ele.Describe().c_str());41+ "rankId={}, expectedStartAddr=0x{:x}, actualStartAddr=0x{:x}, actualBufferRange=[0x{:x},0x{:x})"
42+ "\nCurrent result range detail:\n{}",
43+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_MISSING), rankId, totalSize, ele.startAddr,
44+ ele.startAddr, rangeEnd, ele.Describe());
36 return HcclResult::HCCL_E_PARA;45 return HcclResult::HCCL_E_PARA;
37 }46 }
38 47 
39 if (ele.srcBufs.size() > 1 && ele.reduceType != reduceType) {48 if (ele.srcBufs.size() > 1 && ele.reduceType != reduceType) {
40- HCCL_ERROR("[rankId:%u]cur buffer semantic reduceType %d is unequal to expected reduceType %d, cur buffer semantic is %s",49+ HCCL_VM_ERROR("{} Reduce mode of this AllReduce result range does not match the "
41- rankId, ele.reduceType, reduceType, ele.Describe().c_str());50+ "operator setting, rankId={}, actualReduceType={}, expectedReduceType={}, "
51+ "outputRange=[0x{:x},0x{:x})\nCurrent result range detail:\n{}",
52+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_REDUCE_ERROR), rankId,
53+ DumpReduceOpToString(ele.reduceType), DumpReduceOpToString(reduceType),
54+ ele.startAddr, rangeEnd, ele.Describe());
42 return HcclResult::HCCL_E_PARA;55 return HcclResult::HCCL_E_PARA;
43 }56 }
44 57 
45 if (ele.srcBufs.size() != rankSize) {58 if (ele.srcBufs.size() != rankSize) {
46- HCCL_ERROR("[rankId:%u]buffer semantic srcBufs size %u is unequal to rankSize %u, cur buffer semantic is %s",59+ HCCL_VM_ERROR("{} This AllReduce result range does not include the expected number "
47- rankId, ele.srcBufs.size(), rankSize, ele.Describe().c_str());60+ "of source ranks, rankId={}, actualSourceRankCount={}, expectedRankSize={}, "
61+ "outputRange=[0x{:x},0x{:x})\nCurrent result range detail:\n{}",
62+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_REDUCE_ERROR), rankId, ele.srcBufs.size(),
63+ rankSize, ele.startAddr, rangeEnd, ele.Describe());
48 return HcclResult::HCCL_E_PARA;64 return HcclResult::HCCL_E_PARA;
49 }65 }
50 66 
51 if (ele.srcBufs.begin()->rankId != 0 or ele.srcBufs.rbegin()->rankId != (rankSize - 1)) {67 if (ele.srcBufs.begin()->rankId != 0 or ele.srcBufs.rbegin()->rankId != (rankSize - 1)) {
52- HCCL_ERROR("[rankId:%u]cur buffer semantic srcBufs is invalid, cur buffer semantic is %s",68+ HCCL_VM_ERROR("{} Source rank list for this AllReduce result range is not the "
53- rankId, ele.Describe().c_str());69+ "complete expected rank set, rankId={}, firstSourceRank={}, lastSourceRank={}, "
70+ "expectedFirstSourceRank=0, expectedLastSourceRank={}, outputRange=[0x{:x},0x{:x})"
71+ "\nCurrent result range detail:\n{}",
72+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_SRC_ERROR), rankId,
73+ ele.srcBufs.begin()->rankId, ele.srcBufs.rbegin()->rankId, rankSize - 1, ele.startAddr,
74+ rangeEnd, ele.Describe());
54 return HcclResult::HCCL_E_PARA;75 return HcclResult::HCCL_E_PARA;
55 }76 }
56 77 
57 for (auto &srcBuf : ele.srcBufs) {78 for (auto &srcBuf : ele.srcBufs) {
58 if (srcBuf.bufType != BufferType::INPUT) {79 if (srcBuf.bufType != BufferType::INPUT) {
59- HCCL_ERROR("[rankId:%u]Cur buffer semantic srcBufs bufType is not INPUT, cur buffer semantic is %s",80+ HCCL_VM_ERROR("{} This AllReduce result range comes from a non-INPUT buffer, "
60- rankId, ele.Describe().c_str());81+ "but it should come from INPUT, rankId={}, actualSourceRank={}, actualSourceBufferType={}, "
82+ "actualBufferRange=[0x{:x},0x{:x})\nCurrent result range detail:\n{}",
83+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_SRC_ERROR), rankId, srcBuf.rankId,
84+ BufferTypeToString(srcBuf.bufType), ele.startAddr, rangeEnd, ele.Describe());
61 return HcclResult::HCCL_E_PARA;85 return HcclResult::HCCL_E_PARA;
62 }86 }
63 87 
64 if (srcBuf.srcAddr != totalSize) {88 if (srcBuf.srcAddr != totalSize) {
65- HCCL_ERROR("[rankId:%u]Expected semantic srcBuf srcAddr is %llu, while cur srcBuf srcAddr is %llu, cur buffer semantic is %s",89+ HCCL_VM_ERROR("{} Source address for this AllReduce result range does not match "
66- rankId, totalSize, srcBuf.srcAddr, ele.Describe().c_str());90+ "the expected input address, rankId={}, sourceRank={}, expectedAddr=0x{:x}, "
91+ "actualAddr=0x{:x}, actualBufferRange=[0x{:x},0x{:x})\nCurrent result range detail:\n{}",
92+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_SRC_ERROR), rankId, srcBuf.rankId, totalSize,
93+ srcBuf.srcAddr, ele.startAddr, rangeEnd, ele.Describe());
67 return HcclResult::HCCL_E_PARA;94 return HcclResult::HCCL_E_PARA;
68 }95 }
69 }96 }
70 totalSize += ele.size;97 totalSize += ele.size;
71 }98 }
72 if (totalSize != dataSize) {99 if (totalSize != dataSize) {
73- HCCL_ERROR("[rankId:%u]Missing buffer semantics in tail: already checked total size is %llu, while expected total size is %llu",100+ HCCL_VM_ERROR("{} AllReduce result data ends before the expected total size is reached, "
74- rankId, totalSize, dataSize);101+ "rankId={}, checkedSize=0x{:x}, expectedSize=0x{:x}",
102+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_MISSING), rankId, totalSize, dataSize);
75 return HcclResult::HCCL_E_PARA;103 return HcclResult::HCCL_E_PARA;
76 }104 }
77 }105 }
Mtest/hccl_vm/src/plugin/checker/src/framework/semantics_check/batchsendrecv_semantics_checker.cc+43-13
@@ -14,6 +14,9 @@
14 14 
15#include "base.h"15#include "base.h"
16#include "check_utils.h"16#include "check_utils.h"
17+#include "sim_log.h"
18+#include "utils/dump/dump_json_utils.h"
19+#include "utils/error_codes.h"
17 20 
18namespace HcclSim {21namespace HcclSim {
19HcclResult TaskCheckBatchSendRecvSemantics(std::map<RankId, RankMemorySemantics> &allRankMemSemantics, u64 dataSize)22HcclResult TaskCheckBatchSendRecvSemantics(std::map<RankId, RankMemorySemantics> &allRankMemSemantics, u64 dataSize)
@@ -23,7 +26,10 @@ HcclResult TaskCheckBatchSendRecvSemantics(std::map<RankId, RankMemorySemantics>
23 for (RankId rankId = 0; rankId < rankSize; rankId++) {26 for (RankId rankId = 0; rankId < rankSize; rankId++) {
24 // 对应的rank不存在需要报错27 // 对应的rank不存在需要报错
25 if (allRankMemSemantics.count(rankId) == 0) {28 if (allRankMemSemantics.count(rankId) == 0) {
26- HCCL_ERROR("Missing rank %d mem semantics", rankId);29+ HCCL_VM_ERROR("{} BatchSendRecv produced no result data for rank {}, but this rank is "
30+ "expected to receive one segment from each rank, expectedSourceRankCount={}, "
31+ "expectedResultSize=0x{:x}", MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_MISSING), rankId,
32+ rankSize, dataSize * rankSize);
27 return HcclResult::HCCL_E_PARA;33 return HcclResult::HCCL_E_PARA;
28 }34 }
29 35 
@@ -31,31 +37,51 @@ HcclResult TaskCheckBatchSendRecvSemantics(std::map<RankId, RankMemorySemantics>
31 RankId curRankId = 0;37 RankId curRankId = 0;
32 u64 curDataSize = 0;38 u64 curDataSize = 0;
33 for (auto &ele : allRankMemSemantics[rankId][BufferType::OUTPUT]) {39 for (auto &ele : allRankMemSemantics[rankId][BufferType::OUTPUT]) {
40+ const u64 rangeEnd = ele.startAddr + ele.size;
34 if (ele.startAddr != totalSize) {41 if (ele.startAddr != totalSize) {
35- HCCL_ERROR("[rankId:%u]Missing buffer semantic: expected startAddr is %llu, while cur buffer semantic startAddr is %llu, cur buffer semantic is %s",42+ HCCL_VM_ERROR("{} BatchSendRecv result data does not start from the expected "
36- rankId, totalSize, ele.startAddr, ele.Describe().c_str());43+ "address, rankId={}, expectedStartAddr=0x{:x}, actualStartAddr=0x{:x}, "
44+ "actualBufferRange=[0x{:x},0x{:x})\nCurrent result range detail:\n{}",
45+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_MISSING), rankId, totalSize, ele.startAddr,
46+ ele.startAddr, rangeEnd, ele.Describe());
37 return HcclResult::HCCL_E_PARA;47 return HcclResult::HCCL_E_PARA;
38 }48 }
39 49 
40 if (ele.srcBufs.size() != 1) {50 if (ele.srcBufs.size() != 1) {
41- HCCL_ERROR("[rankId:%u]Cur buffer semantic should not be reduce, which mean srcBufs size should be 1, while cur buffer semantic is %s", rankId, ele.Describe().c_str());51+ HCCL_VM_ERROR("{} This BatchSendRecv result range combines multiple sources, but "
52+ "this operator expects exactly one source, rankId={}, actualSourceCount={}, "
53+ "expectedSourceCount=1, outputRange=[0x{:x},0x{:x})\nCurrent result range detail:\n{}",
54+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_REDUCE_ERROR), rankId, ele.srcBufs.size(),
55+ ele.startAddr, rangeEnd, ele.Describe());
42 return HcclResult::HCCL_E_PARA;56 return HcclResult::HCCL_E_PARA;
43 }57 }
44 58 
45 if (ele.srcBufs.begin()->rankId != curRankId) {59 if (ele.srcBufs.begin()->rankId != curRankId) {
46- HCCL_ERROR("[rankId:%u]Cur buffer semantic should come from rank %u, while it come from rank %u, cur buffer semantic is %s", rankId, curRankId, ele.srcBufs.begin()->rankId, ele.Describe().c_str());60+ HCCL_VM_ERROR("{} This BatchSendRecv result range comes from the wrong source "
61+ "rank, rankId={}, actualSourceRank={}, expectedSourceRank={}, actualBufferRange=[0x{:x},0x{:x})"
62+ "\nCurrent result range detail:\n{}",
63+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_SRC_ERROR), rankId,
64+ ele.srcBufs.begin()->rankId, curRankId, ele.startAddr, rangeEnd, ele.Describe());
47 return HcclResult::HCCL_E_PARA;65 return HcclResult::HCCL_E_PARA;
48 }66 }
49 67 
50 if (ele.srcBufs.begin()->bufType != BufferType::INPUT) {68 if (ele.srcBufs.begin()->bufType != BufferType::INPUT) {
51- HCCL_ERROR("[rankId:%u]Cur buffer semantic srcBufs bufType is not INPUT, cur buffer semantic is %s",69+ HCCL_VM_ERROR("{} This BatchSendRecv result range comes from a non-INPUT buffer, "
52- rankId, ele.Describe().c_str());70+ "but it should come from INPUT, rankId={}, actualSourceRank={}, actualSourceBufferType={}, "
71+ "actualBufferRange=[0x{:x},0x{:x})\nCurrent result range detail:\n{}",
72+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_SRC_ERROR), rankId,
73+ ele.srcBufs.begin()->rankId, BufferTypeToString(ele.srcBufs.begin()->bufType),
74+ ele.startAddr, rangeEnd, ele.Describe());
53 return HcclResult::HCCL_E_PARA;75 return HcclResult::HCCL_E_PARA;
54 }76 }
55 77 
56 if (ele.srcBufs.begin()->srcAddr != dataSize * rankId + curDataSize) {78 if (ele.srcBufs.begin()->srcAddr != dataSize * rankId + curDataSize) {
57- HCCL_ERROR("[rankId:%u]Cur buffer semantic srcBufs srcAddr should be %llu, while it is %llu, cur buffer semantic is %s",79+ HCCL_VM_ERROR("{} Source address for this BatchSendRecv result range does not "
58- rankId, dataSize * rankId + curDataSize, ele.srcBufs.begin()->srcAddr, ele.Describe().c_str());80+ "match the expected input address, rankId={}, sourceRank={}, expectedAddr=0x{:x}, "
81+ "actualAddr=0x{:x}, actualBufferRange=[0x{:x},0x{:x})\nCurrent result range detail:\n{}",
82+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_SRC_ERROR), rankId,
83+ ele.srcBufs.begin()->rankId, dataSize * rankId + curDataSize, ele.srcBufs.begin()->srcAddr,
84+ ele.startAddr, rangeEnd, ele.Describe());
59 return HcclResult::HCCL_E_PARA;85 return HcclResult::HCCL_E_PARA;
60 }86 }
61 curDataSize += ele.size;87 curDataSize += ele.size;
@@ -63,16 +89,20 @@ HcclResult TaskCheckBatchSendRecvSemantics(std::map<RankId, RankMemorySemantics>
63 curDataSize = 0;89 curDataSize = 0;
64 curRankId++;90 curRankId++;
65 } else if (curDataSize > dataSize) {91 } else if (curDataSize > dataSize) {
66- HCCL_ERROR("[rankId:%u]Accumulated semantic size from rank %u is %llu, greater than expected %llu",92+ HCCL_VM_ERROR("{} Data taken from one source rank is larger than expected in "
67- rankId, curRankId, curDataSize, dataSize);93+ "BatchSendRecv, rankId={}, sourceRank={}, actualSize=0x{:x}, expectedSize=0x{:x}, "
94+ "actualBufferRange=[0x{:x},0x{:x})\nCurrent result range detail:\n{}",
95+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_SIZE_ERROR), rankId, curRankId, curDataSize,
96+ dataSize, ele.startAddr, rangeEnd, ele.Describe());
68 return HcclResult::HCCL_E_PARA;97 return HcclResult::HCCL_E_PARA;
69 }98 }
70 totalSize += ele.size;99 totalSize += ele.size;
71 }100 }
72 // 如果curRankId等于rankSize,表示已经接受到其他所有rank的数据101 // 如果curRankId等于rankSize,表示已经接受到其他所有rank的数据
73 if (curRankId != rankSize) {102 if (curRankId != rankSize) {
74- HCCL_ERROR("[rankId:%u]Missing buffer semantics in tail: already checked total size is %llu, accumulated semantic size from rank %u is %llu, while rankSize is %u",103+ HCCL_VM_ERROR("{} BatchSendRecv result data ends before the expected total size is "
75- rankId, totalSize, curRankId, curDataSize, rankSize);104+ "reached, rankId={}, checkedSize=0x{:x}, expectedSize=0x{:x}",
105+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_MISSING), rankId, totalSize, dataSize * rankSize);
76 return HcclResult::HCCL_E_PARA;106 return HcclResult::HCCL_E_PARA;
77 }107 }
78 }108 }
Mtest/hccl_vm/src/plugin/checker/src/framework/semantics_check/broadcast_semantics_checker.cc+37-11
@@ -14,6 +14,9 @@
14 14 
15#include "base.h"15#include "base.h"
16#include "check_utils.h"16#include "check_utils.h"
17+#include "sim_log.h"
18+#include "utils/dump/dump_json_utils.h"
19+#include "utils/error_codes.h"
17 20 
18namespace HcclSim {21namespace HcclSim {
19HcclResult TaskCheckBroadcastSemantics(std::map<RankId, RankMemorySemantics> &allRankMemSemantics, u64 dataSize,22HcclResult TaskCheckBroadcastSemantics(std::map<RankId, RankMemorySemantics> &allRankMemSemantics, u64 dataSize,
@@ -24,45 +27,68 @@ HcclResult TaskCheckBroadcastSemantics(std::map<RankId, RankMemorySemantics> &al
24 for (RankId rankId = 0; rankId < rankSize; rankId++) {27 for (RankId rankId = 0; rankId < rankSize; rankId++) {
25 // 对应的rank不存在需要报错28 // 对应的rank不存在需要报错
26 if (allRankMemSemantics.count(rankId) == 0) {29 if (allRankMemSemantics.count(rankId) == 0) {
27- HCCL_ERROR("Missing rank %d mem semantics", rankId);30+ HCCL_VM_ERROR("{} Broadcast produced no result data for rank {}, but this rank is "
31+ "expected to receive the full root input, expectedSourceRank={}, expectedResultSize=0x{:x}",
32+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_MISSING), rankId, root, dataSize);
28 return HcclResult::HCCL_E_PARA;33 return HcclResult::HCCL_E_PARA;
29 }34 }
30 35 
31 u64 totalSize = 0;36 u64 totalSize = 0;
32 for (auto &ele : allRankMemSemantics[rankId][BufferType::INPUT]) {37 for (auto &ele : allRankMemSemantics[rankId][BufferType::INPUT]) {
38+ const u64 rangeEnd = ele.startAddr + ele.size;
33 if (ele.startAddr != totalSize) {39 if (ele.startAddr != totalSize) {
34- HCCL_ERROR("[rankId:%u]Missing buffer semantic: expected startAddr is %llu, while cur buffer semantic startAddr is %llu, cur buffer semantic is %s",40+ HCCL_VM_ERROR("{} Broadcast result data does not start from the expected address, "
35- rankId, totalSize, ele.startAddr, ele.Describe().c_str());41+ "rankId={}, expectedStartAddr=0x{:x}, actualStartAddr=0x{:x}, actualBufferRange=[0x{:x},0x{:x})"
42+ "\nCurrent result range detail:\n{}",
43+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_MISSING), rankId, totalSize, ele.startAddr,
44+ ele.startAddr, rangeEnd, ele.Describe());
36 return HcclResult::HCCL_E_PARA;45 return HcclResult::HCCL_E_PARA;
37 }46 }
38 47 
39 if (ele.srcBufs.size() != 1) {48 if (ele.srcBufs.size() != 1) {
40- HCCL_ERROR("[rankId:%u]Cur buffer semantic should not be reduce, which mean srcBufs size should be 1, while cur buffer semantic is %s", rankId, ele.Describe().c_str());49+ HCCL_VM_ERROR("{} This Broadcast result range combines multiple sources, but this "
50+ "operator expects exactly one source, rankId={}, actualSourceCount={}, expectedSourceCount=1, "
51+ "outputRange=[0x{:x},0x{:x})\nCurrent result range detail:\n{}",
52+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_REDUCE_ERROR), rankId, ele.srcBufs.size(),
53+ ele.startAddr, rangeEnd, ele.Describe());
41 return HcclResult::HCCL_E_PARA;54 return HcclResult::HCCL_E_PARA;
42 }55 }
43 56 
44 if (ele.srcBufs.begin()->rankId != root) {57 if (ele.srcBufs.begin()->rankId != root) {
45- HCCL_ERROR("[rankId:%u]Buffer semantic srcBuf rank[%u] is not from root[%u], cur buffer semantic is %s, cur rank is %u",58+ HCCL_VM_ERROR("{} This Broadcast result range comes from the wrong source rank, "
46- rankId, ele.srcBufs.begin()->rankId, root, ele.Describe().c_str(), rankId);59+ "rankId={}, actualSourceRank={}, expectedSourceRank={}, actualBufferRange=[0x{:x},0x{:x})"
60+ "\nCurrent result range detail:\n{}",
61+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_SRC_ERROR), rankId,
62+ ele.srcBufs.begin()->rankId, root, ele.startAddr, rangeEnd, ele.Describe());
47 return HcclResult::HCCL_E_PARA;63 return HcclResult::HCCL_E_PARA;
48 }64 }
49 65 
50 if (ele.srcBufs.begin()->bufType != BufferType::INPUT) {66 if (ele.srcBufs.begin()->bufType != BufferType::INPUT) {
51- HCCL_ERROR("[rankId:%u]Cur buffer semantic srcBufs bufType is not INPUT, cur buffer semantic is %s",67+ HCCL_VM_ERROR("{} This Broadcast result range comes from a non-INPUT buffer, but "
52- rankId, ele.Describe().c_str());68+ "it should come from INPUT, rankId={}, actualSourceRank={}, actualSourceBufferType={}, "
69+ "actualBufferRange=[0x{:x},0x{:x})\nCurrent result range detail:\n{}",
70+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_SRC_ERROR), rankId,
71+ ele.srcBufs.begin()->rankId, BufferTypeToString(ele.srcBufs.begin()->bufType),
72+ ele.startAddr, rangeEnd, ele.Describe());
53 return HcclResult::HCCL_E_PARA;73 return HcclResult::HCCL_E_PARA;
54 }74 }
55 75 
56 if (ele.srcBufs.begin()->srcAddr != totalSize) {76 if (ele.srcBufs.begin()->srcAddr != totalSize) {
57- HCCL_ERROR("[rankId:%u]Cur buffer semantic srcBufs srcAddr should be %llu, while it is %llu, cur buffer semantic is %s",77+ HCCL_VM_ERROR("{} Source address for this Broadcast result range does not match "
58- rankId, totalSize, ele.srcBufs.begin()->srcAddr, ele.Describe().c_str());78+ "the expected input address, rankId={}, sourceRank={}, expectedAddr=0x{:x}, actualAddr=0x{:x}, "
79+ "actualBufferRange=[0x{:x},0x{:x})\nCurrent result range detail:\n{}",
80+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_SRC_ERROR), rankId,
81+ ele.srcBufs.begin()->rankId, totalSize, ele.srcBufs.begin()->srcAddr, ele.startAddr,
82+ rangeEnd, ele.Describe());
59 return HcclResult::HCCL_E_PARA;83 return HcclResult::HCCL_E_PARA;
60 }84 }
61 85 
62 totalSize += ele.size;86 totalSize += ele.size;
63 }87 }
64 if (totalSize != dataSize) {88 if (totalSize != dataSize) {
65- HCCL_ERROR("[rankId:%u]Missing buffer semantics in tail: already checked total size is %llu, which should be %llu, cur rank is %u", rankId, totalSize, dataSize, rankId);89+ HCCL_VM_ERROR("{} Broadcast result data ends before the expected total size is reached, "
90+ "rankId={}, checkedSize=0x{:x}, expectedSize=0x{:x}",
91+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_MISSING), rankId, totalSize, dataSize);
66 return HcclResult::HCCL_E_PARA;92 return HcclResult::HCCL_E_PARA;
67 }93 }
68 }94 }
Mtest/hccl_vm/src/plugin/checker/src/framework/semantics_check/reduce_scatter_semantics_checker.cc+43-16
@@ -15,6 +15,8 @@
15#include "base.h"15#include "base.h"
16#include "check_utils.h"16#include "check_utils.h"
17#include "sim_log.h"17#include "sim_log.h"
18+#include "utils/dump/dump_json_utils.h"
19+#include "utils/error_codes.h"
18 20 
19namespace HcclSim {21namespace HcclSim {
20HcclResult TaskCheckReduceScatterSemantics(std::map<RankId, RankMemorySemantics> &allRankMemSemantics, u64 dataSize,22HcclResult TaskCheckReduceScatterSemantics(std::map<RankId, RankMemorySemantics> &allRankMemSemantics, u64 dataSize,
@@ -25,55 +27,80 @@ HcclResult TaskCheckReduceScatterSemantics(std::map<RankId, RankMemorySemantics>
25 for (u32 rankId = 0; rankId < rankSize; rankId++) {27 for (u32 rankId = 0; rankId < rankSize; rankId++) {
26 // 对应的rank不存在需要报错28 // 对应的rank不存在需要报错
27 if (allRankMemSemantics.count(rankId) == 0) {29 if (allRankMemSemantics.count(rankId) == 0) {
28- HCCL_ERROR("Missing rank %d mem semantics", rankId);30+ HCCL_VM_ERROR("{} ReduceScatter produced no result data for rank {}, but this rank is "
31+ "expected to have one reduced shard, expectedSourceRankCount={}, expectedResultSize=0x{:x}",
32+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_MISSING), rankId, rankSize, dataSize);
29 return HcclResult::HCCL_E_PARA;33 return HcclResult::HCCL_E_PARA;
30 }34 }
31 35 
32 u64 totalSize = 0;36 u64 totalSize = 0;
33- HCCL_VM_INFO("[INFO][TaskCheckReduceScatterSemantics] rankSize= {}, rankId= {} {}",
34- rankSize, rankId, allRankMemSemantics[rankId][BufferType::OUTPUT].size());
35 for (auto &ele : allRankMemSemantics[rankId][BufferType::OUTPUT]) {37 for (auto &ele : allRankMemSemantics[rankId][BufferType::OUTPUT]) {
38+ const u64 rangeEnd = ele.startAddr + ele.size;
36 if (ele.startAddr != totalSize) {39 if (ele.startAddr != totalSize) {
37- HCCL_ERROR("[rankId:%u]Missing buffer semantic: expected startAddr is %llu, while cur buffer semantic startAddr is %llu, cur buffer semantic is %s",40+ HCCL_VM_ERROR("{} ReduceScatter result data does not start from the expected "
38- rankId, totalSize, ele.startAddr, ele.Describe().c_str());41+ "address, rankId={}, expectedStartAddr=0x{:x}, actualStartAddr=0x{:x}, "
42+ "actualBufferRange=[0x{:x},0x{:x})\nCurrent result range detail:\n{}",
43+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_MISSING), rankId, totalSize, ele.startAddr,
44+ ele.startAddr, rangeEnd, ele.Describe());
39 return HcclResult::HCCL_E_PARA;45 return HcclResult::HCCL_E_PARA;
40 }46 }
41 47 
42 if (ele.srcBufs.size() > 1 && ele.reduceType != reduceType) {48 if (ele.srcBufs.size() > 1 && ele.reduceType != reduceType) {
43- HCCL_ERROR("[rankId:%u]cur buffer semantic reduceType %d is unequal to expected reduceType %d, cur buffer semantic is %s",49+ HCCL_VM_ERROR("{} Reduce mode of this ReduceScatter result range does not match the "
44- rankId, ele.reduceType, reduceType, ele.Describe().c_str());50+ "operator setting, rankId={}, actualReduceType={}, expectedReduceType={}, "
51+ "outputRange=[0x{:x},0x{:x})\nCurrent result range detail:\n{}",
52+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_REDUCE_ERROR), rankId,
53+ DumpReduceOpToString(ele.reduceType), DumpReduceOpToString(reduceType),
54+ ele.startAddr, rangeEnd, ele.Describe());
45 return HcclResult::HCCL_E_PARA;55 return HcclResult::HCCL_E_PARA;
46 }56 }
47 57 
48 if (ele.srcBufs.size() != rankSize) {58 if (ele.srcBufs.size() != rankSize) {
49- HCCL_ERROR("[rankId:%u]buffer semantic srcBufs size %u is unequal to rankSize %u, cur buffer semantic is %s",59+ HCCL_VM_ERROR("{} This ReduceScatter result range does not include the expected "
50- rankId, ele.srcBufs.size(), rankSize, ele.Describe().c_str());60+ "number of source ranks, rankId={}, actualSourceRankCount={}, expectedRankSize={}, "
61+ "outputRange=[0x{:x},0x{:x})\nCurrent result range detail:\n{}",
62+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_REDUCE_ERROR), rankId, ele.srcBufs.size(),
63+ rankSize, ele.startAddr, rangeEnd, ele.Describe());
51 return HcclResult::HCCL_E_PARA;64 return HcclResult::HCCL_E_PARA;
52 }65 }
53 66 
54 if (ele.srcBufs.begin()->rankId != 0 or ele.srcBufs.rbegin()->rankId != static_cast<int>(rankSize - 1)) {67 if (ele.srcBufs.begin()->rankId != 0 or ele.srcBufs.rbegin()->rankId != static_cast<int>(rankSize - 1)) {
55- HCCL_ERROR("[rankId:%u]cur buffer semantic srcBufs is invalid, cur buffer semantic is %s",68+ HCCL_VM_ERROR("{} Source rank list for this ReduceScatter result range is not the "
56- rankId, ele.Describe().c_str());69+ "complete expected rank set, rankId={}, firstSourceRank={}, lastSourceRank={}, "
70+ "expectedFirstSourceRank=0, expectedLastSourceRank={}, outputRange=[0x{:x},0x{:x})"
71+ "\nCurrent result range detail:\n{}",
72+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_SRC_ERROR), rankId,
73+ ele.srcBufs.begin()->rankId, ele.srcBufs.rbegin()->rankId, rankSize - 1, ele.startAddr,
74+ rangeEnd, ele.Describe());
57 return HcclResult::HCCL_E_PARA;75 return HcclResult::HCCL_E_PARA;
58 }76 }
59 77 
60 for (auto &srcBuf : ele.srcBufs) {78 for (auto &srcBuf : ele.srcBufs) {
61 if (srcBuf.bufType != BufferType::INPUT) {79 if (srcBuf.bufType != BufferType::INPUT) {
62- HCCL_ERROR("[rankId:%u]Cur buffer semantic srcBufs bufType is not INPUT, cur buffer semantic is %s",80+ HCCL_VM_ERROR("{} This ReduceScatter result range comes from a non-INPUT "
63- rankId, ele.Describe().c_str());81+ "buffer, but it should come from INPUT, rankId={}, actualSourceRank={}, "
82+ "actualSourceBufferType={}, actualBufferRange=[0x{:x},0x{:x})"
83+ "\nCurrent result range detail:\n{}",
84+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_SRC_ERROR), rankId, srcBuf.rankId,
85+ BufferTypeToString(srcBuf.bufType), ele.startAddr, rangeEnd, ele.Describe());
64 return HcclResult::HCCL_E_PARA;86 return HcclResult::HCCL_E_PARA;
65 }87 }
66 88 
67 if (srcBuf.srcAddr != rankId * dataSize + totalSize) {89 if (srcBuf.srcAddr != rankId * dataSize + totalSize) {
68- HCCL_ERROR("[rankId:%u]Expected semantic srcBuf srcAddr is %llu, while cur srcBuf srcAddr is %llu, cur buffer semantic is %s",90+ HCCL_VM_ERROR("{} Source address for this ReduceScatter result range does not "
69- rankId, rankId * dataSize + totalSize, srcBuf.srcAddr, ele.Describe().c_str());91+ "match the expected input address, rankId={}, sourceRank={}, expectedAddr=0x{:x}, "
92+ "actualAddr=0x{:x}, actualBufferRange=[0x{:x},0x{:x})\nCurrent result range detail:\n{}",
93+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_SRC_ERROR), rankId, srcBuf.rankId,
94+ rankId * dataSize + totalSize, srcBuf.srcAddr, ele.startAddr, rangeEnd, ele.Describe());
70 return HcclResult::HCCL_E_PARA;95 return HcclResult::HCCL_E_PARA;
71 }96 }
72 }97 }
73 totalSize += ele.size;98 totalSize += ele.size;
74 }99 }
75 if (totalSize != dataSize) {100 if (totalSize != dataSize) {
76- HCCL_ERROR("[rankId:%u]Missing buffer semantics in tail: already checked total size is %llu, while expected total size is %llu", rankId, totalSize, dataSize);101+ HCCL_VM_ERROR("{} ReduceScatter result data ends before the expected total size is "
102+ "reached, rankId={}, checkedSize=0x{:x}, expectedSize=0x{:x}",
103+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_MISSING), rankId, totalSize, dataSize);
77 return HcclResult::HCCL_E_PARA;104 return HcclResult::HCCL_E_PARA;
78 }105 }
79 }106 }
Mtest/hccl_vm/src/plugin/checker/src/framework/semantics_check/reduce_scatter_v_semantics_checker.cc+45-21
@@ -15,7 +15,9 @@
15 15 
16#include "base.h"16#include "base.h"
17#include "checker_def.h"17#include "checker_def.h"
18-#include "log.h"18+#include "sim_log.h"
19+#include "utils/dump/dump_json_utils.h"
20+#include "utils/error_codes.h"
19 21 
20namespace HcclSim {22namespace HcclSim {
21HcclResult TaskCheckReduceScatterVSemantics(std::map<RankId, RankMemorySemantics> &allRankMemSemantics,23HcclResult TaskCheckReduceScatterVSemantics(std::map<RankId, RankMemorySemantics> &allRankMemSemantics,
@@ -27,51 +29,72 @@ HcclResult TaskCheckReduceScatterVSemantics(std::map<RankId, RankMemorySemantics
27 for (u32 rankId = 0; rankId < rankSize; rankId++) {29 for (u32 rankId = 0; rankId < rankSize; rankId++) {
28 // 对应的rank不存在需要报错30 // 对应的rank不存在需要报错
29 if (allRankMemSemantics.count(rankId) == 0) {31 if (allRankMemSemantics.count(rankId) == 0) {
30- HCCL_ERROR("Missing rank %d mem semantics", rankId);32+ HCCL_VM_ERROR("{} ReduceScatterV produced no result data for rank {}, but this rank is "
33+ "expected to have one reduced shard, expectedSourceRankCount={}, expectedResultSize=0x{:x}",
34+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_MISSING), rankId, rankSize,
35+ vDataDes.counts[rankId] * CHECK_SIZE_TABLE[vDataDes.dataType]);
31 return HcclResult::HCCL_E_PARA;36 return HcclResult::HCCL_E_PARA;
32 }37 }
33 38 
34 u64 totalSize = 0;39 u64 totalSize = 0;
35 for (auto &ele : allRankMemSemantics[rankId][BufferType::OUTPUT]) {40 for (auto &ele : allRankMemSemantics[rankId][BufferType::OUTPUT]) {
41+ const u64 rangeEnd = ele.startAddr + ele.size;
36 if (ele.startAddr != totalSize) {42 if (ele.startAddr != totalSize) {
37- HCCL_ERROR("[rankId:%u]Missing buffer semantic: exepected startAddr is %llu, "43+ HCCL_VM_ERROR("{} ReduceScatterV result data does not start from the expected "
38- "while cur buffer semantic startAddr is %llu, cur buffer semantic is %s",44+ "address, rankId={}, expectedStartAddr=0x{:x}, actualStartAddr=0x{:x}, "
39- rankId, totalSize, ele.startAddr, ele.Describe().c_str());45+ "actualBufferRange=[0x{:x},0x{:x})\nCurrent result range detail:\n{}",
46+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_MISSING), rankId, totalSize, ele.startAddr,
47+ ele.startAddr, rangeEnd, ele.Describe());
40 return HcclResult::HCCL_E_PARA;48 return HcclResult::HCCL_E_PARA;
41 }49 }
42 50 
43 if (ele.srcBufs.size() > 1 && ele.reduceType != reduceType) {51 if (ele.srcBufs.size() > 1 && ele.reduceType != reduceType) {
44- HCCL_ERROR("[rankId:%u]cur buffer semantic reduceType %d is unequal to expected reduceType %d, "52+ HCCL_VM_ERROR("{} Reduce mode of this ReduceScatterV result range does not match "
45- "cur buffer semantic is %s",53+ "the operator setting, rankId={}, actualReduceType={}, expectedReduceType={}, "
46- rankId, ele.reduceType, reduceType, ele.Describe().c_str());54+ "outputRange=[0x{:x},0x{:x})\nCurrent result range detail:\n{}",
55+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_REDUCE_ERROR), rankId,
56+ DumpReduceOpToString(ele.reduceType), DumpReduceOpToString(reduceType),
57+ ele.startAddr, rangeEnd, ele.Describe());
47 return HcclResult::HCCL_E_PARA;58 return HcclResult::HCCL_E_PARA;
48 }59 }
49 60 
50 if (ele.srcBufs.size() != rankSize) {61 if (ele.srcBufs.size() != rankSize) {
51- HCCL_ERROR("[rankId:%u]buffer semantic srcBufs size %u is unequal to rankSize %u, "62+ HCCL_VM_ERROR("{} This ReduceScatterV result range does not include the expected "
52- "cur buffer semantic is %s",63+ "number of source ranks, rankId={}, actualSourceRankCount={}, expectedRankSize={}, "
53- rankId, ele.srcBufs.size(), rankSize, ele.Describe().c_str());64+ "outputRange=[0x{:x},0x{:x})\nCurrent result range detail:\n{}",
65+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_REDUCE_ERROR), rankId, ele.srcBufs.size(),
66+ rankSize, ele.startAddr, rangeEnd, ele.Describe());
54 return HcclResult::HCCL_E_PARA;67 return HcclResult::HCCL_E_PARA;
55 }68 }
56 if (ele.srcBufs.begin()->rankId != 0 or ele.srcBufs.rbegin()->rankId != static_cast<int>(rankSize - 1)) {69 if (ele.srcBufs.begin()->rankId != 0 or ele.srcBufs.rbegin()->rankId != static_cast<int>(rankSize - 1)) {
57- HCCL_ERROR("[rankId:%u]cur buffer semantic srcBufs is invalid, cur buffer semantic is %s",70+ HCCL_VM_ERROR("{} Source rank list for this ReduceScatterV result range is not the "
58- rankId, ele.Describe().c_str());71+ "complete expected rank set, rankId={}, firstSourceRank={}, lastSourceRank={}, "
72+ "expectedFirstSourceRank=0, expectedLastSourceRank={}, outputRange=[0x{:x},0x{:x})"
73+ "\nCurrent result range detail:\n{}",
74+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_SRC_ERROR), rankId,
75+ ele.srcBufs.begin()->rankId, ele.srcBufs.rbegin()->rankId, rankSize - 1, ele.startAddr,
76+ rangeEnd, ele.Describe());
59 return HcclResult::HCCL_E_PARA;77 return HcclResult::HCCL_E_PARA;
60 }78 }
61 79 
62 for (auto &srcBuf : ele.srcBufs) {80 for (auto &srcBuf : ele.srcBufs) {
63 if (srcBuf.bufType != BufferType::INPUT) {81 if (srcBuf.bufType != BufferType::INPUT) {
64- HCCL_ERROR("[rankId:%u]Cur buffer semantic srcBufs bufType is not INPUT, "82+ HCCL_VM_ERROR("{} This ReduceScatterV result range comes from a non-INPUT "
65- "cur buffer semantic is %s",83+ "buffer, but it should come from INPUT, rankId={}, actualSourceRank={}, "
66- rankId, ele.Describe().c_str());84+ "actualSourceBufferType={}, actualBufferRange=[0x{:x},0x{:x})"
85+ "\nCurrent result range detail:\n{}",
86+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_SRC_ERROR), rankId, srcBuf.rankId,
87+ BufferTypeToString(srcBuf.bufType), ele.startAddr, rangeEnd, ele.Describe());
67 HcclResult::HCCL_E_PARA;88 HcclResult::HCCL_E_PARA;
68 }89 }
69 90 
70 const u64 outputOffset = vDataDes.displs[rankId] * CHECK_SIZE_TABLE[vDataDes.dataType];91 const u64 outputOffset = vDataDes.displs[rankId] * CHECK_SIZE_TABLE[vDataDes.dataType];
71 if (srcBuf.srcAddr != outputOffset + totalSize) {92 if (srcBuf.srcAddr != outputOffset + totalSize) {
72- HCCL_ERROR("[rankId:%u]Exepected semantic srcBuf srcAddr is %llu, "93+ HCCL_VM_ERROR("{} Source address for this ReduceScatterV result range does not "
73- "while cur srcBuf srcAddr is %llu, cur buffer semantic is %s",94+ "match the expected input address, rankId={}, sourceRank={}, expectedAddr=0x{:x}, "
74- rankId, outputOffset + totalSize, srcBuf.srcAddr, ele.Describe().c_str());95+ "actualAddr=0x{:x}, actualBufferRange=[0x{:x},0x{:x})\nCurrent result range detail:\n{}",
96+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_SRC_ERROR), rankId, srcBuf.rankId,
97+ outputOffset + totalSize, srcBuf.srcAddr, ele.startAddr, rangeEnd, ele.Describe());
75 return HcclResult::HCCL_E_PARA;98 return HcclResult::HCCL_E_PARA;
76 }99 }
77 }100 }
@@ -80,8 +103,9 @@ HcclResult TaskCheckReduceScatterVSemantics(std::map<RankId, RankMemorySemantics
80 outputSize = vDataDes.counts[rankId] * CHECK_SIZE_TABLE[vDataDes.dataType];103 outputSize = vDataDes.counts[rankId] * CHECK_SIZE_TABLE[vDataDes.dataType];
81 104 
82 if (totalSize != outputSize) {105 if (totalSize != outputSize) {
83- HCCL_ERROR("[rankId:%u]Missing buffer semantics in tail: already checked total size is %llu, "106+ HCCL_VM_ERROR("{} ReduceScatterV result data ends before the expected total size is "
84- "while expected output size is %lu", rankId, totalSize, outputSize);107+ "reached, rankId={}, checkedSize=0x{:x}, expectedSize=0x{:x}",
108+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_MISSING), rankId, totalSize, outputSize);
85 return HcclResult::HCCL_E_PARA;109 return HcclResult::HCCL_E_PARA;
86 }110 }
87 }111 }
Mtest/hccl_vm/src/plugin/checker/src/framework/semantics_check/reduce_semantics_checker.cc+42-14
@@ -14,6 +14,9 @@
14 14 
15#include "base.h"15#include "base.h"
16#include "check_utils.h"16#include "check_utils.h"
17+#include "sim_log.h"
18+#include "utils/dump/dump_json_utils.h"
19+#include "utils/error_codes.h"
17 20 
18namespace HcclSim {21namespace HcclSim {
19HcclResult TaskCheckReduceSemantics(std::map<RankId, RankMemorySemantics> &allRankMemSemantics, u64 dataSize,22HcclResult TaskCheckReduceSemantics(std::map<RankId, RankMemorySemantics> &allRankMemSemantics, u64 dataSize,
@@ -26,53 +29,78 @@ HcclResult TaskCheckReduceSemantics(std::map<RankId, RankMemorySemantics> &allRa
26 29 
27 // 对应的rank不存在需要报错30 // 对应的rank不存在需要报错
28 if (allRankMemSemantics.count(root) == 0 && dataSize > 0) {31 if (allRankMemSemantics.count(root) == 0 && dataSize > 0) {
29- HCCL_ERROR("Missing rank %d mem semantics", root);32+ HCCL_VM_ERROR("{} Reduce produced no result data for root rank {}, but this rank is "
33+ "expected to hold the reduced result, expectedSourceRankCount={}, expectedResultSize=0x{:x}",
34+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_MISSING), root, rankSize, dataSize);
30 return HcclResult::HCCL_E_PARA;35 return HcclResult::HCCL_E_PARA;
31 }36 }
32 37 
33 u64 totalSize = 0;38 u64 totalSize = 0;
34 for (auto &ele : allRankMemSemantics[root][BufferType::OUTPUT]) {39 for (auto &ele : allRankMemSemantics[root][BufferType::OUTPUT]) {
40+ const u64 rangeEnd = ele.startAddr + ele.size;
35 if (ele.startAddr != totalSize) {41 if (ele.startAddr != totalSize) {
36- HCCL_ERROR("[rankId:%u]Missing buffer semantic: expected startAddr is %llu, while cur buffer semantic startAddr is %llu, cur buffer semantic is %s",42+ HCCL_VM_ERROR("{} Reduce result data does not start from the expected address, "
37- root, totalSize, ele.startAddr, ele.Describe().c_str());43+ "rankId={}, expectedStartAddr=0x{:x}, actualStartAddr=0x{:x}, actualBufferRange=[0x{:x},0x{:x})"
44+ "\nCurrent result range detail:\n{}",
45+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_MISSING), root, totalSize, ele.startAddr,
46+ ele.startAddr, rangeEnd, ele.Describe());
38 return HcclResult::HCCL_E_PARA;47 return HcclResult::HCCL_E_PARA;
39 }48 }
40 49 
41 if (ele.srcBufs.size() > 1 && ele.reduceType != reduceType) {50 if (ele.srcBufs.size() > 1 && ele.reduceType != reduceType) {
42- HCCL_ERROR("[rankId:%u]cur buffer semantic reduceType %d is unequal to expected reduceType %d, cur buffer semantic is %s",51+ HCCL_VM_ERROR("{} Reduce mode of this Reduce result range does not match the operator "
43- root, ele.reduceType, reduceType, ele.Describe().c_str());52+ "setting, rankId={}, actualReduceType={}, expectedReduceType={}, outputRange=[0x{:x},0x{:x})"
53+ "\nCurrent result range detail:\n{}",
54+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_REDUCE_ERROR), root,
55+ DumpReduceOpToString(ele.reduceType), DumpReduceOpToString(reduceType), ele.startAddr, rangeEnd,
56+ ele.Describe());
44 return HcclResult::HCCL_E_PARA;57 return HcclResult::HCCL_E_PARA;
45 }58 }
46 59 
47 if (ele.srcBufs.size() != rankSize) {60 if (ele.srcBufs.size() != rankSize) {
48- HCCL_ERROR("[rankId:%u]buffer semantic srcBufs size %u is unequal to rankSize %u, cur buffer semantic is %s",61+ HCCL_VM_ERROR("{} This Reduce result range does not include the expected number of "
49- root, ele.srcBufs.size(), rankSize, ele.Describe().c_str());62+ "source ranks, rankId={}, actualSourceRankCount={}, expectedRankSize={}, "
63+ "outputRange=[0x{:x},0x{:x})\nCurrent result range detail:\n{}",
64+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_REDUCE_ERROR), root, ele.srcBufs.size(), rankSize,
65+ ele.startAddr, rangeEnd, ele.Describe());
50 return HcclResult::HCCL_E_PARA;66 return HcclResult::HCCL_E_PARA;
51 }67 }
52 68 
53 if (ele.srcBufs.begin()->rankId != 0 or ele.srcBufs.rbegin()->rankId != (rankSize - 1)) {69 if (ele.srcBufs.begin()->rankId != 0 or ele.srcBufs.rbegin()->rankId != (rankSize - 1)) {
54- HCCL_ERROR("[rankId:%u]cur buffer semantic srcBufs is invalid, cur buffer semantic is %s",70+ HCCL_VM_ERROR("{} Source rank list for this Reduce result range is not the complete "
55- root, ele.Describe().c_str());71+ "expected rank set, rankId={}, firstSourceRank={}, lastSourceRank={}, expectedFirstSourceRank=0, "
72+ "expectedLastSourceRank={}, outputRange=[0x{:x},0x{:x})\nCurrent result range detail:\n{}",
73+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_SRC_ERROR), root,
74+ ele.srcBufs.begin()->rankId, ele.srcBufs.rbegin()->rankId, rankSize - 1, ele.startAddr, rangeEnd,
75+ ele.Describe());
56 return HcclResult::HCCL_E_PARA;76 return HcclResult::HCCL_E_PARA;
57 }77 }
58 78 
59 for (auto &srcBuf : ele.srcBufs) {79 for (auto &srcBuf : ele.srcBufs) {
60 if (srcBuf.bufType != BufferType::INPUT) {80 if (srcBuf.bufType != BufferType::INPUT) {
61- HCCL_ERROR("[rankId:%u]Cur buffer semantic srcBufs bufType is not INPUT, cur buffer semantic is %s",81+ HCCL_VM_ERROR("{} This Reduce result range comes from a non-INPUT buffer, but it "
62- root, ele.Describe().c_str());82+ "should come from INPUT, rankId={}, actualSourceRank={}, actualSourceBufferType={}, "
83+ "actualBufferRange=[0x{:x},0x{:x})\nCurrent result range detail:\n{}",
84+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_SRC_ERROR), root, srcBuf.rankId,
85+ BufferTypeToString(srcBuf.bufType), ele.startAddr, rangeEnd, ele.Describe());
63 return HcclResult::HCCL_E_PARA;86 return HcclResult::HCCL_E_PARA;
64 }87 }
65 88 
66 if (srcBuf.srcAddr != totalSize) {89 if (srcBuf.srcAddr != totalSize) {
67- HCCL_ERROR("[rankId:%u]Expected semantic srcBuf srcAddr is %llu, while cur srcBuf srcAddr is %llu, cur buffer semantic is %s",90+ HCCL_VM_ERROR("{} Source address for this Reduce result range does not match the "
68- root, totalSize, srcBuf.srcAddr, ele.Describe().c_str());91+ "expected input address, rankId={}, sourceRank={}, expectedAddr=0x{:x}, actualAddr=0x{:x}, "
92+ "actualBufferRange=[0x{:x},0x{:x})\nCurrent result range detail:\n{}",
93+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_SRC_ERROR), root, srcBuf.rankId, totalSize,
94+ srcBuf.srcAddr, ele.startAddr, rangeEnd, ele.Describe());
69 return HcclResult::HCCL_E_PARA;95 return HcclResult::HCCL_E_PARA;
70 }96 }
71 }97 }
72 totalSize += ele.size;98 totalSize += ele.size;
73 }99 }
74 if (totalSize != dataSize) {100 if (totalSize != dataSize) {
75- HCCL_ERROR("[rankId:%u]Missing buffer semantics in tail: already checked total size is %llu, while expected total size is %llu", root, totalSize, dataSize);101+ HCCL_VM_ERROR("{} Reduce result data ends before the expected total size is reached, "
102+ "rankId={}, checkedSize=0x{:x}, expectedSize=0x{:x}",
103+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_MISSING), root, totalSize, dataSize);
76 return HcclResult::HCCL_E_PARA;104 return HcclResult::HCCL_E_PARA;
77 }105 }
78 106 
Mtest/hccl_vm/src/plugin/checker/src/framework/semantics_check/scatter_semantics_checker.cc+37-12
@@ -14,6 +14,9 @@
14 14 
15#include "base.h"15#include "base.h"
16#include "check_utils.h"16#include "check_utils.h"
17+#include "sim_log.h"
18+#include "utils/dump/dump_json_utils.h"
19+#include "utils/error_codes.h"
17 20 
18namespace HcclSim {21namespace HcclSim {
19HcclResult TaskCheckScatterSemantics(std::map<RankId, RankMemorySemantics> &allRankMemSemantics, u64 dataSize,22HcclResult TaskCheckScatterSemantics(std::map<RankId, RankMemorySemantics> &allRankMemSemantics, u64 dataSize,
@@ -24,45 +27,67 @@ HcclResult TaskCheckScatterSemantics(std::map<RankId, RankMemorySemantics> &allR
24 for (RankId rankId = 0; rankId < rankSize; rankId++) {27 for (RankId rankId = 0; rankId < rankSize; rankId++) {
25 // 对应的rank不存在需要报错28 // 对应的rank不存在需要报错
26 if (allRankMemSemantics.count(rankId) == 0) {29 if (allRankMemSemantics.count(rankId) == 0) {
27- HCCL_ERROR("Missing rank %d mem semantics", rankId);30+ HCCL_VM_ERROR("{} Scatter produced no result data for rank {}, but this rank is "
31+ "expected to receive one shard from root rank {}, expectedResultSize=0x{:x}",
32+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_MISSING), rankId, root, dataSize);
28 return HcclResult::HCCL_E_PARA;33 return HcclResult::HCCL_E_PARA;
29 }34 }
30 35 
31 u64 totalSize = 0;36 u64 totalSize = 0;
32 for (auto &ele : allRankMemSemantics[rankId][BufferType::OUTPUT]) {37 for (auto &ele : allRankMemSemantics[rankId][BufferType::OUTPUT]) {
38+ const u64 rangeEnd = ele.startAddr + ele.size;
33 if (ele.startAddr != totalSize) {39 if (ele.startAddr != totalSize) {
34- HCCL_ERROR("[rankId:%u]Missing buffer semantic: exepected startAddr is %llu, while cur buffer semantic startAddr is %llu, cur buffer semantic is %s",40+ HCCL_VM_ERROR("{} Scatter result data does not start from the expected address, "
35- rankId, totalSize, ele.startAddr, ele.Describe().c_str());41+ "rankId={}, expectedStartAddr=0x{:x}, actualStartAddr=0x{:x}, actualBufferRange=[0x{:x},0x{:x})"
42+ "\nCurrent result range detail:\n{}",
43+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_MISSING), rankId, totalSize, ele.startAddr,
44+ ele.startAddr, rangeEnd, ele.Describe());
36 return HcclResult::HCCL_E_PARA;45 return HcclResult::HCCL_E_PARA;
37 }46 }
38 47 
39 if (ele.srcBufs.size() != 1) {48 if (ele.srcBufs.size() != 1) {
40- HCCL_ERROR("[rankId:%u]Cur buffer semantic should not be reduce, which mean srcBufs size should be 1, while cur buffer semantic is %s", rankId, ele.Describe().c_str());49+ HCCL_VM_ERROR("{} This Scatter result range combines multiple sources, but this "
50+ "operator expects exactly one source, rankId={}, actualSourceCount={}, expectedSourceCount=1, "
51+ "outputRange=[0x{:x},0x{:x})\nCurrent result range detail:\n{}",
52+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_REDUCE_ERROR), rankId, ele.srcBufs.size(),
53+ ele.startAddr, rangeEnd, ele.Describe());
41 return HcclResult::HCCL_E_PARA;54 return HcclResult::HCCL_E_PARA;
42 }55 }
43 56 
44 if (ele.srcBufs.begin()->rankId != root) {57 if (ele.srcBufs.begin()->rankId != root) {
45- HCCL_ERROR("[rankId:%u]Buffer semantic srcBuf rank[%u] is not from root[%u], cur buffer semantic is %s, cur rank is %u",58+ HCCL_VM_ERROR("{} This Scatter result range comes from the wrong source rank, "
46- rankId, ele.srcBufs.begin()->rankId, root, ele.Describe().c_str(), rankId);59+ "rankId={}, actualSourceRank={}, expectedSourceRank={}, actualBufferRange=[0x{:x},0x{:x})"
60+ "\nCurrent result range detail:\n{}",
61+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_SRC_ERROR), rankId,
62+ ele.srcBufs.begin()->rankId, root, ele.startAddr, rangeEnd, ele.Describe());
47 return HcclResult::HCCL_E_PARA;63 return HcclResult::HCCL_E_PARA;
48 }64 }
49 65 
50 if (ele.srcBufs.begin()->bufType != BufferType::INPUT) {66 if (ele.srcBufs.begin()->bufType != BufferType::INPUT) {
51- HCCL_ERROR("[rankId:%u]Cur buffer semantic srcBufs bufType is not INPUT, cur buffer semantic is %s",67+ HCCL_VM_ERROR("{} This Scatter result range comes from a non-INPUT buffer, but it "
52- rankId, ele.Describe().c_str());68+ "should come from INPUT, rankId={}, actualSourceRank={}, actualSourceBufferType={}, "
69+ "actualBufferRange=[0x{:x},0x{:x})\nCurrent result range detail:\n{}",
70+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_SRC_ERROR), rankId,
71+ ele.srcBufs.begin()->rankId, BufferTypeToString(ele.srcBufs.begin()->bufType),
72+ ele.startAddr, rangeEnd, ele.Describe());
53 return HcclResult::HCCL_E_PARA;73 return HcclResult::HCCL_E_PARA;
54 }74 }
55 75 
56 if (ele.srcBufs.begin()->srcAddr != rankId * dataSize + totalSize) {76 if (ele.srcBufs.begin()->srcAddr != rankId * dataSize + totalSize) {
57- HCCL_ERROR("[rankId:%u]Cur buffer semantic srcBufs srcAddr should be %llu,"77+ HCCL_VM_ERROR("{} Source address for this Scatter result range does not match the "
58- "while it is %llu, cur buffer semantic is %s",78+ "expected input address, rankId={}, sourceRank={}, expectedAddr=0x{:x}, actualAddr=0x{:x}, "
59- rankId, rankId * dataSize + totalSize, ele.srcBufs.begin()->srcAddr, ele.Describe().c_str());79+ "actualBufferRange=[0x{:x},0x{:x})\nCurrent result range detail:\n{}",
80+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_SRC_ERROR), rankId,
81+ ele.srcBufs.begin()->rankId, rankId * dataSize + totalSize, ele.srcBufs.begin()->srcAddr,
82+ ele.startAddr, rangeEnd, ele.Describe());
60 return HcclResult::HCCL_E_PARA;83 return HcclResult::HCCL_E_PARA;
61 }84 }
62 totalSize += ele.size;85 totalSize += ele.size;
63 }86 }
64 if (totalSize != dataSize) {87 if (totalSize != dataSize) {
65- HCCL_ERROR("[rankId:%u]Missing buffer semantics in tail: already checked total size is %llu, which should be %llu, cur rank is %u", rankId, totalSize, dataSize, rankId);88+ HCCL_VM_ERROR("{} Scatter result data ends before the expected total size is reached, "
89+ "rankId={}, checkedSize=0x{:x}, expectedSize=0x{:x}",
90+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_MISSING), rankId, totalSize, dataSize);
66 return HcclResult::HCCL_E_PARA;91 return HcclResult::HCCL_E_PARA;
67 }92 }
68 }93 }
Mtest/hccl_vm/src/plugin/checker/src/framework/semantics_check/send_recv_semantics_checker.cc+36-11
@@ -14,6 +14,9 @@
14 14 
15#include "base.h"15#include "base.h"
16#include "check_utils.h"16#include "check_utils.h"
17+#include "sim_log.h"
18+#include "utils/dump/dump_json_utils.h"
19+#include "utils/error_codes.h"
17 20 
18namespace HcclSim {21namespace HcclSim {
19HcclResult TaskCheckSendRecvSemantics(std::map<RankId, RankMemorySemantics> &allRankMemSemantics, u64 dataSize,22HcclResult TaskCheckSendRecvSemantics(std::map<RankId, RankMemorySemantics> &allRankMemSemantics, u64 dataSize,
@@ -23,43 +26,65 @@ HcclResult TaskCheckSendRecvSemantics(std::map<RankId, RankMemorySemantics> &all
23 26 
24 // 对应的rank不存在需要报错27 // 对应的rank不存在需要报错
25 if (allRankMemSemantics.size() != 2) {28 if (allRankMemSemantics.size() != 2) {
26- HCCL_ERROR("sendrecv only support 2 ranks");29+ HCCL_VM_ERROR("{} Send/Recv final output validation supports exactly 2 ranks, but got "
30+ "expectedRankSize=2, actualRankSize={}, sourceRank={}, targetRank={}",
31+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_CHECK_FAILED), rankSize, srcRank, dstRank);
27 return HcclResult::HCCL_E_PARA;32 return HcclResult::HCCL_E_PARA;
28 }33 }
29 34 
30 u64 totalSize = 0;35 u64 totalSize = 0;
31 for (auto &ele : allRankMemSemantics[dstRank][BufferType::OUTPUT]) {36 for (auto &ele : allRankMemSemantics[dstRank][BufferType::OUTPUT]) {
37+ const u64 rangeEnd = ele.startAddr + ele.size;
32 if (ele.startAddr != totalSize) {38 if (ele.startAddr != totalSize) {
33- HCCL_ERROR("[rankId:%u]Missing buffer semantic: expected startAddr is %llu, while cur buffer semantic startAddr is %llu, cur buffer semantic is %s",39+ HCCL_VM_ERROR("{} Send/Recv result data does not start from the expected address, "
34- dstRank, totalSize, ele.startAddr, ele.Describe().c_str());40+ "rankId={}, expectedStartAddr=0x{:x}, actualStartAddr=0x{:x}, actualBufferRange=[0x{:x},0x{:x})"
41+ "\nCurrent result range detail:\n{}",
42+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_MISSING), dstRank, totalSize, ele.startAddr,
43+ ele.startAddr, rangeEnd, ele.Describe());
35 return HcclResult::HCCL_E_PARA;44 return HcclResult::HCCL_E_PARA;
36 }45 }
37 46 
38 if (ele.srcBufs.size() != 1) {47 if (ele.srcBufs.size() != 1) {
39- HCCL_ERROR("[rankId:%u]Cur buffer semantic should not be reduce, which mean srcBufs size should be 1, while cur buffer semantic is %s", dstRank, ele.Describe().c_str());48+ HCCL_VM_ERROR("{} This Send/Recv result range combines multiple sources, but this "
49+ "operator expects exactly one source, rankId={}, actualSourceCount={}, expectedSourceCount=1, "
50+ "outputRange=[0x{:x},0x{:x})\nCurrent result range detail:\n{}",
51+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_REDUCE_ERROR), dstRank, ele.srcBufs.size(),
52+ ele.startAddr, rangeEnd, ele.Describe());
40 return HcclResult::HCCL_E_PARA;53 return HcclResult::HCCL_E_PARA;
41 }54 }
42 55 
43 if (ele.srcBufs.begin()->rankId != srcRank) {56 if (ele.srcBufs.begin()->rankId != srcRank) {
44- HCCL_ERROR("[rankId:%u]Buffer semantic srcBuf rank[%u] is not from srcRank[%u], cur buffer semantic is %s",57+ HCCL_VM_ERROR("{} This Send/Recv result range comes from the wrong source rank, "
45- dstRank, ele.srcBufs.begin()->rankId, srcRank, ele.Describe().c_str());58+ "rankId={}, actualSourceRank={}, expectedSourceRank={}, actualBufferRange=[0x{:x},0x{:x})"
59+ "\nCurrent result range detail:\n{}",
60+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_SRC_ERROR), dstRank,
61+ ele.srcBufs.begin()->rankId, srcRank, ele.startAddr, rangeEnd, ele.Describe());
46 return HcclResult::HCCL_E_PARA;62 return HcclResult::HCCL_E_PARA;
47 }63 }
48 64 
49 if (ele.srcBufs.begin()->bufType != BufferType::INPUT) {65 if (ele.srcBufs.begin()->bufType != BufferType::INPUT) {
50- HCCL_ERROR("[rankId:%u]Cur buffer semantic srcBufs bufType is not INPUT, cur buffer semantic is %s",66+ HCCL_VM_ERROR("{} This Send/Recv result range comes from a non-INPUT buffer, but it "
51- dstRank, ele.Describe().c_str());67+ "should come from INPUT, rankId={}, actualSourceRank={}, actualSourceBufferType={}, "
68+ "actualBufferRange=[0x{:x},0x{:x})\nCurrent result range detail:\n{}",
69+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_SRC_ERROR), dstRank,
70+ ele.srcBufs.begin()->rankId, BufferTypeToString(ele.srcBufs.begin()->bufType),
71+ ele.startAddr, rangeEnd, ele.Describe());
52 return HcclResult::HCCL_E_PARA;72 return HcclResult::HCCL_E_PARA;
53 }73 }
54 if (ele.srcBufs.begin()->srcAddr != totalSize) {74 if (ele.srcBufs.begin()->srcAddr != totalSize) {
55- HCCL_ERROR("[rankId:%u]Cur buffer semantic srcBufs srcAddr should be %llu, while it is %llu, cur buffer semantic is %s",75+ HCCL_VM_ERROR("{} Source address for this Send/Recv result range does not match the "
56- dstRank, totalSize, ele.srcBufs.begin()->srcAddr, ele.Describe().c_str());76+ "expected input address, rankId={}, sourceRank={}, expectedAddr=0x{:x}, actualAddr=0x{:x}, "
77+ "actualBufferRange=[0x{:x},0x{:x})\nCurrent result range detail:\n{}",
78+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_SRC_ERROR), dstRank, ele.srcBufs.begin()->rankId,
79+ totalSize, ele.srcBufs.begin()->srcAddr, ele.startAddr, rangeEnd, ele.Describe());
57 return HcclResult::HCCL_E_PARA;80 return HcclResult::HCCL_E_PARA;
58 }81 }
59 totalSize += ele.size;82 totalSize += ele.size;
60 }83 }
61 if (totalSize != dataSize) {84 if (totalSize != dataSize) {
62- HCCL_ERROR("[rankId:%u]Missing buffer semantics in tail: already checked total size is %llu, which should be %llu", dstRank, totalSize, dataSize);85+ HCCL_VM_ERROR("{} Send/Recv result data ends before the expected total size is reached, "
86+ "rankId={}, checkedSize=0x{:x}, expectedSize=0x{:x}",
87+ MakeErrorCodeText(ErrorCode::SEMANTIC_FINAL_MISSING), dstRank, totalSize, dataSize);
63 return HcclResult::HCCL_E_PARA;88 return HcclResult::HCCL_E_PARA;
64 }89 }
65 90 
Mtest/hccl_vm/src/plugin/checker/src/framework/task_graph_generator/ccu_all_rank_param_recorder.cc+1-1
@@ -105,7 +105,7 @@ HcclResult AllRankParamRecorder::CheckAllPostMatch()
105 for (const auto& regPair: diePair.second) {105 for (const auto& regPair: diePair.second) {
106 uint16_t regId = regPair.first;106 uint16_t regId = regPair.first;
107 for (auto& post : seenPost[rank][dieId][regId]) {107 for (auto& post : seenPost[rank][dieId][regId]) {
108- HCCL_WARNING("unmatched LocalPost/Post: {}", post->task->Describe().c_str());108+ HCCL_WARNING("unmatched LocalPost/Post: %s", post->task->Describe().c_str());
109 }109 }
110 }110 }
111 }111 }
Mtest/hccl_vm/src/plugin/checker/src/framework/task_graph_generator/ccu_task_transform.cc+5-5
@@ -118,7 +118,7 @@ void GenWaitNode(TaskStubCcuGraph *curCcuTask, uint32_t queId, uint16_t waitCKEI
118HcclResult ProcessWaitMask(RankId rankId, uint32_t dieId, TaskStubCcuGraph *curCcuTask, uint32_t queId,118HcclResult ProcessWaitMask(RankId rankId, uint32_t dieId, TaskStubCcuGraph *curCcuTask, uint32_t queId,
119 uint16_t waitCKEId, uint16_t waitCKEMask, bool& isContinue)119 uint16_t waitCKEId, uint16_t waitCKEMask, bool& isContinue)
120{120{
121- HCCL_VM_DEBUG("[ProcessWaitMask] Enter...rank {:d}, die{:d}, que{:d}, waitCke{:d}, waitMask{:d}",121+ HCCL_VM_DEBUG("Enter...rank {:d}, die{:d}, que{:d}, waitCke{:d}, waitMask{:d}",
122 rankId, static_cast<uint32_t>(dieId), queId, waitCKEId, waitCKEMask);122 rankId, static_cast<uint32_t>(dieId), queId, waitCKEId, waitCKEMask);
123 if (waitCKEMask != 0x0000) {123 if (waitCKEMask != 0x0000) {
124 uint16_t ckeValue = 0;124 uint16_t ckeValue = 0;
@@ -126,7 +126,7 @@ HcclResult ProcessWaitMask(RankId rankId, uint32_t dieId, TaskStubCcuGraph *curC
126 // 条件不满足,继续等待126 // 条件不满足,继续等待
127 if ((ckeValue & waitCKEMask) != waitCKEMask) {127 if ((ckeValue & waitCKEMask) != waitCKEMask) {
128 isContinue = 0;128 isContinue = 0;
129- HCCL_VM_DEBUG("[ProcessWaitMask] Does not meet the condition, continue: ckeId: {:d}, ckeValue: {:d}, mask: {:x}", waitCKEId, ckeValue, waitCKEMask);129+ HCCL_VM_DEBUG("Does not meet the condition, continue: ckeId: {:d}, ckeValue: {:d}, mask: {:x}", waitCKEId, ckeValue, waitCKEMask);
130 return HCCL_SUCCESS;130 return HCCL_SUCCESS;
131 } else {131 } else {
132 GenWaitNode(curCcuTask, queId, waitCKEId, waitCKEMask);132 GenWaitNode(curCcuTask, queId, waitCKEId, waitCKEMask);
@@ -216,7 +216,7 @@ HcclResult TransformInstrQue(TaskNodePtr node, TaskStubCcuGraph *curCcuTask, uin
216 AddCcuSubGraphEnd(node, curCcuTask);216 AddCcuSubGraphEnd(node, curCcuTask);
217 }217 }
218 218 
219- HCCL_VM_DEBUG("[TransformInstrQue] end...");219+ HCCL_VM_DEBUG("end...");
220 return HCCL_SUCCESS;220 return HCCL_SUCCESS;
221}221}
222 222 
@@ -360,7 +360,7 @@ HcclResult GenCcuGraph(TaskNode* dummyStart) {
360 AllRankParamRecorder::Global()->InitParam();360 AllRankParamRecorder::Global()->InitParam();
361 HcclSim::StorageManager::GetInstance().InitCcuInfo(AllRankParamRecorder::Global()->devType_,361 HcclSim::StorageManager::GetInstance().InitCcuInfo(AllRankParamRecorder::Global()->devType_,
362 AllRankParamRecorder::Global()->ccu_resource_base_addr_);362 AllRankParamRecorder::Global()->ccu_resource_base_addr_);
363- HCCL_VM_INFO("[GenCcuGraph] dummyStart children size: {}", dummyStart->children.size());363+ HCCL_VM_INFO("dummyStart children size: {}", dummyStart->children.size());
364 364 
365 bool isPrintCcuGraph = std::getenv("CCU_TASK_PRINT");365 bool isPrintCcuGraph = std::getenv("CCU_TASK_PRINT");
366 completedNodes.insert(dummyStart);366 completedNodes.insert(dummyStart);
@@ -370,7 +370,7 @@ HcclResult GenCcuGraph(TaskNode* dummyStart) {
370 candNode.push_back(child);370 candNode.push_back(child);
371 }371 }
372 372 
373- HCCL_VM_INFO("[GenCcuGraph] start......{}", candNode.size());373+ HCCL_VM_INFO("start......{}", candNode.size());
374 u32 unmatchedCnt = 0;374 u32 unmatchedCnt = 0;
375 while(!candNode.empty()) {375 while(!candNode.empty()) {
376 // 先判断是否存在死锁的情况376 // 先判断是否存在死锁的情况
Mtest/hccl_vm/src/plugin/checker/src/framework/task_graph_generator/ccu_task_transform_instruct_a5.cc+9-9
@@ -386,7 +386,7 @@ HcclResult TransformTransLocMemToRmtMemInstr(const CcuRep::CcuInstr *instr, Task
386 CHK_RET(StorageManager::GetInstance().GetSlice(rmtAddr, len, dstSlice, &rId));386 CHK_RET(StorageManager::GetInstance().GetSlice(rmtAddr, len, dstSlice, &rId));
387 387 
388 RankId rmtRankId = g_allRankChannelInfo[rankId][dieId][channelId].dstRank;388 RankId rmtRankId = g_allRankChannelInfo[rankId][dieId][channelId].dstRank;
389- HCCL_VM_DEBUG("[TransformTransLocMemToRmtMemInstr] curRank= {}, dstRank= {}", rankId, rmtRankId);389+ HCCL_VM_DEBUG("curRank= {}, dstRank= {}", rankId, rmtRankId);
390 // 说明transportId写错了390 // 说明transportId写错了
391 if (rId != rmtRankId) {391 if (rId != rmtRankId) {
392 HCCL_ERROR("remoteRankId calculated by rmtAddr and channelId %u is not equal,"392 HCCL_ERROR("remoteRankId calculated by rmtAddr and channelId %u is not equal,"
@@ -523,12 +523,12 @@ HcclResult TransformSyncCKEInstr(const CcuRep::CcuInstr *instr, TaskStubCcuGraph
523 uint16_t ckeValue = 0;523 uint16_t ckeValue = 0;
524 uint16_t localCkeValue = 0;524 uint16_t localCkeValue = 0;
525 CHK_RET(AllRankParamRecorder::Global()->GetCKE(rankId, dieId, locCKEId, localCkeValue));525 CHK_RET(AllRankParamRecorder::Global()->GetCKE(rankId, dieId, locCKEId, localCkeValue));
526- HCCL_VM_DEBUG("[TransformSyncCKEInstr] Get loc rank: {:d}, die: {:d} CKE: {:d}, value: {:d}, mask: {:x}", rankId, static_cast<uint32_t>(dieId), locCKEId, localCkeValue, locCKEMask);526+ HCCL_VM_DEBUG("Get loc rank: {:d}, die: {:d} CKE: {:d}, value: {:d}, mask: {:x}", rankId, static_cast<uint32_t>(dieId), locCKEId, localCkeValue, locCKEMask);
527 localCkeValue = localCkeValue & locCKEMask;527 localCkeValue = localCkeValue & locCKEMask;
528 CHK_RET(AllRankParamRecorder::Global()->GetCKE(rmtRankId, rmtDieId, rmtCKEId, ckeValue));528 CHK_RET(AllRankParamRecorder::Global()->GetCKE(rmtRankId, rmtDieId, rmtCKEId, ckeValue));
529- HCCL_VM_DEBUG("[TransformSyncCKEInstr] Get rmt rank: {:d}, die: {:d} CKE: {:d}, value: {:d}, mask: {:x}", rmtRankId, static_cast<uint32_t>(rmtDieId), rmtCKEId, ckeValue, locCKEMask);529+ HCCL_VM_DEBUG("Get rmt rank: {:d}, die: {:d} CKE: {:d}, value: {:d}, mask: {:x}", rmtRankId, static_cast<uint32_t>(rmtDieId), rmtCKEId, ckeValue, locCKEMask);
530 CHK_RET(AllRankParamRecorder::Global()->SetCKE(rmtRankId, rmtDieId, rmtCKEId, ckeValue | localCkeValue));530 CHK_RET(AllRankParamRecorder::Global()->SetCKE(rmtRankId, rmtDieId, rmtCKEId, ckeValue | localCkeValue));
531- HCCL_VM_DEBUG("[TransformSyncCKEInstr] Set rank: {:d}, die: {:d} CKE: {:d}, value: {:d}, mask: {:x}", rmtRankId, static_cast<uint32_t>(rmtDieId), rmtCKEId, localCkeValue, locCKEMask);531+ HCCL_VM_DEBUG("Set rank: {:d}, die: {:d} CKE: {:d}, value: {:d}, mask: {:x}", rmtRankId, static_cast<uint32_t>(rmtDieId), rmtCKEId, localCkeValue, locCKEMask);
532 }532 }
533 533 
534 CHK_RET(ProcessSetMask(rankId, dieId, curCcuTask, queId, setCKEId, setCKEMask, true));534 CHK_RET(ProcessSetMask(rankId, dieId, curCcuTask, queId, setCKEId, setCKEMask, true));
@@ -1084,7 +1084,7 @@ HcclResult TransformTransRmtMemToLocMemInstr(const CcuRep::CcuInstr *instr, Task
1084 CHK_RET(StorageManager::GetInstance().GetSlice(localAddr, len, dstSlice));1084 CHK_RET(StorageManager::GetInstance().GetSlice(localAddr, len, dstSlice));
1085 1085 
1086 RankId rmtRankId = g_allRankChannelInfo[rankId][dieId][channelId].dstRank;1086 RankId rmtRankId = g_allRankChannelInfo[rankId][dieId][channelId].dstRank;
1087- HCCL_VM_DEBUG("[TransformTransRmtMemToLocMemInstr] curRank= {}, dstRank= {}", rankId, rmtRankId);1087+ HCCL_VM_DEBUG("curRank= {}, dstRank= {}", rankId, rmtRankId);
1088 // 说明transportId写错了1088 // 说明transportId写错了
1089 if (rId != rmtRankId) {1089 if (rId != rmtRankId) {
1090 HCCL_ERROR("remoteRankId calculated by rmtAddr and channelId %u is not equal,"1090 HCCL_ERROR("remoteRankId calculated by rmtAddr and channelId %u is not equal,"
@@ -1150,7 +1150,7 @@ HcclResult TransformLoopGroupInstr(const CcuRep::CcuInstr *instr, TaskStubCcuGra
1150 hcomm::CcuRep::CcuInstrInfo &microCodeQue = curCcuTask->instrInfo[queId];1150 hcomm::CcuRep::CcuInstrInfo &microCodeQue = curCcuTask->instrInfo[queId];
1151 auto loopGroupIdx = curCcuTask->loopGroupIdx++;1151 auto loopGroupIdx = curCcuTask->loopGroupIdx++;
1152 uint64_t loopCnt = loopGroupParam.loopGroupXn.loopInsCnt;1152 uint64_t loopCnt = loopGroupParam.loopGroupXn.loopInsCnt;
1153- HCCL_VM_DEBUG("[LoopGroup]loop cnt= {}, loop offset= {}, expand cnt= {}",1153+ HCCL_VM_DEBUG("loop cnt= {}, loop offset= {}, expand cnt= {}",
1154 loopCnt, static_cast<uint64_t>(loopGroupParam.loopGroupXn.expandOffset), static_cast<uint64_t>(loopGroupParam.loopGroupXn.expandCnt));1154 loopCnt, static_cast<uint64_t>(loopGroupParam.loopGroupXn.expandOffset), static_cast<uint64_t>(loopGroupParam.loopGroupXn.expandCnt));
1155 1155 
1156 for (u32 curLoopIdx = 0; curLoopIdx < loopCnt; curLoopIdx++) {1156 for (u32 curLoopIdx = 0; curLoopIdx < loopCnt; curLoopIdx++) {
@@ -1662,15 +1662,15 @@ HcclResult ProcessLoopIns(const CcuRep::CcuInstr *instr, TaskStubCcuGraph *curCc
1662 xm.value = value;1662 xm.value = value;
1663 loopGroupParam->loopXms.push_back(xm);1663 loopGroupParam->loopXms.push_back(xm);
1664 1664 
1665- HCCL_VM_DEBUG("[ProcessLoopIns] debug...{}, {}", loopGroupIdx, curCcuTask->loopGroupInfo_.size());1665+ HCCL_VM_DEBUG("debug...{}, {}", loopGroupIdx, curCcuTask->loopGroupInfo_.size());
1666 auto loopIdx = curCcuTask->loopIdx[loopGroupIdx]++;1666 auto loopIdx = curCcuTask->loopIdx[loopGroupIdx]++;
1667 auto loopStart = AddLoopStartTask(queId, loopIdx, loopGroupIdx, curCcuTask); // loop前新增loopStart标记节点1667 auto loopStart = AddLoopStartTask(queId, loopIdx, loopGroupIdx, curCcuTask); // loop前新增loopStart标记节点
1668 1668 
1669- HCCL_VM_DEBUG("[LOOP] loop cnt = {}", static_cast<uint64_t>(xm.loopCnt));1669+ HCCL_VM_DEBUG("loop cnt = {}", static_cast<uint64_t>(xm.loopCnt));
1670 hcomm::CcuRep::CcuInstrInfo& microCodeQue = curCcuTask->instrInfo[queId];1670 hcomm::CcuRep::CcuInstrInfo& microCodeQue = curCcuTask->instrInfo[queId];
1671 for (u32 curLoopCnt = 0; curLoopCnt < xm.loopCnt; curLoopCnt++) {1671 for (u32 curLoopCnt = 0; curLoopCnt < xm.loopCnt; curLoopCnt++) {
1672 loopGroupParam->curLoopCnt = curLoopCnt;1672 loopGroupParam->curLoopCnt = curLoopCnt;
1673- HCCL_VM_DEBUG("[LOOP] loop cnt = {}, cur loop= {}", static_cast<uint64_t>(xm.loopCnt), curLoopCnt);1673+ HCCL_VM_DEBUG("loop cnt = {}, cur loop= {}", static_cast<uint64_t>(xm.loopCnt), curLoopCnt);
1674 for (uint16_t insId = startInstrId; insId <= endInstrId; insId++) {1674 for (uint16_t insId = startInstrId; insId <= endInstrId; insId++) {
1675 // 获取当前要处理的指令1675 // 获取当前要处理的指令
1676 const CcuRep::CcuInstr* instr = &microCodeQue.instrVec[insId - curCcuTask->startInstrIdInQue[queId]];1676 const CcuRep::CcuInstr* instr = &microCodeQue.instrVec[insId - curCcuTask->startInstrIdInQue[queId]];
Mtest/hccl_vm/src/plugin/checker/src/framework/task_graph_generator/task_graph_generator.cc+6-6文件内容审核中,请稍后刷新重试
Mtest/hccl_vm/src/plugin/checker/src/framework/task_graph_generator/task_graph_generator_aiv.cc+3-3文件内容审核中,请稍后刷新重试
Mtest/hccl_vm/src/plugin/checker/src/framework/task_graph_generator_v3/aiv_graph_generator_v3/aiv_snapshot_json_loader_v3.cc+5-14
@@ -17,6 +17,7 @@
17#include <sstream>17#include <sstream>
18#include <sys/stat.h>18#include <sys/stat.h>
19 19 
20+#include "sim_common_api.h"
20#include "sim_log.h"21#include "sim_log.h"
21 22 
22namespace HcclSim {23namespace HcclSim {
@@ -41,15 +42,9 @@ void SetError(std::string &errorMessage, const std::string &message)
41 42 
42std::string ResolveAivTaskFilePath(RankId rankId, uint64_t launchIndex)43std::string ResolveAivTaskFilePath(RankId rankId, uint64_t launchIndex)
43{44{
44- const char *installDir = std::getenv("HCCL_VM_INSTALL_DIR");45+ const std::string dataRelPath = std::string("data/") + AIV_RANK_TASK_FILE_PREFIX + std::to_string(rankId)
45- if (installDir == nullptr || installDir[0] == '\0') {46+ + AIV_RANK_TASK_FILE_LAUNCH_MARKER + std::to_string(launchIndex) + AIV_RANK_TASK_FILE_SUFFIX;
46- return "";47+ return InstallPath::ResolveToInstallRoot(dataRelPath);
47- }
48- 
49- std::ostringstream os;
50- os << installDir << "/data/" << AIV_RANK_TASK_FILE_PREFIX << rankId
51- << AIV_RANK_TASK_FILE_LAUNCH_MARKER << launchIndex << AIV_RANK_TASK_FILE_SUFFIX;
52- return os.str();
53}48}
54 49 
55bool CheckObjectField(const Json &json, const char *fieldName, std::string &errorMessage)50bool CheckObjectField(const Json &json, const char *fieldName, std::string &errorMessage)
@@ -225,10 +220,6 @@ HcclResult AivSnapshotJsonLoaderV3::LoadByRankAndLaunch(RankId rankId, uint64_t
225{220{
226 snapshot = AivRuntimeTaskSnapshotV3 {};221 snapshot = AivRuntimeTaskSnapshotV3 {};
227 const std::string filePath = ResolveAivTaskFilePath(rankId, launchIndex);222 const std::string filePath = ResolveAivTaskFilePath(rankId, launchIndex);
228- if (filePath.empty()) {
229- SetError(errorMessage, "HCCL_VM_INSTALL_DIR is not set");
230- return HCCL_E_NOT_FOUND;
231- }
232 if (!FileExists(filePath)) {223 if (!FileExists(filePath)) {
233 SetError(errorMessage, "AIV task json file does not exist: " + filePath);224 SetError(errorMessage, "AIV task json file does not exist: " + filePath);
234 return HCCL_E_NOT_FOUND;225 return HCCL_E_NOT_FOUND;
@@ -263,7 +254,7 @@ HcclResult AivSnapshotJsonLoaderV3::LoadByRankAndLaunch(RankId rankId, uint64_t
263 return HCCL_E_PARA;254 return HCCL_E_PARA;
264 }255 }
265 256 
266- HCCL_VM_INFO("[AivSnapshotJsonLoaderV3] Loaded AIV snapshot, rankId={}, launchIndex={}, blockCount={}, file={}",257+ HCCL_VM_INFO("Loaded AIV snapshot, rankId={}, launchIndex={}, blockCount={}, file={}",
267 rankId, launchIndex, snapshot.blocks.size(), filePath);258 rankId, launchIndex, snapshot.blocks.size(), filePath);
268 return HCCL_SUCCESS;259 return HCCL_SUCCESS;
269}260}
Mtest/hccl_vm/src/plugin/checker/src/framework/task_graph_generator_v3/aiv_graph_generator_v3/aiv_task_transform_v3.cc+133-48
@@ -25,6 +25,7 @@
25 25 
26#include "aiv_snapshot_json_loader_v3.h"26#include "aiv_snapshot_json_loader_v3.h"
27#include "sim_log.h"27#include "sim_log.h"
28+#include "utils/error_codes.h"
28 29 
29namespace HcclSim {30namespace HcclSim {
30namespace TaskGraphGeneratorV3 {31namespace TaskGraphGeneratorV3 {
@@ -353,8 +354,9 @@ HcclResult ValidateSnapshot(const AivLaunchContext &ctx)
353 }354 }
354 const CheckerParam param = ctx.storage->GetCheckerParam();355 const CheckerParam param = ctx.storage->GetCheckerParam();
355 if (ctx.snapshot.rankSize != 0 && param.rankSize != 0 && ctx.snapshot.rankSize != param.rankSize) {356 if (ctx.snapshot.rankSize != 0 && param.rankSize != 0 && ctx.snapshot.rankSize != param.rankSize) {
356- HCCL_VM_ERROR("[AivTaskTransformV3] AIV snapshot rankSize mismatch, rank={}, launch={}, "357+ HCCL_VM_ERROR("{} The AIV snapshot was captured for a different rank count than the current "
357- "snapshotRankSize={}, checkerRankSize={}, file={}", ctx.placeholder->GetRankId(),358+ "checker input, rankId={}, launchId={}, snapshotRankCount={}, currentRankCount={}, snapshotFile={}",
359+ MakeErrorCodeText(ErrorCode::GRAPH_SNAPSHOT_MISMATCH), ctx.placeholder->GetRankId(),
358 ctx.placeholder->GetLaunchIdx(), ctx.snapshot.rankSize, param.rankSize, ctx.snapshot.filePath);360 ctx.placeholder->GetLaunchIdx(), ctx.snapshot.rankSize, param.rankSize, ctx.snapshot.filePath);
359 return HCCL_E_PARA;361 return HCCL_E_PARA;
360 }362 }
@@ -375,9 +377,10 @@ HcclResult UpdateAivBufferSize(uint64_t snapshotSize, const char *fieldName, con
375 return HCCL_SUCCESS;377 return HCCL_SUCCESS;
376 }378 }
377 379 
378- HCCL_VM_ERROR("[AivTaskTransformV3] AIV {} mismatch, rank={}, launch={}, expected={}, actual={}, file={}",380+ HCCL_VM_ERROR("{} AIV buffer size is inconsistent across snapshots, field={}, rankId={}, "
379- fieldName, ctx.placeholder->GetRankId(), ctx.placeholder->GetLaunchIdx(), bufferSize, snapshotSize,381+ "launchId={}, expectedSize={}, actualSize={}, expectedSource=firstNonZeroSnapshot, snapshotFile={}",
380- ctx.snapshot.filePath);382+ MakeErrorCodeText(ErrorCode::GRAPH_SNAPSHOT_MISMATCH), fieldName, ctx.placeholder->GetRankId(),
383+ ctx.placeholder->GetLaunchIdx(), bufferSize, snapshotSize, ctx.snapshot.filePath);
381 return HCCL_E_PARA;384 return HCCL_E_PARA;
382}385}
383 386 
@@ -385,9 +388,11 @@ HcclResult AppendRuntimeTask(AivLaunchContext &ctx, const AivRuntimeTaskV3 &task
385{388{
386 std::unique_ptr<TaskNode> node = TranslateAivRuntimeTask(task, ctx.placeholder->GetLaunchIdx());389 std::unique_ptr<TaskNode> node = TranslateAivRuntimeTask(task, ctx.placeholder->GetLaunchIdx());
387 if (node == nullptr) {390 if (node == nullptr) {
388- HCCL_VM_ERROR("[AivTaskTransformV3] Translate AIV task failed, rank={}, launch={}, taskId={}, taskType={}, "391+ HCCL_VM_ERROR("{} One AIV runtime task type is not supported, "
389- "file={}", ctx.placeholder->GetRankId(), ctx.placeholder->GetLaunchIdx(), task.taskId,392+ "rankId={}, launchId={}, taskId={}, taskType={}, snapshotFile={}",
390- static_cast<uint32_t>(task.taskType), ctx.snapshot.filePath);393+ MakeErrorCodeText(ErrorCode::GRAPH_UNSUPPORTED), ctx.placeholder->GetRankId(),
394+ ctx.placeholder->GetLaunchIdx(), task.taskId, static_cast<uint32_t>(task.taskType),
395+ ctx.snapshot.filePath);
391 return HCCL_E_NOT_SUPPORT;396 return HCCL_E_NOT_SUPPORT;
392 }397 }
393 398 
@@ -507,8 +512,21 @@ HcclResult AddSetWaitFlagEdges(AivLaunchContext &ctx)
507 while (!readyQueue.empty()) {512 while (!readyQueue.empty()) {
508 if (unmatchedCnt >= readyQueue.size()) {513 if (unmatchedCnt >= readyQueue.size()) {
509 TaskNode *node = ctx.graph->GetNode(readyQueue.front());514 TaskNode *node = ctx.graph->GetNode(readyQueue.front());
510- HCCL_VM_ERROR("[AivTaskTransformV3] SetFlag/WaitFlag matching deadlock, firstUnmatched={}, file={}",515+ size_t seenMatchingSetFlagCount = 0;
511- node == nullptr ? "null" : node->Describe(), ctx.snapshot.filePath);516+ if (node != nullptr && node->GetType() == TaskType::AIV_WAIT_FLAG) {
517+ const auto *waitFlag = dynamic_cast<const TaskAivWaitFlag *>(node);
518+ if (waitFlag != nullptr) {
519+ const SetWaitKey key = MakeSetWaitKey(waitFlag->GetEvent());
520+ const auto iter = seenSetFlags.find(key);
521+ seenMatchingSetFlagCount = (iter == seenSetFlags.end()) ? 0U : iter->second.size();
522+ }
523+ }
524+ HCCL_VM_ERROR("{} AIV SetFlag/WaitFlag matching is stuck. Some WaitFlag tasks are still "
525+ "blocked, but no matching SetFlag has become available, firstBlockedWaitFlagNode={}, "
526+ "blockedWaitFlagNodeCount={}, availableSetFlagCountForThisWait={}, snapshotFile={}",
527+ MakeErrorCodeText(ErrorCode::GRAPH_DEADLOCK), node == nullptr ? "node=null" : node->Describe(),
528+ readyQueue.size(),
529+ seenMatchingSetFlagCount, ctx.snapshot.filePath);
512 return HCCL_E_INTERNAL;530 return HCCL_E_INTERNAL;
513 }531 }
514 532 
@@ -584,8 +602,11 @@ HcclResult AddSetWaitFlagEdges(AivLaunchContext &ctx)
584 for (const auto &entry : seenSetFlags) {602 for (const auto &entry : seenSetFlags) {
585 if (!entry.second.empty()) {603 if (!entry.second.empty()) {
586 TaskNode *node = ctx.graph->GetNode(entry.second.front());604 TaskNode *node = ctx.graph->GetNode(entry.second.front());
587- HCCL_VM_WARN("[AivTaskTransformV3] Unconsumed SetFlag, node={}, file={}",605+ HCCL_VM_WARN("{} Found SetFlag tasks that were never consumed by any WaitFlag task, "
588- node == nullptr ? "null" : node->Describe(), ctx.snapshot.filePath);606+ "firstUnconsumedSetFlagNode={}, unconsumedSetFlagCount={}, snapshotFile={}",
607+ MakeErrorCodeText(ErrorCode::GRAPH_UNMATCHED), node == nullptr ? "node=null" : node->Describe(),
608+ entry.second.size(),
609+ ctx.snapshot.filePath);
589 }610 }
590 }611 }
591 return HCCL_SUCCESS;612 return HCCL_SUCCESS;
@@ -1535,8 +1556,9 @@ HcclResult MergeCpGM2GMLoops(AivLaunchContext &ctx)
1535 for (const auto &block : ctx.snapshot.blocks) {1556 for (const auto &block : ctx.snapshot.blocks) {
1536 std::vector<NodeId> topo;1557 std::vector<NodeId> topo;
1537 if (!CollectCpGmBlockTopo(ctx, block.blockIdx, topo)) {1558 if (!CollectCpGmBlockTopo(ctx, block.blockIdx, topo)) {
1538- HCCL_VM_WARN("[AivTaskTransformV3][CpGM2GMMerge] Skip block due to topo failure, rank={}, launch={}, "1559+ HCCL_VM_WARN("Skip CpGM-to-GM merge for one block because the block DAG order could not "
1539- "block={}, file={}", ctx.placeholder->GetRankId(), ctx.placeholder->GetLaunchIdx(), block.blockIdx,1560+ "be determined, rankId={}, launchId={}, blockId={}, snapshotFile={}",
1561+ ctx.placeholder->GetRankId(), ctx.placeholder->GetLaunchIdx(), block.blockIdx,
1540 ctx.snapshot.filePath);1562 ctx.snapshot.filePath);
1541 continue;1563 continue;
1542 }1564 }
@@ -1570,10 +1592,11 @@ HcclResult MergeCpGM2GMLoops(AivLaunchContext &ctx)
1570 }1592 }
1571 }1593 }
1572 ctx.dagNodeCountAfterCpGmMerge = CountReachableActiveInternalNodes(ctx);1594 ctx.dagNodeCountAfterCpGmMerge = CountReachableActiveInternalNodes(ctx);
1573- HCCL_VM_INFO("[AivTaskTransformV3][CpGM2GMMerge] rank={}, launch={}, taskJsonTotalTaskCount={}, "1595+ HCCL_VM_INFO("Finished CpGM-to-GM merge analysis, rankId={}, launchId={}, "
1574- "dagNodeCountBeforeMerge={}, dagNodeCountAfterMerge={}, cpGmLoopMergeCount={}, "1596+ "taskJsonTotalTaskCount={}, dagNodeCountBeforeMerge={}, dagNodeCountAfterMerge={}, "
1575- "cpGmMergedIterationCount={}, cpGmMergedOriginalNodeCount={}, cpGmGeneratedNodeCount={}, "1597+ "cpGmLoopMergeCount={}, cpGmMergedIterationCount={}, cpGmMergedOriginalNodeCount={}, "
1576- "cpGmInactiveNodeCount={}", ctx.placeholder->GetRankId(), ctx.placeholder->GetLaunchIdx(),1598+ "cpGmGeneratedNodeCount={}, cpGmInactiveNodeCount={}", ctx.placeholder->GetRankId(),
1599+ ctx.placeholder->GetLaunchIdx(),
1577 ctx.taskJsonTotalTaskCount, ctx.dagNodeCountBeforeCpGmMerge, ctx.dagNodeCountAfterCpGmMerge,1600 ctx.taskJsonTotalTaskCount, ctx.dagNodeCountBeforeCpGmMerge, ctx.dagNodeCountAfterCpGmMerge,
1578 ctx.cpGmLoopMergeCount, ctx.cpGmMergedIterationCount, ctx.cpGmMergedOriginalNodeCount,1601 ctx.cpGmLoopMergeCount, ctx.cpGmMergedIterationCount, ctx.cpGmMergedOriginalNodeCount,
1579 ctx.cpGmGeneratedNodeCount, ctx.cpGmInactiveNodeCount);1602 ctx.cpGmGeneratedNodeCount, ctx.cpGmInactiveNodeCount);
@@ -1710,23 +1733,26 @@ HcclResult MergePipeBarrierGroups(AivLaunchContext &ctx)
1710 TaskNode *member = ctx.graph->GetNode(memberNodeId);1733 TaskNode *member = ctx.graph->GetNode(memberNodeId);
1711 const auto *barrier = dynamic_cast<const TaskAivPipeBarrier *>(member);1734 const auto *barrier = dynamic_cast<const TaskAivPipeBarrier *>(member);
1712 if (barrier == nullptr) {1735 if (barrier == nullptr) {
1713- HCCL_VM_ERROR("[AivTaskTransformV3] Invalid PipeBarrier member node, rank={}, launch={}, "1736+ HCCL_VM_ERROR("{} One PipeBarrier group member is not a valid PipeBarrier node, "
1714- "nodeId={}, file={}", ctx.placeholder->GetRankId(), ctx.placeholder->GetLaunchIdx(),1737+ "rankId={}, launchId={}, nodeId={}, snapshotFile={}",
1715- memberNodeId, ctx.snapshot.filePath);1738+ MakeErrorCodeText(ErrorCode::GRAPH_STRUCTURE_INVALID), ctx.placeholder->GetRankId(),
1739+ ctx.placeholder->GetLaunchIdx(), memberNodeId, ctx.snapshot.filePath);
1716 return HCCL_E_INTERNAL;1740 return HCCL_E_INTERNAL;
1717 }1741 }
1718 const AivBarrierInfo &memberInfo = barrier->GetInfo();1742 const AivBarrierInfo &memberInfo = barrier->GetInfo();
1719 if (memberInfo.taskLoc.blockId != group.blockId) {1743 if (memberInfo.taskLoc.blockId != group.blockId) {
1720- HCCL_VM_ERROR("[AivTaskTransformV3] PipeBarrier group crosses block, rank={}, launch={}, "1744+ HCCL_VM_ERROR("{} One PipeBarrier group mixes tasks from different blocks, rankId={}, "
1721- "groupBlock={}, memberBlock={}, memberTaskId={}, file={}", ctx.placeholder->GetRankId(),1745+ "launchId={}, expectedBlockId={}, actualBlockId={}, memberTaskId={}, snapshotFile={}",
1746+ MakeErrorCodeText(ErrorCode::GRAPH_STRUCTURE_INVALID), ctx.placeholder->GetRankId(),
1722 ctx.placeholder->GetLaunchIdx(), group.blockId, memberInfo.taskLoc.blockId,1747 ctx.placeholder->GetLaunchIdx(), group.blockId, memberInfo.taskLoc.blockId,
1723 memberInfo.taskLoc.taskId, ctx.snapshot.filePath);1748 memberInfo.taskLoc.taskId, ctx.snapshot.filePath);
1724 return HCCL_E_INTERNAL;1749 return HCCL_E_INTERNAL;
1725 }1750 }
1726 if (!memberPipes.insert(memberInfo.taskLoc.pipe).second) {1751 if (!memberPipes.insert(memberInfo.taskLoc.pipe).second) {
1727- HCCL_VM_ERROR("[AivTaskTransformV3] Duplicate PipeBarrier member pipe, rank={}, launch={}, "1752+ HCCL_VM_ERROR("{} One PipeBarrier group contains two tasks from the same pipe, "
1728- "block={}, pipe={}, file={}", ctx.placeholder->GetRankId(), ctx.placeholder->GetLaunchIdx(),1753+ "rankId={}, launchId={}, blockId={}, duplicatedPipeId={}, snapshotFile={}",
1729- group.blockId, memberInfo.taskLoc.pipe, ctx.snapshot.filePath);1754+ MakeErrorCodeText(ErrorCode::GRAPH_STRUCTURE_INVALID), ctx.placeholder->GetRankId(),
1755+ ctx.placeholder->GetLaunchIdx(), group.blockId, memberInfo.taskLoc.pipe, ctx.snapshot.filePath);
1730 return HCCL_E_INTERNAL;1756 return HCCL_E_INTERNAL;
1731 }1757 }
1732 }1758 }
@@ -1735,9 +1761,30 @@ HcclResult MergePipeBarrierGroups(AivLaunchContext &ctx)
1735 if (memberPipes.count(pipe) != 0) {1761 if (memberPipes.count(pipe) != 0) {
1736 continue;1762 continue;
1737 }1763 }
1738- HCCL_VM_ERROR("[AivTaskTransformV3] Missing PIPE_ALL PipeBarrier member, rank={}, launch={}, "1764+ std::ostringstream existingPipeIds;
1739- "block={}, pipe={}, file={}", ctx.placeholder->GetRankId(), ctx.placeholder->GetLaunchIdx(),1765+ existingPipeIds << "[";
1740- group.blockId, pipe, ctx.snapshot.filePath);1766+ for (size_t i = 0; i < group.memberPipes.size(); ++i) {
1767+ if (i != 0) {
1768+ existingPipeIds << ",";
1769+ }
1770+ existingPipeIds << group.memberPipes[i];
1771+ }
1772+ existingPipeIds << "]";
1773+ std::ostringstream memberTaskIds;
1774+ memberTaskIds << "[";
1775+ for (size_t i = 0; i < group.memberTaskIds.size(); ++i) {
1776+ if (i != 0) {
1777+ memberTaskIds << ",";
1778+ }
1779+ memberTaskIds << group.memberTaskIds[i];
1780+ }
1781+ memberTaskIds << "]";
1782+ HCCL_VM_ERROR("{} One PIPE_ALL PipeBarrier is incomplete because one pipe is missing, "
1783+ "rankId={}, launchId={}, blockId={}, missingPipeId={}, existingPipeIds={}, memberTaskIds={}, "
1784+ "snapshotFile={}",
1785+ MakeErrorCodeText(ErrorCode::GRAPH_MEMBER_MISSING), ctx.placeholder->GetRankId(),
1786+ ctx.placeholder->GetLaunchIdx(), group.blockId, pipe, existingPipeIds.str(), memberTaskIds.str(),
1787+ ctx.snapshot.filePath);
1741 return HCCL_E_INTERNAL;1788 return HCCL_E_INTERNAL;
1742 }1789 }
1743 }1790 }
@@ -1781,9 +1828,12 @@ HcclResult MergeSyncAllGroups(AivLaunchContext &ctx)
1781 continue;1828 continue;
1782 }1829 }
1783 if (expectedMemberCount != 0 && group.memberNodeIds.size() != expectedMemberCount) {1830 if (expectedMemberCount != 0 && group.memberNodeIds.size() != expectedMemberCount) {
1784- HCCL_VM_ERROR("[AivTaskTransformV3] SyncAll member count mismatch, rank={}, launch={}, syncRound={}, "1831+ HCCL_VM_ERROR("{} One SyncAll group is incomplete because it does not contain one member "
1785- "actual={}, expected={}, file={}", ctx.placeholder->GetRankId(), ctx.placeholder->GetLaunchIdx(),1832+ "for every expected block/pipe pair, rankId={}, launchId={}, syncRound={}, actualCount={}, "
1786- group.syncRound, group.memberNodeIds.size(), expectedMemberCount, ctx.snapshot.filePath);1833+ "expectedCount={}, snapshotFile={}",
1834+ MakeErrorCodeText(ErrorCode::GRAPH_MEMBER_MISSING),
1835+ ctx.placeholder->GetRankId(), ctx.placeholder->GetLaunchIdx(), group.syncRound,
1836+ group.memberNodeIds.size(), expectedMemberCount, ctx.snapshot.filePath);
1787 return HCCL_E_INTERNAL;1837 return HCCL_E_INTERNAL;
1788 }1838 }
1789 std::set<std::pair<uint32_t, uint32_t>> members;1839 std::set<std::pair<uint32_t, uint32_t>> members;
@@ -1791,16 +1841,19 @@ HcclResult MergeSyncAllGroups(AivLaunchContext &ctx)
1791 TaskNode *member = ctx.graph->GetNode(memberNodeId);1841 TaskNode *member = ctx.graph->GetNode(memberNodeId);
1792 const auto *syncAll = dynamic_cast<const TaskAivSyncAll *>(member);1842 const auto *syncAll = dynamic_cast<const TaskAivSyncAll *>(member);
1793 if (syncAll == nullptr) {1843 if (syncAll == nullptr) {
1794- HCCL_VM_ERROR("[AivTaskTransformV3] Invalid SyncAll member node, rank={}, launch={}, syncRound={}, "1844+ HCCL_VM_ERROR("{} One SyncAll group member is not a valid SyncAll node, rankId={}, "
1795- "nodeId={}, file={}", ctx.placeholder->GetRankId(), ctx.placeholder->GetLaunchIdx(),1845+ "launchId={}, syncRound={}, nodeId={}, snapshotFile={}",
1796- group.syncRound, memberNodeId, ctx.snapshot.filePath);1846+ MakeErrorCodeText(ErrorCode::GRAPH_STRUCTURE_INVALID), ctx.placeholder->GetRankId(),
1847+ ctx.placeholder->GetLaunchIdx(), group.syncRound, memberNodeId, ctx.snapshot.filePath);
1797 return HCCL_E_INTERNAL;1848 return HCCL_E_INTERNAL;
1798 }1849 }
1799 const TaskPosition &taskLoc = syncAll->GetInfo().taskLoc;1850 const TaskPosition &taskLoc = syncAll->GetInfo().taskLoc;
1800 if (!members.insert({taskLoc.blockId, taskLoc.pipe}).second) {1851 if (!members.insert({taskLoc.blockId, taskLoc.pipe}).second) {
1801- HCCL_VM_ERROR("[AivTaskTransformV3] Duplicate SyncAll member, rank={}, launch={}, syncRound={}, "1852+ HCCL_VM_ERROR("{} One SyncAll group contains duplicate members for the same "
1802- "block={}, pipe={}, file={}", ctx.placeholder->GetRankId(), ctx.placeholder->GetLaunchIdx(),1853+ "block/pipe pair, rankId={}, launchId={}, syncRound={}, blockId={}, pipeId={}, snapshotFile={}",
1803- group.syncRound, taskLoc.blockId, taskLoc.pipe, ctx.snapshot.filePath);1854+ MakeErrorCodeText(ErrorCode::GRAPH_STRUCTURE_INVALID), ctx.placeholder->GetRankId(),
1855+ ctx.placeholder->GetLaunchIdx(), group.syncRound, taskLoc.blockId, taskLoc.pipe,
1856+ ctx.snapshot.filePath);
1804 return HCCL_E_INTERNAL;1857 return HCCL_E_INTERNAL;
1805 }1858 }
1806 }1859 }
@@ -1809,9 +1862,12 @@ HcclResult MergeSyncAllGroups(AivLaunchContext &ctx)
1809 if (members.count({block.blockIdx, pipe}) != 0) {1862 if (members.count({block.blockIdx, pipe}) != 0) {
1810 continue;1863 continue;
1811 }1864 }
1812- HCCL_VM_ERROR("[AivTaskTransformV3] Missing SyncAll member, rank={}, launch={}, syncRound={}, "1865+ HCCL_VM_ERROR("{} One SyncAll group is incomplete because a block/pipe member is "
1813- "block={}, pipe={}, file={}", ctx.placeholder->GetRankId(), ctx.placeholder->GetLaunchIdx(),1866+ "missing, rankId={}, launchId={}, syncRound={}, missingBlockId={}, missingPipeId={}, "
1814- group.syncRound, block.blockIdx, pipe, ctx.snapshot.filePath);1867+ "snapshotFile={}",
1868+ MakeErrorCodeText(ErrorCode::GRAPH_MEMBER_MISSING),
1869+ ctx.placeholder->GetRankId(), ctx.placeholder->GetLaunchIdx(), group.syncRound,
1870+ block.blockIdx, pipe, ctx.snapshot.filePath);
1815 return HCCL_E_INTERNAL;1871 return HCCL_E_INTERNAL;
1816 }1872 }
1817 }1873 }
@@ -1955,7 +2011,8 @@ HcclResult ExpandOneAivGraph(TaskGraphGeneratorV3 *graph, StorageManager *storag
1955 HcclResult ret = loader.LoadByRankAndLaunch(aivGraph->GetRankId(), aivGraph->GetLaunchIdx(), ctx.snapshot,2011 HcclResult ret = loader.LoadByRankAndLaunch(aivGraph->GetRankId(), aivGraph->GetLaunchIdx(), ctx.snapshot,
1956 errorMessage);2012 errorMessage);
1957 if (ret != HCCL_SUCCESS) {2013 if (ret != HCCL_SUCCESS) {
1958- HCCL_VM_ERROR("[AivTaskTransformV3] Load AIV snapshot failed, rank={}, launch={}, ret={}, err={}",2014+ HCCL_VM_ERROR("{} Failed to load the AIV runtime snapshot file, rankId={}, launchId={}, "
2015+ "ret={}, loaderMessage={}", MakeErrorCodeText(ErrorCode::GRAPH_RESOURCE_NOT_FOUND),
1959 aivGraph->GetRankId(), aivGraph->GetLaunchIdx(), static_cast<uint32_t>(ret), errorMessage);2016 aivGraph->GetRankId(), aivGraph->GetLaunchIdx(), static_cast<uint32_t>(ret), errorMessage);
1960 return ret;2017 return ret;
1961 }2018 }
@@ -2079,7 +2136,8 @@ HcclResult EnqueueChildrenForFlagMatch(const TaskGraphGeneratorV3 *graph, NodeId
2079 return HCCL_SUCCESS;2136 return HCCL_SUCCESS;
2080}2137}
2081 2138 
2082-HcclResult MatchSendRecvFlagEdges(TaskGraphGeneratorV3 *graph, size_t &matchedEdgeCount)2139+HcclResult MatchSendRecvFlagEdges(TaskGraphGeneratorV3 *graph, const std::string &snapshotFile,
2140+ size_t &matchedEdgeCount)
2083{2141{
2084 matchedEdgeCount = 0;2142 matchedEdgeCount = 0;
2085 if (graph == nullptr || graph->GetMainStartNode() == nullptr) {2143 if (graph == nullptr || graph->GetMainStartNode() == nullptr) {
@@ -2104,8 +2162,33 @@ HcclResult MatchSendRecvFlagEdges(TaskGraphGeneratorV3 *graph, size_t &matchedEd
2104 while (!readyQueue.empty()) {2162 while (!readyQueue.empty()) {
2105 if (unmatchedCnt >= readyQueue.size()) {2163 if (unmatchedCnt >= readyQueue.size()) {
2106 const TaskNode *node = graph->GetNode(readyQueue.front());2164 const TaskNode *node = graph->GetNode(readyQueue.front());
2107- HCCL_VM_ERROR("[AivTaskTransformV3] Send/RecvFlag matching deadlock, firstUnmatched={}",2165+ std::string flagCellStateText = "{}";
2108- node == nullptr ? "null" : node->Describe());2166+ if (node != nullptr && node->GetType() == TaskType::AIV_RECV_FLAG) {
2167+ const auto *recvFlag = dynamic_cast<const TaskAivRecvFlag *>(node);
2168+ if (recvFlag != nullptr) {
2169+ const auto stateIter = flagStates.find(MakeFlagCellKey(recvFlag->GetFlag()));
2170+ if (stateIter != flagStates.end()) {
2171+ std::ostringstream os;
2172+ os << "{currentValue=" << stateIter->second.currentValue
2173+ << ", producerCount=" << stateIter->second.producerNodes.size()
2174+ << ", producerNodeIds=[";
2175+ for (size_t i = 0; i < stateIter->second.producerNodes.size(); ++i) {
2176+ if (i != 0) {
2177+ os << ",";
2178+ }
2179+ os << stateIter->second.producerNodes[i];
2180+ }
2181+ os << "]}";
2182+ flagCellStateText = os.str();
2183+ }
2184+ }
2185+ }
2186+ HCCL_VM_ERROR("{} AIV SendFlag/RecvFlag matching is stuck. Some RecvFlag tasks are still "
2187+ "blocked, but no matching SendFlag value has become available, firstBlockedRecvFlagNode={}, "
2188+ "blockedRecvFlagNodeCount={}, currentFlagCellState={}, snapshotFile={}",
2189+ MakeErrorCodeText(ErrorCode::GRAPH_DEADLOCK), node == nullptr ? "node=null" : node->Describe(),
2190+ readyQueue.size(),
2191+ flagCellStateText, snapshotFile);
2109 return HCCL_E_INTERNAL;2192 return HCCL_E_INTERNAL;
2110 }2193 }
2111 2194 
@@ -2134,8 +2217,8 @@ HcclResult MatchSendRecvFlagEdges(TaskGraphGeneratorV3 *graph, size_t &matchedEd
2134 }2217 }
2135 FlagCellState &state = flagStates[MakeFlagCellKey(sendFlag->GetFlag())];2218 FlagCellState &state = flagStates[MakeFlagCellKey(sendFlag->GetFlag())];
2136 if (state.currentValue == sendFlag->GetFlag().value && !state.producerNodes.empty()) {2219 if (state.currentValue == sendFlag->GetFlag().value && !state.producerNodes.empty()) {
2137- HCCL_VM_WARN("[AivTaskTransformV3] Repeated SendFlag value on same flag cell, node={}",2220+ HCCL_VM_WARN("The same flag cell received the same SendFlag value again before any "
2138- node->Describe());2221+ "new RecvFlag consumed it, node={}", node == nullptr ? "node=null" : node->Describe());
2139 }2222 }
2140 state.currentValue = sendFlag->GetFlag().value;2223 state.currentValue = sendFlag->GetFlag().value;
2141 state.producerNodes.clear();2224 state.producerNodes.clear();
@@ -2188,7 +2271,9 @@ HcclResult ExpandAivGraphsV3(const AivGraphsGenerateInputV3 &input, AivGraphGene
2188 }2271 }
2189 }2272 }
2190 2273 
2191- HcclResult ret = MatchSendRecvFlagEdges(input.graph, output.sendRecvEdgeCount);2274+ const std::string snapshotFileHint =
2275+ input.aivGraphs.empty() ? "aiv_snapshot_unavailable" : "multiple_aiv_snapshots";
2276+ HcclResult ret = MatchSendRecvFlagEdges(input.graph, snapshotFileHint, output.sendRecvEdgeCount);
2192 output.expandNs = ElapsedNs(start, AivExpandClock::now());2277 output.expandNs = ElapsedNs(start, AivExpandClock::now());
2193 return ret;2278 return ret;
2194}2279}
Mtest/hccl_vm/src/plugin/checker/src/framework/task_graph_generator_v3/ccu_graph_generator_v3/ccu_all_rank_param_recorder_v3.cc+23-16
@@ -10,6 +10,8 @@
10 10 
11#include "ccu_all_rank_param_recorder_v3.h"11#include "ccu_all_rank_param_recorder_v3.h"
12#include <sys/types.h>12#include <sys/types.h>
13+#include "sim_log.h"
14+#include "utils/error_codes.h"
13 15 
14namespace HcclSim {16namespace HcclSim {
15namespace TaskGraphGeneratorV3 {17namespace TaskGraphGeneratorV3 {
@@ -46,22 +48,22 @@ void AllRankParamRecorder::RegisterPostNode(TaskNode *node, const CcuPostNodeMet
46 postNodeMeta[node] = meta;48 postNodeMeta[node] = meta;
47}49}
48 50 
49-uint32_t AllRankParamRecorder::GetPostTopicId(const TaskNode *node) const51+uint32_t AllRankParamRecorder::GetPostRemainingCkeMask(const TaskNode *node) const
50{52{
51 auto it = postNodeMeta.find(node);53 auto it = postNodeMeta.find(node);
52 if (it == postNodeMeta.end()) {54 if (it == postNodeMeta.end()) {
53 return 0;55 return 0;
54 }56 }
55- return it->second.topicId;57+ return it->second.remainingCkeMask;
56}58}
57 59 
58-void AllRankParamRecorder::SetPostTopicId(TaskNode *node, uint32_t topicId)60+void AllRankParamRecorder::SetPostRemainingCkeMask(TaskNode *node, uint32_t remainingCkeMask)
59{61{
60 auto it = postNodeMeta.find(node);62 auto it = postNodeMeta.find(node);
61 if (it == postNodeMeta.end()) {63 if (it == postNodeMeta.end()) {
62 return;64 return;
63 }65 }
64- it->second.topicId = static_cast<uint16_t>(topicId);66+ it->second.remainingCkeMask = static_cast<uint16_t>(remainingCkeMask);
65}67}
66 68 
67const CcuPostNodeMetaV3 *AllRankParamRecorder::GetPostNodeMeta(const TaskNode *node) const69const CcuPostNodeMetaV3 *AllRankParamRecorder::GetPostNodeMeta(const TaskNode *node) const
@@ -101,7 +103,7 @@ HcclResult AllRankParamRecorder::GetXn(uint32_t rankId, uint32_t dieId, uint16_t
101 }103 }
102 }104 }
103 }105 }
104- HCCL_ERROR("[GetXn]curXn is not be initialized, rankId[%u], dieId[%u], xnId[%hu]", rankId, dieId, xnId);106+ xnValue = 0;
105 return HCCL_E_PARA;107 return HCCL_E_PARA;
106}108}
107 109 
@@ -115,7 +117,7 @@ HcclResult AllRankParamRecorder::GetGSA(uint32_t rankId, uint32_t dieId, uint16
115 }117 }
116 }118 }
117 }119 }
118- HCCL_ERROR("[GetGSA]curGSA is not be initialized, rankId[%u], dieId[%u], gsaId[%hu]", rankId, dieId, gsaId);120+ gsaValue = 0;
119 return HCCL_E_PARA;121 return HCCL_E_PARA;
120}122}
121 123 
@@ -184,14 +186,21 @@ HcclResult AllRankParamRecorder::CheckAllPostMatch()
184 for (const auto *post : regPair.second) {186 for (const auto *post : regPair.second) {
185 const auto *meta = GetPostNodeMeta(post);187 const auto *meta = GetPostNodeMeta(post);
186 if (meta != nullptr) {188 if (meta != nullptr) {
187- HCCL_WARNING("unmatched LocalPost/Post: node={:p}, waitRank={}, dieId={}, ckeId={}, "189+ HCCL_VM_WARN("{} Found CCU post/local-post tasks that were never consumed by "
188- "recordRank={}, topicId=0x{:x}, isLocal={}", static_cast<const void *>(post),190+ "any Wait task, firstUnconsumedPostNode={}, unconsumedPostCount={}, waitRankId={}, "
189- meta->waitRankId, meta->dieId, meta->ckeId, meta->recordRankId, meta->topicId,191+ "dieId={}, "
190- meta->isLocal);192+ "ckeId={}, recordRankId={}, remainingCkeMask=0x{:x}, isLocal={}",
193+ MakeErrorCodeText(ErrorCode::GRAPH_UNMATCHED).c_str(),
194+ post == nullptr ? "node=null" : post->Describe().c_str(), regPair.second.size(),
195+ meta->waitRankId, meta->dieId, meta->ckeId,
196+ meta->recordRankId, meta->remainingCkeMask, meta->isLocal);
191 continue;197 continue;
192 }198 }
193- HCCL_WARNING("unmatched LocalPost/Post: node={:p}, waitRank={}, dieId={}, ckeId={}",199+ HCCL_VM_WARN("{} Found CCU post/local-post tasks that were never consumed by any "
194- static_cast<const void *>(post), rank, dieId, regId);200+ "Wait task, firstUnconsumedPostNode={}, unconsumedPostCount={}, waitRankId={}, dieId={}, "
201+ "ckeId={}", MakeErrorCodeText(ErrorCode::GRAPH_UNMATCHED).c_str(),
202+ post == nullptr ? "node=null" : post->Describe().c_str(), regPair.second.size(), rank,
203+ dieId, regId);
195 }204 }
196 }205 }
197 }206 }
@@ -202,9 +211,7 @@ HcclResult AllRankParamRecorder::CheckAllPostMatch()
202 211 
203HcclResult AllRankParamRecorder::SetHBM(uint32_t rankId, uint32_t dieId, uint64_t hbmAddr,212HcclResult AllRankParamRecorder::SetHBM(uint32_t rankId, uint32_t dieId, uint64_t hbmAddr,
204 const std::vector<uint64_t>& data) {213 const std::vector<uint64_t>& data) {
205- // 合法性校验214+ if (data.size() % 8 != 0) {
206- if (data.size() % 8 != 0){
207- HCCL_ERROR("hbm data size is not 8 bytes aligned");
208 return HCCL_E_PARA;215 return HCCL_E_PARA;
209 }216 }
210 curHBM[rankId][dieId][hbmAddr] = data;217 curHBM[rankId][dieId][hbmAddr] = data;
@@ -220,7 +227,7 @@ HcclResult AllRankParamRecorder::GetHBM(uint32_t rankId, uint32_t dieId, uint64_
220 }227 }
221 }228 }
222 }229 }
223- HCCL_ERROR("[GetGSA]curGSA is not be initialized, rankId[%u], dieId[%u], hbmAddr[%llu]", rankId, dieId, hbmAddr);230+ data.clear();
224 return HCCL_E_PARA;231 return HCCL_E_PARA;
225}232}
226 233 
Mtest/hccl_vm/src/plugin/checker/src/framework/task_graph_generator_v3/ccu_graph_generator_v3/ccu_all_rank_param_recorder_v3.h+3-3
@@ -28,7 +28,7 @@ struct CcuPostNodeMetaV3 {
28 RankId waitRankId{INVALID_RANK_ID};28 RankId waitRankId{INVALID_RANK_ID};
29 uint32_t dieId{INVALID_DIE_ID};29 uint32_t dieId{INVALID_DIE_ID};
30 uint16_t ckeId{INVALID_CCU_CKE};30 uint16_t ckeId{INVALID_CCU_CKE};
31- uint16_t topicId{0};31+ uint16_t remainingCkeMask{0};
32 bool isLocal{false};32 bool isLocal{false};
33};33};
34 34 
@@ -39,8 +39,8 @@ public:
39 void Reset();39 void Reset();
40 HcclResult CheckAllPostMatch();40 HcclResult CheckAllPostMatch();
41 void RegisterPostNode(TaskNode *node, const CcuPostNodeMetaV3 &meta);41 void RegisterPostNode(TaskNode *node, const CcuPostNodeMetaV3 &meta);
42- uint32_t GetPostTopicId(const TaskNode *node) const;42+ uint32_t GetPostRemainingCkeMask(const TaskNode *node) const;
43- void SetPostTopicId(TaskNode *node, uint32_t topicId);43+ void SetPostRemainingCkeMask(TaskNode *node, uint32_t remainingCkeMask);
44 const CcuPostNodeMetaV3 *GetPostNodeMeta(const TaskNode *node) const;44 const CcuPostNodeMetaV3 *GetPostNodeMeta(const TaskNode *node) const;
45 45 
46 HcclResult SetXn(uint32_t rankId, uint32_t dieId, uint16_t xnId, uint64_t xnValue);46 HcclResult SetXn(uint32_t rankId, uint32_t dieId, uint16_t xnId, uint64_t xnValue);
Mtest/hccl_vm/src/plugin/checker/src/framework/task_graph_generator_v3/ccu_graph_generator_v3/ccu_loop_merge_v3.cc+83-20
@@ -17,6 +17,7 @@
17 17 
18#include "ccu_task_transform_v3.h"18#include "ccu_task_transform_v3.h"
19#include "sim_log.h"19#include "sim_log.h"
20+#include "utils/error_codes.h"
20 21 
21namespace HcclSim {22namespace HcclSim {
22namespace TaskGraphGeneratorV3 {23namespace TaskGraphGeneratorV3 {
@@ -256,7 +257,8 @@ bool CcuLoopInstrV3::MergeMemSlices(std::vector<MemSlice> &slices, bool allowOve
256 const uint64_t lastEnd = last.offset + last.len;257 const uint64_t lastEnd = last.offset + last.len;
257 const uint64_t curEnd = cur.offset + cur.len;258 const uint64_t curEnd = cur.offset + cur.len;
258 if (lastEnd > cur.offset && !allowOverlap) {259 if (lastEnd > cur.offset && !allowOverlap) {
259- HCCL_VM_INFO("[ccu_loop_merge_V3] MemSlice overlap between loop expands, instrId={}", instrId);260+ HCCL_VM_WARN("Skip merged-loop optimization because memory slices from different "
261+ "loop expansions overlap, loopInstrId={}", instrId);
260 return false;262 return false;
261 }263 }
262 // 相邻或重叠的切片合并为一个,取并集区间264 // 相邻或重叠的切片合并为一个,取并集区间
@@ -478,14 +480,15 @@ bool CcuLoopV3::CheckResourceConflict() const
478 for (const auto &loopInstrs : loopExpands) {480 for (const auto &loopInstrs : loopExpands) {
479 for (uint16_t ckeId : loopInstrs.GetUsedCKE()) {481 for (uint16_t ckeId : loopInstrs.GetUsedCKE()) {
480 if (!allCKEs.insert(ckeId).second) {482 if (!allCKEs.insert(ckeId).second) {
481- HCCL_VM_INFO("[ccu_loop_merge_V3] loop expand cke conflict, loopInstrId={}, ckeId={}", instrId,483+ HCCL_VM_WARN("Skip merged-loop optimization because different loop expansions reuse "
482- ckeId);484+ "the same CKE resource, loopInstrId={}, ckeId={}", instrId, ckeId);
483 return false;485 return false;
484 }486 }
485 }487 }
486 for (uint16_t msId : loopInstrs.GetUsedMS()) {488 for (uint16_t msId : loopInstrs.GetUsedMS()) {
487 if (!allMSs.insert(msId).second) {489 if (!allMSs.insert(msId).second) {
488- HCCL_VM_INFO("[ccu_loop_merge_V3] loop expand ms conflict, loopInstrId={}, msId={}", instrId, msId);490+ HCCL_VM_WARN("Skip merged-loop optimization because different loop expansions reuse "
491+ "the same MS resource, loopInstrId={}, msId={}", instrId, msId);
489 return false;492 return false;
490 }493 }
491 }494 }
@@ -497,7 +500,8 @@ bool CcuLoopV3::LoopIterationMerge()
497{500{
498 for (auto &loopInstrs : loopExpands) {501 for (auto &loopInstrs : loopExpands) {
499 if (!loopInstrs.MemMerge(true)) {502 if (!loopInstrs.MemMerge(true)) {
500- HCCL_VM_INFO("[ccu_loop_merge_V3] loop iteration merge failed, loopInstrId={}", instrId);503+ HCCL_VM_WARN("Skip merged-loop optimization because memory slices inside one loop "
504+ "iteration cannot be merged, loopInstrId={}", instrId);
501 return false;505 return false;
502 }506 }
503 }507 }
@@ -514,7 +518,8 @@ bool CcuLoopV3::LoopExpandMerge()
514 const size_t instrCnt = loopExpands.front().instrs.size();518 const size_t instrCnt = loopExpands.front().instrs.size();
515 for (const auto &expand : loopExpands) {519 for (const auto &expand : loopExpands) {
516 if (expand.instrs.size() != instrCnt) {520 if (expand.instrs.size() != instrCnt) {
517- HCCL_VM_INFO("[ccu_loop_merge_V3] loop expand instr count mismatch, loopInstrId={}", instrId);521+ HCCL_VM_WARN("Skip merged-loop optimization because loop expansions contain different "
522+ "numbers of instructions, loopInstrId={}", instrId);
518 return false;523 return false;
519 }524 }
520 }525 }
@@ -534,14 +539,15 @@ bool CcuLoopV3::LoopExpandMerge()
534 }539 }
535 auto mergedInstr = loopExpands.front().instrs[instrIdx]->InstrMerge(parallelInstrs);540 auto mergedInstr = loopExpands.front().instrs[instrIdx]->InstrMerge(parallelInstrs);
536 if (mergedInstr == nullptr) {541 if (mergedInstr == nullptr) {
537- HCCL_VM_INFO("[ccu_loop_merge_V3] loop expand instr merge failed, loopInstrId={}, instrIdx={}", instrId,542+ HCCL_VM_WARN("Skip merged-loop optimization because one instruction position cannot be "
538- instrIdx);543+ "merged across loop expansions, loopInstrId={}, instructionIndex={}", instrId, instrIdx);
539 return false;544 return false;
540 }545 }
541 merged.instrs.push_back(std::move(mergedInstr));546 merged.instrs.push_back(std::move(mergedInstr));
542 }547 }
543 if (!merged.MemMerge(false)) {548 if (!merged.MemMerge(false)) {
544- HCCL_VM_INFO("[ccu_loop_merge_V3] loop expand mem merge failed, loopInstrId={}", instrId);549+ HCCL_VM_WARN("Skip merged-loop optimization because merged instruction memory ranges still "
550+ "overlap, loopInstrId={}", instrId);
545 return false;551 return false;
546 }552 }
547 553 
@@ -555,14 +561,26 @@ HcclResult EmitMergedLoopInstrsV3(CcuGraphStateV3 *curCcuTask, uint32_t queId,
555 const CcuLoopInstrsV3 &mergedInstrs, bool &isContinue)561 const CcuLoopInstrsV3 &mergedInstrs, bool &isContinue)
556{562{
557 if (curCcuTask == nullptr) {563 if (curCcuTask == nullptr) {
564+ HCCL_VM_ERROR("{} Failed to emit merged loop instructions because loop graph state is null, "
565+ "queueId={}", MakeErrorCodeText(ErrorCode::GRAPH_LOOP_MERGE_ERROR).c_str(), queId);
558 return HCCL_E_PTR;566 return HCCL_E_PTR;
559 }567 }
560 const RankId rankId = curCcuTask->GetRankId();568 const RankId rankId = curCcuTask->GetRankId();
561 for (const auto &instr : mergedInstrs.instrs) {569 for (const auto &instr : mergedInstrs.instrs) {
562 if (instr == nullptr) {570 if (instr == nullptr) {
571+ HCCL_VM_ERROR("{} Failed to emit one merged loop instruction because the merged instruction "
572+ "entry is null, rankId={}, queueId={}", MakeErrorCodeText(ErrorCode::GRAPH_LOOP_MERGE_ERROR).c_str(),
573+ static_cast<uint32_t>(rankId), queId);
563 return HCCL_E_PTR;574 return HCCL_E_PTR;
564 }575 }
565- CHK_RET(ProcessWaitOps(curCcuTask, queId, instr->waitOps, isContinue));576+ HcclResult ret = ProcessWaitOps(curCcuTask, queId, instr->waitOps, isContinue);
577+ if (ret != HCCL_SUCCESS) {
578+ HCCL_VM_ERROR("{} Failed to process wait conditions before emitting one merged loop "
579+ "instruction, rankId={}, queueId={}, mergedLoopInstr={}",
580+ MakeErrorCodeText(ErrorCode::GRAPH_LOOP_MERGE_ERROR).c_str(), static_cast<uint32_t>(rankId), queId,
581+ instr->Describe());
582+ return ret;
583+ }
566 if (!isContinue) {584 if (!isContinue) {
567 return HCCL_SUCCESS;585 return HCCL_SUCCESS;
568 }586 }
@@ -570,27 +588,72 @@ HcclResult EmitMergedLoopInstrsV3(CcuGraphStateV3 *curCcuTask, uint32_t queId,
570 if (auto trans = std::dynamic_pointer_cast<CcuLoopTransMemV3>(instr)) {588 if (auto trans = std::dynamic_pointer_cast<CcuLoopTransMemV3>(instr)) {
571 CcuNodeRoleV3 role = CcuNodeRoleV3::UNKNOWN;589 CcuNodeRoleV3 role = CcuNodeRoleV3::UNKNOWN;
572 RankId peerRank = INVALID_RANK_ID;590 RankId peerRank = INVALID_RANK_ID;
573- CHK_RET(ResolveTransRole(rankId, trans->srcs, trans->dsts, role, peerRank));591+ ret = ResolveTransRole(rankId, trans->srcs, trans->dsts, role, peerRank);
574- CHK_RET(AddBatchTransMem(rankId, queId, curCcuTask, trans->srcs, trans->dsts, trans->mergedSrcs,592+ if (ret != HCCL_SUCCESS) {
575- trans->mergedDsts, role, peerRank));593+ HCCL_VM_ERROR("{} Failed to determine the transfer direction for one merged loop "
594+ "instruction, rankId={}, queueId={}, mergedLoopInstr={}",
595+ MakeErrorCodeText(ErrorCode::GRAPH_LOOP_MERGE_ERROR).c_str(),
596+ static_cast<uint32_t>(rankId), queId, trans->Describe());
597+ return ret;
598+ }
599+ ret = AddBatchTransMem(rankId, queId, curCcuTask, trans->srcs, trans->dsts, trans->mergedSrcs,
600+ trans->mergedDsts, role, peerRank);
601+ if (ret != HCCL_SUCCESS) {
602+ HCCL_VM_ERROR("{} Failed to emit one merged loop transfer task, rankId={}, queueId={}, "
603+ "mergedLoopInstr={}", MakeErrorCodeText(ErrorCode::GRAPH_LOOP_MERGE_ERROR).c_str(),
604+ static_cast<uint32_t>(rankId), queId, trans->Describe());
605+ return ret;
606+ }
576 } else if (auto reduce = std::dynamic_pointer_cast<CcuLoopReduceV3>(instr)) {607 } else if (auto reduce = std::dynamic_pointer_cast<CcuLoopReduceV3>(instr)) {
577 CcuNodeRoleV3 role = CcuNodeRoleV3::UNKNOWN;608 CcuNodeRoleV3 role = CcuNodeRoleV3::UNKNOWN;
578 RankId peerRank = INVALID_RANK_ID;609 RankId peerRank = INVALID_RANK_ID;
579- CHK_RET(ResolveReduceRole(rankId, reduce->srcs, reduce->dsts, role, peerRank));610+ ret = ResolveReduceRole(rankId, reduce->srcs, reduce->dsts, role, peerRank);
580- CHK_RET(AddBatchReduce(rankId, queId, curCcuTask, reduce->srcs, reduce->dsts, reduce->mergedSrcs,611+ if (ret != HCCL_SUCCESS) {
581- reduce->mergedDsts, reduce->dataType, reduce->reduceOp, role, peerRank));612+ HCCL_VM_ERROR("{} Failed to determine the reduce direction for one merged loop "
613+ "instruction, rankId={}, queueId={}, mergedLoopInstr={}",
614+ MakeErrorCodeText(ErrorCode::GRAPH_LOOP_MERGE_ERROR).c_str(),
615+ static_cast<uint32_t>(rankId), queId, reduce->Describe());
616+ return ret;
617+ }
618+ ret = AddBatchReduce(rankId, queId, curCcuTask, reduce->srcs, reduce->dsts, reduce->mergedSrcs,
619+ reduce->mergedDsts, reduce->dataType, reduce->reduceOp, role, peerRank);
620+ if (ret != HCCL_SUCCESS) {
621+ HCCL_VM_ERROR("{} Failed to emit one merged loop reduce task, rankId={}, queueId={}, "
622+ "mergedLoopInstr={}", MakeErrorCodeText(ErrorCode::GRAPH_LOOP_MERGE_ERROR).c_str(),
623+ static_cast<uint32_t>(rankId), queId, reduce->Describe());
624+ return ret;
625+ }
582 } else if (auto clear = std::dynamic_pointer_cast<CcuLoopClearCkeV3>(instr)) {626 } else if (auto clear = std::dynamic_pointer_cast<CcuLoopClearCkeV3>(instr)) {
583 if (clear->clearMask != 0) {627 if (clear->clearMask != 0) {
584- HCCL_VM_INFO("[ccu_loop_merge_V3] clear cke mask is not supported in merged loop, instrId={}",628+ HCCL_VM_WARN("{} Skip merged-loop optimization because ClearCKE uses a non-zero "
585- clear->instrId);629+ "mask, instrId={}, clearMask=0x{:x}", MakeErrorCodeText(ErrorCode::GRAPH_UNSUPPORTED),
630+ clear->instrId, clear->clearMask);
586 return HCCL_E_NOT_SUPPORT;631 return HCCL_E_NOT_SUPPORT;
587 }632 }
588 } else {633 } else {
634+ HCCL_VM_ERROR("{} Failed to emit one merged loop instruction because its merged instruction "
635+ "type is not supported, rankId={}, queueId={}, mergedLoopInstr={}",
636+ MakeErrorCodeText(ErrorCode::GRAPH_LOOP_MERGE_ERROR).c_str(), static_cast<uint32_t>(rankId), queId,
637+ instr->Describe());
589 return HCCL_E_NOT_SUPPORT;638 return HCCL_E_NOT_SUPPORT;
590 }639 }
591 640 
592- CHK_RET(ProcessSetOps(curCcuTask, queId, instr->setOps));641+ ret = ProcessSetOps(curCcuTask, queId, instr->setOps);
593- CHK_RET(ClearWaitOps(curCcuTask, queId, instr->waitOps));642+ if (ret != HCCL_SUCCESS) {
643+ HCCL_VM_ERROR("{} Failed to apply set masks after emitting one merged loop instruction, "
644+ "rankId={}, queueId={}, mergedLoopInstr={}",
645+ MakeErrorCodeText(ErrorCode::GRAPH_LOOP_MERGE_ERROR).c_str(), static_cast<uint32_t>(rankId), queId,
646+ instr->Describe());
647+ return ret;
648+ }
649+ ret = ClearWaitOps(curCcuTask, queId, instr->waitOps);
650+ if (ret != HCCL_SUCCESS) {
651+ HCCL_VM_ERROR("{} Failed to clear consumed wait masks after emitting one merged loop "
652+ "instruction, rankId={}, queueId={}, mergedLoopInstr={}",
653+ MakeErrorCodeText(ErrorCode::GRAPH_LOOP_MERGE_ERROR).c_str(), static_cast<uint32_t>(rankId), queId,
654+ instr->Describe());
655+ return ret;
656+ }
594 }657 }
595 return HCCL_SUCCESS;658 return HCCL_SUCCESS;
596}659}
Mtest/hccl_vm/src/plugin/checker/src/framework/task_graph_generator_v3/ccu_graph_generator_v3/ccu_task_common_v3.cc+46-20文件内容审核中,请稍后刷新重试
Mtest/hccl_vm/src/plugin/checker/src/framework/task_graph_generator_v3/ccu_graph_generator_v3/ccu_task_common_v3.h+6-5
@@ -59,7 +59,7 @@ struct CcuNodeMetaV3 {
59 RankId peerRank{INVALID_RANK_ID};59 RankId peerRank{INVALID_RANK_ID};
60 uint32_t dieId{INVALID_DIE_ID};60 uint32_t dieId{INVALID_DIE_ID};
61 uint16_t ckeId{INVALID_CCU_CKE};61 uint16_t ckeId{INVALID_CCU_CKE};
62- uint16_t topicId{0};62+ uint16_t remainingCkeMask{0};
63 bool invalidPost{false};63 bool invalidPost{false};
64};64};
65 65 
@@ -98,20 +98,21 @@ public:
98 HcclResult InitInstrInfo(StorageManager &storage);98 HcclResult InitInstrInfo(StorageManager &storage);
99 HcclResult CreateHeadNode();99 HcclResult CreateHeadNode();
100 HcclResult AppendGeneratedNode(std::unique_ptr<TaskNode> node, RankId rankId, uint32_t queId,100 HcclResult AppendGeneratedNode(std::unique_ptr<TaskNode> node, RankId rankId, uint32_t queId,
101- CcuNodeRoleV3 role, TaskNode *&outNode, RankId peerRank = INVALID_RANK_ID, uint16_t topicId = 0,101+ CcuNodeRoleV3 role, TaskNode *&outNode, RankId peerRank = INVALID_RANK_ID, uint16_t remainingCkeMask = 0,
102 uint32_t dieId = INVALID_DIE_ID, uint16_t ckeId = INVALID_CCU_CKE, bool invalidPost = false);102 uint32_t dieId = INVALID_DIE_ID, uint16_t ckeId = INVALID_CCU_CKE, bool invalidPost = false);
103 HcclResult AppendGeneratedNode(std::unique_ptr<TaskNode> node, const TaskPosition &position,103 HcclResult AppendGeneratedNode(std::unique_ptr<TaskNode> node, const TaskPosition &position,
104- CcuNodeRoleV3 role, TaskNode *&outNode, RankId peerRank = INVALID_RANK_ID, uint16_t topicId = 0,104+ CcuNodeRoleV3 role, TaskNode *&outNode, RankId peerRank = INVALID_RANK_ID, uint16_t remainingCkeMask = 0,
105 uint32_t dieId = INVALID_DIE_ID, uint16_t ckeId = INVALID_CCU_CKE, bool invalidPost = false);105 uint32_t dieId = INVALID_DIE_ID, uint16_t ckeId = INVALID_CCU_CKE, bool invalidPost = false);
106 106 
107 void SetNodeMeta(TaskNode *node, const CcuNodeMetaV3 &meta);107 void SetNodeMeta(TaskNode *node, const CcuNodeMetaV3 &meta);
108 const CcuNodeMetaV3 *GetNodeMeta(const TaskNode *node) const;108 const CcuNodeMetaV3 *GetNodeMeta(const TaskNode *node) const;
109 CcuNodeRoleV3 GetNodeRole(const TaskNode *node) const;109 CcuNodeRoleV3 GetNodeRole(const TaskNode *node) const;
110 RankId GetNodePeerRank(const TaskNode *node) const;110 RankId GetNodePeerRank(const TaskNode *node) const;
111- uint16_t GetNodeTopicId(const TaskNode *node) const;111+ uint16_t GetNodeRemainingCkeMask(const TaskNode *node) const;
112- void SetNodeTopicId(TaskNode *node, uint16_t topicId);112+ void SetNodeRemainingCkeMask(TaskNode *node, uint16_t remainingCkeMask);
113 void SetNodePeerRank(TaskNode *node, RankId peerRank);113 void SetNodePeerRank(TaskNode *node, RankId peerRank);
114 std::string Describe() const;114 std::string Describe() const;
115+ std::string DescribeNextInstructionByQueue() const;
115 116 
116 void GetSqe(uint32_t queId, uint16_t sqeArgsId, uint64_t &argVal);117 void GetSqe(uint32_t queId, uint16_t sqeArgsId, uint64_t &argVal);
117 void GetDieId(uint32_t queId, uint32_t &dieId) const;118 void GetDieId(uint32_t queId, uint32_t &dieId) const;
Mtest/hccl_vm/src/plugin/checker/src/framework/task_graph_generator_v3/ccu_graph_generator_v3/ccu_task_transform_instruct_a5_v3.cc+373-182
Mtest/hccl_vm/src/plugin/checker/src/framework/task_graph_generator_v3/ccu_graph_generator_v3/ccu_task_transform_v3.cc+95-35
@@ -25,22 +25,23 @@
25#include "ccu_all_rank_param_recorder_v3.h"25#include "ccu_all_rank_param_recorder_v3.h"
26#include "sim_log.h"26#include "sim_log.h"
27#include "storage_manager.h"27#include "storage_manager.h"
28+#include "utils/error_codes.h"
28 29 
29namespace HcclSim {30namespace HcclSim {
30namespace TaskGraphGeneratorV3 {31namespace TaskGraphGeneratorV3 {
31 32 
32std::unique_ptr<InstructMapBase> g_instrMap;33std::unique_ptr<InstructMapBase> g_instrMap;
33 34 
34-// 作用:读取 post 节点当前挂着topicId35+// 作用:读取 post 节点当前还未被 wait 消费掉CKE mask
35-u32 GetTopicId(TaskNode *post)36+u32 GetPostRemainingCkeMask(TaskNode *post)
36{37{
37- return AllRankParamRecorder::Global()->GetPostTopicId(post);38+ return AllRankParamRecorder::Global()->GetPostRemainingCkeMask(post);
38}39}
39 40 
40-// 作用:更新 post 节点剩余未消费的 topicId41+// 作用:更新 post 节点当前还被 wait 消费CKE mask
41-void SetTopicId(TaskNode *post, u32 topicId)42+void SetPostRemainingCkeMask(TaskNode *post, u32 remainingCkeMask)
42{43{
43- AllRankParamRecorder::Global()->SetPostTopicId(post, topicId);44+ AllRankParamRecorder::Global()->SetPostRemainingCkeMask(post, remainingCkeMask);
44}45}
45 46 
46// 作用:按 wait mask 生成本地/远端 wait 节点,并补齐 post 到 wait 的依赖边。47// 作用:按 wait mask 生成本地/远端 wait 节点,并补齐 post 到 wait 的依赖边。
@@ -55,13 +56,13 @@ void GenWaitNode(CcuGraphStateV3 *curCcuTask, uint32_t queId, uint16_t waitCKEId
55 TaskNode *localWaitNode = nullptr;56 TaskNode *localWaitNode = nullptr;
56 TaskNode *remoteWaitNode = nullptr;57 TaskNode *remoteWaitNode = nullptr;
57 58 
58- // 只有目标 CKE bit 全部就绪后才真正生成 wait 节点;此时会把待消费的 topic bit59+ // 只有目标 CKE bit 全部就绪后才真正生成 wait 节点;此时会把待消费的 CKE mask bit
59 // 反查回它实际依赖的那些 post 节点。60 // 反查回它实际依赖的那些 post 节点。
60 std::set<TaskNode *> &seenPosts = AllRankParamRecorder::Global()->seenPost[rankId][dieId][waitCKEId];61 std::set<TaskNode *> &seenPosts = AllRankParamRecorder::Global()->seenPost[rankId][dieId][waitCKEId];
61 std::vector<TaskNode *> localPosts;62 std::vector<TaskNode *> localPosts;
62 for (auto *post : seenPosts) {63 for (auto *post : seenPosts) {
63- u32 topicId = GetTopicId(post);64+ u32 remainingCkeMask = GetPostRemainingCkeMask(post);
64- uint16_t curPostMask = topicId & waitCKEMask;65+ uint16_t curPostMask = remainingCkeMask & waitCKEMask;
65 if (curPostMask == 0) {66 if (curPostMask == 0) {
66 continue;67 continue;
67 }68 }
@@ -88,8 +89,8 @@ void GenWaitNode(CcuGraphStateV3 *curCcuTask, uint32_t queId, uint16_t waitCKEId
88 89 
89 for (auto it = seenPosts.begin(); it != seenPosts.end();) {90 for (auto it = seenPosts.begin(); it != seenPosts.end();) {
90 auto *post = *it;91 auto *post = *it;
91- u32 topicId = GetTopicId(post);92+ u32 remainingCkeMask = GetPostRemainingCkeMask(post);
92- uint16_t curPostMask = topicId & waitCKEMask;93+ uint16_t curPostMask = remainingCkeMask & waitCKEMask;
93 if (curPostMask == 0) {94 if (curPostMask == 0) {
94 ++it;95 ++it;
95 continue;96 continue;
@@ -106,9 +107,9 @@ void GenWaitNode(CcuGraphStateV3 *curCcuTask, uint32_t queId, uint16_t waitCKEId
106 }107 }
107 (void)CollectBilateralWaitInfo(curCcuTask, queId, remoteWaitNode);108 (void)CollectBilateralWaitInfo(curCcuTask, queId, remoteWaitNode);
108 }109 }
109- topicId = topicId & (~waitCKEMask);110+ remainingCkeMask = remainingCkeMask & (~waitCKEMask);
110- SetTopicId(post, topicId);111+ SetPostRemainingCkeMask(post, remainingCkeMask);
111- if (topicId == 0) {112+ if (remainingCkeMask == 0) {
112 it = seenPosts.erase(it);113 it = seenPosts.erase(it);
113 } else {114 } else {
114 ++it;115 ++it;
@@ -120,16 +121,17 @@ void GenWaitNode(CcuGraphStateV3 *curCcuTask, uint32_t queId, uint16_t waitCKEId
120HcclResult ProcessWaitMask(RankId rankId, uint32_t dieId, CcuGraphStateV3 *curCcuTask, uint32_t queId,121HcclResult ProcessWaitMask(RankId rankId, uint32_t dieId, CcuGraphStateV3 *curCcuTask, uint32_t queId,
121 uint16_t waitCKEId, uint16_t waitCKEMask, bool &isContinue)122 uint16_t waitCKEId, uint16_t waitCKEMask, bool &isContinue)
122{123{
123- HCCL_VM_DEBUG("[ProcessWaitMask] Enter...rank {:d}, die{:d}, que{:d}, waitCke{:d}, waitMask{:d}",124+ HCCL_VM_DEBUG("Evaluate CCU wait condition, rankId={}, dieId={}, queueId={}, waitCkeId={}, "
124- rankId, static_cast<uint32_t>(dieId), queId, waitCKEId, waitCKEMask);125+ "requiredMask=0x{:04x}", rankId, static_cast<uint32_t>(dieId), queId, waitCKEId, waitCKEMask);
125 if (waitCKEMask != 0x0000) {126 if (waitCKEMask != 0x0000) {
126 uint16_t ckeValue = 0;127 uint16_t ckeValue = 0;
127 CHK_RET(AllRankParamRecorder::Global()->GetCKE(rankId, dieId, waitCKEId, ckeValue));128 CHK_RET(AllRankParamRecorder::Global()->GetCKE(rankId, dieId, waitCKEId, ckeValue));
128 if ((ckeValue & waitCKEMask) != waitCKEMask) {129 if ((ckeValue & waitCKEMask) != waitCKEMask) {
129 // 当前 queue 在这里暂停,直到生产侧把所需的 wait bit 全部 post 完成。130 // 当前 queue 在这里暂停,直到生产侧把所需的 wait bit 全部 post 完成。
130 isContinue = false;131 isContinue = false;
131- HCCL_VM_DEBUG("[ProcessWaitMask] Does not meet the condition, continue: ckeId: {:d}, "132+ HCCL_VM_DEBUG("Pause this CCU queue because the wait condition is not satisfied yet, "
132- "ckeValue: {:d}, mask: {:x}", waitCKEId, ckeValue, waitCKEMask);133+ "rankId={}, dieId={}, queueId={}, waitCkeId={}, currentCkeValue=0x{:04x}, requiredMask=0x{:04x}",
134+ rankId, static_cast<uint32_t>(dieId), queId, waitCKEId, ckeValue, waitCKEMask);
133 return HCCL_SUCCESS;135 return HCCL_SUCCESS;
134 }136 }
135 GenWaitNode(curCcuTask, queId, waitCKEId, waitCKEMask);137 GenWaitNode(curCcuTask, queId, waitCKEId, waitCKEMask);
@@ -142,11 +144,17 @@ HcclResult TransformInstr(const CcuRep::CcuInstr *instr, CcuGraphStateV3 *curCcu
142 bool &isContinue)144 bool &isContinue)
143{145{
144 if (g_instrMap == nullptr) {146 if (g_instrMap == nullptr) {
145- HCCL_ERROR("g_instrMap error");147+ HCCL_VM_ERROR("{} Failed to create the CCU instruction translator, rankId={}, queueId={}",
148+ MakeErrorCodeText(ErrorCode::CHECKER_RUNTIME_ERROR).c_str(),
149+ curCcuTask == nullptr ? std::string("null") : std::to_string(curCcuTask->GetRankId()), queId);
146 return HCCL_E_INTERNAL;150 return HCCL_E_INTERNAL;
147 }151 }
148 if (!g_instrMap->IsSupported(instr->header.header)) {152 if (!g_instrMap->IsSupported(instr->header.header)) {
149- HCCL_ERROR("ins type not supported %hu", instr->header.header);153+ HCCL_VM_ERROR("{} This CCU instruction type is not supported by CheckerV3 graph expansion, "
154+ "rankId={}, queueId={}, instructionHeader=0x{:04x}",
155+ MakeErrorCodeText(ErrorCode::GRAPH_UNSUPPORTED).c_str(),
156+ curCcuTask == nullptr ? std::string("null") : std::to_string(curCcuTask->GetRankId()), queId,
157+ instr->header.header);
150 return HCCL_E_INTERNAL;158 return HCCL_E_INTERNAL;
151 }159 }
152 return g_instrMap.get()->Transform(instr, curCcuTask, queId, isContinue, nullptr);160 return g_instrMap.get()->Transform(instr, curCcuTask, queId, isContinue, nullptr);
@@ -177,9 +185,19 @@ HcclResult TransformInstrQue(TaskNode *node, CcuGraphStateV3 *curCcuTask, uint32
177 // 单个 queue 严格按指令顺序推进;跨 queue 的协同只通过生成出来的 wait/post 边表达。185 // 单个 queue 严格按指令顺序推进;跨 queue 的协同只通过生成出来的 wait/post 边表达。
178 while (pos < endInstrId) {186 while (pos < endInstrId) {
179 u32 prePos = pos;187 u32 prePos = pos;
180- HCCL_VM_DEBUG("Current process ccu graph: {}, instruction id={}, header={}",188+ const uint32_t instrId = curCcuTask->startInstrIdInQue[queId] + pos;
181- curCcuTask->Describe(), pos, microCodeQue.instrVec[pos].header.header);189+ HCCL_VM_DEBUG("Translating one CCU instruction, graph={}, instructionId={}, "
182- CHK_RET(TransformInstr(&microCodeQue.instrVec[pos], curCcuTask, queId, isContinue));190+ "instructionHeader=0x{:04x}",
191+ curCcuTask->Describe(), instrId, microCodeQue.instrVec[pos].header.header);
192+ HcclResult ret = TransformInstr(&microCodeQue.instrVec[pos], curCcuTask, queId, isContinue);
193+ if (ret != HCCL_SUCCESS) {
194+ HCCL_VM_ERROR("{} Failed to translate one CCU instruction, rankId={}, queueId={}, "
195+ "instructionId={}, instructionHeader=0x{:04x}, ret={}, graph={}",
196+ MakeErrorCodeText(ErrorCode::GRAPH_TRANSLATE_FAILED).c_str(), curCcuTask->GetRankId(), queId,
197+ instrId, microCodeQue.instrVec[pos].header.header, static_cast<uint32_t>(ret),
198+ curCcuTask->Describe().c_str());
199+ return ret;
200+ }
183 if (!isContinue) {201 if (!isContinue) {
184 return HCCL_SUCCESS;202 return HCCL_SUCCESS;
185 }203 }
@@ -192,7 +210,8 @@ HcclResult TransformInstrQue(TaskNode *node, CcuGraphStateV3 *curCcuTask, uint32
192 curCcuTask->isGenGraphed = std::all_of(curCcuTask->instrQueStatus.begin(),210 curCcuTask->isGenGraphed = std::all_of(curCcuTask->instrQueStatus.begin(),
193 curCcuTask->instrQueStatus.end(), [](bool b) { return b; });211 curCcuTask->instrQueStatus.end(), [](bool b) { return b; });
194 212 
195- HCCL_VM_DEBUG("[TransformInstrQue] end...");213+ HCCL_VM_DEBUG("Finished translating one CCU queue, rankId={}, queueId={}",
214+ curCcuTask->GetRankId(), queId);
196 return HCCL_SUCCESS;215 return HCCL_SUCCESS;
197}216}
198 217 
@@ -566,7 +585,8 @@ HcclResult BuildLoopParallelGroupV3(const std::vector<LoopInfoV3> &loopGroup,
566 group.beforeNodes = GetLoopBeforeNodesV3(loopGroup.front().loopStart, originalQueueId, compressedBoundary);585 group.beforeNodes = GetLoopBeforeNodesV3(loopGroup.front().loopStart, originalQueueId, compressedBoundary);
567 group.afterNodes = GetSameQueueChildrenV3(loopGroup.back().loopEnd, originalQueueId);586 group.afterNodes = GetSameQueueChildrenV3(loopGroup.back().loopEnd, originalQueueId);
568 if (group.beforeNodes.empty() || group.afterNodes.empty()) {587 if (group.beforeNodes.empty() || group.afterNodes.empty()) {
569- HCCL_ERROR("Loop parallel group boundary is invalid.");588+ HCCL_VM_ERROR("{} Failed to split one CCU loop-parallel group because its entry or exit "
589+ "boundary could not be determined.", MakeErrorCodeText(ErrorCode::GRAPH_STRUCTURE_INVALID).c_str());
570 return HCCL_E_INTERNAL;590 return HCCL_E_INTERNAL;
571 }591 }
572 592 
@@ -1078,7 +1098,11 @@ HcclResult GenerateRawCcuGraphStateV3(TaskNode *node, CcuGraphStateV3 &state)
1078 ++unmatchedCnt;1098 ++unmatchedCnt;
1079 }1099 }
1080 if (unmatchedCnt > state.queueNum_) {1100 if (unmatchedCnt > state.queueNum_) {
1081- HCCL_ERROR("deadLocking occurs due to mismatch.");1101+ HCCL_VM_ERROR("{} CCU raw graph generation is stuck inside one graph. All queues are "
1102+ "blocked and no instruction can advance, rankId={}, queueCount={}, nextInstructionByQueue={}, "
1103+ "blockedGraphSummary={}",
1104+ MakeErrorCodeText(ErrorCode::GRAPH_DEADLOCK).c_str(), state.GetRankId(), state.queueNum_,
1105+ state.DescribeNextInstructionByQueue().c_str(), state.Describe().c_str());
1082 return HCCL_E_INTERNAL;1106 return HCCL_E_INTERNAL;
1083 }1107 }
1084 }1108 }
@@ -1111,18 +1135,16 @@ void PrintGeneratedCcuGraphsV3(const std::vector<std::unique_ptr<CcuGraphRuntime
1111 return lhs->ccuGraph->GetNodeId() < rhs->ccuGraph->GetNodeId();1135 return lhs->ccuGraph->GetNodeId() < rhs->ccuGraph->GetNodeId();
1112 });1136 });
1113 1137 
1114- HCCL_VM_INFO("=======================================================");1138+ HCCL_VM_INFO("Printing all generated CCU graphs before queue parallelization, graphCount={}",
1115- HCCL_VM_INFO("[PrintCcuGraphV3] All generated CCU graphs before queue parallelization, graphCount={}",
1116 orderedRuntimes.size());1139 orderedRuntimes.size());
1117 for (const CcuGraphRuntimeV3 *runtime : orderedRuntimes) {1140 for (const CcuGraphRuntimeV3 *runtime : orderedRuntimes) {
1118 if (runtime == nullptr || runtime->state == nullptr) {1141 if (runtime == nullptr || runtime->state == nullptr) {
1119 continue;1142 continue;
1120 }1143 }
1121- HCCL_VM_INFO("[PrintCcuGraphV3] ccuGraphNodeId={}, rankId={}", runtime->ccuGraph->GetNodeId(),1144+ HCCL_VM_INFO("CCU graph detail, ccuGraphNodeId={}, rankId={}", runtime->ccuGraph->GetNodeId(),
1122 runtime->ccuGraph->GetRankId());1145 runtime->ccuGraph->GetRankId());
1123 PrintCcuGraph(runtime->state->ccuHeadTaskNode);1146 PrintCcuGraph(runtime->state->ccuHeadTaskNode);
1124 }1147 }
1125- HCCL_VM_INFO("=======================================================");
1126}1148}
1127 1149 
1128// 作用:把已经展开完成的内部 CCU 子图重新接回主图中的占位节点位置。1150// 作用:把已经展开完成的内部 CCU 子图重新接回主图中的占位节点位置。
@@ -1205,7 +1227,24 @@ HcclResult GenerateRawCcuGraphsV3(const CcuGraphsGenerateInputV3 &input,
1205 size_t generatedCount = 0;1227 size_t generatedCount = 0;
1206 while (!candNode.empty()) {1228 while (!candNode.empty()) {
1207 if (unmatchedCnt >= candNode.size()) {1229 if (unmatchedCnt >= candNode.size()) {
1208- HCCL_ERROR("deadLocking occurs due to mismatch.");1230+ TaskNode *firstBlockedGraph = candNode.front();
1231+ const CcuGraphRuntimeV3 *blockedRuntime = nullptr;
1232+ auto runtimeIter = runtimeByNodeId.find(firstBlockedGraph == nullptr ?
1233+ INVALID_NODE_ID : firstBlockedGraph->GetNodeId());
1234+ if (runtimeIter != runtimeByNodeId.end()) {
1235+ blockedRuntime = runtimeIter->second;
1236+ }
1237+ HCCL_VM_ERROR("{} CCU multi-graph raw generation is stuck. Some CCU graphs are still "
1238+ "blocked by dependencies, but no graph can advance, blockedGraphCount={}, rankId={}, queueCount={}, "
1239+ "nextInstructionByQueue={}, firstBlockedGraph={}",
1240+ MakeErrorCodeText(ErrorCode::GRAPH_DEADLOCK).c_str(), candNode.size(),
1241+ blockedRuntime == nullptr || blockedRuntime->state == nullptr ? std::string("null") :
1242+ std::to_string(blockedRuntime->state->GetRankId()),
1243+ blockedRuntime == nullptr || blockedRuntime->state == nullptr ? 0U :
1244+ blockedRuntime->state->queueNum_,
1245+ blockedRuntime == nullptr || blockedRuntime->state == nullptr ? "[]" :
1246+ blockedRuntime->state->DescribeNextInstructionByQueue().c_str(),
1247+ firstBlockedGraph == nullptr ? "node=null" : firstBlockedGraph->Describe().c_str());
1209 return HCCL_E_INTERNAL;1248 return HCCL_E_INTERNAL;
1210 }1249 }
1211 1250 
@@ -1273,8 +1312,19 @@ HcclResult GenerateRawCcuGraphsV3(const CcuGraphsGenerateInputV3 &input,
1273 }1312 }
1274 1313 
1275 if (generatedCount != input.ccuGraphs.size()) {1314 if (generatedCount != input.ccuGraphs.size()) {
1276- HCCL_ERROR("CCU graph global expansion finished with unfinished graph, generated=%zu, total=%zu",1315+ const CcuGraphRuntimeV3 *firstUnfinishedRuntime = nullptr;
1277- generatedCount, input.ccuGraphs.size());1316+ for (const auto &runtime : runtimes) {
1317+ if (runtime != nullptr && !runtime->generated) {
1318+ firstUnfinishedRuntime = runtime.get();
1319+ break;
1320+ }
1321+ }
1322+ HCCL_VM_ERROR("{} CCU raw graph generation ended before all CCU graphs were finished, "
1323+ "generatedGraphCount={}, totalGraphCount={}, firstUnfinishedGraph={}",
1324+ MakeErrorCodeText(ErrorCode::GRAPH_STRUCTURE_INVALID).c_str(),
1325+ generatedCount, input.ccuGraphs.size(), firstUnfinishedRuntime == nullptr ||
1326+ firstUnfinishedRuntime->ccuGraph == nullptr ? "node=null" :
1327+ firstUnfinishedRuntime->ccuGraph->Describe().c_str());
1278 return HCCL_E_INTERNAL;1328 return HCCL_E_INTERNAL;
1279 }1329 }
1280 return HCCL_SUCCESS;1330 return HCCL_SUCCESS;
@@ -1293,8 +1343,18 @@ HcclResult FinalizeGeneratedCcuGraphsV3(TaskGraphGeneratorV3 &graph,
1293 ++finalizedCount;1343 ++finalizedCount;
1294 }1344 }
1295 if (finalizedCount != runtimes.size()) {1345 if (finalizedCount != runtimes.size()) {
1296- HCCL_ERROR("CCU graph global finalization finished with unfinished graph, finalized=%zu, total=%zu",1346+ const CcuGraphRuntimeV3 *firstUnfinishedRuntime = nullptr;
1297- finalizedCount, runtimes.size());1347+ for (const auto &runtime : runtimes) {
1348+ if (runtime != nullptr && !runtime->finalized) {
1349+ firstUnfinishedRuntime = runtime.get();
1350+ break;
1351+ }
1352+ }
1353+ HCCL_VM_ERROR("{} CCU graph finalization ended before all generated graphs were attached back "
1354+ "to the main graph, finalizedGraphCount={}, totalGraphCount={}, firstUnfinishedGraph={}",
1355+ MakeErrorCodeText(ErrorCode::GRAPH_STRUCTURE_INVALID).c_str(), finalizedCount, runtimes.size(),
1356+ firstUnfinishedRuntime == nullptr || firstUnfinishedRuntime->ccuGraph == nullptr ? "node=null" :
1357+ firstUnfinishedRuntime->ccuGraph->Describe().c_str());
1298 return HCCL_E_INTERNAL;1358 return HCCL_E_INTERNAL;
1299 }1359 }
1300 return HCCL_SUCCESS;1360 return HCCL_SUCCESS;
Mtest/hccl_vm/src/plugin/checker/src/framework/task_graph_generator_v3/ccu_graph_generator_v3/ccu_task_transform_v3.h+2-2
@@ -56,9 +56,9 @@ HcclResult ExpandCcuGraphsV3(const CcuGraphsGenerateInputV3 &input, CcuGraphGene
56 56 
57HcclResult GetHcclDataTypeFromCCUDataType(uint16_t ccuDataType, uint16_t ccuReduceType, DataType& dataType);57HcclResult GetHcclDataTypeFromCCUDataType(uint16_t ccuDataType, uint16_t ccuReduceType, DataType& dataType);
58 58 
59-uint32_t GetTopicId(TaskNode* post);59+uint32_t GetPostRemainingCkeMask(TaskNode* post);
60 60 
61-void SetTopicId(TaskNode* post, u32 topicId);61+void SetPostRemainingCkeMask(TaskNode* post, u32 remainingCkeMask);
62 62 
63HcclResult ProcessWaitMask(RankId rankId, uint32_t dieId, CcuGraphStateV3 *curCcuTask, uint32_t queId,63HcclResult ProcessWaitMask(RankId rankId, uint32_t dieId, CcuGraphStateV3 *curCcuTask, uint32_t queId,
64 uint16_t waitCKEId, uint16_t waitCKEMask, bool& isContinue);64 uint16_t waitCKEId, uint16_t waitCKEMask, bool& isContinue);
Mtest/hccl_vm/src/plugin/checker/src/framework/task_graph_generator_v3/task_def_v3.h+20-0
@@ -55,6 +55,26 @@ enum class MemType : uint8_t {
55 INVALID = UINT8_MAX,55 INVALID = UINT8_MAX,
56};56};
57 57 
58+inline const char *DescribeMemType(MemType memType)
59+{
60+ switch (memType) {
61+ case MemType::INPUT:
62+ return "INPUT";
63+ case MemType::OUTPUT:
64+ return "OUTPUT";
65+ case MemType::CCL:
66+ return "CCL";
67+ case MemType::UB_AIV:
68+ return "UB_AIV";
69+ case MemType::FLAG_AIV:
70+ return "FLAG_AIV";
71+ case MemType::MS_CCU:
72+ return "MS_CCU";
73+ default:
74+ return "invalid";
75+ }
76+}
77+ 
58enum class BoundaryType : uint8_t {78enum class BoundaryType : uint8_t {
59 MAIN_GRAPH = 0,79 MAIN_GRAPH = 0,
60 CCU_SUB_GRAPH,80 CCU_SUB_GRAPH,
Mtest/hccl_vm/src/plugin/checker/src/framework/task_graph_generator_v3/task_graph_generator_v3.cc+62-84
@@ -20,6 +20,7 @@
20#include "ccu_graph_generator_v3/ccu_task_transform_v3.h"20#include "ccu_graph_generator_v3/ccu_task_transform_v3.h"
21#include "sim_log.h"21#include "sim_log.h"
22#include "storage_manager.h"22#include "storage_manager.h"
23+#include "utils/error_codes.h"
23 24 
24namespace HcclSim {25namespace HcclSim {
25namespace TaskGraphGeneratorV3 {26namespace TaskGraphGeneratorV3 {
@@ -27,46 +28,6 @@ namespace {
27using SeenLocalRecords = std::vector<NodeId>;28using SeenLocalRecords = std::vector<NodeId>;
28using SeenInterRankRecords = std::map<RankId, std::map<RankId, std::vector<NodeId>>>;29using SeenInterRankRecords = std::map<RankId, std::map<RankId, std::vector<NodeId>>>;
29 30 
30-const char *TaskTypeName(TaskType type)
31-{
32- switch (type) {
33- case TaskType::TRANS_MEM:
34- return "TRANS_MEM";
35- case TaskType::BATCH_TRANS_MEM:
36- return "BATCH_TRANS_MEM";
37- case TaskType::REDUCE:
38- return "REDUCE";
39- case TaskType::BATCH_REDUCE:
40- return "BATCH_REDUCE";
41- case TaskType::RECORD:
42- return "RECORD";
43- case TaskType::WAIT:
44- return "WAIT";
45- case TaskType::CCU_GRAPH:
46- return "CCU_GRAPH";
47- case TaskType::AIV_GRAPH:
48- return "AIV_GRAPH";
49- case TaskType::AIV_SET_FLAG:
50- return "AIV_SET_FLAG";
51- case TaskType::AIV_WAIT_FLAG:
52- return "AIV_WAIT_FLAG";
53- case TaskType::AIV_PIPE_BARRIER:
54- return "AIV_PIPE_BARRIER";
55- case TaskType::AIV_SYNC_ALL:
56- return "AIV_SYNC_ALL";
57- case TaskType::AIV_SEND_FLAG:
58- return "AIV_SEND_FLAG";
59- case TaskType::AIV_RECV_FLAG:
60- return "AIV_RECV_FLAG";
61- case TaskType::START:
62- return "START";
63- case TaskType::END:
64- return "END";
65- default:
66- return "INVALID";
67- }
68-}
69- 
70std::string NodeIdsToString(const std::vector<TaskNode *> &nodes)31std::string NodeIdsToString(const std::vector<TaskNode *> &nodes)
71{32{
72 std::ostringstream os;33 std::ostringstream os;
@@ -182,8 +143,11 @@ HcclResult TaskGraphGeneratorV3::RemoveEdge(NodeId parentNodeId, NodeId childNod
182 TaskNode *parentNode = GetNode(parentNodeId);143 TaskNode *parentNode = GetNode(parentNodeId);
183 TaskNode *childNode = GetNode(childNodeId);144 TaskNode *childNode = GetNode(childNodeId);
184 if (parentNode == nullptr || childNode == nullptr) {145 if (parentNode == nullptr || childNode == nullptr) {
185- HCCL_VM_ERROR("[TaskGraphGeneratorV3][RemoveEdge] Invalid edge, parent={}, child={}",146+ HCCL_VM_ERROR("{} Failed to remove one graph edge because the parent or child node does not "
186- parentNodeId, childNodeId);147+ "exist, parentNodeId={}, childNodeId={}, parentNode={}, childNode={}",
148+ MakeErrorCodeText(ErrorCode::GRAPH_STRUCTURE_INVALID), parentNodeId, childNodeId,
149+ parentNode == nullptr ? "null" : parentNode->Describe(),
150+ childNode == nullptr ? "null" : childNode->Describe());
187 return HCCL_E_PARA;151 return HCCL_E_PARA;
188 }152 }
189 153 
@@ -403,8 +367,8 @@ HcclResult TaskGraphGeneratorV3::GenGraph(std::vector<std::unique_ptr<TaskNode>>
403 AllRankNodeQueues translatedTaskQueues)367 AllRankNodeQueues translatedTaskQueues)
404{368{
405 Reset();369 Reset();
406- HCCL_VM_INFO("[TaskGraphGeneratorV3][GenGraph] Start gen graph from translated task nodes, nodeCount={}, "370+ HCCL_VM_INFO("Start building the CheckerV3 graph from translated nodes, rankCount={}, nodeCount={}",
407- "rankCount={}", translatedNodes.size(), translatedTaskQueues.size());371+ translatedTaskQueues.size(), translatedNodes.size());
408 nodes_ = std::move(translatedNodes);372 nodes_ = std::move(translatedNodes);
409 taskQueues_ = std::move(translatedTaskQueues);373 taskQueues_ = std::move(translatedTaskQueues);
410 hasAiv_ = std::any_of(nodes_.begin(), nodes_.end(), [](const std::unique_ptr<TaskNode> &node) {374 hasAiv_ = std::any_of(nodes_.begin(), nodes_.end(), [](const std::unique_ptr<TaskNode> &node) {
@@ -435,8 +399,8 @@ HcclResult TaskGraphGeneratorV3::CreateMainStartNode()
435 399 
436HcclResult TaskGraphGeneratorV3::BuildDagEdges()400HcclResult TaskGraphGeneratorV3::BuildDagEdges()
437{401{
438- HCCL_VM_INFO("[TaskGraphGeneratorV3][BuildDagEdges] Start DAG edge build, nodeCount={}, rankCount={}, "402+ HCCL_VM_INFO("Start building graph edges, rankCount={}, nodeCount={}, mainStartNodeId={}",
439- "mainStartNodeId={}", nodes_.size(), taskQueues_.size(), mainStartNodeId_);403+ taskQueues_.size(), nodes_.size(), mainStartNodeId_);
440 404 
441 HcclResult ret = HCCL_SUCCESS;405 HcclResult ret = HCCL_SUCCESS;
442 for (const auto &rankEntry : taskQueues_) {406 for (const auto &rankEntry : taskQueues_) {
@@ -472,8 +436,8 @@ HcclResult TaskGraphGeneratorV3::BuildDagEdges()
472 }436 }
473 }437 }
474 438 
475- HCCL_VM_INFO("[TaskGraphGeneratorV3][BuildDagEdges] DAG edge build success, nodeCount={}, edgeCount={}, "439+ HCCL_VM_INFO("Finished building graph edges, nodeCount={}, edgeCount={}, mainStartNodeId={}",
476- "mainStartNodeId={}", nodes_.size(), CountEdges(), mainStartNodeId_);440+ nodes_.size(), CountEdges(), mainStartNodeId_);
477 return HCCL_SUCCESS;441 return HCCL_SUCCESS;
478}442}
479 443 
@@ -492,8 +456,10 @@ HcclResult TaskGraphGeneratorV3::ExpandAivSubGraphs()
492 456 
493 auto *aivGraph = dynamic_cast<TaskAivGraph *>(node);457 auto *aivGraph = dynamic_cast<TaskAivGraph *>(node);
494 if (aivGraph == nullptr) {458 if (aivGraph == nullptr) {
495- HCCL_VM_ERROR("[TaskGraphGeneratorV3][ExpandAivSubGraphs] AIV node cast failed, nodeId={}",459+ HCCL_VM_ERROR("{} One node expected to be an AIV subgraph entry is actually another node "
496- node->GetNodeId());460+ "type, nodeId={}, node={}", MakeErrorCodeText(ErrorCode::CHECKER_RUNTIME_ERROR),
461+ node == nullptr ? std::string("null") : std::to_string(node->GetNodeId()),
462+ node == nullptr ? "node=null" : node->Describe());
497 return HCCL_E_INTERNAL;463 return HCCL_E_INTERNAL;
498 }464 }
499 aivGraphs.push_back(aivGraph);465 aivGraphs.push_back(aivGraph);
@@ -506,8 +472,9 @@ HcclResult TaskGraphGeneratorV3::ExpandAivSubGraphs()
506 AivGraphGenerateOutputV3 result;472 AivGraphGenerateOutputV3 result;
507 HcclResult ret = ExpandAivGraphsV3(input, result);473 HcclResult ret = ExpandAivGraphsV3(input, result);
508 if (ret != HCCL_SUCCESS) {474 if (ret != HCCL_SUCCESS) {
509- HCCL_VM_ERROR("[TaskGraphGeneratorV3][ExpandAivSubGraphs] Expand AIV graphs failed, graphCount={}, ret={}",475+ HCCL_VM_ERROR("{} Failed to expand AIV Graph nodes, "
510- aivGraphs.size(), static_cast<uint32_t>(ret));476+ "aivGraphCount={}, ret={}", MakeErrorCodeText(ErrorCode::GRAPH_TRANSLATE_FAILED), aivGraphs.size(),
477+ static_cast<uint32_t>(ret));
511 return ret;478 return ret;
512 }479 }
513 480 
@@ -527,12 +494,12 @@ HcclResult TaskGraphGeneratorV3::ExpandAivSubGraphs()
527 stats.cpGmInactiveNodeCount = result.cpGmInactiveNodeCount;494 stats.cpGmInactiveNodeCount = result.cpGmInactiveNodeCount;
528 stats.totalExpandNs = result.expandNs;495 stats.totalExpandNs = result.expandNs;
529 aivExpandStats_ = stats;496 aivExpandStats_ = stats;
530- HCCL_VM_INFO("[TaskGraphGeneratorV3][ExpandAivSubGraphs] AIV subgraph expansion finished, aivGraphCount={}, "497+ HCCL_VM_INFO("Finished expanding AIV Graph nodes, aivGraphCount={}, internalNodeCount={}, "
531- "internalNodeCount={}, setWaitEdgeCount={}, pipeBarrierMergeCount={}, syncAllMergeCount={}, "498+ "setWaitEdgeCount={}, pipeBarrierMergeCount={}, syncAllMergeCount={}, sendRecvEdgeCount={}, "
532- "sendRecvEdgeCount={}, taskJsonTotalTaskCount={}, dagNodeCountBeforeCpGmMerge={}, "499+ "taskJsonTotalTaskCount={}, dagNodeCountBeforeCpGmMerge={}, dagNodeCountAfterCpGmMerge={}, "
533- "dagNodeCountAfterCpGmMerge={}, cpGmLoopMergeCount={}, cpGmMergedIterationCount={}, "500+ "cpGmLoopMergeCount={}, cpGmMergedIterationCount={}, cpGmMergedOriginalNodeCount={}, "
534- "cpGmMergedOriginalNodeCount={}, cpGmGeneratedNodeCount={}, cpGmInactiveNodeCount={}, "501+ "cpGmGeneratedNodeCount={}, cpGmInactiveNodeCount={}, ubBufferSize={}, flagBufferSize={}, "
535- "ubBufferSize={}, flagBufferSize={}, expandTotalMs={}", stats.graphCount, stats.internalNodeCount,502+ "expandTotalMs={}", stats.graphCount, stats.internalNodeCount,
536 stats.setWaitEdgeCount, stats.pipeBarrierMergeCount, stats.syncAllMergeCount, stats.sendRecvEdgeCount,503 stats.setWaitEdgeCount, stats.pipeBarrierMergeCount, stats.syncAllMergeCount, stats.sendRecvEdgeCount,
537 stats.taskJsonTotalTaskCount, stats.dagNodeCountBeforeCpGmMerge, stats.dagNodeCountAfterCpGmMerge,504 stats.taskJsonTotalTaskCount, stats.dagNodeCountBeforeCpGmMerge, stats.dagNodeCountAfterCpGmMerge,
538 stats.cpGmLoopMergeCount, stats.cpGmMergedIterationCount, stats.cpGmMergedOriginalNodeCount,505 stats.cpGmLoopMergeCount, stats.cpGmMergedIterationCount, stats.cpGmMergedOriginalNodeCount,
@@ -556,8 +523,10 @@ HcclResult TaskGraphGeneratorV3::ExpandCcuSubGraphs()
556 523 
557 auto *ccuGraph = dynamic_cast<TaskCcuGraph *>(node);524 auto *ccuGraph = dynamic_cast<TaskCcuGraph *>(node);
558 if (ccuGraph == nullptr) {525 if (ccuGraph == nullptr) {
559- HCCL_VM_ERROR("[TaskGraphGeneratorV3][ExpandCcuSubGraphs] CCU node cast failed, nodeId={}",526+ HCCL_VM_ERROR("{} One node expected to be a CCU subgraph entry is actually another node "
560- node->GetNodeId());527+ "type, nodeId={}, node={}", MakeErrorCodeText(ErrorCode::CHECKER_RUNTIME_ERROR),
528+ node == nullptr ? std::string("null") : std::to_string(node->GetNodeId()),
529+ node == nullptr ? "node=null" : node->Describe());
561 return HCCL_E_INTERNAL;530 return HCCL_E_INTERNAL;
562 }531 }
563 532 
@@ -571,8 +540,9 @@ HcclResult TaskGraphGeneratorV3::ExpandCcuSubGraphs()
571 CcuGraphGenerateOutputV3 result;540 CcuGraphGenerateOutputV3 result;
572 HcclResult ret = ExpandCcuGraphsV3(input, result);541 HcclResult ret = ExpandCcuGraphsV3(input, result);
573 if (ret != HCCL_SUCCESS) {542 if (ret != HCCL_SUCCESS) {
574- HCCL_VM_ERROR("[TaskGraphGeneratorV3][ExpandCcuSubGraphs] Expand CCU graphs failed, graphCount={}, ret={}",543+ HCCL_VM_ERROR("{} Failed to expand CCU Graph nodes, "
575- ccuGraphs.size(), static_cast<uint32_t>(ret));544+ "ccuGraphCount={}, ret={}", MakeErrorCodeText(ErrorCode::GRAPH_TRANSLATE_FAILED), ccuGraphs.size(),
545+ static_cast<uint32_t>(ret));
576 return ret;546 return ret;
577 }547 }
578 stats.graphCount = ccuGraphs.size();548 stats.graphCount = ccuGraphs.size();
@@ -581,8 +551,8 @@ HcclResult TaskGraphGeneratorV3::ExpandCcuSubGraphs()
581 stats.totalExpandNs = result.expandNs;551 stats.totalExpandNs = result.expandNs;
582 552 
583 ccuExpandStats_ = stats;553 ccuExpandStats_ = stats;
584- HCCL_VM_INFO("[TaskGraphGeneratorV3][ExpandCcuSubGraphs] CCU subgraph expansion finished, ccuGraphCount={}, "554+ HCCL_VM_INFO("Finished expanding CCU Graph nodes, ccuGraphCount={}, internalNodeCount={}, "
585- "internalNodeCount={}, recordWaitEdgeCount={}, expandTotalMs={}", stats.graphCount,555+ "recordWaitEdgeCount={}, expandTotalMs={}", stats.graphCount,
586 stats.internalNodeCount, stats.recordWaitEdgeCount, stats.totalExpandNs / 1000000ULL);556 stats.internalNodeCount, stats.recordWaitEdgeCount, stats.totalExpandNs / 1000000ULL);
587 return HCCL_SUCCESS;557 return HCCL_SUCCESS;
588}558}
@@ -648,8 +618,8 @@ HcclResult TaskGraphGeneratorV3::GenGraph4Rank(RankId rankId, const RankNodeQueu
648 ++streamOrderEdgeCount;618 ++streamOrderEdgeCount;
649 }619 }
650 }620 }
651- HCCL_VM_DEBUG("[TaskGraphGeneratorV3][RankSkeleton] Rank {}, streamCount={}, taskNodeCount={}, "621+ HCCL_VM_DEBUG("Built per-rank skeleton edges, rankId={}, nonEmptyStreamCount={}, taskNodeCount={}, "
652- "startEdges={}, streamOrderEdges={}", rankId, nonEmptyStreamCount, taskNodeCount, startEdgeCount,622+ "startEdgeCount={}, streamOrderEdgeCount={}", rankId, nonEmptyStreamCount, taskNodeCount, startEdgeCount,
653 streamOrderEdgeCount);623 streamOrderEdgeCount);
654 return HCCL_SUCCESS;624 return HCCL_SUCCESS;
655}625}
@@ -660,7 +630,6 @@ HcclResult TaskGraphGeneratorV3::AddLocalNotifyEdges(RankId rankId, const RankNo
660 SeenLocalRecords seenLocalRecords;630 SeenLocalRecords seenLocalRecords;
661 uint64_t unmatchedCnt = 0;631 uint64_t unmatchedCnt = 0;
662 size_t matchedEdgeCount = 0;632 size_t matchedEdgeCount = 0;
663- size_t requeueCount = 0;
664 633 
665 if (!rankTaskQueues.empty() && !rankTaskQueues[0].empty()) {634 if (!rankTaskQueues.empty() && !rankTaskQueues[0].empty()) {
666 rankNodeQue.push_back(rankTaskQueues[0].front());635 rankNodeQue.push_back(rankTaskQueues[0].front());
@@ -674,8 +643,11 @@ HcclResult TaskGraphGeneratorV3::AddLocalNotifyEdges(RankId rankId, const RankNo
674 while (!rankNodeQue.empty()) {643 while (!rankNodeQue.empty()) {
675 if (unmatchedCnt >= rankNodeQue.size()) {644 if (unmatchedCnt >= rankNodeQue.size()) {
676 const TaskNode *node = GetNode(rankNodeQue.front());645 const TaskNode *node = GetNode(rankNodeQue.front());
677- HCCL_VM_ERROR("[TaskGraphGeneratorV3][LocalNotify] Rank {} local notify deadlock, firstUnmatched={}",646+ HCCL_VM_ERROR("{} Local Record/Wait matching is stuck on this rank. Some Wait tasks are "
678- rankId, node == nullptr ? "null" : node->Describe());647+ "still blocked, but no new local Record task can unblock them, rankId={}, firstBlockedWaitNode={}, "
648+ "blockedWaitNodeCount={}",
649+ MakeErrorCodeText(ErrorCode::GRAPH_DEADLOCK), rankId,
650+ node == nullptr ? "null" : node->Describe(), rankNodeQue.size());
679 return HCCL_E_INTERNAL;651 return HCCL_E_INTERNAL;
680 }652 }
681 653 
@@ -723,7 +695,6 @@ HcclResult TaskGraphGeneratorV3::AddLocalNotifyEdges(RankId rankId, const RankNo
723 if (!matched) {695 if (!matched) {
724 rankNodeQue.push_back(currNodeId);696 rankNodeQue.push_back(currNodeId);
725 ++unmatchedCnt;697 ++unmatchedCnt;
726- ++requeueCount;
727 }698 }
728 continue;699 continue;
729 }700 }
@@ -737,13 +708,16 @@ HcclResult TaskGraphGeneratorV3::AddLocalNotifyEdges(RankId rankId, const RankNo
737 708 
738 if (!seenLocalRecords.empty()) {709 if (!seenLocalRecords.empty()) {
739 const TaskNode *node = GetNode(seenLocalRecords.front());710 const TaskNode *node = GetNode(seenLocalRecords.front());
740- HCCL_VM_ERROR("[TaskGraphGeneratorV3][LocalNotify] Rank {} has unmatched local record, firstUnmatched={}",711+ HCCL_VM_ERROR("{} Found local Record tasks that were never consumed by any local Wait task, "
741- rankId, node == nullptr ? "null" : node->Describe());712+ "rankId={}, firstUnconsumedRecordNode={}, unconsumedRecordCount={}",
713+ MakeErrorCodeText(ErrorCode::GRAPH_UNMATCHED), rankId,
714+ node == nullptr ? "node=null" : node->Describe(), seenLocalRecords.size());
742 return HCCL_E_INTERNAL;715 return HCCL_E_INTERNAL;
743 }716 }
744 717 
745- HCCL_VM_DEBUG("[TaskGraphGeneratorV3][LocalNotify] Rank {} local notify edges generated, matchedEdges={}, "718+ HCCL_VM_DEBUG("Finished matching local Record/Wait edges on one rank, rankId={}, "
746- "requeueCount={}", rankId, matchedEdgeCount, requeueCount);719+ "matchedRecordWaitEdgeCount={}",
720+ rankId, matchedEdgeCount);
747 return HCCL_SUCCESS;721 return HCCL_SUCCESS;
748}722}
749 723 
@@ -755,7 +729,6 @@ HcclResult TaskGraphGeneratorV3::AddInterRankNotifyEdges()
755 std::vector<uint8_t> traverseFlags(nodes_.size(), 0);729 std::vector<uint8_t> traverseFlags(nodes_.size(), 0);
756 uint64_t unmatchedCnt = 0;730 uint64_t unmatchedCnt = 0;
757 size_t matchedEdgeCount = 0;731 size_t matchedEdgeCount = 0;
758- size_t requeueCount = 0;
759 732 
760 HcclResult ret = ExecuteNode(mainStartNodeId_, graphNodeQue, execFlags, traverseFlags);733 HcclResult ret = ExecuteNode(mainStartNodeId_, graphNodeQue, execFlags, traverseFlags);
761 if (ret != HCCL_SUCCESS) {734 if (ret != HCCL_SUCCESS) {
@@ -765,8 +738,11 @@ HcclResult TaskGraphGeneratorV3::AddInterRankNotifyEdges()
765 while (!graphNodeQue.empty()) {738 while (!graphNodeQue.empty()) {
766 if (unmatchedCnt >= graphNodeQue.size()) {739 if (unmatchedCnt >= graphNodeQue.size()) {
767 const TaskNode *node = GetNode(graphNodeQue.front());740 const TaskNode *node = GetNode(graphNodeQue.front());
768- HCCL_VM_ERROR("[TaskGraphGeneratorV3][InterRankNotify] Inter-rank notify deadlock, firstUnmatched={}",741+ HCCL_VM_ERROR("{} Cross-rank Record/Wait matching is stuck. Some Wait tasks are still "
769- node == nullptr ? "null" : node->Describe());742+ "blocked, but no new cross-rank Record task can unblock them, firstBlockedWaitNode={}, "
743+ "blockedWaitNodeCount={}",
744+ MakeErrorCodeText(ErrorCode::GRAPH_DEADLOCK),
745+ node == nullptr ? "node=null" : node->Describe(), graphNodeQue.size());
770 return HCCL_E_INTERNAL;746 return HCCL_E_INTERNAL;
771 }747 }
772 748 
@@ -780,7 +756,6 @@ HcclResult TaskGraphGeneratorV3::AddInterRankNotifyEdges()
780 if (!IsExecutable(currNodeId, execFlags)) {756 if (!IsExecutable(currNodeId, execFlags)) {
781 graphNodeQue.push_back(currNodeId);757 graphNodeQue.push_back(currNodeId);
782 ++unmatchedCnt;758 ++unmatchedCnt;
783- ++requeueCount;
784 continue;759 continue;
785 }760 }
786 761 
@@ -829,7 +804,6 @@ HcclResult TaskGraphGeneratorV3::AddInterRankNotifyEdges()
829 804 
830 graphNodeQue.push_back(currNodeId);805 graphNodeQue.push_back(currNodeId);
831 ++unmatchedCnt;806 ++unmatchedCnt;
832- ++requeueCount;
833 continue;807 continue;
834 }808 }
835 809 
@@ -844,16 +818,20 @@ HcclResult TaskGraphGeneratorV3::AddInterRankNotifyEdges()
844 for (const auto &peerEntry : rankEntry.second) {818 for (const auto &peerEntry : rankEntry.second) {
845 if (!peerEntry.second.empty()) {819 if (!peerEntry.second.empty()) {
846 const TaskNode *node = GetNode(peerEntry.second.front());820 const TaskNode *node = GetNode(peerEntry.second.front());
847- HCCL_VM_ERROR("[TaskGraphGeneratorV3][InterRankNotify] Unmatched inter-rank record, recordRank={}, "821+ const AicpuNotify *recordNotify = GetRecordNotify(node);
848- "waitRank={}, firstUnmatched={}", rankEntry.first, peerEntry.first,822+ HCCL_VM_ERROR("{} Found cross-rank Record tasks that were never consumed by any matching "
849- node == nullptr ? "null" : node->Describe());823+ "Wait task, recordRankId={}, waitRankId={}, notifyId={}, firstUnconsumedRecordNode={}, "
824+ "unconsumedRecordCount={}", MakeErrorCodeText(ErrorCode::GRAPH_UNMATCHED), rankEntry.first,
825+ peerEntry.first, recordNotify == nullptr ? std::string("null") : std::to_string(recordNotify->notifyId),
826+ node == nullptr ? "null" : node->Describe(), peerEntry.second.size());
850 return HCCL_E_INTERNAL;827 return HCCL_E_INTERNAL;
851 }828 }
852 }829 }
853 }830 }
854 831 
855- HCCL_VM_DEBUG("[TaskGraphGeneratorV3][InterRankNotify] Inter-rank notify edges generated, matchedEdges={}, "832+ HCCL_VM_DEBUG("Finished matching cross-rank Record/Wait edges, "
856- "requeueCount={}", matchedEdgeCount, requeueCount);833+ "matchedCrossRankRecordWaitEdgeCount={}",
834+ matchedEdgeCount);
857 return HCCL_SUCCESS;835 return HCCL_SUCCESS;
858}836}
859} // namespace TaskGraphGeneratorV3837} // namespace TaskGraphGeneratorV3
Mtest/hccl_vm/src/plugin/checker/src/framework/task_graph_generator_v3/task_graph_mem_conflict_v3.cc+46-41
@@ -24,6 +24,7 @@
24 24 
25#include "sim_log.h"25#include "sim_log.h"
26#include "task_graph_reachability_v3.h"26#include "task_graph_reachability_v3.h"
27+#include "utils/error_codes.h"
27 28 
28namespace HcclSim {29namespace HcclSim {
29namespace TaskGraphGeneratorV3 {30namespace TaskGraphGeneratorV3 {
@@ -143,26 +144,6 @@ struct ConflictCandidate {
143 144 
144using ConflictCandidates = std::vector<ConflictCandidate>;145using ConflictCandidates = std::vector<ConflictCandidate>;
145 146 
146-const char *MemTypeName(MemType memType)
147-{
148- switch (memType) {
149- case MemType::INPUT:
150- return "INPUT";
151- case MemType::OUTPUT:
152- return "OUTPUT";
153- case MemType::CCL:
154- return "CCL";
155- case MemType::UB_AIV:
156- return "UB_AIV";
157- case MemType::FLAG_AIV:
158- return "FLAG_AIV";
159- case MemType::MS_CCU:
160- return "MS_CCU";
161- default:
162- return "INVALID";
163- }
164-}
165- 
166const char *AccessKindName(MemoryAccessKind kind)147const char *AccessKindName(MemoryAccessKind kind)
167{148{
168 return kind == MemoryAccessKind::WRITE ? "WRITE" : "READ";149 return kind == MemoryAccessKind::WRITE ? "WRITE" : "READ";
@@ -216,7 +197,7 @@ uint64_t ElapsedMemConflictMs(MemConflictClock::time_point start, MemConflictClo
216 197 
217void LogMemConflictStageTime(const char *stage, const char *status, MemConflictClock::time_point start)198void LogMemConflictStageTime(const char *stage, const char *status, MemConflictClock::time_point start)
218{199{
219- HCCL_VM_WARN("[TaskGraphMemConflictV3][StageTime] {} {}, costMs={}", stage, status,200+ HCCL_VM_INFO("Stage finished, stage={}, status={}, costMs={}", stage, status,
220 ElapsedMemConflictMs(start, MemConflictClock::now()));201 ElapsedMemConflictMs(start, MemConflictClock::now()));
221}202}
222 203 
@@ -230,18 +211,29 @@ bool IsDataMoveTaskNode(const TaskNode *node)
230std::string DescribeKey(const MemoryKey &key)211std::string DescribeKey(const MemoryKey &key)
231{212{
232 std::ostringstream os;213 std::ostringstream os;
233- os << "{rank=" << key.rankId << ", mem=" << MemTypeName(key.memType) << ", aivUbIdx=" << key.aivUbIdx << "}";214+ os << "{rankId=";
215+ if (key.rankId == INVALID_RANK_ID) {
216+ os << "invalid";
217+ } else {
218+ os << key.rankId;
219+ }
220+ os << ", memoryType=" << DescribeMemType(key.memType);
221+ if (key.memType == MemType::UB_AIV) {
222+ os << ", aivBlockIndex=" << key.aivUbIdx;
223+ }
224+ os << "}";
234 return os.str();225 return os.str();
235}226}
236 227 
237std::string DescribeAccess(const MemoryAccess &access)228std::string DescribeAccess(const MemoryAccess &access)
238{229{
239 std::ostringstream os;230 std::ostringstream os;
240- os << "{nodeId=" << access.nodeId << ", access=" << AccessKindName(access.kind)231+ os << "{taskNodeId=" << access.nodeId << ", taskAction="
241- << ", range=[0x" << std::hex << access.start << ",0x" << access.end << ")" << std::dec232+ << (access.kind == MemoryAccessKind::WRITE ? "write" : "read")
242- << ", key=" << DescribeKey(access.key);233+ << ", memory=" << DescribeKey(access.key)
234+ << ", byteRange=[0x" << std::hex << access.start << ",0x" << access.end << ")" << std::dec;
243 if (access.node != nullptr) {235 if (access.node != nullptr) {
244- os << ", desc=" << access.node->Describe();236+ os << ", task=" << access.node->Describe();
245 }237 }
246 os << "}";238 os << "}";
247 return os.str();239 return os.str();
@@ -284,7 +276,8 @@ HcclResult CollectReachableTaskNodes(const TaskNode *start, std::vector<const Ta
284 276 
285 for (const TaskNode *childNode : node->GetChildren()) {277 for (const TaskNode *childNode : node->GetChildren()) {
286 if (childNode == nullptr) {278 if (childNode == nullptr) {
287- HCCL_VM_ERROR("[TaskGraphMemConflictV3] Invalid null child, parent={}", node->Describe());279+ HCCL_VM_ERROR("{} Graph structure is broken because one child node is null, "
280+ "parent={}", MakeErrorCodeText(ErrorCode::MEMCONFLICT_DAG_INVALID), node->Describe());
288 return HCCL_E_PTR;281 return HCCL_E_PTR;
289 }282 }
290 if (visited.count(childNode) != 0) {283 if (visited.count(childNode) != 0) {
@@ -331,20 +324,26 @@ HcclResult AddAccess(const MemSlice &slice, MemoryAccessKind kind, const TaskNod
331 return HCCL_E_PTR;324 return HCCL_E_PTR;
332 }325 }
333 if (node->GetNodeId() < 0) {326 if (node->GetNodeId() < 0) {
334- HCCL_VM_ERROR("[TaskGraphMemConflictV3] Invalid data task node id, node={}", node->Describe());327+ HCCL_VM_ERROR("{} Data task node id is invalid, nodeId={}, task={}",
328+ MakeErrorCodeText(ErrorCode::CHECKER_RUNTIME_ERROR), node->GetNodeId(), node->Describe());
335 return HCCL_E_PARA;329 return HCCL_E_PARA;
336 }330 }
337 if (slice.len == 0) {331 if (slice.len == 0) {
338 return HCCL_SUCCESS;332 return HCCL_SUCCESS;
339 }333 }
340 if (slice.rankId == INVALID_RANK_ID || slice.memType == MemType::INVALID) {334 if (slice.rankId == INVALID_RANK_ID || slice.memType == MemType::INVALID) {
341- HCCL_VM_ERROR("[TaskGraphMemConflictV3] Invalid memory slice, node={}, rank={}, memType={}",335+ HCCL_VM_ERROR("{} One memory slice is missing a valid rank or memory type, "
342- node->Describe(), slice.rankId, static_cast<uint32_t>(slice.memType));336+ "task={}, rankId={}, memType={}, offset=0x{:x}, length=0x{:x}",
337+ MakeErrorCodeText(ErrorCode::SINGLETASK_SLICE_INVALID), node->Describe(),
338+ slice.rankId == INVALID_RANK_ID ? "invalid" : std::to_string(slice.rankId),
339+ DescribeMemType(slice.memType), slice.offset, slice.len);
343 return HCCL_E_PARA;340 return HCCL_E_PARA;
344 }341 }
345 if (slice.len > std::numeric_limits<uint64_t>::max() - slice.offset) {342 if (slice.len > std::numeric_limits<uint64_t>::max() - slice.offset) {
346- HCCL_VM_ERROR("[TaskGraphMemConflictV3] Memory slice overflow, node={}, offset=0x{:x}, len=0x{:x}",343+ HCCL_VM_ERROR("{} One memory slice is invalid because offset + length overflowed, "
347- node->Describe(), slice.offset, slice.len);344+ "task={}, offset=0x{:x}, length=0x{:x}, endLimit=0x{:x}",
345+ MakeErrorCodeText(ErrorCode::SINGLETASK_SLICE_INVALID), node->Describe(), slice.offset, slice.len,
346+ std::numeric_limits<uint64_t>::max());
348 return HCCL_E_PARA;347 return HCCL_E_PARA;
349 }348 }
350 349 
@@ -387,7 +386,8 @@ HcclResult BuildTaskMemoryAccesses(const TaskNode *node, TaskMemoryAccesses &acc
387 }386 }
388 const std::vector<MemSlice> &srcs = reduce->GetSrcs();387 const std::vector<MemSlice> &srcs = reduce->GetSrcs();
389 if (srcs.empty()) {388 if (srcs.empty()) {
390- HCCL_VM_ERROR("[TaskGraphMemConflictV3] Reduce source list is empty, node={}", node->Describe());389+ HCCL_VM_ERROR("{} This reduce task has no source data, task={}",
390+ MakeErrorCodeText(ErrorCode::SINGLETASK_SLICE_INVALID), node->Describe());
391 return HCCL_E_PARA;391 return HCCL_E_PARA;
392 }392 }
393 for (size_t index = 0; index < srcs.size(); ++index) {393 for (size_t index = 0; index < srcs.size(); ++index) {
@@ -429,13 +429,17 @@ HcclResult BuildTaskMemoryAccesses(const TaskNode *node, TaskMemoryAccesses &acc
429 const auto &srcGroups = reduce->GetMergedSrcs();429 const auto &srcGroups = reduce->GetMergedSrcs();
430 const auto &dsts = reduce->GetMergedDsts();430 const auto &dsts = reduce->GetMergedDsts();
431 if (srcGroups.empty() || srcGroups.size() != dsts.size()) {431 if (srcGroups.empty() || srcGroups.size() != dsts.size()) {
432- HCCL_VM_ERROR("[TaskGraphMemConflictV3] Batch reduce group size mismatch, node={}", node->Describe());432+ HCCL_VM_ERROR("{} Batch reduce has different counts of source groups and target "
433+ "memory slices, task={}, srcGroupCount={}, dstCount={}",
434+ MakeErrorCodeText(ErrorCode::SINGLETASK_SLICE_INVALID), node->Describe(), srcGroups.size(),
435+ dsts.size());
433 return HCCL_E_PARA;436 return HCCL_E_PARA;
434 }437 }
435 for (size_t groupIndex = 0; groupIndex < srcGroups.size(); ++groupIndex) {438 for (size_t groupIndex = 0; groupIndex < srcGroups.size(); ++groupIndex) {
436 if (srcGroups[groupIndex].empty()) {439 if (srcGroups[groupIndex].empty()) {
437- HCCL_VM_ERROR("[TaskGraphMemConflictV3] Batch reduce source list is empty, node={}",440+ HCCL_VM_ERROR("{} One reduce group has no source data at all, task={}, "
438- node->Describe());441+ "groupIndex={}", MakeErrorCodeText(ErrorCode::SINGLETASK_SLICE_INVALID), node->Describe(),
442+ groupIndex);
439 return HCCL_E_PARA;443 return HCCL_E_PARA;
440 }444 }
441 for (const auto &src : srcGroups[groupIndex]) {445 for (const auto &src : srcGroups[groupIndex]) {
@@ -632,9 +636,10 @@ HcclResult CheckCollectedConflictCandidates(const ConflictCandidates &candidates
632 ++stats.parallelCandidatePairCount;636 ++stats.parallelCandidatePairCount;
633 const uint64_t overlapStart = std::max(candidate.current->start, candidate.history->start);637 const uint64_t overlapStart = std::max(candidate.current->start, candidate.history->start);
634 const uint64_t overlapEnd = std::min(candidate.current->end, candidate.history->end);638 const uint64_t overlapEnd = std::min(candidate.current->end, candidate.history->end);
635- HCCL_VM_ERROR("[TaskGraphMemConflictV3] Memory conflict detected, memory={}, overlap=[0x{:x},0x{:x}), "639+ HCCL_VM_ERROR("{} Memory conflict detected, memory={}, overlap=[0x{:x},0x{:x}), "
636- "current={}, history={}", DescribeKey(candidate.current->key), overlapStart, overlapEnd,640+ "conflictTask1={}, conflictTask2={}", MakeErrorCodeText(ErrorCode::MEMCONFLICT_DETECTED),
637- DescribeAccess(*candidate.current), DescribeAccess(*candidate.history));641+ DescribeKey(candidate.current->key), overlapStart, overlapEnd, DescribeAccess(*candidate.current),
642+ DescribeAccess(*candidate.history));
638 return HCCL_E_MEMORY;643 return HCCL_E_MEMORY;
639 }644 }
640 return HCCL_SUCCESS;645 return HCCL_SUCCESS;
@@ -687,8 +692,8 @@ HcclResult CollectMemoryAccessBuckets(const std::vector<const TaskNode *> &nodes
687 accessCountByTaskType[TaskTypeName(node->GetType())]++;692 accessCountByTaskType[TaskTypeName(node->GetType())]++;
688 accessCountByTaskTypeAndKind[std::string(TaskTypeName(node->GetType())) + ":" +693 accessCountByTaskTypeAndKind[std::string(TaskTypeName(node->GetType())) + ":" +
689 AccessKindName(access.kind)]++;694 AccessKindName(access.kind)]++;
690- accessCountByMemType[MemTypeName(access.key.memType)]++;695+ accessCountByMemType[DescribeMemType(access.key.memType)]++;
691- accessCountByMemTypeAndKind[std::string(MemTypeName(access.key.memType)) + ":" +696+ accessCountByMemTypeAndKind[std::string(DescribeMemType(access.key.memType)) + ":" +
692 AccessKindName(access.kind)]++;697 AccessKindName(access.kind)]++;
693 accessCountByBucket[DescribeKey(access.key)]++;698 accessCountByBucket[DescribeKey(access.key)]++;
694 }699 }
Mtest/hccl_vm/src/plugin/checker/src/framework/task_graph_generator_v3/task_graph_reachability_v3.cc+59-27
@@ -10,13 +10,16 @@
10 10 
11#include "task_graph_reachability_v3.h"11#include "task_graph_reachability_v3.h"
12 12 
13+#include <algorithm>
13#include <cstdint>14#include <cstdint>
14#include <map>15#include <map>
15#include <queue>16#include <queue>
16#include <set>17#include <set>
18+#include <sstream>
17#include <utility>19#include <utility>
18 20 
19#include "sim_log.h"21#include "sim_log.h"
22+#include "utils/error_codes.h"
20 23 
21namespace HcclSim {24namespace HcclSim {
22namespace TaskGraphGeneratorV3 {25namespace TaskGraphGeneratorV3 {
@@ -88,8 +91,8 @@ HcclResult AddReachableTaskNode(const TaskNode *node, ReachableTaskNodes &reacha
88 91 
89 const NodeId nodeId = node->GetNodeId();92 const NodeId nodeId = node->GetNodeId();
90 if (nodeId < 0) {93 if (nodeId < 0) {
91- HCCL_VM_ERROR("[TaskGraphReachabilityV3] Invalid task node id, nodeId={}, node={}",94+ HCCL_VM_ERROR("{} Task node id is invalid, nodeId={}, node={}",
92- nodeId, node->Describe());95+ MakeErrorCodeText(ErrorCode::MEMCONFLICT_DAG_INVALID), nodeId, node->Describe());
93 return HCCL_E_PARA;96 return HCCL_E_PARA;
94 }97 }
95 98 
@@ -99,14 +102,16 @@ HcclResult AddReachableTaskNode(const TaskNode *node, ReachableTaskNodes &reacha
99 const size_t previousIndex = idResult.first->second;102 const size_t previousIndex = idResult.first->second;
100 const TaskNode *previousNode = (previousIndex < reachable.nodes.size()) ?103 const TaskNode *previousNode = (previousIndex < reachable.nodes.size()) ?
101 reachable.nodes[previousIndex] : nullptr;104 reachable.nodes[previousIndex] : nullptr;
102- HCCL_VM_ERROR("[TaskGraphReachabilityV3] Duplicate task node id, nodeId={}, lhs={}, rhs={}",105+ HCCL_VM_ERROR("{} Two different nodes share the same task node id, nodeId={}, "
103- nodeId, (previousNode == nullptr) ? "null" : previousNode->Describe(), node->Describe());106+ "firstNode={}, secondNode={}", MakeErrorCodeText(ErrorCode::MEMCONFLICT_DAG_INVALID), nodeId,
107+ (previousNode == nullptr) ? "null" : previousNode->Describe(), node->Describe());
104 return HCCL_E_PARA;108 return HCCL_E_PARA;
105 }109 }
106 110 
107 const auto nodeResult = reachable.indexByNode.insert(std::make_pair(node, nodeIndex));111 const auto nodeResult = reachable.indexByNode.insert(std::make_pair(node, nodeIndex));
108 if (!nodeResult.second) {112 if (!nodeResult.second) {
109- HCCL_VM_ERROR("[TaskGraphReachabilityV3] Duplicate task node pointer, node={}", node->Describe());113+ HCCL_VM_ERROR("{} The same task node object was inserted more than once, node={}",
114+ MakeErrorCodeText(ErrorCode::MEMCONFLICT_DAG_INVALID), node->Describe());
110 return HCCL_E_INTERNAL;115 return HCCL_E_INTERNAL;
111 }116 }
112 117 
@@ -138,7 +143,9 @@ HcclResult CollectReachableTaskNodes(const TaskNode *start, ReachableTaskNodes &
138{143{
139 reachable = ReachableTaskNodes();144 reachable = ReachableTaskNodes();
140 if (!IsMainGraphStartNode(start)) {145 if (!IsMainGraphStartNode(start)) {
141- HCCL_VM_ERROR("[TaskGraphReachabilityV3] Invalid main start node.");146+ HCCL_VM_ERROR("{} Reachability analysis cannot start because the main start node is "
147+ "invalid, mainStartNode={}", MakeErrorCodeText(ErrorCode::MEMCONFLICT_DAG_INVALID),
148+ start == nullptr ? "null" : start->Describe());
142 return HCCL_E_PARA;149 return HCCL_E_PARA;
143 }150 }
144 151 
@@ -158,7 +165,8 @@ HcclResult CollectReachableTaskNodes(const TaskNode *start, ReachableTaskNodes &
158 165 
159 for (const TaskNode *childNode : node->GetChildren()) {166 for (const TaskNode *childNode : node->GetChildren()) {
160 if (childNode == nullptr) {167 if (childNode == nullptr) {
161- HCCL_VM_ERROR("[TaskGraphReachabilityV3] Invalid null child, parent={}", node->Describe());168+ HCCL_VM_ERROR("{} Graph structure is broken because one child node is null, "
169+ "parent={}", MakeErrorCodeText(ErrorCode::MEMCONFLICT_DAG_INVALID), node->Describe());
162 return HCCL_E_PTR;170 return HCCL_E_PTR;
163 }171 }
164 if (visited.count(childNode) != 0) {172 if (visited.count(childNode) != 0) {
@@ -178,14 +186,16 @@ HcclResult GetValidChildIndex(const TaskNode *parent, const TaskNode *child,
178 return HCCL_E_PTR;186 return HCCL_E_PTR;
179 }187 }
180 if (child == nullptr) {188 if (child == nullptr) {
181- HCCL_VM_ERROR("[TaskGraphReachabilityV3] Invalid null child, parent={}", parent->Describe());189+ HCCL_VM_ERROR("{} Graph structure is broken because one child node is null, parent={}",
190+ MakeErrorCodeText(ErrorCode::MEMCONFLICT_DAG_INVALID), parent->Describe());
182 return HCCL_E_PTR;191 return HCCL_E_PTR;
183 }192 }
184 193 
185 const auto iter = reachable.indexByNode.find(child);194 const auto iter = reachable.indexByNode.find(child);
186 if (iter == reachable.indexByNode.end()) {195 if (iter == reachable.indexByNode.end()) {
187- HCCL_VM_ERROR("[TaskGraphReachabilityV3] Invalid edge, parent={}, child={}",196+ HCCL_VM_ERROR("{} One graph edge is invalid, so reachability analysis cannot continue, "
188- parent->Describe(), child->Describe());197+ "parent={}, child={}", MakeErrorCodeText(ErrorCode::GRAPH_STRUCTURE_INVALID), parent->Describe(),
198+ child->Describe());
189 return HCCL_E_PARA;199 return HCCL_E_PARA;
190 }200 }
191 201 
@@ -268,10 +278,24 @@ HcclResult BuildReachabilityContext(const TaskNode *start, ReachabilityBuildCont
268 }278 }
269 279 
270 if (context.taskTopoNodes.size() != nodeCount) {280 if (context.taskTopoNodes.size() != nodeCount) {
271- HCCL_VM_ERROR("[TaskGraphReachabilityV3] V3 graph is not fully reachable from mainStart or is not a DAG, "281+ const TaskNode *firstUnreachedNode = nullptr;
272- "topoSize={}, expectedTopoSize={}, taskTopoSize={}, taskNodeCount={}, mainStartNodeId={}",282+ for (const TaskNode *node : context.reachable.nodes) {
283+ if (node == nullptr) {
284+ continue;
285+ }
286+ const bool found = std::find(context.taskTopoNodes.begin(), context.taskTopoNodes.end(), node) !=
287+ context.taskTopoNodes.end();
288+ if (!found) {
289+ firstUnreachedNode = node;
290+ break;
291+ }
292+ }
293+ HCCL_VM_ERROR("{} This V3 graph is not a complete DAG from the main start node, "
294+ "topoSize={}, expectedTopoSize={}, reachableTaskCount={}, taskNodeCount={}, mainStartNodeId={}",
295+ MakeErrorCodeText(ErrorCode::MEMCONFLICT_DAG_INVALID),
273 context.fullTopoOrder.size(), nodeCount + 1U, context.taskTopoNodes.size(), nodeCount,296 context.fullTopoOrder.size(), nodeCount + 1U, context.taskTopoNodes.size(), nodeCount,
274- MAIN_START_NODE_ID);297+ MAIN_START_NODE_ID,
298+ firstUnreachedNode == nullptr ? "null" : firstUnreachedNode->Describe());
275 return HCCL_E_INTERNAL;299 return HCCL_E_INTERNAL;
276 }300 }
277 301 
@@ -295,7 +319,8 @@ HcclResult BuildDenseReachabilityClosure(const ReachabilityBuildContext &context
295 const TaskNode *node = *iter;319 const TaskNode *node = *iter;
296 const auto nodeIter = context.reachable.indexByNode.find(node);320 const auto nodeIter = context.reachable.indexByNode.find(node);
297 if (nodeIter == context.reachable.indexByNode.end()) {321 if (nodeIter == context.reachable.indexByNode.end()) {
298- HCCL_VM_ERROR("[TaskGraphReachabilityV3] Missing topo node index, node={}", node->Describe());322+ HCCL_VM_ERROR("{} This node is missing its topological index, node={}",
323+ MakeErrorCodeText(ErrorCode::MEMCONFLICT_DAG_INVALID), node->Describe());
299 return HCCL_E_INTERNAL;324 return HCCL_E_INTERNAL;
300 }325 }
301 const size_t nodeIndex = nodeIter->second;326 const size_t nodeIndex = nodeIter->second;
@@ -323,7 +348,8 @@ HcclResult BuildDenseReachabilityClosure(const ReachabilityBuildContext &context
323 const size_t dataIndex = dataItem.second;348 const size_t dataIndex = dataItem.second;
324 const auto nodeIter = context.reachable.indexByNode.find(node);349 const auto nodeIter = context.reachable.indexByNode.find(node);
325 if (nodeIter == context.reachable.indexByNode.end() || dataIndex >= closure.matrix.size()) {350 if (nodeIter == context.reachable.indexByNode.end() || dataIndex >= closure.matrix.size()) {
326- HCCL_VM_ERROR("[TaskGraphReachabilityV3] Missing data move node index, node={}", node->Describe());351+ HCCL_VM_ERROR("{} This data-move node is missing its reachability index, node={}",
352+ MakeErrorCodeText(ErrorCode::MEMCONFLICT_DAG_INVALID), node->Describe());
327 return HCCL_E_INTERNAL;353 return HCCL_E_INTERNAL;
328 }354 }
329 closure.matrix[dataIndex] = reachableDataByNode[nodeIter->second];355 closure.matrix[dataIndex] = reachableDataByNode[nodeIter->second];
@@ -361,23 +387,29 @@ HcclResult IsReachable(const ReachabilityClosure &closure, NodeId fromNodeId, No
361 isReachable = false;387 isReachable = false;
362 const auto fromIter = closure.dataIndexByNodeId.find(fromNodeId);388 const auto fromIter = closure.dataIndexByNodeId.find(fromNodeId);
363 if (fromIter == closure.dataIndexByNodeId.end()) {389 if (fromIter == closure.dataIndexByNodeId.end()) {
364- HCCL_VM_ERROR("[TaskGraphReachabilityV3] Reachability query only supports data move task nodes, "390+ HCCL_VM_ERROR("{} Task dependency query only supports memory-copy or reduce tasks, "
365- "fromNodeId={}, toNodeId={}", fromNodeId, toNodeId);391+ "but the source task is not in that set, sourceTaskId={}, targetTaskId={}, "
392+ "sourceReachabilityIndex=null",
393+ MakeErrorCodeText(ErrorCode::CHECKER_RUNTIME_ERROR), fromNodeId, toNodeId);
366 return HCCL_E_PARA;394 return HCCL_E_PARA;
367 }395 }
368 const auto toIter = closure.dataIndexByNodeId.find(toNodeId);396 const auto toIter = closure.dataIndexByNodeId.find(toNodeId);
369 if (toIter == closure.dataIndexByNodeId.end()) {397 if (toIter == closure.dataIndexByNodeId.end()) {
370- HCCL_VM_ERROR("[TaskGraphReachabilityV3] Reachability query only supports data move task nodes, "398+ HCCL_VM_ERROR("{} Task dependency query only supports memory-copy or reduce tasks, "
371- "fromNodeId={}, toNodeId={}", fromNodeId, toNodeId);399+ "but the target task is not in that set, sourceTaskId={}, targetTaskId={}, "
400+ "sourceReachabilityIndex={}, targetReachabilityIndex=null",
401+ MakeErrorCodeText(ErrorCode::CHECKER_RUNTIME_ERROR), fromNodeId, toNodeId, fromIter->second);
372 return HCCL_E_PARA;402 return HCCL_E_PARA;
373 }403 }
374 404 
375 const size_t fromIndex = fromIter->second;405 const size_t fromIndex = fromIter->second;
376 const size_t toIndex = toIter->second;406 const size_t toIndex = toIter->second;
377 if (fromIndex >= closure.matrix.size() || toIndex / BITS_PER_WORD >= closure.matrix[fromIndex].size()) {407 if (fromIndex >= closure.matrix.size() || toIndex / BITS_PER_WORD >= closure.matrix[fromIndex].size()) {
378- HCCL_VM_ERROR("[TaskGraphReachabilityV3] Invalid data move reachability index, fromNodeId={}, "408+ HCCL_VM_ERROR("{} Reachability index is invalid for this data-move task pair, "
379- "toNodeId={}, fromIndex={}, toIndex={}, matrixSize={}",409+ "sourceTaskId={}, targetTaskId={}, sourceIndex={}, targetIndex={}, matrixRowCount={}, "
380- fromNodeId, toNodeId, fromIndex, toIndex, closure.matrix.size());410+ "matrixWordCountInSourceRow={}",
411+ MakeErrorCodeText(ErrorCode::MEMCONFLICT_DAG_INVALID), fromNodeId, toNodeId, fromIndex, toIndex,
412+ closure.matrix.size(), fromIndex < closure.matrix.size() ? closure.matrix[fromIndex].size() : 0U);
381 return HCCL_E_INTERNAL;413 return HCCL_E_INTERNAL;
382 }414 }
383 isReachable = TestBit(closure.matrix[fromIndex], toIndex);415 isReachable = TestBit(closure.matrix[fromIndex], toIndex);
@@ -390,13 +422,13 @@ HcclResult IsReachable(const ReachabilityClosure &closure, NodeId fromNodeId, No
390// isReachable = false;422// isReachable = false;
391// const auto fromIter = closure.dataIndexByNodeId.find(fromNodeId);423// const auto fromIter = closure.dataIndexByNodeId.find(fromNodeId);
392// if (fromIter == closure.dataIndexByNodeId.end()) {424// if (fromIter == closure.dataIndexByNodeId.end()) {
393-// HCCL_VM_ERROR("[TaskGraphReachabilityV3] Roaring reachability query only supports data move task nodes, "425+// HCCL_VM_ERROR("Roaring reachability query only supports data move task nodes, "
394// "fromNodeId={}, toNodeId={}", fromNodeId, toNodeId);426// "fromNodeId={}, toNodeId={}", fromNodeId, toNodeId);
395// return HCCL_E_PARA;427// return HCCL_E_PARA;
396// }428// }
397// const auto toIter = closure.dataIndexByNodeId.find(toNodeId);429// const auto toIter = closure.dataIndexByNodeId.find(toNodeId);
398// if (toIter == closure.dataIndexByNodeId.end()) {430// if (toIter == closure.dataIndexByNodeId.end()) {
399-// HCCL_VM_ERROR("[TaskGraphReachabilityV3] Roaring reachability query only supports data move task nodes, "431+// HCCL_VM_ERROR("Roaring reachability query only supports data move task nodes, "
400// "fromNodeId={}, toNodeId={}", fromNodeId, toNodeId);432// "fromNodeId={}, toNodeId={}", fromNodeId, toNodeId);
401// return HCCL_E_PARA;433// return HCCL_E_PARA;
402// }434// }
@@ -404,7 +436,7 @@ HcclResult IsReachable(const ReachabilityClosure &closure, NodeId fromNodeId, No
404// const size_t fromIndex = fromIter->second;436// const size_t fromIndex = fromIter->second;
405// const size_t toIndex = toIter->second;437// const size_t toIndex = toIter->second;
406// if (fromIndex >= closure.matrix.size()) {438// if (fromIndex >= closure.matrix.size()) {
407-// HCCL_VM_ERROR("[TaskGraphReachabilityV3] Invalid roaring data move reachability index, fromNodeId={}, "439+// HCCL_VM_ERROR("Invalid roaring data move reachability index, fromNodeId={}, "
408// "toNodeId={}, fromIndex={}, toIndex={}, matrixSize={}",440// "toNodeId={}, fromIndex={}, toIndex={}, matrixSize={}",
409// fromNodeId, toNodeId, fromIndex, toIndex, closure.matrix.size());441// fromNodeId, toNodeId, fromIndex, toIndex, closure.matrix.size());
410// return HCCL_E_INTERNAL;442// return HCCL_E_INTERNAL;
@@ -472,8 +504,8 @@ HcclResult CheckDataMoveTaskReachability(const TaskNode *start, DataTaskReachabi
472 }504 }
473 }505 }
474 506 
475- HCCL_VM_INFO("[TaskGraphReachabilityV3] V3 data move reachability generated, nodeCount={}, edgeCount={}, "507+ HCCL_VM_INFO("Task dependency summary generated, nodeCount={}, edgeCount={}, "
476- "dataTaskNodeCount={}, dataTaskPairs={}, reachablePairs={}, unreachablePairs={}",508+ "memoryTaskCount={}, taskPairsChecked={}, orderedTaskPairs={}, parallelTaskPairs={}",
477 localStats.nodeCount, localStats.edgeCount, localStats.dataTaskNodeCount,509 localStats.nodeCount, localStats.edgeCount, localStats.dataTaskNodeCount,
478 localStats.dataTaskPairCount, localStats.reachablePairCount, localStats.unreachablePairCount);510 localStats.dataTaskPairCount, localStats.reachablePairCount, localStats.unreachablePairCount);
479 511 
Mtest/hccl_vm/src/plugin/checker/src/framework/task_graph_generator_v3/task_graph_semantic_check_v3.cc+96-49
@@ -34,6 +34,8 @@
34#include "framework/semantics_check/send_recv_semantics_checker.h"34#include "framework/semantics_check/send_recv_semantics_checker.h"
35#include "sim_log.h"35#include "sim_log.h"
36#include "utils/check_utils.h"36#include "utils/check_utils.h"
37+#include "utils/dump/dump_json_utils.h"
38+#include "utils/error_codes.h"
37#include "utils/storage_manager.h"39#include "utils/storage_manager.h"
38 40 
39namespace HcclSim {41namespace HcclSim {
@@ -46,6 +48,20 @@ enum class SliceOpV3 : uint8_t {
46 REDUCE,48 REDUCE,
47};49};
48 50 
51+std::string DescribeMemSliceForSemantic(const MemSlice &slice)
52+{
53+ std::ostringstream os;
54+ os << "{rankId=";
55+ if (slice.rankId == INVALID_RANK_ID) {
56+ os << "invalid";
57+ } else {
58+ os << slice.rankId;
59+ }
60+ os << ", memoryType=" << DescribeMemType(slice.memType)
61+ << ", offset=0x" << std::hex << slice.offset << ", length=0x" << slice.len << std::dec << "}";
62+ return os.str();
63+}
64+ 
49struct SliceOpPairV3 {65struct SliceOpPairV3 {
50 RankId srcRank{INVALID_RANK_ID};66 RankId srcRank{INVALID_RANK_ID};
51 RankId dstRank{INVALID_RANK_ID};67 RankId dstRank{INVALID_RANK_ID};
@@ -60,15 +76,20 @@ struct SliceOpPairV3 {
60 std::string Describe() const76 std::string Describe() const
61 {77 {
62 std::ostringstream os;78 std::ostringstream os;
63- os << "{srcRank=" << srcRank << ", dstRank=" << dstRank 79+ os << "{operation=" << (op == SliceOpV3::REDUCE ? "reduce" : "overwrite")
64- << ", srcMemType=" << (int)(src.memType) << ", dstMemType=" << (int)(dst.memType)80+ << ", sourceMemorySlice=" << DescribeMemSliceForSemantic(src)
65- << ", srcOffset=0x" << std::hex << src.offset81+ << ", targetMemorySlice=" << DescribeMemSliceForSemantic(dst)
66- << ", dstOffset=0x" << dst.offset << std::dec << ", len=" << src.len82+ << ", launchIdx=" << position.launchIdx
67- << ", launch=" << position.launchIdx << ", block=" << position.blockId83+ << ", blockId=" << position.blockId
68- << ", pipe=" << position.pipe << ", taskId=" << position.taskId84+ << ", pipeId=" << position.pipe
69- << ", index=" << index << ", op=" << static_cast<uint32_t>(op)85+ << ", taskId=" << position.taskId;
70- << ", useBatchIndex=" << useBatchIndex86+ if (useBatchIndex) {
71- << ", reduceType=" << static_cast<uint32_t>(reduceType) << "}";87+ os << ", batchGroup=" << index;
88+ }
89+ if (op == SliceOpV3::REDUCE) {
90+ os << ", reduceType=" << DumpReduceOpToString(reduceType);
91+ }
92+ os << "}";
72 return os.str();93 return os.str();
73 }94 }
74};95};
@@ -269,7 +290,11 @@ HcclResult InitState(SemanticState &state)
269{290{
270 state.param = StorageManager::GetInstance().GetCheckerParam();291 state.param = StorageManager::GetInstance().GetCheckerParam();
271 if (state.param.rankSize == 0) {292 if (state.param.rankSize == 0) {
272- HCCL_VM_ERROR("[TaskGraphSemanticCheckV3] rankSize is zero");293+ HCCL_VM_ERROR("{} Semantic check initialization failed because the rank count is 0, "
294+ "collectiveType={}, dataType={}, elementCount={}, reduceType={}",
295+ MakeErrorCodeText(ErrorCode::CHECKER_RUNTIME_ERROR), HcclCmdTypeToString(state.param.cmdType),
296+ HcclDataTypeToString(state.param.dataType), state.param.dataCount,
297+ DumpReduceOpToString(state.param.reduceType));
273 return HCCL_E_PARA;298 return HCCL_E_PARA;
274 }299 }
275 CalcDataSize(state.param.cmdType, state.param.dataCount, state.param.dataType, state.dataSize);300 CalcDataSize(state.param.cmdType, state.param.dataCount, state.param.dataType, state.dataSize);
@@ -500,8 +525,9 @@ HcclResult CheckBufSemantics(const std::vector<const BufferSemantic *> &bufSeman
500 }525 }
501 if (bufSemantics.empty()) {526 if (bufSemantics.empty()) {
502 if (!ignoreError) {527 if (!ignoreError) {
503- HCCL_VM_ERROR("[TaskGraphSemanticCheckV3] Empty buffer semantics, start=0x{:x}, size=0x{:x}",528+ HCCL_VM_ERROR("{} No source/output information was found for the target memory range, "
504- startAddr, size);529+ "startAddr=0x{:x}, size=0x{:x}", MakeErrorCodeText(ErrorCode::SEMANTIC_BUFFER_EMPTY), startAddr,
530+ size);
505 }531 }
506 return HCCL_E_PARA;532 return HCCL_E_PARA;
507 }533 }
@@ -510,7 +536,8 @@ HcclResult CheckBufSemantics(const std::vector<const BufferSemantic *> &bufSeman
510 const BufferSemantic *prev = bufSemantics.front();536 const BufferSemantic *prev = bufSemantics.front();
511 if (prev->startAddr > startAddr) {537 if (prev->startAddr > startAddr) {
512 if (!ignoreError) {538 if (!ignoreError) {
513- HCCL_VM_ERROR("[TaskGraphSemanticCheckV3] Head gap, expect start=0x{:x}, actual start=0x{:x}",539+ HCCL_VM_ERROR("{} Output data does not start from the expected address; the beginning is "
540+ "missing, expectedStart=0x{:x}, actualStart=0x{:x}", MakeErrorCodeText(ErrorCode::SEMANTIC_GAP),
514 startAddr, prev->startAddr);541 startAddr, prev->startAddr);
515 }542 }
516 return HCCL_E_PARA;543 return HCCL_E_PARA;
@@ -525,7 +552,8 @@ HcclResult CheckBufSemantics(const std::vector<const BufferSemantic *> &bufSeman
525 const BufferSemantic *cur = bufSemantics[index];552 const BufferSemantic *cur = bufSemantics[index];
526 if (cur->startAddr != prev->startAddr + prev->size) {553 if (cur->startAddr != prev->startAddr + prev->size) {
527 if (!ignoreError) {554 if (!ignoreError) {
528- HCCL_VM_ERROR("[TaskGraphSemanticCheckV3] Middle gap, prev end=0x{:x}, cur start=0x{:x}",555+ HCCL_VM_ERROR("{} Output data is broken in the middle; one piece ends at 0x{:x} "
556+ "but the next starts at 0x{:x}", MakeErrorCodeText(ErrorCode::SEMANTIC_GAP),
529 prev->startAddr + prev->size, cur->startAddr);557 prev->startAddr + prev->size, cur->startAddr);
530 }558 }
531 return HCCL_E_PARA;559 return HCCL_E_PARA;
@@ -540,8 +568,9 @@ HcclResult CheckBufSemantics(const std::vector<const BufferSemantic *> &bufSeman
540 568 
541 if (totalSize != size) {569 if (totalSize != size) {
542 if (!ignoreError) {570 if (!ignoreError) {
543- HCCL_VM_ERROR("[TaskGraphSemanticCheckV3] Tail gap, expect end=0x{:x}, actual end=0x{:x}",571+ HCCL_VM_ERROR("{} Output data ends too early; the tail is missing, expectedEnd=0x{:x}, "
544- startAddr + size, startAddr + totalSize);572+ "actualEnd=0x{:x}", MakeErrorCodeText(ErrorCode::SEMANTIC_GAP), startAddr + size,
573+ startAddr + totalSize);
545 }574 }
546 return HCCL_E_PARA;575 return HCCL_E_PARA;
547 }576 }
@@ -594,14 +623,18 @@ HcclResult AddReduceSourcesToDstSegments(const SliceOpPairV3 &pair, const Buffer
594 }623 }
595 if (dstSemantic->srcBufs.size() == 1) {624 if (dstSemantic->srcBufs.size() == 1) {
596 if (dstSemantic->isReduce) {625 if (dstSemantic->isReduce) {
597- HCCL_VM_ERROR("[TaskGraphSemanticCheckV3] Invalid dst reduce semantic, pair={}", pair.Describe());626+ HCCL_VM_ERROR("{} Target output range is already marked as a reduce result before "
627+ "enough source data is merged, dataMapping={}",
628+ MakeErrorCodeText(ErrorCode::SEMANTIC_REDUCE_ERROR), pair.Describe());
598 return HCCL_E_PARA;629 return HCCL_E_PARA;
599 }630 }
600 dstSemantic->isReduce = true;631 dstSemantic->isReduce = true;
601 dstSemantic->reduceType = pair.reduceType;632 dstSemantic->reduceType = pair.reduceType;
602 }633 }
603 if (srcSemantic.srcBufs.size() > 1 && dstSemantic->reduceType != srcSemantic.reduceType) {634 if (srcSemantic.srcBufs.size() > 1 && dstSemantic->reduceType != srcSemantic.reduceType) {
604- HCCL_VM_ERROR("[TaskGraphSemanticCheckV3] Reduce type mismatch, pair={}", pair.Describe());635+ HCCL_VM_ERROR("{} Reduce result type is inconsistent while merging one source data "
636+ "range, dataMapping={}", MakeErrorCodeText(ErrorCode::SEMANTIC_REDUCE_ERROR),
637+ pair.Describe());
605 return HCCL_E_PARA;638 return HCCL_E_PARA;
606 }639 }
607 640 
@@ -610,12 +643,10 @@ HcclResult AddReduceSourcesToDstSegments(const SliceOpPairV3 &pair, const Buffer
610 const size_t before = dstSemantic->srcBufs.size();643 const size_t before = dstSemantic->srcBufs.size();
611 dstSemantic->srcBufs.insert(srcBuf);644 dstSemantic->srcBufs.insert(srcBuf);
612 if (before == dstSemantic->srcBufs.size()) {645 if (before == dstSemantic->srcBufs.size()) {
613- 646+ HCCL_VM_ERROR("{} The same source data is added twice to one output range during "
614- 647+ "reduce, dataMapping={}, sourceOffsetInThisRange=0x{:x}, outputRange=[0x{:x},0x{:x})",
615- 648+ MakeErrorCodeText(ErrorCode::SEMANTIC_REDUCE_ERROR), pair.Describe(), srcOffset,
616- HCCL_VM_ERROR("[TaskGraphSemanticCheckV3] Duplicate reduce source, pair={}, srcOffset=0x{:x}, "649+ dstSemantic->startAddr, GetSegmentEndAddr(*dstSemantic));
617- "dstSegmentStart=0x{:x}, dstSegmentEnd=0x{:x}",
618- pair.Describe(), srcOffset, dstSemantic->startAddr, GetSegmentEndAddr(*dstSemantic));
619 return HCCL_E_PARA;650 return HCCL_E_PARA;
620 }651 }
621 }652 }
@@ -648,17 +679,19 @@ HcclResult ApplyReduceSemantic(const SliceOpPairV3 &pair, SemanticState &state,
648 }679 }
649 HcclResult ret = CheckBufSemantics(affectedViews, dstStartAddr, dstEndAddr - dstStartAddr, false);680 HcclResult ret = CheckBufSemantics(affectedViews, dstStartAddr, dstEndAddr - dstStartAddr, false);
650 if (ret != HCCL_SUCCESS) {681 if (ret != HCCL_SUCCESS) {
651- HCCL_VM_ERROR("[TaskGraphSemanticCheckV3] Destination reduce semantic incomplete, pair={}, dstRange=[0x{:x},0x{:x}), "682+ HCCL_VM_ERROR("{} Target output range is only partially filled before reduce continues, "
652- "dstSegmentCnt={}",683+ "dataMapping={}, outputRange=[0x{:x},0x{:x}), pieceCount={}",
653- pair.Describe(), dstStartAddr, dstEndAddr, affectedDstSemantics.size());684+ MakeErrorCodeText(ErrorCode::SEMANTIC_REDUCE_ERROR), pair.Describe(), dstStartAddr, dstEndAddr,
685+ affectedDstSemantics.size());
654 return ret;686 return ret;
655 }687 }
656 688 
657 ret = AddReduceSourcesToDstSegments(pair, srcSemantic, affectedDstSemantics, srcStartAddr);689 ret = AddReduceSourcesToDstSegments(pair, srcSemantic, affectedDstSemantics, srcStartAddr);
658 if (ret != HCCL_SUCCESS) {690 if (ret != HCCL_SUCCESS) {
659- HCCL_VM_ERROR("[TaskGraphSemanticCheckV3] Add reduce sources to dst segments failed, pair={}, "691+ HCCL_VM_ERROR("{} Failed to merge one source data range into the target output range during "
660- "srcRange=[0x{:x},0x{:x}), dstRange=[0x{:x},0x{:x}), ret={}",692+ "reduce, dataMapping={}, srcRange=[0x{:x},0x{:x}), dstRange=[0x{:x},0x{:x}), ret={}",
661- pair.Describe(), srcStartAddr, srcEndAddr, dstStartAddr, dstEndAddr, static_cast<int32_t>(ret));693+ MakeErrorCodeText(ErrorCode::SEMANTIC_REDUCE_ERROR), pair.Describe(), srcStartAddr, srcEndAddr,
694+ dstStartAddr, dstEndAddr, static_cast<int32_t>(ret));
662 return ret;695 return ret;
663 }696 }
664 697 
@@ -693,11 +726,13 @@ HcclResult LoadSliceOpPairSrcSemantics(const SliceOpPairV3 &pair, const Semantic
693 return HCCL_SUCCESS;726 return HCCL_SUCCESS;
694 }727 }
695 if (!IsValidMemorySlice(pair.src) || !IsValidMemorySlice(pair.dst)) {728 if (!IsValidMemorySlice(pair.src) || !IsValidMemorySlice(pair.dst)) {
696- HCCL_VM_ERROR("[TaskGraphSemanticCheckV3] Invalid memory slice, pair={}", pair.Describe());729+ HCCL_VM_ERROR("{} One source-to-target mapping uses an invalid memory slice, "
730+ "dataMapping={}", MakeErrorCodeText(ErrorCode::SINGLETASK_SLICE_INVALID), pair.Describe());
697 return HCCL_E_PARA;731 return HCCL_E_PARA;
698 }732 }
699 if (!IsSupportedSemanticMemType(pair.src.memType) || !IsSupportedSemanticMemType(pair.dst.memType)) {733 if (!IsSupportedSemanticMemType(pair.src.memType) || !IsSupportedSemanticMemType(pair.dst.memType)) {
700- HCCL_VM_ERROR("[TaskGraphSemanticCheckV3] Unsupported memory type, pair={}", pair.Describe());734+ HCCL_VM_ERROR("{} This data mapping uses a memory type that semantic check does not support, "
735+ "dataMapping={}", MakeErrorCodeText(ErrorCode::SINGLETASK_SLICE_INVALID), pair.Describe());
701 return HCCL_E_NOT_SUPPORT;736 return HCCL_E_NOT_SUPPORT;
702 }737 }
703 738 
@@ -709,14 +744,14 @@ HcclResult LoadSliceOpPairSrcSemantics(const SliceOpPairV3 &pair, const Semantic
709 HcclResult ret = CheckBufSemantics(srcViews, pair.src.offset, pair.src.len, true);744 HcclResult ret = CheckBufSemantics(srcViews, pair.src.offset, pair.src.len, true);
710 if (ret != HCCL_SUCCESS) {745 if (ret != HCCL_SUCCESS) {
711 if (pair.op == SliceOpV3::REDUCE) {746 if (pair.op == SliceOpV3::REDUCE) {
712- HCCL_VM_ERROR("[TaskGraphSemanticCheckV3] Incomplete reduce source semantics, pair={}",747+ HCCL_VM_ERROR("{} Source data needed by this reduce is missing, dataMapping={}",
713- pair.Describe());748+ MakeErrorCodeText(ErrorCode::SEMANTIC_REDUCE_ERROR), pair.Describe());
714 return ret;749 return ret;
715 }750 }
716 // 老语义实现对 override 的不完整源语义是放行但告警;751 // 老语义实现对 override 的不完整源语义是放行但告警;
717 // 这里保持同样语义,至少把“并非完整 memcpy 语义”的风险显式打出来。752 // 这里保持同样语义,至少把“并非完整 memcpy 语义”的风险显式打出来。
718- HCCL_VM_WARN("[TaskGraphSemanticCheckV3] Incomplete override source semantics, pair={}",753+ HCCL_VM_WARN("{} Source data needed by this overwrite is missing, dataMapping={}",
719- pair.Describe());754+ MakeErrorCodeText(ErrorCode::SEMANTIC_REDUCE_ERROR), pair.Describe());
720 }755 }
721 756 
722 srcSemantics.reserve(srcViews.size());757 srcSemantics.reserve(srcViews.size());
@@ -741,7 +776,6 @@ HcclResult ApplyPreparedSliceOpPair(const PreparedSliceOpPairV3 &prepared, Seman
741 }776 }
742 HcclResult ret = ApplySrcSemanticsToDst(prepared.pair, state, srcViews);777 HcclResult ret = ApplySrcSemanticsToDst(prepared.pair, state, srcViews);
743 if (ret != HCCL_SUCCESS) {778 if (ret != HCCL_SUCCESS) {
744- HCCL_VM_ERROR("[TaskGraphSemanticCheckV3] Apply src semantics failed, pair={}", prepared.pair.Describe());
745 return ret;779 return ret;
746 }780 }
747 return HCCL_SUCCESS;781 return HCCL_SUCCESS;
@@ -844,8 +878,6 @@ HcclResult SimulateTask(const TaskNode *node, SemanticState &state)
844 prepared.pair = pair;878 prepared.pair = pair;
845 HcclResult ret = LoadSliceOpPairSrcSemantics(pair, state, prepared.srcSemantics);879 HcclResult ret = LoadSliceOpPairSrcSemantics(pair, state, prepared.srcSemantics);
846 if (ret != HCCL_SUCCESS) {880 if (ret != HCCL_SUCCESS) {
847- HCCL_VM_ERROR("[TaskGraphSemanticCheckV3] Load slice op pair src semantics failed, node={}, pair={}",
848- node->Describe(), pair.Describe());
849 return ret;881 return ret;
850 }882 }
851 preparedPairs.push_back(std::move(prepared));883 preparedPairs.push_back(std::move(prepared));
@@ -853,8 +885,6 @@ HcclResult SimulateTask(const TaskNode *node, SemanticState &state)
853 for (const auto &prepared : preparedPairs) {885 for (const auto &prepared : preparedPairs) {
854 HcclResult ret = ApplyPreparedSliceOpPair(prepared, state);886 HcclResult ret = ApplyPreparedSliceOpPair(prepared, state);
855 if (ret != HCCL_SUCCESS) {887 if (ret != HCCL_SUCCESS) {
856- HCCL_VM_ERROR("[TaskGraphSemanticCheckV3] Process slice op pair failed, node={}, pair={}",
857- node->Describe(), prepared.pair.Describe());
858 return ret;888 return ret;
859 }889 }
860 }890 }
@@ -914,8 +944,12 @@ HcclResult CheckFinalOutput(const SemanticState &state)
914 case HCCL_CMD_BATCH_SEND_RECV:944 case HCCL_CMD_BATCH_SEND_RECV:
915 return TaskCheckBatchSendRecvSemantics(allRankMemSemantics, state.dataSize);945 return TaskCheckBatchSendRecvSemantics(allRankMemSemantics, state.dataSize);
916 default:946 default:
917- HCCL_VM_WARN("[TaskGraphSemanticCheckV3] Unsupported op type for final semantic check, op={}",947+ HCCL_VM_WARN("{} Final output validation does not support this collective type yet, "
918- static_cast<uint32_t>(state.param.cmdType));948+ "collectiveType={}, rankCount={}, dataType={}, elementCount={}, reduceType={}",
949+ MakeErrorCodeText(ErrorCode::CHECKER_RUNTIME_ERROR),
950+ HcclCmdTypeToString(state.param.cmdType), state.param.rankSize,
951+ HcclDataTypeToString(state.param.dataType), state.param.dataCount,
952+ DumpReduceOpToString(state.param.reduceType));
919 return HCCL_E_NOT_SUPPORT;953 return HCCL_E_NOT_SUPPORT;
920 }954 }
921}955}
@@ -944,7 +978,8 @@ void FillStats(const SemanticState &state, size_t handledNodeCount, SemanticChec
944HcclResult CheckSemantics(const TaskNode *start, SemanticCheckStats *stats)978HcclResult CheckSemantics(const TaskNode *start, SemanticCheckStats *stats)
945{979{
946 if (start == nullptr) {980 if (start == nullptr) {
947- HCCL_VM_ERROR("[TaskGraphSemanticCheckV3] Null start node");981+ HCCL_VM_ERROR("{} Semantic check cannot start because the main start node is missing, "
982+ "mainStartNode=null", MakeErrorCodeText(ErrorCode::CHECKER_RUNTIME_ERROR));
948 return HCCL_E_PTR;983 return HCCL_E_PTR;
949 }984 }
950 985 
@@ -993,12 +1028,12 @@ HcclResult CheckSemantics(const TaskNode *start, SemanticCheckStats *stats)
993 ++handled;1028 ++handled;
994 ret = SimulateTask(node, state);1029 ret = SimulateTask(node, state);
995 if (ret != HCCL_SUCCESS) {1030 if (ret != HCCL_SUCCESS) {
996- HCCL_VM_ERROR("[TaskGraphSemanticCheckV3] Semantic simulate failed, node={}", node->Describe());
997 return ret;1031 return ret;
998 }1032 }
999 for (const TaskNode *child : node->GetChildren()) {1033 for (const TaskNode *child : node->GetChildren()) {
1000 if (child == nullptr) {1034 if (child == nullptr) {
1001- HCCL_VM_ERROR("[TaskGraphSemanticCheckV3] Invalid null child, parent={}", node->Describe());1035+ HCCL_VM_ERROR("{} Graph structure is broken because one child node is null, parent={}",
1036+ MakeErrorCodeText(ErrorCode::CHECKER_RUNTIME_ERROR), node->Describe());
1002 return HCCL_E_PTR;1037 return HCCL_E_PTR;
1003 }1038 }
1004 auto iter = pendingParents.find(child->GetNodeId());1039 auto iter = pendingParents.find(child->GetNodeId());
@@ -1013,14 +1048,26 @@ HcclResult CheckSemantics(const TaskNode *start, SemanticCheckStats *stats)
1013 }1048 }
1014 1049 
1015 if (handled != nodes.size()) {1050 if (handled != nodes.size()) {
1016- HCCL_VM_ERROR("[TaskGraphSemanticCheckV3] Graph has unresolved dependencies, handled={}, total={}",1051+ const TaskNode *firstRemainingNode = nullptr;
1017- handled, nodes.size());1052+ for (const auto &entry : pendingParents) {
1053+ if (entry.second != 0) {
1054+ auto nodeIter = nodes.find(entry.first);
1055+ if (nodeIter != nodes.end()) {
1056+ firstRemainingNode = nodeIter->second;
1057+ break;
1058+ }
1059+ }
1060+ }
1061+ HCCL_VM_ERROR("{} Output simulation stopped because some tasks still have unresolved "
1062+ "dependencies, handledNodeCount={}, totalNodeCount={}, firstRemainingNode={}",
1063+ MakeErrorCodeText(ErrorCode::CHECKER_RUNTIME_ERROR), handled, nodes.size(),
1064+ firstRemainingNode == nullptr ? std::string("node=null") : firstRemainingNode->Describe());
1018 return HCCL_E_INTERNAL;1065 return HCCL_E_INTERNAL;
1019 }1066 }
1020 1067 
1021 ret = CheckFinalOutput(state);1068 ret = CheckFinalOutput(state);
1022 if (ret != HCCL_SUCCESS && ret != HCCL_E_NOT_SUPPORT) {1069 if (ret != HCCL_SUCCESS && ret != HCCL_E_NOT_SUPPORT) {
1023- HCCL_VM_ERROR("[TaskGraphSemanticCheckV3] Final semantic check failed, ret={}",1070+ HCCL_VM_ERROR("Final semantic check failed, ret={}",
1024 static_cast<uint32_t>(ret));1071 static_cast<uint32_t>(ret));
1025 return ret;1072 return ret;
1026 }1073 }
Mtest/hccl_vm/src/plugin/checker/src/framework/task_graph_generator_v3/task_graph_single_task_check_v3.cc+116-52
@@ -23,6 +23,8 @@
23 23 
24#include "sim_log.h"24#include "sim_log.h"
25#include "storage_manager.h"25#include "storage_manager.h"
26+#include "utils/check_utils.h"
27+#include "utils/error_codes.h"
26 28 
27namespace HcclSim {29namespace HcclSim {
28namespace TaskGraphGeneratorV3 {30namespace TaskGraphGeneratorV3 {
@@ -80,15 +82,32 @@ const char *TaskTypeName(TaskType taskType)
80std::string DescribePosition(const TaskPosition &loc)82std::string DescribePosition(const TaskPosition &loc)
81{83{
82 std::ostringstream os;84 std::ostringstream os;
83- os << "rank=" << loc.rankId << ", stream=" << loc.streamId;85+ os << "rankId=";
86+ if (loc.rankId == INVALID_RANK_ID) {
87+ os << "invalid";
88+ } else {
89+ os << loc.rankId;
90+ }
91+ os << ", streamId=";
92+ if (loc.streamId == INVALID_STREAM_ID) {
93+ os << "invalid";
94+ } else {
95+ os << loc.streamId;
96+ }
84 return os.str();97 return os.str();
85}98}
86 99 
87std::string DescribeMemSlice(const MemSlice &slice)100std::string DescribeMemSlice(const MemSlice &slice)
88{101{
89 std::ostringstream os;102 std::ostringstream os;
90- os << "{rank=" << slice.rankId << ", mem=" << static_cast<uint32_t>(slice.memType)103+ os << "{rankId=";
91- << ", offset=0x" << std::hex << slice.offset << ", len=0x" << slice.len << std::dec << "}";104+ if (slice.rankId == INVALID_RANK_ID) {
105+ os << "invalid";
106+ } else {
107+ os << slice.rankId;
108+ }
109+ os << ", memType=" << DescribeMemType(slice.memType)
110+ << ", offset=0x" << std::hex << slice.offset << ", length=0x" << slice.len << std::dec << "}";
92 return os.str();111 return os.str();
93}112}
94 113 
@@ -224,8 +243,10 @@ HcclResult AccumulateCoveredSizeByKey(const TaskNode *node, const std::vector<Me
224 continue;243 continue;
225 }244 }
226 if (slice.len > std::numeric_limits<uint64_t>::max() - slice.offset) {245 if (slice.len > std::numeric_limits<uint64_t>::max() - slice.offset) {
227- HCCL_VM_ERROR("[TaskGraphSingleTaskCheckV3] Memory slice overflow while accumulating coverage, "246+ HCCL_VM_ERROR("{} One memory slice is invalid because its end address overflows "
228- "node={}, slice={}", node->Describe(), DescribeMemSlice(slice));247+ "while total coverage is being calculated, task={}, memorySlice={}, offset=0x{:x}, length=0x{:x}",
248+ MakeErrorCodeText(ErrorCode::SINGLETASK_SLICE_INVALID), node->Describe(),
249+ DescribeMemSlice(slice), slice.offset, slice.len);
229 return HCCL_E_PARA;250 return HCCL_E_PARA;
230 }251 }
231 const MemoryKey key{slice.rankId, slice.memType};252 const MemoryKey key{slice.rankId, slice.memType};
@@ -293,7 +314,7 @@ HcclResult BuildCoverageSummary(const TaskNode *node, const std::vector<std::vec
293}314}
294 315 
295HcclResult CheckCoverageMultiple(const TaskNode *node, uint64_t srcSize, uint64_t dstSize,316HcclResult CheckCoverageMultiple(const TaskNode *node, uint64_t srcSize, uint64_t dstSize,
296- const std::string &srcName, const std::string &dstName)317+ const std::string &group)
297{318{
298 if (node == nullptr) {319 if (node == nullptr) {
299 return HCCL_E_PTR;320 return HCCL_E_PTR;
@@ -302,15 +323,17 @@ HcclResult CheckCoverageMultiple(const TaskNode *node, uint64_t srcSize, uint64_
302 if (srcSize == 0) {323 if (srcSize == 0) {
303 return HCCL_SUCCESS;324 return HCCL_SUCCESS;
304 }325 }
305- HCCL_VM_ERROR("[TaskGraphSingleTaskCheckV3] Zero destination coverage with non-zero source coverage, "326+ HCCL_VM_ERROR("{} Source data exists but the target range is empty, task={}, "
306- "node={}, {}={}, {}={}", node->Describe(), srcName, srcSize, dstName, dstSize);327+ "srcDataSize={}, dstDataSize={}, group={}",
328+ MakeErrorCodeText(ErrorCode::SINGLETASK_SLICE_INVALID), node->Describe(), srcSize, dstSize, group);
307 return HCCL_E_INTERNAL;329 return HCCL_E_INTERNAL;
308 }330 }
309 if (srcSize % dstSize == 0) {331 if (srcSize % dstSize == 0) {
310 return HCCL_SUCCESS;332 return HCCL_SUCCESS;
311 }333 }
312- HCCL_VM_ERROR("[TaskGraphSingleTaskCheckV3] Source coverage is not a multiple of destination coverage, "334+ HCCL_VM_ERROR("{} Source data size is not an integer multiple of target data size, "
313- "node={}, {}={}, {}={}", node->Describe(), srcName, srcSize, dstName, dstSize);335+ "task={}, srcDataSize={}, dstDataSize={}, group={}",
336+ MakeErrorCodeText(ErrorCode::SINGLETASK_SLICE_INVALID), node->Describe(), srcSize, dstSize, group);
314 return HCCL_E_INTERNAL;337 return HCCL_E_INTERNAL;
315}338}
316 339 
@@ -323,8 +346,10 @@ HcclResult CheckSliceLenEqual(const TaskNode *node, const MemSlice &src, const M
323 if (src.len == dst.len) {346 if (src.len == dst.len) {
324 return HCCL_SUCCESS;347 return HCCL_SUCCESS;
325 }348 }
326- HCCL_VM_ERROR("[TaskGraphSingleTaskCheckV3] Batch trans slice length mismatch, node={}, label={}, index={}, "349+ HCCL_VM_ERROR("{} In one batch transfer pair, source length and target length are "
327- "src={}, dst={}", node->Describe(), label, index, DescribeMemSlice(src), DescribeMemSlice(dst));350+ "different, task={}, group={}, pairIndex={}, sourceMemorySlice={}, targetMemorySlice={}",
351+ MakeErrorCodeText(ErrorCode::SINGLETASK_SLICE_INVALID), node->Describe(), label, index,
352+ DescribeMemSlice(src), DescribeMemSlice(dst));
328 return HCCL_E_INTERNAL;353 return HCCL_E_INTERNAL;
329}354}
330 355 
@@ -365,8 +390,10 @@ HcclResult CheckBatchTransSlices(const TaskNode *node, const std::vector<MemSlic
365 // 原始 src/dst 负责逐对检查 memcpy 语义是否满足“长度一一对应”;390 // 原始 src/dst 负责逐对检查 memcpy 语义是否满足“长度一一对应”;
366 // merged src/dst 只用于更快地检查 batch 节点内部是否存在自冲突。391 // merged src/dst 只用于更快地检查 batch 节点内部是否存在自冲突。
367 if (srcs.size() != dsts.size()) {392 if (srcs.size() != dsts.size()) {
368- HCCL_VM_ERROR("[TaskGraphSingleTaskCheckV3] Batch trans pair size mismatch, node={}, label={}, "393+ HCCL_VM_ERROR("{} Batch transfer has different counts of source memory slices and "
369- "srcCount={}, dstCount={}", node->Describe(), label, srcs.size(), dsts.size());394+ "target memory slices, task={}, group={}, sourceMemorySliceCount={}, targetMemorySliceCount={}",
395+ MakeErrorCodeText(ErrorCode::SINGLETASK_SLICE_INVALID), node->Describe(), label, srcs.size(),
396+ dsts.size());
370 return HCCL_E_PARA;397 return HCCL_E_PARA;
371 }398 }
372 for (size_t index = 0; index < srcs.size(); ++index) {399 for (size_t index = 0; index < srcs.size(); ++index) {
@@ -409,9 +436,10 @@ HcclResult CheckBatchTransSlices(const TaskNode *node, const std::vector<MemSlic
409 }436 }
410 }437 }
411 }438 }
412- HCCL_VM_DEBUG("[TaskGraphSingleTaskCheckV3] Batch trans merged slices checked, "439+ HCCL_VM_DEBUG("Batch transfer merged-slice check finished, "
413- "nodeId={}, label={}, originalSrcCount={}, originalDstCount={}, mergedSrcCount={}, mergedDstCount={}, "440+ "taskId={}, group={}, originalSourceMemorySliceCount={}, originalTargetMemorySliceCount={}, "
414- "dedupMergedSrcCount={}, dedupMergedDstCount={}", node->GetNodeId(), label, srcs.size(), dsts.size(),441+ "mergedSourceMemorySliceCount={}, mergedTargetMemorySliceCount={}, uniqueMergedSourceMemorySliceCount={}, "
442+ "uniqueMergedTargetMemorySliceCount={}", node->GetNodeId(), label, srcs.size(), dsts.size(),
415 mergedSrcs.size(), mergedDsts.size(), dedupMergedSrcs.size(), dedupMergedDsts.size());443 mergedSrcs.size(), mergedDsts.size(), dedupMergedSrcs.size(), dedupMergedDsts.size());
416 return HCCL_SUCCESS;444 return HCCL_SUCCESS;
417}445}
@@ -426,8 +454,10 @@ HcclResult CheckBatchReduceSlices(const TaskNode *node, const std::vector<std::v
426 // batch reduce 仍然按原始 group 校验:每组 src 必须能落到同一个 dst,454 // batch reduce 仍然按原始 group 校验:每组 src 必须能落到同一个 dst,
427 // 且所有 src 总覆盖量应当是 dst 总覆盖量的整倍数。455 // 且所有 src 总覆盖量应当是 dst 总覆盖量的整倍数。
428 if (srcGroups.empty() || srcGroups.size() != dsts.size()) {456 if (srcGroups.empty() || srcGroups.size() != dsts.size()) {
429- HCCL_VM_ERROR("[TaskGraphSingleTaskCheckV3] Batch reduce group size mismatch, node={}, label={}, "457+ HCCL_VM_ERROR("{} Batch reduce has different counts of source groups and target "
430- "srcGroupCount={}, dstCount={}", node->Describe(), label, srcGroups.size(), dsts.size());458+ "memory slices, task={}, group={}, sourceGroupCount={}, targetMemorySliceCount={}",
459+ MakeErrorCodeText(ErrorCode::SINGLETASK_SLICE_INVALID), node->Describe(), label, srcGroups.size(),
460+ dsts.size());
431 return HCCL_E_PARA;461 return HCCL_E_PARA;
432 }462 }
433 for (size_t groupIndex = 0; groupIndex < srcGroups.size(); ++groupIndex) {463 for (size_t groupIndex = 0; groupIndex < srcGroups.size(); ++groupIndex) {
@@ -436,8 +466,9 @@ HcclResult CheckBatchReduceSlices(const TaskNode *node, const std::vector<std::v
436 return ret;466 return ret;
437 }467 }
438 if (srcGroups[groupIndex].empty()) {468 if (srcGroups[groupIndex].empty()) {
439- HCCL_VM_ERROR("[TaskGraphSingleTaskCheckV3] Batch reduce source list is empty, node={}, label={}, "469+ HCCL_VM_ERROR("{} One reduce group has no source data at all, task={}, group={}, "
440- "groupIndex={}", node->Describe(), label, groupIndex);470+ "groupIndex={}", MakeErrorCodeText(ErrorCode::SINGLETASK_SLICE_INVALID), node->Describe(),
471+ label, groupIndex);
441 return HCCL_E_PARA;472 return HCCL_E_PARA;
442 }473 }
443 for (const auto &src : srcGroups[groupIndex]) {474 for (const auto &src : srcGroups[groupIndex]) {
@@ -460,8 +491,7 @@ HcclResult CheckBatchReduceSlices(const TaskNode *node, const std::vector<std::v
460 if (ret != HCCL_SUCCESS) {491 if (ret != HCCL_SUCCESS) {
461 return ret;492 return ret;
462 }493 }
463- return CheckCoverageMultiple(node, srcSummary.totalCoveredSize, dstSummary.totalCoveredSize,494+ return CheckCoverageMultiple(node, srcSummary.totalCoveredSize, dstSummary.totalCoveredSize, label);
464- label + " srcCoveredSize", label + " dstCoveredSize");
465}495}
466 496 
467HcclResult CheckSingleSlice(const TaskNode *node, const MemSlice &slice)497HcclResult CheckSingleSlice(const TaskNode *node, const MemSlice &slice)
@@ -473,21 +503,28 @@ HcclResult CheckSingleSlice(const TaskNode *node, const MemSlice &slice)
473 return HCCL_SUCCESS;503 return HCCL_SUCCESS;
474 }504 }
475 if (slice.rankId == INVALID_RANK_ID || slice.memType == MemType::INVALID) {505 if (slice.rankId == INVALID_RANK_ID || slice.memType == MemType::INVALID) {
476- HCCL_VM_ERROR("[TaskGraphSingleTaskCheckV3] Invalid memory slice, node={}, slice={}",506+ HCCL_VM_ERROR("{} One memory slice is missing a valid rank or memory type, task={}, "
477- node->Describe(), DescribeMemSlice(slice));507+ "memorySlice={}, rankId={}, memType={}",
508+ MakeErrorCodeText(ErrorCode::SINGLETASK_SLICE_INVALID), node->Describe(),
509+ DescribeMemSlice(slice), slice.rankId == INVALID_RANK_ID ? "invalid" : std::to_string(slice.rankId),
510+ DescribeMemType(slice.memType));
478 return HCCL_E_PARA;511 return HCCL_E_PARA;
479 }512 }
480 if (slice.len > std::numeric_limits<uint64_t>::max() - slice.offset) {513 if (slice.len > std::numeric_limits<uint64_t>::max() - slice.offset) {
481- HCCL_VM_ERROR("[TaskGraphSingleTaskCheckV3] Memory slice overflow, node={}, slice={}",514+ HCCL_VM_ERROR("{} One memory slice is invalid because offset + length exceeds the "
482- node->Describe(), DescribeMemSlice(slice));515+ "numeric limit, task={}, memorySlice={}, offset=0x{:x}, length=0x{:x}",
516+ MakeErrorCodeText(ErrorCode::SINGLETASK_SLICE_INVALID), node->Describe(),
517+ DescribeMemSlice(slice), slice.offset, slice.len);
483 return HCCL_E_PARA;518 return HCCL_E_PARA;
484 }519 }
485 520 
486 if (slice.memType == MemType::UB_AIV || slice.memType == MemType::FLAG_AIV) {521 if (slice.memType == MemType::UB_AIV || slice.memType == MemType::FLAG_AIV) {
487 const uint64_t boundSize = GetAivSliceBoundSize(slice.memType);522 const uint64_t boundSize = GetAivSliceBoundSize(slice.memType);
488 if (boundSize != 0 && slice.offset + slice.len > boundSize) {523 if (boundSize != 0 && slice.offset + slice.len > boundSize) {
489- HCCL_VM_ERROR("[TaskGraphSingleTaskCheckV3] AIV slice out-of-bounds, node={}, slice={}, boundSize={}",524+ HCCL_VM_ERROR("{} One AIV memory slice goes past the valid AIV buffer boundary, "
490- node->Describe(), DescribeMemSlice(slice), boundSize);525+ "task={}, memorySlice={}, bufferLimit={}, memType={}",
526+ MakeErrorCodeText(ErrorCode::SINGLETASK_SLICE_INVALID), node->Describe(),
527+ DescribeMemSlice(slice), boundSize, DescribeMemType(slice.memType));
491 return HCCL_E_INTERNAL;528 return HCCL_E_INTERNAL;
492 }529 }
493 return HCCL_SUCCESS;530 return HCCL_SUCCESS;
@@ -495,15 +532,20 @@ HcclResult CheckSingleSlice(const TaskNode *node, const MemSlice &slice)
495 532 
496 const BufferType storageType = ConvertMemTypeToBufferType(slice.memType);533 const BufferType storageType = ConvertMemTypeToBufferType(slice.memType);
497 if (storageType == BufferType::RESERVED) {534 if (storageType == BufferType::RESERVED) {
498- HCCL_VM_ERROR("[TaskGraphSingleTaskCheckV3] Unsupported memory type, node={}, slice={}",535+ HCCL_VM_ERROR("{} This task uses a memory type that the checker does not support, "
499- node->Describe(), DescribeMemSlice(slice));536+ "task={}, memorySlice={}, memType={}",
537+ MakeErrorCodeText(ErrorCode::SINGLETASK_SLICE_INVALID), node->Describe(),
538+ DescribeMemSlice(slice), DescribeMemType(slice.memType));
500 return HCCL_E_NOT_SUPPORT;539 return HCCL_E_NOT_SUPPORT;
501 }540 }
502 541 
503 const uint64_t blockSize = StorageManager::GetInstance().GetBlockSize(slice.rankId, storageType);542 const uint64_t blockSize = StorageManager::GetInstance().GetBlockSize(slice.rankId, storageType);
504 if (slice.offset + slice.len > blockSize) {543 if (slice.offset + slice.len > blockSize) {
505- HCCL_VM_ERROR("[TaskGraphSingleTaskCheckV3] Failed to check slice, {}, node={}, slice={}, blockSize={}",544+ HCCL_VM_ERROR("{} One memory slice goes past the end of its buffer, task={}, "
506- DescribePosition(node->GetPosition()), node->Describe(), DescribeMemSlice(slice), blockSize);545+ "memorySlice={}, bufferType={}, bufferSize={}, position={}",
546+ MakeErrorCodeText(ErrorCode::SINGLETASK_SLICE_INVALID), node->Describe(),
547+ DescribeMemSlice(slice), BufferTypeToString(storageType), blockSize,
548+ DescribePosition(node->GetPosition()));
507 return HCCL_E_INTERNAL;549 return HCCL_E_INTERNAL;
508 }550 }
509 return HCCL_SUCCESS;551 return HCCL_SUCCESS;
@@ -519,15 +561,18 @@ HcclResult CheckTwoSliceOverlap(const TaskNode *node, const MemSlice &lhs, const
519 }561 }
520 if (lhs.len > std::numeric_limits<uint64_t>::max() - lhs.offset ||562 if (lhs.len > std::numeric_limits<uint64_t>::max() - lhs.offset ||
521 rhs.len > std::numeric_limits<uint64_t>::max() - rhs.offset) {563 rhs.len > std::numeric_limits<uint64_t>::max() - rhs.offset) {
522- HCCL_VM_ERROR("[TaskGraphSingleTaskCheckV3] Memory slice overflow, node={}, sliceA={}, sliceB={}",564+ HCCL_VM_ERROR("{} Two memory slices cannot be compared because one slice already "
523- node->Describe(), DescribeMemSlice(lhs), DescribeMemSlice(rhs));565+ "overflowed, task={}, memorySlice1={}, memorySlice2={}",
566+ MakeErrorCodeText(ErrorCode::SINGLETASK_SLICE_INVALID), node->Describe(),
567+ DescribeMemSlice(lhs), DescribeMemSlice(rhs));
524 return HCCL_E_PARA;568 return HCCL_E_PARA;
525 }569 }
526 570 
527 if (node->HasCcuTrace()) {571 if (node->HasCcuTrace()) {
528 // CCU mode下,允许src == dst572 // CCU mode下,允许src == dst
529 if (lhs.rankId == rhs.rankId && IsSameMemoryType(lhs, rhs) && lhs.offset == rhs.offset && lhs.len == rhs.len) {573 if (lhs.rankId == rhs.rankId && IsSameMemoryType(lhs, rhs) && lhs.offset == rhs.offset && lhs.len == rhs.len) {
530- HCCL_VM_WARN("[TaskGraphSingleTaskCheckV3] MemSlice are same (src == dst), which may affect performance, {}, node={}, sliceA={}, sliceB={}",574+ HCCL_VM_WARN("Source and destination use the same memory slice, which may hurt "
575+ "performance, position={}, task={}, memorySlice1={}, memorySlice2={}",
531 DescribePosition(node->GetPosition()), node->Describe(), DescribeMemSlice(lhs), DescribeMemSlice(rhs));576 DescribePosition(node->GetPosition()), node->Describe(), DescribeMemSlice(lhs), DescribeMemSlice(rhs));
532 return HCCL_SUCCESS;577 return HCCL_SUCCESS;
533 }578 }
@@ -536,8 +581,10 @@ HcclResult CheckTwoSliceOverlap(const TaskNode *node, const MemSlice &lhs, const
536 const bool conflictCase1 = lhs.offset >= rhs.offset && lhs.offset < (rhs.offset + rhs.len);581 const bool conflictCase1 = lhs.offset >= rhs.offset && lhs.offset < (rhs.offset + rhs.len);
537 const bool conflictCase2 = rhs.offset >= lhs.offset && rhs.offset < (lhs.offset + lhs.len);582 const bool conflictCase2 = rhs.offset >= lhs.offset && rhs.offset < (lhs.offset + lhs.len);
538 if (conflictCase1 || conflictCase2) {583 if (conflictCase1 || conflictCase2) {
539- HCCL_VM_ERROR("[TaskGraphSingleTaskCheckV3] Slice conflict, {}, node={}, sliceA={}, sliceB={}",584+ HCCL_VM_ERROR("{} Two memory slices overlap inside the same task, task={}, "
540- DescribePosition(node->GetPosition()), node->Describe(), DescribeMemSlice(lhs), DescribeMemSlice(rhs));585+ "memorySlice1={}, memorySlice2={}, position={}",
586+ MakeErrorCodeText(ErrorCode::SINGLETASK_SLICE_CONFLICT), node->Describe(),
587+ DescribeMemSlice(lhs), DescribeMemSlice(rhs), DescribePosition(node->GetPosition()));
541 return HCCL_E_INTERNAL;588 return HCCL_E_INTERNAL;
542 }589 }
543 return HCCL_SUCCESS;590 return HCCL_SUCCESS;
@@ -566,14 +613,18 @@ HcclResult CheckAivFlagNodeMem(const TaskNode *node)
566 return HCCL_SUCCESS;613 return HCCL_SUCCESS;
567 }614 }
568 if (flag->flagOffset > (std::numeric_limits<uint64_t>::max() - sizeof(int32_t)) / AIV_FLAG_SLOT_BYTES) {615 if (flag->flagOffset > (std::numeric_limits<uint64_t>::max() - sizeof(int32_t)) / AIV_FLAG_SLOT_BYTES) {
569- HCCL_VM_ERROR("[TaskGraphSingleTaskCheckV3] AIV flag offset overflow, node={}", node->Describe());616+ HCCL_VM_ERROR("{} AIV flag address overflowed while calculating its byte offset, "
617+ "task={}, flagOffset={}", MakeErrorCodeText(ErrorCode::SINGLETASK_SLICE_INVALID), node->Describe(),
618+ flag->flagOffset);
570 return HCCL_E_PARA;619 return HCCL_E_PARA;
571 }620 }
572 const uint64_t flagByteOffset = flag->flagOffset * AIV_FLAG_SLOT_BYTES;621 const uint64_t flagByteOffset = flag->flagOffset * AIV_FLAG_SLOT_BYTES;
573 const uint64_t flagBufferSize = GetAivSliceBoundSize(MemType::FLAG_AIV);622 const uint64_t flagBufferSize = GetAivSliceBoundSize(MemType::FLAG_AIV);
574 if (flagBufferSize != 0 && flagByteOffset + sizeof(int32_t) > flagBufferSize) {623 if (flagBufferSize != 0 && flagByteOffset + sizeof(int32_t) > flagBufferSize) {
575- HCCL_VM_ERROR("[TaskGraphSingleTaskCheckV3] AIV flag offset out-of-bounds, node={}, "624+ HCCL_VM_ERROR("{} AIV flag address goes past the valid flag buffer boundary, task={}, "
576- "flagByteOffset={}, flagBufferSize={}", node->Describe(), flagByteOffset, flagBufferSize);625+ "flagByteOffset={}, flagBufferSize={}",
626+ MakeErrorCodeText(ErrorCode::SINGLETASK_SLICE_INVALID), node->Describe(), flagByteOffset,
627+ flagBufferSize);
577 return HCCL_E_INTERNAL;628 return HCCL_E_INTERNAL;
578 }629 }
579 return HCCL_SUCCESS;630 return HCCL_SUCCESS;
@@ -611,7 +662,8 @@ HcclResult CheckDataMoveTaskMem(const TaskNode *node)
611 }662 }
612 const std::vector<MemSlice> &srcs = reduce->GetSrcs();663 const std::vector<MemSlice> &srcs = reduce->GetSrcs();
613 if (srcs.empty()) {664 if (srcs.empty()) {
614- HCCL_VM_ERROR("[TaskGraphSingleTaskCheckV3] Reduce source list is empty, node={}", node->Describe());665+ HCCL_VM_ERROR("{} This reduce task has no source data at all, task={}",
666+ MakeErrorCodeText(ErrorCode::SINGLETASK_SLICE_INVALID), node->Describe());
615 return HCCL_E_PARA;667 return HCCL_E_PARA;
616 }668 }
617 for (const auto &src : srcs) {669 for (const auto &src : srcs) {
@@ -674,8 +726,8 @@ HcclResult CheckSlaveTaskQueue(const std::vector<std::unique_ptr<TaskNode>> &nod
674 const auto &stream = taskQueue[streamId];726 const auto &stream = taskQueue[streamId];
675 const size_t taskSize = stream.size();727 const size_t taskSize = stream.size();
676 if (StreamHasAivGraph(nodes, stream)) {728 if (StreamHasAivGraph(nodes, stream)) {
677- HCCL_VM_DEBUG("[TaskGraphSingleTaskCheckV3] Skip legacy slave stream shape check for AIV stream, "729+ HCCL_VM_DEBUG("Skip this slave-stream structure check because the stream contains "
678- "rank={}, stream={}", rankId, streamId);730+ "an AIV graph task, rankId={}, streamId={}", rankId, streamId);
679 continue;731 continue;
680 }732 }
681 if (taskSize < 2) {733 if (taskSize < 2) {
@@ -685,31 +737,43 @@ HcclResult CheckSlaveTaskQueue(const std::vector<std::unique_ptr<TaskNode>> &nod
685 const TaskNode *firstTask = GetQueueNode(nodes, stream.front());737 const TaskNode *firstTask = GetQueueNode(nodes, stream.front());
686 const TaskNode *lastTask = GetQueueNode(nodes, stream.back());738 const TaskNode *lastTask = GetQueueNode(nodes, stream.back());
687 if (firstTask == nullptr || lastTask == nullptr) {739 if (firstTask == nullptr || lastTask == nullptr) {
688- HCCL_VM_ERROR("[TaskGraphSingleTaskCheckV3] Invalid slave stream node, rank={}, stream={}",740+ HCCL_VM_ERROR("{} This slave stream is missing its start node "
689- rankId, streamId);741+ "or end node, rankId={}, streamId={}, taskCount={}, startNode={}, endNode={}",
742+ MakeErrorCodeText(ErrorCode::SINGLETASK_SLAVE_STREAM_INVALID), rankId, streamId, taskSize,
743+ firstTask == nullptr ? "null" : firstTask->Describe(),
744+ lastTask == nullptr ? "null" : lastTask->Describe());
690 return HCCL_E_PTR;745 return HCCL_E_PTR;
691 }746 }
692 747 
693 size_t backStep = 1;748 size_t backStep = 1;
749+ size_t skippedEmptyLocalCopyCount = 0;
694 while (IsEmptyLocalCopy(lastTask) && backStep < taskSize) {750 while (IsEmptyLocalCopy(lastTask) && backStep < taskSize) {
695 lastTask = GetQueueNode(nodes, stream[taskSize - 1 - backStep]);751 lastTask = GetQueueNode(nodes, stream[taskSize - 1 - backStep]);
696 if (lastTask == nullptr) {752 if (lastTask == nullptr) {
697- HCCL_VM_ERROR("[TaskGraphSingleTaskCheckV3] Invalid slave stream tail node, rank={}, stream={}",753+ HCCL_VM_ERROR("{} This slave stream still has no valid end "
698- rankId, streamId);754+ "node after empty local-copy tasks are skipped, rankId={}, streamId={}, "
755+ "skippedEmptyLocalCopyCount={}, currentTailNode={}",
756+ MakeErrorCodeText(ErrorCode::SINGLETASK_SLAVE_STREAM_INVALID), rankId, streamId,
757+ skippedEmptyLocalCopyCount, "null");
699 return HCCL_E_PTR;758 return HCCL_E_PTR;
700 }759 }
760+ ++skippedEmptyLocalCopyCount;
701 ++backStep;761 ++backStep;
702 }762 }
703 763 
704 if (!IsLocalAicpuWait(firstTask)) {764 if (!IsLocalAicpuWait(firstTask)) {
705- HCCL_VM_ERROR("[TaskGraphSingleTaskCheckV3] RankId:{}, streamId:{} first task type should be "765+ HCCL_VM_ERROR("{} The first task in this slave stream is not a "
706- "local WAIT, while is {}", rankId, streamId, TaskTypeName(firstTask->GetType()));766+ "local WAIT task, rankId={}, streamId={}, actualFirstTaskType={}, firstTask={}",
767+ MakeErrorCodeText(ErrorCode::SINGLETASK_SLAVE_STREAM_INVALID), rankId, streamId,
768+ TaskTypeName(firstTask->GetType()), firstTask->Describe());
707 return HCCL_E_INTERNAL;769 return HCCL_E_INTERNAL;
708 }770 }
709 771 
710 if (!IsLocalAicpuRecord(lastTask)) {772 if (!IsLocalAicpuRecord(lastTask)) {
711- HCCL_VM_ERROR("[TaskGraphSingleTaskCheckV3] RankId:{}, streamId:{} last task type should be "773+ HCCL_VM_ERROR("{} The last task in this slave stream is not a "
712- "local RECORD, while is {}", rankId, streamId, TaskTypeName(lastTask->GetType()));774+ "local RECORD task, rankId={}, streamId={}, actualLastTaskType={}, lastTask={}",
775+ MakeErrorCodeText(ErrorCode::SINGLETASK_SLAVE_STREAM_INVALID), rankId, streamId,
776+ TaskTypeName(lastTask->GetType()), lastTask->Describe());
713 return HCCL_E_INTERNAL;777 return HCCL_E_INTERNAL;
714 }778 }
715 }779 }
Mtest/hccl_vm/src/plugin/checker/src/framework/task_graph_generator_v3/task_meta_translator_v3.cc+66-9
@@ -13,10 +13,12 @@
13#include <algorithm>13#include <algorithm>
14#include <cstdint>14#include <cstdint>
15#include <limits>15#include <limits>
16+#include <sstream>
16#include <utility>17#include <utility>
17 18 
18#include "data_slice.h"19#include "data_slice.h"
19#include "sim_log.h"20#include "sim_log.h"
21+#include "utils/error_codes.h"
20 22 
21namespace HcclSim {23namespace HcclSim {
22namespace TaskGraphGeneratorV3 {24namespace TaskGraphGeneratorV3 {
@@ -112,6 +114,56 @@ CcuSqeParam MakeCcuSqeParam(const CcuTask &ccuTask)
112 return param;114 return param;
113}115}
114 116 
117+std::string DescribeTaskMetaForLog(const HcclTaskMetaData &taskMeta)
118+{
119+ std::ostringstream os;
120+ os << "{taskType=" << static_cast<int32_t>(taskMeta.taskType)
121+ << ", rankId=" << taskMeta.rankId
122+ << ", streamId=" << taskMeta.streamId;
123+ switch (taskMeta.taskType) {
124+ case HccLTaskMetaType::MEM_CPY:
125+ os << ", srcRankId=" << taskMeta.taskData.transMem.srcRankId
126+ << ", dstRankId=" << taskMeta.taskData.transMem.dstRankId
127+ << ", srcOffset=" << taskMeta.taskData.transMem.srcOffset
128+ << ", dstOffset=" << taskMeta.taskData.transMem.dstOffset
129+ << ", len=" << taskMeta.taskData.transMem.len
130+ << ", protocol=" << static_cast<uint32_t>(taskMeta.taskData.transMem.protocol);
131+ break;
132+ case HccLTaskMetaType::REDUCE:
133+ os << ", srcRankId=" << taskMeta.taskData.reduce.srcRankId
134+ << ", dstRankId=" << taskMeta.taskData.reduce.dstRankId
135+ << ", srcOffset=" << taskMeta.taskData.reduce.srcOffset
136+ << ", dstOffset=" << taskMeta.taskData.reduce.dstOffset
137+ << ", dataCount=" << taskMeta.taskData.reduce.dataCount
138+ << ", dataType=" << static_cast<uint32_t>(taskMeta.taskData.reduce.dataType)
139+ << ", reduceOp=" << static_cast<uint32_t>(taskMeta.taskData.reduce.reduceOp);
140+ break;
141+ case HccLTaskMetaType::NOTIFY_RECORD:
142+ case HccLTaskMetaType::NOTIFY_WAIT:
143+ os << ", srcRankId=" << taskMeta.taskData.notify.srcRankId
144+ << ", dstRankId=" << taskMeta.taskData.notify.dstRankId
145+ << ", notifyId=" << taskMeta.taskData.notify.notifyId
146+ << ", notifyCount=" << taskMeta.taskData.notify.notifyCount
147+ << ", protocol=" << static_cast<uint32_t>(taskMeta.taskData.notify.protocol);
148+ break;
149+ case HccLTaskMetaType::CCU_GRAPH:
150+ os << ", dieId=" << taskMeta.taskData.ccu.dieId
151+ << ", missionId=" << taskMeta.taskData.ccu.missionId
152+ << ", instStartId=" << taskMeta.taskData.ccu.instStartId
153+ << ", instCnt=" << taskMeta.taskData.ccu.instCnt
154+ << ", argSize=" << taskMeta.taskData.ccu.argSize
155+ << ", key=" << taskMeta.taskData.ccu.key;
156+ break;
157+ case HccLTaskMetaType::AIV_GRAPH:
158+ os << ", launchId=" << taskMeta.taskData.aiv.launchIdx;
159+ break;
160+ default:
161+ break;
162+ }
163+ os << "}";
164+ return os.str();
165+}
166+ 
115bool IsContiguousCcuSqe(const TaskCcuGraph &missionNode, const CcuSqeParam &sqe)167bool IsContiguousCcuSqe(const TaskCcuGraph &missionNode, const CcuSqeParam &sqe)
116{168{
117 constexpr size_t DEFAULT_CCU_QUEUE_ID = 0U;169 constexpr size_t DEFAULT_CCU_QUEUE_ID = 0U;
@@ -160,14 +212,17 @@ HcclResult TaskMetaTranslatorV3::Translate(StorageManager &storage)
160 212 
161 const HcclVmTaskMetaData &taskMetaData = storage.GetHvmTaskMetaData();213 const HcclVmTaskMetaData &taskMetaData = storage.GetHvmTaskMetaData();
162 const auto &taskMetaVec = taskMetaData.task_meta;214 const auto &taskMetaVec = taskMetaData.task_meta;
163- HCCL_VM_INFO("[TaskMetaTranslatorV3][Translate] Start translate task meta, taskMetaCount={}",215+ HCCL_VM_INFO("Start converting task metadata into graph nodes, taskMetaCount={}",
164 taskMetaVec.size());216 taskMetaVec.size());
165 for (uint32_t i = 0; i < taskMetaVec.size(); ++i) {217 for (uint32_t i = 0; i < taskMetaVec.size(); ++i) {
166 NodeId nodeId = INVALID_NODE_ID;218 NodeId nodeId = INVALID_NODE_ID;
167- const HcclResult ret = TranslateOneTaskMeta(taskMetaVec[i], storage, nodeId);219+ const HcclResult ret = TranslateOneTaskMeta(taskMetaVec[i], storage, i, nodeId);
168 if (ret != HCCL_SUCCESS) {220 if (ret != HCCL_SUCCESS) {
169- HCCL_VM_ERROR("[TaskMetaTranslatorV3] Translate task meta failed, index={}, taskType={}, ret={}",221+ HCCL_VM_ERROR("{} Failed to convert one task into a graph node, taskIndex={}, "
170- i, static_cast<int32_t>(taskMetaVec[i].taskType), static_cast<uint32_t>(ret));222+ "taskType={}, rankId={}, streamId={}, ret={}, taskMeta={}",
223+ MakeErrorCodeText(ErrorCode::GRAPH_TRANSLATE_FAILED), i,
224+ static_cast<int32_t>(taskMetaVec[i].taskType), taskMetaVec[i].rankId, taskMetaVec[i].streamId,
225+ static_cast<uint32_t>(ret), DescribeTaskMetaForLog(taskMetaVec[i]));
171 return ret;226 return ret;
172 }227 }
173 }228 }
@@ -180,8 +235,8 @@ HcclResult TaskMetaTranslatorV3::Translate(StorageManager &storage)
180 }235 }
181 }236 }
182 }237 }
183- HCCL_VM_INFO("[TaskMetaTranslatorV3][Translate] Translate task meta success, taskMetaCount={}, nodeCount={}, "238+ HCCL_VM_INFO("Finished converting task metadata into graph nodes, taskMetaCount={}, nodeCount={}, "
184- "rankCount={}, streamCount={}", taskMetaVec.size(), nodes_.size(), taskQueues_.size(), streamCount);239+ "rankCount={}, nonEmptyStreamCount={}", taskMetaVec.size(), nodes_.size(), taskQueues_.size(), streamCount);
185 return HCCL_SUCCESS;240 return HCCL_SUCCESS;
186}241}
187 242 
@@ -209,7 +264,7 @@ HcclResult TaskMetaTranslatorV3::AddTaskNode(const TaskPosition &position, std::
209}264}
210 265 
211HcclResult TaskMetaTranslatorV3::TranslateOneTaskMeta(const HcclTaskMetaData &taskMeta, StorageManager &storage,266HcclResult TaskMetaTranslatorV3::TranslateOneTaskMeta(const HcclTaskMetaData &taskMeta, StorageManager &storage,
212- NodeId &nodeId)267+ uint32_t taskIndex, NodeId &nodeId)
213{268{
214 TaskPosition position;269 TaskPosition position;
215 HcclResult ret = MakeTaskPosition(taskMeta, position);270 HcclResult ret = MakeTaskPosition(taskMeta, position);
@@ -322,8 +377,10 @@ HcclResult TaskMetaTranslatorV3::TranslateOneTaskMeta(const HcclTaskMetaData &ta
322 case HccLTaskMetaType::EVENT_WAIT:377 case HccLTaskMetaType::EVENT_WAIT:
323 case HccLTaskMetaType::EVENT_RECORD:378 case HccLTaskMetaType::EVENT_RECORD:
324 default:379 default:
325- HCCL_VM_WARN("[TaskMetaTranslatorV3] Unsupported task meta type: {}",380+ HCCL_VM_WARN("{} This task type is not supported for CheckerV3 graph generation, "
326- static_cast<int32_t>(taskMeta.taskType));381+ "taskIndex={}, taskType={}, rankId={}, streamId={}, taskMeta={}",
382+ MakeErrorCodeText(ErrorCode::GRAPH_UNSUPPORTED), taskIndex, static_cast<int32_t>(taskMeta.taskType),
383+ taskMeta.rankId, taskMeta.streamId, DescribeTaskMetaForLog(taskMeta));
327 return HCCL_E_NOT_SUPPORT;384 return HCCL_E_NOT_SUPPORT;
328 }385 }
329}386}
Mtest/hccl_vm/src/plugin/checker/src/framework/task_graph_generator_v3/task_meta_translator_v3.h+2-1
@@ -59,7 +59,8 @@ private:
59 }59 }
60 };60 };
61 61 
62- HcclResult TranslateOneTaskMeta(const HcclTaskMetaData &taskMeta, StorageManager &storage, NodeId &nodeId);62+ HcclResult TranslateOneTaskMeta(const HcclTaskMetaData &taskMeta, StorageManager &storage, uint32_t taskIndex,
63+ NodeId &nodeId);
63 HcclResult AddTaskNode(const TaskPosition &position, std::unique_ptr<TaskNode> node, NodeId &nodeId);64 HcclResult AddTaskNode(const TaskPosition &position, std::unique_ptr<TaskNode> node, NodeId &nodeId);
64 65 
65 std::vector<std::unique_ptr<TaskNode>> nodes_;66 std::vector<std::unique_ptr<TaskNode>> nodes_;
Mtest/hccl_vm/src/plugin/checker/src/log/log_stub.cc+5-5
@@ -30,19 +30,19 @@ void DlogPrintStub(int level, const char *msgBuffer)
30{30{
31 switch (level) {31 switch (level) {
32 case 0:32 case 0:
33- HCCL_VM_DEBUG(msgBuffer);33+ HCCL_VM_DEBUG("{}", msgBuffer);
34 break;34 break;
35 case 1:35 case 1:
36- HCCL_VM_INFO(msgBuffer);36+ HCCL_VM_INFO("{}", msgBuffer);
37 break;37 break;
38 case 2:38 case 2:
39- HCCL_VM_WARN(msgBuffer);39+ HCCL_VM_WARN("{}", msgBuffer);
40 break;40 break;
41 case 3:41 case 3:
42- HCCL_VM_ERROR(msgBuffer);42+ HCCL_VM_ERROR("{}", msgBuffer);
43 break;43 break;
44 default:44 default:
45- HCCL_VM_TRACE(msgBuffer);45+ HCCL_VM_TRACE("{}", msgBuffer);
46 }46 }
47}47}
48 48 
Mtest/hccl_vm/src/plugin/checker/src/utils/binary_data_operator.cc+21-21
@@ -45,7 +45,7 @@ HcclResult FileHeaderRead(FILE *fp, FileHeader &header, uint32_t magic)
45 return HcclResult::HCCL_E_INTERNAL;45 return HcclResult::HCCL_E_INTERNAL;
46 }46 }
47 if (header.magic != magic) {47 if (header.magic != magic) {
48- HCCL_VM_ERROR("[FileHeaderRead] Unmatched magic number:{:#x}≠{:#x}", header.magic, magic);48+ HCCL_VM_ERROR("Unmatched magic number:{:#x}≠{:#x}", header.magic, magic);
49 return HcclResult::HCCL_E_NOT_SUPPORT;49 return HcclResult::HCCL_E_NOT_SUPPORT;
50 }50 }
51 51 
@@ -56,13 +56,13 @@ HcclResult HcclVmSynDataRead(FILE *fp, HcclVmSynData &synData, uint32_t magic)
56{56{
57 auto ret = FileHeaderRead(fp, synData.header, magic);57 auto ret = FileHeaderRead(fp, synData.header, magic);
58 if (ret != HcclResult::HCCL_SUCCESS) {58 if (ret != HcclResult::HCCL_SUCCESS) {
59- HCCL_VM_ERROR("[HcclVmSynDataRead] Read file header failed. ");59+ HCCL_VM_ERROR("Read file header failed. ");
60 return HcclResult::HCCL_E_INTERNAL;60 return HcclResult::HCCL_E_INTERNAL;
61 }61 }
62 62 
63 ret = ModelInfoRead(fp, synData.model_info);63 ret = ModelInfoRead(fp, synData.model_info);
64 if (ret != HcclResult::HCCL_SUCCESS) {64 if (ret != HcclResult::HCCL_SUCCESS) {
65- HCCL_VM_ERROR("[HcclVmSynDataRead] Read file model info failed. ");65+ HCCL_VM_ERROR("Read file model info failed. ");
66 return HcclResult::HCCL_E_INTERNAL;66 return HcclResult::HCCL_E_INTERNAL;
67 }67 }
68 68 
@@ -71,7 +71,7 @@ HcclResult HcclVmSynDataRead(FILE *fp, HcclVmSynData &synData, uint32_t magic)
71 71 
72 ret = MemLayoutRead(fp, synData.memory_info);72 ret = MemLayoutRead(fp, synData.memory_info);
73 if (ret != HcclResult::HCCL_SUCCESS) {73 if (ret != HcclResult::HCCL_SUCCESS) {
74- HCCL_VM_ERROR("[HcclVmSynDataRead] Read file memory layout info failed. ");74+ HCCL_VM_ERROR("Read file memory layout info failed. ");
75 return HcclResult::HCCL_E_INTERNAL;75 return HcclResult::HCCL_E_INTERNAL;
76 }76 }
77 77 
@@ -82,13 +82,13 @@ HcclResult HcclVmSynDataWrite(FILE *fp, const HcclVmSynData &synData)
82{82{
83 auto ret = FileHeaderWrite(fp, synData.header);83 auto ret = FileHeaderWrite(fp, synData.header);
84 if (ret != HcclResult::HCCL_SUCCESS) {84 if (ret != HcclResult::HCCL_SUCCESS) {
85- HCCL_VM_ERROR("[HcclVmSynDataWrite] Write file header failed. ");85+ HCCL_VM_ERROR("Write file header failed. ");
86 return HcclResult::HCCL_E_INTERNAL;86 return HcclResult::HCCL_E_INTERNAL;
87 }87 }
88 88 
89 ret = ModelInfoWrite(fp, synData.model_info);89 ret = ModelInfoWrite(fp, synData.model_info);
90 if (ret != HcclResult::HCCL_SUCCESS) {90 if (ret != HcclResult::HCCL_SUCCESS) {
91- HCCL_VM_ERROR("[HcclVmSynDataWrite] Write file model info failed. ");91+ HCCL_VM_ERROR("Write file model info failed. ");
92 return HcclResult::HCCL_E_INTERNAL;92 return HcclResult::HCCL_E_INTERNAL;
93 }93 }
94 94 
@@ -97,7 +97,7 @@ HcclResult HcclVmSynDataWrite(FILE *fp, const HcclVmSynData &synData)
97 97 
98 ret = MemLayoutWrite(fp, synData.memory_info);98 ret = MemLayoutWrite(fp, synData.memory_info);
99 if (ret != HcclResult::HCCL_SUCCESS) {99 if (ret != HcclResult::HCCL_SUCCESS) {
100- HCCL_VM_ERROR("[HcclVmSynDataWrite] Write file memory layout info failed. ");100+ HCCL_VM_ERROR("Write file memory layout info failed. ");
101 return HcclResult::HCCL_E_INTERNAL;101 return HcclResult::HCCL_E_INTERNAL;
102 }102 }
103 103 
@@ -108,7 +108,7 @@ HcclResult HcclVmInstrDataRead(FILE *fp, HcclVmInstrData &instrData, uint32_t ma
108{108{
109 auto ret = FileHeaderRead(fp, instrData.header, magic);109 auto ret = FileHeaderRead(fp, instrData.header, magic);
110 if (ret != HcclResult::HCCL_SUCCESS) {110 if (ret != HcclResult::HCCL_SUCCESS) {
111- HCCL_VM_ERROR("[HcclVmInstrDataRead] Read file header failed. ");111+ HCCL_VM_ERROR("Read file header failed. ");
112 return HcclResult::HCCL_E_INTERNAL;112 return HcclResult::HCCL_E_INTERNAL;
113 }113 }
114 114 
@@ -116,7 +116,7 @@ HcclResult HcclVmInstrDataRead(FILE *fp, HcclVmInstrData &instrData, uint32_t ma
116 for (uint32_t i = 0; i < instrData.header.count; i++) {116 for (uint32_t i = 0; i < instrData.header.count; i++) {
117 ret = MicrocodeInstrRead(fp, instrData.instr_data[i]);117 ret = MicrocodeInstrRead(fp, instrData.instr_data[i]);
118 if (ret != HcclResult::HCCL_SUCCESS) {118 if (ret != HcclResult::HCCL_SUCCESS) {
119- HCCL_VM_ERROR("[HcclVmInstrDataRead] Read file microcode instruction info failed. times= {}", i);119+ HCCL_VM_ERROR("Read file microcode instruction info failed. times= {}", i);
120 return HcclResult::HCCL_E_INTERNAL;120 return HcclResult::HCCL_E_INTERNAL;
121 }121 }
122 }122 }
@@ -128,14 +128,14 @@ HcclResult HcclVmInstrDataWrite(FILE *fp, const HcclVmInstrData &instrData)
128{128{
129 auto ret = FileHeaderWrite(fp, instrData.header);129 auto ret = FileHeaderWrite(fp, instrData.header);
130 if (ret != HcclResult::HCCL_SUCCESS) {130 if (ret != HcclResult::HCCL_SUCCESS) {
131- HCCL_VM_ERROR("[HcclVmInstrDataWrite] Write file header failed. ");131+ HCCL_VM_ERROR("Write file header failed. ");
132 return HcclResult::HCCL_E_INTERNAL;132 return HcclResult::HCCL_E_INTERNAL;
133 }133 }
134 134 
135 for (uint32_t i = 0; i < instrData.header.count; i++) {135 for (uint32_t i = 0; i < instrData.header.count; i++) {
136 ret = MicrocodeInstrWrite(fp, instrData.instr_data[i]);136 ret = MicrocodeInstrWrite(fp, instrData.instr_data[i]);
137 if (ret != HcclResult::HCCL_SUCCESS) {137 if (ret != HcclResult::HCCL_SUCCESS) {
138- HCCL_VM_ERROR("[HcclVmInstrDataWrite] Write file microcode instruction info failed. times= {}", i);138+ HCCL_VM_ERROR("Write file microcode instruction info failed. times= {}", i);
139 return HcclResult::HCCL_E_INTERNAL;139 return HcclResult::HCCL_E_INTERNAL;
140 }140 }
141 }141 }
@@ -268,19 +268,19 @@ HcclResult ModelInfoWrite(FILE *fp, const ModelInfoInner &modelInfo)
268{268{
269 auto ret = ModelInfoCommWrite(fp, modelInfo.comm);269 auto ret = ModelInfoCommWrite(fp, modelInfo.comm);
270 if (ret != HcclResult::HCCL_SUCCESS) {270 if (ret != HcclResult::HCCL_SUCCESS) {
271- HCCL_VM_ERROR("[ModelInfoWrite] Write model info comm data failed. ");271+ HCCL_VM_ERROR("Write model info comm data failed. ");
272 return HcclResult::HCCL_E_INTERNAL;272 return HcclResult::HCCL_E_INTERNAL;
273 }273 }
274 274 
275 ret = VDataDesTagWrite(fp, modelInfo.vDataDes);275 ret = VDataDesTagWrite(fp, modelInfo.vDataDes);
276 if (ret != HcclResult::HCCL_SUCCESS) {276 if (ret != HcclResult::HCCL_SUCCESS) {
277- HCCL_VM_ERROR("[ModelInfoWrite] Write model info vdatades data failed. ");277+ HCCL_VM_ERROR("Write model info vdatades data failed. ");
278 return HcclResult::HCCL_E_INTERNAL;278 return HcclResult::HCCL_E_INTERNAL;
279 }279 }
280 280 
281 ret = All2AllDataDesTagWrite(fp, modelInfo.all2AllDataDes);281 ret = All2AllDataDesTagWrite(fp, modelInfo.all2AllDataDes);
282 if (ret != HcclResult::HCCL_SUCCESS) {282 if (ret != HcclResult::HCCL_SUCCESS) {
283- HCCL_VM_ERROR("[ModelInfoWrite] Write model info all2allDataDes data failed. ");283+ HCCL_VM_ERROR("Write model info all2allDataDes data failed. ");
284 return HcclResult::HCCL_E_INTERNAL;284 return HcclResult::HCCL_E_INTERNAL;
285 }285 }
286 286 
@@ -291,19 +291,19 @@ HcclResult ModelInfoRead(FILE *fp, ModelInfoInner &modelInfo)
291{291{
292 auto ret = ModelInfoCommRead(fp, modelInfo.comm);292 auto ret = ModelInfoCommRead(fp, modelInfo.comm);
293 if (ret != HcclResult::HCCL_SUCCESS) {293 if (ret != HcclResult::HCCL_SUCCESS) {
294- HCCL_VM_ERROR("[ModelInfoRead] Read model info comm data failed ");294+ HCCL_VM_ERROR("Read model info comm data failed ");
295 return HcclResult::HCCL_E_INTERNAL;295 return HcclResult::HCCL_E_INTERNAL;
296 }296 }
297 297 
298 ret = VDataDesTagRead(fp, modelInfo.vDataDes);298 ret = VDataDesTagRead(fp, modelInfo.vDataDes);
299 if (ret != HcclResult::HCCL_SUCCESS) {299 if (ret != HcclResult::HCCL_SUCCESS) {
300- HCCL_VM_ERROR("[ModelInfoRead] Read model info vdatades data failed. ");300+ HCCL_VM_ERROR("Read model info vdatades data failed. ");
301 return HcclResult::HCCL_E_INTERNAL;301 return HcclResult::HCCL_E_INTERNAL;
302 }302 }
303 303 
304 ret = All2AllDataDesTagRead(fp, modelInfo.all2AllDataDes);304 ret = All2AllDataDesTagRead(fp, modelInfo.all2AllDataDes);
305 if (ret != HcclResult::HCCL_SUCCESS) {305 if (ret != HcclResult::HCCL_SUCCESS) {
306- HCCL_VM_ERROR("[ModelInfoRead] Read model info all2allDataDes data failed.");306+ HCCL_VM_ERROR("Read model info all2allDataDes data failed.");
307 return HcclResult::HCCL_E_INTERNAL;307 return HcclResult::HCCL_E_INTERNAL;
308 }308 }
309 309 
@@ -441,7 +441,7 @@ HcclResult HcclVmTaskMetaDataWrite(FILE *fp, const HcclVmTaskMetaData &taskMeta)
441{441{
442 auto ret = FileHeaderWrite(fp, taskMeta.header);442 auto ret = FileHeaderWrite(fp, taskMeta.header);
443 if (ret != HcclResult::HCCL_SUCCESS) {443 if (ret != HcclResult::HCCL_SUCCESS) {
444- HCCL_VM_ERROR("[TaskMetaDataWrite] Read file header failed. ");444+ HCCL_VM_ERROR("Read file header failed. ");
445 return HcclResult::HCCL_E_INTERNAL;445 return HcclResult::HCCL_E_INTERNAL;
446 }446 }
447 447 
@@ -456,7 +456,7 @@ HcclResult HcclVmTaskMetaDataRead(FILE *fp, HcclVmTaskMetaData &taskMeta, uint32
456{456{
457 auto ret = FileHeaderRead(fp, taskMeta.header, magic);457 auto ret = FileHeaderRead(fp, taskMeta.header, magic);
458 if (ret != HcclResult::HCCL_SUCCESS) {458 if (ret != HcclResult::HCCL_SUCCESS) {
459- HCCL_VM_ERROR("[HcclVmTaskMetaDataRead] Read file header failed. ");459+ HCCL_VM_ERROR("Read file header failed. ");
460 return HcclResult::HCCL_E_INTERNAL;460 return HcclResult::HCCL_E_INTERNAL;
461 }461 }
462 462 
@@ -472,7 +472,7 @@ HcclResult HcclVmFlagDataWrite(FILE *fp, const HcclVmFlagData &flagData)
472{472{
473 auto ret = FileHeaderWrite(fp, flagData.header);473 auto ret = FileHeaderWrite(fp, flagData.header);
474 if (ret != HcclResult::HCCL_SUCCESS) {474 if (ret != HcclResult::HCCL_SUCCESS) {
475- HCCL_VM_ERROR("[HcclVmFlagDataWrite] Read file header failed. ");475+ HCCL_VM_ERROR("Read file header failed. ");
476 return HcclResult::HCCL_E_INTERNAL;476 return HcclResult::HCCL_E_INTERNAL;
477 }477 }
478 478 
@@ -487,7 +487,7 @@ HcclResult HcclVmFlagDataRead(FILE *fp, HcclVmFlagData &flagData, uint32_t magic
487{487{
488 auto ret = FileHeaderRead(fp, flagData.header, magic);488 auto ret = FileHeaderRead(fp, flagData.header, magic);
489 if (ret != HcclResult::HCCL_SUCCESS) {489 if (ret != HcclResult::HCCL_SUCCESS) {
490- HCCL_VM_ERROR("[HcclVmFlagDataRead] Read file header failed. ");490+ HCCL_VM_ERROR("Read file header failed. ");
491 return HcclResult::HCCL_E_INTERNAL;491 return HcclResult::HCCL_E_INTERNAL;
492 }492 }
493 493 
Mtest/hccl_vm/src/plugin/checker/src/utils/check_utils.h+122-11文件内容审核中,请稍后刷新重试
Mtest/hccl_vm/src/plugin/checker/src/utils/dump/dump_graph.cc+32-32
@@ -197,7 +197,7 @@ static Json MakeTaskStubFields(TaskStub *task, const std::map<TaskNode *, std::s
197 case TaskTypeStub::LOCAL_COPY: {197 case TaskTypeStub::LOCAL_COPY: {
198 TaskStubLocalCopy *localCopyTask = dynamic_cast<TaskStubLocalCopy *>(task);198 TaskStubLocalCopy *localCopyTask = dynamic_cast<TaskStubLocalCopy *>(task);
199 if (localCopyTask == nullptr) {199 if (localCopyTask == nullptr) {
200- HCCL_VM_WARN("[MakeTaskStubFields] failed to cast task [{}] to TaskStubLocalCopy.",200+ HCCL_VM_WARN("failed to cast task [{}] to TaskStubLocalCopy.",
201 taskType);201 taskType);
202 break;202 break;
203 }203 }
@@ -209,7 +209,7 @@ static Json MakeTaskStubFields(TaskStub *task, const std::map<TaskNode *, std::s
209 case TaskTypeStub::LOCAL_REDUCE: {209 case TaskTypeStub::LOCAL_REDUCE: {
210 TaskStubLocalReduce *localReduceTask = dynamic_cast<TaskStubLocalReduce *>(task);210 TaskStubLocalReduce *localReduceTask = dynamic_cast<TaskStubLocalReduce *>(task);
211 if (localReduceTask == nullptr) {211 if (localReduceTask == nullptr) {
212- HCCL_VM_WARN("[MakeTaskStubFields] failed to cast task [{}] to TaskStubLocalReduce.",212+ HCCL_VM_WARN("failed to cast task [{}] to TaskStubLocalReduce.",
213 taskType);213 taskType);
214 break;214 break;
215 }215 }
@@ -223,7 +223,7 @@ static Json MakeTaskStubFields(TaskStub *task, const std::map<TaskNode *, std::s
223 case TaskTypeStub::LOCAL_BATCH_REDUCE: {223 case TaskTypeStub::LOCAL_BATCH_REDUCE: {
224 TaskStubLocalBatchReduce *localBatchReduceTask = dynamic_cast<TaskStubLocalBatchReduce *>(task);224 TaskStubLocalBatchReduce *localBatchReduceTask = dynamic_cast<TaskStubLocalBatchReduce *>(task);
225 if (localBatchReduceTask == nullptr) {225 if (localBatchReduceTask == nullptr) {
226- HCCL_VM_WARN("[MakeTaskStubFields] failed to cast task [{}] to TaskStubLocalBatchReduce.",226+ HCCL_VM_WARN("failed to cast task [{}] to TaskStubLocalBatchReduce.",
227 taskType);227 taskType);
228 break;228 break;
229 }229 }
@@ -239,7 +239,7 @@ static Json MakeTaskStubFields(TaskStub *task, const std::map<TaskNode *, std::s
239 case TaskTypeStub::LOCAL_POST_TO: {239 case TaskTypeStub::LOCAL_POST_TO: {
240 TaskStubLocalPostTo *localPostToTask = dynamic_cast<TaskStubLocalPostTo *>(task);240 TaskStubLocalPostTo *localPostToTask = dynamic_cast<TaskStubLocalPostTo *>(task);
241 if (localPostToTask == nullptr) {241 if (localPostToTask == nullptr) {
242- HCCL_VM_WARN("[MakeTaskStubFields] failed to cast task [{}] to TaskStubLocalPostTo.",242+ HCCL_VM_WARN("failed to cast task [{}] to TaskStubLocalPostTo.",
243 taskType);243 taskType);
244 break;244 break;
245 }245 }
@@ -255,7 +255,7 @@ static Json MakeTaskStubFields(TaskStub *task, const std::map<TaskNode *, std::s
255 case TaskTypeStub::LOCAL_WAIT_FROM: {255 case TaskTypeStub::LOCAL_WAIT_FROM: {
256 TaskStubLocalWaitFrom *localWaitFromTask = dynamic_cast<TaskStubLocalWaitFrom *>(task);256 TaskStubLocalWaitFrom *localWaitFromTask = dynamic_cast<TaskStubLocalWaitFrom *>(task);
257 if (localWaitFromTask == nullptr) {257 if (localWaitFromTask == nullptr) {
258- HCCL_VM_WARN("[MakeTaskStubFields] failed to cast task [{}] to TaskStubLocalWaitFrom.",258+ HCCL_VM_WARN("failed to cast task [{}] to TaskStubLocalWaitFrom.",
259 taskType);259 taskType);
260 break;260 break;
261 }261 }
@@ -265,7 +265,7 @@ static Json MakeTaskStubFields(TaskStub *task, const std::map<TaskNode *, std::s
265 case TaskTypeStub::POST: {265 case TaskTypeStub::POST: {
266 TaskStubPost *postTask = dynamic_cast<TaskStubPost *>(task);266 TaskStubPost *postTask = dynamic_cast<TaskStubPost *>(task);
267 if (postTask == nullptr) {267 if (postTask == nullptr) {
268- HCCL_VM_WARN("[MakeTaskStubFields] failed to cast task [{}] to TaskStubPost.",268+ HCCL_VM_WARN("failed to cast task [{}] to TaskStubPost.",
269 taskType);269 taskType);
270 break;270 break;
271 }271 }
@@ -279,7 +279,7 @@ static Json MakeTaskStubFields(TaskStub *task, const std::map<TaskNode *, std::s
279 case TaskTypeStub::WAIT: {279 case TaskTypeStub::WAIT: {
280 TaskStubWait *waitTask = dynamic_cast<TaskStubWait *>(task);280 TaskStubWait *waitTask = dynamic_cast<TaskStubWait *>(task);
281 if (waitTask == nullptr) {281 if (waitTask == nullptr) {
282- HCCL_VM_WARN("[MakeTaskStubFields] failed to cast task [{}] to TaskStubWait.",282+ HCCL_VM_WARN("failed to cast task [{}] to TaskStubWait.",
283 taskType);283 taskType);
284 break;284 break;
285 }285 }
@@ -292,7 +292,7 @@ static Json MakeTaskStubFields(TaskStub *task, const std::map<TaskNode *, std::s
292 case TaskTypeStub::READ: {292 case TaskTypeStub::READ: {
293 TaskStubRead *readTask = dynamic_cast<TaskStubRead *>(task);293 TaskStubRead *readTask = dynamic_cast<TaskStubRead *>(task);
294 if (readTask == nullptr) {294 if (readTask == nullptr) {
295- HCCL_VM_WARN("[MakeTaskStubFields] failed to cast task [{}] to TaskStubRead.",295+ HCCL_VM_WARN("failed to cast task [{}] to TaskStubRead.",
296 taskType);296 taskType);
297 break;297 break;
298 }298 }
@@ -306,7 +306,7 @@ static Json MakeTaskStubFields(TaskStub *task, const std::map<TaskNode *, std::s
306 case TaskTypeStub::READ_REDUCE: {306 case TaskTypeStub::READ_REDUCE: {
307 TaskStubReadReduce *readReduceTask = dynamic_cast<TaskStubReadReduce *>(task);307 TaskStubReadReduce *readReduceTask = dynamic_cast<TaskStubReadReduce *>(task);
308 if (readReduceTask == nullptr) {308 if (readReduceTask == nullptr) {
309- HCCL_VM_WARN("[MakeTaskStubFields] failed to cast task [{}] to TaskStubReadReduce.",309+ HCCL_VM_WARN("failed to cast task [{}] to TaskStubReadReduce.",
310 taskType);310 taskType);
311 break;311 break;
312 }312 }
@@ -322,7 +322,7 @@ static Json MakeTaskStubFields(TaskStub *task, const std::map<TaskNode *, std::s
322 case TaskTypeStub::WRITE: {322 case TaskTypeStub::WRITE: {
323 TaskStubWrite *writeTask = dynamic_cast<TaskStubWrite *>(task);323 TaskStubWrite *writeTask = dynamic_cast<TaskStubWrite *>(task);
324 if (writeTask == nullptr) {324 if (writeTask == nullptr) {
325- HCCL_VM_WARN("[MakeTaskStubFields] failed to cast task [{}] to TaskStubWrite.",325+ HCCL_VM_WARN("failed to cast task [{}] to TaskStubWrite.",
326 taskType);326 taskType);
327 break;327 break;
328 }328 }
@@ -336,7 +336,7 @@ static Json MakeTaskStubFields(TaskStub *task, const std::map<TaskNode *, std::s
336 case TaskTypeStub::WRITE_REDUCE: {336 case TaskTypeStub::WRITE_REDUCE: {
337 TaskStubWriteReduce *writeReduceTask = dynamic_cast<TaskStubWriteReduce *>(task);337 TaskStubWriteReduce *writeReduceTask = dynamic_cast<TaskStubWriteReduce *>(task);
338 if (writeReduceTask == nullptr) {338 if (writeReduceTask == nullptr) {
339- HCCL_VM_WARN("[MakeTaskStubFields] failed to cast task [{}] to TaskStubWriteReduce.",339+ HCCL_VM_WARN("failed to cast task [{}] to TaskStubWriteReduce.",
340 taskType);340 taskType);
341 break;341 break;
342 }342 }
@@ -352,7 +352,7 @@ static Json MakeTaskStubFields(TaskStub *task, const std::map<TaskNode *, std::s
352 case TaskTypeStub::LOCAL_POST_TO_SHADOW: {352 case TaskTypeStub::LOCAL_POST_TO_SHADOW: {
353 TaskStubLocalPostToShadow *localPostToShadowTask = dynamic_cast<TaskStubLocalPostToShadow *>(task);353 TaskStubLocalPostToShadow *localPostToShadowTask = dynamic_cast<TaskStubLocalPostToShadow *>(task);
354 if (localPostToShadowTask == nullptr) {354 if (localPostToShadowTask == nullptr) {
355- HCCL_VM_WARN("[MakeTaskStubFields] failed to cast task [{}] to TaskStubLocalPostToShadow.",355+ HCCL_VM_WARN("failed to cast task [{}] to TaskStubLocalPostToShadow.",
356 taskType);356 taskType);
357 break;357 break;
358 }358 }
@@ -362,7 +362,7 @@ static Json MakeTaskStubFields(TaskStub *task, const std::map<TaskNode *, std::s
362 case TaskTypeStub::LOCAL_WAIT_FROM_SHADOW: {362 case TaskTypeStub::LOCAL_WAIT_FROM_SHADOW: {
363 TaskStubLocalWaitFromShadow *localWaitFromShadowTask = dynamic_cast<TaskStubLocalWaitFromShadow *>(task);363 TaskStubLocalWaitFromShadow *localWaitFromShadowTask = dynamic_cast<TaskStubLocalWaitFromShadow *>(task);
364 if (localWaitFromShadowTask == nullptr) {364 if (localWaitFromShadowTask == nullptr) {
365- HCCL_VM_WARN("[MakeTaskStubFields] failed to cast task [{}] to TaskStubLocalWaitFromShadow.",365+ HCCL_VM_WARN("failed to cast task [{}] to TaskStubLocalWaitFromShadow.",
366 taskType);366 taskType);
367 break;367 break;
368 }368 }
@@ -372,7 +372,7 @@ static Json MakeTaskStubFields(TaskStub *task, const std::map<TaskNode *, std::s
372 case TaskTypeStub::BEING_READ: {372 case TaskTypeStub::BEING_READ: {
373 TaskStubBeingRead *beingReadTask = dynamic_cast<TaskStubBeingRead *>(task);373 TaskStubBeingRead *beingReadTask = dynamic_cast<TaskStubBeingRead *>(task);
374 if (beingReadTask == nullptr) {374 if (beingReadTask == nullptr) {
375- HCCL_VM_WARN("[MakeTaskStubFields] failed to cast task [{}] to TaskStubBeingRead.",375+ HCCL_VM_WARN("failed to cast task [{}] to TaskStubBeingRead.",
376 taskType);376 taskType);
377 break;377 break;
378 }378 }
@@ -386,7 +386,7 @@ static Json MakeTaskStubFields(TaskStub *task, const std::map<TaskNode *, std::s
386 case TaskTypeStub::BEING_READ_REDUCE: {386 case TaskTypeStub::BEING_READ_REDUCE: {
387 TaskStubBeingReadReduce *beingReadReduceTask = dynamic_cast<TaskStubBeingReadReduce *>(task);387 TaskStubBeingReadReduce *beingReadReduceTask = dynamic_cast<TaskStubBeingReadReduce *>(task);
388 if (beingReadReduceTask == nullptr) {388 if (beingReadReduceTask == nullptr) {
389- HCCL_VM_WARN("[MakeTaskStubFields] failed to cast task [{}] to TaskStubBeingReadReduce.",389+ HCCL_VM_WARN("failed to cast task [{}] to TaskStubBeingReadReduce.",
390 taskType);390 taskType);
391 break;391 break;
392 }392 }
@@ -402,7 +402,7 @@ static Json MakeTaskStubFields(TaskStub *task, const std::map<TaskNode *, std::s
402 case TaskTypeStub::BEING_WRITTEN: {402 case TaskTypeStub::BEING_WRITTEN: {
403 TaskStubBeingWritten *beingWrittenTask = dynamic_cast<TaskStubBeingWritten *>(task);403 TaskStubBeingWritten *beingWrittenTask = dynamic_cast<TaskStubBeingWritten *>(task);
404 if (beingWrittenTask == nullptr) {404 if (beingWrittenTask == nullptr) {
405- HCCL_VM_WARN("[MakeTaskStubFields] failed to cast task [{}] to TaskStubBeingWritten.",405+ HCCL_VM_WARN("failed to cast task [{}] to TaskStubBeingWritten.",
406 taskType);406 taskType);
407 break;407 break;
408 }408 }
@@ -416,7 +416,7 @@ static Json MakeTaskStubFields(TaskStub *task, const std::map<TaskNode *, std::s
416 case TaskTypeStub::BEING_WRITTEN_REDUCE: {416 case TaskTypeStub::BEING_WRITTEN_REDUCE: {
417 TaskStubBeingWrittenReduce *beingWrittenReduceTask = dynamic_cast<TaskStubBeingWrittenReduce *>(task);417 TaskStubBeingWrittenReduce *beingWrittenReduceTask = dynamic_cast<TaskStubBeingWrittenReduce *>(task);
418 if (beingWrittenReduceTask == nullptr) {418 if (beingWrittenReduceTask == nullptr) {
419- HCCL_VM_WARN("[MakeTaskStubFields] failed to cast task [{}] to TaskStubBeingWrittenReduce.",419+ HCCL_VM_WARN("failed to cast task [{}] to TaskStubBeingWrittenReduce.",
420 taskType);420 taskType);
421 break;421 break;
422 }422 }
@@ -432,7 +432,7 @@ static Json MakeTaskStubFields(TaskStub *task, const std::map<TaskNode *, std::s
432 case TaskTypeStub::CCU_GRAPH: {432 case TaskTypeStub::CCU_GRAPH: {
433 TaskStubCcuGraph *ccuGraphTask = dynamic_cast<TaskStubCcuGraph *>(task);433 TaskStubCcuGraph *ccuGraphTask = dynamic_cast<TaskStubCcuGraph *>(task);
434 if (ccuGraphTask == nullptr) {434 if (ccuGraphTask == nullptr) {
435- HCCL_VM_WARN("[MakeTaskStubFields] failed to cast task [{}] to TaskStubCcuGraph.",435+ HCCL_VM_WARN("failed to cast task [{}] to TaskStubCcuGraph.",
436 taskType);436 taskType);
437 break;437 break;
438 }438 }
@@ -444,7 +444,7 @@ static Json MakeTaskStubFields(TaskStub *task, const std::map<TaskNode *, std::s
444 case TaskTypeStub::LOOP_START: {444 case TaskTypeStub::LOOP_START: {
445 TaskStubLoopStart *loopStartTask = dynamic_cast<TaskStubLoopStart *>(task);445 TaskStubLoopStart *loopStartTask = dynamic_cast<TaskStubLoopStart *>(task);
446 if (loopStartTask == nullptr) {446 if (loopStartTask == nullptr) {
447- HCCL_VM_WARN("[MakeTaskStubFields] failed to cast task [{}] to TaskStubLoopStart.",447+ HCCL_VM_WARN("failed to cast task [{}] to TaskStubLoopStart.",
448 taskType);448 taskType);
449 break;449 break;
450 }450 }
@@ -455,7 +455,7 @@ static Json MakeTaskStubFields(TaskStub *task, const std::map<TaskNode *, std::s
455 case TaskTypeStub::LOOP_END: {455 case TaskTypeStub::LOOP_END: {
456 TaskStubLoopEnd *loopEndTask = dynamic_cast<TaskStubLoopEnd *>(task);456 TaskStubLoopEnd *loopEndTask = dynamic_cast<TaskStubLoopEnd *>(task);
457 if (loopEndTask == nullptr) {457 if (loopEndTask == nullptr) {
458- HCCL_VM_WARN("[MakeTaskStubFields] failed to cast task [{}] to TaskStubLoopEnd.",458+ HCCL_VM_WARN("failed to cast task [{}] to TaskStubLoopEnd.",
459 taskType);459 taskType);
460 break;460 break;
461 }461 }
@@ -568,7 +568,7 @@ void CollectTaskNodesWithGlobalIds(TaskNode *dummyStart, std::vector<TaskNode *>
568 }568 }
569 std::string nodeId = node->task->GetTaskId();569 std::string nodeId = node->task->GetTaskId();
570 if (nodeId.empty()) {570 if (nodeId.empty()) {
571- HCCL_VM_ERROR("[CollectTaskNodesWithGlobalIds] task id not found, fallback to local node id.");571+ HCCL_VM_ERROR("task id not found, fallback to local node id.");
572 const uint64_t nodeIndex = rankNodeCounters[node->rankIdx]++;572 const uint64_t nodeIndex = rankNodeCounters[node->rankIdx]++;
573 nodeId = BuildNodeId(node->rankIdx, nodeIndex);573 nodeId = BuildNodeId(node->rankIdx, nodeIndex);
574 }574 }
@@ -726,7 +726,7 @@ static nlohmann::json DumpTaskNodeToJson(TaskNode *node, const std::string &node
726 }726 }
727 auto iter = nodeIdMap.find(parent);727 auto iter = nodeIdMap.find(parent);
728 if (iter == nodeIdMap.end()) {728 if (iter == nodeIdMap.end()) {
729- HCCL_VM_ERROR("[TaskNodeToJson] parent node id not found.");729+ HCCL_VM_ERROR("parent node id not found.");
730 continue;730 continue;
731 }731 }
732 nodeJson["parents"].push_back(iter->second);732 nodeJson["parents"].push_back(iter->second);
@@ -740,7 +740,7 @@ static nlohmann::json DumpTaskNodeToJson(TaskNode *node, const std::string &node
740 }740 }
741 auto iter = nodeIdMap.find(child);741 auto iter = nodeIdMap.find(child);
742 if (iter == nodeIdMap.end()) {742 if (iter == nodeIdMap.end()) {
743- HCCL_VM_ERROR("[TaskNodeToJson] child node id not found.");743+ HCCL_VM_ERROR("child node id not found.");
744 continue;744 continue;
745 }745 }
746 nodeJson["children"].push_back(iter->second);746 nodeJson["children"].push_back(iter->second);
@@ -793,7 +793,7 @@ static Json DumpCcuSubGraph(TaskStubCcuGraph *ccuGraphTask, const std::map<TaskN
793 continue;793 continue;
794 }794 }
795 if (node->queIdx >= queueBuckets.size()) {795 if (node->queIdx >= queueBuckets.size()) {
796- HCCL_VM_WARN("[DumpCcuSubGraph] queue id[%u] is out of range[%zu], skip node.",796+ HCCL_VM_WARN("queue id[{}] is out of range[{}], skip node.",
797 node->queIdx, queueBuckets.size());797 node->queIdx, queueBuckets.size());
798 continue;798 continue;
799 }799 }
@@ -805,7 +805,7 @@ static Json DumpCcuSubGraph(TaskStubCcuGraph *ccuGraphTask, const std::map<TaskN
805 for (auto *node : queueBuckets[queueIdx]) {805 for (auto *node : queueBuckets[queueIdx]) {
806 auto nodeIdIter = resolvedNodeIdMap->find(node);806 auto nodeIdIter = resolvedNodeIdMap->find(node);
807 if (nodeIdIter == resolvedNodeIdMap->end()) {807 if (nodeIdIter == resolvedNodeIdMap->end()) {
808- HCCL_VM_ERROR("[DumpCcuSubGraph] node id not found.");808+ HCCL_VM_ERROR("node id not found.");
809 continue;809 continue;
810 }810 }
811 queueJson.push_back(DumpTaskNodeToJson(node, nodeIdIter->second, *resolvedNodeIdMap,811 queueJson.push_back(DumpTaskNodeToJson(node, nodeIdIter->second, *resolvedNodeIdMap,
@@ -869,7 +869,7 @@ static std::map<RankId, Json> BuildTaskGraphJsonMap(TaskNode *dummyStart)
869 for (auto *currentNode : graphNodes) {869 for (auto *currentNode : graphNodes) {
870 auto currentNodeIdIter = nodeIdMap.find(currentNode);870 auto currentNodeIdIter = nodeIdMap.find(currentNode);
871 if (currentNodeIdIter == nodeIdMap.end()) {871 if (currentNodeIdIter == nodeIdMap.end()) {
872- HCCL_VM_ERROR("[DumpTaskGraph] current node id not found.");872+ HCCL_VM_ERROR("current node id not found.");
873 continue;873 continue;
874 }874 }
875 875 
@@ -889,7 +889,7 @@ static std::map<RankId, Json> BuildTaskGraphJsonMap(TaskNode *dummyStart)
889 }889 }
890 auto parentIdIter = nodeIdMap.find(parent);890 auto parentIdIter = nodeIdMap.find(parent);
891 if (parentIdIter == nodeIdMap.end()) {891 if (parentIdIter == nodeIdMap.end()) {
892- HCCL_VM_ERROR("[DumpTaskGraph] parent node id not found.");892+ HCCL_VM_ERROR("parent node id not found.");
893 continue;893 continue;
894 }894 }
895 895 
@@ -908,7 +908,7 @@ static std::map<RankId, Json> BuildTaskGraphJsonMap(TaskNode *dummyStart)
908 }908 }
909 auto childIdIter = nodeIdMap.find(child);909 auto childIdIter = nodeIdMap.find(child);
910 if (childIdIter == nodeIdMap.end()) {910 if (childIdIter == nodeIdMap.end()) {
911- HCCL_VM_ERROR("[DumpTaskGraph] child node id not found.");911+ HCCL_VM_ERROR("child node id not found.");
912 continue;912 continue;
913 }913 }
914 914 
@@ -926,7 +926,7 @@ static HcclResult DumpRankJsonMapByStage(const std::map<RankId, Json> &rankJsonM
926 const std::string &stage)926 const std::string &stage)
927{927{
928 if (stage.empty()) {928 if (stage.empty()) {
929- HCCL_VM_WARN("[DumpRankJsonMapByStage] stage is empty.");929+ HCCL_VM_WARN("stage is empty.");
930 return HcclResult::HCCL_E_PARA;930 return HcclResult::HCCL_E_PARA;
931 }931 }
932 932 
@@ -962,14 +962,14 @@ static HcclResult DumpRankJsonMapByStage(const std::map<RankId, Json> &rankJsonM
962 }962 }
963 963 
964 if (unlink(childPath.c_str()) != 0) {964 if (unlink(childPath.c_str()) != 0) {
965- HCCL_VM_WARN("[DumpRankJsonMapByStage] failed to remove file[%s], errno[%d].",965+ HCCL_VM_WARN("failed to remove file[{}], errno[{}].",
966 childPath.c_str(), errno);966 childPath.c_str(), errno);
967 }967 }
968 }968 }
969 969 
970 closedir(dir);970 closedir(dir);
971 if (rmdir(path.c_str()) != 0 && errno != ENOENT) {971 if (rmdir(path.c_str()) != 0 && errno != ENOENT) {
972- HCCL_VM_WARN("[DumpRankJsonMapByStage] failed to remove dir[%s], errno[%d].",972+ HCCL_VM_WARN("failed to remove dir[{}], errno[{}].",
973 path.c_str(), errno);973 path.c_str(), errno);
974 }974 }
975 };975 };
@@ -993,7 +993,7 @@ static HcclResult DumpRankJsonMapByStage(const std::map<RankId, Json> &rankJsonM
993 const std::string rankDumpPath = BuildStageRankDumpPath(stage, rankJsonPair.first);993 const std::string rankDumpPath = BuildStageRankDumpPath(stage, rankJsonPair.first);
994 const HcclResult writeRet = dumpManager.Write(rankDumpPath, rankDumpJson);994 const HcclResult writeRet = dumpManager.Write(rankDumpPath, rankDumpJson);
995 if (writeRet != HcclResult::HCCL_SUCCESS) {995 if (writeRet != HcclResult::HCCL_SUCCESS) {
996- HCCL_VM_WARN("[DumpRankJsonMapByStage] failed to dump rank[%u], stage[%s], ret[%u].",996+ HCCL_VM_WARN("failed to dump rank[{}], stage[{}], ret[{}].",
997 rankJsonPair.first, stage.c_str(), static_cast<u32>(writeRet));997 rankJsonPair.first, stage.c_str(), static_cast<u32>(writeRet));
998 return writeRet;998 return writeRet;
999 }999 }
Mtest/hccl_vm/src/plugin/checker/src/utils/dump/dump_manager.cc+14-14
@@ -25,21 +25,21 @@ HcclResult DumpManager::Initialize(const std::string& dataId)
25{25{
26 std::lock_guard<std::mutex> lock(m_mutex);26 std::lock_guard<std::mutex> lock(m_mutex);
27 if (dataId.empty()) {27 if (dataId.empty()) {
28- HCCL_VM_ERROR("[DumpManager::Initialize] dataId is empty.");28+ HCCL_VM_ERROR("dataId is empty.");
29 return HcclResult::HCCL_E_PARA;29 return HcclResult::HCCL_E_PARA;
30 }30 }
31 31 
32 m_dataId = dataId;32 m_dataId = dataId;
33 m_pluginRootDir = HcclSim::GetCurrentPath();33 m_pluginRootDir = HcclSim::GetCurrentPath();
34 if (m_pluginRootDir.empty()) {34 if (m_pluginRootDir.empty()) {
35- HCCL_VM_ERROR("[DumpManager::Initialize] failed to get current path.");35+ HCCL_VM_ERROR("failed to get current path.");
36 return HcclResult::HCCL_E_INTERNAL;36 return HcclResult::HCCL_E_INTERNAL;
37 }37 }
38 m_dataDir = HcclSim::JoinPath(m_pluginRootDir, DATA_DIR_NAME);38 m_dataDir = HcclSim::JoinPath(m_pluginRootDir, DATA_DIR_NAME);
39 m_dumpRootDir = HcclSim::JoinPath(HcclSim::JoinPath(m_dataDir, INSIGHT_DIR_NAME), m_dataId);39 m_dumpRootDir = HcclSim::JoinPath(HcclSim::JoinPath(m_dataDir, INSIGHT_DIR_NAME), m_dataId);
40 40 
41 if (!SettingManager::GetInstance().IsInsightDumpEnabled()) {41 if (!SettingManager::GetInstance().IsInsightDumpEnabled()) {
42- HCCL_VM_INFO("[DumpManager] insight dump disabled by manifest setting.");42+ HCCL_VM_INFO("insight dump disabled by manifest setting.");
43 return HcclResult::HCCL_SUCCESS;43 return HcclResult::HCCL_SUCCESS;
44 }44 }
45 return PrepareDumpDirs();45 return PrepareDumpDirs();
@@ -141,7 +141,7 @@ HcclResult DumpManager::WriteMsgpackFile(const std::string& dumpRootDir, const s
141 141 
142 std::ofstream out(fullPath.c_str(), std::ios::out | std::ios::trunc | std::ios::binary);142 std::ofstream out(fullPath.c_str(), std::ios::out | std::ios::trunc | std::ios::binary);
143 if (!out.is_open()) {143 if (!out.is_open()) {
144- HCCL_VM_ERROR("[DumpManager::WriteMsgpackFile] failed to open file: {}", fullPath);144+ HCCL_VM_ERROR("failed to open file: {}", fullPath);
145 return HcclResult::HCCL_E_INTERNAL;145 return HcclResult::HCCL_E_INTERNAL;
146 }146 }
147 147 
@@ -149,7 +149,7 @@ HcclResult DumpManager::WriteMsgpackFile(const std::string& dumpRootDir, const s
149 try {149 try {
150 nlohmann::json::to_msgpack(document, nlohmann::detail::output_adapter<char>(out));150 nlohmann::json::to_msgpack(document, nlohmann::detail::output_adapter<char>(out));
151 } catch (const std::exception &ex) {151 } catch (const std::exception &ex) {
152- HCCL_VM_ERROR("[DumpManager::WriteMsgpackFile] stream msgpack serialize failed: {}, file: {}", ex.what(),152+ HCCL_VM_ERROR("stream msgpack serialize failed: {}, file: {}", ex.what(),
153 fullPath);153 fullPath);
154 return HcclResult::HCCL_E_INTERNAL;154 return HcclResult::HCCL_E_INTERNAL;
155 }155 }
@@ -158,7 +158,7 @@ HcclResult DumpManager::WriteMsgpackFile(const std::string& dumpRootDir, const s
158 const uint64_t writeBeginMs = NowMs();158 const uint64_t writeBeginMs = NowMs();
159 out.flush();159 out.flush();
160 if (!out.good()) {160 if (!out.good()) {
161- HCCL_VM_ERROR("[DumpManager::WriteMsgpackFile] failed to flush file: {}", fullPath);161+ HCCL_VM_ERROR("failed to flush file: {}", fullPath);
162 return HcclResult::HCCL_E_INTERNAL;162 return HcclResult::HCCL_E_INTERNAL;
163 }163 }
164 const uint64_t writeCostMs = NowMs() - writeBeginMs;164 const uint64_t writeCostMs = NowMs() - writeBeginMs;
@@ -167,7 +167,7 @@ HcclResult DumpManager::WriteMsgpackFile(const std::string& dumpRootDir, const s
167 static_cast<uint64_t>(finalPos) : 0;167 static_cast<uint64_t>(finalPos) : 0;
168 168 
169 if (relativePath.compare(0, std::string("memory/").size(), "memory/") == 0) {169 if (relativePath.compare(0, std::string("memory/").size(), "memory/") == 0) {
170- HCCL_VM_INFO("[DumpManager::WriteMsgpackFile][perf] path={}, ensure_dir_ms={}, serialize_ms={}, "170+ HCCL_VM_INFO("path={}, ensure_dir_ms={}, serialize_ms={}, "
171 "write_file_ms={}, msgpack_bytes={}", relativePath, ensureDirCostMs, serializeCostMs, writeCostMs,171 "write_file_ms={}, msgpack_bytes={}", relativePath, ensureDirCostMs, serializeCostMs, writeCostMs,
172 msgpackBytes);172 msgpackBytes);
173 }173 }
@@ -185,19 +185,19 @@ HcclResult DumpManager::WriteJsonFile(const std::string& dumpRootDir, const std:
185 185 
186 std::ofstream out(fullPath.c_str(), std::ios::out | std::ios::trunc);186 std::ofstream out(fullPath.c_str(), std::ios::out | std::ios::trunc);
187 if (!out.is_open()) {187 if (!out.is_open()) {
188- HCCL_VM_ERROR("[DumpManager::WriteJsonFile] failed to open file: {}", fullPath);188+ HCCL_VM_ERROR("failed to open file: {}", fullPath);
189 return HcclResult::HCCL_E_INTERNAL;189 return HcclResult::HCCL_E_INTERNAL;
190 }190 }
191 191 
192 try {192 try {
193 out << document.dump(2) << std::endl;193 out << document.dump(2) << std::endl;
194 } catch (const std::exception &ex) {194 } catch (const std::exception &ex) {
195- HCCL_VM_ERROR("[DumpManager::WriteJsonFile] json serialize failed: {}, file: {}", ex.what(), fullPath);195+ HCCL_VM_ERROR("json serialize failed: {}, file: {}", ex.what(), fullPath);
196 return HcclResult::HCCL_E_INTERNAL;196 return HcclResult::HCCL_E_INTERNAL;
197 }197 }
198 198 
199 if (!out.good()) {199 if (!out.good()) {
200- HCCL_VM_ERROR("[DumpManager::WriteJsonFile] failed to write file: {}", fullPath);200+ HCCL_VM_ERROR("failed to write file: {}", fullPath);
201 return HcclResult::HCCL_E_INTERNAL;201 return HcclResult::HCCL_E_INTERNAL;
202 }202 }
203 return HcclResult::HCCL_SUCCESS;203 return HcclResult::HCCL_SUCCESS;
@@ -221,7 +221,7 @@ HcclResult DumpManager::WriteMsgpackStreamFile(const std::string& dumpRootDir, c
221 221 
222 std::ofstream out(fullPath.c_str(), std::ios::out | std::ios::trunc | std::ios::binary);222 std::ofstream out(fullPath.c_str(), std::ios::out | std::ios::trunc | std::ios::binary);
223 if (!out.is_open()) {223 if (!out.is_open()) {
224- HCCL_VM_ERROR("[DumpManager::WriteMsgpackStreamFile] failed to open file: {}", fullPath);224+ HCCL_VM_ERROR("failed to open file: {}", fullPath);
225 return HcclResult::HCCL_E_INTERNAL;225 return HcclResult::HCCL_E_INTERNAL;
226 }226 }
227 227 
@@ -229,14 +229,14 @@ HcclResult DumpManager::WriteMsgpackStreamFile(const std::string& dumpRootDir, c
229 ret = writer(out);229 ret = writer(out);
230 const uint64_t serializeCostMs = NowMs() - serializeBeginMs;230 const uint64_t serializeCostMs = NowMs() - serializeBeginMs;
231 if (ret != HcclResult::HCCL_SUCCESS) {231 if (ret != HcclResult::HCCL_SUCCESS) {
232- HCCL_VM_ERROR("[DumpManager::WriteMsgpackStreamFile] custom writer failed. path: {}", fullPath);232+ HCCL_VM_ERROR("custom writer failed. path: {}", fullPath);
233 return ret;233 return ret;
234 }234 }
235 235 
236 const uint64_t writeBeginMs = NowMs();236 const uint64_t writeBeginMs = NowMs();
237 out.flush();237 out.flush();
238 if (!out.good()) {238 if (!out.good()) {
239- HCCL_VM_ERROR("[DumpManager::WriteMsgpackStreamFile] failed to flush file: {}", fullPath);239+ HCCL_VM_ERROR("failed to flush file: {}", fullPath);
240 return HcclResult::HCCL_E_INTERNAL;240 return HcclResult::HCCL_E_INTERNAL;
241 }241 }
242 const uint64_t writeCostMs = NowMs() - writeBeginMs;242 const uint64_t writeCostMs = NowMs() - writeBeginMs;
@@ -245,7 +245,7 @@ HcclResult DumpManager::WriteMsgpackStreamFile(const std::string& dumpRootDir, c
245 static_cast<uint64_t>(finalPos) : 0;245 static_cast<uint64_t>(finalPos) : 0;
246 246 
247 if (relativePath.compare(0, std::string("memory/").size(), "memory/") == 0) {247 if (relativePath.compare(0, std::string("memory/").size(), "memory/") == 0) {
248- HCCL_VM_INFO("[DumpManager::WriteMsgpackStreamFile][perf] path={}, ensure_dir_ms={}, serialize_ms={}, "248+ HCCL_VM_INFO("path={}, ensure_dir_ms={}, serialize_ms={}, "
249 "write_file_ms={}, msgpack_bytes={}", relativePath, ensureDirCostMs, serializeCostMs, writeCostMs,249 "write_file_ms={}, msgpack_bytes={}", relativePath, ensureDirCostMs, serializeCostMs, writeCostMs,
250 msgpackBytes);250 msgpackBytes);
251 }251 }
Mtest/hccl_vm/src/plugin/checker/src/utils/dump/dump_memory_timeline.cc+4-4
@@ -285,7 +285,7 @@ HcclResult MemoryTimelineDumpStream::Initialize(const std::vector<RankId> &rankI
285 285 
286 const EventIdToEventMap eventIdToEvent = BuildEventIdToEventMap(timelineEvents);286 const EventIdToEventMap eventIdToEvent = BuildEventIdToEventMap(timelineEvents);
287 m_globalStepToEvent = BuildGlobalStepToEventMap(globalStepToEventId, eventIdToEvent);287 m_globalStepToEvent = BuildGlobalStepToEventMap(globalStepToEventId, eventIdToEvent);
288- HCCL_VM_INFO("[DumpMemoryTimelines] start dump memory timeline, rank_count={}, event_count={}, step_event_count={}",288+ HCCL_VM_INFO("start dump memory timeline, rank_count={}, event_count={}, step_event_count={}",
289 rankIds.size(), eventIdToEvent.size(), m_globalStepToEvent.size());289 rankIds.size(), eventIdToEvent.size(), m_globalStepToEvent.size());
290 290 
291 for (RankId rankId : rankIds) {291 for (RankId rankId : rankIds) {
@@ -312,7 +312,7 @@ HcclResult MemoryTimelineDumpStream::FlushOpenChunk(RankId rankId, RankDumpState
312 rankDumpState.openChunkSnapshotEntries);312 rankDumpState.openChunkSnapshotEntries);
313 });313 });
314 if (ret != HcclResult::HCCL_SUCCESS) {314 if (ret != HcclResult::HCCL_SUCCESS) {
315- HCCL_VM_WARN("[DumpMemoryTimelines] failed to dump rank [%u] snapshot chunk [%zu].", rankId, chunkId);315+ HCCL_VM_WARN("failed to dump rank [{}] snapshot chunk [{}].", rankId, chunkId);
316 return ret;316 return ret;
317 }317 }
318 318 
@@ -397,7 +397,7 @@ HcclResult MemoryTimelineDumpStream::Finalize()
397 const std::string snapshotManifestPath = BuildRankSnapshotManifestPath(rankId);397 const std::string snapshotManifestPath = BuildRankSnapshotManifestPath(rankId);
398 ret = DumpManager::GetInstance().Write(snapshotManifestPath, snapshotManifestJson);398 ret = DumpManager::GetInstance().Write(snapshotManifestPath, snapshotManifestJson);
399 if (ret != HcclResult::HCCL_SUCCESS) {399 if (ret != HcclResult::HCCL_SUCCESS) {
400- HCCL_VM_WARN("[DumpMemoryTimelines] failed to dump rank [%u] snapshot manifest.", rankId);400+ HCCL_VM_WARN("failed to dump rank [{}] snapshot manifest.", rankId);
401 return ret;401 return ret;
402 }402 }
403 403 
@@ -413,7 +413,7 @@ HcclResult MemoryTimelineDumpStream::Finalize()
413 memoryStatsJson["snapshot_count"] = totalSnapshotCount;413 memoryStatsJson["snapshot_count"] = totalSnapshotCount;
414 memoryStatsJson["chunk_count"] = totalChunkCount;414 memoryStatsJson["chunk_count"] = totalChunkCount;
415 DumpRunManifest::GetInstance().SetMemorySnapshotStats(memoryStatsJson);415 DumpRunManifest::GetInstance().SetMemorySnapshotStats(memoryStatsJson);
416- HCCL_VM_INFO("[DumpMemoryTimelines] finish dump memory timeline, rank_count={}", m_rankDumpStates.size());416+ HCCL_VM_INFO("finish dump memory timeline, rank_count={}", m_rankDumpStates.size());
417 return HcclResult::HCCL_SUCCESS;417 return HcclResult::HCCL_SUCCESS;
418}418}
419} // namespace HcclSim419} // namespace HcclSim
Mtest/hccl_vm/src/plugin/checker/src/utils/dump_v3/dump_v3_dag.cc+14-6
@@ -25,6 +25,7 @@
25#include "framework/task_graph_generator_v3/task_graph_reachability_v3.h"25#include "framework/task_graph_generator_v3/task_graph_reachability_v3.h"
26#include "sim_log.h"26#include "sim_log.h"
27#include "storage_manager.h"27#include "storage_manager.h"
28+#include "utils/error_codes.h"
28 29 
29namespace HcclSim {30namespace HcclSim {
30namespace {31namespace {
@@ -521,7 +522,9 @@ HcclResult CollectTopoNodes(const TaskGraphGeneratorV3::TaskGraphGeneratorV3 &gr
521 topoNodes.clear();522 topoNodes.clear();
522 const TaskNode *mainStart = graph.GetMainStartNode();523 const TaskNode *mainStart = graph.GetMainStartNode();
523 if (!IsMainGraphStartNode(mainStart)) {524 if (!IsMainGraphStartNode(mainStart)) {
524- HCCL_VM_ERROR("[DumpV3Dag] invalid V3 main start node.");525+ HCCL_VM_ERROR("{} Main start node of the V3 graph is invalid, mainStartNodeId={}",
526+ MakeErrorCodeText(ErrorCode::DUMP_FAILED),
527+ mainStart == nullptr ? MAIN_START_NODE_ID : mainStart->GetNodeId());
525 return HCCL_E_PARA;528 return HCCL_E_PARA;
526 }529 }
527 530 
@@ -529,7 +532,9 @@ HcclResult CollectTopoNodes(const TaskGraphGeneratorV3::TaskGraphGeneratorV3 &gr
529 std::vector<NodeId> topoOrder;532 std::vector<NodeId> topoOrder;
530 HcclResult ret = GenReachabilityClosure(mainStart, closure, &topoOrder);533 HcclResult ret = GenReachabilityClosure(mainStart, closure, &topoOrder);
531 if (ret != HCCL_SUCCESS) {534 if (ret != HCCL_SUCCESS) {
532- HCCL_VM_ERROR("[DumpV3Dag] failed to build topo order, ret={}.", static_cast<uint32_t>(ret));535+ HCCL_VM_ERROR("{} Failed to build the V3 graph topological order for dump output, "
536+ "mainStartNodeId={}, ret={}", MakeErrorCodeText(ErrorCode::DUMP_FAILED),
537+ mainStart->GetNodeId(), static_cast<uint32_t>(ret));
533 return ret;538 return ret;
534 }539 }
535 540 
@@ -540,7 +545,8 @@ HcclResult CollectTopoNodes(const TaskGraphGeneratorV3::TaskGraphGeneratorV3 &gr
540 }545 }
541 const TaskNode *node = graph.GetNode(nodeId);546 const TaskNode *node = graph.GetNode(nodeId);
542 if (node == nullptr) {547 if (node == nullptr) {
543- HCCL_VM_ERROR("[DumpV3Dag] topo node is null, nodeId={}.", nodeId);548+ HCCL_VM_ERROR("{} One node in topological order is missing, nodeId={}, "
549+ "processedTopoNodeCount={}", MakeErrorCodeText(ErrorCode::DUMP_FAILED), nodeId, topoNodes.size());
544 return HCCL_E_PTR;550 return HCCL_E_PTR;
545 }551 }
546 topoNodes.push_back(node);552 topoNodes.push_back(node);
@@ -616,7 +622,9 @@ HcclResult DumpV3Dag(const TaskGraphGeneratorV3::TaskGraphGeneratorV3 &graph, co
616 return HCCL_SUCCESS;622 return HCCL_SUCCESS;
617 }623 }
618 if (stage.empty()) {624 if (stage.empty()) {
619- HCCL_VM_ERROR("[DumpV3Dag] stage is empty.");625+ HCCL_VM_ERROR("{} Dump stage name is empty, dataId={}, graphOutputPath={}, layoutOutputPath={}",
626+ MakeErrorCodeText(ErrorCode::DUMP_FAILED), StorageManager::GetInstance().GetDataId(), kGraphPath,
627+ kLayoutPath);
620 return HCCL_E_PARA;628 return HCCL_E_PARA;
621 }629 }
622 630 
@@ -847,8 +855,8 @@ HcclResult DumpV3Dag(const TaskGraphGeneratorV3::TaskGraphGeneratorV3 &graph, co
847 MakeStageStats(graphDoc["lanes"].get<std::vector<Json>>(),855 MakeStageStats(graphDoc["lanes"].get<std::vector<Json>>(),
848 graphDoc["nodes"].get<std::vector<Json>>(), graphDoc["edges"].get<std::vector<Json>>()));856 graphDoc["nodes"].get<std::vector<Json>>(), graphDoc["edges"].get<std::vector<Json>>()));
849 857 
850- HCCL_VM_INFO("[DumpV3Dag] dumped V3 DAG, stage={}, laneCount={}, nodeCount={}, edgeCount={}",858+ HCCL_VM_INFO("V3 DAG dump finished, laneCount={}, nodeCount={}, edgeCount={}",
851- stage, laneIdByKey.size(), topoNodes.size(), edgeInfos.size());859+ laneIdByKey.size(), topoNodes.size(), edgeInfos.size());
852 return HCCL_SUCCESS;860 return HCCL_SUCCESS;
853}861}
854} // namespace HcclSim862} // namespace HcclSim
Mtest/hccl_vm/src/plugin/checker/src/utils/dump_v3/dump_v3_manager.cc+18-10
@@ -15,6 +15,7 @@
15#include "file_utils.h"15#include "file_utils.h"
16#include "sim_log.h"16#include "sim_log.h"
17#include "setting_manager.h"17#include "setting_manager.h"
18+#include "utils/error_codes.h"
18 19 
19namespace {20namespace {
20const std::string kDataDirName = "data";21const std::string kDataDirName = "data";
@@ -26,14 +27,16 @@ HcclResult DumpV3Manager::Initialize(const std::string &dataId)
26{27{
27 std::lock_guard<std::mutex> lock(m_mutex);28 std::lock_guard<std::mutex> lock(m_mutex);
28 if (dataId.empty()) {29 if (dataId.empty()) {
29- HCCL_VM_ERROR("[DumpV3Manager::Initialize] dataId is empty.");30+ HCCL_VM_ERROR("{} dataId is empty, the dump output path cannot be built, outputRoot=null",
31+ MakeErrorCodeText(ErrorCode::DUMP_FAILED));
30 return HcclResult::HCCL_E_PARA;32 return HcclResult::HCCL_E_PARA;
31 }33 }
32 34 
33 m_dataId = dataId;35 m_dataId = dataId;
34 m_pluginRootDir = HcclSim::GetCurrentPath();36 m_pluginRootDir = HcclSim::GetCurrentPath();
35 if (m_pluginRootDir.empty()) {37 if (m_pluginRootDir.empty()) {
36- HCCL_VM_ERROR("[DumpV3Manager::Initialize] failed to get current path.");38+ HCCL_VM_ERROR("{} Failed to get the current working directory for dump output, dataId={}",
39+ MakeErrorCodeText(ErrorCode::DUMP_FAILED), m_dataId);
37 return HcclResult::HCCL_E_INTERNAL;40 return HcclResult::HCCL_E_INTERNAL;
38 }41 }
39 42 
@@ -41,7 +44,7 @@ HcclResult DumpV3Manager::Initialize(const std::string &dataId)
41 m_rootDir = HcclSim::JoinPath(HcclSim::JoinPath(m_dataDir, kInsightDirName), m_dataId);44 m_rootDir = HcclSim::JoinPath(HcclSim::JoinPath(m_dataDir, kInsightDirName), m_dataId);
42 45 
43 if (!SettingManager::GetInstance().IsInsightDumpEnabled()) {46 if (!SettingManager::GetInstance().IsInsightDumpEnabled()) {
44- HCCL_VM_INFO("[DumpV3Manager] insight dump disabled by manifest setting.");47+ HCCL_VM_INFO("Insight dump is disabled by manifest setting");
45 return HcclResult::HCCL_SUCCESS;48 return HcclResult::HCCL_SUCCESS;
46 }49 }
47 return PrepareDirs();50 return PrepareDirs();
@@ -86,21 +89,23 @@ HcclResult DumpV3Manager::WriteMsgpack(const std::string &relativePath, const nl
86 89 
87 std::ofstream out(fullPath, std::ios::out | std::ios::trunc | std::ios::binary);90 std::ofstream out(fullPath, std::ios::out | std::ios::trunc | std::ios::binary);
88 if (!out.is_open()) {91 if (!out.is_open()) {
89- HCCL_VM_ERROR("[DumpV3Manager::WriteMsgpack] failed to open file: {}", fullPath);92+ HCCL_VM_ERROR("{} Failed to open dump file, file={}, format=msgpack, dataId={}",
93+ MakeErrorCodeText(ErrorCode::DUMP_FAILED), fullPath, m_dataId);
90 return HcclResult::HCCL_E_INTERNAL;94 return HcclResult::HCCL_E_INTERNAL;
91 }95 }
92 96 
93 try {97 try {
94 nlohmann::json::to_msgpack(document, nlohmann::detail::output_adapter<char>(out));98 nlohmann::json::to_msgpack(document, nlohmann::detail::output_adapter<char>(out));
95 } catch (const std::exception &ex) {99 } catch (const std::exception &ex) {
96- HCCL_VM_ERROR("[DumpV3Manager::WriteMsgpack] msgpack serialize failed: {}, file: {}", ex.what(),100+ HCCL_VM_ERROR("{} Failed to serialize dump content into msgpack, file={}, dataId={}, reason={}",
97- fullPath);101+ MakeErrorCodeText(ErrorCode::DUMP_FAILED), fullPath, m_dataId, ex.what());
98 return HcclResult::HCCL_E_INTERNAL;102 return HcclResult::HCCL_E_INTERNAL;
99 }103 }
100 104 
101 out.flush();105 out.flush();
102 if (!out.good()) {106 if (!out.good()) {
103- HCCL_VM_ERROR("[DumpV3Manager::WriteMsgpack] failed to flush file: {}", fullPath);107+ HCCL_VM_ERROR("{} Failed to flush dump file to disk, file={}, format=msgpack, dataId={}",
108+ MakeErrorCodeText(ErrorCode::DUMP_FAILED), fullPath, m_dataId);
104 return HcclResult::HCCL_E_INTERNAL;109 return HcclResult::HCCL_E_INTERNAL;
105 }110 }
106 return HcclResult::HCCL_SUCCESS;111 return HcclResult::HCCL_SUCCESS;
@@ -119,19 +124,22 @@ HcclResult DumpV3Manager::WriteJson(const std::string &relativePath, const nlohm
119 124 
120 std::ofstream out(fullPath, std::ios::out | std::ios::trunc);125 std::ofstream out(fullPath, std::ios::out | std::ios::trunc);
121 if (!out.is_open()) {126 if (!out.is_open()) {
122- HCCL_VM_ERROR("[DumpV3Manager::WriteJson] failed to open file: {}", fullPath);127+ HCCL_VM_ERROR("{} Failed to open dump file, file={}, format=json, dataId={}",
128+ MakeErrorCodeText(ErrorCode::DUMP_FAILED), fullPath, m_dataId);
123 return HcclResult::HCCL_E_INTERNAL;129 return HcclResult::HCCL_E_INTERNAL;
124 }130 }
125 131 
126 try {132 try {
127 out << document.dump(2) << std::endl;133 out << document.dump(2) << std::endl;
128 } catch (const std::exception &ex) {134 } catch (const std::exception &ex) {
129- HCCL_VM_ERROR("[DumpV3Manager::WriteJson] json serialize failed: {}, file: {}", ex.what(), fullPath);135+ HCCL_VM_ERROR("{} Failed to serialize dump content into JSON, file={}, dataId={}, reason={}",
136+ MakeErrorCodeText(ErrorCode::DUMP_FAILED), fullPath, m_dataId, ex.what());
130 return HcclResult::HCCL_E_INTERNAL;137 return HcclResult::HCCL_E_INTERNAL;
131 }138 }
132 139 
133 if (!out.good()) {140 if (!out.good()) {
134- HCCL_VM_ERROR("[DumpV3Manager::WriteJson] failed to write file: {}", fullPath);141+ HCCL_VM_ERROR("{} Failed to write dump file content, file={}, format=json, dataId={}",
142+ MakeErrorCodeText(ErrorCode::DUMP_FAILED), fullPath, m_dataId);
135 return HcclResult::HCCL_E_INTERNAL;143 return HcclResult::HCCL_E_INTERNAL;
136 }144 }
137 return HcclResult::HCCL_SUCCESS;145 return HcclResult::HCCL_SUCCESS;
Atest/hccl_vm/src/plugin/checker/src/utils/error_codes.h+69-0
@@ -0,0 +1,69 @@
1+/**
2+ * Copyright (c) 2026 Huawei Technologies Co., Ltd.
3+ * This program is free software, you can redistribute it and/or modify it under the terms and conditions of
4+ * CANN Open Software License Agreement Version 2.0 (the "License").
5+ * Please refer to the License for details. You may not use this file except in compliance with the License.
6+ * THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED,
7+ * INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE.
8+ * See LICENSE in the root of the software repository for the full text of the License.
9+ */
10+ 
11+#ifndef ERROR_CODES_H
12+#define ERROR_CODES_H
13+ 
14+#include <cstdint>
15+#include <string>
16+ 
17+namespace HcclSim {
18+ 
19+enum class ErrorCode {
20+ GRAPH_TRANSLATE_FAILED = 101,
21+ GRAPH_DEADLOCK = 102,
22+ GRAPH_UNMATCHED = 103,
23+ GRAPH_MEMBER_MISSING = 104,
24+ GRAPH_STRUCTURE_INVALID = 105,
25+ GRAPH_SNAPSHOT_MISMATCH = 106,
26+ GRAPH_RESOURCE_NOT_FOUND = 107,
27+ GRAPH_REGISTER_UNINITIALIZED = 108,
28+ GRAPH_OUT_OF_RANGE = 109,
29+ GRAPH_ADDRESS_INVALID = 110,
30+ GRAPH_UNSUPPORTED = 111,
31+ GRAPH_REMOTE_RANK_MISMATCH = 112,
32+ GRAPH_LOOP_MERGE_ERROR = 113,
33+ 
34+ SINGLETASK_SLICE_INVALID = 201,
35+ SINGLETASK_SLICE_CONFLICT = 202,
36+ SINGLETASK_SLAVE_STREAM_INVALID = 203,
37+ 
38+ MEMCONFLICT_DAG_INVALID = 301,
39+ MEMCONFLICT_DETECTED = 302,
40+ 
41+ SEMANTIC_BUFFER_EMPTY = 401,
42+ SEMANTIC_GAP = 402,
43+ SEMANTIC_REDUCE_ERROR = 403,
44+ SEMANTIC_SIMULATE_FAILED = 404,
45+ SEMANTIC_FINAL_CHECK_FAILED = 405,
46+ SEMANTIC_FINAL_MISSING = 406,
47+ SEMANTIC_FINAL_SRC_ERROR = 407,
48+ SEMANTIC_FINAL_SIZE_ERROR = 408,
49+ SEMANTIC_FINAL_REDUCE_ERROR = 409,
50+ 
51+ DUMP_FAILED = 501,
52+ 
53+ CHECKER_RUNTIME_ERROR = 901,
54+ SETTING_WARNING = 902
55+};
56+ 
57+constexpr uint32_t ToErrorCodeValue(ErrorCode code)
58+{
59+ return static_cast<uint32_t>(code);
60+}
61+ 
62+inline std::string MakeErrorCodeText(ErrorCode code)
63+{
64+ return "[ErrorCode: " + std::to_string(ToErrorCodeValue(code)) + "]";
65+}
66+ 
67+} // namespace HcclSim
68+ 
69+#endif
Mtest/hccl_vm/src/plugin/checker/src/utils/file_utils.cc+3-3
@@ -23,7 +23,7 @@ std::string GetCurrentPath()
23{23{
24 char absPath[PATH_MAX];24 char absPath[PATH_MAX];
25 if (realpath(".", absPath) == nullptr) {25 if (realpath(".", absPath) == nullptr) {
26- HCCL_VM_ERROR("[GetCurrentPath] realpath failed for current path.");26+ HCCL_VM_ERROR("realpath failed for current path.");
27 return "";27 return "";
28 }28 }
29 return std::string(absPath);29 return std::string(absPath);
@@ -45,7 +45,7 @@ std::string GetRootPath(const std::string& anchorPath, int maxDepth)
45 currentSearchPath = GetParentPath(currentSearchPath);45 currentSearchPath = GetParentPath(currentSearchPath);
46 }46 }
47 47 
48- HCCL_VM_INFO("[GetRootPath] RootPath NOT found.");48+ HCCL_VM_INFO("RootPath NOT found.");
49 return "";49 return "";
50}50}
51 51 
@@ -91,7 +91,7 @@ HcclResult EnsureDirectory(const std::string& path)
91 current = JoinPath(current, part);91 current = JoinPath(current, part);
92 if (!DirExists(current)) {92 if (!DirExists(current)) {
93 if (mkdir(current.c_str(), 0755) != 0 && errno != EEXIST) {93 if (mkdir(current.c_str(), 0755) != 0 && errno != EEXIST) {
94- HCCL_VM_ERROR("[EnsureDirectory] mkdir failed for {}: {}", current, strerror(errno));94+ HCCL_VM_ERROR("mkdir failed for {}: {}", current, strerror(errno));
95 return HcclResult::HCCL_E_INTERNAL;95 return HcclResult::HCCL_E_INTERNAL;
96 }96 }
97 }97 }
Mtest/hccl_vm/src/plugin/checker/src/utils/setting_manager.cc+5-5
@@ -33,7 +33,7 @@ HcclResult SettingManager::Refresh()
33{33{
34 const std::string pluginRootDir = GetCurrentPath();34 const std::string pluginRootDir = GetCurrentPath();
35 if (pluginRootDir.empty()) {35 if (pluginRootDir.empty()) {
36- HCCL_VM_ERROR("[SettingManager::Refresh] failed to get current path.");36+ HCCL_VM_ERROR("failed to get current path.");
37 return HcclResult::HCCL_E_INTERNAL;37 return HcclResult::HCCL_E_INTERNAL;
38 }38 }
39 39 
@@ -41,7 +41,7 @@ HcclResult SettingManager::Refresh()
41 CheckerSettings newSettings;41 CheckerSettings newSettings;
42 42 
43 if (!FileExists(manifestPath)) {43 if (!FileExists(manifestPath)) {
44- HCCL_VM_WARN("[SettingManager::Refresh] manifest.json not found, use default settings.");44+ HCCL_VM_WARN("manifest.json not found, use default settings.");
45 std::lock_guard<std::mutex> lock(m_mutex);45 std::lock_guard<std::mutex> lock(m_mutex);
46 m_settings = newSettings;46 m_settings = newSettings;
47 m_pluginRootDir = pluginRootDir;47 m_pluginRootDir = pluginRootDir;
@@ -51,7 +51,7 @@ HcclResult SettingManager::Refresh()
51 51 
52 std::ifstream manifestStream(manifestPath.c_str());52 std::ifstream manifestStream(manifestPath.c_str());
53 if (!manifestStream.is_open()) {53 if (!manifestStream.is_open()) {
54- HCCL_VM_ERROR("[SettingManager::Refresh] failed to open manifest file: {}", manifestPath);54+ HCCL_VM_ERROR("failed to open manifest file: {}", manifestPath);
55 return HcclResult::HCCL_E_INTERNAL;55 return HcclResult::HCCL_E_INTERNAL;
56 }56 }
57 57 
@@ -64,7 +64,7 @@ HcclResult SettingManager::Refresh()
64 newSettings.enableNewChecker = settings.value(SETTING_KEY_ENABLE_NEW_CHECKER, true);64 newSettings.enableNewChecker = settings.value(SETTING_KEY_ENABLE_NEW_CHECKER, true);
65 newSettings.enableOldChecker = settings.value(SETTING_KEY_ENABLE_OLD_CHECKER, true);65 newSettings.enableOldChecker = settings.value(SETTING_KEY_ENABLE_OLD_CHECKER, true);
66 } catch (const std::exception &ex) {66 } catch (const std::exception &ex) {
67- HCCL_VM_ERROR("[SettingManager::Refresh] parse manifest failed: {}", ex.what());67+ HCCL_VM_ERROR("parse manifest failed: {}", ex.what());
68 return HcclResult::HCCL_E_INTERNAL;68 return HcclResult::HCCL_E_INTERNAL;
69 }69 }
70 70 
@@ -75,7 +75,7 @@ HcclResult SettingManager::Refresh()
75 m_manifestPath = manifestPath;75 m_manifestPath = manifestPath;
76 }76 }
77 77 
78- HCCL_VM_INFO("[SettingManager::Refresh] settings refreshed: insight_dump={}, memory_snapshot_dump={}, "78+ HCCL_VM_INFO("settings refreshed: insight_dump={}, memory_snapshot_dump={}, "
79 "new_checker={}, old_checker={}", newSettings.enableInsightDump, newSettings.enableMemorySnapshotDump,79 "new_checker={}, old_checker={}", newSettings.enableInsightDump, newSettings.enableMemorySnapshotDump,
80 newSettings.enableNewChecker, newSettings.enableOldChecker);80 newSettings.enableNewChecker, newSettings.enableOldChecker);
81 return HcclResult::HCCL_SUCCESS;81 return HcclResult::HCCL_SUCCESS;
Mtest/hccl_vm/src/plugin/checker/src/utils/storage_manager.cc+14-14
@@ -97,7 +97,7 @@ HcclResult StorageManager::Trans2CheckerParam(sim::OpDetailTab& detailTab, ::OpD
97 m_checker_param.all2AllDataDes.sendCountMatrix = currentMatrix;97 m_checker_param.all2AllDataDes.sendCountMatrix = currentMatrix;
98 }98 }
99 99 
100- HCCL_VM_INFO("[Trans2CheckerParam] opIter={}, rankId={}", detailTab.opIter, detailTab.rankId);100+ HCCL_VM_INFO("opIter={}, rankId={}", detailTab.opIter, detailTab.rankId);
101 return HcclResult::HCCL_SUCCESS;101 return HcclResult::HCCL_SUCCESS;
102}102}
103 103 
@@ -125,7 +125,7 @@ void StorageManager::MergeAll2AllVSendCountMatrix()
125 m_checker_param.all2AllDataDes.count = matrixSize;125 m_checker_param.all2AllDataDes.count = matrixSize;
126 m_all2AllvSendMatrices.clear();126 m_all2AllvSendMatrices.clear();
127 127 
128- HCCL_VM_INFO("[MergeAll2AllVSendCountMatrix] Merged {} ranks into {}x{} matrix", numRanks, rankSize, rankSize);128+ HCCL_VM_INFO("Merged {} ranks into {}x{} matrix", numRanks, rankSize, rankSize);
129}129}
130 130 
131HcclResult StorageManager::LoadHcclVmSynthesisData(uint32_t rankId, sim::OpMemInfoTab memInfo, std::vector<sim::CcuChannelTab>& channels)131HcclResult StorageManager::LoadHcclVmSynthesisData(uint32_t rankId, sim::OpMemInfoTab memInfo, std::vector<sim::CcuChannelTab>& channels)
@@ -188,7 +188,7 @@ HcclResult StorageManager::LoadHcclVmInstrData(std::vector<sim::CcuInstrResTab>&
188{188{
189 for (auto& instr : instrRes) {189 for (auto& instr : instrRes) {
190 if (instr.instrCount == 0 || instr.instrCount > 32 * 1024) {190 if (instr.instrCount == 0 || instr.instrCount > 32 * 1024) {
191- HCCL_VM_WARN("[LoadHcclVmInstrData] invalid instrCount={}, rankId={}, dieId={}",191+ HCCL_VM_WARN("invalid instrCount={}, rankId={}, dieId={}",
192 instr.instrCount, instr.rankId, static_cast<uint32_t>(instr.dieId));192 instr.instrCount, instr.rankId, static_cast<uint32_t>(instr.dieId));
193 continue;193 continue;
194 }194 }
@@ -207,7 +207,7 @@ HcclResult StorageManager::LoadHcclVmInstrData(std::vector<sim::CcuInstrResTab>&
207 207 
208 m_instrData.instr_data.push_back(std::move(instrInner));208 m_instrData.instr_data.push_back(std::move(instrInner));
209 209 
210- HCCL_VM_INFO("[LoadHcclVmInstrData] rankId={}, dieId={}, count={}",210+ HCCL_VM_INFO("rankId={}, dieId={}, count={}",
211 instr.rankId, static_cast<uint32_t>(instr.dieId), instr.instrCount);211 instr.rankId, static_cast<uint32_t>(instr.dieId), instr.instrCount);
212 }212 }
213 213 
@@ -220,7 +220,7 @@ HcclResult StorageManager::LoadHcclVmTaskMetaData(std::vector<std::vector<sim::O
220 for (const auto& rankTasks : allTasks) {220 for (const auto& rankTasks : allTasks) {
221 totalTasks += rankTasks.size();221 totalTasks += rankTasks.size();
222 }222 }
223- HCCL_VM_INFO("[LoadHcclVmTaskMetaData] total tasks: {}, ranks: {}", totalTasks, allTasks.size());223+ HCCL_VM_INFO("total tasks: {}, ranks: {}", totalTasks, allTasks.size());
224 HcclVmTaskMetaData taskMeataData;224 HcclVmTaskMetaData taskMeataData;
225 for (const auto& rankTasks : allTasks) {225 for (const auto& rankTasks : allTasks) {
226 for (const auto &task : rankTasks) {226 for (const auto &task : rankTasks) {
@@ -229,7 +229,7 @@ HcclResult StorageManager::LoadHcclVmTaskMetaData(std::vector<std::vector<sim::O
229 std::memcpy(&metaData, task.optaskMeta.data(), sizeof(HcclTaskMetaData));229 std::memcpy(&metaData, task.optaskMeta.data(), sizeof(HcclTaskMetaData));
230 taskMeataData.task_meta.push_back(metaData);230 taskMeataData.task_meta.push_back(metaData);
231 } else {231 } else {
232- HCCL_VM_WARN("[LoadHcclVmTaskMetaData] optaskMeta too small, taskSeq={} src:{:d}, dst:{:d}", task.taskSeq, task.optaskMeta.size(), sizeof(HcclTaskMetaData));232+ HCCL_VM_WARN("optaskMeta too small, taskSeq={} src:{:d}, dst:{:d}", task.taskSeq, task.optaskMeta.size(), sizeof(HcclTaskMetaData));
233 }233 }
234 }234 }
235 }235 }
@@ -269,21 +269,21 @@ uint64_t StorageManager::GetBlockSize(uint32_t rankId, BufferType bufferType) {
269 // 1. 定位 Rank269 // 1. 定位 Rank
270 auto rankIt = m_mem_layout.find(rankId);270 auto rankIt = m_mem_layout.find(rankId);
271 if (rankIt == m_mem_layout.end()) {271 if (rankIt == m_mem_layout.end()) {
272- HCCL_VM_INFO("[GetBlockSize] Cannot find rank id from memory layout");272+ HCCL_VM_INFO("Cannot find rank id from memory layout");
273 return 0;273 return 0;
274 }274 }
275 275 
276 // 2. 定位 BufferType276 // 2. 定位 BufferType
277 auto typeIt = rankIt->second.find(bufferType);277 auto typeIt = rankIt->second.find(bufferType);
278 if (typeIt == rankIt->second.end()) {278 if (typeIt == rankIt->second.end()) {
279- HCCL_VM_INFO("[GetBlockSize] Cannot find buffer type from memory layout");279+ HCCL_VM_INFO("Cannot find buffer type from memory layout");
280 return 0;280 return 0;
281 }281 }
282 282 
283 // 3. 获取该类型下的最后一个块 (map 的 rbegin)283 // 3. 获取该类型下的最后一个块 (map 的 rbegin)
284 const auto& addrMap = typeIt->second;284 const auto& addrMap = typeIt->second;
285 if (addrMap.empty()) {285 if (addrMap.empty()) {
286- HCCL_VM_INFO("[GetBlockSize] Cannot find addr from memory layout");286+ HCCL_VM_INFO("Cannot find addr from memory layout");
287 return 0;287 return 0;
288 }288 }
289 289 
@@ -302,7 +302,7 @@ DataSlice StorageManager::GetDataSlice(uint32_t rankId, uint64_t addr, uint64_t
302 // 1. 定位该 Rank 的内存布局302 // 1. 定位该 Rank 的内存布局
303 auto rankIter = m_mem_layout.find(rankId);303 auto rankIter = m_mem_layout.find(rankId);
304 if (rankIter == m_mem_layout.end()) {304 if (rankIter == m_mem_layout.end()) {
305- HCCL_VM_ERROR("[GetDataSlice] Cannot find rank {} from memory layout.", rankId);305+ HCCL_VM_ERROR("Cannot find rank {} from memory layout.", rankId);
306 return slice;306 return slice;
307 }307 }
308 308 
@@ -332,7 +332,7 @@ DataSlice StorageManager::GetDataSlice(uint32_t rankId, uint64_t addr, uint64_t
332 }332 }
333 }333 }
334 }334 }
335- HCCL_VM_ERROR("[GetDataSlice] Cannot find rank {}, addr {}, size {} from memory layout.", rankId, addr, size);335+ HCCL_VM_ERROR("Cannot find rank {}, addr {}, size {} from memory layout.", rankId, addr, size);
336 return slice; // 未找到匹配的物理区间336 return slice; // 未找到匹配的物理区间
337}337}
338 338 
@@ -371,7 +371,7 @@ HcclResult StorageManager::GetSlice(uint64_t addr, uint64_t len, DataSlice& data
371 }371 }
372 }372 }
373 373 
374- HCCL_VM_ERROR("[GetSlice] Cannot find addr...:addr= {}, len={}", addr, len);374+ HCCL_VM_ERROR("Cannot find addr...:addr= {}, len={}", addr, len);
375 return HcclResult::HCCL_E_MEMORY;375 return HcclResult::HCCL_E_MEMORY;
376}376}
377 377 
@@ -391,14 +391,14 @@ std::string StorageManager::FindRootPath()
391 }391 }
392 } else {392 } else {
393 // 如果 realpath 失败(例如路径被删除或权限不足)393 // 如果 realpath 失败(例如路径被删除或权限不足)
394- HCCL_VM_ERROR("[FindRootPath] Iteration {}: realpath failed for {}", i, current_search_path);394+ HCCL_VM_ERROR("Iteration {}: realpath failed for {}", i, current_search_path);
395 }395 }
396 396 
397 // 没找到,将探测路径向上推一级397 // 没找到,将探测路径向上推一级
398 current_search_path += "/..";398 current_search_path += "/..";
399 }399 }
400 400 
401- HCCL_VM_INFO("[FindRootPath] RootPath NOT found.");401+ HCCL_VM_INFO("RootPath NOT found.");
402 return ""; 402 return "";
403}403}
404 404 
Mtest/hccl_vm/src/plugin/checker/src/utils/task_ccu.cc+1-1文件内容审核中,请稍后刷新重试
Mtest/hccl_vm/src/plugin/checker/src/utils/task_utils.cc+3-3
@@ -352,7 +352,7 @@ std::shared_ptr<TaskStub> ConvertTask(const HcclSim::StorageManager& storage, Hc
352 memcpy(ccuParam.args, hcclTask.taskData.ccu.args, sizeof(uint64_t) * ccuParam.argSize);352 memcpy(ccuParam.args, hcclTask.taskData.ccu.args, sizeof(uint64_t) * ccuParam.argSize);
353 missionParam.push_back(ccuParam);353 missionParam.push_back(ccuParam);
354 HCCL_VM_INFO("rank {}, dieId= {}", hcclTask.rankId, static_cast<uint32_t>(dieId));354 HCCL_VM_INFO("rank {}, dieId= {}", hcclTask.rankId, static_cast<uint32_t>(dieId));
355- HCCL_VM_INFO("[ConvertTask] Get sqe info: missionId= {:d}, startId= {:d}, cnt= {:d}, argSize= {:d}",355+ HCCL_VM_INFO("Get sqe info: missionId= {:d}, startId= {:d}, cnt= {:d}, argSize= {:d}",
356 missionId, ccuParam.instStartId, ccuParam.instCnt, ccuParam.argSize);356 missionId, ccuParam.instStartId, ccuParam.instCnt, ccuParam.argSize);
357 357 
358 if (g_missionTask.find(hcclTask.rankId) != g_missionTask.end()) {358 if (g_missionTask.find(hcclTask.rankId) != g_missionTask.end()) {
@@ -378,7 +378,7 @@ std::shared_ptr<TaskStub> ConvertTask(const HcclSim::StorageManager& storage, Hc
378 }378 }
379 hcomm::CcuRep::CcuInstrInfo ccuInstrInfo;379 hcomm::CcuRep::CcuInstrInfo ccuInstrInfo;
380 ccuInstrInfo.instrVec = ccuInstr.data;380 ccuInstrInfo.instrVec = ccuInstr.data;
381- HCCL_VM_INFO("[ConvertTask] Create new ccu graph base node, rank_id= {}, die_id= {} - {}",381+ HCCL_VM_INFO("Create new ccu graph base node, rank_id= {}, die_id= {} - {}",
382 ccuInstr.desc.rank_id, static_cast<uint32_t>(ccuInstr.desc.die_id), static_cast<uint32_t>(dieId));382 ccuInstr.desc.rank_id, static_cast<uint32_t>(ccuInstr.desc.die_id), static_cast<uint32_t>(dieId));
383 auto taskPtr = std::make_shared<HcclSim::TaskStubCcuGraph>(ccuInstrInfo, missionParam, ccuInstr.desc.rank_id);383 auto taskPtr = std::make_shared<HcclSim::TaskStubCcuGraph>(ccuInstrInfo, missionParam, ccuInstr.desc.rank_id);
384 g_missionTask[hcclTask.rankId][dieId][missionId] = taskPtr;384 g_missionTask[hcclTask.rankId][dieId][missionId] = taskPtr;
@@ -410,7 +410,7 @@ void ConvertTaskQueue(AllRankTaskQueues& allRankTaskQueues)
410 continue;410 continue;
411 }411 }
412 if (size > 10000 && i % outputInterval == 0) {412 if (size > 10000 && i % outputInterval == 0) {
413- HCCL_VM_INFO("[ConvertTaskQueue] Processing at index: {:d} / {:d} ({:f}% complete)",413+ HCCL_VM_INFO("Processing at index: {:d} / {:d} ({:f}% complete)",
414 i, size, i * 100.0 / size);414 i, size, i * 100.0 / size);
415 }415 }
416 uint32_t rankId = taskMetaVec[i].rankId;416 uint32_t rankId = taskMetaVec[i].rankId;
Mtest/hccl_vm/src/plugin/hccl_plugin_manager.cc+13-12
@@ -29,6 +29,7 @@
29#include <unistd.h>29#include <unistd.h>
30 30 
31#include "cmd_base_utils.h"31#include "cmd_base_utils.h"
32+#include "sim_common_api.h"
32#include "store_dump_shm_data.h"33#include "store_dump_shm_data.h"
33#include "sim_log.h"34#include "sim_log.h"
34#include "sim_process_syncer.h"35#include "sim_process_syncer.h"
@@ -69,7 +70,7 @@ bool HcclPluginManager::IsMatchingPlugin(const std::string& manifestPath, const
69 if (currentPluginName == targetTag) {70 if (currentPluginName == targetTag) {
70 // 如果必需字段缺失,认为是一个损坏的插件,返回 false 并记录日志71 // 如果必需字段缺失,认为是一个损坏的插件,返回 false 并记录日志
71 if (!data.contains(HcclPlugin::Manifest::pluginEntry)) {72 if (!data.contains(HcclPlugin::Manifest::pluginEntry)) {
72- HCCL_VM_ERROR("[Plugin] [{}] '{}' missing entry field.", __func__, targetTag);73+ HCCL_VM_ERROR("[{}] '{}' missing entry field.", __func__, targetTag);
73 return false;74 return false;
74 }75 }
75 return true;76 return true;
@@ -77,11 +78,11 @@ bool HcclPluginManager::IsMatchingPlugin(const std::string& manifestPath, const
77 }78 }
78 } catch (const nlohmann::json::parse_error& e) {79 } catch (const nlohmann::json::parse_error& e) {
79 // 生产环境建议记录具体的解析错误位置80 // 生产环境建议记录具体的解析错误位置
80- HCCL_VM_ERROR("[Plugin] [{}] JSON Parse Error at byte {}", __func__, e.byte);81+ HCCL_VM_ERROR("[{}] JSON Parse Error at byte {}", __func__, e.byte);
81 return false;82 return false;
82 } catch (const std::exception& e) {83 } catch (const std::exception& e) {
83 // 处理其他可能的异常(如文件读取中途断开等)84 // 处理其他可能的异常(如文件读取中途断开等)
84- HCCL_VM_ERROR("[Plugin] [{}] Error: ", __func__, e.what());85+ HCCL_VM_ERROR("[{}] Error: {}", __func__, e.what());
85 return false;86 return false;
86 }87 }
87 88 
@@ -90,7 +91,7 @@ bool HcclPluginManager::IsMatchingPlugin(const std::string& manifestPath, const
90 91 
91bool HcclPluginManager::GetPluginFolderPath(const std::string& pluginTag, std::string& pluginFolderPath)92bool HcclPluginManager::GetPluginFolderPath(const std::string& pluginTag, std::string& pluginFolderPath)
92{93{
93- std::string rootDir = GetBinLocation() + HcclPlugin::PLUGIN_PATH;94+ std::string rootDir = InstallPath::ResolveToInstallRoot(HcclPlugin::PLUGIN_PATH.substr(1));
94 std::string foundPath = "";95 std::string foundPath = "";
95 bool hasConflict = false;96 bool hasConflict = false;
96 97 
@@ -115,7 +116,7 @@ bool HcclPluginManager::GetPluginFolderPath(const std::string& pluginTag, std::s
115 116
116 // 冲突检查:如果之前已经找到了一个路径,且不是当前路径117 // 冲突检查:如果之前已经找到了一个路径,且不是当前路径
117 if (!foundPath.empty() && foundPath != currentAbsPath) {118 if (!foundPath.empty() && foundPath != currentAbsPath) {
118- HCCL_VM_ERROR("[Plugin] [{}] Conflict detected: Tag '{}' exists in both: {} and {}",119+ HCCL_VM_ERROR("[{}] Conflict detected: Tag '{}' exists in both: {} and {}",
119 __func__, pluginTag, foundPath, currentAbsPath);120 __func__, pluginTag, foundPath, currentAbsPath);
120 hasConflict = true;121 hasConflict = true;
121 break; 122 break;
@@ -203,7 +204,7 @@ HcclVmResult HcclPluginManager::RegisterPlugin(const std::string& pluginTag)
203 204
204 // 防止重复注册205 // 防止重复注册
205 if (m_plugins.find(pluginTag) != m_plugins.end()) {206 if (m_plugins.find(pluginTag) != m_plugins.end()) {
206- HCCL_VM_WARN("[PluginManager] [{}] Plugin {} already registered", __func__, pluginTag);207+ HCCL_VM_WARN("[{}] Plugin {} already registered", __func__, pluginTag);
207 return HcclSim::HcclVmResult::HCCL_SIM_SUCCESS;208 return HcclSim::HcclVmResult::HCCL_SIM_SUCCESS;
208 }209 }
209 210 
@@ -217,7 +218,7 @@ HcclVmResult HcclPluginManager::RegisterPlugin(const std::string& pluginTag)
217 auto plugin = std::make_shared<HcclPlugin>(folderPath);218 auto plugin = std::make_shared<HcclPlugin>(folderPath);
218 m_plugins[pluginTag] = plugin;219 m_plugins[pluginTag] = plugin;
219 } catch (const std::exception& e) {220 } catch (const std::exception& e) {
220- HCCL_VM_ERROR("[PluginManager] [{}] Failed to register plugin {}: {}", __func__, pluginTag, e.what());221+ HCCL_VM_ERROR("[{}] Failed to register plugin {}: {}", __func__, pluginTag, e.what());
221 return HcclSim::HcclVmResult::HCCL_SIM_E_INTERNAL;222 return HcclSim::HcclVmResult::HCCL_SIM_E_INTERNAL;
222 }223 }
223 224 
@@ -229,12 +230,12 @@ HcclVmResult HcclPluginManager::SendMessageToPlugin(const std::string& pluginTag
229 std::lock_guard<std::mutex> lock(m_mutex);230 std::lock_guard<std::mutex> lock(m_mutex);
230 auto it = m_plugins.find(pluginTag);231 auto it = m_plugins.find(pluginTag);
231 if (it == m_plugins.end()) {232 if (it == m_plugins.end()) {
232- HCCL_VM_ERROR("[PluginManager] [{}] Plugin {} not found", __func__, pluginTag);233+ HCCL_VM_ERROR("[{}] Plugin {} not found", __func__, pluginTag);
233 return HcclVmResult::HCCL_SIM_E_NOT_FOUND;234 return HcclVmResult::HCCL_SIM_E_NOT_FOUND;
234 }235 }
235 HcclVmResult ret = it->second->SendMessage(PLUGIN_MESSAGE_TYPE::BROADCAST, action, payload);236 HcclVmResult ret = it->second->SendMessage(PLUGIN_MESSAGE_TYPE::BROADCAST, action, payload);
236 if (ret != HcclVmResult::HCCL_SIM_SUCCESS) {237 if (ret != HcclVmResult::HCCL_SIM_SUCCESS) {
237- HCCL_VM_ERROR("[PluginManager] [{}] Failed to send message to plugin {}: {}", __func__, pluginTag, static_cast<int>(ret));238+ HCCL_VM_ERROR("[{}] Failed to send message to plugin {}: {}", __func__, pluginTag, static_cast<int>(ret));
238 }239 }
239 return ret;240 return ret;
240}241}
@@ -341,7 +342,7 @@ HcclSim::HcclVmResult HcclPluginManager::StopAllPlugins() {
341 342 
342HcclPluginManager::HcclPluginManager()343HcclPluginManager::HcclPluginManager()
343{344{
344- HCCL_VM_INFO("[PluginManager]HcclPluginManager");345+ HCCL_VM_INFO("HcclPluginManager");
345 m_monitorThreadStop = false;346 m_monitorThreadStop = false;
346 m_monitorThread = std::thread([this]() {347 m_monitorThread = std::thread([this]() {
347 this->MonitorThread();348 this->MonitorThread();
@@ -369,10 +370,10 @@ void HcclPluginManager::MonitorThread()
369 if (it.second->GetPid() == terminatedPid) {370 if (it.second->GetPid() == terminatedPid) {
370 // 解析退出原因371 // 解析退出原因
371 if (WIFEXITED(status)) {372 if (WIFEXITED(status)) {
372- HCCL_VM_INFO("[PluginManager] [{}] Plugin [{}] (PID: {}) exit normally. Code {}", __func__,373+ HCCL_VM_INFO("[{}] Plugin [{}] (PID: {}) exit normally. Code {}", __func__,
373 it.second->GetTag(), terminatedPid, WEXITSTATUS(status));374 it.second->GetTag(), terminatedPid, WEXITSTATUS(status));
374 } else if (WIFSIGNALED(status)) {375 } else if (WIFSIGNALED(status)) {
375- HCCL_VM_ERROR("[PluginManager] [{}] Plugin [{}] (PID: {}) exit with failure. Code {} ({})", __func__,376+ HCCL_VM_ERROR("[{}] Plugin [{}] (PID: {}) exit with failure. Code {} ({})", __func__,
376 it.second->GetTag(), terminatedPid, WTERMSIG(status), strsignal(WTERMSIG(status)));377 it.second->GetTag(), terminatedPid, WTERMSIG(status), strsignal(WTERMSIG(status)));
377 }378 }
378 m_plugins.erase(it.first);379 m_plugins.erase(it.first);
Mtest/hccl_vm/src/plugin/hccl_virtual_runtime_main.cc+11-11
@@ -85,10 +85,10 @@ void RecordRunVirtualRuntimeTime(RunVirtualRuntimeTimeStats& stats, uint64_t cos
85void DumpRunVirtualRuntimeTimeStats(const RunVirtualRuntimeTimeStats& stats)85void DumpRunVirtualRuntimeTimeStats(const RunVirtualRuntimeTimeStats& stats)
86{86{
87 if (stats.runCount == 0) {87 if (stats.runCount == 0) {
88- HCCL_VM_INFO("[Runner][StageTime] RunVirtualRuntime no execution, runCount=0");88+ HCCL_VM_INFO("RunVirtualRuntime no execution, runCount=0");
89 return;89 return;
90 }90 }
91- HCCL_VM_INFO("[Runner][StageTime] RunVirtualRuntime summary, runCount={}, totalCostMs={}, lastCostMs={}, "91+ HCCL_VM_INFO("RunVirtualRuntime summary, runCount={}, totalCostMs={}, lastCostMs={}, "
92 "maxCostMs={}", stats.runCount, stats.totalCostMs, stats.lastCostMs, stats.maxCostMs);92 "maxCostMs={}", stats.runCount, stats.totalCostMs, stats.lastCostMs, stats.maxCostMs);
93}93}
94}94}
@@ -152,12 +152,12 @@ void ProcessCommand(const std::string& line) {
152 std::string action = j.value("action", "");152 std::string action = j.value("action", "");
153 153 
154 if (action == "stop") {154 if (action == "stop") {
155- HCCL_VM_INFO("[Runner] Stop signal received. Initiating shutdown...");155+ HCCL_VM_INFO("Stop signal received. Initiating shutdown...");
156 g_keep_running.store(false);156 g_keep_running.store(false);
157 }157 }
158 // 未来可扩展其他 action...158 // 未来可扩展其他 action...
159 } catch (const std::exception& e) {159 } catch (const std::exception& e) {
160- HCCL_VM_ERROR("[Runner] Command processing failed: {}", e.what());160+ HCCL_VM_ERROR("Command processing failed: {}", e.what());
161 }161 }
162}162}
163 163 
@@ -210,10 +210,10 @@ void RunVirtualRuntime(HcclSim::StorageManager& storage) {
210 210 
211 HcclSim::CheckerParam param = storage.GetCheckerParam();211 HcclSim::CheckerParam param = storage.GetCheckerParam();
212 HcclCMDType cmdType = param.cmdType;212 HcclCMDType cmdType = param.cmdType;
213- HCCL_VM_INFO("[RunVirtualRuntime] start check semantic ....: cmdType = {}, reduceOp = {}",213+ HCCL_VM_INFO("start check semantic ....: cmdType = {}, reduceOp = {}",
214 static_cast<uint32_t>(cmdType), static_cast<uint32_t>(param.reduceType));214 static_cast<uint32_t>(cmdType), static_cast<uint32_t>(param.reduceType));
215 215 
216- HCCL_VM_INFO("[Runner][OpInfo] syncIter = {}, op = {}, rankSize = {}, dataType = {}, dataCount = {}, "216+ HCCL_VM_INFO("syncIter = {}, op = {}, rankSize = {}, dataType = {}, dataCount = {}, "
217 "reduceType = {}, srcRank = {}, dstRank = {}, root = {}",217 "reduceType = {}, srcRank = {}, dstRank = {}, root = {}",
218 records[0].syncIter, HcclCmdTypeToString(cmdType), param.rankSize,218 records[0].syncIter, HcclCmdTypeToString(cmdType), param.rankSize,
219 HcclDataTypeToString(param.dataType), param.dataCount,219 HcclDataTypeToString(param.dataType), param.dataCount,
@@ -275,7 +275,7 @@ int main(int argc, char* argv[])
275 }275 }
276 // EOF 或 stop 命令后,确保标志被设置276 // EOF 或 stop 命令后,确保标志被设置
277 g_keep_running.store(false);277 g_keep_running.store(false);
278- HCCL_VM_INFO("[Runner] stdin reader thread exiting.");278+ HCCL_VM_INFO("stdin reader thread exiting.");
279 }).detach();279 }).detach();
280 280 
281 while (g_keep_running.load()) {281 while (g_keep_running.load()) {
@@ -286,16 +286,16 @@ int main(int argc, char* argv[])
286 continue;286 continue;
287 }287 }
288 if (targetRound == sim::kRunnerExitSignal) {288 if (targetRound == sim::kRunnerExitSignal) {
289- HCCL_VM_INFO("[Runner] Exit signal received from hccl-vm.");289+ HCCL_VM_INFO("Exit signal received from hccl-vm.");
290 break;290 break;
291 }291 }
292- HCCL_VM_INFO("[Runner] Task received, targetRound={}", targetRound);292+ HCCL_VM_INFO("Task received, targetRound={}", targetRound);
293 RunVirtualRuntime(storage);293 RunVirtualRuntime(storage);
294 syncer.notifyProxyToContinue(targetRound);294 syncer.notifyProxyToContinue(targetRound);
295- HCCL_VM_INFO("[Runner] Round {} completed.", targetRound);295+ HCCL_VM_INFO("Round {} completed.", targetRound);
296 FlushLog(); // 将本轮完整日志落盘296 FlushLog(); // 将本轮完整日志落盘
297 }297 }
298 298 
299- HCCL_VM_INFO("[Runner] Exiting...");299+ HCCL_VM_INFO("Exiting...");
300 return 0;300 return 0;
301}301}
Mtest/hccl_vm/src/plugin/runner/CMakeLists.txt+3-0
@@ -89,3 +89,6 @@ install(FILES "${CMAKE_CURRENT_SOURCE_DIR}/../manifest.json"
89 GROUP_READ GROUP_EXECUTE89 GROUP_READ GROUP_EXECUTE
90 WORLD_READ WORLD_EXECUTE90 WORLD_READ WORLD_EXECUTE
91)91)
92+ 
93+# 日志染色: 模块 tag
94+target_compile_definitions(runner PRIVATE HCCL_VM_MODULE="RUNNER")
Mtest/hccl_vm/src/plugin/runner/aiv_executor/CMakeLists.txt+3-0
@@ -56,3 +56,6 @@ target_link_libraries(aiv_executor
56 log56 log
57 rt57 rt
58)58)
59+ 
60+# 日志染色: 模块 tag
61+target_compile_definitions(aiv_executor PRIVATE HCCL_VM_MODULE="RUNNER")
Mtest/hccl_vm/src/plugin/runner/aiv_executor/aiv_graph_executor.cc+11-11
@@ -78,7 +78,7 @@ AivBlock::~AivBlock() {
78}78}
79 79 
80bool AivGraphExecutor::Init(uint32_t rankId, uint32_t launchIdx) {80bool AivGraphExecutor::Init(uint32_t rankId, uint32_t launchIdx) {
81- HCCL_VM_DEBUG("[AivGraphExecutor][Init] begin, launchIdx={}, rankId={}", launchIdx_, rankId_);81+ HCCL_VM_DEBUG("begin, launchIdx={}, rankId={}", launchIdx_, rankId_);
82 rankId_ = rankId;82 rankId_ = rankId;
83 launchIdx_ = launchIdx;83 launchIdx_ = launchIdx;
84 84 
@@ -86,7 +86,7 @@ bool AivGraphExecutor::Init(uint32_t rankId, uint32_t launchIdx) {
86 std::string errorMessage;86 std::string errorMessage;
87 if (!AivTaskSnapshotLoader::LoadRuntimeTaskSnapshotByLaunchDirect(87 if (!AivTaskSnapshotLoader::LoadRuntimeTaskSnapshotByLaunchDirect(
88 rankId_, static_cast<uint32_t>(launchIdx_), taskSnapshot, &errorMessage)) {88 rankId_, static_cast<uint32_t>(launchIdx_), taskSnapshot, &errorMessage)) {
89- HCCL_VM_ERROR("[AivGraphExecutor][Init] failed to load runtime snapshot, launchIdx={}, rankId={}, reason={}",89+ HCCL_VM_ERROR("failed to load runtime snapshot, launchIdx={}, rankId={}, reason={}",
90 launchIdx_, rankId_, errorMessage);90 launchIdx_, rankId_, errorMessage);
91 return false;91 return false;
92 }92 }
@@ -99,7 +99,7 @@ bool AivGraphExecutor::Init(uint32_t rankId, uint32_t launchIdx) {
99 99 
100 rankId_ = taskSnapshot.rankId;100 rankId_ = taskSnapshot.rankId;
101 rankSize_ = taskSnapshot.rankSize;101 rankSize_ = taskSnapshot.rankSize;
102- HCCL_VM_DEBUG("[AivGraphExecutor][Init] snapshot loaded, launchIdx={}, rankId={}, rankSize={}, file={}, aivBlockNum={}",102+ HCCL_VM_DEBUG("snapshot loaded, launchIdx={}, rankId={}, rankSize={}, file={}, aivBlockNum={}",
103 launchIdx_, rankId_, rankSize_, taskSnapshot.filePath, taskSnapshot.blocks.size());103 launchIdx_, rankId_, rankSize_, taskSnapshot.filePath, taskSnapshot.blocks.size());
104 104 
105 uint32_t maxTaskId = 0;105 uint32_t maxTaskId = 0;
@@ -116,7 +116,7 @@ bool AivGraphExecutor::Init(uint32_t rankId, uint32_t launchIdx) {
116 aivTaskQueues_.emplace_back();116 aivTaskQueues_.emplace_back();
117 AppendPipeTasksToQueue(block.mte3Tasks, aivTaskQueues_.back(), maxTaskId, maxEventId, maxSyncRound);117 AppendPipeTasksToQueue(block.mte3Tasks, aivTaskQueues_.back(), maxTaskId, maxEventId, maxSyncRound);
118 118 
119- HCCL_VM_TRACE("[AivGraphExecutor][Init] blockIdx={}, scalarTaskCount={}, mte2TaskCount={}, mte3TaskCount={}, maxEventId={}",119+ HCCL_VM_TRACE("blockIdx={}, scalarTaskCount={}, mte2TaskCount={}, mte3TaskCount={}, maxEventId={}",
120 block.blockIdx, block.scalarTasks.size(), block.mte2Tasks.size(), block.mte3Tasks.size(), maxEventId);120 block.blockIdx, block.scalarTasks.size(), block.mte2Tasks.size(), block.mte3Tasks.size(), maxEventId);
121 aivBlocks_.emplace_back(std::make_unique<AivBlock>(block.blockIdx, maxEventId, AivSim::AIV_UB_SIZE));121 aivBlocks_.emplace_back(std::make_unique<AivBlock>(block.blockIdx, maxEventId, AivSim::AIV_UB_SIZE));
122 }122 }
@@ -129,11 +129,11 @@ bool AivGraphExecutor::Init(uint32_t rankId, uint32_t launchIdx) {
129 syncAllRegisters_.emplace_back(pipeSize, false);129 syncAllRegisters_.emplace_back(pipeSize, false);
130 }130 }
131 131 
132- HCCL_VM_DEBUG("[AivGraphExecutor][Init] queueCount={}, maxTaskId={}, pipeBarrierRegisterCount={}, syncAllRegisterCount={}, aivBlockNum={}",132+ HCCL_VM_DEBUG("queueCount={}, maxTaskId={}, pipeBarrierRegisterCount={}, syncAllRegisterCount={}, aivBlockNum={}",
133 aivTaskQueues_.size(), maxTaskId, pipeBarrierRegisters_.size(), syncAllRegisters_.size(), aivBlocks_.size());133 aivTaskQueues_.size(), maxTaskId, pipeBarrierRegisters_.size(), syncAllRegisters_.size(), aivBlocks_.size());
134 134 
135 isInitialized_ = true;135 isInitialized_ = true;
136- HCCL_VM_INFO("[AivGraphExecutor][Init] success, launchIdx={}, rankId={}", launchIdx_, rankId_);136+ HCCL_VM_INFO("success, launchIdx={}, rankId={}", launchIdx_, rankId_);
137 return true;137 return true;
138}138}
139 139 
@@ -204,7 +204,7 @@ HcclVmResult AivGraphExecutor::ExecuteTask(std::shared_ptr<AivSim::AivTask> task
204}204}
205 205 
206HcclVmResult AivGraphExecutor::ExecuteTask(std::shared_ptr<AivSim::AivTaskMemCopy> task) {206HcclVmResult AivGraphExecutor::ExecuteTask(std::shared_ptr<AivSim::AivTaskMemCopy> task) {
207- HCCL_VM_DEBUG(task->Describe());207+ HCCL_VM_DEBUG("{}", task->Describe());
208 208 
209 if (task->GetSrc().GetSize() != task->GetDst().GetSize()) {209 if (task->GetSrc().GetSize() != task->GetDst().GetSize()) {
210 HCCL_VM_ERROR("task invalid, len not match, taskId={:d}", task->GetTaskId());210 HCCL_VM_ERROR("task invalid, len not match, taskId={:d}", task->GetTaskId());
@@ -270,7 +270,7 @@ void* AivGraphExecutor::GetMemPtr(std::shared_ptr<T> task, bool isSrc) {
270}270}
271 271 
272HcclVmResult AivGraphExecutor::ExecuteTask(std::shared_ptr<AivSim::AivTaskReduce> task) {272HcclVmResult AivGraphExecutor::ExecuteTask(std::shared_ptr<AivSim::AivTaskReduce> task) {
273- HCCL_VM_DEBUG(task->Describe());273+ HCCL_VM_DEBUG("{}", task->Describe());
274 274 
275 if (task->GetSrc().GetSize() != task->GetDst().GetSize()) {275 if (task->GetSrc().GetSize() != task->GetDst().GetSize()) {
276 HCCL_VM_ERROR("task invalid, len not match, taskId={:d}", task->GetTaskId());276 HCCL_VM_ERROR("task invalid, len not match, taskId={:d}", task->GetTaskId());
@@ -399,7 +399,7 @@ HcclVmResult AivGraphExecutor::ExecuteTask(std::shared_ptr<AivSim::AivTaskSyncAl
399}399}
400 400 
401HcclVmResult AivGraphExecutor::ExecuteTask(std::shared_ptr<AivSim::AivTaskSendFlag> task) {401HcclVmResult AivGraphExecutor::ExecuteTask(std::shared_ptr<AivSim::AivTaskSendFlag> task) {
402- HCCL_VM_DEBUG(task->Describe());402+ HCCL_VM_DEBUG("{}", task->Describe());
403 403 
404 auto* rankResource = AivResourceManager::GetInstance().GetRankResource(task->GetRank());404 auto* rankResource = AivResourceManager::GetInstance().GetRankResource(task->GetRank());
405 if (rankResource == nullptr) {405 if (rankResource == nullptr) {
@@ -417,7 +417,7 @@ HcclVmResult AivGraphExecutor::ExecuteTask(std::shared_ptr<AivSim::AivTaskSendFl
417}417}
418 418 
419HcclVmResult AivGraphExecutor::ExecuteTask(std::shared_ptr<AivSim::AivTaskRecvFlag> task) {419HcclVmResult AivGraphExecutor::ExecuteTask(std::shared_ptr<AivSim::AivTaskRecvFlag> task) {
420- HCCL_VM_DEBUG(task->Describe());420+ HCCL_VM_DEBUG("{}", task->Describe());
421 421 
422 auto* rankResource = AivResourceManager::GetInstance().GetRankResource(task->GetRank());422 auto* rankResource = AivResourceManager::GetInstance().GetRankResource(task->GetRank());
423 if (rankResource == nullptr) {423 if (rankResource == nullptr) {
@@ -456,5 +456,5 @@ void AivGraphExecutor::ShowCurrentTaskStatus() {
456 ss << "]";456 ss << "]";
457 }457 }
458 ss << "}";458 ss << "}";
459- HCCL_VM_DEBUG(ss.str());459+ HCCL_VM_DEBUG("{}", ss.str());
460}460}
Mtest/hccl_vm/src/plugin/runner/aiv_executor/aiv_resource_manager.cc+14-14
@@ -74,7 +74,7 @@ HcclVmResult AivResourceManager::Init(uint32_t rankId, const sim::OpMemInfoTab &
74 return ret;74 return ret;
75 }75 }
76 if (rankId >= rankSize) {76 if (rankId >= rankSize) {
77- HCCL_VM_ERROR("[AivResourceManager][Init] rankId={} out of range, rankSize={}", rankId, rankSize);77+ HCCL_VM_ERROR("rankId={} out of range, rankSize={}", rankId, rankSize);
78 return HcclVmResult::HCCL_SIM_E_PARA;78 return HcclVmResult::HCCL_SIM_E_PARA;
79 }79 }
80 80 
@@ -91,7 +91,7 @@ HcclVmResult AivResourceManager::Init(uint32_t rankId, const sim::OpMemInfoTab &
91 return ret;91 return ret;
92 }92 }
93 93 
94- HCCL_VM_DEBUG("[AivResourceManager][Init] init rank op mem success, rankId={}, rankSize={}, "94+ HCCL_VM_DEBUG("init rank op mem success, rankId={}, rankSize={}, "
95 "input={:x}/{} output={:x}/{} ccl={:x}/{}",95 "input={:x}/{} output={:x}/{} ccl={:x}/{}",
96 rankId,96 rankId,
97 rankSize,97 rankSize,
@@ -130,7 +130,7 @@ const AivRankResource *AivResourceManager::GetRankResource(uint32_t rankId) cons
130HcclVmResult AivResourceManager::EnsureInitialized(uint32_t rankSize)130HcclVmResult AivResourceManager::EnsureInitialized(uint32_t rankSize)
131{131{
132 if (rankSize == 0) {132 if (rankSize == 0) {
133- HCCL_VM_ERROR("[AivResourceManager][EnsureInitialized] rankSize is 0");133+ HCCL_VM_ERROR("rankSize is 0");
134 return HcclVmResult::HCCL_SIM_E_PARA;134 return HcclVmResult::HCCL_SIM_E_PARA;
135 }135 }
136 136 
@@ -141,12 +141,12 @@ HcclVmResult AivResourceManager::EnsureInitialized(uint32_t rankSize)
141 Reset();141 Reset();
142 return ret;142 return ret;
143 }143 }
144- HCCL_VM_INFO("[AivResourceManager][EnsureInitialized] init base resource success, rankSize={}", rankSize);144+ HCCL_VM_INFO("init base resource success, rankSize={}", rankSize);
145 return HcclVmResult::HCCL_SIM_SUCCESS;145 return HcclVmResult::HCCL_SIM_SUCCESS;
146 }146 }
147 147 
148 if (rankResources_.size() != rankSize) {148 if (rankResources_.size() != rankSize) {
149- HCCL_VM_ERROR("[AivResourceManager][EnsureInitialized] rankSize mismatch, current={}, incoming={}",149+ HCCL_VM_ERROR("rankSize mismatch, current={}, incoming={}",
150 rankResources_.size(), rankSize);150 rankResources_.size(), rankSize);
151 return HcclVmResult::HCCL_SIM_E_PARA;151 return HcclVmResult::HCCL_SIM_E_PARA;
152 }152 }
@@ -157,7 +157,7 @@ HcclVmResult AivResourceManager::MapBuffer(
157 uint32_t rankSize, uint32_t rankId, uint8_t bufferType, uint64_t startAddr, uint64_t size, bool allowDuplicateSame)157 uint32_t rankSize, uint32_t rankId, uint8_t bufferType, uint64_t startAddr, uint64_t size, bool allowDuplicateSame)
158{158{
159 if (rankId >= rankSize) {159 if (rankId >= rankSize) {
160- HCCL_VM_ERROR("[AivResourceManager][MapBuffer] rankId={} out of range, rankSize={}",160+ HCCL_VM_ERROR("rankId={} out of range, rankSize={}",
161 rankId, rankSize);161 rankId, rankSize);
162 return HcclVmResult::HCCL_SIM_E_PARA;162 return HcclVmResult::HCCL_SIM_E_PARA;
163 }163 }
@@ -165,18 +165,18 @@ HcclVmResult AivResourceManager::MapBuffer(
165 auto &rankResource = rankResources_[rankId];165 auto &rankResource = rankResources_[rankId];
166 AivBufferResource *bufferSlot = GetBufferSlot(rankResource, bufferType);166 AivBufferResource *bufferSlot = GetBufferSlot(rankResource, bufferType);
167 if (bufferSlot == nullptr) {167 if (bufferSlot == nullptr) {
168- HCCL_VM_ERROR("[AivResourceManager][MapBuffer] unsupported buffer type={}, rankId={}",168+ HCCL_VM_ERROR("unsupported buffer type={}, rankId={}",
169 bufferType, rankId);169 bufferType, rankId);
170 return HcclVmResult::HCCL_SIM_E_PARA;170 return HcclVmResult::HCCL_SIM_E_PARA;
171 }171 }
172 if (bufferSlot->realAddr != nullptr) {172 if (bufferSlot->realAddr != nullptr) {
173 if (allowDuplicateSame && bufferSlot->virtualAddr == startAddr && bufferSlot->size == size) {173 if (allowDuplicateSame && bufferSlot->virtualAddr == startAddr && bufferSlot->size == size) {
174- HCCL_VM_DEBUG("[AivResourceManager][MapBuffer] duplicate same {} buffer skipped, "174+ HCCL_VM_DEBUG("duplicate same {} buffer skipped, "
175 "rankId={}, virtualAddr={:x}, size={}",175 "rankId={}, virtualAddr={:x}, size={}",
176 GetBufferTypeName(bufferType), rankId, startAddr, size);176 GetBufferTypeName(bufferType), rankId, startAddr, size);
177 return HcclVmResult::HCCL_SIM_SUCCESS;177 return HcclVmResult::HCCL_SIM_SUCCESS;
178 }178 }
179- HCCL_VM_ERROR("[AivResourceManager][MapBuffer] duplicate {} buffer found, rankId={}, virtualAddr={:x}, size={}",179+ HCCL_VM_ERROR("duplicate {} buffer found, rankId={}, virtualAddr={:x}, size={}",
180 GetBufferTypeName(bufferType), rankId, startAddr, size);180 GetBufferTypeName(bufferType), rankId, startAddr, size);
181 return HcclVmResult::HCCL_SIM_E_PARA;181 return HcclVmResult::HCCL_SIM_E_PARA;
182 }182 }
@@ -184,7 +184,7 @@ HcclVmResult AivResourceManager::MapBuffer(
184 sim::PhyMemBlock phyMem {};184 sim::PhyMemBlock phyMem {};
185 void *realAddr = sim::AcquireDevPtrInNoHostProcess(reinterpret_cast<void *>(startAddr), phyMem);185 void *realAddr = sim::AcquireDevPtrInNoHostProcess(reinterpret_cast<void *>(startAddr), phyMem);
186 if (realAddr == nullptr) {186 if (realAddr == nullptr) {
187- HCCL_VM_ERROR("[AivResourceManager][MapBuffer] translate addr failed, rankId={}, type={}, addr={:x}",187+ HCCL_VM_ERROR("translate addr failed, rankId={}, type={}, addr={:x}",
188 rankId, bufferType, startAddr);188 rankId, bufferType, startAddr);
189 return HcclVmResult::HCCL_SIM_E_INTERNAL;189 return HcclVmResult::HCCL_SIM_E_INTERNAL;
190 }190 }
@@ -197,7 +197,7 @@ HcclVmResult AivResourceManager::MapBuffer(
197 *bufferSlot = bufferResource;197 *bufferSlot = bufferResource;
198 198 
199 acquiredPhyMemBlocks_.push_back(phyMem);199 acquiredPhyMemBlocks_.push_back(phyMem);
200- HCCL_VM_DEBUG("[AivResourceManager][MapBuffer] rankId={}, type={}, virtualAddr={:x}, realAddr={}, size={}",200+ HCCL_VM_DEBUG("rankId={}, type={}, virtualAddr={:x}, realAddr={}, size={}",
201 rankId, bufferType, startAddr, realAddr, size);201 rankId, bufferType, startAddr, realAddr, size);
202 return HcclVmResult::HCCL_SIM_SUCCESS;202 return HcclVmResult::HCCL_SIM_SUCCESS;
203}203}
@@ -207,7 +207,7 @@ HcclVmResult AivResourceManager::MapOpMemBuffer(
207{207{
208 if (startAddr == 0 || size == 0) {208 if (startAddr == 0 || size == 0) {
209 if (startAddr != 0 || size != 0) {209 if (startAddr != 0 || size != 0) {
210- HCCL_VM_WARN("[AivResourceManager][MapOpMemBuffer] incomplete {} buffer skipped, "210+ HCCL_VM_WARN("incomplete {} buffer skipped, "
211 "rankId={}, virtualAddr={:x}, size={}",211 "rankId={}, virtualAddr={:x}, size={}",
212 GetBufferTypeName(bufferType), rankId, startAddr, size);212 GetBufferTypeName(bufferType), rankId, startAddr, size);
213 }213 }
@@ -223,7 +223,7 @@ HcclVmResult AivResourceManager::InitFlagBuffers(uint32_t rankSize)
223 for (uint32_t rankId = 0; rankId < rankSize; ++rankId) {223 for (uint32_t rankId = 0; rankId < rankSize; ++rankId) {
224 auto flagBuffer = std::make_unique<uint8_t[]>(AIV_FLAG_BUFFER_SIZE);224 auto flagBuffer = std::make_unique<uint8_t[]>(AIV_FLAG_BUFFER_SIZE);
225 if (flagBuffer == nullptr) {225 if (flagBuffer == nullptr) {
226- HCCL_VM_ERROR("[AivResourceManager][InitFlagBuffers] alloc flag buffer failed, rankId={}, size={}",226+ HCCL_VM_ERROR("alloc flag buffer failed, rankId={}, size={}",
227 rankId, AIV_FLAG_BUFFER_SIZE);227 rankId, AIV_FLAG_BUFFER_SIZE);
228 return HcclVmResult::HCCL_SIM_E_INTERNAL;228 return HcclVmResult::HCCL_SIM_E_INTERNAL;
229 }229 }
@@ -232,7 +232,7 @@ HcclVmResult AivResourceManager::InitFlagBuffers(uint32_t rankSize)
232 rankResources_[rankId].flagBuffer.size = AIV_FLAG_BUFFER_SIZE;232 rankResources_[rankId].flagBuffer.size = AIV_FLAG_BUFFER_SIZE;
233 flagBufferOwners_[rankId] = std::move(flagBuffer);233 flagBufferOwners_[rankId] = std::move(flagBuffer);
234 234 
235- HCCL_VM_DEBUG("[AivResourceManager][InitFlagBuffers] rankId={}, realAddr={}, size={}",235+ HCCL_VM_DEBUG("rankId={}, realAddr={}, size={}",
236 rankId,236 rankId,
237 rankResources_[rankId].flagBuffer.realAddr,237 rankResources_[rankId].flagBuffer.realAddr,
238 rankResources_[rankId].flagBuffer.size);238 rankResources_[rankId].flagBuffer.size);
Mtest/hccl_vm/src/plugin/runner/aiv_executor/aiv_task_snapshot_loader.cc+2-7
@@ -9,6 +9,7 @@
9 */9 */
10 10 
11#include "aiv_task_snapshot_loader.h"11#include "aiv_task_snapshot_loader.h"
12+#include "sim_common_api.h"
12 13 
13#include <cstdint>14#include <cstdint>
14#include <cstdlib>15#include <cstdlib>
@@ -33,13 +34,7 @@ static void SetCommonError(std::string *errorMessage, const std::string &message
33 34 
34static bool ResolveAivSnapshotDirectory(fs::path &snapshotDir, std::string *errorMessage)35static bool ResolveAivSnapshotDirectory(fs::path &snapshotDir, std::string *errorMessage)
35{36{
36- const char *installDirEnv = std::getenv("HCCL_VM_INSTALL_DIR");37+ snapshotDir = fs::path(InstallPath::ResolveToInstallRoot("data"));
37- if (installDirEnv == nullptr || installDirEnv[0] == '\0') {
38- SetCommonError(errorMessage, "HCCL_VM_INSTALL_DIR is not set");
39- return false;
40- }
41- 
42- snapshotDir = fs::path(installDirEnv) / "data";
43 std::error_code ec;38 std::error_code ec;
44 if (!fs::exists(snapshotDir, ec) || ec || !fs::is_directory(snapshotDir, ec)) {39 if (!fs::exists(snapshotDir, ec) || ec || !fs::is_directory(snapshotDir, ec)) {
45 SetCommonError(errorMessage, "AIV data directory does not exist: " + snapshotDir.string());40 SetCommonError(errorMessage, "AIV data directory does not exist: " + snapshotDir.string());
Mtest/hccl_vm/src/plugin/runner/ccu_executor/CMakeLists.txt+3-0
@@ -42,3 +42,6 @@ target_link_libraries(ccu_executor PRIVATE
42 log42 log
43 rt43 rt
44)44)
45+ 
46+# 日志染色: 模块 tag
47+target_compile_definitions(ccu_executor PRIVATE HCCL_VM_MODULE="RUNNER")
Mtest/hccl_vm/src/plugin/runner/ccu_executor/ccu_executor_base.cc+9-9文件内容审核中,请稍后刷新重试
Mtest/hccl_vm/src/plugin/runner/ccu_executor/ccu_resource_manager.cc+36-36
@@ -26,7 +26,7 @@ using namespace hcomm::CcuRep;
26 26 
27void CcuResourceManager::Reset()27void CcuResourceManager::Reset()
28{28{
29- HCCL_VM_DEBUG("[CcuResourceManager][Reset] Clearing all CCU resource state (KE/XN/MS/simulators)");29+ HCCL_VM_DEBUG("Clearing all CCU resource state (KE/XN/MS/simulators)");
30 for (size_t rankId = 0; rankId < ccuResData_.v1Res.size(); rankId++) {30 for (size_t rankId = 0; rankId < ccuResData_.v1Res.size(); rankId++) {
31 if (ccuResData_.v1Res[rankId] != nullptr) {31 if (ccuResData_.v1Res[rankId] != nullptr) {
32 ccuResData_.v1Res[rankId]->Reset();32 ccuResData_.v1Res[rankId]->Reset();
@@ -43,7 +43,7 @@ void CcuResourceManager::Init(int rankId, int rankSize, RunnerCcuVersion version
43 }43 }
44 ccuResData_.v1Res[rankId] = std::make_unique<CcuResourceV1>(rankId, rankSize);44 ccuResData_.v1Res[rankId] = std::make_unique<CcuResourceV1>(rankId, rankSize);
45 } else {45 } else {
46- HCCL_VM_ERROR("[CcuResourceManager][Init] ccu version {} not supported", static_cast<int>(ccuResData_.version));46+ HCCL_VM_ERROR("ccu version {} not supported", static_cast<int>(ccuResData_.version));
47 }47 }
48 ccuResourceBaseAddr_ = ccuResourceBaseAddr;48 ccuResourceBaseAddr_ = ccuResourceBaseAddr;
49}49}
@@ -54,7 +54,7 @@ void CcuResourceManager::InitInstrInfo(int rankId, int dieId, const CcuInstrData
54 if (version == RunnerCcuVersion::CCU_V1) {54 if (version == RunnerCcuVersion::CCU_V1) {
55 ccuResData_.v1Res[rankId]->instrSpace_[dieId] = ccuInstrInfo;55 ccuResData_.v1Res[rankId]->instrSpace_[dieId] = ccuInstrInfo;
56 } else {56 } else {
57- HCCL_VM_ERROR("[CcuResourceManager][InitInstrInfo] ccu version {} not supported", static_cast<int>(version));57+ HCCL_VM_ERROR("ccu version {} not supported", static_cast<int>(version));
58 }58 }
59}59}
60 60 
@@ -64,7 +64,7 @@ void CcuResourceManager::InitChannelInfo(int rankId, const RankChannelInfo &chan
64 if (version == RunnerCcuVersion::CCU_V1) {64 if (version == RunnerCcuVersion::CCU_V1) {
65 ccuResData_.v1Res[rankId]->channelId2RmtRankMap_ = channelInfo;65 ccuResData_.v1Res[rankId]->channelId2RmtRankMap_ = channelInfo;
66 } else {66 } else {
67- HCCL_VM_ERROR("[CcuResourceManager][InitInstrInfo] ccu version {} not supported", static_cast<int>(version));67+ HCCL_VM_ERROR("ccu version {} not supported", static_cast<int>(version));
68 }68 }
69}69}
70 70 
@@ -72,14 +72,14 @@ void CcuResourceManager::AddTaskInfo(int rankId, const HcclTaskMetaData &task)
72{72{
73 auto dieId = task.taskData.ccu.dieId;73 auto dieId = task.taskData.ccu.dieId;
74 if (dieId >= HcclSim::DIE_NUM) {74 if (dieId >= HcclSim::DIE_NUM) {
75- HCCL_VM_ERROR("[CcuResourceManager][AddTaskInfo] invalid dieId[{}].", dieId);75+ HCCL_VM_ERROR("invalid dieId[{}].", dieId);
76 return;76 return;
77 }77 }
78 auto version = ccuResData_.version;78 auto version = ccuResData_.version;
79 if (version == RunnerCcuVersion::CCU_V1) {79 if (version == RunnerCcuVersion::CCU_V1) {
80 ccuResData_.v1Res[rankId]->ccuTaskInfos_[dieId] = task.taskData.ccu;80 ccuResData_.v1Res[rankId]->ccuTaskInfos_[dieId] = task.taskData.ccu;
81 } else {81 } else {
82- HCCL_VM_ERROR("[CcuResourceManager][AddTaskInfo] ccu version {} not supported", static_cast<int>(version));82+ HCCL_VM_ERROR("ccu version {} not supported", static_cast<int>(version));
83 }83 }
84}84}
85 85 
@@ -89,7 +89,7 @@ uint64_t CcuResourceManager::GetSqeArgValue(int rankId, int dieId, uint16_t argI
89 if (version == RunnerCcuVersion::CCU_V1) {89 if (version == RunnerCcuVersion::CCU_V1) {
90 return ccuResData_.v1Res[rankId]->ccuTaskInfos_[dieId].args[argId];90 return ccuResData_.v1Res[rankId]->ccuTaskInfos_[dieId].args[argId];
91 } else {91 } else {
92- HCCL_VM_ERROR("[CcuResourceManager][GetSqeArgValue] ccu version {} not supported", static_cast<int>(version));92+ HCCL_VM_ERROR("ccu version {} not supported", static_cast<int>(version));
93 return U64_INVALID;93 return U64_INVALID;
94 }94 }
95}95}
@@ -100,7 +100,7 @@ uint64_t CcuResourceManager::GetXnValue(int rankId, int dieId, uint16_t xnId) co
100 if (version == RunnerCcuVersion::CCU_V1) {100 if (version == RunnerCcuVersion::CCU_V1) {
101 return ccuResData_.v1Res[rankId]->xn_[dieId][xnId];101 return ccuResData_.v1Res[rankId]->xn_[dieId][xnId];
102 } else {102 } else {
103- HCCL_VM_ERROR("[CcuResourceManager][GetXnValue] ccu version {} not supported", static_cast<int>(version));103+ HCCL_VM_ERROR("ccu version {} not supported", static_cast<int>(version));
104 return U64_INVALID;104 return U64_INVALID;
105 }105 }
106}106}
@@ -111,7 +111,7 @@ uint64_t CcuResourceManager::GetGsaValue(int rankId, int dieId, uint16_t gsaId)
111 if (version == RunnerCcuVersion::CCU_V1) {111 if (version == RunnerCcuVersion::CCU_V1) {
112 return ccuResData_.v1Res[rankId]->gsa_[dieId][gsaId];112 return ccuResData_.v1Res[rankId]->gsa_[dieId][gsaId];
113 } else {113 } else {
114- HCCL_VM_ERROR("[CcuResourceManager][GetGsaValue] ccu version {} not supported", static_cast<int>(version));114+ HCCL_VM_ERROR("ccu version {} not supported", static_cast<int>(version));
115 return U64_INVALID;115 return U64_INVALID;
116 }116 }
117}117}
@@ -122,7 +122,7 @@ uint16_t CcuResourceManager::GetCkeValue(int rankId, int dieId, uint16_t ckeId)
122 if (version == RunnerCcuVersion::CCU_V1) {122 if (version == RunnerCcuVersion::CCU_V1) {
123 return ccuResData_.v1Res[rankId]->cke_[dieId][ckeId];123 return ccuResData_.v1Res[rankId]->cke_[dieId][ckeId];
124 } else {124 } else {
125- HCCL_VM_ERROR("[CcuResourceManager][GetCkeValue] ccu version {} not supported", static_cast<int>(version));125+ HCCL_VM_ERROR("ccu version {} not supported", static_cast<int>(version));
126 return U16_INVALID;126 return U16_INVALID;
127 }127 }
128}128}
@@ -133,7 +133,7 @@ std::pair<int, int> CcuResourceManager::GetRmtCcu(int rankId, int dieId, uint16_
133 int rmtDie = S32_INVALID;133 int rmtDie = S32_INVALID;
134 134 
135 if (channelId >= SimCcuV1::MAX_CCU_CHANNEL_NUM) {135 if (channelId >= SimCcuV1::MAX_CCU_CHANNEL_NUM) {
136- HCCL_VM_ERROR("[CcuResourceManager][GetRmtCcu] invalid channelId[{}], max={}", channelId, SimCcuV1::MAX_CCU_CHANNEL_NUM);136+ HCCL_VM_ERROR("invalid channelId[{}], max={}", channelId, SimCcuV1::MAX_CCU_CHANNEL_NUM);
137 return std::make_pair(S32_INVALID, S32_INVALID);137 return std::make_pair(S32_INVALID, S32_INVALID);
138 }138 }
139 139 
@@ -142,12 +142,12 @@ std::pair<int, int> CcuResourceManager::GetRmtCcu(int rankId, int dieId, uint16_
142 rmtRank = ccuResData_.v1Res[rankId]->channelId2RmtRankMap_[dieId][channelId].rankId;142 rmtRank = ccuResData_.v1Res[rankId]->channelId2RmtRankMap_[dieId][channelId].rankId;
143 rmtDie = ccuResData_.v1Res[rankId]->channelId2RmtRankMap_[dieId][channelId].dieId;143 rmtDie = ccuResData_.v1Res[rankId]->channelId2RmtRankMap_[dieId][channelId].dieId;
144 } else {144 } else {
145- HCCL_VM_ERROR("[CcuResourceManager][GetRmtCcu] ccu version {} not supported", static_cast<int>(version));145+ HCCL_VM_ERROR("ccu version {} not supported", static_cast<int>(version));
146 return std::make_pair(S32_INVALID, S32_INVALID);146 return std::make_pair(S32_INVALID, S32_INVALID);
147 }147 }
148 148 
149 if (dieId >= HcclSim::DIE_NUM || rmtRank == S32_INVALID || rmtDie == S32_INVALID) {149 if (dieId >= HcclSim::DIE_NUM || rmtRank == S32_INVALID || rmtDie == S32_INVALID) {
150- HCCL_VM_ERROR("[CcuResourceManager][GetRmtCcu] get invalid value, dieId{}, rmtRank{}, rmtDie{}", dieId, rmtRank, rmtDie);150+ HCCL_VM_ERROR("get invalid value, dieId{}, rmtRank{}, rmtDie{}", dieId, rmtRank, rmtDie);
151 return std::make_pair(S32_INVALID, S32_INVALID);151 return std::make_pair(S32_INVALID, S32_INVALID);
152 }152 }
153 153 
@@ -160,7 +160,7 @@ void CcuResourceManager::UpdateXnValue(int rankId, int dieId, uint16_t xnId, uin
160 if (version == RunnerCcuVersion::CCU_V1) {160 if (version == RunnerCcuVersion::CCU_V1) {
161 ccuResData_.v1Res[rankId]->xn_[dieId][xnId] = value;161 ccuResData_.v1Res[rankId]->xn_[dieId][xnId] = value;
162 } else {162 } else {
163- HCCL_VM_ERROR("[CcuResourceManager][UpdateXnValue] ccu version {} not supported", static_cast<int>(version));163+ HCCL_VM_ERROR("ccu version {} not supported", static_cast<int>(version));
164 }164 }
165}165}
166 166 
@@ -170,7 +170,7 @@ void CcuResourceManager::UpdateGsaValue(int rankId, int dieId, uint16_t gsaId, u
170 if (version == RunnerCcuVersion::CCU_V1) {170 if (version == RunnerCcuVersion::CCU_V1) {
171 ccuResData_.v1Res[rankId]->gsa_[dieId][gsaId] = value;171 ccuResData_.v1Res[rankId]->gsa_[dieId][gsaId] = value;
172 } else {172 } else {
173- HCCL_VM_ERROR("[CcuResourceManager][UpdateGsaValue] ccu version {} not supported", static_cast<int>(version));173+ HCCL_VM_ERROR("ccu version {} not supported", static_cast<int>(version));
174 }174 }
175}175}
176 176 
@@ -180,34 +180,34 @@ void CcuResourceManager::UpdateCkeValue(int rankId, int dieId, uint16_t ckeId, u
180 if (version == RunnerCcuVersion::CCU_V1) {180 if (version == RunnerCcuVersion::CCU_V1) {
181 ccuResData_.v1Res[rankId]->cke_[dieId][ckeId] = value;181 ccuResData_.v1Res[rankId]->cke_[dieId][ckeId] = value;
182 } else {182 } else {
183- HCCL_VM_ERROR("[CcuResourceManager][UpdateCkeValue] ccu version {} not supported", static_cast<int>(version));183+ HCCL_VM_ERROR("ccu version {} not supported", static_cast<int>(version));
184 }184 }
185}185}
186 186 
187void CcuResourceManager::TransMemToMem(void *srcBuf, void *dstBuf, uint64_t length, bool reduceEn, uint16_t reduceOp, uint16_t dataType)187void CcuResourceManager::TransMemToMem(void *srcBuf, void *dstBuf, uint64_t length, bool reduceEn, uint16_t reduceOp, uint16_t dataType)
188{188{
189- HCCL_VM_DEBUG("[CcuResourceManager][TransMemToMem] srcBuf={}, dstBuf={}, length={}", srcBuf, dstBuf, length);189+ HCCL_VM_DEBUG("srcBuf={}, dstBuf={}, length={}", srcBuf, dstBuf, length);
190 if (srcBuf == nullptr) {190 if (srcBuf == nullptr) {
191- HCCL_VM_ERROR("[CcuResourceManager][TransMemToMem] srcBuf invalid...");191+ HCCL_VM_ERROR("srcBuf invalid...");
192 return;192 return;
193 }193 }
194 194 
195 if (dstBuf == nullptr) {195 if (dstBuf == nullptr) {
196- HCCL_VM_ERROR("[CcuResourceManager][TransMemToMem] dstBuf invalid...");196+ HCCL_VM_ERROR("dstBuf invalid...");
197 return;197 return;
198 }198 }
199 199 
200 sim::PhyMemBlock srcPhyMem{};200 sim::PhyMemBlock srcPhyMem{};
201 auto srcAddr = sim::AcquireDevPtrInNoHostProcess((void*)srcBuf, srcPhyMem);201 auto srcAddr = sim::AcquireDevPtrInNoHostProcess((void*)srcBuf, srcPhyMem);
202 if (srcAddr == nullptr) {202 if (srcAddr == nullptr) {
203- HCCL_VM_ERROR("[TransMemToMem] 无法获取srcBuf的设备地址(addr= {:x})!", reinterpret_cast<uintptr_t>(srcBuf));203+ HCCL_VM_ERROR("无法获取srcBuf的设备地址(addr= {:x})!", reinterpret_cast<uintptr_t>(srcBuf));
204 return;204 return;
205 }205 }
206 206 
207 sim::PhyMemBlock dstPhyMem{};207 sim::PhyMemBlock dstPhyMem{};
208 auto dstAddr = sim::AcquireDevPtrInNoHostProcess((void*)dstBuf, dstPhyMem);208 auto dstAddr = sim::AcquireDevPtrInNoHostProcess((void*)dstBuf, dstPhyMem);
209 if (dstAddr == nullptr) {209 if (dstAddr == nullptr) {
210- HCCL_VM_ERROR("[TransMemToMem] 无法获取dstBuf的设备地址(addr= {:x})!", reinterpret_cast<uintptr_t>(dstBuf));210+ HCCL_VM_ERROR("无法获取dstBuf的设备地址(addr= {:x})!", reinterpret_cast<uintptr_t>(dstBuf));
211 return;211 return;
212 }212 }
213 213 
@@ -229,14 +229,14 @@ void CcuResourceManager::TransMSToMS(int srcRank, int srcDie, int dstRank, int d
229bool CcuResourceManager::TransMSToMem(int rankId, int dieId, uint16_t msId, void *buf, uint16_t length)229bool CcuResourceManager::TransMSToMem(int rankId, int dieId, uint16_t msId, void *buf, uint16_t length)
230{230{
231 if (buf == nullptr) {231 if (buf == nullptr) {
232- HCCL_VM_ERROR("[CcuResourceManager][TransMSToMem] param invalid...");232+ HCCL_VM_ERROR("param invalid...");
233 return false;233 return false;
234 }234 }
235 235 
236 sim::PhyMemBlock dstPhyMem{};236 sim::PhyMemBlock dstPhyMem{};
237 auto dstAddr = sim::AcquireDevPtrInNoHostProcess((void*)buf, dstPhyMem);237 auto dstAddr = sim::AcquireDevPtrInNoHostProcess((void*)buf, dstPhyMem);
238 if (dstAddr == nullptr) {238 if (dstAddr == nullptr) {
239- HCCL_VM_ERROR("[TransMSToMem] 无法获取buf的设备地址(addr= {:x})!", reinterpret_cast<uintptr_t>(buf));239+ HCCL_VM_ERROR("无法获取buf的设备地址(addr= {:x})!", reinterpret_cast<uintptr_t>(buf));
240 return false;240 return false;
241 }241 }
242 242 
@@ -251,7 +251,7 @@ bool CcuResourceManager::TransMemToXn(int rankId, int dieId, uint16_t xnId, uint
251 sim::PhyMemBlock srcPhyMem{};251 sim::PhyMemBlock srcPhyMem{};
252 auto srcAddr = sim::AcquireDevPtrInNoHostProcess((void*)buf, srcPhyMem);252 auto srcAddr = sim::AcquireDevPtrInNoHostProcess((void*)buf, srcPhyMem);
253 if (srcAddr == nullptr) {253 if (srcAddr == nullptr) {
254- HCCL_VM_ERROR("[TransMemToMS] 无法获取buf的设备地址(addr= {:x})!", buf);254+ HCCL_VM_ERROR("无法获取buf的设备地址(addr= {:x})!", buf);
255 return false;255 return false;
256 }256 }
257 memcpy(GetXnAddr(rankId, dieId, xnId), srcAddr, length);257 memcpy(GetXnAddr(rankId, dieId, xnId), srcAddr, length);
@@ -265,7 +265,7 @@ bool CcuResourceManager::TransXnToMem(int rankId, int dieId, uint16_t xnId, uint
265 sim::PhyMemBlock dstPhyMem{};265 sim::PhyMemBlock dstPhyMem{};
266 auto dstAddr = sim::AcquireDevPtrInNoHostProcess((void*)buf, dstPhyMem);266 auto dstAddr = sim::AcquireDevPtrInNoHostProcess((void*)buf, dstPhyMem);
267 if (dstAddr == nullptr) {267 if (dstAddr == nullptr) {
268- HCCL_VM_ERROR("[TransXnToMem] 无法获取buf的设备地址(addr= {:x})!", buf);268+ HCCL_VM_ERROR("无法获取buf的设备地址(addr= {:x})!", buf);
269 return false;269 return false;
270 }270 }
271 memcpy(dstAddr, GetXnAddr(rankId, dieId, xnId), length);271 memcpy(dstAddr, GetXnAddr(rankId, dieId, xnId), length);
@@ -277,14 +277,14 @@ bool CcuResourceManager::TransXnToMem(int rankId, int dieId, uint16_t xnId, uint
277bool CcuResourceManager::TransMemToMS(int rankId, int dieId, uint16_t msId, void *buf, uint16_t length)277bool CcuResourceManager::TransMemToMS(int rankId, int dieId, uint16_t msId, void *buf, uint16_t length)
278{278{
279 if (buf == nullptr) {279 if (buf == nullptr) {
280- HCCL_VM_ERROR("[CcuResourceManager][TransMemToMS] param invalid...");280+ HCCL_VM_ERROR("param invalid...");
281 return false;281 return false;
282 }282 }
283 283 
284 sim::PhyMemBlock srcPhyMem{};284 sim::PhyMemBlock srcPhyMem{};
285 auto srcAddr = sim::AcquireDevPtrInNoHostProcess((void*)buf, srcPhyMem);285 auto srcAddr = sim::AcquireDevPtrInNoHostProcess((void*)buf, srcPhyMem);
286 if (srcAddr == nullptr) {286 if (srcAddr == nullptr) {
287- HCCL_VM_ERROR("[TransMemToMS] 无法获取buf的设备地址(addr= {:x})!", reinterpret_cast<uintptr_t>(buf));287+ HCCL_VM_ERROR("无法获取buf的设备地址(addr= {:x})!", reinterpret_cast<uintptr_t>(buf));
288 return false;288 return false;
289 }289 }
290 memcpy(GetMsAddr(rankId, dieId, msId), srcAddr, length);290 memcpy(GetMsAddr(rankId, dieId, msId), srcAddr, length);
@@ -300,7 +300,7 @@ char *CcuResourceManager::GetMsAddr(int rankId, int dieId, uint16_t msId) const
300 if (version == RunnerCcuVersion::CCU_V1) {300 if (version == RunnerCcuVersion::CCU_V1) {
301 return (ccuResData_.v1Res[rankId]->ms_[dieId].data() + offset);301 return (ccuResData_.v1Res[rankId]->ms_[dieId].data() + offset);
302 } else {302 } else {
303- HCCL_VM_ERROR("[CcuResourceManager][GetMsAddr] ccu version {} not supported", static_cast<int>(version));303+ HCCL_VM_ERROR("ccu version {} not supported", static_cast<int>(version));
304 return nullptr;304 return nullptr;
305 }305 }
306}306}
@@ -312,7 +312,7 @@ uint64_t *CcuResourceManager::GetXnAddr(int rankId, int dieId, uint16_t xnId) co
312 if (version == RunnerCcuVersion::CCU_V1) {312 if (version == RunnerCcuVersion::CCU_V1) {
313 return (ccuResData_.v1Res[rankId]->xn_[dieId].data() + offset);313 return (ccuResData_.v1Res[rankId]->xn_[dieId].data() + offset);
314 } else {314 } else {
315- HCCL_VM_ERROR("[CcuResourceManager][GetXnAddr] ccu version {} not supported", static_cast<int>(version));315+ HCCL_VM_ERROR("ccu version {} not supported", static_cast<int>(version));
316 return nullptr;316 return nullptr;
317 }317 }
318}318}
@@ -323,7 +323,7 @@ uint16_t CcuResourceManager::GetInstrCnt(int rankId, int dieId) const
323 if (version == RunnerCcuVersion::CCU_V1) {323 if (version == RunnerCcuVersion::CCU_V1) {
324 return ccuResData_.v1Res[rankId]->instrSpace_[dieId].instrCnt;324 return ccuResData_.v1Res[rankId]->instrSpace_[dieId].instrCnt;
325 } else {325 } else {
326- HCCL_VM_ERROR("[CcuResourceManager][GetInstrCnt] ccu version {} not supported", static_cast<int>(version));326+ HCCL_VM_ERROR("ccu version {} not supported", static_cast<int>(version));
327 return U16_INVALID;327 return U16_INVALID;
328 }328 }
329}329}
@@ -334,7 +334,7 @@ std::vector<hcomm::CcuRep::CcuInstr> CcuResourceManager::GetInstrData(int rankId
334 if (version == RunnerCcuVersion::CCU_V1) {334 if (version == RunnerCcuVersion::CCU_V1) {
335 return ccuResData_.v1Res[rankId]->instrSpace_[dieId].instrData;335 return ccuResData_.v1Res[rankId]->instrSpace_[dieId].instrData;
336 } else {336 } else {
337- HCCL_VM_ERROR("[CcuResourceManager][GetInstrData] ccu version {} not supported", static_cast<int>(version));337+ HCCL_VM_ERROR("ccu version {} not supported", static_cast<int>(version));
338 return {};338 return {};
339 }339 }
340}340}
@@ -346,7 +346,7 @@ std::string CcuResourceManager::GetInstrDescribe(int rankId, int dieId, int inst
346 if (version == RunnerCcuVersion::CCU_V1) {346 if (version == RunnerCcuVersion::CCU_V1) {
347 // return hcomm::CcuRep::ParseInstr(&(ccuResData_.v1Res[rankId]->instrSpace_[dieId].instrData[instrId]));347 // return hcomm::CcuRep::ParseInstr(&(ccuResData_.v1Res[rankId]->instrSpace_[dieId].instrData[instrId]));
348 } else {348 } else {
349- HCCL_VM_ERROR("[CcuResourceManager][GetInstrDescribe] ccu version {} not supported", static_cast<int>(version));349+ HCCL_VM_ERROR("ccu version {} not supported", static_cast<int>(version));
350 return "";350 return "";
351 }351 }
352#endif352#endif
@@ -365,7 +365,7 @@ std::shared_ptr<CcuSimulator> CcuResourceManager::InitSimulator(int rankId, int
365 return ccuResData_.v1Res[rankId]->simulators_[dieId]; 365 return ccuResData_.v1Res[rankId]->simulators_[dieId];
366 }366 }
367 } else {367 } else {
368- HCCL_VM_ERROR("[CcuResourceManager][GetInstrData] ccu version {} not supported", static_cast<int>(version));368+ HCCL_VM_ERROR("ccu version {} not supported", static_cast<int>(version));
369 return nullptr;369 return nullptr;
370 }370 }
371}371}
@@ -440,7 +440,7 @@ void CcuResourceManager::DumpCcuCkeResouceInfo(int rankId) const
440void CcuResourceManager::DumpCcuChannelResouceInfo(int rankId) const440void CcuResourceManager::DumpCcuChannelResouceInfo(int rankId) const
441{441{
442 for (int dieId = 0; dieId < HcclSim::DIE_NUM; dieId++) {442 for (int dieId = 0; dieId < HcclSim::DIE_NUM; dieId++) {
443- HCCL_VM_DEBUG("\n==========================All CCU CHANNEL Resouce Info Start: rank[{}], dieId[{}]===========================", rankId, dieId);443+ HCCL_VM_DEBUG("==========================All CCU CHANNEL Resouce Info Start: rank[{}], dieId[{}]===========================", rankId, dieId);
444 HCCL_VM_DEBUG("-------------------------DieId[{}] CHANNEL Resouce Info Start-------------------------", dieId);444 HCCL_VM_DEBUG("-------------------------DieId[{}] CHANNEL Resouce Info Start-------------------------", dieId);
445 for (uint32_t chId = 0; chId < SimCcuV1::MAX_CCU_CHANNEL_NUM; chId++) {445 for (uint32_t chId = 0; chId < SimCcuV1::MAX_CCU_CHANNEL_NUM; chId++) {
446 auto rmtCcu = GetRmtCcu(rankId, dieId, chId);446 auto rmtCcu = GetRmtCcu(rankId, dieId, chId);
@@ -449,7 +449,7 @@ void CcuResourceManager::DumpCcuChannelResouceInfo(int rankId) const
449 }449 }
450 }450 }
451 HCCL_VM_DEBUG("-------------------------DieId[{}] CHANNEL Resouce Info End-------------------------", dieId);451 HCCL_VM_DEBUG("-------------------------DieId[{}] CHANNEL Resouce Info End-------------------------", dieId);
452- HCCL_VM_DEBUG("==========================All CCU CHANNEL Resouce Info End: rank[{}], dieId[{}]===========================\n", rankId, dieId);452+ HCCL_VM_DEBUG("==========================All CCU CHANNEL Resouce Info End: rank[{}], dieId[{}]===========================", rankId, dieId);
453 }453 }
454}454}
455 455 
@@ -465,7 +465,7 @@ void CcuResourceManager::DumpChannelId2RmtRank(int rankId, int dieId) const
465 if (rmtCcu.first == INT32_MAX || rmtCcu.second == INT32_MAX) {465 if (rmtCcu.first == INT32_MAX || rmtCcu.second == INT32_MAX) {
466 continue;466 continue;
467 }467 }
468- HCCL_VM_DEBUG("[CcuResourceManager][DumpChannelId2RmtRank] channelId[{}], ccu[{}:{} --> {}:{}]", i, rankId, dieId, rmtCcu.first, rmtCcu.second);468+ HCCL_VM_DEBUG("channelId[{}], ccu[{}:{} --> {}:{}]", i, rankId, dieId, rmtCcu.first, rmtCcu.second);
469 }469 }
470}470}
471 471 
Mtest/hccl_vm/src/plugin/runner/ccu_executor/ccu_simulator.cc+9-9
@@ -28,13 +28,13 @@ void CcuSimulator::InitLoopGroupInfo(uint16_t startLoopId, uint64_t offsetCfg, u
28 loopGroupInfo_.loopNum_ = (repeatCfg >> 41) & 0x7F; // 0x7F: 取loop指令个数,即[47:41]位28 loopGroupInfo_.loopNum_ = (repeatCfg >> 41) & 0x7F; // 0x7F: 取loop指令个数,即[47:41]位
29 loopGroupInfo_.loopOffset_ = (repeatCfg >> 48) & 0x7F; // 0x7F: 取loop偏移,即[54:48]位29 loopGroupInfo_.loopOffset_ = (repeatCfg >> 48) & 0x7F; // 0x7F: 取loop偏移,即[54:48]位
30 loopGroupInfo_.loopExtendNum_ = (repeatCfg >> 55) & 0x7F; // 0x7F: 取loop展开次数,即[61:55]位30 loopGroupInfo_.loopExtendNum_ = (repeatCfg >> 55) & 0x7F; // 0x7F: 取loop展开次数,即[61:55]位
31- HCCL_VM_DEBUG("[CcuSimulator][InitLoopGroupInfo] locCcu[{}:{}], curInstrId_=[{}], loopNum=[{}], loopOffset=[{}], loopExtendNum=[{}]",31+ HCCL_VM_DEBUG("locCcu[{}:{}], curInstrId_=[{}], loopNum=[{}], loopOffset=[{}], loopExtendNum=[{}]",
32 rankId_, dieId_, curInstrId_, loopGroupInfo_.loopNum_, loopGroupInfo_.loopOffset_, loopGroupInfo_.loopExtendNum_);32 rankId_, dieId_, curInstrId_, loopGroupInfo_.loopNum_, loopGroupInfo_.loopOffset_, loopGroupInfo_.loopExtendNum_);
33 33 
34 loopGroupInfo_.ckeOffset_ = offsetCfg & 0x3FF; // 0x3FF: 取低[9:0]位34 loopGroupInfo_.ckeOffset_ = offsetCfg & 0x3FF; // 0x3FF: 取低[9:0]位
35 loopGroupInfo_.msOffset_ = (offsetCfg >> 10) & 0x7FF; // 0x7FF: 取低[20:10]位35 loopGroupInfo_.msOffset_ = (offsetCfg >> 10) & 0x7FF; // 0x7FF: 取低[20:10]位
36 loopGroupInfo_.gsaOffset_ = (offsetCfg >> 21) & 0xFFFFFFFF; // 0xFFFFFFFF: 取低[52:21]位36 loopGroupInfo_.gsaOffset_ = (offsetCfg >> 21) & 0xFFFFFFFF; // 0xFFFFFFFF: 取低[52:21]位
37- HCCL_VM_DEBUG("[CcuSimulator][InitLoopGroupInfo] locCcu[{}:{}], ckeOffset=[{}], msOffset=[{}], gsaOffset=[{}]",37+ HCCL_VM_DEBUG("locCcu[{}:{}], ckeOffset=[{}], msOffset=[{}], gsaOffset=[{}]",
38 rankId_, dieId_, loopGroupInfo_.ckeOffset_, loopGroupInfo_.msOffset_, loopGroupInfo_.gsaOffset_);38 rankId_, dieId_, loopGroupInfo_.ckeOffset_, loopGroupInfo_.msOffset_, loopGroupInfo_.gsaOffset_);
39}39}
40 40 
@@ -47,7 +47,7 @@ void CcuSimulator::InitLoopInfo(uint16_t startInstrId, uint16_t endInstrId, uint
47 loopGroupInfo_.loopStatus_.curLoopRound = 0;47 loopGroupInfo_.loopStatus_.curLoopRound = 0;
48 loopGroupInfo_.loopStatus_.loopGsaIterStep = addrStep;48 loopGroupInfo_.loopStatus_.loopGsaIterStep = addrStep;
49 49 
50- HCCL_VM_DEBUG("[CcuSimulator][InitLoopInfo] locCcu[{}:{}], loopStartInstrId=[{}], loopCurInstrId=[{}], "50+ HCCL_VM_DEBUG("locCcu[{}:{}], loopStartInstrId=[{}], loopCurInstrId=[{}], "
51 "loopEndInstrId=[{}], loopExecCount=[{}], loopGsaIterStep=[{}]",51 "loopEndInstrId=[{}], loopExecCount=[{}], loopGsaIterStep=[{}]",
52 rankId_, dieId_, startInstrId, startInstrId, endInstrId, execCount, addrStep);52 rankId_, dieId_, startInstrId, startInstrId, endInstrId, execCount, addrStep);
53}53}
@@ -192,16 +192,16 @@ bool CcuSimulator::ExecuteInstr(uint16_t curInstrId)
192{192{
193 auto &ccuResMgr = CcuResourceManager::GetInstance();193 auto &ccuResMgr = CcuResourceManager::GetInstance();
194 auto instrData = ccuResMgr.GetInstrData(rankId_, dieId_);194 auto instrData = ccuResMgr.GetInstrData(rankId_, dieId_);
195- HCCL_VM_DEBUG("[CcuSimulator][ExecuteInstr] locCcu[{}:{}], current instr[{}], state=[{}]", rankId_, dieId_, curInstrId, static_cast<int>(state_));195+ HCCL_VM_DEBUG("locCcu[{}:{}], current instr[{}], state=[{}]", rankId_, dieId_, curInstrId, static_cast<int>(state_));
196 if (curInstrId >= endInstrId_) {196 if (curInstrId >= endInstrId_) {
197 state_ = CcuExecState::EXEC_FAIL;197 state_ = CcuExecState::EXEC_FAIL;
198- HCCL_VM_ERROR("[CcuSimulator][ExecuteInstr] locCcu[{}:{}], Invalid curInstrId_[{}], endInstrId[{}]", rankId_, dieId_, curInstrId, endInstrId_);198+ HCCL_VM_ERROR("locCcu[{}:{}], Invalid curInstrId_[{}], endInstrId[{}]", rankId_, dieId_, curInstrId, endInstrId_);
199 return false;199 return false;
200 }200 }
201 auto executor = CcuExecutorFactory::MakeCcuExecutorInstance(version_, instrData[curInstrId].header.header, 0, rankId_, dieId_, instrData[curInstrId], this);201 auto executor = CcuExecutorFactory::MakeCcuExecutorInstance(version_, instrData[curInstrId].header.header, 0, rankId_, dieId_, instrData[curInstrId], this);
202 if (executor == nullptr) {202 if (executor == nullptr) {
203 UpdateLoopStatus();203 UpdateLoopStatus();
204- HCCL_VM_DEBUG("[CcuSimulator][ExecuteInstr] locCcu[{}:{}], curInstrId_[{}] executor is null.", rankId_, dieId_, curInstrId);204+ HCCL_VM_DEBUG("locCcu[{}:{}], curInstrId_[{}] executor is null.", rankId_, dieId_, curInstrId);
205 return true; // 让还未添加的Executor先跑下去,看下是否能从start->loop->end205 return true; // 让还未添加的Executor先跑下去,看下是否能从start->loop->end
206 }206 }
207 executor->Parser();207 executor->Parser();
@@ -210,7 +210,7 @@ bool CcuSimulator::ExecuteInstr(uint16_t curInstrId)
210 return false;210 return false;
211 }211 }
212 212 
213- HCCL_VM_DEBUG("[CcuSimulator][ExecuteInstr] locCcu[{}:{}], end instr[{}], {}", rankId_, dieId_, curInstrId, executor->Describe());213+ HCCL_VM_DEBUG("locCcu[{}:{}], end instr[{}], {}", rankId_, dieId_, curInstrId, executor->Describe());
214 return true;214 return true;
215}215}
216 216 
@@ -219,7 +219,7 @@ bool CcuSimulator::ExecuteLoop()
219{219{
220 auto instrCnt = loopGroupInfo_.loopStatus_.loopEndInstrId - loopGroupInfo_.loopStatus_.loopStartInstrId + 1;220 auto instrCnt = loopGroupInfo_.loopStatus_.loopEndInstrId - loopGroupInfo_.loopStatus_.loopStartInstrId + 1;
221 for (uint32_t i = 0; i < loopGroupInfo_.loopStatus_.loopExecCount; i++) {221 for (uint32_t i = 0; i < loopGroupInfo_.loopStatus_.loopExecCount; i++) {
222- HCCL_VM_DEBUG("[CcuSimulator][ExecuteLoop] locCcu[{}:{}], loopStartInstrId=[{}], loopCurInstrId=[{}], instrCnt=[{}], loopRound=[{}].",222+ HCCL_VM_DEBUG("locCcu[{}:{}], loopStartInstrId=[{}], loopCurInstrId=[{}], instrCnt=[{}], loopRound=[{}].",
223 rankId_, dieId_, loopGroupInfo_.loopStatus_.loopStartInstrId, curInstrId_, instrCnt, i);223 rankId_, dieId_, loopGroupInfo_.loopStatus_.loopStartInstrId, curInstrId_, instrCnt, i);
224 auto simulator = std::make_unique<CcuSimulator>(224 auto simulator = std::make_unique<CcuSimulator>(
225 rankId_, dieId_, loopGroupInfo_.loopStatus_.loopStartInstrId, loopGroupInfo_.loopStatus_.loopEndInstrId + 1, instrCnt, version_);225 rankId_, dieId_, loopGroupInfo_.loopStatus_.loopStartInstrId, loopGroupInfo_.loopStatus_.loopEndInstrId + 1, instrCnt, version_);
@@ -234,7 +234,7 @@ bool CcuSimulator::ExecuteLoop()
234// loopGroup内所有展开的loop执行,后续改为多线程并行执行234// loopGroup内所有展开的loop执行,后续改为多线程并行执行
235bool CcuSimulator::ExecuteLoopGroup()235bool CcuSimulator::ExecuteLoopGroup()
236{236{
237- HCCL_VM_DEBUG("[CcuSimulator][ExecuteLoopGroup] locCcu[{}:{}], lopNum=[{}], loopExtendNum=[{}], loopOffset=[{}].",237+ HCCL_VM_DEBUG("locCcu[{}:{}], lopNum=[{}], loopExtendNum=[{}], loopOffset=[{}].",
238 rankId_, dieId_, loopGroupInfo_.loopNum_, loopGroupInfo_.loopExtendNum_, loopGroupInfo_.loopOffset_);238 rankId_, dieId_, loopGroupInfo_.loopNum_, loopGroupInfo_.loopExtendNum_, loopGroupInfo_.loopOffset_);
239 if (loopGroupInfo_.loopNum_ == 0) {239 if (loopGroupInfo_.loopNum_ == 0) {
240 return true;240 return true;
Mtest/hccl_vm/src/plugin/runner/ccu_executor/load_executor/load_imd_to_xn_executor.cc+1-1
@@ -39,7 +39,7 @@ void LoadImdToXnExecutor::Run()
39{39{
40 auto &ccuResMgr = CcuResourceManager::GetInstance();40 auto &ccuResMgr = CcuResourceManager::GetInstance();
41 ccuResMgr.UpdateXnValue(rankId_, dieId_, xnId_, immediate_);41 ccuResMgr.UpdateXnValue(rankId_, dieId_, xnId_, immediate_);
42- HCCL_VM_DEBUG("[LoadImdToXnExecutor] Load immediate: locCcu[{}:{}], XnId=[{}], immediate=[{}]",42+ HCCL_VM_DEBUG("Load immediate: locCcu[{}:{}], XnId=[{}], immediate=[{}]",
43 rankId_, dieId_, xnId_, immediate_);43 rankId_, dieId_, xnId_, immediate_);
44}44}
45 45 
Mtest/hccl_vm/src/plugin/runner/ccu_executor/load_executor/load_sqe_args_to_gsa_executor.cc+1-1
@@ -37,7 +37,7 @@ void LoadSqeArgsToGsaExecutor::Run() {
37 uint64_t sqeArgValue = ccuResMgr.GetSqeArgValue(rankId_, dieId_, sqeArgId_);37 uint64_t sqeArgValue = ccuResMgr.GetSqeArgValue(rankId_, dieId_, sqeArgId_);
38 38 
39 ccuResMgr.UpdateXnValue(rankId_, dieId_, gsaId_, sqeArgValue);39 ccuResMgr.UpdateXnValue(rankId_, dieId_, gsaId_, sqeArgValue);
40- HCCL_VM_DEBUG("[LoadSqeArgsToGsaExecutor] Load SqeArg: locCcu[{}:{}], GSAId=[{}], sqeArgValue=[{}]",40+ HCCL_VM_DEBUG("Load SqeArg: locCcu[{}:{}], GSAId=[{}], sqeArgValue=[{}]",
41 rankId_, dieId_, gsaId_, sqeArgValue);41 rankId_, dieId_, gsaId_, sqeArgValue);
42}42}
43 43 
Mtest/hccl_vm/src/plugin/runner/ccu_executor/load_executor/load_sqe_args_to_xn_executor.cc+2-2
@@ -30,7 +30,7 @@ void LoadSqeArgsToXnExecutor::Parser() {
30 xnId_ = instr_.v1.loadSqeArgsToXn.xnId;30 xnId_ = instr_.v1.loadSqeArgsToXn.xnId;
31 sqeArgId_ = instr_.v1.loadSqeArgsToXn.sqeArgsId;31 sqeArgId_ = instr_.v1.loadSqeArgsToXn.sqeArgsId;
32 } else {32 } else {
33- HCCL_VM_ERROR("[LoadSqeArgsToXnExecutor] Unsupported CCU version: {}", RunnerCcuVersionToString(version_));33+ HCCL_VM_ERROR("Unsupported CCU version: {}", RunnerCcuVersionToString(version_));
34 }34 }
35}35}
36 36 
@@ -39,7 +39,7 @@ void LoadSqeArgsToXnExecutor::Run() {
39 auto &ccuResMgr = CcuResourceManager::GetInstance();39 auto &ccuResMgr = CcuResourceManager::GetInstance();
40 uint64_t sqeArgValue = ccuResMgr.GetSqeArgValue(rankId_, dieId_, sqeArgId_);40 uint64_t sqeArgValue = ccuResMgr.GetSqeArgValue(rankId_, dieId_, sqeArgId_);
41 41 
42- HCCL_VM_DEBUG("[LoadSqeArgsToXnExecutor] Load arg: locCcu[{}:{}], XnId=[{}], argId=[{}], value=[{}]",42+ HCCL_VM_DEBUG("Load arg: locCcu[{}:{}], XnId=[{}], argId=[{}], value=[{}]",
43 rankId_, dieId_, xnId_, sqeArgId_, sqeArgValue);43 rankId_, dieId_, xnId_, sqeArgId_, sqeArgValue);
44 ccuResMgr.UpdateXnValue(rankId_, dieId_, xnId_, sqeArgValue);44 ccuResMgr.UpdateXnValue(rankId_, dieId_, xnId_, sqeArgValue);
45}45}
Mtest/hccl_vm/src/plugin/runner/ccu_executor/load_executor/load_xn_xn_executor.cc+1-1
@@ -41,7 +41,7 @@ void LoadXnXnExecutor::Run()
41 uint64_t val = xn1 + xn2;41 uint64_t val = xn1 + xn2;
42 ccuResMgr.UpdateXnValue(rankId_, dieId_, xdId_, val);42 ccuResMgr.UpdateXnValue(rankId_, dieId_, xdId_, val);
43 43 
44- HCCL_VM_DEBUG("[LoadXnXnExecutor] Load Xn[%u] + Xn[%u] to Xn[%u]",44+ HCCL_VM_DEBUG("Load Xn[{}] + Xn[{}] to Xn[{}]",
45 xmId_, xnId_, xdId_);45 xmId_, xnId_, xdId_);
46}46}
47 47 
Mtest/hccl_vm/src/plugin/runner/ccu_executor/reduce_executor/reduce_add_executor.cc+2-2
@@ -51,7 +51,7 @@ void ReduceAddExecutor::Parser()
51// Reduce Add操作51// Reduce Add操作
52void ReduceAddExecutor::Process(CcuResourceManager &ccuResMgr)52void ReduceAddExecutor::Process(CcuResourceManager &ccuResMgr)
53{53{
54- HCCL_VM_DEBUG("[ReduceAddExecutor][Process] Reduce Add info, locCcu[{}:{}], count=[{}], castEn=[{}], dataType=[{}]",54+ HCCL_VM_DEBUG("Reduce Add info, locCcu[{}:{}], count=[{}], castEn=[{}], dataType=[{}]",
55 rankId_, dieId_, count_, castEn_, dataType_);55 rankId_, dieId_, count_, castEn_, dataType_);
56 for (uint32_t i = 0; i < CCU_REDUCE_MAX_MS; i++) {56 for (uint32_t i = 0; i < CCU_REDUCE_MAX_MS; i++) {
57 HCCL_VM_TRACE("msId_[{}]:dieId[{}], msId[{}]", i, msId_[i] >> 15, msId_[i] & 0x7FFF);57 HCCL_VM_TRACE("msId_[{}]:dieId[{}], msId[{}]", i, msId_[i] >> 15, msId_[i] & 0x7FFF);
@@ -68,7 +68,7 @@ void ReduceAddExecutor::Process(CcuResourceManager &ccuResMgr)
68 for (uint32_t i = 0; i < hcomm::CcuRep::CCU_REDUCE_MAX_MS; i++) {68 for (uint32_t i = 0; i < hcomm::CcuRep::CCU_REDUCE_MAX_MS; i++) {
69 msId_[i] += msOffset;69 msId_[i] += msOffset;
70 }70 }
71- HCCL_VM_DEBUG("[ReduceAddExecutor][Process] locCcu[{}:{}], ms offset=[{}], ckeOffset[{}]", rankId_, dieId_, msOffset, ckeOffset);71+ HCCL_VM_DEBUG("locCcu[{}:{}], ms offset=[{}], ckeOffset[{}]", rankId_, dieId_, msOffset, ckeOffset);
72 }72 }
73 // 2. reduce操作73 // 2. reduce操作
74 ReduceAddDataType type = static_cast<ReduceAddDataType>(dataType_);74 ReduceAddDataType type = static_cast<ReduceAddDataType>(dataType_);
Mtest/hccl_vm/src/plugin/runner/ccu_executor/reduce_executor/reduce_max_executor.cc+2-2
@@ -49,7 +49,7 @@ void ReduceMaxExecutor::Parser()
49// Reduce Max操作49// Reduce Max操作
50void ReduceMaxExecutor::Process(CcuResourceManager &ccuResMgr)50void ReduceMaxExecutor::Process(CcuResourceManager &ccuResMgr)
51{51{
52- HCCL_VM_DEBUG("[ReduceMaxExecutor][Process] Reduce Max info, locCcu[{}:{}], count:[{}], dataType:[{}]", rankId_, dieId_, count_, dataType_);52+ HCCL_VM_DEBUG("Reduce Max info, locCcu[{}:{}], count:[{}], dataType:[{}]", rankId_, dieId_, count_, dataType_);
53 if (dataType_ >= ReduceMaxMinDataType::MAX_MIN_RESERVED4 || dataType_ == ReduceMaxMinDataType::MAX_MIN_RESERVED1 ||53 if (dataType_ >= ReduceMaxMinDataType::MAX_MIN_RESERVED4 || dataType_ == ReduceMaxMinDataType::MAX_MIN_RESERVED1 ||
54 dataType_ == ReduceMaxMinDataType::MAX_MIN_RESERVED2 || dataType_ == ReduceMaxMinDataType::MAX_MIN_RESERVED3) {54 dataType_ == ReduceMaxMinDataType::MAX_MIN_RESERVED2 || dataType_ == ReduceMaxMinDataType::MAX_MIN_RESERVED3) {
55 return;55 return;
@@ -66,7 +66,7 @@ void ReduceMaxExecutor::Process(CcuResourceManager &ccuResMgr)
66 for (uint32_t i = 0; i < hcomm::CcuRep::CCU_REDUCE_MAX_MS; i++) {66 for (uint32_t i = 0; i < hcomm::CcuRep::CCU_REDUCE_MAX_MS; i++) {
67 msId_[i] += msOffset;67 msId_[i] += msOffset;
68 }68 }
69- HCCL_VM_DEBUG("[ReduceMaxExecutor][Process] locCcu[{}:{}], ms offset=[{}]", rankId_, dieId_, msOffset);69+ HCCL_VM_DEBUG("locCcu[{}:{}], ms offset=[{}]", rankId_, dieId_, msOffset);
70 }70 }
71 // 2. reduce操作71 // 2. reduce操作
72 ReduceMaxMinDataType type = static_cast<ReduceMaxMinDataType>(dataType_);72 ReduceMaxMinDataType type = static_cast<ReduceMaxMinDataType>(dataType_);
Mtest/hccl_vm/src/plugin/runner/ccu_executor/reduce_executor/reduce_min_executor.cc+2-2
@@ -49,7 +49,7 @@ void ReduceMinExecutor::Parser()
49// Reduce Min操作49// Reduce Min操作
50void ReduceMinExecutor::Process(CcuResourceManager &ccuResMgr)50void ReduceMinExecutor::Process(CcuResourceManager &ccuResMgr)
51{51{
52- HCCL_VM_DEBUG("[ReduceMinExecutor][Process] Reduce Min info, locCcu[{}:{}], count:[{}], dataType:[{}]", rankId_, dieId_, count_, dataType_);52+ HCCL_VM_DEBUG("Reduce Min info, locCcu[{}:{}], count:[{}], dataType:[{}]", rankId_, dieId_, count_, dataType_);
53 if (dataType_ >= ReduceMaxMinDataType::MAX_MIN_RESERVED4 || dataType_ == ReduceMaxMinDataType::MAX_MIN_RESERVED1 ||53 if (dataType_ >= ReduceMaxMinDataType::MAX_MIN_RESERVED4 || dataType_ == ReduceMaxMinDataType::MAX_MIN_RESERVED1 ||
54 dataType_ == ReduceMaxMinDataType::MAX_MIN_RESERVED2 || dataType_ == ReduceMaxMinDataType::MAX_MIN_RESERVED3) {54 dataType_ == ReduceMaxMinDataType::MAX_MIN_RESERVED2 || dataType_ == ReduceMaxMinDataType::MAX_MIN_RESERVED3) {
55 return;55 return;
@@ -66,7 +66,7 @@ void ReduceMinExecutor::Process(CcuResourceManager &ccuResMgr)
66 for (uint32_t i = 0; i < hcomm::CcuRep::CCU_REDUCE_MAX_MS; i++) {66 for (uint32_t i = 0; i < hcomm::CcuRep::CCU_REDUCE_MAX_MS; i++) {
67 msId_[i] += msOffset;67 msId_[i] += msOffset;
68 }68 }
69- HCCL_VM_DEBUG("[ReduceMinExecutor][Process] locCcu[{}:{}], ms offset=[{}]", rankId_, dieId_, msOffset);69+ HCCL_VM_DEBUG("locCcu[{}:{}], ms offset=[{}]", rankId_, dieId_, msOffset);
70 }70 }
71 // 2. reduce操作71 // 2. reduce操作
72 ReduceMaxMinDataType type = static_cast<ReduceMaxMinDataType>(dataType_);72 ReduceMaxMinDataType type = static_cast<ReduceMaxMinDataType>(dataType_);
Mtest/hccl_vm/src/plugin/runner/ccu_executor/trans_executor/sync_cke_executor.cc+1-1
@@ -47,7 +47,7 @@ void SyncCkeExecutor::Process(CcuResourceManager &ccuResMgr) {
47 uint16_t newRmtCke = rmtCke | (locCKE & locCKEMask_);47 uint16_t newRmtCke = rmtCke | (locCKE & locCKEMask_);
48 ccuResMgr.UpdateCkeValue(rmtCcu.first, rmtCcu.second, rmtCKEId_, newRmtCke);48 ccuResMgr.UpdateCkeValue(rmtCcu.first, rmtCcu.second, rmtCKEId_, newRmtCke);
49 49 
50- HCCL_VM_DEBUG("[SyncCkeExecutor][Process] locCcu[{}:{}], channelId=[{}], "50+ HCCL_VM_DEBUG("locCcu[{}:{}], channelId=[{}], "
51 "rmtDevice[{}:{}], locCKE=[{}:{:04x}], "51 "rmtDevice[{}:{}], locCKE=[{}:{:04x}], "
52 "value=[{}], rmtCKE=[{}:{:04x}], value=[{} --> {}].",52 "value=[{}], rmtCKE=[{}:{:04x}], value=[{} --> {}].",
53 rankId_, dieId_, channelId_, rmtCcu.first, rmtCcu.second, locCKEId_,53 rankId_, dieId_, channelId_, rmtCcu.first, rmtCcu.second, locCKEId_,
Mtest/hccl_vm/src/plugin/runner/ccu_executor/trans_executor/sync_xn_executor.cc+1-1
@@ -42,7 +42,7 @@ void SyncXnExecutor::Process(CcuResourceManager &ccuResMgr) {
42 auto rmtCcu = ccuResMgr.GetRmtCcu(rankId_, dieId_, channelId_);42 auto rmtCcu = ccuResMgr.GetRmtCcu(rankId_, dieId_, channelId_);
43 // 将本端的xn内容写到目的端的xn中43 // 将本端的xn内容写到目的端的xn中
44 auto locXnValue = ccuResMgr.GetXnValue(rankId_, dieId_, locXnId_);44 auto locXnValue = ccuResMgr.GetXnValue(rankId_, dieId_, locXnId_);
45- HCCL_VM_INFO("[SyncXnExecutor]sync xn, rankId{}, dieId{}, xnId{}, channelId{}, locXnValue: {}", rmtCcu.first, rmtCcu.second, rmtXnId_, channelId_, locXnValue);45+ HCCL_VM_INFO("sync xn, rankId{}, dieId{}, xnId{}, channelId{}, locXnValue: {}", rmtCcu.first, rmtCcu.second, rmtXnId_, channelId_, locXnValue);
46 ccuResMgr.UpdateXnValue(rmtCcu.first, rmtCcu.second, rmtXnId_, locXnValue);46 ccuResMgr.UpdateXnValue(rmtCcu.first, rmtCcu.second, rmtXnId_, locXnValue);
47 47 
48 // 设置目的端的cke48 // 设置目的端的cke
Mtest/hccl_vm/src/plugin/runner/ccu_executor/trans_executor/trans_loc_mem_to_loc_mem_executor.cc+2-2
@@ -56,9 +56,9 @@ void TransLocMemToLocMemExecutor::Process(CcuResourceManager &ccuResMgr)
56 srcLocAddr += gsaOffset;56 srcLocAddr += gsaOffset;
57 dstLocAddr += gsaOffset;57 dstLocAddr += gsaOffset;
58 setCKEId_ += ckeOffset;58 setCKEId_ += ckeOffset;
59- HCCL_VM_DEBUG("[TransLocMemToLocMemExecutor][Process] locCcu[{}:{}], Get gsa addr offset = [{:x}], cke id offset = [{}]", rankId_, dieId_, gsaOffset, ckeOffset);59+ HCCL_VM_DEBUG("locCcu[{}:{}], Get gsa addr offset = [{:x}], cke id offset = [{}]", rankId_, dieId_, gsaOffset, ckeOffset);
60 }60 }
61- HCCL_VM_DEBUG("[TransLocMemToLocMemExecutor][Process] locCcu[{}:{}] Trans data from srcLocGSAId_[{}] srcLocAddr[{:x}] to dstLocGSAId_[{}] "61+ HCCL_VM_DEBUG("locCcu[{}:{}] Trans data from srcLocGSAId_[{}] srcLocAddr[{:x}] to dstLocGSAId_[{}] "
62 "dstLocAddr[{:x}], with lengthXnId[{}] transLength[{}].",62 "dstLocAddr[{:x}], with lengthXnId[{}] transLength[{}].",
63 rankId_, dieId_, srcGSAId_, srcLocAddr, dstGSAId_, dstLocAddr, lengthXnId_, transLength_);63 rankId_, dieId_, srcGSAId_, srcLocAddr, dstGSAId_, dstLocAddr, lengthXnId_, transLength_);
64 ccuResMgr.TransMemToMem(reinterpret_cast<void *>(srcLocAddr), reinterpret_cast<void *>(dstLocAddr), transLength_, false, 0, 0);64 ccuResMgr.TransMemToMem(reinterpret_cast<void *>(srcLocAddr), reinterpret_cast<void *>(dstLocAddr), transLength_, false, 0, 0);
Mtest/hccl_vm/src/plugin/runner/ccu_executor/trans_executor/trans_loc_mem_to_loc_ms_executor.cc+2-2
@@ -56,11 +56,11 @@ void TransLocMemToLocMSExecutor::Process(CcuResourceManager &ccuResMgr)
56 locAddr += addrOffset;56 locAddr += addrOffset;
57 locMSId_ += msOffset;57 locMSId_ += msOffset;
58 setCKEId_ += ckeOffset;58 setCKEId_ += ckeOffset;
59- HCCL_VM_DEBUG("[TransLocMemToLocMSExecutor][Process] locCcu[{}:{}], Get gsa "59+ HCCL_VM_DEBUG("locCcu[{}:{}], Get gsa "
60 "addr offset = [{:04x}], ms offset = [{:04x}], cke offset = [{:04x}]",60 "addr offset = [{:04x}], ms offset = [{:04x}], cke offset = [{:04x}]",
61 rankId_, dieId_, addrOffset, msOffset, ckeOffset);61 rankId_, dieId_, addrOffset, msOffset, ckeOffset);
62 }62 }
63- HCCL_VM_DEBUG("[TransLocMemToLocMSExecutor][Process] locCcu[{}:{}] Trans data "63+ HCCL_VM_DEBUG("locCcu[{}:{}] Trans data "
64 "from locGSAId_[{}] locAddr[{:x}] to locMsId[{:04x}], "64 "from locGSAId_[{}] locAddr[{:x}] to locMsId[{:04x}], "
65 "with lengthXnId[{}] transLength[{}], lengthEn_[{}].",65 "with lengthXnId[{}] transLength[{}], lengthEn_[{}].",
66 rankId_, dieId_, locGSAId_, locAddr, locMSId_, lengthXnId_,66 rankId_, dieId_, locGSAId_, locAddr, locMSId_, lengthXnId_,
Mtest/hccl_vm/src/plugin/runner/ccu_executor/trans_executor/trans_loc_mem_to_rmt_mem_executor.cc+2-2
@@ -59,12 +59,12 @@ void TransLocMemToRmtMemExecutor::Process(CcuResourceManager &ccuResMgr)
59 rmtAddr += offset;59 rmtAddr += offset;
60 locAddr += offset;60 locAddr += offset;
61 setCKEId_ += ckeOffset;61 setCKEId_ += ckeOffset;
62- HCCL_VM_DEBUG("[TransLocMemToRmtMemExecutor][Process] locCcu[{}:{}], Get gsa "62+ HCCL_VM_DEBUG("locCcu[{}:{}], Get gsa "
63 "addr offset = [{:04x}], cke offset = [{:04x}]",63 "addr offset = [{:04x}], cke offset = [{:04x}]",
64 rankId_, dieId_, offset, ckeOffset);64 rankId_, dieId_, offset, ckeOffset);
65 }65 }
66 // 3.搬运动作66 // 3.搬运动作
67- HCCL_VM_DEBUG("[TransLocMemToRmtMemExecutor][Process] locCcu[{}:{}] Trans data "67+ HCCL_VM_DEBUG("locCcu[{}:{}] Trans data "
68 "from locGSAId_[{}] locAddr[{:x}] to rmtGSAId_[{}] rmtAddr[{:x}], "68 "from locGSAId_[{}] locAddr[{:x}] to rmtGSAId_[{}] rmtAddr[{:x}], "
69 "with lengthXnId[{}] transLength[{}], reduceEn[{}], reduceOp[{}].",69 "with lengthXnId[{}] transLength[{}], reduceEn[{}], reduceOp[{}].",
70 rankId_, dieId_, locGSAId_, locAddr, rmtGSAId_, rmtAddr, lengthXnId_, transLength_, reduceEn_, reduceOpCode_);70 rankId_, dieId_, locGSAId_, locAddr, rmtGSAId_, rmtAddr, lengthXnId_, transLength_, reduceEn_, reduceOpCode_);
Mtest/hccl_vm/src/plugin/runner/ccu_executor/trans_executor/trans_loc_ms_to_loc_mem_executor.cc+2-2
@@ -56,10 +56,10 @@ void TransLocMSToLocMemExecutor::Process(CcuResourceManager &ccuResMgr)
56 locAddr += addrOffset;56 locAddr += addrOffset;
57 locMSId_ += msOffset;57 locMSId_ += msOffset;
58 setCKEId_ += ckeOffset;58 setCKEId_ += ckeOffset;
59- HCCL_VM_DEBUG("[TransLocMSToLocMemExecutor][Process] locCcu[{}:{}], Get gsa addr offset = [{:04x}], ms offset = [{}], "59+ HCCL_VM_DEBUG("locCcu[{}:{}], Get gsa addr offset = [{:04x}], ms offset = [{}], "
60 "cke offset = [{:04x}]", rankId_, dieId_, addrOffset, msOffset, ckeOffset);60 "cke offset = [{:04x}]", rankId_, dieId_, addrOffset, msOffset, ckeOffset);
61 }61 }
62- HCCL_VM_DEBUG("[TransLocMSToLocMemExecutor][Process] locCcu[{}:{}] Trans data "62+ HCCL_VM_DEBUG("locCcu[{}:{}] Trans data "
63 "from locMsId[{}] to locGSAId[{}] locAddr[{:x}], "63 "from locMsId[{}] to locGSAId[{}] locAddr[{:x}], "
64 "with lengthXnId[{}] transLength[{}], lengthEn_[{}].",64 "with lengthXnId[{}] transLength[{}], lengthEn_[{}].",
65 rankId_, dieId_, locMSId_, locGSAId_, locAddr, lengthXnId_,65 rankId_, dieId_, locMSId_, locGSAId_, locAddr, lengthXnId_,
Mtest/hccl_vm/src/plugin/runner/ccu_executor/trans_executor/trans_loc_ms_to_loc_ms_executor.cc+2-2
@@ -54,10 +54,10 @@ void TransLocMSToLocMSExecutor::Process(CcuResourceManager &ccuResMgr)
54 srcMSId_ += msOffset;54 srcMSId_ += msOffset;
55 dstMSId_ += msOffset;55 dstMSId_ += msOffset;
56 setCKEId_ += ckeOffset;56 setCKEId_ += ckeOffset;
57- HCCL_VM_DEBUG("[TransLocMSToLocMSExecutor][Process] locCcu[{}:{}], Get ms offset = [{:04x}], cke offset = [{:04x}]",57+ HCCL_VM_DEBUG("locCcu[{}:{}], Get ms offset = [{:04x}], cke offset = [{:04x}]",
58 rankId_, dieId_, msOffset, ckeOffset);58 rankId_, dieId_, msOffset, ckeOffset);
59 }59 }
60- HCCL_VM_DEBUG("[TransLocMSToLocMSExecutor][Process] locCcu[{}:{}] Trans data "60+ HCCL_VM_DEBUG("locCcu[{}:{}] Trans data "
61 "from locSrcMsId[{}] to locDstMsId[{}], "61 "from locSrcMsId[{}] to locDstMsId[{}], "
62 "with lengthXnId[{}] transLength[{}].",62 "with lengthXnId[{}] transLength[{}].",
63 rankId_, dieId_, srcMSId_, dstMSId_, lengthXnId_, transLength_);63 rankId_, dieId_, srcMSId_, dstMSId_, lengthXnId_, transLength_);
Mtest/hccl_vm/src/plugin/runner/ccu_executor/trans_executor/trans_loc_ms_to_rmt_mem_executor.cc+2-2
@@ -56,12 +56,12 @@ void TransLocMSToRmtMemExecutor::Process(CcuResourceManager &ccuResMgr)
56 rmtAddr += addrOffset;56 rmtAddr += addrOffset;
57 locMSId_ += msOffset;57 locMSId_ += msOffset;
58 setCKEId_ += ckeOffset;58 setCKEId_ += ckeOffset;
59- HCCL_VM_DEBUG("[TransLocMSToRmtMemExecutor][Process] locCcu[{}:{}], Get gsa "59+ HCCL_VM_DEBUG("locCcu[{}:{}], Get gsa "
60 "addr offset = [{:04x}], ms offset = [{:04x}], cke offset = [{:04x}]",60 "addr offset = [{:04x}], ms offset = [{:04x}], cke offset = [{:04x}]",
61 rankId_, dieId_, addrOffset, msOffset, ckeOffset);61 rankId_, dieId_, addrOffset, msOffset, ckeOffset);
62 }62 }
63 // 3.搬运动作63 // 3.搬运动作
64- HCCL_VM_DEBUG("[TransLocMSToRmtMemExecutor][Process] locCcu[{}:{}] Trans data "64+ HCCL_VM_DEBUG("locCcu[{}:{}] Trans data "
65 "from locMsId[{}] to rmtGSAId_[{}] rmtAddr[{:x}], "65 "from locMsId[{}] to rmtGSAId_[{}] rmtAddr[{:x}], "
66 "with lengthXnId[{}] transLength[{}].",66 "with lengthXnId[{}] transLength[{}].",
67 rankId_, dieId_, locMSId_, rmtGSAId_, rmtAddr, lengthXnId_, transLength_);67 rankId_, dieId_, locMSId_, rmtGSAId_, rmtAddr, lengthXnId_, transLength_);
Mtest/hccl_vm/src/plugin/runner/ccu_executor/trans_executor/trans_loc_ms_to_rmt_ms_executor.cc+3-3
@@ -48,7 +48,7 @@ void TransLocMSToRmtMSExecutor::Process(CcuResourceManager &ccuResMgr)
48 // 1.根据channel id获取remote rank id48 // 1.根据channel id获取remote rank id
49 auto rmtCcu = ccuResMgr.GetRmtCcu(rankId_, dieId_, channelId_);49 auto rmtCcu = ccuResMgr.GetRmtCcu(rankId_, dieId_, channelId_);
50 if (rmtCcu.second != rmtDieId_) {50 if (rmtCcu.second != rmtDieId_) {
51- HCCL_VM_WARN("[TransLocMSToRmtMSExecutor][Process] dieId[{}] from channel is not same as rmtDieId[{}]. curCcu[{}:{}], rmtCcu[{}:{}]",51+ HCCL_VM_WARN("dieId[{}] from channel is not same as rmtDieId[{}]. curCcu[{}:{}], rmtCcu[{}:{}]",
52 rmtCcu.second, rmtDieId_, rankId_, dieId_, rmtCcu.first, rmtCcu.second);52 rmtCcu.second, rmtDieId_, rankId_, dieId_, rmtCcu.first, rmtCcu.second);
53 return;53 return;
54 }54 }
@@ -59,13 +59,13 @@ void TransLocMSToRmtMSExecutor::Process(CcuResourceManager &ccuResMgr)
59 rmtMSId_ += msOffset;59 rmtMSId_ += msOffset;
60 locMSId_ += msOffset;60 locMSId_ += msOffset;
61 setCKEId_ += ckeOffset;61 setCKEId_ += ckeOffset;
62- HCCL_VM_DEBUG("[TransLocMSToRmtMSExecutor][Process] curCcu[{}:{}], Get ms offset = [{:04x}], cke offset = [{:04x}]",62+ HCCL_VM_DEBUG("curCcu[{}:{}], Get ms offset = [{:04x}], cke offset = [{:04x}]",
63 rankId_, dieId_, msOffset, ckeOffset);63 rankId_, dieId_, msOffset, ckeOffset);
64 }64 }
65 // 3.要搬运的本端内存地址及数据长度65 // 3.要搬运的本端内存地址及数据长度
66 transLength_ = (lengthEn_ == 0) ? HcclSim::BYTE_NUM_4K : ccuResMgr.GetXnValue(rankId_, dieId_, lengthXnId_);66 transLength_ = (lengthEn_ == 0) ? HcclSim::BYTE_NUM_4K : ccuResMgr.GetXnValue(rankId_, dieId_, lengthXnId_);
67 // 4.搬运动作67 // 4.搬运动作
68- HCCL_VM_DEBUG("[TransLocMSToRmtMSExecutor][Process] ccuId[{}:{}-{}:{}] Trans data "68+ HCCL_VM_DEBUG("ccuId[{}:{}-{}:{}] Trans data "
69 "from locSrcMsId[{}] to rmtDstMsId[{}], "69 "from locSrcMsId[{}] to rmtDstMsId[{}], "
70 "with lengthXnId[{}] transLength[{}].",70 "with lengthXnId[{}] transLength[{}].",
71 rankId_, dieId_, rmtCcu.first, rmtCcu.second, locMSId_, rmtMSId_, lengthXnId_, transLength_);71 rankId_, dieId_, rmtCcu.first, rmtCcu.second, locMSId_, rmtMSId_, lengthXnId_, transLength_);
Mtest/hccl_vm/src/plugin/runner/ccu_executor/trans_executor/trans_rmt_mem_to_loc_mem_executor.cc+2-2
@@ -59,11 +59,11 @@ void TransRmtMemToLocMemExecutor::Process(CcuResourceManager &ccuResMgr)
59 rmtAddr += offset;59 rmtAddr += offset;
60 locAddr += offset;60 locAddr += offset;
61 setCKEId_ += ckeOffset;61 setCKEId_ += ckeOffset;
62- HCCL_VM_DEBUG("[TransRmtMemToLocMemExecutor][Process] locCcu[{}:{}], Get gsa addr offset = [{:04x}], cke offset = [{:04x}]",62+ HCCL_VM_DEBUG("locCcu[{}:{}], Get gsa addr offset = [{:04x}], cke offset = [{:04x}]",
63 rankId_, dieId_, offset, ckeOffset);63 rankId_, dieId_, offset, ckeOffset);
64 }64 }
65 // 3.搬运动作65 // 3.搬运动作
66- HCCL_VM_DEBUG("[TransRmtMemToLocMemExecutor][Process] locCcu[{}:{}] Trans data "66+ HCCL_VM_DEBUG("locCcu[{}:{}] Trans data "
67 "from rmtGSAId[{}] rmtAddr[{:x}] to locGSAId[{}] locAddr[{:x}], "67 "from rmtGSAId[{}] rmtAddr[{:x}] to locGSAId[{}] locAddr[{:x}], "
68 "with lengthXnId[{}] transLength[{}], reduceEn[{}], reduceOp[{}].",68 "with lengthXnId[{}] transLength[{}], reduceEn[{}], reduceOp[{}].",
69 rankId_, dieId_, rmtGSAId_, rmtAddr, locGSAId_, locAddr, lengthXnId_, transLength_, reduceEn_, reduceOpCode_);69 rankId_, dieId_, rmtGSAId_, rmtAddr, locGSAId_, locAddr, lengthXnId_, transLength_, reduceEn_, reduceOpCode_);
Mtest/hccl_vm/src/plugin/runner/ccu_executor/trans_executor/trans_rmt_mem_to_loc_ms_executor.cc+2-2
@@ -56,11 +56,11 @@ void TransRmtMemToLocMSExecutor::Process(CcuResourceManager &ccuResMgr)
56 rmtAddr += addrOffset;56 rmtAddr += addrOffset;
57 locMSId_ += msOffset;57 locMSId_ += msOffset;
58 setCKEId_ += ckeOffset;58 setCKEId_ += ckeOffset;
59- HCCL_VM_DEBUG("[TransRmtMemToLocMSExecutor][Process] locCcu[{}:{}], Get gsa addr offset = [{:04x}], ms offset = [{:04x}], cke offset = [{:04x}]",59+ HCCL_VM_DEBUG("locCcu[{}:{}], Get gsa addr offset = [{:04x}], ms offset = [{:04x}], cke offset = [{:04x}]",
60 rankId_, dieId_, addrOffset, msOffset, ckeOffset);60 rankId_, dieId_, addrOffset, msOffset, ckeOffset);
61 }61 }
62 // 3.搬运动作62 // 3.搬运动作
63- HCCL_VM_DEBUG("[TransRmtMemToLocMSExecutor][Process] locCcu[{}:{}] Trans data "63+ HCCL_VM_DEBUG("locCcu[{}:{}] Trans data "
64 "from rmtGSAId[{}] rmtAddr[{:x}] to locMsId[{}], "64 "from rmtGSAId[{}] rmtAddr[{:x}] to locMsId[{}], "
65 "with lengthXnId[{}] transLength[{}].",65 "with lengthXnId[{}] transLength[{}].",
66 rankId_, dieId_, rmtGSAId_, rmtAddr, locMSId_, lengthXnId_, transLength_);66 rankId_, dieId_, rmtGSAId_, rmtAddr, locMSId_, lengthXnId_, transLength_);
Mtest/hccl_vm/src/plugin/runner/ccu_executor/trans_executor/trans_rmt_ms_to_loc_mem_executor.cc+3-3
@@ -49,7 +49,7 @@ void TransRmtMSToLocMemExecutor::Process(CcuResourceManager &ccuResMgr)
49 // 1.根据channel id获取remote rank id49 // 1.根据channel id获取remote rank id
50 auto rmtCcu = ccuResMgr.GetRmtCcu(rankId_, dieId_, channelId_);50 auto rmtCcu = ccuResMgr.GetRmtCcu(rankId_, dieId_, channelId_);
51 if (rmtCcu.second != rmtDieId_) {51 if (rmtCcu.second != rmtDieId_) {
52- HCCL_VM_WARN("[TransRmtMSToLocMemExecutor][Process] dieId[{}] from channel is not same as rmtDieId[{}]. curCcu[{}:{}], rmtCcu[{}:{}]",52+ HCCL_VM_WARN("dieId[{}] from channel is not same as rmtDieId[{}]. curCcu[{}:{}], rmtCcu[{}:{}]",
53 rmtCcu.second, rmtDieId_, rankId_, dieId_, rmtCcu.first, rmtCcu.second);53 rmtCcu.second, rmtDieId_, rankId_, dieId_, rmtCcu.first, rmtCcu.second);
54 return;54 return;
55 }55 }
@@ -63,13 +63,13 @@ void TransRmtMSToLocMemExecutor::Process(CcuResourceManager &ccuResMgr)
63 locAddr += addrOffset;63 locAddr += addrOffset;
64 rmtMSId_ += msOffset;64 rmtMSId_ += msOffset;
65 setCKEId_ += ckeOffset;65 setCKEId_ += ckeOffset;
66- HCCL_VM_DEBUG("[TransRmtMSToLocMemExecutor][Process] ccuId=[{}:{}], Get gsa addr offset = [{:04x}], ms offset = [{:04x}], cke offset = [{:04x}]",66+ HCCL_VM_DEBUG("ccuId=[{}:{}], Get gsa addr offset = [{:04x}], ms offset = [{:04x}], cke offset = [{:04x}]",
67 rankId_, dieId_, addrOffset, msOffset, ckeOffset);67 rankId_, dieId_, addrOffset, msOffset, ckeOffset);
68 }68 }
69 // 4.要搬运的本端内存地址及数据长度69 // 4.要搬运的本端内存地址及数据长度
70 transLength_ = (lengthEn_ == 0) ? HcclSim::BYTE_NUM_4K : ccuResMgr.GetXnValue(rankId_, dieId_, lengthXnId_);70 transLength_ = (lengthEn_ == 0) ? HcclSim::BYTE_NUM_4K : ccuResMgr.GetXnValue(rankId_, dieId_, lengthXnId_);
71 // 5.搬运动作71 // 5.搬运动作
72- HCCL_VM_DEBUG("[TransRmtMSToLocMemExecutor][Process] ccuId=[{}:{}-{}:{}] Trans data "72+ HCCL_VM_DEBUG("ccuId=[{}:{}-{}:{}] Trans data "
73 "from rmtMsId[{}] to locGSAId[{}] locAddr[{:x}], "73 "from rmtMsId[{}] to locGSAId[{}] locAddr[{:x}], "
74 "with lengthXnId[{}] transLength[{}].",74 "with lengthXnId[{}] transLength[{}].",
75 rankId_, dieId_, rmtCcu.first, rmtCcu.second, rmtMSId_, locGSAId_, locAddr, lengthXnId_, transLength_);75 rankId_, dieId_, rmtCcu.first, rmtCcu.second, rmtMSId_, locGSAId_, locAddr, lengthXnId_, transLength_);
Mtest/hccl_vm/src/plugin/runner/ccu_executor/trans_executor/trans_rmt_ms_to_loc_ms_executor.cc+3-3
@@ -46,7 +46,7 @@ void TransRmtMSToLocMSExecutor::Process(CcuResourceManager &ccuResMgr)
46 // 1.根据channel id获取remote rank id46 // 1.根据channel id获取remote rank id
47 auto rmtCcu = ccuResMgr.GetRmtCcu(rankId_, dieId_, channelId_);47 auto rmtCcu = ccuResMgr.GetRmtCcu(rankId_, dieId_, channelId_);
48 if (rmtCcu.second != rmtDieId_) {48 if (rmtCcu.second != rmtDieId_) {
49- HCCL_VM_WARN("[TransRmtMSToLocMSExecutor][Process] dieId[{}] from channel is not same as rmtDieId[{}]. curCcu[{}:{}], rmtCcu[{}:{}]",49+ HCCL_VM_WARN("dieId[{}] from channel is not same as rmtDieId[{}]. curCcu[{}:{}], rmtCcu[{}:{}]",
50 rmtCcu.second, rmtDieId_, rankId_, dieId_, rmtCcu.first, rmtCcu.second);50 rmtCcu.second, rmtDieId_, rankId_, dieId_, rmtCcu.first, rmtCcu.second);
51 return;51 return;
52 }52 }
@@ -57,13 +57,13 @@ void TransRmtMSToLocMSExecutor::Process(CcuResourceManager &ccuResMgr)
57 locMSId_ += msOffset;57 locMSId_ += msOffset;
58 rmtMSId_ += msOffset;58 rmtMSId_ += msOffset;
59 setCKEId_ += ckeOffset;59 setCKEId_ += ckeOffset;
60- HCCL_VM_DEBUG("[TransRmtMSToLocMSExecutor][Process] ccuId=[{}:{}], Get ms offset = [{:04x}], cke offset = [{:04x}]",60+ HCCL_VM_DEBUG("ccuId=[{}:{}], Get ms offset = [{:04x}], cke offset = [{:04x}]",
61 rankId_, dieId_, msOffset, ckeOffset);61 rankId_, dieId_, msOffset, ckeOffset);
62 }62 }
63 // 3.要搬运的本端内存地址及数据长度63 // 3.要搬运的本端内存地址及数据长度
64 transLength_ = (lengthEn_ == 0) ? HcclSim::BYTE_NUM_4K : ccuResMgr.GetXnValue(rankId_, dieId_, lengthXnId_);64 transLength_ = (lengthEn_ == 0) ? HcclSim::BYTE_NUM_4K : ccuResMgr.GetXnValue(rankId_, dieId_, lengthXnId_);
65 // 4.搬运动作65 // 4.搬运动作
66- HCCL_VM_DEBUG("[TransRmtMSToLocMSExecutor][Process] ccuId=[{}:{}-{}:{}] Trans data "66+ HCCL_VM_DEBUG("ccuId=[{}:{}-{}:{}] Trans data "
67 "from rmtMsId[{}] to locMsId[{}], "67 "from rmtMsId[{}] to locMsId[{}], "
68 "with lengthXnId[{}] transLength[{}].",68 "with lengthXnId[{}] transLength[{}].",
69 rankId_, dieId_, rmtCcu.first, rmtCcu.second, rmtMSId_, locMSId_, lengthXnId_, transLength_);69 rankId_, dieId_, rmtCcu.first, rmtCcu.second, rmtMSId_, locMSId_, lengthXnId_, transLength_);
Mtest/hccl_vm/src/plugin/runner/runner_utils/storage_manager.cc+34-34
@@ -70,16 +70,16 @@ HcclVmResult StorageManager::PrintAllRankInputBuffer()
70 sim::PhyMemBlock srcPhyMem{};70 sim::PhyMemBlock srcPhyMem{};
71 auto srcAddr = sim::AcquireDevPtrInNoHostProcess((void*)startAddr, srcPhyMem);71 auto srcAddr = sim::AcquireDevPtrInNoHostProcess((void*)startAddr, srcPhyMem);
72 if (srcAddr == nullptr) {72 if (srcAddr == nullptr) {
73- HCCL_VM_ERROR("[PrintAllRankInputBuffer] 无法获取startAddr的设备地址(addr= {:x})!", reinterpret_cast<uintptr_t>((void*)startAddr));73+ HCCL_VM_ERROR("无法获取startAddr的设备地址(addr= {:x})!", reinterpret_cast<uintptr_t>((void*)startAddr));
74 return HcclVmResult::HCCL_SIM_E_INTERNAL;74 return HcclVmResult::HCCL_SIM_E_INTERNAL;
75 }75 }
76 m_allPhyMem.push_back(srcPhyMem);76 m_allPhyMem.push_back(srcPhyMem);
77 77 
78 auto size = m_synData.memory_info.data[i].size;78 auto size = m_synData.memory_info.data[i].size;
79- HCCL_VM_DEBUG("[PrintAllRankInputBuffer] rankId={}, input buffer= {:x}, size={}",79+ HCCL_VM_DEBUG("rankId={}, input buffer= {:x}, size={}",
80 m_synData.memory_info.data[i].rank_id, startAddr, size);80 m_synData.memory_info.data[i].rank_id, startAddr, size);
81 for (uint32_t idx = 0; idx < size; idx++) {81 for (uint32_t idx = 0; idx < size; idx++) {
82- HCCL_VM_TRACE("[PrintAllRankInputBuffer] idx={}, value={:x}", idx, *((char*)srcAddr + idx));82+ HCCL_VM_TRACE("idx={}, value={:x}", idx, *((char*)srcAddr + idx));
83 }83 }
84 }84 }
85 }85 }
@@ -116,7 +116,7 @@ HcclVmResult StorageManager::Trans2CheckerParam(sim::OpDetailTab& detailTab, ::O
116 }116 }
117 }117 }
118 118 
119- HCCL_VM_DEBUG("[Trans2CheckerParam] Success");119+ HCCL_VM_DEBUG("Success");
120 120 
121 return HcclVmResult::HCCL_SIM_SUCCESS;121 return HcclVmResult::HCCL_SIM_SUCCESS;
122}122}
@@ -124,7 +124,7 @@ HcclVmResult StorageManager::Trans2CheckerParam(sim::OpDetailTab& detailTab, ::O
124HcclVmResult StorageManager::LoadHcclVmSynthesisData(sim::OpDetailTab& detailTab, std::vector<sim::CcuChannelTab>& channels)124HcclVmResult StorageManager::LoadHcclVmSynthesisData(sim::OpDetailTab& detailTab, std::vector<sim::CcuChannelTab>& channels)
125{125{
126 if (detailTab.opDetail.size() < sizeof(::OpDetails)) {126 if (detailTab.opDetail.size() < sizeof(::OpDetails)) {
127- HCCL_VM_ERROR("[LoadHcclVmSynthesisData] opDetail BLOB too small");127+ HCCL_VM_ERROR("opDetail BLOB too small");
128 return HcclVmResult::HCCL_SIM_E_INTERNAL;128 return HcclVmResult::HCCL_SIM_E_INTERNAL;
129 }129 }
130 ::OpDetails opDetails{};130 ::OpDetails opDetails{};
@@ -151,7 +151,7 @@ HcclVmResult StorageManager::LoadHcclVmSynthesisData(sim::OpDetailTab& detailTab
151HcclVmResult StorageManager::LoadHcclVmInstrData(std::vector<sim::CcuInstrResTab>& instrRes)151HcclVmResult StorageManager::LoadHcclVmInstrData(std::vector<sim::CcuInstrResTab>& instrRes)
152{152{
153 for (auto &ccuInstr : instrRes) {153 for (auto &ccuInstr : instrRes) {
154- HCCL_VM_DEBUG("[LoadHcclVmInstrData] rankId={}, dieId={}, count={}",154+ HCCL_VM_DEBUG("rankId={}, dieId={}, count={}",
155 ccuInstr.rankId, ccuInstr.dieId, ccuInstr.instrCount);155 ccuInstr.rankId, ccuInstr.dieId, ccuInstr.instrCount);
156 }156 }
157 157
@@ -167,11 +167,11 @@ HcclVmResult StorageManager::LoadHcclVmTaskMetaData(std::vector<sim::OpTaskTab>&
167 std::memcpy(&metaData, task.optaskMeta.data(), sizeof(HcclTaskMetaData));167 std::memcpy(&metaData, task.optaskMeta.data(), sizeof(HcclTaskMetaData));
168 taskMeataData.task_meta.push_back(metaData);168 taskMeataData.task_meta.push_back(metaData);
169 } else {169 } else {
170- HCCL_VM_WARN("[LoadHcclVmTaskMetaData] optaskMeta too small, taskSeq={} src:{:d}, dst:{:d}", task.taskSeq, task.optaskMeta.size(), sizeof(HcclTaskMetaData));170+ HCCL_VM_WARN("optaskMeta too small, taskSeq={} src:{:d}, dst:{:d}", task.taskSeq, task.optaskMeta.size(), sizeof(HcclTaskMetaData));
171 }171 }
172 }172 }
173 for (auto &taskMeta : taskMeataData.task_meta) {173 for (auto &taskMeta : taskMeataData.task_meta) {
174- HCCL_VM_DEBUG("[LoadHcclVmTaskMetaData] rankId={}, dieId={}, instrCnt={}, argSize={}, streamId={}",174+ HCCL_VM_DEBUG("rankId={}, dieId={}, instrCnt={}, argSize={}, streamId={}",
175 taskMeta.rankId, static_cast<uint32_t>(taskMeta.taskData.ccu.dieId), 175 taskMeta.rankId, static_cast<uint32_t>(taskMeta.taskData.ccu.dieId),
176 taskMeta.taskData.ccu.instCnt, taskMeta.taskData.ccu.argSize, taskMeta.streamId);176 taskMeta.taskData.ccu.instCnt, taskMeta.taskData.ccu.argSize, taskMeta.streamId);
177 }177 }
@@ -184,20 +184,20 @@ HcclVmResult StorageManager::GetHcclVmFlagData(HcclSim::HcclVmFlagData &waitFlag
184 // 1. 构造路径184 // 1. 构造路径
185 std::string rootPath = FindRootPath();185 std::string rootPath = FindRootPath();
186 if (rootPath.empty()) {186 if (rootPath.empty()) {
187- HCCL_VM_ERROR("[StorageManager][GetHcclVmFlagData] Failed to find root path");187+ HCCL_VM_ERROR("Failed to find root path");
188 return HcclVmResult::HCCL_SIM_E_INTERNAL;188 return HcclVmResult::HCCL_SIM_E_INTERNAL;
189 }189 }
190 std::string fullPath = rootPath + DATA_FILE_PATH + HCCLVM_FLAG_DATA_FILE;190 std::string fullPath = rootPath + DATA_FILE_PATH + HCCLVM_FLAG_DATA_FILE;
191 // 2. 写文件,通知runner启动191 // 2. 写文件,通知runner启动
192 FILE *fp = fopen(fullPath.c_str(), "rb");192 FILE *fp = fopen(fullPath.c_str(), "rb");
193 if (!fp) {193 if (!fp) {
194- HCCL_VM_ERROR("[StorageManager][GetHcclVmFlagData] Open file failed: {}", fullPath);194+ HCCL_VM_ERROR("Open file failed: {}", fullPath);
195 return HcclVmResult::HCCL_SIM_E_INTERNAL;195 return HcclVmResult::HCCL_SIM_E_INTERNAL;
196 }196 }
197 197 
198 auto ret = HcclVmFlagDataRead(fp, waitFlag, HCCLVM_FLAG_FILE_MAGIC);198 auto ret = HcclVmFlagDataRead(fp, waitFlag, HCCLVM_FLAG_FILE_MAGIC);
199 if (ret != HcclVmResult::HCCL_SIM_SUCCESS) {199 if (ret != HcclVmResult::HCCL_SIM_SUCCESS) {
200- HCCL_VM_ERROR("[StorageManager][GetHcclVmFlagData] HcclVmFlagDataRead failed");200+ HCCL_VM_ERROR("HcclVmFlagDataRead failed");
201 fclose(fp);201 fclose(fp);
202 return HcclVmResult::HCCL_SIM_E_INTERNAL;202 return HcclVmResult::HCCL_SIM_E_INTERNAL;
203 }203 }
@@ -207,18 +207,18 @@ HcclVmResult StorageManager::GetHcclVmFlagData(HcclSim::HcclVmFlagData &waitFlag
207 207 
208HcclVmResult StorageManager::DumpHcclVmFlagData(uint16_t status)208HcclVmResult StorageManager::DumpHcclVmFlagData(uint16_t status)
209{209{
210- HCCL_VM_DEBUG("[StorageManager][DumpHcclVmFlagData] Start dumping hccl vm flag data...");210+ HCCL_VM_DEBUG("Start dumping hccl vm flag data...");
211 // 1. 构造完整路径211 // 1. 构造完整路径
212 std::string rootPath = FindRootPath();212 std::string rootPath = FindRootPath();
213 if (rootPath.empty()) {213 if (rootPath.empty()) {
214- HCCL_VM_ERROR("[StorageManager][GetHcclVmFlagData] Failed to find root path");214+ HCCL_VM_ERROR("Failed to find root path");
215 return HcclVmResult::HCCL_SIM_E_INTERNAL;215 return HcclVmResult::HCCL_SIM_E_INTERNAL;
216 }216 }
217 std::string fullPath = rootPath + DATA_FILE_PATH + HCCLVM_FLAG_DATA_FILE;217 std::string fullPath = rootPath + DATA_FILE_PATH + HCCLVM_FLAG_DATA_FILE;
218 // 2. 写文件,通知runner启动218 // 2. 写文件,通知runner启动
219 FILE *fp = fopen(fullPath.c_str(), "wb");219 FILE *fp = fopen(fullPath.c_str(), "wb");
220 if (!fp) {220 if (!fp) {
221- HCCL_VM_ERROR("[StorageManager][DumpHcclVmFlagData] Open file failed: {}", fullPath);221+ HCCL_VM_ERROR("Open file failed: {}", fullPath);
222 return HcclVmResult::HCCL_SIM_E_INTERNAL;222 return HcclVmResult::HCCL_SIM_E_INTERNAL;
223 }223 }
224 224 
@@ -227,7 +227,7 @@ HcclVmResult StorageManager::DumpHcclVmFlagData(uint16_t status)
227 flagData.runner_status = status;227 flagData.runner_status = status;
228 auto ret = HcclVmFlagDataWrite(fp, flagData);228 auto ret = HcclVmFlagDataWrite(fp, flagData);
229 if (ret != HcclVmResult::HCCL_SIM_SUCCESS) {229 if (ret != HcclVmResult::HCCL_SIM_SUCCESS) {
230- HCCL_VM_ERROR("[StorageManager][DumpHcclVmFlagData] HcclVmFlagDataWrite failed");230+ HCCL_VM_ERROR("HcclVmFlagDataWrite failed");
231 fclose(fp);231 fclose(fp);
232 return HcclVmResult::HCCL_SIM_E_INTERNAL;232 return HcclVmResult::HCCL_SIM_E_INTERNAL;
233 }233 }
@@ -252,14 +252,14 @@ std::string StorageManager::FindRootPath()
252 }252 }
253 } else {253 } else {
254 // 如果 realpath 失败(例如路径被删除或权限不足)254 // 如果 realpath 失败(例如路径被删除或权限不足)
255- HCCL_VM_WARN("[FindRootPath] Iteration {}: realpath failed for {}", i, current_search_path);255+ HCCL_VM_WARN("Iteration {}: realpath failed for {}", i, current_search_path);
256 }256 }
257 257 
258 // 没找到,将探测路径向上推一级258 // 没找到,将探测路径向上推一级
259 current_search_path += "/..";259 current_search_path += "/..";
260 }260 }
261 261 
262- HCCL_VM_WARN("[FindRootPath] RootPath NOT found");262+ HCCL_VM_WARN("RootPath NOT found");
263 return ""; 263 return "";
264}264}
265 265 
@@ -267,14 +267,14 @@ HcclVmResult StorageManager::ConvertTaskQueue(const HcclVmTaskMetaData &taskMeat
267{267{
268 auto rankSize = GetRankSize();268 auto rankSize = GetRankSize();
269 m_allRankTaskQueues.resize(rankSize);269 m_allRankTaskQueues.resize(rankSize);
270- HCCL_VM_INFO("[ConvertTaskQueue] rankSize={}", rankSize);270+ HCCL_VM_INFO("rankSize={}", rankSize);
271 271 
272 auto taskMetaVec = taskMeataData.task_meta;272 auto taskMetaVec = taskMeataData.task_meta;
273 uint32_t size = taskMetaVec.size();273 uint32_t size = taskMetaVec.size();
274 for (uint32_t i = 0; i < size; i++) {274 for (uint32_t i = 0; i < size; i++) {
275 uint32_t rankId = taskMetaVec[i].rankId;275 uint32_t rankId = taskMetaVec[i].rankId;
276 if (rankId >= rankSize) {276 if (rankId >= rankSize) {
277- HCCL_VM_ERROR("[ConvertTaskQueue] rankId={} is out of range. rankSize={}", rankId, rankSize);277+ HCCL_VM_ERROR("rankId={} is out of range. rankSize={}", rankId, rankSize);
278 return HcclVmResult::HCCL_SIM_E_PARA;278 return HcclVmResult::HCCL_SIM_E_PARA;
279 }279 }
280 uint32_t streamId = (taskMetaVec[i].streamId);280 uint32_t streamId = (taskMetaVec[i].streamId);
@@ -315,7 +315,7 @@ HcclVmResult StorageManager::InitCcuResource(std::vector<sim::CcuInstrResTab>& i
315 if (devType_ == DevType::DEV_TYPE_950) {315 if (devType_ == DevType::DEV_TYPE_950) {
316 ccuVersion = RunnerCcuVersion::CCU_V1;316 ccuVersion = RunnerCcuVersion::CCU_V1;
317 } else {317 } else {
318- HCCL_VM_ERROR("[InitCcuResource] Wrong devive type: {:d}", static_cast<int>(devType_));318+ HCCL_VM_ERROR("Wrong devive type: {:d}", static_cast<int>(devType_));
319 return HcclVmResult::HCCL_SIM_E_INTERNAL;319 return HcclVmResult::HCCL_SIM_E_INTERNAL;
320 }320 }
321 std::vector<uint64_t> ccuResourceBaseAddr{};321 std::vector<uint64_t> ccuResourceBaseAddr{};
@@ -346,7 +346,7 @@ HcclVmResult StorageManager::InitAivResourceFromCompositeOpDetail(const sim::Com
346{346{
347 auto ret = AivResourceManager::GetInstance().Init(opDetail.rankId, opDetail.memInfo, GetRankSize());347 auto ret = AivResourceManager::GetInstance().Init(opDetail.rankId, opDetail.memInfo, GetRankSize());
348 if (ret != HcclVmResult::HCCL_SIM_SUCCESS) {348 if (ret != HcclVmResult::HCCL_SIM_SUCCESS) {
349- HCCL_VM_ERROR("[InitAivResourceFromCompositeOpDetail] init aiv resource failed, "349+ HCCL_VM_ERROR("init aiv resource failed, "
350 "rankId={}, opDetailId={}, memInfoId={}, ret={}",350 "rankId={}, opDetailId={}, memInfoId={}, ret={}",
351 opDetail.rankId, opDetail.detail.id, opDetail.memInfo.id, static_cast<int>(ret));351 opDetail.rankId, opDetail.detail.id, opDetail.memInfo.id, static_cast<int>(ret));
352 return ret;352 return ret;
@@ -364,27 +364,27 @@ void StorageManager::DumpAllRankInputOutput(std::vector<std::map<uint32_t, sim::
364{364{
365 const char* enableDumpData = std::getenv("HCCLVM_ENABLE_DUMP_DATA");365 const char* enableDumpData = std::getenv("HCCLVM_ENABLE_DUMP_DATA");
366 if (enableDumpData == nullptr || std::string(enableDumpData).empty() || std::string(enableDumpData) == "0") {366 if (enableDumpData == nullptr || std::string(enableDumpData).empty() || std::string(enableDumpData) == "0") {
367- HCCL_VM_DEBUG("[DumpAllRankInputOutput] HCCLVM_ENABLE_DUMP_DATA is not set");367+ HCCL_VM_DEBUG("HCCLVM_ENABLE_DUMP_DATA is not set");
368 return;368 return;
369 }369 }
370 auto rankSize = GetRankSize();370 auto rankSize = GetRankSize();
371 if (compositeDataMap.empty() || compositeDataMap[0].empty()) {371 if (compositeDataMap.empty() || compositeDataMap[0].empty()) {
372- HCCL_VM_ERROR("[DumpAllRankInputOutput] compositeDataMap is empty");372+ HCCL_VM_ERROR("compositeDataMap is empty");
373 return;373 return;
374 }374 }
375 std::string rootPath = FindRootPath();375 std::string rootPath = FindRootPath();
376 if (rootPath.empty()) {376 if (rootPath.empty()) {
377- HCCL_VM_ERROR("[DumpAllRankInputOutput] Failed to find root path");377+ HCCL_VM_ERROR("Failed to find root path");
378 return;378 return;
379 }379 }
380 std::string fullPath = rootPath + HCCLVM_ALL_RANK_INPUT_OUTPUT_FILE;380 std::string fullPath = rootPath + HCCLVM_ALL_RANK_INPUT_OUTPUT_FILE;
381 std::ofstream ofs(fullPath);381 std::ofstream ofs(fullPath);
382 if (!ofs.is_open()) {382 if (!ofs.is_open()) {
383- HCCL_VM_ERROR("[DumpAllRankInputOutput] Open file failed: {}", fullPath);383+ HCCL_VM_ERROR("Open file failed: {}", fullPath);
384 return;384 return;
385 }385 }
386 386 
387- HCCL_VM_INFO("[DumpAllRankInputOutput] Dump All Rank Input Output Data. rankSize={}", rankSize);387+ HCCL_VM_INFO("Dump All Rank Input Output Data. rankSize={}", rankSize);
388 for (size_t opIdx = 0; opIdx < compositeDataMap.size(); opIdx++) {388 for (size_t opIdx = 0; opIdx < compositeDataMap.size(); opIdx++) {
389 auto& rankTask = compositeDataMap[opIdx];389 auto& rankTask = compositeDataMap[opIdx];
390 if (rankTask.empty()) {390 if (rankTask.empty()) {
@@ -399,7 +399,7 @@ void StorageManager::DumpAllRankInputOutput(std::vector<std::map<uint32_t, sim::
399 sim::PhyMemBlock srcPhyMem{};399 sim::PhyMemBlock srcPhyMem{};
400 auto startAddr = sim::AcquireDevPtrInNoHostProcess((void*)(memInfo.inputAddr), srcPhyMem);400 auto startAddr = sim::AcquireDevPtrInNoHostProcess((void*)(memInfo.inputAddr), srcPhyMem);
401 if (startAddr == nullptr) {401 if (startAddr == nullptr) {
402- HCCL_VM_ERROR("[DumpAllRankInputOutput] fail addr(type={}, addr={:x})!",402+ HCCL_VM_ERROR("fail addr(type={}, addr={:x})!",
403 static_cast<int>(BufferType::INPUT),403 static_cast<int>(BufferType::INPUT),
404 memInfo.inputAddr);404 memInfo.inputAddr);
405 return;405 return;
@@ -408,7 +408,7 @@ void StorageManager::DumpAllRankInputOutput(std::vector<std::map<uint32_t, sim::
408 allRankInput[it.first].virtualAddr = memInfo.inputAddr;408 allRankInput[it.first].virtualAddr = memInfo.inputAddr;
409 allRankInput[it.first].phyAddr = (char*)startAddr;409 allRankInput[it.first].phyAddr = (char*)startAddr;
410 allRankInput[it.first].size = memInfo.inputSize;410 allRankInput[it.first].size = memInfo.inputSize;
411- HCCL_VM_INFO("[DumpAllRankInputOutput] opIdx={}, rankId={}, inputAddr={:x}, size={}",411+ HCCL_VM_INFO("opIdx={}, rankId={}, inputAddr={:x}, size={}",
412 opIdx, it.first, memInfo.inputAddr, memInfo.inputSize);412 opIdx, it.first, memInfo.inputAddr, memInfo.inputSize);
413 }413 }
414 414 
@@ -416,7 +416,7 @@ void StorageManager::DumpAllRankInputOutput(std::vector<std::map<uint32_t, sim::
416 sim::PhyMemBlock srcPhyMem{};416 sim::PhyMemBlock srcPhyMem{};
417 auto startAddr = sim::AcquireDevPtrInNoHostProcess((void*)(memInfo.outputAddr), srcPhyMem);417 auto startAddr = sim::AcquireDevPtrInNoHostProcess((void*)(memInfo.outputAddr), srcPhyMem);
418 if (startAddr == nullptr) {418 if (startAddr == nullptr) {
419- HCCL_VM_ERROR("[DumpAllRankInputOutput] fail addr(type={}, addr={:x})!",419+ HCCL_VM_ERROR("fail addr(type={}, addr={:x})!",
420 static_cast<int>(BufferType::OUTPUT),420 static_cast<int>(BufferType::OUTPUT),
421 memInfo.outputAddr);421 memInfo.outputAddr);
422 return;422 return;
@@ -425,7 +425,7 @@ void StorageManager::DumpAllRankInputOutput(std::vector<std::map<uint32_t, sim::
425 allRankOutput[it.first].virtualAddr = memInfo.outputAddr;425 allRankOutput[it.first].virtualAddr = memInfo.outputAddr;
426 allRankOutput[it.first].phyAddr = (char*)startAddr;426 allRankOutput[it.first].phyAddr = (char*)startAddr;
427 allRankOutput[it.first].size = memInfo.outputSize;427 allRankOutput[it.first].size = memInfo.outputSize;
428- HCCL_VM_INFO("[DumpAllRankInputOutput] opIdx={}, rankId={}, outputAddr={:x}, size={}",428+ HCCL_VM_INFO("opIdx={}, rankId={}, outputAddr={:x}, size={}",
429 opIdx, it.first, memInfo.outputAddr, memInfo.outputSize);429 opIdx, it.first, memInfo.outputAddr, memInfo.outputSize);
430 }430 }
431 431 
@@ -433,7 +433,7 @@ void StorageManager::DumpAllRankInputOutput(std::vector<std::map<uint32_t, sim::
433 sim::PhyMemBlock srcPhyMem{};433 sim::PhyMemBlock srcPhyMem{};
434 auto startAddr = sim::AcquireDevPtrInNoHostProcess((void*)(memInfo.cclAddr), srcPhyMem);434 auto startAddr = sim::AcquireDevPtrInNoHostProcess((void*)(memInfo.cclAddr), srcPhyMem);
435 if (startAddr == nullptr) {435 if (startAddr == nullptr) {
436- HCCL_VM_ERROR("[DumpAllRankInputOutput] fail addr(type={}, addr={:x})!",436+ HCCL_VM_ERROR("fail addr(type={}, addr={:x})!",
437 static_cast<int>(BufferType::CCL),437 static_cast<int>(BufferType::CCL),
438 memInfo.cclAddr);438 memInfo.cclAddr);
439 return;439 return;
@@ -442,14 +442,14 @@ void StorageManager::DumpAllRankInputOutput(std::vector<std::map<uint32_t, sim::
442 allRankCCL[it.first].virtualAddr = memInfo.cclAddr;442 allRankCCL[it.first].virtualAddr = memInfo.cclAddr;
443 allRankCCL[it.first].phyAddr = (char*)startAddr;443 allRankCCL[it.first].phyAddr = (char*)startAddr;
444 allRankCCL[it.first].size = memInfo.cclSize;444 allRankCCL[it.first].size = memInfo.cclSize;
445- HCCL_VM_INFO("[DumpAllRankInputOutput] opIdx={}, rankId={}, cclAddr={:x}, size={}",445+ HCCL_VM_INFO("opIdx={}, rankId={}, cclAddr={:x}, size={}",
446 opIdx, it.first, memInfo.cclAddr, memInfo.cclSize);446 opIdx, it.first, memInfo.cclAddr, memInfo.cclSize);
447 }447 }
448 }448 }
449 449 
450 const auto& firstComp = rankTask.begin()->second;450 const auto& firstComp = rankTask.begin()->second;
451 if (firstComp.detail.opDetail.size() < sizeof(::OpDetails)) {451 if (firstComp.detail.opDetail.size() < sizeof(::OpDetails)) {
452- HCCL_VM_ERROR("[DumpAllRankInputOutput] opDetail BLOB too small, opIdx={}, size={}",452+ HCCL_VM_ERROR("opDetail BLOB too small, opIdx={}, size={}",
453 opIdx, firstComp.detail.opDetail.size());453 opIdx, firstComp.detail.opDetail.size());
454 continue;454 continue;
455 }455 }
@@ -475,7 +475,7 @@ void StorageManager::DumpAllRankInputOutput(std::vector<std::map<uint32_t, sim::
475 }475 }
476 }476 }
477 ofs.close();477 ofs.close();
478- HCCL_VM_INFO("[DumpAllRankInputOutput] Dump All Rank Input Output Data success.");478+ HCCL_VM_INFO("Dump All Rank Input Output Data success.");
479}479}
480 480 
481void StorageManager::FlexiblePrintData(std::ofstream &ofs, const BufferInfo &buffer)481void StorageManager::FlexiblePrintData(std::ofstream &ofs, const BufferInfo &buffer)
Mtest/hccl_vm/src/plugin/runner/runner_worker/hccl_task_sequential_execute.cc+2-2
@@ -57,8 +57,8 @@ HcclVmResult SqeuentialExecutor::Execute()
57 }57 }
58 }58 }
59 }59 }
60- 60+ 
61- return HcclVmResult::HCCL_SIM_SUCCESS; 61+ return HcclVmResult::HCCL_SIM_SUCCESS;
62}62}
63 63 
64HcclVmResult SqeuentialExecutor::ExecuteOneTask(HcclTaskMetaData& task)64HcclVmResult SqeuentialExecutor::ExecuteOneTask(HcclTaskMetaData& task)
Mtest/hccl_vm/src/proxy/aclrt_context_stub.cc+7-9
@@ -8,6 +8,9 @@
8 * See LICENSE in the root of the software repository for the full text of the License.8 * See LICENSE in the root of the software repository for the full text of the License.
9 */9 */
10 10 
11+// 日志染色: 模块 tag (须在 include sim_log.h 之前)
12+#define HCCL_VM_MODULE "CTX_STUB"
13+ 
11#include <atomic>14#include <atomic>
12#include <cstdint>15#include <cstdint>
13#include <iostream>16#include <iostream>
@@ -21,11 +24,6 @@
21#include "sim_log.h"24#include "sim_log.h"
22#include "db_sim_runner_ops.h"25#include "db_sim_runner_ops.h"
23 26 
24-#define CTX_STUB_ERROR(format, ...) HCCL_VM_ERROR("[CTX_STUB]" format, ##__VA_ARGS__)
25-#define CTX_STUB_DEBUG(format, ...) HCCL_VM_DEBUG("[CTX_STUB]" format, ##__VA_ARGS__)
26-#define CTX_STUB_INFO(format, ...) HCCL_VM_INFO("[CTX_STUB]" format, ##__VA_ARGS__)
27-#define CTX_STUB_WARN(format, ...) HCCL_VM_WARN("[CTX_STUB]" format, ##__VA_ARGS__)
28-#define CTX_STUB_TRACE(format, ...) HCCL_VM_TRACE("[CTX_STUB]" format, ##__VA_ARGS__)
29 27 
30#ifdef __cplusplus28#ifdef __cplusplus
31extern "C" {29extern "C" {
@@ -40,7 +38,7 @@ aclError aclrtCreateContext(aclrtContext *context, int32_t deviceId)
40 return d.logic_id == (uint32_t)deviceId;38 return d.logic_id == (uint32_t)deviceId;
41 });39 });
42 if (!ret.second) {40 if (!ret.second) {
43- CTX_STUB_ERROR("device not found:{:d}", deviceId);41+ HCCL_VM_ERROR("device not found:{:d}", deviceId);
44 return HcclResult::HCCL_E_NOT_FOUND;42 return HcclResult::HCCL_E_NOT_FOUND;
45 }43 }
46 // 找不到context,则默认新增一个context44 // 找不到context,则默认新增一个context
@@ -67,7 +65,7 @@ aclError aclrtDestroyContext(aclrtContext context)
67 return stm.ctx_id == ctxId && stm.is_primary_default == 1;65 return stm.ctx_id == ctxId && stm.is_primary_default == 1;
68 });66 });
69 if (!ret.second) {67 if (!ret.second) {
70- CTX_STUB_ERROR("stream not found ctxId:{:d}", ctxId);68+ HCCL_VM_ERROR("stream not found ctxId:{:d}", ctxId);
71 return HcclResult::HCCL_E_NOT_FOUND;69 return HcclResult::HCCL_E_NOT_FOUND;
72 }70 }
73 71 
@@ -104,7 +102,7 @@ aclError aclrtCtxGetCurrentDefaultStream(aclrtStream *stream)
104 return stm.ctx_id == curCtxId && stm.is_primary_default == 1;102 return stm.ctx_id == curCtxId && stm.is_primary_default == 1;
105 });103 });
106 if (!stm.second) {104 if (!stm.second) {
107- CTX_STUB_ERROR("stream not found ctxId:{:d}", curCtxId);105+ HCCL_VM_ERROR("stream not found ctxId:{:d}", curCtxId);
108 return HcclResult::HCCL_E_NOT_FOUND;106 return HcclResult::HCCL_E_NOT_FOUND;
109 }107 }
110 108 
@@ -123,7 +121,7 @@ aclError aclrtGetPrimaryCtxState(int32_t deviceId, uint32_t *flags, int32_t *act
123 auto currCtx = RunnerDB::GetById<sim::Context>(runner.current_ctx_id);121 auto currCtx = RunnerDB::GetById<sim::Context>(runner.current_ctx_id);
124 if (!currCtx.has_value()) {122 if (!currCtx.has_value()) {
125 // not find123 // not find
126- CTX_STUB_ERROR("ctx not found:{:d}", runner.current_ctx_id);124+ HCCL_VM_ERROR("ctx not found:{:d}", runner.current_ctx_id);
127 return ACL_ERROR_INVALID_PARAM;125 return ACL_ERROR_INVALID_PARAM;
128 }126 }
129 127 
Mtest/hccl_vm/src/proxy/aclrt_device_stub.cc+72-74
@@ -8,6 +8,9 @@
8 * See LICENSE in the root of the software repository for the full text of the License.8 * See LICENSE in the root of the software repository for the full text of the License.
9 */9 */
10 10 
11+// 日志染色: 模块 tag (须在 include sim_log.h 之前)
12+#define HCCL_VM_MODULE "DEVICE_STUB"
13+ 
11#include <atomic>14#include <atomic>
12#include <csignal>15#include <csignal>
13#include <cstdint>16#include <cstdint>
@@ -30,11 +33,6 @@ uint64_t g_host_id;
30extern uint64_t g_cur_server_key;33extern uint64_t g_cur_server_key;
31 34 
32extern pid_t g_devicePid;35extern pid_t g_devicePid;
33-#define DEVICE_STUB_ERROR(format, ...) HCCL_VM_ERROR("[DEVICE_STUB]" format, ##__VA_ARGS__)
34-#define DEVICE_STUB_DEBUG(format, ...) HCCL_VM_DEBUG("[DEVICE_STUB]" format, ##__VA_ARGS__)
35-#define DEVICE_STUB_INFO(format, ...) HCCL_VM_INFO("[DEVICE_STUB]" format, ##__VA_ARGS__)
36-#define DEVICE_STUB_WARN(format, ...) HCCL_VM_WARN("[DEVICE_STUB]" format, ##__VA_ARGS__)
37-#define DEVICE_STUB_TRACE(format, ...) HCCL_VM_TRACE("[DEVICE_STUB]" format, ##__VA_ARGS__)
38 36 
39#ifdef __cplusplus37#ifdef __cplusplus
40extern "C" {38extern "C" {
@@ -56,13 +54,13 @@ HcclResult hrtGetDeviceIndexByPhyId(uint32_t devicePhyId, uint32_t &deviceLogicI
56 return d.server_id == g_cur_server_key && d.physical_id == (uint32_t)devicePhyId;54 return d.server_id == g_cur_server_key && d.physical_id == (uint32_t)devicePhyId;
57 });55 });
58 if (!ret.second) {56 if (!ret.second) {
59- DEVICE_STUB_ERROR("device not found by phyId:{:d}", devicePhyId);57+ HCCL_VM_ERROR("device not found by phyId:{:d}", devicePhyId);
60 return HcclResult::HCCL_E_NOT_FOUND;58 return HcclResult::HCCL_E_NOT_FOUND;
61 }59 }
62 deviceLogicId = ret.first.logic_id;60 deviceLogicId = ret.first.logic_id;
63 return HCCL_SUCCESS;61 return HCCL_SUCCESS;
64 } catch (const std::exception &e) {62 } catch (const std::exception &e) {
65- DEVICE_STUB_ERROR("exception:{}", e.what());63+ HCCL_VM_ERROR("exception:{}", e.what());
66 return HcclResult::HCCL_E_INTERNAL;64 return HcclResult::HCCL_E_INTERNAL;
67 }65 }
68}66}
@@ -70,11 +68,11 @@ HcclResult hrtGetDeviceIndexByPhyId(uint32_t devicePhyId, uint32_t &deviceLogicI
70aclError aclrtSetDevice(int32_t deviceId)68aclError aclrtSetDevice(int32_t deviceId)
71{69{
72 try {70 try {
73- DEVICE_STUB_DEBUG("set id:{:d}", deviceId);71+ HCCL_VM_DEBUG("set id:{:d}", deviceId);
74 uint32_t rankId;72 uint32_t rankId;
75 uint64_t serverId = 0;73 uint64_t serverId = 0;
76 if (!sim::GetRankIdByMPI(rankId, serverId)) {74 if (!sim::GetRankIdByMPI(rankId, serverId)) {
77- DEVICE_STUB_ERROR("get rankId by MPI fail serverId:{:d}", serverId);75+ HCCL_VM_ERROR("get rankId by MPI fail serverId:{:d}", serverId);
78 return ACL_ERROR_INVALID_PARAM;76 return ACL_ERROR_INVALID_PARAM;
79 }77 }
80 78 
@@ -82,21 +80,21 @@ aclError aclrtSetDevice(int32_t deviceId)
82 if (serverId == 0) {80 if (serverId == 0) {
83 auto devRet = sim::GetDeviceByRankId(rankId, device);81 auto devRet = sim::GetDeviceByRankId(rankId, device);
84 if (devRet != ACL_SUCCESS) {82 if (devRet != ACL_SUCCESS) {
85- DEVICE_STUB_ERROR("device not found by rankId:{:d}", rankId);83+ HCCL_VM_ERROR("device not found by rankId:{:d}", rankId);
86 return devRet;84 return devRet;
87 }85 }
88 serverId = device.server_id;86 serverId = device.server_id;
89- DEVICE_STUB_DEBUG("logicId:{:d} rankId:{:d} key:{:d}", device.logic_id, rankId, device.id);87+ HCCL_VM_DEBUG("logicId:{:d} rankId:{:d} key:{:d}", device.logic_id, rankId, device.id);
90 } else {88 } else {
91 auto ret = RunnerDB::GetOneByPred<sim::Device>([serverId, deviceId](const sim::Device &d) {89 auto ret = RunnerDB::GetOneByPred<sim::Device>([serverId, deviceId](const sim::Device &d) {
92 return d.server_id == serverId && d.logic_id == (uint32_t)deviceId;90 return d.server_id == serverId && d.logic_id == (uint32_t)deviceId;
93 });91 });
94 if (!ret.second) {92 if (!ret.second) {
95- DEVICE_STUB_ERROR("device not found logicId:{:d} serverId:{:d}", deviceId, serverId);93+ HCCL_VM_ERROR("device not found logicId:{:d} serverId:{:d}", deviceId, serverId);
96 return ACL_ERROR_INVALID_PARAM;94 return ACL_ERROR_INVALID_PARAM;
97 }95 }
98 device = ret.first;96 device = ret.first;
99- DEVICE_STUB_DEBUG("logicId:{:d} serverId:{:d} key:{:d}", device.logic_id, serverId, device.id);97+ HCCL_VM_DEBUG("logicId:{:d} serverId:{:d} key:{:d}", device.logic_id, serverId, device.id);
100 }98 }
101 99 
102 sim::Runner runner{};100 sim::Runner runner{};
@@ -132,7 +130,7 @@ aclError aclrtSetDevice(int32_t deviceId)
132 sim::SetCurrCtxTls(currCtxId);130 sim::SetCurrCtxTls(currCtxId);
133 return ACL_SUCCESS;131 return ACL_SUCCESS;
134 } catch (const std::exception &e) {132 } catch (const std::exception &e) {
135- DEVICE_STUB_ERROR("exception:{}", e.what());133+ HCCL_VM_ERROR("exception:{}", e.what());
136 return ACL_ERROR_INTERNAL_ERROR;134 return ACL_ERROR_INTERNAL_ERROR;
137 }135 }
138}136}
@@ -140,7 +138,7 @@ aclError aclrtSetDevice(int32_t deviceId)
140aclError aclrtResetDevice(int32_t deviceId)138aclError aclrtResetDevice(int32_t deviceId)
141{139{
142 try {140 try {
143- DEVICE_STUB_INFO("deviceId:{:d}", deviceId);141+ HCCL_VM_INFO("deviceId:{:d}", deviceId);
144 sim::Runner runner{};142 sim::Runner runner{};
145 if (!sim::GetCurrRunnerTls(0, runner)) {143 if (!sim::GetCurrRunnerTls(0, runner)) {
146 return ACL_ERROR_INVALID_PARAM;144 return ACL_ERROR_INVALID_PARAM;
@@ -151,7 +149,7 @@ aclError aclrtResetDevice(int32_t deviceId)
151 }149 }
152 auto currCtx = RunnerDB::GetById<sim::Context>(curCtxId);150 auto currCtx = RunnerDB::GetById<sim::Context>(curCtxId);
153 if (!currCtx.has_value()) {151 if (!currCtx.has_value()) {
154- DEVICE_STUB_ERROR("ctx not found:{:d}", runner.current_ctx_id);152+ HCCL_VM_ERROR("ctx not found:{:d}", runner.current_ctx_id);
155 return ACL_ERROR_INVALID_PARAM;153 return ACL_ERROR_INVALID_PARAM;
156 }154 }
157 155 
@@ -159,7 +157,7 @@ aclError aclrtResetDevice(int32_t deviceId)
159 return stream.ctx_id == curCtxId && stream.is_primary_default == 1;157 return stream.ctx_id == curCtxId && stream.is_primary_default == 1;
160 });158 });
161 if (!streamRet.second) {159 if (!streamRet.second) {
162- DEVICE_STUB_ERROR("stream not found ctxId:{:d}", curCtxId);160+ HCCL_VM_ERROR("stream not found ctxId:{:d}", curCtxId);
163 return ACL_ERROR_INVALID_PARAM;161 return ACL_ERROR_INVALID_PARAM;
164 }162 }
165 163 
@@ -174,7 +172,7 @@ aclError aclrtResetDevice(int32_t deviceId)
174 sim::SetCurrCtxTls(curCtxId);172 sim::SetCurrCtxTls(curCtxId);
175 return ACL_SUCCESS;173 return ACL_SUCCESS;
176 } catch (const std::exception &e) {174 } catch (const std::exception &e) {
177- DEVICE_STUB_ERROR("exception:{}", e.what());175+ HCCL_VM_ERROR("exception:{}", e.what());
178 return ACL_ERROR_INTERNAL_ERROR;176 return ACL_ERROR_INTERNAL_ERROR;
179 }177 }
180}178}
@@ -182,7 +180,7 @@ aclError aclrtResetDevice(int32_t deviceId)
182aclError aclrtResetDeviceForce(int32_t deviceId)180aclError aclrtResetDeviceForce(int32_t deviceId)
183{181{
184 try {182 try {
185- DEVICE_STUB_DEBUG("stream not found deviceId:{:d}", deviceId);183+ HCCL_VM_DEBUG("stream not found deviceId:{:d}", deviceId);
186 sim::Runner runner{};184 sim::Runner runner{};
187 if (!sim::GetCurrRunnerTls(0, runner)) {185 if (!sim::GetCurrRunnerTls(0, runner)) {
188 return ACL_ERROR_INVALID_PARAM;186 return ACL_ERROR_INVALID_PARAM;
@@ -193,7 +191,7 @@ aclError aclrtResetDeviceForce(int32_t deviceId)
193 }191 }
194 auto currCtx = RunnerDB::GetById<sim::Context>(curCtxId);192 auto currCtx = RunnerDB::GetById<sim::Context>(curCtxId);
195 if (!currCtx.has_value()) {193 if (!currCtx.has_value()) {
196- DEVICE_STUB_ERROR("ctx not found:{:d}", runner.current_ctx_id);194+ HCCL_VM_ERROR("ctx not found:{:d}", runner.current_ctx_id);
197 return ACL_ERROR_INVALID_PARAM;195 return ACL_ERROR_INVALID_PARAM;
198 }196 }
199 197 
@@ -201,7 +199,7 @@ aclError aclrtResetDeviceForce(int32_t deviceId)
201 return stream.ctx_id == curCtxId && stream.is_primary_default == 1;199 return stream.ctx_id == curCtxId && stream.is_primary_default == 1;
202 });200 });
203 if (!streamRet.second) {201 if (!streamRet.second) {
204- DEVICE_STUB_ERROR("stream not found ctxId:{:d}", curCtxId);202+ HCCL_VM_ERROR("stream not found ctxId:{:d}", curCtxId);
205 return ACL_ERROR_INVALID_PARAM;203 return ACL_ERROR_INVALID_PARAM;
206 }204 }
207 205 
@@ -211,7 +209,7 @@ aclError aclrtResetDeviceForce(int32_t deviceId)
211 sim::SetCurrCtxTls(curCtxId);209 sim::SetCurrCtxTls(curCtxId);
212 return ACL_SUCCESS;210 return ACL_SUCCESS;
213 } catch (const std::exception &e) {211 } catch (const std::exception &e) {
214- DEVICE_STUB_ERROR("exception:{}", e.what());212+ HCCL_VM_ERROR("exception:{}", e.what());
215 return ACL_ERROR_INTERNAL_ERROR;213 return ACL_ERROR_INTERNAL_ERROR;
216 }214 }
217}215}
@@ -225,20 +223,20 @@ aclError aclrtGetDevice(int32_t* device)
225 }223 }
226 auto currCtx = RunnerDB::GetById<sim::Context>(runner.current_ctx_id);224 auto currCtx = RunnerDB::GetById<sim::Context>(runner.current_ctx_id);
227 if (!currCtx.has_value()) {225 if (!currCtx.has_value()) {
228- DEVICE_STUB_ERROR("ctx not found:{:d}", runner.current_ctx_id);226+ HCCL_VM_ERROR("ctx not found:{:d}", runner.current_ctx_id);
229 return ACL_ERROR_INVALID_PARAM;227 return ACL_ERROR_INVALID_PARAM;
230 }228 }
231 229 
232 auto devRes = RunnerDB::GetById<sim::Device>(currCtx->device_id);230 auto devRes = RunnerDB::GetById<sim::Device>(currCtx->device_id);
233 if (!devRes.has_value()) {231 if (!devRes.has_value()) {
234- DEVICE_STUB_ERROR("device not found:{:d}", currCtx->device_id);232+ HCCL_VM_ERROR("device not found:{:d}", currCtx->device_id);
235 return ACL_ERROR_INVALID_PARAM;233 return ACL_ERROR_INVALID_PARAM;
236 }234 }
237 *device = devRes->logic_id;235 *device = devRes->logic_id;
238- DEVICE_STUB_DEBUG("id:{:d}", devRes->logic_id);236+ HCCL_VM_DEBUG("id:{:d}", devRes->logic_id);
239 return ACL_SUCCESS;237 return ACL_SUCCESS;
240 } catch (const std::exception &e) {238 } catch (const std::exception &e) {
241- DEVICE_STUB_ERROR("exception:{}", e.what());239+ HCCL_VM_ERROR("exception:{}", e.what());
242 return ACL_ERROR_INTERNAL_ERROR;240 return ACL_ERROR_INTERNAL_ERROR;
243 }241 }
244}242}
@@ -262,13 +260,13 @@ aclError aclrtSetTsDevice(aclrtTsId tsId)
262 }260 }
263 auto currCtx = RunnerDB::GetById<sim::Context>(runner.current_ctx_id);261 auto currCtx = RunnerDB::GetById<sim::Context>(runner.current_ctx_id);
264 if (!currCtx.has_value()) {262 if (!currCtx.has_value()) {
265- DEVICE_STUB_ERROR("ctx not found:{:d}", runner.current_ctx_id);263+ HCCL_VM_ERROR("ctx not found:{:d}", runner.current_ctx_id);
266 return ACL_ERROR_INVALID_PARAM;264 return ACL_ERROR_INVALID_PARAM;
267 }265 }
268 266 
269 auto devRes = RunnerDB::GetById<sim::Device>(currCtx->device_id);267 auto devRes = RunnerDB::GetById<sim::Device>(currCtx->device_id);
270 if (!devRes.has_value()) {268 if (!devRes.has_value()) {
271- DEVICE_STUB_ERROR("device not found:{:d}", currCtx->device_id);269+ HCCL_VM_ERROR("device not found:{:d}", currCtx->device_id);
272 return ACL_ERROR_INVALID_PARAM;270 return ACL_ERROR_INVALID_PARAM;
273 }271 }
274 272 
@@ -279,7 +277,7 @@ aclError aclrtSetTsDevice(aclrtTsId tsId)
279 277 
280 return ACL_SUCCESS;278 return ACL_SUCCESS;
281 } catch (const std::exception &e) {279 } catch (const std::exception &e) {
282- DEVICE_STUB_ERROR("exception:{}", e.what());280+ HCCL_VM_ERROR("exception:{}", e.what());
283 return ACL_ERROR_INTERNAL_ERROR;281 return ACL_ERROR_INTERNAL_ERROR;
284 }282 }
285}283}
@@ -292,13 +290,13 @@ aclError aclrtGetDeviceCount(uint32_t *count)
292 });290 });
293 291 
294 if (devs.empty()) {292 if (devs.empty()) {
295- DEVICE_STUB_ERROR("devices not found");293+ HCCL_VM_ERROR("devices not found");
296 return ACL_ERROR_INVALID_PARAM;294 return ACL_ERROR_INVALID_PARAM;
297 }295 }
298 *count = devs.size();296 *count = devs.size();
299 return ACL_SUCCESS;297 return ACL_SUCCESS;
300 } catch (const std::exception &e) {298 } catch (const std::exception &e) {
301- DEVICE_STUB_ERROR("exception:{}", e.what());299+ HCCL_VM_ERROR("exception:{}", e.what());
302 return ACL_ERROR_INTERNAL_ERROR;300 return ACL_ERROR_INTERNAL_ERROR;
303 }301 }
304}302}
@@ -310,7 +308,7 @@ aclError aclrtGetDeviceUtilizationRate(int32_t deviceId, aclrtUtilizationInfo *u
310 return d.logic_id == deviceId;308 return d.logic_id == deviceId;
311 });309 });
312 if (!ret.second) {310 if (!ret.second) {
313- DEVICE_STUB_ERROR("device not found by phyId:{:d}", deviceId);311+ HCCL_VM_ERROR("device not found by phyId:{:d}", deviceId);
314 return 0;312 return 0;
315 }313 }
316 314 
@@ -320,7 +318,7 @@ aclError aclrtGetDeviceUtilizationRate(int32_t deviceId, aclrtUtilizationInfo *u
320 utilizationInfo->memoryUtilization = 20;318 utilizationInfo->memoryUtilization = 20;
321 return ACL_SUCCESS;319 return ACL_SUCCESS;
322 } catch (const std::exception &e) {320 } catch (const std::exception &e) {
323- DEVICE_STUB_ERROR("exception:{}", e.what());321+ HCCL_VM_ERROR("exception:{}", e.what());
324 return ACL_ERROR_INTERNAL_ERROR;322 return ACL_ERROR_INTERNAL_ERROR;
325 }323 }
326}324}
@@ -332,13 +330,13 @@ aclError aclrtQueryDeviceStatus(int32_t deviceId, aclrtDeviceStatus *deviceStatu
332 return d.logic_id == (uint32_t)deviceId;330 return d.logic_id == (uint32_t)deviceId;
333 });331 });
334 if (!ret.second) {332 if (!ret.second) {
335- DEVICE_STUB_ERROR("device not found logicId:{:d}", deviceId);333+ HCCL_VM_ERROR("device not found logicId:{:d}", deviceId);
336 return HcclResult::HCCL_E_NOT_FOUND;334 return HcclResult::HCCL_E_NOT_FOUND;
337 }335 }
338 *deviceStatus = (aclrtDeviceStatus)ret.first.status;336 *deviceStatus = (aclrtDeviceStatus)ret.first.status;
339 return HcclVmResult::HCCL_SIM_SUCCESS;337 return HcclVmResult::HCCL_SIM_SUCCESS;
340 } catch (const std::exception &e) {338 } catch (const std::exception &e) {
341- DEVICE_STUB_ERROR("exception:{}", e.what());339+ HCCL_VM_ERROR("exception:{}", e.what());
342 return ACL_ERROR_INTERNAL_ERROR;340 return ACL_ERROR_INTERNAL_ERROR;
343 }341 }
344}342}
@@ -351,17 +349,17 @@ const char *aclrtGetSocName()
351 return d.server_id == 1;349 return d.server_id == 1;
352 });350 });
353 if (!devRes.second) {351 if (!devRes.second) {
354- DEVICE_STUB_ERROR("device not found serverId:1");352+ HCCL_VM_ERROR("device not found serverId:1");
355 return "";353 return "";
356 }354 }
357 355 
358 thread_local static char SocName[128] = {0};356 thread_local static char SocName[128] = {0};
359 memcpy(SocName, devRes.first.soc_version, strlen(devRes.first.soc_version));357 memcpy(SocName, devRes.first.soc_version, strlen(devRes.first.soc_version));
360 SocName[strlen(devRes.first.soc_version)] = '\0';358 SocName[strlen(devRes.first.soc_version)] = '\0';
361- DEVICE_STUB_DEBUG("soc:{}", devRes.first.soc_version);359+ HCCL_VM_DEBUG("soc:{}", devRes.first.soc_version);
362 return SocName;360 return SocName;
363 } catch (const std::exception &e) {361 } catch (const std::exception &e) {
364- DEVICE_STUB_ERROR("exception:{}", e.what());362+ HCCL_VM_ERROR("exception:{}", e.what());
365 static thread_local char s_errMsg[] = "";363 static thread_local char s_errMsg[] = "";
366 return s_errMsg;364 return s_errMsg;
367 }365 }
@@ -376,7 +374,7 @@ aclError aclrtSetDeviceSatMode(aclrtFloatOverflowMode mode)
376 }374 }
377 auto currCtx = RunnerDB::GetById<sim::Context>(runner.current_ctx_id);375 auto currCtx = RunnerDB::GetById<sim::Context>(runner.current_ctx_id);
378 if (!currCtx.has_value()) {376 if (!currCtx.has_value()) {
379- DEVICE_STUB_ERROR("ctx not found:{:d}", runner.current_ctx_id);377+ HCCL_VM_ERROR("ctx not found:{:d}", runner.current_ctx_id);
380 return ACL_ERROR_INVALID_PARAM;378 return ACL_ERROR_INVALID_PARAM;
381 }379 }
382 380 
@@ -384,7 +382,7 @@ aclError aclrtSetDeviceSatMode(aclrtFloatOverflowMode mode)
384 RunnerDB::Update<sim::Device>(curDevId, [curDevId, mode](sim::Device &dev) { dev.overflow_mode = mode; });382 RunnerDB::Update<sim::Device>(curDevId, [curDevId, mode](sim::Device &dev) { dev.overflow_mode = mode; });
385 return ACL_SUCCESS;383 return ACL_SUCCESS;
386 } catch (const std::exception &e) {384 } catch (const std::exception &e) {
387- DEVICE_STUB_ERROR("exception:{}", e.what());385+ HCCL_VM_ERROR("exception:{}", e.what());
388 return ACL_ERROR_INTERNAL_ERROR;386 return ACL_ERROR_INTERNAL_ERROR;
389 }387 }
390}388}
@@ -398,19 +396,19 @@ aclError aclrtGetDeviceSatMode(aclrtFloatOverflowMode *mode)
398 }396 }
399 auto currCtx = RunnerDB::GetById<sim::Context>(runner.current_ctx_id);397 auto currCtx = RunnerDB::GetById<sim::Context>(runner.current_ctx_id);
400 if (!currCtx.has_value()) {398 if (!currCtx.has_value()) {
401- DEVICE_STUB_ERROR("ctx not found:{:d}", runner.current_ctx_id);399+ HCCL_VM_ERROR("ctx not found:{:d}", runner.current_ctx_id);
402 return ACL_ERROR_INVALID_PARAM;400 return ACL_ERROR_INVALID_PARAM;
403 }401 }
404 402 
405 auto dev = RunnerDB::GetById<sim::Device>(currCtx->device_id);403 auto dev = RunnerDB::GetById<sim::Device>(currCtx->device_id);
406 if (!dev.has_value()) {404 if (!dev.has_value()) {
407- DEVICE_STUB_ERROR("device not found:{:d}", currCtx->device_id);405+ HCCL_VM_ERROR("device not found:{:d}", currCtx->device_id);
408 return ACL_ERROR_INVALID_PARAM;406 return ACL_ERROR_INVALID_PARAM;
409 }407 }
410 *mode = (aclrtFloatOverflowMode)dev->overflow_mode;408 *mode = (aclrtFloatOverflowMode)dev->overflow_mode;
411 return ACL_SUCCESS;409 return ACL_SUCCESS;
412 } catch (const std::exception &e) {410 } catch (const std::exception &e) {
413- DEVICE_STUB_ERROR("exception:{}", e.what());411+ HCCL_VM_ERROR("exception:{}", e.what());
414 return ACL_ERROR_INTERNAL_ERROR;412 return ACL_ERROR_INTERNAL_ERROR;
415 }413 }
416}414}
@@ -422,7 +420,7 @@ aclError aclrtDeviceCanAccessPeer(int32_t *canAccessPeer, int32_t deviceId, int3
422 return d.logic_id == deviceId;420 return d.logic_id == deviceId;
423 });421 });
424 if (!dev1.second) {422 if (!dev1.second) {
425- DEVICE_STUB_ERROR("device not found logicId:{:d}", deviceId);423+ HCCL_VM_ERROR("device not found logicId:{:d}", deviceId);
426 return ACL_ERROR_INVALID_PARAM;424 return ACL_ERROR_INVALID_PARAM;
427 }425 }
428 426 
@@ -430,7 +428,7 @@ aclError aclrtDeviceCanAccessPeer(int32_t *canAccessPeer, int32_t deviceId, int3
430 return d.logic_id == peerDeviceId;428 return d.logic_id == peerDeviceId;
431 });429 });
432 if (!dev2.second) {430 if (!dev2.second) {
433- DEVICE_STUB_ERROR("device not found logicId:{:d}", peerDeviceId);431+ HCCL_VM_ERROR("device not found logicId:{:d}", peerDeviceId);
434 return ACL_ERROR_INVALID_PARAM;432 return ACL_ERROR_INVALID_PARAM;
435 }433 }
436 434 
@@ -441,14 +439,14 @@ aclError aclrtDeviceCanAccessPeer(int32_t *canAccessPeer, int32_t deviceId, int3
441 return devConn.src_dev_id == dev1Id && devConn.dst_dev_id == dev2Id;439 return devConn.src_dev_id == dev1Id && devConn.dst_dev_id == dev2Id;
442 });440 });
443 if (!ret.second) {441 if (!ret.second) {
444- DEVICE_STUB_ERROR("connection not found src:{:d} dst:{:d}", dev1Id, dev2Id);442+ HCCL_VM_ERROR("connection not found src:{:d} dst:{:d}", dev1Id, dev2Id);
445 return HcclResult::HCCL_E_NOT_FOUND;443 return HcclResult::HCCL_E_NOT_FOUND;
446 }444 }
447 445 
448 *canAccessPeer = (int32_t)ret.first.access_by_remote;446 *canAccessPeer = (int32_t)ret.first.access_by_remote;
449 return HcclVmResult::HCCL_SIM_SUCCESS;447 return HcclVmResult::HCCL_SIM_SUCCESS;
450 } catch (const std::exception &e) {448 } catch (const std::exception &e) {
451- DEVICE_STUB_ERROR("exception:{}", e.what());449+ HCCL_VM_ERROR("exception:{}", e.what());
452 return ACL_ERROR_INTERNAL_ERROR;450 return ACL_ERROR_INTERNAL_ERROR;
453 }451 }
454}452}
@@ -463,18 +461,18 @@ aclError aclrtDeviceEnablePeerAccess(int32_t peerDeviceId, uint32_t flags)
463 }461 }
464 auto currCtx = RunnerDB::GetById<sim::Context>(runner.current_ctx_id);462 auto currCtx = RunnerDB::GetById<sim::Context>(runner.current_ctx_id);
465 if (!currCtx.has_value()) {463 if (!currCtx.has_value()) {
466- DEVICE_STUB_ERROR("ctx not found:{:d}", runner.current_ctx_id);464+ HCCL_VM_ERROR("ctx not found:{:d}", runner.current_ctx_id);
467 return ACL_ERROR_INVALID_PARAM;465 return ACL_ERROR_INVALID_PARAM;
468 }466 }
469 467 
470 auto dev1 = RunnerDB::GetById<sim::Device>(currCtx->device_id);468 auto dev1 = RunnerDB::GetById<sim::Device>(currCtx->device_id);
471 if (!dev1.has_value()) {469 if (!dev1.has_value()) {
472- DEVICE_STUB_ERROR("device not found:{:d}", currCtx->device_id);470+ HCCL_VM_ERROR("device not found:{:d}", currCtx->device_id);
473 return ACL_ERROR_INVALID_PARAM;471 return ACL_ERROR_INVALID_PARAM;
474 }472 }
475 473 
476 if (dev1->logic_id == peerDeviceId) {474 if (dev1->logic_id == peerDeviceId) {
477- DEVICE_STUB_ERROR("invalid peerId:{:d}", peerDeviceId);475+ HCCL_VM_ERROR("invalid peerId:{:d}", peerDeviceId);
478 return ACL_ERROR_INVALID_PARAM;476 return ACL_ERROR_INVALID_PARAM;
479 }477 }
480 478 
@@ -482,7 +480,7 @@ aclError aclrtDeviceEnablePeerAccess(int32_t peerDeviceId, uint32_t flags)
482 return d.logic_id == peerDeviceId;480 return d.logic_id == peerDeviceId;
483 });481 });
484 if (!dev2.second) {482 if (!dev2.second) {
485- DEVICE_STUB_ERROR("device not found logicId:{:d}", peerDeviceId);483+ HCCL_VM_ERROR("device not found logicId:{:d}", peerDeviceId);
486 return ACL_ERROR_INVALID_PARAM;484 return ACL_ERROR_INVALID_PARAM;
487 }485 }
488 486 
@@ -492,14 +490,14 @@ aclError aclrtDeviceEnablePeerAccess(int32_t peerDeviceId, uint32_t flags)
492 return devConn.src_dev_id == dev1Id && devConn.dst_dev_id == dev2Id;490 return devConn.src_dev_id == dev1Id && devConn.dst_dev_id == dev2Id;
493 });491 });
494 if (!ret.second) {492 if (!ret.second) {
495- DEVICE_STUB_ERROR("connection not found src:{:d} dst:{:d}", dev1Id, dev2Id);493+ HCCL_VM_ERROR("connection not found src:{:d} dst:{:d}", dev1Id, dev2Id);
496 return HcclResult::HCCL_E_NOT_FOUND;494 return HcclResult::HCCL_E_NOT_FOUND;
497 }495 }
498 496 
499 RunnerDB::Update<sim::DeviceConnection>(ret.first.id, [](sim::DeviceConnection &devConn) { devConn.access_by_remote = 1;});497 RunnerDB::Update<sim::DeviceConnection>(ret.first.id, [](sim::DeviceConnection &devConn) { devConn.access_by_remote = 1;});
500 return ACL_SUCCESS;498 return ACL_SUCCESS;
501 } catch (const std::exception &e) {499 } catch (const std::exception &e) {
502- DEVICE_STUB_ERROR("exception:{}", e.what());500+ HCCL_VM_ERROR("exception:{}", e.what());
503 return ACL_ERROR_INTERNAL_ERROR;501 return ACL_ERROR_INTERNAL_ERROR;
504 }502 }
505}503}
@@ -513,18 +511,18 @@ aclError aclrtDeviceDisablePeerAccess(int32_t peerDeviceId)
513 }511 }
514 auto currCtx = RunnerDB::GetById<sim::Context>(runner.current_ctx_id);512 auto currCtx = RunnerDB::GetById<sim::Context>(runner.current_ctx_id);
515 if (!currCtx.has_value()) {513 if (!currCtx.has_value()) {
516- DEVICE_STUB_ERROR("ctx not found:{:d}", runner.current_ctx_id);514+ HCCL_VM_ERROR("ctx not found:{:d}", runner.current_ctx_id);
517 return ACL_ERROR_INVALID_PARAM;515 return ACL_ERROR_INVALID_PARAM;
518 }516 }
519 517 
520 auto dev1 = RunnerDB::GetById<sim::Device>(currCtx->device_id);518 auto dev1 = RunnerDB::GetById<sim::Device>(currCtx->device_id);
521 if (!dev1.has_value()) {519 if (!dev1.has_value()) {
522- DEVICE_STUB_ERROR("device not found:{:d}", currCtx->device_id);520+ HCCL_VM_ERROR("device not found:{:d}", currCtx->device_id);
523 return ACL_ERROR_INVALID_PARAM;521 return ACL_ERROR_INVALID_PARAM;
524 }522 }
525 523 
526 if (dev1->logic_id == peerDeviceId) {524 if (dev1->logic_id == peerDeviceId) {
527- DEVICE_STUB_ERROR("invalid peerId:{:d}", peerDeviceId);525+ HCCL_VM_ERROR("invalid peerId:{:d}", peerDeviceId);
528 return ACL_ERROR_INVALID_PARAM;526 return ACL_ERROR_INVALID_PARAM;
529 }527 }
530 528 
@@ -532,7 +530,7 @@ aclError aclrtDeviceDisablePeerAccess(int32_t peerDeviceId)
532 return d.logic_id == peerDeviceId;530 return d.logic_id == peerDeviceId;
533 });531 });
534 if (!dev2.second) {532 if (!dev2.second) {
535- DEVICE_STUB_ERROR("device not found logicId:{:d}", peerDeviceId);533+ HCCL_VM_ERROR("device not found logicId:{:d}", peerDeviceId);
536 return ACL_ERROR_INVALID_PARAM;534 return ACL_ERROR_INVALID_PARAM;
537 }535 }
538 536 
@@ -542,14 +540,14 @@ aclError aclrtDeviceDisablePeerAccess(int32_t peerDeviceId)
542 return devConn.src_dev_id == dev1Id && devConn.dst_dev_id == dev2Id;540 return devConn.src_dev_id == dev1Id && devConn.dst_dev_id == dev2Id;
543 });541 });
544 if (!ret.second) {542 if (!ret.second) {
545- DEVICE_STUB_ERROR("connection not found src:{:d} dst:{:d}", dev1Id, dev2Id);543+ HCCL_VM_ERROR("connection not found src:{:d} dst:{:d}", dev1Id, dev2Id);
546 return ACL_ERROR_INVALID_PARAM;544 return ACL_ERROR_INVALID_PARAM;
547 }545 }
548 546 
549 RunnerDB::Update<sim::DeviceConnection>(ret.first.id, [](sim::DeviceConnection &devConn) { devConn.access_by_remote = 0;});547 RunnerDB::Update<sim::DeviceConnection>(ret.first.id, [](sim::DeviceConnection &devConn) { devConn.access_by_remote = 0;});
550 return ACL_SUCCESS;548 return ACL_SUCCESS;
551 } catch (const std::exception &e) {549 } catch (const std::exception &e) {
552- DEVICE_STUB_ERROR("exception:{}", e.what());550+ HCCL_VM_ERROR("exception:{}", e.what());
553 return ACL_ERROR_INTERNAL_ERROR;551 return ACL_ERROR_INTERNAL_ERROR;
554 }552 }
555}553}
@@ -561,13 +559,13 @@ aclError aclrtGetOverflowStatus(void *outputAddr, size_t outputSize, aclrtStream
561 uint64_t streamIdx = (uint64_t)(uintptr_t)stream;559 uint64_t streamIdx = (uint64_t)(uintptr_t)stream;
562 auto stmRes = RunnerDB::GetById<sim::Stream>(streamIdx);560 auto stmRes = RunnerDB::GetById<sim::Stream>(streamIdx);
563 if (!stmRes.has_value()) {561 if (!stmRes.has_value()) {
564- DEVICE_STUB_ERROR("stream not found:{:d}", streamIdx);562+ HCCL_VM_ERROR("stream not found:{:d}", streamIdx);
565 return ACL_ERROR_INVALID_PARAM;563 return ACL_ERROR_INVALID_PARAM;
566 }564 }
567 565 
568 auto ctxRes = RunnerDB::GetById<sim::Context>(stmRes->ctx_id);566 auto ctxRes = RunnerDB::GetById<sim::Context>(stmRes->ctx_id);
569 if (!ctxRes.has_value()) {567 if (!ctxRes.has_value()) {
570- DEVICE_STUB_ERROR("ctx not found:{:d}", stmRes->ctx_id);568+ HCCL_VM_ERROR("ctx not found:{:d}", stmRes->ctx_id);
571 return ACL_ERROR_INVALID_PARAM;569 return ACL_ERROR_INVALID_PARAM;
572 }570 }
573 571 
@@ -576,7 +574,7 @@ aclError aclrtGetOverflowStatus(void *outputAddr, size_t outputSize, aclrtStream
576 return dev.device_id == deviceIdx;574 return dev.device_id == deviceIdx;
577 });575 });
578 if (!devStatusRes.second) {576 if (!devStatusRes.second) {
579- DEVICE_STUB_ERROR("device not found:{:d}", deviceIdx);577+ HCCL_VM_ERROR("device not found:{:d}", deviceIdx);
580 return ACL_ERROR_INVALID_PARAM;578 return ACL_ERROR_INVALID_PARAM;
581 }579 }
582 580 
@@ -585,7 +583,7 @@ aclError aclrtGetOverflowStatus(void *outputAddr, size_t outputSize, aclrtStream
585 583 
586 return ACL_SUCCESS;584 return ACL_SUCCESS;
587 } catch (const std::exception &e) {585 } catch (const std::exception &e) {
588- DEVICE_STUB_ERROR("exception:{}", e.what());586+ HCCL_VM_ERROR("exception:{}", e.what());
589 return ACL_ERROR_INTERNAL_ERROR;587 return ACL_ERROR_INTERNAL_ERROR;
590 }588 }
591}589}
@@ -596,13 +594,13 @@ aclError aclrtResetOverflowStatus(aclrtStream stream)
596 uint64_t streamIdx = (uint64_t)(uintptr_t)stream;594 uint64_t streamIdx = (uint64_t)(uintptr_t)stream;
597 auto stmRes = RunnerDB::GetById<sim::Stream>(streamIdx);595 auto stmRes = RunnerDB::GetById<sim::Stream>(streamIdx);
598 if (!stmRes.has_value()) {596 if (!stmRes.has_value()) {
599- DEVICE_STUB_ERROR("stream not found:{:d}", streamIdx);597+ HCCL_VM_ERROR("stream not found:{:d}", streamIdx);
600 return ACL_ERROR_INVALID_PARAM;598 return ACL_ERROR_INVALID_PARAM;
601 }599 }
602 600 
603 auto ctxRes = RunnerDB::GetById<sim::Context>(stmRes->ctx_id);601 auto ctxRes = RunnerDB::GetById<sim::Context>(stmRes->ctx_id);
604 if (!ctxRes.has_value()) {602 if (!ctxRes.has_value()) {
605- DEVICE_STUB_ERROR("ctx not found:{:d}", stmRes->ctx_id);603+ HCCL_VM_ERROR("ctx not found:{:d}", stmRes->ctx_id);
606 return ACL_ERROR_INVALID_PARAM;604 return ACL_ERROR_INVALID_PARAM;
607 }605 }
608 606 
@@ -610,13 +608,13 @@ aclError aclrtResetOverflowStatus(aclrtStream stream)
610 auto devStatusRes = RunnerDB::GetOneByPred<sim::DeviceStatus>(608 auto devStatusRes = RunnerDB::GetOneByPred<sim::DeviceStatus>(
611 [deviceIdx](const sim::DeviceStatus &dev) { return dev.device_id == deviceIdx; });609 [deviceIdx](const sim::DeviceStatus &dev) { return dev.device_id == deviceIdx; });
612 if (!devStatusRes.second) {610 if (!devStatusRes.second) {
613- DEVICE_STUB_ERROR("device not found:{:d}", deviceIdx);611+ HCCL_VM_ERROR("device not found:{:d}", deviceIdx);
614 return ACL_ERROR_INVALID_PARAM;612 return ACL_ERROR_INVALID_PARAM;
615 }613 }
616 RunnerDB::Update<sim::DeviceStatus>(devStatusRes.first.id, [](sim::DeviceStatus &devStatus) { devStatus.overflow_status = 0;});614 RunnerDB::Update<sim::DeviceStatus>(devStatusRes.first.id, [](sim::DeviceStatus &devStatus) { devStatus.overflow_status = 0;});
617 return ACL_SUCCESS;615 return ACL_SUCCESS;
618 } catch (const std::exception &e) {616 } catch (const std::exception &e) {
619- DEVICE_STUB_ERROR("exception:{}", e.what());617+ HCCL_VM_ERROR("exception:{}", e.what());
620 return ACL_ERROR_INTERNAL_ERROR;618 return ACL_ERROR_INTERNAL_ERROR;
621 }619 }
622}620}
@@ -668,7 +666,7 @@ aclError aclrtGetDevicesTopo(uint32_t deviceId, uint32_t otherDeviceId, uint64_t
668 return d.logic_id == deviceId;666 return d.logic_id == deviceId;
669 });667 });
670 if (!dev1.second) {668 if (!dev1.second) {
671- DEVICE_STUB_ERROR("device not found logicId:{:d}", deviceId);669+ HCCL_VM_ERROR("device not found logicId:{:d}", deviceId);
672 return ACL_ERROR_INVALID_PARAM;670 return ACL_ERROR_INVALID_PARAM;
673 }671 }
674 672 
@@ -676,7 +674,7 @@ aclError aclrtGetDevicesTopo(uint32_t deviceId, uint32_t otherDeviceId, uint64_t
676 return d.logic_id == otherDeviceId;674 return d.logic_id == otherDeviceId;
677 });675 });
678 if (!dev2.second) {676 if (!dev2.second) {
679- DEVICE_STUB_ERROR("device not found logicId:{:d}", otherDeviceId);677+ HCCL_VM_ERROR("device not found logicId:{:d}", otherDeviceId);
680 return ACL_ERROR_INVALID_PARAM;678 return ACL_ERROR_INVALID_PARAM;
681 }679 }
682 680 
@@ -687,14 +685,14 @@ aclError aclrtGetDevicesTopo(uint32_t deviceId, uint32_t otherDeviceId, uint64_t
687 return devConn.src_dev_id == dev1Id && devConn.dst_dev_id == dev2Id;685 return devConn.src_dev_id == dev1Id && devConn.dst_dev_id == dev2Id;
688 });686 });
689 if (!ret.second) {687 if (!ret.second) {
690- DEVICE_STUB_ERROR("connection not found src:{:d} dst:{:d}", dev1Id, dev2Id);688+ HCCL_VM_ERROR("connection not found src:{:d} dst:{:d}", dev1Id, dev2Id);
691 return HcclResult::HCCL_E_NOT_FOUND;689 return HcclResult::HCCL_E_NOT_FOUND;
692 }690 }
693 691 
694 *value = (uint64_t)ret.first.link_type;692 *value = (uint64_t)ret.first.link_type;
695 return HcclVmResult::HCCL_SIM_SUCCESS;693 return HcclVmResult::HCCL_SIM_SUCCESS;
696 } catch (const std::exception &e) {694 } catch (const std::exception &e) {
697- DEVICE_STUB_ERROR("exception:{}", e.what());695+ HCCL_VM_ERROR("exception:{}", e.what());
698 return ACL_ERROR_INTERNAL_ERROR;696 return ACL_ERROR_INTERNAL_ERROR;
699 }697 }
700}698}
@@ -727,7 +725,7 @@ aclError aclrtGetPhyDevIdByLogicDevId(int32_t logicDevId, int32_t *const phyDevI
727 }725 }
728 726 
729 *phyDevId = (int32_t)device.physical_id;727 *phyDevId = (int32_t)device.physical_id;
730- DEVICE_STUB_DEBUG("server:{:d} logicId:{:d} phyId:{:d}",g_cur_server_key, logicDevId, *phyDevId);728+ HCCL_VM_DEBUG("server:{:d} logicId:{:d} phyId:{:d}",g_cur_server_key, logicDevId, *phyDevId);
731 return ACL_SUCCESS;729 return ACL_SUCCESS;
732}730}
733 731 
@@ -778,13 +776,13 @@ rtError_t rtGetDeviceIndexByPhyId(uint32_t phyId, uint32_t *devIndex)
778 return d.physical_id == phyId;776 return d.physical_id == phyId;
779 });777 });
780 if (!ret.second) {778 if (!ret.second) {
781- DEVICE_STUB_ERROR("device not found by phyId:{:d}", phyId);779+ HCCL_VM_ERROR("device not found by phyId:{:d}", phyId);
782 return HcclResult::HCCL_E_NOT_FOUND;780 return HcclResult::HCCL_E_NOT_FOUND;
783 }781 }
784 *devIndex = ret.first.logic_id;782 *devIndex = ret.first.logic_id;
785 return ACL_SUCCESS;783 return ACL_SUCCESS;
786 } catch (const std::exception &e) {784 } catch (const std::exception &e) {
787- DEVICE_STUB_ERROR("exception:{}", e.what());785+ HCCL_VM_ERROR("exception:{}", e.what());
788 return ACL_ERROR_INTERNAL_ERROR;786 return ACL_ERROR_INTERNAL_ERROR;
789 }787 }
790}788}
Mtest/hccl_vm/src/proxy/aclrt_event_stub.cc+12-14
@@ -8,6 +8,9 @@
8 * See LICENSE in the root of the software repository for the full text of the License.8 * See LICENSE in the root of the software repository for the full text of the License.
9 */9 */
10 10 
11+// 日志染色: 模块 tag (须在 include sim_log.h 之前)
12+#define HCCL_VM_MODULE "EVENT_STUB"
13+ 
11#include <atomic>14#include <atomic>
12#include <chrono>15#include <chrono>
13#include <cstdint>16#include <cstdint>
@@ -21,11 +24,6 @@
21#include "sim_log.h"24#include "sim_log.h"
22#include "db_sim_runner_ops.h"25#include "db_sim_runner_ops.h"
23 26 
24-#define EVENT_STUB_ERROR(format, ...) HCCL_VM_ERROR("[EVENT_STUB]" format, ##__VA_ARGS__)
25-#define EVENT_STUB_DEBUG(format, ...) HCCL_VM_DEBUG("[EVENT_STUB]" format, ##__VA_ARGS__)
26-#define EVENT_STUB_INFO(format, ...) HCCL_VM_INFO("[EVENT_STUB]" format, ##__VA_ARGS__)
27-#define EVENT_STUB_WARN(format, ...) HCCL_VM_WARN("[EVENT_STUB]" format, ##__VA_ARGS__)
28-#define EVENT_STUB_TRACE(format, ...) HCCL_VM_TRACE("[EVENT_STUB]" format, ##__VA_ARGS__)
29 27 
30#ifdef __cplusplus28#ifdef __cplusplus
31extern "C" {29extern "C" {
@@ -40,7 +38,7 @@ aclError aclrtCreateEventWithFlag(aclrtEvent *event, uint32_t flag)
40 auto currCtx = RunnerDB::GetById<sim::Context>(runner.current_ctx_id);38 auto currCtx = RunnerDB::GetById<sim::Context>(runner.current_ctx_id);
41 if (!currCtx.has_value()) {39 if (!currCtx.has_value()) {
42 // not find40 // not find
43- EVENT_STUB_ERROR("ctx not found:{:d}", runner.current_ctx_id);41+ HCCL_VM_ERROR("ctx not found:{:d}", runner.current_ctx_id);
44 return ACL_ERROR_INVALID_PARAM;42 return ACL_ERROR_INVALID_PARAM;
45 }43 }
46 44
@@ -57,7 +55,7 @@ aclError aclrtCreateEventWithFlag(aclrtEvent *event, uint32_t flag)
57 auto res = RunnerDB::Add<sim::Event>(tmp);55 auto res = RunnerDB::Add<sim::Event>(tmp);
58 56 
59 *event = (aclrtEvent)res;57 *event = (aclrtEvent)res;
60- EVENT_STUB_INFO("id:{:d}", res);58+ HCCL_VM_INFO("id:{:d}", res);
61 return ACL_SUCCESS; 59 return ACL_SUCCESS;
62}60}
63 61 
@@ -74,7 +72,7 @@ aclError aclrtCreateEvent(aclrtEvent *event)
74aclError aclrtDestroyEvent(aclrtEvent event)72aclError aclrtDestroyEvent(aclrtEvent event)
75{73{
76 uint64_t eventId = (uint32_t)(uintptr_t)event;74 uint64_t eventId = (uint32_t)(uintptr_t)event;
77- EVENT_STUB_INFO("id:{:d}", eventId);75+ HCCL_VM_INFO("id:{:d}", eventId);
78 RunnerDB::Delete<sim::Event>(eventId);76 RunnerDB::Delete<sim::Event>(eventId);
79 return ACL_SUCCESS;77 return ACL_SUCCESS;
80}78}
@@ -87,14 +85,14 @@ aclError aclrtRecordEvent(aclrtEvent event, aclrtStream stream)
87 auto currEvent = RunnerDB::GetById<sim::Event>(eventIdx);85 auto currEvent = RunnerDB::GetById<sim::Event>(eventIdx);
88 if (!currEvent.has_value()) {86 if (!currEvent.has_value()) {
89 // not find87 // not find
90- EVENT_STUB_ERROR("event not found:{:d}", eventIdx);88+ HCCL_VM_ERROR("event not found:{:d}", eventIdx);
91 return ACL_ERROR_INVALID_PARAM;89 return ACL_ERROR_INVALID_PARAM;
92 }90 }
93 91 
94 auto currStm = RunnerDB::GetById<sim::Stream>(streamIdx);92 auto currStm = RunnerDB::GetById<sim::Stream>(streamIdx);
95 if (!currStm.has_value()) {93 if (!currStm.has_value()) {
96 // not find94 // not find
97- EVENT_STUB_ERROR("stream not found:{:d}", streamIdx);95+ HCCL_VM_ERROR("stream not found:{:d}", streamIdx);
98 return ACL_ERROR_INVALID_PARAM;96 return ACL_ERROR_INVALID_PARAM;
99 }97 }
100 98 
@@ -110,7 +108,7 @@ aclError aclrtResetEvent(aclrtEvent event, aclrtStream stream)
110 uint64_t eventIdx = (uint32_t)(uintptr_t)event;108 uint64_t eventIdx = (uint32_t)(uintptr_t)event;
111 auto currEvent = RunnerDB::GetById<sim::Event>(eventIdx);109 auto currEvent = RunnerDB::GetById<sim::Event>(eventIdx);
112 if (!currEvent.has_value()) {110 if (!currEvent.has_value()) {
113- EVENT_STUB_ERROR("event not found:{:d}", eventIdx);111+ HCCL_VM_ERROR("event not found:{:d}", eventIdx);
114 return ACL_ERROR_INVALID_PARAM;112 return ACL_ERROR_INVALID_PARAM;
115 }113 }
116 RunnerDB::Update<sim::Event>(eventIdx, [](sim::Event &evt) { evt.status = ACL_EVENT_RECORDED_STATUS_COMPLETE;});114 RunnerDB::Update<sim::Event>(eventIdx, [](sim::Event &evt) { evt.status = ACL_EVENT_RECORDED_STATUS_COMPLETE;});
@@ -122,7 +120,7 @@ aclError aclrtQueryEventStatus(aclrtEvent event, aclrtEventRecordedStatus *statu
122 uint64_t eventId = (uint32_t)(uintptr_t)event;120 uint64_t eventId = (uint32_t)(uintptr_t)event;
123 auto res = RunnerDB::GetById<sim::Event>(eventId);121 auto res = RunnerDB::GetById<sim::Event>(eventId);
124 if (!res.has_value()) {122 if (!res.has_value()) {
125- EVENT_STUB_ERROR("event not found:{:d}", eventId);123+ HCCL_VM_ERROR("event not found:{:d}", eventId);
126 return ACL_ERROR_INVALID_PARAM;124 return ACL_ERROR_INVALID_PARAM;
127 }125 }
128 126 
@@ -197,7 +195,7 @@ aclError aclrtGetEventAvailNum(uint32_t *eventCount)
197 auto currCtx = RunnerDB::GetById<sim::Context>(runner.current_ctx_id);195 auto currCtx = RunnerDB::GetById<sim::Context>(runner.current_ctx_id);
198 if (!currCtx.has_value()) {196 if (!currCtx.has_value()) {
199 // not find197 // not find
200- EVENT_STUB_ERROR("ctx not found:{:d}", runner.current_ctx_id);198+ HCCL_VM_ERROR("ctx not found:{:d}", runner.current_ctx_id);
201 return ACL_ERROR_INVALID_PARAM;199 return ACL_ERROR_INVALID_PARAM;
202 }200 }
203 201
@@ -205,7 +203,7 @@ aclError aclrtGetEventAvailNum(uint32_t *eventCount)
205 203 
206 auto device = RunnerDB::GetById<sim::Device>(devId);204 auto device = RunnerDB::GetById<sim::Device>(devId);
207 if (!device.has_value()) {205 if (!device.has_value()) {
208- EVENT_STUB_ERROR("device not found:{:d}", devId);206+ HCCL_VM_ERROR("device not found:{:d}", devId);
209 return ACL_ERROR_INVALID_PARAM;207 return ACL_ERROR_INVALID_PARAM;
210 }208 }
211 209 
Mtest/hccl_vm/src/proxy/aclrt_exception_stub.cc+17-19
@@ -8,6 +8,9 @@
8 * See LICENSE in the root of the software repository for the full text of the License.8 * See LICENSE in the root of the software repository for the full text of the License.
9 */9 */
10 10 
11+// 日志染色: 模块 tag (须在 include sim_log.h 之前)
12+#define HCCL_VM_MODULE "EXCEPTION_STUB"
13+ 
11#include <atomic>14#include <atomic>
12#include <chrono>15#include <chrono>
13#include <cstdint>16#include <cstdint>
@@ -21,11 +24,6 @@
21#include "sim_log.h"24#include "sim_log.h"
22#include "db_sim_runner_ops.h"25#include "db_sim_runner_ops.h"
23 26 
24-#define EXC_STUB_TRACE(format, ...) HCCL_VM_TRACE("[EXCEPTION_STUB]" format, ##__VA_ARGS__)
25-#define EXC_STUB_DEBUG(format, ...) HCCL_VM_DEBUG("[EXCEPTION_STUB]" format, ##__VA_ARGS__)
26-#define EXC_STUB_INFO(format, ...) HCCL_VM_INFO("[EXCEPTION_STUB]" format, ##__VA_ARGS__)
27-#define EXC_STUB_WARN(format, ...) HCCL_VM_WARN("[EXCEPTION_STUB]" format, ##__VA_ARGS__)
28-#define EXC_STUB_ERROR(format, ...) HCCL_VM_ERROR("[EXCEPTION_STUB]" format, ##__VA_ARGS__)
29 27 
30#ifdef __cplusplus28#ifdef __cplusplus
31extern "C" {29extern "C" {
@@ -34,56 +32,56 @@ extern "C" {
34aclError aclrtSetExceptionInfoCallback(aclrtExceptionInfoCallback callback)32aclError aclrtSetExceptionInfoCallback(aclrtExceptionInfoCallback callback)
35{33{
36 (void) callback;34 (void) callback;
37- EXC_STUB_TRACE("not supported");35+ HCCL_VM_TRACE("not supported");
38 return ACL_SUCCESS;36 return ACL_SUCCESS;
39}37}
40 38 
41uint32_t aclrtGetTaskIdFromExceptionInfo(const aclrtExceptionInfo *info)39uint32_t aclrtGetTaskIdFromExceptionInfo(const aclrtExceptionInfo *info)
42{40{
43 (void) info;41 (void) info;
44- EXC_STUB_TRACE("not supported");42+ HCCL_VM_TRACE("not supported");
45 return ACL_SUCCESS;43 return ACL_SUCCESS;
46}44}
47 45 
48uint32_t aclrtGetStreamIdFromExceptionInfo(const aclrtExceptionInfo *info)46uint32_t aclrtGetStreamIdFromExceptionInfo(const aclrtExceptionInfo *info)
49{47{
50 (void) info;48 (void) info;
51- EXC_STUB_TRACE("not supported");49+ HCCL_VM_TRACE("not supported");
52 return ACL_SUCCESS;50 return ACL_SUCCESS;
53}51}
54 52 
55uint32_t aclrtGetThreadIdFromExceptionInfo(const aclrtExceptionInfo *info)53uint32_t aclrtGetThreadIdFromExceptionInfo(const aclrtExceptionInfo *info)
56{54{
57 (void) info;55 (void) info;
58- EXC_STUB_TRACE("not supported");56+ HCCL_VM_TRACE("not supported");
59 return ACL_SUCCESS;57 return ACL_SUCCESS;
60}58}
61 59 
62uint32_t aclrtGetDeviceIdFromExceptionInfo(const aclrtExceptionInfo *info)60uint32_t aclrtGetDeviceIdFromExceptionInfo(const aclrtExceptionInfo *info)
63{61{
64 (void) info;62 (void) info;
65- EXC_STUB_TRACE("not supported");63+ HCCL_VM_TRACE("not supported");
66 return ACL_SUCCESS;64 return ACL_SUCCESS;
67}65}
68 66 
69uint32_t aclrtGetErrorCodeFromExceptionInfo(const aclrtExceptionInfo *info)67uint32_t aclrtGetErrorCodeFromExceptionInfo(const aclrtExceptionInfo *info)
70{68{
71 (void) info;69 (void) info;
72- EXC_STUB_TRACE("not supported");70+ HCCL_VM_TRACE("not supported");
73 return ACL_SUCCESS;71 return ACL_SUCCESS;
74}72}
75 73 
76aclError aclrtPeekAtLastError(aclrtLastErrLevel level)74aclError aclrtPeekAtLastError(aclrtLastErrLevel level)
77{75{
78 (void) level;76 (void) level;
79- EXC_STUB_TRACE("not supported");77+ HCCL_VM_TRACE("not supported");
80 return ACL_SUCCESS;78 return ACL_SUCCESS;
81}79}
82 80 
83aclError aclrtGetLastError(aclrtLastErrLevel level)81aclError aclrtGetLastError(aclrtLastErrLevel level)
84{82{
85 (void) level;83 (void) level;
86- EXC_STUB_TRACE("not supported");84+ HCCL_VM_TRACE("not supported");
87 return ACL_SUCCESS;85 return ACL_SUCCESS;
88}86}
89 87 
@@ -93,7 +91,7 @@ aclError aclrtGetMemUceInfo(int32_t deviceId, aclrtMemUceInfo *memUceInfoArray,
93 (void) memUceInfoArray;91 (void) memUceInfoArray;
94 (void) arraySize;92 (void) arraySize;
95 (void) retSize;93 (void) retSize;
96- EXC_STUB_TRACE("not supported");94+ HCCL_VM_TRACE("not supported");
97 return ACL_SUCCESS;95 return ACL_SUCCESS;
98}96}
99 97 
@@ -102,7 +100,7 @@ aclError aclrtMemUceRepair(int32_t deviceId, aclrtMemUceInfo *memUceInfoArray, s
102 (void) deviceId;100 (void) deviceId;
103 (void) memUceInfoArray;101 (void) memUceInfoArray;
104 (void) arraySize;102 (void) arraySize;
105- EXC_STUB_TRACE("not supported");103+ HCCL_VM_TRACE("not supported");
106 return ACL_SUCCESS;104 return ACL_SUCCESS;
107}105}
108 106 
@@ -110,14 +108,14 @@ aclError aclrtDeviceTaskAbort(int32_t deviceId, uint32_t timeout)
110{108{
111 (void) deviceId;109 (void) deviceId;
112 (void) timeout;110 (void) timeout;
113- EXC_STUB_TRACE("not supported");111+ HCCL_VM_TRACE("not supported");
114 return ACL_SUCCESS;112 return ACL_SUCCESS;
115}113}
116 114 
117aclError aclRecoverAllHcclTasks(int32_t deviceId)115aclError aclRecoverAllHcclTasks(int32_t deviceId)
118{116{
119 (void) deviceId;117 (void) deviceId;
120- EXC_STUB_TRACE("not supported");118+ HCCL_VM_TRACE("not supported");
121 return ACL_SUCCESS;119 return ACL_SUCCESS;
122}120}
123 121 
@@ -125,7 +123,7 @@ aclError aclrtGetErrorVerbose(int32_t deviceId, aclrtErrorInfo *errorInfo)
125{123{
126 (void) deviceId;124 (void) deviceId;
127 (void) errorInfo;125 (void) errorInfo;
128- EXC_STUB_TRACE("not supported");126+ HCCL_VM_TRACE("not supported");
129 return ACL_SUCCESS;127 return ACL_SUCCESS;
130}128}
131 129 
@@ -133,7 +131,7 @@ aclError aclrtRepairError(int32_t deviceId, const aclrtErrorInfo *errorInfo)
133{131{
134 (void) deviceId;132 (void) deviceId;
135 (void) errorInfo;133 (void) errorInfo;
136- EXC_STUB_TRACE("not supported");134+ HCCL_VM_TRACE("not supported");
137 return ACL_SUCCESS;135 return ACL_SUCCESS;
138}136}
139#ifdef __cplusplus137#ifdef __cplusplus
Mtest/hccl_vm/src/proxy/aclrt_exec_control.cc+16-18
@@ -8,6 +8,9 @@
8 * See LICENSE in the root of the software repository for the full text of the License.8 * See LICENSE in the root of the software repository for the full text of the License.
9 */9 */
10 10 
11+// 日志染色: 模块 tag (须在 include sim_log.h 之前)
12+#define HCCL_VM_MODULE "EXEC_STUB"
13+ 
11#include <atomic>14#include <atomic>
12#include <chrono>15#include <chrono>
13#include <cstdint>16#include <cstdint>
@@ -21,11 +24,6 @@
21#include "sim_log.h"24#include "sim_log.h"
22#include "db_sim_runner_ops.h"25#include "db_sim_runner_ops.h"
23 26 
24-#define EXEC_STUB_ERROR(format, ...) HCCL_VM_ERROR("[EXEC_STUB]" format, #__VA_ARGS__)
25-#define EXEC_STUB_DEBUG(format, ...) HCCL_VM_DEBUG("[EXEC_STUB]" format, ##__VA_ARGS__)
26-#define EXEC_STUB_INFO(format, ...) HCCL_VM_INFO("[EXEC_STUB]" format, #__VA_ARGS__)
27-#define EXEC_STUB_WARN(format, ...) HCCL_VM_WARN("[EXEC_STUB]" format, ##__VA_ARGS__)
28-#define EXEC_STUB_TRACE(format, ...) HCCL_VM_TRACE("[EXEC_STUB]" format, ##__VA_ARGS__)
29 27 
30#ifdef __cplusplus28#ifdef __cplusplus
31extern "C" {29extern "C" {
@@ -37,7 +35,7 @@ aclError aclrtLaunchCallback(aclrtCallback fn, void *userData, aclrtCallbackBloc
37 (void) userData;35 (void) userData;
38 (void) blockType;36 (void) blockType;
39 (void) stream;37 (void) stream;
40- EXEC_STUB_TRACE("not supported");38+ HCCL_VM_TRACE("not supported");
41 return ACL_SUCCESS;39 return ACL_SUCCESS;
42}40}
43 41 
@@ -45,14 +43,14 @@ aclError aclrtSubscribeReport(uint64_t threadId, aclrtStream stream)
45{43{
46 (void) threadId;44 (void) threadId;
47 (void) stream;45 (void) stream;
48- EXEC_STUB_TRACE("not supported");46+ HCCL_VM_TRACE("not supported");
49 return ACL_SUCCESS;47 return ACL_SUCCESS;
50}48}
51 49 
52aclError aclrtProcessReport(int32_t timeout)50aclError aclrtProcessReport(int32_t timeout)
53{51{
54 (void) timeout;52 (void) timeout;
55- EXEC_STUB_TRACE("not supported");53+ HCCL_VM_TRACE("not supported");
56 return ACL_SUCCESS;54 return ACL_SUCCESS;
57}55}
58 56 
@@ -60,7 +58,7 @@ aclError aclrtUnSubscribeReport(uint64_t threadId, aclrtStream stream)
60{58{
61 (void) threadId;59 (void) threadId;
62 (void) stream;60 (void) stream;
63- EXEC_STUB_TRACE("not supported");61+ HCCL_VM_TRACE("not supported");
64 return ACL_SUCCESS;62 return ACL_SUCCESS;
65}63}
66 64 
@@ -68,14 +66,14 @@ aclError aclrtSubscribeHostFunc(uint64_t hostFuncThreadId, aclrtStream exeStream
68{66{
69 (void) hostFuncThreadId;67 (void) hostFuncThreadId;
70 (void) exeStream;68 (void) exeStream;
71- EXEC_STUB_TRACE("not supported");69+ HCCL_VM_TRACE("not supported");
72 return ACL_SUCCESS;70 return ACL_SUCCESS;
73}71}
74 72 
75aclError aclrtProcessHostFunc(int32_t timeout)73aclError aclrtProcessHostFunc(int32_t timeout)
76{74{
77 (void) timeout;75 (void) timeout;
78- EXEC_STUB_TRACE("not supported");76+ HCCL_VM_TRACE("not supported");
79 return ACL_SUCCESS;77 return ACL_SUCCESS;
80}78}
81 79 
@@ -83,21 +81,21 @@ aclError aclrtUnSubscribeHostFunc(uint64_t hostFuncThreadId, aclrtStream exeStre
83{81{
84 (void) hostFuncThreadId;82 (void) hostFuncThreadId;
85 (void) exeStream;83 (void) exeStream;
86- EXEC_STUB_TRACE("not supported");84+ HCCL_VM_TRACE("not supported");
87 return ACL_SUCCESS;85 return ACL_SUCCESS;
88}86}
89 87 
90aclError aclrtGetOpTimeoutInterval(uint64_t *interval)88aclError aclrtGetOpTimeoutInterval(uint64_t *interval)
91{89{
92 (void) interval;90 (void) interval;
93- EXEC_STUB_TRACE("not supported");91+ HCCL_VM_TRACE("not supported");
94 return ACL_SUCCESS;92 return ACL_SUCCESS;
95}93}
96 94 
97aclError aclrtSetOpExecuteTimeOut(uint32_t timeout)95aclError aclrtSetOpExecuteTimeOut(uint32_t timeout)
98{96{
99 (void) timeout;97 (void) timeout;
100- EXEC_STUB_TRACE("not supported");98+ HCCL_VM_TRACE("not supported");
101 return ACL_SUCCESS;99 return ACL_SUCCESS;
102}100}
103 101 
@@ -105,21 +103,21 @@ aclError aclrtSetOpExecuteTimeOutV2(uint64_t timeout, uint64_t *actualTimeout)
105{103{
106 (void) timeout;104 (void) timeout;
107 (void) actualTimeout;105 (void) actualTimeout;
108- EXEC_STUB_TRACE("not supported");106+ HCCL_VM_TRACE("not supported");
109 return ACL_SUCCESS;107 return ACL_SUCCESS;
110}108}
111 109 
112aclError aclrtSetOpExecuteTimeOutWithMs(uint32_t timeout)110aclError aclrtSetOpExecuteTimeOutWithMs(uint32_t timeout)
113{111{
114 (void) timeout;112 (void) timeout;
115- EXEC_STUB_TRACE("not supported");113+ HCCL_VM_TRACE("not supported");
116 return ACL_SUCCESS;114 return ACL_SUCCESS;
117}115}
118 116 
119aclError aclrtGetThreadLastTaskId(uint32_t *taskId)117aclError aclrtGetThreadLastTaskId(uint32_t *taskId)
120{118{
121 (void) taskId;119 (void) taskId;
122- EXEC_STUB_TRACE("not supported");120+ HCCL_VM_TRACE("not supported");
123 return ACL_SUCCESS;121 return ACL_SUCCESS;
124}122}
125 123 
@@ -128,7 +126,7 @@ aclError aclrtLaunchHostFunc(aclrtStream stream, aclrtHostFunc fn, void *args)
128 (void) stream;126 (void) stream;
129 (void) fn;127 (void) fn;
130 (void) args;128 (void) args;
131- EXEC_STUB_TRACE("not supported");129+ HCCL_VM_TRACE("not supported");
132 return ACL_SUCCESS;130 return ACL_SUCCESS;
133}131}
134#ifdef __cplusplus132#ifdef __cplusplus
Mtest/hccl_vm/src/proxy/aclrt_kernel_stub.cc+2004-54
Mtest/hccl_vm/src/proxy/aclrt_memory_stub.cc+85-86
@@ -8,6 +8,9 @@
8 * See LICENSE in the root of the software repository for the full text of the License.8 * See LICENSE in the root of the software repository for the full text of the License.
9 */9 */
10 10 
11+// 日志染色: 模块 tag (须在 include sim_log.h 之前)
12+#define HCCL_VM_MODULE "MEM_STUB"
13+ 
11#include <atomic>14#include <atomic>
12#include <cstdint>15#include <cstdint>
13#include <iostream>16#include <iostream>
@@ -27,10 +30,6 @@
27#include "sim_models.h"30#include "sim_models.h"
28#include "db_sim_runner_ops.h"31#include "db_sim_runner_ops.h"
29 32 
30-#define MEM_STUB_ERROR(format, ...) HCCL_VM_ERROR("[MEM_STUB]" format, ##__VA_ARGS__)
31-#define MEM_STUB_DEBUG(format, ...) HCCL_VM_DEBUG("[MEM_STUB]" format, ##__VA_ARGS__)
32-#define MEM_STUB_INFO(format, ...) HCCL_VM_INFO("[MEM_STUB]" format, ##__VA_ARGS__)
33-#define MEM_STUB_WARN(format, ...) HCCL_VM_WARN("[MEM_STUB]" format, ##__VA_ARGS__)
34 33 
35// rank 进程加载本库时先于 main 预热仅校验模式缓存,使后续引流判定拿到确定值。34// rank 进程加载本库时先于 main 预热仅校验模式缓存,使后续引流判定拿到确定值。
36// 只读 DB、不打日志(此刻日志组件可能尚未初始化)。35// 只读 DB、不打日志(此刻日志组件可能尚未初始化)。
@@ -73,14 +72,14 @@ bool GetPtrNameByVirPtr(const void* virPtr, uint32_t& offset, sim::PhyMemBlock &
73 (devPtr < (virMem.start_ptr + virMem.size)) &&72 (devPtr < (virMem.start_ptr + virMem.size)) &&
74 (virMem.src_type == (uint8_t)sim::VIR_MEM_TYPE_DEV)); });73 (virMem.src_type == (uint8_t)sim::VIR_MEM_TYPE_DEV)); });
75 if (!virMemRes.second) {74 if (!virMemRes.second) {
76- MEM_STUB_ERROR("can not find this buff offset ptr:{:p}", virPtr);75+ HCCL_VM_ERROR("can not find this buff offset ptr:{:p}", virPtr);
77 return false;76 return false;
78 }77 }
79 78 
80 auto phyMemId = virMemRes.first.phy_mem_id;79 auto phyMemId = virMemRes.first.phy_mem_id;
81 auto phyMemRes = RunnerDB::GetById<sim::PhyMemBlock>(phyMemId);80 auto phyMemRes = RunnerDB::GetById<sim::PhyMemBlock>(phyMemId);
82 if (!phyMemRes.has_value()) {81 if (!phyMemRes.has_value()) {
83- MEM_STUB_ERROR("can not find phy Mem id:{:d}", phyMemId);82+ HCCL_VM_ERROR("can not find phy Mem id:{:d}", phyMemId);
84 return false;83 return false;
85 };84 };
86 offset = (uint32_t)(devPtr - virMemRes.first.start_ptr);85 offset = (uint32_t)(devPtr - virMemRes.first.start_ptr);
@@ -99,7 +98,7 @@ uint32_t GetRankIdByVirAddr(const void* virAddr)
99 uint32_t deviceId = phyMem.device_id;98 uint32_t deviceId = phyMem.device_id;
100 auto rank = RunnerDB::GetOneByPred<sim::Rank>([deviceId](const sim::Rank &rank) { return rank.device_id == deviceId; });99 auto rank = RunnerDB::GetOneByPred<sim::Rank>([deviceId](const sim::Rank &rank) { return rank.device_id == deviceId; });
101 if (!rank.second) {100 if (!rank.second) {
102- MEM_STUB_ERROR("cannot find rank by device id[{}]", deviceId);101+ HCCL_VM_ERROR("cannot find rank by device id[{}]", deviceId);
103 return 0;102 return 0;
104 }103 }
105 104 
@@ -115,14 +114,14 @@ void* GetRealPtrByAddr(const void* virPtr)
115 (devPtr < (virMem.start_ptr + virMem.size)) &&114 (devPtr < (virMem.start_ptr + virMem.size)) &&
116 (virMem.src_type == (uint8_t)sim::VIR_MEM_TYPE_DEV)); });115 (virMem.src_type == (uint8_t)sim::VIR_MEM_TYPE_DEV)); });
117 if (!virMemRes.second) {116 if (!virMemRes.second) {
118- MEM_STUB_ERROR("can not find this buff offset ptr:{:p}", virPtr);117+ HCCL_VM_ERROR("can not find this buff offset ptr:{:p}", virPtr);
119 return nullptr;118 return nullptr;
120 }119 }
121 120 
122 void* devStartPtr = (void*)(uintptr_t)virMemRes.first.start_ptr;121 void* devStartPtr = (void*)(uintptr_t)virMemRes.first.start_ptr;
123 auto hostPtr = sim::DeviceMemoryManager::GetInstance().GetHostPtrByDevPtr(devStartPtr);122 auto hostPtr = sim::DeviceMemoryManager::GetInstance().GetHostPtrByDevPtr(devStartPtr);
124 if (hostPtr == nullptr) {123 if (hostPtr == nullptr) {
125- MEM_STUB_ERROR("can not find host ptr by dev ptr:{:p}", devStartPtr);124+ HCCL_VM_ERROR("can not find host ptr by dev ptr:{:p}", devStartPtr);
126 return nullptr;125 return nullptr;
127 }126 }
128 127 
@@ -146,7 +145,7 @@ aclError aclrtMallocHost(void **hostPtr, size_t size)
146 bool checkOnlyMode = sim::IsCheckOnlyMode();145 bool checkOnlyMode = sim::IsCheckOnlyMode();
147 // 超过复用区上界直接报错,不回退真实分配。146 // 超过复用区上界直接报错,不回退真实分配。
148 if (sim::CommPoolPolicy::ExceedsCeiling(size, checkOnlyMode)) {147 if (sim::CommPoolPolicy::ExceedsCeiling(size, checkOnlyMode)) {
149- MEM_STUB_ERROR("malloc host size:{:d} exceeds pool ceiling, reject", size);148+ HCCL_VM_ERROR("malloc host size:{:d} exceeds pool ceiling, reject", size);
150 return ACL_ERROR_INTERNAL_ERROR;149 return ACL_ERROR_INTERNAL_ERROR;
151 }150 }
152 // 仅校验模式下大块引流到共享复用区 HcclCommPool151 // 仅校验模式下大块引流到共享复用区 HcclCommPool
@@ -155,16 +154,16 @@ aclError aclrtMallocHost(void **hostPtr, size_t size)
155 if (base == nullptr) {154 if (base == nullptr) {
156 base = sim::MemoryManager::GetInstance().AcquireMemByName(sim::CommPoolPolicy::kPoolName);155 base = sim::MemoryManager::GetInstance().AcquireMemByName(sim::CommPoolPolicy::kPoolName);
157 if (base == nullptr) {156 if (base == nullptr) {
158- MEM_STUB_ERROR("malloc host acquire HcclCommPool failed, size:{:d}", size);157+ HCCL_VM_ERROR("malloc host acquire HcclCommPool failed, size:{:d}", size);
159 return ACL_ERROR_INTERNAL_ERROR;158 return ACL_ERROR_INTERNAL_ERROR;
160 }159 }
161 CacheCommPoolBase(base);160 CacheCommPoolBase(base);
162 }161 }
163 *hostPtr = base;162 *hostPtr = base;
164- MEM_STUB_INFO("malloc host (SHM) addr:{:p}, size:{:d}", *hostPtr, size);163+ HCCL_VM_INFO("malloc host (SHM) addr:{:p}, size:{:d}", *hostPtr, size);
165 } else {164 } else {
166 *hostPtr = malloc(size);165 *hostPtr = malloc(size);
167- MEM_STUB_INFO("malloc host addr:{:p}, size:{:d}", *hostPtr, size);166+ HCCL_VM_INFO("malloc host addr:{:p}, size:{:d}", *hostPtr, size);
168 }167 }
169 return ACL_SUCCESS;168 return ACL_SUCCESS;
170}169}
@@ -177,10 +176,10 @@ aclError aclrtMallocHostWithCfg(void **ptr, uint64_t size, aclrtMallocConfig *cf
177 176 
178aclError aclrtFreeHost(void *hostPtr)177aclError aclrtFreeHost(void *hostPtr)
179{178{
180- MEM_STUB_INFO("[MEM] free host addr:{:p}", hostPtr);179+ HCCL_VM_INFO("free host addr:{:p}", hostPtr);
181 // 池内地址:空操作(复用区生命周期由主进程管理,不在此 free)180 // 池内地址:空操作(复用区生命周期由主进程管理,不在此 free)
182 if (IsInCommPool(hostPtr)) {181 if (IsInCommPool(hostPtr)) {
183- MEM_STUB_INFO("free host (SHM noop) addr:{:p}", hostPtr);182+ HCCL_VM_INFO("free host (SHM noop) addr:{:p}", hostPtr);
184 return ACL_SUCCESS;183 return ACL_SUCCESS;
185 }184 }
186 free(hostPtr);185 free(hostPtr);
@@ -195,7 +194,7 @@ aclError aclrtMalloc(void **devPtr, size_t size, aclrtMemMallocPolicy policy)
195 }194 }
196 auto currCtx = RunnerDB::GetById<sim::Context>(runner.current_ctx_id);195 auto currCtx = RunnerDB::GetById<sim::Context>(runner.current_ctx_id);
197 if (!currCtx.has_value()) {196 if (!currCtx.has_value()) {
198- MEM_STUB_ERROR("can not find current context:{:d}", runner.current_ctx_id);197+ HCCL_VM_ERROR("can not find current context:{:d}", runner.current_ctx_id);
199 return ACL_ERROR_INTERNAL_ERROR;198 return ACL_ERROR_INTERNAL_ERROR;
200 }199 }
201 200 
@@ -203,7 +202,7 @@ aclError aclrtMalloc(void **devPtr, size_t size, aclrtMemMallocPolicy policy)
203 202 
204 auto dev = RunnerDB::GetById<sim::Device>(deviceId);203 auto dev = RunnerDB::GetById<sim::Device>(deviceId);
205 if (!dev.has_value()) {204 if (!dev.has_value()) {
206- MEM_STUB_ERROR("can not find device id:{:d}", deviceId);205+ HCCL_VM_ERROR("can not find device id:{:d}", deviceId);
207 return ACL_ERROR_INTERNAL_ERROR;206 return ACL_ERROR_INTERNAL_ERROR;
208 }207 }
209 208 
@@ -211,7 +210,7 @@ aclError aclrtMalloc(void **devPtr, size_t size, aclrtMemMallocPolicy policy)
211 210 
212 auto virPtr = sim::DeviceMemoryManager::GetInstance().AllocVirMem(rankId, size);211 auto virPtr = sim::DeviceMemoryManager::GetInstance().AllocVirMem(rankId, size);
213 if (virPtr == nullptr) {212 if (virPtr == nullptr) {
214- MEM_STUB_ERROR("can not alloc vir mem deviceId:{:d}, phyMemId:{:d}, size:{:d}", deviceId, rankId, size);213+ HCCL_VM_ERROR("can not alloc vir mem deviceId:{:d}, phyMemId:{:d}, size:{:d}", deviceId, rankId, size);
215 return ACL_ERROR_INTERNAL_ERROR;214 return ACL_ERROR_INTERNAL_ERROR;
216 }215 }
217 216 
@@ -219,7 +218,7 @@ aclError aclrtMalloc(void **devPtr, size_t size, aclrtMemMallocPolicy policy)
219 auto hostPtr = sim::DeviceMemoryManager::GetInstance().AllocPhyMem(memName.c_str(), deviceId, size);218 auto hostPtr = sim::DeviceMemoryManager::GetInstance().AllocPhyMem(memName.c_str(), deviceId, size);
220 if (hostPtr == nullptr) {219 if (hostPtr == nullptr) {
221 sim::DeviceMemoryManager::GetInstance().FreeVirMem(rankId, devPtr);220 sim::DeviceMemoryManager::GetInstance().FreeVirMem(rankId, devPtr);
222- MEM_STUB_ERROR("can not alloc phy mem deviceId:{:d}, size:{:d}", deviceId, size);221+ HCCL_VM_ERROR("can not alloc phy mem deviceId:{:d}, size:{:d}", deviceId, size);
223 return ACL_ERROR_INTERNAL_ERROR;222 return ACL_ERROR_INTERNAL_ERROR;
224 }223 }
225 // 大块引流到复用区时缓存池基址,供 IsInCommPool 判定(与主机侧共用同一基址)。224 // 大块引流到复用区时缓存池基址,供 IsInCommPool 判定(与主机侧共用同一基址)。
@@ -249,20 +248,20 @@ aclError aclrtMalloc(void **devPtr, size_t size, aclrtMemMallocPolicy policy)
249 *devPtr = virPtr;248 *devPtr = virPtr;
250 // 缓存设备地址到host地址249 // 缓存设备地址到host地址
251 sim::DeviceMemoryManager::GetInstance().MapDevPtrHostPtr(virPtr, hostPtr);250 sim::DeviceMemoryManager::GetInstance().MapDevPtrHostPtr(virPtr, hostPtr);
252- MEM_STUB_INFO("malloc dev addr:{:p}, memName:{}, size:{:d}", virPtr, memName, size);251+ HCCL_VM_INFO("malloc dev addr:{:p}, memName:{}, size:{:d}", virPtr, memName, size);
253 return ACL_SUCCESS;252 return ACL_SUCCESS;
254}253}
255 254 
256aclError aclrtMallocAlign32(void **devPtr, size_t size, aclrtMemMallocPolicy policy)255aclError aclrtMallocAlign32(void **devPtr, size_t size, aclrtMemMallocPolicy policy)
257{256{
258- MEM_STUB_INFO("..");257+ HCCL_VM_INFO("..");
259 size_t realSize = ((size + 31) / 32) * 32;258 size_t realSize = ((size + 31) / 32) * 32;
260 return aclrtMalloc(devPtr, realSize, policy);259 return aclrtMalloc(devPtr, realSize, policy);
261}260}
262 261 
263aclError aclrtMallocCached(void **devPtr, size_t size, aclrtMemMallocPolicy policy)262aclError aclrtMallocCached(void **devPtr, size_t size, aclrtMemMallocPolicy policy)
264{263{
265- MEM_STUB_INFO("..");264+ HCCL_VM_INFO("..");
266 return aclrtMalloc(devPtr, size, policy);265 return aclrtMalloc(devPtr, size, policy);
267}266}
268 267 
@@ -270,7 +269,7 @@ aclError aclrtMemFlush(void *devPtr, size_t size)
270{269{
271 (void) devPtr;270 (void) devPtr;
272 (void) size;271 (void) size;
273- MEM_STUB_INFO("..");272+ HCCL_VM_INFO("..");
274 return ACL_SUCCESS;273 return ACL_SUCCESS;
275}274}
276 275 
@@ -278,21 +277,21 @@ aclError aclrtMemInvalidate(void *devPtr, size_t size)
278{277{
279 (void) devPtr;278 (void) devPtr;
280 (void) size;279 (void) size;
281- MEM_STUB_INFO("..");280+ HCCL_VM_INFO("..");
282 return ACL_SUCCESS;281 return ACL_SUCCESS;
283}282}
284 283 
285aclError aclrtMallocWithCfg(void **devPtr, size_t size, aclrtMemMallocPolicy policy, aclrtMallocConfig *cfg)284aclError aclrtMallocWithCfg(void **devPtr, size_t size, aclrtMemMallocPolicy policy, aclrtMallocConfig *cfg)
286{285{
287 (void) cfg;286 (void) cfg;
288- MEM_STUB_INFO("..");287+ HCCL_VM_INFO("..");
289 return aclrtMalloc(devPtr, size, policy);288 return aclrtMalloc(devPtr, size, policy);
290}289}
291 290 
292aclError aclrtMallocForTaskScheduler(void **devPtr, size_t size, aclrtMemMallocPolicy policy, aclrtMallocConfig *cfg)291aclError aclrtMallocForTaskScheduler(void **devPtr, size_t size, aclrtMemMallocPolicy policy, aclrtMallocConfig *cfg)
293{292{
294 (void) cfg;293 (void) cfg;
295- MEM_STUB_INFO("..");294+ HCCL_VM_INFO("..");
296 return aclrtMalloc(devPtr, size, policy);295 return aclrtMalloc(devPtr, size, policy);
297}296}
298 297 
@@ -303,21 +302,21 @@ aclError aclrtFree(void* devPtr)
303 [startPtr](const sim::VirtualMemBlock &virMem)302 [startPtr](const sim::VirtualMemBlock &virMem)
304 { return virMem.start_ptr == startPtr && virMem.src_type == (uint8_t)sim::VIR_MEM_TYPE_DEV; });303 { return virMem.start_ptr == startPtr && virMem.src_type == (uint8_t)sim::VIR_MEM_TYPE_DEV; });
305 if (!virMemRes.second) {304 if (!virMemRes.second) {
306- MEM_STUB_ERROR("can not find this buff offset ptr:0x{:x}", startPtr);305+ HCCL_VM_ERROR("can not find this buff offset ptr:0x{:x}", startPtr);
307 return ACL_ERROR_INTERNAL_ERROR;306 return ACL_ERROR_INTERNAL_ERROR;
308 }307 }
309 308 
310 auto phyMemId = virMemRes.first.phy_mem_id;309 auto phyMemId = virMemRes.first.phy_mem_id;
311 auto phyMemRes = RunnerDB::GetById<sim::PhyMemBlock>(phyMemId);310 auto phyMemRes = RunnerDB::GetById<sim::PhyMemBlock>(phyMemId);
312 if (!phyMemRes.has_value()) {311 if (!phyMemRes.has_value()) {
313- MEM_STUB_ERROR("can not find phy Mem id:{:d}", phyMemId);312+ HCCL_VM_ERROR("can not find phy Mem id:{:d}", phyMemId);
314 return ACL_ERROR_INTERNAL_ERROR;313 return ACL_ERROR_INTERNAL_ERROR;
315 };314 };
316 315 
317 auto deviceId = phyMemRes->device_id;316 auto deviceId = phyMemRes->device_id;
318 auto dev = RunnerDB::GetById<sim::Device>(deviceId);317 auto dev = RunnerDB::GetById<sim::Device>(deviceId);
319 if (!dev.has_value()) {318 if (!dev.has_value()) {
320- MEM_STUB_ERROR("can not find device id:{:d}", deviceId);319+ HCCL_VM_ERROR("can not find device id:{:d}", deviceId);
321 return ACL_ERROR_INTERNAL_ERROR;320 return ACL_ERROR_INTERNAL_ERROR;
322 }321 }
323 322 
@@ -342,7 +341,7 @@ aclError aclrtFree(void* devPtr)
342 RunnerDB::Update<sim::PhyMemBlock>(phyMemId, [](sim::PhyMemBlock &memBlock) { memBlock.is_freed = 1; });341 RunnerDB::Update<sim::PhyMemBlock>(phyMemId, [](sim::PhyMemBlock &memBlock) { memBlock.is_freed = 1; });
343 RunnerDB::Update<sim::VirtualMemBlock>(virMemRes.first.id, [](sim::VirtualMemBlock &virtMemBlock) { virtMemBlock.is_freed = 1; });342 RunnerDB::Update<sim::VirtualMemBlock>(virMemRes.first.id, [](sim::VirtualMemBlock &virtMemBlock) { virtMemBlock.is_freed = 1; });
344 343 
345- MEM_STUB_INFO("free dev addr:{:p}, memName:{}, size:{:d}", devPtr, phyMemRes->name, virMemRes.first.size);344+ HCCL_VM_INFO("free dev addr:{:p}, memName:{}, size:{:d}", devPtr, phyMemRes->name, virMemRes.first.size);
346 return ACL_SUCCESS;345 return ACL_SUCCESS;
347}346}
348 347 
@@ -352,20 +351,20 @@ aclError aclrtMemset(void *devPtr, size_t maxCount, int32_t value, size_t count)
352 uint32_t offset = 0;351 uint32_t offset = 0;
353 if (!GetPtrNameByVirPtr(devPtr, offset, phyMem)) {352 if (!GetPtrNameByVirPtr(devPtr, offset, phyMem)) {
354 memset(devPtr, value, count);353 memset(devPtr, value, count);
355- MEM_STUB_INFO("sys mem memset ptr:{:p}, maxCount: {:d}, value: {:d}, count: {:d}", devPtr, maxCount, value, count);354+ HCCL_VM_INFO("sys mem memset ptr:{:p}, maxCount: {:d}, value: {:d}, count: {:d}", devPtr, maxCount, value, count);
356 return ACL_SUCCESS;355 return ACL_SUCCESS;
357 }356 }
358 357 
359 char* hostPtr = (char*)sim::DeviceMemoryManager::GetInstance().AcquirePhyMem(phyMem.name, phyMem.device_id, phyMem.size);358 char* hostPtr = (char*)sim::DeviceMemoryManager::GetInstance().AcquirePhyMem(phyMem.name, phyMem.device_id, phyMem.size);
360 if (hostPtr == nullptr) {359 if (hostPtr == nullptr) {
361- MEM_STUB_INFO("dev mem memset ptr:{:p} name:{} can not get host ptr", devPtr, phyMem.name);360+ HCCL_VM_INFO("dev mem memset ptr:{:p} name:{} can not get host ptr", devPtr, phyMem.name);
362 return ACL_ERROR_INTERNAL_ERROR;361 return ACL_ERROR_INTERNAL_ERROR;
363 }362 }
364 363 
365 // 目的设备块按 size 判是否在复用区,与申请一致,命中则跳过实际写入并按池释放。364 // 目的设备块按 size 判是否在复用区,与申请一致,命中则跳过实际写入并按池释放。
366 bool pooled = sim::CommPoolPolicy::ShouldRedirect(phyMem.size, sim::IsCheckOnlyMode());365 bool pooled = sim::CommPoolPolicy::ShouldRedirect(phyMem.size, sim::IsCheckOnlyMode());
367 if (pooled) {366 if (pooled) {
368- MEM_STUB_INFO("memset skip (pool) ptr:{:p}, size:{:d}, count:{:d}", devPtr, phyMem.size, count);367+ HCCL_VM_INFO("memset skip (pool) ptr:{:p}, size:{:d}, count:{:d}", devPtr, phyMem.size, count);
369 } else {368 } else {
370 memset(hostPtr + offset, value, count);369 memset(hostPtr + offset, value, count);
371 }370 }
@@ -374,14 +373,14 @@ aclError aclrtMemset(void *devPtr, size_t maxCount, int32_t value, size_t count)
374 } else {373 } else {
375 sim::DeviceMemoryManager::GetInstance().ReleasePhyMem(phyMem.name, phyMem.device_id);374 sim::DeviceMemoryManager::GetInstance().ReleasePhyMem(phyMem.name, phyMem.device_id);
376 }375 }
377- MEM_STUB_INFO("dev mem memset ptr:{:p}, memName:{}, count: {:d}", devPtr, phyMem.name, count);376+ HCCL_VM_INFO("dev mem memset ptr:{:p}, memName:{}, count: {:d}", devPtr, phyMem.name, count);
378 return ACL_SUCCESS;377 return ACL_SUCCESS;
379}378}
380 379 
381aclError aclrtMemsetAsync(void *devPtr, size_t maxCount, int32_t value, size_t count, aclrtStream stream)380aclError aclrtMemsetAsync(void *devPtr, size_t maxCount, int32_t value, size_t count, aclrtStream stream)
382{381{
383 (void) stream;382 (void) stream;
384- MEM_STUB_INFO("..");383+ HCCL_VM_INFO("..");
385 // 当前先不生成任务,后续有需要再根据实际情况生成任务384 // 当前先不生成任务,后续有需要再根据实际情况生成任务
386 return aclrtMemset(devPtr, maxCount, value, count);385 return aclrtMemset(devPtr, maxCount, value, count);
387}386}
@@ -403,37 +402,37 @@ aclError aclrtMemcpy(void *dst, size_t destMax, const void *src, size_t count, a
403 } else if (kind == ACL_MEMCPY_DEVICE_TO_DEVICE) {402 } else if (kind == ACL_MEMCPY_DEVICE_TO_DEVICE) {
404 auto dstAddr = GetRealPtrByAddr(dst);403 auto dstAddr = GetRealPtrByAddr(dst);
405 if (dstAddr == nullptr) {404 if (dstAddr == nullptr) {
406- MEM_STUB_WARN(" only support self D2D memcpy");405+ HCCL_VM_WARN(" only support self D2D memcpy");
407 return ACL_ERROR_RT_FEATURE_NOT_SUPPORT;406 return ACL_ERROR_RT_FEATURE_NOT_SUPPORT;
408 }407 }
409 auto srcAddr = GetRealPtrByAddr(src);408 auto srcAddr = GetRealPtrByAddr(src);
410 if (srcAddr == nullptr) {409 if (srcAddr == nullptr) {
411- MEM_STUB_WARN("only support self D2D memcpy");410+ HCCL_VM_WARN("only support self D2D memcpy");
412 return ACL_ERROR_RT_FEATURE_NOT_SUPPORT;411 return ACL_ERROR_RT_FEATURE_NOT_SUPPORT;
413 }412 }
414 // 目的设备块按 size 判是否在复用区,与申请一致,命中则跳过实际拷贝。413 // 目的设备块按 size 判是否在复用区,与申请一致,命中则跳过实际拷贝。
415 if (IsInCommPool(dstAddr)) {414 if (IsInCommPool(dstAddr)) {
416- MEM_STUB_INFO("memcpy D2D skip (dst big->pool) count:{:d}", count);415+ HCCL_VM_INFO("memcpy D2D skip (dst big->pool) count:{:d}", count);
417 } else {416 } else {
418 memcpy(dstAddr, srcAddr, count);417 memcpy(dstAddr, srcAddr, count);
419 }418 }
420 return ACL_SUCCESS;419 return ACL_SUCCESS;
421 } else {420 } else {
422- MEM_STUB_INFO("src:{:p} to dst:{:p}, size:{:d} type: {:d} not support", src, dst, count, (int)kind);421+ HCCL_VM_INFO("src:{:p} to dst:{:p}, size:{:d} type: {:d} not support", src, dst, count, (int)kind);
423 return ACL_ERROR_RT_FEATURE_NOT_SUPPORT;422 return ACL_ERROR_RT_FEATURE_NOT_SUPPORT;
424 }423 }
425 424 
426 sim::PhyMemBlock phyMem{};425 sim::PhyMemBlock phyMem{};
427 uint32_t offset = 0;426 uint32_t offset = 0;
428 if (!GetPtrNameByVirPtr(devPtr, offset, phyMem)) { 427 if (!GetPtrNameByVirPtr(devPtr, offset, phyMem)) {
429- MEM_STUB_INFO("dev mem memcpy ptr:{:p}, count: {:d}", devPtr, count);428+ HCCL_VM_INFO("dev mem memcpy ptr:{:p}, count: {:d}", devPtr, count);
430 return ACL_ERROR_INTERNAL_ERROR;429 return ACL_ERROR_INTERNAL_ERROR;
431 }430 }
432 431 
433- MEM_STUB_INFO("host:{:p} dev:{:p} memName:{} size:{:d} type:{:d}", hostPtr, devPtr, phyMem.name, count, (int)kind);432+ HCCL_VM_INFO("host:{:p} dev:{:p} memName:{} size:{:d} type:{:d}", hostPtr, devPtr, phyMem.name, count, (int)kind);
434 char* hostDevPtr = (char*)sim::DeviceMemoryManager::GetInstance().AcquirePhyMem(phyMem.name, phyMem.device_id, phyMem.size);433 char* hostDevPtr = (char*)sim::DeviceMemoryManager::GetInstance().AcquirePhyMem(phyMem.name, phyMem.device_id, phyMem.size);
435 if (hostDevPtr == nullptr) {434 if (hostDevPtr == nullptr) {
436- MEM_STUB_INFO(" dev mem memcpy ptr:{:p} name:{} can not get host ptr", devPtr, phyMem.name);435+ HCCL_VM_INFO(" dev mem memcpy ptr:{:p} name:{} can not get host ptr", devPtr, phyMem.name);
437 return ACL_ERROR_INTERNAL_ERROR;436 return ACL_ERROR_INTERNAL_ERROR;
438 }437 }
439 438 
@@ -442,13 +441,13 @@ aclError aclrtMemcpy(void *dst, size_t destMax, const void *src, size_t count, a
442 bool devPooled = sim::CommPoolPolicy::ShouldRedirect(phyMem.size, sim::IsCheckOnlyMode());441 bool devPooled = sim::CommPoolPolicy::ShouldRedirect(phyMem.size, sim::IsCheckOnlyMode());
443 if (kind == ACL_MEMCPY_HOST_TO_DEVICE) {442 if (kind == ACL_MEMCPY_HOST_TO_DEVICE) {
444 if (devPooled) {443 if (devPooled) {
445- MEM_STUB_INFO("memcpy H2D skip (dst big->pool) count:{:d}", count);444+ HCCL_VM_INFO("memcpy H2D skip (dst big->pool) count:{:d}", count);
446 } else {445 } else {
447 memcpy(hostDevPtr + offset, hostPtr, count);446 memcpy(hostDevPtr + offset, hostPtr, count);
448 }447 }
449 } else if (kind == ACL_MEMCPY_DEVICE_TO_HOST) {448 } else if (kind == ACL_MEMCPY_DEVICE_TO_HOST) {
450 if (IsInCommPool(hostPtr)) {449 if (IsInCommPool(hostPtr)) {
451- MEM_STUB_INFO("memcpy D2H skip (dst host pool) count:{:d}", count);450+ HCCL_VM_INFO("memcpy D2H skip (dst host pool) count:{:d}", count);
452 } else {451 } else {
453 memcpy(hostPtr, hostDevPtr + offset, count);452 memcpy(hostPtr, hostDevPtr + offset, count);
454 }453 }
@@ -493,7 +492,7 @@ aclError aclrtMemcpyAsync(void *dst, size_t destMax, const void *src, size_t cou
493 uint32_t index{0};492 uint32_t index{0};
494 auto ret = InsertTaskToCollection(&taskMetaData, &index);493 auto ret = InsertTaskToCollection(&taskMetaData, &index);
495 if (ret != HcclSim::HcclVmResult::HCCL_SIM_SUCCESS) {494 if (ret != HcclSim::HcclVmResult::HCCL_SIM_SUCCESS) {
496- MEM_STUB_ERROR("InsertTaskToCollection fail");495+ HCCL_VM_ERROR("InsertTaskToCollection fail");
497 return ACL_ERROR_INTERNAL_ERROR;496 return ACL_ERROR_INTERNAL_ERROR;
498 }497 }
499 498 
@@ -507,22 +506,22 @@ aclError aclrtMemcpyAsync(void *dst, size_t destMax, const void *src, size_t cou
507 auto taskId = RunnerDB::Add<sim::Task>(task);506 auto taskId = RunnerDB::Add<sim::Task>(task);
508 return ACL_SUCCESS;507 return ACL_SUCCESS;
509 } else {508 } else {
510- MEM_STUB_INFO("src:{:p} to dst:{:p}, size:{:d} type: {:d} not support", src, dst, count, (int)kind);509+ HCCL_VM_INFO("src:{:p} to dst:{:p}, size:{:d} type: {:d} not support", src, dst, count, (int)kind);
511 return ACL_ERROR_RT_FEATURE_NOT_SUPPORT;510 return ACL_ERROR_RT_FEATURE_NOT_SUPPORT;
512 }511 }
513 512 
514 sim::PhyMemBlock phyMem{};513 sim::PhyMemBlock phyMem{};
515 uint32_t offset = 0;514 uint32_t offset = 0;
516 if (!GetPtrNameByVirPtr(devPtr, offset, phyMem)) {515 if (!GetPtrNameByVirPtr(devPtr, offset, phyMem)) {
517- MEM_STUB_INFO("dev mem memcpy ptr:{:p}, count: {:d}", devPtr, count);516+ HCCL_VM_INFO("dev mem memcpy ptr:{:p}, count: {:d}", devPtr, count);
518 return ACL_ERROR_INTERNAL_ERROR;517 return ACL_ERROR_INTERNAL_ERROR;
519 }518 }
520 519 
521- MEM_STUB_INFO("host:{:p} dev:{:p} memName:{} size:{:d} type:{:d}", hostPtr, devPtr, phyMem.name, count, (int)kind);520+ HCCL_VM_INFO("host:{:p} dev:{:p} memName:{} size:{:d} type:{:d}", hostPtr, devPtr, phyMem.name, count, (int)kind);
522 521 
523 char* hostDevPtr = (char*)sim::DeviceMemoryManager::GetInstance().AcquirePhyMem(phyMem.name, phyMem.device_id, phyMem.size);522 char* hostDevPtr = (char*)sim::DeviceMemoryManager::GetInstance().AcquirePhyMem(phyMem.name, phyMem.device_id, phyMem.size);
524 if (hostDevPtr == nullptr) {523 if (hostDevPtr == nullptr) {
525- MEM_STUB_INFO("dev mem memcpy ptr:{:p} name:{} can not get host ptr", devPtr, phyMem.name);524+ HCCL_VM_INFO("dev mem memcpy ptr:{:p} name:{} can not get host ptr", devPtr, phyMem.name);
526 return ACL_ERROR_INTERNAL_ERROR;525 return ACL_ERROR_INTERNAL_ERROR;
527 }526 }
528 527 
@@ -531,13 +530,13 @@ aclError aclrtMemcpyAsync(void *dst, size_t destMax, const void *src, size_t cou
531 bool devPooled = sim::CommPoolPolicy::ShouldRedirect(phyMem.size, sim::IsCheckOnlyMode());530 bool devPooled = sim::CommPoolPolicy::ShouldRedirect(phyMem.size, sim::IsCheckOnlyMode());
532 if (kind == ACL_MEMCPY_HOST_TO_DEVICE) {531 if (kind == ACL_MEMCPY_HOST_TO_DEVICE) {
533 if (devPooled) {532 if (devPooled) {
534- MEM_STUB_INFO("memcpy H2D skip (dst big->pool) count:{:d}", count);533+ HCCL_VM_INFO("memcpy H2D skip (dst big->pool) count:{:d}", count);
535 } else {534 } else {
536 memcpy(hostDevPtr + offset, hostPtr, count);535 memcpy(hostDevPtr + offset, hostPtr, count);
537 }536 }
538 } else if (kind == ACL_MEMCPY_DEVICE_TO_HOST) {537 } else if (kind == ACL_MEMCPY_DEVICE_TO_HOST) {
539 if (IsInCommPool(hostPtr)) {538 if (IsInCommPool(hostPtr)) {
540- MEM_STUB_INFO("memcpy D2H skip (dst host pool) count:{:d}", count);539+ HCCL_VM_INFO("memcpy D2H skip (dst host pool) count:{:d}", count);
541 } else {540 } else {
542 memcpy(hostPtr, hostDevPtr + offset, count);541 memcpy(hostPtr, hostDevPtr + offset, count);
543 }542 }
@@ -647,7 +646,7 @@ aclError aclrtMallocPhysical(aclrtDrvMemHandle *handle, size_t size, const aclrt
647 }646 }
648 auto currCtx = RunnerDB::GetById<sim::Context>(runner.current_ctx_id);647 auto currCtx = RunnerDB::GetById<sim::Context>(runner.current_ctx_id);
649 if (!currCtx.has_value()) {648 if (!currCtx.has_value()) {
650- MEM_STUB_ERROR("can not find current context:{:d}", runner.current_ctx_id);649+ HCCL_VM_ERROR("can not find current context:{:d}", runner.current_ctx_id);
651 return ACL_ERROR_INTERNAL_ERROR;650 return ACL_ERROR_INTERNAL_ERROR;
652 }651 }
653 652 
@@ -656,7 +655,7 @@ aclError aclrtMallocPhysical(aclrtDrvMemHandle *handle, size_t size, const aclrt
656 std::string memName = GenDevMemName(deviceId);655 std::string memName = GenDevMemName(deviceId);
657 auto hostPtr = sim::DeviceMemoryManager::GetInstance().AllocPhyMem(memName.c_str(), deviceId, size);656 auto hostPtr = sim::DeviceMemoryManager::GetInstance().AllocPhyMem(memName.c_str(), deviceId, size);
658 if (hostPtr == nullptr) {657 if (hostPtr == nullptr) {
659- MEM_STUB_ERROR("can not alloc phy mem deviceId:{:d}, size:{:d}", deviceId, size);658+ HCCL_VM_ERROR("can not alloc phy mem deviceId:{:d}, size:{:d}", deviceId, size);
660 return ACL_ERROR_INTERNAL_ERROR;659 return ACL_ERROR_INTERNAL_ERROR;
661 }660 }
662 // 大块物理内存同样引流到复用区,缓存池基址,使其被 map 后的 memcpy/memset 能按地址判定。661 // 大块物理内存同样引流到复用区,缓存池基址,使其被 map 后的 memcpy/memset 能按地址判定。
@@ -672,7 +671,7 @@ aclError aclrtMallocPhysical(aclrtDrvMemHandle *handle, size_t size, const aclrt
672 phyMem.ref_count = 1;671 phyMem.ref_count = 1;
673 auto phyMemId = RunnerDB::Add<sim::PhyMemBlock>(phyMem);672 auto phyMemId = RunnerDB::Add<sim::PhyMemBlock>(phyMem);
674 673 
675- MEM_STUB_INFO("malloc phy mem name:{}, size:{:d}, id:{:d}", memName, size, phyMemId);674+ HCCL_VM_INFO("malloc phy mem name:{}, size:{:d}, id:{:d}", memName, size, phyMemId);
676 *handle = (aclrtDrvMemHandle)phyMemId;675 *handle = (aclrtDrvMemHandle)phyMemId;
677 return ACL_SUCCESS;676 return ACL_SUCCESS;
678}677}
@@ -682,7 +681,7 @@ aclError aclrtFreePhysical(aclrtDrvMemHandle handle)
682 auto phyMemId = (uint64_t)(uintptr_t)handle;681 auto phyMemId = (uint64_t)(uintptr_t)handle;
683 auto phyMemRes = RunnerDB::GetById<sim::PhyMemBlock>(phyMemId);682 auto phyMemRes = RunnerDB::GetById<sim::PhyMemBlock>(phyMemId);
684 if (!phyMemRes.has_value()) {683 if (!phyMemRes.has_value()) {
685- MEM_STUB_ERROR("can not find phy Mem id:{:d}", phyMemId);684+ HCCL_VM_ERROR("can not find phy Mem id:{:d}", phyMemId);
686 return ACL_ERROR_INTERNAL_ERROR;685 return ACL_ERROR_INTERNAL_ERROR;
687 };686 };
688 687 
@@ -694,7 +693,7 @@ aclError aclrtFreePhysical(aclrtDrvMemHandle handle)
694 }693 }
695 // 删除数据库记录694 // 删除数据库记录
696 RunnerDB::Delete<sim::PhyMemBlock>(phyMemId);695 RunnerDB::Delete<sim::PhyMemBlock>(phyMemId);
697- MEM_STUB_INFO("free phy mem, id:{:d}", phyMemId);696+ HCCL_VM_INFO("free phy mem, id:{:d}", phyMemId);
698 697
699 return ACL_SUCCESS;698 return ACL_SUCCESS;
700}699}
@@ -710,7 +709,7 @@ aclError aclrtReserveMemAddress(void **virPtr, size_t size, size_t alignment, vo
710 }709 }
711 auto currCtx = RunnerDB::GetById<sim::Context>(runner.current_ctx_id);710 auto currCtx = RunnerDB::GetById<sim::Context>(runner.current_ctx_id);
712 if (!currCtx.has_value()) {711 if (!currCtx.has_value()) {
713- MEM_STUB_ERROR("can not find current context:{:d}", runner.current_ctx_id);712+ HCCL_VM_ERROR("can not find current context:{:d}", runner.current_ctx_id);
714 return ACL_ERROR_INTERNAL_ERROR;713 return ACL_ERROR_INTERNAL_ERROR;
715 }714 }
716 715 
@@ -718,7 +717,7 @@ aclError aclrtReserveMemAddress(void **virPtr, size_t size, size_t alignment, vo
718 717 
719 auto dev = RunnerDB::GetById<sim::Device>(deviceId);718 auto dev = RunnerDB::GetById<sim::Device>(deviceId);
720 if (!dev.has_value()) {719 if (!dev.has_value()) {
721- MEM_STUB_ERROR("can not find device id:{:d}", deviceId);720+ HCCL_VM_ERROR("can not find device id:{:d}", deviceId);
722 return ACL_ERROR_INTERNAL_ERROR;721 return ACL_ERROR_INTERNAL_ERROR;
723 }722 }
724 723 
@@ -727,7 +726,7 @@ aclError aclrtReserveMemAddress(void **virPtr, size_t size, size_t alignment, vo
727 std::string memName = GenDevMemName(deviceId);726 std::string memName = GenDevMemName(deviceId);
728 auto devPtr = sim::DeviceMemoryManager::GetInstance().AllocVirMem( devPhyId, size);727 auto devPtr = sim::DeviceMemoryManager::GetInstance().AllocVirMem( devPhyId, size);
729 if (devPtr == nullptr) {728 if (devPtr == nullptr) {
730- MEM_STUB_ERROR("can not alloc vir mem deviceId:{:d}, phyMemId:{:d}, size:{:d}", deviceId, devPhyId, size);729+ HCCL_VM_ERROR("can not alloc vir mem deviceId:{:d}, phyMemId:{:d}, size:{:d}", deviceId, devPhyId, size);
731 return ACL_ERROR_INTERNAL_ERROR;730 return ACL_ERROR_INTERNAL_ERROR;
732 }731 }
733 732 
@@ -741,7 +740,7 @@ aclError aclrtReserveMemAddress(void **virPtr, size_t size, size_t alignment, vo
741 virMem.src_type = (uint8_t)sim::VIR_MEM_TYPE_DEV;740 virMem.src_type = (uint8_t)sim::VIR_MEM_TYPE_DEV;
742 virMem.policy = 0;741 virMem.policy = 0;
743 RunnerDB::Add<sim::VirtualMemBlock>(virMem);742 RunnerDB::Add<sim::VirtualMemBlock>(virMem);
744- MEM_STUB_INFO("reserve vir mem, size:{:d}, devPtr:{:p}", size, devPtr);743+ HCCL_VM_INFO("reserve vir mem, size:{:d}, devPtr:{:p}", size, devPtr);
745 *virPtr = devPtr;744 *virPtr = devPtr;
746 return ACL_SUCCESS;745 return ACL_SUCCESS;
747}746}
@@ -753,13 +752,13 @@ aclError aclrtReleaseMemAddress(void *virPtr)
753 [devPtr](const sim::VirtualMemBlock &virMem)752 [devPtr](const sim::VirtualMemBlock &virMem)
754 { return virMem.start_ptr == devPtr && virMem.src_type == (uint8_t)sim::VIR_MEM_TYPE_DEV; });753 { return virMem.start_ptr == devPtr && virMem.src_type == (uint8_t)sim::VIR_MEM_TYPE_DEV; });
755 if (!virMemRes.second) {754 if (!virMemRes.second) {
756- MEM_STUB_ERROR("can not find this buff offset ptr:{:p}", devPtr);755+ HCCL_VM_ERROR("can not find this buff offset ptr:{:p}", devPtr);
757 return ACL_ERROR_INTERNAL_ERROR;756 return ACL_ERROR_INTERNAL_ERROR;
758 }757 }
759 auto ctxId = virMemRes.first.ctx_id;758 auto ctxId = virMemRes.first.ctx_id;
760 auto currCtx = RunnerDB::GetById<sim::Context>(ctxId);759 auto currCtx = RunnerDB::GetById<sim::Context>(ctxId);
761 if (!currCtx.has_value()) {760 if (!currCtx.has_value()) {
762- MEM_STUB_ERROR("can not find current context:{:d}", ctxId);761+ HCCL_VM_ERROR("can not find current context:{:d}", ctxId);
763 return ACL_ERROR_INTERNAL_ERROR;762 return ACL_ERROR_INTERNAL_ERROR;
764 }763 }
765 764 
@@ -767,7 +766,7 @@ aclError aclrtReleaseMemAddress(void *virPtr)
767 766 
768 auto dev = RunnerDB::GetById<sim::Device>(deviceId);767 auto dev = RunnerDB::GetById<sim::Device>(deviceId);
769 if (!dev.has_value()) {768 if (!dev.has_value()) {
770- MEM_STUB_ERROR("can not find device id:{:d}", deviceId);769+ HCCL_VM_ERROR("can not find device id:{:d}", deviceId);
771 return ACL_ERROR_INTERNAL_ERROR;770 return ACL_ERROR_INTERNAL_ERROR;
772 }771 }
773 772 
@@ -775,7 +774,7 @@ aclError aclrtReleaseMemAddress(void *virPtr)
775 774 
776 sim::DeviceMemoryManager::GetInstance().FreeVirMem(devPhyId, virPtr);775 sim::DeviceMemoryManager::GetInstance().FreeVirMem(devPhyId, virPtr);
777 RunnerDB::Delete<sim::VirtualMemBlock>(virMemRes.first.id);776 RunnerDB::Delete<sim::VirtualMemBlock>(virMemRes.first.id);
778- MEM_STUB_INFO("release vir mem, devPtr:{:p}", virPtr);777+ HCCL_VM_INFO("release vir mem, devPtr:{:p}", virPtr);
779 778
780 return ACL_SUCCESS;779 return ACL_SUCCESS;
781}780}
@@ -791,13 +790,13 @@ aclError aclrtMapMem(void *virPtr, size_t size, size_t offset, aclrtDrvMemHandle
791 [devPtr](const sim::VirtualMemBlock &virMem)790 [devPtr](const sim::VirtualMemBlock &virMem)
792 { return virMem.start_ptr == devPtr && virMem.src_type == (uint8_t)sim::VIR_MEM_TYPE_DEV; });791 { return virMem.start_ptr == devPtr && virMem.src_type == (uint8_t)sim::VIR_MEM_TYPE_DEV; });
793 if (!virMemRes.second) {792 if (!virMemRes.second) {
794- MEM_STUB_ERROR("can not find this buff offset ptr:{:p}", devPtr);793+ HCCL_VM_ERROR("can not find this buff offset ptr:{:p}", devPtr);
795 return ACL_ERROR_INTERNAL_ERROR;794 return ACL_ERROR_INTERNAL_ERROR;
796 }795 }
797 796 
798 auto phyMemRes = RunnerDB::GetById<sim::PhyMemBlock>(phyMemId);797 auto phyMemRes = RunnerDB::GetById<sim::PhyMemBlock>(phyMemId);
799 if (!phyMemRes.has_value()) {798 if (!phyMemRes.has_value()) {
800- MEM_STUB_ERROR("can not find phy Mem id:{:d}", phyMemId);799+ HCCL_VM_ERROR("can not find phy Mem id:{:d}", phyMemId);
801 return ACL_ERROR_INTERNAL_ERROR;800 return ACL_ERROR_INTERNAL_ERROR;
802 };801 };
803 802 
@@ -816,14 +815,14 @@ aclError aclrtUnmapMem(void *virPtr)
816 [devPtr](const sim::VirtualMemBlock &virMem)815 [devPtr](const sim::VirtualMemBlock &virMem)
817 { return virMem.start_ptr == devPtr && virMem.src_type == (uint8_t)sim::VIR_MEM_TYPE_DEV; });816 { return virMem.start_ptr == devPtr && virMem.src_type == (uint8_t)sim::VIR_MEM_TYPE_DEV; });
818 if (!virMemRes.second) {817 if (!virMemRes.second) {
819- MEM_STUB_ERROR("can not find this buff offset ptr:{:p}", devPtr);818+ HCCL_VM_ERROR("can not find this buff offset ptr:{:p}", devPtr);
820 return ACL_ERROR_INTERNAL_ERROR;819 return ACL_ERROR_INTERNAL_ERROR;
821 }820 }
822 821 
823 auto phyMemId = virMemRes.first.phy_mem_id;822 auto phyMemId = virMemRes.first.phy_mem_id;
824 auto phyMemRes = RunnerDB::GetById<sim::PhyMemBlock>(phyMemId);823 auto phyMemRes = RunnerDB::GetById<sim::PhyMemBlock>(phyMemId);
825 if (!phyMemRes.has_value()) {824 if (!phyMemRes.has_value()) {
826- MEM_STUB_ERROR("can not find phy Mem id:{:d}", phyMemId);825+ HCCL_VM_ERROR("can not find phy Mem id:{:d}", phyMemId);
827 return ACL_ERROR_INTERNAL_ERROR;826 return ACL_ERROR_INTERNAL_ERROR;
828 };827 };
829 828 
@@ -842,7 +841,7 @@ aclError aclrtMemExportToShareableHandle(aclrtDrvMemHandle handle, aclrtMemHandl
842 841 
843 auto phyMemRes = RunnerDB::GetById<sim::PhyMemBlock>(phyMemId);842 auto phyMemRes = RunnerDB::GetById<sim::PhyMemBlock>(phyMemId);
844 if (!phyMemRes.has_value()) {843 if (!phyMemRes.has_value()) {
845- MEM_STUB_ERROR("cannot find phy Mem offset: {:d}", phyMemId);844+ HCCL_VM_ERROR("cannot find phy Mem offset: {:d}", phyMemId);
846 return ACL_ERROR_INVALID_PARAM;845 return ACL_ERROR_INVALID_PARAM;
847 }846 }
848 847 
@@ -884,7 +883,7 @@ aclError aclrtMemImportFromShareableHandle(uint64_t shareableHandle, int32_t dev
884 883 
885 auto phyMemRes = RunnerDB::GetById<sim::PhyMemBlock>(phyMemId);884 auto phyMemRes = RunnerDB::GetById<sim::PhyMemBlock>(phyMemId);
886 if (!phyMemRes.has_value()) {885 if (!phyMemRes.has_value()) {
887- MEM_STUB_ERROR("cannot find phy Mem offset: {:d}", phyMemId);886+ HCCL_VM_ERROR("cannot find phy Mem offset: {:d}", phyMemId);
888 return ACL_ERROR_INVALID_PARAM;887 return ACL_ERROR_INVALID_PARAM;
889 }888 }
890 889 
@@ -934,20 +933,20 @@ aclError aclrtPointerGetAttributes(const void *ptr, aclrtPtrAttributes *attribut
934 auto virMemRes = RunnerDB::GetOneByPred<sim::VirtualMemBlock>(933 auto virMemRes = RunnerDB::GetOneByPred<sim::VirtualMemBlock>(
935 [startPtr](const sim::VirtualMemBlock &virMem) { return virMem.start_ptr == startPtr && virMem.src_type == (uint8_t)sim::VIR_MEM_TYPE_DEV;});934 [startPtr](const sim::VirtualMemBlock &virMem) { return virMem.start_ptr == startPtr && virMem.src_type == (uint8_t)sim::VIR_MEM_TYPE_DEV;});
936 if (!virMemRes.second) {935 if (!virMemRes.second) {
937- MEM_STUB_ERROR("can not find this buff offset ptr: 0x{:x}", startPtr);936+ HCCL_VM_ERROR("can not find this buff offset ptr: 0x{:x}", startPtr);
938 return ACL_ERROR_INVALID_PARAM;937 return ACL_ERROR_INVALID_PARAM;
939 }938 }
940 939 
941 auto phyMemId = virMemRes.first.phy_mem_id;940 auto phyMemId = virMemRes.first.phy_mem_id;
942 auto phyMemRes = RunnerDB::GetById<sim::PhyMemBlock>(phyMemId);941 auto phyMemRes = RunnerDB::GetById<sim::PhyMemBlock>(phyMemId);
943 if (!phyMemRes.has_value()) {942 if (!phyMemRes.has_value()) {
944- MEM_STUB_ERROR("cannot find phy Mem offset: {:d}", phyMemId);943+ HCCL_VM_ERROR("cannot find phy Mem offset: {:d}", phyMemId);
945 return ACL_ERROR_INVALID_PARAM;944 return ACL_ERROR_INVALID_PARAM;
946 }945 }
947 946 
948 auto devRes = RunnerDB::GetById<sim::Device>(phyMemRes->device_id);947 auto devRes = RunnerDB::GetById<sim::Device>(phyMemRes->device_id);
949 if (!devRes.has_value()) {948 if (!devRes.has_value()) {
950- MEM_STUB_ERROR("cannot find phy Mem device: {:d}", phyMemRes->device_id);949+ HCCL_VM_ERROR("cannot find phy Mem device: {:d}", phyMemRes->device_id);
951 return ACL_ERROR_INVALID_PARAM;950 return ACL_ERROR_INVALID_PARAM;
952 }951 }
953 952 
@@ -964,7 +963,7 @@ aclError aclrtHostRegister(void *ptr, uint64_t size, aclrtHostRegisterType type,
964 auto virMemRes = RunnerDB::GetOneByPred<sim::VirtualMemBlock>(963 auto virMemRes = RunnerDB::GetOneByPred<sim::VirtualMemBlock>(
965 [startPtr](const sim::VirtualMemBlock &virMem) { return virMem.start_ptr == startPtr && virMem.src_type == (uint8_t)sim::VIR_MEM_TYPE_HOST;});964 [startPtr](const sim::VirtualMemBlock &virMem) { return virMem.start_ptr == startPtr && virMem.src_type == (uint8_t)sim::VIR_MEM_TYPE_HOST;});
966 if (!virMemRes.second) {965 if (!virMemRes.second) {
967- MEM_STUB_ERROR("can not find this buff offset ptr: 0x{:x}", startPtr);966+ HCCL_VM_ERROR("can not find this buff offset ptr: 0x{:x}", startPtr);
968 return ACL_ERROR_INVALID_PARAM;967 return ACL_ERROR_INVALID_PARAM;
969 }968 }
970 *devPtr = ptr;969 *devPtr = ptr;
@@ -977,7 +976,7 @@ aclError aclrtHostUnregister(void *ptr)
977 auto virMemRes = RunnerDB::GetOneByPred<sim::VirtualMemBlock>(976 auto virMemRes = RunnerDB::GetOneByPred<sim::VirtualMemBlock>(
978 [startPtr](const sim::VirtualMemBlock &virMem) { return virMem.start_ptr == startPtr && virMem.src_type == (uint8_t)sim::VIR_MEM_TYPE_HOST;});977 [startPtr](const sim::VirtualMemBlock &virMem) { return virMem.start_ptr == startPtr && virMem.src_type == (uint8_t)sim::VIR_MEM_TYPE_HOST;});
979 if (!virMemRes.second) {978 if (!virMemRes.second) {
980- MEM_STUB_ERROR("can not find this buff offset ptr: 0x{:x}", startPtr);979+ HCCL_VM_ERROR("can not find this buff offset ptr: 0x{:x}", startPtr);
981 return ACL_ERROR_INVALID_PARAM;980 return ACL_ERROR_INVALID_PARAM;
982 }981 }
983 return ACL_SUCCESS;982 return ACL_SUCCESS;
@@ -1008,15 +1007,15 @@ aclError aclrtIpcMemGetExportKey(void *devPtr, size_t size, char *key, size_t le
1008 (void) flags;1007 (void) flags;
1009 uint64_t startPtr = static_cast<uint64_t>(reinterpret_cast<uintptr_t>(devPtr));1008 uint64_t startPtr = static_cast<uint64_t>(reinterpret_cast<uintptr_t>(devPtr));
1010 auto virMemRes = RunnerDB::GetOneByPred<sim::VirtualMemBlock>(1009 auto virMemRes = RunnerDB::GetOneByPred<sim::VirtualMemBlock>(
1011- [startPtr](const sim::VirtualMemBlock &virMem) { return virMem.start_ptr == startPtr;});1010+ [startPtr](const sim::VirtualMemBlock &virMem) { return virMem.start_ptr == startPtr; });
1012 if (!virMemRes.second) {1011 if (!virMemRes.second) {
1013- MEM_STUB_ERROR("can not find this buff offset ptr: 0x{:x}", startPtr);1012+ HCCL_VM_ERROR("can not find this buff offset ptr: 0x{:x}", startPtr);
1014 return ACL_ERROR_INVALID_PARAM;1013 return ACL_ERROR_INVALID_PARAM;
1015 }1014 }
1016 auto phyMemId = virMemRes.first.phy_mem_id;1015 auto phyMemId = virMemRes.first.phy_mem_id;
1017 auto phyMemRes = RunnerDB::GetById<sim::PhyMemBlock>(phyMemId);1016 auto phyMemRes = RunnerDB::GetById<sim::PhyMemBlock>(phyMemId);
1018 if (!phyMemRes.has_value()) {1017 if (!phyMemRes.has_value()) {
1019- MEM_STUB_ERROR("cannot find phy Mem offset: {:d}", phyMemId);1018+ HCCL_VM_ERROR("cannot find phy Mem offset: {:d}", phyMemId);
1020 return ACL_ERROR_INVALID_PARAM;1019 return ACL_ERROR_INVALID_PARAM;
1021 }1020 }
1022 1021 
@@ -1056,13 +1055,13 @@ aclError aclrtIpcMemImportByKey(void **devPtr, const char *key, uint64_t flags)
1056 uint64_t ipcRecordIdx = *(const uint64_t *)key;1055 uint64_t ipcRecordIdx = *(const uint64_t *)key;
1057 auto recordRes = RunnerDB::GetById<sim::IpcMemRecord>(ipcRecordIdx);1056 auto recordRes = RunnerDB::GetById<sim::IpcMemRecord>(ipcRecordIdx);
1058 if (!recordRes.has_value()) {1057 if (!recordRes.has_value()) {
1059- MEM_STUB_ERROR("cannot find ipc record: {:d}", ipcRecordIdx);1058+ HCCL_VM_ERROR("cannot find ipc record: {:d}", ipcRecordIdx);
1060 return ACL_ERROR_INVALID_PARAM;1059 return ACL_ERROR_INVALID_PARAM;
1061 }1060 }
1062 1061 
1063 auto virMemRes = RunnerDB::GetById<sim::VirtualMemBlock>(recordRes->vir_mem_id);1062 auto virMemRes = RunnerDB::GetById<sim::VirtualMemBlock>(recordRes->vir_mem_id);
1064 if (!virMemRes.has_value()) {1063 if (!virMemRes.has_value()) {
1065- MEM_STUB_ERROR("cannot find vir mem: {:d}", recordRes->vir_mem_id);1064+ HCCL_VM_ERROR("cannot find vir mem: {:d}", recordRes->vir_mem_id);
1066 return ACL_ERROR_INVALID_PARAM;1065 return ACL_ERROR_INVALID_PARAM;
1067 }1066 }
1068 *devPtr = (void *)virMemRes->start_ptr;1067 *devPtr = (void *)virMemRes->start_ptr;
@@ -1075,7 +1074,7 @@ aclError aclrtIpcMemClose(const char *key)
1075 uint64_t ipcRecordIdx = *reinterpret_cast<const uint64_t*>(key);1074 uint64_t ipcRecordIdx = *reinterpret_cast<const uint64_t*>(key);
1076 auto recordRes = RunnerDB::GetById<sim::IpcMemRecord>(ipcRecordIdx);1075 auto recordRes = RunnerDB::GetById<sim::IpcMemRecord>(ipcRecordIdx);
1077 if (!recordRes.has_value()) {1076 if (!recordRes.has_value()) {
1078- MEM_STUB_ERROR("cannot find ipc record: {:d}", ipcRecordIdx);1077+ HCCL_VM_ERROR("cannot find ipc record: {:d}", ipcRecordIdx);
1079 return ACL_ERROR_INVALID_PARAM;1078 return ACL_ERROR_INVALID_PARAM;
1080 }1079 }
1081 1080 
@@ -1156,14 +1155,14 @@ rtError_t rtMalloc(void **devPtr, uint64_t size, rtMemType_t type, const uint16_
1156 (void) type;1155 (void) type;
1157 (void) moduleId;1156 (void) moduleId;
1158 int ret = aclrtMalloc(devPtr, size, aclrtMemMallocPolicy::ACL_MEM_MALLOC_HUGE_FIRST);1157 int ret = aclrtMalloc(devPtr, size, aclrtMemMallocPolicy::ACL_MEM_MALLOC_HUGE_FIRST);
1159- MEM_STUB_INFO("enter into stub rtMalloc.... addr:{:p}", *devPtr);1158+ HCCL_VM_INFO("enter into stub rtMalloc.... addr:{:p}", *devPtr);
1160 return ret;1159 return ret;
1161}1160}
1162 1161 
1163aclError rtFree(void* devPtr)1162aclError rtFree(void* devPtr)
1164{1163{
1165 (void) devPtr;1164 (void) devPtr;
1166- MEM_STUB_ERROR("enter into rtFree not support");1165+ HCCL_VM_ERROR("enter into rtFree not support");
1167 return ACL_ERROR_INTERNAL_ERROR;1166 return ACL_ERROR_INTERNAL_ERROR;
1168}1167}
1169 1168 
@@ -1190,7 +1189,7 @@ aclError aclrtReduceAsync(void *dst, const void *src, uint64_t count, aclrtReduc
1190 1189 
1191 auto iter = rtDataType2CheckerDataType.find(type);1190 auto iter = rtDataType2CheckerDataType.find(type);
1192 if (iter == rtDataType2CheckerDataType.end()) {1191 if (iter == rtDataType2CheckerDataType.end()) {
1193- MEM_STUB_ERROR("not support data type: {}", static_cast<uint32_t>(type));1192+ HCCL_VM_ERROR("not support data type: {}", static_cast<uint32_t>(type));
1194 return ACL_ERROR_RT_FEATURE_NOT_SUPPORT;1193 return ACL_ERROR_RT_FEATURE_NOT_SUPPORT;
1195 }1194 }
1196 1195 
@@ -1218,10 +1217,10 @@ aclError aclrtReduceAsync(void *dst, const void *src, uint64_t count, aclrtReduc
1218 taskMetaData.taskData.reduce.reduceOp = static_cast<uint8_t>(kind);1217 taskMetaData.taskData.reduce.reduceOp = static_cast<uint8_t>(kind);
1219 1218 
1220 uint32_t index{0};1219 uint32_t index{0};
1221- MEM_STUB_DEBUG("Get reduce task, streamId={:d}", streamId);1220+ HCCL_VM_DEBUG("Get reduce task, streamId={:d}", streamId);
1222 auto ret = InsertTaskToCollection(&taskMetaData, &index);1221 auto ret = InsertTaskToCollection(&taskMetaData, &index);
1223 if (ret != HcclSim::HcclVmResult::HCCL_SIM_SUCCESS) {1222 if (ret != HcclSim::HcclVmResult::HCCL_SIM_SUCCESS) {
1224- MEM_STUB_ERROR("InsertTaskToCollection fail");1223+ HCCL_VM_ERROR("InsertTaskToCollection fail");
1225 return ACL_ERROR_INTERNAL_ERROR;1224 return ACL_ERROR_INTERNAL_ERROR;
1226 }1225 }
1227 1226 
Mtest/hccl_vm/src/proxy/aclrt_new_stub.cc+6-8
@@ -8,6 +8,9 @@
8 * See LICENSE in the root of the software repository for the full text of the License.8 * See LICENSE in the root of the software repository for the full text of the License.
9 */9 */
10 10 
11+// 日志染色: 模块 tag (须在 include sim_log.h 之前)
12+#define HCCL_VM_MODULE "NEW_STUB"
13+ 
11#include <atomic>14#include <atomic>
12#include <cstdint>15#include <cstdint>
13#include <iostream>16#include <iostream>
@@ -27,11 +30,6 @@
27#include "db_sim_runner_common.h"30#include "db_sim_runner_common.h"
28#include "db_sim_runner_ops.h"31#include "db_sim_runner_ops.h"
29 32 
30-#define NEW_STUB_ERROR(format, ...) HCCL_VM_ERROR("[NEW_STUB]" format, ##__VA_ARGS__)
31-#define NEW_STUB_DEBUG(format, ...) HCCL_VM_DEBUG("[NEW_STUB]" format, ##__VA_ARGS__)
32-#define NEW_STUB_INFO(format, ...) HCCL_VM_INFO("[NEW_STUB]" format, ##__VA_ARGS__)
33-#define NEW_STUB_WARN(format, ...) HCCL_VM_WARN("[NEW_STUB]" format, ##__VA_ARGS__)
34-#define NEW_STUB_TRACE(format, ...) HCCL_VM_TRACE("[NEW_STUB]" format, ##__VA_ARGS__)
35 33 
36#ifdef __cplusplus34#ifdef __cplusplus
37extern "C" {35extern "C" {
@@ -43,16 +41,16 @@ HcclResult hrtGetDeviceType(DevType &devType)
43 auto device = RunnerDB::GetById<sim::Device>(devKey);41 auto device = RunnerDB::GetById<sim::Device>(devKey);
44 if (!device.has_value()) {42 if (!device.has_value()) {
45 // not find43 // not find
46- NEW_STUB_ERROR("current device type not found: {:d}", devKey);44+ HCCL_VM_ERROR("current device type not found: {:d}", devKey);
47 return HCCL_E_NOT_FOUND;45 return HCCL_E_NOT_FOUND;
48 }46 }
49 if (strcmp(device->soc_version, "Ascend950") == 0) {47 if (strcmp(device->soc_version, "Ascend950") == 0) {
50 devType = DevType::DEV_TYPE_950;48 devType = DevType::DEV_TYPE_950;
51 } else {49 } else {
52- NEW_STUB_ERROR("not support device soc version: {:s}", device->soc_version);50+ HCCL_VM_ERROR("not support device soc version: {:s}", device->soc_version);
53 return HCCL_E_NOT_SUPPORT;51 return HCCL_E_NOT_SUPPORT;
54 }52 }
55- NEW_STUB_TRACE("Get current device type: {}", static_cast<int>(devType));53+ HCCL_VM_TRACE("Get current device type: {}", static_cast<int>(devType));
56 return HCCL_SUCCESS;54 return HCCL_SUCCESS;
57}55}
58 56 
Mtest/hccl_vm/src/proxy/aclrt_notify_stub.cc+19-21
@@ -8,6 +8,9 @@
8 * See LICENSE in the root of the software repository for the full text of the License.8 * See LICENSE in the root of the software repository for the full text of the License.
9 */9 */
10 10 
11+// 日志染色: 模块 tag (须在 include sim_log.h 之前)
12+#define HCCL_VM_MODULE "NOTIFY_STUB"
13+ 
11#include <atomic>14#include <atomic>
12#include <cstdint>15#include <cstdint>
13#include <iostream>16#include <iostream>
@@ -23,11 +26,6 @@
23#include "runtime/event.h"26#include "runtime/event.h"
24#include "db_sim_runner_ops.h"27#include "db_sim_runner_ops.h"
25 28 
26-#define NOTIFY_STUB_ERROR(format, ...) HCCL_VM_ERROR("[NOTIFY_STUB]" format, ##__VA_ARGS__)
27-#define NOTIFY_STUB_DEBUG(format, ...) HCCL_VM_DEBUG("[NOTIFY_STUB]" format, ##__VA_ARGS__)
28-#define NOTIFY_STUB_INFO(format, ...) HCCL_VM_INFO("[NOTIFY_STUB]" format, ##__VA_ARGS__)
29-#define NOTIFY_STUB_WARN(format, ...) HCCL_VM_WARN("[NOTIFY_STUB]" format, ##__VA_ARGS__)
30-#define NOTIFY_STUB_TRACE(format, ...) HCCL_VM_TRACE("[NOTIFY_STUB]" format, ##__VA_ARGS__)
31 29 
32#ifdef __cplusplus30#ifdef __cplusplus
33extern "C" {31extern "C" {
@@ -38,21 +36,21 @@ void PrintTaskMetaData(const HcclTaskMetaData &taskMeta)
38 pid_t pid = getpid();36 pid_t pid = getpid();
39 switch (taskMeta.taskType) {37 switch (taskMeta.taskType) {
40 case HccLTaskMetaType::MEM_CPY:38 case HccLTaskMetaType::MEM_CPY:
41- NOTIFY_STUB_INFO("pid[{}]: rankId[{}], streamId[{}], taskType[MEM_CPY], srcOffset[{}], dstOffset[{}], len[{}], srcRankId[{}], dstRankId[{}]\n",39+ HCCL_VM_INFO("pid[{}]: rankId[{}], streamId[{}], taskType[MEM_CPY], srcOffset[{}], dstOffset[{}], len[{}], srcRankId[{}], dstRankId[{}]",
42 pid, taskMeta.rankId, taskMeta.streamId, taskMeta.taskData.transMem.srcOffset, taskMeta.taskData.transMem.dstOffset, taskMeta.taskData.transMem.len,40 pid, taskMeta.rankId, taskMeta.streamId, taskMeta.taskData.transMem.srcOffset, taskMeta.taskData.transMem.dstOffset, taskMeta.taskData.transMem.len,
43 taskMeta.taskData.transMem.srcRankId, taskMeta.taskData.transMem.dstRankId);41 taskMeta.taskData.transMem.srcRankId, taskMeta.taskData.transMem.dstRankId);
44 break;42 break;
45- case HccLTaskMetaType::REDUCE: 43+ case HccLTaskMetaType::REDUCE:
46- NOTIFY_STUB_INFO("pid[{}]: rankId[{}], streamId[{}], taskType[REDUCE], srcOffset[{}], dstOffset[{}], len[{}], srcRankId[{}], dstRankId[{}], reduceOp[{}], dataType[{}]\n",44+ HCCL_VM_INFO("pid[{}]: rankId[{}], streamId[{}], taskType[REDUCE], srcOffset[{}], dstOffset[{}], len[{}], srcRankId[{}], dstRankId[{}], reduceOp[{}], dataType[{}]",
47 pid, taskMeta.rankId, taskMeta.streamId, taskMeta.taskData.reduce.srcOffset, taskMeta.taskData.reduce.dstOffset, taskMeta.taskData.reduce.dataCount,45 pid, taskMeta.rankId, taskMeta.streamId, taskMeta.taskData.reduce.srcOffset, taskMeta.taskData.reduce.dstOffset, taskMeta.taskData.reduce.dataCount,
48 taskMeta.taskData.reduce.srcRankId, taskMeta.taskData.reduce.dstRankId, taskMeta.taskData.reduce.reduceOp, taskMeta.taskData.reduce.dataType);46 taskMeta.taskData.reduce.srcRankId, taskMeta.taskData.reduce.dstRankId, taskMeta.taskData.reduce.reduceOp, taskMeta.taskData.reduce.dataType);
49 break;47 break;
50 case HccLTaskMetaType::NOTIFY_WAIT:48 case HccLTaskMetaType::NOTIFY_WAIT:
51- NOTIFY_STUB_INFO("pid[{}]: rankId[{}], streamId[{}], taskType[NOTIFY_WAIT], notifyId[{}], srcRankId[{}], dstRankId[{}]\n",49+ HCCL_VM_INFO("pid[{}]: rankId[{}], streamId[{}], taskType[NOTIFY_WAIT], notifyId[{}], srcRankId[{}], dstRankId[{}]",
52 pid, taskMeta.rankId, taskMeta.streamId, taskMeta.taskData.notify.notifyId, taskMeta.taskData.notify.srcRankId, taskMeta.taskData.notify.dstRankId);50 pid, taskMeta.rankId, taskMeta.streamId, taskMeta.taskData.notify.notifyId, taskMeta.taskData.notify.srcRankId, taskMeta.taskData.notify.dstRankId);
53 break;51 break;
54 case HccLTaskMetaType::NOTIFY_RECORD:52 case HccLTaskMetaType::NOTIFY_RECORD:
55- NOTIFY_STUB_INFO("pid[{}]: rankId[{}], streamId[{}], taskType[NOTIFY_RECORD], notifyId[{}], srcRankId[{}], dstRankId[{}]\n",53+ HCCL_VM_INFO("pid[{}]: rankId[{}], streamId[{}], taskType[NOTIFY_RECORD], notifyId[{}], srcRankId[{}], dstRankId[{}]",
56 pid, taskMeta.rankId, taskMeta.streamId, taskMeta.taskData.notify.notifyId, taskMeta.taskData.notify.srcRankId, taskMeta.taskData.notify.dstRankId);54 pid, taskMeta.rankId, taskMeta.streamId, taskMeta.taskData.notify.notifyId, taskMeta.taskData.notify.srcRankId, taskMeta.taskData.notify.dstRankId);
57 break;55 break;
58 default:56 default:
@@ -68,7 +66,7 @@ aclError aclrtCreateNotify(aclrtNotify *notify, uint64_t flag)
68 return ACL_ERROR_INVALID_PARAM;66 return ACL_ERROR_INVALID_PARAM;
69 }67 }
70 if (runner.current_ctx_id == 0) {68 if (runner.current_ctx_id == 0) {
71- HCCL_VM_ERROR("[aclrtCreateNotify] invalid param");69+ HCCL_VM_ERROR("invalid param");
72 return ACL_ERROR_INVALID_PARAM;70 return ACL_ERROR_INVALID_PARAM;
73 }71 }
74 auto currCtx = RunnerDB::GetById<sim::Context>(runner.current_ctx_id);72 auto currCtx = RunnerDB::GetById<sim::Context>(runner.current_ctx_id);
@@ -84,14 +82,14 @@ aclError aclrtCreateNotify(aclrtNotify *notify, uint64_t flag)
84 auto res = RunnerDB::Add<sim::Notify>(tmp);82 auto res = RunnerDB::Add<sim::Notify>(tmp);
85 83 
86 *notify = (aclrtNotify)res;84 *notify = (aclrtNotify)res;
87- HCCL_VM_DEBUG("[aclstub][aclrtCreateNotify] notify: {:d}", res);85+ HCCL_VM_DEBUG("notify: {:d}", res);
88 return ACL_SUCCESS;86 return ACL_SUCCESS;
89}87}
90 88 
91aclError aclrtDestroyNotify(aclrtNotify notify)89aclError aclrtDestroyNotify(aclrtNotify notify)
92{90{
93 uint64_t notifyId = (uint32_t)(uintptr_t)notify;91 uint64_t notifyId = (uint32_t)(uintptr_t)notify;
94- HCCL_VM_DEBUG("[aclstub][aclrtDestroyNotify] notify: {:d}", notifyId);92+ HCCL_VM_DEBUG("notify: {:d}", notifyId);
95 RunnerDB::Delete<sim::Notify>(notifyId);93 RunnerDB::Delete<sim::Notify>(notifyId);
96 return ACL_SUCCESS;94 return ACL_SUCCESS;
97}95}
@@ -99,7 +97,7 @@ aclError aclrtDestroyNotify(aclrtNotify notify)
99aclError aclrtGetNotifyId(aclrtNotify notify, uint32_t *notifyId)97aclError aclrtGetNotifyId(aclrtNotify notify, uint32_t *notifyId)
100{98{
101 *notifyId = (uint32_t)(uintptr_t)notify;99 *notifyId = (uint32_t)(uintptr_t)notify;
102- HCCL_VM_DEBUG("[aclstub][aclrtGetNotifyId] notifyId: {:d}", *notifyId);100+ HCCL_VM_DEBUG("notifyId: {:d}", *notifyId);
103 return ACL_SUCCESS;101 return ACL_SUCCESS;
104}102}
105 103 
@@ -122,10 +120,10 @@ aclError aclrtRecordNotify(aclrtNotify notify, aclrtStream stream)
122 PrintTaskMetaData(taskMetaData);120 PrintTaskMetaData(taskMetaData);
123 121 
124 uint32_t index{0};122 uint32_t index{0};
125- HCCL_VM_DEBUG("[aclstub][aclrtRecordNotify] Get notify task, id={:d}, streamId={:d}", notifyId, streamId);123+ HCCL_VM_DEBUG("Get notify task, id={:d}, streamId={:d}", notifyId, streamId);
126 auto ret = InsertTaskToCollection(&taskMetaData, &index);124 auto ret = InsertTaskToCollection(&taskMetaData, &index);
127 if (ret != HcclSim::HcclVmResult::HCCL_SIM_SUCCESS) {125 if (ret != HcclSim::HcclVmResult::HCCL_SIM_SUCCESS) {
128- HCCL_VM_ERROR("[aclstub] InsertTaskToCollection fail");126+ HCCL_VM_ERROR("InsertTaskToCollection fail");
129 return ACL_ERROR_INTERNAL_ERROR;127 return ACL_ERROR_INTERNAL_ERROR;
130 }128 }
131 129 
@@ -159,10 +157,10 @@ aclError aclrtWaitAndResetNotify(aclrtNotify notify, aclrtStream stream, uint32_
159 PrintTaskMetaData(taskMetaData);157 PrintTaskMetaData(taskMetaData);
160 158 
161 uint32_t index{0};159 uint32_t index{0};
162- HCCL_VM_DEBUG("[aclstub][aclrtWaitAndResetNotify] Get notify task, id={:d}, streamId={:d}", notifyId, streamId);160+ HCCL_VM_DEBUG("Get notify task, id={:d}, streamId={:d}", notifyId, streamId);
163 auto ret = InsertTaskToCollection(&taskMetaData, &index);161 auto ret = InsertTaskToCollection(&taskMetaData, &index);
164 if (ret != HcclSim::HcclVmResult::HCCL_SIM_SUCCESS) {162 if (ret != HcclSim::HcclVmResult::HCCL_SIM_SUCCESS) {
165- HCCL_VM_ERROR("[aclstub] InsertTaskToCollection fail");163+ HCCL_VM_ERROR("InsertTaskToCollection fail");
166 return ACL_ERROR_INTERNAL_ERROR;164 return ACL_ERROR_INTERNAL_ERROR;
167 }165 }
168 166 
@@ -199,14 +197,14 @@ aclError aclrtNotifyGetExportKey(aclrtNotify notify, char *key, size_t len, uint
199 memcpy(key, notifyKeyStr.data(), notifyKeyStr.length());197 memcpy(key, notifyKeyStr.data(), notifyKeyStr.length());
200 tmp.create_pid = getpid();198 tmp.create_pid = getpid();
201 auto res = RunnerDB::Add<sim::IpcNotify>(tmp);199 auto res = RunnerDB::Add<sim::IpcNotify>(tmp);
202- HCCL_VM_DEBUG("[aclstub][aclrtNotifyGetExportKey] notify: {:d}", notifyId);200+ HCCL_VM_DEBUG("notify: {:d}", notifyId);
203 return ACL_SUCCESS;201 return ACL_SUCCESS;
204}202}
205 203 
206aclError aclrtNotifySetImportPid(aclrtNotify notify, int32_t *pid, size_t num)204aclError aclrtNotifySetImportPid(aclrtNotify notify, int32_t *pid, size_t num)
207{205{
208 uint64_t notifyId = (uint32_t)(uintptr_t)notify;206 uint64_t notifyId = (uint32_t)(uintptr_t)notify;
209- HCCL_VM_DEBUG("[aclstub][aclrtNotifySetImportPid] notify: {:d}", notifyId);207+ HCCL_VM_DEBUG("notify: {:d}", notifyId);
210 208 
211 auto ipcNotify = RunnerDB::GetOneByPred<sim::IpcNotify>([notifyId](const sim::IpcNotify& ipc) {209 auto ipcNotify = RunnerDB::GetOneByPred<sim::IpcNotify>([notifyId](const sim::IpcNotify& ipc) {
212 return ipc.notify_id == notifyId;210 return ipc.notify_id == notifyId;
@@ -237,7 +235,7 @@ aclError aclrtNotifyImportByKey(aclrtNotify *notify, const char *key, uint64_t f
237 return ACL_ERROR_INVALID_PARAM;235 return ACL_ERROR_INVALID_PARAM;
238 }236 }
239 *notify = (aclrtNotify)ipcNotify.first.notify_id;237 *notify = (aclrtNotify)ipcNotify.first.notify_id;
240- HCCL_VM_DEBUG("[aclstub][aclrtNotifyImportByKey] notify: {:d}", ipcNotify.first.notify_id);238+ HCCL_VM_DEBUG("notify: {:d}", ipcNotify.first.notify_id);
241 return ACL_SUCCESS;239 return ACL_SUCCESS;
242}240}
243 241 
Mtest/hccl_vm/src/proxy/aclrt_runtime_config.cc+13-15
@@ -8,6 +8,9 @@
8 * See LICENSE in the root of the software repository for the full text of the License.8 * See LICENSE in the root of the software repository for the full text of the License.
9 */9 */
10 10 
11+// 日志染色: 模块 tag (须在 include sim_log.h 之前)
12+#define HCCL_VM_MODULE "CONFIG_STUB"
13+ 
11#include <atomic>14#include <atomic>
12#include <cstdint>15#include <cstdint>
13#include <iostream>16#include <iostream>
@@ -22,11 +25,6 @@
22#include "db_sim_runner_common.h"25#include "db_sim_runner_common.h"
23#include "db_sim_runner_ops.h"26#include "db_sim_runner_ops.h"
24 27 
25-#define CONFIG_STUB_ERROR(format, ...) HCCL_VM_ERROR("[CONFIG_STUB]" format, ##__VA_ARGS__)
26-#define CONFIG_STUB_DEBUG(format, ...) HCCL_VM_DEBUG("[CONFIG_STUB]" format, ##__VA_ARGS__)
27-#define CONFIG_STUB_INFO(format, ...) HCCL_VM_INFO("[CONFIG_STUB]" format, ##__VA_ARGS__)
28-#define CONFIG_STUB_WARN(format, ...) HCCL_VM_WARN("[CONFIG_STUB]" format, ##__VA_ARGS__)
29-#define CONFIG_STUB_TRACE(format, ...) HCCL_VM_TRACE("[CONFIG_STUB]" format, ##__VA_ARGS__)
30 28 
31#ifdef __cplusplus29#ifdef __cplusplus
32extern "C" {30extern "C" {
@@ -36,7 +34,7 @@ aclError aclrtSetSysParamOpt(aclSysParamOpt opt, int64_t value)
36{34{
37 (void) opt;35 (void) opt;
38 (void) value;36 (void) value;
39- CONFIG_STUB_WARN("not support");37+ HCCL_VM_WARN("not support");
40 return ACL_SUCCESS;38 return ACL_SUCCESS;
41}39}
42 40 
@@ -44,7 +42,7 @@ aclError aclrtGetSysParamOpt(aclSysParamOpt opt, int64_t *value)
44{42{
45 (void) opt;43 (void) opt;
46 (void) value;44 (void) value;
47- CONFIG_STUB_WARN("not support");45+ HCCL_VM_WARN("not support");
48 return ACL_SUCCESS;46 return ACL_SUCCESS;
49}47}
50 48 
@@ -53,7 +51,7 @@ aclError aclrtGetDeviceResLimit(int32_t deviceId, aclrtDevResLimitType type, uin
53 (void) deviceId;51 (void) deviceId;
54 (void) type;52 (void) type;
55 (void) value;53 (void) value;
56- CONFIG_STUB_WARN("not support");54+ HCCL_VM_WARN("not support");
57 return ACL_SUCCESS;55 return ACL_SUCCESS;
58}56}
59 57 
@@ -62,14 +60,14 @@ aclError aclrtSetDeviceResLimit(int32_t deviceId, aclrtDevResLimitType type, uin
62 (void) deviceId;60 (void) deviceId;
63 (void) type;61 (void) type;
64 (void) value;62 (void) value;
65- CONFIG_STUB_WARN("not support");63+ HCCL_VM_WARN("not support");
66 return ACL_SUCCESS;64 return ACL_SUCCESS;
67}65}
68 66 
69aclError aclrtResetDeviceResLimit(int32_t deviceId)67aclError aclrtResetDeviceResLimit(int32_t deviceId)
70{68{
71 (void) deviceId;69 (void) deviceId;
72- CONFIG_STUB_WARN("not support");70+ HCCL_VM_WARN("not support");
73 return ACL_SUCCESS;71 return ACL_SUCCESS;
74}72}
75 73 
@@ -78,7 +76,7 @@ aclError aclrtGetStreamResLimit(aclrtStream stream, aclrtDevResLimitType type, u
78 (void) stream;76 (void) stream;
79 (void) type;77 (void) type;
80 (void) value;78 (void) value;
81- CONFIG_STUB_WARN("not support");79+ HCCL_VM_WARN("not support");
82 return ACL_SUCCESS;80 return ACL_SUCCESS;
83}81}
84 82 
@@ -87,28 +85,28 @@ aclError aclrtSetStreamResLimit(aclrtStream stream, aclrtDevResLimitType type, u
87 (void) stream;85 (void) stream;
88 (void) type;86 (void) type;
89 (void) value;87 (void) value;
90- CONFIG_STUB_WARN("not support");88+ HCCL_VM_WARN("not support");
91 return ACL_SUCCESS;89 return ACL_SUCCESS;
92}90}
93 91 
94aclError aclrtResetStreamResLimit(aclrtStream stream)92aclError aclrtResetStreamResLimit(aclrtStream stream)
95{93{
96 (void) stream;94 (void) stream;
97- CONFIG_STUB_WARN("not support");95+ HCCL_VM_WARN("not support");
98 return ACL_SUCCESS;96 return ACL_SUCCESS;
99}97}
100 98 
101aclError aclrtUseStreamResInCurrentThread(aclrtStream stream)99aclError aclrtUseStreamResInCurrentThread(aclrtStream stream)
102{100{
103 (void) stream;101 (void) stream;
104- CONFIG_STUB_WARN("not support");102+ HCCL_VM_WARN("not support");
105 return ACL_SUCCESS;103 return ACL_SUCCESS;
106}104}
107 105 
108aclError aclrtUnuseStreamResInCurrentThread(aclrtStream stream)106aclError aclrtUnuseStreamResInCurrentThread(aclrtStream stream)
109{107{
110 (void) stream;108 (void) stream;
111- CONFIG_STUB_WARN("not support");109+ HCCL_VM_WARN("not support");
112 return ACL_SUCCESS;110 return ACL_SUCCESS;
113}111}
114 112 
Mtest/hccl_vm/src/proxy/aclrt_stream_stub.cc+31-33
@@ -8,6 +8,9 @@
8 * See LICENSE in the root of the software repository for the full text of the License.8 * See LICENSE in the root of the software repository for the full text of the License.
9 */9 */
10 10 
11+// 日志染色: 模块 tag (须在 include sim_log.h 之前)
12+#define HCCL_VM_MODULE "STREAM_STUB"
13+ 
11#include <atomic>14#include <atomic>
12#include <cstdint>15#include <cstdint>
13#include <ctime>16#include <ctime>
@@ -28,11 +31,6 @@
28 31 
29static const uint32_t WAIT_FALG_TIMEOUT = 600;32static const uint32_t WAIT_FALG_TIMEOUT = 600;
30 33 
31-#define STREAM_STUB_ERROR(format, ...) HCCL_VM_ERROR("[STREAM_STUB]" format, ##__VA_ARGS__)
32-#define STREAM_STUB_DEBUG(format, ...) HCCL_VM_DEBUG("[STREAM_STUB]" format, ##__VA_ARGS__)
33-#define STREAM_STUB_INFO(format, ...) HCCL_VM_INFO("[STREAM_STUB]" format, ##__VA_ARGS__)
34-#define STREAM_STUB_WARN(format, ...) HCCL_VM_WARN("[STREAM_STUB]" format, ##__VA_ARGS__)
35-#define STREAM_STUB_TRACE(format, ...) HCCL_VM_TRACE("[STREAM_STUB]" format, ##__VA_ARGS__)
36 34 
37#ifdef __cplusplus35#ifdef __cplusplus
38extern "C" {36extern "C" {
@@ -47,12 +45,12 @@ aclError aclrtCreateStreamWithConfig(aclrtStream *stream, uint32_t priority, uin
47 return ACL_ERROR_INVALID_PARAM;45 return ACL_ERROR_INVALID_PARAM;
48 }46 }
49 if (runner.current_ctx_id == 0) {47 if (runner.current_ctx_id == 0) {
50- STREAM_STUB_ERROR("invalid param");48+ HCCL_VM_ERROR("invalid param");
51 return ACL_ERROR_INVALID_PARAM;49 return ACL_ERROR_INVALID_PARAM;
52 }50 }
53 auto currCtx = RunnerDB::GetById<sim::Context>(runner.current_ctx_id);51 auto currCtx = RunnerDB::GetById<sim::Context>(runner.current_ctx_id);
54 if (!currCtx.has_value()) {52 if (!currCtx.has_value()) {
55- STREAM_STUB_ERROR("ctx not found:{:d}", runner.current_ctx_id);53+ HCCL_VM_ERROR("ctx not found:{:d}", runner.current_ctx_id);
56 return ACL_ERROR_INVALID_PARAM;54 return ACL_ERROR_INVALID_PARAM;
57 }55 }
58 56 
@@ -66,7 +64,7 @@ aclError aclrtCreateStreamWithConfig(aclrtStream *stream, uint32_t priority, uin
66 auto res = RunnerDB::Add<sim::Stream>(streamTmp);64 auto res = RunnerDB::Add<sim::Stream>(streamTmp);
67 65 
68 *stream = (aclrtStream)res;66 *stream = (aclrtStream)res;
69- STREAM_STUB_DEBUG("id:{:d}", res);67+ HCCL_VM_DEBUG("id:{:d}", res);
70 sim::SetLastStreamIdTls(res);68 sim::SetLastStreamIdTls(res);
71 return ACL_SUCCESS;69 return ACL_SUCCESS;
72}70}
@@ -79,7 +77,7 @@ aclError aclrtCreateStream(aclrtStream *stream)
79aclError aclrtDestroyStream(aclrtStream stream)77aclError aclrtDestroyStream(aclrtStream stream)
80{78{
81 uint64_t streamId = (uint64_t)(uintptr_t)stream;79 uint64_t streamId = (uint64_t)(uintptr_t)stream;
82- STREAM_STUB_DEBUG("id:{:d}", streamId);80+ HCCL_VM_DEBUG("id:{:d}", streamId);
83 RunnerDB::Delete<sim::Stream>(streamId);81 RunnerDB::Delete<sim::Stream>(streamId);
84 return ACL_SUCCESS;82 return ACL_SUCCESS;
85}83}
@@ -93,10 +91,10 @@ aclError aclrtActiveStream(aclrtStream activeStream, aclrtStream stream)
93{91{
94 (void) stream;92 (void) stream;
95 uint64_t activStreamId = (uint64_t)(uintptr_t)activeStream;93 uint64_t activStreamId = (uint64_t)(uintptr_t)activeStream;
96- STREAM_STUB_DEBUG("id:{:d}", activStreamId);94+ HCCL_VM_DEBUG("id:{:d}", activStreamId);
97 auto res = RunnerDB::Update<sim::Stream>(activStreamId, [](sim::Stream &stm) { stm.activated = 1; });95 auto res = RunnerDB::Update<sim::Stream>(activStreamId, [](sim::Stream &stm) { stm.activated = 1; });
98 if (!res) {96 if (!res) {
99- STREAM_STUB_ERROR("stream not found:{:d}", activStreamId);97+ HCCL_VM_ERROR("stream not found:{:d}", activStreamId);
100 return ACL_ERROR_INVALID_PARAM;98 return ACL_ERROR_INVALID_PARAM;
101 }99 }
102 return ACL_SUCCESS;100 return ACL_SUCCESS;
@@ -105,7 +103,7 @@ aclError aclrtActiveStream(aclrtStream activeStream, aclrtStream stream)
105aclError aclrtSetStreamFailureMode(aclrtStream stream, uint64_t mode)103aclError aclrtSetStreamFailureMode(aclrtStream stream, uint64_t mode)
106{104{
107 uint64_t streamId = (uint64_t)(uintptr_t)stream;105 uint64_t streamId = (uint64_t)(uintptr_t)stream;
108- STREAM_STUB_DEBUG("id:{:d} mode:{:d}", streamId, mode);106+ HCCL_VM_DEBUG("id:{:d} mode:{:d}", streamId, mode);
109 RunnerDB::Update<sim::Stream>(streamId, [streamId, mode](sim::Stream &stm) { stm.failure_mode = mode; });107 RunnerDB::Update<sim::Stream>(streamId, [streamId, mode](sim::Stream &stm) { stm.failure_mode = mode; });
110 108 
111 return ACL_SUCCESS;109 return ACL_SUCCESS;
@@ -118,7 +116,7 @@ aclError aclrtSynchronizeStreamWithTimeout(aclrtStream stream, int32_t timeout)
118 // GetMode();116 // GetMode();
119 std::string mode = "checker";117 std::string mode = "checker";
120 const int WAIT_COUNTDOWN = 10; // 等待20s118 const int WAIT_COUNTDOWN = 10; // 等待20s
121- STREAM_STUB_DEBUG("id:{:d}", streamId);119+ HCCL_VM_DEBUG("id:{:d}", streamId);
122 return ACL_SUCCESS;120 return ACL_SUCCESS;
123}121}
124 122 
@@ -157,21 +155,21 @@ aclError aclrtSynchronizeStream(aclrtStream stream)
157 syncRecord.streamId = (uint64_t)(uintptr_t)stream;155 syncRecord.streamId = (uint64_t)(uintptr_t)stream;
158 syncRecord.status = 0;156 syncRecord.status = 0;
159 if (sim::InsertSyncRecord(syncRecord) != 0) {157 if (sim::InsertSyncRecord(syncRecord) != 0) {
160- STREAM_STUB_ERROR("insert sync record fail");158+ HCCL_VM_ERROR("insert sync record fail");
161 return ACL_ERROR_INTERNAL_ERROR;159 return ACL_ERROR_INTERNAL_ERROR;
162 }160 }
163 161 
164 // 4. 阻塞等待本轮同步完成(runner 模式等 runner;no-runner 模式等 host barrier-only)162 // 4. 阻塞等待本轮同步完成(runner 模式等 runner;no-runner 模式等 host barrier-only)
165 bool done = syncer.tryBlockProxyUntilRunnerDone(targetRound, WAIT_FALG_TIMEOUT * 1000);163 bool done = syncer.tryBlockProxyUntilRunnerDone(targetRound, WAIT_FALG_TIMEOUT * 1000);
166 if (!done) {164 if (!done) {
167- STREAM_STUB_ERROR("sync timeout round:{}", targetRound);165+ HCCL_VM_ERROR("sync timeout round:{}", targetRound);
168 return ACL_ERROR_INTERNAL_ERROR;166 return ACL_ERROR_INTERNAL_ERROR;
169 }167 }
170 168 
171 syncRecord.status = 1;169 syncRecord.status = 1;
172 std::vector<sim::SyncRecordTab> syncRecords = {syncRecord};170 std::vector<sim::SyncRecordTab> syncRecords = {syncRecord};
173 if (sim::UpdateSyncRecordStatus(syncRecords) != 0) {171 if (sim::UpdateSyncRecordStatus(syncRecords) != 0) {
174- STREAM_STUB_ERROR("update sync record fail");172+ HCCL_VM_ERROR("update sync record fail");
175 return ACL_ERROR_INTERNAL_ERROR;173 return ACL_ERROR_INTERNAL_ERROR;
176 }174 }
177 175 
@@ -179,7 +177,7 @@ aclError aclrtSynchronizeStream(aclrtStream stream)
179 clock_gettime(CLOCK_MONOTONIC, &tsEnd);177 clock_gettime(CLOCK_MONOTONIC, &tsEnd);
180 double elapsedMs = (tsEnd.tv_sec - tsStart.tv_sec) * 1000.0178 double elapsedMs = (tsEnd.tv_sec - tsStart.tv_sec) * 1000.0
181 + (tsEnd.tv_nsec - tsStart.tv_nsec) / 1e6;179 + (tsEnd.tv_nsec - tsStart.tv_nsec) / 1e6;
182- STREAM_STUB_INFO("sync round:{:d} done {:.3f}ms", targetRound, elapsedMs);180+ HCCL_VM_INFO("sync round:{:d} done {:.3f}ms", targetRound, elapsedMs);
183 181 
184 return ACL_SUCCESS;182 return ACL_SUCCESS;
185}183}
@@ -187,7 +185,7 @@ aclError aclrtSynchronizeStream(aclrtStream stream)
187aclError aclrtStreamAbort(aclrtStream stream)185aclError aclrtStreamAbort(aclrtStream stream)
188{186{
189 uint64_t streamId = (uint64_t)(uintptr_t)stream;187 uint64_t streamId = (uint64_t)(uintptr_t)stream;
190- STREAM_STUB_DEBUG("id:{:d}", streamId);188+ HCCL_VM_DEBUG("id:{:d}", streamId);
191 RunnerDB::Update<sim::Stream>(streamId, [streamId](sim::Stream &stm) { stm.activated = 0; });189 RunnerDB::Update<sim::Stream>(streamId, [streamId](sim::Stream &stm) { stm.activated = 0; });
192 190 
193 return ACL_SUCCESS;191 return ACL_SUCCESS;
@@ -196,10 +194,10 @@ aclError aclrtStreamAbort(aclrtStream stream)
196aclError aclrtStreamQuery(aclrtStream stream, aclrtStreamStatus *status)194aclError aclrtStreamQuery(aclrtStream stream, aclrtStreamStatus *status)
197{195{
198 uint64_t streamId = (uint64_t)(uintptr_t)stream;196 uint64_t streamId = (uint64_t)(uintptr_t)stream;
199- STREAM_STUB_DEBUG("id:{:d}", streamId);197+ HCCL_VM_DEBUG("id:{:d}", streamId);
200 auto res = RunnerDB::GetById<sim::Stream>(streamId);198 auto res = RunnerDB::GetById<sim::Stream>(streamId);
201 if (!res.has_value()) {199 if (!res.has_value()) {
202- STREAM_STUB_ERROR("stream not found:{:d}", streamId);200+ HCCL_VM_ERROR("stream not found:{:d}", streamId);
203 return ACL_ERROR_INVALID_PARAM;201 return ACL_ERROR_INVALID_PARAM;
204 }202 }
205 *status = (aclrtStreamStatus)res->task_complete_status;203 *status = (aclrtStreamStatus)res->task_complete_status;
@@ -214,12 +212,12 @@ aclError aclrtGetStreamAvailableNum(uint32_t *streamCount)
214 return ACL_ERROR_INVALID_PARAM;212 return ACL_ERROR_INVALID_PARAM;
215 }213 }
216 if (runner.current_ctx_id == 0) {214 if (runner.current_ctx_id == 0) {
217- STREAM_STUB_ERROR("invalid param");215+ HCCL_VM_ERROR("invalid param");
218 return ACL_ERROR_INVALID_PARAM;216 return ACL_ERROR_INVALID_PARAM;
219 }217 }
220 auto currCtx = RunnerDB::GetById<sim::Context>(runner.current_ctx_id);218 auto currCtx = RunnerDB::GetById<sim::Context>(runner.current_ctx_id);
221 if (!currCtx.has_value()) {219 if (!currCtx.has_value()) {
222- STREAM_STUB_ERROR("ctx not found:{:d}", runner.current_ctx_id);220+ HCCL_VM_ERROR("ctx not found:{:d}", runner.current_ctx_id);
223 return ACL_ERROR_INVALID_PARAM;221 return ACL_ERROR_INVALID_PARAM;
224 }222 }
225 223 
@@ -227,7 +225,7 @@ aclError aclrtGetStreamAvailableNum(uint32_t *streamCount)
227 225 
228 auto device = RunnerDB::GetById<sim::Device>(devId);226 auto device = RunnerDB::GetById<sim::Device>(devId);
229 if (!device.has_value()) {227 if (!device.has_value()) {
230- STREAM_STUB_ERROR("device not found:{:d}", devId);228+ HCCL_VM_ERROR("device not found:{:d}", devId);
231 return ACL_ERROR_INVALID_PARAM;229 return ACL_ERROR_INVALID_PARAM;
232 }230 }
233 231 
@@ -254,14 +252,14 @@ aclError aclrtGetStreamAvailableNum(uint32_t *streamCount)
254aclError aclrtStreamGetId(aclrtStream stream, int32_t *streamId)252aclError aclrtStreamGetId(aclrtStream stream, int32_t *streamId)
255{253{
256 *streamId = (uint32_t)(uintptr_t)stream;254 *streamId = (uint32_t)(uintptr_t)stream;
257- STREAM_STUB_DEBUG("id:{:d}", *streamId);255+ HCCL_VM_DEBUG("id:{:d}", *streamId);
258 return ACL_SUCCESS;256 return ACL_SUCCESS;
259}257}
260 258 
261aclError aclrtSetStreamOverflowSwitch(aclrtStream stream, uint32_t flag)259aclError aclrtSetStreamOverflowSwitch(aclrtStream stream, uint32_t flag)
262{260{
263 uint64_t streamId = (uint64_t)(uintptr_t)stream;261 uint64_t streamId = (uint64_t)(uintptr_t)stream;
264- STREAM_STUB_DEBUG("id:{:d} flag:{:d}", streamId, flag);262+ HCCL_VM_DEBUG("id:{:d} flag:{:d}", streamId, flag);
265 RunnerDB::Update<sim::Stream>(streamId, [streamId, flag](sim::Stream &stm) { stm.overflow_switch = flag; });263 RunnerDB::Update<sim::Stream>(streamId, [streamId, flag](sim::Stream &stm) { stm.overflow_switch = flag; });
266 return ACL_SUCCESS;264 return ACL_SUCCESS;
267}265}
@@ -269,10 +267,10 @@ aclError aclrtSetStreamOverflowSwitch(aclrtStream stream, uint32_t flag)
269aclError aclrtGetStreamOverflowSwitch(aclrtStream stream, uint32_t *flag)267aclError aclrtGetStreamOverflowSwitch(aclrtStream stream, uint32_t *flag)
270{268{
271 uint64_t streamId = (uint64_t)(uintptr_t)stream;269 uint64_t streamId = (uint64_t)(uintptr_t)stream;
272- STREAM_STUB_DEBUG("id:{:d}", streamId);270+ HCCL_VM_DEBUG("id:{:d}", streamId);
273 auto res = RunnerDB::GetById<sim::Stream>(streamId);271 auto res = RunnerDB::GetById<sim::Stream>(streamId);
274 if (!res.has_value()) {272 if (!res.has_value()) {
275- STREAM_STUB_ERROR("stream not found:{:d}", streamId);273+ HCCL_VM_ERROR("stream not found:{:d}", streamId);
276 return ACL_ERROR_INVALID_PARAM;274 return ACL_ERROR_INVALID_PARAM;
277 }275 }
278 *flag = res->overflow_switch;276 *flag = res->overflow_switch;
@@ -282,10 +280,10 @@ aclError aclrtGetStreamOverflowSwitch(aclrtStream stream, uint32_t *flag)
282aclError aclrtSetStreamAttribute(aclrtStream stream, aclrtStreamAttr stmAttrType, aclrtStreamAttrValue *value)280aclError aclrtSetStreamAttribute(aclrtStream stream, aclrtStreamAttr stmAttrType, aclrtStreamAttrValue *value)
283{281{
284 uint64_t streamId = (uint64_t)(uintptr_t)stream;282 uint64_t streamId = (uint64_t)(uintptr_t)stream;
285- STREAM_STUB_DEBUG("id:{:d}", streamId);283+ HCCL_VM_DEBUG("id:{:d}", streamId);
286 auto res = RunnerDB::GetById<sim::Stream>(streamId);284 auto res = RunnerDB::GetById<sim::Stream>(streamId);
287 if (!res.has_value()) {285 if (!res.has_value()) {
288- STREAM_STUB_ERROR("stream not found:{:d}", streamId);286+ HCCL_VM_ERROR("stream not found:{:d}", streamId);
289 return ACL_ERROR_INVALID_PARAM;287 return ACL_ERROR_INVALID_PARAM;
290 }288 }
291 289 
@@ -306,10 +304,10 @@ aclError aclrtSetStreamAttribute(aclrtStream stream, aclrtStreamAttr stmAttrType
306aclError aclrtGetStreamAttribute(aclrtStream stream, aclrtStreamAttr stmAttrType, aclrtStreamAttrValue *value)304aclError aclrtGetStreamAttribute(aclrtStream stream, aclrtStreamAttr stmAttrType, aclrtStreamAttrValue *value)
307{305{
308 uint64_t streamId = (uint64_t)(uintptr_t)stream;306 uint64_t streamId = (uint64_t)(uintptr_t)stream;
309- STREAM_STUB_DEBUG("id:{:d}", streamId);307+ HCCL_VM_DEBUG("id:{:d}", streamId);
310 auto res = RunnerDB::GetById<sim::Stream>(streamId);308 auto res = RunnerDB::GetById<sim::Stream>(streamId);
311 if (!res.has_value()) {309 if (!res.has_value()) {
312- STREAM_STUB_ERROR("stream not found:{:d}", streamId);310+ HCCL_VM_ERROR("stream not found:{:d}", streamId);
313 return ACL_ERROR_INVALID_PARAM;311 return ACL_ERROR_INVALID_PARAM;
314 }312 }
315 313 
@@ -332,7 +330,7 @@ aclError aclrtStreamStop(aclrtStream stream)
332rtError_t rtStreamSynchronize(rtStream_t stream)330rtError_t rtStreamSynchronize(rtStream_t stream)
333{331{
334 uint64_t streamId = (uint64_t)(uintptr_t)stream;332 uint64_t streamId = (uint64_t)(uintptr_t)stream;
335- STREAM_STUB_DEBUG("id:{:d}", streamId);333+ HCCL_VM_DEBUG("id:{:d}", streamId);
336 return aclrtSynchronizeStream((aclrtStream)stream);334 return aclrtSynchronizeStream((aclrtStream)stream);
337}335}
338 336 
@@ -345,7 +343,7 @@ rtError_t rtStreamGetSqid(const rtStream_t stream, uint32_t *sqId)
345{343{
346 uint64_t streamId = (uint64_t)(uintptr_t)stream;344 uint64_t streamId = (uint64_t)(uintptr_t)stream;
347 *sqId = streamId;345 *sqId = streamId;
348- STREAM_STUB_DEBUG("id:{:d}", streamId);346+ HCCL_VM_DEBUG("id:{:d}", streamId);
349 return ACL_SUCCESS;347 return ACL_SUCCESS;
350}348}
351 349 
Mtest/hccl_vm/src/proxy/aclrt_stub.cc+4-6
@@ -8,6 +8,9 @@
8 * See LICENSE in the root of the software repository for the full text of the License.8 * See LICENSE in the root of the software repository for the full text of the License.
9 */9 */
10 10 
11+// 日志染色: 模块 tag (须在 include sim_log.h 之前)
12+#define HCCL_VM_MODULE "ACLRT_STUB"
13+ 
11#include <atomic>14#include <atomic>
12#include <cstdint>15#include <cstdint>
13#include <iostream>16#include <iostream>
@@ -19,11 +22,6 @@
19#include "runtime/base.h"22#include "runtime/base.h"
20#include "db_sim_runner_ops.h"23#include "db_sim_runner_ops.h"
21 24 
22-#define ACLRT_STUB_ERROR(format, ...) HCCL_VM_ERROR("[ACLRT_STUB]" format, ##__VA_ARGS__)
23-#define ACLRT_STUB_DEBUG(format, ...) HCCL_VM_DEBUG("[ACLRT_STUB]" format, ##__VA_ARGS__)
24-#define ACLRT_STUB_INFO(format, ...) HCCL_VM_INFO("[ACLRT_STUB]" format, ##__VA_ARGS__)
25-#define ACLRT_STUB_WARN(format, ...) HCCL_VM_WARN("[ACLRT_STUB]" format, ##__VA_ARGS__)
26-#define ACLRT_STUB_TRACE(format, ...) HCCL_VM_TRACE("[ACLRT_STUB]" format, ##__VA_ARGS__)
27 25 
28#ifdef __cplusplus26#ifdef __cplusplus
29extern "C" {27extern "C" {
@@ -33,7 +31,7 @@ aclError aclsysGetVersionStr(char *pkgName, char *versionStr)
33{31{
34 (void) pkgName;32 (void) pkgName;
35 memcpy(versionStr, "9.0.0", sizeof("9.0.0"));33 memcpy(versionStr, "9.0.0", sizeof("9.0.0"));
36- ACLRT_STUB_DEBUG("get version:{}", versionStr);34+ HCCL_VM_DEBUG("get version:{}", versionStr);
37 return ACL_SUCCESS;35 return ACL_SUCCESS;
38}36}
39 37 
Mtest/hccl_vm/src/proxy/adapter_rts_stub.cc+19-16
@@ -8,6 +8,9 @@
8 * See LICENSE in the root of the software repository for the full text of the License.8 * See LICENSE in the root of the software repository for the full text of the License.
9 */9 */
10 10 
11+// 日志染色: 模块 tag (须在 include sim_log.h 之前)
12+#define HCCL_VM_MODULE "RTS_STUB"
13+ 
11#include <atomic>14#include <atomic>
12#include <cstdint>15#include <cstdint>
13#include <cstdio>16#include <cstdio>
@@ -44,14 +47,14 @@ extern "C" {
44int RaIsFirstUsed(int insId)47int RaIsFirstUsed(int insId)
45{48{
46 (void) insId;49 (void) insId;
47- HCCL_VM_INFO("[RTS_STUB] stub is empty");50+ HCCL_VM_WARN("is empty");
48 return 0;51 return 0;
49}52}
50 53 
51int RaIsLastUsed(int insId)54int RaIsLastUsed(int insId)
52{55{
53 (void) insId;56 (void) insId;
54- HCCL_VM_INFO("[RTS_STUB] stub is empty");57+ HCCL_VM_WARN("is empty");
55 return 0;58 return 0;
56}59}
57 60 
@@ -59,8 +62,9 @@ int ibv_get_cq_event_stub(struct ibv_comp_channel *channel, struct ibv_cq **cq,
59{62{
60 (void) cq;63 (void) cq;
61 (void) cq_context;64 (void) cq_context;
62- HCCL_VM_INFO("[RTS_STUB] stub is empty");65+ HCCL_VM_WARN("is empty");
63 if (!channel) {66 if (!channel) {
67+ HCCL_VM_ERROR("channel is null");
64 return -1;68 return -1;
65 }69 }
66 return 0;70 return 0;
@@ -70,7 +74,7 @@ void ibv_ack_cq_events_stub(struct ibv_cq *cq, unsigned int nevents)
70{74{
71 (void) cq;75 (void) cq;
72 (void) nevents;76 (void) nevents;
73- HCCL_VM_INFO("[RTS_STUB] stub is empty");77+ HCCL_VM_WARN("is empty");
74 return;78 return;
75}79}
76 80 
@@ -80,7 +84,7 @@ void ibv_query_qp_stub(struct ibv_qp *qp, struct ibv_qp_attr *attr, int attr_mas
80 (void) attr;84 (void) attr;
81 (void) attr_mask;85 (void) attr_mask;
82 (void) init_attr;86 (void) init_attr;
83- HCCL_VM_INFO("[RTS_STUB] stub is empty");87+ HCCL_VM_WARN("is empty");
84 return;88 return;
85}89}
86 90 
@@ -89,7 +93,7 @@ int32_t UtraceCreateWithAttr(int32_t tracerType, const char *objName, const Trac
89 (void)(tracerType);93 (void)(tracerType);
90 (void)(objName);94 (void)(objName);
91 (void)(attr);95 (void)(attr);
92- HCCL_VM_INFO("[RTS_STUB] stub is empty");96+ HCCL_VM_WARN("is empty");
93 return 0;97 return 0;
94}98}
95 99 
@@ -98,14 +102,14 @@ HcclResult UtraceSubmit(int32_t handle, const void *buffer, uint32_t bufSize)
98 (void)(handle);102 (void)(handle);
99 (void)(buffer);103 (void)(buffer);
100 (void)(bufSize);104 (void)(bufSize);
101- HCCL_VM_INFO("[RTS_STUB] stub is empty");105+ HCCL_VM_WARN("is empty");
102 return HCCL_SUCCESS;106 return HCCL_SUCCESS;
103}107}
104 108 
105int32_t UtraceSetGlobalAttr(const TraceGlobalAttr *attr)109int32_t UtraceSetGlobalAttr(const TraceGlobalAttr *attr)
106{110{
107 (void)(attr);111 (void)(attr);
108- HCCL_VM_INFO("[RTS_STUB] stub is empty");112+ HCCL_VM_WARN("is empty");
109 return 0;113 return 0;
110}114}
111 115 
@@ -113,14 +117,14 @@ int32_t UtraceSave(TracerType tracerType, bool syncFlag)
113{117{
114 (void)(tracerType);118 (void)(tracerType);
115 (void)(syncFlag);119 (void)(syncFlag);
116- HCCL_VM_INFO("[RTS_STUB] stub is empty");120+ HCCL_VM_WARN("is empty");
117 return 0;121 return 0;
118}122}
119 123 
120void UtraceDestroy(int32_t handle)124void UtraceDestroy(int32_t handle)
121{125{
122 (void)(handle);126 (void)(handle);
123- HCCL_VM_INFO("[RTS_STUB] stub is empty");127+ HCCL_VM_WARN("is empty");
124 return;128 return;
125}129}
126 130 
@@ -132,7 +136,7 @@ int ibv_exp_post_send_stub(struct ibv_qp *qp, struct ibv_send_wr *wr, struct ibv
132 (void) bad_wr;136 (void) bad_wr;
133 (void) ext_attr;137 (void) ext_attr;
134 (void) ext_resp;138 (void) ext_resp;
135- HCCL_VM_INFO("[RTS_STUB] stub is empty");139+ HCCL_VM_WARN("is empty");
136 return 0;140 return 0;
137}141}
138 142 
@@ -144,20 +148,20 @@ int ibv_ext_post_send_stub(struct ibv_qp *qp, struct ibv_send_wr *wr, struct ibv
144 (void) bad_wr;148 (void) bad_wr;
145 (void) ext_attr;149 (void) ext_attr;
146 (void) ext_resp;150 (void) ext_resp;
147- HCCL_VM_INFO("[RTS_STUB] stub is empty");151+ HCCL_VM_WARN("is empty");
148 return 0;152 return 0;
149}153}
150 154 
151HcclResult hrtTsdCapabilityGet(uint32_t deviceLogicId, int32_t type, uint64_t ptr)155HcclResult hrtTsdCapabilityGet(uint32_t deviceLogicId, int32_t type, uint64_t ptr)
152{156{
153- HCCL_VM_INFO("[RTS_STUB] deviceLogicId:[{}] type:[{}]], ptr:[{}]\n", deviceLogicId, type, ptr);157+ HCCL_VM_INFO("deviceLogicId:[{}] type:[{}]], ptr:[{}]\n", deviceLogicId, type, ptr);
154 return (HcclResult)0;158 return (HcclResult)0;
155}159}
156 160 
157HcclResult hrtGetDeviceInfo(u32 deviceId, HcclRtDeviceModuleType hcclModuleType, HcclRtDeviceInfoType hcclInfoType,161HcclResult hrtGetDeviceInfo(u32 deviceId, HcclRtDeviceModuleType hcclModuleType, HcclRtDeviceInfoType hcclInfoType,
158 s64 &val)162 s64 &val)
159{163{
160- HCCL_VM_INFO("[RTS_STUB] deviceId:[{}] hcclModuleType:[{}], HcclRtDeviceInfoType:[{}]\n", deviceId,164+ HCCL_VM_INFO("deviceId:[{}] hcclModuleType:[{}], HcclRtDeviceInfoType:[{}]\n", deviceId,
161 (int)hcclModuleType, (int)hcclInfoType);165 (int)hcclModuleType, (int)hcclInfoType);
162 return (HcclResult)0;166 return (HcclResult)0;
163}167}
@@ -177,10 +181,9 @@ void *__HcclDlsymSub(void *handle, const char *funcName)
177 (void) handle;181 (void) handle;
178 void *addr = dlsym(RTLD_DEFAULT, funcName);182 void *addr = dlsym(RTLD_DEFAULT, funcName);
179 if (addr == nullptr) {183 if (addr == nullptr) {
180- HCCL_VM_INFO("[RTS_STUB] not find {} Error: [{}]\n", funcName, dlerror());184+ HCCL_VM_ERROR("not find {} Error: [{}]\n", funcName, dlerror());
181 return nullptr;185 return nullptr;
182 }186 }
183- // CCU_STUB_DEBUG("[HcclDlsymSub] [{}]\n", funcName);
184 return addr;187 return addr;
185}188}
186 189 
Mtest/hccl_vm/src/proxy/aiv_kernel/CMakeLists.txt+15-0
@@ -6,6 +6,19 @@
6# INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE.6# INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE.
7# See LICENSE in the root of the software repository for the full text of the License.7# See LICENSE in the root of the software repository for the full text of the License.
8 8 
9+add_compile_options(
10+ -fno-common
11+ -fno-strict-aliasing
12+ -pipe
13+ -O0
14+ -g
15+)
16+# 日志染色: aiv_kernel 整片统一 [AIV]
17+add_compile_definitions(
18+ HCCL_VM_MODULE="AIV"
19+ _GLIBCXX_USE_CXX11_ABI=0
20+)
21+ 
9if(NOT DEFINED HCCL_CODE_ROOT)22if(NOT DEFINED HCCL_CODE_ROOT)
10 if(NOT DEFINED ENV{HCCL_CODE_HOME})23 if(NOT DEFINED ENV{HCCL_CODE_HOME})
11 message(FATAL_ERROR "Error: BUILD_AIV is ON, but HCCL_CODE_HOME environment variable is not set. Please set it to the HCCL source path.")24 message(FATAL_ERROR "Error: BUILD_AIV is ON, but HCCL_CODE_HOME environment variable is not set. Please set it to the HCCL source path.")
@@ -41,6 +54,8 @@ add_library(hccl_aiv_kernel SHARED
41 $<TARGET_OBJECTS:hccl_aiv_reduce_op>54 $<TARGET_OBJECTS:hccl_aiv_reduce_op>
42 $<TARGET_OBJECTS:hccl_aiv_reduce_scatter_op>55 $<TARGET_OBJECTS:hccl_aiv_reduce_scatter_op>
43 $<TARGET_OBJECTS:hccl_aiv_scatter_op>56 $<TARGET_OBJECTS:hccl_aiv_scatter_op>
57+ 
58+ ${HCCL_VM_SRC}/utils/sim_common_api.cc
44)59)
45 60 
46target_link_libraries(hccl_aiv_kernel PRIVATE log)61target_link_libraries(hccl_aiv_kernel PRIVATE log)
Mtest/hccl_vm/src/proxy/aiv_kernel/aiv_mode_stub/ai_core_stub.cc+4-4
@@ -140,7 +140,7 @@ AivDataSlice AivKernelExecutor::ResolveGlobalDataSlice(uint64_t addr, uint64_t s
140}140}
141 141 
142void AivKernelExecutor::DumpAllTasks() const {142void AivKernelExecutor::DumpAllTasks() const {
143- HCCL_VM_DEBUG("[DumpAivTasks]");143+ HCCL_VM_DEBUG("");
144 HCCL_VM_DEBUG("[rankId={}]", GetRankId());144 HCCL_VM_DEBUG("[rankId={}]", GetRankId());
145 145 
146 for (const auto& core : aivCores_) {146 for (const auto& core : aivCores_) {
@@ -175,17 +175,17 @@ void AivCore::AppendMTE3(const std::shared_ptr<AivTask>& task) {
175void AivCore::DumpAllTasks() const {175void AivCore::DumpAllTasks() const {
176 HCCL_VM_DEBUG("[blockIdx={:d}]", blockIdx_);176 HCCL_VM_DEBUG("[blockIdx={:d}]", blockIdx_);
177 177 
178- HCCL_VM_DEBUG("[Scalar]");178+ HCCL_VM_DEBUG("");
179 for (const auto& task : pipeScalar_) {179 for (const auto& task : pipeScalar_) {
180 HCCL_VM_DEBUG(" {:s}", task->Describe());180 HCCL_VM_DEBUG(" {:s}", task->Describe());
181 }181 }
182 182 
183- HCCL_VM_DEBUG("[MTE2]");183+ HCCL_VM_DEBUG("MTE2 ");
184 for (const auto& task : pipeMTE2_) {184 for (const auto& task : pipeMTE2_) {
185 HCCL_VM_DEBUG(" {:s}", task->Describe());185 HCCL_VM_DEBUG(" {:s}", task->Describe());
186 }186 }
187 187 
188- HCCL_VM_DEBUG("[MTE3]");188+ HCCL_VM_DEBUG("MTE3 ");
189 for (const auto& task : pipeMTE3_) {189 for (const auto& task : pipeMTE3_) {
190 HCCL_VM_DEBUG(" {:s}", task->Describe());190 HCCL_VM_DEBUG(" {:s}", task->Describe());
191 }191 }
Mtest/hccl_vm/src/proxy/aiv_kernel/aiv_mode_stub/aiv_task_json.cc+4-9
@@ -9,6 +9,7 @@
9 */9 */
10 10 
11#include "aiv_task_json.h"11#include "aiv_task_json.h"
12+#include "sim_common_api.h"
12 13 
13#include <cstdint>14#include <cstdint>
14#include <cstdlib>15#include <cstdlib>
@@ -210,15 +211,9 @@ bool WriteJsonAtomically(const json &content, const std::string &filePath)
210 211 
211bool ResolveExecutorJsonFilePath(const AivKernelExecutor &executor, uint32_t launchIndex, std::string &filePath)212bool ResolveExecutorJsonFilePath(const AivKernelExecutor &executor, uint32_t launchIndex, std::string &filePath)
212{213{
213- const char *installDirEnv = std::getenv("HCCL_VM_INSTALL_DIR");214+ const std::string dataRelPath = "data/hcclvm_aiv_rank" + std::to_string(executor.GetRankId()) +
214- if (installDirEnv == nullptr || installDirEnv[0] == '\0') {215+ "_launch" + std::to_string(launchIndex) + "_task.json";
215- return false;216+ filePath = InstallPath::ResolveToInstallRoot(dataRelPath);
216- }
217- 
218- const fs::path outputPath = fs::path(installDirEnv) / "data" /
219- ("hcclvm_aiv_rank" + std::to_string(executor.GetRankId()) +
220- "_launch" + std::to_string(launchIndex) + "_task.json");
221- filePath = outputPath.string();
222 return true;217 return true;
223}218}
224 219 
Mtest/hccl_vm/src/proxy/aiv_kernel/hccl_op_stub/aiv_model_init.cc+7-7
@@ -38,15 +38,15 @@ void aiv_env_init(uint32_t rankId,
38 executor.SetCurOp(opParam);38 executor.SetCurOp(opParam);
39 const std::string inputMemDesc = AivSim::Mem{input, inputSize}.Describe();39 const std::string inputMemDesc = AivSim::Mem{input, inputSize}.Describe();
40 const std::string outputMemDesc = AivSim::Mem{output, outputSize}.Describe();40 const std::string outputMemDesc = AivSim::Mem{output, outputSize}.Describe();
41- HCCL_VM_DEBUG("[aiv_env_init] begin");41+ HCCL_VM_DEBUG("begin");
42 HCCL_VM_DEBUG(42 HCCL_VM_DEBUG(
43- "[aiv_env_init] rankId={}, blockNum={}, buffIn={:p}, rankSize={}",43+ "rankId={}, blockNum={}, buffIn={:p}, rankSize={}",
44 rankId,44 rankId,
45 blockNum,45 blockNum,
46 buffIn,46 buffIn,
47 rankSize);47 rankSize);
48 HCCL_VM_DEBUG(48 HCCL_VM_DEBUG(
49- "[aiv_env_init] input={}, output={}, "49+ "input={}, output={}, "
50 "inputGlobalOffsetBase={}, outputGlobalOffsetBase={}, cclBufferSize={}, flagBufferSize={}",50 "inputGlobalOffsetBase={}, outputGlobalOffsetBase={}, cclBufferSize={}, flagBufferSize={}",
51 inputMemDesc,51 inputMemDesc,
52 outputMemDesc,52 outputMemDesc,
@@ -55,7 +55,7 @@ void aiv_env_init(uint32_t rankId,
55 cclBufferSize,55 cclBufferSize,
56 flagBufferSize);56 flagBufferSize);
57 HCCL_VM_DEBUG(57 HCCL_VM_DEBUG(
58- "[aiv_env_init] curOp: dataType={}, len={}, reduceOp={}, root={}, sliceId={}, inputStride={}, "58+ "curOp: dataType={}, len={}, reduceOp={}, root={}, sliceId={}, inputStride={}, "
59 "outputStride={}, kernelName={}",59 "outputStride={}, kernelName={}",
60 opParam.dataType,60 opParam.dataType,
61 opParam.len,61 opParam.len,
@@ -66,7 +66,7 @@ void aiv_env_init(uint32_t rankId,
66 opParam.outputStride,66 opParam.outputStride,
67 opParam.kernelName);67 opParam.kernelName);
68 68 
69- HCCL_VM_DEBUG("[aiv_env_init] SetIoBuffer");69+ HCCL_VM_DEBUG("SetIoBuffer");
70 executor.SetIoBuffer(70 executor.SetIoBuffer(
71 input,71 input,
72 inputSize,72 inputSize,
@@ -76,7 +76,7 @@ void aiv_env_init(uint32_t rankId,
76 outputGlobalOffsetBase);76 outputGlobalOffsetBase);
77 77 
78 if (buffIn == nullptr) {78 if (buffIn == nullptr) {
79- HCCL_VM_DEBUG("[aiv_env_init] buffIn is null, skip SetCommBuffer initialization");79+ HCCL_VM_DEBUG("buffIn is null, skip SetCommBuffer initialization");
80 return;80 return;
81 }81 }
82 82 
@@ -89,7 +89,7 @@ void aiv_env_init(uint32_t rankId,
89 const std::string cclMemDesc = AivSim::Mem{cclBuffer, cclBufferSize}.Describe();89 const std::string cclMemDesc = AivSim::Mem{cclBuffer, cclBufferSize}.Describe();
90 const std::string flagMemDesc = AivSim::Mem{flagBuffer, flagBufferSize}.Describe();90 const std::string flagMemDesc = AivSim::Mem{flagBuffer, flagBufferSize}.Describe();
91 HCCL_VM_DEBUG(91 HCCL_VM_DEBUG(
92- "[aiv_env_init] SetCommBuffer rank={}, cclBuffer={}, flagBuffer={}",92+ "SetCommBuffer rank={}, cclBuffer={}, flagBuffer={}",
93 i,93 i,
94 cclMemDesc,94 cclMemDesc,
95 flagMemDesc);95 flagMemDesc);
Mtest/hccl_vm/src/proxy/aprofiling_stub.cc+42-0
@@ -78,6 +78,48 @@ int32_t MsprofReportCompactInfo(uint32_t agingFlag, const VOID_PTR data, uint32_
78 return 0;78 return 0;
79}79}
80 80 
81+int32_t MsprofReportBatchAdditionalInfo(uint32_t nonPersistantFlag, const VOID_PTR data, uint32_t length)
82+{
83+ (void) nonPersistantFlag;
84+ (void) data;
85+ (void) length;
86+ HCCL_VM_INFO("[APROF] stub");
87+ return 0;
88+}
89+ 
90+int32_t AdprofReportAdditionalInfo(uint32_t agingFlag, const void *data, uint32_t length)
91+{
92+ (void) agingFlag;
93+ (void) data;
94+ (void) length;
95+ HCCL_VM_INFO("[APROF] stub");
96+ return 0;
97+}
98+ 
99+int32_t AdprofReportBatchAdditionalInfo(uint32_t nonPersistantFlag, const void *data, uint32_t length)
100+{
101+ (void) nonPersistantFlag;
102+ (void) data;
103+ (void) length;
104+ HCCL_VM_INFO("[APROF] stub");
105+ return 0;
106+}
107+ 
108+uint64_t AdprofGetHashId(const char *hashInfo, size_t length)
109+{
110+ (void) hashInfo;
111+ (void) length;
112+ HCCL_VM_INFO("[APROF] stub");
113+ return 1;
114+}
115+ 
116+int32_t AdprofCheckFeatureIsOn(uint64_t feature)
117+{
118+ (void) feature;
119+ HCCL_VM_INFO("[APROF] stub");
120+ return 0;
121+}
122+ 
81#ifdef __cplusplus123#ifdef __cplusplus
82}124}
83#endif // __cplusplus125#endif // __cplusplus
Mtest/hccl_vm/src/proxy/ascend_hal_stub.cc+334-334
Mtest/hccl_vm/src/proxy/hccl_comm_stub.cc+3-6
@@ -7,6 +7,8 @@
7 * INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE.7 * INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE.
8 * See LICENSE in the root of the software repository for the full text of the License.8 * See LICENSE in the root of the software repository for the full text of the License.
9 */9 */
10+// 日志染色: 模块 tag (须在 include sim_log.h 之前)
11+#define HCCL_VM_MODULE "COMM_STUB"
10 12 
11#include <atomic>13#include <atomic>
12#include <cstdint>14#include <cstdint>
@@ -25,11 +27,6 @@
25#include "db_sim_runner_common.h"27#include "db_sim_runner_common.h"
26#include "db_sim_runner_ops.h"28#include "db_sim_runner_ops.h"
27 29 
28-#define COMM_STUB_ERROR(format, ...) HCCL_VM_ERROR("[COMM_STUB]" format, ##__VA_ARGS__)
29-#define COM_STUB_DEBUG(format, ...) HCCL_VM_DEBUG("[COMM_STUB]" format, ##__VA_ARGS__)
30-#define COMM_STUB_INFO(format, ...) HCCL_VM_INFO("[COMM_STUB]" format, ##__VA_ARGS__)
31-#define COM_STUB_WARN(format, ...) HCCL_VM_WARN("[COMM_STUB]" format, ##__VA_ARGS__)
32-#define COM_STUB_TRACE(format, ...) HCCL_VM_TRACE("[COMM_STUB]" format, ##__VA_ARGS__)
33 30 
34#ifdef __cplusplus31#ifdef __cplusplus
35extern "C" {32extern "C" {
@@ -55,7 +52,7 @@ HcclResult HcclCommInitRootInfo(uint32_t nRanks, const HcclRootInfo *rootInfo, u
55 // 解析ranktable文件,初始化通信域相关数据库表项52 // 解析ranktable文件,初始化通信域相关数据库表项
56 const char* clusterInfo = std::getenv("RANK_TABLE_FILE");53 const char* clusterInfo = std::getenv("RANK_TABLE_FILE");
57 if (!clusterInfo) {54 if (!clusterInfo) {
58- COMM_STUB_ERROR("RANK_TABLE_FILE env not set, please check your config.");55+ HCCL_VM_ERROR("RANK_TABLE_FILE env not set, please check your config.");
59 return HcclResult::HCCL_E_INTERNAL;56 return HcclResult::HCCL_E_INTERNAL;
60 }57 }
61 58 
Mtest/hccl_vm/src/proxy/hccl_op_stub.cc+96-2010
Mtest/hccl_vm/src/proxy/hccl_proxy_common.cc+5-7
@@ -8,6 +8,9 @@
8 * See LICENSE in the root of the software repository for the full text of the License.8 * See LICENSE in the root of the software repository for the full text of the License.
9 */9 */
10 10 
11+// 日志染色: 模块 tag (须在 include sim_log.h 之前)
12+#define HCCL_VM_MODULE "PROXY_STUB"
13+ 
11#include <cstdint>14#include <cstdint>
12#include <iostream>15#include <iostream>
13#include <map>16#include <map>
@@ -16,11 +19,6 @@
16#include "sim_log.h"19#include "sim_log.h"
17#include "db_sim_runner_db.h"20#include "db_sim_runner_db.h"
18 21 
19-#define PROXY_STUB_ERROR(format, ...) HCCL_VM_ERROR("[PROXY_STUB]" format, ##__VA_ARGS__)
20-#define PROXY_STUB_DEBUG(format, ...) HCCL_VM_DEBUG("[PROXY_STUB]" format, ##__VA_ARGS__)
21-#define PROXY_STUB_INFO(format, ...) HCCL_VM_INFO("[PROXY_STUB]" format, ##__VA_ARGS__)
22-#define PROXY_STUB_WARN(format, ...) HCCL_VM_WARN("[PROXY_STUB]" format, ##__VA_ARGS__)
23-#define PROXY_STUB_TRACE(format, ...) HCCL_VM_TRACE("[PROXY_STUB]" format, ##__VA_ARGS__)
24 22 
25namespace sim {23namespace sim {
26const std::map<HcclDataType, u32> DATA_TYPE_SIZE_MAP = {24const std::map<HcclDataType, u32> DATA_TYPE_SIZE_MAP = {
@@ -47,7 +45,7 @@ int GetDataTypeSize(HcclDataType dataType, uint32_t &size)
47{45{
48 auto iter = DATA_TYPE_SIZE_MAP.find(dataType);46 auto iter = DATA_TYPE_SIZE_MAP.find(dataType);
49 if (iter == DATA_TYPE_SIZE_MAP.end()) {47 if (iter == DATA_TYPE_SIZE_MAP.end()) {
50- PROXY_STUB_ERROR("not support data type: {}", static_cast<uint32_t>(dataType));48+ HCCL_VM_ERROR("not support data type: {}", static_cast<uint32_t>(dataType));
51 return 1;49 return 1;
52 }50 }
53 size = iter->second;51 size = iter->second;
@@ -63,7 +61,7 @@ bool IsDeviceAddress(void *addr)
63 (devPtr < (virMem.start_ptr + virMem.size)) &&61 (devPtr < (virMem.start_ptr + virMem.size)) &&
64 (virMem.src_type == (uint8_t)sim::VIR_MEM_TYPE_DEV)); });62 (virMem.src_type == (uint8_t)sim::VIR_MEM_TYPE_DEV)); });
65 if (!virMemRes.second) {63 if (!virMemRes.second) {
66- PROXY_STUB_INFO("this buff offset ptr not found:{:p} in VirtualMemBlock.", addr);64+ HCCL_VM_INFO("this buff offset ptr not found:{:p} in VirtualMemBlock.", addr);
67 return false;65 return false;
68 }66 }
69 67 
Mtest/hccl_vm/src/proxy/hccp_ccu_stub.cc+38-40
@@ -8,6 +8,9 @@
8 * See LICENSE in the root of the software repository for the full text of the License.8 * See LICENSE in the root of the software repository for the full text of the License.
9 */9 */
10 10 
11+// 日志染色: 模块 tag (须在 include sim_log.h 之前)
12+#define HCCL_VM_MODULE "CCU_STUB"
13+ 
11#include <atomic>14#include <atomic>
12#include <cstdint>15#include <cstdint>
13#include <filesystem>16#include <filesystem>
@@ -34,11 +37,6 @@
34#include "db_sim_runner_ops.h"37#include "db_sim_runner_ops.h"
35#include "sim_common_defs.h"38#include "sim_common_defs.h"
36 39 
37-#define CCU_STUB_ERROR(format, ...) HCCL_VM_ERROR("[CCU_STUB]" format, ##__VA_ARGS__)
38-#define CCU_STUB_DEBUG(format, ...) HCCL_VM_DEBUG("[CCU_STUB]" format, ##__VA_ARGS__)
39-#define CCU_STUB_INFO(format, ...) HCCL_VM_INFO("[CCU_STUB]" format, ##__VA_ARGS__)
40-#define CCU_STUB_WARN(format, ...) HCCL_VM_WARN("[CCU_STUB]" format, ##__VA_ARGS__)
41-#define CCU_STUB_TRACE(format, ...) HCCL_VM_TRACE("[CCU_STUB]" format, ##__VA_ARGS__)
42 40 
43extern uint64_t g_cur_server_key;41extern uint64_t g_cur_server_key;
44extern thread_local std::string g_currentOpFastLaunchTag;42extern thread_local std::string g_currentOpFastLaunchTag;
@@ -51,7 +49,7 @@ extern "C" {
51 49 
52int GetEnableCcuDie(channel_info_out *output, uint8_t dieId)50int GetEnableCcuDie(channel_info_out *output, uint8_t dieId)
53{51{
54- CCU_STUB_INFO("dieId:{}.", dieId);52+ HCCL_VM_INFO("dieId:{}.", dieId);
55 output->data.data_info.data_array[0].dieinfo.enable_flag = 1;53 output->data.data_info.data_array[0].dieinfo.enable_flag = 1;
56 return 0;54 return 0;
57}55}
@@ -90,13 +88,13 @@ int SetCcuResourceBasicInfo(channel_info_out* output, uint8_t dieId, uint32_t de
90{88{
91 sim::Device device{};89 sim::Device device{};
92 if (GetDeviceByPhysicalId(devId, device) != ACL_SUCCESS) {90 if (GetDeviceByPhysicalId(devId, device) != ACL_SUCCESS) {
93- CCU_STUB_ERROR("get device by logic id {} failed.", devId);91+ HCCL_VM_ERROR("get device by logic id {} failed.", devId);
94 return -1;92 return -1;
95 }93 }
96 if (strcmp(device.soc_version, "Ascend950") == 0) {94 if (strcmp(device.soc_version, "Ascend950") == 0) {
97 SetCcuV1ResourceBasicInfo(output, dieId, devId);95 SetCcuV1ResourceBasicInfo(output, dieId, devId);
98 } else {96 } else {
99- CCU_STUB_ERROR("unknown device version: {} for devId: {}", device.soc_version, devId);97+ HCCL_VM_ERROR("unknown device version: {} for devId: {}", device.soc_version, devId);
100 return -1;98 return -1;
101 }99 }
102 return 0;100 return 0;
@@ -160,25 +158,25 @@ bool write_or_overwrite_in_cwd(const std::string& filename, const std::string &d
160// input->op == Hccl::CcuOpcodeType::CCU_U_OP_SET_INSTRUCTION158// input->op == Hccl::CcuOpcodeType::CCU_U_OP_SET_INSTRUCTION
161int LoadMicrocodeInstructionStub(uint32_t devId, uint8_t dieId, const channel_info_in *input)159int LoadMicrocodeInstructionStub(uint32_t devId, uint8_t dieId, const channel_info_in *input)
162{160{
163- CCU_STUB_INFO("enter LoadMicrocodeInstruction .....");161+ HCCL_VM_INFO("enter LoadMicrocodeInstruction .....");
164 if (dieId >= DIE_NUM) {162 if (dieId >= DIE_NUM) {
165- CCU_STUB_ERROR("wrong param of die id: {}", dieId);163+ HCCL_VM_ERROR("wrong param of die id: {}", dieId);
166 return -1;164 return -1;
167 }165 }
168 166 
169 sim::Device device{};167 sim::Device device{};
170 if (GetDeviceByPhysicalId(devId, device) != ACL_SUCCESS) {168 if (GetDeviceByPhysicalId(devId, device) != ACL_SUCCESS) {
171- CCU_STUB_ERROR("get device by logic id {} failed.", devId);169+ HCCL_VM_ERROR("get device by logic id {} failed.", devId);
172 return -1;170 return -1;
173 }171 }
174 sim::Ccu ccu{};172 sim::Ccu ccu{};
175 if (GetCcuFromDeviceByDieId(device.id, dieId, ccu) != ACL_SUCCESS) {173 if (GetCcuFromDeviceByDieId(device.id, dieId, ccu) != ACL_SUCCESS) {
176- CCU_STUB_ERROR("get ccu from device by die id {} failed.", dieId);174+ HCCL_VM_ERROR("get ccu from device by die id {} failed.", dieId);
177 return -1;175 return -1;
178 }176 }
179 sim::CcuResource ccuRes;177 sim::CcuResource ccuRes;
180 if (GetCcuResourceByCcu(ccu.id, ccuRes) != ACL_SUCCESS) {178 if (GetCcuResourceByCcu(ccu.id, ccuRes) != ACL_SUCCESS) {
181- CCU_STUB_ERROR("get ccu resource by ccu {} failed.", ccu.id);179+ HCCL_VM_ERROR("get ccu resource by ccu {} failed.", ccu.id);
182 return -1;180 return -1;
183 }181 }
184 182 
@@ -187,7 +185,7 @@ int LoadMicrocodeInstructionStub(uint32_t devId, uint8_t dieId, const channel_in
187 return r.device_id == devKey;185 return r.device_id == devKey;
188 });186 });
189 if (!rank.second) {187 if (!rank.second) {
190- CCU_STUB_ERROR("can not find any rank");188+ HCCL_VM_ERROR("can not find any rank");
191 return -1;189 return -1;
192 }190 }
193 auto rankId = rank.first.rank_id;191 auto rankId = rank.first.rank_id;
@@ -200,11 +198,11 @@ int LoadMicrocodeInstructionStub(uint32_t devId, uint8_t dieId, const channel_in
200 auto ccuDataTmp = (ccu_data_type_union)(input->data.data_info.data_array[0]);198 auto ccuDataTmp = (ccu_data_type_union)(input->data.data_info.data_array[0]);
201 auto instructionData = reinterpret_cast<hcomm::CcuRep::CcuInstr*>(GetRealPtrByAddr((void *)ccuDataTmp.insinfo.resourceAddr));199 auto instructionData = reinterpret_cast<hcomm::CcuRep::CcuInstr*>(GetRealPtrByAddr((void *)ccuDataTmp.insinfo.resourceAddr));
202 if (instructionData == nullptr) {200 if (instructionData == nullptr) {
203- CCU_STUB_ERROR("get ccu instruction data by resourceAddr failed addr:0x{:x}", ccuDataTmp.insinfo.resourceAddr);201+ HCCL_VM_ERROR("get ccu instruction data by resourceAddr failed addr:0x{:x}", ccuDataTmp.insinfo.resourceAddr);
204 return -1;202 return -1;
205 }203 }
206 if (sim::UpdateAndInsertByCcuId(ccuId, devKey, rankId, dieId, instrCnt, instrOffset, instrInfoSize, instructionData) != 0) {204 if (sim::UpdateAndInsertByCcuId(ccuId, devKey, rankId, dieId, instrCnt, instrOffset, instrInfoSize, instructionData) != 0) {
207- CCU_STUB_ERROR("update ccu failed");205+ HCCL_VM_ERROR("update ccu failed");
208 return -1;206 return -1;
209 }207 }
210 208 
@@ -218,7 +216,7 @@ int LoadMicrocodeInstructionStub(uint32_t devId, uint8_t dieId, const channel_in
218 mcData << "[InstrData][ " + std::to_string(startId + idx) + "]" + hcomm::CcuRep::ParseInstr(&instructionData[idx]) + "\n";216 mcData << "[InstrData][ " + std::to_string(startId + idx) + "]" + hcomm::CcuRep::ParseInstr(&instructionData[idx]) + "\n";
219 }217 }
220 } else {218 } else {
221- CCU_STUB_ERROR("not support device soc version: {:s}", device.soc_version);219+ HCCL_VM_ERROR("not support device soc version: {:s}", device.soc_version);
222 return HCCL_E_NOT_SUPPORT;220 return HCCL_E_NOT_SUPPORT;
223 }221 }
224 222 
@@ -231,7 +229,7 @@ int LoadMicrocodeInstructionStub(uint32_t devId, uint8_t dieId, const channel_in
231 instr.startId = startId;229 instr.startId = startId;
232 instr.instrInfoSize = instrInfoSize;230 instr.instrInfoSize = instrInfoSize;
233 if(sim::InsertCcuInstr(instr) != 0) {231 if(sim::InsertCcuInstr(instr) != 0) {
234- CCU_STUB_ERROR("insert instr failed");232+ HCCL_VM_ERROR("insert instr failed");
235 return -1;233 return -1;
236 }234 }
237 return 0;235 return 0;
@@ -254,20 +252,20 @@ int GetLocalEndPointByJetty(uint64_t jettyId, uint16_t dieId, sim::EndPoint &end
254 return jetty.jetty_id == jettyId && jetty.pid == getpid() && jetty.dieId == dieId;252 return jetty.jetty_id == jettyId && jetty.pid == getpid() && jetty.dieId == dieId;
255 });253 });
256 if (!localJetty.second) {254 if (!localJetty.second) {
257- CCU_STUB_ERROR("can not find jetty {} die:{} in local jetty map", jettyId);255+ HCCL_VM_ERROR("can not find jetty {} die:{} in local jetty map", jettyId, dieId);
258 return -1;256 return -1;
259 }257 }
260 258 
261 auto ctxRes = RunnerDB::GetById<sim::RaContext>(localJetty.first.ctx_handle);259 auto ctxRes = RunnerDB::GetById<sim::RaContext>(localJetty.first.ctx_handle);
262 if (!ctxRes.has_value()) {260 if (!ctxRes.has_value()) {
263- CCU_STUB_ERROR("can not find context {} in local context map", localJetty.first.ctx_handle);261+ HCCL_VM_ERROR("can not find context {} in local context map", localJetty.first.ctx_handle);
264 return -1;262 return -1;
265 }263 }
266 264 
267 auto localEp = ctxRes->endpoint_id;265 auto localEp = ctxRes->endpoint_id;
268 auto endPointOpt = RunnerDB::GetById<sim::EndPoint>(localEp);266 auto endPointOpt = RunnerDB::GetById<sim::EndPoint>(localEp);
269 if (!endPointOpt.has_value()) {267 if (!endPointOpt.has_value()) {
270- CCU_STUB_ERROR("can not find endpoint:{:d}", localEp);268+ HCCL_VM_ERROR("can not find endpoint:{:d}", localEp);
271 return -1;269 return -1;
272 }270 }
273 endPoint = *endPointOpt;271 endPoint = *endPointOpt;
@@ -279,7 +277,7 @@ int ConfigChannelInfo(channel_info_in *input, uint32_t deviceId)
279{277{
280 sim::Device locDevice{};278 sim::Device locDevice{};
281 if (GetDeviceByPhysicalId(deviceId, locDevice) != ACL_SUCCESS) {279 if (GetDeviceByPhysicalId(deviceId, locDevice) != ACL_SUCCESS) {
282- CCU_STUB_ERROR("get device by physic id {} failed.", deviceId);280+ HCCL_VM_ERROR("get device by physic id {} failed.", deviceId);
283 return -1;281 return -1;
284 }282 }
285 uint8_t dieId = input->data.data_info.udie_idx;283 uint8_t dieId = input->data.data_info.udie_idx;
@@ -300,11 +298,11 @@ int ConfigChannelInfo(channel_info_in *input, uint32_t deviceId)
300 298 
301 sim::EndPoint rmtEndPoint{};299 sim::EndPoint rmtEndPoint{};
302 if (GetEndPointByIpAddr(ipAddr, rmtEndPoint) != 0) {300 if (GetEndPointByIpAddr(ipAddr, rmtEndPoint) != 0) {
303- CCU_STUB_ERROR("Get remote endpoint failed. ip:{}, eid:{}", ipAddr.c_str(), addr.EidToHexString());301+ HCCL_VM_ERROR("Get remote endpoint failed. ip:{}, eid:{}", ipAddr.c_str(), addr.EidToHexString());
304 return -1;302 return -1;
305 }303 }
306 304 
307- CCU_STUB_INFO(305+ HCCL_VM_INFO(
308 "channel info: loc phyId: {:d}, loc devKey: {:d}, loc dieId: {:d}, chId: {:d}, rmt devKey: {:d}, rmt dieId: {:d}, rmt ip: {}",306 "channel info: loc phyId: {:d}, loc devKey: {:d}, loc dieId: {:d}, chId: {:d}, rmt devKey: {:d}, rmt dieId: {:d}, rmt ip: {}",
309 deviceId,307 deviceId,
310 locDevice.id,308 locDevice.id,
@@ -319,7 +317,7 @@ int ConfigChannelInfo(channel_info_in *input, uint32_t deviceId)
319 return r.device_id == locDevKey;317 return r.device_id == locDevKey;
320 });318 });
321 if (!locRank.second) {319 if (!locRank.second) {
322- CCU_STUB_ERROR("can not find loc rank by device key: {}", locDevKey);320+ HCCL_VM_ERROR("can not find loc rank by device key: {}", locDevKey);
323 return -1;321 return -1;
324 }322 }
325 323 
@@ -328,7 +326,7 @@ int ConfigChannelInfo(channel_info_in *input, uint32_t deviceId)
328 return r.device_id == rmtDevKey;326 return r.device_id == rmtDevKey;
329 });327 });
330 if (!rmtRank.second) {328 if (!rmtRank.second) {
331- CCU_STUB_ERROR("can not find rmt rank by device key: {}", rmtDevKey);329+ HCCL_VM_ERROR("can not find rmt rank by device key: {}", rmtDevKey);
332 return -1;330 return -1;
333 }331 }
334 uint16_t srcJettyId {0};332 uint16_t srcJettyId {0};
@@ -343,11 +341,11 @@ int ConfigChannelInfo(channel_info_in *input, uint32_t deviceId)
343 // 根据endPointPair获取src eid341 // 根据endPointPair获取src eid
344 sim::EndPoint localEndPoint{};342 sim::EndPoint localEndPoint{};
345 if (GetLocalEndPointByJetty(srcJettyId, srcDieId, localEndPoint) != 0) {343 if (GetLocalEndPointByJetty(srcJettyId, srcDieId, localEndPoint) != 0) {
346- CCU_STUB_ERROR("can not find local endPoint by jettyId: {}", srcJettyId);344+ HCCL_VM_ERROR("can not find local endPoint by jettyId: {}", srcJettyId);
347 return -1;345 return -1;
348 }346 }
349 347 
350- CCU_STUB_INFO("[CHANNEL] add chn:{:d}, srcJetty:{:d},jettyNum:{:d}, srcAddr:{}, dstAddr:{}, {:d}<-->{:d}",348+ HCCL_VM_INFO("add chn:{:d}, srcJetty:{:d},jettyNum:{:d}, srcAddr:{}, dstAddr:{}, {:d}<-->{:d}",
351 chId, srcJettyId, jettyNum, localEndPoint.ip_addr, rmtEndPoint.ip_addr, localEndPoint.id, rmtEndPoint.id);349 chId, srcJettyId, jettyNum, localEndPoint.ip_addr, rmtEndPoint.ip_addr, localEndPoint.id, rmtEndPoint.id);
352 350 
353 sim::CcuChannelTab ccuChannelTab{};351 sim::CcuChannelTab ccuChannelTab{};
@@ -366,7 +364,7 @@ int ConfigChannelInfo(channel_info_in *input, uint32_t deviceId)
366 }364 }
367 auto ret = sim::InsertCcuChannel(ccuChannelTab); 365 auto ret = sim::InsertCcuChannel(ccuChannelTab);
368 if (ret != 0) {366 if (ret != 0) {
369- CCU_STUB_ERROR("insert ccu channel table failed for channel id: {}", chId);367+ HCCL_VM_ERROR("insert ccu channel table failed for channel id: {}", chId);
370 return -1;368 return -1;
371 }369 }
372 370 
@@ -375,7 +373,7 @@ int ConfigChannelInfo(channel_info_in *input, uint32_t deviceId)
375 373 
376int ConfigJettyInfo(channel_info_in *input, uint32_t deviceId)374int ConfigJettyInfo(channel_info_in *input, uint32_t deviceId)
377{375{
378- CCU_STUB_INFO("Enter into config jetty info...");376+ HCCL_VM_INFO("Enter into config jetty info...");
379 uint8_t dieId = input->data.data_info.udie_idx;377 uint8_t dieId = input->data.data_info.udie_idx;
380 uint32_t jettyNum = input->data.data_info.data_array_size;378 uint32_t jettyNum = input->data.data_info.data_array_size;
381 uint32_t startJettyCtxId = input->offset_start;379 uint32_t startJettyCtxId = input->offset_start;
@@ -388,20 +386,20 @@ int ConfigJettyInfo(channel_info_in *input, uint32_t deviceId)
388 }386 }
389 387 
390 for (auto &tmp : jettyCtxData) {388 for (auto &tmp : jettyCtxData) {
391- CCU_STUB_DEBUG("doorbellAddr: [3]0x{:04x}, [2]0x{:04x}, [1]0x{:04x}, [0]0x{:04x}",389+ HCCL_VM_DEBUG("doorbellAddr: [3]0x{:04x}, [2]0x{:04x}, [1]0x{:04x}, [0]0x{:04x}",
392 tmp.doorbellAddr[3], // 3: doorbell ��ַ����390 tmp.doorbellAddr[3], // 3: doorbell ��ַ����
393 tmp.doorbellAddr[2], // 2: doorbell ��ַ����391 tmp.doorbellAddr[2], // 2: doorbell ��ַ����
394 tmp.doorbellAddr[1],392 tmp.doorbellAddr[1],
395 tmp.doorbellAddr[0]);393 tmp.doorbellAddr[0]);
396 394 
397 // ��ȫ���⣺��ֹ��ӡtoken�����Ϣ395 // ��ȫ���⣺��ֹ��ӡtoken�����Ϣ
398- CCU_STUB_DEBUG("pfeIdx: 0x{:04x}, ioDieId: 0x{:04x}, doorbellAddrType: 0x{:04x}, tokenValueIsValid: 0x{:04x}",396+ HCCL_VM_DEBUG("pfeIdx: 0x{:04x}, ioDieId: 0x{:04x}, doorbellAddrType: 0x{:04x}, tokenValueIsValid: 0x{:04x}",
399 static_cast<uint16_t>(tmp.pfeIdx),397 static_cast<uint16_t>(tmp.pfeIdx),
400 static_cast<uint16_t>(tmp.ioDieId),398 static_cast<uint16_t>(tmp.ioDieId),
401 static_cast<uint16_t>(tmp.doorbellAddrType),399 static_cast<uint16_t>(tmp.doorbellAddrType),
402 static_cast<uint16_t>(tmp.tokenValueIsValid));400 static_cast<uint16_t>(tmp.tokenValueIsValid));
403 401 
404- CCU_STUB_DEBUG("sqeBasicBlockLeftShifts: 0x{:04x}, pi: 0x{:04x}, ci: 0x{:04x}, "402+ HCCL_VM_DEBUG("sqeBasicBlockLeftShifts: 0x{:04x}, pi: 0x{:04x}, ci: 0x{:04x}, "
405 "maxCi: 0x{:04x}, oooCqeCnt: 0x{:04x}, startWqeBasicBlockIdxLow: 0x{:04x}, "403 "maxCi: 0x{:04x}, oooCqeCnt: 0x{:04x}, startWqeBasicBlockIdxLow: 0x{:04x}, "
406 "startWqeBasicBlockIdxHigh: 0x{:04x}, doorbellSendState: 0x{:04x}",404 "startWqeBasicBlockIdxHigh: 0x{:04x}, doorbellSendState: 0x{:04x}",
407 static_cast<uint16_t>(tmp.sqeBasicBlockLeftShifts),405 static_cast<uint16_t>(tmp.sqeBasicBlockLeftShifts),
@@ -421,13 +419,13 @@ int GetCcuVersion(channel_info_out* output, uint32_t deviceId)
421{419{
422 sim::Device locDevice{};420 sim::Device locDevice{};
423 if (GetDeviceByPhysicalId(deviceId, locDevice) != ACL_SUCCESS) {421 if (GetDeviceByPhysicalId(deviceId, locDevice) != ACL_SUCCESS) {
424- CCU_STUB_ERROR("get device by physic id {} failed.", deviceId);422+ HCCL_VM_ERROR("get device by physic id {} failed.", deviceId);
425 return -1;423 return -1;
426 }424 }
427 if (strcmp(locDevice.soc_version, "Ascend950") == 0) {425 if (strcmp(locDevice.soc_version, "Ascend950") == 0) {
428 output->data.data_info.data_array[0].version = static_cast<ccu_version_e>(CcuVersion::CCU_V1);426 output->data.data_info.data_array[0].version = static_cast<ccu_version_e>(CcuVersion::CCU_V1);
429 } else {427 } else {
430- CCU_STUB_ERROR("unknown soc version: {}", locDevice.soc_version);428+ HCCL_VM_ERROR("unknown soc version: {}", locDevice.soc_version);
431 return -1;429 return -1;
432 }430 }
433 return 0;431 return 0;
@@ -469,7 +467,7 @@ int RaCtxGetAsyncEvents(void *ctxHandle, struct AsyncEvent events[], unsigned in
469 467 
470int RaGetEidByIp(void *ctxHandle, struct IpInfo ip[], union HccpEid eid[], unsigned int *num)468int RaGetEidByIp(void *ctxHandle, struct IpInfo ip[], union HccpEid eid[], unsigned int *num)
471{469{
472- CCU_STUB_ERROR("not support for now.");470+ HCCL_VM_ERROR("not support for now.");
473 return -1;471 return -1;
474}472}
475 473 
@@ -477,7 +475,7 @@ int GetAllUsedEndPoint(uint32_t phyDevId, std::vector<sim::EndPoint> &endPoints)
477{475{
478 sim::Device device{};476 sim::Device device{};
479 if (GetDeviceByPhysicalId(phyDevId, device) != 0) {477 if (GetDeviceByPhysicalId(phyDevId, device) != 0) {
480- CCU_STUB_ERROR("can not find device by id:{:d}", phyDevId);478+ HCCL_VM_ERROR("can not find device by id:{:d}", phyDevId);
481 return -1;479 return -1;
482 }480 }
483 481 
@@ -494,7 +492,7 @@ int RaGetDevEidInfoNum(struct RaInfo info, unsigned int *num)
494 if (GetAllUsedEndPoint(info.phyId, endPoints) != 0) {492 if (GetAllUsedEndPoint(info.phyId, endPoints) != 0) {
495 return -1;493 return -1;
496 }494 }
497- CCU_STUB_INFO("return success {:d}", endPoints.size());495+ HCCL_VM_INFO("return success {:d}", endPoints.size());
498 *num = endPoints.size();496 *num = endPoints.size();
499 return 0;497 return 0;
500}498}
@@ -503,7 +501,7 @@ int RaGetDevEidInfoList(struct RaInfo info, struct HccpDevEidInfo infoList[], un
503{501{
504 std::vector<sim::EndPoint> endPoints;502 std::vector<sim::EndPoint> endPoints;
505 if (GetAllUsedEndPoint(info.phyId, endPoints) != 0) {503 if (GetAllUsedEndPoint(info.phyId, endPoints) != 0) {
506- CCU_STUB_ERROR("get all EndPoint failed phyId:{:d}", info.phyId);504+ HCCL_VM_ERROR("get all EndPoint failed phyId:{:d}", info.phyId);
507 return -1;505 return -1;
508 }506 }
509 507 
@@ -547,7 +545,7 @@ int rtCCULaunch(rtCcuTaskInfo_t *taskInfo, rtStream_t const stream)
547 uint32_t index{0};545 uint32_t index{0};
548 auto ret = InsertTaskToCollection(&taskMetaData, &index);546 auto ret = InsertTaskToCollection(&taskMetaData, &index);
549 if (ret != HcclSim::HcclVmResult::HCCL_SIM_SUCCESS) {547 if (ret != HcclSim::HcclVmResult::HCCL_SIM_SUCCESS) {
550- CCU_STUB_ERROR("insert task fail");548+ HCCL_VM_ERROR("insert task fail");
551 return ACL_ERROR_INTERNAL_ERROR;549 return ACL_ERROR_INTERNAL_ERROR;
552 }550 }
553 551 
Mtest/hccl_vm/src/proxy/hccp_ra_socket_stub.cc+45-46
@@ -8,6 +8,9 @@
8 * See LICENSE in the root of the software repository for the full text of the License.8 * See LICENSE in the root of the software repository for the full text of the License.
9 */9 */
10 10 
11+// 日志染色: 模块 tag (须在 include sim_log.h 之前)
12+#define HCCL_VM_MODULE "RASOCKET_STUB"
13+ 
11#include <atomic>14#include <atomic>
12#include <cstdint>15#include <cstdint>
13#include <cstdio>16#include <cstdio>
@@ -33,10 +36,6 @@ constexpr uint32_t RA_SOCKET_BUF_SIZE = (64 * 1024);
33 36 
34constexpr const char* RA_SOCK_BUF_PREFIX = "ra_sock_";37constexpr const char* RA_SOCK_BUF_PREFIX = "ra_sock_";
35 38 
36-#define RA_SOCK_STUB_ERROR(format, ...) HCCL_VM_ERROR("[RASOCKET_STUB]" format, ##__VA_ARGS__)
37-#define RA_SOCK_STUB_DEBUG(format, ...) HCCL_VM_DEBUG("[RASOCKET_STUB]" format, ##__VA_ARGS__)
38-#define RA_SOCK_STUB_INFO(format, ...) HCCL_VM_INFO("[RASOCKET_STUB]" format, ##__VA_ARGS__)
39-#define RA_SOCK_STUB_WARN(format, ...) HCCL_VM_WARN("[RASOCKET_STUB]" format, ##__VA_ARGS__)
40 39 
41#define RS_FD_CLIENT 0x01U40#define RS_FD_CLIENT 0x01U
42#define RS_FD_SERVER 0x00U41#define RS_FD_SERVER 0x00U
@@ -65,7 +64,7 @@ static bool GenRaSocketBufKeyByPairId(uint64_t pairId)
65 std::string c2s = RA_SOCK_BUF_PREFIX + std::to_string(pairId) + "_c2s";64 std::string c2s = RA_SOCK_BUF_PREFIX + std::to_string(pairId) + "_c2s";
66 void *ptrC2s = sim::CommunicationMemoryManager::GetInstance().AllocCommMem(c2s.data());65 void *ptrC2s = sim::CommunicationMemoryManager::GetInstance().AllocCommMem(c2s.data());
67 if (ptrC2s == nullptr) {66 if (ptrC2s == nullptr) {
68- RA_SOCK_STUB_ERROR(" alloc sock name {} mem failed", c2s.data());67+ HCCL_VM_ERROR(" alloc sock name {} mem failed", c2s.data());
69 return false;68 return false;
70 }69 }
71 70 
@@ -73,7 +72,7 @@ static bool GenRaSocketBufKeyByPairId(uint64_t pairId)
73 void *ptrS2c = sim::CommunicationMemoryManager::GetInstance().AllocCommMem(s2c.data());72 void *ptrS2c = sim::CommunicationMemoryManager::GetInstance().AllocCommMem(s2c.data());
74 if (ptrS2c == nullptr) {73 if (ptrS2c == nullptr) {
75 sim::CommunicationMemoryManager::GetInstance().ReleaseCommMem(c2s.data());74 sim::CommunicationMemoryManager::GetInstance().ReleaseCommMem(c2s.data());
76- RA_SOCK_STUB_ERROR(" alloc sock name {} mem failed", s2c.data());75+ HCCL_VM_ERROR(" alloc sock name {} mem failed", s2c.data());
77 return false;76 return false;
78 }77 }
79 return true;78 return true;
@@ -84,7 +83,7 @@ static bool AcquireRaSocketBufKeyByPairId(uint64_t pairId)
84 std::string c2s = RA_SOCK_BUF_PREFIX + std::to_string(pairId) + "_c2s";83 std::string c2s = RA_SOCK_BUF_PREFIX + std::to_string(pairId) + "_c2s";
85 void *ptrC2s = sim::CommunicationMemoryManager::GetInstance().AcquireCommMem(c2s.data());84 void *ptrC2s = sim::CommunicationMemoryManager::GetInstance().AcquireCommMem(c2s.data());
86 if (ptrC2s == nullptr) {85 if (ptrC2s == nullptr) {
87- RA_SOCK_STUB_ERROR(" acquire sock name {} mem failed", c2s.data());86+ HCCL_VM_ERROR(" acquire sock name {} mem failed", c2s.data());
88 return false;87 return false;
89 }88 }
90 89 
@@ -92,7 +91,7 @@ static bool AcquireRaSocketBufKeyByPairId(uint64_t pairId)
92 void *ptrS2c = sim::CommunicationMemoryManager::GetInstance().AcquireCommMem(s2c.data());91 void *ptrS2c = sim::CommunicationMemoryManager::GetInstance().AcquireCommMem(s2c.data());
93 if (ptrS2c == nullptr) {92 if (ptrS2c == nullptr) {
94 sim::CommunicationMemoryManager::GetInstance().ReleaseCommMem(c2s.data());93 sim::CommunicationMemoryManager::GetInstance().ReleaseCommMem(c2s.data());
95- RA_SOCK_STUB_ERROR(" acquire sock name {} mem failed", s2c.data());94+ HCCL_VM_ERROR(" acquire sock name {} mem failed", s2c.data());
96 return false;95 return false;
97 }96 }
98 return true;97 return true;
@@ -131,7 +130,7 @@ int RaSocketInit(int mode, struct rdev rdevInfo, void **socketHandle)
131 (void) mode;130 (void) mode;
132 sim::Device device{};131 sim::Device device{};
133 if (GetDeviceByPhysicalId(rdevInfo.phyId, device) != ACL_SUCCESS) {132 if (GetDeviceByPhysicalId(rdevInfo.phyId, device) != ACL_SUCCESS) {
134- RA_SOCK_STUB_ERROR(" get device by phy id {} failed.", rdevInfo.phyId);133+ HCCL_VM_ERROR(" get device by phy id {} failed.", rdevInfo.phyId);
135 return -1;134 return -1;
136 }135 }
137 136 
@@ -141,7 +140,7 @@ int RaSocketInit(int mode, struct rdev rdevInfo, void **socketHandle)
141 140
142 sim::EndPoint endPoint{};141 sim::EndPoint endPoint{};
143 if (GetEndPointByIpAddr(ipAddr, endPoint) != 0) {142 if (GetEndPointByIpAddr(ipAddr, endPoint) != 0) {
144- RA_SOCK_STUB_ERROR(" cannot find remote ip {} ", ipAddr);143+ HCCL_VM_ERROR(" cannot find remote ip {} ", ipAddr);
145 return -1;144 return -1;
146 }145 }
147 146 
@@ -152,7 +151,7 @@ int RaSocketInit(int mode, struct rdev rdevInfo, void **socketHandle)
152 [deviceIdx, endpointId](const sim::RaSocket &so) { return (so.device_id == deviceIdx && so.endpoint_id == endpointId); });151 [deviceIdx, endpointId](const sim::RaSocket &so) { return (so.device_id == deviceIdx && so.endpoint_id == endpointId); });
153 if (ret.second) {152 if (ret.second) {
154 *socketHandle = reinterpret_cast<void *>(static_cast<uintptr_t>(ret.first.id));153 *socketHandle = reinterpret_cast<void *>(static_cast<uintptr_t>(ret.first.id));
155- RA_SOCK_STUB_INFO(" Found socket socketFd:{:d}", ret.first.id);154+ HCCL_VM_INFO(" Found socket socketFd:{:d}", ret.first.id);
156 return 0;155 return 0;
157 }156 }
158 157 
@@ -166,15 +165,15 @@ int RaSocketInit(int mode, struct rdev rdevInfo, void **socketHandle)
166 [deviceIdx, endpointId](const sim::RaSocket &so) { return (so.device_id == deviceIdx && so.endpoint_id == endpointId); });165 [deviceIdx, endpointId](const sim::RaSocket &so) { return (so.device_id == deviceIdx && so.endpoint_id == endpointId); });
167 if (retry.second) {166 if (retry.second) {
168 *socketHandle = reinterpret_cast<void *>(static_cast<uintptr_t>(retry.first.id));167 *socketHandle = reinterpret_cast<void *>(static_cast<uintptr_t>(retry.first.id));
169- RA_SOCK_STUB_INFO(" Insert failed, found existing socketFd:{:d}", retry.first.id);168+ HCCL_VM_WARN(" Insert conflict, reused existing socketFd:{:d}", retry.first.id);
170 return 0;169 return 0;
171 }170 }
172- RA_SOCK_STUB_ERROR(" Insert and re-lookup both failed for device:{:d} endpointId:{:d}", deviceIdx, endpointId);171+ HCCL_VM_ERROR(" Insert and re-lookup both failed for device:{:d} endpointId:{:d}", deviceIdx, endpointId);
173 return -1;172 return -1;
174 }173 }
175 174 
176 *socketHandle = reinterpret_cast<void *>(static_cast<uintptr_t>(socketfd));175 *socketHandle = reinterpret_cast<void *>(static_cast<uintptr_t>(socketfd));
177- RA_SOCK_STUB_INFO(" add socketHandle:{:d} device:{:d} ip:{} endpointId:{:d}", socketfd, deviceIdx, ipAddr, endpointId);176+ HCCL_VM_INFO(" add socketHandle:{:d} device:{:d} ip:{} endpointId:{:d}", socketfd, deviceIdx, ipAddr, endpointId);
178 return 0;177 return 0;
179}178}
180 179 
@@ -191,7 +190,7 @@ int RaSocketDeinit(void *socketHandle)
191 uint64_t localFd = static_cast<uint64_t>(reinterpret_cast<uintptr_t>(socketHandle));190 uint64_t localFd = static_cast<uint64_t>(reinterpret_cast<uintptr_t>(socketHandle));
192 RunnerDB::Delete<sim::RaSocket>(localFd);191 RunnerDB::Delete<sim::RaSocket>(localFd);
193 192 
194- RA_SOCK_STUB_INFO(" delete socketHandle:{:d}", localFd);193+ HCCL_VM_INFO(" delete socketHandle:{:d}", localFd);
195 return 0;194 return 0;
196}195}
197 196 
@@ -201,13 +200,13 @@ int RaSocketListenStart(struct SocketListenInfoT conn[], uint32_t num)
201 uint64_t socketHandle = static_cast<uint64_t>(reinterpret_cast<uintptr_t>(conn[i].socketHandle));200 uint64_t socketHandle = static_cast<uint64_t>(reinterpret_cast<uintptr_t>(conn[i].socketHandle));
202 auto socketRes = RunnerDB::GetById<sim::RaSocket>(socketHandle);201 auto socketRes = RunnerDB::GetById<sim::RaSocket>(socketHandle);
203 if (!socketRes.has_value()) {202 if (!socketRes.has_value()) {
204- RA_SOCK_STUB_ERROR(" can not get Socket:{:d}", socketHandle);203+ HCCL_VM_ERROR(" can not get Socket:{:d}", socketHandle);
205 return -1;204 return -1;
206 }205 }
207 RunnerDB::Update<sim::RaSocket>(socketHandle, [](sim::RaSocket &s) {206 RunnerDB::Update<sim::RaSocket>(socketHandle, [](sim::RaSocket &s) {
208 s.state = 1;207 s.state = 1;
209 });208 });
210- RA_SOCK_STUB_INFO(" socket {:d} port:{:d}", socketHandle, conn[i].port);209+ HCCL_VM_INFO(" socket {:d} port:{:d}", socketHandle, conn[i].port);
211 }210 }
212 return 0;211 return 0;
213}212}
@@ -218,13 +217,13 @@ int RaSocketListenStop(struct SocketListenInfoT conn[], unsigned int num)
218 uint64_t socketHandle = static_cast<uint64_t>(reinterpret_cast<uintptr_t>(conn[i].socketHandle));217 uint64_t socketHandle = static_cast<uint64_t>(reinterpret_cast<uintptr_t>(conn[i].socketHandle));
219 auto socketRes = RunnerDB::GetById<sim::RaSocket>(socketHandle);218 auto socketRes = RunnerDB::GetById<sim::RaSocket>(socketHandle);
220 if (!socketRes.has_value()) {219 if (!socketRes.has_value()) {
221- RA_SOCK_STUB_ERROR(" can not get Socket:{:d}", socketHandle);220+ HCCL_VM_ERROR(" can not get Socket:{:d}", socketHandle);
222 return -1;221 return -1;
223 }222 }
224 RunnerDB::Update<sim::RaSocket>(socketHandle, [](sim::RaSocket &s) {223 RunnerDB::Update<sim::RaSocket>(socketHandle, [](sim::RaSocket &s) {
225 s.state = 0;224 s.state = 0;
226 });225 });
227- RA_SOCK_STUB_INFO(" socket {:d} port:{:d}", socketHandle, conn[i].port);226+ HCCL_VM_INFO(" socket {:d} port:{:d}", socketHandle, conn[i].port);
228 }227 }
229 return 0;228 return 0;
230}229}
@@ -235,7 +234,7 @@ int RaSocketBatchConnect(struct SocketConnectInfoT conn[], unsigned int num)
235 uint64_t localFd = static_cast<uint64_t>(reinterpret_cast<uintptr_t>(conn[i].socketHandle));234 uint64_t localFd = static_cast<uint64_t>(reinterpret_cast<uintptr_t>(conn[i].socketHandle));
236 auto localSock = RunnerDB::GetById<sim::RaSocket>(localFd);235 auto localSock = RunnerDB::GetById<sim::RaSocket>(localFd);
237 if (!localSock.has_value()) {236 if (!localSock.has_value()) {
238- RA_SOCK_STUB_ERROR(" can not get Local Ra Socket:{:d}", localFd);237+ HCCL_VM_ERROR(" can not get Local Ra Socket:{:d}", localFd);
239 return -1;238 return -1;
240 }239 }
241 240 
@@ -244,7 +243,7 @@ int RaSocketBatchConnect(struct SocketConnectInfoT conn[], unsigned int num)
244 auto ipAddr = IpAddress(ba, AF_INET6).GetIpStr().substr(2);243 auto ipAddr = IpAddress(ba, AF_INET6).GetIpStr().substr(2);
245 sim::EndPoint endPoint{};244 sim::EndPoint endPoint{};
246 if (GetEndPointByIpAddr(ipAddr, endPoint) != 0) {245 if (GetEndPointByIpAddr(ipAddr, endPoint) != 0) {
247- RA_SOCK_STUB_ERROR(" cannot find remote ip:{}", ipAddr);246+ HCCL_VM_ERROR(" cannot find remote ip:{}", ipAddr);
248 return -1;247 return -1;
249 }248 }
250 249 
@@ -253,11 +252,11 @@ int RaSocketBatchConnect(struct SocketConnectInfoT conn[], unsigned int num)
253 uint64_t tagHash = ComputeTagHash(conn[i].tag);252 uint64_t tagHash = ComputeTagHash(conn[i].tag);
254 uint32_t remotePort = conn[i].port;253 uint32_t remotePort = conn[i].port;
255 254 
256- RA_SOCK_STUB_INFO(" dev:{} sock:{} connect dev:{} ip addr:{} tag:{}", localSock->device_id, localFd, remoteDevId, ipAddr, conn[i].tag);255+ HCCL_VM_INFO(" dev:{} sock:{} connect dev:{} ip addr:{} tag:{}", localSock->device_id, localFd, remoteDevId, ipAddr, conn[i].tag);
257 uint32_t count = 0;256 uint32_t count = 0;
258 while (true) {257 while (true) {
259 if (count++ >= 600) { // 超时60s258 if (count++ >= 600) { // 超时60s
260- RA_SOCK_STUB_ERROR(" can not get break dev:{} sock:{} connect dev:{} ip addr:{} tag:{}", localSock->device_id, localFd, remoteDevId, ipAddr, conn[i].tag);259+ HCCL_VM_ERROR(" can not get break dev:{} sock:{} connect dev:{} ip addr:{} tag:{}", localSock->device_id, localFd, remoteDevId, ipAddr, conn[i].tag);
261 break;260 break;
262 }261 }
263 auto remoteSockRes = RunnerDB::GetOneByPred<sim::RaSocket>(262 auto remoteSockRes = RunnerDB::GetOneByPred<sim::RaSocket>(
@@ -266,7 +265,7 @@ int RaSocketBatchConnect(struct SocketConnectInfoT conn[], unsigned int num)
266 });265 });
267 if (!remoteSockRes.second) {266 if (!remoteSockRes.second) {
268 if (count % 100 == 0) {267 if (count % 100 == 0) {
269- RA_SOCK_STUB_WARN(" can not find remote dev:{}, endpoint:{}", remoteDevId, remoteEndPointId);268+ HCCL_VM_WARN(" can not find remote dev:{}, endpoint:{}", remoteDevId, remoteEndPointId);
270 }269 }
271 270 
272 std::this_thread::sleep_for(std::chrono::milliseconds(100));271 std::this_thread::sleep_for(std::chrono::milliseconds(100));
@@ -285,7 +284,7 @@ int RaSocketBatchConnect(struct SocketConnectInfoT conn[], unsigned int num)
285 uint64_t pairId = 0;284 uint64_t pairId = 0;
286 if (existPairRes.second) {285 if (existPairRes.second) {
287 pairId = existPairRes.first.id;286 pairId = existPairRes.first.id;
288- RA_SOCK_STUB_INFO(" pair already exists id:{:d}", pairId);287+ HCCL_VM_INFO(" pair already exists id:{:d}", pairId);
289 } else {288 } else {
290 sim::RaSocketPair socketpair{};289 sim::RaSocketPair socketpair{};
291 socketpair.client_id = localFd;290 socketpair.client_id = localFd;
@@ -296,7 +295,7 @@ int RaSocketBatchConnect(struct SocketConnectInfoT conn[], unsigned int num)
296 socketpair.buf_status = 0;295 socketpair.buf_status = 0;
297 pairId = RunnerDB::Add<sim::RaSocketPair>(socketpair);296 pairId = RunnerDB::Add<sim::RaSocketPair>(socketpair);
298 if (pairId == 0) {297 if (pairId == 0) {
299- RA_SOCK_STUB_WARN(" add failed retry");298+ HCCL_VM_WARN(" add failed retry");
300 std::this_thread::sleep_for(std::chrono::milliseconds(100));299 std::this_thread::sleep_for(std::chrono::milliseconds(100));
301 continue;300 continue;
302 }301 }
@@ -311,7 +310,7 @@ int RaSocketBatchConnect(struct SocketConnectInfoT conn[], unsigned int num)
311 std::string sendBuf, recvBuf;310 std::string sendBuf, recvBuf;
312 GetRaSendSocketkeyByFd(pairId, sendBuf);311 GetRaSendSocketkeyByFd(pairId, sendBuf);
313 GetRaRecvSocketkeyByFd(pairId, recvBuf);312 GetRaRecvSocketkeyByFd(pairId, recvBuf);
314- RA_SOCK_STUB_INFO(" add pair id:{:d} local:{:d} peer:{:d} port:{:d} sendBuf:{}, recvBuf:{}",313+ HCCL_VM_INFO(" add pair id:{:d} local:{:d} peer:{:d} port:{:d} sendBuf:{}, recvBuf:{}",
315 pairId, localFd, peerFd, remotePort, sendBuf, recvBuf);314 pairId, localFd, peerFd, remotePort, sendBuf, recvBuf);
316 }315 }
317 break;316 break;
@@ -327,7 +326,7 @@ int RaGetSockets(unsigned int role, struct SocketInfoT conn[], unsigned int num,
327 uint64_t loadFd = static_cast<uint64_t>(reinterpret_cast<uintptr_t>(conn[i].socketHandle));326 uint64_t loadFd = static_cast<uint64_t>(reinterpret_cast<uintptr_t>(conn[i].socketHandle));
328 auto localSock = RunnerDB::GetById<sim::RaSocket>(loadFd);327 auto localSock = RunnerDB::GetById<sim::RaSocket>(loadFd);
329 if (!localSock.has_value()) {328 if (!localSock.has_value()) {
330- RA_SOCK_STUB_ERROR(" can not get local socket fd:{:d}", loadFd);329+ HCCL_VM_ERROR(" can not get local socket fd:{:d}", loadFd);
331 return -1;330 return -1;
332 }331 }
333 332 
@@ -336,7 +335,7 @@ int RaGetSockets(unsigned int role, struct SocketInfoT conn[], unsigned int num,
336 auto ipAddr = IpAddress(ba, AF_INET6).GetIpStr().substr(2);335 auto ipAddr = IpAddress(ba, AF_INET6).GetIpStr().substr(2);
337 sim::EndPoint endPoint{};336 sim::EndPoint endPoint{};
338 if (GetEndPointByIpAddr(ipAddr, endPoint) != 0) {337 if (GetEndPointByIpAddr(ipAddr, endPoint) != 0) {
339- RA_SOCK_STUB_ERROR(" cannot find remote ip:{}", ipAddr);338+ HCCL_VM_ERROR(" cannot find remote ip:{}", ipAddr);
340 continue;339 continue;
341 }340 }
342 341 
@@ -344,7 +343,7 @@ int RaGetSockets(unsigned int role, struct SocketInfoT conn[], unsigned int num,
344 auto remoteEndpointId = endPoint.id;343 auto remoteEndpointId = endPoint.id;
345 uint64_t tagHash = ComputeTagHash(conn[i].tag);344 uint64_t tagHash = ComputeTagHash(conn[i].tag);
346 345 
347- RA_SOCK_STUB_INFO(" dev:{} sock:{} get remote dev:{} addr:{} tag:{}", localSock->device_id, loadFd, remoteDevId, ipAddr, conn[i].tag);346+ HCCL_VM_INFO(" dev:{} sock:{} get remote dev:{} addr:{} tag:{}", localSock->device_id, loadFd, remoteDevId, ipAddr, conn[i].tag);
348 347 
349 uint32_t count = 0;348 uint32_t count = 0;
350 bool found = false;349 bool found = false;
@@ -355,7 +354,7 @@ int RaGetSockets(unsigned int role, struct SocketInfoT conn[], unsigned int num,
355 });354 });
356 if (!remoteSockRes.second) {355 if (!remoteSockRes.second) {
357 if (count % 60 == 0) {356 if (count % 60 == 0) {
358- RA_SOCK_STUB_WARN(" waiting for remote socket dev:{}, endpoint:{}, attempt:{}/180",357+ HCCL_VM_WARN(" waiting for remote socket dev:{}, endpoint:{}, attempt:{}/180",
359 remoteDevId, remoteEndpointId, count);358 remoteDevId, remoteEndpointId, count);
360 }359 }
361 std::this_thread::sleep_for(std::chrono::milliseconds(500));360 std::this_thread::sleep_for(std::chrono::milliseconds(500));
@@ -373,7 +372,7 @@ int RaGetSockets(unsigned int role, struct SocketInfoT conn[], unsigned int num,
373 372 
374 if (!pairRes.second) {373 if (!pairRes.second) {
375 if (count % 60 == 0) {374 if (count % 60 == 0) {
376- RA_SOCK_STUB_WARN(" waiting for socket pair local:{:d} peer:{:d} tag_hash:{:d}, attempt:{}/180",375+ HCCL_VM_WARN(" waiting for socket pair local:{:d} peer:{:d} tag_hash:{:d}, attempt:{}/180",
377 loadFd, peerFd, tagHash, count);376 loadFd, peerFd, tagHash, count);
378 }377 }
379 std::this_thread::sleep_for(std::chrono::milliseconds(500));378 std::this_thread::sleep_for(std::chrono::milliseconds(500));
@@ -393,13 +392,13 @@ int RaGetSockets(unsigned int role, struct SocketInfoT conn[], unsigned int num,
393 *connectedNum += 1;392 *connectedNum += 1;
394 found = true;393 found = true;
395 394 
396- RA_SOCK_STUB_INFO(" get socket local:{:d} peer:{:d} pairId:{:d} role:{:d}",395+ HCCL_VM_INFO(" get socket local:{:d} peer:{:d} pairId:{:d} role:{:d}",
397 loadFd, peerFd, pairId, role);396 loadFd, peerFd, pairId, role);
398 break;397 break;
399 }398 }
400 399 
401 if (!found) {400 if (!found) {
402- RA_SOCK_STUB_ERROR(" get socket failed local:{:d} peerAddr:{} role:{:d}", loadFd, ipAddr, role);401+ HCCL_VM_ERROR(" get socket failed local:{:d} peerAddr:{} role:{:d}", loadFd, ipAddr, role);
403 }402 }
404 }403 }
405 return 0;404 return 0;
@@ -413,7 +412,7 @@ int RaSocketBatchClose(struct SocketCloseInfoT conn[], unsigned int num)
413 412 
414 auto pairRes = RunnerDB::GetById<sim::RaSocketPair>(pairId);413 auto pairRes = RunnerDB::GetById<sim::RaSocketPair>(pairId);
415 if (!pairRes.has_value()) {414 if (!pairRes.has_value()) {
416- RA_SOCK_STUB_WARN(" pair {:d} not found during close", pairId);415+ HCCL_VM_WARN(" pair {:d} not found during close", pairId);
417 continue;416 continue;
418 }417 }
419 418 
@@ -423,10 +422,10 @@ int RaSocketBatchClose(struct SocketCloseInfoT conn[], unsigned int num)
423 422 
424 if (pairRes->ref_cnt > 1) {423 if (pairRes->ref_cnt > 1) {
425 RunnerDB::Update<sim::RaSocketPair>(pairId, [](sim::RaSocketPair &pair) { pair.ref_cnt -= 1; });424 RunnerDB::Update<sim::RaSocketPair>(pairId, [](sim::RaSocketPair &pair) { pair.ref_cnt -= 1; });
426- RA_SOCK_STUB_INFO(" reduce pair:{:d} ref_cnt:{:d} sendBuf:{}, recvBuf:{}", pairId, pairRes->ref_cnt, sendBuf, recvBuf);425+ HCCL_VM_INFO(" reduce pair:{:d} ref_cnt:{:d} sendBuf:{}, recvBuf:{}", pairId, pairRes->ref_cnt, sendBuf, recvBuf);
427 } else {426 } else {
428 RunnerDB::Delete<sim::RaSocketPair>(pairId);427 RunnerDB::Delete<sim::RaSocketPair>(pairId);
429- RA_SOCK_STUB_INFO(" delete pair:{:d} ref_cnt:{:d} sendBuf:{}, recvBuf:{}", pairId, pairRes->ref_cnt, sendBuf, recvBuf);428+ HCCL_VM_INFO(" delete pair:{:d} ref_cnt:{:d} sendBuf:{}, recvBuf:{}", pairId, pairRes->ref_cnt, sendBuf, recvBuf);
430 }429 }
431 430 
432 DestoryRaSocketBufKeyByPairId(pairId);431 DestoryRaSocketBufKeyByPairId(pairId);
@@ -449,13 +448,13 @@ int RaSocketSend(const void *fdHandle, const void *data, unsigned long long size
449 448 
450 int ret = sim::CommunicationMemoryManager::GetInstance().WriteCommMem(socketKey.data(), data, size);449 int ret = sim::CommunicationMemoryManager::GetInstance().WriteCommMem(socketKey.data(), data, size);
451 if (ret != 0) {450 if (ret != 0) {
452- RA_SOCK_STUB_ERROR(" cannot pair socket:{:d} role:{:d}, key={}", FD_PAIR_ID(socketFd),451+ HCCL_VM_ERROR(" cannot pair socket:{:d} role:{:d}, key={}", FD_PAIR_ID(socketFd),
453 FD_ROLE(socketFd), socketKey);452 FD_ROLE(socketFd), socketKey);
454 return -1;453 return -1;
455 }454 }
456 455 
457 *sentSize = size;456 *sentSize = size;
458- RA_SOCK_STUB_INFO(" pair socket:{:d} role:{:d} key={} Send:{:d}", FD_PAIR_ID(socketFd),457+ HCCL_VM_INFO(" pair socket:{:d} role:{:d} key={} Send:{:d}", FD_PAIR_ID(socketFd),
459 FD_ROLE(socketFd), socketKey.data(), size);458 FD_ROLE(socketFd), socketKey.data(), size);
460 459 
461 return 0;460 return 0;
@@ -469,16 +468,16 @@ int RaSocketRecv(const void *fdHandle, void *data, unsigned long long size, unsi
469 468 
470 int ret = sim::CommunicationMemoryManager::GetInstance().ReadCommMem(socketKey.data(), data, size);469 int ret = sim::CommunicationMemoryManager::GetInstance().ReadCommMem(socketKey.data(), data, size);
471 if (ret == 0) {470 if (ret == 0) {
472- RA_SOCK_STUB_WARN(" socket pair:{:d} role:{:d}, key={} read try again",471+ HCCL_VM_WARN(" socket pair:{:d} role:{:d}, key={} read try again",
473 FD_PAIR_ID(socketFd), FD_ROLE(socketFd), socketKey);472 FD_PAIR_ID(socketFd), FD_ROLE(socketFd), socketKey);
474 std::this_thread::sleep_for(std::chrono::milliseconds(500));473 std::this_thread::sleep_for(std::chrono::milliseconds(500));
475 return 0;474 return 0;
476 } else if (ret == -1) {475 } else if (ret == -1) {
477- RA_SOCK_STUB_ERROR(" socket pair:{:d} role:{:d}, key={} recv failed",476+ HCCL_VM_ERROR(" socket pair:{:d} role:{:d}, key={} recv failed",
478 FD_PAIR_ID(socketFd), FD_ROLE(socketFd), socketKey);477 FD_PAIR_ID(socketFd), FD_ROLE(socketFd), socketKey);
479 return -1;478 return -1;
480 } else if (ret > 0) {479 } else if (ret > 0) {
481- RA_SOCK_STUB_INFO(" socket pair:{:d} role:{:d}, key={} recv bytes:{:d} ",480+ HCCL_VM_INFO(" socket pair:{:d} role:{:d}, key={} recv bytes:{:d} ",
482 FD_PAIR_ID(socketFd), FD_ROLE(socketFd), socketKey, ret);481 FD_PAIR_ID(socketFd), FD_ROLE(socketFd), socketKey, ret);
483 }482 }
484 *receivedSize = ret;483 *receivedSize = ret;
@@ -545,7 +544,7 @@ int RaSocketListenStopAsync(struct SocketListenInfoT conn[], unsigned int num, v
545int RaSocketBatchCloseAsync(struct SocketCloseInfoT conn[], unsigned int num, void **reqHandle)544int RaSocketBatchCloseAsync(struct SocketCloseInfoT conn[], unsigned int num, void **reqHandle)
546{545{
547 (void) reqHandle;546 (void) reqHandle;
548- RA_SOCK_STUB_INFO(" close socket async");547+ HCCL_VM_INFO(" close socket async");
549 return RaSocketBatchClose(conn, num);548 return RaSocketBatchClose(conn, num);
550}549}
551 550 
@@ -567,18 +566,18 @@ int RaSocketRecvAsync(const void *fdHandle, void *data, unsigned long long size,
567 566 
568 int ret = sim::CommunicationMemoryManager::GetInstance().ReadCommMem(socketKey.data(), data, size); 567 int ret = sim::CommunicationMemoryManager::GetInstance().ReadCommMem(socketKey.data(), data, size);
569 if (ret == 0) {568 if (ret == 0) {
570- RA_SOCK_STUB_WARN(" socket pair:{:d} role:{:d} key:{} try again",569+ HCCL_VM_WARN(" socket pair:{:d} role:{:d} key:{} try again",
571 FD_PAIR_ID(socketFd), FD_ROLE(socketFd), socketKey);570 FD_PAIR_ID(socketFd), FD_ROLE(socketFd), socketKey);
572 std::this_thread::sleep_for(std::chrono::milliseconds(100));571 std::this_thread::sleep_for(std::chrono::milliseconds(100));
573 return 0;572 return 0;
574 } else if (ret == -1) {573 } else if (ret == -1) {
575- RA_SOCK_STUB_ERROR(" socket pair:{:d} role:{:d} key:{} recv failed",574+ HCCL_VM_ERROR(" socket pair:{:d} role:{:d} key:{} recv failed",
576 FD_PAIR_ID(socketFd), FD_ROLE(socketFd), socketKey);575 FD_PAIR_ID(socketFd), FD_ROLE(socketFd), socketKey);
577 return -1;576 return -1;
578 }577 }
579 578 
580 *receivedSize = ret;579 *receivedSize = ret;
581- RA_SOCK_STUB_INFO(" socket pair:{:d} role:{:d} key:{} recv bytes:{:d} ",580+ HCCL_VM_INFO(" socket pair:{:d} role:{:d} key:{} recv bytes:{:d} ",
582 FD_PAIR_ID(socketFd), FD_ROLE(socketFd), socketKey, ret);581 FD_PAIR_ID(socketFd), FD_ROLE(socketFd), socketKey, ret);
583 return 0;582 return 0;
584}583}
Mtest/hccl_vm/src/proxy/hccp_stub.cc+211-213
Mtest/hccl_vm/src/proxy/log_stub.cc+9-5
@@ -8,6 +8,9 @@
8 * See LICENSE in the root of the software repository for the full text of the License.8 * See LICENSE in the root of the software repository for the full text of the License.
9 */9 */
10 10 
11+// 日志染色: 模块 tag (须在 include sim_log.h 之前)
12+#define HCCL_VM_MODULE "HCCL_LOG"
13+ 
11#include <cstdarg>14#include <cstdarg>
12#include <cstdint>15#include <cstdint>
13#include <cstdio>16#include <cstdio>
@@ -17,6 +20,7 @@
17 20 
18#include "sim_log.h"21#include "sim_log.h"
19 22 
23+ 
20constexpr size_t LOG_MSG_BUFFER_SIZE = 1024;24constexpr size_t LOG_MSG_BUFFER_SIZE = 1024;
21 25 
22std::once_flag log_initialized_flag;26std::once_flag log_initialized_flag;
@@ -44,19 +48,19 @@ void DlogPrintStub(int level, const char *msgBuffer)
44 48 
45 switch (level) {49 switch (level) {
46 case 0:50 case 0:
47- HCCL_VM_DEBUG("[HCCL_LOG]{}", msgBuffer);51+ HCCL_VM_DEBUG("{}", msgBuffer);
48 break;52 break;
49 case 1:53 case 1:
50- HCCL_VM_INFO("[HCCL_LOG]{}", msgBuffer);54+ HCCL_VM_INFO("{}", msgBuffer);
51 break;55 break;
52 case 2:56 case 2:
53- HCCL_VM_WARN("[HCCL_LOG]{}", msgBuffer);57+ HCCL_VM_WARN("{}", msgBuffer);
54 break;58 break;
55 case 3:59 case 3:
56- HCCL_VM_ERROR("[HCCL_LOG]{}", msgBuffer);60+ HCCL_VM_ERROR("{}", msgBuffer);
57 break;61 break;
58 default:62 default:
59- HCCL_VM_TRACE("[HCCL_LOG]{}", msgBuffer);63+ HCCL_VM_TRACE("{}", msgBuffer);
60 }64 }
61}65}
62 66 
Mtest/hccl_vm/src/proxy/scatter_stub.cc+5-7
@@ -10,23 +10,21 @@
10 10 
11// 为确保Scatter正常执行打的临时桩,需要适时移除11// 为确保Scatter正常执行打的临时桩,需要适时移除
12 12 
13+// 日志染色: 模块 tag (须在 include sim_log.h 之前)
14+#define HCCL_VM_MODULE "SCATTER_STUB"
15+ 
13#include <cstdio>16#include <cstdio>
14#include <cstring>17#include <cstring>
15 18 
16#include "hccl/dtype_common.h"19#include "hccl/dtype_common.h"
17#include "sim_log.h"20#include "sim_log.h"
18 21 
19-#define SCATTER_STUB_ERROR(format, ...) HCCL_VM_ERROR("[SCATTER_STUB]" format, ##_VA_ARGS__)
20-#define SCATTER_STUB_DEBUG(format, ...) HCCL_VM_DEBUG("[SCATTER_STUB]" format, ##_VA_ARGS__)
21-#define SCATTER_STUB_INFO(format, ...) HCCL_VM_INFO("[SCATTER_STUB]" format, ##__VA_ARGS__)
22-#define SCATTER_STUB_WARN(format, ...) HCCL_VM_WARN("[SCATTER_STUB]" format, ##__VA_ARGS__)
23-#define SCATTER_STUB_TRACE(format, ...) HCCL_VM_TRACE("[SCATTER_STUB]" format, ##__VA_ARGS__)
24 22 
25namespace ops_hccl {23namespace ops_hccl {
26bool RunIndependentOpExpansion(DevType deviceType)24bool RunIndependentOpExpansion(DevType deviceType)
27{25{
28 (void) deviceType;26 (void) deviceType;
29- SCATTER_STUB_INFO("return true");27+ HCCL_VM_INFO("return true");
30 return true;28 return true;
31}29}
32 30 
@@ -35,7 +33,7 @@ bool IsAiCpuMode(DevType deviceType, u32 rankSize)
35 (void) deviceType;33 (void) deviceType;
36 (void) rankSize;34 (void) rankSize;
37 const char* env = getenv("HCCL_OP_EXPANSION_MODE");35 const char* env = getenv("HCCL_OP_EXPANSION_MODE");
38- SCATTER_STUB_INFO("env = {}", env);36+ HCCL_VM_INFO("env = {}", env);
39 // 判断是否是AI_CPU37 // 判断是否是AI_CPU
40 if (env != nullptr && strcmp(env, "AI_CPU") == 0) {38 if (env != nullptr && strcmp(env, "AI_CPU") == 0) {
41 return true;39 return true;
Mtest/hccl_vm/src/proxy/tsd_client_stub.cc+5-1
@@ -8,6 +8,9 @@
8 * See LICENSE in the root of the software repository for the full text of the License.8 * See LICENSE in the root of the software repository for the full text of the License.
9 */9 */
10 10 
11+// 日志染色: 模块 tag (须在 include sim_log.h 之前)
12+#define HCCL_VM_MODULE "TSD_STUB"
13+ 
11#include <atomic>14#include <atomic>
12#include <cstdint>15#include <cstdint>
13#include <iostream>16#include <iostream>
@@ -16,6 +19,7 @@
16 19 
17#include "sim_log.h"20#include "sim_log.h"
18 21 
22+ 
19#ifdef __cplusplus23#ifdef __cplusplus
20extern "C" {24extern "C" {
21#endif // __cplusplus25#endif // __cplusplus
@@ -25,7 +29,7 @@ uint32_t TsdCapabilityGet(const uint32_t logicDeviceId, const int32_t type, cons
25 (void) logicDeviceId;29 (void) logicDeviceId;
26 (void) type;30 (void) type;
27 (void) ptr;31 (void) ptr;
28- HCCL_VM_INFO("[TSD_STUB]stub");32+ HCCL_VM_INFO("stub");
29 return 0;33 return 0;
30}34}
31#ifdef __cplusplus35#ifdef __cplusplus
Mtest/hccl_vm/src/runnerdb/CMakeLists.txt+3-0
@@ -59,3 +59,6 @@ find_package(SQLite3 REQUIRED)
59 59 
60target_link_libraries(runnerdb PUBLIC SQLite::SQLite3)60target_link_libraries(runnerdb PUBLIC SQLite::SQLite3)
61 61 
62+ 
63+# 日志染色: 模块 tag
64+target_compile_definitions(runnerdb PRIVATE HCCL_VM_MODULE="RUNNERDB")
Mtest/hccl_vm/src/runnerdb/modeldb/CMakeLists.txt+3-0
@@ -36,3 +36,6 @@ find_package(SQLite3 REQUIRED)
36 36 
37target_link_libraries(modeldb PUBLIC SQLite::SQLite3)37target_link_libraries(modeldb PUBLIC SQLite::SQLite3)
38 38 
39+ 
40+# 日志染色: 模块 tag
41+target_compile_definitions(modeldb PRIVATE HCCL_VM_MODULE="RUNNERDB")
Mtest/hccl_vm/src/runnerdb/modeldb/db_hccl_db_sqlite.cc+19-19
@@ -32,7 +32,7 @@ HcclVmResult HcclDBSqlite::Connect(const sim::DBConfig& config) {
32 int ret = sqlite3_open(config.dbPath.c_str(), &m_db);32 int ret = sqlite3_open(config.dbPath.c_str(), &m_db);
33 if (ret != SQLITE_OK) {33 if (ret != SQLITE_OK) {
34 m_lastError = sqlite3_errmsg(m_db);34 m_lastError = sqlite3_errmsg(m_db);
35- HCCL_VM_ERROR("[HcclDBSqlite::Connect] Failed: {}, path: {}", m_lastError, config.dbPath);35+ HCCL_VM_ERROR("Failed: {}, path: {}", m_lastError, config.dbPath);
36 sqlite3_close(m_db);36 sqlite3_close(m_db);
37 m_db = nullptr;37 m_db = nullptr;
38 return HCCL_SIM_E_OPEN_FILE_FAILURE;38 return HCCL_SIM_E_OPEN_FILE_FAILURE;
@@ -50,7 +50,7 @@ HcclVmResult HcclDBSqlite::Connect(const sim::DBConfig& config) {
50 sqlite3_exec(m_db, "PRAGMA wal_autocheckpoint = 200;", nullptr, nullptr, nullptr);50 sqlite3_exec(m_db, "PRAGMA wal_autocheckpoint = 200;", nullptr, nullptr, nullptr);
51 sqlite3_exec(m_db, "PRAGMA mmap_size = 268435456;", nullptr, nullptr, nullptr);51 sqlite3_exec(m_db, "PRAGMA mmap_size = 268435456;", nullptr, nullptr, nullptr);
52 52 
53- HCCL_VM_INFO("[HcclDBSqlite::Connect] Connected: {}", config.dbPath);53+ HCCL_VM_INFO("Connected: {}", config.dbPath);
54 return HCCL_SIM_SUCCESS;54 return HCCL_SIM_SUCCESS;
55}55}
56 56 
@@ -116,7 +116,7 @@ HcclVmResult HcclDBSqlite::Query(const std::string& sql, const std::vector<Value
116 int ret = sqlite3_prepare_v2(m_db, sql.c_str(), -1, &stmt, nullptr);116 int ret = sqlite3_prepare_v2(m_db, sql.c_str(), -1, &stmt, nullptr);
117 if (ret != SQLITE_OK) {117 if (ret != SQLITE_OK) {
118 m_lastError = sqlite3_errmsg(m_db);118 m_lastError = sqlite3_errmsg(m_db);
119- HCCL_VM_ERROR("[HcclDBSqlite::Query] Prepare failed: {}", m_lastError);119+ HCCL_VM_ERROR("Prepare failed: {}", m_lastError);
120 return HCCL_SIM_E_INTERNAL;120 return HCCL_SIM_E_INTERNAL;
121 }121 }
122 122 
@@ -148,7 +148,7 @@ HcclVmResult HcclDBSqlite::Query(const std::string& sql, const std::vector<Value
148 148 
149 if (ret != SQLITE_DONE) {149 if (ret != SQLITE_DONE) {
150 m_lastError = sqlite3_errmsg(m_db);150 m_lastError = sqlite3_errmsg(m_db);
151- HCCL_VM_ERROR("[HcclDBSqlite::Query] Step failed: {}", m_lastError);151+ HCCL_VM_ERROR("Step failed: {}", m_lastError);
152 return HCCL_SIM_E_INTERNAL;152 return HCCL_SIM_E_INTERNAL;
153 }153 }
154 154 
@@ -164,7 +164,7 @@ HcclVmResult HcclDBSqlite::QueryBlob(const std::string& sql, const std::vector<V
164 int ret = sqlite3_prepare_v2(m_db, sql.c_str(), -1, &stmt, nullptr);164 int ret = sqlite3_prepare_v2(m_db, sql.c_str(), -1, &stmt, nullptr);
165 if (ret != SQLITE_OK) {165 if (ret != SQLITE_OK) {
166 m_lastError = sqlite3_errmsg(m_db);166 m_lastError = sqlite3_errmsg(m_db);
167- HCCL_VM_ERROR("[HcclDBSqlite::QueryBlob] Prepare failed: {}", m_lastError);167+ HCCL_VM_ERROR("Prepare failed: {}", m_lastError);
168 return HCCL_SIM_E_INTERNAL;168 return HCCL_SIM_E_INTERNAL;
169 }169 }
170 170 
@@ -193,7 +193,7 @@ HcclVmResult HcclDBSqlite::QueryBlob(const std::string& sql, const std::vector<V
193 193 
194 if (ret != SQLITE_ROW && ret != SQLITE_DONE) {194 if (ret != SQLITE_ROW && ret != SQLITE_DONE) {
195 m_lastError = sqlite3_errmsg(m_db);195 m_lastError = sqlite3_errmsg(m_db);
196- HCCL_VM_ERROR("[HcclDBSqlite::QueryBlob] Step failed: {}", m_lastError);196+ HCCL_VM_ERROR("Step failed: {}", m_lastError);
197 return HCCL_SIM_E_INTERNAL;197 return HCCL_SIM_E_INTERNAL;
198 }198 }
199 199 
@@ -209,7 +209,7 @@ HcclVmResult HcclDBSqlite::QueryBlobColumns(const std::string& sql, const std::v
209 int ret = sqlite3_prepare_v2(m_db, sql.c_str(), -1, &stmt, nullptr);209 int ret = sqlite3_prepare_v2(m_db, sql.c_str(), -1, &stmt, nullptr);
210 if (ret != SQLITE_OK) {210 if (ret != SQLITE_OK) {
211 m_lastError = sqlite3_errmsg(m_db);211 m_lastError = sqlite3_errmsg(m_db);
212- HCCL_VM_ERROR("[HcclDBSqlite::QueryBlobColumns] Prepare failed: {}", m_lastError);212+ HCCL_VM_ERROR("Prepare failed: {}", m_lastError);
213 return HCCL_SIM_E_INTERNAL;213 return HCCL_SIM_E_INTERNAL;
214 }214 }
215 215 
@@ -242,7 +242,7 @@ HcclVmResult HcclDBSqlite::QueryBlobColumns(const std::string& sql, const std::v
242 242 
243 if (ret != SQLITE_ROW && ret != SQLITE_DONE) {243 if (ret != SQLITE_ROW && ret != SQLITE_DONE) {
244 m_lastError = sqlite3_errmsg(m_db);244 m_lastError = sqlite3_errmsg(m_db);
245- HCCL_VM_ERROR("[HcclDBSqlite::QueryBlobColumns] Step failed: {}", m_lastError);245+ HCCL_VM_ERROR("Step failed: {}", m_lastError);
246 return HCCL_SIM_E_INTERNAL;246 return HCCL_SIM_E_INTERNAL;
247 }247 }
248 248 
@@ -258,7 +258,7 @@ HcclVmResult HcclDBSqlite::QueryEx(const std::string& sql, const std::vector<Val
258 int ret = sqlite3_prepare_v2(m_db, sql.c_str(), -1, &stmt, nullptr);258 int ret = sqlite3_prepare_v2(m_db, sql.c_str(), -1, &stmt, nullptr);
259 if (ret != SQLITE_OK) {259 if (ret != SQLITE_OK) {
260 m_lastError = sqlite3_errmsg(m_db);260 m_lastError = sqlite3_errmsg(m_db);
261- HCCL_VM_ERROR("[HcclDBSqlite::QueryEx] Prepare failed: {}", m_lastError);261+ HCCL_VM_ERROR("Prepare failed: {}", m_lastError);
262 return HCCL_SIM_E_INTERNAL;262 return HCCL_SIM_E_INTERNAL;
263 }263 }
264 264 
@@ -306,7 +306,7 @@ HcclVmResult HcclDBSqlite::QueryEx(const std::string& sql, const std::vector<Val
306 306 
307 if (ret != SQLITE_DONE) {307 if (ret != SQLITE_DONE) {
308 m_lastError = sqlite3_errmsg(m_db);308 m_lastError = sqlite3_errmsg(m_db);
309- HCCL_VM_ERROR("[HcclDBSqlite::QueryEx] Step failed: {}", m_lastError);309+ HCCL_VM_ERROR("Step failed: {}", m_lastError);
310 return HCCL_SIM_E_INTERNAL;310 return HCCL_SIM_E_INTERNAL;
311 }311 }
312 312 
@@ -323,7 +323,7 @@ HcclVmResult HcclDBSqlite::Execute(const std::string& sql) {
323 323 
324 if (ret != SQLITE_OK) {324 if (ret != SQLITE_OK) {
325 m_lastError = errMsg ? errMsg : "Unknown error";325 m_lastError = errMsg ? errMsg : "Unknown error";
326- HCCL_VM_ERROR("[HcclDBSqlite::Execute] SQL error: {}, sql: {}", m_lastError, sql);326+ HCCL_VM_ERROR("SQL error: {}, sql: {}", m_lastError, sql);
327 if (errMsg) {327 if (errMsg) {
328 sqlite3_free(errMsg);328 sqlite3_free(errMsg);
329 }329 }
@@ -342,7 +342,7 @@ HcclVmResult HcclDBSqlite::Query(const std::string& sql, std::vector<std::vector
342 int ret = sqlite3_prepare_v2(m_db, sql.c_str(), -1, &stmt, nullptr);342 int ret = sqlite3_prepare_v2(m_db, sql.c_str(), -1, &stmt, nullptr);
343 if (ret != SQLITE_OK) {343 if (ret != SQLITE_OK) {
344 m_lastError = sqlite3_errmsg(m_db);344 m_lastError = sqlite3_errmsg(m_db);
345- HCCL_VM_ERROR("[HcclDBSqlite::Query] Prepare failed: {}", m_lastError);345+ HCCL_VM_ERROR("Prepare failed: {}", m_lastError);
346 return HCCL_SIM_E_INTERNAL;346 return HCCL_SIM_E_INTERNAL;
347 }347 }
348 348 
@@ -361,7 +361,7 @@ HcclVmResult HcclDBSqlite::Query(const std::string& sql, std::vector<std::vector
361 361 
362 if (ret != SQLITE_DONE) {362 if (ret != SQLITE_DONE) {
363 m_lastError = sqlite3_errmsg(m_db);363 m_lastError = sqlite3_errmsg(m_db);
364- HCCL_VM_ERROR("[HcclDBSqlite::Query] Step failed: {}", m_lastError);364+ HCCL_VM_ERROR("Step failed: {}", m_lastError);
365 return HCCL_SIM_E_INTERNAL;365 return HCCL_SIM_E_INTERNAL;
366 }366 }
367 367 
@@ -410,7 +410,7 @@ HcclVmResult HcclDBSqlite::Backup(const std::string& destPath) {
410 rc = sqlite3_open(destPath.c_str(), &pDest);410 rc = sqlite3_open(destPath.c_str(), &pDest);
411 if (rc != SQLITE_OK) {411 if (rc != SQLITE_OK) {
412 m_lastError = sqlite3_errstr(rc);412 m_lastError = sqlite3_errstr(rc);
413- HCCL_VM_ERROR("[HcclDBSqlite::Backup] Failed to open destination DB: {}", m_lastError);413+ HCCL_VM_ERROR("Failed to open destination DB: {}", m_lastError);
414 return HCCL_SIM_E_OPEN_FILE_FAILURE;414 return HCCL_SIM_E_OPEN_FILE_FAILURE;
415 }415 }
416 416 
@@ -428,18 +428,18 @@ HcclVmResult HcclDBSqlite::Backup(const std::string& destPath) {
428 } else {428 } else {
429 rc = sqlite3_errcode(pDest);429 rc = sqlite3_errcode(pDest);
430 m_lastError = sqlite3_errmsg(pDest);430 m_lastError = sqlite3_errmsg(pDest);
431- HCCL_VM_ERROR("[HcclDBSqlite::Backup] Backup init failed: {}", m_lastError);431+ HCCL_VM_ERROR("Backup init failed: {}", m_lastError);
432 }432 }
433 433 
434 // 5. Close destination434 // 5. Close destination
435 sqlite3_close(pDest);435 sqlite3_close(pDest);
436 436 
437 if (rc == SQLITE_OK) {437 if (rc == SQLITE_OK) {
438- HCCL_VM_INFO("[HcclDBSqlite::Backup] Backup completed successfully to: {}", destPath);438+ HCCL_VM_INFO("Backup completed successfully to: {}", destPath);
439 return HCCL_SIM_SUCCESS;439 return HCCL_SIM_SUCCESS;
440 } else {440 } else {
441 m_lastError = sqlite3_errstr(rc);441 m_lastError = sqlite3_errstr(rc);
442- HCCL_VM_ERROR("[HcclDBSqlite::Backup] Backup failed: {}", m_lastError);442+ HCCL_VM_ERROR("Backup failed: {}", m_lastError);
443 return HCCL_SIM_E_INTERNAL;443 return HCCL_SIM_E_INTERNAL;
444 }444 }
445}445}
@@ -453,7 +453,7 @@ HcclVmResult HcclDBSqlite::RunInTransaction(std::function<HcclVmResult()> fn) {
453 int rc = sqlite3_exec(m_db, "BEGIN IMMEDIATE", nullptr, nullptr, &errMsg);453 int rc = sqlite3_exec(m_db, "BEGIN IMMEDIATE", nullptr, nullptr, &errMsg);
454 if (rc != SQLITE_OK) {454 if (rc != SQLITE_OK) {
455 m_lastError = errMsg ? errMsg : "BEGIN IMMEDIATE failed";455 m_lastError = errMsg ? errMsg : "BEGIN IMMEDIATE failed";
456- HCCL_VM_ERROR("[HcclDBSqlite::RunInTransaction] BEGIN failed: {}", m_lastError);456+ HCCL_VM_ERROR("BEGIN failed: {}", m_lastError);
457 if (errMsg) {457 if (errMsg) {
458 sqlite3_free(errMsg);458 sqlite3_free(errMsg);
459 }459 }
@@ -466,7 +466,7 @@ HcclVmResult HcclDBSqlite::RunInTransaction(std::function<HcclVmResult()> fn) {
466 rc = sqlite3_exec(m_db, "COMMIT", nullptr, nullptr, &errMsg);466 rc = sqlite3_exec(m_db, "COMMIT", nullptr, nullptr, &errMsg);
467 if (rc != SQLITE_OK) {467 if (rc != SQLITE_OK) {
468 m_lastError = errMsg ? errMsg : "COMMIT failed";468 m_lastError = errMsg ? errMsg : "COMMIT failed";
469- HCCL_VM_ERROR("[HcclDBSqlite::RunInTransaction] COMMIT failed: {}, rolling back", m_lastError);469+ HCCL_VM_ERROR("COMMIT failed: {}, rolling back", m_lastError);
470 if (errMsg) {470 if (errMsg) {
471 sqlite3_free(errMsg);471 sqlite3_free(errMsg);
472 }472 }
Mtest/hccl_vm/src/runnerdb/modeldb/db_hccl_op_db_ops.cc+4-4
@@ -36,16 +36,16 @@ OpDbOps::OpDbOps()
36 36 
37 auto db = HcclSim::DB::HcclDBFactory::Instance().CreateDB(config.type);37 auto db = HcclSim::DB::HcclDBFactory::Instance().CreateDB(config.type);
38 if (!db || db->Connect(config) != HcclSim::HCCL_SIM_SUCCESS) {38 if (!db || db->Connect(config) != HcclSim::HCCL_SIM_SUCCESS) {
39- HCCL_VM_ERROR("[dbInit] Connect database failed");39+ HCCL_VM_ERROR("Connect database failed");
40 return;40 return;
41 }41 }
42 42 
43 SetDB(std::move(db));43 SetDB(std::move(db));
44 44 
45 if (InitSchema() != HcclSim::HCCL_SIM_SUCCESS) {45 if (InitSchema() != HcclSim::HCCL_SIM_SUCCESS) {
46- HCCL_VM_ERROR("[dbInit] Init schema failed");46+ HCCL_VM_ERROR("Init schema failed");
47 } else {47 } else {
48- HCCL_VM_INFO("[dbInit] OpDbOps tables initialized successfully");48+ HCCL_VM_INFO("OpDbOps tables initialized successfully");
49 }49 }
50}50}
51 51 
@@ -150,7 +150,7 @@ HcclVmResult OpDbOps::InitSchema() {
150 });150 });
151 151 
152 if (ret == HCCL_SIM_SUCCESS) {152 if (ret == HCCL_SIM_SUCCESS) {
153- HCCL_VM_INFO("[OpDbOps::InitSchema] All tables created");153+ HCCL_VM_INFO("All tables created");
154 }154 }
155 return ret;155 return ret;
156}156}
Mtest/hccl_vm/src/runnerdb/tracedb/db_sim_op_db_ops.cc+6-6
@@ -180,7 +180,7 @@ int UpdateAndInsertByCcuId(uint64_t& ccuId, uint32_t deviceId, uint32_t rankId,
180 ExRows rows;180 ExRows rows;
181 int ret = OpDbOps::Instance().ExecQueryEx(querySql, queryParams, rows);181 int ret = OpDbOps::Instance().ExecQueryEx(querySql, queryParams, rows);
182 if (ret != 0) {182 if (ret != 0) {
183- HCCL_VM_ERROR("[UpdateAndInsertByCcuId] query ccuInstrRes failed for deviceId: {} dieId: {}", deviceId, dieId);183+ HCCL_VM_ERROR("query ccuInstrRes failed for deviceId: {} dieId: {}", deviceId, dieId);
184 return -1;184 return -1;
185 }185 }
186 186 
@@ -303,14 +303,14 @@ int QueryCurrentOpMemInfoByRank(uint32_t rankId, OpMemInfoTab& out)
303 std::vector<Value> params = {(int64_t)g_currOpDetailId, (int64_t)rankId};303 std::vector<Value> params = {(int64_t)g_currOpDetailId, (int64_t)rankId};
304 std::vector<std::vector<std::string>> rows;304 std::vector<std::vector<std::string>> rows;
305 if (OpDbOps::Instance().ExecQuery(sql, params, rows) != 0 || rows.empty()) {305 if (OpDbOps::Instance().ExecQuery(sql, params, rows) != 0 || rows.empty()) {
306- HCCL_VM_ERROR("[QueryCurrentOpMemInfoByRank] query failed, currentOpDetailId={}, rankId={}",306+ HCCL_VM_ERROR("query failed, currentOpDetailId={}, rankId={}",
307 g_currOpDetailId, rankId);307 g_currOpDetailId, rankId);
308 return -1;308 return -1;
309 }309 }
310 310 
311 const auto& row = rows[0];311 const auto& row = rows[0];
312 if (row.size() < 8) {312 if (row.size() < 8) {
313- HCCL_VM_ERROR("[QueryCurrentOpMemInfoByRank] invalid row size={}, currentOpDetailId={}, rankId={}",313+ HCCL_VM_ERROR("invalid row size={}, currentOpDetailId={}, rankId={}",
314 row.size(), g_currOpDetailId, rankId);314 row.size(), g_currOpDetailId, rankId);
315 return -1;315 return -1;
316 }316 }
@@ -332,7 +332,7 @@ int UpdateOpExpansionMode(uint8_t mode)
332 std::vector<Value> params = {(int64_t)mode, (int64_t)g_currOpDetailId};332 std::vector<Value> params = {(int64_t)mode, (int64_t)g_currOpDetailId};
333 int ret = OpDbOps::Instance().ExecUpdate(sql, params);333 int ret = OpDbOps::Instance().ExecUpdate(sql, params);
334 if (ret != 0) {334 if (ret != 0) {
335- HCCL_VM_ERROR("[UpdateOpExpansionMode] Failed to update mode={}, id={}", mode, g_currOpDetailId);335+ HCCL_VM_ERROR("Failed to update mode={}, id={}", mode, g_currOpDetailId);
336 }336 }
337 return ret;337 return ret;
338}338}
@@ -343,7 +343,7 @@ int QueryLatestOpExpansionMode()
343 std::vector<Value> params = {(int64_t)g_currOpDetailId};343 std::vector<Value> params = {(int64_t)g_currOpDetailId};
344 std::vector<std::vector<std::string>> rows;344 std::vector<std::vector<std::string>> rows;
345 if (OpDbOps::Instance().ExecQuery(sql, params, rows) != 0 || rows.empty() || rows[0].empty()) {345 if (OpDbOps::Instance().ExecQuery(sql, params, rows) != 0 || rows.empty() || rows[0].empty()) {
346- HCCL_VM_ERROR("[QueryLatestOpExpansionMode] Failed to query, id={}", g_currOpDetailId);346+ HCCL_VM_ERROR("Failed to query, id={}", g_currOpDetailId);
347 return static_cast<int>(SimOpExpansionMode::SIM_OP_EXPANSION_MODE_RESERVED);347 return static_cast<int>(SimOpExpansionMode::SIM_OP_EXPANSION_MODE_RESERVED);
348 }348 }
349 return static_cast<int>(std::stoul(rows[0][0]));349 return static_cast<int>(std::stoul(rows[0][0]));
@@ -376,7 +376,7 @@ int QueryCompositeOpDetailBySyncIter(uint32_t syncIter, std::map<uint32_t, std::
376 std::vector<Value> opParams = {(int64_t)syncIter};376 std::vector<Value> opParams = {(int64_t)syncIter};
377 HcclSim::DB::ExRows opRows;377 HcclSim::DB::ExRows opRows;
378 if (OpDbOps::Instance().ExecQueryEx(opSql, opParams, opRows) != 0) {378 if (OpDbOps::Instance().ExecQueryEx(opSql, opParams, opRows) != 0) {
379- HCCL_VM_ERROR("[QueryCompositeOpDetailBySyncIter] Query opDetails failed for syncIter: {}", syncIter);379+ HCCL_VM_ERROR("Query opDetails failed for syncIter: {}", syncIter);
380 return -1;380 return -1;
381 }381 }
382 382 
Mtest/hccl_vm/src/runnerdb/tracedb/db_sim_runner_common.cc+3-3
@@ -371,7 +371,7 @@ std::set<uint64_t> GetUsedServerNum()
371 }371 }
372 return usedServerIds;372 return usedServerIds;
373 } catch (const std::exception &e) {373 } catch (const std::exception &e) {
374- HCCL_VM_ERROR("[GetUsedServerNum] exception: {}", e.what());374+ HCCL_VM_ERROR("exception: {}", e.what());
375 return {};375 return {};
376 }376 }
377}377}
@@ -385,13 +385,13 @@ bool GetRankIdByMPI(uint32_t &rankId, uint64_t &serverId)
385 auto usedServerIds = GetUsedServerNum();385 auto usedServerIds = GetUsedServerNum();
386 if (usedServerIds.size() == 1) {386 if (usedServerIds.size() == 1) {
387 if (*usedServerIds.begin() != 1) {387 if (*usedServerIds.begin() != 1) {
388- HCCL_VM_ERROR("[HVM] env OMPI_COMM_WORLD_RANK or PMI_RANK are not found, serverId is not 1.");388+ HCCL_VM_ERROR("env OMPI_COMM_WORLD_RANK or PMI_RANK are not found, serverId is not 1.");
389 return false;389 return false;
390 }390 }
391 serverId = 1;391 serverId = 1;
392 return true;392 return true;
393 }393 }
394- HCCL_VM_ERROR("[HVM] env OMPI_COMM_WORLD_RANK or PMI_RANK are not found, usedServerIds size is not 1.");394+ HCCL_VM_ERROR("env OMPI_COMM_WORLD_RANK or PMI_RANK are not found, usedServerIds size is not 1.");
395 return false;395 return false;
396 } else if (ompiRankStr != nullptr) {396 } else if (ompiRankStr != nullptr) {
397 rankId = static_cast<uint32_t>(atoi(ompiRankStr));397 rankId = static_cast<uint32_t>(atoi(ompiRankStr));
Mtest/hccl_vm/src/runnerdb/tracedb/db_sim_runner_ops.cc+5-5
@@ -60,12 +60,12 @@ bool InsertRunner(uint64_t serverKey)
60 g_runner.thread_id = pthread_self();60 g_runner.thread_id = pthread_self();
61 61 
62 g_cur_server_key = serverKey;62 g_cur_server_key = serverKey;
63- HCCL_VM_INFO("[InsertRunner] Init host ip= {}, server key= {}.", ret.first.ip_addr, g_cur_server_key);63+ HCCL_VM_INFO("Init host ip= {}, server key= {}.", ret.first.ip_addr, g_cur_server_key);
64 64 
65 g_runner.id = RunnerDB::Add<sim::Runner>(runner);65 g_runner.id = RunnerDB::Add<sim::Runner>(runner);
66 return true;66 return true;
67 } catch (const std::exception &e) {67 } catch (const std::exception &e) {
68- HCCL_VM_ERROR("[InsertRunner] SQLite exception: {}", e.what());68+ HCCL_VM_ERROR("SQLite exception: {}", e.what());
69 return false;69 return false;
70 }70 }
71}71}
@@ -93,7 +93,7 @@ bool SetCurrCtxTls(uint64_t ctx)
93 [currRunnerId, ctx](sim::Runner &runner) { runner.current_ctx_id = ctx; });93 [currRunnerId, ctx](sim::Runner &runner) { runner.current_ctx_id = ctx; });
94 return true;94 return true;
95 } catch (const std::exception &e) {95 } catch (const std::exception &e) {
96- HCCL_VM_ERROR("[SetCurrCtxTls] SQLite exception: {}", e.what());96+ HCCL_VM_ERROR("SQLite exception: {}", e.what());
97 return false;97 return false;
98 }98 }
99}99}
@@ -139,7 +139,7 @@ uint64_t GetCurrRankId()
139 return r.device_id == devKey;139 return r.device_id == devKey;
140 });140 });
141 if (!rank.second) {141 if (!rank.second) {
142- HCCL_VM_ERROR("[GetCurrRankId] can not find any rank");142+ HCCL_VM_ERROR("can not find any rank");
143 return 0;143 return 0;
144 }144 }
145 145 
@@ -216,7 +216,7 @@ uint64_t GetServerKeyById(uint32_t superPodIdx, uint32_t serverIdx)
216 return r.pod_id == superPodIdx && r.server_id == serverIdx;216 return r.pod_id == superPodIdx && r.server_id == serverIdx;
217 });217 });
218 if (!serverRet.second) {218 if (!serverRet.second) {
219- HCCL_VM_ERROR("[GetServerKeyById] can not find server by id: {:d}, {:d}", superPodIdx, serverIdx);219+ HCCL_VM_ERROR("can not find server by id: {:d}, {:d}", superPodIdx, serverIdx);
220 return 0;220 return 0;
221 }221 }
222 return serverRet.first.id;222 return serverRet.first.id;
Mtest/hccl_vm/src/store/CMakeLists.txt+3-0
@@ -46,3 +46,6 @@ target_include_directories(store PRIVATE
46)46)
47 47 
48target_link_libraries(store PUBLIC log loader)48target_link_libraries(store PUBLIC log loader)
49+ 
50+# 日志染色: 模块 tag
51+target_compile_definitions(store PRIVATE HCCL_VM_MODULE="STORE")
Mtest/hccl_vm/src/store/store_dump_shm_data.cc+13-13
@@ -53,7 +53,7 @@ std::string GenDataId()
53 int32_t retryCount = 10;53 int32_t retryCount = 10;
54 while (true) {54 while (true) {
55 if (retryCount == 0) {55 if (retryCount == 0) {
56- HCCL_VM_ERROR("[GenDataId] failed to gen dataId.");56+ HCCL_VM_ERROR("failed to gen dataId.");
57 break;57 break;
58 }58 }
59 retryCount--;59 retryCount--;
@@ -100,7 +100,7 @@ HcclVmResult DumpHcclVmFlagData(HcclSim::HcclVmFlagData &flagData)
100 100 
101 auto ret = HcclVmFlagDataWrite(fp, flagData);101 auto ret = HcclVmFlagDataWrite(fp, flagData);
102 if (ret != HcclVmResult::HCCL_SIM_SUCCESS) {102 if (ret != HcclVmResult::HCCL_SIM_SUCCESS) {
103- HCCL_VM_ERROR("[DumpHcclVmFlagData] HcclVmFlagDataWrite failed. ");103+ HCCL_VM_ERROR("HcclVmFlagDataWrite failed. ");
104 fclose(fp);104 fclose(fp);
105 return HcclVmResult::HCCL_SIM_E_INTERNAL;105 return HcclVmResult::HCCL_SIM_E_INTERNAL;
106 }106 }
@@ -123,7 +123,7 @@ HcclVmResult GetHcclVmFlagData(HcclSim::HcclVmFlagData &waitFlag)
123 123 
124 auto ret = HcclVmFlagDataRead(fp, waitFlag, HCCLVM_FLAG_FILE_MAGIC);124 auto ret = HcclVmFlagDataRead(fp, waitFlag, HCCLVM_FLAG_FILE_MAGIC);
125 if (ret != HcclVmResult::HCCL_SIM_SUCCESS) {125 if (ret != HcclVmResult::HCCL_SIM_SUCCESS) {
126- HCCL_VM_ERROR("[GetHcclVmFlagData] HcclVmFlagDataRead failed. ");126+ HCCL_VM_ERROR("HcclVmFlagDataRead failed. ");
127 fclose(fp);127 fclose(fp);
128 return HcclVmResult::HCCL_SIM_E_INTERNAL;128 return HcclVmResult::HCCL_SIM_E_INTERNAL;
129 }129 }
@@ -158,7 +158,7 @@ HcclVmResult DumpDataToFile(const std::string &dataId)
158 158 
159HcclVmResult CreateMemoryInfo(HcclVmSynData &hvmSynData, const sim::OpMemInfoTab &memInfo, uint32_t rankId)159HcclVmResult CreateMemoryInfo(HcclVmSynData &hvmSynData, const sim::OpMemInfoTab &memInfo, uint32_t rankId)
160{160{
161- HCCL_VM_INFO("[STUB] Enter into create memory info...");161+ HCCL_VM_INFO("Enter into create memory info...");
162 auto addBuf = [&](uint8_t bufType, uint64_t addr, uint64_t size) {162 auto addBuf = [&](uint8_t bufType, uint64_t addr, uint64_t size) {
163 if (addr != 0 && size > 0) {163 if (addr != 0 && size > 0) {
164 MemLayoutData memLayoutData{};164 MemLayoutData memLayoutData{};
@@ -175,14 +175,14 @@ HcclVmResult CreateMemoryInfo(HcclVmSynData &hvmSynData, const sim::OpMemInfoTab
175 addBuf(static_cast<uint8_t>(BufferType::CCL), memInfo.cclAddr, memInfo.cclSize);175 addBuf(static_cast<uint8_t>(BufferType::CCL), memInfo.cclAddr, memInfo.cclSize);
176 176 
177 hvmSynData.memory_info.count = hvmSynData.memory_info.data.size();177 hvmSynData.memory_info.count = hvmSynData.memory_info.data.size();
178- HCCL_VM_INFO("{}", hvmSynData.memory_info.count, hvmSynData.memory_info.data.size());178+ HCCL_VM_INFO("memory_info count={}, dataSize={}", hvmSynData.memory_info.count, hvmSynData.memory_info.data.size());
179 179 
180 return HcclVmResult::HCCL_SIM_SUCCESS;180 return HcclVmResult::HCCL_SIM_SUCCESS;
181}181}
182 182 
183HcclVmResult CreateChannelInfo(HcclVmSynData &hvmSynData)183HcclVmResult CreateChannelInfo(HcclVmSynData &hvmSynData)
184{184{
185- HCCL_VM_INFO("[STUB] Enter into create channel info...");185+ HCCL_VM_INFO("Enter into create channel info...");
186 186 
187 std::vector<sim::CcuChannelTab> channels;187 std::vector<sim::CcuChannelTab> channels;
188 loader::Loader dataLoader;188 loader::Loader dataLoader;
@@ -251,7 +251,7 @@ HcclVmResult CreateChannelInfo(HcclVmSynData &hvmSynData)
251 251 
252HcclVmResult CreateJettyInfo(HcclVmSynData &hvmSynData)252HcclVmResult CreateJettyInfo(HcclVmSynData &hvmSynData)
253{253{
254- HCCL_VM_INFO("[STUB] Enter into create jetty info...");254+ HCCL_VM_INFO("Enter into create jetty info...");
255 auto endpointPairs = RunnerDB::GetByPred<sim::EndPointPair>([](auto &&) { return true; });255 auto endpointPairs = RunnerDB::GetByPred<sim::EndPointPair>([](auto &&) { return true; });
256 hvmSynData.channel_info.count = endpointPairs.size();256 hvmSynData.channel_info.count = endpointPairs.size();
257 for (auto &endpoint : endpointPairs) {257 for (auto &endpoint : endpointPairs) {
@@ -334,7 +334,7 @@ HcclVmResult CreateJettyInfo(HcclVmSynData &hvmSynData)
334 334 
335HcclVmResult CreateSimSynData(HcclVmSynData &hvmSynData)335HcclVmResult CreateSimSynData(HcclVmSynData &hvmSynData)
336{336{
337- HCCL_VM_INFO("[CreateSimSynData] Start get simulator synthesis data...");337+ HCCL_VM_INFO("Start get simulator synthesis data...");
338 // header338 // header
339 hvmSynData.header.magic = HCCLVM_SYN_FILE_MAGIC;339 hvmSynData.header.magic = HCCLVM_SYN_FILE_MAGIC;
340 hvmSynData.header.version = 1;340 hvmSynData.header.version = 1;
@@ -345,7 +345,7 @@ HcclVmResult CreateSimSynData(HcclVmSynData &hvmSynData)
345 constexpr uint32_t kDumpSyncIter = 0; // 仅在 round 0 时 dump,syncIter 固定为 0345 constexpr uint32_t kDumpSyncIter = 0; // 仅在 round 0 时 dump,syncIter 固定为 0
346 loader::Loader dataLoader;346 loader::Loader dataLoader;
347 if (dataLoader.LoadCompositeOpDetailBySyncIter(kDumpSyncIter, compositeDataMap) != HcclResult::HCCL_SUCCESS) {347 if (dataLoader.LoadCompositeOpDetailBySyncIter(kDumpSyncIter, compositeDataMap) != HcclResult::HCCL_SUCCESS) {
348- HCCL_VM_ERROR("LoadCompositeOpDetailBySyncIter({}, {}) failed.", kDumpSyncIter);348+ HCCL_VM_ERROR("LoadCompositeOpDetailBySyncIter({}) failed.", kDumpSyncIter);
349 return HcclVmResult::HCCL_SIM_E_INTERNAL;349 return HcclVmResult::HCCL_SIM_E_INTERNAL;
350 }350 }
351 if (compositeDataMap.count(0) == 0 || compositeDataMap[0].empty()) {351 if (compositeDataMap.count(0) == 0 || compositeDataMap[0].empty()) {
@@ -415,7 +415,7 @@ HcclVmResult CreateSimSynData(HcclVmSynData &hvmSynData)
415 415 
416HcclVmResult DumpHcclVmSynthesisData(const std::string &dataId)416HcclVmResult DumpHcclVmSynthesisData(const std::string &dataId)
417{417{
418- HCCL_VM_INFO("[DumpHcclVmSynthesisData] Start dumping hccl vm synthesis data...");418+ HCCL_VM_INFO("Start dumping hccl vm synthesis data...");
419 // 1. 构造完整路径419 // 1. 构造完整路径
420 char fileName[256];420 char fileName[256];
421 snprintf(fileName, sizeof(fileName), HCCLVM_SYN_DATA_FILE.c_str(), dataId.c_str());421 snprintf(fileName, sizeof(fileName), HCCLVM_SYN_DATA_FILE.c_str(), dataId.c_str());
@@ -436,7 +436,7 @@ HcclVmResult DumpHcclVmSynthesisData(const std::string &dataId)
436 436 
437 FILE *fp = fopen(fullPath.c_str(), "wb");437 FILE *fp = fopen(fullPath.c_str(), "wb");
438 if (!fp) {438 if (!fp) {
439- HCCL_VM_ERROR("Open file failed: {}", fullPath, strerror(errno));439+ HCCL_VM_ERROR("Open file failed: {}, err: {}", fullPath, strerror(errno));
440 return HcclVmResult::HCCL_SIM_E_INTERNAL;440 return HcclVmResult::HCCL_SIM_E_INTERNAL;
441 }441 }
442 442 
@@ -511,7 +511,7 @@ HcclVmResult DumpHcclVmInstrData(const std::string &dataId)
511 std::string fullPath = InstallPath::ResolveToInstallRoot("data" + std::string(fileName));511 std::string fullPath = InstallPath::ResolveToInstallRoot("data" + std::string(fileName));
512 FILE *fp = fopen(fullPath.c_str(), "wb");512 FILE *fp = fopen(fullPath.c_str(), "wb");
513 if (!fp) {513 if (!fp) {
514- HCCL_VM_ERROR("Open file failed: {}", fullPath, strerror(errno));514+ HCCL_VM_ERROR("Open file failed: {}, err: {}", fullPath, strerror(errno));
515 return HcclVmResult::HCCL_SIM_E_INTERNAL;515 return HcclVmResult::HCCL_SIM_E_INTERNAL;
516 }516 }
517 517 
@@ -601,7 +601,7 @@ HcclVmResult DumpHcclVmTask(const std::string &dataId)
601 601 
602 FILE *fp = fopen(fullPath.c_str(), "wb");602 FILE *fp = fopen(fullPath.c_str(), "wb");
603 if (!fp) {603 if (!fp) {
604- HCCL_VM_ERROR("Open file failed: {}", fullPath, strerror(errno));604+ HCCL_VM_ERROR("Open file failed: {}, err: {}", fullPath, strerror(errno));
605 return HcclVmResult::HCCL_SIM_E_INTERNAL;605 return HcclVmResult::HCCL_SIM_E_INTERNAL;
606 }606 }
607 607 
Mtest/hccl_vm/src/store/store_sim_comm_memory_manager.cc+15-15
@@ -48,7 +48,7 @@ CommunicationMemoryManager::~CommunicationMemoryManager()
48void* CommunicationMemoryManager::AllocCommMem(const char* name) 48void* CommunicationMemoryManager::AllocCommMem(const char* name)
49{49{
50 if (!name) {50 if (!name) {
51- HCCL_VM_ERROR("[COMM_MEM] acquire name is null");51+ HCCL_VM_ERROR("acquire name is null");
52 return nullptr;52 return nullptr;
53 }53 }
54 54 
@@ -56,7 +56,7 @@ void* CommunicationMemoryManager::AllocCommMem(const char* name)
56 56 
57 void* ptr = MemoryManager::GetInstance().AllocMemByName(name, sizeof(CommMemHead));57 void* ptr = MemoryManager::GetInstance().AllocMemByName(name, sizeof(CommMemHead));
58 if (ptr == nullptr) {58 if (ptr == nullptr) {
59- HCCL_VM_ERROR("[COMM_MEM] alloc failed, name: {}", name);59+ HCCL_VM_ERROR("alloc failed, name: {}", name);
60 return nullptr;60 return nullptr;
61 }61 }
62 62 
@@ -64,7 +64,7 @@ void* CommunicationMemoryManager::AllocCommMem(const char* name)
64 head->bufferSize = 0;64 head->bufferSize = 0;
65 head->writeTotalBytes = 0;65 head->writeTotalBytes = 0;
66 head->readTotalBytes = 0;66 head->readTotalBytes = 0;
67- HCCL_VM_INFO("[COMM_MEM] acquire name: {}, ptr: {:p}", name, ptr);67+ HCCL_VM_INFO("acquire name: {}, ptr: {:p}", name, ptr);
68 m_commMemMap[name] = ptr;68 m_commMemMap[name] = ptr;
69 return head->data;69 return head->data;
70}70}
@@ -72,7 +72,7 @@ void* CommunicationMemoryManager::AllocCommMem(const char* name)
72void* CommunicationMemoryManager::AcquireCommMem(const char* name) 72void* CommunicationMemoryManager::AcquireCommMem(const char* name)
73{73{
74 if (!name) {74 if (!name) {
75- HCCL_VM_ERROR("[COMM_MEM] acquire name is null");75+ HCCL_VM_ERROR("acquire name is null");
76 return nullptr;76 return nullptr;
77 }77 }
78 void* ptr = nullptr;78 void* ptr = nullptr;
@@ -83,7 +83,7 @@ void* CommunicationMemoryManager::AcquireCommMem(const char* name)
83 } else {83 } else {
84 ptr = MemoryManager::GetInstance().AcquireMemByName(name);84 ptr = MemoryManager::GetInstance().AcquireMemByName(name);
85 if (ptr == nullptr) {85 if (ptr == nullptr) {
86- HCCL_VM_ERROR("[COMM_MEM] acquire failed, name: {}", name);86+ HCCL_VM_ERROR("acquire failed, name: {}", name);
87 return nullptr;87 return nullptr;
88 }88 }
89 m_commMemMap[name] = ptr;89 m_commMemMap[name] = ptr;
@@ -95,17 +95,17 @@ void* CommunicationMemoryManager::AcquireCommMem(const char* name)
95 95 
96int CommunicationMemoryManager::ReleaseCommMem(const char* name) {96int CommunicationMemoryManager::ReleaseCommMem(const char* name) {
97 if (!name) {97 if (!name) {
98- HCCL_VM_ERROR("[COMM_MEM] release name is nullptr");98+ HCCL_VM_ERROR("release name is nullptr");
99 return -1;99 return -1;
100 }100 }
101 101 
102 std::lock_guard<std::mutex> lock(m_mutex);102 std::lock_guard<std::mutex> lock(m_mutex);
103 if (m_commMemMap.find(name) == m_commMemMap.end()) {103 if (m_commMemMap.find(name) == m_commMemMap.end()) {
104- HCCL_VM_ERROR("[COMM_MEM] release comm mem not found, name: {}", name);104+ HCCL_VM_ERROR("release comm mem not found, name: {}", name);
105 return -1;105 return -1;
106 }106 }
107 m_commMemMap.erase(name);107 m_commMemMap.erase(name);
108- HCCL_VM_INFO("[COMM_MEM] release name: {}", name);108+ HCCL_VM_INFO("release name: {}", name);
109 MemoryManager::GetInstance().ReleaseMemByName(name);109 MemoryManager::GetInstance().ReleaseMemByName(name);
110 return 0;110 return 0;
111}111}
@@ -113,13 +113,13 @@ int CommunicationMemoryManager::ReleaseCommMem(const char* name) {
113int CommunicationMemoryManager::WriteCommMem(const char* name, const void* dataPtr, size_t size)113int CommunicationMemoryManager::WriteCommMem(const char* name, const void* dataPtr, size_t size)
114{114{
115 if (!name || !dataPtr || size == 0) {115 if (!name || !dataPtr || size == 0) {
116- HCCL_VM_ERROR("[COMM_MEM] write invalid params");116+ HCCL_VM_ERROR("write invalid params, name: {}, dataPtr: {}, size: {}", name, dataPtr, size);
117 return -1;117 return -1;
118 }118 }
119 119 
120 std::lock_guard<std::mutex> lock(m_mutex);120 std::lock_guard<std::mutex> lock(m_mutex);
121 if (m_commMemMap.find(name) == m_commMemMap.end()) {121 if (m_commMemMap.find(name) == m_commMemMap.end()) {
122- HCCL_VM_ERROR("[COMM_MEM] write comm mem not found, name: {}", name);122+ HCCL_VM_ERROR("write comm mem not found, name: {}", name);
123 return -1;123 return -1;
124 }124 }
125 void* ptr = m_commMemMap[name];125 void* ptr = m_commMemMap[name];
@@ -129,7 +129,7 @@ int CommunicationMemoryManager::WriteCommMem(const char* name, const void* dataP
129 129 
130 if (size + head->bufferSize > RA_SOCKET_BUF_SIZE) {130 if (size + head->bufferSize > RA_SOCKET_BUF_SIZE) {
131 MemoryManager::GetInstance().UnlockMemByName(name);131 MemoryManager::GetInstance().UnlockMemByName(name);
132- HCCL_VM_ERROR("[COMM_MEM] write size too large, size: {}, max: {}", size, RA_SOCKET_BUF_SIZE);132+ HCCL_VM_ERROR("write size too large, size: {}, max: {}", size, RA_SOCKET_BUF_SIZE);
133 return -1;133 return -1;
134 }134 }
135 135 
@@ -139,20 +139,20 @@ int CommunicationMemoryManager::WriteCommMem(const char* name, const void* dataP
139 uint64_t totalWrite = head->writeTotalBytes;139 uint64_t totalWrite = head->writeTotalBytes;
140 140 
141 MemoryManager::GetInstance().UnlockMemByName(name);141 MemoryManager::GetInstance().UnlockMemByName(name);
142- HCCL_VM_INFO("[COMM_MEM] write name: {}, size: {}, totalWriteBytes: {}", name, size, totalWrite);142+ HCCL_VM_INFO("write name: {}, size: {}, totalWriteBytes: {}", name, size, totalWrite);
143 return 0;143 return 0;
144}144}
145 145 
146int CommunicationMemoryManager::ReadCommMem(const char* name, void* dataPtr, size_t size)146int CommunicationMemoryManager::ReadCommMem(const char* name, void* dataPtr, size_t size)
147{147{
148 if (!name || !dataPtr || size == 0) {148 if (!name || !dataPtr || size == 0) {
149- HCCL_VM_ERROR("[COMM_MEM] read invalid params");149+ HCCL_VM_ERROR("read invalid params, name: {}, dataPtr: {}, size: {}", name, dataPtr, size);
150 return -1;150 return -1;
151 }151 }
152 152 
153 std::lock_guard<std::mutex> lock(m_mutex);153 std::lock_guard<std::mutex> lock(m_mutex);
154 if (m_commMemMap.find(name) == m_commMemMap.end()) {154 if (m_commMemMap.find(name) == m_commMemMap.end()) {
155- HCCL_VM_ERROR("[COMM_MEM] read comm mem not found, name: {}", name);155+ HCCL_VM_ERROR("read comm mem not found, name: {}", name);
156 return -1;156 return -1;
157 }157 }
158 void* ptr = m_commMemMap[name];158 void* ptr = m_commMemMap[name];
@@ -177,7 +177,7 @@ int CommunicationMemoryManager::ReadCommMem(const char* name, void* dataPtr, siz
177 177 
178 MemoryManager::GetInstance().UnlockMemByName(name);178 MemoryManager::GetInstance().UnlockMemByName(name);
179 179 
180- HCCL_VM_INFO("[COMM_MEM] read name: {}, size: {}, totalReadBytes: {}", name, recvSize, totalRead);180+ HCCL_VM_INFO("read name: {}, size: {}, totalReadBytes: {}", name, recvSize, totalRead);
181 return recvSize;181 return recvSize;
182}182}
183 183 
Mtest/hccl_vm/src/store/store_sim_memory_manager.cc+4-4
@@ -65,7 +65,7 @@ void* MemoryManager::AllocMemByName(const char* name, size_t size) {
65 memInfo.size = size;65 memInfo.size = size;
66 memInfo.type = 0; // 0: 共享内存66 memInfo.type = 0; // 0: 共享内存
67 memInfo.refCount = 1;67 memInfo.refCount = 1;
68- HCCL_VM_INFO("[MEM_MANAGER] alloc name: {}, size: {:d}, ptr: {:p}", name, size, ptr);68+ HCCL_VM_INFO("alloc name: {}, size: {:d}, ptr: {:p}", name, size, ptr);
69 m_memMap[nameStr] = memInfo;69 m_memMap[nameStr] = memInfo;
70 return ptr;70 return ptr;
71}71}
@@ -81,7 +81,7 @@ void MemoryManager::FreeMemByName(const char* name) {
81 auto it = m_memMap.find(nameStr);81 auto it = m_memMap.find(nameStr);
82 if (it != m_memMap.end()) {82 if (it != m_memMap.end()) {
83 // 关闭共享内存83 // 关闭共享内存
84- HCCL_VM_INFO("[MEM_MANAGER] free name: {}, size: {:d}, ptr: {:p}", name, it->second.size, it->second.ptr);84+ HCCL_VM_INFO("free name: {}, size: {:d}, ptr: {:p}", name, it->second.size, it->second.ptr);
85 ShmClose(it->second.ptr);85 ShmClose(it->second.ptr);
86 // 从映射表中移除86 // 从映射表中移除
87 m_memMap.erase(it);87 m_memMap.erase(it);
@@ -145,7 +145,7 @@ void* MemoryManager::AcquireMemByName(const char* name) {
145 memInfo.size = size; // 使用ShmOpen返回的大小145 memInfo.size = size; // 使用ShmOpen返回的大小
146 memInfo.type = 0; // 0: 共享内存146 memInfo.type = 0; // 0: 共享内存
147 memInfo.refCount = 1;147 memInfo.refCount = 1;
148- HCCL_VM_INFO("[MEM_MANAGER] acquire name: {}, size: {:d}, ptr: {:p}", name, size, ptr);148+ HCCL_VM_INFO("acquire name: {}, size: {:d}, ptr: {:p}", name, size, ptr);
149 m_memMap[nameStr] = memInfo;149 m_memMap[nameStr] = memInfo;
150 return ptr;150 return ptr;
151}151}
@@ -164,7 +164,7 @@ void MemoryManager::ReleaseMemByName(const char* name) {
164 int refCount = __sync_fetch_and_sub(&it->second.refCount, 1);164 int refCount = __sync_fetch_and_sub(&it->second.refCount, 1);
165 if (refCount == 1) {165 if (refCount == 1) {
166 // 引用计数归0,关闭共享内存并从映射表中移除166 // 引用计数归0,关闭共享内存并从映射表中移除
167- HCCL_VM_INFO("[MEM_MANAGER] release name: {}, ptr: {:p}", name, it->second.ptr);167+ HCCL_VM_INFO("release name: {}, ptr: {:p}", name, it->second.ptr);
168 ShmClose(it->second.ptr);168 ShmClose(it->second.ptr);
169 m_memMap.erase(it);169 m_memMap.erase(it);
170 }170 }
Mtest/hccl_vm/src/store/store_sim_shm_ops.cc+14-14
@@ -92,13 +92,13 @@ static uint64_t FromLe64(uint64_t value) {
92 92 
93void* ShmCreate(const char* name, size_t size) {93void* ShmCreate(const char* name, size_t size) {
94 if (!name || size == 0) {94 if (!name || size == 0) {
95- HCCL_VM_ERROR("[SHM_OPS] create: name is nullptr or size is 0");95+ HCCL_VM_ERROR("create: name is nullptr or size is 0");
96 return nullptr;96 return nullptr;
97 }97 }
98 98 
99 int shmFd = shm_open(name, O_CREAT | O_RDWR | O_EXCL, 0666);99 int shmFd = shm_open(name, O_CREAT | O_RDWR | O_EXCL, 0666);
100 if (shmFd == -1) {100 if (shmFd == -1) {
101- HCCL_VM_ERROR("[SHM_OPS] create: shm_open failed, name: {}", name);101+ HCCL_VM_ERROR("create: shm_open failed, name: {}", name);
102 return nullptr;102 return nullptr;
103 }103 }
104 104 
@@ -106,7 +106,7 @@ void* ShmCreate(const char* name, size_t size) {
106 if (ftruncate(shmFd, totalSize) == -1) {106 if (ftruncate(shmFd, totalSize) == -1) {
107 close(shmFd);107 close(shmFd);
108 shm_unlink(name);108 shm_unlink(name);
109- HCCL_VM_ERROR("[SHM_OPS] create: ftruncate failed, name: {}", name);109+ HCCL_VM_ERROR("create: ftruncate failed, name: {}", name);
110 return nullptr;110 return nullptr;
111 }111 }
112 112 
@@ -115,7 +115,7 @@ void* ShmCreate(const char* name, size_t size) {
115 115 
116 if (addr == MAP_FAILED) {116 if (addr == MAP_FAILED) {
117 shm_unlink(name);117 shm_unlink(name);
118- HCCL_VM_ERROR("[SHM_OPS] create: mmap failed, name: {}", name);118+ HCCL_VM_ERROR("create: mmap failed, name: {}", name);
119 return nullptr;119 return nullptr;
120 }120 }
121 121 
@@ -127,25 +127,25 @@ void* ShmCreate(const char* name, size_t size) {
127 head->size = ToLe64(size);127 head->size = ToLe64(size);
128 head->lock = 0;128 head->lock = 0;
129 head->refCount = 1;129 head->refCount = 1;
130- HCCL_VM_INFO("[SHM_OPS] create name: {}, size: {:d}, ptr: {:p} ref:{:d}", head->name, size, (char*)addr + SHM_HEAD_SIZE, (int)head->refCount);130+ HCCL_VM_INFO("create name: {}, size: {:d}, ptr: {:p} ref:{:d}", head->name, size, (char*)addr + SHM_HEAD_SIZE, (int)head->refCount);
131 return (char*)addr + SHM_HEAD_SIZE;131 return (char*)addr + SHM_HEAD_SIZE;
132}132}
133 133 
134void* ShmOpen(const char* name, size_t* size) {134void* ShmOpen(const char* name, size_t* size) {
135 if (!name) {135 if (!name) {
136- HCCL_VM_ERROR("[SHM_OPS] open: name is nullptr");136+ HCCL_VM_ERROR("open: name is nullptr");
137 return nullptr;137 return nullptr;
138 }138 }
139 139 
140 int shmFd = shm_open(name, O_RDWR, 0666);140 int shmFd = shm_open(name, O_RDWR, 0666);
141 if (shmFd == -1) {141 if (shmFd == -1) {
142- HCCL_VM_ERROR("[SHM_OPS] open: shm_open failed, name: {}", name);142+ HCCL_VM_ERROR("open: shm_open failed, name: {}", name);
143 return nullptr;143 return nullptr;
144 }144 }
145 145 
146 struct stat statBuf;146 struct stat statBuf;
147 if (fstat(shmFd, &statBuf) == -1) {147 if (fstat(shmFd, &statBuf) == -1) {
148- HCCL_VM_ERROR("[SHM_OPS] open: fstat failed, name: {}", name);148+ HCCL_VM_ERROR("open: fstat failed, name: {}", name);
149 close(shmFd);149 close(shmFd);
150 return nullptr;150 return nullptr;
151 }151 }
@@ -154,13 +154,13 @@ void* ShmOpen(const char* name, size_t* size) {
154 close(shmFd);154 close(shmFd);
155 155 
156 if (addr == MAP_FAILED) { 156 if (addr == MAP_FAILED) {
157- HCCL_VM_ERROR("[SHM_OPS] open: mmap failed, name: {}", name);157+ HCCL_VM_ERROR("open: mmap failed, name: {}", name);
158 return nullptr;158 return nullptr;
159 }159 }
160 160 
161 ShmHead* head = (ShmHead*)addr;161 ShmHead* head = (ShmHead*)addr;
162 if (FromLe32(head->magic) != SHM_MAGIC) {162 if (FromLe32(head->magic) != SHM_MAGIC) {
163- HCCL_VM_ERROR("[SHM_OPS] open: invalid magic number, name: {}", name);163+ HCCL_VM_ERROR("open: invalid magic number, name: {}", name);
164 munmap(addr, statBuf.st_size);164 munmap(addr, statBuf.st_size);
165 return nullptr;165 return nullptr;
166 }166 }
@@ -173,14 +173,14 @@ void* ShmOpen(const char* name, size_t* size) {
173 current_count = FromLe32(head->refCount);173 current_count = FromLe32(head->refCount);
174 new_count = current_count + 1;174 new_count = current_count + 1;
175 }175 }
176- HCCL_VM_INFO("[SHM_OPS] open name: {}, size: {:d}, ptr: {:p}", head->name, *size, (char*)addr + SHM_HEAD_SIZE);176+ HCCL_VM_INFO("open name: {}, size: {:d}, ptr: {:p}", head->name, *size, (char*)addr + SHM_HEAD_SIZE);
177 return (void*)((char*)addr + SHM_HEAD_SIZE);177 return (void*)((char*)addr + SHM_HEAD_SIZE);
178}178}
179 179 
180void ShmClose(void* shm)180void ShmClose(void* shm)
181{181{
182 if (!shm) {182 if (!shm) {
183- HCCL_VM_ERROR("[SHM_OPS] close: shm is nullptr");183+ HCCL_VM_ERROR("close: shm is nullptr");
184 return;184 return;
185 }185 }
186 186 
@@ -195,10 +195,10 @@ void ShmClose(void* shm)
195 munmap(head, totalSize);195 munmap(head, totalSize);
196 if (refCount == 1) {196 if (refCount == 1) {
197 // 最后一个引用,删除共享内存197 // 最后一个引用,删除共享内存
198- HCCL_VM_INFO("[SHM_OPS] close name: {}, ptr: {:p}", tmp, (char*)head);198+ HCCL_VM_INFO("close name: {}, ptr: {:p}", tmp, (char*)head);
199 shm_unlink(tmp);199 shm_unlink(tmp);
200 } else {200 } else {
201- HCCL_VM_INFO("[SHM_OPS] close name: {}, ptr: {:p}, refCount: {} {}", tmp, (char*)head, refCount, ref);201+ HCCL_VM_INFO("close name: {}, ptr: {:p}, refCount: {} {}", tmp, (char*)head, refCount, ref);
202 }202 }
203}203}
204 204 
Mtest/hccl_vm/src/store/store_sim_store_pub.cc+6-6
@@ -32,7 +32,7 @@ HcclVmResult GetAddrByOffset(uint64_t offset, VmUniquePtr& addrPtr) // todo
32{32{
33 // 1. 入参合法性检查(避免空指针和无效输出)33 // 1. 入参合法性检查(避免空指针和无效输出)
34 if (addrPtr) {34 if (addrPtr) {
35- HCCL_VM_ERROR("[GetAddrByOffset] 错误:指针已持有资源,不可重复获取!");35+ HCCL_VM_ERROR("错误:指针已持有资源,不可重复获取!");
36 return HcclVmResult::HCCL_SIM_E_PARA;36 return HcclVmResult::HCCL_SIM_E_PARA;
37 }37 }
38 38 
@@ -40,12 +40,12 @@ HcclVmResult GetAddrByOffset(uint64_t offset, VmUniquePtr& addrPtr) // todo
40 sim::PhyMemBlock phyMem{};40 sim::PhyMemBlock phyMem{};
41 void *addr = sim::AcquireDevPtrInNoHostProcess(offserPtr, phyMem);41 void *addr = sim::AcquireDevPtrInNoHostProcess(offserPtr, phyMem);
42 if (addr == nullptr) {42 if (addr == nullptr) {
43- HCCL_VM_ERROR("[GetAddrByOffset] 错误:无法获取设备地址(addr= {})!", offserPtr);43+ HCCL_VM_ERROR("错误:无法获取设备地址(addr= {})!", offserPtr);
44 return HcclVmResult::HCCL_SIM_E_PTR;44 return HcclVmResult::HCCL_SIM_E_PTR;
45 }45 }
46 46 
47 // 5. 日志打印(可选,便于问题排查)47 // 5. 日志打印(可选,便于问题排查)
48- HCCL_VM_INFO("[GetAddrByOffset] 计算成功, 虚拟地址: {}, 物理地址: {}", offset, addr);48+ HCCL_VM_INFO("计算成功, 虚拟地址: {}, 物理地址: {}", offset, addr);
49 49 
50 addrPtr = VmUniquePtr(addr, VmPtrReleaser{phyMem});50 addrPtr = VmUniquePtr(addr, VmPtrReleaser{phyMem});
51 51 
@@ -58,11 +58,11 @@ HcclVmResult InsertTaskToCollection(HcclTaskMetaData *task, uint32_t *index)
58 58 
59 // 1. 入参合法性检查(避免空指针访问)59 // 1. 入参合法性检查(避免空指针访问)
60 if (task == nullptr) {60 if (task == nullptr) {
61- HCCL_VM_ERROR("[InsertTaskToCollection] 错误:输入任务指针 task 不能为空!");61+ HCCL_VM_ERROR("错误:输入任务指针 task 不能为空!");
62 return HcclVmResult::HCCL_SIM_E_PARA;62 return HcclVmResult::HCCL_SIM_E_PARA;
63 }63 }
64 if (index == nullptr) {64 if (index == nullptr) {
65- HCCL_VM_ERROR("[InsertTaskToCollection] 错误:输出索引指针 index 不能为空!");65+ HCCL_VM_ERROR("错误:输出索引指针 index 不能为空!");
66 return HcclVmResult::HCCL_SIM_E_PARA;66 return HcclVmResult::HCCL_SIM_E_PARA;
67 }67 }
68 68 
@@ -81,7 +81,7 @@ HcclVmResult InsertTaskToCollection(HcclTaskMetaData *task, uint32_t *index)
81 81
82 auto ret = sim::InsertOpTask(opTaskInfo);82 auto ret = sim::InsertOpTask(opTaskInfo);
83 if (ret != 0) {83 if (ret != 0) {
84- HCCL_VM_ERROR("[InsertTaskToCollection] 错误:插入任务到数据库失败 - {}", ret);84+ HCCL_VM_ERROR("错误:插入任务到数据库失败 - {}", ret);
85 return HcclVmResult::HCCL_SIM_SHM_FAIL;85 return HcclVmResult::HCCL_SIM_SHM_FAIL;
86 }86 }
87 return HcclVmResult::HCCL_SIM_SUCCESS;87 return HcclVmResult::HCCL_SIM_SUCCESS;
Mtest/hccl_vm/src/topo/CMakeLists.txt+3-0
@@ -40,3 +40,6 @@ target_compile_options(topo PRIVATE
40)40)
41 41 
42target_link_libraries(topo PRIVATE log)42target_link_libraries(topo PRIVATE log)
43+ 
44+# 日志染色: 模块 tag
45+target_compile_definitions(topo PRIVATE HCCL_VM_MODULE="TOPO")
Mtest/hccl_vm/src/topo/topo_ascend_cluster_parser.cc+6-9
@@ -11,6 +11,7 @@
11#include "topo_ascend_cluster_parser.h"11#include "topo_ascend_cluster_parser.h"
12 12 
13#include <cstdint>13#include <cstdint>
14+#include <cstdlib>
14#include <fstream>15#include <fstream>
15#include <iomanip>16#include <iomanip>
16#include <map>17#include <map>
@@ -137,7 +138,7 @@ HcclVmResult AscendClusterTopoParser::InitClusterTopo(const std::string &cluster
137 */138 */
138HcclVmResult AscendClusterTopoParser::ParseRanktableAndInitCommDomain(const std::string &ranktablePath)139HcclVmResult AscendClusterTopoParser::ParseRanktableAndInitCommDomain(const std::string &ranktablePath)
139{140{
140- HCCL_VM_DEBUG("Enter ParseRanktableAndInitCommDomain, status: {}");141+ HCCL_VM_DEBUG("Enter ParseRanktableAndInitCommDomain, ranktablePath: {}", ranktablePath);
141 142 
142 TopoMeta topoMeta;143 TopoMeta topoMeta;
143 if (ParseRanktable(ranktablePath, topoMeta) != HcclVmResult::HCCL_SIM_SUCCESS) {144 if (ParseRanktable(ranktablePath, topoMeta) != HcclVmResult::HCCL_SIM_SUCCESS) {
@@ -416,13 +417,7 @@ HcclVmResult AscendClusterTopoParser::CreateRankTableFile(const TopoMeta &topoMe
416 }417 }
417 }418 }
418 419 
419- std::error_code ec;420+ fs::path dataDir = fs::path(InstallPath::ResolveToInstallRoot("data"));
420- fs::path exePath = fs::read_symlink("/proc/self/exe", ec);
421- fs::path binDir = ec ? fs::current_path() : exePath.parent_path();
422- if (binDir.filename() == "bin") {
423- binDir = binDir.parent_path();
424- }
425- fs::path dataDir = binDir / "data";
426 std::error_code ec2;421 std::error_code ec2;
427 fs::create_directories(dataDir, ec2);422 fs::create_directories(dataDir, ec2);
428 std::string outputPath = (dataDir / "ranktable.json").string();423 std::string outputPath = (dataDir / "ranktable.json").string();
@@ -435,6 +430,8 @@ HcclVmResult AscendClusterTopoParser::CreateRankTableFile(const TopoMeta &topoMe
435 ofs << rankTable.dump(4) << std::endl;430 ofs << rankTable.dump(4) << std::endl;
436 ofs.close();431 ofs.close();
437 432 
433+ setenv("RANK_TABLE_FILE", outputPath.c_str(), 1);
434+ 
438 HCCL_VM_DEBUG("[{}] ranktable.json generated with {:d} ranks", __func__, rankId);435 HCCL_VM_DEBUG("[{}] ranktable.json generated with {:d} ranks", __func__, rankId);
439 return HcclVmResult::HCCL_SIM_SUCCESS;436 return HcclVmResult::HCCL_SIM_SUCCESS;
440}437}
@@ -685,7 +682,7 @@ HcclVmResult AscendClusterTopoParser::ParseRanktable(const std::string &ranktabl
685 !network_.superPods[0].servers[0].devices.empty()) {682 !network_.superPods[0].servers[0].devices.empty()) {
686 }683 }
687 684 
688- HCCL_VM_DEBUG("[{}] parsed ranktable: podNum={}, serverCount={}, rankNum={}",685+ HCCL_VM_DEBUG("[{}] parsed ranktable: topoMetaSize={}",
689 __func__, topoMeta.size());686 __func__, topoMeta.size());
690 return HcclVmResult::HCCL_SIM_SUCCESS;687 return HcclVmResult::HCCL_SIM_SUCCESS;
691}688}
Mtest/hccl_vm/src/topo/topo_cluster_dumper.cc+3-2
@@ -12,6 +12,7 @@
12#include <iostream>12#include <iostream>
13 13 
14#include "topo_cluster_dumper.h"14#include "topo_cluster_dumper.h"
15+#include "sim_log.h"
15 16 
16std::string ClusterTopoDumper::Indent(int level) {17std::string ClusterTopoDumper::Indent(int level) {
17 return std::string(level * 2, ' ');18 return std::string(level * 2, ' ');
@@ -20,12 +21,12 @@ std::string ClusterTopoDumper::Indent(int level) {
20void ClusterTopoDumper::DumpToFile(const Network& network, const std::string& filePath) {21void ClusterTopoDumper::DumpToFile(const Network& network, const std::string& filePath) {
21 std::ofstream ofs(filePath);22 std::ofstream ofs(filePath);
22 if (!ofs.is_open()) {23 if (!ofs.is_open()) {
23- fprintf(stderr, "Error: cannot open file for writing: %s\n", filePath.c_str());24+ HCCL_VM_ERROR("cannot open file for writing: {}", filePath.c_str());
24 return;25 return;
25 }26 }
26 DumpToStream(network, ofs);27 DumpToStream(network, ofs);
27 ofs.close();28 ofs.close();
28- printf("IR data dumped to: %s\n", filePath.c_str());29+ HCCL_VM_INFO("IR data dumped to: {}", filePath.c_str());
29}30}
30 31 
31void ClusterTopoDumper::DumpToStream(const Network& network, std::ostream& os) {32void ClusterTopoDumper::DumpToStream(const Network& network, std::ostream& os) {
Mtest/hccl_vm/src/topo/topo_cluster_parser.cc+11-10
@@ -18,6 +18,7 @@
18#include <sys/stat.h>18#include <sys/stat.h>
19 19 
20#include "topo_cluster_parser.h"20#include "topo_cluster_parser.h"
21+#include "sim_log.h"
21 22 
22using json = nlohmann::json;23using json = nlohmann::json;
23 24 
@@ -36,7 +37,7 @@ ParseStatus ClusterTopoParser::ParseClusterDir(const std::string& clusterDir) {
36 std::string serversInfoPath = clusterDir + "/servers_info.json";37 std::string serversInfoPath = clusterDir + "/servers_info.json";
37 ParseStatus siStatus = ParseServersInfoJson(serversInfoPath);38 ParseStatus siStatus = ParseServersInfoJson(serversInfoPath);
38 if (siStatus != ParseStatus::OK) {39 if (siStatus != ParseStatus::OK) {
39- fprintf(stderr, "Warning: failed to parse servers_info.json\n");40+ HCCL_VM_WARN("failed to parse servers_info.json");
40 }41 }
41 42 
42 return network_.BuildIndexes();43 return network_.BuildIndexes();
@@ -81,7 +82,7 @@ ParseStatus ClusterTopoParser::ScanSuperPodDirs(const std::string& clusterDir) {
81 try { 82 try {
82 superpod.superPodId = std::stoi(numStr);83 superpod.superPodId = std::stoi(numStr);
83 } catch (...) {84 } catch (...) {
84- fprintf(stderr, "Convert numStr[%s] to int error.\n", numStr.c_str());85+ HCCL_VM_ERROR("Convert numStr[{}] to int error", numStr.c_str());
85 }86 }
86 }87 }
87 88 
@@ -139,7 +140,7 @@ ParseStatus ClusterTopoParser::ScanServerDirs(const std::string& superpodDir, Su
139 try { 140 try {
140 server.serverId = std::stoi(numStr);141 server.serverId = std::stoi(numStr);
141 } catch (...) {142 } catch (...) {
142- fprintf(stderr, "Convert numStr[%s] to int error.\n", numStr.c_str());143+ HCCL_VM_ERROR("Convert numStr[{}] to int error", numStr.c_str());
143 }144 }
144 }145 }
145 146 
@@ -159,10 +160,10 @@ ParseStatus ClusterTopoParser::ParseServer(const std::string& serverDir, Server&
159 std::string rootinfoPath = FindFileBySuffix(serverDir, "rootinfo");160 std::string rootinfoPath = FindFileBySuffix(serverDir, "rootinfo");
160 161 
161 if (topoPath.empty()) {162 if (topoPath.empty()) {
162- fprintf(stderr, "Warning: no topo.json found in %s\n", serverDir.c_str());163+ HCCL_VM_WARN("no topo.json found in {}", serverDir.c_str());
163 }164 }
164 if (rootinfoPath.empty()) {165 if (rootinfoPath.empty()) {
165- fprintf(stderr, "Warning: no rootinfo.json found in %s\n", serverDir.c_str());166+ HCCL_VM_WARN("no rootinfo.json found in {}", serverDir.c_str());
166 }167 }
167 168 
168 if (!topoPath.empty()) {169 if (!topoPath.empty()) {
@@ -299,7 +300,7 @@ ParseStatus ClusterTopoParser::ParseTopoJson(const std::string& topoPath, Server
299 }300 }
300 }301 }
301 } catch (const json::exception& e) {302 } catch (const json::exception& e) {
302- fprintf(stderr, "JSON parse error in %s: %s\n", topoPath.c_str(), e.what());303+ HCCL_VM_ERROR("JSON parse error in {}: {}", topoPath.c_str(), e.what());
303 return ParseStatus::PARSE_ERROR;304 return ParseStatus::PARSE_ERROR;
304 }305 }
305 306 
@@ -413,7 +414,7 @@ ParseStatus ClusterTopoParser::ParseRootinfoJson(const std::string& rootinfoPath
413 }414 }
414 }415 }
415 } catch (const json::exception& e) {416 } catch (const json::exception& e) {
416- fprintf(stderr, "JSON parse error in %s: %s\n", rootinfoPath.c_str(), e.what());417+ HCCL_VM_ERROR("JSON parse error in {}: {}", rootinfoPath.c_str(), e.what());
417 return ParseStatus::PARSE_ERROR;418 return ParseStatus::PARSE_ERROR;
418 }419 }
419 420 
@@ -649,7 +650,7 @@ int ClusterTopoParser::ExtractDieIdFromPortId(const std::string& portId) {
649 try {650 try {
650 return std::stoi(portId.substr(0, slashPos));651 return std::stoi(portId.substr(0, slashPos));
651 } catch (...) {652 } catch (...) {
652- fprintf(stderr, "ExtractDieIdFromPortId error.\n");653+ HCCL_VM_ERROR("ExtractDieIdFromPortId error");
653 return -1;654 return -1;
654 }655 }
655 }656 }
@@ -659,7 +660,7 @@ int ClusterTopoParser::ExtractDieIdFromPortId(const std::string& portId) {
659ParseStatus ClusterTopoParser::ParseServersInfoJson(const std::string& serversInfoPath) {660ParseStatus ClusterTopoParser::ParseServersInfoJson(const std::string& serversInfoPath) {
660 std::ifstream ifs(serversInfoPath);661 std::ifstream ifs(serversInfoPath);
661 if (!ifs.is_open()) {662 if (!ifs.is_open()) {
662- fprintf(stderr, "Warning: servers_info.json not found: %s\n", serversInfoPath.c_str());663+ HCCL_VM_WARN("servers_info.json not found: {}", serversInfoPath.c_str());
663 return ParseStatus::FILE_NOT_FOUND;664 return ParseStatus::FILE_NOT_FOUND;
664 }665 }
665 666 
@@ -704,7 +705,7 @@ ParseStatus ClusterTopoParser::ParseServersInfoJson(const std::string& serversIn
704 }705 }
705 }706 }
706 } catch (const json::exception& e) {707 } catch (const json::exception& e) {
707- fprintf(stderr, "JSON parse error in %s: %s\n", serversInfoPath.c_str(), e.what());708+ HCCL_VM_ERROR("JSON parse error in {}: {}", serversInfoPath.c_str(), e.what());
708 return ParseStatus::PARSE_ERROR;709 return ParseStatus::PARSE_ERROR;
709 }710 }
710 711 
Mtest/hccl_vm/src/utils/CMakeLists.txt+6-0
@@ -82,3 +82,9 @@ target_link_libraries(loader PRIVATE
82 runnerdb82 runnerdb
83 rt83 rt
84)84)
85+ 
86+# 日志染色: 模块 tag
87+target_compile_definitions(log PRIVATE HCCL_VM_MODULE="UTILS")
88+ 
89+# 日志染色: 模块 tag
90+target_compile_definitions(loader PRIVATE HCCL_VM_MODULE="UTILS")
Mtest/hccl_vm/src/utils/sim_loader.cc+16-16
@@ -31,15 +31,15 @@ HcclResult Loader::LoadOpTaskFile(const std::string dbPath)
31 31 
32 if (!dbPath.empty()) {32 if (!dbPath.empty()) {
33 if (!std::filesystem::exists(dbPath)) {33 if (!std::filesystem::exists(dbPath)) {
34- HCCL_VM_ERROR("[Loader] Backup database file not found: {}", dbPath);34+ HCCL_VM_ERROR("Backup database file not found: {}", dbPath);
35 return HcclResult::HCCL_E_PARA;35 return HcclResult::HCCL_E_PARA;
36 }36 }
37 targetPath = dbPath;37 targetPath = dbPath;
38- HCCL_VM_INFO("[Loader] Loading from specific backup path: {}", targetPath);38+ HCCL_VM_INFO("Loading from specific backup path: {}", targetPath);
39 } else {39 } else {
40 targetPath = InstallPath::ResolveToInstallRoot("data/hccl_vm_data.db");40 targetPath = InstallPath::ResolveToInstallRoot("data/hccl_vm_data.db");
41 std::string absPath = std::filesystem::absolute(targetPath).string();41 std::string absPath = std::filesystem::absolute(targetPath).string();
42- HCCL_VM_INFO("[Loader] Loading using default configuration path: {}", absPath);42+ HCCL_VM_INFO("Loading using default configuration path: {}", absPath);
43 }43 }
44 44 
45 config.dbPath = targetPath;45 config.dbPath = targetPath;
@@ -62,67 +62,67 @@ HcclResult Loader::LoadRunnerSingleSync(const uint32_t& outSyncIter,
62 HCCL_VM_ERROR("QueryCompositeOpDetailBySyncIter failed for syncIter: {}", outSyncIter);62 HCCL_VM_ERROR("QueryCompositeOpDetailBySyncIter failed for syncIter: {}", outSyncIter);
63 return HcclResult::HCCL_E_PARA;63 return HcclResult::HCCL_E_PARA;
64 }64 }
65- HCCL_VM_INFO("[LoadRunnerSingleSync] syncIter={} loaded. Rank Count: {}", outSyncIter, compositeDataMap.size());65+ HCCL_VM_INFO("syncIter={} loaded. Rank Count: {}", outSyncIter, compositeDataMap.size());
66 return HcclResult::HCCL_SUCCESS;66 return HcclResult::HCCL_SUCCESS;
67}67}
68 68 
69HcclResult Loader::GetCcuChannelInfo(std::vector<sim::CcuChannelTab>& channels)69HcclResult Loader::GetCcuChannelInfo(std::vector<sim::CcuChannelTab>& channels)
70{70{
71 if (sim::QueryCcuChannelAll(channels) != 0) {71 if (sim::QueryCcuChannelAll(channels) != 0) {
72- HCCL_VM_ERROR("[GetCcuChannelInfo] QueryCcuChannelAll failed");72+ HCCL_VM_ERROR("QueryCcuChannelAll failed");
73 return HcclResult::HCCL_E_PARA;73 return HcclResult::HCCL_E_PARA;
74 }74 }
75- HCCL_VM_INFO("[GetCcuChannelInfo] Get {} channels", channels.size());75+ HCCL_VM_INFO("Get {} channels", channels.size());
76 return HcclResult::HCCL_SUCCESS;76 return HcclResult::HCCL_SUCCESS;
77}77}
78 78 
79HcclResult Loader::GetJettyMapInfo(std::vector<sim::JettyMapTab>& jettyMaps)79HcclResult Loader::GetJettyMapInfo(std::vector<sim::JettyMapTab>& jettyMaps)
80{80{
81 if (sim::QueryJettyMapAll(jettyMaps) != 0) {81 if (sim::QueryJettyMapAll(jettyMaps) != 0) {
82- HCCL_VM_ERROR("[GetJettyMapInfo] QueryJettyMapAll failed");82+ HCCL_VM_ERROR("QueryJettyMapAll failed");
83 return HcclResult::HCCL_E_PARA;83 return HcclResult::HCCL_E_PARA;
84 }84 }
85- HCCL_VM_INFO("[GetJettyMapInfo] Get {} jetty maps", jettyMaps.size());85+ HCCL_VM_INFO("Get {} jetty maps", jettyMaps.size());
86 return HcclResult::HCCL_SUCCESS;86 return HcclResult::HCCL_SUCCESS;
87}87}
88 88 
89HcclResult Loader::GetInstrResInfo(std::vector<sim::CcuInstrResTab>& instrRes)89HcclResult Loader::GetInstrResInfo(std::vector<sim::CcuInstrResTab>& instrRes)
90{90{
91 if (sim::QueryCcuInstrResAll(instrRes) != 0) {91 if (sim::QueryCcuInstrResAll(instrRes) != 0) {
92- HCCL_VM_ERROR("[GetInstrResInfo] QueryCcuInstrResAll failed");92+ HCCL_VM_ERROR("QueryCcuInstrResAll failed");
93 return HcclResult::HCCL_E_PARA;93 return HcclResult::HCCL_E_PARA;
94 }94 }
95- HCCL_VM_INFO("[GetInstrResInfo] Get {} instr res records", instrRes.size());95+ HCCL_VM_INFO("Get {} instr res records", instrRes.size());
96 return HcclResult::HCCL_SUCCESS;96 return HcclResult::HCCL_SUCCESS;
97}97}
98 98 
99HcclResult Loader::GetSyncInfo(std::vector<sim::SyncRecordTab>& syncRecords)99HcclResult Loader::GetSyncInfo(std::vector<sim::SyncRecordTab>& syncRecords)
100{100{
101 if (sim::QuerySyncRecordAll(syncRecords) != 0) {101 if (sim::QuerySyncRecordAll(syncRecords) != 0) {
102- HCCL_VM_ERROR("[GetSyncInfo] QuerySyncRecordAll failed");102+ HCCL_VM_ERROR("QuerySyncRecordAll failed");
103 return HcclResult::HCCL_E_PARA;103 return HcclResult::HCCL_E_PARA;
104 }104 }
105- HCCL_VM_INFO("[GetSyncInfo] Get {} sync records", syncRecords.size());105+ HCCL_VM_INFO("Get {} sync records", syncRecords.size());
106 return HcclResult::HCCL_SUCCESS;106 return HcclResult::HCCL_SUCCESS;
107}107}
108 108 
109HcclResult Loader::GetSyncRecordsByStatus(uint8_t status, std::vector<sim::SyncRecordTab>& syncRecords)109HcclResult Loader::GetSyncRecordsByStatus(uint8_t status, std::vector<sim::SyncRecordTab>& syncRecords)
110{110{
111 if (sim::QuerySyncRecordByStatus(status, syncRecords) != 0) {111 if (sim::QuerySyncRecordByStatus(status, syncRecords) != 0) {
112- HCCL_VM_ERROR("[GetSyncRecordsByStatus] QuerySyncRecordByStatus failed for status: {}", status);112+ HCCL_VM_ERROR("QuerySyncRecordByStatus failed for status: {}", status);
113 return HcclResult::HCCL_E_PARA;113 return HcclResult::HCCL_E_PARA;
114 }114 }
115- HCCL_VM_INFO("[GetSyncRecordsByStatus] Get {} sync records for status: {}", syncRecords.size(), status);115+ HCCL_VM_INFO("Get {} sync records for status: {}", syncRecords.size(), status);
116 return HcclResult::HCCL_SUCCESS;116 return HcclResult::HCCL_SUCCESS;
117}117}
118 118 
119HcclResult Loader::LoadCompositeOpDetailBySyncIter(uint32_t syncIter, std::map<uint32_t, std::vector<sim::CompositeOpDetail>>& compositeDataMap)119HcclResult Loader::LoadCompositeOpDetailBySyncIter(uint32_t syncIter, std::map<uint32_t, std::vector<sim::CompositeOpDetail>>& compositeDataMap)
120{120{
121 if (sim::QueryCompositeOpDetailBySyncIter(syncIter, compositeDataMap) != 0) {121 if (sim::QueryCompositeOpDetailBySyncIter(syncIter, compositeDataMap) != 0) {
122- HCCL_VM_ERROR("[LoadCompositeOpDetailBySyncIter] QueryCompositeOpDetailsBySyncIter failed for syncIter: {}", syncIter);122+ HCCL_VM_ERROR("QueryCompositeOpDetailsBySyncIter failed for syncIter: {}", syncIter);
123 return HcclResult::HCCL_E_PARA;123 return HcclResult::HCCL_E_PARA;
124 }124 }
125- HCCL_VM_INFO("[LoadCompositeOpDetailBySyncIter] syncIter={}, Rank Count: {}", syncIter, compositeDataMap.size());125+ HCCL_VM_INFO("syncIter={}, Rank Count: {}", syncIter, compositeDataMap.size());
126 return HcclResult::HCCL_SUCCESS;126 return HcclResult::HCCL_SUCCESS;
127}127}
128} // namespace loader128} // namespace loader
Mtest/hccl_vm/src/utils/sim_yaml_config.cc+1-1
@@ -38,7 +38,7 @@ bool LoadYamlStringMap(const std::string& yamlPath,
38 }38 }
39 return true;39 return true;
40 } catch (const std::exception& e) {40 } catch (const std::exception& e) {
41- HCCL_VM_ERROR("[SimYamlConfig] Failed to parse YAML file {}: {}", yamlPath, e.what());41+ HCCL_VM_ERROR("Failed to parse YAML file {}: {}", yamlPath, e.what());
42 return false;42 return false;
43 }43 }
44#else44#else
Mtest/hccl_vm/test/cmd/cmd_base_utils_test.cc+6-34
@@ -1468,13 +1468,11 @@ protected:
1468 sim::MemoryManager::GetInstance().FreeMemByName(sim::CommPoolPolicy::kPoolName);1468 sim::MemoryManager::GetInstance().FreeMemByName(sim::CommPoolPolicy::kPoolName);
1469 shm_unlink(sim::CommPoolPolicy::kPoolName);1469 shm_unlink(sim::CommPoolPolicy::kPoolName);
1470 unsetenv("HCCL_OP_EXPANSION_MODE");1470 unsetenv("HCCL_OP_EXPANSION_MODE");
1471- unsetenv("HCCL_VM_INSTALL_DIR");
1472 }1471 }
1473 void TearDown() override {1472 void TearDown() override {
1474 sim::MemoryManager::GetInstance().FreeMemByName("HcclAicpuData");1473 sim::MemoryManager::GetInstance().FreeMemByName("HcclAicpuData");
1475 sim::MemoryManager::GetInstance().FreeMemByName(sim::CommPoolPolicy::kPoolName);1474 sim::MemoryManager::GetInstance().FreeMemByName(sim::CommPoolPolicy::kPoolName);
1476 unsetenv("HCCL_OP_EXPANSION_MODE");1475 unsetenv("HCCL_OP_EXPANSION_MODE");
1477- unsetenv("HCCL_VM_INSTALL_DIR");
1478 }1476 }
1479};1477};
1480 1478 
@@ -1490,13 +1488,6 @@ TEST_F(InitHvmEnvTest, InitializesSharedMemoryWithoutAivValidation) {
1490 EXPECT_EQ(sim::MemoryManager::GetInstance().AcquireMemByName(sim::CommPoolPolicy::kPoolName), nullptr);1488 EXPECT_EQ(sim::MemoryManager::GetInstance().AcquireMemByName(sim::CommPoolPolicy::kPoolName), nullptr);
1491}1489}
1492 1490 
1493-TEST_F(InitHvmEnvTest, FailsInAivModeWithoutInstallDir) {
1494- setenv("HCCL_OP_EXPANSION_MODE", "AIV", 1);
1495- 
1496- HcclVmResult ret = InitHvmEnv("/nonexistent/path/for/ut", 2, false);
1497- EXPECT_EQ(ret, HCCL_SIM_HOST_ERROR_CMD);
1498-}
1499- 
1500TEST_F(InitHvmEnvTest, FailsWhenCommPoolNameAlreadyExists) {1491TEST_F(InitHvmEnvTest, FailsWhenCommPoolNameAlreadyExists) {
1501 // 仅校验模式下池名被预先占用时,InitHvmEnv 建池失败、返回错误,且不创建 HcclAicpuData。1492 // 仅校验模式下池名被预先占用时,InitHvmEnv 建池失败、返回错误,且不创建 HcclAicpuData。
1502 ASSERT_NE(sim::MemoryManager::GetInstance().AllocMemByName(1493 ASSERT_NE(sim::MemoryManager::GetInstance().AllocMemByName(
@@ -1553,33 +1544,14 @@ TEST_F(IsAivExpansionModeEnabledTest, ToggleOnOff) {
1553class ValidateAivTaskJsonByRankTest : public testing::Test {1544class ValidateAivTaskJsonByRankTest : public testing::Test {
1554protected:1545protected:
1555 void SetUp() override {1546 void SetUp() override {
1556- unsetenv("HCCL_VM_INSTALL_DIR");1547+ unsetenv("HCCL_VM_INSTALL_ROOT");
1557- setenv("HCCL_VM_INSTALL_DIR", "/tmp/hvm_ut_aiv_dummy", 1);1548+ setenv("HCCL_VM_INSTALL_ROOT", "/tmp/hvm_ut_aiv_dummy", 1);
1558 }1549 }
1559 void TearDown() override {1550 void TearDown() override {
1560- unsetenv("HCCL_VM_INSTALL_DIR");1551+ unsetenv("HCCL_VM_INSTALL_ROOT");
1561 }1552 }
1562};1553};
1563 1554 
1564-TEST_F(ValidateAivTaskJsonByRankTest, FailsWithoutInstallDir) {
1565- std::vector<sim::Rank> allRank;
1566- sim::Rank r;
1567- r.rank_id = 0;
1568- allRank.push_back(r);
1569- HcclVmResult ret = ValidateAivTaskJsonByRank(allRank);
1570- EXPECT_EQ(ret, HcclVmResult::HCCL_SIM_HOST_ERROR_CMD);
1571-}
1572- 
1573-TEST_F(ValidateAivTaskJsonByRankTest, FailsWithEmptyInstallDir) {
1574- setenv("HCCL_VM_INSTALL_DIR", "", 1);
1575- std::vector<sim::Rank> allRank;
1576- sim::Rank r;
1577- r.rank_id = 0;
1578- allRank.push_back(r);
1579- HcclVmResult ret = ValidateAivTaskJsonByRank(allRank);
1580- EXPECT_EQ(ret, HcclVmResult::HCCL_SIM_HOST_ERROR_CMD);
1581-}
1582- 
1583TEST_F(ValidateAivTaskJsonByRankTest, SucceedsWithEmptyRankList) {1555TEST_F(ValidateAivTaskJsonByRankTest, SucceedsWithEmptyRankList) {
1584 std::vector<sim::Rank> allRank;1556 std::vector<sim::Rank> allRank;
1585 HcclVmResult ret = ValidateAivTaskJsonByRank(allRank);1557 HcclVmResult ret = ValidateAivTaskJsonByRank(allRank);
@@ -1587,7 +1559,7 @@ TEST_F(ValidateAivTaskJsonByRankTest, SucceedsWithEmptyRankList) {
1587}1559}
1588 1560 
1589TEST_F(ValidateAivTaskJsonByRankTest, FailsWithNonExistentDataDir) {1561TEST_F(ValidateAivTaskJsonByRankTest, FailsWithNonExistentDataDir) {
1590- setenv("HCCL_VM_INSTALL_DIR", "/tmp/hvm_ut_no_such_dir_validate", 1);1562+ setenv("HCCL_VM_INSTALL_ROOT", "/tmp/hvm_ut_no_such_dir_validate", 1);
1591 std::vector<sim::Rank> allRank;1563 std::vector<sim::Rank> allRank;
1592 sim::Rank r;1564 sim::Rank r;
1593 r.rank_id = 0;1565 r.rank_id = 0;
@@ -1602,10 +1574,10 @@ protected:
1602 void SetUp() override {1574 void SetUp() override {
1603 installDir_ = std::filesystem::temp_directory_path() / "hvm_ut_aiv_validate";1575 installDir_ = std::filesystem::temp_directory_path() / "hvm_ut_aiv_validate";
1604 std::filesystem::create_directories(installDir_ / "data");1576 std::filesystem::create_directories(installDir_ / "data");
1605- setenv("HCCL_VM_INSTALL_DIR", installDir_.string().c_str(), 1);1577+ setenv("HCCL_VM_INSTALL_ROOT", installDir_.string().c_str(), 1);
1606 }1578 }
1607 void TearDown() override {1579 void TearDown() override {
1608- unsetenv("HCCL_VM_INSTALL_DIR");1580+ unsetenv("HCCL_VM_INSTALL_ROOT");
1609 std::filesystem::remove_all(installDir_);1581 std::filesystem::remove_all(installDir_);
1610 }1582 }
1611 void WriteAivTaskFile(uint32_t rankId, uint32_t launchIdx) {1583 void WriteAivTaskFile(uint32_t rankId, uint32_t launchIdx) {
Mtest/hccl_vm/test/plugin/checker/framework/CMakeLists.txt+4-0
@@ -68,6 +68,7 @@ set_target_properties(test_allgather_v_semantics_checker PROPERTIES
68add_executable(test_allreduce_semantics_checker semantics_check/allreduce_semantics_checker_test.cc main.cc)68add_executable(test_allreduce_semantics_checker semantics_check/allreduce_semantics_checker_test.cc main.cc)
69target_sources(test_allreduce_semantics_checker PRIVATE69target_sources(test_allreduce_semantics_checker PRIVATE
70 ${CMAKE_SOURCE_DIR}/src/plugin/checker/src/framework/semantics_check/allreduce_semantics_checker.cc70 ${CMAKE_SOURCE_DIR}/src/plugin/checker/src/framework/semantics_check/allreduce_semantics_checker.cc
71+ ${CMAKE_SOURCE_DIR}/src/plugin/checker/src/utils/dump/dump_json_utils.cc
71 ${CHECKER_LOG_SRC}72 ${CHECKER_LOG_SRC}
72)73)
73set_target_properties(test_allreduce_semantics_checker PROPERTIES74set_target_properties(test_allreduce_semantics_checker PROPERTIES
@@ -86,6 +87,7 @@ set_target_properties(test_broadcast_semantics_checker PROPERTIES
86add_executable(test_reduce_semantics_checker semantics_check/reduce_semantics_checker_test.cc main.cc)87add_executable(test_reduce_semantics_checker semantics_check/reduce_semantics_checker_test.cc main.cc)
87target_sources(test_reduce_semantics_checker PRIVATE88target_sources(test_reduce_semantics_checker PRIVATE
88 ${CMAKE_SOURCE_DIR}/src/plugin/checker/src/framework/semantics_check/reduce_semantics_checker.cc89 ${CMAKE_SOURCE_DIR}/src/plugin/checker/src/framework/semantics_check/reduce_semantics_checker.cc
90+ ${CMAKE_SOURCE_DIR}/src/plugin/checker/src/utils/dump/dump_json_utils.cc
89 ${CHECKER_LOG_SRC}91 ${CHECKER_LOG_SRC}
90)92)
91set_target_properties(test_reduce_semantics_checker PROPERTIES93set_target_properties(test_reduce_semantics_checker PROPERTIES
@@ -122,6 +124,7 @@ set_target_properties(test_all2all_semantics_checker PROPERTIES
122add_executable(test_reduce_scatter_semantics_checker semantics_check/reduce_scatter_semantics_checker_test.cc main.cc)124add_executable(test_reduce_scatter_semantics_checker semantics_check/reduce_scatter_semantics_checker_test.cc main.cc)
123target_sources(test_reduce_scatter_semantics_checker PRIVATE125target_sources(test_reduce_scatter_semantics_checker PRIVATE
124 ${CMAKE_SOURCE_DIR}/src/plugin/checker/src/framework/semantics_check/reduce_scatter_semantics_checker.cc126 ${CMAKE_SOURCE_DIR}/src/plugin/checker/src/framework/semantics_check/reduce_scatter_semantics_checker.cc
127+ ${CMAKE_SOURCE_DIR}/src/plugin/checker/src/utils/dump/dump_json_utils.cc
125 ${CHECKER_LOG_SRC}128 ${CHECKER_LOG_SRC}
126)129)
127set_target_properties(test_reduce_scatter_semantics_checker PROPERTIES130set_target_properties(test_reduce_scatter_semantics_checker PROPERTIES
@@ -131,6 +134,7 @@ set_target_properties(test_reduce_scatter_semantics_checker PROPERTIES
131add_executable(test_reduce_scatter_v_semantics_checker semantics_check/reduce_scatter_v_semantics_checker_test.cc main.cc)134add_executable(test_reduce_scatter_v_semantics_checker semantics_check/reduce_scatter_v_semantics_checker_test.cc main.cc)
132target_sources(test_reduce_scatter_v_semantics_checker PRIVATE135target_sources(test_reduce_scatter_v_semantics_checker PRIVATE
133 ${CMAKE_SOURCE_DIR}/src/plugin/checker/src/framework/semantics_check/reduce_scatter_v_semantics_checker.cc136 ${CMAKE_SOURCE_DIR}/src/plugin/checker/src/framework/semantics_check/reduce_scatter_v_semantics_checker.cc
137+ ${CMAKE_SOURCE_DIR}/src/plugin/checker/src/utils/dump/dump_json_utils.cc
134 ${CHECKER_LOG_SRC}138 ${CHECKER_LOG_SRC}
135)139)
136set_target_properties(test_reduce_scatter_v_semantics_checker PROPERTIES140set_target_properties(test_reduce_scatter_v_semantics_checker PROPERTIES
Mtest/hccl_vm/test/plugin/checker/st/aiv/aiv_st_test.cc+5-5
@@ -107,22 +107,22 @@ public:
107 dataDir_ = root_ + "/data";107 dataDir_ = root_ + "/data";
108 EnsureDir(dataDir_);108 EnsureDir(dataDir_);
109 109 
110- const char *oldEnv = std::getenv("HCCL_VM_INSTALL_DIR");110+ const char *oldEnv = std::getenv("HCCL_VM_INSTALL_ROOT");
111 if (oldEnv != nullptr) {111 if (oldEnv != nullptr) {
112 hadOldEnv_ = true;112 hadOldEnv_ = true;
113 oldEnv_ = oldEnv;113 oldEnv_ = oldEnv;
114 }114 }
115- if (setenv("HCCL_VM_INSTALL_DIR", root_.c_str(), 1) != 0) {115+ if (setenv("HCCL_VM_INSTALL_ROOT", root_.c_str(), 1) != 0) {
116- throw std::runtime_error("failed to set HCCL_VM_INSTALL_DIR");116+ throw std::runtime_error("failed to set HCCL_VM_INSTALL_ROOT");
117 }117 }
118 }118 }
119 119 
120 ~TempAivInstallDir()120 ~TempAivInstallDir()
121 {121 {
122 if (hadOldEnv_) {122 if (hadOldEnv_) {
123- setenv("HCCL_VM_INSTALL_DIR", oldEnv_.c_str(), 1);123+ setenv("HCCL_VM_INSTALL_ROOT", oldEnv_.c_str(), 1);
124 } else {124 } else {
125- unsetenv("HCCL_VM_INSTALL_DIR");125+ unsetenv("HCCL_VM_INSTALL_ROOT");
126 }126 }
127 if (!root_.empty()) {127 if (!root_.empty()) {
128 RemoveTree(root_);128 RemoveTree(root_);
Mtest/hccl_vm/test/plugin/runner/aiv_graph_executor_test.cc+2-2
@@ -30,12 +30,12 @@ protected:
30 void SetUp() override {30 void SetUp() override {
31 testDir_ = fs::temp_directory_path() / ("aiv_exec_test_" + std::to_string(::getpid()));31 testDir_ = fs::temp_directory_path() / ("aiv_exec_test_" + std::to_string(::getpid()));
32 fs::create_directories(testDir_ / "data");32 fs::create_directories(testDir_ / "data");
33- setenv("HCCL_VM_INSTALL_DIR", testDir_.c_str(), 1);33+ setenv("HCCL_VM_INSTALL_ROOT", testDir_.c_str(), 1);
34 AivResourceManager::GetInstance().Reset();34 AivResourceManager::GetInstance().Reset();
35 }35 }
36 36 
37 void TearDown() override {37 void TearDown() override {
38- unsetenv("HCCL_VM_INSTALL_DIR");38+ unsetenv("HCCL_VM_INSTALL_ROOT");
39 AivResourceManager::GetInstance().Reset();39 AivResourceManager::GetInstance().Reset();
40 fs::remove_all(testDir_);40 fs::remove_all(testDir_);
41 }41 }
Mtest/hccl_vm/test/plugin/runner/aiv_task_json_test.cc+4-29
@@ -440,32 +440,14 @@ TEST_F(AivTaskJsonTest, SerializeExecutor_WithTasks) {
440 440 
441// ========== ResolveExecutorJsonFilePath ==========441// ========== ResolveExecutorJsonFilePath ==========
442 442 
443-TEST_F(AivTaskJsonTest, ResolveExecutorJsonFilePath_NoEnvVar) {
444- unsetenv("HCCL_VM_INSTALL_DIR");
445- auto &executor = AivKernelExecutor::GetInstance();
446- std::string filePath;
447- bool result = ResolveExecutorJsonFilePath(executor, 0, filePath);
448- EXPECT_FALSE(result);
449- EXPECT_TRUE(filePath.empty());
450-}
451- 
452-TEST_F(AivTaskJsonTest, ResolveExecutorJsonFilePath_EmptyEnvVar) {
453- setenv("HCCL_VM_INSTALL_DIR", "", 1);
454- auto &executor = AivKernelExecutor::GetInstance();
455- std::string filePath;
456- bool result = ResolveExecutorJsonFilePath(executor, 0, filePath);
457- EXPECT_FALSE(result);
458- unsetenv("HCCL_VM_INSTALL_DIR");
459-}
460- 
461TEST_F(AivTaskJsonTest, ResolveExecutorJsonFilePath_Success) {443TEST_F(AivTaskJsonTest, ResolveExecutorJsonFilePath_Success) {
462- setenv("HCCL_VM_INSTALL_DIR", "/tmp/hccl_test", 1);444+ setenv("HCCL_VM_INSTALL_ROOT", "/tmp/hccl_test", 1);
463 auto &executor = AivKernelExecutor::GetInstance();445 auto &executor = AivKernelExecutor::GetInstance();
464 std::string filePath;446 std::string filePath;
465 bool result = ResolveExecutorJsonFilePath(executor, 3, filePath);447 bool result = ResolveExecutorJsonFilePath(executor, 3, filePath);
466 EXPECT_TRUE(result);448 EXPECT_TRUE(result);
467 EXPECT_NE(filePath.find("hcclvm_aiv_rank0_launch3_task.json"), std::string::npos);449 EXPECT_NE(filePath.find("hcclvm_aiv_rank0_launch3_task.json"), std::string::npos);
468- unsetenv("HCCL_VM_INSTALL_DIR");450+ unsetenv("HCCL_VM_INSTALL_ROOT");
469}451}
470 452 
471// ========== WriteJsonAtomically ==========453// ========== WriteJsonAtomically ==========
@@ -524,16 +506,9 @@ TEST_F(AivTaskJsonTest, WriteJsonAtomically_RenameFallback) {
524 506 
525// ========== DumpExecutorToJsonFile ==========507// ========== DumpExecutorToJsonFile ==========
526 508 
527-TEST_F(AivTaskJsonTest, DumpExecutorToJsonFile_NoEnvVar) {
528- unsetenv("HCCL_VM_INSTALL_DIR");
529- auto &executor = AivKernelExecutor::GetInstance();
530- bool result = DumpExecutorToJsonFile(executor, 0);
531- EXPECT_FALSE(result);
532-}
533- 
534TEST_F(AivTaskJsonTest, DumpExecutorToJsonFile_Success) {509TEST_F(AivTaskJsonTest, DumpExecutorToJsonFile_Success) {
535 std::error_code ec;510 std::error_code ec;
536- setenv("HCCL_VM_INSTALL_DIR", "/tmp/hccl_dump_test", 1);511+ setenv("HCCL_VM_INSTALL_ROOT", "/tmp/hccl_dump_test", 1);
537 fs::remove_all("/tmp/hccl_dump_test", ec);512 fs::remove_all("/tmp/hccl_dump_test", ec);
538 513 
539 auto &executor = AivKernelExecutor::GetInstance();514 auto &executor = AivKernelExecutor::GetInstance();
@@ -554,7 +529,7 @@ TEST_F(AivTaskJsonTest, DumpExecutorToJsonFile_Success) {
554 EXPECT_NE(content.find("\"aiv\""), std::string::npos);529 EXPECT_NE(content.find("\"aiv\""), std::string::npos);
555 530 
556 fs::remove_all("/tmp/hccl_dump_test", ec);531 fs::remove_all("/tmp/hccl_dump_test", ec);
557- unsetenv("HCCL_VM_INSTALL_DIR");532+ unsetenv("HCCL_VM_INSTALL_ROOT");
558}533}
559 534 
560// ========== aiv_task.h coverage: AivTask base ==========535// ========== aiv_task.h coverage: AivTask base ==========
Mtest/hccl_vm/test/plugin/runner/aiv_task_snapshot_loader_test.cc+2-10
@@ -25,11 +25,11 @@ protected:
25 void SetUp() override {25 void SetUp() override {
26 testDir_ = fs::temp_directory_path() / ("aiv_test_" + std::to_string(::getpid()));26 testDir_ = fs::temp_directory_path() / ("aiv_test_" + std::to_string(::getpid()));
27 fs::create_directories(testDir_ / "data");27 fs::create_directories(testDir_ / "data");
28- setenv("HCCL_VM_INSTALL_DIR", testDir_.c_str(), 1);28+ setenv("HCCL_VM_INSTALL_ROOT", testDir_.c_str(), 1);
29 }29 }
30 30 
31 void TearDown() override {31 void TearDown() override {
32- unsetenv("HCCL_VM_INSTALL_DIR");32+ unsetenv("HCCL_VM_INSTALL_ROOT");
33 fs::remove_all(testDir_);33 fs::remove_all(testDir_);
34 }34 }
35 35 
@@ -300,14 +300,6 @@ TEST_F(AivTaskSnapshotLoaderTest, LoadByLaunchDirect_ErrorNullPointer_Succeeds)
300 300 
301// ==================== LoadRuntimeTaskSnapshotByLaunchDirect Error Tests ====================301// ==================== LoadRuntimeTaskSnapshotByLaunchDirect Error Tests ====================
302 302 
303-TEST_F(AivTaskSnapshotLoaderTest, LoadByLaunchDirect_EnvNotSet_ReturnsFalse) {
304- unsetenv("HCCL_VM_INSTALL_DIR");
305- AivRuntimeTaskSnapshot snapshot;
306- std::string errorMsg;
307- EXPECT_FALSE(AivTaskSnapshotLoader::LoadRuntimeTaskSnapshotByLaunchDirect(0, 0, snapshot, &errorMsg));
308- EXPECT_FALSE(errorMsg.empty());
309-}
310- 
311TEST_F(AivTaskSnapshotLoaderTest, LoadByLaunchDirect_FileNotExist_ReturnsFalse) {303TEST_F(AivTaskSnapshotLoaderTest, LoadByLaunchDirect_FileNotExist_ReturnsFalse) {
312 AivRuntimeTaskSnapshot snapshot;304 AivRuntimeTaskSnapshot snapshot;
313 std::string errorMsg;305 std::string errorMsg;
Mtest/hccl_vm/test/run_ut_README.md+28-28
@@ -41,9 +41,9 @@ cd {HCCL_VM路径}/test
41./run_ut.sh41./run_ut.sh
42```42```
43 43 
44-- 执行所有 `test` 目录下的测试44+- 执行所有 `test` 目录下的测试
45-- 完整三步流程:编译 → 生成可执行文件 → 执行所有测试45+- 完整三步流程:编译 → 生成可执行文件 → 执行所有测试
46-- 适用于完整回归测试46+- 适用于完整回归测试
47 47 
48### 2. 覆盖率报告48### 2. 覆盖率报告
49 49 
@@ -51,11 +51,11 @@ cd {HCCL_VM路径}/test
51./run_ut.sh --cov51./run_ut.sh --cov
52```52```
53 53 
54-- 全量编译 + 执行所有测试 + 生成 gcov/lcov 覆盖率 HTML 报告54+- 全量编译 + 执行所有测试 + 生成 gcov/lcov 覆盖率 HTML 报告
55-- 自动启用 `--coverage` 编译选项,生成 `.gcno`/`.gcda` 文件55+- 自动启用 `--coverage` 编译选项,生成 `.gcno`/`.gcda` 文件
56-- 四步流程:编译(含覆盖率插桩)→ 执行 → 收集覆盖率数据 → 生成 HTML 报告56+- 四步流程:编译(含覆盖率插桩)→ 执行 → 收集覆盖率数据 → 生成 HTML 报告
57-- 报告输出路径:`$CODE_DIR/coverage_report/html/index.html`57+- 报告输出路径:`$CODE_DIR/coverage_report/html/index.html`
58-- 会自动过滤系统头文件、第三方库、stub 文件等非业务代码58+- 会自动过滤系统头文件、第三方库、stub 文件等非业务代码
59 59 
60### 3. 目录执行60### 3. 目录执行
61 61 
@@ -65,9 +65,9 @@ cd {HCCL_VM路径}/test
65./run_ut.sh store65./run_ut.sh store
66```66```
67 67 
68-- 递归查找指定目录下所有 `*_test.cc` 文件68+- 递归查找指定目录下所有 `*_test.cc` 文件
69-- 编译并执行该目录下所有测试69+- 编译并执行该目录下所有测试
70-- 适用于模块级测试70+- 适用于模块级测试
71 71 
72### 4. 二进制执行72### 4. 二进制执行
73 73 
@@ -76,8 +76,8 @@ cd {HCCL_VM路径}/test
76./run_ut.sh test_allgather_semantics_checker76./run_ut.sh test_allgather_semantics_checker
77```77```
78 78 
79-- 编译并执行指定的测试二进制79+- 编译并执行指定的测试二进制
80-- 二进制名以 `test_` 开头80+- 二进制名以 `test_` 开头
81 81 
82### 5. 文件执行82### 5. 文件执行
83 83 
@@ -86,8 +86,8 @@ cd {HCCL_VM路径}/test
86./run_ut.sh allgather_semantics_checker_test.cc86./run_ut.sh allgather_semantics_checker_test.cc
87```87```
88 88 
89-- 根据测试文件名自动匹配对应二进制89+- 根据测试文件名自动匹配对应二进制
90-- 编译并执行90+- 编译并执行
91 91 
92### 6. 单个用例执行92### 6. 单个用例执行
93 93 
@@ -96,8 +96,8 @@ cd {HCCL_VM路径}/test
96./run_ut.sh allgather_semantics_checker_test.cc AllgatherSemanticsCheckerTest.CheckBasic96./run_ut.sh allgather_semantics_checker_test.cc AllgatherSemanticsCheckerTest.CheckBasic
97```97```
98 98 
99-- 执行指定测试文件中的单个测试用例99+- 执行指定测试文件中的单个测试用例
100-- 用例名格式:`TestSuiteName.TestCaseName`100+- 用例名格式:`TestSuiteName.TestCaseName`
101 101 
102### 7. 列出测试102### 7. 列出测试
103 103 
@@ -106,8 +106,8 @@ cd {HCCL_VM路径}/test
106./run_ut.sh --list106./run_ut.sh --list
107```107```
108 108 
109-- 列出所有可用测试文件及其状态109+- 列出所有可用测试文件及其状态
110-- 显示用例数量和对应二进制名110+- 显示用例数量和对应二进制名
111 111 
112## 示例112## 示例
113 113 
@@ -244,10 +244,10 @@ source /home/workspace/Ascend/cann/set_env.sh
244 244 
245## 注意事项245## 注意事项
246 246 
247-1. **首次执行**:首次执行会进行完整的 CMake 配置,耗时较长247+1. **首次执行**:首次执行会进行完整的 CMake 配置,耗时较长
248-2. **编译失败**:如果编译失败,请检查 `build.log` 查看详细错误信息248+2. **编译失败**:如果编译失败,请检查 `build.log` 查看详细错误信息
249-3. **测试失败**:如果测试失败,请检查 `run.log` 查看具体失败的用例249+3. **测试失败**:如果测试失败,请检查 `run.log` 查看具体失败的用例
250-4. **日志清理**:日志目录按时间戳命名,定期清理旧日志以节省空间250+4. **日志清理**:日志目录按时间戳命名,定期清理旧日志以节省空间
251 251 
252## 常见问题252## 常见问题
253 253 
@@ -281,8 +281,8 @@ cd <coverage_report/html所在路径>
281python3 -m http.server 8080281python3 -m http.server 8080
282```282```
283 283 
284-- 本机浏览器访问:`http://localhost:8080`284+- 本机浏览器访问:`http://localhost:8080`
285-- 远程服务器:通过 SSH 端口转发后本地访问285+- 远程服务器:通过 SSH 端口转发后本地访问
286 286 
287```bash287```bash
288ssh -L 8080:localhost:8080 <user>@<server_ip>288ssh -L 8080:localhost:8080 <user>@<server_ip>
@@ -293,6 +293,6 @@ ssh -L 8080:localhost:8080 <user>@<server_ip>
293 293 
294## 版本历史294## 版本历史
295 295 
296-- v1.0 - 初始版本,支持全量/目录/文件/用例级测试执行296+- v1.0 - 初始版本,支持全量/目录/文件/用例级测试执行
297-- v2.0 - 优化命令行参数,支持自动路径推导,三步流程日志记录297+- v2.0 - 优化命令行参数,支持自动路径推导,三步流程日志记录
298-- v2.1 - 新增 `--cov` 参数,支持 gcov/lcov 覆盖率 HTML 报告生成298+- v2.1 - 新增 `--cov` 参数,支持 gcov/lcov 覆盖率 HTML 报告生成