已合并
fix:修复HCCL软件目录重构后遗留问题 #2945
fix:修复HCCL软件目录重构后遗留问题 #2945
已合并
miaoziyi1031创建于 9月7日
共 7 个文件变更+15-15
@@ -26,7 +26,7 @@ A5 注册方式时选用的载体,本身不是本目录的产出目标。
26## 1. 动机26## 1. 动机
27 27 
28- **验证 A5 注册方式在 experimental 下的可用性(主动机)**:主链路算法(如28- **验证 A5 注册方式在 experimental 下的可用性(主动机)**:主链路算法(如
29- `src/ops/all_reduce/executor/ins_v2_all_reduce_sole_executor.cc` 的 `CcuMSAllReduceSoleMesh`)通过29+ `src/ops/all_reduce/algorithm/executor/ins_v2_all_reduce_sole_executor.cc` 的 `CcuMSAllReduceSoleMesh`)通过
30 `REGISTER_EXEC_V2` 宏注册进 `CollAlgExecRegistryV2`,运行时由 `GetAlgExec` 按算法名查找执行器。30 `REGISTER_EXEC_V2` 宏注册进 `CollAlgExecRegistryV2`,运行时由 `GetAlgExec` 按算法名查找执行器。
31 本目录在experimental/ 下新增一个同样走 A5 注册的算法,以测试 A5 注册方式是否可用。31 本目录在experimental/ 下新增一个同样走 A5 注册的算法,以测试 A5 注册方式是否可用。
32- **以最小样例验证全链路**:不修改主链路算法,而是在experimental 下复制同构的 executor/template/kernel 三层32- **以最小样例验证全链路**:不修改主链路算法,而是在experimental 下复制同构的 executor/template/kernel 三层
@@ -37,7 +37,7 @@ A5 注册方式时选用的载体,本身不是本目录的产出目标。
37## 2. 目录结构与设计37## 2. 目录结构与设计
38 38 
39下图为以 `hccl/` 仓库根为起点的完整结构,省略的无关文件/目录统一用 `...` 标注。本目录结构与主链路39下图为以 `hccl/` 仓库根为起点的完整结构,省略的无关文件/目录统一用 `...` 标注。本目录结构与主链路
40-`src/ops/all_reduce/`(`executor/` + `template/ccu/kernel/`)同构,证明相同的 A5 注册代码在40+`src/ops/all_reduce/`(`algorithm/executor/` + `algorithm/template/ccu/kernel/`)同构,证明相同的 A5 注册代码在
41experimental 目录下可以原样工作;图中同时标注了与本目录直接相关的关键上下文(注册表、主链路41experimental 目录下可以原样工作;图中同时标注了与本目录直接相关的关键上下文(注册表、主链路
42对照样例、父级 CMakeLists)。42对照样例、父级 CMakeLists)。
43 43 
@@ -46,10 +46,10 @@ hccl/ # 仓库根
46├── CMakeLists.txt # 顶层:option(ENABLE_EXPERIMENTAL);条件 add_subdirectory(experimental/ops/)46├── CMakeLists.txt # 顶层:option(ENABLE_EXPERIMENTAL);条件 add_subdirectory(experimental/ops/)
47├── build.sh # 编包入口:--experimental → -DENABLE_EXPERIMENTAL=ON47├── build.sh # 编包入口:--experimental → -DENABLE_EXPERIMENTAL=ON
48├── src/ # 主链路(商用代码,位于 experimental 之外)48├── src/ # 主链路(商用代码,位于 experimental 之外)
49-│ ├── ops/op_common/executor/registry/49+│ ├── ops/op_common/algorithm/executor/registry/
50│ │ └── coll_alg_v2_exec_registry.* # 执行器注册表:REGISTER_EXEC_V2 写入 / GetAlgExec 查找50│ │ └── coll_alg_v2_exec_registry.* # 执行器注册表:REGISTER_EXEC_V2 写入 / GetAlgExec 查找
51│ ├── ops/all_reduce/51│ ├── ops/all_reduce/
52-│ │ ├── executor/ins_v2_all_reduce_sole_executor.cc # 主链路对照样例(注册 52+│ │ ├── algorithm/executor/ins_v2_all_reduce_sole_executor.cc # 主链路对照样例(注册
53│ │ └── ... # 其余 executor/template/kernel 文件53│ │ └── ... # 其余 executor/template/kernel 文件
54│ └── ... # 其余 src 子模块54│ └── ... # 其余 src 子模块
55├── include/ # 对外头文件(hccl.h / hccl_mc2.h 等)55├── include/ # 对外头文件(hccl.h / hccl_mc2.h 等)
@@ -207,11 +207,11 @@ template 继承自引擎模板基类(以 `CcuAlgTemplateBase` 为例),须
207| `CalcScratchMultiple` | 返回 scratch 倍率,供 executor 计算单 loop 数据上界 | `inBuffType`/`outBuffType` 输入输出 buffer 类型 |207| `CalcScratchMultiple` | 返回 scratch 倍率,供 executor 计算单 loop 数据上界 | `inBuffType`/`outBuffType` 输入输出 buffer 类型 |
208| `FastLaunch` | 快速下发:用预存 `submitInfos` 改写地址后直发内核 | `param`;`tempFastLaunchCtx` threads/ccuKernelSubmitInfos/buffInfo |208| `FastLaunch` | 快速下发:用预存 `submitInfos` 改写地址后直发内核 | `param`;`tempFastLaunchCtx` threads/ccuKernelSubmitInfos/buffInfo |
209 209 
210-`CalcCostCoeff`(executor 与 template)对应基类 `CalcCostCoeffParam` 字段的完整说明可见 `src/ops/op_common/template/` 下模板头文件注释。210+`CalcCostCoeff`(executor 与 template)对应基类 `CalcCostCoeffParam` 字段的完整说明可见 `src/ops/op_common/algorithm/template/` 下模板头文件注释。
211 211 
212#### REGISTER_EXEC_V2 宏参数212#### REGISTER_EXEC_V2 宏参数
213 213 
214-宏定义见 `src/ops/op_common/executor/registry/coll_alg_v2_exec_registry.h`,签名:214+宏定义见 `src/ops/op_common/algorithm/executor/registry/coll_alg_v2_exec_registry.h`,签名:
215 215 
216```cpp216```cpp
217REGISTER_EXEC_V2(type, name, insCollAlgBase, AlgTopoMatch, InsAlgTemplate)217REGISTER_EXEC_V2(type, name, insCollAlgBase, AlgTopoMatch, InsAlgTemplate)
@@ -14,24 +14,24 @@ Besides this algorithm itself, Section 2.5 provides the general guide on "implem
14 14 
15## 1. Motivation15## 1. Motivation
16 16 
17-- **Verify the usability of the A5 registration approach under experimental (primary motivation)**: Main-pathway algorithms (e.g., `CcuMSAllReduceSoleMesh` in `src/ops/all_reduce/executor/ins_v2_all_reduce_sole_executor.cc`) register into `CollAlgExecRegistryV2` via the `REGISTER_EXEC_V2` macro, and are looked up at runtime by `GetAlgExec` based on the algorithm name. This directory adds an algorithm under experimental/ that also follows A5 registration, to test whether the A5 registration approach is usable.17+- **Verify the usability of the A5 registration approach under experimental (primary motivation)**: Main-pathway algorithms (e.g., `CcuMSAllReduceSoleMesh` in `src/ops/all_reduce/algorithm/executor/ins_v2_all_reduce_sole_executor.cc`) register into `CollAlgExecRegistryV2` via the `REGISTER_EXEC_V2` macro, and are looked up at runtime by `GetAlgExec` based on the algorithm name. This directory adds an algorithm under experimental/ that also follows A5 registration, to test whether the A5 registration approach is usable.
18- **Verify the full pathway with a minimal sample**: Without modifying main-pathway algorithms, replicate the isomorphic executor/template/kernel three-layer structure under experimental/ as a carrier, verifying that every step from build to checker is usable.18- **Verify the full pathway with a minimal sample**: Without modifying main-pathway algorithms, replicate the isomorphic executor/template/kernel three-layer structure under experimental/ as a carrier, verifying that every step from build to checker is usable.
19 19 
20---20---
21 21 
22## 2. Directory Structure and Design22## 2. Directory Structure and Design
23 23 
24-The diagram below shows the complete structure starting from the `hccl/` repository root; unrelated files/directories are uniformly marked with `...`. This directory's structure is isomorphic to the main pathway `src/ops/all_reduce/` (`executor/` + `template/ccu/kernel/`), demonstrating that the same A5 registration code works unchanged under the experimental directory; the diagram also annotates key context directly related to this directory (the registry, the main-pathway reference sample, and the parent CMakeLists).24+The diagram below shows the complete structure starting from the `hccl/` repository root; unrelated files/directories are uniformly marked with `...`. This directory's structure is isomorphic to the main pathway `src/ops/all_reduce/` (`algorithm/executor/` + `algorithm/template/ccu/kernel/`), demonstrating that the same A5 registration code works unchanged under the experimental directory; the diagram also annotates key context directly related to this directory (the registry, the main-pathway reference sample, and the parent CMakeLists).
25 25 
26```plaintext26```plaintext
27hccl/ # Repository root27hccl/ # Repository root
28├── CMakeLists.txt # Top-level: option(ENABLE_EXPERIMENTAL); conditional add_subdirectory(experimental/ops/)28├── CMakeLists.txt # Top-level: option(ENABLE_EXPERIMENTAL); conditional add_subdirectory(experimental/ops/)
29├── build.sh # Build entry: --experimental → -DENABLE_EXPERIMENTAL=ON29├── build.sh # Build entry: --experimental → -DENABLE_EXPERIMENTAL=ON
30├── src/ # Main pathway (commercial code, outside experimental)30├── src/ # Main pathway (commercial code, outside experimental)
31-│ ├── ops/op_common/executor/registry/31+│ ├── ops/op_common/algorithm/executor/registry/
32│ │ └── coll_alg_v2_exec_registry.* # Executor registry: REGISTER_EXEC_V2 writes / GetAlgExec looks up32│ │ └── coll_alg_v2_exec_registry.* # Executor registry: REGISTER_EXEC_V2 writes / GetAlgExec looks up
33│ ├── ops/all_reduce/33│ ├── ops/all_reduce/
34-│ │ ├── executor/ins_v2_all_reduce_sole_executor.cc # Main-pathway reference sample (registers CcuMSAllReduceSoleMesh)34+│ │ ├── algorithm/executor/ins_v2_all_reduce_sole_executor.cc # Main-pathway reference sample (registers CcuMSAllReduceSoleMesh)
35│ │ └── ... # Other executor/template/kernel files35│ │ └── ... # Other executor/template/kernel files
36│ └── ... # Other src submodules36│ └── ... # Other src submodules
37├── include/ # Public headers (hccl.h / hccl_mc2.h, etc.)37├── include/ # Public headers (hccl.h / hccl_mc2.h, etc.)
@@ -189,11 +189,11 @@ The template inherits from the engine template base class (using `CcuAlgTemplate
189| `CalcScratchMultiple` | Returns the scratch multiplier for the executor to compute the per-loop data upper bound | `inBuffType`/`outBuffType` input/output buffer types |189| `CalcScratchMultiple` | Returns the scratch multiplier for the executor to compute the per-loop data upper bound | `inBuffType`/`outBuffType` input/output buffer types |
190| `FastLaunch` | Fast dispatch: rewrites addresses using pre-stored `submitInfos` and dispatches the kernel directly | `param`; `tempFastLaunchCtx` threads/ccuKernelSubmitInfos/buffInfo |190| `FastLaunch` | Fast dispatch: rewrites addresses using pre-stored `submitInfos` and dispatches the kernel directly | `param`; `tempFastLaunchCtx` threads/ccuKernelSubmitInfos/buffInfo |
191 191 
192-For the full description of the `CalcCostCoeffParam` fields of the base class used by `CalcCostCoeff` (both executor and template), see the template header file comments under `src/ops/op_common/template/`.192+For the full description of the `CalcCostCoeffParam` fields of the base class used by `CalcCostCoeff` (both executor and template), see the template header file comments under `src/ops/op_common/algorithm/template/`.
193 193 
194#### REGISTER_EXEC_V2 Macro Parameters194#### REGISTER_EXEC_V2 Macro Parameters
195 195 
196-The macro is defined in `src/ops/op_common/executor/registry/coll_alg_v2_exec_registry.h`, signature:196+The macro is defined in `src/ops/op_common/algorithm/executor/registry/coll_alg_v2_exec_registry.h`, signature:
197 197 
198```cpp198```cpp
199REGISTER_EXEC_V2(type, name, insCollAlgBase, AlgTopoMatch, InsAlgTemplate)199REGISTER_EXEC_V2(type, name, insCollAlgBase, AlgTopoMatch, InsAlgTemplate)
@@ -13,9 +13,6 @@ set(src_list
13 ${CMAKE_CURRENT_SOURCE_DIR}/topo_match_base.cc13 ${CMAKE_CURRENT_SOURCE_DIR}/topo_match_base.cc
14 ${CMAKE_CURRENT_SOURCE_DIR}/topo_match_base_v2.cc14 ${CMAKE_CURRENT_SOURCE_DIR}/topo_match_base_v2.cc
15 ${CMAKE_CURRENT_SOURCE_DIR}/topo_match_concurrent.cc15 ${CMAKE_CURRENT_SOURCE_DIR}/topo_match_concurrent.cc
16- # PhysicalLevel标准化: 不依赖版本宏, 因此放在无条件块
17- ${CMAKE_CURRENT_SOURCE_DIR}/physical_level_build.cc
18- ${CMAKE_CURRENT_SOURCE_DIR}/physical_level_normalize.cc
19 ${CMAKE_CURRENT_SOURCE_DIR}/topo_match_one_level.cc16 ${CMAKE_CURRENT_SOURCE_DIR}/topo_match_one_level.cc
20 ${CMAKE_CURRENT_SOURCE_DIR}/topo_match_two_level.cc17 ${CMAKE_CURRENT_SOURCE_DIR}/topo_match_two_level.cc
21 ${CMAKE_CURRENT_SOURCE_DIR}/topo_match_three_level.cc18 ${CMAKE_CURRENT_SOURCE_DIR}/topo_match_three_level.cc
@@ -11,6 +11,9 @@
11set(src_list11set(src_list
12 ${CMAKE_CURRENT_SOURCE_DIR}/topo.cc12 ${CMAKE_CURRENT_SOURCE_DIR}/topo.cc
13 ${CMAKE_CURRENT_SOURCE_DIR}/topo_host.cc13 ${CMAKE_CURRENT_SOURCE_DIR}/topo_host.cc
14+ # PhysicalLevel标准化: 不依赖版本宏, 因此放在无条件块
15+ ${CMAKE_CURRENT_SOURCE_DIR}/physical_level_build.cc
16+ ${CMAKE_CURRENT_SOURCE_DIR}/physical_level_normalize.cc
14)17)
15 18 
16if(TARGET hccl)19if(TARGET hccl)
Rsrc/ops/op_common/algorithm/topo_match/physical_level.h→src/ops/op_common/topo_info/physical_level.h+0-0
文件重命名但无更改。
Rsrc/ops/op_common/algorithm/topo_match/physical_level_build.cc→src/ops/op_common/topo_info/physical_level_build.cc+0-0
文件重命名但无更改。
Rsrc/ops/op_common/algorithm/topo_match/physical_level_normalize.cc→src/ops/op_common/topo_info/physical_level_normalize.cc+0-0
文件重命名但无更改。