已合并
fix:修复HCCL软件目录重构后遗留问题 #2945
miaoziyi1031创建于 9月7日
fix:修复HCCL软件目录重构后遗留问题 #2945
已合并
共 7 个文件变更+15-15
| @@ -26,7 +26,7 @@ A5 注册方式时选用的载体,本身不是本目录的产出目标。 | |||
| 26 | ## 1. 动机 | 26 | ## 1. 动机 |
| 27 | 27 | ||
| 28 | - **验证 A5 注册方式在 experimental 下的可用性(主动机)**:主链路算法(如 | 28 | - **验证 A5 注册方式在 experimental 下的可用性(主动机)**:主链路算法(如 |
| 29 | - `src/ops/all_reduce/executor/ins_v2_all_reduce_sole_executor.cc` 的 `CcuMSAllReduceSoleMesh`)通过 | 29 | + `src/ops/all_reduce/algorithm/executor/ins_v2_all_reduce_sole_executor.cc` 的 `CcuMSAllReduceSoleMesh`)通过 |
| 30 | `REGISTER_EXEC_V2` 宏注册进 `CollAlgExecRegistryV2`,运行时由 `GetAlgExec` 按算法名查找执行器。 | 30 | `REGISTER_EXEC_V2` 宏注册进 `CollAlgExecRegistryV2`,运行时由 `GetAlgExec` 按算法名查找执行器。 |
| 31 | 本目录在experimental/ 下新增一个同样走 A5 注册的算法,以测试 A5 注册方式是否可用。 | 31 | 本目录在experimental/ 下新增一个同样走 A5 注册的算法,以测试 A5 注册方式是否可用。 |
| 32 | - **以最小样例验证全链路**:不修改主链路算法,而是在experimental 下复制同构的 executor/template/kernel 三层 | 32 | - **以最小样例验证全链路**:不修改主链路算法,而是在experimental 下复制同构的 executor/template/kernel 三层 |
| @@ -37,7 +37,7 @@ A5 注册方式时选用的载体,本身不是本目录的产出目标。 | |||
| 37 | ## 2. 目录结构与设计 | 37 | ## 2. 目录结构与设计 |
| 38 | 38 | ||
| 39 | 下图为以 `hccl/` 仓库根为起点的完整结构,省略的无关文件/目录统一用 `...` 标注。本目录结构与主链路 | 39 | 下图为以 `hccl/` 仓库根为起点的完整结构,省略的无关文件/目录统一用 `...` 标注。本目录结构与主链路 |
| 40 | -`src/ops/all_reduce/`(`executor/` + `template/ccu/kernel/`)同构,证明相同的 A5 注册代码在 | 40 | +`src/ops/all_reduce/`(`algorithm/executor/` + `algorithm/template/ccu/kernel/`)同构,证明相同的 A5 注册代码在 |
| 41 | experimental 目录下可以原样工作;图中同时标注了与本目录直接相关的关键上下文(注册表、主链路 | 41 | experimental 目录下可以原样工作;图中同时标注了与本目录直接相关的关键上下文(注册表、主链路 |
| 42 | 对照样例、父级 CMakeLists)。 | 42 | 对照样例、父级 CMakeLists)。 |
| 43 | 43 | ||
| @@ -46,10 +46,10 @@ hccl/ # 仓库根 | |||
| 46 | ├── CMakeLists.txt # 顶层:option(ENABLE_EXPERIMENTAL);条件 add_subdirectory(experimental/ops/) | 46 | ├── CMakeLists.txt # 顶层:option(ENABLE_EXPERIMENTAL);条件 add_subdirectory(experimental/ops/) |
| 47 | ├── build.sh # 编包入口:--experimental → -DENABLE_EXPERIMENTAL=ON | 47 | ├── build.sh # 编包入口:--experimental → -DENABLE_EXPERIMENTAL=ON |
| 48 | ├── src/ # 主链路(商用代码,位于 experimental 之外) | 48 | ├── src/ # 主链路(商用代码,位于 experimental 之外) |
| 49 | -│ ├── ops/op_common/executor/registry/ | 49 | +│ ├── ops/op_common/algorithm/executor/registry/ |
| 50 | │ │ └── coll_alg_v2_exec_registry.* # 执行器注册表:REGISTER_EXEC_V2 写入 / GetAlgExec 查找 | 50 | │ │ └── coll_alg_v2_exec_registry.* # 执行器注册表:REGISTER_EXEC_V2 写入 / GetAlgExec 查找 |
| 51 | │ ├── ops/all_reduce/ | 51 | │ ├── ops/all_reduce/ |
| 52 | -│ │ ├── executor/ins_v2_all_reduce_sole_executor.cc # 主链路对照样例(注册 | 52 | +│ │ ├── algorithm/executor/ins_v2_all_reduce_sole_executor.cc # 主链路对照样例(注册 |
| 53 | │ │ └── ... # 其余 executor/template/kernel 文件 | 53 | │ │ └── ... # 其余 executor/template/kernel 文件 |
| 54 | │ └── ... # 其余 src 子模块 | 54 | │ └── ... # 其余 src 子模块 |
| 55 | ├── include/ # 对外头文件(hccl.h / hccl_mc2.h 等) | 55 | ├── include/ # 对外头文件(hccl.h / hccl_mc2.h 等) |
| @@ -207,11 +207,11 @@ template 继承自引擎模板基类(以 `CcuAlgTemplateBase` 为例),须 | |||
| 207 | | `CalcScratchMultiple` | 返回 scratch 倍率,供 executor 计算单 loop 数据上界 | `inBuffType`/`outBuffType` 输入输出 buffer 类型 | | 207 | | `CalcScratchMultiple` | 返回 scratch 倍率,供 executor 计算单 loop 数据上界 | `inBuffType`/`outBuffType` 输入输出 buffer 类型 | |
| 208 | | `FastLaunch` | 快速下发:用预存 `submitInfos` 改写地址后直发内核 | `param`;`tempFastLaunchCtx` threads/ccuKernelSubmitInfos/buffInfo | | 208 | | `FastLaunch` | 快速下发:用预存 `submitInfos` 改写地址后直发内核 | `param`;`tempFastLaunchCtx` threads/ccuKernelSubmitInfos/buffInfo | |
| 209 | 209 | ||
| 210 | -`CalcCostCoeff`(executor 与 template)对应基类 `CalcCostCoeffParam` 字段的完整说明可见 `src/ops/op_common/template/` 下模板头文件注释。 | 210 | +`CalcCostCoeff`(executor 与 template)对应基类 `CalcCostCoeffParam` 字段的完整说明可见 `src/ops/op_common/algorithm/template/` 下模板头文件注释。 |
| 211 | 211 | ||
| 212 | #### REGISTER_EXEC_V2 宏参数 | 212 | #### REGISTER_EXEC_V2 宏参数 |
| 213 | 213 | ||
| 214 | -宏定义见 `src/ops/op_common/executor/registry/coll_alg_v2_exec_registry.h`,签名: | 214 | +宏定义见 `src/ops/op_common/algorithm/executor/registry/coll_alg_v2_exec_registry.h`,签名: |
| 215 | 215 | ||
| 216 | ```cpp | 216 | ```cpp |
| 217 | REGISTER_EXEC_V2(type, name, insCollAlgBase, AlgTopoMatch, InsAlgTemplate) | 217 | REGISTER_EXEC_V2(type, name, insCollAlgBase, AlgTopoMatch, InsAlgTemplate) |
| @@ -14,24 +14,24 @@ Besides this algorithm itself, Section 2.5 provides the general guide on "implem | |||
| 14 | 14 | ||
| 15 | ## 1. Motivation | 15 | ## 1. Motivation |
| 16 | 16 | ||
| 17 | -- **Verify the usability of the A5 registration approach under experimental (primary motivation)**: Main-pathway algorithms (e.g., `CcuMSAllReduceSoleMesh` in `src/ops/all_reduce/executor/ins_v2_all_reduce_sole_executor.cc`) register into `CollAlgExecRegistryV2` via the `REGISTER_EXEC_V2` macro, and are looked up at runtime by `GetAlgExec` based on the algorithm name. This directory adds an algorithm under experimental/ that also follows A5 registration, to test whether the A5 registration approach is usable. | 17 | +- **Verify the usability of the A5 registration approach under experimental (primary motivation)**: Main-pathway algorithms (e.g., `CcuMSAllReduceSoleMesh` in `src/ops/all_reduce/algorithm/executor/ins_v2_all_reduce_sole_executor.cc`) register into `CollAlgExecRegistryV2` via the `REGISTER_EXEC_V2` macro, and are looked up at runtime by `GetAlgExec` based on the algorithm name. This directory adds an algorithm under experimental/ that also follows A5 registration, to test whether the A5 registration approach is usable. |
| 18 | - **Verify the full pathway with a minimal sample**: Without modifying main-pathway algorithms, replicate the isomorphic executor/template/kernel three-layer structure under experimental/ as a carrier, verifying that every step from build to checker is usable. | 18 | - **Verify the full pathway with a minimal sample**: Without modifying main-pathway algorithms, replicate the isomorphic executor/template/kernel three-layer structure under experimental/ as a carrier, verifying that every step from build to checker is usable. |
| 19 | 19 | ||
| 20 | --- | 20 | --- |
| 21 | 21 | ||
| 22 | ## 2. Directory Structure and Design | 22 | ## 2. Directory Structure and Design |
| 23 | 23 | ||
| 24 | -The diagram below shows the complete structure starting from the `hccl/` repository root; unrelated files/directories are uniformly marked with `...`. This directory's structure is isomorphic to the main pathway `src/ops/all_reduce/` (`executor/` + `template/ccu/kernel/`), demonstrating that the same A5 registration code works unchanged under the experimental directory; the diagram also annotates key context directly related to this directory (the registry, the main-pathway reference sample, and the parent CMakeLists). | 24 | +The diagram below shows the complete structure starting from the `hccl/` repository root; unrelated files/directories are uniformly marked with `...`. This directory's structure is isomorphic to the main pathway `src/ops/all_reduce/` (`algorithm/executor/` + `algorithm/template/ccu/kernel/`), demonstrating that the same A5 registration code works unchanged under the experimental directory; the diagram also annotates key context directly related to this directory (the registry, the main-pathway reference sample, and the parent CMakeLists). |
| 25 | 25 | ||
| 26 | ```plaintext | 26 | ```plaintext |
| 27 | hccl/ # Repository root | 27 | hccl/ # Repository root |
| 28 | ├── CMakeLists.txt # Top-level: option(ENABLE_EXPERIMENTAL); conditional add_subdirectory(experimental/ops/) | 28 | ├── CMakeLists.txt # Top-level: option(ENABLE_EXPERIMENTAL); conditional add_subdirectory(experimental/ops/) |
| 29 | ├── build.sh # Build entry: --experimental → -DENABLE_EXPERIMENTAL=ON | 29 | ├── build.sh # Build entry: --experimental → -DENABLE_EXPERIMENTAL=ON |
| 30 | ├── src/ # Main pathway (commercial code, outside experimental) | 30 | ├── src/ # Main pathway (commercial code, outside experimental) |
| 31 | -│ ├── ops/op_common/executor/registry/ | 31 | +│ ├── ops/op_common/algorithm/executor/registry/ |
| 32 | │ │ └── coll_alg_v2_exec_registry.* # Executor registry: REGISTER_EXEC_V2 writes / GetAlgExec looks up | 32 | │ │ └── coll_alg_v2_exec_registry.* # Executor registry: REGISTER_EXEC_V2 writes / GetAlgExec looks up |
| 33 | │ ├── ops/all_reduce/ | 33 | │ ├── ops/all_reduce/ |
| 34 | -│ │ ├── executor/ins_v2_all_reduce_sole_executor.cc # Main-pathway reference sample (registers CcuMSAllReduceSoleMesh) | 34 | +│ │ ├── algorithm/executor/ins_v2_all_reduce_sole_executor.cc # Main-pathway reference sample (registers CcuMSAllReduceSoleMesh) |
| 35 | │ │ └── ... # Other executor/template/kernel files | 35 | │ │ └── ... # Other executor/template/kernel files |
| 36 | │ └── ... # Other src submodules | 36 | │ └── ... # Other src submodules |
| 37 | ├── include/ # Public headers (hccl.h / hccl_mc2.h, etc.) | 37 | ├── include/ # Public headers (hccl.h / hccl_mc2.h, etc.) |
| @@ -189,11 +189,11 @@ The template inherits from the engine template base class (using `CcuAlgTemplate | |||
| 189 | | `CalcScratchMultiple` | Returns the scratch multiplier for the executor to compute the per-loop data upper bound | `inBuffType`/`outBuffType` input/output buffer types | | 189 | | `CalcScratchMultiple` | Returns the scratch multiplier for the executor to compute the per-loop data upper bound | `inBuffType`/`outBuffType` input/output buffer types | |
| 190 | | `FastLaunch` | Fast dispatch: rewrites addresses using pre-stored `submitInfos` and dispatches the kernel directly | `param`; `tempFastLaunchCtx` threads/ccuKernelSubmitInfos/buffInfo | | 190 | | `FastLaunch` | Fast dispatch: rewrites addresses using pre-stored `submitInfos` and dispatches the kernel directly | `param`; `tempFastLaunchCtx` threads/ccuKernelSubmitInfos/buffInfo | |
| 191 | 191 | ||
| 192 | -For the full description of the `CalcCostCoeffParam` fields of the base class used by `CalcCostCoeff` (both executor and template), see the template header file comments under `src/ops/op_common/template/`. | 192 | +For the full description of the `CalcCostCoeffParam` fields of the base class used by `CalcCostCoeff` (both executor and template), see the template header file comments under `src/ops/op_common/algorithm/template/`. |
| 193 | 193 | ||
| 194 | #### REGISTER_EXEC_V2 Macro Parameters | 194 | #### REGISTER_EXEC_V2 Macro Parameters |
| 195 | 195 | ||
| 196 | -The macro is defined in `src/ops/op_common/executor/registry/coll_alg_v2_exec_registry.h`, signature: | 196 | +The macro is defined in `src/ops/op_common/algorithm/executor/registry/coll_alg_v2_exec_registry.h`, signature: |
| 197 | 197 | ||
| 198 | ```cpp | 198 | ```cpp |
| 199 | REGISTER_EXEC_V2(type, name, insCollAlgBase, AlgTopoMatch, InsAlgTemplate) | 199 | REGISTER_EXEC_V2(type, name, insCollAlgBase, AlgTopoMatch, InsAlgTemplate) |
| @@ -13,9 +13,6 @@ set(src_list | |||
| 13 | ${CMAKE_CURRENT_SOURCE_DIR}/topo_match_base.cc | 13 | ${CMAKE_CURRENT_SOURCE_DIR}/topo_match_base.cc |
| 14 | ${CMAKE_CURRENT_SOURCE_DIR}/topo_match_base_v2.cc | 14 | ${CMAKE_CURRENT_SOURCE_DIR}/topo_match_base_v2.cc |
| 15 | ${CMAKE_CURRENT_SOURCE_DIR}/topo_match_concurrent.cc | 15 | ${CMAKE_CURRENT_SOURCE_DIR}/topo_match_concurrent.cc |
| 16 | - # PhysicalLevel标准化: 不依赖版本宏, 因此放在无条件块 | ||
| 17 | - ${CMAKE_CURRENT_SOURCE_DIR}/physical_level_build.cc | ||
| 18 | - ${CMAKE_CURRENT_SOURCE_DIR}/physical_level_normalize.cc | ||
| 19 | ${CMAKE_CURRENT_SOURCE_DIR}/topo_match_one_level.cc | 16 | ${CMAKE_CURRENT_SOURCE_DIR}/topo_match_one_level.cc |
| 20 | ${CMAKE_CURRENT_SOURCE_DIR}/topo_match_two_level.cc | 17 | ${CMAKE_CURRENT_SOURCE_DIR}/topo_match_two_level.cc |
| 21 | ${CMAKE_CURRENT_SOURCE_DIR}/topo_match_three_level.cc | 18 | ${CMAKE_CURRENT_SOURCE_DIR}/topo_match_three_level.cc |
| @@ -11,6 +11,9 @@ | |||
| 11 | set(src_list | 11 | set(src_list |
| 12 | ${CMAKE_CURRENT_SOURCE_DIR}/topo.cc | 12 | ${CMAKE_CURRENT_SOURCE_DIR}/topo.cc |
| 13 | ${CMAKE_CURRENT_SOURCE_DIR}/topo_host.cc | 13 | ${CMAKE_CURRENT_SOURCE_DIR}/topo_host.cc |
| 14 | + # PhysicalLevel标准化: 不依赖版本宏, 因此放在无条件块 | ||
| 15 | + ${CMAKE_CURRENT_SOURCE_DIR}/physical_level_build.cc | ||
| 16 | + ${CMAKE_CURRENT_SOURCE_DIR}/physical_level_normalize.cc | ||
| 14 | ) | 17 | ) |
| 15 | 18 | ||
| 16 | if(TARGET hccl) | 19 | if(TARGET hccl) |
Rsrc/ops/op_common/algorithm/topo_match/physical_level.h→src/ops/op_common/topo_info/physical_level.h+0-0
文件重命名但无更改。
Rsrc/ops/op_common/algorithm/topo_match/physical_level_build.cc→src/ops/op_common/topo_info/physical_level_build.cc+0-0
文件重命名但无更改。
Rsrc/ops/op_common/algorithm/topo_match/physical_level_normalize.cc→src/ops/op_common/topo_info/physical_level_normalize.cc+0-0
文件重命名但无更改。