已合并
[feat]ccu resource management compatibility new impl #4420
[feat]ccu resource management compatibility new impl #4420
已合并
guoziqi创建于 25 天前
20 个文件变更+627-233
@@ -20,14 +20,14 @@ Kernel注册和启动接口需要传入`CcuInsHandle`(CCU实例句柄)。当
203. **绑定CCU实例**203. **绑定CCU实例**
21 21 
22 - 调用[HcclCommAssignCcuIns](../comms_domain_resource_mgmt/HcclCommAssignCcuIns.md)将实例绑定到通信域。绑定成功后,实例所有权和销毁责任转移给通信域。22 - 调用[HcclCommAssignCcuIns](../comms_domain_resource_mgmt/HcclCommAssignCcuIns.md)将实例绑定到通信域。绑定成功后,实例所有权和销毁责任转移给通信域。
23- - 创建实例的调用方可继续使用`HcommCcuInsCreate`返回的句柄注册和启动Kernel,但不得再自行销毁实例。仅持有通信域句柄时,可调用[HcclCommQueryCcuIns](../comms_domain_resource_mgmt/HcclCommQueryCcuIns.md)获取已绑定的实例句柄。23+ - 创建实例的调用方可继续使用`HcommCcuInsCreate`返回的句柄注册和启动Kernel,但不得再自行销毁实例。仅持有通信域句柄时,可调用[HcclCommQueryAssignedCcuIns](../comms_domain_resource_mgmt/HcclCommQueryAssignedCcuIns.md)获取已绑定的实例句柄。
24 24 
25- 通过通信域查询实例句柄的典型用法:25+ 通过通信域查询已绑定实例句柄的典型用法:
26 26 
27 ```c27 ```c
28 CcuInsHandle insHandle = 0;28 CcuInsHandle insHandle = 0;
29 uint32_t insNum = 0;29 uint32_t insNum = 0;
30- HcclResult ret = HcclCommQueryCcuIns(comm, &insHandle, &insNum);30+ HcclResult ret = HcclCommQueryAssignedCcuIns(comm, &insHandle, &insNum);
31 // 当前一个通信域最多绑定1个CCU实例,查询成功时insNum为1。31 // 当前一个通信域最多绑定1个CCU实例,查询成功时insNum为1。
32 if (ret != HCCL_SUCCESS || insNum != 1) {32 if (ret != HCCL_SUCCESS || insNum != 1) {
33 // 错误处理33 // 错误处理
@@ -36,7 +36,8 @@ Kernel注册和启动接口需要传入`CcuInsHandle`(CCU实例句柄)。当
36 36 
37> 该接口属于HCCL层(不在`Hcomm*`/`Ccu*`系列内),暂未提供独立API参考页面。37> 该接口属于HCCL层(不在`Hcomm*`/`Ccu*`系列内),暂未提供独立API参考页面。
38> 完整签名以头文件`include/hccl/hccl_ccu_res.h`为准。38> 完整签名以头文件`include/hccl/hccl_ccu_res.h`为准。
39-> 如果通信域未绑定CCU实例,`HcclCommQueryCcuIns`返回`HCCL_E_UNAVAIL`。39+> 如果通信域未绑定CCU实例,`HcclCommQueryAssignedCcuIns`返回`HCCL_E_UNAVAIL`。
40+> 此外,[HcclCommQueryCcuIns](../comms_domain_resource_mgmt/HcclCommQueryCcuIns.md)用于查询通信域自有的CCU实例(未创建时创建CCU实例),与本接口查询的绑定实例相互独立。
40 41 
41## 接口调用顺序42## 接口调用顺序
42 43 
@@ -20,7 +20,7 @@
20 20 
21## 功能说明21## 功能说明
22 22 
23-将调用方已创建的CCU实例绑定到指定HCCL通信域。绑定成功后,通信域保存该实例句柄,并接管CCU实例的所有权和销毁责任;后续可通过`HcclCommQueryCcuIns`查询该实例。23+将调用方已创建的CCU实例绑定到指定HCCL通信域。绑定成功后,通信域保存该实例句柄,并接管CCU实例的所有权和销毁责任;后续可通过`HcclCommQueryAssignedCcuIns`查询该实例。
24 24 
25每个通信域只能绑定一个CCU实例。通信域已有绑定实例时,本接口不会覆盖原实例。25每个通信域只能绑定一个CCU实例。通信域已有绑定实例时,本接口不会覆盖原实例。
26 26 
@@ -56,7 +56,7 @@ HcclResult HcclCommAssignCcuIns(HcclComm comm, CcuInsHandle insHandle)
56- 绑定失败时,`insHandle`的所有权仍归调用方,调用方负责继续使用或销毁该实例。56- 绑定失败时,`insHandle`的所有权仍归调用方,调用方负责继续使用或销毁该实例。
57- 同一个`insHandle`只能绑定到一个通信域,不得重复绑定到其他通信域。57- 同一个`insHandle`只能绑定到一个通信域,不得重复绑定到其他通信域。
58- 同一个通信域不支持重复绑定。无论新旧句柄是否相同,再次调用本接口均返回`HCCL_E_PARA`,原绑定关系保持不变。58- 同一个通信域不支持重复绑定。无论新旧句柄是否相同,再次调用本接口均返回`HCCL_E_PARA`,原绑定关系保持不变。
59-- 本接口只保证多个`HcclCommAssignCcuIns`调用之间的并发安全。调用方须保证本接口不与`HcclCommQueryCcuIns`或`HcclCommDestroy`并发执行。59+- 本接口只保证多个`HcclCommAssignCcuIns`调用之间的并发安全。调用方须保证本接口不与`HcclCommQueryCcuIns`、`HcclCommQueryAssignedCcuIns`或`HcclCommDestroy`并发执行。
60- `insHandle`须在`comm`所在的当前Device上创建。60- `insHandle`须在`comm`所在的当前Device上创建。
61 61 
62## 调用示例62## 调用示例
@@ -79,7 +79,7 @@ if (hcclRet != HCCL_SUCCESS) {
79// 绑定成功,实例所有权已转移给通信域,调用方不再销毁insHandle。79// 绑定成功,实例所有权已转移给通信域,调用方不再销毁insHandle。
80CcuInsHandle queriedInsHandle = 0;80CcuInsHandle queriedInsHandle = 0;
81uint32_t insNum = 0;81uint32_t insNum = 0;
82-hcclRet = HcclCommQueryCcuIns(comm, &queriedInsHandle, &insNum);82+hcclRet = HcclCommQueryAssignedCcuIns(comm, &queriedInsHandle, &insNum);
83if (hcclRet != HCCL_SUCCESS || insNum != 1 ||83if (hcclRet != HCCL_SUCCESS || insNum != 1 ||
84 queriedInsHandle != insHandle) {84 queriedInsHandle != insHandle) {
85 HcclCommDestroy(comm);85 HcclCommDestroy(comm);
@@ -0,0 +1,107 @@
1+# HcclCommQueryAssignedCcuIns
2+ 
3+## 产品支持情况
4+ 
5+<!-- npu="950" id1 -->
6+- Ascend 950PR/Ascend 950DT:支持
7+<!-- end id1 -->
8+<!-- npu="A3" id2 -->
9+- Atlas A3 训练系列产品/Atlas A3 推理系列产品:不支持
10+<!-- end id2 -->
11+<!-- npu="910b" id3 -->
12+- Atlas A2 训练系列产品/Atlas A2 推理系列产品:不支持
13+<!-- end id3 -->
14+<!-- npu="910" id4 -->
15+- Atlas 训练系列产品:不支持
16+<!-- end id4 -->
17+<!-- npu="310p" id5 -->
18+- Atlas 推理系列产品:不支持
19+<!-- end id5 -->
20+ 
21+## 功能说明
22+ 
23+查询指定HCCL通信域通过新方式绑定的CCU实例句柄,返回实例句柄用于后续CCU Kernel的注册与下发。
24+ 
25+新方式指:调用方先通过`HcommCcuInsCreate``HcommCcuInsCreateDefault`创建CCU实例,再通过[HcclCommAssignCcuIns](HcclCommAssignCcuIns.md)将实例绑定到通信域。本接口查询的就是该绑定关系所对应的实例句柄(assignedCcuInsHandle)。查询成功时固定返回1个实例:insHandles[0]为已绑定的实例句柄,*insNum为1。
26+ 
27+本接口不会创建CCU实例。若通信域尚未通过`HcclCommAssignCcuIns`绑定任何CCU实例,本接口返回HCCL_E_UNAVAIL。
28+ 
29+> [!NOTE]注意
30+> - 本接口查询的是通过`HcclCommAssignCcuIns`绑定的CCU实例,与通信域自有的CCU实例(由[HcclCommQueryCcuIns](HcclCommQueryCcuIns.md)查询/创建)相互独立。若需查询通信域自有的实例,请使用`HcclCommQueryCcuIns`。
31+> - 查询结果仅供借用,不转移实例所有权。已绑定的CCU实例所有权归通信域,由通信域负责释放;调用者不可销毁该实例,否则会造成对同一实例的重复释放,破坏通信域的资源管理。
32+ 
33+## 函数原型
34+ 
35+```c
36+HcclResult HcclCommQueryAssignedCcuIns(HcclComm comm, CcuInsHandle *insHandles, uint32_t *insNum)
37+```
38+ 
39+## 参数说明
40+ 
41+| 参数名 | 输入/输出 | 描述 |
42+| --- | --- | --- |
43+| comm | 输入 | 通信域句柄,不可为nullptr。<br>HcclComm类型的定义如下:<br>typedef void *HcclComm; |
44+| insHandles | 输出 | CCU实例句柄数组,不可为nullptr。查询成功后insHandles[0]返回已绑定的CCU实例句柄。<br>调用方需分配至少1个CcuInsHandle元素的空间。<br>CcuInsHandle类型的定义如下:<br>typedef uint64_t CcuInsHandle; |
45+| insNum | 输出 | CCU实例数量,不可为nullptr。查询成功后返回值固定为1。 |
46+ 
47+## 返回值
48+ 
49+[HcclResult](../../../comm_mgr_c/data_type_definition/HcclResult.md):接口成功返回HCCL_SUCCESS,其他失败。
50+ 
51+- 当comm、insHandles或insNum为nullptr时,返回HCCL_E_PTR。
52+- 当通信域代际不支持CCU(早于Ascend 950PR/Ascend 950DT)时,返回HCCL_E_NOT_SUPPORT。
53+- 当通信域尚未通过`HcclCommAssignCcuIns`绑定CCU实例时,返回HCCL_E_UNAVAIL,本接口不会创建实例。
54+ 
55+## 约束说明
56+ 
57+1. CCU特性仅在Ascend 950PR/Ascend 950DT及以上代际支持,在更早代际的通信域上调用返回HCCL_E_NOT_SUPPORT。
58+2. 调用本接口前,须已通过`HcommCcuInsCreate``HcommCcuInsCreateDefault`创建CCU实例,并通过`HcclCommAssignCcuIns`绑定到通信域;否则返回HCCL_E_UNAVAIL。
59+3. 本接口查询的CCU实例与通信域自有的CCU实例(由`HcclCommQueryCcuIns`查询/创建)相互独立,二者不会相互覆盖或影响。
60+4. 返回的CCU实例句柄仅供借用,所有权仍归通信域所有,由通信域负责释放。调用者不可销毁该实例,否则会造成对同一实例的重复释放,破坏通信域的资源管理。
61+5. 本接口不保证与`HcclCommAssignCcuIns``HcclCommDestroy`并发执行的安全。调用方须保证本接口不与`HcclCommAssignCcuIns``HcclCommDestroy`并发执行。
62+ 
63+## 调用示例
64+ 
65+```c
66+// comm 需通过 HcclCommInitClusterInfo 等接口完成初始化,此处仅为示例占位
67+HcclComm comm = /* 已初始化的通信域句柄 */;
68+ 
69+// 1. 新方式:先创建CCU实例,再绑定到通信域
70+CcuInsHandle insHandle = 0;
71+CcuResult ccuRet = HcommCcuInsCreateDefault(NULL, 0, &insHandle);
72+if (ccuRet != CCU_SUCCESS) {
73+ return ccuRet;
74+}
75+ 
76+HcclResult hcclRet = HcclCommAssignCcuIns(comm, insHandle);
77+if (hcclRet != HCCL_SUCCESS) {
78+ // 绑定失败,实例所有权仍归调用方。
79+ HcommCcuInsDestroy(insHandle);
80+ return hcclRet;
81+}
82+// 绑定成功,实例所有权已转移给通信域,调用方不再销毁insHandle。
83+ 
84+// 2. 查询已绑定的CCU实例(新方式)
85+CcuInsHandle queriedInsHandle = 0;
86+uint32_t insNum = 0;
87+hcclRet = HcclCommQueryAssignedCcuIns(comm, &queriedInsHandle, &insNum);
88+// 当前实现固定返回1个CCU实例,insNum != 1视为失败
89+if (hcclRet != HCCL_SUCCESS || insNum != 1) {
90+ // 错误处理:接口失败返回原始错误码,接口成功但实例数异常返回内部错误码
91+ HcclCommDestroy(comm);
92+ return (hcclRet != HCCL_SUCCESS) ? hcclRet : HCCL_E_INTERNAL;
93+}
94+ 
95+// 校验查询到的句柄与绑定时的句柄一致
96+if (queriedInsHandle != insHandle) {
97+ HcclCommDestroy(comm);
98+ return HCCL_E_INTERNAL;
99+}
100+ 
101+// 使用queriedInsHandle注册并下发CCU Kernel
102+CcuResult regStartRet = HcommCcuKernelRegisterStart(queriedInsHandle);
103+// ...
104+ 
105+// 销毁通信域时,由通信域销毁已绑定的CCU实例。
106+return HcclCommDestroy(comm);
107+```
@@ -20,12 +20,15 @@
20 20 
21## 功能说明21## 功能说明
22 22 
23-查询指定HCCL通信域已绑定的CCU实例,返回实例句柄用于后续CCU Kernel的注册与下发。23+查询指定HCCL通信域自有的CCU实例句柄,返回实例句柄用于后续CCU Kernel的注册与下发。
24 24 
25-当前一个通信域最多绑定一个CCU实例,因此查询成功时固定返回1个实例:insHandles[0]为已绑定的实例句柄,*insNum为1。25+通信域在初始化阶段不再主动创建CCU实例。本接口查询通信域自有的CCU实例句柄(即以固资源量创建、由通信域自有的句柄);当该句柄尚未存在(值为`0`)时,接口会根据通信域的算子扩展模式(opExpansionMode)创建一个CCU实例并保存到通信域随后返回该句柄。查询成功时固定返回1个实例:insHandles[0]为通信域自有的实例句柄,*insNum为1。
26+ 
27+当前一个通信域最多持有一个自有的CCU实例,多次查询返回同一句柄,不会重复创建。
26 28 
27> [!NOTE]注意29> [!NOTE]注意
28-> 查询结果仅供借用,不转移实例所权。返回的CCU实例所有权归通信域信域负责释放;调用者不可销毁该实例,否则会造成对同一实例的重复释放破坏通信域的资源管理30+> - 本接口查询的是通信域自有的CCU实例,过[HcclCommAssignCcuIns](HcclCommAssignCcuIns.md)绑定的CCU实例(新版本)相互独立。若需查询通过`HcclCommAssignCcuIns`绑定的实例,请使用[HcclCommQueryAssignedCcuIns](HcclCommQueryAssignedCcuIns.md)
31+> - 查询结果仅供借用,不转移实例所有权。创建的CCU实例所有权归通信域,由通信域负责释放;调用者不可销毁该实例,否则会造成对同一实例的重复释放,破坏通信域的资源管理。
29 32 
30## 函数原型33## 函数原型
31 34 
@@ -38,7 +41,7 @@ HcclResult HcclCommQueryCcuIns(HcclComm comm, CcuInsHandle *insHandles, uint32_t
38| 参数名 | 输入/输出 | 描述 |41| 参数名 | 输入/输出 | 描述 |
39| --- | --- | --- |42| --- | --- | --- |
40| comm | 输入 | 通信域句柄,不可为nullptr。<br>HcclComm类型的定义如下:<br>typedef void *HcclComm; |43| comm | 输入 | 通信域句柄,不可为nullptr。<br>HcclComm类型的定义如下:<br>typedef void *HcclComm; |
41-| insHandles | 输出 | CCU实例句柄数组,不可为nullptr。查询成功后insHandles[0]返回已绑定的CCU实例句柄。<br>CcuInsHandle类型的定义如下:<br>typedef uint64_t CcuInsHandle; |44+| insHandles | 输出 | CCU实例句柄数组,不可为nullptr。查询成功后insHandles[0]返回通信域自有的CCU实例句柄(若调用前不存在则创建)。<br>调用方需分配至少1个CcuInsHandle元素的空间。<br>CcuInsHandle类型的定义如下:<br>typedef uint64_t CcuInsHandle; |
42| insNum | 输出 | CCU实例数量,不可为nullptr。查询成功后返回值固定为1。 |45| insNum | 输出 | CCU实例数量,不可为nullptr。查询成功后返回值固定为1。 |
43 46 
44## 返回值47## 返回值
@@ -47,13 +50,21 @@ HcclResult HcclCommQueryCcuIns(HcclComm comm, CcuInsHandle *insHandles, uint32_t
47 50 
48- 当comm、insHandles或insNum为nullptr时,返回HCCL_E_PTR。51- 当comm、insHandles或insNum为nullptr时,返回HCCL_E_PTR。
49- 当通信域代际不支持CCU(早于Ascend 950PR/Ascend 950DT)时,返回HCCL_E_NOT_SUPPORT。52- 当通信域代际不支持CCU(早于Ascend 950PR/Ascend 950DT)时,返回HCCL_E_NOT_SUPPORT。
50-- 当通信域未绑定CCU实例时,返回HCCL_E_UNAVAIL。53+- 当通信域的自有CCU实例句柄为`0`、且算子扩展模式未使能CCU时,返回HCCL_E_UNAVAIL,不创建实例
54+- 当创建CCU实例失败时,返回创建接口的原始错误码(透传CcuResult,如CCU_E_DRV_BUSY、CCU_E_UNAVAIL等)。
55+ 
56+> [!NOTE]资源不足处理
57+> - 当算子扩展模式为CCU_MS且CCU资源不足以创建CCU_MS实例时,本接口会自动降级到CCU_SCHED重试创建一次。
58+> - 当CCU_SCHED实例也无法创建(资源仍不足)时,返回HCCL_E_UNAVAIL。**本接口不会自动降级到AICPU_TS**,调用方需在收到HCCL_E_UNAVAIL后自行决定是否降级。
51 59 
52## 约束说明60## 约束说明
53 61 
541. CCU特性仅在Ascend 950PR/Ascend 950DT及以上代际支持,在更早代际的通信域上调用返回HCCL_E_NOT_SUPPORT。621. CCU特性仅在Ascend 950PR/Ascend 950DT及以上代际支持,在更早代际的通信域上调用返回HCCL_E_NOT_SUPPORT。
55-2. 需在通信域完成初始化并绑定CCU实例后调用,否则返回HCCL_E_UNAVAIL。63+2. 需在通信域完成初始化后调用。当通信域的自有CCU实例尚未创建时本接口会创建;若通信域展开模式未使能CCU,返回HCCL_E_UNAVAIL。
56-3. 返回的CCU实例句柄仅供借用,所有权仍归信域所有,由通信域负责释放。调用者不可销毁该实例,否则造成对同一实例的重复释放,破坏通信域的资源管理64+3. 本接口查询的CCU实例过`HcclCommAssignCcuIns`绑定的CCU实例相互独立二者不相互覆盖或影响
65+4. 返回的CCU实例句柄仅供借用,所有权仍归通信域所有,由通信域负责释放。调用者不可销毁该实例,否则会造成对同一实例的重复释放,破坏通信域的资源管理。
66+5. 本接口仅保证多次`HcclCommQueryCcuIns`调用之间的幂等性(已创建则直接返回,不重复创建)。调用方须保证本接口不与`HcclCommAssignCcuIns``HcclCommDestroy`并发执行。
67+6. CCU资源不足时本接口仅在CCU_MS→CCU_SCHED之间自动降级一次,不会自动降级到AICPU_TS。调用方收到HCCL_E_UNAVAIL后需自行处理是否降级到AICPU_TS模式。
57 68 
58## 调用示例69## 调用示例
59 70 
@@ -63,8 +74,12 @@ HcclComm comm = /* 已初始化的通信域句柄 */;
63CcuInsHandle insHandle = 0;74CcuInsHandle insHandle = 0;
64uint32_t insNum = 0;75uint32_t insNum = 0;
65 76 
66-// 查询通信域已绑定的CCU实例77+// 查询通信域自有的CCU实例;若尚未创建则按通信域展开模式创建
67HcclResult ret = HcclCommQueryCcuIns(comm, &insHandle, &insNum);78HcclResult ret = HcclCommQueryCcuIns(comm, &insHandle, &insNum);
79+if (ret == HCCL_E_UNAVAIL) {
80+ // CCU资源不足,无法创建CCU实例,调用方需自行处理是否降级到AICPU_TS模式
81+ return fallbackToAicpuTs(comm, ...);
82+}
68// 当前实现固定返回1个CCU实例,insNum != 1视为失败83// 当前实现固定返回1个CCU实例,insNum != 1视为失败
69if (ret != HCCL_SUCCESS || insNum != 1) {84if (ret != HCCL_SUCCESS || insNum != 1) {
Z
Zzangyan11 天前

🔴 [严重] [样例代码完整性] HcclCommQueryCcuIns.md:80

文档新增约束 #6(第67行)和 NOTE 块(第56-58行)明确要求:

调用方收到HCCL_E_UNAVAIL后需自行处理是否降级到AICPU_TS模式

但样例代码仅简单 return 错误码,未展示 HCCL_E_UNAVAIL 时的降级处理模式。

建议在错误处理分支中补充 HCCL_E_UNAVAIL 的降级示例:

HcclResult ret = HcclCommQueryCcuIns(comm, &insHandle, &insNum);
if (ret == HCCL_E_UNAVAIL) {
    // CCU资源不足且无法自动降级到AICPU_TS,调用方自行处理降级
    // 例如:回退到AICPU_TS模式执行
    return fallbackToAicpuTs(comm, ...);
}
if (ret != HCCL_SUCCESS || insNum != 1) {
    return (ret != HCCL_SUCCESS) ? ret : HCCL_E_INTERNAL;
}
likedislike
70 // 错误处理:接口失败返回原始错误码,接口成功但实例数异常返回内部错误码85 // 错误处理:接口失败返回原始错误码,接口成功但实例数异常返回内部错误码
@@ -24,6 +24,7 @@
24- [HcclCommMemReg](HcclCommMemReg.md)24- [HcclCommMemReg](HcclCommMemReg.md)
25- [HcclCommAssignCcuIns](HcclCommAssignCcuIns.md)25- [HcclCommAssignCcuIns](HcclCommAssignCcuIns.md)
26- [HcclCommQueryCcuIns](HcclCommQueryCcuIns.md)26- [HcclCommQueryCcuIns](HcclCommQueryCcuIns.md)
27+- [HcclCommQueryAssignedCcuIns](HcclCommQueryAssignedCcuIns.md)
27- [HcclTeamCreateDescInit](HcclTeamCreateDescInit.md)28- [HcclTeamCreateDescInit](HcclTeamCreateDescInit.md)
28- [HcclTeamCreateChannelsDescInit](HcclTeamCreateChannelsDescInit.md)29- [HcclTeamCreateChannelsDescInit](HcclTeamCreateChannelsDescInit.md)
29- [HcclWorldTeamCreate](HcclWorldTeamCreate.md)30- [HcclWorldTeamCreate](HcclWorldTeamCreate.md)
@@ -32,16 +32,31 @@ extern "C" {
32extern HcclResult HcclCommAssignCcuIns(HcclComm comm, CcuInsHandle insHandle);32extern HcclResult HcclCommAssignCcuIns(HcclComm comm, CcuInsHandle insHandle);
33 33 
34/**34/**
35- * @brief 查询指定 HCCL 通信域已绑定的 CCU 实例。35+ * @brief 查询指定 HCCL 通信域自有的 CCU 实例句柄
36 * @param[in] comm HCCL 通信域句柄,不可为 nullptr。36 * @param[in] comm HCCL 通信域句柄,不可为 nullptr。
37 * @param[out] insHandles 查询成功后返回 CCU 实例句柄数组,不可为 nullptr。37 * @param[out] insHandles 查询成功后返回 CCU 实例句柄数组,不可为 nullptr。
38 * @param[out] insNum 查询成功后返回 CCU 实例数量,不可为 nullptr。38 * @param[out] insNum 查询成功后返回 CCU 实例数量,不可为 nullptr。
39 * @return HcclResult。39 * @return HcclResult。
40+ * @note 查询通信域自有的
41+ * ccuInsHandle;若通信域未创建CCU实例且通信域展开模式为CCU,则按通信域展开模式创建CCU实例后返回。
40 * @note 返回的 CCU 实例句柄仅供借用,不转移所有权,调用方不得销毁该句柄。42 * @note 返回的 CCU 实例句柄仅供借用,不转移所有权,调用方不得销毁该句柄。
41 * @note 调用方必须保证本接口不与 HcclCommAssignCcuIns 或 HcclCommDestroy 并发执行。43 * @note 调用方必须保证本接口不与 HcclCommAssignCcuIns 或 HcclCommDestroy 并发执行。
42 */44 */
43extern HcclResult HcclCommQueryCcuIns(HcclComm comm, CcuInsHandle* insHandles, uint32_t* insNum);45extern HcclResult HcclCommQueryCcuIns(HcclComm comm, CcuInsHandle* insHandles, uint32_t* insNum);
44 46 
47+/**
48+ * @brief 查询指定 HCCL 通信域通过HcclCommAssignCcuIns绑定的 CCU 实例句柄。
49+ * @param[in] comm HCCL 通信域句柄,不可为 nullptr。
50+ * @param[out] insHandles 查询成功后返回 CCU 实例句柄数组,不可为 nullptr。
51+ * @param[out] insNum 查询成功后返回 CCU 实例数量,不可为 nullptr。
52+ * @return HcclResult。
53+ * @note 查询通过HcclCommAssignCcuIns绑定给通信域的CCU实例句柄;未绑定时返回 HCCL_E_UNAVAIL,不创建句柄。
54+ * 调用方须先通过 HcommCcuInsCreate + HcclCommAssignCcuIns 绑定。
55+ * @note 返回的 CCU 实例句柄仅供借用,不转移所有权,调用方不得销毁该句柄。
56+ * @note 调用方必须保证本接口不与 HcclCommAssignCcuIns 或 HcclCommDestroy 并发执行。
57+ */
58+extern HcclResult HcclCommQueryAssignedCcuIns(HcclComm comm, CcuInsHandle* insHandles, uint32_t* insNum);
59+ 
45#ifdef __cplusplus60#ifdef __cplusplus
46}61}
47#endif // __cplusplus62#endif // __cplusplus
@@ -65,7 +65,7 @@ static CcuResult CcuKernelTryRegister(
65{65{
66 CCU_EXCEPTION_HANDLE_BEGIN66 CCU_EXCEPTION_HANDLE_BEGIN
67 auto& kernelMgr = hcomm::CcuKernelMgr::GetInstance(devLogicId);67 auto& kernelMgr = hcomm::CcuKernelMgr::GetInstance(devLogicId);
68- CCU_CHK_RET(kernelMgr.Register(*resPack, dieId, kernelFuncName, kernelFunc, kernelArgs, argNum, newHandle));68+ CCU_CHK_RET(kernelMgr.Register(*resPack, dieId, kernelFuncName, kernelFunc, kernelArgs, argNum, ccuIns, newHandle));
69 CCU_CHK_RET(ccuIns->SaveKernel(newHandle));69 CCU_CHK_RET(ccuIns->SaveKernel(newHandle));
70 CCU_EXCEPTION_HANDLE_END70 CCU_EXCEPTION_HANDLE_END
71 return CcuResult::CCU_SUCCESS;71 return CcuResult::CCU_SUCCESS;
@@ -191,7 +191,7 @@ CcuResult HcommCcuKernelQueryResReq(
191 return CcuResult::CCU_SUCCESS;191 return CcuResult::CCU_SUCCESS;
192}192}
193 193 
194-// Deprecated: legacy internal CCU instance creation path.194+// CCU 实例类型创建 ccu 实例(兼容 hccl 旧版本)
195CcuResult HcommCcuInsCreateLegacy(const CcuInstanceType insType, CcuInsHandle* ccuInsHandle)195CcuResult HcommCcuInsCreateLegacy(const CcuInstanceType insType, CcuInsHandle* ccuInsHandle)
196{196{
197 CCU_CHK_PTR_NULL(ccuInsHandle);197 CCU_CHK_PTR_NULL(ccuInsHandle);
@@ -66,6 +66,7 @@ CcuResult CcuInstance::InitByInsType(const CcuInstanceType insType)
66 CCU_CHK_RET(FillTotalResDescs());66 CCU_CHK_RET(FillTotalResDescs());
67 }67 }
68 68 
69+ isFixedResNum_ = true;
69 return CcuResult::CCU_SUCCESS;70 return CcuResult::CCU_SUCCESS;
70}71}
71 72 
@@ -44,6 +44,9 @@ public:
44 const std::vector<CcuKernelHandle>& GetUntranslatedKernels();44 const std::vector<CcuKernelHandle>& GetUntranslatedKernels();
45 void SetHandle(CcuInsHandle insHandle);45 void SetHandle(CcuInsHandle insHandle);
46 46 
47+ // 是否使用固定资源数量创建(InitByInsType 路径置 true,其余路径保持 false)
48+ bool IsFixedResNum() const { return isFixedResNum_; }
49+ 
47 CcuResult BeginRegister();50 CcuResult BeginRegister();
48 CcuResult CheckRegistering() const;51 CcuResult CheckRegistering() const;
49 CcuResult EndRegister();52 CcuResult EndRegister();
@@ -62,6 +65,7 @@ private:
62 std::vector<CcuKernelHandle> kernelHandles_{};65 std::vector<CcuKernelHandle> kernelHandles_{};
63 std::vector<CcuKernelHandle> untranslatedKernelHandles_{};66 std::vector<CcuKernelHandle> untranslatedKernelHandles_{};
64 std::array<CcuResDesc, CCU_MAX_IODIE_NUM> totalResDescs_{}; // ccuIns持有的全部资源描述符67 std::array<CcuResDesc, CCU_MAX_IODIE_NUM> totalResDescs_{}; // ccuIns持有的全部资源描述符
68+ bool isFixedResNum_{false}; // InitByInsType 路径置 true(使用固定资源数量创建),其余保持 false
65};69};
66 70 
67} // namespace hcomm71} // namespace hcomm
@@ -130,7 +130,7 @@ HcclResult CcuKernelMgr::Deinit()
130 130 
131CcuResult CcuKernelMgr::Register(131CcuResult CcuKernelMgr::Register(
132 CcuResPack& resPack, const uint32_t dieId, const char* kernelFuncName, const void* kernelFunc,132 CcuResPack& resPack, const uint32_t dieId, const char* kernelFuncName, const void* kernelFunc,
133- const void** kernelArgs, const uint32_t argNum, CcuKernelHandle& kernelHandle)133+ const void** kernelArgs, const uint32_t argNum, CcuInstance* ccuIns, CcuKernelHandle& kernelHandle)
134{134{
135 // 允许kernelFuncName为空,此时传递默认名称135 // 允许kernelFuncName为空,此时传递默认名称
136 CCU_CHK_PTR_NULL(kernelFunc);136 CCU_CHK_PTR_NULL(kernelFunc);
@@ -143,7 +143,7 @@ CcuResult CcuKernelMgr::Register(
143 143 
144 // 注意处理时序,需要先重置后处理rep144 // 注意处理时序,需要先重置后处理rep
145 std::unique_lock<std::mutex> lock(kernelMapMutex_);145 std::unique_lock<std::mutex> lock(kernelMapMutex_);
146- CCU_CHK_RET(BuildKernel(dieId, kernelFuncName, kernelFunc, kernelArgs, argNum));146+ CCU_CHK_RET(BuildKernel(dieId, kernelFuncName, kernelFunc, kernelArgs, argNum, ccuIns));
147 147 
148 CcuResult ret = AllocRes(resPack);148 CcuResult ret = AllocRes(resPack);
149 if (ret != CcuResult::CCU_SUCCESS) {149 if (ret != CcuResult::CCU_SUCCESS) {
@@ -160,7 +160,7 @@ CcuResult CcuKernelMgr::Register(
160 160 
161CcuResult CcuKernelMgr::BuildKernel(161CcuResult CcuKernelMgr::BuildKernel(
162 const uint32_t dieId, const char* kernelFuncName, const void* kernelFunc, const void** kernelArgs,162 const uint32_t dieId, const char* kernelFuncName, const void* kernelFunc, const void** kernelArgs,
163- const uint32_t argNum)163+ const uint32_t argNum, CcuInstance* ccuIns)
164{164{
165 currKernel_ = std::make_unique<CcuKernel>(); // 重置待构建kernel165 currKernel_ = std::make_unique<CcuKernel>(); // 重置待构建kernel
166 // 执行算法流程时将资源占用临时记录在 die 0,后续确定实际 die 并迁移资源166 // 执行算法流程时将资源占用临时记录在 die 0,后续确定实际 die 并迁移资源
@@ -184,13 +184,11 @@ CcuResult CcuKernelMgr::BuildKernel(
184 }184 }
185 185 
186 currKernel_->FlushClosablePendingIfs(); // 处理未闭合的if186 currKernel_->FlushClosablePendingIfs(); // 处理未闭合的if
187- int hcclVersion = 0;187+ if (ccuIns != nullptr && ccuIns->IsFixedResNum()) {
188- CCU_CHK_RET(GetHcclVersionForCcuKernelMgr(hcclVersion));188+ // 按固定资源数量创建的 ccu instance,外部 dieId 始终为 0,从 channel 获取实际 dieId
L
Llaodazhao111 天前

可以在通信域的ccu ins时候把die id保存到ins里吗

likedislike
189- if (hcclVersion <= MAX_HCCL_VERSION_USING_CCU_RES_STATIC_ALLOC) {
190- // 9.1.0 及之前版本的外部 dieId 始终为 0,需要从 channel 中获取实际 dieId
191 CCU_CHK_RET(currKernel_->ApplyDieFromChannels());189 CCU_CHK_RET(currKernel_->ApplyDieFromChannels());
192 } else {190 } else {
193- // 校验所有 channel 使用相同的 die,然后将资源占用从 die 0 迁移到指定 die191+ // 按需创建的 ccu instance,校验所有 channel 使用相同的 die,然后将资源占用从 die 0 迁移到指定 die
194 CCU_CHK_RET(currKernel_->ValidateAndApplyDie(dieId));192 CCU_CHK_RET(currKernel_->ValidateAndApplyDie(dieId));
195 }193 }
196 CCU_CHK_RET(PrepareConstValueResources()); // 记录翻译过程所需常量并申请对应资源194 CCU_CHK_RET(PrepareConstValueResources()); // 记录翻译过程所需常量并申请对应资源
@@ -219,7 +217,7 @@ CcuResult CcuKernelMgr::GetKernelResourceRequest(
219 std::unique_ptr<CcuKernel>& kernel_;217 std::unique_ptr<CcuKernel>& kernel_;
220 } guard(currKernel_);218 } guard(currKernel_);
221 219 
222- CCU_CHK_RET(BuildKernel(dieId, kernelFuncName, kernelFunc, kernelArgs, argNum));220+ CCU_CHK_RET(BuildKernel(dieId, kernelFuncName, kernelFunc, kernelArgs, argNum, nullptr));
223 resReq = currKernel_->GetResourceRequest();221 resReq = currKernel_->GetResourceRequest();
224 const uint32_t kernelInstrCount = currKernel_->GetInstrCount();222 const uint32_t kernelInstrCount = currKernel_->GetInstrCount();
225 const uint32_t translatorInstrCount = CcuRepTranslator::GetInstrNum(devLogicId_);223 const uint32_t translatorInstrCount = CcuRepTranslator::GetInstrNum(devLogicId_);
@@ -16,6 +16,7 @@
16 16 
17#include "ccu_kernel.h"17#include "ccu_kernel.h"
18#include "../ccu_instance/ccu_res_pack.h"18#include "../ccu_instance/ccu_res_pack.h"
19+#include "../ccu_instance/ccu_instance.h"
19 20 
20#include "ccu_dev_mgr_imp.h"21#include "ccu_dev_mgr_imp.h"
21#include "../ccu_representation/reps/translator/ccu_rep_translator_v1.h"22#include "../ccu_representation/reps/translator/ccu_rep_translator_v1.h"
@@ -34,12 +35,12 @@ public:
34 HcclResult Deinit();35 HcclResult Deinit();
35 36 
36 CcuResult Register(37 CcuResult Register(
37- CcuResPack& resPack, uint32_t dieId, const char* kernelFuncName, const void* kernelFunc,38+ CcuResPack& resPack, const uint32_t dieId, const char* kernelFuncName, const void* kernelFunc,
38- const void** kernelArgs, const uint32_t argNum, CcuKernelHandle& kernelHandle);39+ const void** kernelArgs, const uint32_t argNum, CcuInstance* ccuIns, CcuKernelHandle& kernelHandle);
39 40 
40 CcuResult GetKernelResourceRequest(41 CcuResult GetKernelResourceRequest(
41- uint32_t dieId, const char* kernelFuncName, const void* kernelFunc, const void** kernelArgs, uint32_t argNum,42+ const uint32_t dieId, const char* kernelFuncName, const void* kernelFunc, const void** kernelArgs,
42- CcuResReq& resReq, uint32_t& instrCount);43+ const uint32_t argNum, CcuResReq& resReq, uint32_t& instrCount);
43 44 
44 CcuResult Translate(const std::vector<CcuKernelHandle>& kernelHandles);45 CcuResult Translate(const std::vector<CcuKernelHandle>& kernelHandles);
45 46 
@@ -64,7 +65,8 @@ private:
64 65 
65private:66private:
66 CcuResult BuildKernel(67 CcuResult BuildKernel(
67- uint32_t dieId, const char* kernelFuncName, const void* kernelFunc, const void** kernelArgs, uint32_t argNum);68+ uint32_t dieId, const char* kernelFuncName, const void* kernelFunc, const void** kernelArgs, uint32_t argNum,
69+ CcuInstance* ccuIns);
68 CcuResult PrepareConstValueResources();70 CcuResult PrepareConstValueResources();
69 CcuResult AllocRes(CcuResPack& resPack);71 CcuResult AllocRes(CcuResPack& resPack);
70 72 
@@ -27,7 +27,7 @@ typedef enum {
27} CcuInstanceType;27} CcuInstanceType;
28 28 
29/**29/**
30- * @brief 创建ccu实例旧接口,兼容hccl旧版本30+ * @brief 按 CCU 实例类型创建 ccu 实例兼容 hccl 旧版本)。
31 * @param[in] insType CCU实例类型。31 * @param[in] insType CCU实例类型。
32 * @param[out] ccuInsHandle 创建成功后返回的 CCU 实例句柄,不可为 nullptr。32 * @param[out] ccuInsHandle 创建成功后返回的 CCU 实例句柄,不可为 nullptr。
33 * @return CcuResult。33 * @return CcuResult。
@@ -16,6 +16,50 @@
16 16 
17#include "exception_handler.h"17#include "exception_handler.h"
18#include "ccu_instance_mgr.h"18#include "ccu_instance_mgr.h"
19+#include "ccu_device_res.h"
20+ 
21+static HcclResult CreateCcuInsByFixedResNum(
22+ const char* funcName, const std::string& commId, hccl::MyRank* myRank, CcuInsHandle& ccuInsHandle)
23+{
24+ const auto opExpansionMode = myRank->GetOpExpansionMode();
25+ auto ccuInsType = hccl::OpExpansionModeToCcuInstanceType(opExpansionMode);
26+ if (ccuInsType == CcuInstanceType::CCU_UNUSED) {
27+ HCCL_WARNING(
28+ "[%s] failed to get ccu instance, commId[%s] op expansion mode[%u].", funcName, commId.c_str(),
29+ opExpansionMode);
30+ return HcclResult::HCCL_E_UNAVAIL;
31+ }
32+ 
33+ CcuInsHandle newHandle = 0;
34+ auto ccuRet = CcuResult::CCU_SUCCESS;
35+ // CCU_MS 资源不足(CCU_E_UNAVAIL)时降级到 CCU_SCHED 重试一次;CCU_SCHED 不再继续降级
36+ while (true) {
37+ ccuRet = HcommCcuInsCreateLegacy(ccuInsType, &newHandle);
38+ if (ccuRet != CcuResult::CCU_E_UNAVAIL || ccuInsType != CcuInstanceType::CCU_MS) {
39+ break;
40+ }
41+ HCCL_WARNING(
42+ "[%s] ccu instance resource unavailable for CCU_MS, fallback to CCU_SCHED, commId[%s].", funcName,
43+ commId.c_str());
44+ ccuInsType = CcuInstanceType::CCU_SCHED;
45+ }
46+ if (ccuRet == CcuResult::CCU_E_UNAVAIL) {
严正行
严正行严正行20 天前

[LOW] CCU_MS 资源不足时不再回退到 CCU_SCHED 模式

问题分析

  • 第 50 行:CreateCcuInsByFixedResNumCCU_E_UNAVAIL 直接返回错误,不尝试降级。
  • 旧代码 TryInitCcuInstanceLegacy(已删除)在 CCU_E_UNAVAIL 时,若 opExpansionMode_ == CCU_MS_MODE,会将模式降级为 CCU_SCHED_MODE 重试,或回退到 AICPU_TS_MODE
  • 新代码将 CCU 实例创建从 Init 阶段推迟到 QueryCcuIns 阶段,在 Query 时静默修改 opExpansionMode_ 确实不合适,因此移除降级逻辑有一定合理性。
  • 但对使用 CCU_MS 模式的调用方而言,资源不足时从"自动降级到 SCHED"变为"直接报错",是行为变更。调用方需自行处理 HCCL_E_UNAVAIL 并决定降级策略。

建议: 在 HcclCommQueryCcuIns 的对外文档中明确标注"资源不足时返回 HCCL_E_UNAVAIL,不自动降级",帮助调用方知晓需自行处理降级。若确需保留降级能力,可考虑在 CreateCcuInsByFixedResNum 中对 CCU_MS→CCU_SCHED 做一次重试。

likedislike
47+ HCCL_WARNING(
48+ "[%s] failed to create ccu instance, resources are unavailable, "
49+ "commId[%s] insType[%d] ret[%d].",
50+ funcName, commId.c_str(), ccuInsType, ccuRet);
51+ return static_cast<HcclResult>(ccuRet);
52+ }
53+ if (ccuRet != CcuResult::CCU_SUCCESS) {
54+ HCCL_ERROR(
55+ "[%s] failed to create ccu instance, commId[%s] insType[%d] ret[%d].", funcName, commId.c_str(), ccuInsType,
56+ ccuRet);
57+ return static_cast<HcclResult>(ccuRet);
58+ }
59+ myRank->SetCcuInstance(newHandle);
60+ ccuInsHandle = newHandle;
61+ return HcclResult::HCCL_SUCCESS;
62+}
19 63 
20/**64/**
21 * @note 职责:集合通信的通信域CCU管理的C接口的C到C++适配65 * @note 职责:集合通信的通信域CCU管理的C接口的C到C++适配
@@ -27,16 +71,15 @@ HcclResult HcclCommQueryCcuIns(HcclComm comm, CcuInsHandle* insHandles, uint32_t
27 HcclUs startut = TIME_NOW();71 HcclUs startut = TIME_NOW();
28 72 
29 CHK_PTR_NULL(comm);73 CHK_PTR_NULL(comm);
74+ CHK_PTR_NULL(insHandles);
75+ CHK_PTR_NULL(insNum);
30 auto* hcclComm = static_cast<hccl::hcclComm*>(comm);76 auto* hcclComm = static_cast<hccl::hcclComm*>(comm);
31 const auto& commId = hcclComm->GetIdentifier();77 const auto& commId = hcclComm->GetIdentifier();
32 HCCL_INFO("[%s] CommId[%s] query ccu instance.", __func__, commId.c_str());78 HCCL_INFO("[%s] CommId[%s] query ccu instance.", __func__, commId.c_str());
33 79 
34- CHK_PTR_NULL(insHandles);
35- CHK_PTR_NULL(insNum);
36- 
37 // CCU不支持A5之前代际80 // CCU不支持A5之前代际
38 if (!hcclComm->IsCommunicatorV2()) {81 if (!hcclComm->IsCommunicatorV2()) {
39- HCCL_WARNING("[%s] is not supported.", __func__);82+ HCCL_RUN_WARNING("[%s] is not supported.", __func__);
40 return HcclResult::HCCL_E_NOT_SUPPORT;83 return HcclResult::HCCL_E_NOT_SUPPORT;
41 }84 }
42 85 
@@ -45,14 +88,14 @@ HcclResult HcclCommQueryCcuIns(HcclComm comm, CcuInsHandle* insHandles, uint32_t
45 auto* myRank = collComm->GetMyRank();88 auto* myRank = collComm->GetMyRank();
46 CHK_PTR_NULL(myRank);89 CHK_PTR_NULL(myRank);
47 90 
48- // 非CCU通信域允许查询,不认是错误91+ // 查询通信域自有的 ccuInsHandle_; 0 时按 opExpansionMode 创建
49 auto ccuInsHandle = myRank->GetCcuInstance();92 auto ccuInsHandle = myRank->GetCcuInstance();
50 if (ccuInsHandle == 0) {93 if (ccuInsHandle == 0) {
51- auto opExpansionMode = myRank->GetOpExpansionMode();94+ auto ret = CreateCcuInsByFixedResNum(__func__, commId, myRank, ccuInsHandle);
52- HCCL_WARNING(95+ if (ret != HcclResult::HCCL_SUCCESS) {
53- "[%s] failed to get ccu instance, commId[%s] op expansion mode[%u].", __func__, commId.c_str(),96+ HCCL_WARNING("[%s] failed to get ccu instance, commId[%s] ret[%d]", __func__, commId.c_str(), ret);
54- opExpansionMode);97+ return ret;
55- return HcclResult::HCCL_E_UNAVAIL;98+ }
56 }99 }
57 100 
58 insHandles[0] = ccuInsHandle;101 insHandles[0] = ccuInsHandle;
@@ -63,6 +106,45 @@ HcclResult HcclCommQueryCcuIns(HcclComm comm, CcuInsHandle* insHandles, uint32_t
63 return HcclResult::HCCL_SUCCESS;106 return HcclResult::HCCL_SUCCESS;
64}107}
65 108 
109+HcclResult HcclCommQueryAssignedCcuIns(HcclComm comm, CcuInsHandle* insHandles, uint32_t* insNum)
110+{
111+ EXCEPTION_HANDLE_BEGIN
112+ 
113+ HcclUs startut = TIME_NOW();
114+ 
115+ CHK_PTR_NULL(comm);
116+ CHK_PTR_NULL(insHandles);
117+ CHK_PTR_NULL(insNum);
118+ auto* hcclComm = static_cast<hccl::hcclComm*>(comm);
119+ const auto& commId = hcclComm->GetIdentifier();
120+ HCCL_INFO("[%s] CommId[%s] query assigned ccu instance.", __func__, commId.c_str());
121+ 
122+ // CCU不支持A5之前代际
123+ if (!hcclComm->IsCommunicatorV2()) {
124+ HCCL_RUN_WARNING("[%s] is not supported.", __func__);
125+ return HcclResult::HCCL_E_NOT_SUPPORT;
126+ }
127+ 
128+ auto* collComm = hcclComm->GetCollComm();
129+ CHK_PTR_NULL(collComm);
130+ auto* myRank = collComm->GetMyRank();
131+ CHK_PTR_NULL(myRank);
132+ 
133+ // 查询绑定的 assignedCcuInsHandle_;未绑定时返回 UNAVAIL,不创建
134+ auto assignedCcuInsHandle = myRank->GetAssignedCcuInstance();
135+ if (assignedCcuInsHandle == 0) {
136+ HCCL_WARNING("[%s] assigned ccu instance not exist, commId[%s].", __func__, commId.c_str());
137+ return HcclResult::HCCL_E_UNAVAIL;
138+ }
139+ 
140+ insHandles[0] = assignedCcuInsHandle;
141+ *insNum = 1;
142+ HCCL_INFO("[%s] success, take time [%lld]us.", __func__, DURATION_US(TIME_NOW() - startut).count());
143+ 
144+ EXCEPTION_HANDLE_END
145+ return HcclResult::HCCL_SUCCESS;
146+}
147+ 
66HcclResult HcclCommAssignCcuIns(HcclComm comm, CcuInsHandle insHandle)148HcclResult HcclCommAssignCcuIns(HcclComm comm, CcuInsHandle insHandle)
67{149{
68 EXCEPTION_HANDLE_BEGIN150 EXCEPTION_HANDLE_BEGIN
@@ -84,7 +166,7 @@ HcclResult HcclCommAssignCcuIns(HcclComm comm, CcuInsHandle insHandle)
84 }166 }
85 167 
86 if (!hcclComm->IsCommunicatorV2()) {168 if (!hcclComm->IsCommunicatorV2()) {
87- HCCL_WARNING("[%s] is not supported.", __func__);169+ HCCL_RUN_WARNING("[%s] is not supported.", __func__);
88 return HcclResult::HCCL_E_NOT_SUPPORT;170 return HcclResult::HCCL_E_NOT_SUPPORT;
89 }171 }
90 172 
@@ -98,10 +180,10 @@ HcclResult HcclCommAssignCcuIns(HcclComm comm, CcuInsHandle insHandle)
98 static std::mutex assignCcuInsMutex;180 static std::mutex assignCcuInsMutex;
严正行
严正行严正行20 天前

[MEDIUM] static std::mutex assignCcuInsMutex 跨通信域序列化 Assign 调用

问题分析

  • 第 183 行:static std::mutex assignCcuInsMutex 是函数内静态变量,所有通信域、所有 Device 上的 HcclCommAssignCcuIns 调用共享同一把锁。
  • 锁内仅操作 myRank->GetAssignedCcuInstance() / SetAssignedCcuInstance(),而 myRank 是每个通信域独立的实例,不存在跨通信域共享状态。
  • 不同通信域的 Assign 调用本可并行执行,却被 static mutex 强制串行化。
  • 虽然 Assign 调用频率低(每个通信域仅一次),影响有限,但设计上仍不必要地扩大了锁粒度。

建议修复: 将锁改为 MyRank 的成员变量(如 std::mutex assignCcuInsMutex_),或使用 per-communicator 锁,使不同通信域的 Assign 可并行。若维持现有设计,在注释中说明为何选用 static mutex。

likedislike
guoziqi
guoziqi
12 天前 评论:
99 std::lock_guard<std::mutex> lock(assignCcuInsMutex);181 std::lock_guard<std::mutex> lock(assignCcuInsMutex);
100 182 
101- auto oldInsHandle = myRank->GetCcuInstance();183+ auto oldInsHandle = myRank->GetAssignedCcuInstance();
102 if (oldInsHandle != 0) {184 if (oldInsHandle != 0) {
103 HCCL_ERROR(185 HCCL_ERROR(
104- "[%s] failed, commId[%s] already has ccu instance[%llu], "186+ "[%s] failed, commId[%s] already has assigned ccu instance[%llu], "
105 "new instance[%llu] will not be assigned.",187 "new instance[%llu] will not be assigned.",
106 __func__, commId.c_str(), static_cast<unsigned long long>(oldInsHandle),188 __func__, commId.c_str(), static_cast<unsigned long long>(oldInsHandle),
107 static_cast<unsigned long long>(insHandle));189 static_cast<unsigned long long>(insHandle));
@@ -117,7 +199,7 @@ HcclResult HcclCommAssignCcuIns(HcclComm comm, CcuInsHandle insHandle)
117 return HcclResult::HCCL_E_NOT_FOUND;199 return HcclResult::HCCL_E_NOT_FOUND;
118 }200 }
119 201 
120- myRank->SetCcuInstance(insHandle);202+ myRank->SetAssignedCcuInstance(insHandle);
121 }203 }
122 204 
123 HCCL_INFO(205 HCCL_INFO(
@@ -121,8 +121,9 @@ MyRank::~MyRank()
121 ~ResourceCleanupGuard() noexcept121 ~ResourceCleanupGuard() noexcept
122 {122 {
123 myRank_.ccuInsHandle_ = 0;123 myRank_.ccuInsHandle_ = 0;
124+ myRank_.assignedCcuInsHandle_ = 0;
124 125 
125- if (!myRank_.useCcuResStaticAlloc_ && myRank_.ccuDrvHandle_) {126+ if (myRank_.ccuDrvHandle_) {
126 myRank_.ccuDrvHandle_ = nullptr; // 先减少引用计数,再尝试关闭127 myRank_.ccuDrvHandle_ = nullptr; // 先减少引用计数,再尝试关闭
127 (void)CcuDeinitFeature(myRank_.devLogicId_);128 (void)CcuDeinitFeature(myRank_.devLogicId_);
128 // 尝试关闭CCU功能,最后一个调用时会关闭CCU驱动129 // 尝试关闭CCU功能,最后一个调用时会关闭CCU驱动
@@ -137,7 +138,7 @@ MyRank::~MyRank()
137 MyRank& myRank_;138 MyRank& myRank_;
138 } cleanupGuard(*this);139 } cleanupGuard(*this);
139 140 
140- if (ccuInsHandle_ != 0) { // 内部清理CCU资源,关闭CCU通道141+ if (ccuInsHandle_ != 0 || assignedCcuInsHandle_ != 0) { // 内部清理CCU资源,关闭CCU通道
141 // 刷新并获取当前线程的 DeviceId142 // 刷新并获取当前线程的 DeviceId
142 int32_t threadDevId = INVALID_INT;143 int32_t threadDevId = INVALID_INT;
143 CHK_RET_NULL(HcclDeviceRefresh(threadDevId));144 CHK_RET_NULL(HcclDeviceRefresh(threadDevId));
@@ -148,10 +149,13 @@ MyRank::~MyRank()
148 CHK_RET_NULL(hrtSetDevice(devLogicId_));149 CHK_RET_NULL(hrtSetDevice(devLogicId_));
149 isDiffDevId = true;150 isDiffDevId = true;
150 }151 }
151- // 销毁 CcuInstance152+ // 销毁通信域自有的 ccuInstance(QueryCcuIns 创建)
152- CcuResult ret = HcommCcuInsDestroy(ccuInsHandle_);153+ if (ccuInsHandle_ != 0) {
153- if (ret != CCU_SUCCESS) {154+ CHK_PRT(static_cast<HcclResult>(HcommCcuInsDestroy(ccuInsHandle_)));
154- HCCL_ERROR("[%s] HcommCcuInsDestroy failed, ret[%d]", __func__, ret);155+ }
156+ // 销毁通过 Assign 绑定的 ccuInstance(所有权已转移给通信域)
157+ if (assignedCcuInsHandle_ != 0 && assignedCcuInsHandle_ != ccuInsHandle_) {
158+ CHK_PRT(static_cast<HcclResult>(HcommCcuInsDestroy(assignedCcuInsHandle_)));
155 }159 }
156 // 切换回原来的 DeviceId160 // 切换回原来的 DeviceId
157 if (isDiffDevId) {161 if (isDiffDevId) {
@@ -208,70 +212,6 @@ HcclResult MyRank::UnregMemByTag(const std::string& tag)
208 return endpointMgr_->UnregMemByTag(tag);212 return endpointMgr_->UnregMemByTag(tag);
209}213}
210 214 
211-constexpr uint32_t DEFAULT_MODE = 0;
212-constexpr uint32_t AICPU_TS_MODE = 2;
213-constexpr uint32_t CCU_MS_MODE = 5;
214-constexpr uint32_t CCU_SCHED_MODE = 6;
215-inline CcuInstanceType OpExpansionModeToCcuInstanceType(uint32_t opExpansionMode)
216-{
217- // 仅作数据类型转换,不做逻辑处理
218- if (opExpansionMode == CCU_SCHED_MODE) {
219- return CcuInstanceType::CCU_SCHED;
220- }
221- 
222- if (opExpansionMode == CCU_MS_MODE) {
223- return CcuInstanceType::CCU_MS;
224- }
225- 
226- return CcuInstanceType::CCU_UNUSED;
227-}
228- 
229-HcclResult MyRank::TryInitCcuInstanceLegacy()
230-{
231- auto ccuInsType = OpExpansionModeToCcuInstanceType(opExpansionMode_);
232- if (ccuInsType == CcuInstanceType::CCU_UNUSED) {
233- ccuInsHandle_ = 0;
234- return HcclResult::HCCL_SUCCESS;
235- }
236- 
237- auto ccuInitRet = HcommCcuInsCreateLegacy(ccuInsType, &ccuInsHandle_);
238- // ccu驱动拉起失败,直接回退至aicpu ts
239- if (ccuInitRet == CcuResult::CCU_E_DRV_BUSY) {
240- opExpansionMode_ = AICPU_TS_MODE;
241- ccuInsHandle_ = 0;
242- HCCL_RUN_WARNING("[MyRank][%s] failed to init ccu driver, fallback to aicpu, rankId[%u].", __func__, rankId_);
243- return HcclResult::HCCL_SUCCESS;
244- }
245- 
246- // ccu通信域数量过多,导致资源不足
247- if (CCU_CHK_RES_UNAVAIL(ccuInitRet)) {
248- // 如果是ccu ms模式,回退至ccu调度模式重试
249- // 复用原有的ccuResContainer,回退到ccu sched时不需要重复拉起ccu驱动
250- if (opExpansionMode_ == CCU_MS_MODE) {
251- opExpansionMode_ = CCU_SCHED_MODE;
252- CHK_RET(TryInitCcuInstanceLegacy()); // 至多递归一次
253- return HcclResult::HCCL_SUCCESS;
254- }
255- 
256- // 其余模式资源不足回退至aicpu ts
257- opExpansionMode_ = AICPU_TS_MODE;
258- ccuInsHandle_ = 0;
259- HCCL_RUN_WARNING(
260- "[MyRank][%s] ccu resources are unavailable, fallback to aicpu, rankId[%u].", __func__, rankId_);
261- return HcclResult::HCCL_SUCCESS;
262- }
263- 
264- // 预期外返回值属于错误
265- if (ccuInitRet != CcuResult::CCU_SUCCESS) {
266- HCCL_ERROR("[%s] failed, ret[%d] is not expected.", __func__, ccuInitRet);
267- ccuInsHandle_ = 0;
268- return static_cast<HcclResult>(ccuInitRet);
269- }
270- 
271- // ccu资源申请成功
272- return HcclResult::HCCL_SUCCESS;
273-}
274- 
275HcclResult MyRank::ReserveCcuMsCommOrFallback()215HcclResult MyRank::ReserveCcuMsCommOrFallback()
276{216{
277 if (opExpansionMode_ != CCU_MS_MODE) {217 if (opExpansionMode_ != CCU_MS_MODE) {
@@ -310,7 +250,6 @@ void MyRank::ReconcileCcuMsCommReservation(HcclResult initRet)
310 250 
311HcclResult MyRank::TryInitCcuInstanceOnDemand()251HcclResult MyRank::TryInitCcuInstanceOnDemand()
312{252{
313- // 以下为ccu新接口流程
314 auto ccuInsType = OpExpansionModeToCcuInstanceType(opExpansionMode_);253 auto ccuInsType = OpExpansionModeToCcuInstanceType(opExpansionMode_);
315 if (ccuInsType == CcuInstanceType::CCU_UNUSED) {254 if (ccuInsType == CcuInstanceType::CCU_UNUSED) {
316 ccuInsHandle_ = 0;255 ccuInsHandle_ = 0;
@@ -358,15 +297,9 @@ HcclResult MyRank::TryInitCcuInstance()
358{297{
359 CHK_RET(ReserveCcuMsCommOrFallback());298 CHK_RET(ReserveCcuMsCommOrFallback());
360 299 
361- HcclResult ret = HCCL_SUCCESS;300+ // ccu instance 不在 init 时创建,由 QueryCcuIns 创建或由 Assign 显式绑定;
362- if (useCcuResStaticAlloc_) {301+ // 此处仅拉起 ccu 驱动。
363- HCCL_RUN_INFO(302+ HcclResult ret = TryInitCcuInstanceOnDemand();
364- "[MyRank][%s] HCCL version does not support CCU on-demand resource allocation, use legacy allocation.",
365- __func__);
366- ret = TryInitCcuInstanceLegacy();
367- } else {
368- ret = TryInitCcuInstanceOnDemand();
369- }
370 ReconcileCcuMsCommReservation(ret);303 ReconcileCcuMsCommReservation(ret);
371 return ret;304 return ret;
372}305}
@@ -389,28 +322,11 @@ HcclResult MyRank::GetDevicePortInternal(uint32_t rank, uint32_t* devPort, Endpo
389 return HCCL_SUCCESS;322 return HCCL_SUCCESS;
390}323}
391 324 
392-inline HcclResult GetHcclVersion(int& hcclVersion)
393-{
394- char hcclPkgName[] = "hccl";
395- aclError aclRet = aclsysGetVersionNum(hcclPkgName, &hcclVersion);
396- CHK_PRT_RET(
397- aclRet != ACL_SUCCESS, HCCL_ERROR("[GetHcclVersion] aclsysGetVersionNum failed, aclRet[%d].", aclRet),
398- HCCL_E_INTERNAL);
399- HCCL_RUN_INFO("[GetHcclVersion] hccl version is %d.", hcclVersion);
400- return HCCL_SUCCESS;
401-}
402- 
403-constexpr int MAX_HCCL_VERSION_USING_CCU_RES_STATIC_ALLOC = 90100000;
404HcclResult MyRank::Init(HcclMem cclBuffer, const uint32_t opExpansionMode, uint32_t rankNum)325HcclResult MyRank::Init(HcclMem cclBuffer, const uint32_t opExpansionMode, uint32_t rankNum)
405{326{
406 // EXCEPTION_HANDLE_BEGIN327 // EXCEPTION_HANDLE_BEGIN
407 CHK_RET(hrtGetDevice(&devLogicId_));328 CHK_RET(hrtGetDevice(&devLogicId_));
408 329 
409- // 获取hccl版本
410- int hcclVersion = 0;
411- CHK_RET(GetHcclVersion(hcclVersion));
412- useCcuResStaticAlloc_ = hcclVersion <= MAX_HCCL_VERSION_USING_CCU_RES_STATIC_ALLOC;
413- 
414 // ns recovery processor初始化330 // ns recovery processor初始化
415 EXCEPTION_CATCH(nsRecoveryProcessor_ = std::make_unique<NsRecoveryProcessor>(), return HCCL_E_PTR);331 EXCEPTION_CATCH(nsRecoveryProcessor_ = std::make_unique<NsRecoveryProcessor>(), return HCCL_E_PTR);
416 332 
@@ -43,10 +43,28 @@
43 43 
44#include "ccu_types.h"44#include "ccu_types.h"
45#include "ccu_drv_handle.h"45#include "ccu_drv_handle.h"
46+#include "ccu_device_res.h"
46#include "dev_type.h"47#include "dev_type.h"
47 48 
48namespace hccl {49namespace hccl {
49 50 
51+constexpr uint32_t DEFAULT_MODE = 0;
52+constexpr uint32_t AICPU_TS_MODE = 2;
53+constexpr uint32_t CCU_MS_MODE = 5;
54+constexpr uint32_t CCU_SCHED_MODE = 6;
L
Llaodazhao111 天前

为啥和这个不一致 HcclOpExpansionMode

likedislike
guoziqi
guoziqi
10 天前 评论:
55+// opExpansionMode 到 CcuInstanceType 的映射,仅作数据类型转换,不做逻辑处理。
56+// 供 MyRank 内部及外部适配层(如 coll_comm_ccu_c_adpt.cc)共用。
57+inline CcuInstanceType OpExpansionModeToCcuInstanceType(uint32_t opExpansionMode)
58+{
59+ if (opExpansionMode == CCU_SCHED_MODE) {
60+ return CcuInstanceType::CCU_SCHED;
61+ }
62+ if (opExpansionMode == CCU_MS_MODE) {
63+ return CcuInstanceType::CCU_MS;
64+ }
65+ return CcuInstanceType::CCU_UNUSED;
66+}
67+ 
50/**68/**
51 * @note 职责:管理当前通信域下本Rank的信息和通信资源69 * @note 职责:管理当前通信域下本Rank的信息和通信资源
52 */70 */
@@ -67,6 +85,8 @@ public:
67 uint32_t GetOpExpansionMode() { return opExpansionMode_; }85 uint32_t GetOpExpansionMode() { return opExpansionMode_; }
68 CcuInsHandle GetCcuInstance() const { return ccuInsHandle_; }86 CcuInsHandle GetCcuInstance() const { return ccuInsHandle_; }
69 void SetCcuInstance(CcuInsHandle ccuInsHandle) { ccuInsHandle_ = ccuInsHandle; }87 void SetCcuInstance(CcuInsHandle ccuInsHandle) { ccuInsHandle_ = ccuInsHandle; }
88+ CcuInsHandle GetAssignedCcuInstance() const { return assignedCcuInsHandle_; }
89+ void SetAssignedCcuInstance(CcuInsHandle ccuInsHandle) { assignedCcuInsHandle_ = ccuInsHandle; }
70 90 
71 CollCommConfigConsistency& GetCollCommConfigConsistency();91 CollCommConfigConsistency& GetCollCommConfigConsistency();
72 92 
@@ -150,7 +170,6 @@ private:
150 const EndpointDesc& remoteEndpointDesc, uint32_t& listenPort, HcommChannelDesc& hcommDesc);170 const EndpointDesc& remoteEndpointDesc, uint32_t& listenPort, HcommChannelDesc& hcommDesc);
151 HcclResult GetLocalTlsStatus(Hccl::TlsStatus& tlsStatus) const;171 HcclResult GetLocalTlsStatus(Hccl::TlsStatus& tlsStatus) const;
152 HcclResult RegisterCommMemsToEndpoint(EndpointHandle epHandle);172 HcclResult RegisterCommMemsToEndpoint(EndpointHandle epHandle);
153- HcclResult TryInitCcuInstanceLegacy();
154 HcclResult TryInitCcuInstance();173 HcclResult TryInitCcuInstance();
155 HcclResult ReserveCcuMsCommOrFallback();174 HcclResult ReserveCcuMsCommOrFallback();
156 HcclResult TryInitCcuInstanceOnDemand();175 HcclResult TryInitCcuInstanceOnDemand();
@@ -172,7 +191,8 @@ private:
172 std::unique_ptr<CommMems> commMems_{nullptr};191 std::unique_ptr<CommMems> commMems_{nullptr};
173 std::unique_ptr<EngineCtxs> engineCtxs_{nullptr};192 std::unique_ptr<EngineCtxs> engineCtxs_{nullptr};
174 193 
175- CcuInsHandle ccuInsHandle_{0};194+ CcuInsHandle ccuInsHandle_{0}; // 按固定量CCU资源方式创建的 ccuInsHandle(HcclCommQueryCcuIns)(兼容旧版本)
195+ CcuInsHandle assignedCcuInsHandle_{0}; // 按需申请CCU资源方式绑定的 ccuInsHandle(HcommCcuInsCreate + Assign)
176 196 
177 ManagerCallbacks callbacks_;197 ManagerCallbacks callbacks_;
178 198 
@@ -192,7 +212,6 @@ private:
192 CollCommConfigConsistency collCommConfigConsistency_;212 CollCommConfigConsistency collCommConfigConsistency_;
193 ExchangeInfoMgr exchangeInfoMgr_;213 ExchangeInfoMgr exchangeInfoMgr_;
194 std::shared_ptr<hcomm::CcuDrvHandle> ccuDrvHandle_{};214 std::shared_ptr<hcomm::CcuDrvHandle> ccuDrvHandle_{};
195- bool useCcuResStaticAlloc_{false}; // HCCL版本不支持CCU资源按需申请
196 bool ccuMsCommReserved_{false};215 bool ccuMsCommReserved_{false};
197 Hccl::HcclMainboardId mainBoardType_{Hccl::HcclMainboardId::MAINBOARD_OTHERS};216 Hccl::HcclMainboardId mainBoardType_{Hccl::HcclMainboardId::MAINBOARD_OTHERS};
198};217};
@@ -26,6 +26,8 @@
26 26 
27#include "ccu/ccu_res.h"27#include "ccu/ccu_res.h"
28#include "hccl/hccl_ccu_res.h"28#include "hccl/hccl_ccu_res.h"
29+#include "ccu_device_res.h"
30+#include "ccu_instance.h"
29 31 
30using namespace hccl;32using namespace hccl;
31using namespace hcomm;33using namespace hcomm;
@@ -39,7 +41,8 @@ protected:
39 }41 }
40 42 
41 HcclComm BuildComm(43 HcclComm BuildComm(
42- bool isCommunicatorV2 = true, bool hasCollComm = true, bool hasMyRank = true, CcuInsHandle currentInsHandle = 0)44+ bool isCommunicatorV2 = true, bool hasCollComm = true, bool hasMyRank = true,
45+ CcuInsHandle currentAssignedInsHandle = 0)
43 {46 {
44 comm_ = std::make_unique<hcclComm>(0, 0, "assign_ccu_ut");47 comm_ = std::make_unique<hcclComm>(0, 0, "assign_ccu_ut");
45 comm_->devType_ = isCommunicatorV2 ? DevType::DEV_TYPE_950 : DevType::DEV_TYPE_COUNT;48 comm_->devType_ = isCommunicatorV2 ? DevType::DEV_TYPE_950 : DevType::DEV_TYPE_COUNT;
@@ -56,7 +59,8 @@ protected:
56 }59 }
57 60 
58 myRank_ = std::make_shared<MyRank>(nullptr, 0, comm_->collComm_->config_, ManagerCallbacks{}, nullptr, nullptr);61 myRank_ = std::make_shared<MyRank>(nullptr, 0, comm_->collComm_->config_, ManagerCallbacks{}, nullptr, nullptr);
59- myRank_->ccuInsHandle_ = currentInsHandle;62+ // 语义变更:HcclCommAssignCcuIns 改写 assignedCcuInsHandle_(不再写 ccuInsHandle_)
63+ myRank_->assignedCcuInsHandle_ = currentAssignedInsHandle;
60 comm_->collComm_->myRank_ = myRank_;64 comm_->collComm_->myRank_ = myRank_;
61 return static_cast<HcclComm>(comm_.get());65 return static_cast<HcclComm>(comm_.get());
62 }66 }
@@ -82,7 +86,8 @@ TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommAssignCcuIns_When_CommHasNoCcuInstan
82 HcclResult ret = HcclCommAssignCcuIns(comm, insHandle);86 HcclResult ret = HcclCommAssignCcuIns(comm, insHandle);
83 87 
84 EXPECT_EQ(ret, HCCL_SUCCESS);88 EXPECT_EQ(ret, HCCL_SUCCESS);
85- EXPECT_EQ(myRank_->GetCcuInstance(), insHandle);89+ EXPECT_EQ(myRank_->GetAssignedCcuInstance(), insHandle);
90+ EXPECT_EQ(myRank_->GetCcuInstance(), 0); // ccuInsHandle_ 不被 Assign 改写
86}91}
87 92 
88TEST_F(93TEST_F(
@@ -97,7 +102,7 @@ TEST_F(
97 HcclResult ret = HcclCommAssignCcuIns(comm, insHandle);102 HcclResult ret = HcclCommAssignCcuIns(comm, insHandle);
98 103 
99 EXPECT_EQ(ret, HCCL_SUCCESS);104 EXPECT_EQ(ret, HCCL_SUCCESS);
100- EXPECT_EQ(myRank_->GetCcuInstance(), insHandle);105+ EXPECT_EQ(myRank_->GetAssignedCcuInstance(), insHandle);
101 myRank_.reset();106 myRank_.reset();
102 comm_.reset();107 comm_.reset();
103 EXPECT_TRUE(myRankWeak.expired());108 EXPECT_TRUE(myRankWeak.expired());
@@ -119,7 +124,7 @@ TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommAssignCcuIns_When_InsHandleIsZero_Ex
119 HcclResult ret = HcclCommAssignCcuIns(comm, 0);124 HcclResult ret = HcclCommAssignCcuIns(comm, 0);
120 125 
121 EXPECT_EQ(ret, HCCL_E_PARA);126 EXPECT_EQ(ret, HCCL_E_PARA);
122- EXPECT_EQ(myRank_->GetCcuInstance(), 0);127+ EXPECT_EQ(myRank_->GetAssignedCcuInstance(), 0);
123}128}
124 129 
125TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommAssignCcuIns_When_CommIsNotV2_Expect_ReturnIsHCCL_E_NOT_SUPPORT)130TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommAssignCcuIns_When_CommIsNotV2_Expect_ReturnIsHCCL_E_NOT_SUPPORT)
@@ -130,7 +135,7 @@ TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommAssignCcuIns_When_CommIsNotV2_Expect
130 HcclResult ret = HcclCommAssignCcuIns(comm, insHandle);135 HcclResult ret = HcclCommAssignCcuIns(comm, insHandle);
131 136 
132 EXPECT_EQ(ret, HCCL_E_NOT_SUPPORT);137 EXPECT_EQ(ret, HCCL_E_NOT_SUPPORT);
133- EXPECT_EQ(myRank_->GetCcuInstance(), 0);138+ EXPECT_EQ(myRank_->GetAssignedCcuInstance(), 0);
134}139}
135 140 
136TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommAssignCcuIns_When_CollCommIsNull_Expect_ReturnIsHCCL_E_PTR)141TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommAssignCcuIns_When_CollCommIsNull_Expect_ReturnIsHCCL_E_PTR)
@@ -162,7 +167,7 @@ TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommAssignCcuIns_When_InsHandleNotFound_
162 HcclResult ret = HcclCommAssignCcuIns(comm, insHandle);167 HcclResult ret = HcclCommAssignCcuIns(comm, insHandle);
163 168 
164 EXPECT_EQ(ret, HCCL_E_NOT_FOUND);169 EXPECT_EQ(ret, HCCL_E_NOT_FOUND);
165- EXPECT_EQ(myRank_->GetCcuInstance(), 0);170+ EXPECT_EQ(myRank_->GetAssignedCcuInstance(), 0);
166}171}
167 172 
168TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommAssignCcuIns_When_CommHasDifferentCcuInstance_Expect_ReturnIsHCCL_E_PARA)173TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommAssignCcuIns_When_CommHasDifferentCcuInstance_Expect_ReturnIsHCCL_E_PARA)
@@ -175,7 +180,7 @@ TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommAssignCcuIns_When_CommHasDifferentCc
175 HcclResult ret = HcclCommAssignCcuIns(comm, newInsHandle);180 HcclResult ret = HcclCommAssignCcuIns(comm, newInsHandle);
176 181 
177 EXPECT_EQ(ret, HCCL_E_PARA);182 EXPECT_EQ(ret, HCCL_E_PARA);
178- EXPECT_EQ(myRank_->GetCcuInstance(), oldInsHandle);183+ EXPECT_EQ(myRank_->GetAssignedCcuInstance(), oldInsHandle);
179}184}
180 185 
181TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommAssignCcuIns_When_CommHasSameCcuInstance_Expect_ReturnIsHCCL_E_PARA)186TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommAssignCcuIns_When_CommHasSameCcuInstance_Expect_ReturnIsHCCL_E_PARA)
@@ -187,7 +192,7 @@ TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommAssignCcuIns_When_CommHasSameCcuInst
187 HcclResult ret = HcclCommAssignCcuIns(comm, insHandle);192 HcclResult ret = HcclCommAssignCcuIns(comm, insHandle);
188 193 
189 EXPECT_EQ(ret, HCCL_E_PARA);194 EXPECT_EQ(ret, HCCL_E_PARA);
190- EXPECT_EQ(myRank_->GetCcuInstance(), insHandle);195+ EXPECT_EQ(myRank_->GetAssignedCcuInstance(), insHandle);
191}196}
192 197 
193TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommAssignCcuIns_When_InternalExceptionOccurs_Expect_ReturnIsHCCL_E_INTERNAL)198TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommAssignCcuIns_When_InternalExceptionOccurs_Expect_ReturnIsHCCL_E_INTERNAL)
@@ -200,7 +205,7 @@ TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommAssignCcuIns_When_InternalExceptionO
200 HcclResult ret = HcclCommAssignCcuIns(comm, insHandle);205 HcclResult ret = HcclCommAssignCcuIns(comm, insHandle);
201 206 
202 EXPECT_EQ(ret, HCCL_E_INTERNAL);207 EXPECT_EQ(ret, HCCL_E_INTERNAL);
203- EXPECT_EQ(myRank_->GetCcuInstance(), 0);208+ EXPECT_EQ(myRank_->GetAssignedCcuInstance(), 0);
204}209}
205 210 
206TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommAssignCcuIns_When_TwoThreadsAssignConcurrently_Expect_OnlyOneSucceeds)211TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommAssignCcuIns_When_TwoThreadsAssignConcurrently_Expect_OnlyOneSucceeds)
@@ -235,5 +240,287 @@ TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommAssignCcuIns_When_TwoThreadsAssignCo
235 const bool firstSucceeded = firstRet == HCCL_SUCCESS && secondRet == HCCL_E_PARA;240 const bool firstSucceeded = firstRet == HCCL_SUCCESS && secondRet == HCCL_E_PARA;
236 const bool secondSucceeded = firstRet == HCCL_E_PARA && secondRet == HCCL_SUCCESS;241 const bool secondSucceeded = firstRet == HCCL_E_PARA && secondRet == HCCL_SUCCESS;
237 EXPECT_TRUE(firstSucceeded || secondSucceeded);242 EXPECT_TRUE(firstSucceeded || secondSucceeded);
238- EXPECT_EQ(myRank_->GetCcuInstance(), firstSucceeded ? firstInsHandle : secondInsHandle);243+ EXPECT_EQ(myRank_->GetAssignedCcuInstance(), firstSucceeded ? firstInsHandle : secondInsHandle);
244+}
245+ 
246+// ============================================================================================
247+// HcclCommQueryCcuIns 单元测试(阶段2新语义:查/建通信域自有的 ccuInsHandle_,旧方式)
248+// 对应业务代码:src/coll_communicator_mgr/api_c_adpt/coll_comm_ccu_c_adpt.cc
249+// ============================================================================================
250+ 
251+TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommQueryCcuIns_When_SelfHandleExists_Expect_ReturnSuccessAndNoCreate)
252+{
253+ constexpr CcuInsHandle selfHandle = 0xAB;
254+ HcclComm comm = BuildComm(true, true, true, 0);
255+ myRank_->ccuInsHandle_ = selfHandle; // 通信域自有 handle 已存在
256+ // 按需创建不应被调用
257+ MOCKER(HcommCcuInsCreateLegacy).expects(never());
258+ 
259+ CcuInsHandle insHandles[1] = {0};
260+ uint32_t insNum = 0;
261+ HcclResult ret = HcclCommQueryCcuIns(comm, insHandles, &insNum);
262+ 
263+ EXPECT_EQ(ret, HCCL_SUCCESS);
264+ EXPECT_EQ(insHandles[0], selfHandle);
265+ EXPECT_EQ(insNum, 1U);
266+ EXPECT_EQ(myRank_->GetCcuInstance(), selfHandle); // 未被改写
267+}
268+ 
269+TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommQueryCcuIns_When_SelfHandleZero_And_ModeValid_Expect_CreateAndWriteback)
270+{
271+ CcuInsHandle createdHandle = 0xCD;
272+ HcclComm comm = BuildComm(true, true, true, 0);
273+ myRank_->ccuInsHandle_ = 0;
274+ myRank_->opExpansionMode_ = 6; // CCU_SCHED_MODE 映射为 CCU_SCHED
275+ CcuInsHandle insHandles[1] = {0};
276+ uint32_t insNum = 0;
277+ // mock 按需创建:通过 outBoundP 把 createdHandle 拷贝到出参指针 *ccuInsHandle,并返回成功
278+ MOCKER(HcommCcuInsCreateLegacy)
279+ .expects(once())
280+ .with(mockcpp::any(), mockcpp::outBoundP(&createdHandle, sizeof(CcuInsHandle)))
281+ .will(returnValue(CcuResult::CCU_SUCCESS));
282+ 
283+ HcclResult ret = HcclCommQueryCcuIns(comm, insHandles, &insNum);
284+ EXPECT_EQ(ret, HCCL_SUCCESS);
285+ EXPECT_EQ(insHandles[0], createdHandle);
286+ EXPECT_EQ(insNum, 1U);
287+ EXPECT_EQ(myRank_->GetCcuInstance(), createdHandle); // 已回写
288+}
289+ 
290+TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommQueryCcuIns_When_SelfHandleZero_And_ModeUnused_Expect_ReturnUnavail)
291+{
292+ HcclComm comm = BuildComm(true, true, true, 0);
293+ myRank_->ccuInsHandle_ = 0;
294+ myRank_->opExpansionMode_ = 0; // DEFAULT_MODE 映射为 CCU_UNUSED
295+ MOCKER(HcommCcuInsCreateLegacy).expects(never());
296+ 
297+ CcuInsHandle insHandles[1] = {0};
298+ uint32_t insNum = 0;
299+ HcclResult ret = HcclCommQueryCcuIns(comm, insHandles, &insNum);
300+ 
301+ EXPECT_EQ(ret, HCCL_E_UNAVAIL);
302+ EXPECT_EQ(insHandles[0], static_cast<CcuInsHandle>(0)); // 出参未修改
303+ EXPECT_EQ(insNum, 0U);
304+ EXPECT_EQ(myRank_->GetCcuInstance(), 0); // 未创建
305+}
306+ 
307+TEST_F(
308+ HcclCommAssignCcuInsTest,
309+ Ut_HcclCommQueryCcuIns_When_SelfHandleZero_And_CreateFails_Expect_ReturnErrorAndNoWriteback)
310+{
311+ HcclComm comm = BuildComm(true, true, true, 0);
312+ myRank_->ccuInsHandle_ = 0;
313+ myRank_->opExpansionMode_ = 5; // CCU_MS_MODE 映射为 CCU_MS
314+ MOCKER(HcommCcuInsCreateLegacy).expects(once()).will(returnValue(CcuResult::CCU_E_DRV_BUSY));
315+ 
316+ CcuInsHandle insHandles[1] = {0};
317+ uint32_t insNum = 0;
318+ HcclResult ret = HcclCommQueryCcuIns(comm, insHandles, &insNum);
319+ 
320+ EXPECT_EQ(ret, static_cast<HcclResult>(CcuResult::CCU_E_DRV_BUSY));
321+ EXPECT_EQ(myRank_->GetCcuInstance(), 0); // 失败不回写
322+}
323+ 
324+// CCU_MS 资源不足时降级到 CCU_SCHED 重试,CCU_SCHED 创建成功 → 返回成功并回写降级后创建的 handle
325+TEST_F(
326+ HcclCommAssignCcuInsTest, Ut_HcclCommQueryCcuIns_When_CcuMsUnavail_And_FallbackCcuSchedSuccess_Expect_ReturnSuccess)
327+{
328+ CcuInsHandle createdHandle = 0xCD;
329+ HcclComm comm = BuildComm(true, true, true, 0);
330+ myRank_->ccuInsHandle_ = 0;
331+ myRank_->opExpansionMode_ = 5; // CCU_MS_MODE 映射为 CCU_MS
332+ // 首次 CCU_MS 返回资源不足,降级到 CCU_SCHED 重试一次成功。
333+ // 用 eq(CCU_MS)/eq(CCU_SCHED) 区分入参,使 mockcpp 按参数精确匹配两次调用
334+ // (多次 expects(once()) 无 with 时会误匹配首条,导致第二次调用报错)
335+ MOCKER(HcommCcuInsCreateLegacy)
336+ .expects(once())
337+ .with(mockcpp::eq(CcuInstanceType::CCU_MS), mockcpp::any())
338+ .will(returnValue(CcuResult::CCU_E_UNAVAIL));
339+ MOCKER(HcommCcuInsCreateLegacy)
340+ .expects(once())
341+ .with(mockcpp::eq(CcuInstanceType::CCU_SCHED), mockcpp::outBoundP(&createdHandle, sizeof(CcuInsHandle)))
342+ .will(returnValue(CcuResult::CCU_SUCCESS));
343+ 
344+ CcuInsHandle insHandles[1] = {0};
345+ uint32_t insNum = 0;
346+ HcclResult ret = HcclCommQueryCcuIns(comm, insHandles, &insNum);
347+ 
348+ EXPECT_EQ(ret, HCCL_SUCCESS);
349+ EXPECT_EQ(insHandles[0], createdHandle);
350+ EXPECT_EQ(insNum, 1U);
351+ EXPECT_EQ(myRank_->GetCcuInstance(), createdHandle); // 降级后创建的 handle 已回写
352+}
353+ 
354+// CCU_MS 资源不足降级到 CCU_SCHED 仍资源不足 → 返回 HCCL_E_UNAVAIL,不创建、不回写
355+TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommQueryCcuIns_When_CcuMsAndSchedBothUnavail_Expect_ReturnUnavail)
356+{
357+ HcclComm comm = BuildComm(true, true, true, 0);
358+ myRank_->ccuInsHandle_ = 0;
359+ myRank_->opExpansionMode_ = 5; // CCU_MS_MODE 映射为 CCU_MS
360+ // 首次 CCU_MS 资源不足,降级 CCU_SCHED 仍资源不足(共调用两次)。
361+ // 用 eq(CCU_MS)/eq(CCU_SCHED) 区分入参,避免两次 expects(once()) 互相误匹配
362+ MOCKER(HcommCcuInsCreateLegacy)
363+ .expects(once())
364+ .with(mockcpp::eq(CcuInstanceType::CCU_MS), mockcpp::any())
365+ .will(returnValue(CcuResult::CCU_E_UNAVAIL));
366+ MOCKER(HcommCcuInsCreateLegacy)
367+ .expects(once())
368+ .with(mockcpp::eq(CcuInstanceType::CCU_SCHED), mockcpp::any())
369+ .will(returnValue(CcuResult::CCU_E_UNAVAIL));
370+ 
371+ CcuInsHandle insHandles[1] = {0};
372+ uint32_t insNum = 0;
373+ HcclResult ret = HcclCommQueryCcuIns(comm, insHandles, &insNum);
374+ 
375+ EXPECT_EQ(ret, HCCL_E_UNAVAIL);
376+ EXPECT_EQ(insHandles[0], static_cast<CcuInsHandle>(0)); // 出参未修改
377+ EXPECT_EQ(insNum, 0U);
378+ EXPECT_EQ(myRank_->GetCcuInstance(), 0); // 失败不回写
379+}
380+ 
381+TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommQueryCcuIns_When_CommIsNotV2_Expect_ReturnNotSupport)
382+{
383+ HcclComm comm = BuildComm(false, true, true, 0);
384+ MOCKER(HcommCcuInsCreateLegacy).expects(never());
385+ 
386+ CcuInsHandle insHandles[1] = {0};
387+ uint32_t insNum = 0;
388+ HcclResult ret = HcclCommQueryCcuIns(comm, insHandles, &insNum);
389+ 
390+ EXPECT_EQ(ret, HCCL_E_NOT_SUPPORT);
391+}
392+ 
393+TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommQueryCcuIns_When_CommIsNull_Expect_ReturnPtr)
394+{
395+ CcuInsHandle insHandles[1] = {0};
396+ uint32_t insNum = 0;
397+ EXPECT_EQ(HcclCommQueryCcuIns(nullptr, insHandles, &insNum), HCCL_E_PTR);
398+}
399+ 
400+TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommQueryCcuIns_When_InsHandlesIsNull_Expect_ReturnPtr)
401+{
402+ HcclComm comm = BuildComm();
403+ uint32_t insNum = 0;
404+ EXPECT_EQ(HcclCommQueryCcuIns(comm, nullptr, &insNum), HCCL_E_PTR);
405+}
406+ 
407+TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommQueryCcuIns_When_InsNumIsNull_Expect_ReturnPtr)
408+{
409+ HcclComm comm = BuildComm();
410+ CcuInsHandle insHandles[1] = {0};
411+ EXPECT_EQ(HcclCommQueryCcuIns(comm, insHandles, nullptr), HCCL_E_PTR);
412+}
413+ 
414+TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommQueryCcuIns_When_RepeatQuery_Expect_IdempotentAndNoSecondCreate)
415+{
416+ CcuInsHandle createdHandle = 0xCD;
417+ HcclComm comm = BuildComm(true, true, true, 0);
418+ myRank_->ccuInsHandle_ = 0;
419+ myRank_->opExpansionMode_ = 6; // CCU_SCHED
420+ MOCKER(HcommCcuInsCreateLegacy)
421+ .expects(once()) // 仅首次创建
422+ .with(mockcpp::any(), mockcpp::outBoundP(&createdHandle, sizeof(CcuInsHandle)))
423+ .will(returnValue(CcuResult::CCU_SUCCESS));
424+ 
425+ CcuInsHandle insHandles[1] = {0};
426+ uint32_t insNum = 0;
427+ EXPECT_EQ(HcclCommQueryCcuIns(comm, insHandles, &insNum), HCCL_SUCCESS);
428+ EXPECT_EQ(insHandles[0], createdHandle);
429+ // 第二次 Query:handle 已存在,不应再创建
430+ CcuInsHandle insHandles2[1] = {0};
431+ uint32_t insNum2 = 0;
432+ EXPECT_EQ(HcclCommQueryCcuIns(comm, insHandles2, &insNum2), HCCL_SUCCESS);
433+ EXPECT_EQ(insHandles2[0], createdHandle);
434+ EXPECT_EQ(myRank_->GetCcuInstance(), createdHandle);
435+}
436+ 
437+// ============================================================================================
438+// HcclCommQueryAssignedCcuIns 单元测试(新增接口:查询新方式绑定的 assignedCcuInsHandle_)
439+// ============================================================================================
440+ 
441+TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommQueryAssignedCcuIns_When_Bound_Expect_ReturnSuccess)
442+{
443+ constexpr CcuInsHandle assignedHandle = 0xEF;
444+ HcclComm comm = BuildComm(true, true, true, assignedHandle);
445+ // 不应创建任何 instance
446+ MOCKER(HcommCcuInsCreateLegacy).expects(never());
447+ 
448+ CcuInsHandle insHandles[1] = {0};
449+ uint32_t insNum = 0;
450+ HcclResult ret = HcclCommQueryAssignedCcuIns(comm, insHandles, &insNum);
451+ 
452+ EXPECT_EQ(ret, HCCL_SUCCESS);
453+ EXPECT_EQ(insHandles[0], assignedHandle);
454+ EXPECT_EQ(insNum, 1U);
455+}
456+ 
457+TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommQueryAssignedCcuIns_When_NotBound_Expect_ReturnUnavail)
458+{
459+ HcclComm comm = BuildComm(true, true, true, 0); // assignedCcuInsHandle_ = 0
460+ MOCKER(HcommCcuInsCreateLegacy).expects(never()); // 不创建
461+ 
462+ CcuInsHandle insHandles[1] = {0};
463+ uint32_t insNum = 0;
464+ HcclResult ret = HcclCommQueryAssignedCcuIns(comm, insHandles, &insNum);
465+ 
466+ EXPECT_EQ(ret, HCCL_E_UNAVAIL);
467+ EXPECT_EQ(insHandles[0], static_cast<CcuInsHandle>(0)); // 出参未修改
468+ EXPECT_EQ(insNum, 0U);
469+}
470+ 
471+TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommQueryAssignedCcuIns_When_CommIsNotV2_Expect_ReturnNotSupport)
472+{
473+ HcclComm comm = BuildComm(false, true, true, 0xEF);
474+ CcuInsHandle insHandles[1] = {0};
475+ uint32_t insNum = 0;
476+ EXPECT_EQ(HcclCommQueryAssignedCcuIns(comm, insHandles, &insNum), HCCL_E_NOT_SUPPORT);
477+}
478+ 
479+TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommQueryAssignedCcuIns_When_CommIsNull_Expect_ReturnPtr)
480+{
481+ CcuInsHandle insHandles[1] = {0};
482+ uint32_t insNum = 0;
483+ EXPECT_EQ(HcclCommQueryAssignedCcuIns(nullptr, insHandles, &insNum), HCCL_E_PTR);
484+}
485+ 
486+TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommQueryAssignedCcuIns_When_InsHandlesIsNull_Expect_ReturnPtr)
487+{
488+ HcclComm comm = BuildComm();
489+ uint32_t insNum = 0;
490+ EXPECT_EQ(HcclCommQueryAssignedCcuIns(comm, nullptr, &insNum), HCCL_E_PTR);
491+}
492+ 
493+TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommQueryAssignedCcuIns_When_InsNumIsNull_Expect_ReturnPtr)
494+{
495+ HcclComm comm = BuildComm();
496+ CcuInsHandle insHandles[1] = {0};
497+ EXPECT_EQ(HcclCommQueryAssignedCcuIns(comm, insHandles, nullptr), HCCL_E_PTR);
498+}
499+ 
500+// ============================================================================================
501+// HcclCommAssignCcuIns 语义变更补充:写 assignedCcuInsHandle_,ccuInsHandle_ 不变
502+// ============================================================================================
503+ 
504+TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommAssignCcuIns_When_Success_Expect_CcuInsHandleUnchanged)
505+{
506+ constexpr CcuInsHandle selfHandle = 0xA;
507+ constexpr CcuInsHandle assignedHandle = 0xB;
508+ HcclComm comm = BuildComm(true, true, true, 0);
509+ myRank_->ccuInsHandle_ = selfHandle; // 通信域自有 handle
510+ MockCcuInstanceFound(assignedHandle);
511+ 
512+ HcclResult ret = HcclCommAssignCcuIns(comm, assignedHandle);
513+ 
514+ EXPECT_EQ(ret, HCCL_SUCCESS);
515+ EXPECT_EQ(myRank_->GetAssignedCcuInstance(), assignedHandle);
516+ EXPECT_EQ(myRank_->GetCcuInstance(), selfHandle); // 自有 handle 不被改写
517+}
518+ 
519+TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommQueryAssignedCcuIns_When_CalledBeforeAssign_Expect_ReturnUnavail)
520+{
521+ // 组合:未 Assign 直接 QueryAssigned
522+ HcclComm comm = BuildComm(true, true, true, 0);
523+ CcuInsHandle insHandles[1] = {0};
524+ uint32_t insNum = 0;
525+ EXPECT_EQ(HcclCommQueryAssignedCcuIns(comm, insHandles, &insNum), HCCL_E_UNAVAIL);
239}526}
@@ -65,10 +65,12 @@ CcuResult StubDeinitCcuFeature(int32_t deviceLogicId)
65 return CcuResult::CCU_SUCCESS;65 return CcuResult::CCU_SUCCESS;
66}66}
67 67 
68-CcuResult StubCreateCcuInstanceFailure(CcuInstanceType ccuInsType, CcuInsHandle* ccuInsHandle)68+// 简化后 TryInitCcuInstance 只走 OnDemand(拉起驱动 CcuInitFeature),不再创建 ccu instance;
69+// 驱动拉起失败时的预约对账行为改由 CcuInitFeature 返回值驱动
70+CcuResult StubCcuInitFeatureFailure(const int32_t devLogicId, std::shared_ptr<hcomm::CcuDrvHandle>& ccuDrvHandle)
69{71{
70- (void)ccuInsType;72+ EXPECT_EQ(devLogicId, DEVICE_0);
71- (void)ccuInsHandle;73+ (void)ccuDrvHandle;
72 bool reserved = false;74 bool reserved = false;
73 EXPECT_EQ(75 EXPECT_EQ(
74 hccl::CollCommMgr::GetInstance().TryReserveCcuMsComm(DEVICE_0, "init_failure_probe", reserved), HCCL_SUCCESS);76 hccl::CollCommMgr::GetInstance().TryReserveCcuMsComm(DEVICE_0, "init_failure_probe", reserved), HCCL_SUCCESS);
@@ -207,8 +209,9 @@ TEST_F(CcuMsPerDeviceExclusiveTest, Ut_TryInitCcuInstance_When_SameDeviceAlready
207 ASSERT_TRUE(ownerReserved);209 ASSERT_TRUE(ownerReserved);
208 myRank_->devLogicId_ = DEVICE_0;210 myRank_->devLogicId_ = DEVICE_0;
209 myRank_->opExpansionMode_ = CCU_MS_MODE;211 myRank_->opExpansionMode_ = CCU_MS_MODE;
210- myRank_->useCcuResStaticAlloc_ = true;212+ // 简化后:MS 回退到 SCHED 由 ReserveCcuMsCommOrFallback 完成(预约失败即回退),
211- MOCKER(HcommCcuInsCreateLegacy).expects(once()).will(returnValue(CcuResult::CCU_SUCCESS));213+ // 不再依赖 ccu instance 创建;驱动拉起走 CcuInitFeature
214+ MOCKER(hcomm::CcuInitFeature).expects(once()).will(returnValue(CcuResult::CCU_SUCCESS));
212 215 
213 EXPECT_EQ(myRank_->TryInitCcuInstance(), HCCL_SUCCESS);216 EXPECT_EQ(myRank_->TryInitCcuInstance(), HCCL_SUCCESS);
214 EXPECT_EQ(myRank_->opExpansionMode_, CCU_SCHED_MODE);217 EXPECT_EQ(myRank_->opExpansionMode_, CCU_SCHED_MODE);
@@ -224,9 +227,9 @@ TEST_F(CcuMsPerDeviceExclusiveTest, Ut_TryInitCcuInstance_When_CcuInitFails_Expe
224{227{
225 myRank_->devLogicId_ = DEVICE_0;228 myRank_->devLogicId_ = DEVICE_0;
226 myRank_->opExpansionMode_ = CCU_MS_MODE;229 myRank_->opExpansionMode_ = CCU_MS_MODE;
227- myRank_->useCcuResStaticAlloc_ = true;
228 g_reservationHeldDuringInitFailure = false;230 g_reservationHeldDuringInitFailure = false;
229- MOCKER(HcommCcuInsCreateLegacy).expects(once()).will(invoke(StubCreateCcuInstanceFailure));231+ // 简化后 ccu instance 不在 init 时创建,驱动拉起失败由 CcuInitFeature 返回
232+ MOCKER(hcomm::CcuInitFeature).expects(once()).will(invoke(StubCcuInitFeatureFailure));
230 233 
231 EXPECT_EQ(myRank_->TryInitCcuInstance(), HCCL_E_PARA);234 EXPECT_EQ(myRank_->TryInitCcuInstance(), HCCL_E_PARA);
232 EXPECT_TRUE(g_reservationHeldDuringInitFailure);235 EXPECT_TRUE(g_reservationHeldDuringInitFailure);
@@ -237,31 +240,12 @@ TEST_F(CcuMsPerDeviceExclusiveTest, Ut_TryInitCcuInstance_When_CcuInitFails_Expe
237 EXPECT_TRUE(nextReserved);240 EXPECT_TRUE(nextReserved);
238}241}
239 242 
240-TEST_F(CcuMsPerDeviceExclusiveTest, Ut_TryInitCcuInstance_When_MsFallsBack_Expect_ReservationReleased)
241-{
242- myRank_->devLogicId_ = DEVICE_0;
243- myRank_->opExpansionMode_ = CCU_MS_MODE;
244- myRank_->useCcuResStaticAlloc_ = true;
245- MOCKER(HcommCcuInsCreateLegacy)
246- .expects(exactly(2))
247- .will(returnValue(CcuResult::CCU_E_UNAVAIL))
248- .then(returnValue(CcuResult::CCU_SUCCESS));
249- 
250- EXPECT_EQ(myRank_->TryInitCcuInstance(), HCCL_SUCCESS);
251- EXPECT_EQ(myRank_->opExpansionMode_, CCU_SCHED_MODE);
252- EXPECT_FALSE(myRank_->ccuMsCommReserved_);
253- 
254- bool nextReserved = false;
255- EXPECT_EQ(hccl::CollCommMgr::GetInstance().TryReserveCcuMsComm(DEVICE_0, "next_owner", nextReserved), HCCL_SUCCESS);
256- EXPECT_TRUE(nextReserved);
257-}
258- 
259TEST_F(CcuMsPerDeviceExclusiveTest, Ut_TryInitCcuInstance_When_DriverBusyFallsBackToAicpu_Expect_ReservationReleased)243TEST_F(CcuMsPerDeviceExclusiveTest, Ut_TryInitCcuInstance_When_DriverBusyFallsBackToAicpu_Expect_ReservationReleased)
260{244{
261 myRank_->devLogicId_ = DEVICE_0;245 myRank_->devLogicId_ = DEVICE_0;
262 myRank_->opExpansionMode_ = CCU_MS_MODE;246 myRank_->opExpansionMode_ = CCU_MS_MODE;
263- myRank_->useCcuResStaticAlloc_ = true;247+ // 简化后驱动忙回退到 aicpu 由 TryInitCcuInstanceOnDemand 中 CcuInitFeature 返回 CCU_E_DRV_BUSY 触发
264- MOCKER(HcommCcuInsCreateLegacy).expects(once()).will(returnValue(CcuResult::CCU_E_DRV_BUSY));248+ MOCKER(hcomm::CcuInitFeature).expects(once()).will(returnValue(CcuResult::CCU_E_DRV_BUSY));
265 249 
266 EXPECT_EQ(myRank_->TryInitCcuInstance(), HCCL_SUCCESS);250 EXPECT_EQ(myRank_->TryInitCcuInstance(), HCCL_SUCCESS);
267 EXPECT_EQ(myRank_->opExpansionMode_, AICPU_TS_MODE);251 EXPECT_EQ(myRank_->opExpansionMode_, AICPU_TS_MODE);
@@ -282,7 +266,7 @@ TEST_F(CcuMsPerDeviceExclusiveTest, Ut_MyRankDestructor_When_HoldsMsReservation_
282 myRank_->devLogicId_ = DEVICE_0;266 myRank_->devLogicId_ = DEVICE_0;
283 myRank_->ccuMsCommReserved_ = true;267 myRank_->ccuMsCommReserved_ = true;
284 myRank_->ccuInsHandle_ = VALID_CCU_INS_HANDLE;268 myRank_->ccuInsHandle_ = VALID_CCU_INS_HANDLE;
285- myRank_->useCcuResStaticAlloc_ = false;269+ // 删除 useCcuResStaticAlloc_ 成员;析构路径按 ccuInsHandle_/assignedCcuInsHandle_ 是否非 0 决定清理
286 auto* fakeDrvHandle = reinterpret_cast<hcomm::CcuDrvHandle*>(0x1);270 auto* fakeDrvHandle = reinterpret_cast<hcomm::CcuDrvHandle*>(0x1);
287 myRank_->ccuDrvHandle_ = std::shared_ptr<hcomm::CcuDrvHandle>(fakeDrvHandle, [](hcomm::CcuDrvHandle*) {});271 myRank_->ccuDrvHandle_ = std::shared_ptr<hcomm::CcuDrvHandle>(fakeDrvHandle, [](hcomm::CcuDrvHandle*) {});
288 g_reservationHeldDuringInstanceDestroy.store(false);272 g_reservationHeldDuringInstanceDestroy.store(false);
@@ -20,6 +20,7 @@
20#include "channel_process.h"20#include "channel_process.h"
21#include "base_config.h"21#include "base_config.h"
22#include "ccu_device_res.h"22#include "ccu_device_res.h"
23+#include "ccu_device_pub.h"
23#include "config/env_config.h"24#include "config/env_config.h"
24#include "env_config/env_config_v2.h"25#include "env_config/env_config_v2.h"
25#define private public26#define private public
@@ -320,10 +321,11 @@ TEST_F(MyRankTest, Ut_Init_When_Default_Mode_Expect_Set_By_Env)
320}321}
321 322 
322// 测试Init时ccu驱动拉起失败回退到aicpu323// 测试Init时ccu驱动拉起失败回退到aicpu
324+// 简化后 TryInitCcuInstance 只走 OnDemand(拉起驱动 CcuInitFeature),不再调 HcommCcuInsCreateLegacy 创建实例
323TEST_F(MyRankTest, Ut_Init_When_Ccu_Driver_Fail_Expect_Fallback_Aicpu)325TEST_F(MyRankTest, Ut_Init_When_Ccu_Driver_Fail_Expect_Fallback_Aicpu)
324{326{
325 setenv("HCCL_CCU_CUSTOM_OP_MODE", "1", 1);327 setenv("HCCL_CCU_CUSTOM_OP_MODE", "1", 1);
326- MOCKER(HcommCcuInsCreateLegacy).stubs().will(returnValue(CcuResult::CCU_E_DRV_BUSY));328+ MOCKER(hcomm::CcuInitFeature).stubs().will(returnValue(CcuResult::CCU_E_DRV_BUSY));
327 329 
328 HcclMem cclBuffer;330 HcclMem cclBuffer;
329 CreateCclBuffer(cclBuffer);331 CreateCclBuffer(cclBuffer);
@@ -334,44 +336,13 @@ TEST_F(MyRankTest, Ut_Init_When_Ccu_Driver_Fail_Expect_Fallback_Aicpu)
334 unsetenv("HCCL_CCU_CUSTOM_OP_MODE");336 unsetenv("HCCL_CCU_CUSTOM_OP_MODE");
335}337}
336 338 
337-// 测试Init时ccu ms资源不足退到sched339+// 测试Init时ccu驱动拉起返非预期错误时失败
338-TEST_F(MyRankTest, Ut_Init_When_Ccu_Ms_Insufficient_Expect_Fallback_Sched)340+// 简化后 ccu instance 不在 init 时创建,MS->SCH 回退由 ReserveCcuMsCommOrFallback 在预约失败时完成;
341+// 此用例验证驱动返回非预期错误(CCU_E_PARA)时直接失败
342+TEST_F(MyRankTest, Ut_Init_When_Ccu_Driver_Unexpected_Fail_Expect_Fail)
339{343{
340 setenv("HCCL_CCU_CUSTOM_OP_MODE", "1", 1);344 setenv("HCCL_CCU_CUSTOM_OP_MODE", "1", 1);
341- MOCKER(HcommCcuInsCreateLegacy)345+ MOCKER(hcomm::CcuInitFeature).stubs().will(returnValue(CcuResult::CCU_E_PARA));
342- .stubs()
343- .will(returnValue(CcuResult::CCU_E_UNAVAIL))
344- .then(returnValue(CcuResult::CCU_SUCCESS));
345- 
346- HcclMem cclBuffer;
347- CreateCclBuffer(cclBuffer);
348- 
349- uint32_t opExpansionModeMs = CCU_MS_MODE;
350- EXPECT_EQ(myRank->Init(cclBuffer, opExpansionModeMs, 2), HCCL_SUCCESS);
351- EXPECT_EQ(myRank->opExpansionMode_, CCU_SCHED_MODE);
352- unsetenv("HCCL_CCU_CUSTOM_OP_MODE");
353-}
354- 
355-// 测试Init时ccu ms和sched资源不足回退到aicpu
356-TEST_F(MyRankTest, Ut_Init_When_Ccu_Ms_And_Sched_Insufficient_Expect_Fallback_Aicpu)
357-{
358- setenv("HCCL_CCU_CUSTOM_OP_MODE", "1", 1);
359- MOCKER(HcommCcuInsCreateLegacy).stubs().will(returnValue(CcuResult::CCU_E_UNAVAIL));
360- 
361- HcclMem cclBuffer;
362- CreateCclBuffer(cclBuffer);
363- 
364- uint32_t opExpansionModeMs = CCU_MS_MODE;
365- EXPECT_EQ(myRank->Init(cclBuffer, opExpansionModeMs, 2), HCCL_SUCCESS);
366- EXPECT_EQ(myRank->opExpansionMode_, AICPU_TS_MODE);
367- unsetenv("HCCL_CCU_CUSTOM_OP_MODE");
368-}
369- 
370-// 测试Init在申请资源时出现其他报错时失败
371-TEST_F(MyRankTest, Ut_Init_When_Resource_Fail_Expect_Fail)
372-{
373- setenv("HCCL_CCU_CUSTOM_OP_MODE", "1", 1);
374- MOCKER(HcommCcuInsCreateLegacy).stubs().will(returnValue(CcuResult::CCU_E_PARA));
375 346 
376 HcclMem cclBuffer;347 HcclMem cclBuffer;
377 CreateCclBuffer(cclBuffer);348 CreateCclBuffer(cclBuffer);
@@ -32,14 +32,9 @@
32#include "ccu_comp.h"32#include "ccu_comp.h"
33#include "mocks/ccu_device_mock_utils.h"33#include "mocks/ccu_device_mock_utils.h"
34 34 
35-namespace hcomm {
36-HcclResult GetHcclVersionForCcuKernelMgr(int& hcclVersion);
37-}
38- 
39namespace {35namespace {
40constexpr int32_t TEST_DEVICE_LOGIC_ID = 0;36constexpr int32_t TEST_DEVICE_LOGIC_ID = 0;
41constexpr int32_t OTHER_TEST_DEVICE_LOGIC_ID = 1;37constexpr int32_t OTHER_TEST_DEVICE_LOGIC_ID = 1;
42-constexpr int32_t HCCL_VERSION_USING_VALIDATE_AND_APPLY_DIE = 90100001;
43constexpr uint32_t VALID_DIE_ID = 0;38constexpr uint32_t VALID_DIE_ID = 0;
44constexpr uint32_t OTHER_VALID_DIE_ID = 1;39constexpr uint32_t OTHER_VALID_DIE_ID = 1;
45constexpr uint32_t INVALID_DIE_ID = hcomm::CCU_MAX_IODIE_NUM;40constexpr uint32_t INVALID_DIE_ID = hcomm::CCU_MAX_IODIE_NUM;
@@ -326,10 +321,6 @@ public:
326 g_deviceRefreshCalls = 0;321 g_deviceRefreshCalls = 0;
327 g_deviceRefreshResult = HcclResult::HCCL_SUCCESS;322 g_deviceRefreshResult = HcclResult::HCCL_SUCCESS;
328 323 
329- MOCKER(hcomm::GetHcclVersionForCcuKernelMgr)
330- .stubs()
331- .with(outBound(HCCL_VERSION_USING_VALIDATE_AND_APPLY_DIE))
332- .will(returnValue(HCCL_SUCCESS));
333 MOCKER(hrtGetDeviceRefresh).stubs().with(mockcpp::any()).will(invoke(MockHrtGetDeviceRefresh));324 MOCKER(hrtGetDeviceRefresh).stubs().with(mockcpp::any()).will(invoke(MockHrtGetDeviceRefresh));
334 int32_t seededDeviceLogicId = INVALID_INT;325 int32_t seededDeviceLogicId = INVALID_INT;
335 ASSERT_EQ(HcclDeviceRefresh(seededDeviceLogicId), HcclResult::HCCL_SUCCESS);326 ASSERT_EQ(HcclDeviceRefresh(seededDeviceLogicId), HcclResult::HCCL_SUCCESS);