已合并
[feat]ccu resource management compatibility new impl #4420
guoziqi创建于 25 天前
[feat]ccu resource management compatibility new impl #4420
已合并
共 20 个文件变更+627-233
| @@ -20,14 +20,14 @@ Kernel注册和启动接口需要传入`CcuInsHandle`(CCU实例句柄)。当 | |||
| 20 | 3. **绑定CCU实例**: | 20 | 3. **绑定CCU实例**: |
| 21 | 21 | ||
| 22 | - 调用[HcclCommAssignCcuIns](../comms_domain_resource_mgmt/HcclCommAssignCcuIns.md)将实例绑定到通信域。绑定成功后,实例所有权和销毁责任转移给通信域。 | 22 | - 调用[HcclCommAssignCcuIns](../comms_domain_resource_mgmt/HcclCommAssignCcuIns.md)将实例绑定到通信域。绑定成功后,实例所有权和销毁责任转移给通信域。 |
| 23 | - - 创建实例的调用方可继续使用`HcommCcuInsCreate`返回的句柄注册和启动Kernel,但不得再自行销毁实例。仅持有通信域句柄时,可调用[HcclCommQueryCcuIns](../comms_domain_resource_mgmt/HcclCommQueryCcuIns.md)获取已绑定的实例句柄。 | 23 | + - 创建实例的调用方可继续使用`HcommCcuInsCreate`返回的句柄注册和启动Kernel,但不得再自行销毁实例。仅持有通信域句柄时,可调用[HcclCommQueryAssignedCcuIns](../comms_domain_resource_mgmt/HcclCommQueryAssignedCcuIns.md)获取已绑定的实例句柄。 |
| 24 | 24 | ||
| 25 | - 通过通信域查询实例句柄的典型用法: | 25 | + 通过通信域查询已绑定实例句柄的典型用法: |
| 26 | 26 | ||
| 27 | ```c | 27 | ```c |
| 28 | CcuInsHandle insHandle = 0; | 28 | CcuInsHandle insHandle = 0; |
| 29 | uint32_t insNum = 0; | 29 | uint32_t insNum = 0; |
| 30 | - HcclResult ret = HcclCommQueryCcuIns(comm, &insHandle, &insNum); | 30 | + HcclResult ret = HcclCommQueryAssignedCcuIns(comm, &insHandle, &insNum); |
| 31 | // 当前一个通信域最多绑定1个CCU实例,查询成功时insNum为1。 | 31 | // 当前一个通信域最多绑定1个CCU实例,查询成功时insNum为1。 |
| 32 | if (ret != HCCL_SUCCESS || insNum != 1) { | 32 | if (ret != HCCL_SUCCESS || insNum != 1) { |
| 33 | // 错误处理 | 33 | // 错误处理 |
| @@ -36,7 +36,8 @@ Kernel注册和启动接口需要传入`CcuInsHandle`(CCU实例句柄)。当 | |||
| 36 | 36 | ||
| 37 | > 该接口属于HCCL层(不在`Hcomm*`/`Ccu*`系列内),暂未提供独立API参考页面。 | 37 | > 该接口属于HCCL层(不在`Hcomm*`/`Ccu*`系列内),暂未提供独立API参考页面。 |
| 38 | > 完整签名以头文件`include/hccl/hccl_ccu_res.h`为准。 | 38 | > 完整签名以头文件`include/hccl/hccl_ccu_res.h`为准。 |
| 39 | -> 如果通信域未绑定CCU实例,`HcclCommQueryCcuIns`返回`HCCL_E_UNAVAIL`。 | 39 | +> 如果通信域未绑定CCU实例,`HcclCommQueryAssignedCcuIns`返回`HCCL_E_UNAVAIL`。 |
| 40 | +> 此外,[HcclCommQueryCcuIns](../comms_domain_resource_mgmt/HcclCommQueryCcuIns.md)用于查询通信域自有的CCU实例(未创建时创建CCU实例),与本接口查询的绑定实例相互独立。 | ||
| 40 | 41 | ||
| 41 | ## 接口调用顺序 | 42 | ## 接口调用顺序 |
| 42 | 43 | ||
Mdocs/zh/api_ref/comm_opdev/control_plane_api/comms_domain_resource_mgmt/HcclCommAssignCcuIns.md+3-3
| @@ -20,7 +20,7 @@ | |||
| 20 | 20 | ||
| 21 | ## 功能说明 | 21 | ## 功能说明 |
| 22 | 22 | ||
| 23 | -将调用方已创建的CCU实例绑定到指定HCCL通信域。绑定成功后,通信域保存该实例句柄,并接管CCU实例的所有权和销毁责任;后续可通过`HcclCommQueryCcuIns`查询该实例。 | 23 | +将调用方已创建的CCU实例绑定到指定HCCL通信域。绑定成功后,通信域保存该实例句柄,并接管CCU实例的所有权和销毁责任;后续可通过`HcclCommQueryAssignedCcuIns`查询该实例。 |
| 24 | 24 | ||
| 25 | 每个通信域只能绑定一个CCU实例。通信域已有绑定实例时,本接口不会覆盖原实例。 | 25 | 每个通信域只能绑定一个CCU实例。通信域已有绑定实例时,本接口不会覆盖原实例。 |
| 26 | 26 | ||
| @@ -56,7 +56,7 @@ HcclResult HcclCommAssignCcuIns(HcclComm comm, CcuInsHandle insHandle) | |||
| 56 | - 绑定失败时,`insHandle`的所有权仍归调用方,调用方负责继续使用或销毁该实例。 | 56 | - 绑定失败时,`insHandle`的所有权仍归调用方,调用方负责继续使用或销毁该实例。 |
| 57 | - 同一个`insHandle`只能绑定到一个通信域,不得重复绑定到其他通信域。 | 57 | - 同一个`insHandle`只能绑定到一个通信域,不得重复绑定到其他通信域。 |
| 58 | - 同一个通信域不支持重复绑定。无论新旧句柄是否相同,再次调用本接口均返回`HCCL_E_PARA`,原绑定关系保持不变。 | 58 | - 同一个通信域不支持重复绑定。无论新旧句柄是否相同,再次调用本接口均返回`HCCL_E_PARA`,原绑定关系保持不变。 |
| 59 | -- 本接口只保证多个`HcclCommAssignCcuIns`调用之间的并发安全。调用方须保证本接口不与`HcclCommQueryCcuIns`或`HcclCommDestroy`并发执行。 | 59 | +- 本接口只保证多个`HcclCommAssignCcuIns`调用之间的并发安全。调用方须保证本接口不与`HcclCommQueryCcuIns`、`HcclCommQueryAssignedCcuIns`或`HcclCommDestroy`并发执行。 |
| 60 | - `insHandle`须在`comm`所在的当前Device上创建。 | 60 | - `insHandle`须在`comm`所在的当前Device上创建。 |
| 61 | 61 | ||
| 62 | ## 调用示例 | 62 | ## 调用示例 |
| @@ -79,7 +79,7 @@ if (hcclRet != HCCL_SUCCESS) { | |||
| 79 | // 绑定成功,实例所有权已转移给通信域,调用方不再销毁insHandle。 | 79 | // 绑定成功,实例所有权已转移给通信域,调用方不再销毁insHandle。 |
| 80 | CcuInsHandle queriedInsHandle = 0; | 80 | CcuInsHandle queriedInsHandle = 0; |
| 81 | uint32_t insNum = 0; | 81 | uint32_t insNum = 0; |
| 82 | -hcclRet = HcclCommQueryCcuIns(comm, &queriedInsHandle, &insNum); | 82 | +hcclRet = HcclCommQueryAssignedCcuIns(comm, &queriedInsHandle, &insNum); |
| 83 | if (hcclRet != HCCL_SUCCESS || insNum != 1 || | 83 | if (hcclRet != HCCL_SUCCESS || insNum != 1 || |
| 84 | queriedInsHandle != insHandle) { | 84 | queriedInsHandle != insHandle) { |
| 85 | HcclCommDestroy(comm); | 85 | HcclCommDestroy(comm); |
| @@ -0,0 +1,107 @@ | |||
| 1 | +# HcclCommQueryAssignedCcuIns | ||
| 2 | + | ||
| 3 | +## 产品支持情况 | ||
| 4 | + | ||
| 5 | +<!-- npu="950" id1 --> | ||
| 6 | +- Ascend 950PR/Ascend 950DT:支持 | ||
| 7 | +<!-- end id1 --> | ||
| 8 | +<!-- npu="A3" id2 --> | ||
| 9 | +- Atlas A3 训练系列产品/Atlas A3 推理系列产品:不支持 | ||
| 10 | +<!-- end id2 --> | ||
| 11 | +<!-- npu="910b" id3 --> | ||
| 12 | +- Atlas A2 训练系列产品/Atlas A2 推理系列产品:不支持 | ||
| 13 | +<!-- end id3 --> | ||
| 14 | +<!-- npu="910" id4 --> | ||
| 15 | +- Atlas 训练系列产品:不支持 | ||
| 16 | +<!-- end id4 --> | ||
| 17 | +<!-- npu="310p" id5 --> | ||
| 18 | +- Atlas 推理系列产品:不支持 | ||
| 19 | +<!-- end id5 --> | ||
| 20 | + | ||
| 21 | +## 功能说明 | ||
| 22 | + | ||
| 23 | +查询指定HCCL通信域通过新方式绑定的CCU实例句柄,返回实例句柄用于后续CCU Kernel的注册与下发。 | ||
| 24 | + | ||
| 25 | +新方式指:调用方先通过`HcommCcuInsCreate`或`HcommCcuInsCreateDefault`创建CCU实例,再通过[HcclCommAssignCcuIns](HcclCommAssignCcuIns.md)将实例绑定到通信域。本接口查询的就是该绑定关系所对应的实例句柄(assignedCcuInsHandle)。查询成功时固定返回1个实例:insHandles[0]为已绑定的实例句柄,*insNum为1。 | ||
| 26 | + | ||
| 27 | +本接口不会创建CCU实例。若通信域尚未通过`HcclCommAssignCcuIns`绑定任何CCU实例,本接口返回HCCL_E_UNAVAIL。 | ||
| 28 | + | ||
| 29 | +> [!NOTE]注意 | ||
| 30 | +> - 本接口查询的是通过`HcclCommAssignCcuIns`绑定的CCU实例,与通信域自有的CCU实例(由[HcclCommQueryCcuIns](HcclCommQueryCcuIns.md)查询/创建)相互独立。若需查询通信域自有的实例,请使用`HcclCommQueryCcuIns`。 | ||
| 31 | +> - 查询结果仅供借用,不转移实例所有权。已绑定的CCU实例所有权归通信域,由通信域负责释放;调用者不可销毁该实例,否则会造成对同一实例的重复释放,破坏通信域的资源管理。 | ||
| 32 | + | ||
| 33 | +## 函数原型 | ||
| 34 | + | ||
| 35 | +```c | ||
| 36 | +HcclResult HcclCommQueryAssignedCcuIns(HcclComm comm, CcuInsHandle *insHandles, uint32_t *insNum) | ||
| 37 | +``` | ||
| 38 | + | ||
| 39 | +## 参数说明 | ||
| 40 | + | ||
| 41 | +| 参数名 | 输入/输出 | 描述 | | ||
| 42 | +| --- | --- | --- | | ||
| 43 | +| comm | 输入 | 通信域句柄,不可为nullptr。<br>HcclComm类型的定义如下:<br>typedef void *HcclComm; | | ||
| 44 | +| insHandles | 输出 | CCU实例句柄数组,不可为nullptr。查询成功后insHandles[0]返回已绑定的CCU实例句柄。<br>调用方需分配至少1个CcuInsHandle元素的空间。<br>CcuInsHandle类型的定义如下:<br>typedef uint64_t CcuInsHandle; | | ||
| 45 | +| insNum | 输出 | CCU实例数量,不可为nullptr。查询成功后返回值固定为1。 | | ||
| 46 | + | ||
| 47 | +## 返回值 | ||
| 48 | + | ||
| 49 | +[HcclResult](../../../comm_mgr_c/data_type_definition/HcclResult.md):接口成功返回HCCL_SUCCESS,其他失败。 | ||
| 50 | + | ||
| 51 | +- 当comm、insHandles或insNum为nullptr时,返回HCCL_E_PTR。 | ||
| 52 | +- 当通信域代际不支持CCU(早于Ascend 950PR/Ascend 950DT)时,返回HCCL_E_NOT_SUPPORT。 | ||
| 53 | +- 当通信域尚未通过`HcclCommAssignCcuIns`绑定CCU实例时,返回HCCL_E_UNAVAIL,本接口不会创建实例。 | ||
| 54 | + | ||
| 55 | +## 约束说明 | ||
| 56 | + | ||
| 57 | +1. CCU特性仅在Ascend 950PR/Ascend 950DT及以上代际支持,在更早代际的通信域上调用返回HCCL_E_NOT_SUPPORT。 | ||
| 58 | +2. 调用本接口前,须已通过`HcommCcuInsCreate`或`HcommCcuInsCreateDefault`创建CCU实例,并通过`HcclCommAssignCcuIns`绑定到通信域;否则返回HCCL_E_UNAVAIL。 | ||
| 59 | +3. 本接口查询的CCU实例与通信域自有的CCU实例(由`HcclCommQueryCcuIns`查询/创建)相互独立,二者不会相互覆盖或影响。 | ||
| 60 | +4. 返回的CCU实例句柄仅供借用,所有权仍归通信域所有,由通信域负责释放。调用者不可销毁该实例,否则会造成对同一实例的重复释放,破坏通信域的资源管理。 | ||
| 61 | +5. 本接口不保证与`HcclCommAssignCcuIns`或`HcclCommDestroy`并发执行的安全。调用方须保证本接口不与`HcclCommAssignCcuIns`或`HcclCommDestroy`并发执行。 | ||
| 62 | + | ||
| 63 | +## 调用示例 | ||
| 64 | + | ||
| 65 | +```c | ||
| 66 | +// comm 需通过 HcclCommInitClusterInfo 等接口完成初始化,此处仅为示例占位 | ||
| 67 | +HcclComm comm = /* 已初始化的通信域句柄 */; | ||
| 68 | + | ||
| 69 | +// 1. 新方式:先创建CCU实例,再绑定到通信域 | ||
| 70 | +CcuInsHandle insHandle = 0; | ||
| 71 | +CcuResult ccuRet = HcommCcuInsCreateDefault(NULL, 0, &insHandle); | ||
| 72 | +if (ccuRet != CCU_SUCCESS) { | ||
| 73 | + return ccuRet; | ||
| 74 | +} | ||
| 75 | + | ||
| 76 | +HcclResult hcclRet = HcclCommAssignCcuIns(comm, insHandle); | ||
| 77 | +if (hcclRet != HCCL_SUCCESS) { | ||
| 78 | + // 绑定失败,实例所有权仍归调用方。 | ||
| 79 | + HcommCcuInsDestroy(insHandle); | ||
| 80 | + return hcclRet; | ||
| 81 | +} | ||
| 82 | +// 绑定成功,实例所有权已转移给通信域,调用方不再销毁insHandle。 | ||
| 83 | + | ||
| 84 | +// 2. 查询已绑定的CCU实例(新方式) | ||
| 85 | +CcuInsHandle queriedInsHandle = 0; | ||
| 86 | +uint32_t insNum = 0; | ||
| 87 | +hcclRet = HcclCommQueryAssignedCcuIns(comm, &queriedInsHandle, &insNum); | ||
| 88 | +// 当前实现固定返回1个CCU实例,insNum != 1视为失败 | ||
| 89 | +if (hcclRet != HCCL_SUCCESS || insNum != 1) { | ||
| 90 | + // 错误处理:接口失败返回原始错误码,接口成功但实例数异常返回内部错误码 | ||
| 91 | + HcclCommDestroy(comm); | ||
| 92 | + return (hcclRet != HCCL_SUCCESS) ? hcclRet : HCCL_E_INTERNAL; | ||
| 93 | +} | ||
| 94 | + | ||
| 95 | +// 校验查询到的句柄与绑定时的句柄一致 | ||
| 96 | +if (queriedInsHandle != insHandle) { | ||
| 97 | + HcclCommDestroy(comm); | ||
| 98 | + return HCCL_E_INTERNAL; | ||
| 99 | +} | ||
| 100 | + | ||
| 101 | +// 使用queriedInsHandle注册并下发CCU Kernel | ||
| 102 | +CcuResult regStartRet = HcommCcuKernelRegisterStart(queriedInsHandle); | ||
| 103 | +// ... | ||
| 104 | + | ||
| 105 | +// 销毁通信域时,由通信域销毁已绑定的CCU实例。 | ||
| 106 | +return HcclCommDestroy(comm); | ||
| 107 | +``` | ||
Mdocs/zh/api_ref/comm_opdev/control_plane_api/comms_domain_resource_mgmt/HcclCommQueryCcuIns.md+23-8
| @@ -20,12 +20,15 @@ | |||
| 20 | 20 | ||
| 21 | ## 功能说明 | 21 | ## 功能说明 |
| 22 | 22 | ||
| 23 | -查询指定HCCL通信域已绑定的CCU实例,返回实例句柄用于后续CCU Kernel的注册与下发。 | 23 | +查询指定HCCL通信域自有的CCU实例句柄,返回实例句柄用于后续CCU Kernel的注册与下发。 |
| 24 | 24 | ||
| 25 | -当前一个通信域最多绑定一个CCU实例,因此查询成功时固定返回1个实例:insHandles[0]为已绑定的实例句柄,*insNum为1。 | 25 | +通信域在初始化阶段不再主动创建CCU实例。本接口查询通信域自有的CCU实例句柄(即以固定资源量创建、由通信域自有的句柄);当该句柄尚未存在(值为`0`)时,接口会根据通信域的算子扩展模式(opExpansionMode)创建一个CCU实例并保存到通信域,随后返回该句柄。查询成功时固定返回1个实例:insHandles[0]为通信域自有的实例句柄,*insNum为1。 |
| 26 | + | ||
| 27 | +当前一个通信域最多持有一个自有的CCU实例,多次查询返回同一句柄,不会重复创建。 | ||
| 26 | 28 | ||
| 27 | > [!NOTE]注意 | 29 | > [!NOTE]注意 |
| 28 | -> 查询结果仅供借用,不转移实例所有权。返回的CCU实例所有权归通信域,由通信域负责释放;调用者不可销毁该实例,否则会造成对同一实例的重复释放,破坏通信域的资源管理。 | 30 | +> - 本接口查询的是通信域自有的CCU实例,与通过[HcclCommAssignCcuIns](HcclCommAssignCcuIns.md)绑定的CCU实例(新版本)相互独立。若需查询通过`HcclCommAssignCcuIns`绑定的实例,请使用[HcclCommQueryAssignedCcuIns](HcclCommQueryAssignedCcuIns.md)。 |
| 31 | +> - 查询结果仅供借用,不转移实例所有权。创建的CCU实例所有权归通信域,由通信域负责释放;调用者不可销毁该实例,否则会造成对同一实例的重复释放,破坏通信域的资源管理。 | ||
| 29 | 32 | ||
| 30 | ## 函数原型 | 33 | ## 函数原型 |
| 31 | 34 | ||
| @@ -38,7 +41,7 @@ HcclResult HcclCommQueryCcuIns(HcclComm comm, CcuInsHandle *insHandles, uint32_t | |||
| 38 | | 参数名 | 输入/输出 | 描述 | | 41 | | 参数名 | 输入/输出 | 描述 | |
| 39 | | --- | --- | --- | | 42 | | --- | --- | --- | |
| 40 | | comm | 输入 | 通信域句柄,不可为nullptr。<br>HcclComm类型的定义如下:<br>typedef void *HcclComm; | | 43 | | comm | 输入 | 通信域句柄,不可为nullptr。<br>HcclComm类型的定义如下:<br>typedef void *HcclComm; | |
| 41 | -| insHandles | 输出 | CCU实例句柄数组,不可为nullptr。查询成功后insHandles[0]返回已绑定的CCU实例句柄。<br>CcuInsHandle类型的定义如下:<br>typedef uint64_t CcuInsHandle; | | 44 | +| insHandles | 输出 | CCU实例句柄数组,不可为nullptr。查询成功后insHandles[0]返回通信域自有的CCU实例句柄(若调用前不存在则创建)。<br>调用方需分配至少1个CcuInsHandle元素的空间。<br>CcuInsHandle类型的定义如下:<br>typedef uint64_t CcuInsHandle; | |
| 42 | | insNum | 输出 | CCU实例数量,不可为nullptr。查询成功后返回值固定为1。 | | 45 | | insNum | 输出 | CCU实例数量,不可为nullptr。查询成功后返回值固定为1。 | |
| 43 | 46 | ||
| 44 | ## 返回值 | 47 | ## 返回值 |
| @@ -47,13 +50,21 @@ HcclResult HcclCommQueryCcuIns(HcclComm comm, CcuInsHandle *insHandles, uint32_t | |||
| 47 | 50 | ||
| 48 | - 当comm、insHandles或insNum为nullptr时,返回HCCL_E_PTR。 | 51 | - 当comm、insHandles或insNum为nullptr时,返回HCCL_E_PTR。 |
| 49 | - 当通信域代际不支持CCU(早于Ascend 950PR/Ascend 950DT)时,返回HCCL_E_NOT_SUPPORT。 | 52 | - 当通信域代际不支持CCU(早于Ascend 950PR/Ascend 950DT)时,返回HCCL_E_NOT_SUPPORT。 |
| 50 | -- 当通信域未绑定CCU实例时,返回HCCL_E_UNAVAIL。 | 53 | +- 当通信域的自有CCU实例句柄为`0`、且算子扩展模式未使能CCU时,返回HCCL_E_UNAVAIL,不创建实例。 |
| 54 | +- 当创建CCU实例失败时,返回创建接口的原始错误码(透传CcuResult,如CCU_E_DRV_BUSY、CCU_E_UNAVAIL等)。 | ||
| 55 | + | ||
| 56 | +> [!NOTE]资源不足处理 | ||
| 57 | +> - 当算子扩展模式为CCU_MS且CCU资源不足以创建CCU_MS实例时,本接口会自动降级到CCU_SCHED重试创建一次。 | ||
| 58 | +> - 当CCU_SCHED实例也无法创建(资源仍不足)时,返回HCCL_E_UNAVAIL。**本接口不会自动降级到AICPU_TS**,调用方需在收到HCCL_E_UNAVAIL后自行决定是否降级。 | ||
| 51 | 59 | ||
| 52 | ## 约束说明 | 60 | ## 约束说明 |
| 53 | 61 | ||
| 54 | 1. CCU特性仅在Ascend 950PR/Ascend 950DT及以上代际支持,在更早代际的通信域上调用返回HCCL_E_NOT_SUPPORT。 | 62 | 1. CCU特性仅在Ascend 950PR/Ascend 950DT及以上代际支持,在更早代际的通信域上调用返回HCCL_E_NOT_SUPPORT。 |
| 55 | -2. 需在通信域完成初始化并绑定CCU实例后调用,否则返回HCCL_E_UNAVAIL。 | 63 | +2. 需在通信域完成初始化后调用。当通信域的自有CCU实例尚未创建时,本接口会创建;若通信域展开模式未使能CCU,返回HCCL_E_UNAVAIL。 |
| 56 | -3. 返回的CCU实例句柄仅供借用,所有权仍归通信域所有,由通信域负责释放。调用者不可销毁该实例,否则会造成对同一实例的重复释放,破坏通信域的资源管理。 | 64 | +3. 本接口查询的CCU实例与通过`HcclCommAssignCcuIns`绑定的CCU实例相互独立,二者不会相互覆盖或影响。 |
| 65 | +4. 返回的CCU实例句柄仅供借用,所有权仍归通信域所有,由通信域负责释放。调用者不可销毁该实例,否则会造成对同一实例的重复释放,破坏通信域的资源管理。 | ||
| 66 | +5. 本接口仅保证多次`HcclCommQueryCcuIns`调用之间的幂等性(已创建则直接返回,不重复创建)。调用方须保证本接口不与`HcclCommAssignCcuIns`或`HcclCommDestroy`并发执行。 | ||
| 67 | +6. CCU资源不足时本接口仅在CCU_MS→CCU_SCHED之间自动降级一次,不会自动降级到AICPU_TS。调用方收到HCCL_E_UNAVAIL后需自行处理是否降级到AICPU_TS模式。 | ||
| 57 | 68 | ||
| 58 | ## 调用示例 | 69 | ## 调用示例 |
| 59 | 70 | ||
| @@ -63,8 +74,12 @@ HcclComm comm = /* 已初始化的通信域句柄 */; | |||
| 63 | CcuInsHandle insHandle = 0; | 74 | CcuInsHandle insHandle = 0; |
| 64 | uint32_t insNum = 0; | 75 | uint32_t insNum = 0; |
| 65 | 76 | ||
| 66 | -// 查询通信域已绑定的CCU实例 | 77 | +// 查询通信域自有的CCU实例;若尚未创建则按通信域展开模式创建 |
| 67 | HcclResult ret = HcclCommQueryCcuIns(comm, &insHandle, &insNum); | 78 | HcclResult ret = HcclCommQueryCcuIns(comm, &insHandle, &insNum); |
| 79 | +if (ret == HCCL_E_UNAVAIL) { | ||
| 80 | + // CCU资源不足,无法创建CCU实例,调用方需自行处理是否降级到AICPU_TS模式 | ||
| 81 | + return fallbackToAicpuTs(comm, ...); | ||
| 82 | +} | ||
| 68 | // 当前实现固定返回1个CCU实例,insNum != 1视为失败 | 83 | // 当前实现固定返回1个CCU实例,insNum != 1视为失败 |
| 69 | if (ret != HCCL_SUCCESS || insNum != 1) { | 84 | if (ret != HCCL_SUCCESS || insNum != 1) { |
Z | |||
| 70 | // 错误处理:接口失败返回原始错误码,接口成功但实例数异常返回内部错误码 | 85 | // 错误处理:接口失败返回原始错误码,接口成功但实例数异常返回内部错误码 |
| @@ -24,6 +24,7 @@ | |||
| 24 | - [HcclCommMemReg](HcclCommMemReg.md) | 24 | - [HcclCommMemReg](HcclCommMemReg.md) |
| 25 | - [HcclCommAssignCcuIns](HcclCommAssignCcuIns.md) | 25 | - [HcclCommAssignCcuIns](HcclCommAssignCcuIns.md) |
| 26 | - [HcclCommQueryCcuIns](HcclCommQueryCcuIns.md) | 26 | - [HcclCommQueryCcuIns](HcclCommQueryCcuIns.md) |
| 27 | +- [HcclCommQueryAssignedCcuIns](HcclCommQueryAssignedCcuIns.md) | ||
| 27 | - [HcclTeamCreateDescInit](HcclTeamCreateDescInit.md) | 28 | - [HcclTeamCreateDescInit](HcclTeamCreateDescInit.md) |
| 28 | - [HcclTeamCreateChannelsDescInit](HcclTeamCreateChannelsDescInit.md) | 29 | - [HcclTeamCreateChannelsDescInit](HcclTeamCreateChannelsDescInit.md) |
| 29 | - [HcclWorldTeamCreate](HcclWorldTeamCreate.md) | 30 | - [HcclWorldTeamCreate](HcclWorldTeamCreate.md) |
| @@ -32,16 +32,31 @@ extern "C" { | |||
| 32 | extern HcclResult HcclCommAssignCcuIns(HcclComm comm, CcuInsHandle insHandle); | 32 | extern HcclResult HcclCommAssignCcuIns(HcclComm comm, CcuInsHandle insHandle); |
| 33 | 33 | ||
| 34 | /** | 34 | /** |
| 35 | - * @brief 查询指定 HCCL 通信域已绑定的 CCU 实例。 | 35 | + * @brief 查询指定 HCCL 通信域自有的 CCU 实例句柄。 |
| 36 | * @param[in] comm HCCL 通信域句柄,不可为 nullptr。 | 36 | * @param[in] comm HCCL 通信域句柄,不可为 nullptr。 |
| 37 | * @param[out] insHandles 查询成功后返回 CCU 实例句柄数组,不可为 nullptr。 | 37 | * @param[out] insHandles 查询成功后返回 CCU 实例句柄数组,不可为 nullptr。 |
| 38 | * @param[out] insNum 查询成功后返回 CCU 实例数量,不可为 nullptr。 | 38 | * @param[out] insNum 查询成功后返回 CCU 实例数量,不可为 nullptr。 |
| 39 | * @return HcclResult。 | 39 | * @return HcclResult。 |
| 40 | + * @note 查询通信域自有的 | ||
| 41 | + * ccuInsHandle;若通信域未创建CCU实例且通信域展开模式为CCU,则按通信域展开模式创建CCU实例后返回。 | ||
| 40 | * @note 返回的 CCU 实例句柄仅供借用,不转移所有权,调用方不得销毁该句柄。 | 42 | * @note 返回的 CCU 实例句柄仅供借用,不转移所有权,调用方不得销毁该句柄。 |
| 41 | * @note 调用方必须保证本接口不与 HcclCommAssignCcuIns 或 HcclCommDestroy 并发执行。 | 43 | * @note 调用方必须保证本接口不与 HcclCommAssignCcuIns 或 HcclCommDestroy 并发执行。 |
| 42 | */ | 44 | */ |
| 43 | extern HcclResult HcclCommQueryCcuIns(HcclComm comm, CcuInsHandle* insHandles, uint32_t* insNum); | 45 | extern HcclResult HcclCommQueryCcuIns(HcclComm comm, CcuInsHandle* insHandles, uint32_t* insNum); |
| 44 | 46 | ||
| 47 | +/** | ||
| 48 | + * @brief 查询指定 HCCL 通信域通过HcclCommAssignCcuIns绑定的 CCU 实例句柄。 | ||
| 49 | + * @param[in] comm HCCL 通信域句柄,不可为 nullptr。 | ||
| 50 | + * @param[out] insHandles 查询成功后返回 CCU 实例句柄数组,不可为 nullptr。 | ||
| 51 | + * @param[out] insNum 查询成功后返回 CCU 实例数量,不可为 nullptr。 | ||
| 52 | + * @return HcclResult。 | ||
| 53 | + * @note 查询通过HcclCommAssignCcuIns绑定给通信域的CCU实例句柄;未绑定时返回 HCCL_E_UNAVAIL,不创建句柄。 | ||
| 54 | + * 调用方须先通过 HcommCcuInsCreate + HcclCommAssignCcuIns 绑定。 | ||
| 55 | + * @note 返回的 CCU 实例句柄仅供借用,不转移所有权,调用方不得销毁该句柄。 | ||
| 56 | + * @note 调用方必须保证本接口不与 HcclCommAssignCcuIns 或 HcclCommDestroy 并发执行。 | ||
| 57 | + */ | ||
| 58 | +extern HcclResult HcclCommQueryAssignedCcuIns(HcclComm comm, CcuInsHandle* insHandles, uint32_t* insNum); | ||
| 59 | + | ||
| 45 | 60 | ||
| 46 | } | 61 | } |
| 47 | 62 | ||
| @@ -65,7 +65,7 @@ static CcuResult CcuKernelTryRegister( | |||
| 65 | { | 65 | { |
| 66 | CCU_EXCEPTION_HANDLE_BEGIN | 66 | CCU_EXCEPTION_HANDLE_BEGIN |
| 67 | auto& kernelMgr = hcomm::CcuKernelMgr::GetInstance(devLogicId); | 67 | auto& kernelMgr = hcomm::CcuKernelMgr::GetInstance(devLogicId); |
| 68 | - CCU_CHK_RET(kernelMgr.Register(*resPack, dieId, kernelFuncName, kernelFunc, kernelArgs, argNum, newHandle)); | 68 | + CCU_CHK_RET(kernelMgr.Register(*resPack, dieId, kernelFuncName, kernelFunc, kernelArgs, argNum, ccuIns, newHandle)); |
| 69 | CCU_CHK_RET(ccuIns->SaveKernel(newHandle)); | 69 | CCU_CHK_RET(ccuIns->SaveKernel(newHandle)); |
| 70 | CCU_EXCEPTION_HANDLE_END | 70 | CCU_EXCEPTION_HANDLE_END |
| 71 | return CcuResult::CCU_SUCCESS; | 71 | return CcuResult::CCU_SUCCESS; |
| @@ -191,7 +191,7 @@ CcuResult HcommCcuKernelQueryResReq( | |||
| 191 | return CcuResult::CCU_SUCCESS; | 191 | return CcuResult::CCU_SUCCESS; |
| 192 | } | 192 | } |
| 193 | 193 | ||
| 194 | -// Deprecated: legacy internal CCU instance creation path. | 194 | +// 按 CCU 实例类型创建 ccu 实例(兼容 hccl 旧版本) |
| 195 | CcuResult HcommCcuInsCreateLegacy(const CcuInstanceType insType, CcuInsHandle* ccuInsHandle) | 195 | CcuResult HcommCcuInsCreateLegacy(const CcuInstanceType insType, CcuInsHandle* ccuInsHandle) |
| 196 | { | 196 | { |
| 197 | CCU_CHK_PTR_NULL(ccuInsHandle); | 197 | CCU_CHK_PTR_NULL(ccuInsHandle); |
| @@ -66,6 +66,7 @@ CcuResult CcuInstance::InitByInsType(const CcuInstanceType insType) | |||
| 66 | CCU_CHK_RET(FillTotalResDescs()); | 66 | CCU_CHK_RET(FillTotalResDescs()); |
| 67 | } | 67 | } |
| 68 | 68 | ||
| 69 | + isFixedResNum_ = true; | ||
| 69 | return CcuResult::CCU_SUCCESS; | 70 | return CcuResult::CCU_SUCCESS; |
| 70 | } | 71 | } |
| 71 | 72 | ||
| @@ -44,6 +44,9 @@ public: | |||
| 44 | const std::vector<CcuKernelHandle>& GetUntranslatedKernels(); | 44 | const std::vector<CcuKernelHandle>& GetUntranslatedKernels(); |
| 45 | void SetHandle(CcuInsHandle insHandle); | 45 | void SetHandle(CcuInsHandle insHandle); |
| 46 | 46 | ||
| 47 | + // 是否使用固定资源数量创建(InitByInsType 路径置 true,其余路径保持 false) | ||
| 48 | + bool IsFixedResNum() const { return isFixedResNum_; } | ||
| 49 | + | ||
| 47 | CcuResult BeginRegister(); | 50 | CcuResult BeginRegister(); |
| 48 | CcuResult CheckRegistering() const; | 51 | CcuResult CheckRegistering() const; |
| 49 | CcuResult EndRegister(); | 52 | CcuResult EndRegister(); |
| @@ -62,6 +65,7 @@ private: | |||
| 62 | std::vector<CcuKernelHandle> kernelHandles_{}; | 65 | std::vector<CcuKernelHandle> kernelHandles_{}; |
| 63 | std::vector<CcuKernelHandle> untranslatedKernelHandles_{}; | 66 | std::vector<CcuKernelHandle> untranslatedKernelHandles_{}; |
| 64 | std::array<CcuResDesc, CCU_MAX_IODIE_NUM> totalResDescs_{}; // ccuIns持有的全部资源描述符 | 67 | std::array<CcuResDesc, CCU_MAX_IODIE_NUM> totalResDescs_{}; // ccuIns持有的全部资源描述符 |
| 68 | + bool isFixedResNum_{false}; // InitByInsType 路径置 true(使用固定资源数量创建),其余保持 false | ||
| 65 | }; | 69 | }; |
| 66 | 70 | ||
| 67 | } // namespace hcomm | 71 | } // namespace hcomm |
| @@ -130,7 +130,7 @@ HcclResult CcuKernelMgr::Deinit() | |||
| 130 | 130 | ||
| 131 | CcuResult CcuKernelMgr::Register( | 131 | CcuResult CcuKernelMgr::Register( |
| 132 | CcuResPack& resPack, const uint32_t dieId, const char* kernelFuncName, const void* kernelFunc, | 132 | CcuResPack& resPack, const uint32_t dieId, const char* kernelFuncName, const void* kernelFunc, |
| 133 | - const void** kernelArgs, const uint32_t argNum, CcuKernelHandle& kernelHandle) | 133 | + const void** kernelArgs, const uint32_t argNum, CcuInstance* ccuIns, CcuKernelHandle& kernelHandle) |
| 134 | { | 134 | { |
| 135 | // 允许kernelFuncName为空,此时传递默认名称 | 135 | // 允许kernelFuncName为空,此时传递默认名称 |
| 136 | CCU_CHK_PTR_NULL(kernelFunc); | 136 | CCU_CHK_PTR_NULL(kernelFunc); |
| @@ -143,7 +143,7 @@ CcuResult CcuKernelMgr::Register( | |||
| 143 | 143 | ||
| 144 | // 注意处理时序,需要先重置后处理rep | 144 | // 注意处理时序,需要先重置后处理rep |
| 145 | std::unique_lock<std::mutex> lock(kernelMapMutex_); | 145 | std::unique_lock<std::mutex> lock(kernelMapMutex_); |
| 146 | - CCU_CHK_RET(BuildKernel(dieId, kernelFuncName, kernelFunc, kernelArgs, argNum)); | 146 | + CCU_CHK_RET(BuildKernel(dieId, kernelFuncName, kernelFunc, kernelArgs, argNum, ccuIns)); |
| 147 | 147 | ||
| 148 | CcuResult ret = AllocRes(resPack); | 148 | CcuResult ret = AllocRes(resPack); |
| 149 | if (ret != CcuResult::CCU_SUCCESS) { | 149 | if (ret != CcuResult::CCU_SUCCESS) { |
| @@ -160,7 +160,7 @@ CcuResult CcuKernelMgr::Register( | |||
| 160 | 160 | ||
| 161 | CcuResult CcuKernelMgr::BuildKernel( | 161 | CcuResult CcuKernelMgr::BuildKernel( |
| 162 | const uint32_t dieId, const char* kernelFuncName, const void* kernelFunc, const void** kernelArgs, | 162 | const uint32_t dieId, const char* kernelFuncName, const void* kernelFunc, const void** kernelArgs, |
| 163 | - const uint32_t argNum) | 163 | + const uint32_t argNum, CcuInstance* ccuIns) |
| 164 | { | 164 | { |
| 165 | currKernel_ = std::make_unique<CcuKernel>(); // 重置待构建kernel | 165 | currKernel_ = std::make_unique<CcuKernel>(); // 重置待构建kernel |
| 166 | // 执行算法流程时将资源占用临时记录在 die 0,后续确定实际 die 并迁移资源 | 166 | // 执行算法流程时将资源占用临时记录在 die 0,后续确定实际 die 并迁移资源 |
| @@ -184,13 +184,11 @@ CcuResult CcuKernelMgr::BuildKernel( | |||
| 184 | } | 184 | } |
| 185 | 185 | ||
| 186 | currKernel_->FlushClosablePendingIfs(); // 处理未闭合的if | 186 | currKernel_->FlushClosablePendingIfs(); // 处理未闭合的if |
| 187 | - int hcclVersion = 0; | 187 | + if (ccuIns != nullptr && ccuIns->IsFixedResNum()) { |
| 188 | - CCU_CHK_RET(GetHcclVersionForCcuKernelMgr(hcclVersion)); | 188 | + // 按固定资源数量创建的 ccu instance,外部 dieId 始终为 0,从 channel 获取实际 dieId |
L 可以在通信域的ccu ins时候把die id保存到ins里吗 ![]() ![]() | |||
| 189 | - if (hcclVersion <= MAX_HCCL_VERSION_USING_CCU_RES_STATIC_ALLOC) { | ||
| 190 | - // 9.1.0 及之前版本的外部 dieId 始终为 0,需要从 channel 中获取实际 dieId | ||
| 191 | CCU_CHK_RET(currKernel_->ApplyDieFromChannels()); | 189 | CCU_CHK_RET(currKernel_->ApplyDieFromChannels()); |
| 192 | } else { | 190 | } else { |
| 193 | - // 校验所有 channel 使用相同的 die,然后将资源占用从 die 0 迁移到指定 die | 191 | + // 按需创建的 ccu instance,校验所有 channel 使用相同的 die,然后将资源占用从 die 0 迁移到指定 die |
| 194 | CCU_CHK_RET(currKernel_->ValidateAndApplyDie(dieId)); | 192 | CCU_CHK_RET(currKernel_->ValidateAndApplyDie(dieId)); |
| 195 | } | 193 | } |
| 196 | CCU_CHK_RET(PrepareConstValueResources()); // 记录翻译过程所需常量并申请对应资源 | 194 | CCU_CHK_RET(PrepareConstValueResources()); // 记录翻译过程所需常量并申请对应资源 |
| @@ -219,7 +217,7 @@ CcuResult CcuKernelMgr::GetKernelResourceRequest( | |||
| 219 | std::unique_ptr<CcuKernel>& kernel_; | 217 | std::unique_ptr<CcuKernel>& kernel_; |
| 220 | } guard(currKernel_); | 218 | } guard(currKernel_); |
| 221 | 219 | ||
| 222 | - CCU_CHK_RET(BuildKernel(dieId, kernelFuncName, kernelFunc, kernelArgs, argNum)); | 220 | + CCU_CHK_RET(BuildKernel(dieId, kernelFuncName, kernelFunc, kernelArgs, argNum, nullptr)); |
| 223 | resReq = currKernel_->GetResourceRequest(); | 221 | resReq = currKernel_->GetResourceRequest(); |
| 224 | const uint32_t kernelInstrCount = currKernel_->GetInstrCount(); | 222 | const uint32_t kernelInstrCount = currKernel_->GetInstrCount(); |
| 225 | const uint32_t translatorInstrCount = CcuRepTranslator::GetInstrNum(devLogicId_); | 223 | const uint32_t translatorInstrCount = CcuRepTranslator::GetInstrNum(devLogicId_); |
| @@ -16,6 +16,7 @@ | |||
| 16 | 16 | ||
| 17 | 17 | ||
| 18 | 18 | ||
| 19 | + | ||
| 19 | 20 | ||
| 20 | 21 | ||
| 21 | 22 | ||
| @@ -34,12 +35,12 @@ public: | |||
| 34 | HcclResult Deinit(); | 35 | HcclResult Deinit(); |
| 35 | 36 | ||
| 36 | CcuResult Register( | 37 | CcuResult Register( |
| 37 | - CcuResPack& resPack, uint32_t dieId, const char* kernelFuncName, const void* kernelFunc, | 38 | + CcuResPack& resPack, const uint32_t dieId, const char* kernelFuncName, const void* kernelFunc, |
| 38 | - const void** kernelArgs, const uint32_t argNum, CcuKernelHandle& kernelHandle); | 39 | + const void** kernelArgs, const uint32_t argNum, CcuInstance* ccuIns, CcuKernelHandle& kernelHandle); |
| 39 | 40 | ||
| 40 | CcuResult GetKernelResourceRequest( | 41 | CcuResult GetKernelResourceRequest( |
| 41 | - uint32_t dieId, const char* kernelFuncName, const void* kernelFunc, const void** kernelArgs, uint32_t argNum, | 42 | + const uint32_t dieId, const char* kernelFuncName, const void* kernelFunc, const void** kernelArgs, |
| 42 | - CcuResReq& resReq, uint32_t& instrCount); | 43 | + const uint32_t argNum, CcuResReq& resReq, uint32_t& instrCount); |
| 43 | 44 | ||
| 44 | CcuResult Translate(const std::vector<CcuKernelHandle>& kernelHandles); | 45 | CcuResult Translate(const std::vector<CcuKernelHandle>& kernelHandles); |
| 45 | 46 | ||
| @@ -64,7 +65,8 @@ private: | |||
| 64 | 65 | ||
| 65 | private: | 66 | private: |
| 66 | CcuResult BuildKernel( | 67 | CcuResult BuildKernel( |
| 67 | - uint32_t dieId, const char* kernelFuncName, const void* kernelFunc, const void** kernelArgs, uint32_t argNum); | 68 | + uint32_t dieId, const char* kernelFuncName, const void* kernelFunc, const void** kernelArgs, uint32_t argNum, |
| 69 | + CcuInstance* ccuIns); | ||
| 68 | CcuResult PrepareConstValueResources(); | 70 | CcuResult PrepareConstValueResources(); |
| 69 | CcuResult AllocRes(CcuResPack& resPack); | 71 | CcuResult AllocRes(CcuResPack& resPack); |
| 70 | 72 | ||
| @@ -27,7 +27,7 @@ typedef enum { | |||
| 27 | } CcuInstanceType; | 27 | } CcuInstanceType; |
| 28 | 28 | ||
| 29 | /** | 29 | /** |
| 30 | - * @brief 创建ccu实例旧接口,兼容hccl旧版本 | 30 | + * @brief 按 CCU 实例类型创建 ccu 实例(兼容 hccl 旧版本)。 |
| 31 | * @param[in] insType CCU实例类型。 | 31 | * @param[in] insType CCU实例类型。 |
| 32 | * @param[out] ccuInsHandle 创建成功后返回的 CCU 实例句柄,不可为 nullptr。 | 32 | * @param[out] ccuInsHandle 创建成功后返回的 CCU 实例句柄,不可为 nullptr。 |
| 33 | * @return CcuResult。 | 33 | * @return CcuResult。 |
| @@ -16,6 +16,50 @@ | |||
| 16 | 16 | ||
| 17 | 17 | ||
| 18 | 18 | ||
| 19 | + | ||
| 20 | + | ||
| 21 | +static HcclResult CreateCcuInsByFixedResNum( | ||
| 22 | + const char* funcName, const std::string& commId, hccl::MyRank* myRank, CcuInsHandle& ccuInsHandle) | ||
| 23 | +{ | ||
| 24 | + const auto opExpansionMode = myRank->GetOpExpansionMode(); | ||
| 25 | + auto ccuInsType = hccl::OpExpansionModeToCcuInstanceType(opExpansionMode); | ||
| 26 | + if (ccuInsType == CcuInstanceType::CCU_UNUSED) { | ||
| 27 | + HCCL_WARNING( | ||
| 28 | + "[%s] failed to get ccu instance, commId[%s] op expansion mode[%u].", funcName, commId.c_str(), | ||
| 29 | + opExpansionMode); | ||
| 30 | + return HcclResult::HCCL_E_UNAVAIL; | ||
| 31 | + } | ||
| 32 | + | ||
| 33 | + CcuInsHandle newHandle = 0; | ||
| 34 | + auto ccuRet = CcuResult::CCU_SUCCESS; | ||
| 35 | + // CCU_MS 资源不足(CCU_E_UNAVAIL)时降级到 CCU_SCHED 重试一次;CCU_SCHED 不再继续降级 | ||
| 36 | + while (true) { | ||
| 37 | + ccuRet = HcommCcuInsCreateLegacy(ccuInsType, &newHandle); | ||
| 38 | + if (ccuRet != CcuResult::CCU_E_UNAVAIL || ccuInsType != CcuInstanceType::CCU_MS) { | ||
| 39 | + break; | ||
| 40 | + } | ||
| 41 | + HCCL_WARNING( | ||
| 42 | + "[%s] ccu instance resource unavailable for CCU_MS, fallback to CCU_SCHED, commId[%s].", funcName, | ||
| 43 | + commId.c_str()); | ||
| 44 | + ccuInsType = CcuInstanceType::CCU_SCHED; | ||
| 45 | + } | ||
| 46 | + if (ccuRet == CcuResult::CCU_E_UNAVAIL) { | ||
[LOW] CCU_MS 资源不足时不再回退到 CCU_SCHED 模式 问题分析:
建议:
在 ![]() ![]() | |||
| 47 | + HCCL_WARNING( | ||
| 48 | + "[%s] failed to create ccu instance, resources are unavailable, " | ||
| 49 | + "commId[%s] insType[%d] ret[%d].", | ||
| 50 | + funcName, commId.c_str(), ccuInsType, ccuRet); | ||
| 51 | + return static_cast<HcclResult>(ccuRet); | ||
| 52 | + } | ||
| 53 | + if (ccuRet != CcuResult::CCU_SUCCESS) { | ||
| 54 | + HCCL_ERROR( | ||
| 55 | + "[%s] failed to create ccu instance, commId[%s] insType[%d] ret[%d].", funcName, commId.c_str(), ccuInsType, | ||
| 56 | + ccuRet); | ||
| 57 | + return static_cast<HcclResult>(ccuRet); | ||
| 58 | + } | ||
| 59 | + myRank->SetCcuInstance(newHandle); | ||
| 60 | + ccuInsHandle = newHandle; | ||
| 61 | + return HcclResult::HCCL_SUCCESS; | ||
| 62 | +} | ||
| 19 | 63 | ||
| 20 | /** | 64 | /** |
| 21 | * @note 职责:集合通信的通信域CCU管理的C接口的C到C++适配 | 65 | * @note 职责:集合通信的通信域CCU管理的C接口的C到C++适配 |
| @@ -27,16 +71,15 @@ HcclResult HcclCommQueryCcuIns(HcclComm comm, CcuInsHandle* insHandles, uint32_t | |||
| 27 | HcclUs startut = TIME_NOW(); | 71 | HcclUs startut = TIME_NOW(); |
| 28 | 72 | ||
| 29 | CHK_PTR_NULL(comm); | 73 | CHK_PTR_NULL(comm); |
| 74 | + CHK_PTR_NULL(insHandles); | ||
| 75 | + CHK_PTR_NULL(insNum); | ||
| 30 | auto* hcclComm = static_cast<hccl::hcclComm*>(comm); | 76 | auto* hcclComm = static_cast<hccl::hcclComm*>(comm); |
| 31 | const auto& commId = hcclComm->GetIdentifier(); | 77 | const auto& commId = hcclComm->GetIdentifier(); |
| 32 | HCCL_INFO("[%s] CommId[%s] query ccu instance.", __func__, commId.c_str()); | 78 | HCCL_INFO("[%s] CommId[%s] query ccu instance.", __func__, commId.c_str()); |
| 33 | 79 | ||
| 34 | - CHK_PTR_NULL(insHandles); | ||
| 35 | - CHK_PTR_NULL(insNum); | ||
| 36 | - | ||
| 37 | // CCU不支持A5之前代际 | 80 | // CCU不支持A5之前代际 |
| 38 | if (!hcclComm->IsCommunicatorV2()) { | 81 | if (!hcclComm->IsCommunicatorV2()) { |
| 39 | - HCCL_WARNING("[%s] is not supported.", __func__); | 82 | + HCCL_RUN_WARNING("[%s] is not supported.", __func__); |
| 40 | return HcclResult::HCCL_E_NOT_SUPPORT; | 83 | return HcclResult::HCCL_E_NOT_SUPPORT; |
| 41 | } | 84 | } |
| 42 | 85 | ||
| @@ -45,14 +88,14 @@ HcclResult HcclCommQueryCcuIns(HcclComm comm, CcuInsHandle* insHandles, uint32_t | |||
| 45 | auto* myRank = collComm->GetMyRank(); | 88 | auto* myRank = collComm->GetMyRank(); |
| 46 | CHK_PTR_NULL(myRank); | 89 | CHK_PTR_NULL(myRank); |
| 47 | 90 | ||
| 48 | - // 非CCU通信域允许查询,不认为是错误 | 91 | + // 查询通信域自有的 ccuInsHandle_;为 0 时按 opExpansionMode 创建 |
| 49 | auto ccuInsHandle = myRank->GetCcuInstance(); | 92 | auto ccuInsHandle = myRank->GetCcuInstance(); |
| 50 | if (ccuInsHandle == 0) { | 93 | if (ccuInsHandle == 0) { |
| 51 | - auto opExpansionMode = myRank->GetOpExpansionMode(); | 94 | + auto ret = CreateCcuInsByFixedResNum(__func__, commId, myRank, ccuInsHandle); |
| 52 | - HCCL_WARNING( | 95 | + if (ret != HcclResult::HCCL_SUCCESS) { |
| 53 | - "[%s] failed to get ccu instance, commId[%s] op expansion mode[%u].", __func__, commId.c_str(), | 96 | + HCCL_WARNING("[%s] failed to get ccu instance, commId[%s] ret[%d]", __func__, commId.c_str(), ret); |
| 54 | - opExpansionMode); | 97 | + return ret; |
| 55 | - return HcclResult::HCCL_E_UNAVAIL; | 98 | + } |
| 56 | } | 99 | } |
| 57 | 100 | ||
| 58 | insHandles[0] = ccuInsHandle; | 101 | insHandles[0] = ccuInsHandle; |
| @@ -63,6 +106,45 @@ HcclResult HcclCommQueryCcuIns(HcclComm comm, CcuInsHandle* insHandles, uint32_t | |||
| 63 | return HcclResult::HCCL_SUCCESS; | 106 | return HcclResult::HCCL_SUCCESS; |
| 64 | } | 107 | } |
| 65 | 108 | ||
| 109 | +HcclResult HcclCommQueryAssignedCcuIns(HcclComm comm, CcuInsHandle* insHandles, uint32_t* insNum) | ||
| 110 | +{ | ||
| 111 | + EXCEPTION_HANDLE_BEGIN | ||
| 112 | + | ||
| 113 | + HcclUs startut = TIME_NOW(); | ||
| 114 | + | ||
| 115 | + CHK_PTR_NULL(comm); | ||
| 116 | + CHK_PTR_NULL(insHandles); | ||
| 117 | + CHK_PTR_NULL(insNum); | ||
| 118 | + auto* hcclComm = static_cast<hccl::hcclComm*>(comm); | ||
| 119 | + const auto& commId = hcclComm->GetIdentifier(); | ||
| 120 | + HCCL_INFO("[%s] CommId[%s] query assigned ccu instance.", __func__, commId.c_str()); | ||
| 121 | + | ||
| 122 | + // CCU不支持A5之前代际 | ||
| 123 | + if (!hcclComm->IsCommunicatorV2()) { | ||
| 124 | + HCCL_RUN_WARNING("[%s] is not supported.", __func__); | ||
| 125 | + return HcclResult::HCCL_E_NOT_SUPPORT; | ||
| 126 | + } | ||
| 127 | + | ||
| 128 | + auto* collComm = hcclComm->GetCollComm(); | ||
| 129 | + CHK_PTR_NULL(collComm); | ||
| 130 | + auto* myRank = collComm->GetMyRank(); | ||
| 131 | + CHK_PTR_NULL(myRank); | ||
| 132 | + | ||
| 133 | + // 查询绑定的 assignedCcuInsHandle_;未绑定时返回 UNAVAIL,不创建 | ||
| 134 | + auto assignedCcuInsHandle = myRank->GetAssignedCcuInstance(); | ||
| 135 | + if (assignedCcuInsHandle == 0) { | ||
| 136 | + HCCL_WARNING("[%s] assigned ccu instance not exist, commId[%s].", __func__, commId.c_str()); | ||
| 137 | + return HcclResult::HCCL_E_UNAVAIL; | ||
| 138 | + } | ||
| 139 | + | ||
| 140 | + insHandles[0] = assignedCcuInsHandle; | ||
| 141 | + *insNum = 1; | ||
| 142 | + HCCL_INFO("[%s] success, take time [%lld]us.", __func__, DURATION_US(TIME_NOW() - startut).count()); | ||
| 143 | + | ||
| 144 | + EXCEPTION_HANDLE_END | ||
| 145 | + return HcclResult::HCCL_SUCCESS; | ||
| 146 | +} | ||
| 147 | + | ||
| 66 | HcclResult HcclCommAssignCcuIns(HcclComm comm, CcuInsHandle insHandle) | 148 | HcclResult HcclCommAssignCcuIns(HcclComm comm, CcuInsHandle insHandle) |
| 67 | { | 149 | { |
| 68 | EXCEPTION_HANDLE_BEGIN | 150 | EXCEPTION_HANDLE_BEGIN |
| @@ -84,7 +166,7 @@ HcclResult HcclCommAssignCcuIns(HcclComm comm, CcuInsHandle insHandle) | |||
| 84 | } | 166 | } |
| 85 | 167 | ||
| 86 | if (!hcclComm->IsCommunicatorV2()) { | 168 | if (!hcclComm->IsCommunicatorV2()) { |
| 87 | - HCCL_WARNING("[%s] is not supported.", __func__); | 169 | + HCCL_RUN_WARNING("[%s] is not supported.", __func__); |
| 88 | return HcclResult::HCCL_E_NOT_SUPPORT; | 170 | return HcclResult::HCCL_E_NOT_SUPPORT; |
| 89 | } | 171 | } |
| 90 | 172 | ||
| @@ -98,10 +180,10 @@ HcclResult HcclCommAssignCcuIns(HcclComm comm, CcuInsHandle insHandle) | |||
| 98 | static std::mutex assignCcuInsMutex; | 180 | static std::mutex assignCcuInsMutex; |
[MEDIUM] static std::mutex assignCcuInsMutex 跨通信域序列化 Assign 调用 问题分析:
建议修复:
将锁改为 ![]() ![]() | |||
| 99 | std::lock_guard<std::mutex> lock(assignCcuInsMutex); | 181 | std::lock_guard<std::mutex> lock(assignCcuInsMutex); |
| 100 | 182 | ||
| 101 | - auto oldInsHandle = myRank->GetCcuInstance(); | 183 | + auto oldInsHandle = myRank->GetAssignedCcuInstance(); |
| 102 | if (oldInsHandle != 0) { | 184 | if (oldInsHandle != 0) { |
| 103 | HCCL_ERROR( | 185 | HCCL_ERROR( |
| 104 | - "[%s] failed, commId[%s] already has ccu instance[%llu], " | 186 | + "[%s] failed, commId[%s] already has assigned ccu instance[%llu], " |
| 105 | "new instance[%llu] will not be assigned.", | 187 | "new instance[%llu] will not be assigned.", |
| 106 | __func__, commId.c_str(), static_cast<unsigned long long>(oldInsHandle), | 188 | __func__, commId.c_str(), static_cast<unsigned long long>(oldInsHandle), |
| 107 | static_cast<unsigned long long>(insHandle)); | 189 | static_cast<unsigned long long>(insHandle)); |
| @@ -117,7 +199,7 @@ HcclResult HcclCommAssignCcuIns(HcclComm comm, CcuInsHandle insHandle) | |||
| 117 | return HcclResult::HCCL_E_NOT_FOUND; | 199 | return HcclResult::HCCL_E_NOT_FOUND; |
| 118 | } | 200 | } |
| 119 | 201 | ||
| 120 | - myRank->SetCcuInstance(insHandle); | 202 | + myRank->SetAssignedCcuInstance(insHandle); |
| 121 | } | 203 | } |
| 122 | 204 | ||
| 123 | HCCL_INFO( | 205 | HCCL_INFO( |
| @@ -121,8 +121,9 @@ MyRank::~MyRank() | |||
| 121 | ~ResourceCleanupGuard() noexcept | 121 | ~ResourceCleanupGuard() noexcept |
| 122 | { | 122 | { |
| 123 | myRank_.ccuInsHandle_ = 0; | 123 | myRank_.ccuInsHandle_ = 0; |
| 124 | + myRank_.assignedCcuInsHandle_ = 0; | ||
| 124 | 125 | ||
| 125 | - if (!myRank_.useCcuResStaticAlloc_ && myRank_.ccuDrvHandle_) { | 126 | + if (myRank_.ccuDrvHandle_) { |
| 126 | myRank_.ccuDrvHandle_ = nullptr; // 先减少引用计数,再尝试关闭 | 127 | myRank_.ccuDrvHandle_ = nullptr; // 先减少引用计数,再尝试关闭 |
| 127 | (void)CcuDeinitFeature(myRank_.devLogicId_); | 128 | (void)CcuDeinitFeature(myRank_.devLogicId_); |
| 128 | // 尝试关闭CCU功能,最后一个调用时会关闭CCU驱动 | 129 | // 尝试关闭CCU功能,最后一个调用时会关闭CCU驱动 |
| @@ -137,7 +138,7 @@ MyRank::~MyRank() | |||
| 137 | MyRank& myRank_; | 138 | MyRank& myRank_; |
| 138 | } cleanupGuard(*this); | 139 | } cleanupGuard(*this); |
| 139 | 140 | ||
| 140 | - if (ccuInsHandle_ != 0) { // 内部清理CCU资源,关闭CCU通道 | 141 | + if (ccuInsHandle_ != 0 || assignedCcuInsHandle_ != 0) { // 内部清理CCU资源,关闭CCU通道 |
| 141 | // 刷新并获取当前线程的 DeviceId | 142 | // 刷新并获取当前线程的 DeviceId |
| 142 | int32_t threadDevId = INVALID_INT; | 143 | int32_t threadDevId = INVALID_INT; |
| 143 | CHK_RET_NULL(HcclDeviceRefresh(threadDevId)); | 144 | CHK_RET_NULL(HcclDeviceRefresh(threadDevId)); |
| @@ -148,10 +149,13 @@ MyRank::~MyRank() | |||
| 148 | CHK_RET_NULL(hrtSetDevice(devLogicId_)); | 149 | CHK_RET_NULL(hrtSetDevice(devLogicId_)); |
| 149 | isDiffDevId = true; | 150 | isDiffDevId = true; |
| 150 | } | 151 | } |
| 151 | - // 销毁 CcuInstance | 152 | + // 销毁通信域自有的 ccuInstance(QueryCcuIns 创建) |
| 152 | - CcuResult ret = HcommCcuInsDestroy(ccuInsHandle_); | 153 | + if (ccuInsHandle_ != 0) { |
| 153 | - if (ret != CCU_SUCCESS) { | 154 | + CHK_PRT(static_cast<HcclResult>(HcommCcuInsDestroy(ccuInsHandle_))); |
| 154 | - HCCL_ERROR("[%s] HcommCcuInsDestroy failed, ret[%d]", __func__, ret); | 155 | + } |
| 156 | + // 销毁通过 Assign 绑定的 ccuInstance(所有权已转移给通信域) | ||
| 157 | + if (assignedCcuInsHandle_ != 0 && assignedCcuInsHandle_ != ccuInsHandle_) { | ||
| 158 | + CHK_PRT(static_cast<HcclResult>(HcommCcuInsDestroy(assignedCcuInsHandle_))); | ||
| 155 | } | 159 | } |
| 156 | // 切换回原来的 DeviceId | 160 | // 切换回原来的 DeviceId |
| 157 | if (isDiffDevId) { | 161 | if (isDiffDevId) { |
| @@ -208,70 +212,6 @@ HcclResult MyRank::UnregMemByTag(const std::string& tag) | |||
| 208 | return endpointMgr_->UnregMemByTag(tag); | 212 | return endpointMgr_->UnregMemByTag(tag); |
| 209 | } | 213 | } |
| 210 | 214 | ||
| 211 | -constexpr uint32_t DEFAULT_MODE = 0; | ||
| 212 | -constexpr uint32_t AICPU_TS_MODE = 2; | ||
| 213 | -constexpr uint32_t CCU_MS_MODE = 5; | ||
| 214 | -constexpr uint32_t CCU_SCHED_MODE = 6; | ||
| 215 | -inline CcuInstanceType OpExpansionModeToCcuInstanceType(uint32_t opExpansionMode) | ||
| 216 | -{ | ||
| 217 | - // 仅作数据类型转换,不做逻辑处理 | ||
| 218 | - if (opExpansionMode == CCU_SCHED_MODE) { | ||
| 219 | - return CcuInstanceType::CCU_SCHED; | ||
| 220 | - } | ||
| 221 | - | ||
| 222 | - if (opExpansionMode == CCU_MS_MODE) { | ||
| 223 | - return CcuInstanceType::CCU_MS; | ||
| 224 | - } | ||
| 225 | - | ||
| 226 | - return CcuInstanceType::CCU_UNUSED; | ||
| 227 | -} | ||
| 228 | - | ||
| 229 | -HcclResult MyRank::TryInitCcuInstanceLegacy() | ||
| 230 | -{ | ||
| 231 | - auto ccuInsType = OpExpansionModeToCcuInstanceType(opExpansionMode_); | ||
| 232 | - if (ccuInsType == CcuInstanceType::CCU_UNUSED) { | ||
| 233 | - ccuInsHandle_ = 0; | ||
| 234 | - return HcclResult::HCCL_SUCCESS; | ||
| 235 | - } | ||
| 236 | - | ||
| 237 | - auto ccuInitRet = HcommCcuInsCreateLegacy(ccuInsType, &ccuInsHandle_); | ||
| 238 | - // ccu驱动拉起失败,直接回退至aicpu ts | ||
| 239 | - if (ccuInitRet == CcuResult::CCU_E_DRV_BUSY) { | ||
| 240 | - opExpansionMode_ = AICPU_TS_MODE; | ||
| 241 | - ccuInsHandle_ = 0; | ||
| 242 | - HCCL_RUN_WARNING("[MyRank][%s] failed to init ccu driver, fallback to aicpu, rankId[%u].", __func__, rankId_); | ||
| 243 | - return HcclResult::HCCL_SUCCESS; | ||
| 244 | - } | ||
| 245 | - | ||
| 246 | - // ccu通信域数量过多,导致资源不足 | ||
| 247 | - if (CCU_CHK_RES_UNAVAIL(ccuInitRet)) { | ||
| 248 | - // 如果是ccu ms模式,回退至ccu调度模式重试 | ||
| 249 | - // 复用原有的ccuResContainer,回退到ccu sched时不需要重复拉起ccu驱动 | ||
| 250 | - if (opExpansionMode_ == CCU_MS_MODE) { | ||
| 251 | - opExpansionMode_ = CCU_SCHED_MODE; | ||
| 252 | - CHK_RET(TryInitCcuInstanceLegacy()); // 至多递归一次 | ||
| 253 | - return HcclResult::HCCL_SUCCESS; | ||
| 254 | - } | ||
| 255 | - | ||
| 256 | - // 其余模式资源不足回退至aicpu ts | ||
| 257 | - opExpansionMode_ = AICPU_TS_MODE; | ||
| 258 | - ccuInsHandle_ = 0; | ||
| 259 | - HCCL_RUN_WARNING( | ||
| 260 | - "[MyRank][%s] ccu resources are unavailable, fallback to aicpu, rankId[%u].", __func__, rankId_); | ||
| 261 | - return HcclResult::HCCL_SUCCESS; | ||
| 262 | - } | ||
| 263 | - | ||
| 264 | - // 预期外返回值属于错误 | ||
| 265 | - if (ccuInitRet != CcuResult::CCU_SUCCESS) { | ||
| 266 | - HCCL_ERROR("[%s] failed, ret[%d] is not expected.", __func__, ccuInitRet); | ||
| 267 | - ccuInsHandle_ = 0; | ||
| 268 | - return static_cast<HcclResult>(ccuInitRet); | ||
| 269 | - } | ||
| 270 | - | ||
| 271 | - // ccu资源申请成功 | ||
| 272 | - return HcclResult::HCCL_SUCCESS; | ||
| 273 | -} | ||
| 274 | - | ||
| 275 | HcclResult MyRank::ReserveCcuMsCommOrFallback() | 215 | HcclResult MyRank::ReserveCcuMsCommOrFallback() |
| 276 | { | 216 | { |
| 277 | if (opExpansionMode_ != CCU_MS_MODE) { | 217 | if (opExpansionMode_ != CCU_MS_MODE) { |
| @@ -310,7 +250,6 @@ void MyRank::ReconcileCcuMsCommReservation(HcclResult initRet) | |||
| 310 | 250 | ||
| 311 | HcclResult MyRank::TryInitCcuInstanceOnDemand() | 251 | HcclResult MyRank::TryInitCcuInstanceOnDemand() |
| 312 | { | 252 | { |
| 313 | - // 以下为ccu新接口流程 | ||
| 314 | auto ccuInsType = OpExpansionModeToCcuInstanceType(opExpansionMode_); | 253 | auto ccuInsType = OpExpansionModeToCcuInstanceType(opExpansionMode_); |
| 315 | if (ccuInsType == CcuInstanceType::CCU_UNUSED) { | 254 | if (ccuInsType == CcuInstanceType::CCU_UNUSED) { |
| 316 | ccuInsHandle_ = 0; | 255 | ccuInsHandle_ = 0; |
| @@ -358,15 +297,9 @@ HcclResult MyRank::TryInitCcuInstance() | |||
| 358 | { | 297 | { |
| 359 | CHK_RET(ReserveCcuMsCommOrFallback()); | 298 | CHK_RET(ReserveCcuMsCommOrFallback()); |
| 360 | 299 | ||
| 361 | - HcclResult ret = HCCL_SUCCESS; | 300 | + // ccu instance 不在 init 时创建,由 QueryCcuIns 创建或由 Assign 显式绑定; |
| 362 | - if (useCcuResStaticAlloc_) { | 301 | + // 此处仅拉起 ccu 驱动。 |
| 363 | - HCCL_RUN_INFO( | 302 | + HcclResult ret = TryInitCcuInstanceOnDemand(); |
| 364 | - "[MyRank][%s] HCCL version does not support CCU on-demand resource allocation, use legacy allocation.", | ||
| 365 | - __func__); | ||
| 366 | - ret = TryInitCcuInstanceLegacy(); | ||
| 367 | - } else { | ||
| 368 | - ret = TryInitCcuInstanceOnDemand(); | ||
| 369 | - } | ||
| 370 | ReconcileCcuMsCommReservation(ret); | 303 | ReconcileCcuMsCommReservation(ret); |
| 371 | return ret; | 304 | return ret; |
| 372 | } | 305 | } |
| @@ -389,28 +322,11 @@ HcclResult MyRank::GetDevicePortInternal(uint32_t rank, uint32_t* devPort, Endpo | |||
| 389 | return HCCL_SUCCESS; | 322 | return HCCL_SUCCESS; |
| 390 | } | 323 | } |
| 391 | 324 | ||
| 392 | -inline HcclResult GetHcclVersion(int& hcclVersion) | ||
| 393 | -{ | ||
| 394 | - char hcclPkgName[] = "hccl"; | ||
| 395 | - aclError aclRet = aclsysGetVersionNum(hcclPkgName, &hcclVersion); | ||
| 396 | - CHK_PRT_RET( | ||
| 397 | - aclRet != ACL_SUCCESS, HCCL_ERROR("[GetHcclVersion] aclsysGetVersionNum failed, aclRet[%d].", aclRet), | ||
| 398 | - HCCL_E_INTERNAL); | ||
| 399 | - HCCL_RUN_INFO("[GetHcclVersion] hccl version is %d.", hcclVersion); | ||
| 400 | - return HCCL_SUCCESS; | ||
| 401 | -} | ||
| 402 | - | ||
| 403 | -constexpr int MAX_HCCL_VERSION_USING_CCU_RES_STATIC_ALLOC = 90100000; | ||
| 404 | HcclResult MyRank::Init(HcclMem cclBuffer, const uint32_t opExpansionMode, uint32_t rankNum) | 325 | HcclResult MyRank::Init(HcclMem cclBuffer, const uint32_t opExpansionMode, uint32_t rankNum) |
| 405 | { | 326 | { |
| 406 | // EXCEPTION_HANDLE_BEGIN | 327 | // EXCEPTION_HANDLE_BEGIN |
| 407 | CHK_RET(hrtGetDevice(&devLogicId_)); | 328 | CHK_RET(hrtGetDevice(&devLogicId_)); |
| 408 | 329 | ||
| 409 | - // 获取hccl版本 | ||
| 410 | - int hcclVersion = 0; | ||
| 411 | - CHK_RET(GetHcclVersion(hcclVersion)); | ||
| 412 | - useCcuResStaticAlloc_ = hcclVersion <= MAX_HCCL_VERSION_USING_CCU_RES_STATIC_ALLOC; | ||
| 413 | - | ||
| 414 | // ns recovery processor初始化 | 330 | // ns recovery processor初始化 |
| 415 | EXCEPTION_CATCH(nsRecoveryProcessor_ = std::make_unique<NsRecoveryProcessor>(), return HCCL_E_PTR); | 331 | EXCEPTION_CATCH(nsRecoveryProcessor_ = std::make_unique<NsRecoveryProcessor>(), return HCCL_E_PTR); |
| 416 | 332 | ||
| @@ -43,10 +43,28 @@ | |||
| 43 | 43 | ||
| 44 | 44 | ||
| 45 | 45 | ||
| 46 | + | ||
| 46 | 47 | ||
| 47 | 48 | ||
| 48 | namespace hccl { | 49 | namespace hccl { |
| 49 | 50 | ||
| 51 | +constexpr uint32_t DEFAULT_MODE = 0; | ||
| 52 | +constexpr uint32_t AICPU_TS_MODE = 2; | ||
| 53 | +constexpr uint32_t CCU_MS_MODE = 5; | ||
| 54 | +constexpr uint32_t CCU_SCHED_MODE = 6; | ||
L 为啥和这个不一致 HcclOpExpansionMode ![]() ![]() | |||
| 55 | +// opExpansionMode 到 CcuInstanceType 的映射,仅作数据类型转换,不做逻辑处理。 | ||
| 56 | +// 供 MyRank 内部及外部适配层(如 coll_comm_ccu_c_adpt.cc)共用。 | ||
| 57 | +inline CcuInstanceType OpExpansionModeToCcuInstanceType(uint32_t opExpansionMode) | ||
| 58 | +{ | ||
| 59 | + if (opExpansionMode == CCU_SCHED_MODE) { | ||
| 60 | + return CcuInstanceType::CCU_SCHED; | ||
| 61 | + } | ||
| 62 | + if (opExpansionMode == CCU_MS_MODE) { | ||
| 63 | + return CcuInstanceType::CCU_MS; | ||
| 64 | + } | ||
| 65 | + return CcuInstanceType::CCU_UNUSED; | ||
| 66 | +} | ||
| 67 | + | ||
| 50 | /** | 68 | /** |
| 51 | * @note 职责:管理当前通信域下本Rank的信息和通信资源 | 69 | * @note 职责:管理当前通信域下本Rank的信息和通信资源 |
| 52 | */ | 70 | */ |
| @@ -67,6 +85,8 @@ public: | |||
| 67 | uint32_t GetOpExpansionMode() { return opExpansionMode_; } | 85 | uint32_t GetOpExpansionMode() { return opExpansionMode_; } |
| 68 | CcuInsHandle GetCcuInstance() const { return ccuInsHandle_; } | 86 | CcuInsHandle GetCcuInstance() const { return ccuInsHandle_; } |
| 69 | void SetCcuInstance(CcuInsHandle ccuInsHandle) { ccuInsHandle_ = ccuInsHandle; } | 87 | void SetCcuInstance(CcuInsHandle ccuInsHandle) { ccuInsHandle_ = ccuInsHandle; } |
| 88 | + CcuInsHandle GetAssignedCcuInstance() const { return assignedCcuInsHandle_; } | ||
| 89 | + void SetAssignedCcuInstance(CcuInsHandle ccuInsHandle) { assignedCcuInsHandle_ = ccuInsHandle; } | ||
| 70 | 90 | ||
| 71 | CollCommConfigConsistency& GetCollCommConfigConsistency(); | 91 | CollCommConfigConsistency& GetCollCommConfigConsistency(); |
| 72 | 92 | ||
| @@ -150,7 +170,6 @@ private: | |||
| 150 | const EndpointDesc& remoteEndpointDesc, uint32_t& listenPort, HcommChannelDesc& hcommDesc); | 170 | const EndpointDesc& remoteEndpointDesc, uint32_t& listenPort, HcommChannelDesc& hcommDesc); |
| 151 | HcclResult GetLocalTlsStatus(Hccl::TlsStatus& tlsStatus) const; | 171 | HcclResult GetLocalTlsStatus(Hccl::TlsStatus& tlsStatus) const; |
| 152 | HcclResult RegisterCommMemsToEndpoint(EndpointHandle epHandle); | 172 | HcclResult RegisterCommMemsToEndpoint(EndpointHandle epHandle); |
| 153 | - HcclResult TryInitCcuInstanceLegacy(); | ||
| 154 | HcclResult TryInitCcuInstance(); | 173 | HcclResult TryInitCcuInstance(); |
| 155 | HcclResult ReserveCcuMsCommOrFallback(); | 174 | HcclResult ReserveCcuMsCommOrFallback(); |
| 156 | HcclResult TryInitCcuInstanceOnDemand(); | 175 | HcclResult TryInitCcuInstanceOnDemand(); |
| @@ -172,7 +191,8 @@ private: | |||
| 172 | std::unique_ptr<CommMems> commMems_{nullptr}; | 191 | std::unique_ptr<CommMems> commMems_{nullptr}; |
| 173 | std::unique_ptr<EngineCtxs> engineCtxs_{nullptr}; | 192 | std::unique_ptr<EngineCtxs> engineCtxs_{nullptr}; |
| 174 | 193 | ||
| 175 | - CcuInsHandle ccuInsHandle_{0}; | 194 | + CcuInsHandle ccuInsHandle_{0}; // 按固定量CCU资源方式创建的 ccuInsHandle(HcclCommQueryCcuIns)(兼容旧版本) |
| 195 | + CcuInsHandle assignedCcuInsHandle_{0}; // 按需申请CCU资源方式绑定的 ccuInsHandle(HcommCcuInsCreate + Assign) | ||
| 176 | 196 | ||
| 177 | ManagerCallbacks callbacks_; | 197 | ManagerCallbacks callbacks_; |
| 178 | 198 | ||
| @@ -192,7 +212,6 @@ private: | |||
| 192 | CollCommConfigConsistency collCommConfigConsistency_; | 212 | CollCommConfigConsistency collCommConfigConsistency_; |
| 193 | ExchangeInfoMgr exchangeInfoMgr_; | 213 | ExchangeInfoMgr exchangeInfoMgr_; |
| 194 | std::shared_ptr<hcomm::CcuDrvHandle> ccuDrvHandle_{}; | 214 | std::shared_ptr<hcomm::CcuDrvHandle> ccuDrvHandle_{}; |
| 195 | - bool useCcuResStaticAlloc_{false}; // HCCL版本不支持CCU资源按需申请 | ||
| 196 | bool ccuMsCommReserved_{false}; | 215 | bool ccuMsCommReserved_{false}; |
| 197 | Hccl::HcclMainboardId mainBoardType_{Hccl::HcclMainboardId::MAINBOARD_OTHERS}; | 216 | Hccl::HcclMainboardId mainBoardType_{Hccl::HcclMainboardId::MAINBOARD_OTHERS}; |
| 198 | }; | 217 | }; |
| @@ -26,6 +26,8 @@ | |||
| 26 | 26 | ||
| 27 | 27 | ||
| 28 | 28 | ||
| 29 | + | ||
| 30 | + | ||
| 29 | 31 | ||
| 30 | using namespace hccl; | 32 | using namespace hccl; |
| 31 | using namespace hcomm; | 33 | using namespace hcomm; |
| @@ -39,7 +41,8 @@ protected: | |||
| 39 | } | 41 | } |
| 40 | 42 | ||
| 41 | HcclComm BuildComm( | 43 | HcclComm BuildComm( |
| 42 | - bool isCommunicatorV2 = true, bool hasCollComm = true, bool hasMyRank = true, CcuInsHandle currentInsHandle = 0) | 44 | + bool isCommunicatorV2 = true, bool hasCollComm = true, bool hasMyRank = true, |
| 45 | + CcuInsHandle currentAssignedInsHandle = 0) | ||
| 43 | { | 46 | { |
| 44 | comm_ = std::make_unique<hcclComm>(0, 0, "assign_ccu_ut"); | 47 | comm_ = std::make_unique<hcclComm>(0, 0, "assign_ccu_ut"); |
| 45 | comm_->devType_ = isCommunicatorV2 ? DevType::DEV_TYPE_950 : DevType::DEV_TYPE_COUNT; | 48 | comm_->devType_ = isCommunicatorV2 ? DevType::DEV_TYPE_950 : DevType::DEV_TYPE_COUNT; |
| @@ -56,7 +59,8 @@ protected: | |||
| 56 | } | 59 | } |
| 57 | 60 | ||
| 58 | myRank_ = std::make_shared<MyRank>(nullptr, 0, comm_->collComm_->config_, ManagerCallbacks{}, nullptr, nullptr); | 61 | myRank_ = std::make_shared<MyRank>(nullptr, 0, comm_->collComm_->config_, ManagerCallbacks{}, nullptr, nullptr); |
| 59 | - myRank_->ccuInsHandle_ = currentInsHandle; | 62 | + // 语义变更:HcclCommAssignCcuIns 改写 assignedCcuInsHandle_(不再写 ccuInsHandle_) |
| 63 | + myRank_->assignedCcuInsHandle_ = currentAssignedInsHandle; | ||
| 60 | comm_->collComm_->myRank_ = myRank_; | 64 | comm_->collComm_->myRank_ = myRank_; |
| 61 | return static_cast<HcclComm>(comm_.get()); | 65 | return static_cast<HcclComm>(comm_.get()); |
| 62 | } | 66 | } |
| @@ -82,7 +86,8 @@ TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommAssignCcuIns_When_CommHasNoCcuInstan | |||
| 82 | HcclResult ret = HcclCommAssignCcuIns(comm, insHandle); | 86 | HcclResult ret = HcclCommAssignCcuIns(comm, insHandle); |
| 83 | 87 | ||
| 84 | EXPECT_EQ(ret, HCCL_SUCCESS); | 88 | EXPECT_EQ(ret, HCCL_SUCCESS); |
| 85 | - EXPECT_EQ(myRank_->GetCcuInstance(), insHandle); | 89 | + EXPECT_EQ(myRank_->GetAssignedCcuInstance(), insHandle); |
| 90 | + EXPECT_EQ(myRank_->GetCcuInstance(), 0); // ccuInsHandle_ 不被 Assign 改写 | ||
| 86 | } | 91 | } |
| 87 | 92 | ||
| 88 | TEST_F( | 93 | TEST_F( |
| @@ -97,7 +102,7 @@ TEST_F( | |||
| 97 | HcclResult ret = HcclCommAssignCcuIns(comm, insHandle); | 102 | HcclResult ret = HcclCommAssignCcuIns(comm, insHandle); |
| 98 | 103 | ||
| 99 | EXPECT_EQ(ret, HCCL_SUCCESS); | 104 | EXPECT_EQ(ret, HCCL_SUCCESS); |
| 100 | - EXPECT_EQ(myRank_->GetCcuInstance(), insHandle); | 105 | + EXPECT_EQ(myRank_->GetAssignedCcuInstance(), insHandle); |
| 101 | myRank_.reset(); | 106 | myRank_.reset(); |
| 102 | comm_.reset(); | 107 | comm_.reset(); |
| 103 | EXPECT_TRUE(myRankWeak.expired()); | 108 | EXPECT_TRUE(myRankWeak.expired()); |
| @@ -119,7 +124,7 @@ TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommAssignCcuIns_When_InsHandleIsZero_Ex | |||
| 119 | HcclResult ret = HcclCommAssignCcuIns(comm, 0); | 124 | HcclResult ret = HcclCommAssignCcuIns(comm, 0); |
| 120 | 125 | ||
| 121 | EXPECT_EQ(ret, HCCL_E_PARA); | 126 | EXPECT_EQ(ret, HCCL_E_PARA); |
| 122 | - EXPECT_EQ(myRank_->GetCcuInstance(), 0); | 127 | + EXPECT_EQ(myRank_->GetAssignedCcuInstance(), 0); |
| 123 | } | 128 | } |
| 124 | 129 | ||
| 125 | TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommAssignCcuIns_When_CommIsNotV2_Expect_ReturnIsHCCL_E_NOT_SUPPORT) | 130 | TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommAssignCcuIns_When_CommIsNotV2_Expect_ReturnIsHCCL_E_NOT_SUPPORT) |
| @@ -130,7 +135,7 @@ TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommAssignCcuIns_When_CommIsNotV2_Expect | |||
| 130 | HcclResult ret = HcclCommAssignCcuIns(comm, insHandle); | 135 | HcclResult ret = HcclCommAssignCcuIns(comm, insHandle); |
| 131 | 136 | ||
| 132 | EXPECT_EQ(ret, HCCL_E_NOT_SUPPORT); | 137 | EXPECT_EQ(ret, HCCL_E_NOT_SUPPORT); |
| 133 | - EXPECT_EQ(myRank_->GetCcuInstance(), 0); | 138 | + EXPECT_EQ(myRank_->GetAssignedCcuInstance(), 0); |
| 134 | } | 139 | } |
| 135 | 140 | ||
| 136 | TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommAssignCcuIns_When_CollCommIsNull_Expect_ReturnIsHCCL_E_PTR) | 141 | TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommAssignCcuIns_When_CollCommIsNull_Expect_ReturnIsHCCL_E_PTR) |
| @@ -162,7 +167,7 @@ TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommAssignCcuIns_When_InsHandleNotFound_ | |||
| 162 | HcclResult ret = HcclCommAssignCcuIns(comm, insHandle); | 167 | HcclResult ret = HcclCommAssignCcuIns(comm, insHandle); |
| 163 | 168 | ||
| 164 | EXPECT_EQ(ret, HCCL_E_NOT_FOUND); | 169 | EXPECT_EQ(ret, HCCL_E_NOT_FOUND); |
| 165 | - EXPECT_EQ(myRank_->GetCcuInstance(), 0); | 170 | + EXPECT_EQ(myRank_->GetAssignedCcuInstance(), 0); |
| 166 | } | 171 | } |
| 167 | 172 | ||
| 168 | TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommAssignCcuIns_When_CommHasDifferentCcuInstance_Expect_ReturnIsHCCL_E_PARA) | 173 | TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommAssignCcuIns_When_CommHasDifferentCcuInstance_Expect_ReturnIsHCCL_E_PARA) |
| @@ -175,7 +180,7 @@ TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommAssignCcuIns_When_CommHasDifferentCc | |||
| 175 | HcclResult ret = HcclCommAssignCcuIns(comm, newInsHandle); | 180 | HcclResult ret = HcclCommAssignCcuIns(comm, newInsHandle); |
| 176 | 181 | ||
| 177 | EXPECT_EQ(ret, HCCL_E_PARA); | 182 | EXPECT_EQ(ret, HCCL_E_PARA); |
| 178 | - EXPECT_EQ(myRank_->GetCcuInstance(), oldInsHandle); | 183 | + EXPECT_EQ(myRank_->GetAssignedCcuInstance(), oldInsHandle); |
| 179 | } | 184 | } |
| 180 | 185 | ||
| 181 | TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommAssignCcuIns_When_CommHasSameCcuInstance_Expect_ReturnIsHCCL_E_PARA) | 186 | TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommAssignCcuIns_When_CommHasSameCcuInstance_Expect_ReturnIsHCCL_E_PARA) |
| @@ -187,7 +192,7 @@ TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommAssignCcuIns_When_CommHasSameCcuInst | |||
| 187 | HcclResult ret = HcclCommAssignCcuIns(comm, insHandle); | 192 | HcclResult ret = HcclCommAssignCcuIns(comm, insHandle); |
| 188 | 193 | ||
| 189 | EXPECT_EQ(ret, HCCL_E_PARA); | 194 | EXPECT_EQ(ret, HCCL_E_PARA); |
| 190 | - EXPECT_EQ(myRank_->GetCcuInstance(), insHandle); | 195 | + EXPECT_EQ(myRank_->GetAssignedCcuInstance(), insHandle); |
| 191 | } | 196 | } |
| 192 | 197 | ||
| 193 | TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommAssignCcuIns_When_InternalExceptionOccurs_Expect_ReturnIsHCCL_E_INTERNAL) | 198 | TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommAssignCcuIns_When_InternalExceptionOccurs_Expect_ReturnIsHCCL_E_INTERNAL) |
| @@ -200,7 +205,7 @@ TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommAssignCcuIns_When_InternalExceptionO | |||
| 200 | HcclResult ret = HcclCommAssignCcuIns(comm, insHandle); | 205 | HcclResult ret = HcclCommAssignCcuIns(comm, insHandle); |
| 201 | 206 | ||
| 202 | EXPECT_EQ(ret, HCCL_E_INTERNAL); | 207 | EXPECT_EQ(ret, HCCL_E_INTERNAL); |
| 203 | - EXPECT_EQ(myRank_->GetCcuInstance(), 0); | 208 | + EXPECT_EQ(myRank_->GetAssignedCcuInstance(), 0); |
| 204 | } | 209 | } |
| 205 | 210 | ||
| 206 | TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommAssignCcuIns_When_TwoThreadsAssignConcurrently_Expect_OnlyOneSucceeds) | 211 | TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommAssignCcuIns_When_TwoThreadsAssignConcurrently_Expect_OnlyOneSucceeds) |
| @@ -235,5 +240,287 @@ TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommAssignCcuIns_When_TwoThreadsAssignCo | |||
| 235 | const bool firstSucceeded = firstRet == HCCL_SUCCESS && secondRet == HCCL_E_PARA; | 240 | const bool firstSucceeded = firstRet == HCCL_SUCCESS && secondRet == HCCL_E_PARA; |
| 236 | const bool secondSucceeded = firstRet == HCCL_E_PARA && secondRet == HCCL_SUCCESS; | 241 | const bool secondSucceeded = firstRet == HCCL_E_PARA && secondRet == HCCL_SUCCESS; |
| 237 | EXPECT_TRUE(firstSucceeded || secondSucceeded); | 242 | EXPECT_TRUE(firstSucceeded || secondSucceeded); |
| 238 | - EXPECT_EQ(myRank_->GetCcuInstance(), firstSucceeded ? firstInsHandle : secondInsHandle); | 243 | + EXPECT_EQ(myRank_->GetAssignedCcuInstance(), firstSucceeded ? firstInsHandle : secondInsHandle); |
| 244 | +} | ||
| 245 | + | ||
| 246 | +// ============================================================================================ | ||
| 247 | +// HcclCommQueryCcuIns 单元测试(阶段2新语义:查/建通信域自有的 ccuInsHandle_,旧方式) | ||
| 248 | +// 对应业务代码:src/coll_communicator_mgr/api_c_adpt/coll_comm_ccu_c_adpt.cc | ||
| 249 | +// ============================================================================================ | ||
| 250 | + | ||
| 251 | +TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommQueryCcuIns_When_SelfHandleExists_Expect_ReturnSuccessAndNoCreate) | ||
| 252 | +{ | ||
| 253 | + constexpr CcuInsHandle selfHandle = 0xAB; | ||
| 254 | + HcclComm comm = BuildComm(true, true, true, 0); | ||
| 255 | + myRank_->ccuInsHandle_ = selfHandle; // 通信域自有 handle 已存在 | ||
| 256 | + // 按需创建不应被调用 | ||
| 257 | + MOCKER(HcommCcuInsCreateLegacy).expects(never()); | ||
| 258 | + | ||
| 259 | + CcuInsHandle insHandles[1] = {0}; | ||
| 260 | + uint32_t insNum = 0; | ||
| 261 | + HcclResult ret = HcclCommQueryCcuIns(comm, insHandles, &insNum); | ||
| 262 | + | ||
| 263 | + EXPECT_EQ(ret, HCCL_SUCCESS); | ||
| 264 | + EXPECT_EQ(insHandles[0], selfHandle); | ||
| 265 | + EXPECT_EQ(insNum, 1U); | ||
| 266 | + EXPECT_EQ(myRank_->GetCcuInstance(), selfHandle); // 未被改写 | ||
| 267 | +} | ||
| 268 | + | ||
| 269 | +TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommQueryCcuIns_When_SelfHandleZero_And_ModeValid_Expect_CreateAndWriteback) | ||
| 270 | +{ | ||
| 271 | + CcuInsHandle createdHandle = 0xCD; | ||
| 272 | + HcclComm comm = BuildComm(true, true, true, 0); | ||
| 273 | + myRank_->ccuInsHandle_ = 0; | ||
| 274 | + myRank_->opExpansionMode_ = 6; // CCU_SCHED_MODE 映射为 CCU_SCHED | ||
| 275 | + CcuInsHandle insHandles[1] = {0}; | ||
| 276 | + uint32_t insNum = 0; | ||
| 277 | + // mock 按需创建:通过 outBoundP 把 createdHandle 拷贝到出参指针 *ccuInsHandle,并返回成功 | ||
| 278 | + MOCKER(HcommCcuInsCreateLegacy) | ||
| 279 | + .expects(once()) | ||
| 280 | + .with(mockcpp::any(), mockcpp::outBoundP(&createdHandle, sizeof(CcuInsHandle))) | ||
| 281 | + .will(returnValue(CcuResult::CCU_SUCCESS)); | ||
| 282 | + | ||
| 283 | + HcclResult ret = HcclCommQueryCcuIns(comm, insHandles, &insNum); | ||
| 284 | + EXPECT_EQ(ret, HCCL_SUCCESS); | ||
| 285 | + EXPECT_EQ(insHandles[0], createdHandle); | ||
| 286 | + EXPECT_EQ(insNum, 1U); | ||
| 287 | + EXPECT_EQ(myRank_->GetCcuInstance(), createdHandle); // 已回写 | ||
| 288 | +} | ||
| 289 | + | ||
| 290 | +TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommQueryCcuIns_When_SelfHandleZero_And_ModeUnused_Expect_ReturnUnavail) | ||
| 291 | +{ | ||
| 292 | + HcclComm comm = BuildComm(true, true, true, 0); | ||
| 293 | + myRank_->ccuInsHandle_ = 0; | ||
| 294 | + myRank_->opExpansionMode_ = 0; // DEFAULT_MODE 映射为 CCU_UNUSED | ||
| 295 | + MOCKER(HcommCcuInsCreateLegacy).expects(never()); | ||
| 296 | + | ||
| 297 | + CcuInsHandle insHandles[1] = {0}; | ||
| 298 | + uint32_t insNum = 0; | ||
| 299 | + HcclResult ret = HcclCommQueryCcuIns(comm, insHandles, &insNum); | ||
| 300 | + | ||
| 301 | + EXPECT_EQ(ret, HCCL_E_UNAVAIL); | ||
| 302 | + EXPECT_EQ(insHandles[0], static_cast<CcuInsHandle>(0)); // 出参未修改 | ||
| 303 | + EXPECT_EQ(insNum, 0U); | ||
| 304 | + EXPECT_EQ(myRank_->GetCcuInstance(), 0); // 未创建 | ||
| 305 | +} | ||
| 306 | + | ||
| 307 | +TEST_F( | ||
| 308 | + HcclCommAssignCcuInsTest, | ||
| 309 | + Ut_HcclCommQueryCcuIns_When_SelfHandleZero_And_CreateFails_Expect_ReturnErrorAndNoWriteback) | ||
| 310 | +{ | ||
| 311 | + HcclComm comm = BuildComm(true, true, true, 0); | ||
| 312 | + myRank_->ccuInsHandle_ = 0; | ||
| 313 | + myRank_->opExpansionMode_ = 5; // CCU_MS_MODE 映射为 CCU_MS | ||
| 314 | + MOCKER(HcommCcuInsCreateLegacy).expects(once()).will(returnValue(CcuResult::CCU_E_DRV_BUSY)); | ||
| 315 | + | ||
| 316 | + CcuInsHandle insHandles[1] = {0}; | ||
| 317 | + uint32_t insNum = 0; | ||
| 318 | + HcclResult ret = HcclCommQueryCcuIns(comm, insHandles, &insNum); | ||
| 319 | + | ||
| 320 | + EXPECT_EQ(ret, static_cast<HcclResult>(CcuResult::CCU_E_DRV_BUSY)); | ||
| 321 | + EXPECT_EQ(myRank_->GetCcuInstance(), 0); // 失败不回写 | ||
| 322 | +} | ||
| 323 | + | ||
| 324 | +// CCU_MS 资源不足时降级到 CCU_SCHED 重试,CCU_SCHED 创建成功 → 返回成功并回写降级后创建的 handle | ||
| 325 | +TEST_F( | ||
| 326 | + HcclCommAssignCcuInsTest, Ut_HcclCommQueryCcuIns_When_CcuMsUnavail_And_FallbackCcuSchedSuccess_Expect_ReturnSuccess) | ||
| 327 | +{ | ||
| 328 | + CcuInsHandle createdHandle = 0xCD; | ||
| 329 | + HcclComm comm = BuildComm(true, true, true, 0); | ||
| 330 | + myRank_->ccuInsHandle_ = 0; | ||
| 331 | + myRank_->opExpansionMode_ = 5; // CCU_MS_MODE 映射为 CCU_MS | ||
| 332 | + // 首次 CCU_MS 返回资源不足,降级到 CCU_SCHED 重试一次成功。 | ||
| 333 | + // 用 eq(CCU_MS)/eq(CCU_SCHED) 区分入参,使 mockcpp 按参数精确匹配两次调用 | ||
| 334 | + // (多次 expects(once()) 无 with 时会误匹配首条,导致第二次调用报错) | ||
| 335 | + MOCKER(HcommCcuInsCreateLegacy) | ||
| 336 | + .expects(once()) | ||
| 337 | + .with(mockcpp::eq(CcuInstanceType::CCU_MS), mockcpp::any()) | ||
| 338 | + .will(returnValue(CcuResult::CCU_E_UNAVAIL)); | ||
| 339 | + MOCKER(HcommCcuInsCreateLegacy) | ||
| 340 | + .expects(once()) | ||
| 341 | + .with(mockcpp::eq(CcuInstanceType::CCU_SCHED), mockcpp::outBoundP(&createdHandle, sizeof(CcuInsHandle))) | ||
| 342 | + .will(returnValue(CcuResult::CCU_SUCCESS)); | ||
| 343 | + | ||
| 344 | + CcuInsHandle insHandles[1] = {0}; | ||
| 345 | + uint32_t insNum = 0; | ||
| 346 | + HcclResult ret = HcclCommQueryCcuIns(comm, insHandles, &insNum); | ||
| 347 | + | ||
| 348 | + EXPECT_EQ(ret, HCCL_SUCCESS); | ||
| 349 | + EXPECT_EQ(insHandles[0], createdHandle); | ||
| 350 | + EXPECT_EQ(insNum, 1U); | ||
| 351 | + EXPECT_EQ(myRank_->GetCcuInstance(), createdHandle); // 降级后创建的 handle 已回写 | ||
| 352 | +} | ||
| 353 | + | ||
| 354 | +// CCU_MS 资源不足降级到 CCU_SCHED 仍资源不足 → 返回 HCCL_E_UNAVAIL,不创建、不回写 | ||
| 355 | +TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommQueryCcuIns_When_CcuMsAndSchedBothUnavail_Expect_ReturnUnavail) | ||
| 356 | +{ | ||
| 357 | + HcclComm comm = BuildComm(true, true, true, 0); | ||
| 358 | + myRank_->ccuInsHandle_ = 0; | ||
| 359 | + myRank_->opExpansionMode_ = 5; // CCU_MS_MODE 映射为 CCU_MS | ||
| 360 | + // 首次 CCU_MS 资源不足,降级 CCU_SCHED 仍资源不足(共调用两次)。 | ||
| 361 | + // 用 eq(CCU_MS)/eq(CCU_SCHED) 区分入参,避免两次 expects(once()) 互相误匹配 | ||
| 362 | + MOCKER(HcommCcuInsCreateLegacy) | ||
| 363 | + .expects(once()) | ||
| 364 | + .with(mockcpp::eq(CcuInstanceType::CCU_MS), mockcpp::any()) | ||
| 365 | + .will(returnValue(CcuResult::CCU_E_UNAVAIL)); | ||
| 366 | + MOCKER(HcommCcuInsCreateLegacy) | ||
| 367 | + .expects(once()) | ||
| 368 | + .with(mockcpp::eq(CcuInstanceType::CCU_SCHED), mockcpp::any()) | ||
| 369 | + .will(returnValue(CcuResult::CCU_E_UNAVAIL)); | ||
| 370 | + | ||
| 371 | + CcuInsHandle insHandles[1] = {0}; | ||
| 372 | + uint32_t insNum = 0; | ||
| 373 | + HcclResult ret = HcclCommQueryCcuIns(comm, insHandles, &insNum); | ||
| 374 | + | ||
| 375 | + EXPECT_EQ(ret, HCCL_E_UNAVAIL); | ||
| 376 | + EXPECT_EQ(insHandles[0], static_cast<CcuInsHandle>(0)); // 出参未修改 | ||
| 377 | + EXPECT_EQ(insNum, 0U); | ||
| 378 | + EXPECT_EQ(myRank_->GetCcuInstance(), 0); // 失败不回写 | ||
| 379 | +} | ||
| 380 | + | ||
| 381 | +TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommQueryCcuIns_When_CommIsNotV2_Expect_ReturnNotSupport) | ||
| 382 | +{ | ||
| 383 | + HcclComm comm = BuildComm(false, true, true, 0); | ||
| 384 | + MOCKER(HcommCcuInsCreateLegacy).expects(never()); | ||
| 385 | + | ||
| 386 | + CcuInsHandle insHandles[1] = {0}; | ||
| 387 | + uint32_t insNum = 0; | ||
| 388 | + HcclResult ret = HcclCommQueryCcuIns(comm, insHandles, &insNum); | ||
| 389 | + | ||
| 390 | + EXPECT_EQ(ret, HCCL_E_NOT_SUPPORT); | ||
| 391 | +} | ||
| 392 | + | ||
| 393 | +TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommQueryCcuIns_When_CommIsNull_Expect_ReturnPtr) | ||
| 394 | +{ | ||
| 395 | + CcuInsHandle insHandles[1] = {0}; | ||
| 396 | + uint32_t insNum = 0; | ||
| 397 | + EXPECT_EQ(HcclCommQueryCcuIns(nullptr, insHandles, &insNum), HCCL_E_PTR); | ||
| 398 | +} | ||
| 399 | + | ||
| 400 | +TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommQueryCcuIns_When_InsHandlesIsNull_Expect_ReturnPtr) | ||
| 401 | +{ | ||
| 402 | + HcclComm comm = BuildComm(); | ||
| 403 | + uint32_t insNum = 0; | ||
| 404 | + EXPECT_EQ(HcclCommQueryCcuIns(comm, nullptr, &insNum), HCCL_E_PTR); | ||
| 405 | +} | ||
| 406 | + | ||
| 407 | +TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommQueryCcuIns_When_InsNumIsNull_Expect_ReturnPtr) | ||
| 408 | +{ | ||
| 409 | + HcclComm comm = BuildComm(); | ||
| 410 | + CcuInsHandle insHandles[1] = {0}; | ||
| 411 | + EXPECT_EQ(HcclCommQueryCcuIns(comm, insHandles, nullptr), HCCL_E_PTR); | ||
| 412 | +} | ||
| 413 | + | ||
| 414 | +TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommQueryCcuIns_When_RepeatQuery_Expect_IdempotentAndNoSecondCreate) | ||
| 415 | +{ | ||
| 416 | + CcuInsHandle createdHandle = 0xCD; | ||
| 417 | + HcclComm comm = BuildComm(true, true, true, 0); | ||
| 418 | + myRank_->ccuInsHandle_ = 0; | ||
| 419 | + myRank_->opExpansionMode_ = 6; // CCU_SCHED | ||
| 420 | + MOCKER(HcommCcuInsCreateLegacy) | ||
| 421 | + .expects(once()) // 仅首次创建 | ||
| 422 | + .with(mockcpp::any(), mockcpp::outBoundP(&createdHandle, sizeof(CcuInsHandle))) | ||
| 423 | + .will(returnValue(CcuResult::CCU_SUCCESS)); | ||
| 424 | + | ||
| 425 | + CcuInsHandle insHandles[1] = {0}; | ||
| 426 | + uint32_t insNum = 0; | ||
| 427 | + EXPECT_EQ(HcclCommQueryCcuIns(comm, insHandles, &insNum), HCCL_SUCCESS); | ||
| 428 | + EXPECT_EQ(insHandles[0], createdHandle); | ||
| 429 | + // 第二次 Query:handle 已存在,不应再创建 | ||
| 430 | + CcuInsHandle insHandles2[1] = {0}; | ||
| 431 | + uint32_t insNum2 = 0; | ||
| 432 | + EXPECT_EQ(HcclCommQueryCcuIns(comm, insHandles2, &insNum2), HCCL_SUCCESS); | ||
| 433 | + EXPECT_EQ(insHandles2[0], createdHandle); | ||
| 434 | + EXPECT_EQ(myRank_->GetCcuInstance(), createdHandle); | ||
| 435 | +} | ||
| 436 | + | ||
| 437 | +// ============================================================================================ | ||
| 438 | +// HcclCommQueryAssignedCcuIns 单元测试(新增接口:查询新方式绑定的 assignedCcuInsHandle_) | ||
| 439 | +// ============================================================================================ | ||
| 440 | + | ||
| 441 | +TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommQueryAssignedCcuIns_When_Bound_Expect_ReturnSuccess) | ||
| 442 | +{ | ||
| 443 | + constexpr CcuInsHandle assignedHandle = 0xEF; | ||
| 444 | + HcclComm comm = BuildComm(true, true, true, assignedHandle); | ||
| 445 | + // 不应创建任何 instance | ||
| 446 | + MOCKER(HcommCcuInsCreateLegacy).expects(never()); | ||
| 447 | + | ||
| 448 | + CcuInsHandle insHandles[1] = {0}; | ||
| 449 | + uint32_t insNum = 0; | ||
| 450 | + HcclResult ret = HcclCommQueryAssignedCcuIns(comm, insHandles, &insNum); | ||
| 451 | + | ||
| 452 | + EXPECT_EQ(ret, HCCL_SUCCESS); | ||
| 453 | + EXPECT_EQ(insHandles[0], assignedHandle); | ||
| 454 | + EXPECT_EQ(insNum, 1U); | ||
| 455 | +} | ||
| 456 | + | ||
| 457 | +TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommQueryAssignedCcuIns_When_NotBound_Expect_ReturnUnavail) | ||
| 458 | +{ | ||
| 459 | + HcclComm comm = BuildComm(true, true, true, 0); // assignedCcuInsHandle_ = 0 | ||
| 460 | + MOCKER(HcommCcuInsCreateLegacy).expects(never()); // 不创建 | ||
| 461 | + | ||
| 462 | + CcuInsHandle insHandles[1] = {0}; | ||
| 463 | + uint32_t insNum = 0; | ||
| 464 | + HcclResult ret = HcclCommQueryAssignedCcuIns(comm, insHandles, &insNum); | ||
| 465 | + | ||
| 466 | + EXPECT_EQ(ret, HCCL_E_UNAVAIL); | ||
| 467 | + EXPECT_EQ(insHandles[0], static_cast<CcuInsHandle>(0)); // 出参未修改 | ||
| 468 | + EXPECT_EQ(insNum, 0U); | ||
| 469 | +} | ||
| 470 | + | ||
| 471 | +TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommQueryAssignedCcuIns_When_CommIsNotV2_Expect_ReturnNotSupport) | ||
| 472 | +{ | ||
| 473 | + HcclComm comm = BuildComm(false, true, true, 0xEF); | ||
| 474 | + CcuInsHandle insHandles[1] = {0}; | ||
| 475 | + uint32_t insNum = 0; | ||
| 476 | + EXPECT_EQ(HcclCommQueryAssignedCcuIns(comm, insHandles, &insNum), HCCL_E_NOT_SUPPORT); | ||
| 477 | +} | ||
| 478 | + | ||
| 479 | +TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommQueryAssignedCcuIns_When_CommIsNull_Expect_ReturnPtr) | ||
| 480 | +{ | ||
| 481 | + CcuInsHandle insHandles[1] = {0}; | ||
| 482 | + uint32_t insNum = 0; | ||
| 483 | + EXPECT_EQ(HcclCommQueryAssignedCcuIns(nullptr, insHandles, &insNum), HCCL_E_PTR); | ||
| 484 | +} | ||
| 485 | + | ||
| 486 | +TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommQueryAssignedCcuIns_When_InsHandlesIsNull_Expect_ReturnPtr) | ||
| 487 | +{ | ||
| 488 | + HcclComm comm = BuildComm(); | ||
| 489 | + uint32_t insNum = 0; | ||
| 490 | + EXPECT_EQ(HcclCommQueryAssignedCcuIns(comm, nullptr, &insNum), HCCL_E_PTR); | ||
| 491 | +} | ||
| 492 | + | ||
| 493 | +TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommQueryAssignedCcuIns_When_InsNumIsNull_Expect_ReturnPtr) | ||
| 494 | +{ | ||
| 495 | + HcclComm comm = BuildComm(); | ||
| 496 | + CcuInsHandle insHandles[1] = {0}; | ||
| 497 | + EXPECT_EQ(HcclCommQueryAssignedCcuIns(comm, insHandles, nullptr), HCCL_E_PTR); | ||
| 498 | +} | ||
| 499 | + | ||
| 500 | +// ============================================================================================ | ||
| 501 | +// HcclCommAssignCcuIns 语义变更补充:写 assignedCcuInsHandle_,ccuInsHandle_ 不变 | ||
| 502 | +// ============================================================================================ | ||
| 503 | + | ||
| 504 | +TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommAssignCcuIns_When_Success_Expect_CcuInsHandleUnchanged) | ||
| 505 | +{ | ||
| 506 | + constexpr CcuInsHandle selfHandle = 0xA; | ||
| 507 | + constexpr CcuInsHandle assignedHandle = 0xB; | ||
| 508 | + HcclComm comm = BuildComm(true, true, true, 0); | ||
| 509 | + myRank_->ccuInsHandle_ = selfHandle; // 通信域自有 handle | ||
| 510 | + MockCcuInstanceFound(assignedHandle); | ||
| 511 | + | ||
| 512 | + HcclResult ret = HcclCommAssignCcuIns(comm, assignedHandle); | ||
| 513 | + | ||
| 514 | + EXPECT_EQ(ret, HCCL_SUCCESS); | ||
| 515 | + EXPECT_EQ(myRank_->GetAssignedCcuInstance(), assignedHandle); | ||
| 516 | + EXPECT_EQ(myRank_->GetCcuInstance(), selfHandle); // 自有 handle 不被改写 | ||
| 517 | +} | ||
| 518 | + | ||
| 519 | +TEST_F(HcclCommAssignCcuInsTest, Ut_HcclCommQueryAssignedCcuIns_When_CalledBeforeAssign_Expect_ReturnUnavail) | ||
| 520 | +{ | ||
| 521 | + // 组合:未 Assign 直接 QueryAssigned | ||
| 522 | + HcclComm comm = BuildComm(true, true, true, 0); | ||
| 523 | + CcuInsHandle insHandles[1] = {0}; | ||
| 524 | + uint32_t insNum = 0; | ||
| 525 | + EXPECT_EQ(HcclCommQueryAssignedCcuIns(comm, insHandles, &insNum), HCCL_E_UNAVAIL); | ||
| 239 | } | 526 | } |
| @@ -65,10 +65,12 @@ CcuResult StubDeinitCcuFeature(int32_t deviceLogicId) | |||
| 65 | return CcuResult::CCU_SUCCESS; | 65 | return CcuResult::CCU_SUCCESS; |
| 66 | } | 66 | } |
| 67 | 67 | ||
| 68 | -CcuResult StubCreateCcuInstanceFailure(CcuInstanceType ccuInsType, CcuInsHandle* ccuInsHandle) | 68 | +// 简化后 TryInitCcuInstance 只走 OnDemand(拉起驱动 CcuInitFeature),不再创建 ccu instance; |
| 69 | +// 驱动拉起失败时的预约对账行为改由 CcuInitFeature 返回值驱动 | ||
| 70 | +CcuResult StubCcuInitFeatureFailure(const int32_t devLogicId, std::shared_ptr<hcomm::CcuDrvHandle>& ccuDrvHandle) | ||
| 69 | { | 71 | { |
| 70 | - (void)ccuInsType; | 72 | + EXPECT_EQ(devLogicId, DEVICE_0); |
| 71 | - (void)ccuInsHandle; | 73 | + (void)ccuDrvHandle; |
| 72 | bool reserved = false; | 74 | bool reserved = false; |
| 73 | EXPECT_EQ( | 75 | EXPECT_EQ( |
| 74 | hccl::CollCommMgr::GetInstance().TryReserveCcuMsComm(DEVICE_0, "init_failure_probe", reserved), HCCL_SUCCESS); | 76 | hccl::CollCommMgr::GetInstance().TryReserveCcuMsComm(DEVICE_0, "init_failure_probe", reserved), HCCL_SUCCESS); |
| @@ -207,8 +209,9 @@ TEST_F(CcuMsPerDeviceExclusiveTest, Ut_TryInitCcuInstance_When_SameDeviceAlready | |||
| 207 | ASSERT_TRUE(ownerReserved); | 209 | ASSERT_TRUE(ownerReserved); |
| 208 | myRank_->devLogicId_ = DEVICE_0; | 210 | myRank_->devLogicId_ = DEVICE_0; |
| 209 | myRank_->opExpansionMode_ = CCU_MS_MODE; | 211 | myRank_->opExpansionMode_ = CCU_MS_MODE; |
| 210 | - myRank_->useCcuResStaticAlloc_ = true; | 212 | + // 简化后:MS 回退到 SCHED 由 ReserveCcuMsCommOrFallback 完成(预约失败即回退), |
| 211 | - MOCKER(HcommCcuInsCreateLegacy).expects(once()).will(returnValue(CcuResult::CCU_SUCCESS)); | 213 | + // 不再依赖 ccu instance 创建;驱动拉起走 CcuInitFeature |
| 214 | + MOCKER(hcomm::CcuInitFeature).expects(once()).will(returnValue(CcuResult::CCU_SUCCESS)); | ||
| 212 | 215 | ||
| 213 | EXPECT_EQ(myRank_->TryInitCcuInstance(), HCCL_SUCCESS); | 216 | EXPECT_EQ(myRank_->TryInitCcuInstance(), HCCL_SUCCESS); |
| 214 | EXPECT_EQ(myRank_->opExpansionMode_, CCU_SCHED_MODE); | 217 | EXPECT_EQ(myRank_->opExpansionMode_, CCU_SCHED_MODE); |
| @@ -224,9 +227,9 @@ TEST_F(CcuMsPerDeviceExclusiveTest, Ut_TryInitCcuInstance_When_CcuInitFails_Expe | |||
| 224 | { | 227 | { |
| 225 | myRank_->devLogicId_ = DEVICE_0; | 228 | myRank_->devLogicId_ = DEVICE_0; |
| 226 | myRank_->opExpansionMode_ = CCU_MS_MODE; | 229 | myRank_->opExpansionMode_ = CCU_MS_MODE; |
| 227 | - myRank_->useCcuResStaticAlloc_ = true; | ||
| 228 | g_reservationHeldDuringInitFailure = false; | 230 | g_reservationHeldDuringInitFailure = false; |
| 229 | - MOCKER(HcommCcuInsCreateLegacy).expects(once()).will(invoke(StubCreateCcuInstanceFailure)); | 231 | + // 简化后 ccu instance 不在 init 时创建,驱动拉起失败由 CcuInitFeature 返回 |
| 232 | + MOCKER(hcomm::CcuInitFeature).expects(once()).will(invoke(StubCcuInitFeatureFailure)); | ||
| 230 | 233 | ||
| 231 | EXPECT_EQ(myRank_->TryInitCcuInstance(), HCCL_E_PARA); | 234 | EXPECT_EQ(myRank_->TryInitCcuInstance(), HCCL_E_PARA); |
| 232 | EXPECT_TRUE(g_reservationHeldDuringInitFailure); | 235 | EXPECT_TRUE(g_reservationHeldDuringInitFailure); |
| @@ -237,31 +240,12 @@ TEST_F(CcuMsPerDeviceExclusiveTest, Ut_TryInitCcuInstance_When_CcuInitFails_Expe | |||
| 237 | EXPECT_TRUE(nextReserved); | 240 | EXPECT_TRUE(nextReserved); |
| 238 | } | 241 | } |
| 239 | 242 | ||
| 240 | -TEST_F(CcuMsPerDeviceExclusiveTest, Ut_TryInitCcuInstance_When_MsFallsBack_Expect_ReservationReleased) | ||
| 241 | -{ | ||
| 242 | - myRank_->devLogicId_ = DEVICE_0; | ||
| 243 | - myRank_->opExpansionMode_ = CCU_MS_MODE; | ||
| 244 | - myRank_->useCcuResStaticAlloc_ = true; | ||
| 245 | - MOCKER(HcommCcuInsCreateLegacy) | ||
| 246 | - .expects(exactly(2)) | ||
| 247 | - .will(returnValue(CcuResult::CCU_E_UNAVAIL)) | ||
| 248 | - .then(returnValue(CcuResult::CCU_SUCCESS)); | ||
| 249 | - | ||
| 250 | - EXPECT_EQ(myRank_->TryInitCcuInstance(), HCCL_SUCCESS); | ||
| 251 | - EXPECT_EQ(myRank_->opExpansionMode_, CCU_SCHED_MODE); | ||
| 252 | - EXPECT_FALSE(myRank_->ccuMsCommReserved_); | ||
| 253 | - | ||
| 254 | - bool nextReserved = false; | ||
| 255 | - EXPECT_EQ(hccl::CollCommMgr::GetInstance().TryReserveCcuMsComm(DEVICE_0, "next_owner", nextReserved), HCCL_SUCCESS); | ||
| 256 | - EXPECT_TRUE(nextReserved); | ||
| 257 | -} | ||
| 258 | - | ||
| 259 | TEST_F(CcuMsPerDeviceExclusiveTest, Ut_TryInitCcuInstance_When_DriverBusyFallsBackToAicpu_Expect_ReservationReleased) | 243 | TEST_F(CcuMsPerDeviceExclusiveTest, Ut_TryInitCcuInstance_When_DriverBusyFallsBackToAicpu_Expect_ReservationReleased) |
| 260 | { | 244 | { |
| 261 | myRank_->devLogicId_ = DEVICE_0; | 245 | myRank_->devLogicId_ = DEVICE_0; |
| 262 | myRank_->opExpansionMode_ = CCU_MS_MODE; | 246 | myRank_->opExpansionMode_ = CCU_MS_MODE; |
| 263 | - myRank_->useCcuResStaticAlloc_ = true; | 247 | + // 简化后驱动忙回退到 aicpu 由 TryInitCcuInstanceOnDemand 中 CcuInitFeature 返回 CCU_E_DRV_BUSY 触发 |
| 264 | - MOCKER(HcommCcuInsCreateLegacy).expects(once()).will(returnValue(CcuResult::CCU_E_DRV_BUSY)); | 248 | + MOCKER(hcomm::CcuInitFeature).expects(once()).will(returnValue(CcuResult::CCU_E_DRV_BUSY)); |
| 265 | 249 | ||
| 266 | EXPECT_EQ(myRank_->TryInitCcuInstance(), HCCL_SUCCESS); | 250 | EXPECT_EQ(myRank_->TryInitCcuInstance(), HCCL_SUCCESS); |
| 267 | EXPECT_EQ(myRank_->opExpansionMode_, AICPU_TS_MODE); | 251 | EXPECT_EQ(myRank_->opExpansionMode_, AICPU_TS_MODE); |
| @@ -282,7 +266,7 @@ TEST_F(CcuMsPerDeviceExclusiveTest, Ut_MyRankDestructor_When_HoldsMsReservation_ | |||
| 282 | myRank_->devLogicId_ = DEVICE_0; | 266 | myRank_->devLogicId_ = DEVICE_0; |
| 283 | myRank_->ccuMsCommReserved_ = true; | 267 | myRank_->ccuMsCommReserved_ = true; |
| 284 | myRank_->ccuInsHandle_ = VALID_CCU_INS_HANDLE; | 268 | myRank_->ccuInsHandle_ = VALID_CCU_INS_HANDLE; |
| 285 | - myRank_->useCcuResStaticAlloc_ = false; | 269 | + // 删除 useCcuResStaticAlloc_ 成员;析构路径按 ccuInsHandle_/assignedCcuInsHandle_ 是否非 0 决定清理 |
| 286 | auto* fakeDrvHandle = reinterpret_cast<hcomm::CcuDrvHandle*>(0x1); | 270 | auto* fakeDrvHandle = reinterpret_cast<hcomm::CcuDrvHandle*>(0x1); |
| 287 | myRank_->ccuDrvHandle_ = std::shared_ptr<hcomm::CcuDrvHandle>(fakeDrvHandle, [](hcomm::CcuDrvHandle*) {}); | 271 | myRank_->ccuDrvHandle_ = std::shared_ptr<hcomm::CcuDrvHandle>(fakeDrvHandle, [](hcomm::CcuDrvHandle*) {}); |
| 288 | g_reservationHeldDuringInstanceDestroy.store(false); | 272 | g_reservationHeldDuringInstanceDestroy.store(false); |
| @@ -20,6 +20,7 @@ | |||
| 20 | 20 | ||
| 21 | 21 | ||
| 22 | 22 | ||
| 23 | + | ||
| 23 | 24 | ||
| 24 | 25 | ||
| 25 | 26 | ||
| @@ -320,10 +321,11 @@ TEST_F(MyRankTest, Ut_Init_When_Default_Mode_Expect_Set_By_Env) | |||
| 320 | } | 321 | } |
| 321 | 322 | ||
| 322 | // 测试Init时ccu驱动拉起失败回退到aicpu | 323 | // 测试Init时ccu驱动拉起失败回退到aicpu |
| 324 | +// 简化后 TryInitCcuInstance 只走 OnDemand(拉起驱动 CcuInitFeature),不再调 HcommCcuInsCreateLegacy 创建实例 | ||
| 323 | TEST_F(MyRankTest, Ut_Init_When_Ccu_Driver_Fail_Expect_Fallback_Aicpu) | 325 | TEST_F(MyRankTest, Ut_Init_When_Ccu_Driver_Fail_Expect_Fallback_Aicpu) |
| 324 | { | 326 | { |
| 325 | setenv("HCCL_CCU_CUSTOM_OP_MODE", "1", 1); | 327 | setenv("HCCL_CCU_CUSTOM_OP_MODE", "1", 1); |
| 326 | - MOCKER(HcommCcuInsCreateLegacy).stubs().will(returnValue(CcuResult::CCU_E_DRV_BUSY)); | 328 | + MOCKER(hcomm::CcuInitFeature).stubs().will(returnValue(CcuResult::CCU_E_DRV_BUSY)); |
| 327 | 329 | ||
| 328 | HcclMem cclBuffer; | 330 | HcclMem cclBuffer; |
| 329 | CreateCclBuffer(cclBuffer); | 331 | CreateCclBuffer(cclBuffer); |
| @@ -334,44 +336,13 @@ TEST_F(MyRankTest, Ut_Init_When_Ccu_Driver_Fail_Expect_Fallback_Aicpu) | |||
| 334 | unsetenv("HCCL_CCU_CUSTOM_OP_MODE"); | 336 | unsetenv("HCCL_CCU_CUSTOM_OP_MODE"); |
| 335 | } | 337 | } |
| 336 | 338 | ||
| 337 | -// 测试Init时ccu ms资源不足回退到sched | 339 | +// 测试Init时ccu驱动拉起返回非预期错误时失败 |
| 338 | -TEST_F(MyRankTest, Ut_Init_When_Ccu_Ms_Insufficient_Expect_Fallback_Sched) | 340 | +// 简化后 ccu instance 不在 init 时创建,MS->SCH 回退由 ReserveCcuMsCommOrFallback 在预约失败时完成; |
| 341 | +// 此用例验证驱动返回非预期错误(CCU_E_PARA)时直接失败 | ||
| 342 | +TEST_F(MyRankTest, Ut_Init_When_Ccu_Driver_Unexpected_Fail_Expect_Fail) | ||
| 339 | { | 343 | { |
| 340 | setenv("HCCL_CCU_CUSTOM_OP_MODE", "1", 1); | 344 | setenv("HCCL_CCU_CUSTOM_OP_MODE", "1", 1); |
| 341 | - MOCKER(HcommCcuInsCreateLegacy) | 345 | + MOCKER(hcomm::CcuInitFeature).stubs().will(returnValue(CcuResult::CCU_E_PARA)); |
| 342 | - .stubs() | ||
| 343 | - .will(returnValue(CcuResult::CCU_E_UNAVAIL)) | ||
| 344 | - .then(returnValue(CcuResult::CCU_SUCCESS)); | ||
| 345 | - | ||
| 346 | - HcclMem cclBuffer; | ||
| 347 | - CreateCclBuffer(cclBuffer); | ||
| 348 | - | ||
| 349 | - uint32_t opExpansionModeMs = CCU_MS_MODE; | ||
| 350 | - EXPECT_EQ(myRank->Init(cclBuffer, opExpansionModeMs, 2), HCCL_SUCCESS); | ||
| 351 | - EXPECT_EQ(myRank->opExpansionMode_, CCU_SCHED_MODE); | ||
| 352 | - unsetenv("HCCL_CCU_CUSTOM_OP_MODE"); | ||
| 353 | -} | ||
| 354 | - | ||
| 355 | -// 测试Init时ccu ms和sched资源不足回退到aicpu | ||
| 356 | -TEST_F(MyRankTest, Ut_Init_When_Ccu_Ms_And_Sched_Insufficient_Expect_Fallback_Aicpu) | ||
| 357 | -{ | ||
| 358 | - setenv("HCCL_CCU_CUSTOM_OP_MODE", "1", 1); | ||
| 359 | - MOCKER(HcommCcuInsCreateLegacy).stubs().will(returnValue(CcuResult::CCU_E_UNAVAIL)); | ||
| 360 | - | ||
| 361 | - HcclMem cclBuffer; | ||
| 362 | - CreateCclBuffer(cclBuffer); | ||
| 363 | - | ||
| 364 | - uint32_t opExpansionModeMs = CCU_MS_MODE; | ||
| 365 | - EXPECT_EQ(myRank->Init(cclBuffer, opExpansionModeMs, 2), HCCL_SUCCESS); | ||
| 366 | - EXPECT_EQ(myRank->opExpansionMode_, AICPU_TS_MODE); | ||
| 367 | - unsetenv("HCCL_CCU_CUSTOM_OP_MODE"); | ||
| 368 | -} | ||
| 369 | - | ||
| 370 | -// 测试Init在申请资源时出现其他报错时失败 | ||
| 371 | -TEST_F(MyRankTest, Ut_Init_When_Resource_Fail_Expect_Fail) | ||
| 372 | -{ | ||
| 373 | - setenv("HCCL_CCU_CUSTOM_OP_MODE", "1", 1); | ||
| 374 | - MOCKER(HcommCcuInsCreateLegacy).stubs().will(returnValue(CcuResult::CCU_E_PARA)); | ||
| 375 | 346 | ||
| 376 | HcclMem cclBuffer; | 347 | HcclMem cclBuffer; |
| 377 | CreateCclBuffer(cclBuffer); | 348 | CreateCclBuffer(cclBuffer); |
| @@ -32,14 +32,9 @@ | |||
| 32 | 32 | ||
| 33 | 33 | ||
| 34 | 34 | ||
| 35 | -namespace hcomm { | ||
| 36 | -HcclResult GetHcclVersionForCcuKernelMgr(int& hcclVersion); | ||
| 37 | -} | ||
| 38 | - | ||
| 39 | namespace { | 35 | namespace { |
| 40 | constexpr int32_t TEST_DEVICE_LOGIC_ID = 0; | 36 | constexpr int32_t TEST_DEVICE_LOGIC_ID = 0; |
| 41 | constexpr int32_t OTHER_TEST_DEVICE_LOGIC_ID = 1; | 37 | constexpr int32_t OTHER_TEST_DEVICE_LOGIC_ID = 1; |
| 42 | -constexpr int32_t HCCL_VERSION_USING_VALIDATE_AND_APPLY_DIE = 90100001; | ||
| 43 | constexpr uint32_t VALID_DIE_ID = 0; | 38 | constexpr uint32_t VALID_DIE_ID = 0; |
| 44 | constexpr uint32_t OTHER_VALID_DIE_ID = 1; | 39 | constexpr uint32_t OTHER_VALID_DIE_ID = 1; |
| 45 | constexpr uint32_t INVALID_DIE_ID = hcomm::CCU_MAX_IODIE_NUM; | 40 | constexpr uint32_t INVALID_DIE_ID = hcomm::CCU_MAX_IODIE_NUM; |
| @@ -326,10 +321,6 @@ public: | |||
| 326 | g_deviceRefreshCalls = 0; | 321 | g_deviceRefreshCalls = 0; |
| 327 | g_deviceRefreshResult = HcclResult::HCCL_SUCCESS; | 322 | g_deviceRefreshResult = HcclResult::HCCL_SUCCESS; |
| 328 | 323 | ||
| 329 | - MOCKER(hcomm::GetHcclVersionForCcuKernelMgr) | ||
| 330 | - .stubs() | ||
| 331 | - .with(outBound(HCCL_VERSION_USING_VALIDATE_AND_APPLY_DIE)) | ||
| 332 | - .will(returnValue(HCCL_SUCCESS)); | ||
| 333 | MOCKER(hrtGetDeviceRefresh).stubs().with(mockcpp::any()).will(invoke(MockHrtGetDeviceRefresh)); | 324 | MOCKER(hrtGetDeviceRefresh).stubs().with(mockcpp::any()).will(invoke(MockHrtGetDeviceRefresh)); |
| 334 | int32_t seededDeviceLogicId = INVALID_INT; | 325 | int32_t seededDeviceLogicId = INVALID_INT; |
| 335 | ASSERT_EQ(HcclDeviceRefresh(seededDeviceLogicId), HcclResult::HCCL_SUCCESS); | 326 | ASSERT_EQ(HcclDeviceRefresh(seededDeviceLogicId), HcclResult::HCCL_SUCCESS); |


🔴 [严重] [样例代码完整性] HcclCommQueryCcuIns.md:80
文档新增约束 #6(第67行)和 NOTE 块(第56-58行)明确要求:
但样例代码仅简单
return错误码,未展示 HCCL_E_UNAVAIL 时的降级处理模式。建议在错误处理分支中补充 HCCL_E_UNAVAIL 的降级示例:
HcclResult ret = HcclCommQueryCcuIns(comm, &insHandle, &insNum); if (ret == HCCL_E_UNAVAIL) { // CCU资源不足且无法自动降级到AICPU_TS,调用方自行处理降级 // 例如:回退到AICPU_TS模式执行 return fallbackToAicpuTs(comm, ...); } if (ret != HCCL_SUCCESS || insNum != 1) { return (ret != HCCL_SUCCESS) ? ret : HCCL_E_INTERNAL; }