feat(comm): add async RDMA support for HNS1825-pto-isa-AtomGit

Mdocs/isa/comm/README.md+1-0

	@@ -83,6 +83,7 @@ DMA backend selection for `TPUT_ASYNC` and `TGET_ASYNC`:
83	\|-------\|-------------\|	83	\|-------\|-------------\|
84	\| `DmaEngine::SDMA` \| SDMA engine (supports 1D transfer) \|	84	\| `DmaEngine::SDMA` \| SDMA engine (supports 1D transfer) \|
85	\| `DmaEngine::URMA` \| URMA engine (supports 1D transfer, Ascend950 / NPU_ARCH 3510 only; requires CANN >= 9.1.0) \|	85	\| `DmaEngine::URMA` \| URMA engine (supports 1D transfer, Ascend950 / NPU_ARCH 3510 only; requires CANN >= 9.1.0) \|
		86	+\| `DmaEngine::RDMA` \| RDMA engine (supports 1D transfer, Ascend950 / NPU_ARCH 3510 only; currently supports only the HNS1825 NIC platform) \|
86		87
87	### AsyncEvent	88	### AsyncEvent
88		89

Mdocs/isa/comm/README_zh.md+1-0

	@@ -83,6 +83,7 @@ comm::TTEST(signal, 1, comm::WaitCmp::GE);
83	\|-------\|-------------\|	83	\|-------\|-------------\|
84	\| `DmaEngine::SDMA` \| SDMA 引擎（支持一维传输，Ascend950 上仅支持TGET\|	84	\| `DmaEngine::SDMA` \| SDMA 引擎（支持一维传输，Ascend950 上仅支持TGET\|
85	\| `DmaEngine::URMA` \| URMA 引擎（支持一维传输，仅 Ascend950 / NPU_ARCH 3510；要求 CANN >= 9.1.0）\|	85	\| `DmaEngine::URMA` \| URMA 引擎（支持一维传输，仅 Ascend950 / NPU_ARCH 3510；要求 CANN >= 9.1.0）\|
		86	+\| `DmaEngine::RDMA` \| RDMA 引擎（支持一维传输，仅 Ascend950 / NPU_ARCH 3510；当前网卡平台仅支持 HNS1825）\|
86		87
87	### AsyncEvent	88	### AsyncEvent
88		89

Mdocs/isa/comm/TGET_ASYNC.md+40-0

	@@ -13,6 +13,7 @@ Data flow:
13	- `engine`:	13	- `engine`:
14	- `DmaEngine::SDMA` (default)	14	- `DmaEngine::SDMA` (default)
15	- `DmaEngine::URMA` (Ascend950, NPU_ARCH 3510 only)	15	- `DmaEngine::URMA` (Ascend950, NPU_ARCH 3510 only)
		16	+ - `DmaEngine::RDMA` (Ascend950, NPU_ARCH 3510 only; currently supports only the HNS1825 NIC platform)
16		17
17	> Important (SDMA path)	18	> Important (SDMA path)
18	> `TGET_ASYNC` with `DmaEngine::SDMA` currently supports only flat contiguous logical 1D tensors.	19	> `TGET_ASYNC` with `DmaEngine::SDMA` currently supports only flat contiguous logical 1D tensors.
	@@ -82,6 +83,39 @@ PTO_INTERNAL bool BuildAsyncSession(__gm__ uint8_t *workspace,
82		83
83	URMA does not require `scratchTile` — polling uses `ld_dev`/`st_dev` hardware intrinsics directly.	84	URMA does not require `scratchTile` — polling uses `ld_dev`/`st_dev` hardware intrinsics directly.
84		85
		86	+### RDMA Construction (NPU_ARCH 3510 only)
		87	+
		88	+RDMA is intended for standard cross-node networking and currently supports only the HNS1825 NIC platform.
		89	+
		90	+```cpp
		91	+#ifdef PTO_RDMA_SUPPORTED
		92	+template <DmaEngine engine, typename ScratchTile>
		93	+PTO_INTERNAL bool BuildAsyncSession(ScratchTile &scratchTile,
		94	+ __gm__ uint8_t *workspace,
		95	+ uint32_t myPe,
		96	+ AsyncSession &session,
		97	+ uint32_t syncId = 0);
		98	+#endif
		99	+```
		100	+
		101	+\| Parameter \| Description \|
		102	+\|---\|---\|
		103	+\| `scratchTile` \| UB/Vec tile used for WQE/CQE control data; at least 64 bytes. \|
		104	+\| `workspace` \| GM pointer returned by the host-side RDMA initialization flow. \|
		105	+\| `myPe` \| Local rank id used to select the registered local memory region. \|
		106	+\| `session` \| Output `AsyncSession` object. \|
		107	+\| `syncId` \| MTE/scalar synchronization event id in the range 0-7. \|
		108	+
		109	+Build one peer-independent session, then select the source rank with the explicit `peer` argument:
		110	+
		111	+```cpp
		112	+comm::AsyncSession session;
		113	+if (comm::BuildAsyncSession<comm::DmaEngine::RDMA>(scratchTile, rdmaWorkspace, myPe, session, syncId)) {
		114	+ auto event = comm::TGET_ASYNC<comm::DmaEngine::RDMA>(dstG, srcG, session, peer);
		115	+ (void)event.Wait(session);
		116	+}
		117	+```
		118	+
85	## Constraints	119	## Constraints
86		120
87	- `GlobalSrcData::RawDType == GlobalDstData::RawDType`	121	- `GlobalSrcData::RawDType == GlobalDstData::RawDType`
	@@ -93,6 +127,9 @@ URMA does not require `scratchTile` — polling uses `ld_dev`/`st_dev` hardware
93	- URMA is only available on NPU_ARCH 3510 (Ascend950)	127	- URMA is only available on NPU_ARCH 3510 (Ascend950)
94	- URMA requires CANN Toolkit >= 9.1.0	128	- URMA requires CANN Toolkit >= 9.1.0
95	- The symmetric data buffer passed to `UrmaWorkspaceManager::Init()` must be backed by huge-page memory (allocate with `ACL_MEM_MALLOC_HUGE_ONLY`). The underlying MR registration requires huge-page backing; `ACL_MEM_MALLOC_HUGE_FIRST` may silently fall back to 4KB pages for small allocations, causing registration to fail	129	- The symmetric data buffer passed to `UrmaWorkspaceManager::Init()` must be backed by huge-page memory (allocate with `ACL_MEM_MALLOC_HUGE_ONLY`). The underlying MR registration requires huge-page backing; `ACL_MEM_MALLOC_HUGE_FIRST` may silently fall back to 4KB pages for small allocations, causing registration to fail
		130	+- RDMA is available only on Ascend950 / NPU_ARCH 3510 and currently supports only the HNS1825 NIC platform
		131	+- RDMA source and destination tensors must be flat contiguous logical 1D, and the complete local and remote ranges must lie within memory regions registered during host initialization
		132	+- One RDMA transfer must not exceed `0x7fffffff` bytes
96		133
97	If the 1D contiguous requirement is not met, current implementation returns an invalid async event (`handle == 0`).	134	If the 1D contiguous requirement is not met, current implementation returns an invalid async event (`handle == 0`).
98		135
	@@ -121,11 +158,14 @@ The completion mechanism differs by engine, but user-facing quiet semantics are
121		158
122	- SDMA: Each `TGET_ASYNC` submits data-transfer SQEs and flag SQEs that mark completion of that operation. `Wait` or `Test` on its returned event polls the corresponding flags to determine whether that `TGET_ASYNC` has completed; completion also guarantees that all earlier SDMA operations in the same session have completed.	159	- SDMA: Each `TGET_ASYNC` submits data-transfer SQEs and flag SQEs that mark completion of that operation. `Wait` or `Test` on its returned event polls the corresponding flags to determine whether that `TGET_ASYNC` has completed; completion also guarantees that all earlier SDMA operations in the same session have completed.
123	- URMA: `TGET_ASYNC` submits an RDMA READ WQE and rings the doorbell immediately. `Wait` polls the Completion Queue (CQ) until all expected CQEs have been consumed.	160	- URMA: `TGET_ASYNC` submits an RDMA READ WQE and rings the doorbell immediately. `Wait` polls the Completion Queue (CQ) until all expected CQEs have been consumed.
		161	+- RDMA: `TGET_ASYNC` submits an RDMA READ WQE to the queue selected by `peer`. Completion is tracked independently for each peer/queue.
124		162
125	- `event.Wait(session)` — blocks until all async operations issued since the last Wait are complete	163	- `event.Wait(session)` — blocks until all async operations issued since the last Wait are complete
126		164
127	This means after multiple `TGET_ASYNC` calls, a single `Wait` on the last returned `AsyncEvent` drains all pending operations (similar to shmem's quiet semantics).	165	This means after multiple `TGET_ASYNC` calls, a single `Wait` on the last returned `AsyncEvent` drains all pending operations (similar to shmem's quiet semantics).
128		166
		167	+For RDMA operations targeting different peers, wait for the last event of each peer separately.
		168	+
129	Up to 64 operations may be outstanding in one session before submission can apply backpressure.	169	Up to 64 operations may be outstanding in one session before submission can apply backpressure.
130		170
131	After wait succeeds, all issued reads into `dstGlobalData` are complete.	171	After wait succeeds, all issued reads into `dstGlobalData` are complete.

Mdocs/isa/comm/TGET_ASYNC_zh.md+40-0

	@@ -13,6 +13,7 @@
13	- `engine`：	13	- `engine`：
14	- `DmaEngine::SDMA`（默认）	14	- `DmaEngine::SDMA`（默认）
15	- `DmaEngine::URMA`（Ascend 950PR/Ascend 950DT，仅NPU_ARCH 3510）	15	- `DmaEngine::URMA`（Ascend 950PR/Ascend 950DT，仅NPU_ARCH 3510）
		16	+ - `DmaEngine::RDMA`（Ascend 950PR/Ascend 950DT，仅NPU_ARCH 3510；当前网卡平台仅支持 HNS1825）
16		17
17	> 注意（SDMA路径）	18	> 注意（SDMA路径）
18	> `TGET_ASYNC` 配合 `DmaEngine::SDMA` 目前仅支持扁平连续的逻辑一维tensor。	19	> `TGET_ASYNC` 配合 `DmaEngine::SDMA` 目前仅支持扁平连续的逻辑一维tensor。
	@@ -79,6 +80,39 @@ PTO_INTERNAL bool BuildAsyncSession(__gm__ uint8_t *workspace,
79		80
80	URMA不需要 `scratchTile`——轮询通过 `ld_dev`/`st_dev` 硬件原语直接操作。	81	URMA不需要 `scratchTile`——轮询通过 `ld_dev`/`st_dev` 硬件原语直接操作。
81		82
		83	+### RDMA构建（仅NPU_ARCH 3510）
		84	+
		85	+RDMA面向经典跨机场景组网，当前网卡平台仅支持 HNS1825。
		86	+
		87	+```cpp
		88	+#ifdef PTO_RDMA_SUPPORTED
		89	+template <DmaEngine engine, typename ScratchTile>
		xun8月14日考虑需要推荐用户使用新接口，在这里不说明老接口 xun 8月14日评论： 8月14日评论： xun 8月14日评论： 8月14日评论：
		90	+PTO_INTERNAL bool BuildAsyncSession(ScratchTile &scratchTile,
		91	+ __gm__ uint8_t *workspace,
		92	+ uint32_t myPe,
		93	+ AsyncSession &session,
		94	+ uint32_t syncId = 0);
		95	+#endif
		96	+```
		97	+
		98	+\| 参数 \| 说明 \|
		99	+\|---\|---\|
		100	+\| `scratchTile` \| 用于 WQE/CQE 控制数据的 UB/Vec tile，至少 64 字节。\|
		101	+\| `workspace` \| Host侧 RDMA 初始化流程返回的 GM 指针。\|
		102	+\| `myPe` \| 本地 rank id，用于选择已注册的本地内存区域。\|
		103	+\| `session` \| 输出的 `AsyncSession` 对象。\|
		104	+\| `syncId` \| MTE/Scalar 同步事件 ID，取值范围为 0-7。\|
		105	+
		106	+构建一次不绑定 peer 的Session，再通过显式 `peer` 参数选择源rank：
		107	+
		108	+```cpp
		109	+comm::AsyncSession session;
		110	+if (comm::BuildAsyncSession<comm::DmaEngine::RDMA>(scratchTile, rdmaWorkspace, myPe, session, syncId)) {
		111	+ auto event = comm::TGET_ASYNC<comm::DmaEngine::RDMA>(dstG, srcG, session, peer);
		112	+ (void)event.Wait(session);
		113	+}
		114	+```
		115	+
82	## 约束	116	## 约束
83		117
84	- `GlobalSrcData::RawDType == GlobalDstData::RawDType`	118	- `GlobalSrcData::RawDType == GlobalDstData::RawDType`
	@@ -90,6 +124,9 @@ URMA不需要 `scratchTile`——轮询通过 `ld_dev`/`st_dev` 硬件原语直
90	- URMA仅在NPU_ARCH 3510（Ascend 950PR/Ascend 950DT）上可用	124	- URMA仅在NPU_ARCH 3510（Ascend 950PR/Ascend 950DT）上可用
91	- URMA要求CANN Toolkit >= 9.1.0	125	- URMA要求CANN Toolkit >= 9.1.0
92	- 传给 `UrmaWorkspaceManager::Init()` 的对称数据缓冲区必须由大页内存支撑（使用 `ACL_MEM_MALLOC_HUGE_ONLY` 分配）。底层MR注册要求大页背景；`ACL_MEM_MALLOC_HUGE_FIRST` 在小尺寸分配时可能静默回退到4KB小页，导致注册失败	126	- 传给 `UrmaWorkspaceManager::Init()` 的对称数据缓冲区必须由大页内存支撑（使用 `ACL_MEM_MALLOC_HUGE_ONLY` 分配）。底层MR注册要求大页背景；`ACL_MEM_MALLOC_HUGE_FIRST` 在小尺寸分配时可能静默回退到4KB小页，导致注册失败
		127	+- RDMA仅在Ascend 950PR/Ascend 950DT（NPU_ARCH 3510）上可用，当前网卡平台仅支持 HNS1825
		128	+- RDMA源、目的tensor必须是扁平连续的逻辑一维，且本地和远端完整传输范围必须位于Host初始化阶段注册的内存区域内
		129	+- 单次RDMA传输不能超过 `0x7fffffff` 字节
93		130
94	若不满足一维连续要求，当前实现返回无效async event（`handle == 0`）。	131	若不满足一维连续要求，当前实现返回无效async event（`handle == 0`）。
95		132
	@@ -118,11 +155,14 @@ URMA不需要 `scratchTile`——轮询通过 `ld_dev`/`st_dev` 硬件原语直
118		155
119	- SDMA：每次`TGET_ASYNC`都会提交数据传输SQE和用于标记本次操作完成的flag SQE。对其返回的Event调用`Wait`或`Test`时，通过轮询对应flag判断该次`TGET_ASYNC`是否完成；完成后也能保证同一Session中此前提交的所有SDMA操作均已完成。	156	- SDMA：每次`TGET_ASYNC`都会提交数据传输SQE和用于标记本次操作完成的flag SQE。对其返回的Event调用`Wait`或`Test`时，通过轮询对应flag判断该次`TGET_ASYNC`是否完成；完成后也能保证同一Session中此前提交的所有SDMA操作均已完成。
120	- URMA：`TGET_ASYNC` 立即提交RDMA READ WQE并敲门铃。`Wait` 通过轮询Completion Queue（CQ）等待所有预期的CQE被消费。	157	- URMA：`TGET_ASYNC` 立即提交RDMA READ WQE并敲门铃。`Wait` 通过轮询Completion Queue（CQ）等待所有预期的CQE被消费。
		158	+- RDMA：`TGET_ASYNC` 向 `peer` 选择的队列提交RDMA READ WQE；不同peer/queue分别跟踪完成状态。
121		159
122	- `event.Wait(session)` —阻塞，直到自上次Wait以来所有已发出的异步操作全部完成	160	- `event.Wait(session)` —阻塞，直到自上次Wait以来所有已发出的异步操作全部完成
123		161
124	这意味着多次 `TGET_ASYNC` 调用后，只需对最后一个返回的 `AsyncEvent` 调用一次 `Wait`，即可等待所有pending操作完成（类似shmem的quiet语义）。	162	这意味着多次 `TGET_ASYNC` 调用后，只需对最后一个返回的 `AsyncEvent` 调用一次 `Wait`，即可等待所有pending操作完成（类似shmem的quiet语义）。
125		163
		164	+RDMA操作涉及不同peer时，必须分别等待每个peer的最后一个Event。
		165	+
126	同一Session最多可有64个未完成操作，超过后提交可能产生背压。	166	同一Session最多可有64个未完成操作，超过后提交可能产生背压。
127		167
128	wait成功后，所有已发出的 `dstGlobalData` 读入数据均已全部就绪。	168	wait成功后，所有已发出的 `dstGlobalData` 读入数据均已全部就绪。

Mdocs/isa/comm/TPUT_ASYNC.md+40-0

	@@ -14,6 +14,7 @@ Data flow:
14	- `engine`:	14	- `engine`:
15	- `DmaEngine::SDMA` (default)	15	- `DmaEngine::SDMA` (default)
16	- `DmaEngine::URMA` (Ascend950, NPU_ARCH 3510 only)	16	- `DmaEngine::URMA` (Ascend950, NPU_ARCH 3510 only)
		17	+ - `DmaEngine::RDMA` (Ascend950, NPU_ARCH 3510 only; currently supports only the HNS1825 NIC platform)
17		18
18	> Important (SDMA path)	19	> Important (SDMA path)
19	> `TPUT_ASYNC` with `DmaEngine::SDMA` currently supports only flat contiguous logical 1D tensors.	20	> `TPUT_ASYNC` with `DmaEngine::SDMA` currently supports only flat contiguous logical 1D tensors.
	@@ -84,6 +85,39 @@ PTO_INTERNAL bool BuildAsyncSession(__gm__ uint8_t *workspace,
84		85
85	URMA does not require `scratchTile` — polling uses `ld_dev`/`st_dev` hardware intrinsics directly.	86	URMA does not require `scratchTile` — polling uses `ld_dev`/`st_dev` hardware intrinsics directly.
86		87
		88	+### RDMA Construction (NPU_ARCH 3510 only)
		89	+
		90	+RDMA is intended for standard cross-node networking and currently supports only the HNS1825 NIC platform.
		91	+
		92	+```cpp
		93	+#ifdef PTO_RDMA_SUPPORTED
		94	+template <DmaEngine engine, typename ScratchTile>
		95	+PTO_INTERNAL bool BuildAsyncSession(ScratchTile &scratchTile,
		96	+ __gm__ uint8_t *workspace,
		97	+ uint32_t myPe,
		98	+ AsyncSession &session,
		99	+ uint32_t syncId = 0);
		100	+#endif
		101	+```
		102	+
		103	+\| Parameter \| Description \|
		104	+\|---\|---\|
		105	+\| `scratchTile` \| UB/Vec tile used for WQE/CQE control data; at least 64 bytes. \|
		106	+\| `workspace` \| GM pointer returned by the host-side RDMA initialization flow. \|
		107	+\| `myPe` \| Local rank id used to select the registered local memory region. \|
		108	+\| `session` \| Output `AsyncSession` object. \|
		109	+\| `syncId` \| MTE/scalar synchronization event id in the range 0-7. \|
		110	+
		111	+Build one peer-independent session, then select the destination rank with the explicit `peer` argument:
		112	+
		113	+```cpp
		114	+comm::AsyncSession session;
		115	+if (comm::BuildAsyncSession<comm::DmaEngine::RDMA>(scratchTile, rdmaWorkspace, myPe, session, syncId)) {
		116	+ auto event = comm::TPUT_ASYNC<comm::DmaEngine::RDMA>(dstG, srcG, session, peer);
		117	+ (void)event.Wait(session);
		118	+}
		119	+```
		120	+
87	## Constraints	121	## Constraints
88		122
89	- `GlobalSrcData::RawDType == GlobalDstData::RawDType`	123	- `GlobalSrcData::RawDType == GlobalDstData::RawDType`
	@@ -95,6 +129,9 @@ URMA does not require `scratchTile` — polling uses `ld_dev`/`st_dev` hardware
95	- URMA is only available on NPU_ARCH 3510 (Ascend950)	129	- URMA is only available on NPU_ARCH 3510 (Ascend950)
96	- URMA requires CANN Toolkit >= 9.1.0	130	- URMA requires CANN Toolkit >= 9.1.0
97	- The symmetric data buffer passed to `UrmaWorkspaceManager::Init()` must be backed by huge-page memory (allocate with `ACL_MEM_MALLOC_HUGE_ONLY`). The underlying MR registration requires huge-page backing; `ACL_MEM_MALLOC_HUGE_FIRST` may silently fall back to 4KB pages for small allocations, causing registration to fail	131	- The symmetric data buffer passed to `UrmaWorkspaceManager::Init()` must be backed by huge-page memory (allocate with `ACL_MEM_MALLOC_HUGE_ONLY`). The underlying MR registration requires huge-page backing; `ACL_MEM_MALLOC_HUGE_FIRST` may silently fall back to 4KB pages for small allocations, causing registration to fail
		132	+- RDMA is available only on Ascend950 / NPU_ARCH 3510 and currently supports only the HNS1825 NIC platform
		133	+- RDMA source and destination tensors must be flat contiguous logical 1D, and the complete local and remote ranges must lie within memory regions registered during host initialization
		134	+- One RDMA transfer must not exceed `0x7fffffff` bytes
98		135
99	If the 1D contiguous requirement is not met, current implementation returns an invalid async event (`handle == 0`).	136	If the 1D contiguous requirement is not met, current implementation returns an invalid async event (`handle == 0`).
100		137
	@@ -123,11 +160,14 @@ The completion mechanism differs by engine, but user-facing quiet semantics are
123		160
124	- SDMA: Each `TPUT_ASYNC` submits data-transfer SQEs and flag SQEs that mark completion of that operation. `Wait` or `Test` on its returned event polls the corresponding flags to determine whether that `TPUT_ASYNC` has completed; completion also guarantees that all earlier SDMA operations in the same session have completed.	161	- SDMA: Each `TPUT_ASYNC` submits data-transfer SQEs and flag SQEs that mark completion of that operation. `Wait` or `Test` on its returned event polls the corresponding flags to determine whether that `TPUT_ASYNC` has completed; completion also guarantees that all earlier SDMA operations in the same session have completed.
125	- URMA: `TPUT_ASYNC` submits an RDMA WRITE WQE and rings the doorbell immediately. `Wait` polls the Completion Queue (CQ) until all expected CQEs have been consumed.	162	- URMA: `TPUT_ASYNC` submits an RDMA WRITE WQE and rings the doorbell immediately. `Wait` polls the Completion Queue (CQ) until all expected CQEs have been consumed.
		163	+- RDMA: `TPUT_ASYNC` submits an RDMA WRITE WQE to the queue selected by `peer`. Completion is tracked independently for each peer/queue.
126		164
127	- `event.Wait(session)` — blocks until all async operations issued since the last Wait are complete	165	- `event.Wait(session)` — blocks until all async operations issued since the last Wait are complete
128		166
129	This means after multiple `TPUT_ASYNC` calls, a single `Wait` on the last returned `AsyncEvent` drains all pending operations (similar to shmem's quiet semantics).	167	This means after multiple `TPUT_ASYNC` calls, a single `Wait` on the last returned `AsyncEvent` drains all pending operations (similar to shmem's quiet semantics).
130		168
		169	+For RDMA operations targeting different peers, wait for the last event of each peer separately.
		170	+
131	Up to 64 operations may be outstanding in one session before submission can apply backpressure.	171	Up to 64 operations may be outstanding in one session before submission can apply backpressure.
132		172
133	After wait succeeds, all issued writes to `dstGlobalData` are complete.	173	After wait succeeds, all issued writes to `dstGlobalData` are complete.

Mdocs/isa/comm/TPUT_ASYNC_zh.md+40-0

	@@ -13,6 +13,7 @@
13	- `engine`：	13	- `engine`：
14	- `DmaEngine::SDMA`（默认）	14	- `DmaEngine::SDMA`（默认）
15	- `DmaEngine::URMA`（Ascend 950PR/Ascend 950DT，仅NPU_ARCH 3510）	15	- `DmaEngine::URMA`（Ascend 950PR/Ascend 950DT，仅NPU_ARCH 3510）
		16	+ - `DmaEngine::RDMA`（Ascend 950PR/Ascend 950DT，仅NPU_ARCH 3510；当前网卡平台仅支持 HNS1825）
16		17
17	> 注意（SDMA路径）	18	> 注意（SDMA路径）
18	> `TPUT_ASYNC` 配合 `DmaEngine::SDMA` 目前仅支持扁平连续的逻辑一维tensor。	19	> `TPUT_ASYNC` 配合 `DmaEngine::SDMA` 目前仅支持扁平连续的逻辑一维tensor。
	@@ -79,6 +80,39 @@ PTO_INTERNAL bool BuildAsyncSession(__gm__ uint8_t *workspace,
79		80
80	URMA不需要 `scratchTile`——轮询通过 `ld_dev`/`st_dev` 硬件原语直接操作。	81	URMA不需要 `scratchTile`——轮询通过 `ld_dev`/`st_dev` 硬件原语直接操作。
81		82
		83	+### RDMA构建（仅NPU_ARCH 3510）
		84	+
		85	+RDMA面向经典跨机场景组网，当前网卡平台仅支持 HNS1825。
		86	+
		87	+```cpp
		88	+#ifdef PTO_RDMA_SUPPORTED
		89	+template <DmaEngine engine, typename ScratchTile>
		90	+PTO_INTERNAL bool BuildAsyncSession(ScratchTile &scratchTile,
		91	+ __gm__ uint8_t *workspace,
		92	+ uint32_t myPe,
		93	+ AsyncSession &session,
		94	+ uint32_t syncId = 0);
		95	+#endif
		96	+```
		97	+
		98	+\| 参数 \| 说明 \|
		99	+\|---\|---\|
		100	+\| `scratchTile` \| 用于 WQE/CQE 控制数据的 UB/Vec tile，至少 64 字节。\|
		101	+\| `workspace` \| Host侧 RDMA 初始化流程返回的 GM 指针。\|
		102	+\| `myPe` \| 本地 rank id，用于选择已注册的本地内存区域。\|
		103	+\| `session` \| 输出的 `AsyncSession` 对象。\|
		104	+\| `syncId` \| MTE/Scalar 同步事件 ID，取值范围为 0-7。\|
		105	+
		106	+构建一次不绑定 peer 的Session，再通过显式 `peer` 参数选择目标rank：
		107	+
		108	+```cpp
		109	+comm::AsyncSession session;
		110	+if (comm::BuildAsyncSession<comm::DmaEngine::RDMA>(scratchTile, rdmaWorkspace, myPe, session, syncId)) {
		111	+ auto event = comm::TPUT_ASYNC<comm::DmaEngine::RDMA>(dstG, srcG, session, peer);
		112	+ (void)event.Wait(session);
		113	+}
		114	+```
		115	+
82	## 约束	116	## 约束
83		117
84	- `GlobalSrcData::RawDType == GlobalDstData::RawDType`	118	- `GlobalSrcData::RawDType == GlobalDstData::RawDType`
	@@ -90,6 +124,9 @@ URMA不需要 `scratchTile`——轮询通过 `ld_dev`/`st_dev` 硬件原语直
90	- URMA仅在NPU_ARCH 3510（Ascend 950PR/Ascend 950DT）上可用	124	- URMA仅在NPU_ARCH 3510（Ascend 950PR/Ascend 950DT）上可用
91	- URMA要求CANN Toolkit >= 9.1.0	125	- URMA要求CANN Toolkit >= 9.1.0
92	- 传给 `UrmaWorkspaceManager::Init()` 的对称数据缓冲区必须由大页内存支撑（使用 `ACL_MEM_MALLOC_HUGE_ONLY` 分配）。底层MR注册要求大页背景；`ACL_MEM_MALLOC_HUGE_FIRST` 在小尺寸分配时可能静默回退到4KB小页，导致注册失败	126	- 传给 `UrmaWorkspaceManager::Init()` 的对称数据缓冲区必须由大页内存支撑（使用 `ACL_MEM_MALLOC_HUGE_ONLY` 分配）。底层MR注册要求大页背景；`ACL_MEM_MALLOC_HUGE_FIRST` 在小尺寸分配时可能静默回退到4KB小页，导致注册失败
		127	+- RDMA仅在Ascend 950PR/Ascend 950DT（NPU_ARCH 3510）上可用，当前网卡平台仅支持 HNS1825
		128	+- RDMA源、目的tensor必须是扁平连续的逻辑一维，且本地和远端完整传输范围必须位于Host初始化阶段注册的内存区域内
		129	+- 单次RDMA传输不能超过 `0x7fffffff` 字节
93		130
94	若不满足一维连续要求，当前实现返回无效async event（`handle == 0`）。	131	若不满足一维连续要求，当前实现返回无效async event（`handle == 0`）。
95		132
	@@ -118,11 +155,14 @@ URMA不需要 `scratchTile`——轮询通过 `ld_dev`/`st_dev` 硬件原语直
118		155
119	- SDMA：每次`TPUT_ASYNC`都会提交数据传输SQE和用于标记本次操作完成的flag SQE。对其返回的Event调用`Wait`或`Test`时，通过轮询对应flag判断该次`TPUT_ASYNC`是否完成；完成后也能保证同一Session中此前提交的所有SDMA操作均已完成。	156	- SDMA：每次`TPUT_ASYNC`都会提交数据传输SQE和用于标记本次操作完成的flag SQE。对其返回的Event调用`Wait`或`Test`时，通过轮询对应flag判断该次`TPUT_ASYNC`是否完成；完成后也能保证同一Session中此前提交的所有SDMA操作均已完成。
120	- URMA：`TPUT_ASYNC` 立即提交RDMA WRITE WQE并敲门铃。`Wait` 通过轮询Completion Queue（CQ）等待所有预期的CQE被消费。	157	- URMA：`TPUT_ASYNC` 立即提交RDMA WRITE WQE并敲门铃。`Wait` 通过轮询Completion Queue（CQ）等待所有预期的CQE被消费。
		158	+- RDMA：`TPUT_ASYNC` 向 `peer` 选择的队列提交RDMA WRITE WQE；不同peer/queue分别跟踪完成状态。
121		159
122	- `event.Wait(session)` —阻塞，直到自上次Wait以来所有已发出的异步操作全部完成	160	- `event.Wait(session)` —阻塞，直到自上次Wait以来所有已发出的异步操作全部完成
123		161
124	这意味着多次 `TPUT_ASYNC` 调用后，只需对最后一个返回的 `AsyncEvent` 调用一次 `Wait`，即可等待所有pending操作完成（类似shmem的quiet语义）。	162	这意味着多次 `TPUT_ASYNC` 调用后，只需对最后一个返回的 `AsyncEvent` 调用一次 `Wait`，即可等待所有pending操作完成（类似shmem的quiet语义）。
125		163
		164	+RDMA操作涉及不同peer时，必须分别等待每个peer的最后一个Event。
		165	+
126	同一Session最多可有64个未完成操作，超过后提交可能产生背压。	166	同一Session最多可有64个未完成操作，超过后提交可能产生背压。
127		167
128	wait成功后，所有已发出的 `dstGlobalData` 写入均已全部完成。	168	wait成功后，所有已发出的 `dstGlobalData` 写入均已全部完成。

Minclude/pto/comm/README.md+4-4

Minclude/pto/comm/README_zh.md+4-4

Minclude/pto/comm/a5/async/TGetAsync.hpp+58-2

	@@ -15,6 +15,9 @@ See LICENSE in the root of the software repository for the full text of the Lice
15	#ifdef PTO_URMA_SUPPORTED	15	#ifdef PTO_URMA_SUPPORTED
16	#include "pto/comm/async/urma/urma_async_intrin.hpp"	16	#include "pto/comm/async/urma/urma_async_intrin.hpp"
17	#endif	17	#endif
		18	+#ifdef PTO_RDMA_SUPPORTED
		19	+#include "pto/comm/async/rdma/rdma_async_intrin.hpp"
		20	+#endif
18		21
19	namespace pto {	22	namespace pto {
20	namespace comm {	23	namespace comm {
	@@ -60,11 +63,50 @@ TGET_ASYNC_URMA_IMPL(GlobalDstData& dstGlobalData, GlobalSrcData& srcGlobalData,
60	}	63	}
61	#endif	64	#endif
62		65
		66	+#ifdef PTO_RDMA_SUPPORTED
		67	+// RDMA GET: async remote read. dst is local, src is remote.
		68	+template <typename GlobalDstData, typename GlobalSrcData>
		69	+PTO_INTERNAL AsyncEvent TGET_ASYNC_RDMA_IMPL(
		70	+ GlobalDstData& dstGlobalData, GlobalSrcData& srcGlobalData, const AsyncSession& session, uint32_t peer)
		71	+{
		72	+ (void)TGetAsyncCheckTensorCompatibility<GlobalDstData, GlobalSrcData>();
		73	+
		74	+ PTO_ASSERT(
		75	+ TGetAsyncIsFlatContiguous1D(srcGlobalData),
		76	+ "TGET_ASYNC RDMA: src tensor must be flat contiguous 1D (packed layout, single logical line).");
		77	+ PTO_ASSERT(
		78	+ TGetAsyncIsFlatContiguous1D(dstGlobalData),
		79	+ "TGET_ASYNC RDMA: dst tensor must be flat contiguous 1D (packed layout, single logical line).");
		80	+
		81	+ const uint32_t srcElems = TGetAsyncGetTotalElemCount(srcGlobalData);
		82	+ const uint32_t dstElems = TGetAsyncGetTotalElemCount(dstGlobalData);
		83	+ PTO_ASSERT(dstElems >= srcElems, "TGET_ASYNC RDMA: dst buffer too small for src data");
		84	+
		85	+ using T = typename GlobalSrcData::RawDType;
		86	+ const uint64_t transferSize = static_cast<uint64_t>(srcElems) * sizeof(T);
		87	+ if (transferSize == 0) {
		88	+ return AsyncEvent(0, DmaEngine::RDMA);
		89	+ }
		90	+
		91	+ const uint64_t eventHandle = rdma::Read(
		92	+ session, reinterpret_cast<__gm__ uint8_t*>(dstGlobalData.data()),
		93	+ reinterpret_cast<__gm__ uint8_t*>(srcGlobalData.data()), transferSize, peer);
		94	+ return AsyncEvent(eventHandle, DmaEngine::RDMA);
		95	+}
		96	+
		97	+template <typename GlobalDstData, typename GlobalSrcData>
		98	+PTO_INTERNAL AsyncEvent
		99	+TGET_ASYNC_RDMA_IMPL(GlobalDstData& dstGlobalData, GlobalSrcData& srcGlobalData, const AsyncSession& session)
		100	+{
		101	+ return TGET_ASYNC_RDMA_IMPL(dstGlobalData, srcGlobalData, session, session.destRankId);
		102	+}
		103	+#endif
		104	+
63	} // namespace detail	105	} // namespace detail
64		106
65	// ============================================================================	107	// ============================================================================
66	// Main TGET_ASYNC_IMPL with DmaEngine template parameter	108	// Main TGET_ASYNC_IMPL with DmaEngine template parameter
67	-// A5: SDMA and URMA engines are supported	109	+// A5: SDMA, URMA, and RDMA engines are supported
68	// ============================================================================	110	// ============================================================================
69		111
70	template <DmaEngine engine = DmaEngine::SDMA, typename GlobalDstData, typename GlobalSrcData>	112	template <DmaEngine engine = DmaEngine::SDMA, typename GlobalDstData, typename GlobalSrcData>
	@@ -79,6 +121,13 @@ TGET_ASYNC_IMPL(GlobalDstData& dstGlobalData, GlobalSrcData& srcGlobalData, cons
79	#else	121	#else
80	static_assert(engine != DmaEngine::URMA, "TGET_ASYNC: URMA engine requires NPU_ARCH 3510");	122	static_assert(engine != DmaEngine::URMA, "TGET_ASYNC: URMA engine requires NPU_ARCH 3510");
81	return AsyncEvent(0, engine);	123	return AsyncEvent(0, engine);
		124	+#endif
		125	+ } else if constexpr (engine == DmaEngine::RDMA) {
		126	+#ifdef PTO_RDMA_SUPPORTED
		127	+ return detail::TGET_ASYNC_RDMA_IMPL(dstGlobalData, srcGlobalData, session);
		128	+#else
		129	+ static_assert(engine != DmaEngine::RDMA, "TGET_ASYNC: RDMA support is not enabled for this build");
		130	+ return AsyncEvent(0, engine);
82	#endif	131	#endif
83	} else {	132	} else {
84	PTO_ASSERT(false, "TGET_ASYNC: unsupported engine");	133	PTO_ASSERT(false, "TGET_ASYNC: unsupported engine");
	@@ -86,7 +135,7 @@ TGET_ASYNC_IMPL(GlobalDstData& dstGlobalData, GlobalSrcData& srcGlobalData, cons
86	}	135	}
87	}	136	}
88		137
89	-// peer overload: URMA uses peer for SQ/CQ/MemInfo; SDMA ignores peer.	138	+// peer overload: URMA and RDMA use peer for queue/memory selection; SDMA ignores peer.
90	template <DmaEngine engine = DmaEngine::SDMA, typename GlobalDstData, typename GlobalSrcData>	139	template <DmaEngine engine = DmaEngine::SDMA, typename GlobalDstData, typename GlobalSrcData>
91	PTO_INTERNAL AsyncEvent	140	PTO_INTERNAL AsyncEvent
92	TGET_ASYNC_IMPL(GlobalDstData& dstGlobalData, GlobalSrcData& srcGlobalData, const AsyncSession& session, uint32_t peer)	141	TGET_ASYNC_IMPL(GlobalDstData& dstGlobalData, GlobalSrcData& srcGlobalData, const AsyncSession& session, uint32_t peer)
	@@ -99,6 +148,13 @@ TGET_ASYNC_IMPL(GlobalDstData& dstGlobalData, GlobalSrcData& srcGlobalData, cons
99	#else	148	#else
100	static_assert(engine != DmaEngine::URMA, "TGET_ASYNC: URMA engine requires NPU_ARCH 3510");	149	static_assert(engine != DmaEngine::URMA, "TGET_ASYNC: URMA engine requires NPU_ARCH 3510");
101	return AsyncEvent(0, engine);	150	return AsyncEvent(0, engine);
		151	+#endif
		152	+ } else if constexpr (engine == DmaEngine::RDMA) {
		153	+#ifdef PTO_RDMA_SUPPORTED
		154	+ return detail::TGET_ASYNC_RDMA_IMPL(dstGlobalData, srcGlobalData, session, peer);
		155	+#else
		156	+ static_assert(engine != DmaEngine::RDMA, "TGET_ASYNC: RDMA support is not enabled for this build");
		157	+ return AsyncEvent(0, engine);
102	#endif	158	#endif
103	} else {	159	} else {
104	PTO_ASSERT(false, "TGET_ASYNC: unsupported engine");	160	PTO_ASSERT(false, "TGET_ASYNC: unsupported engine");

Minclude/pto/comm/a5/async/TPutAsync.hpp+58-2

	@@ -15,6 +15,9 @@ See LICENSE in the root of the software repository for the full text of the Lice
15	#ifdef PTO_URMA_SUPPORTED	15	#ifdef PTO_URMA_SUPPORTED
16	#include "pto/comm/async/urma/urma_async_intrin.hpp"	16	#include "pto/comm/async/urma/urma_async_intrin.hpp"
17	#endif	17	#endif
		18	+#ifdef PTO_RDMA_SUPPORTED
		19	+#include "pto/comm/async/rdma/rdma_async_intrin.hpp"
		20	+#endif
18		21
19	namespace pto {	22	namespace pto {
20	namespace comm {	23	namespace comm {
	@@ -128,11 +131,50 @@ TPUT_ASYNC_URMA_IMPL(GlobalDstData& dstGlobalData, GlobalSrcData& srcGlobalData,
128	}	131	}
129	#endif	132	#endif
130		133
		134	+#ifdef PTO_RDMA_SUPPORTED
		135	+// RDMA PUT: async remote write. dst is remote, src is local.
		136	+template <typename GlobalDstData, typename GlobalSrcData>
		137	+PTO_INTERNAL AsyncEvent TPUT_ASYNC_RDMA_IMPL(
		138	+ GlobalDstData& dstGlobalData, GlobalSrcData& srcGlobalData, const AsyncSession& session, uint32_t peer)
		139	+{
		140	+ (void)TPutAsyncCheckTensorCompatibility<GlobalDstData, GlobalSrcData>();
		141	+
		142	+ PTO_ASSERT(
		143	+ TPutAsyncIsFlatContiguous1D(srcGlobalData),
		144	+ "TPUT_ASYNC RDMA: src tensor must be flat contiguous 1D (packed layout, single logical line).");
		145	+ PTO_ASSERT(
		146	+ TPutAsyncIsFlatContiguous1D(dstGlobalData),
		147	+ "TPUT_ASYNC RDMA: dst tensor must be flat contiguous 1D (packed layout, single logical line).");
		148	+
		149	+ const uint32_t srcElems = TPutAsyncGetTotalElemCount(srcGlobalData);
		150	+ const uint32_t dstElems = TPutAsyncGetTotalElemCount(dstGlobalData);
		151	+ PTO_ASSERT(dstElems >= srcElems, "TPUT_ASYNC RDMA: dst buffer too small for src data");
		152	+
		153	+ using T = typename GlobalSrcData::RawDType;
		154	+ const uint64_t transferSize = static_cast<uint64_t>(srcElems) * sizeof(T);
		155	+ if (transferSize == 0) {
		156	+ return AsyncEvent(0, DmaEngine::RDMA);
		157	+ }
		158	+
		159	+ const uint64_t eventHandle = rdma::Write(
		160	+ session, reinterpret_cast<__gm__ uint8_t*>(dstGlobalData.data()),
		161	+ reinterpret_cast<__gm__ uint8_t*>(srcGlobalData.data()), transferSize, peer);
		162	+ return AsyncEvent(eventHandle, DmaEngine::RDMA);
		163	+}
		164	+
		165	+template <typename GlobalDstData, typename GlobalSrcData>
		166	+PTO_INTERNAL AsyncEvent
		167	+TPUT_ASYNC_RDMA_IMPL(GlobalDstData& dstGlobalData, GlobalSrcData& srcGlobalData, const AsyncSession& session)
		168	+{
		169	+ return TPUT_ASYNC_RDMA_IMPL(dstGlobalData, srcGlobalData, session, session.destRankId);
		170	+}
		171	+#endif
		172	+
131	} // namespace detail	173	} // namespace detail
132		174
133	// ============================================================================	175	// ============================================================================
134	// Main TPUT_ASYNC_IMPL with DmaEngine template parameter	176	// Main TPUT_ASYNC_IMPL with DmaEngine template parameter
135	-// A5: SDMA uses MTE fallback for PUT direction; URMA is also supported	177	+// A5: SDMA uses MTE fallback for PUT direction; URMA and RDMA are also supported
136	// ============================================================================	178	// ============================================================================
137		179
138	template <DmaEngine engine = DmaEngine::SDMA, typename GlobalDstData, typename GlobalSrcData>	180	template <DmaEngine engine = DmaEngine::SDMA, typename GlobalDstData, typename GlobalSrcData>
	@@ -147,6 +189,13 @@ TPUT_ASYNC_IMPL(GlobalDstData& dstGlobalData, GlobalSrcData& srcGlobalData, cons
147	#else	189	#else
148	static_assert(engine != DmaEngine::URMA, "TPUT_ASYNC: URMA engine requires NPU_ARCH 3510");	190	static_assert(engine != DmaEngine::URMA, "TPUT_ASYNC: URMA engine requires NPU_ARCH 3510");
149	return AsyncEvent(0, engine);	191	return AsyncEvent(0, engine);
		192	+#endif
		193	+ } else if constexpr (engine == DmaEngine::RDMA) {
		194	+#ifdef PTO_RDMA_SUPPORTED
		195	+ return detail::TPUT_ASYNC_RDMA_IMPL(dstGlobalData, srcGlobalData, session);
		196	+#else
		197	+ static_assert(engine != DmaEngine::RDMA, "TPUT_ASYNC: RDMA support is not enabled for this build");
		198	+ return AsyncEvent(0, engine);
150	#endif	199	#endif
151	} else {	200	} else {
152	PTO_ASSERT(false, "TPUT_ASYNC: unsupported engine");	201	PTO_ASSERT(false, "TPUT_ASYNC: unsupported engine");
	@@ -154,7 +203,7 @@ TPUT_ASYNC_IMPL(GlobalDstData& dstGlobalData, GlobalSrcData& srcGlobalData, cons
154	}	203	}
155	}	204	}
156		205
157	-// peer overload: URMA uses peer for SQ/CQ/MemInfo; SDMA ignores peer.	206	+// peer overload: URMA and RDMA use peer for queue/memory selection; SDMA ignores peer.
158	template <DmaEngine engine = DmaEngine::SDMA, typename GlobalDstData, typename GlobalSrcData>	207	template <DmaEngine engine = DmaEngine::SDMA, typename GlobalDstData, typename GlobalSrcData>
159	PTO_INTERNAL AsyncEvent	208	PTO_INTERNAL AsyncEvent
160	TPUT_ASYNC_IMPL(GlobalDstData& dstGlobalData, GlobalSrcData& srcGlobalData, const AsyncSession& session, uint32_t peer)	209	TPUT_ASYNC_IMPL(GlobalDstData& dstGlobalData, GlobalSrcData& srcGlobalData, const AsyncSession& session, uint32_t peer)
	@@ -167,6 +216,13 @@ TPUT_ASYNC_IMPL(GlobalDstData& dstGlobalData, GlobalSrcData& srcGlobalData, cons
167	#else	216	#else
168	static_assert(engine != DmaEngine::URMA, "TPUT_ASYNC: URMA engine requires NPU_ARCH 3510");	217	static_assert(engine != DmaEngine::URMA, "TPUT_ASYNC: URMA engine requires NPU_ARCH 3510");
169	return AsyncEvent(0, engine);	218	return AsyncEvent(0, engine);
		219	+#endif
		220	+ } else if constexpr (engine == DmaEngine::RDMA) {
		221	+#ifdef PTO_RDMA_SUPPORTED
		222	+ return detail::TPUT_ASYNC_RDMA_IMPL(dstGlobalData, srcGlobalData, session, peer);
		223	+#else
		224	+ static_assert(engine != DmaEngine::RDMA, "TPUT_ASYNC: RDMA support is not enabled for this build");
		225	+ return AsyncEvent(0, engine);
170	#endif	226	#endif
171	} else {	227	} else {
172	PTO_ASSERT(false, "TPUT_ASYNC: unsupported engine");	228	PTO_ASSERT(false, "TPUT_ASYNC: unsupported engine");

Ainclude/pto/comm/async/rdma/rdma_workspace_manager.hpp+174-0

@@ -0,0 +1,174 @@
	1	+/**
	2	+Copyright (c) 2026 Huawei Technologies Co., Ltd.
	3	+This program is free software, you can redistribute it and/or modify it under the terms and conditions of
	4	+CANN Open Software License Agreement Version 2.0 (the "License").
	5	+Please refer to the License for details. You may not use this file except in compliance with the License.
	6	+THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED,
	7	+INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE.
	8	+See LICENSE in the root of the software repository for the full text of the License.
	9	+*/
	10	+
	11	+// RDMA host control plane. The backend selected at build time determines which
	12	+// manager creates and releases the underlying resources.
	13	+
	14	+#ifndef PTO_COMM_ASYNC_RDMA_WORKSPACE_MANAGER_HPP
	15	+#define PTO_COMM_ASYNC_RDMA_WORKSPACE_MANAGER_HPP
	16	+
	17	+#if defined(__CCE_KT_TEST__)
	18	+#error "rdma_workspace_manager.hpp is a host-only header and cannot be included in device code."
	19	+#endif
	20	+
	21	+#include <cstdint>
	22	+#include <iostream>
	23	+#include <string>
	24	+#include <vector>
	25	+
	26	+#include "pto/comm/rdma_backend.hpp"
	27	+#ifdef PTO_RDMA_BACKEND_HNS_1825_SUPPORTED
	28	+#include "pto/comm/async/rdma/backends/hns_1825/hns_1825_arch.hpp"
	29	+#include "pto/comm/async/rdma/backends/hns_1825/hns_1825_workspace_manager.hpp"
	30	+#endif
	31	+
	32	+namespace pto {
	33	+namespace comm {
	34	+namespace rdma {
	35	+
	36	+enum class WorkspaceInitResult : uint8_t {
	37	+ DISABLED = 0,
	38	+ READY,
	39	+ ERROR,
	40	+};
	41	+
	42	+struct WorkspaceConfig {
	43	+ uint32_t rankId{0};
	44	+ uint32_t rankCount{0};
	45	+ uint32_t phyId{0};
	46	+ std::string localIp;
	47	+ uint16_t basePort{60032};
	48	+ std::vector<std::string> peerIps;
	49	+ std::vector<uint32_t> peerPhyIds;
	50	+ std::vector<uint64_t> peerSymAddrs;
	51	+ void* symmetricAddr{nullptr};
	52	+ uint64_t symmetricSize{0};
	53	+};
	54	+
	55	+class RdmaWorkspaceManager {
	56	+public:
	57	+ RdmaWorkspaceManager() = default;
	58	+ ~RdmaWorkspaceManager() = default;
	59	+
	60	+ RdmaWorkspaceManager(const RdmaWorkspaceManager&) = delete;
	61	+ RdmaWorkspaceManager& operator=(const RdmaWorkspaceManager&) = delete;
	62	+
	63	+ static RdmaBackend ConfiguredBackend()
	64	+ {
	65	+#ifdef PTO_RDMA_BACKEND_HNS_1825_SUPPORTED
	66	+ return RdmaBackend::HNS_1825;
	67	+#else
	68	+ return RdmaBackend::NONE;
	69	+#endif
	70	+ }
	71	+
	72	+ static const char* ConfiguredBackendName()
	73	+ {
	74	+#ifdef PTO_RDMA_BACKEND_HNS_1825_SUPPORTED
	75	+ return "HNS_1825";
	76	+#else
	77	+ return "NONE";
	78	+#endif
	79	+ }
	80	+
	81	+ static WorkspaceInitResult Preflight()
	82	+ {
	83	+ const RdmaBackend backend = ConfiguredBackend();
	84	+ if (backend == RdmaBackend::NONE) {
	85	+ return WorkspaceInitResult::DISABLED;
	86	+ }
	87	+
	88	+ switch (backend) {
	89	+#ifdef PTO_RDMA_BACKEND_HNS_1825_SUPPORTED
	90	+ case RdmaBackend::HNS_1825: {
	91	+ const hns_1825::ArchitectureProbe arch = hns_1825::ProbeArchitecture();
	92	+ if (!arch.supported) {
	93	+ std::cerr << "[RDMA] the compiled HNS_1825 backend is only supported on A5; "
	94	+ "current architecture is "
	95	+ << hns_1825::DescribeArchitecture(arch) << std::endl;
	96	+ return WorkspaceInitResult::ERROR;
	97	+ }
	98	+ }
	99	+ return WorkspaceInitResult::READY;
	100	+#endif
	101	+ default:
	102	+ return WorkspaceInitResult::DISABLED;
	103	+ }
	104	+ }
	105	+
	106	+ WorkspaceInitResult Init(const WorkspaceConfig& config)
	107	+ {
	108	+ (void)config;
	109	+ const WorkspaceInitResult preflight = Preflight();
	110	+ if (preflight != WorkspaceInitResult::READY) {
	111	+ return preflight;
	112	+ }
	113	+
	114	+ switch (ConfiguredBackend()) {
	115	+#ifdef PTO_RDMA_BACKEND_HNS_1825_SUPPORTED
	116	+ case RdmaBackend::HNS_1825: {
	117	+ hns1825Backend_.SetTraceId(traceId_);
	118	+ const bool initialized = hns1825Backend_.Init(
	119	+ config.rankId, config.rankCount, config.phyId, config.localIp, config.basePort, config.peerIps,
	120	+ config.peerPhyIds, config.peerSymAddrs, config.symmetricAddr, config.symmetricSize);
	121	+ activeBackend_ = initialized ? RdmaBackend::HNS_1825 : RdmaBackend::NONE;
	122	+ return initialized ? WorkspaceInitResult::READY : WorkspaceInitResult::ERROR;
	123	+ }
	124	+#endif
	125	+ default:
	126	+ return WorkspaceInitResult::DISABLED;
	127	+ }
	128	+ }
	129	+
	130	+ bool Finalize()
	131	+ {
	132	+ bool ok = true;
	133	+ switch (activeBackend_) {
	134	+#ifdef PTO_RDMA_BACKEND_HNS_1825_SUPPORTED
	135	+ case RdmaBackend::HNS_1825:
	136	+ ok = hns1825Backend_.Finalize();
	137	+ break;
	138	+#endif
	139	+ default:
	140	+ break;
	141	+ }
	142	+ activeBackend_ = RdmaBackend::NONE;
	143	+ return ok;
	144	+ }
	145	+
	146	+ void SetTraceId(uint32_t traceId) { traceId_ = traceId; }
	147	+
	148	+ void* GetWorkspaceAddr() const
	149	+ {
	150	+ switch (activeBackend_) {
	151	+#ifdef PTO_RDMA_BACKEND_HNS_1825_SUPPORTED
	152	+ case RdmaBackend::HNS_1825:
	153	+ return hns1825Backend_.GetWorkspaceAddr();
	154	+#endif
	155	+ default:
	156	+ return nullptr;
	157	+ }
	158	+ }
	159	+
	160	+ RdmaBackend ActiveBackend() const { return activeBackend_; }
	161	+
	162	+private:
	163	+ RdmaBackend activeBackend_{RdmaBackend::NONE};
	164	+ uint32_t traceId_{0};
	165	+#ifdef PTO_RDMA_BACKEND_HNS_1825_SUPPORTED
	166	+ hns_1825::WorkspaceManager hns1825Backend_;
	167	+#endif
	168	+};
	169	+
	170	+} // namespace rdma
	171	+} // namespace comm
	172	+} // namespace pto
	173	+
	174	+#endif // PTO_COMM_ASYNC_RDMA_WORKSPACE_MANAGER_HPP

Minclude/pto/comm/async_common/async_event_impl.hpp+45-1

	@@ -17,6 +17,9 @@ See LICENSE in the root of the software repository for the full text of the Lice
17	#ifdef PTO_URMA_SUPPORTED	17	#ifdef PTO_URMA_SUPPORTED
18	#include "pto/comm/async/urma/urma_async_intrin.hpp"	18	#include "pto/comm/async/urma/urma_async_intrin.hpp"
19	#endif	19	#endif
		20	+#ifdef PTO_RDMA_SUPPORTED
		21	+#include "pto/comm/async/rdma/rdma_async_intrin.hpp"
		22	+#endif
20		23
21	namespace pto {	24	namespace pto {
22	namespace comm {	25	namespace comm {
	@@ -34,7 +37,7 @@ PTO_INTERNAL bool BuildAsyncSession(
34	} else {	37	} else {
35	static_assert(	38	static_assert(
36	engine == DmaEngine::SDMA,	39	engine == DmaEngine::SDMA,
37	- "This overload is for SDMA; use the URMA-specific BuildAsyncSession for DmaEngine::URMA");	40	+ "This overload is for SDMA; use the engine-specific BuildAsyncSession overload for URMA or RDMA");
38	return false;	41	return false;
39	}	42	}
40	}	43	}
	@@ -65,6 +68,39 @@ PTO_INTERNAL bool BuildAsyncSession(__gm__ uint8_t* workspace, uint32_t destRank
65	}	68	}
66	#endif	69	#endif
67		70
		71	+#ifdef PTO_RDMA_SUPPORTED
		72	+// Peer-independent RDMA session builder. The explicit-peer TPUT_ASYNC and
		73	+// TGET_ASYNC overloads can reuse this session for multiple remote ranks.
		74	+template <DmaEngine engine, typename ScratchTile>
		75	+PTO_INTERNAL bool BuildAsyncSession(
		76	+ ScratchTile& scratchTile, __gm__ uint8_t* workspace, uint32_t myPe, AsyncSession& session, uint32_t syncId = 0)
		77	+{
		78	+ static_assert(engine == DmaEngine::RDMA, "This overload is for RDMA only");
		79	+ rdma::RdmaTmpBuffer tmpBuf{};
		80	+ if (!rdma::detail::MakeTmpBufferFromTile(scratchTile, tmpBuf)) {
		81	+ session = AsyncSession{};
		82	+ return false;
		83	+ }
		84	+ return rdma::BuildSession(workspace, myPe, tmpBuf, syncId, session);
		85	+}
		86	+
		87	+// Peer-bound RDMA builder retained for callers that use the original async
		88	+// overload without an explicit peer.
		89	+template <DmaEngine engine, typename ScratchTile>
		90	+PTO_INTERNAL bool BuildAsyncSession(
		91	+ ScratchTile& scratchTile, __gm__ uint8_t* workspace, uint32_t destRankId, uint32_t myPe, AsyncSession& session,
		92	+ uint32_t syncId = 0)
		93	+{
		94	+ static_assert(engine == DmaEngine::RDMA, "This overload is for RDMA only");
		95	+ rdma::RdmaTmpBuffer tmpBuf{};
		96	+ if (!rdma::detail::MakeTmpBufferFromTile(scratchTile, tmpBuf)) {
		97	+ session = AsyncSession{};
		98	+ return false;
		99	+ }
		100	+ return rdma::BuildSession(workspace, destRankId, myPe, tmpBuf, syncId, session);
		101	+}
		102	+#endif
		103	+
68	// ============================================================================	104	// ============================================================================
69	// AsyncEvent::Wait / Test — AsyncSession overloads (primary user API)	105	// AsyncEvent::Wait / Test — AsyncSession overloads (primary user API)
70	// ============================================================================	106	// ============================================================================
	@@ -80,6 +116,10 @@ PTO_INTERNAL bool AsyncEvent::Wait(const AsyncSession& session) const
80	#ifdef PTO_URMA_SUPPORTED	116	#ifdef PTO_URMA_SUPPORTED
81	case DmaEngine::URMA:	117	case DmaEngine::URMA:
82	return urma::detail::UrmaWaitEvent(handle, session);	118	return urma::detail::UrmaWaitEvent(handle, session);
		119	+#endif
		120	+#ifdef PTO_RDMA_SUPPORTED
		121	+ case DmaEngine::RDMA:
		122	+ return rdma::WaitEvent(handle, session);
83	#endif	123	#endif
84	default:	124	default:
85	return false;	125	return false;
	@@ -97,6 +137,10 @@ PTO_INTERNAL bool AsyncEvent::Test(const AsyncSession& session) const
97	#ifdef PTO_URMA_SUPPORTED	137	#ifdef PTO_URMA_SUPPORTED
98	case DmaEngine::URMA:	138	case DmaEngine::URMA:
99	return urma::detail::UrmaTestEvent(handle, session);	139	return urma::detail::UrmaTestEvent(handle, session);
		140	+#endif
		141	+#ifdef PTO_RDMA_SUPPORTED
		142	+ case DmaEngine::RDMA:
		143	+ return rdma::TestEvent(handle, session);
100	#endif	144	#endif
101	default:	145	default:
102	return false;	146	return false;

Minclude/pto/comm/async_common/async_types.hpp+3-0

	@@ -145,6 +145,9 @@ struct AsyncSession {
145		145
146	uint32_t destRankId{0};	146	uint32_t destRankId{0};
147	uint32_t qpIdx{0};	147	uint32_t qpIdx{0};
		148	+
		149	+ RdmaBackend rdmaBackend{RdmaBackend::NONE};
		150	+ uint32_t myPe{0};
148	};	151	};
149		152
150	} // namespace comm	153	} // namespace comm

Minclude/pto/comm/comm_types.hpp+2-0

	@@ -17,6 +17,7 @@ See LICENSE in the root of the software repository for the full text of the Lice
17	#include "pto/common/debug.h"	17	#include "pto/common/debug.h"
18	#include "pto/common/type.hpp"	18	#include "pto/common/type.hpp"
19	#include "pto/common/pto_tile.hpp"	19	#include "pto/common/pto_tile.hpp"
		20	+#include "pto/comm/rdma_backend.hpp"
20		21
21	namespace pto {	22	namespace pto {
22	namespace comm {	23	namespace comm {
	@@ -121,6 +122,7 @@ enum class ReduceOp : uint8_t {
121	enum class DmaEngine : uint8_t {	122	enum class DmaEngine : uint8_t {
122	SDMA = 0, // Supports 2D transfer	123	SDMA = 0, // Supports 2D transfer
123	URMA = 1, // Supports 1D transfer (HCCP V2 Jetty, NPU_ARCH 3510 only)	124	URMA = 1, // Supports 1D transfer (HCCP V2 Jetty, NPU_ARCH 3510 only)
		125	+ RDMA = 2, // RDMA engine; RdmaBackend identifies the NIC implementation compiled into the binary
124	};	126	};
125		127
126	// ============================================================================	128	// ============================================================================

Minclude/pto/comm/pto_comm_inst.hpp+2-2

	@@ -352,7 +352,7 @@ PTO_INST AsyncEvent TPUT_ASYNC(
352	/**	352	/**
353	* @brief Asynchronous remote write with explicit peer (A5 / CPU stub).	353	* @brief Asynchronous remote write with explicit peer (A5 / CPU stub).
354	*	354	*
355	- * For URMA: @p peer selects SQ/CQ/MR in UrmaInfo (session need not bind peer).	355	+ * For URMA and RDMA: @p peer selects the per-peer queue and memory metadata (session need not bind peer).
356	* For SDMA: @p peer is ignored; addressing comes from the GlobalTensor VA.	356	* For SDMA: @p peer is ignored; addressing comes from the GlobalTensor VA.
357	* For CPU: @p peer is ignored; use default implementation	357	* For CPU: @p peer is ignored; use default implementation
358	*/	358	*/
	@@ -383,7 +383,7 @@ PTO_INST AsyncEvent TGET_ASYNC(
383	/**	383	/**
384	* @brief Asynchronous remote read with explicit peer (A5 / CPU stub).	384	* @brief Asynchronous remote read with explicit peer (A5 / CPU stub).
385	*	385	*
386	- * For URMA: @p peer selects SQ/CQ/MR in UrmaInfo (session need not bind peer).	386	+ * For URMA and RDMA: @p peer selects the per-peer queue and memory metadata (session need not bind peer).
387	* For SDMA: @p peer is ignored; addressing comes from the GlobalTensor VA.	387	* For SDMA: @p peer is ignored; addressing comes from the GlobalTensor VA.
388	* For CPU: @p peer is ignored; use default implementation	388	* For CPU: @p peer is ignored; use default implementation
389	*/	389	*/

Ainclude/pto/comm/rdma_backend.hpp+29-0

@@ -0,0 +1,29 @@
	1	+/**
	2	+Copyright (c) 2026 Huawei Technologies Co., Ltd.
	3	+This program is free software, you can redistribute it and/or modify it under the terms and conditions of
	4	+CANN Open Software License Agreement Version 2.0 (the "License").
	5	+Please refer to the License for details. You may not use this file except in compliance with the License.
	6	+THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED,
	7	+INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE.
	8	+See LICENSE in the root of the software repository for the full text of the License.
	9	+*/
	10	+
	11	+#ifndef PTO_COMM_RDMA_BACKEND_HPP
	12	+#define PTO_COMM_RDMA_BACKEND_HPP
	13	+
	14	+#include <cstdint>
	15	+
	16	+namespace pto {
	17	+namespace comm {
	18	+
	19	+// A process may activate at most one RDMA backend. NONE means that no RDMA
	20	+// control plane has been initialized for the session.
	21	+enum class RdmaBackend : uint32_t {
	22	+ NONE = 0,
	23	+ HNS_1825 = 1,
	24	+};
	25	+
	26	+} // namespace comm
	27	+} // namespace pto
	28	+
	29	+#endif // PTO_COMM_RDMA_BACKEND_HPP

Apkg_inc/pto/comm/async/rdma/backends/hns_1825/hns_1825_arch.hpp+85-0

@@ -0,0 +1,85 @@
	1	+/**
	2	+Copyright (c) 2026 Huawei Technologies Co., Ltd.
	3	+This program is free software, you can redistribute it and/or modify it under the terms and conditions of
	4	+CANN Open Software License Agreement Version 2.0 (the "License").
	5	+Please refer to the License for details. You may not use this file except in compliance with the License.
	6	+THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED,
	7	+INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE.
	8	+See LICENSE in the root of the software repository for the full text of the License.
	9	+*/
	10	+
	11	+#ifndef PTO_COMM_ASYNC_RDMA_BACKENDS_HNS_1825_ARCH_HPP
	12	+#define PTO_COMM_ASYNC_RDMA_BACKENDS_HNS_1825_ARCH_HPP
	13	+
	14	+#if defined(__CCE_KT_TEST__)
	15	+#error "hns_1825_arch.hpp is a host-only header and cannot be included in device code."
	16	+#endif
	17	+
	18	+#include <dlfcn.h>
	19	+
	20	+#include <cstdint>
	21	+#include <string>
	22	+
	23	+namespace pto {
	24	+namespace comm {
	25	+namespace rdma {
	26	+namespace hns_1825 {
	27	+
	28	+struct ArchitectureProbe {
	29	+ uint32_t compiledNpuArch{0};
	30	+ std::string runtimeSocName;
	31	+ bool runtimeResolved{false};
	32	+ bool supported{false};
	33	+};
	34	+
	35	+// Accept HNS1825 only when every available signal agrees with 3510. Host-only
	36	+// translation units may not define __NPU_ARCH__, so the runtime SoC remains
	37	+// authoritative in that case.
	38	+inline ArchitectureProbe ProbeArchitecture()
	39	+{
	40	+ ArchitectureProbe result{};
	41	+#if defined(__NPU_ARCH__) && (__NPU_ARCH__ == 3510)
	42	+ result.compiledNpuArch = 3510;
	43	+#elif defined(__NPU_ARCH__)
	44	+ result.compiledNpuArch = static_cast<uint32_t>(__NPU_ARCH__);
	45	+#endif
	46	+
	47	+ using GetSocNameFn = const char* (*)();
	48	+ auto getSocName = reinterpret_cast<GetSocNameFn>(dlsym(RTLD_DEFAULT, "aclrtGetSocName"));
	49	+ if (getSocName != nullptr) {
	50	+ const char* name = getSocName();
	51	+ if (name != nullptr && name[0] != '\0') {
	52	+ result.runtimeSocName = name;
	53	+ result.runtimeResolved = true;
	54	+ const bool runtimeIs3510 = result.runtimeSocName.find("Ascend950") != std::string::npos \|\|
	55	+ result.runtimeSocName.find("dav-3510") != std::string::npos \|\|
	56	+ result.runtimeSocName.find("3510") != std::string::npos;
	57	+ result.supported = runtimeIs3510 && (result.compiledNpuArch == 0 \|\| result.compiledNpuArch == 3510);
	58	+ return result;
	59	+ }
	60	+ }
	61	+
	62	+ result.supported = result.compiledNpuArch == 3510;
	63	+ return result;
	64	+}
	65	+
	66	+inline std::string DescribeArchitecture(const ArchitectureProbe& probe)
	67	+{
	68	+ if (probe.runtimeResolved) {
	69	+ if (probe.compiledNpuArch != 0) {
	70	+ return probe.runtimeSocName + " (build NPU_ARCH " + std::to_string(probe.compiledNpuArch) + ')';
	71	+ }
	72	+ return probe.runtimeSocName + " (build NPU_ARCH unavailable)";
	73	+ }
	74	+ if (probe.compiledNpuArch != 0) {
	75	+ return std::string("NPU_ARCH ") + std::to_string(probe.compiledNpuArch) + " (runtime SoC unavailable)";
	76	+ }
	77	+ return "unknown (runtime SoC and NPU_ARCH unavailable)";
	78	+}
	79	+
	80	+} // namespace hns_1825
	81	+} // namespace rdma
	82	+} // namespace comm
	83	+} // namespace pto
	84	+
	85	+#endif // PTO_COMM_ASYNC_RDMA_BACKENDS_HNS_1825_ARCH_HPP

Apkg_inc/pto/comm/async/rdma/backends/hns_1825/hns_1825_backend.hpp+538-0

@@ -0,0 +1,538 @@
	1	+/**
	2	+Copyright (c) 2026 Huawei Technologies Co., Ltd.
	3	+This program is free software, you can redistribute it and/or modify it under the terms and conditions of
	4	+CANN Open Software License Agreement Version 2.0 (the "License").
	5	+Please refer to the License for details. You may not use this file except in compliance with the License.
	6	+THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED,
	7	+INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE.
	8	+See LICENSE in the root of the software repository for the full text of the License.
	9	+*/
	10	+
	11	+// HNS_1825 (Hi1825) RoCE RDMA device-side backend for A5.
	12	+//
	13	+// Posts RDMA WRITE / READ from AIV using the RdmaInfo table supplied via RdmaExecContext.
	14	+// WQE/CQE staging uses the RDMA session's UB scratch; atomics are not supported.
	15	+
	16	+#ifndef PTO_COMM_ASYNC_RDMA_BACKENDS_HNS_1825_BACKEND_HPP
	17	+#define PTO_COMM_ASYNC_RDMA_BACKENDS_HNS_1825_BACKEND_HPP
	18	+
	19	+#include "pto/pto-inst.hpp"
	20	+#include "pto/comm/async_common/async_types.hpp"
	21	+#include "pto/comm/async/rdma/rdma_device_common.hpp"
	22	+#include "pto/comm/async/rdma/backends/hns_1825/hns_1825_types.hpp"
	23	+// AscendC::GetSystemCycle only — do NOT include full kernel_operator.h here.
	24	+// That pulls adv_api/hccl (`using namespace HcclApi`) and collides with ST
	25	+// common.hpp symbols (MAX_CC_TILING_NUM / GROUP_NAME_SIZE / ALG_CONFIG_SIZE).
	26	+#if !defined(__CPU_SIM) && !defined(__COSTMODEL)
	27	+#include "kernel_operator_sys_var_intf.h"
	28	+#endif
	29	+
	30	+namespace pto {
	31	+namespace comm {
	32	+namespace rdma {
	33	+namespace hns_1825 {
	34	+namespace detail {
	35	+
	36	+constexpr uint32_t kCqeReadSize = sizeof(Hns1825Cqe);
	37	+
	38	+struct PostSendResult {
	39	+ uint32_t curHead;
	40	+ uint32_t status;
	41	+};
	42	+
	43	+// NIC consumes big-endian; AscendC is little-endian. SQ/CQ head-tail mirrors stay host-order;
	44	+// only NIC-visible payloads (WQE dwords / software doorbell) are byte-swapped.
	45	+AICORE inline uint16_t Htobe16(uint16_t v) { return (uint16_t)(((v & 0x00ffU) << 8) \| ((v & 0xff00U) >> 8)); }
	46	+
	47	+AICORE inline uint32_t Htobe32(uint32_t v)
	48	+{
	49	+ return ((v & 0x000000ffU) << 24) \| ((v & 0x0000ff00U) << 8) \| ((v & 0x00ff0000U) >> 8) \| ((v & 0xff000000U) >> 24);
	50	+}
	51	+
	52	+AICORE inline uint64_t Htobe64(uint64_t v)
	53	+{
	54	+ return ((v & 0x00000000000000ffULL) << 56) \| ((v & 0x000000000000ff00ULL) << 40) \|
	55	+ ((v & 0x0000000000ff0000ULL) << 24) \| ((v & 0x00000000ff000000ULL) << 8) \|
	56	+ ((v & 0x000000ff00000000ULL) >> 8) \| ((v & 0x0000ff0000000000ULL) >> 24) \|
	57	+ ((v & 0x00ff000000000000ULL) >> 40) \| ((v & 0xff00000000000000ULL) >> 56);
	58	+}
	59	+
	60	+// Host-order u32 mirror access (SQ/CQ head-tail): bypass the scalar data cache.
	61	+AICORE inline uint32_t ReadU32Gm(uint64_t addr) { return ld_dev((__gm__ uint32_t*)addr, 0); }
	62	+
	63	+AICORE inline void WriteU32Gm(uint64_t addr, uint32_t value) { st_dev(value, (__gm__ uint32_t*)addr, 0); }
	64	+
	65	+// Order scalar UB writes before MTE3 copies a NIC-visible record (WQE / software doorbell) to GM, then order the
	66	+// MTE3 store before the subsequent scalar dcci / st_dev. Post-copy sync MUST be MTE3->Scalar (not MTE3->MTE2):
	67	+// RoCE consumers are scalar NIC doorbell paths.
	68	+AICORE inline void WriteUbToGmWithSync(uint64_t gmAddr, __ubuf__ uint8_t* ub, uint32_t size, uint32_t syncId)
	69	+{
	70	+ set_flag(PIPE_S, PIPE_MTE3, syncId);
	71	+ wait_flag(PIPE_S, PIPE_MTE3, syncId);
	72	+ copy_ubuf_to_gm_align_v2(
	73	+ reinterpret_cast<__gm__ uint32_t>(gmAddr), reinterpret_cast<__ubuf__ uint32_t>(ub), 0, 1, size, 0, 0, 0);
	74	+ set_flag(PIPE_MTE3, PIPE_S, syncId);
	75	+ wait_flag(PIPE_MTE3, PIPE_S, syncId);
	76	+}
	77	+
	78	+// Copy `size` bytes from GM to the UB scratch, then make them scalar-readable.
	79	+AICORE inline void ReadGmToUb(__ubuf__ uint8_t* ub, uint64_t gmAddr, uint32_t size, uint32_t syncId)
	80	+{
	81	+ copy_gm_to_ubuf_align_v2(
	82	+ reinterpret_cast<__ubuf__ uint32_t>(ub), reinterpret_cast<__gm__ uint32_t>(gmAddr), 0, 1, size, 0, 0, 0, 0, 0,
	83	+ 0);
	84	+ set_flag(PIPE_MTE2, PIPE_S, syncId);
	85	+ wait_flag(PIPE_MTE2, PIPE_S, syncId);
	86	+}
	87	+
	88	+// SQ ring slot (one 64B WQEBB).
	89	+AICORE inline __gm__ uint8_t* GetSendWqe(__gm__ RoceSqCtx* sqCtx, uint32_t idx)
	90	+{
	91	+ return (__gm__ uint8_t)sqCtx->bufAddr + (uint64_t)idx kHns1825WqebbSize;
	92	+}
	93	+
	94	+// A CQE is ready when its owner bit matches the lap parity from the current consumer index.
	95	+AICORE inline bool CheckCqeOwner(__ubuf__ Hns1825Cqe* cqe, uint32_t curTail, uint32_t cqRing)
	96	+{
	97	+ constexpr uint32_t kOwnerShift = 31; // owner bit at dw1[31]
	98	+ uint32_t curOwner = ((cqe->owner_id_qpn & (1U << kOwnerShift)) != 0);
	99	+ uint32_t expectOwner = (uint32_t)(((curTail & cqRing) == 0));
	100	+ return (expectOwner ^ curOwner) != 0;

Apkg_inc/pto/comm/async/rdma/backends/hns_1825/hns_1825_hcomm_defs.hpp+277-0

@@ -0,0 +1,277 @@
	1	+/**
	2	+Copyright (c) 2026 Huawei Technologies Co., Ltd.
	3	+This program is free software, you can redistribute it and/or modify it under the terms and conditions of
	4	+CANN Open Software License Agreement Version 2.0 (the "License").
	5	+Please refer to the License for details. You may not use this file except in compliance with the License.
	6	+THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED,
	7	+INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE.
	8	+See LICENSE in the root of the software repository for the full text of the License.
	9	+*/
	10	+
	11	+// Host-only hcomm ABI definitions for the RoCE (HNS_1825) control plane.
	12	+//
	13	+// Public hcomm types come from <hcomm/hcomm_res.h>. ChannelEntity / SqContext /
	14	+// CqContext / RegedBufferEntity are not in the public include path; they are
	15	+// defined here ABI-compatible with hcomm_res_entity_defs.h. PTO supports the
	16	+// split-doorbell and vendor-specified RoCE context layouts used by
	17	+// HNS1825; legacy HCOMM V1 contexts are outside this backend's scope.
	18	+
	19	+#ifndef PTO_COMM_ASYNC_RDMA_BACKENDS_HNS_1825_HCOMM_DEFS_HPP
	20	+#define PTO_COMM_ASYNC_RDMA_BACKENDS_HNS_1825_HCOMM_DEFS_HPP
	21	+
	22	+#if defined(__CCE_KT_TEST__)
	23	+#error "hns_1825_hcomm_defs.hpp is a host-only header and cannot be included in device code."
	24	+#endif
	25	+
	26	+#include <hcomm/hcomm_res.h>
	27	+
	28	+#include <cstddef>
	29	+#include <cstdint>
	30	+
	31	+#include "securec.h"
	32	+
	33	+namespace pto {
	34	+namespace comm {
	35	+namespace rdma {
	36	+namespace hns_1825 {
	37	+namespace host {
	38	+
	39	+enum ProtectionType {
	40	+ PROTECTION_TYPE_RESERVED = -1,
	41	+ PROTECTION_TYPE_ROCE = 0,
	42	+ PROTECTION_TYPE_UB = 1,
	43	+};
	44	+
	45	+enum RegedBufferType {
	46	+ REGED_BUFFER_TYPE_RESERVED = -1,
	47	+ REGED_BUFFER_TYPE_IPC = 0,
	48	+ REGED_BUFFER_TYPE_RMA = 1,
	49	+};
	50	+
	51	+enum RegedNotifyType {
	52	+ REGED_NOTIFY_TYPE_RESERVED = -1,
	53	+ REGED_NOTIFY_TYPE_IPC_RT = 0,
	54	+ REGED_NOTIFY_TYPE_IPC_MEM = 1,
	55	+ REGED_NOTIFY_TYPE_RMA_RT = 2,
	56	+ REGED_NOTIFY_TYPE_RMA_MEM = 3,
	57	+};
	58	+
	59	+enum SqContextType {
	60	+ SQ_CONTEXT_TYPE_RESERVED = -1,
	61	+ SQ_CONTEXT_TYPE_UB_JFS = 0,
	62	+ SQ_CONTEXT_TYPE_ROCE = 1,
	63	+};
	64	+
	65	+enum CqContextType {
	66	+ CQ_CONTEXT_TYPE_RESERVED = -1,
	67	+ CQ_CONTEXT_TYPE_UB_JFC = 0,
	68	+ CQ_CONTEXT_TYPE_ROCE = 1,
	69	+};
	70	+
	71	+struct ProtectionInfo {
	72	+ ProtectionType type;
	73	+ union {
	74	+ struct {
	75	+ uint32_t lkey;
	76	+ uint32_t rkey;
	77	+ } roce;
	78	+ struct {
	79	+ uint32_t tokenId;
	80	+ uint32_t tokenValue;
	81	+ } ub;
	82	+ uint8_t raws[24];
	83	+ } memInfo;
	84	+};
	85	+
	86	+struct RegedBufferEntity {
	87	+ RegedBufferType type;
	88	+ union {
	89	+ struct {
	90	+ uint64_t addr;
	91	+ uint64_t size;
	92	+ } ipc;
	93	+ struct {
	94	+ uint64_t addr;
	95	+ uint64_t size;
	96	+ ProtectionInfo protectionInfo;
	97	+ } rma;
	98	+ uint8_t raws[56];
	99	+ } bufferInfo;
	100	+};
	101	+
	102	+struct RegedNotifyEntity {
	103	+ RegedNotifyType type;
	104	+ union {
	105	+ struct {
	106	+ uint64_t addr;
	107	+ uint32_t size;
	108	+ int32_t notifyId;
	109	+ } ipcRt;
	110	+ struct {
	111	+ uint64_t addr;
	112	+ uint32_t size;
	113	+ } ipcMem;
	114	+ struct {
	115	+ uint64_t addr;
	116	+ uint32_t size;
	117	+ int32_t notifyId;
	118	+ ProtectionInfo protectionInfo;
	119	+ } rmaRt;
	120	+ struct {
	121	+ uint64_t addr;
	122	+ uint32_t size;
	123	+ ProtectionInfo protectionInfo;
	124	+ } rmaMem;
	125	+ uint8_t raws[56];
	126	+ } notifyInfo;
	127	+};
	128	+
	129	+// HCOMM has shipped both a split-doorbell layout and a vendor-specified QoS
	130	+// layout without adding an ABI tag to these private contexts. Keep explicit
	131	+// byte-layout views and decode from contextInfo.raws below.
	132	+#define PTO_HNS1825_ROCE_SQ_CONTEXT_COMMON_FIELDS \
	133	+ uint64_t sqVa; \
	134	+ uint64_t headAddr; \
	135	+ uint64_t tailAddr; \
	136	+ uint64_t dbHwVa; \
	137	+ uint64_t dbSwVa; \
	138	+ uint32_t qpn; \
	139	+ uint32_t wqeSize; \
	140	+ uint32_t depth; \
	141	+ uint8_t sl
	142	+
	143	+struct RoceSqContextSplitDb {
	144	+ PTO_HNS1825_ROCE_SQ_CONTEXT_COMMON_FIELDS;
	145	+ uint8_t mtuShift;
	146	+};
	147	+
	148	+struct RoceSqContextVendorSpecified {
	149	+ PTO_HNS1825_ROCE_SQ_CONTEXT_COMMON_FIELDS;
	150	+ uint64_t DbVendorSpecified;
	151	+};
	152	+
	153	+#undef PTO_HNS1825_ROCE_SQ_CONTEXT_COMMON_FIELDS
	154	+
	155	+#define PTO_HNS1825_ROCE_CQ_CONTEXT_COMMON_FIELDS \
	156	+ uint64_t cqVa; \
	157	+ uint64_t headAddr; \
	158	+ uint64_t tailAddr; \
	159	+ uint64_t dbHwVa; \
	160	+ uint64_t dbSwVa; \
	161	+ uint32_t cqn; \
	162	+ uint32_t cqeSize; \
	163	+ uint32_t cqDepth
	164	+
	165	+struct RoceCqContextSplitDb {
	166	+ PTO_HNS1825_ROCE_CQ_CONTEXT_COMMON_FIELDS;
	167	+};
	168	+
	169	+struct RoceCqContextVendorSpecified {
	170	+ PTO_HNS1825_ROCE_CQ_CONTEXT_COMMON_FIELDS;
	171	+ uint64_t DbVendorSpecified;
	172	+};
	173	+
	174	+#undef PTO_HNS1825_ROCE_CQ_CONTEXT_COMMON_FIELDS
	175	+
	176	+struct SqContext {
	177	+ SqContextType type;
	178	+ union {
	179	+ struct {
	180	+ uint64_t sqVa;
	181	+ uint64_t headAddr;
	182	+ uint64_t tailAddr;
	183	+ uint64_t dbVa;
	184	+ uint32_t jfsID;
	185	+ uint32_t wqeSize;
	186	+ uint32_t sqDepth;
	187	+ uint32_t tpID;
	188	+ uint8_t remoteEID[16];
	189	+ } ubJfs;
	190	+ RoceSqContextVendorSpecified roceSq;
	191	+ uint8_t raws[120];
	192	+ } contextInfo;
	193	+};
	194	+
	195	+struct CqContext {
	196	+ CqContextType type;
	197	+ union {
	198	+ struct {
	199	+ uint64_t scqVa;
	200	+ uint64_t headAddr;
	201	+ uint64_t tailAddr;
	202	+ uint64_t dbVa;
	203	+ uint32_t jfcID;
	204	+ uint32_t cqeSize;
	205	+ uint32_t cqDepth;
	206	+ } ubJfc;
	207	+ RoceCqContextVendorSpecified roceCq;
	208	+ uint8_t raws[120];
	209	+ } contextInfo;
	210	+};
	211	+
	212	+static_assert(offsetof(RoceSqContextSplitDb, mtuShift) == 53, "unexpected split-db SQ context layout");
	213	+static_assert(
	214	+ offsetof(RoceSqContextVendorSpecified, DbVendorSpecified) == 56, "unexpected vendor-specified SQ context layout");
	215	+static_assert(
	216	+ offsetof(RoceCqContextVendorSpecified, DbVendorSpecified) == 56, "unexpected vendor-specified CQ context layout");
	217	+
	218	+template <typename RoceContext, typename Context>
	219	+inline RoceContext ExtractRoceContext(const Context& context)
	220	+{
	221	+ RoceContext roce{};
	222	+ static_assert(sizeof(roce) <= sizeof(context.contextInfo.raws), "ROCE context view is too large");
	223	+ (void)memcpy_s(&roce, sizeof(roce), context.contextInfo.raws, sizeof(roce));
	224	+ return roce;
	225	+}
	226	+
	227	+inline RoceSqContextSplitDb ExtractRoceSqContextSplitDb(const SqContext& context)
	228	+{
	229	+ return ExtractRoceContext<RoceSqContextSplitDb>(context);
	230	+}
	231	+
	232	+inline RoceSqContextVendorSpecified ExtractRoceSqContextVendorSpecified(const SqContext& context)
	233	+{
	234	+ return ExtractRoceContext<RoceSqContextVendorSpecified>(context);
	235	+}
	236	+
	237	+inline RoceCqContextSplitDb ExtractRoceCqContextSplitDb(const CqContext& context)
	238	+{
	239	+ return ExtractRoceContext<RoceCqContextSplitDb>(context);
	240	+}
	241	+
	242	+struct ChannelEntity {
	243	+ CommAbiHeader abiHeader;
	244	+ CommEngine engine;
	245	+ CommProtocol protocol;
	246	+ uint32_t localNotifyNum;
	247	+ uint32_t remoteNotifyNum;
	248	+ uint32_t localBufferNum;
	249	+ uint32_t remoteBufferNum;
	250	+ uint32_t sqNum;
	251	+ uint32_t cqNum;
	252	+ RegedNotifyEntity* localNotifyAddr;
	253	+ RegedNotifyEntity* remoteNotifyAddr;
	254	+ RegedBufferEntity* localBufferAddr;
	255	+ RegedBufferEntity* remoteBufferAddr;
	256	+ SqContext* sqContextAddr;
	257	+ CqContext* cqContextAddr;
	258	+ uint8_t reserve[160];
	259	+};
	260	+
	261	+static_assert(sizeof(ProtectionInfo) == 28, "unexpected hcomm ProtectionInfo ABI");
	262	+static_assert(sizeof(RegedBufferEntity) == 64, "unexpected hcomm RegedBufferEntity ABI");
	263	+static_assert(sizeof(RegedNotifyEntity) == 64, "unexpected hcomm RegedNotifyEntity ABI");
	264	+static_assert(sizeof(SqContext) == 128, "unexpected hcomm SqContext ABI");
	265	+static_assert(sizeof(CqContext) == 128, "unexpected hcomm CqContext ABI");
	266	+static_assert(sizeof(ChannelEntity) == 256, "unexpected hcomm ChannelEntity ABI");
	267	+static_assert(offsetof(SqContext, contextInfo) == 8, "unexpected hcomm SqContext union offset");
	268	+static_assert(offsetof(CqContext, contextInfo) == 8, "unexpected hcomm CqContext union offset");
	269	+static_assert(offsetof(ChannelEntity, localNotifyAddr) == 48, "unexpected hcomm ChannelEntity pointer offset");
	270	+
	271	+} // namespace host
	272	+} // namespace hns_1825
	273	+} // namespace rdma
	274	+} // namespace comm
	275	+} // namespace pto
	276	+
	277	+#endif // PTO_COMM_ASYNC_RDMA_BACKENDS_HNS_1825_HCOMM_DEFS_HPP

Apkg_inc/pto/comm/async/rdma/backends/hns_1825/hns_1825_types.hpp+187-0

@@ -0,0 +1,187 @@
	1	+/**
	2	+Copyright (c) 2026 Huawei Technologies Co., Ltd.
	3	+This program is free software, you can redistribute it and/or modify it under the terms and conditions of
	4	+CANN Open Software License Agreement Version 2.0 (the "License").
	5	+Please refer to the License for details. You may not use this file except in compliance with the License.
	6	+THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED,
	7	+INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE.
	8	+See LICENSE in the root of the software repository for the full text of the License.
	9	+*/
	10	+
	11	+// HNS_1825 NIC-visible SQ/CQ/WQE/CQE layouts and constants.
	12	+
	13	+#ifndef PTO_COMM_ASYNC_RDMA_BACKENDS_HNS_1825_TYPES_HPP
	14	+#define PTO_COMM_ASYNC_RDMA_BACKENDS_HNS_1825_TYPES_HPP
	15	+
	16	+#include <cstdint>
	17	+
	18	+#include "pto/comm/async/rdma/rdma_workspace_types.hpp"
	19	+
	20	+namespace pto {
	21	+namespace comm {
	22	+namespace rdma {
	23	+namespace hns_1825 {
	24	+
	25	+struct SqContext {
	26	+ uint32_t wqn;
	27	+ uint64_t bufAddr;
	28	+ uint32_t wqeSize;
	29	+ uint32_t depth;
	30	+ uint64_t headAddr;
	31	+ uint64_t tailAddr;
	32	+ RdmaDbMode dbMode;
	33	+ uint64_t dbAddr;
	34	+ uint32_t sl;
	35	+ uint64_t amoAddr;
	36	+ uint32_t amoLkey;
	37	+ uint64_t dbSwAddr;
	38	+ uint8_t mtuShift{4};
	39	+ uint8_t dbCos{0x7};
	40	+ uint8_t reserved[6];
	41	+};
	42	+
	43	+struct CqContext {
	44	+ uint32_t cqn;
	45	+ uint64_t bufAddr;
	46	+ uint32_t cqeSize;
	47	+ uint32_t depth;
	48	+ uint64_t headAddr;
	49	+ uint64_t tailAddr;
	50	+ RdmaDbMode dbMode;
	51	+ uint64_t dbAddr;
	52	+ uint64_t dbSwAddr;
	53	+};
	54	+
	55	+struct WqeCtrlSeg {
	56	+ uint8_t owner_sl;
	57	+ uint8_t df_tsl;
	58	+ uint16_t wf_bdsl;
	59	+ uint32_t cl_pi;
	60	+ uint64_t db;
	61	+};
	62	+
	63	+union WqeTaskCommonSeg {
	64	+ struct {
	65	+ uint32_t xrc_srqn : 18;
	66	+ uint32_t ext : 1;
	67	+ uint32_t dif : 1;
	68	+ uint32_t rsvd : 3;
	69	+ uint32_t so : 1;
	70	+ uint32_t opcode : 5;
	71	+ uint32_t signal : 1;
	72	+ uint32_t fence : 1;
	73	+ uint32_t se : 1;
	74	+ } bs;
	75	+ uint32_t value;
	76	+};
	77	+
	78	+struct WqeRdmaTaskSeg {
	79	+ WqeTaskCommonSeg com_tsk;
	80	+ uint32_t data_len;
	81	+ uint32_t imm_data;
	82	+ union {
	83	+ struct {
	84	+ uint32_t last_ext_len : 8;
	85	+ uint32_t cmd_len : 8;
	86	+ uint32_t pi : 16;
	87	+ } bs;
	88	+ uint32_t value;
	89	+ } dw3;
	90	+ uint64_t va;
	91	+ uint32_t rkey;
	92	+ uint32_t ulp;
	93	+};
	94	+
	95	+struct WqeDataSeg {
	96	+ uint64_t buf_addr;
	97	+ uint32_t r_len;
	98	+ uint32_t le_key;
	99	+};
	100	+
	101	+struct Cqe {
	102	+ uint32_t owner_id_qpn;
	103	+ uint32_t op_sr_wqebb;
	104	+ uint32_t byte_cnt;
	105	+ uint32_t imm_data;
	106	+ uint32_t rsvd_dw5;
	107	+ uint32_t wqe_num;
	108	+ uint32_t vlan_queue_index;
	109	+ uint8_t syndrome;
	110	+ uint8_t rsvd;
	111	+ uint16_t wqe_counter;
	112	+};
	113	+
	114	+union SqDoorbell {
	115	+ struct {
	116	+ uint64_t qpn : 20;
	117	+ uint64_t cntx_size : 2;
	118	+ uint64_t rsvd0 : 1;
	119	+ uint64_t c : 1;
	120	+ uint64_t cos : 3;
	121	+ uint64_t type : 5;
	122	+ uint64_t pi : 8;
	123	+ uint64_t rsvd1 : 8;
	124	+ uint64_t xrc_vld : 1;
	125	+ uint64_t rsvd2 : 1;
	126	+ uint64_t mtu_shift : 3;
	127	+ uint64_t sgid_index : 7;
	128	+ uint64_t sub_type : 4;
	129	+ } bs;
	130	+ uint64_t value;
	131	+};
	132	+
	133	+constexpr uint32_t kWriteReadWqeSize = 64;
	134	+constexpr uint32_t kWqebbSize = 64;
	135	+constexpr uint32_t kDefaultCqeSize = 64;
	136	+constexpr uint32_t kPollCqThreshold = 10;
	137	+constexpr uint64_t kMaxTransferBytes = 0x7fffffffULL;
	138	+constexpr uint32_t kMaxSyncId = 7;
	139	+constexpr uint32_t kPollCqTimeoutError = 0x10000;
	140	+constexpr uint32_t kSessionBuildError = 0x20000;
	141	+constexpr uint32_t kInvalidArgumentError = 0x20001;
	142	+constexpr uint32_t kInvalidContextError = 0x20002;
	143	+constexpr uint32_t kCqeError = 0x20003;
	144	+
	145	+#if defined(PTO_NPU_ARCH_A2A3) \|\| defined(__DAV_C220_VEC__) \|\| defined(__DAV_C220_CUBE__)
	146	+constexpr uint64_t kCycleToTimeBase = 50;
	147	+#else
	148	+constexpr uint64_t kCycleToTimeBase = 1000;
	149	+#endif
	150	+constexpr uint64_t kPollCqTimeoutUs = 60ULL * 1000 * 1000;
	151	+constexpr uint64_t kPollCqTimeoutCycles = kPollCqTimeoutUs * kCycleToTimeBase;
	152	+
	153	+// Backend-local aliases preserve the vocabulary used by the verified HNS WQE
	154	+// implementation while keeping NIC layouts inside the HNS1825 backend.
	155	+using RoceSqCtx = SqContext;
	156	+using RoceCqCtx = CqContext;
	157	+using Hns1825WqeCtrlSeg = WqeCtrlSeg;
	158	+using Hns1825WqeRdmaTaskSeg = WqeRdmaTaskSeg;
	159	+using Hns1825WqeDataSeg = WqeDataSeg;
	160	+using Hns1825Cqe = Cqe;
	161	+using Hns1825SqDb = SqDoorbell;
	162	+constexpr uint32_t kHns1825WriteReadWqeSize = kWriteReadWqeSize;
	163	+constexpr uint32_t kHns1825WqebbSize = kWqebbSize;
	164	+constexpr uint32_t kHns1825DefaultCqeSize = kDefaultCqeSize;
	165	+constexpr uint32_t kHns1825PollCqThreshold = kPollCqThreshold;
	166	+constexpr uint64_t kHns1825MaxTransferBytes = kMaxTransferBytes;
	167	+constexpr uint32_t kHns1825MaxSyncId = kMaxSyncId;
	168	+constexpr uint32_t kHns1825PollCqTimeoutError = kPollCqTimeoutError;
	169	+constexpr uint32_t kHns1825SessionBuildError = kSessionBuildError;
	170	+constexpr uint32_t kHns1825InvalidArgumentError = kInvalidArgumentError;
	171	+constexpr uint32_t kHns1825InvalidContextError = kInvalidContextError;
	172	+constexpr uint32_t kHns1825CqeError = kCqeError;
	173	+constexpr uint64_t kHns1825CycleToTimeBase = kCycleToTimeBase;
	174	+constexpr uint64_t kHns1825PollCqTimeoutCycles = kPollCqTimeoutCycles;
	175	+
	176	+static_assert(sizeof(WqeCtrlSeg) == 16, "unexpected HNS1825 WQE control-segment layout");
	177	+static_assert(sizeof(WqeRdmaTaskSeg) == 32, "unexpected HNS1825 WQE task-segment layout");
	178	+static_assert(sizeof(WqeDataSeg) == 16, "unexpected HNS1825 WQE data-segment layout");
	179	+static_assert(sizeof(Cqe) == 32, "unexpected HNS1825 CQE layout");
	180	+static_assert(sizeof(SqDoorbell) == 8, "unexpected HNS1825 SQ doorbell layout");
	181	+
	182	+} // namespace hns_1825
	183	+} // namespace rdma
	184	+} // namespace comm
	185	+} // namespace pto
	186	+
	187	+#endif // PTO_COMM_ASYNC_RDMA_BACKENDS_HNS_1825_TYPES_HPP

Apkg_inc/pto/comm/async/rdma/backends/hns_1825/hns_1825_workspace_manager.hpp+216-0

@@ -0,0 +1,216 @@
	1	+/**
	2	+Copyright (c) 2026 Huawei Technologies Co., Ltd.
	3	+This program is free software, you can redistribute it and/or modify it under the terms and conditions of
	4	+CANN Open Software License Agreement Version 2.0 (the "License").
	5	+Please refer to the License for details. You may not use this file except in compliance with the License.
	6	+THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED,
	7	+INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE.
	8	+See LICENSE in the root of the software repository for the full text of the License.
	9	+*/
	10	+
	11	+#ifndef PTO_COMM_ASYNC_RDMA_BACKENDS_HNS_1825_WORKSPACE_MANAGER_HPP
	12	+#define PTO_COMM_ASYNC_RDMA_BACKENDS_HNS_1825_WORKSPACE_MANAGER_HPP
	13	+
	14	+#if defined(__CCE_KT_TEST__)
	15	+#error "hns_1825_workspace_manager.hpp is a host-only header and cannot be included in device code."
	16	+#endif
	17	+
	18	+#include <arpa/inet.h>
	19	+
	20	+#include <algorithm>
	21	+#include <chrono>
	22	+#include <cstdint>
	23	+#include <cstdio>
	24	+#include <cstdlib>
	25	+#include <iostream>
	26	+#include <sstream>
	27	+#include <string>
	28	+#include <thread>
	29	+#include <utility>
	30	+#include <vector>
	31	+
	32	+#include "securec.h"
	33	+
	34	+#include "acl/acl.h"
	35	+
	36	+#include "pto/comm/async/rdma/rdma_workspace_types.hpp"
	37	+#include "pto/comm/async/rdma/backends/hns_1825/hns_1825_hcomm_defs.hpp"
	38	+#include "pto/comm/async/rdma/backends/hns_1825/hns_1825_types.hpp"
	39	+
	40	+namespace pto {
	41	+namespace comm {
	42	+namespace rdma {
	43	+namespace hns_1825 {
	44	+
	45	+// HCOMM-based RoCE control plane for the HNS_1825 backend.
	46	+//
	47	+// Flow:
	48	+// 1. Create the local RoCE endpoint.
	49	+// 2. Register the symmetric communication buffer.
	50	+// 3. Create one channel per peer and wait for every channel to become ready.
	51	+// 4. Read ChannelEntity SQ/CQ/MR metadata and copy RdmaInfo to device memory.
	52	+//
	53	+// Teardown releases resources in reverse ownership order: channels, registered
	54	+// memory, endpoint, then device metadata. The caller supplies peer network and
	55	+// symmetric-memory metadata; opt-in progress logs use PTO_ROCE_VERBOSE=1.
	56	+class WorkspaceManager {
	57	+public:
	58	+ WorkspaceManager() = default;
	59	+ ~WorkspaceManager();
	60	+
	61	+ WorkspaceManager(const WorkspaceManager&) = delete;
	62	+ WorkspaceManager& operator=(const WorkspaceManager&) = delete;
	63	+
	64	+ void SetTraceId(uint32_t traceId) { traceId_ = traceId; }
	65	+
	66	+ // rankId/rankCount — this PE and the total number of PEs.
	67	+ // phyId/localIp — this rank's physical device and RoCE IPv4.
	68	+ // basePort — base port for the HCOMM channel handshake.
	69	+ // peerIps/peerPhyIds — every rank's RoCE endpoint information.
	70	+ // peerSymAddrs — every rank's registered communication-buffer base.
	71	+ // symmetricAddr/Size — this rank's symmetric communication buffer.
	72	+ bool Init(
	73	+ uint32_t rankId, uint32_t rankCount, uint32_t phyId, const std::string& localIp, uint16_t basePort,
	74	+ const std::vector<std::string>& peerIps, const std::vector<uint32_t>& peerPhyIds,
	75	+ const std::vector<uint64_t>& peerSymAddrs, void* symmetricAddr, uint64_t symmetricSize);
	76	+
	77	+ static bool Verbose();
	78	+
	79	+ // Best-effort teardown. Every owned resource is attempted once, errors are
	80	+ // accumulated in the return value, and PTO-side handles are always reset.
	81	+ bool Finalize();
	82	+
	83	+ // Device address of the RdmaInfo table passed to the RDMA kernel.
	84	+ void* GetWorkspaceAddr() const { return rdmaInfoDevice_; }
	85	+
	86	+private:
	87	+ struct DecodedRoceSqContext : host::RoceSqContextSplitDb {
	88	+ const char* abiName;
	89	+ uint8_t dbCos;
	90	+ };
	91	+
	92	+ struct RdmaInfoHostLayout {
	93	+ size_t totalSize{0};
	94	+ RdmaInfo* info{nullptr};
	95	+ RoceSqCtx* sq{nullptr};
	96	+ RoceSqCtx* rq{nullptr};
	97	+ RoceCqCtx* scq{nullptr};
	98	+ RoceCqCtx* rcq{nullptr};
	99	+ RdmaMemInfo* memory{nullptr};
	100	+ };
	101	+
	102	+ template <typename... Args>
	103	+ void Trace(Args&&... args) const;
	104	+
	105	+ static uint64_t ElapsedUs(const std::chrono::steady_clock::time_point& start);
	106	+ static std::string Hex(uint64_t value);
	107	+
	108	+ void AssignInitArguments(
	109	+ uint32_t rankId, uint32_t rankCount, uint32_t phyId, const std::string& localIp, uint16_t basePort,
	110	+ const std::vector<std::string>& peerIps, const std::vector<uint32_t>& peerPhyIds,
	111	+ const std::vector<uint64_t>& peerSymAddrs, void* symmetricAddr, uint64_t symmetricSize);
	112	+ void LogInitConfiguration() const;
	113	+ bool RunInitStage(bool (WorkspaceManager::stage)(), const char stageName);
	114	+
	115	+ bool DestroyChannels();
	116	+ bool UnregisterMemory();
	117	+ bool DestroyEndpoint();
	118	+ bool FreeDeviceInfo();
	119	+ bool HasOwnedResources() const;
	120	+ static bool ValidateBasicInitArguments(
	121	+ uint32_t rankId, uint32_t rankCount, uint32_t phyId, const std::string& localIp, uint16_t basePort,
	122	+ const std::vector<std::string>& peerIps, const std::vector<uint32_t>& peerPhyIds,
	123	+ const std::vector<uint64_t>& peerSymAddrs, void* symmetricAddr, uint64_t symmetricSize);
	124	+ static bool ValidatePeerPortMapping(
	125	+ uint32_t rankCount, const std::vector<std::string>& peerIps, const std::vector<uint32_t>& peerPhyIds,
	126	+ const std::vector<uint64_t>& peerSymAddrs);
	127	+ static bool ValidateInitArguments(
	128	+ uint32_t rankId, uint32_t rankCount, uint32_t phyId, const std::string& localIp, uint16_t basePort,
	129	+ const std::vector<std::string>& peerIps, const std::vector<uint32_t>& peerPhyIds,
	130	+ const std::vector<uint64_t>& peerSymAddrs, void* symmetricAddr, uint64_t symmetricSize);
	131	+ void ResetOwnedState();
	132	+
	133	+ bool CreateEndpoint();
	134	+ bool RegisterMemory();
	135	+ bool InitializeChannelDescriptions(std::vector<HcommChannelDesc>& descriptions) const;
	136	+ bool ConfigureChannelDescription(
	137	+ HcommChannelDesc& description, std::string& channelName, uint32_t channelIndex, uint32_t remoteRank, uint8_t tc,
	138	+ uint8_t sl) const;
	139	+ bool ConfigureChannelDescriptions(
	140	+ std::vector<HcommChannelDesc>& descriptions, std::vector<std::string>& channelNames);
	141	+ bool CreateAndValidateChannels(std::vector<HcommChannelDesc>& descriptions);
	142	+ bool BuildChannels();
	143	+ bool WaitChannelsReady();
	144	+ static uint8_t GetEnvU8(const char* name, uint8_t defaultValue, long maximumValue, bool requireMultipleOf4);
	145	+ static void SetEndpointDescDefaults(EndpointDesc& description);
	146	+ static void SetChannelDescDefaults(HcommChannelDesc& description);
	147	+
	148	+ static bool IsDbVendorSpecifiedValid(uint64_t value);
	149	+ static DecodedRoceSqContext DecodeRoceSqContext(const host::SqContext& context);
	150	+ static bool ReadU32Device(uint64_t deviceAddress, uint32_t& value);
	151	+ static bool WriteU32Device(uint64_t deviceAddress, uint32_t value);
	152	+ bool InitializeQueueMirrors(const host::SqContext& sq, const host::CqContext& cq, uint32_t peer) const;
	153	+ bool PrepareRdmaInfoLayout(std::vector<uint8_t>& hostBuffer, RdmaInfoHostLayout& layout);
	154	+ bool FillLocalMemoryInfo(RdmaMemInfo* memory);
	155	+ bool FillPeerRdmaInfo(size_t channelIndex, RdmaInfoHostLayout& layout);
	156	+ bool FillPeerRdmaInfoEntries(RdmaInfoHostLayout& layout);
	157	+ void LogRdmaInfo(const RdmaInfoHostLayout& layout) const;
	158	+ bool CopyRdmaInfoToDevice(const std::vector<uint8_t>& hostBuffer, size_t totalSize) const;
	159	+ bool FillRdmaInfo();
	160	+ static bool IsPowerOfTwo(uint64_t value);
	161	+ static bool IsU32Aligned(uint64_t address);
	162	+ static bool IsCacheLineAligned(uint64_t address);
	163	+ static bool ValidateChannelEntity(const host::ChannelEntity& entity, uint32_t peer);
	164	+ static bool ValidateRegisteredBuffer(
	165	+ const host::RegedBufferEntity& buffer, uint64_t expectedAddr, uint64_t expectedSize, const char* side,
	166	+ uint32_t rank);
	167	+ static bool ValidateSqContext(const host::SqContext& sq, uint32_t peer);
	168	+ static bool ValidateCqContext(const host::CqContext& cq, uint32_t peer);
	169	+ static void CopyRoceSq(RoceSqCtx& destination, const host::SqContext& source);
	170	+ static void CopyRoceCq(RoceCqCtx& destination, const host::CqContext& source);
	171	+ bool ReadChannelEntity(ChannelHandle handle, host::ChannelEntity& output);
	172	+ static bool ReadDeviceStruct(const void* devicePointer, void* host, size_t size);
	173	+
	174	+ static constexpr uint32_t kDbVendorFieldMask = 0x7;
	175	+ static constexpr uint32_t kDbCosShift = 24;
	176	+ static constexpr uint32_t kMtuShiftShift = 50;
	177	+ static constexpr uint8_t kDefaultMtuShift = 4;
	178	+ static constexpr uint8_t kDefaultDbCos = 0x7;
	179	+ static constexpr uint64_t kDbVendorSpecifiedMask = (static_cast<uint64_t>(kDbVendorFieldMask) << kDbCosShift) \|
	180	+ (static_cast<uint64_t>(kDbVendorFieldMask) << kMtuShiftShift);
	181	+ static constexpr uint32_t kMaxRanksPerNic = 16;
	182	+ static constexpr uint32_t kDefaultRoceRetryCount = 7;
	183	+ static constexpr uint32_t kDefaultRoceRetryIntervalMs = 20;
	184	+ static constexpr uint32_t kChannelStatusPollIntervalPerChannelMs = 10;
	185	+ static constexpr uint32_t kChannelStatusPollTimeoutPerChannelMs = 60000;
	186	+
	187	+ uint32_t rankId_{0};
	188	+ uint32_t rankCount_{1};
	189	+ uint32_t phyId_{0};
	190	+ std::string localIp_;
	191	+ uint16_t basePort_{60032};
	192	+ std::vector<std::string> peerIps_;
	193	+ std::vector<uint32_t> peerPhyIds_;
	194	+ std::vector<uint64_t> peerSymAddrs_;
	195	+ void* symmetricAddr_{nullptr};
	196	+ uint64_t symmetricSize_{0};
	197	+
	198	+ EndpointHandle endpoint_{nullptr};
	199	+ HcommMemHandle memHandle_{nullptr};
	200	+ std::vector<ChannelHandle> channelHandles_;
	201	+ std::vector<uint32_t> channelPeer_;
	202	+ void* rdmaInfoDevice_{nullptr};
	203	+ bool initialized_{false};
	204	+ uint32_t traceId_{0};
	205	+};
	206	+
	207	+} // namespace hns_1825
	208	+} // namespace rdma
	209	+} // namespace comm
	210	+} // namespace pto
	211	+
	212	+#include "pto/comm/async/rdma/backends/hns_1825/hns_1825_workspace_manager_lifecycle.hpp"
	213	+#include "pto/comm/async/rdma/backends/hns_1825/hns_1825_workspace_manager_channel.hpp"
	214	+#include "pto/comm/async/rdma/backends/hns_1825/hns_1825_workspace_manager_info.hpp"
	215	+
	216	+#endif // PTO_COMM_ASYNC_RDMA_BACKENDS_HNS_1825_WORKSPACE_MANAGER_HPP

Apkg_inc/pto/comm/async/rdma/backends/hns_1825/hns_1825_workspace_manager_channel.hpp+294-0

@@ -0,0 +1,294 @@
	1	+/**
	2	+Copyright (c) 2026 Huawei Technologies Co., Ltd.
	3	+This program is free software, you can redistribute it and/or modify it under the terms and conditions of
	4	+CANN Open Software License Agreement Version 2.0 (the "License").
	5	+Please refer to the License for details. You may not use this file except in compliance with the License.
	6	+THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED,
	7	+INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE.
	8	+See LICENSE in the root of the software repository for the full text of the License.
	9	+*/
	10	+
	11	+#ifndef PTO_COMM_ASYNC_RDMA_BACKENDS_HNS_1825_WORKSPACE_MANAGER_CHANNEL_HPP
	12	+#define PTO_COMM_ASYNC_RDMA_BACKENDS_HNS_1825_WORKSPACE_MANAGER_CHANNEL_HPP
	13	+
	14	+#ifndef PTO_COMM_ASYNC_RDMA_BACKENDS_HNS_1825_WORKSPACE_MANAGER_HPP
	15	+#error "Include hns_1825_workspace_manager.hpp instead of this internal implementation header."
	16	+#endif
	17	+
	18	+namespace pto {
	19	+namespace comm {
	20	+namespace rdma {
	21	+namespace hns_1825 {
	22	+
	23	+inline bool WorkspaceManager::CreateEndpoint()
	24	+{
	25	+ EndpointDesc description{};
	26	+ HcommResult result = EndpointDescInit(&description, 1);
	27	+ if (result != 0) {
	28	+ std::cerr << "[RoCE] EndpointDescInit failed: " << result << std::endl;
	29	+ return false;
	30	+ }
	31	+ // HCOMM initializes unspecified descriptor bytes to 0xFF. Replace those
	32	+ // sentinels before setting the attributes used by PTO.
	33	+ SetEndpointDescDefaults(description);
	34	+ description.protocol = COMM_PROTOCOL_ROCE;
	35	+ description.commAddr.type = COMM_ADDR_TYPE_IP_V4;
	36	+ if (inet_pton(AF_INET, localIp_.c_str(), &description.commAddr.addr) != 1) {
	37	+ std::cerr << "[RoCE] invalid local ip: " << localIp_ << std::endl;
	38	+ return false;
	39	+ }
	40	+ description.loc.locType = ENDPOINT_LOC_TYPE_HOST;
	41	+ description.loc.device.devPhyId = phyId_;
	42	+ result = HcommEndpointCreate(&description, &endpoint_);
	43	+ if (result != 0 \|\| endpoint_ == nullptr) {
	44	+ std::cerr << "[RoCE] HcommEndpointCreate failed: " << result << std::endl;
	45	+ return false;
	46	+ }
	47	+ Trace("CREATE endpoint handle=", endpoint_, " phyId=", phyId_, " ip=", localIp_);
	48	+ if (Verbose()) {
	49	+ std::cerr << "[RoCE][rank " << rankId_ << "] (1/4) endpoint created (ROCE, phyId=" << phyId_
	50	+ << ", ip=" << localIp_ << ")" << std::endl;
	51	+ }
	52	+ return true;
	53	+}
	54	+
	55	+inline bool WorkspaceManager::RegisterMemory()
	56	+{
	57	+ CommMem memory{};
	58	+ memory.type = COMM_MEM_TYPE_DEVICE;
	59	+ memory.addr = symmetricAddr_;
	60	+ memory.size = symmetricSize_;
	61	+ const HcommResult result = HcommMemReg(endpoint_, "PtoRoceBuffer", &memory, &memHandle_);
	62	+ if (result != 0 \|\| memHandle_ == nullptr) {
	63	+ std::cerr << "[RoCE] HcommMemReg failed: " << result << std::endl;
	64	+ return false;
	65	+ }
	66	+ Trace("CREATE mem-reg handle=", memHandle_, " addr=", symmetricAddr_, " size=", symmetricSize_);
	67	+ if (Verbose()) {
	68	+ std::cerr << "[RoCE][rank " << rankId_ << "] (2/4) MR registered: addr=0x" << std::hex
	69	+ << reinterpret_cast<uint64_t>(symmetricAddr_) << std::dec << " size=" << symmetricSize_ << std::endl;
	70	+ }
	71	+ return true;
	72	+}
	73	+
	74	+inline bool WorkspaceManager::InitializeChannelDescriptions(std::vector<HcommChannelDesc>& descriptions) const
	75	+{
	76	+ const HcommResult result = HcommChannelDescInit(descriptions.data(), static_cast<uint32_t>(descriptions.size()));
	77	+ if (result != 0) {
	78	+ std::cerr << "[RoCE] HcommChannelDescInit failed: " << result << std::endl;
	79	+ return false;
	80	+ }
	81	+ for (HcommChannelDesc& description : descriptions) {
	82	+ SetChannelDescDefaults(description);
	83	+ }
	84	+ return true;
	85	+}
	86	+
	87	+inline bool WorkspaceManager::ConfigureChannelDescription(
	88	+ HcommChannelDesc& description, std::string& channelName, uint32_t channelIndex, uint32_t remoteRank, uint8_t tc,
	89	+ uint8_t sl) const
	90	+{
	91	+ description.remoteEndpoint.protocol = COMM_PROTOCOL_ROCE;
	92	+ description.remoteEndpoint.commAddr.type = COMM_ADDR_TYPE_IP_V4;
	93	+ if (inet_pton(AF_INET, peerIps_[remoteRank].c_str(), &description.remoteEndpoint.commAddr.addr) != 1) {
	94	+ std::cerr << "[RoCE] invalid peer ip[" << remoteRank << "]: " << peerIps_[remoteRank] << std::endl;
	95	+ return false;
	96	+ }
	97	+ description.remoteEndpoint.loc.locType = ENDPOINT_LOC_TYPE_HOST;
	98	+ description.remoteEndpoint.loc.device.devPhyId = peerPhyIds_[remoteRank];
	99	+ description.notifyNum = 3;
	100	+ description.exchangeAllMems = true;
	101	+ description.memHandles = nullptr;
	102	+ description.memHandleNum = 0;
	103	+ description.roceAttr.queueNum = 1;
	104	+ description.roceAttr.retryCnt = kDefaultRoceRetryCount;
	105	+ description.roceAttr.retryInterval = kDefaultRoceRetryIntervalMs;
	106	+ description.roceAttr.tc = tc;
	107	+ description.roceAttr.sl = sl;
	108	+ description.roceAttr.qpThreshold = 0;
	109	+ description.socket = nullptr;
	110	+ description.qos = 0;
	111	+ const bool isServer = rankId_ < remoteRank;
	112	+ description.role = isServer ? HCOMM_SOCKET_ROLE_SERVER : HCOMM_SOCKET_ROLE_CLIENT;
	113	+ const uint32_t serverRank = isServer ? rankId_ : remoteRank;
	114	+ const uint32_t clientRank = isServer ? remoteRank : rankId_;
	115	+ const uint32_t lowRank = std::min(rankId_, remoteRank);
	116	+ const uint32_t highRank = std::max(rankId_, remoteRank);
	117	+ channelName = "pto-rdma/" + std::to_string(lowRank) + "-" + std::to_string(highRank) + "/ch0";
	118	+ description.channelName = channelName.c_str();
	119	+ description.port = static_cast<uint16_t>(
	120	+ basePort_ + (serverRank % kMaxRanksPerNic) * kMaxRanksPerNic + (clientRank % kMaxRanksPerNic));
	121	+ if (Verbose()) {
	122	+ std::cerr << "[RoCE][rank " << rankId_ << "] channel[" << channelIndex << "] -> peer " << remoteRank
	123	+ << " ip=" << peerIps_[remoteRank] << " role=" << (isServer ? "SERVER" : "CLIENT")
	124	+ << " phyId=" << peerPhyIds_[remoteRank] << " port=" << description.port
	125	+ << " tc=" << static_cast<int>(tc) << " sl=" << static_cast<int>(sl)
	126	+ << " retryCnt=" << description.roceAttr.retryCnt
	127	+ << " retryIntervalMs=" << description.roceAttr.retryInterval << std::endl;
	128	+ }
	129	+ return true;
	130	+}
	131	+
	132	+inline bool WorkspaceManager::ConfigureChannelDescriptions(
	133	+ std::vector<HcommChannelDesc>& descriptions, std::vector<std::string>& channelNames)
	134	+{
	135	+ const uint8_t tc = GetEnvU8("HCCL_RDMA_TC", 132, 255, /requireMultipleOf4/ true);
	136	+ const uint8_t sl = GetEnvU8("HCCL_RDMA_SL", 4, 7, /requireMultipleOf4/ false);
	137	+ channelPeer_.clear();
	138	+ uint32_t channelIndex = 0;
	139	+ for (uint32_t remoteRank = 0; remoteRank < rankCount_; ++remoteRank) {
	140	+ if (remoteRank == rankId_) {
	141	+ continue;
	142	+ }
	143	+ if (!ConfigureChannelDescription(
	144	+ descriptions[channelIndex], channelNames[channelIndex], channelIndex, remoteRank, tc, sl)) {
	145	+ return false;
	146	+ }
	147	+ channelPeer_.push_back(remoteRank);
	148	+ ++channelIndex;
	149	+ }
	150	+ return true;
	151	+}
	152	+
	153	+inline bool WorkspaceManager::CreateAndValidateChannels(std::vector<HcommChannelDesc>& descriptions)
	154	+{
	155	+ const uint32_t channelNum = static_cast<uint32_t>(descriptions.size());
	156	+ channelHandles_.assign(channelNum, 0);
	157	+ if (Verbose()) {
	158	+ std::cerr << "[RoCE][rank " << rankId_ << "] (3/4) HcommChannelCreate: " << channelNum
	159	+ << " channel(s), waiting for QP handshake ..." << std::endl;
	160	+ }
	161	+ const HcommResult result = HcommChannelCreate(
	162	+ endpoint_, COMM_ENGINE_AIV, descriptions.data(), channelNum,
	163	+ reinterpret_cast<ChannelHandle*>(channelHandles_.data()));
	164	+ if (result != 0) {
	165	+ std::cerr << "[RoCE] HcommChannelCreate failed: " << result << std::endl;
	166	+ return false;
	167	+ }
	168	+ for (uint32_t index = 0; index < channelNum; ++index) {
	169	+ if (channelHandles_[index] == 0) {
	170	+ std::cerr << "[RoCE] HcommChannelCreate returned a null handle at index " << index << std::endl;
	171	+ return false;
	172	+ }
	173	+ Trace(
	174	+ "CREATE channel index=", index, " peer=", channelPeer_[index],
	175	+ " handle=", Hex(static_cast<uint64_t>(channelHandles_[index])));
	176	+ }
	177	+ if (!WaitChannelsReady()) {
	178	+ return false;
	179	+ }
	180	+ if (Verbose()) {
	181	+ std::cerr << "[RoCE][rank " << rankId_ << "] (3/4) all channels READY" << std::endl;
	182	+ }
	183	+ return true;
	184	+}
	185	+
	186	+inline bool WorkspaceManager::BuildChannels()
	187	+{
	188	+ if (rankCount_ <= 1) {
	189	+ return true;
	190	+ }
	191	+ const uint32_t channelNum = rankCount_ - 1;
	192	+ std::vector<HcommChannelDesc> descriptions(channelNum);
	193	+ std::vector<std::string> channelNames(channelNum);
	194	+ return InitializeChannelDescriptions(descriptions) && ConfigureChannelDescriptions(descriptions, channelNames) &&
	195	+ CreateAndValidateChannels(descriptions);
	196	+}
	197	+
	198	+inline bool WorkspaceManager::WaitChannelsReady()
	199	+{
	200	+ if (channelHandles_.empty()) {
	201	+ return true;
	202	+ }
	203	+ constexpr int32_t kChannelReady = 0;
	204	+ constexpr int32_t kChannelConnecting = 1;
	205	+ constexpr int32_t kChannelFailed = 2;
	206	+ constexpr int32_t kChannelTimeout = 3;
	207	+ const uint32_t channelNum = static_cast<uint32_t>(channelHandles_.size());
	208	+ const uint32_t pollIntervalMs = channelNum * kChannelStatusPollIntervalPerChannelMs;
	209	+ const uint32_t pollTimeoutMs = channelNum * kChannelStatusPollTimeoutPerChannelMs;
	210	+ std::vector<int32_t> statuses(channelNum, kChannelConnecting);
	211	+ uint32_t elapsedMs = 0;
	212	+ while (elapsedMs <= pollTimeoutMs) {
	213	+ const HcommResult result = HcommChannelGetStatus(channelHandles_.data(), channelNum, statuses.data());
	214	+ if (result != 0) {
	215	+ std::cerr << "[RoCE] HcommChannelGetStatus failed: " << result << std::endl;
	216	+ return false;
	217	+ }
	218	+ bool allReady = true;
	219	+ for (uint32_t index = 0; index < channelNum; ++index) {
	220	+ const int32_t status = statuses[index];
	221	+ if (status == kChannelReady) {
	222	+ continue;
	223	+ }
	224	+ allReady = false;
	225	+ if (status == kChannelConnecting) {
	226	+ continue;
	227	+ }
	228	+ const char* reason = status == kChannelFailed ? "failed" :
	229	+ status == kChannelTimeout ? "timed out" :
	230	+ "returned an unknown status";
	231	+ std::cerr << "[RoCE] channel " << index << " (peer " << channelPeer_[index] << ") " << reason << ": "
	232	+ << status << std::endl;
	233	+ return false;
	234	+ }
	235	+ if (allReady) {
	236	+ if (Verbose()) {
	237	+ std::cerr << "[RoCE][rank " << rankId_ << "] channels READY after " << elapsedMs << " ms" << std::endl;
	238	+ }
	239	+ return true;
	240	+ }
	241	+ std::this_thread::sleep_for(std::chrono::milliseconds(pollIntervalMs));
	242	+ elapsedMs += pollIntervalMs;
	243	+ }
	244	+ std::cerr << "[RoCE] wait channel READY timed out after " << elapsedMs << " ms" << std::endl;
	245	+ return false;
	246	+}
	247	+
	248	+inline uint8_t WorkspaceManager::GetEnvU8(
	249	+ const char* name, uint8_t defaultValue, long maximumValue, bool requireMultipleOf4)
	250	+{
	251	+ const char* value = std::getenv(name);
	252	+ if (value == nullptr) {
	253	+ return defaultValue;
	254	+ }
	255	+ char* end = nullptr;
	256	+ const long parsed = std::strtol(value, &end, 10);
	257	+ if (end == value \|\| *end != '\0' \|\| parsed < 0 \|\| parsed > maximumValue) {
	258	+ std::cerr << "[RoCE] env " << name << "='" << value << "' invalid (expect 0.." << maximumValue
	259	+ << "); using default " << static_cast<int>(defaultValue) << std::endl;
	260	+ return defaultValue;
	261	+ }
	262	+ if (requireMultipleOf4 && parsed % 4 != 0) {
	263	+ std::cerr << "[RoCE] env " << name << "=" << parsed << " must be a multiple of 4; using default "
	264	+ << static_cast<int>(defaultValue) << std::endl;
	265	+ return defaultValue;
	266	+ }
	267	+ return static_cast<uint8_t>(parsed);
	268	+}
	269	+
	270	+inline void WorkspaceManager::SetEndpointDescDefaults(EndpointDesc& description)
	271	+{
	272	+ (void)memset_s(&description, sizeof(description), 0, sizeof(description));
	273	+ description.protocol = COMM_PROTOCOL_RESERVED;
	274	+ description.commAddr.type = COMM_ADDR_TYPE_RESERVED;
	275	+ description.loc.locType = ENDPOINT_LOC_TYPE_RESERVED;
	276	+}
	277	+
	278	+inline void WorkspaceManager::SetChannelDescDefaults(HcommChannelDesc& description)
	279	+{
	280	+ // Preserve the ABI header produced by HCOMM while deterministically
	281	+ // initializing every payload and padding byte owned by PTO.
	282	+ const CommAbiHeader header = description.header;
	283	+ (void)memset_s(&description, sizeof(description), 0, sizeof(description));
	284	+ description.header = header;
	285	+ SetEndpointDescDefaults(description.remoteEndpoint);
	286	+ description.role = HCOMM_SOCKET_ROLE_RESERVED;
	287	+}
	288	+
	289	+} // namespace hns_1825
	290	+} // namespace rdma
	291	+} // namespace comm
	292	+} // namespace pto
	293	+
	294	+#endif // PTO_COMM_ASYNC_RDMA_BACKENDS_HNS_1825_WORKSPACE_MANAGER_CHANNEL_HPP

Apkg_inc/pto/comm/async/rdma/backends/hns_1825/hns_1825_workspace_manager_info.hpp+390-0

@@ -0,0 +1,390 @@
	1	+/**
	2	+Copyright (c) 2026 Huawei Technologies Co., Ltd.
	3	+This program is free software, you can redistribute it and/or modify it under the terms and conditions of
	4	+CANN Open Software License Agreement Version 2.0 (the "License").
	5	+Please refer to the License for details. You may not use this file except in compliance with the License.
	6	+THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED,
	7	+INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE.
	8	+See LICENSE in the root of the software repository for the full text of the License.
	9	+*/
	10	+
	11	+#ifndef PTO_COMM_ASYNC_RDMA_BACKENDS_HNS_1825_WORKSPACE_MANAGER_INFO_HPP
	12	+#define PTO_COMM_ASYNC_RDMA_BACKENDS_HNS_1825_WORKSPACE_MANAGER_INFO_HPP
	13	+
	14	+#ifndef PTO_COMM_ASYNC_RDMA_BACKENDS_HNS_1825_WORKSPACE_MANAGER_HPP
	15	+#error "Include hns_1825_workspace_manager.hpp instead of this internal implementation header."
	16	+#endif
	17	+
	18	+namespace pto {
	19	+namespace comm {
	20	+namespace rdma {
	21	+namespace hns_1825 {
	22	+
	23	+inline bool WorkspaceManager::IsDbVendorSpecifiedValid(uint64_t value)
	24	+{
	25	+ if (value == 0) {
	26	+ return false;
	27	+ }
	28	+ if ((value & ~0xffULL) == 0) {
	29	+ return value <= kDbVendorFieldMask;
	30	+ }
	31	+ return (value & ~kDbVendorSpecifiedMask) == 0;
	32	+}
	33	+
	34	+inline WorkspaceManager::DecodedRoceSqContext WorkspaceManager::DecodeRoceSqContext(const host::SqContext& context)
	35	+{
	36	+ const auto splitDb = host::ExtractRoceSqContextSplitDb(context);
	37	+ DecodedRoceSqContext decoded{};
	38	+ static_cast<host::RoceSqContextSplitDb&>(decoded) = splitDb;
	39	+ decoded.abiName = "split-db-mtushift";
	40	+ decoded.dbCos = kDefaultDbCos;
	41	+ const auto vendor = host::ExtractRoceSqContextVendorSpecified(context);
	42	+ if (IsDbVendorSpecifiedValid(vendor.DbVendorSpecified)) {
	43	+ decoded.abiName = "db-vendor-specified";
	44	+ if ((vendor.DbVendorSpecified & ~0xffULL) == 0) {
	45	+ decoded.mtuShift = kDefaultMtuShift;
	46	+ decoded.dbCos = static_cast<uint8_t>(vendor.DbVendorSpecified & kDbVendorFieldMask);
	47	+ } else {
	48	+ decoded.mtuShift = static_cast<uint8_t>((vendor.DbVendorSpecified >> kMtuShiftShift) & kDbVendorFieldMask);
	49	+ decoded.dbCos = static_cast<uint8_t>((vendor.DbVendorSpecified >> kDbCosShift) & kDbVendorFieldMask);
	50	+ }
	51	+ }
	52	+ return decoded;
	53	+}
	54	+
	55	+inline bool WorkspaceManager::ReadU32Device(uint64_t deviceAddress, uint32_t& value)
	56	+{
	57	+ return deviceAddress != 0 &&
	58	+ ReadDeviceStruct(
	59	+ reinterpret_cast<const void*>(static_cast<uintptr_t>(deviceAddress)), &value, sizeof(value));
	60	+}
	61	+
	62	+inline bool WorkspaceManager::WriteU32Device(uint64_t deviceAddress, uint32_t value)
	63	+{
	64	+ return deviceAddress != 0 && aclrtMemcpy(
	65	+ reinterpret_cast<void*>(static_cast<uintptr_t>(deviceAddress)), sizeof(value),
	66	+ &value, sizeof(value), ACL_MEMCPY_HOST_TO_DEVICE) == ACL_SUCCESS;
	67	+}
	68	+
	69	+inline bool WorkspaceManager::InitializeQueueMirrors(
	70	+ const host::SqContext& sq, const host::CqContext& cq, uint32_t peer) const
	71	+{
	72	+ const auto roceSq = host::ExtractRoceSqContextSplitDb(sq);
	73	+ const auto roceCq = host::ExtractRoceCqContextSplitDb(cq);
	74	+ uint32_t sqHead = UINT32_MAX;
	75	+ uint32_t sqTail = UINT32_MAX;
	76	+ uint32_t cqHead = UINT32_MAX;
	77	+ uint32_t cqTail = UINT32_MAX;
	78	+ const bool readOk = ReadU32Device(roceSq.headAddr, sqHead) && ReadU32Device(roceSq.tailAddr, sqTail) &&
	79	+ ReadU32Device(roceCq.headAddr, cqHead) && ReadU32Device(roceCq.tailAddr, cqTail);
	80	+ Trace(
	81	+ "QUEUE initial peer=", peer, " sq(head/tail)=", sqHead, "/", sqTail, " cq(head/tail)=", cqHead, "/", cqTail,
	82	+ " readOk=", readOk, " sqMirror=", Hex(roceSq.headAddr), "/", Hex(roceSq.tailAddr),
	83	+ " cqMirror=", Hex(roceCq.headAddr), "/", Hex(roceCq.tailAddr));
	84	+ if (!readOk) {
	85	+ std::cerr << "[RoCE] peer " << peer << " queue mirror D2H read failed" << std::endl;
	86	+ return false;
	87	+ }
	88	+ // HCOMM exposes freshly allocated PI/CI mirrors but does not guarantee the
	89	+ // allocation contents. A new QP starts at index zero, so initialize every
	90	+ // mirror explicitly before the first PTO WQE is posted.
	91	+ constexpr uint32_t kInitialQueueIndex = 0;
	92	+ const bool writeOk =
	93	+ WriteU32Device(roceSq.headAddr, kInitialQueueIndex) && WriteU32Device(roceSq.tailAddr, kInitialQueueIndex) &&
	94	+ WriteU32Device(roceCq.headAddr, kInitialQueueIndex) && WriteU32Device(roceCq.tailAddr, kInitialQueueIndex);
	95	+ if (!writeOk) {
	96	+ std::cerr << "[RoCE] peer " << peer << " queue mirror initialization failed" << std::endl;
	97	+ return false;
	98	+ }
	99	+ Trace("QUEUE reset peer=", peer, " sq(head/tail)=0/0 cq(head/tail)=0/0");
	100	+ return true;
	101	+}
	102	+
	103	+inline bool WorkspaceManager::PrepareRdmaInfoLayout(std::vector<uint8_t>& hostBuffer, RdmaInfoHostLayout& layout)
	104	+{
	105	+ constexpr uint32_t kQpNum = 1;
	106	+ const size_t sqBytes = sizeof(RoceSqCtx) * rankCount_ * kQpNum;
	107	+ const size_t cqBytes = sizeof(RoceCqCtx) * rankCount_ * kQpNum;
	108	+ const size_t memBytes = sizeof(RdmaMemInfo) * rankCount_;
	109	+ layout.totalSize = sizeof(RdmaInfo) + 2 * sqBytes + 2 * cqBytes + memBytes;
	110	+ if (aclrtMalloc(&rdmaInfoDevice_, layout.totalSize, ACL_MEM_MALLOC_HUGE_FIRST) != 0 \|\| rdmaInfoDevice_ == nullptr) {
	111	+ std::cerr << "[RoCE] aclrtMalloc(rdmaInfo) failed" << std::endl;
	112	+ return false;
	113	+ }
	114	+ hostBuffer.assign(layout.totalSize, 0);
	115	+ layout.info = reinterpret_cast<RdmaInfo*>(hostBuffer.data());
	116	+ layout.info->magic = kRdmaWorkspaceMagic;
	117	+ layout.info->version = kRdmaWorkspaceVersion;
	118	+ layout.info->backend = RdmaBackend::HNS_1825;
	119	+ layout.info->qpNum = kQpNum;
	120	+ layout.info->rankCount = rankCount_;
	121	+ uint8_t* deviceBase = static_cast<uint8_t*>(rdmaInfoDevice_) + sizeof(RdmaInfo);
	122	+ layout.info->sqPtr = reinterpret_cast<uint64_t>(deviceBase);
	123	+ layout.info->rqPtr = reinterpret_cast<uint64_t>(deviceBase + sqBytes);
	124	+ layout.info->scqPtr = reinterpret_cast<uint64_t>(deviceBase + 2 * sqBytes);
	125	+ layout.info->rcqPtr = reinterpret_cast<uint64_t>(deviceBase + 2 * sqBytes + cqBytes);
	126	+ layout.info->memPtr = reinterpret_cast<uint64_t>(deviceBase + 2 * sqBytes + 2 * cqBytes);
	127	+ uint8_t* hostBase = hostBuffer.data() + sizeof(RdmaInfo);
	128	+ layout.sq = reinterpret_cast<RoceSqCtx*>(hostBase);
	129	+ layout.rq = reinterpret_cast<RoceSqCtx*>(hostBase + sqBytes);
	130	+ layout.scq = reinterpret_cast<RoceCqCtx>(hostBase + 2 sqBytes);
	131	+ layout.rcq = reinterpret_cast<RoceCqCtx>(hostBase + 2 sqBytes + cqBytes);
	132	+ layout.memory = reinterpret_cast<RdmaMemInfo>(hostBase + 2 sqBytes + 2 * cqBytes);
	133	+ return true;
	134	+}
	135	+
	136	+inline bool WorkspaceManager::FillLocalMemoryInfo(RdmaMemInfo* memory)
	137	+{
	138	+ bool localMrRead = rankCount_ == 1;
	139	+ if (rankCount_ == 1) {
	140	+ memory[rankId_].addr = reinterpret_cast<uint64_t>(symmetricAddr_);
	141	+ memory[rankId_].size = symmetricSize_;
	142	+ }
	143	+ for (size_t channel = 0; channel < channelHandles_.size() && !localMrRead; ++channel) {
	144	+ host::ChannelEntity entity{};
	145	+ if (!ReadChannelEntity(channelHandles_[channel], entity)) {
	146	+ std::cerr << "[RoCE] failed to read ChannelEntity while resolving the local MR" << std::endl;
	147	+ return false;
	148	+ }
	149	+ if (!ValidateChannelEntity(entity, channelPeer_[channel]) \|\| entity.localBufferNum == 0 \|\|
	150	+ entity.localBufferAddr == nullptr) {
	151	+ std::cerr << "[RoCE] local registered buffer is absent for peer " << channelPeer_[channel] << std::endl;
	152	+ return false;
	153	+ }
	154	+ host::RegedBufferEntity localBuffer{};
	155	+ if (!ReadDeviceStruct(entity.localBufferAddr, &localBuffer, sizeof(localBuffer)) \|\|
	156	+ !ValidateRegisteredBuffer(
	157	+ localBuffer, reinterpret_cast<uint64_t>(symmetricAddr_), symmetricSize_, "local", rankId_)) {
	158	+ return false;
	159	+ }
	160	+ memory[rankId_].addr = localBuffer.bufferInfo.rma.addr;
	161	+ memory[rankId_].size = localBuffer.bufferInfo.rma.size;
	162	+ memory[rankId_].lkey = localBuffer.bufferInfo.rma.protectionInfo.memInfo.roce.lkey;
	163	+ memory[rankId_].rkey = localBuffer.bufferInfo.rma.protectionInfo.memInfo.roce.rkey;
	164	+ localMrRead = true;
	165	+ }
	166	+ if (!localMrRead) {
	167	+ std::cerr << "[RoCE] failed to read the local registered buffer from any channel" << std::endl;
	168	+ return false;
	169	+ }
	170	+ return true;
	171	+}
	172	+
	173	+inline bool WorkspaceManager::FillPeerRdmaInfo(size_t channelIndex, RdmaInfoHostLayout& layout)
	174	+{
	175	+ const uint32_t peer = channelPeer_[channelIndex];
	176	+ host::ChannelEntity entity{};
	177	+ if (!ReadChannelEntity(channelHandles_[channelIndex], entity)) {
	178	+ std::cerr << "[RoCE] read ChannelEntity failed for peer " << peer << std::endl;
	179	+ return false;
	180	+ }
	181	+ if (!ValidateChannelEntity(entity, peer)) {
	182	+ return false;
	183	+ }
	184	+ if (entity.remoteBufferNum == 0 \|\| entity.remoteBufferAddr == nullptr) {
	185	+ std::cerr << "[RoCE] peer " << peer << " has no exchanged remote registered buffer" << std::endl;
	186	+ return false;
	187	+ }
	188	+ host::RegedBufferEntity remoteBuffer{};
	189	+ if (!ReadDeviceStruct(entity.remoteBufferAddr, &remoteBuffer, sizeof(remoteBuffer)) \|\|
	190	+ !ValidateRegisteredBuffer(remoteBuffer, peerSymAddrs_[peer], symmetricSize_, "remote", peer)) {
	191	+ return false;
	192	+ }
	193	+ layout.memory[peer].addr = remoteBuffer.bufferInfo.rma.addr;
	194	+ layout.memory[peer].size = remoteBuffer.bufferInfo.rma.size;
	195	+ layout.memory[peer].lkey = remoteBuffer.bufferInfo.rma.protectionInfo.memInfo.roce.lkey;
	196	+ layout.memory[peer].rkey = remoteBuffer.bufferInfo.rma.protectionInfo.memInfo.roce.rkey;
	197	+ host::SqContext sq{};
	198	+ if (entity.sqNum == 0 \|\| entity.sqContextAddr == nullptr \|\|
	199	+ !ReadDeviceStruct(entity.sqContextAddr, &sq, sizeof(sq)) \|\| !ValidateSqContext(sq, peer)) {
	200	+ return false;
	201	+ }
	202	+ CopyRoceSq(layout.sq[peer], sq);
	203	+ layout.rq[peer] = layout.sq[peer];
	204	+ host::CqContext cq{};
	205	+ if (entity.cqNum == 0 \|\| entity.cqContextAddr == nullptr \|\|
	206	+ !ReadDeviceStruct(entity.cqContextAddr, &cq, sizeof(cq)) \|\| !ValidateCqContext(cq, peer) \|\|
	207	+ !InitializeQueueMirrors(sq, cq, peer)) {
	208	+ return false;
	209	+ }
	210	+ CopyRoceCq(layout.scq[peer], cq);
	211	+ layout.rcq[peer] = layout.scq[peer];
	212	+ return true;
	213	+}
	214	+
	215	+inline bool WorkspaceManager::FillPeerRdmaInfoEntries(RdmaInfoHostLayout& layout)
	216	+{
	217	+ for (size_t channelIndex = 0; channelIndex < channelHandles_.size(); ++channelIndex) {
	218	+ if (!FillPeerRdmaInfo(channelIndex, layout)) {
	219	+ return false;
	220	+ }
	221	+ }
	222	+ return true;
	223	+}
	224	+
	225	+inline void WorkspaceManager::LogRdmaInfo(const RdmaInfoHostLayout& layout) const
	226	+{
	227	+ if (!Verbose()) {
	228	+ return;
	229	+ }
	230	+ std::cerr << "[RoCE][rank " << rankId_ << "] (4/4) RdmaInfo filled (qpNum=" << layout.info->qpNum
	231	+ << "):" << std::endl;
	232	+ std::cerr << "[RoCE][rank " << rankId_ << "] localMR[" << rankId_ << "] addr=0x" << std::hex
	233	+ << layout.memory[rankId_].addr << std::dec << " size=" << layout.memory[rankId_].size << " lkey=0x"
	234	+ << std::hex << layout.memory[rankId_].lkey << " rkey=0x" << layout.memory[rankId_].rkey << std::dec
	235	+ << std::endl;
	236	+ for (uint32_t peer : channelPeer_) {
	237	+ std::cerr << "[RoCE][rank " << rankId_ << "] peer[" << peer << "] MR addr=0x" << std::hex
	238	+ << layout.memory[peer].addr << " rkey=0x" << layout.memory[peer].rkey << std::dec
	239	+ << " \| SQ wqn=" << layout.sq[peer].wqn << " depth=" << layout.sq[peer].depth << " sqVa=0x" << std::hex
	240	+ << layout.sq[peer].bufAddr << " dbHw=0x" << layout.sq[peer].dbAddr << " dbSw=0x"
	241	+ << layout.sq[peer].dbSwAddr << std::dec << " mtuShift=" << static_cast<int>(layout.sq[peer].mtuShift)
	242	+ << " dbCos=" << static_cast<int>(layout.sq[peer].dbCos) << " \| CQ cqn=" << layout.scq[peer].cqn
	243	+ << " depth=" << layout.scq[peer].depth << " cqVa=0x" << std::hex << layout.scq[peer].bufAddr
	244	+ << " dbSw=0x" << layout.scq[peer].dbSwAddr << std::dec << std::endl;
	245	+ }
	246	+}
	247	+
	248	+inline bool WorkspaceManager::CopyRdmaInfoToDevice(const std::vector<uint8_t>& hostBuffer, size_t totalSize) const
	249	+{
	250	+ if (aclrtMemcpy(rdmaInfoDevice_, totalSize, hostBuffer.data(), totalSize, ACL_MEMCPY_HOST_TO_DEVICE) == 0) {
	251	+ return true;
	252	+ }
	253	+ std::cerr << "[RoCE] aclrtMemcpy(rdmaInfo H2D) failed" << std::endl;
	254	+ return false;
	255	+}
	256	+
	257	+inline bool WorkspaceManager::FillRdmaInfo()
	258	+{
	259	+ std::vector<uint8_t> hostBuffer;
	260	+ RdmaInfoHostLayout layout;
	261	+ if (!PrepareRdmaInfoLayout(hostBuffer, layout) \|\| !FillLocalMemoryInfo(layout.memory) \|\|
	262	+ !FillPeerRdmaInfoEntries(layout)) {
	263	+ return false;
	264	+ }
	265	+ LogRdmaInfo(layout);
	266	+ return CopyRdmaInfoToDevice(hostBuffer, layout.totalSize);
	267	+}
	268	+
	269	+inline bool WorkspaceManager::IsPowerOfTwo(uint64_t value) { return value != 0 && (value & (value - 1)) == 0; }
	270	+
	271	+inline bool WorkspaceManager::IsU32Aligned(uint64_t address) { return address % alignof(uint32_t) == 0; }
	272	+
	273	+inline bool WorkspaceManager::IsCacheLineAligned(uint64_t address) { return address % kHns1825WqebbSize == 0; }
	274	+
	275	+inline bool WorkspaceManager::ValidateChannelEntity(const host::ChannelEntity& entity, uint32_t peer)
	276	+{
	277	+ if (entity.engine != COMM_ENGINE_AIV \|\| entity.protocol != COMM_PROTOCOL_ROCE) {
	278	+ std::cerr << "[RoCE] peer " << peer
	279	+ << " ChannelEntity has unexpected engine/protocol: " << static_cast<int>(entity.engine) << "/"
	280	+ << static_cast<int>(entity.protocol) << std::endl;
	281	+ return false;
	282	+ }
	283	+ return true;
	284	+}
	285	+
	286	+inline bool WorkspaceManager::ValidateRegisteredBuffer(
	287	+ const host::RegedBufferEntity& buffer, uint64_t expectedAddr, uint64_t expectedSize, const char* side,
	288	+ uint32_t rank)
	289	+{
	290	+ const auto& rma = buffer.bufferInfo.rma;
	291	+ if (buffer.type != host::REGED_BUFFER_TYPE_RMA \|\| rma.protectionInfo.type != host::PROTECTION_TYPE_ROCE \|\|
	292	+ rma.addr != expectedAddr \|\| rma.size < expectedSize) {
	293	+ std::cerr << "[RoCE] " << side << " MR for rank " << rank
	294	+ << " has invalid type/protection/address/size: type=" << static_cast<int>(buffer.type)
	295	+ << " protection=" << static_cast<int>(rma.protectionInfo.type) << " addr=0x" << std::hex << rma.addr
	296	+ << " expected=0x" << expectedAddr << std::dec << " size=" << rma.size
	297	+ << " expected-at-least=" << expectedSize << std::endl;
	298	+ return false;
	299	+ }
	300	+ return true;
	301	+}
	302	+
	303	+inline bool WorkspaceManager::ValidateSqContext(const host::SqContext& sq, uint32_t peer)
	304	+{
	305	+ const auto context = DecodeRoceSqContext(sq);
	306	+ if (sq.type != host::SQ_CONTEXT_TYPE_ROCE \|\| !IsPowerOfTwo(context.depth) \|\|
	307	+ context.depth <= kHns1825PollCqThreshold \|\| context.sqVa == 0 \|\| context.headAddr == 0 \|\|
	308	+ context.tailAddr == 0 \|\| context.dbHwVa == 0 \|\| context.dbSwVa == 0 \|\| !IsCacheLineAligned(context.sqVa) \|\|
	309	+ !IsU32Aligned(context.headAddr) \|\| !IsU32Aligned(context.tailAddr) \|\| !IsU32Aligned(context.dbSwVa) \|\|
	310	+ context.mtuShift > kDbVendorFieldMask \|\| context.dbCos > kDbVendorFieldMask) {
	311	+ std::cerr << "[RoCE] peer " << peer << " has an unusable RoCE SQ context: type=" << static_cast<int>(sq.type)
	312	+ << " depth=" << context.depth << " wqeSize=" << context.wqeSize << " abi=" << context.abiName
	313	+ << " mtuShift=" << static_cast<int>(context.mtuShift) << " dbCos=" << static_cast<int>(context.dbCos)
	314	+ << std::endl;
	315	+ return false;
	316	+ }
	317	+ return true;
	318	+}
	319	+
	320	+inline bool WorkspaceManager::ValidateCqContext(const host::CqContext& cq, uint32_t peer)
	321	+{
	322	+ const auto context = host::ExtractRoceCqContextSplitDb(cq);
	323	+ const uint32_t cqeSize = context.cqeSize == 0 ? kHns1825DefaultCqeSize : context.cqeSize;
	324	+ const uint64_t cqRing = static_cast<uint64_t>(context.cqDepth);
	325	+ const bool supportedCqeSize = cqeSize == sizeof(Hns1825Cqe) \|\| cqeSize == kHns1825DefaultCqeSize;
	326	+ if (cq.type != host::CQ_CONTEXT_TYPE_ROCE \|\| !IsPowerOfTwo(cqRing) \|\| !supportedCqeSize \|\| context.cqVa == 0 \|\|
	327	+ context.headAddr == 0 \|\| context.tailAddr == 0 \|\| context.dbSwVa == 0 \|\| !IsCacheLineAligned(context.cqVa) \|\|
	328	+ !IsU32Aligned(context.tailAddr) \|\| !IsU32Aligned(context.dbSwVa)) {
	329	+ std::cerr << "[RoCE] peer " << peer << " has an unusable RoCE CQ context: type=" << static_cast<int>(cq.type)
	330	+ << " depth=" << context.cqDepth << " cqeSize=" << context.cqeSize << std::endl;
	331	+ return false;
	332	+ }
	333	+ return true;
	334	+}
	335	+
	336	+inline void WorkspaceManager::CopyRoceSq(RoceSqCtx& destination, const host::SqContext& source)
	337	+{
	338	+ const auto context = DecodeRoceSqContext(source);
	339	+ destination.wqn = context.qpn;
	340	+ destination.bufAddr = context.sqVa;
	341	+ destination.wqeSize = context.wqeSize;
	342	+ destination.depth = context.depth;
	343	+ destination.headAddr = context.headAddr;
	344	+ destination.tailAddr = context.tailAddr;
	345	+ destination.dbMode = RdmaDbMode::SW_DB;
	346	+ destination.dbAddr = context.dbHwVa;
	347	+ destination.sl = context.sl;
	348	+ destination.amoAddr = 0;
	349	+ destination.amoLkey = 0;
	350	+ destination.dbSwAddr = context.dbSwVa;
	351	+ destination.mtuShift = context.mtuShift;
	352	+ destination.dbCos = context.dbCos;
	353	+}
	354	+
	355	+inline void WorkspaceManager::CopyRoceCq(RoceCqCtx& destination, const host::CqContext& source)
	356	+{
	357	+ const auto context = host::ExtractRoceCqContextSplitDb(source);
	358	+ destination.cqn = context.cqn;
	359	+ destination.bufAddr = context.cqVa;
	360	+ destination.cqeSize = context.cqeSize;
	361	+ destination.depth = context.cqDepth;
	362	+ destination.headAddr = context.headAddr;
	363	+ destination.tailAddr = context.tailAddr;
	364	+ destination.dbMode = RdmaDbMode::SW_DB;
	365	+ destination.dbAddr = context.dbHwVa;
	366	+ destination.dbSwAddr = context.dbSwVa;
	367	+}
	368	+
	369	+inline bool WorkspaceManager::ReadChannelEntity(ChannelHandle handle, host::ChannelEntity& output)
	370	+{
	371	+ if (handle == 0) {
	372	+ return false;
	373	+ }
	374	+ return ReadDeviceStruct(reinterpret_cast<void*>(static_cast<uintptr_t>(handle)), &output, sizeof(output));
	375	+}
	376	+
	377	+inline bool WorkspaceManager::ReadDeviceStruct(const void* devicePointer, void* host, size_t size)
	378	+{
	379	+ if (devicePointer == nullptr) {
	380	+ return false;
	381	+ }
	382	+ return aclrtMemcpy(host, size, devicePointer, size, ACL_MEMCPY_DEVICE_TO_HOST) == 0;
	383	+}
	384	+
	385	+} // namespace hns_1825
	386	+} // namespace rdma
	387	+} // namespace comm
	388	+} // namespace pto
	389	+
	390	+#endif // PTO_COMM_ASYNC_RDMA_BACKENDS_HNS_1825_WORKSPACE_MANAGER_INFO_HPP

Apkg_inc/pto/comm/async/rdma/backends/hns_1825/hns_1825_workspace_manager_lifecycle.hpp+382-0

@@ -0,0 +1,382 @@
	1	+/**
	2	+Copyright (c) 2026 Huawei Technologies Co., Ltd.
	3	+This program is free software, you can redistribute it and/or modify it under the terms and conditions of
	4	+CANN Open Software License Agreement Version 2.0 (the "License").
	5	+Please refer to the License for details. You may not use this file except in compliance with the License.
	6	+THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED,
	7	+INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE.
	8	+See LICENSE in the root of the software repository for the full text of the License.
	9	+*/
	10	+
	11	+#ifndef PTO_COMM_ASYNC_RDMA_BACKENDS_HNS_1825_WORKSPACE_MANAGER_LIFECYCLE_HPP
	12	+#define PTO_COMM_ASYNC_RDMA_BACKENDS_HNS_1825_WORKSPACE_MANAGER_LIFECYCLE_HPP
	13	+
	14	+#ifndef PTO_COMM_ASYNC_RDMA_BACKENDS_HNS_1825_WORKSPACE_MANAGER_HPP
	15	+#error "Include hns_1825_workspace_manager.hpp instead of this internal implementation header."
	16	+#endif
	17	+
	18	+namespace pto {
	19	+namespace comm {
	20	+namespace rdma {
	21	+namespace hns_1825 {
	22	+
	23	+inline WorkspaceManager::~WorkspaceManager() { (void)Finalize(); }
	24	+
	25	+inline bool WorkspaceManager::Init(
	26	+ uint32_t rankId, uint32_t rankCount, uint32_t phyId, const std::string& localIp, uint16_t basePort,
	27	+ const std::vector<std::string>& peerIps, const std::vector<uint32_t>& peerPhyIds,
	28	+ const std::vector<uint64_t>& peerSymAddrs, void* symmetricAddr, uint64_t symmetricSize)
	29	+{
	30	+ if (HasOwnedResources() && !Finalize()) {
	31	+ std::cerr << "[RoCE] previous session cleanup reported errors; continuing with a fresh HCOMM session"
	32	+ << std::endl;
	33	+ }
	34	+ if (!ValidateInitArguments(
	35	+ rankId, rankCount, phyId, localIp, basePort, peerIps, peerPhyIds, peerSymAddrs, symmetricAddr,
	36	+ symmetricSize)) {
	37	+ return false;
	38	+ }
	39	+ AssignInitArguments(
	40	+ rankId, rankCount, phyId, localIp, basePort, peerIps, peerPhyIds, peerSymAddrs, symmetricAddr, symmetricSize);
	41	+ const auto initStart = std::chrono::steady_clock::now();
	42	+ Trace(
	43	+ "INIT begin manager=", static_cast<const void*>(this), " endpoint=", endpoint_, " memHandle=", memHandle_,
	44	+ " channels=", channelHandles_.size(), " rdmaInfo=", rdmaInfoDevice_, " symAddr=", symmetricAddr_,
	45	+ " symSize=", symmetricSize_, " basePort=", basePort_);
	46	+ LogInitConfiguration();
	47	+ if (!RunInitStage(&WorkspaceManager::CreateEndpoint, "EndpointCreate") \|\|
	48	+ !RunInitStage(&WorkspaceManager::RegisterMemory, "MemReg") \|\|
	49	+ !RunInitStage(&WorkspaceManager::BuildChannels, "ChannelCreateReady") \|\|
	50	+ !RunInitStage(&WorkspaceManager::FillRdmaInfo, "FillRdmaInfo")) {
	51	+ (void)Finalize();
	52	+ return false;
	53	+ }
	54	+ if (Verbose()) {
	55	+ std::cerr << "[RoCE][rank " << rankId_ << "] connect DONE: workspace(RdmaInfo)=0x" << std::hex
	56	+ << reinterpret_cast<uint64_t>(rdmaInfoDevice_) << std::dec << std::endl;
	57	+ }
	58	+ initialized_ = true;
	59	+ Trace(
	60	+ "INIT success total_us=", ElapsedUs(initStart), " endpoint=", endpoint_, " memHandle=", memHandle_,
	61	+ " channels=", channelHandles_.size(), " rdmaInfo=", rdmaInfoDevice_);
	62	+ return true;
	63	+}
	64	+
	65	+inline bool WorkspaceManager::Verbose()
	66	+{
	67	+ const char* value = std::getenv("PTO_ROCE_VERBOSE");
	68	+ return value != nullptr && value[0] == '1';
	69	+}
	70	+
	71	+inline bool WorkspaceManager::Finalize()
	72	+{
	73	+ const bool hadResources = HasOwnedResources();
	74	+ const auto finalizeStart = std::chrono::steady_clock::now();
	75	+ if (hadResources) {
	76	+ Trace(
	77	+ "FINALIZE begin initialized=", initialized_, " endpoint=", endpoint_, " memHandle=", memHandle_,
	78	+ " channels=", channelHandles_.size(), " rdmaInfo=", rdmaInfoDevice_, " symAddr=", symmetricAddr_);
	79	+ }
	80	+ bool ok = true;
	81	+ auto phaseStart = std::chrono::steady_clock::now();
	82	+ const bool channelsOk = DestroyChannels();
	83	+ if (!channelsOk) {
	84	+ Trace("FINALIZE failed stage=ChannelDestroy elapsed_us=", ElapsedUs(phaseStart));
	85	+ ok = false;
	86	+ } else if (hadResources) {
	87	+ Trace("FINALIZE stage=ChannelDestroy ok elapsed_us=", ElapsedUs(phaseStart));
	88	+ }
	89	+ phaseStart = std::chrono::steady_clock::now();
	90	+ const bool memoryOk = UnregisterMemory();
	91	+ if (!memoryOk) {
	92	+ Trace("FINALIZE failed stage=MemUnreg elapsed_us=", ElapsedUs(phaseStart));
	93	+ ok = false;
	94	+ } else if (hadResources) {
	95	+ Trace("FINALIZE stage=MemUnreg ok elapsed_us=", ElapsedUs(phaseStart));
	96	+ }
	97	+ phaseStart = std::chrono::steady_clock::now();
	98	+ const bool endpointOk = DestroyEndpoint();
	99	+ if (!endpointOk) {
	100	+ Trace("FINALIZE failed stage=EndpointDestroy elapsed_us=", ElapsedUs(phaseStart));
	101	+ ok = false;
	102	+ } else if (hadResources) {
	103	+ Trace("FINALIZE stage=EndpointDestroy ok elapsed_us=", ElapsedUs(phaseStart));
	104	+ }
	105	+ phaseStart = std::chrono::steady_clock::now();
	106	+ const bool deviceInfoOk = FreeDeviceInfo();
	107	+ if (!deviceInfoOk) {
	108	+ Trace("FINALIZE failed stage=FreeRdmaInfo elapsed_us=", ElapsedUs(phaseStart));
	109	+ ok = false;
	110	+ } else if (hadResources) {
	111	+ Trace("FINALIZE stage=FreeRdmaInfo ok elapsed_us=", ElapsedUs(phaseStart));
	112	+ }
	113	+ ResetOwnedState();
	114	+ if (hadResources) {
	115	+ Trace("FINALIZE end ok=", ok, " total_us=", ElapsedUs(finalizeStart));
	116	+ }
	117	+ return ok;
	118	+}
	119	+
	120	+template <typename... Args>
	121	+inline void WorkspaceManager::Trace(Args&&... args) const
	122	+{
	123	+ if (!Verbose()) {
	124	+ return;
	125	+ }
	126	+ std::ostringstream stream;
	127	+ stream << "[RoCE][case " << traceId_ << "][rank " << rankId_ << "] ";
	128	+ (stream << ... << std::forward<Args>(args));
	129	+ stream << '\n';
	130	+ const std::string line = stream.str();
	131	+ (void)std::fwrite(line.data(), 1, line.size(), stderr);
	132	+ std::fflush(stderr);
	133	+}
	134	+
	135	+inline uint64_t WorkspaceManager::ElapsedUs(const std::chrono::steady_clock::time_point& start)
	136	+{
	137	+ return static_cast<uint64_t>(
	138	+ std::chrono::duration_cast<std::chrono::microseconds>(std::chrono::steady_clock::now() - start).count());
	139	+}
	140	+
	141	+inline std::string WorkspaceManager::Hex(uint64_t value)
	142	+{
	143	+ std::ostringstream stream;
	144	+ stream << "0x" << std::hex << value;
	145	+ return stream.str();
	146	+}
	147	+
	148	+inline void WorkspaceManager::AssignInitArguments(
	149	+ uint32_t rankId, uint32_t rankCount, uint32_t phyId, const std::string& localIp, uint16_t basePort,
	150	+ const std::vector<std::string>& peerIps, const std::vector<uint32_t>& peerPhyIds,
	151	+ const std::vector<uint64_t>& peerSymAddrs, void* symmetricAddr, uint64_t symmetricSize)
	152	+{
	153	+ rankId_ = rankId;
	154	+ rankCount_ = rankCount;
	155	+ phyId_ = phyId;
	156	+ localIp_ = localIp;
	157	+ basePort_ = basePort;
	158	+ peerIps_ = peerIps;
	159	+ peerPhyIds_ = peerPhyIds;
	160	+ peerSymAddrs_ = peerSymAddrs;
	161	+ symmetricAddr_ = symmetricAddr;
	162	+ symmetricSize_ = symmetricSize;
	163	+}
	164	+
	165	+inline void WorkspaceManager::LogInitConfiguration() const
	166	+{
	167	+ if (!Verbose()) {
	168	+ return;
	169	+ }
	170	+ std::cerr << "[RoCE][rank " << rankId_ << "/" << rankCount_ << "] connect start: phyId=" << phyId_
	171	+ << " localIp=" << localIp_ << " basePort=" << basePort_ << " symAddr=0x" << std::hex
	172	+ << reinterpret_cast<uint64_t>(symmetricAddr_) << std::dec << " symSize=" << symmetricSize_ << std::endl;
	173	+ for (uint32_t rank = 0; rank < rankCount_; ++rank) {
	174	+ std::cerr << "[RoCE][rank " << rankId_ << "] peer[" << rank << "] ip=" << peerIps_[rank]
	175	+ << " phyId=" << peerPhyIds_[rank] << " symAddr=0x" << std::hex << peerSymAddrs_[rank] << std::dec
	176	+ << std::endl;
	177	+ }
	178	+}
	179	+
	180	+inline bool WorkspaceManager::RunInitStage(bool (WorkspaceManager::stage)(), const char stageName)
	181	+{
	182	+ const auto phaseStart = std::chrono::steady_clock::now();
	183	+ if (!(this->*stage)()) {
	184	+ return false;
	185	+ }
	186	+ Trace("INIT stage=", stageName, " ok elapsed_us=", ElapsedUs(phaseStart));
	187	+ return true;
	188	+}
	189	+
	190	+inline bool WorkspaceManager::DestroyChannels()
	191	+{
	192	+ if (channelHandles_.empty()) {
	193	+ return true;
	194	+ }
	195	+ std::vector<ChannelHandle> valid;
	196	+ valid.reserve(channelHandles_.size());
	197	+ for (ChannelHandle handle : channelHandles_) {
	198	+ if (handle != 0) {
	199	+ valid.push_back(handle);
	200	+ }
	201	+ }
	202	+ if (valid.empty()) {
	203	+ channelHandles_.clear();
	204	+ channelPeer_.clear();
	205	+ return true;
	206	+ }
	207	+ if (Verbose()) {
	208	+ std::cerr << "[RoCE][rank " << rankId_ << "] HcommChannelDestroy n=" << valid.size() << std::endl;
	209	+ }
	210	+ for (size_t index = 0; index < channelHandles_.size(); ++index) {
	211	+ const ChannelHandle handle = channelHandles_[index];
	212	+ if (handle == 0) {
	213	+ continue;
	214	+ }
	215	+ const uint32_t peer = index < channelPeer_.size() ? channelPeer_[index] : UINT32_MAX;
	216	+ Trace("DESTROY channel index=", index, " peer=", peer, " handle=", Hex(static_cast<uint64_t>(handle)));
	217	+ }
	218	+ const HcommResult result = HcommChannelDestroy(valid.data(), static_cast<uint32_t>(valid.size()));
	219	+ channelHandles_.clear();
	220	+ channelPeer_.clear();
	221	+ if (result != 0) {
	222	+ std::cerr << "[RoCE] HcommChannelDestroy failed: " << result << std::endl;
	223	+ return false;
	224	+ }
	225	+ if (Verbose()) {
	226	+ std::cerr << "[RoCE][rank " << rankId_ << "] HcommChannelDestroy ok" << std::endl;
	227	+ }
	228	+ Trace("DESTROY channels API returned success");
	229	+ return true;
	230	+}
	231	+
	232	+inline bool WorkspaceManager::UnregisterMemory()
	233	+{
	234	+ if (memHandle_ == nullptr) {
	235	+ return true;
	236	+ }
	237	+ const HcommMemHandle memHandle = memHandle_;
	238	+ memHandle_ = nullptr;
	239	+ if (endpoint_ == nullptr) {
	240	+ std::cerr << "[RoCE] cannot unregister memory without its endpoint" << std::endl;
	241	+ return false;
	242	+ }
	243	+ const HcommResult result = HcommMemUnreg(endpoint_, memHandle);
	244	+ if (result != 0) {
	245	+ std::cerr << "[RoCE] HcommMemUnreg failed: " << result << std::endl;
	246	+ return false;
	247	+ }
	248	+ Trace("DESTROY mem-unreg API returned success handle=", memHandle);
	249	+ return true;
	250	+}
	251	+
	252	+inline bool WorkspaceManager::DestroyEndpoint()
	253	+{
	254	+ if (endpoint_ == nullptr) {
	255	+ return true;
	256	+ }
	257	+ const EndpointHandle endpoint = endpoint_;
	258	+ endpoint_ = nullptr;
	259	+ const HcommResult result = HcommEndpointDestroy(endpoint);
	260	+ if (result != 0) {
	261	+ std::cerr << "[RoCE] HcommEndpointDestroy failed: " << result << std::endl;
	262	+ return false;
	263	+ }
	264	+ Trace("DESTROY endpoint API returned success handle=", endpoint);
	265	+ return true;
	266	+}
	267	+
	268	+inline bool WorkspaceManager::FreeDeviceInfo()
	269	+{
	270	+ if (rdmaInfoDevice_ == nullptr) {
	271	+ return true;
	272	+ }
	273	+ void* rdmaInfo = rdmaInfoDevice_;
	274	+ rdmaInfoDevice_ = nullptr;
	275	+ const aclError result = aclrtFree(rdmaInfo);
	276	+ if (result != ACL_SUCCESS) {
	277	+ std::cerr << "[RoCE] aclrtFree(rdmaInfo) failed: " << static_cast<int>(result) << std::endl;
	278	+ return false;
	279	+ }
	280	+ Trace("DESTROY rdma-info free returned success addr=", rdmaInfo);
	281	+ return true;
	282	+}
	283	+
	284	+inline bool WorkspaceManager::HasOwnedResources() const
	285	+{
	286	+ return initialized_ \|\| endpoint_ != nullptr \|\| memHandle_ != nullptr \|\| rdmaInfoDevice_ != nullptr \|\|
	287	+ !channelHandles_.empty();
	288	+}
	289	+
	290	+inline bool WorkspaceManager::ValidateBasicInitArguments(
	291	+ uint32_t rankId, uint32_t rankCount, uint32_t phyId, const std::string& localIp, uint16_t basePort,
	292	+ const std::vector<std::string>& peerIps, const std::vector<uint32_t>& peerPhyIds,
	293	+ const std::vector<uint64_t>& peerSymAddrs, void* symmetricAddr, uint64_t symmetricSize)
	294	+{
	295	+ if (rankCount == 0 \|\| rankId >= rankCount \|\| peerIps.size() != rankCount \|\| peerPhyIds.size() != rankCount \|\|
	296	+ peerSymAddrs.size() != rankCount \|\| localIp.empty() \|\| symmetricAddr == nullptr \|\| symmetricSize == 0) {
	297	+ std::cerr << "[RoCE] invalid rank, peer, IP, or symmetric-memory arguments" << std::endl;
	298	+ return false;
	299	+ }
	300	+ if (peerIps[rankId] != localIp \|\| peerPhyIds[rankId] != phyId \|\|
	301	+ peerSymAddrs[rankId] != reinterpret_cast<uint64_t>(symmetricAddr)) {
	302	+ std::cerr << "[RoCE] local IP/phyId/address does not match this rank's gathered peer entry" << std::endl;
	303	+ return false;
	304	+ }
	305	+ if (basePort > UINT16_MAX - (kMaxRanksPerNic * kMaxRanksPerNic - 1)) {
	306	+ std::cerr << "[RoCE] basePort leaves no room for the per-rank channel port range" << std::endl;
	307	+ return false;
	308	+ }
	309	+ return true;
	310	+}
	311	+
	312	+inline bool WorkspaceManager::ValidatePeerPortMapping(
	313	+ uint32_t rankCount, const std::vector<std::string>& peerIps, const std::vector<uint32_t>& peerPhyIds,
	314	+ const std::vector<uint64_t>& peerSymAddrs)
	315	+{
	316	+ for (uint32_t rank = 0; rank < rankCount; ++rank) {
	317	+ if (peerIps[rank].empty() \|\| peerPhyIds[rank] == UINT32_MAX \|\| peerSymAddrs[rank] == 0) {
	318	+ std::cerr << "[RoCE] peer " << rank << " has an empty IP, invalid phyId, or null symmetric address"
	319	+ << std::endl;
	320	+ return false;
	321	+ }
	322	+ uint32_t ranksOnNic = 0;
	323	+ bool occupiedSlots[kMaxRanksPerNic] = {};
	324	+ for (uint32_t peer = 0; peer < rankCount; ++peer) {
	325	+ if (peerIps[peer] != peerIps[rank]) {
	326	+ continue;
	327	+ }
	328	+ ++ranksOnNic;
	329	+ const uint32_t slot = peer % kMaxRanksPerNic;
	330	+ if (occupiedSlots[slot]) {
	331	+ std::cerr << "[RoCE] ranks sharing NIC IP " << peerIps[rank]
	332	+ << " collide after modulo port mapping at slot " << slot << std::endl;
	333	+ return false;
	334	+ }
	335	+ occupiedSlots[slot] = true;
	336	+ }
	337	+ if (ranksOnNic > kMaxRanksPerNic) {
	338	+ std::cerr << "[RoCE] more than " << kMaxRanksPerNic << " ranks share NIC IP " << peerIps[rank]
	339	+ << "; the channel port mapping would collide" << std::endl;
	340	+ return false;
	341	+ }
	342	+ }
	343	+ return true;
	344	+}
	345	+
	346	+inline bool WorkspaceManager::ValidateInitArguments(
	347	+ uint32_t rankId, uint32_t rankCount, uint32_t phyId, const std::string& localIp, uint16_t basePort,
	348	+ const std::vector<std::string>& peerIps, const std::vector<uint32_t>& peerPhyIds,
	349	+ const std::vector<uint64_t>& peerSymAddrs, void* symmetricAddr, uint64_t symmetricSize)
	350	+{
	351	+ return ValidateBasicInitArguments(
	352	+ rankId, rankCount, phyId, localIp, basePort, peerIps, peerPhyIds, peerSymAddrs, symmetricAddr,
	353	+ symmetricSize) &&
	354	+ ValidatePeerPortMapping(rankCount, peerIps, peerPhyIds, peerSymAddrs);
	355	+}
	356	+
	357	+inline void WorkspaceManager::ResetOwnedState()
	358	+{
	359	+ channelHandles_.clear();
	360	+ channelPeer_.clear();
	361	+ peerIps_.clear();
	362	+ peerPhyIds_.clear();
	363	+ peerSymAddrs_.clear();
	364	+ memHandle_ = nullptr;
	365	+ endpoint_ = nullptr;
	366	+ rdmaInfoDevice_ = nullptr;
	367	+ symmetricAddr_ = nullptr;
	368	+ symmetricSize_ = 0;
	369	+ localIp_.clear();
	370	+ rankId_ = 0;
	371	+ rankCount_ = 1;
	372	+ phyId_ = 0;
	373	+ basePort_ = 60032;
	374	+ initialized_ = false;
	375	+}
	376	+
	377	+} // namespace hns_1825
	378	+} // namespace rdma
	379	+} // namespace comm
	380	+} // namespace pto
	381	+
	382	+#endif // PTO_COMM_ASYNC_RDMA_BACKENDS_HNS_1825_WORKSPACE_MANAGER_LIFECYCLE_HPP

Apkg_inc/pto/comm/async/rdma/rdma_async_intrin.hpp+239-0

Apkg_inc/pto/comm/async/rdma/rdma_device_common.hpp+55-0

Apkg_inc/pto/comm/async/rdma/rdma_types.hpp+95-0

Apkg_inc/pto/comm/async/rdma/rdma_workspace_types.hpp+73-0

Mtests/README.md+1-1

Mtests/README_zh.md+1-1

Mtests/npu/a5/comm/st/CMakeLists.txt+17-1

Mtests/npu/a5/comm/st/testcase/CMakeLists.txt+7-1

Atests/npu/a5/comm/st/testcase/tget_async_rdma/CMakeLists.txt+14-0

Atests/npu/a5/comm/st/testcase/tget_async_rdma/README.md+3-0

Atests/npu/a5/comm/st/testcase/tget_async_rdma/README_zh.md+3-0

Atests/npu/a5/comm/st/testcase/tget_async_rdma/gen_data.py+21-0

Atests/npu/a5/comm/st/testcase/tget_async_rdma/main.cpp+113-0

Atests/npu/a5/comm/st/testcase/tput_async_rdma/CMakeLists.txt+11-0

Atests/npu/a5/comm/st/testcase/tput_async_rdma/README.md+51-0

Atests/npu/a5/comm/st/testcase/tput_async_rdma/README_zh.md+51-0

Atests/npu/a5/comm/st/testcase/tput_async_rdma/backends/hns_1825/hns_1825_bootstrap.hpp+442-0

Atests/npu/a5/comm/st/testcase/tput_async_rdma/backends/rdma_test_backend.hpp+60-0

Atests/npu/a5/comm/st/testcase/tput_async_rdma/gen_data.py+21-0

Atests/npu/a5/comm/st/testcase/tput_async_rdma/main.cpp+121-0

Atests/npu/a5/comm/st/testcase/tput_async_rdma/tput_async_rdma_kernel.cpp+1069-0

Atests/npu/a5/comm/st/testcase/tput_async_rdma/tput_async_rdma_kernel.h+40-0

Mtests/run_comm_test.sh+1-1

Mtypos.toml+1-0

	@@ -35,11 +35,11 @@ comm/
35	├── a5/ # A5 (Ascend 950) architecture implementations	35	├── a5/ # A5 (Ascend 950) architecture implementations
36	│ ├── T*.hpp # Sync instructions (include a2a3/ counterparts)	36	│ ├── T*.hpp # Sync instructions (include a2a3/ counterparts)
37	│ └── async/	37	│ └── async/
38	-│ ├── TPutAsync.hpp # TPUT_ASYNC_IMPL (SDMA with MTE fallback + URMA)	38	+│ ├── TPutAsync.hpp # TPUT_ASYNC_IMPL (SDMA with MTE fallback + URMA + RDMA)
39	-│ └── TGetAsync.hpp # TGET_ASYNC_IMPL (SDMA + URMA)	39	+│ └── TGetAsync.hpp # TGET_ASYNC_IMPL (SDMA + URMA + RDMA)
40	│	40	│
41	└── async_common/ # Common async implementations (shared by a2a3/a5)	41	└── async_common/ # Common async implementations (shared by a2a3/a5)
42	- ├── async_types.hpp # SDMA/URMA session and context types	42	+ ├── async_types.hpp # SDMA/URMA/RDMA session and context types
43	├── async_event_impl.hpp # AsyncEvent::Wait/Test, BuildAsyncSession	43	├── async_event_impl.hpp # AsyncEvent::Wait/Test, BuildAsyncSession
44	├── TPutAsyncCommonDetail.hpp # Common TPUT_ASYNC detail helpers + SDMA impl	44	├── TPutAsyncCommonDetail.hpp # Common TPUT_ASYNC detail helpers + SDMA impl
45	└── TGetAsyncCommonDetail.hpp # Common TGET_ASYNC detail helpers + SDMA impl	45	└── TGetAsyncCommonDetail.hpp # Common TGET_ASYNC detail helpers + SDMA impl
	@@ -66,7 +66,7 @@ comm/
66	\| Category \| Instructions \| Description \|	66	\| Category \| Instructions \| Description \|
67	\|---\|---\|---\|	67	\|---\|---\|---\|
68	\| Point-to-Point (sync) \| `TPUT`, `TGET` \| Remote write / read through UB staging tile. Supports single-buffer and ping-pong double-buffering modes. \|	68	\| Point-to-Point (sync) \| `TPUT`, `TGET` \| Remote write / read through UB staging tile. Supports single-buffer and ping-pong double-buffering modes. \|
69	-\| Point-to-Point (async) \| `TPUT_ASYNC`, `TGET_ASYNC` \| GM-to-GM DMA via SDMA or URMA engine. Returns `AsyncEvent` for later Wait/Test. \|	69	+\| Point-to-Point (async) \| `TPUT_ASYNC`, `TGET_ASYNC` \| GM-to-GM DMA via SDMA, URMA, or RDMA engine. Returns `AsyncEvent` for later Wait/Test. \|
70	\| Signal Synchronization \| `TNOTIFY`, `TWAIT`, `TTEST` \| Flag-based cross-NPU synchronization. Signals are `int32_t` scalars or 2D grids. \|	70	\| Signal Synchronization \| `TNOTIFY`, `TWAIT`, `TTEST` \| Flag-based cross-NPU synchronization. Signals are `int32_t` scalars or 2D grids. \|
71	\| Collective \| `TGATHER`, `TSCATTER`, `TBROADCAST`, `TREDUCE` \| Multi-rank operations via `ParallelGroup`. Root-initiated; support chunked 2D sliding and ping-pong. \|	71	\| Collective \| `TGATHER`, `TSCATTER`, `TBROADCAST`, `TREDUCE` \| Multi-rank operations via `ParallelGroup`. Root-initiated; support chunked 2D sliding and ping-pong. \|
72		72

	@@ -35,11 +35,11 @@ comm/
35	├── a5/ # A5（Ascend 950）架构实现	35	├── a5/ # A5（Ascend 950）架构实现
36	│ ├── T*.hpp # 同步指令（include a2a3/ 对应文件）	36	│ ├── T*.hpp # 同步指令（include a2a3/ 对应文件）
37	│ └── async/	37	│ └── async/
38	-│ ├── TPutAsync.hpp # TPUT_ASYNC_IMPL（SDMA + MTE 回退 + URMA）	38	+│ ├── TPutAsync.hpp # TPUT_ASYNC_IMPL（SDMA + MTE 回退 + URMA + RDMA）
39	-│ └── TGetAsync.hpp # TGET_ASYNC_IMPL（SDMA + URMA）	39	+│ └── TGetAsync.hpp # TGET_ASYNC_IMPL（SDMA + URMA + RDMA）
40	│	40	│
41	└── async_common/ # 异步公共实现（a2a3/a5 共享）	41	└── async_common/ # 异步公共实现（a2a3/a5 共享）
42	- ├── async_types.hpp # SDMA/URMA 会话与上下文类型	42	+ ├── async_types.hpp # SDMA/URMA/RDMA 会话与上下文类型
43	├── async_event_impl.hpp # AsyncEvent::Wait/Test、BuildAsyncSession	43	├── async_event_impl.hpp # AsyncEvent::Wait/Test、BuildAsyncSession
44	├── TPutAsyncCommonDetail.hpp # TPUT_ASYNC 公共辅助函数 + SDMA 实现	44	├── TPutAsyncCommonDetail.hpp # TPUT_ASYNC 公共辅助函数 + SDMA 实现
45	└── TGetAsyncCommonDetail.hpp # TGET_ASYNC 公共辅助函数 + SDMA 实现	45	└── TGetAsyncCommonDetail.hpp # TGET_ASYNC 公共辅助函数 + SDMA 实现
	@@ -66,7 +66,7 @@ comm/
66	\| 类别 \| 指令 \| 说明 \|	66	\| 类别 \| 指令 \| 说明 \|
67	\|---\|---\|---\|	67	\|---\|---\|---\|
68	\| 点对点（同步） \| `TPUT`、`TGET` \| 通过 UB 暂存 Tile 的远程写/读。支持单缓冲和 ping-pong 双缓冲模式。 \|	68	\| 点对点（同步） \| `TPUT`、`TGET` \| 通过 UB 暂存 Tile 的远程写/读。支持单缓冲和 ping-pong 双缓冲模式。 \|
69	-\| 点对点（异步） \| `TPUT_ASYNC`、`TGET_ASYNC` \| 通过 SDMA 或 URMA 引擎进行 GM-to-GM DMA。返回 `AsyncEvent` 用于后续 Wait/Test。 \|	69	+\| 点对点（异步） \| `TPUT_ASYNC`、`TGET_ASYNC` \| 通过 SDMA、URMA 或 RDMA 引擎进行 GM-to-GM DMA。返回 `AsyncEvent` 用于后续 Wait/Test。 \|
70	\| 信号同步 \| `TNOTIFY`、`TWAIT`、`TTEST` \| 基于标志的跨 NPU 同步。信号为 `int32_t` 标量或二维网格。 \|	70	\| 信号同步 \| `TNOTIFY`、`TWAIT`、`TTEST` \| 基于标志的跨 NPU 同步。信号为 `int32_t` 标量或二维网格。 \|
71	\| 集合通信 \| `TGATHER`、`TSCATTER`、`TBROADCAST`、`TREDUCE` \| 基于 `ParallelGroup` 的多 rank 操作。由 root 发起，支持 2D 分块滑动和 ping-pong 双缓冲。 \|	71	\| 集合通信 \| `TGATHER`、`TSCATTER`、`TBROADCAST`、`TREDUCE` \| 基于 `ParallelGroup` 的多 rank 操作。由 root 发起，支持 2D 分块滑动和 ping-pong 双缓冲。 \|
72		72