已合并
修改中英文一致性问题 #2608
zangyan创建于 15 天前
修改中英文一致性问题 #2608
已合并
zangyan创建于 15 天前
19 个文件变更+237-234
@@ -6,7 +6,7 @@
6<!-- If this PR does not involve an Issue, enter "NA". -->6<!-- If this PR does not involve an Issue, enter "NA". -->
7 7 
8## Testing8## Testing
9-<!-- Describe what tests you performed to verify your changes. This includes but is not limited to constructing corresponding test cases, secondary smoke tests, operator generalization, and so on. -->9+<!-- Describe what tests you performed to verify your changes. This includes but is not limited to constructing corresponding xx test cases, secondary smoke tests, operator generalization, and so on. -->
10 10 
11## Documentation Update11## Documentation Update
12<!-- If this PR includes documentation updates, indicate them here. For example: Updated the README.md file. -->12<!-- If this PR includes documentation updates, indicate them here. For example: Updated the README.md file. -->
@@ -30,7 +30,7 @@ Community contributions such as new operators, algorithms, or extended features
30- In principle, **avoid modifying** any files under `src/` as much as possible to prevent polluting stable code.30- In principle, **avoid modifying** any files under `src/` as much as possible to prevent polluting stable code.
31- If modifications to `src/` are necessary, explicitly state the reason and impact scope in the PR description and obtain Committer review.31- If modifications to `src/` are necessary, explicitly state the reason and impact scope in the PR description and obtain Committer review.
32- Code changes in a PR should focus on the `experimental/` directory.32- Code changes in a PR should focus on the `experimental/` directory.
33-- Provide a runtime switch for quick rollback. For details, see [experimental/README.md](./experimental/README_en.md).33+- Provide a runtime switch for quick rollback. For details, see [experimental/README_en.md](./experimental/README_en.md).
34 34 
35## Contribution Process35## Contribution Process
36 36 
@@ -24,7 +24,7 @@ HCCL is a core component of CANN. It supports multiple AI frameworks upward and
24HCCL consists of the HCCL collective communication library and the HCOMM (Huawei Communication) basic communication library:24HCCL consists of the HCCL collective communication library and the HCOMM (Huawei Communication) basic communication library:
25 25 
26- HCCL: includes built-in and extended communication operators, and provides external communication operator APIs.26- HCCL: includes built-in and extended communication operators, and provides external communication operator APIs.
27-- [HCOMM](https://gitcode.com/cann/hcomm): adopts a layered and decoupled design, dividing communication capabilities into a control plane and a data plane.27+- [HCOMM](https://gitcode.com/cann/hcomm): adopts a layered and decoupled design, dividing communication capabilities into a control plane and a data plane.
28 28 
29## Directory Structure29## Directory Structure
30 30 
@@ -24,8 +24,8 @@
24 24 
25| 类型 | 开源代码地址 | 文件名 | 公网 IP 地址/公网 URL 地址/域名/邮箱地址/压缩文件地址 | 用途说明 |25| 类型 | 开源代码地址 | 文件名 | 公网 IP 地址/公网 URL 地址/域名/邮箱地址/压缩文件地址 | 用途说明 |
26| :--: | :----------: | :------------------------------------- | :------------------------------------------------------------------------------------------------------------------------------ | :-------------------------------------------- |26| :--: | :----------: | :------------------------------------- | :------------------------------------------------------------------------------------------------------------------------------ | :-------------------------------------------- |
27-| 依赖 | 不涉及 | cmake/third_party/makeself-fetch.cmake | https://gitcode.com/cann-src-third-party/makeself/releases/download/release-2.5.0-patch1.0/makeself-release-2.5.0-patch1.tar.gz | 从 gitcode 下载 makeself 源码,作为编译依赖 |27+| 依赖 | 不涉及 | cmake/third_party/makeself-fetch.cmake | https://gitcode.com/cann-src-third-party/makeself/releases/download/release-2.5.0-patch1.0/makeself-release-2.5.0-patch1.tar.gz | 从 GitCode 下载 makeself 源码,作为编译依赖 |
28-| 依赖 | 不涉及 | cmake/third_party/gtest.cmake | https://gitcode.com/cann-src-third-party/googletest/releases/download/v1.14.0/googletest-1.14.0.tar.gz | 从 gitcode 下载 googletest 源码,作为编译依赖 |28+| 依赖 | 不涉及 | cmake/third_party/gtest.cmake | https://gitcode.com/cann-src-third-party/googletest/releases/download/v1.14.0/googletest-1.14.0.tar.gz | 从 GitCode 下载 Google Test 源码,作为编译依赖 |
29 29 
30---30---
31 31 
@@ -7,7 +7,7 @@ To ensure security and minimize permissions, you are not advised to use administ
7## File Permission Control7## File Permission Control
8 8 
9- You are advised to set the system `umask` value to `0027` or higher on hosts (including host machines) and containers. This ensures that new folders have a default maximum permission of `750` and new files have a default maximum permission of `640`.9- You are advised to set the system `umask` value to `0027` or higher on hosts (including host machines) and containers. This ensures that new folders have a default maximum permission of `750` and new files have a default maximum permission of `640`.
10-- You are advised to take security measures such as permission control on sensitive files, including personal privacy data, business assets, and source files. For example, permissions for the project installation directory and public input data files must follow the recommendations in [A–Recommended Maximum Permissions for Files and Folders in Different Scenarios](#a-recommended-maximum-permissions-for-files-and-folders-in-different-scenarios).10+- You are advised to take security measures such as permission control on sensitive files, including personal privacy data, commercial assets, and source files. For example, permissions for the project installation directory and public input data files must follow the recommendations in [A–Recommended Maximum Permissions for Files and Folders in Different Scenarios](#a-recommended-maximum-permissions-for-files-and-folders-in-different-scenarios).
11- During installation and usage, you must enforce proper permission control, referring to the same [A–Recommended Maximum Permissions for Files and Folders in Different Scenarios](#a-recommended-maximum-permissions-for-files-and-folders-in-different-scenarios).11- During installation and usage, you must enforce proper permission control, referring to the same [A–Recommended Maximum Permissions for Files and Folders in Different Scenarios](#a-recommended-maximum-permissions-for-files-and-folders-in-different-scenarios).
12 12 
13## Build Security Statement13## Build Security Statement
@@ -1,7 +1,7 @@
1# HCCL Document Overview1# HCCL Document Overview
2 2 
3- [Collective Communication Library User Guide](https://www.hiascend.com/document/detail/en/canncommercial/850/commlib/hcclug/hcclug_000001.html): Introduces basic HCCL concepts, how to use HCCL APIs for collective communication development, how to develop communication operators, and fault handling with common configurations.3- [Collective Communication Library User Guide](https://www.hiascend.com/document/detail/en/canncommercial/850/commlib/hcclug/hcclug_000001.html): Introduces basic HCCL concepts, how to use HCCL APIs for collective communication development, how to develop communication operators, and fault handling with common configurations.
4-- [Communication Operator Development Guide (Chinese)](https://gitcode.com/cann/hcomm/blob/master/docs/zh/comm_op_dev_guide/README.md)Introduces how to develop communication operators based on the APIs provided by the HCOMM communication foundation library.4+- [Communication Operator Development Guide (Chinese)](https://gitcode.com/cann/hcomm/blob/master/docs/zh/comm_op_dev_guide/README.md): Introduces how to develop communication operators based on the APIs provided by the HCOMM communication foundation library.
5 5 
6## Technical Articles6## Technical Articles
7 7 
@@ -14,7 +14,7 @@ Introduce BIRS (Batchsize Invariant ReduceScatter) — a novel batch-invariant R
14 14 
15### Industry Demand for Deterministic Collective Communication15### Industry Demand for Deterministic Collective Communication
16 16 
17-In distributed training and inference, **deterministic collective communication** requires that reduction operations (AllReduce, ReduceScatter, etc.) produce **bit-identical** results for the same input, regardless of batch size, process count, or memory sharding strategy. This requirement has become a hard constraint in multiple industry scenarios:17+In distributed training and inference, **deterministic collective communication** requires that reduction operations (AllReduce, ReduceScatter, etc.) produce **bit-identical** results for the same input, regardless of batch size, process count, or memory sharding strategy. This requirement has become a hard constraint in multiple industry scenarios.
18 18 
19#### 1. Training Reproducibility and CI/CD19#### 1. Training Reproducibility and CI/CD
20 20 
@@ -62,9 +62,9 @@ On the A3 server topology (SIO + HCCS hybrid interconnect), RHD cannot simultane
62 62 
63The BIRS algorithm is designed for the 2D topology characteristics of A3 servers, maintaining batch invariance while:63The BIRS algorithm is designed for the 2D topology characteristics of A3 servers, maintaining batch invariance while:
64 64 
65-- **First round**: Performing SendReduce over SIO links (cross X-axis reduction)65+- **First round**: Performing SendReduce over SIO links (cross X-axis reduction).
66-- **Subsequent rounds**: Simultaneously utilizing SIO (reduction) and HCCS (intermediate result transfer) links66+- **Subsequent rounds**: Simultaneously utilizing SIO (reduction) and HCCS (intermediate result transfer) links.
67-- Achieving near-optimal bandwidth utilization, with only the first round not fully utilizing bandwidth67+- Achieving near-optimal bandwidth utilization, with only the first round not fully utilizing bandwidth.
68 68 
69## Detailed Design69## Detailed Design
70 70 
@@ -163,7 +163,7 @@ HcclReduceScatter(sendBuf, recvBuf, recvCount, dataType, op, comm, stream);
163 163 
164BIRS constructs a logical 2D layout over the A3/16P topology:164BIRS constructs a logical 2D layout over the A3/16P topology:
165 165 
166-```166+```text
167rankSizeX = 2 // X-axis direction (SIO links)167rankSizeX = 2 // X-axis direction (SIO links)
168rankSizeY = rankSize / rankSizeX // Y-axis direction (HCCS links)168rankSizeY = rankSize / rankSizeX // Y-axis direction (HCCS links)
169```169```
@@ -183,7 +183,7 @@ Each rank maintains the following topology information:
183 183 
184BIRS uses scratch memory to store intermediate reduction results (IM), with a strided layout to satisfy the 910B minimum slice alignment requirement:184BIRS uses scratch memory to store intermediate reduction results (IM), with a strided layout to satisfy the 910B minimum slice alignment requirement:
185 185 
186-```186+```text
187localStrideSize = RoundUp(sliceSize, HCCL_MIN_SLICE_ALIGN_910B)187localStrideSize = RoundUp(sliceSize, HCCL_MIN_SLICE_ALIGN_910B)
188 188 
189Scratch buffer is divided into 2 regions with N slots each: Region A is used to accumulate intermediate results for HCCS,189Scratch buffer is divided into 2 regions with N slots each: Region A is used to accumulate intermediate results for HCCS,
@@ -228,14 +228,15 @@ Inter-thread synchronization is performed via `PreSyncInterThreads` / `PostSyncI
228The core property of the BIRS algorithm is **batch invariance**: the order of reduction additions on each rank is strictly identical, regardless of batch size or memory slicing.228The core property of the BIRS algorithm is **batch invariance**: the order of reduction additions on each rank is strictly identical, regardless of batch size or memory slicing.
229 229 
230**Notation**:230**Notation**:
231-- `S(d, i)`: The i-th slice of the input message on device d231+ 
232-- `rankSizeX = 2`, `rankSizeY = rankSize / 2`232+- `S(d, i)`: The i-th slice of the input message on device d.
233-- `sio_rank = rank XOR 1` (SIO peer)233+- `rankSizeX = 2`, `rankSizeY = rankSize / 2`.
234-- `hccs_ranks[i] = (rank + rankSizeX * i) % rankSize` (HCCS peer sequence)234+- `sio_rank = rank XOR 1` (SIO peer).
235+- `hccs_ranks[i] = (rank + rankSizeX * i) % rankSize` (HCCS peer sequence).
235 236 
236#### 4.2 Main Communication Loop237#### 4.2 Main Communication Loop
237 238 
238-```239+```c
239// Initial: copy the input slice corresponding to the first HCCS peer into scratch memory240// Initial: copy the input slice corresponding to the first HCCS peer into scratch memory
240LocalCopy(input[S(hccs_ranks[0])], scratch[IM_0])241LocalCopy(input[S(hccs_ranks[0])], scratch[IM_0])
241 242 
@@ -268,7 +269,7 @@ for round in 0 ... hccs_ranks.size():
268 269 
269After all rounds complete, each rank holds `rankSizeY` intermediate results in scratch memory. These are merged via a **tree-based local reduction**:270After all rounds complete, each rank holds `rankSizeY` intermediate results in scratch memory. These are merged via a **tree-based local reduction**:
270 271 
271-```272+```text
272// Collect all intermediate result offsets273// Collect all intermediate result offsets
273vec = [IM_0, IM_1, ..., IM_{rankSizeY-1}] // this rank's result is at the correct position274vec = [IM_0, IM_1, ..., IM_{rankSizeY-1}] // this rank's result is at the correct position
274 275 
@@ -297,14 +298,14 @@ The BIRS algorithm currently has the following constraints:
297 298 
298| Constraint | Description |299| Constraint | Description |
299|------------|-------------|300|------------|-------------|
300-| Platform | A3 servers only (SIO + HCCS hybrid topology) |301+| Platform | A3 servers only (SIO + HCCS hybrid topology). |
301-| rankSize | Must be even (`rankSize % 2 == 0`), typical values: 4, 8, 16 |302+| rankSize | Must be even (`rankSize % 2 == 0`), typical values: 4, 8, 16. |
302-| Communication domain | Both Intra-server and Inter-server are supported |303+| Communication domain | Both Intra-server and Inter-server are supported. |
303-| Data alignment | Slice sizes must satisfy `HCCL_MIN_SLICE_ALIGN_910B` alignment requirements |304+| Data alignment | Slice sizes must satisfy `HCCL_MIN_SLICE_ALIGN_910B` alignment requirements. |
304 305 
305-ReduceScatterBIRS() is the recommended choice for single-server A3 scenario (rankSize <= 16), ReduceScatterBIRSInter() is chosen automatically for multi-server A3 scenario.306+ReduceScatterBIRS() is the recommended choice for the single-server A3 scenario (rankSize <= 16), ReduceScatterBIRSInter() is chosen automatically for multi-server A3 scenario.
306 307 
307-When conditions are not met, the workflow exit and log error messages using hccl. The user must follow the recommendations in the error logs or manually adjust the parameters to comply with the restrictions.308+When conditions are not met, the workflow exits and logs error messages via HCCL. The user must follow the recommendations in the error logs or manually adjust the parameters to comply with the restrictions.
308 309 
309#### 5.3 Rollout Strategy310#### 5.3 Rollout Strategy
310 311 
@@ -325,12 +326,12 @@ When conditions are not met, the workflow exit and log error messages using hccl
325 326 
326#### 6.2 Batch Invariance Verification327#### 6.2 Batch Invariance Verification
327 328 
328-- Execute ReduceScatter with the same input data but different batch sizes329+- Execute ReduceScatter with the same input data but different batch sizes.
329-- Verify that output results are bit-identical330+- Verify that output results are bit-identical.
330 331 
331#### 6.3 Performance Testing332#### 6.3 Performance Testing
332 333 
333-- Compare against the RHD algorithm, measuring Task Duration across different message sizes334+- Compare against the RHD algorithm, measuring Task Duration across different message sizes.
334- Expected: For message sizes >= 16MB, BIRS achieves up to 25% improvement over RHD.335- Expected: For message sizes >= 16MB, BIRS achieves up to 25% improvement over RHD.
335- Note: At the moment of this RFC creation kernel submission mechanism in HCCL is slower than the one of HCOMM, so 25% performance improvement applies only to operator execution time (without submission overhead).336- Note: At the moment of this RFC creation kernel submission mechanism in HCCL is slower than the one of HCOMM, so 25% performance improvement applies only to operator execution time (without submission overhead).
336 337 
@@ -342,10 +343,10 @@ When conditions are not met, the workflow exit and log error messages using hccl
342 343 
343| Risk | Impact | Mitigation |344| Risk | Impact | Mitigation |
344|------|--------|------------|345|------|--------|------------|
345-| BIRS only available for specific rankSize (even numbers) | Odd rank scenarios cannot use BIRS | `MatchBIRS()` check auto-falls back to existing algorithms; document constraints clearly |346+| BIRS only available for specific rankSize (even numbers) | Odd rank scenarios cannot use BIRS | `MatchBIRS()` check auto-falls back to existing algorithms; document constraints clearly. |
346-| Experimental code may introduce stability issues | Affects overall HCCL reliability | Dual gating (compile + runtime) isolation; independent `experimental/` directory; disabled by default |347+| Experimental code may introduce stability issues | Affects overall HCCL reliability | Dual gating (compile + runtime) isolation; independent `experimental/` directory; disabled by default. |
347-| Additional scratch memory overhead | Increased memory usage for large messages | Requires `2 * rankSizeY × localStrideSize` scratch space; pre-allocated via `CalcResRequest` |348+| Additional scratch memory overhead | Increased memory usage for large messages | Requires `2 * rankSizeY × localStrideSize` scratch space; pre-allocated via `CalcResRequest`. |
348-| A3 topology assumption (SIO + HCCS) may not apply to other platforms | Cross-platform compatibility | Algorithm explicitly bound to A3 topology characteristics; other platforms require independent adaptation |349+| A3 topology assumption (SIO + HCCS) may not apply to other platforms | Cross-platform compatibility | Algorithm explicitly bound to A3 topology characteristics; other platforms require independent adaptation. |
349 350 
350## Alternative Approaches351## Alternative Approaches
351 352 
@@ -353,7 +354,7 @@ N/A
353 354 
354## Open Questions355## Open Questions
355 356 
356-1. **AllReduce extension**: Batch-invariant AllReduce which follows the same ideas will be submitted in separate PR357+1. **AllReduce extension**: Batch-invariant AllReduce which follows the same ideas will be submitted in separate PR.
3572. **Efficient support for arbitrary rank enumeration**: Current solution assumes default rank enumeration where rankID of SIO neighbour of RankX can be calculated as (RankX XOR 1). In case of other rank enumerations BIRS is functional but doesn't deliver performance advantage over RHD. Efficient support for custom enumerations have already been implemented and will be submitted in the next PR.3582. **Efficient support for arbitrary rank enumeration**: Current solution assumes default rank enumeration where rankID of SIO neighbour of RankX can be calculated as (RankX XOR 1). In case of other rank enumerations BIRS is functional but doesn't deliver performance advantage over RHD. Efficient support for custom enumerations have already been implemented and will be submitted in the next PR.
358 359 
359---360---
@@ -1,114 +1,115 @@
1-# RFC: Bandwidth-efficient Invariant ReduceScatter (BIRS) Algorithm1+# RFCBandwidth-efficient Invariant ReduceScatter (BIRS)算法
2 2 
3-- Start Date: 2026-04-243+- 起始日期:2026-04-24
4-- RFC PR: cann/hccl#6574+- RFC PR编号:cann/hccl#657
5-- Related Issues: cann/hcomm#139, cann/hccl#965+- 相关Issue:cann/hcomm#139, cann/hccl#96
6 6 
7---7---
8 8 
9-## Summary9+## 概要
10 10 
11-Introduce BIRS (Batchsize Invariant ReduceScatter) — a novel batch-invariant ReduceScatter algorithm for the Ascend A3 server topology in HCCL. While guaranteeing deterministic reduction ordering (bit-level reproducibility), the algorithm achieves up to 25% performance improvement (operator execution time without submission overhead) over the existing RHD (Recursive Halving-Doubling) algorithm for large message sizes by more fully utilizing the SIO + HCCS hybrid interconnect bandwidth.11+BIRSBatchsize Invariant ReduceScatter,批大小不变的 ReduceScatter 算法)是HCCL面向Ascend A3 服务器拓扑提出的一种新型batch大小不变的ReduceScatter算法。该算法在保证确定性归约顺序(即比特级可重现性)的前提下,通过更充分地利用SIO + HCCS混合互连带宽,在大消息场景下相比现有的RHDRecursive Halving-Doubling)算法,可获得最高25%的性能提升(算子执行时间,不含下发开销)。
12 12 
13-## Background and Motivation13+## 背景与动机
14 14 
15-### Industry Demand for Deterministic Collective Communication15+### 业界对确定性集合通信的需求
16 16 
17-In distributed training and inference, **deterministic collective communication** requires that reduction operations (AllReduce, ReduceScatter, etc.) produce **bit-identical** results for the same input, regardless of batch size, process count, or memory sharding strategy. This requirement has become a hard constraint in multiple industry scenarios:17+在分布式训练和推理中,**确定性集合通信**要求归约操作(AllReduceReduceScatter等)在输入相同的情况下,无论批大小、进程数或内存分片策略如何变化,都能产生**比特级**完全相同的结果。这一需求已在多个行业场景中成为硬性约束。
18 18 
19-#### 1. Training Reproducibility & CI/CD19+#### 1. 训练可复现性与CI/CD
20 20 
21-Reproducible training is essential for trustworthy research and production pipelines. Non‑deterministic reductions introduce floating‑point noise that masks bugs and makes results impossible to compare across runs.21+可复现的训练是可信研究和生产流水线的基础。非确定性归约会引入浮点噪声,掩盖缺陷并使不同运行之间的结果无法比较。
22 22 
23-- **Picard (2021)** (“Torch.manual_seed(3407) is all you need”) demonstrates that random seed variations alone can produce statistically significant outliers in final model performance – when reduction ordering is also non‑deterministic, the variance grows even larger. ([arXiv:2109.08203](https://arxiv.org/abs/2109.08203))23+- **Picard (2021)**("Torch.manual_seed(3407) is all you need")证明仅随机种子变化就能在最终模型性能上产生统计显著的离群值 —— 当归约顺序也是非确定性时,方差会进一步增大。([arXiv:2109.08203](https://arxiv.org/abs/2109.08203))
24-- **CI/CD & Debugging**: In continuous integration testing and distributed debugging, any non‑determinism turns a reproducible bug into a ghost. Deterministic collectives guarantee that a failing test will fail identically on every rerun, drastically reducing root‑cause analysis time.24+- **CI/CD与调试**:在持续集成测试和分布式调试中,任何非确定性都会将可复现的缺陷变为“幽灵”问题。确定性集合通信保证失败的测试在每次重运行时以完全相同的方式失败,从而大幅缩短根因分析时间。
25 25 
26-#### 2. Reinforcement Learning (RL / RLHF / PPO)26+#### 2. 强化学习(RLRLHFPPO
27 27 
28-Reinforcement learning training is highly sensitive to consistency in policy evaluation. In PPO and RLHF pipelines, when the same policy is evaluated with different batch sizes, a change in ReduceScatter reduction ordering due to sharding can inject floating‑point noise into gradient/reward signals, destabilizing policy updates.28+强化学习训练对策略评估的一致性高度敏感。在PPORLHF流水线中,当同一策略以不同batch size进行评估时,由于分片导致的ReduceScatter归约顺序变化会向梯度或奖励信号中注入浮点噪声,使策略更新不稳定。
29 29 
30-- **verl** ([github.com/verl-project/verl](https://github.com/verl-project/verl)): A mainstream open‑source RLHF/PPO framework that provides a `full_determinism` configuration option and explicitly sets `HCCL_DETERMINISTIC=1` to guarantee reproducible collective operations.30+- **verl**[github.com/verl-project/verl](https://github.com/verl-project/verl)):主流开源RLHFPPO框架,提供了 `full_determinism` 配置选项,并显式设置`HCCL_DETERMINISTIC=1`以保证可复现的集合操作。
31-- **DeepSpeedChat** and derivative frameworks: Require deterministic reductions in RLHF training to keep reward model training consistent across identical inputs.31+- **DeepSpeed-Chat** 及衍生框架:在RLHF训练中要求确定性归约,以保持奖励模型训练在相同输入上的一致性。
32 32 
33-#### 3. Inference Consistency & Batch Invariance33+#### 3. 推理一致性与 Batch 不变性
34 34 
35-In large‑model serving, users expect the same prompt to always return the same output. However, dynamic batching means a prompt can be grouped with different neighbours on each request. Without deterministic collective communication, floating‑point reduction order can vary with batch composition, breaking this invariance.35+在大模型服务中,用户期望同一prompt始终返回相同输出。然而,动态batching意味着一个prompt在每次请求时可能与不同的邻居组合。如果没有确定性集合通信,浮点归约顺序会随batch组成而变化,破坏这一不变性。
36 36 
37-- **vLLM Batch Invariance**: The vLLM project explicitly calls out that non‑deterministic allreduce backends (e.g., NCCL) can cause different logits for the same prompt depending on batch mates. Their batch invariance guarantee relies on deterministic communication to ensure “the output for a given prompt is the same regardless of what other prompts are in the batch.” ([Motivation](https://docs.vllm.ai/en/latest/features/batch_invariance/#motivation), [Ascend Guide](https://docs.vllm.ai/projects/ascend/en/latest/user_guide/feature_guide/batch_invariance.html))37+- **vLLM Batch不变性**vLLM项目明确指出非确定性all-reduce后端(如 NCCL)会导致同一promptbatch中的其他prompt不同而产生不同的logits。其batch不变性保证依赖于确定性通信,以确保"给定prompt的输出不受batch中其他prompt的影响"。([Motivation](https://docs.vllm.ai/en/latest/features/batch_invariance/#motivation), [Ascend Guide](https://docs.vllm.ai/projects/ascend/en/latest/user_guide/feature_guide/batch_invariance.html))
38-- **SGLang**: Provides an `--enable-deterministic-inference` flag that forces deterministic computation and communication ordering, making inference outputs fully reproducible across different batch sizes and request arrival patterns. ([SGLang deterministic inference](https://sgl-project.github.io/advanced_features/deterministic_inference.html))38+- **SGLang**:提供`--enable-deterministic-inference`标志,强制确定性计算和通信顺序,使推理输出在不同batch size和请求到达模式下完全可复现。([SGLang deterministic inference](https://sgl-project.github.io/advanced_features/deterministic_inference.html))
39-- **OpenAI Community**: Practitioners have long struggled with non‑deterministic GPU operations in production LLM inference, where bit‑for‑bit reproducibility is expected by end‑users and essential for debugging. ([Defeating Nondeterminism in LLM Inference](https://community.openai.com/t/defeating-nondeterminism-in-llm-inference/1358623))39+- **OpenAI 社区**:在生产环境的LLM推理中,从业者长期以来一直受困于非确定性的GPU操作——而终端用户期望的是比特级完全可复现的结果,且这对于调试也至关重要。([Defeating Nondeterminism in LLM Inference](https://community.openai.com/t/defeating-nondeterminism-in-llm-inference/1358623))
40 40 
41-#### 4. Ecosystem API & Framework Support41+#### 4. 生态API与框架支持
42 42 
43-The demand for determinism is reflected in the official APIs and configuration flags of major ML frameworks:43+对确定性的需求已体现在主流 ML 框架的官方 API 和配置标志中:
44 44 
45-- **PyTorch**: `torch.use_deterministic_algorithms(True)` requires all operations – including collectives – to produce the same output given the same input on the same hardware/software. ([PyTorch docs](https://docs.pytorch.org/docs/stable/generated/torch.use_deterministic_algorithms.html))45+- **PyTorch**`torch.use_deterministic_algorithms(True)` 要求所有操作——包括集合通信——在相同软硬件环境下对相同输入产生相同输出。([PyTorch docs](https://docs.pytorch.org/docs/stable/generated/torch.use_deterministic_algorithms.html))
46-- **HuggingFace Transformers / Diffusers**: Provide a standardised `enable_full_determinism()` function that sets `NCCL_DETERMINISTIC=1`, `CUBLAS_WORKSPACE_CONFIG`, and other variables.46+- **HuggingFace TransformersDiffusers**:提供标准化的`enable_full_determinism()`函数,用于设置`NCCL_DETERMINISTIC=1``CUBLAS_WORKSPACE_CONFIG`等变量。
47-- **LlamaFactory**: Large‑model fine‑tuning framework offering an `enable_full_determinism(seed)` interface for reproducible distributed training.47+- **LlamaFactory**:大模型微调框架,提供`enable_full_determinism(seed)`接口用于可复现的分布式训练。
48-- **ByteDance VeOmni**: Enforces `--train.enable_full_determinism true` in CI tests, making deterministic collectives a gate for code acceptance.48+- **ByteDance VeOmni**:在CI测试中强制`--train.enable_full_determinism true`,使确定性集合通信成为代码合入的门槛。
49 49 
50-### Limitations of Existing Batch-Invariant Algorithms in HCCL50+### HCCL现有Batch不变算法的局限性
51 51 
52-HCCL currently provides two batch-invariant algorithms:52+HCCL目前提供两种batch不变算法:
53 53 
54-| Algorithm | Use Case | Limitation |54+| 算法 | 适用场景 | 局限性 |
55-|-----------|----------|------------|55+|------|----------|--------|
56-| **Mesh + Local Reduce** | Small messages (< several MB) | Low bandwidth utilization for large messages |56+| **Mesh + Local Reduce** | 小消息(< MB | 大消息场景带宽利用率低 |
57-| **RHD (Recursive Halving-Doubling)** | Large messages | Utilizes only ~50% of available bandwidth (only half the nodes communicate per round) |57+| **RHD(递归半倍-倍增)** | 大消息 | 仅利用约50%的可用带宽(每轮仅一半节点参与通信) |
58 58 
59-On the A3 server topology (SIO + HCCS hybrid interconnect), RHD cannot simultaneously utilize SIO and HCCS links, resulting in insufficient bandwidth utilization for large message scenarios.59+A3 服务器拓扑(SIO + HCCS 混合互连)上,RHD无法同时利用SIOHCCS链路,导致大消息场景下带宽利用不足。
60 60 
61-### Value of BIRS61+### BIRS的价值
62 62 
63-The BIRS algorithm is designed for the 2D topology characteristics of A3 servers, maintaining batch invariance while:63+BIRS算法针对A3服务器的2D拓扑特性设计,在保持batch不变性的同时:
64-- **First round**: Performing SendReduce over SIO links (cross X-axis reduction)
65-- **Subsequent rounds**: Simultaneously utilizing SIO (reduction) and HCCS (intermediate result transfer) links
66-- Achieving near-optimal bandwidth utilization, with only the first round not fully utilizing bandwidth
67 64 
68-## Detailed Design65+- **第一轮**:通过SIO链路执行SendReduce(跨X轴归约)。
66+- **后续轮次**:同时利用SIO(归约)和HCCS(中间结果传输)链路。
67+- 实现近最优带宽利用,仅第一轮未充分利用带宽。
69 68 
70-### 1. Overall Architecture69+## 详细设计
71 70 
72-The BIRS algorithm is integrated into HCCL as an experimental feature, isolated from existing algorithms through an independent code path and build option.71+### 1. 总体架构
72+ 
73+BIRS算法作为实验特性集成到HCCL中,通过独立代码路径和编译选项与现有算法隔离。
73 74 
74```text75```text
75HCCL76HCCL
76-├── src/ops/reduce_scatter/ # Existing ReduceScatter implementation77+├── src/ops/reduce_scatter/ # 现有 ReduceScatter 实现
77-│ └── reduce_scatter_op.cc/.h # Entry function (with BIRS dispatch logic added)78+│ └── reduce_scatter_op.cc/.h # 入口函数(新增BIRS分发逻辑)
7879
79-├── experimental/ops/ # Experimental features directory (new)80+├── experimental/ops/ # 实验特性目录(新增)
80-│ ├── op_common/ # Common infrastructure81+│ ├── op_common/ # 公共基础设施
81-│ │ ├── op_common_experimental.cc/.h # Experimental op common logic (ProcessA3, etc.)82+│ │ ├── op_common_experimental.cc/.h # 实验op公共逻辑(ProcessA3等)
82-│ │ ├── template/ # Experimental algorithm template base class83+│ │ ├── template/ # 实验算法模板基类
83│ │ │ └── alg_template_base_experimental.cc/.h84│ │ │ └── alg_template_base_experimental.cc/.h
84-│ │ └── topo/ # Experimental topology utilities85+│ │ └── topo/ # 实验拓扑工具
85│ │ └── topo_experimental.cc/.h86│ │ └── topo_experimental.cc/.h
86│ │87│ │
87-│ └── reduce_scatter/ # ReduceScatter experimental algorithms88+│ └── reduce_scatter/ # ReduceScatter实验算法
88-│ ├── reduce_scatter_op_experimental.cc/.h # Experimental entry (MatchBIRS dispatch)89+│ ├── reduce_scatter_op_experimental.cc/.h # 实验入口(MatchBIRS分发)
89-│ └── birs/ # BIRS algorithm implementation90+│ └── birs/ # BIRS算法实现
90-│ ├── reduce_scatter_birs_executor.cc/.h # Executor layer (resource calc, scheduling)91+│ ├── reduce_scatter_birs_executor.cc/.h # Executor层(资源计算、调度)
91-│ ├── reduce_scatter_executor_base.cc/.h # Executor base class92+│ ├── reduce_scatter_executor_base.cc/.h # Executor基类
92│ └── template/93│ └── template/
93-│ ├── reduce_scatter_birs.cc/.h # Core algorithm template (communication loop)94+│ ├── reduce_scatter_birs.cc/.h # 核心算法模板(通信循环)
94-│ └── reduce_scatter_birs_inter.cc/.h # Intermediate result handling95+│ └── reduce_scatter_birs_inter.cc/.h # 中间结果处理
9596
96└── test/st/algorithm/testcase/97└── test/st/algorithm/testcase/
97- └── reduce_scatter_testcase_a3.cc # A3 platform test cases (new)98+ └── reduce_scatter_testcase_a3.cc # A3平台测试用例(新增)
98```99```
99 100 
100-**Data Flow**:101+**数据流**
101 102 
102```text103```text
103-User calls HcclReduceScatter()104+用户调用HcclReduceScatter()
104105
105- ├── HCCL_BIRS_ENABLE != TRUE → Take existing HcclReduceScatterInner() path106+ ├── HCCL_BIRS_ENABLE != TRUE → 走现有 HcclReduceScatterInner() 路径
106107
107 └── HCCL_BIRS_ENABLE == TRUE108 └── HCCL_BIRS_ENABLE == TRUE
108109
109 └── ReduceScatterExperimental()110 └── ReduceScatterExperimental()
110111
111- ├── Parameter validation (reuse existing CheckReduceScatterInputPara, etc.)112+ ├── 参数校验(复用现有 CheckReduceScatterInputPara 等)
112113
113 └── ReduceScatterOutPlaceCustom()114 └── ReduceScatterOutPlaceCustom()
114115
@@ -118,24 +119,24 @@ User calls HcclReduceScatter()
118119
119 └── ReduceScatterBIRS::RunAsync()120 └── ReduceScatterBIRS::RunAsync()
120121
121- ├── Preprocess() — Preprocessing (slice calc, channel validation)122+ ├── Preprocess() — 预处理(切片计算、通道校验)
122- ├── Main comm loop — SIO SendReduce + HCCS transfers123+ ├── Main comm loop — SIO SendReduce + HCCS 传输
123- └── FinalStep() — Local tree reduction + output copy124+ └── FinalStep() — 本地树形归约 + 输出拷贝
124```125```
125 126 
126-### 2. Interface Design127+### 2. 接口设计
127 128 
128-#### 2.1 Environment Variables129+#### 2.1 环境变量
129 130 
130-| Environment Variable | Values | Description |131+| 环境变量 | 取值 | 说明 |
131-|---------------------|--------|-------------|132+|---------|------|------|
132-| `HCCL_BIRS_ENABLE` | `TRUE` / `FALSE` (default) | Enable the BIRS algorithm. When set to `TRUE`, ReduceScatter calls are routed to the BIRS implementation |133+| `HCCL_BIRS_ENABLE` | `TRUE`/`FALSE`(默认) | 启用BIRS算法。设为`TRUE`时,ReduceScatter调用将路由到BIRS实现。|
133 134 
134-#### 2.2 Build Options135+#### 2.2 编译选项
135 136 
136-Run the following command from the root directory of the hccl repository:137+hccl仓库根目录下执行以下命令:
137 138 
138-```139+```bash
139# host + device + experimental140# host + device + experimental
140bash build.sh --pkg --full --experimental141bash build.sh --pkg --full --experimental
141```142```
@@ -144,103 +145,104 @@ bash build.sh --pkg --full --experimental
144option(ENABLE_EXPERIMENTAL "Enable experimental features" OFF)145option(ENABLE_EXPERIMENTAL "Enable experimental features" OFF)
145```146```
146 147 
147-To enable experimental functions, use the `--experimental option`. This sets the compilation flag `-DENABLE_EXPERIMENTAL=ON`, which in turn causes the `experimental/ops/` subdirectory to be compiled. Disabled by default, with no impact on existing builds.148+使用`--experimental option`选项启用实验功能。为该选项设置编译标志`-DENABLE_EXPERIMENTAL=ON`,从而编译`experimental/ops/`子目录中的代码。该功能默认关闭,不影响现有构建。
148 149 
149-#### 2.3 API Compatibility150+#### 2.3 API 兼容性
150 151 
151-BIRS introduces no new user-facing APIs. Users call the standard `HcclReduceScatter()` interface, with algorithm selection entirely controlled by environment variables:152+BIRS未引入新的用户态API。用户调用标准 `HcclReduceScatter()`接口即可,算法选择完全由环境变量控制:
152 153 
153```c154```c
154-// No user code changes required just set the environment variable to enable155+// 无需修改用户代码 只需设置环境变量即可启用
155// export HCCL_BIRS_ENABLE=TRUE156// export HCCL_BIRS_ENABLE=TRUE
156HcclReduceScatter(sendBuf, recvBuf, recvCount, dataType, op, comm, stream);157HcclReduceScatter(sendBuf, recvBuf, recvCount, dataType, op, comm, stream);
157```158```
158 159 
159-### 3. Data Structures160+### 3. 数据结构
160 161 
161-#### 3.1 Logical 2D Topology Layout162+#### 3.1 逻辑2D拓扑布局
162 163 
163-BIRS constructs a logical 2D layout over the A3/16P topology:164+BIRSA3/16P拓扑上构建逻辑2D布局:
164 165 
165-```166+```text
166-rankSizeX = 2 // X-axis direction (SIO links)167+rankSizeX = 2 // X 轴方向(SIO 链路)
167-rankSizeY = rankSize / rankSizeX // Y-axis direction (HCCS links)168+rankSizeY = rankSize / rankSizeX // Y 轴方向(HCCS 链路)
168```169```
169 170 
170-Each rank maintains the following topology information:171+每个rank维护以下拓扑信息:
171 172 
172-| Member | Type | Description |173+| 成员 | 类型 | 说明 |
173-|--------|------|-------------|174+|------|------|------|
174-| `sio_rank` | `u32` | SIO peer rank (`rank XOR 1`) |175+| `sio_rank` | `u32` | SIO对端rank`rank XOR 1` |
175-| `hccs_ranks` | `vector<u32>` | Peer rank list along HCCS direction |176+| `hccs_ranks` | `vector<u32>` | HCCS方向的对端rank列表 |
176-| `hccs_neighbour_rank` | `vector<u32>` | SIO neighbor ranks of HCCS peers |177+| `hccs_neighbour_rank` | `vector<u32>` | HCCS对端的SIO邻居rank |
177-| `sio_link` | `ChannelInfo` | SIO communication channel |178+| `sio_link` | `ChannelInfo` | SIO通信通道 |
178-| `hccs_links` | `vector<ChannelInfo>` | HCCS communication channel list |179+| `hccs_links` | `vector<ChannelInfo>` | HCCS通信通道列表 |
179-| `hccs_links_reversed` | `vector<ChannelInfo>` | Reversed HCCS channels (for receiving) |180+| `hccs_links_reversed` | `vector<ChannelInfo>` | 反向HCCS通道(用于接收) |
180 181 
181-#### 3.2 Scratch Memory Layout182+#### 3.2 Scratch 内存布局
182 183 
183-BIRS uses scratch memory to store intermediate reduction results (IM), with a strided layout to satisfy the 910B minimum slice alignment requirement:184+BIRS使用scratch内存存储中间归约结果(IM),采用步长布局以满足910B最小切片对齐要求:
184 185 
185-```186+```text
186localStrideSize = RoundUp(sliceSize, HCCL_MIN_SLICE_ALIGN_910B)187localStrideSize = RoundUp(sliceSize, HCCL_MIN_SLICE_ALIGN_910B)
187 188 
188-Scratch buffer is divided into 2 regions with N slots each: Region A is used to accumulate intermediate results for HCCS,189+Scratch 缓冲区分为 2 个区域,各含 N 个槽位:区域 A 用于累积 HCCS 中间结果,
189-Region B is used for sendReduce over SIO.190+区域 B 用于 SIO 上的 sendReduce
190 191 
191Scratch Memory:192Scratch Memory:
192┌─────────────────────────────────────────────┐193┌─────────────────────────────────────────────┐
193-│ IM[0]: offset = 0 * localStrideSize │ ← Region A intermediate result194+│ IM[0]: offset = 0 * localStrideSize │ ← 区域 A 中间结果
194├─────────────────────────────────────────────┤195├─────────────────────────────────────────────┤
195-│ IM[1]: offset = 1 * localStrideSize │ ← Region A intermediate result196+│ IM[1]: offset = 1 * localStrideSize │ ← 区域 A 中间结果
196├─────────────────────────────────────────────┤197├─────────────────────────────────────────────┤
197│ ... │198│ ... │
198├─────────────────────────────────────────────┤199├─────────────────────────────────────────────┤
199-│ IM[N]: offset = N * localStrideSize │ ← Region A intermediate result200+│ IM[N]: offset = N * localStrideSize │ ← 区域 A 中间结果
200├─────────────────────────────────────────────┤201├─────────────────────────────────────────────┤
201-│ SIO[0]: offset = (N+1) * localStrideSize │ ← Region B SIO-intermediate result202+│ SIO[0]: offset = (N+1) * localStrideSize │ ← 区域 B SIO 中间结果
202├─────────────────────────────────────────────┤203├─────────────────────────────────────────────┤
203-│ SIO[1]: offset = (N+2) * localStrideSize │ ← Region B SIO-intermediate result204+│ SIO[1]: offset = (N+2) * localStrideSize │ ← 区域 B SIO 中间结果
204├─────────────────────────────────────────────┤205├─────────────────────────────────────────────┤
205│ ... │206│ ... │
206├─────────────────────────────────────────────┤207├─────────────────────────────────────────────┤
207-│ SIO[N]: offset = 2 * N * localStrideSize │ ← Region B SIO-intermediate result208+│ SIO[N]: offset = 2 * N * localStrideSize │ ← 区域 B SIO 中间结果
208└─────────────────────────────────────────────┘209└─────────────────────────────────────────────┘
209```210```
210 211 
211-#### 3.3 Thread Model212+#### 3.3 线程模型
212 213 
213-BIRS uses a 3-thread parallel model:214+BIRS使用三线程并行模型:
214 215 
215-| Thread | Role | Responsibility |216+| 线程 | 角色 | 职责 |
216-|--------|------|----------------|217+|------|------|------|
217-| `mainThread` | Main thread | SIO SendReduce, final local reduction |218+| `mainThread` | 主线程 | SIO SendReduce、最终本地归约 |
218-| `subThreads[0]` | HCCS sub-thread | HCCS link Send/Notify operations |219+| `subThreads[0]` | HCCS子线程 | HCCS链路SendNotify操作 |
219-| `subThreads[1]` | Copy sub-thread | Pre-copy of next round's input data |220+| `subThreads[1]` | 拷贝子线程 | 预拷贝下一轮输入数据 |
220 221 
221-Inter-thread synchronization is performed via `PreSyncInterThreads` / `PostSyncInterThreads`.222+线程间同步通过 `PreSyncInterThreads`/`PostSyncInterThreads`完成。
222 223 
223-### 4. Key Logic224+### 4. 关键逻辑
224 225 
225-#### 4.1 Algorithm Overview226+#### 4.1 算法概述
226 227 
227-The core property of the BIRS algorithm is **batch invariance**: the order of reduction additions on each rank is strictly identical, regardless of batch size or memory slicing.228+BIRS算法的核心性质是**batch 不变性**:每个rank上的归约加法顺序严格一致,不受batch size或内存分片影响。
228 229 
229-**Notation**:230+**符号约定**
230-- `S(d, i)`: The i-th slice of the input message on device d
231-- `rankSizeX = 2`, `rankSizeY = rankSize / 2`
232-- `sio_rank = rank XOR 1` (SIO peer)
233-- `hccs_ranks[i] = (rank + rankSizeX * i) % rankSize` (HCCS peer sequence)
234 231 
235-#### 4.2 Main Communication Loop232+- `S(d, i)`:设备d上输入消息的第i个切片。
233+- `rankSizeX = 2``rankSizeY = rankSize / 2`
234+- `sio_rank = rank XOR 1`(SIO 对端)。
235+- `hccs_ranks[i] = (rank + rankSizeX * i) % rankSize`(HCCS 对端序列)。
236 236 
237-```237+#### 4.2 主通信循环
238-// Initial: copy the input slice corresponding to the first HCCS peer into scratch memory238+ 
239+```c
240+// 初始:将第一个 HCCS 对端对应的输入切片拷贝到 scratch 内存
239LocalCopy(input[S(hccs_ranks[0])], scratch[IM_0])241LocalCopy(input[S(hccs_ranks[0])], scratch[IM_0])
240 242 
241for round in 0 ... hccs_ranks.size():243for round in 0 ... hccs_ranks.size():
242 244 
243- // ── Sub-thread 0: HCCS transfer (when round > 0) ──245+ // ── 子线程 0HCCS 传输(round > 0 时) ──
244 if round > 0:246 if round > 0:
245 Notify(sio → hccs_ack)247 Notify(sio → hccs_ack)
246 Wait(hccs_ack)248 Wait(hccs_ack)
@@ -248,118 +250,118 @@ for round in 0 ... hccs_ranks.size():
248 Notify(data_signal)250 Notify(data_signal)
249 Wait(data_signal)251 Wait(data_signal)
250 252 
251- // ── Main thread: SIO SendReduce ──253+ // ── 主线程:SIO SendReduce ──
252 Notify(sio_ack)254 Notify(sio_ack)
253 Wait(sio_ack)255 Wait(sio_ack)
254 SendReduce(256 SendReduce(
255- local: input[S(hccs_neighbour_rank[round])], // or S(sio_rank) in the last round257+ local: input[S(hccs_neighbour_rank[round])], // 最后一轮为 S(sio_rank)
256 remote: scratch[IM_round on sio_peer]258 remote: scratch[IM_round on sio_peer]
257 ) → scratch[IM_round on sio_peer]259 ) → scratch[IM_round on sio_peer]
258 Notify(data_signal)260 Notify(data_signal)
259 Wait(data_signal)261 Wait(data_signal)
260 262 
261- // ── Sub-thread 1: Pre-copy next round's data ──263+ // ── 子线程 1:预拷贝下一轮数据 ──
262 if round < hccs_ranks.size() - 1:264 if round < hccs_ranks.size() - 1:
263 LocalCopy(input[S(hccs_ranks[round+1])], scratch[next_slot])265 LocalCopy(input[S(hccs_ranks[round+1])], scratch[next_slot])
264```266```
265 267 
266-#### 4.3 Final Reduction (FinalStep)268+#### 4.3 最终归约(FinalStep
267 269 
268-After all rounds complete, each rank holds `rankSizeY` intermediate results in scratch memory. These are merged via a **tree-based local reduction**:270+所有轮次完成后,每个rank在scratch内存中持有`rankSizeY`个中间结果。这些结果通过**树形本地归约**合并:
269 271 
270-```272+```text
271-// Collect all intermediate result offsets273+// 收集所有中间结果偏移
272-vec = [IM_0, IM_1, ..., IM_{rankSizeY-1}] // this rank's result is at the correct position274+vec = [IM_0, IM_1, ..., IM_{rankSizeY-1}] // rank 的结果位于正确位置
273 275 
274-// Tree reduction (guarantees deterministic addition order)276+// 树形归约(保证确定性加法顺序)
275for stride in 1, 2, 4, ...:277for stride in 1, 2, 4, ...:
276 for i in stride, stride+stride, ...:278 for i in stride, stride+stride, ...:
277 LocalReduce(vec[i] → vec[i - stride])279 LocalReduce(vec[i] → vec[i - stride])
278 280 
279-// Copy final result to output281+// 将最终结果拷贝到输出
280LocalCopy(vec[0] → outputMem)282LocalCopy(vec[0] → outputMem)
281```283```
282 284 
283-The tree reduction guarantees deterministic addition ordering: for `rankSizeY = 4`, the reduction order is `(IM_0 + IM_1) + (IM_2 + IM_3)`, independent of rank id. ReduceScatterBIRS() supports reduction for rankSize <= 16, for larger rankSize it is recommended to use ReduceScatterBIRSInter().285+树形归约保证确定性加法顺序:以`rankSizeY = 4`为例,归约顺序为 `(IM_0 + IM_1) + (IM_2 + IM_3)`,与rank id无关。ReduceScatterBIRS()支持rankSize <= 16的归约,对于更大的rankSize,建议使用ReduceScatterBIRSInter()
284 286 
285-### 5. Compatibility Considerations287+### 5. 兼容性考虑
286 288 
287-#### 5.1 Backward Compatibility289+#### 5.1 向后兼容
288 290 
289-- **Fully backward compatible**: BIRS is disabled by default (`HCCL_BIRS_ENABLE` defaults to `FALSE`), with no impact on existing ReduceScatter behavior.291+- **完全向后兼容**BIRS默认关闭(`HCCL_BIRS_ENABLE` 默认为`FALSE`),对现有的ReduceScatter行为无任何影响。
290-- **Build isolation**: Experimental code resides in an independent `experimental/` directory, controlled by the `ENABLE_EXPERIMENTAL` build flag, and is excluded from compilation by default.292+- **构建隔离**:实验代码位于独立的`experimental/`目录,由`ENABLE_EXPERIMENTAL`编译标志控制,默认不参与编译。
291-- **No API changes**: The user-facing API (`HcclReduceScatter`) remains unchanged; algorithm selection is transparent to users.293+- **API变更**:用户态API`HcclReduceScatter`)保持不变,算法选择对用户透明。
292 294 
293-#### 5.2 Applicability Conditions295+#### 5.2 适用条件
294 296 
295-The BIRS algorithm currently has the following constraints:297+BIRS 算法当前有以下约束:
296 298 
297-| Constraint | Description |299+| 约束 | 说明 |
298-|------------|-------------|300+|------|------|
299-| Platform | A3 servers only (SIO + HCCS hybrid topology) |301+| 平台 | A3服务器(SIO + HCCS 混合拓扑)。 |
300-| rankSize | Must be even (`rankSize % 2 == 0`), typical values: 4, 8, 16 |302+| rankSize | 必须为偶数(`rankSize % 2 == 0`),典型值:4816 |
301-| Communication domain | Both Intra-server and Inter-server are supported |303+| 通信域 | 同时支持单服务器内和跨服务器。|
302-| Data alignment | Slice sizes must satisfy `HCCL_MIN_SLICE_ALIGN_910B` alignment requirements |304+| 数据对齐 | 切片大小须满足`HCCL_MIN_SLICE_ALIGN_910B` 对齐要求。 |
303 305 
304-ReduceScatterBIRS() is the recommended choice for single-server A3 scenario (rankSize <= 16), ReduceScatterBIRSInter() is chosen automatically for multi-server A3 scenario.306+ReduceScatterBIRS() A3单服务器场景(rankSize <= 16)的推荐选择,A3多服务器场景自动选择 ReduceScatterBIRSInter()
305 307 
306-When conditions are not met, the workflow exit and log error messages using hccl. The user must follow the recommendations in the error logs or manually adjust the parameters to comply with the restrictions.308+当条件不满足时,流程退出并通过HCCL记录错误日志。用户需根据错误日志中的建议操作或手动调整参数以满足约束条件。
307 309 
308-#### 5.3 Rollout Strategy310+#### 5.3 上线策略
309 311 
310-1. **Phase 1** (current): As an experimental feature, with dual gating via `ENABLE_EXPERIMENTAL=ON` compile flag + `HCCL_BIRS_ENABLE=TRUE` runtime flag.312+1. **阶段 1**(当前):作为实验特性,通过`ENABLE_EXPERIMENTAL=ON`编译标志 + `HCCL_BIRS_ENABLE=TRUE`运行时标志双重门控。
311-2. **Phase 2** (post-validation): Remove compile-time gating, retain only environment variable control.313+2. **阶段 2**(验证后):移除编译期门控,仅保留环境变量控制。
312-3. **Phase 3** (post-stabilization): Automatically select BIRS as the default algorithm when conditions are met; users can disable via `HCCL_BIRS_ENABLE=FALSE`.314+3. **阶段 3**(稳定后):满足条件时自动选择BIRS作为默认算法;用户可通过`HCCL_BIRS_ENABLE=FALSE`关闭。
313 315 
314-### 6. Test Plan316+### 6. 测试方案
315 317 
316-#### 6.1 Functional Correctness Testing318+#### 6.1 功能正确性测试
317 319 
318-- **New test file**: `test/st/algorithm/testcase/reduce_scatter_testcase_a3.cc`320+- **新增测试文件**`test/st/algorithm/testcase/reduce_scatter_testcase_a3.cc`
319-- **Test dimensions**:321+- **测试维度**
320- - Different rankSize values (4, 8, 16)322+ - 不同rankSize值(4816
321- - Different data types (FP16, FP32, BF16)323+ - 不同数据类型(FP16FP32BF16
322- - Different reduction operations (SUM, MAX, MIN, PROD)324+ - 不同归约操作(SUMMAXMINPROD
323- - Different message sizes (from KB-level to tens of MB)325+ - 不同消息大小(从KB级到数十MB
324 326 
325-#### 6.2 Batch Invariance Verification327+#### 6.2 Batch不变性验证
326 328 
327-- Execute ReduceScatter with the same input data but different batch sizes329+- 使用相同输入数据但不同batch size执行ReduceScatter。
328-- Verify that output results are bit-identical330+- 验证输出结果比特一致。
329 331 
330-#### 6.3 Performance Testing332+#### 6.3 性能测试
331 333 
332-- Compare against the RHD algorithm, measuring Task Duration across different message sizes334+- RHD算法进行对比,测量不同消息大小下的任务执行时长(Task Duration)。
333-- Expected: For message sizes ≥ 16MB, BIRS achieves up to 25% improvement over RHD. 335+- 预期:消息大小 >= 16MB时,BIRS相较RHD实现最高25%的提升。
334-- Note: At the moment of this RFC creation kernel submission mechanism in HCCL is slower than the one of HCOMM, so 25% performance improvement applies only to operator execution time (without submission overhead).336+- 说明:在本RFC创建时,HCCL的算子下发机制慢于HCOMM,因此25%的性能提升仅适用于算子执行时间(不含下发开销)。
335 337 
336-#### 6.4 Regression Testing338+#### 6.4 回归测试
337 339 
338-- Ensure all existing ReduceScatter test cases are unaffected when `HCCL_BIRS_ENABLE=FALSE` (default)340+- 确保`HCCL_BIRS_ENABLE=FALSE`(默认)时所有现有 ReduceScatter测试用例不受影响。
339 341 
340-## Risk Assessment342+## 风险评估
341 343 
342-| Risk | Impact | Mitigation |344+| 风险 | 影响 | 缓解措施 |
343-|------|--------|------------|345+|------|------|----------|
344-| BIRS only available for specific rankSize (even numbers) | Odd rank scenarios cannot use BIRS | `MatchBIRS()` check auto-falls back to existing algorithms; document constraints clearly |346+| BIRS仅适用于特定rankSize(偶数) | 奇数rank场景无法使用BIRS | `MatchBIRS()`检查自动回退到现有算法;明确文档化约束条件。 |
345-| Experimental code may introduce stability issues | Affects overall HCCL reliability | Dual gating (compile + runtime) isolation; independent `experimental/` directory; disabled by default |347+| 实验代码可能引入稳定性问题 | 影响HCCL整体可靠性 | 双重门控(编译 + 运行时)隔离;独立`experimental/`目录;默认关闭。 |
346-| Additional scratch memory overhead | Increased memory usage for large messages | Requires `2 * rankSizeY × localStrideSize` scratch space; pre-allocated via `CalcResRequest` |348+| 额外scratch内存开销 | 大消息场景内存使用增加 | 需要`2 * rankSizeY × localStrideSize`scratch空间;通过`CalcResRequest`预分配。 |
347-| A3 topology assumption (SIO + HCCS) may not apply to other platforms | Cross-platform compatibility | Algorithm explicitly bound to A3 topology characteristics; other platforms require independent adaptation |349+| A3拓扑假设(SIO + HCCS)可能不适用于其他平台 | 跨平台兼容性 | 算法明确绑定A3拓扑特性;其他平台需独立适配。 |
348 350 
351+## 替代方案
349 352 
350-## Alternative Approaches353+
351 354 
352-N/A355+## 开放问题
353 356 
354-## Open Questions357+1. **AllReduce扩展**:遵循相同思路的batch不变AllReduce将在单独的PR中提交。
355- 358+2. **高效支持任意rank编号**:当前方案假设默认rank编号,即RankX的SIO邻居rankID可通过(RankX XOR 1)计算。在其他rank编号方式下,BIRS功能正常但无法提供相较RHD的性能优势。对自定义编号的高效支持已实现,将在下一个PR中提交。
356-1. **AllReduce extension**: Batch-invariant AllReduce which follows the same ideas will be submitted in separate PR
357-2. **Efficient support for arbitrary rank enumeration**: Current solution assumes default rank enumeration where rankID of SIO neighbour of RankX can be calculated as (RankX XOR 1). In case of other rank enumerations BIRS is functional but doesn't deliver performance advantage over RHD. Efficient support for custom enumerations have already been implemented and will be submitted in the next PR.
358 359 
359---360---
360 361 
361-## Review Records362+## 评审记录
363+ 
364+检视过程在PR评论区进行。详细检视评论请参考对应PR:
362 365 
363-The review process takes place in the PR comment section. For detailed review comments, please refer to the corresponding PR:
364- PR: [cann/hccl#657](https://gitcode.com/cann/hccl/pull/657)366- PR: [cann/hccl#657](https://gitcode.com/cann/hccl/pull/657)
365- Issues: [cann/hcomm#139](https://gitcode.com/cann/hcomm/issues/139), [cann/hccl#96](https://gitcode.com/cann/hccl/issues/96)367- Issues: [cann/hcomm#139](https://gitcode.com/cann/hcomm/issues/139), [cann/hccl#96](https://gitcode.com/cann/hccl/issues/96)
@@ -2,9 +2,9 @@
2 2 
3## Sample Description3## Sample Description
4 4 
5-This sample demonstrates how to call the `HcclAllReduce()` API to perform an `AllReduce` operation. It covers the following functions: 5+This sample demonstrates how to call the `HcclAllReduce()` API to perform an `AllReduce` operation. It covers the following functions:
6 6 
7-- Call `aclrtGetDeviceCount()` to detect devices and query the number of available devices.7+- Call `aclrtGetDeviceCount()` to detect devices and query the number of available devices.
8- Call `HcclGetRootInfo()` and use `rank 0` as the root rank to generate the rootinfo identifier.8- Call `HcclGetRootInfo()` and use `rank 0` as the root rank to generate the rootinfo identifier.
9 9 
10 > The rootinfo identifier contains the device IP address and device ID. This information must be broadcast to all ranks in the cluster to initialize the communicator.10 > The rootinfo identifier contains the device IP address and device ID. This information must be broadcast to all ranks in the cluster to initialize the communicator.
@@ -9,8 +9,8 @@
9 9 
10 > rootinfo 标识信息主要包含:Device IP、Device ID 等信息,此信息需广播至集群内所有 rank 用来初始化通信域。10 > rootinfo 标识信息主要包含:Device IP、Device ID 等信息,此信息需广播至集群内所有 rank 用来初始化通信域。
11 11 
12-- 在每个线程中,基于 rootinfo 标识信息通过 `HcclCommInitRootInfo()` 接口初始化通信域。12+ - 在每个线程中,基于 rootinfo 标识信息通过 `HcclCommInitRootInfo()` 接口初始化通信域。
13-- 调用 `HcclBroadcast()` 接口,将通信域内 root 节点的数据广播至其他 rank,并打印结果。13+ - 调用 `HcclBroadcast()` 接口,将通信域内 root 节点的数据广播至其他 rank,并打印结果。
14 14 
15## 目录结构15## 目录结构
16 16 
@@ -4,7 +4,7 @@
4 4 
5This sample demonstrates how to call the `HcclAlltoAll()` API to perform an `AlltoAll` operation. It covers the following functions:5This sample demonstrates how to call the `HcclAlltoAll()` API to perform an `AlltoAll` operation. It covers the following functions:
6 6 
7-- Call `aclrtGetDeviceCount()` to detect devices and query the number of available devices.7+- Call `aclrtGetDeviceCount()` to detect devices and query the number of available devices.
8- Call `HcclGetRootInfo()` and use `rank 0` as the root rank to generate the rootinfo identifier.8- Call `HcclGetRootInfo()` and use `rank 0` as the root rank to generate the rootinfo identifier.
9 9 
10 > The rootinfo identifier contains the device IP address and device ID. This information must be broadcast to all ranks in the cluster to initialize the communicator.10 > The rootinfo identifier contains the device IP address and device ID. This information must be broadcast to all ranks in the cluster to initialize the communicator.
@@ -4,10 +4,10 @@
4 4 
5This sample demonstrates how to use the `HcclScatter()` API to perform collective communication. It covers the following functions:5This sample demonstrates how to use the `HcclScatter()` API to perform collective communication. It covers the following functions:
6 6 
7-- Call `aclrtGetDeviceCount()` to detect devices and query the number of available devices.7+- Call `aclrtGetDeviceCount()` to detect devices and query the number of available devices.
8- Call `HcclGetRootInfo()` and use `rank 0` as the root rank to generate the rootinfo identifier.8- Call `HcclGetRootInfo()` and use `rank 0` as the root rank to generate the rootinfo identifier.
9 9 
10-> The rootinfo identifier contains the device IP address and device ID. This information must be broadcast to all ranks in the cluster to initialize the communicator.10+ > The rootinfo identifier contains the device IP address and device ID. This information must be broadcast to all ranks in the cluster to initialize the communicator.
11 11 
12- In each thread, call `HcclCommInitRootInfo()` to initialize the communicator based on the rootinfo identifier.12- In each thread, call `HcclCommInitRootInfo()` to initialize the communicator based on the rootinfo identifier.
13- Call `HcclScatter()` to evenly distribute the data of the root node in the communicator to all other ranks and display the results.13- Call `HcclScatter()` to evenly distribute the data of the root node in the communicator to all other ranks and display the results.
@@ -2,7 +2,7 @@
2 2 
3## Sample Description3## Sample Description
4 4 
5-This sample demonstrates how to perform an AllReduce operation using the PyTorch interface. It covers the following features:5+This sample demonstrates how to perform an AllReduce operation using the PyTorch interface. It covers the following functions:
6 6 
7- Device detection: Query the number of available devices using the `torch_npu.npu.device_count()` interface.7- Device detection: Query the number of available devices using the `torch_npu.npu.device_count()` interface.
8- Start multiple processes using the `torch.multiprocessing.spawn()` interface.8- Start multiple processes using the `torch.multiprocessing.spawn()` interface.
@@ -2,13 +2,13 @@
2 2 
3## Sample Description3## Sample Description
4 4 
5-This sample demonstrates how to perform an AllReduce operation using the TensorFlow interface. It covers the following features:5+This sample demonstrates how to perform an AllReduce operation using the TensorFlow interface. It covers the following functions:
6 6 
7- Initialize the communication domain based on the `ranktable.json` configuration file.7- Initialize the communication domain based on the `ranktable.json` configuration file.
8 8 
9## Environment Preparation9## Environment Preparation
10 10 
11-### Prerequisites11+### Environment Requirements
12 12 
13This sample supports the following products in a single-server 8-card configuration:13This sample supports the following products in a single-server 8-card configuration:
14 14 
@@ -54,7 +54,7 @@ Using the root user default installation path as an example:
54source /usr/local/Ascend/cann/set_env.sh54source /usr/local/Ascend/cann/set_env.sh
55```55```
56 56 
57-Additionally, running test cases requires an MPI environment. Ensure that MPI is installed and configured.57+Additionally, running the test cases requires MPI environment support. Please ensure that MPI has been installed and configured correctly. For MPI configuration, please refer to the "MPI Installation and Configuration" section in the corresponding version of the [Ascend Documentation Center - HCCL Performance Test Tool User Guide](https://hiascend.com/en/document/redirect/CannCommunityToolHcclTest).
58 58 
59## 2. Compilation and Execution59## 2. Compilation and Execution
60 60 
@@ -201,7 +201,7 @@ export LD_LIBRARY_PATH=${ASCEND_HOME_PATH}/opp/vendors/cust/lib64:${LD_LIBRARY_P
201 201 
202```text202```text
203Found 2 NPU device(s) available203Found 2 NPU device(s) available
204-rankId1, input: [1 1]204+rankId: 1, input: [1 1]
205rankId: 0, input: [0 0]205rankId: 0, input: [0 0]
206rankId: 0, output: [ 0 0 1 1 ]206rankId: 0, output: [ 0 0 1 1 ]
207rankId: 1, output: [ 0 0 1 1 ]207rankId: 1, output: [ 0 0 1 1 ]
@@ -1 +1 @@
1-# readme1+# README
@@ -203,7 +203,7 @@ struct SrcBufDes {
203- **错误场景:**203- **错误场景:**
204![](./figures/allgather-3.png)204![](./figures/allgather-3.png)
205 205 
206-#### 5.1.3 定位思路206+#### 5.1.4 定位思路
207 207 
208语义校验阶段可以发现两种类型的错误:208语义校验阶段可以发现两种类型的错误:
209 209 
@@ -182,11 +182,11 @@ The following example illustrates what semantics calculation is.
182 182 
183 ![](./figures/allgather.png)183 ![](./figures/allgather.png)
184 184 
185-2. Action state 1: Move a data block from rank0 Input with offset 20 and size 30 to rank0 Output with offset 35. Result: A semantic block is generated on rank0 Output, recording the movement information.185+2. State 1 action: Move a data block from rank0 Input with offset 20 and size 30 to rank0 Output with offset 35. Result: A semantic block is generated on rank0 Output, recording the movement information.
186 186 
187 ![](./figures/allgather-0.png)187 ![](./figures/allgather-0.png)
188 188 
189-3. Action state 2: Move a data block from rank1 Input with offset 70 and size 15 to rank0 Output with offset 50. Result: The destination memory overlaps with the existing semantic block. The existing semantic block must be split, generating two semantic blocks.189+3. State 2 action: Move a data block from rank1 Input with offset 70 and size 15 to rank0 Output with offset 50. Result: The destination memory overlaps with the existing semantic block. The existing semantic block must be split, generating two semantic blocks.
190 190 
191 ![](./figures/allgather-1.png)191 ![](./figures/allgather-1.png)
192 192