合并受阻
变更摘要
此 PR 新增了基于 Triton 的 _layer_norm_fwd_1pass_kernel 单次前向 LayerNorm/RMSNorm 算子,专为昇腾 910 系列 NPU 优化。主要引入了 API 接口、Triton JIT 内核实现以及完备的单元测试,支持带偏置和 z-gating(激活前/后门控)的层归一化,并通过 UB 容量感知的动态分块策略和发射参数选择来提升性能。
主要改动
-
新增
layer_norm_fwd_1passAPI 函数:在mindspeed_ops/api/triton/layer_norm_fwd_1pass.py中提供高层接口,支持 LayerNorm 与 RMSNorm 两种模式、可选 bias、可选 z-gating(NORM_BEFORE_GATE控制门控在归一化前或后施加),以及多组(ngroups)归一化,并通过_compute_bt根据 192KB UB 容量动态计算每任务处理的 token 行数,通过_select_launch_params根据隐藏维大小动态选择num_warps和num_stages。 -
新增 Triton JIT 内核
_layer_norm_fwd_1pass_kernel:在mindspeed_ops/arch32/triton/layer_norm_fwd_1pass.py中实现核心计算逻辑,采用一维物理核网格 + 任务分发机制,批量处理 BT 行作为单个[BT, BLOCK_N]tile,复用 weight/bias 加载,并融合乘加运算(x_hat * w + b)、支持均值/逆标准差提前写出以优化写流。 -
新增单元测试:在
tests/unit_tests/triton/test_layer_norm_fwd_1pass.py中覆盖多种 shape(2×64 至 64×2048)、三种数据类型(fp32/bf16/fp16)以及 RMS/LayerNorm、有无 z-gating、门控位置等组合,同时提供ngroups多组归一化测试,以 CPU 参考实现作为 golden 对比验证精度。


代码审查
审查关闭总结
-
已审查文件:4 个
docs/triton/layer_norm_fwd_1pass.md— 无问题mindspeed_ops/api/triton/layer_norm_fwd_1pass.py— 无问题mindspeed_ops/arch32/triton/layer_norm_fwd_1pass.py— 无问题tests/unit_tests/triton/test_layer_norm_fwd_1pass.py— 1 个 P3 发现
-
发现统计:
- P0: 0
- P1: 0
- P2: 0
- P3: 1(测试未验证 mean/rstd 输出)
-
总体风险判断:低风险。新增的 Triton LayerNorm 前向算子实现结构清晰、逻辑正确,kernel 内部的计算(均值、方差、归一化、门控、分组)与 CPU 参考实现一致,UB 容量规划有合理的安全余量。API 层的参数校验、contiguous 处理、shape 恢复均正确。唯一的 P3 发现是测试未直接校验 mean/rstd 辅助输出,不影响生产使用。
| 类型 | 数量 |
|---|---|
| 🔴 阻塞 | 0 |
| 🟡 建议 | 2 |
💬 仅评论


Thanks for your pull-request.
The full list of commands accepted by me can be found at here.
You can get sig-info at here.
You can self-configure the PR merge rules for this repository. For more details, please refer to Here.
PR Approval Progress
⚠️ This PR does not yet meet the following requirements:lgtm (requires ≥ 2 person(s) per module)、approve (requires ≥ 1 person(s) per module)
Module Approval Details
| module | lgtm status | approve status |
|---|---|---|
| docs | ❌ (0/2)(You can also ask: 朱彦儒, 孙银磊, bigdog1206, feng0w0, guofanfeng23) | ❌ (0/1)(You can also ask: 华郁秀, 刘哲续, 孙银磊, 刘荣, bigdog1206) |
| repo-Ascend/MindSpeed-Ops | ❌ (0/2)(You can also ask: LinMingZhe, HANHU1CHEN, 朱彦儒, 周蓓蓉, 王晓歆) | ❌ (0/1)(You can also ask: 雷珍珍, 王晓歆, 刘哲续, 周蓓蓉, 刘荣) |
💡 Tip:
- Committer can comment
/approveor/lgtm- Commenting
/approveimplies both code review (lgtm) and intent to merge (approve)
CLA Signature Pass
gcw_K5CvmS79, thanks for your pull request. All authors of the commits have signed the CLA. 👍


compile


Related issue: #10
———
What this PR does / why we need it?
Migrates and optimizes the
_layer_norm_fwd_1pass_kerneloperator from the open-source state-spaces/mamba repo to run efficiently on Ascend NPU (Atlas 800T A2/A3, verified on 910C).The operator implements a fused LayerNorm/RMSNorm forward pass with optional SiLU gating (
z * sigmoid(z)), supportingnorm_before_gateand grouped normalization (ngroups).Key NPU optimizations over the original per-row GPU kernel:
(num_core,)+ in-kernel task dispatch (tl.range(core_id, task_num, num_core)), replacing the original 2D(M, ngroups)grid._compute_bt).[BT, BLOCK_N]tile and reduced in a singletl.sum(axis=1), eliminating the per-row scalar loop (main perf lever).num_warps/num_stagesby hidden size.Does this PR introduce any user-facing change?
Yes — adds a new public Triton operator API:
from mindspeed_ops.api.triton.layer_norm_fwd_1pass import layer_norm_fwd_1pass layer_norm_fwd_1pass(x, weight, bias=None, z=None, eps=1e-5, norm_before_gate=True, is_rms_norm=False, ngroups=1) # Returns: (y, mean, rstd)New files (7):
mindspeed_ops/arch32/triton/layer_norm_fwd_1pass.pymindspeed_ops/api/triton/layer_norm_fwd_1pass.pytests/unit_tests/triton/test_layer_norm_fwd_1pass.pytests/atk_tests/triton/layer_norm_fwd_1pass/generate_layer_norm_fwd_1pass.pytests/atk_tests/triton/layer_norm_fwd_1pass/layer_norm_fwd_1pass.yamltests/atk_tests/triton/layer_norm_fwd_1pass/triton_layer_norm_fwd_1pass.pydocs/triton/layer_norm_fwd_1pass.mdHow was this patch tested?
1. UT 精度测试 (147 passed)
(2,64), (4,128), (8,256), (16,512), (32,1024), (64,2048)2. ATK 精度测试 (23/23 passed)
single_bm(Triton vs CPU golden)3. ATK 性能测试 (23 cases, Triton vs torch_npu)
4. 优化前后对比 (msprof kernel 级, fp16, 910C)
对比原始 Triton 实现(per-row
grid=(M,)基线):5. 竞品性能对比(910C vs A100)
算子分类: VV 类算子,Memory Bound(msprof 显示 aivector compute usage < 20%,瓶颈在 HBM 搬运)
折算方法(按任务书公式):
Constraints / Limitations
NotImplementedError)BLOCK_N = next_power_of_2(N); very large N may hit UB overflowcontiguous(); internal compute in float32, output keeps input dtype