Merged
fix(bn3d_training_reduce): preserve FP16 full-R slot merge precision #13622
fix(bn3d_training_reduce): preserve FP16 full-R slot merge precision #13622
Merged
raoliang_sacAdd on 18 days ago
raoliang_sac
raoliang_sacCollaborators
18 days ago

描述

修复 FP16 普通 full-R 多槽归约在槽间合并时丢失新舍入低位的问题。BN3DTrainingReduce_L1_149 的 sum 因此相差 1 ULP,三方 RMSE ratio 为 2.0,超过 L1 门槛 1.5。该路径改用已有 TwoSum 补偿合并,并补充原场景 ST;Host tiling、UB 分配、golden 和精度阈值不变。

关联的Issue

https://gitcode.com/cann/ops-nn/issues/8106

测试

  • 相同原始输入及 CPU/GPU 参考:旧代码 kernel 3/3、GEIR 1/1 失败,修复后 4/4 通过。
  • 24 条边界、16 条仓内 ST 和固定输入两方验证通过;全量结果如下。

原报告 588 条全部纳入本次计划,取得新的 TTK 结果 586/588。

  • 正向两方 mixed:实际执行 562 条,562 条通过。

  • 正向三方 cross_check L1:实际执行 562 条,562 条通过;两组为同一批正向用例,不累加为独立用例数。

  • 修复版本本次回归精度失败:0。按用户明确要求不测的超大内存用例:2,为 BN3DTrainingReduce_L1_upboundary_highpre_078, BN3DTrainingReduce_L1_upboundary_highpre_094;不计为通过。其他正向用例均已完成两方与三方验证。

  • 24 条反向用例:11 条非法输入被拒绝,13 条合法 GEIR 输入被接受或重新推导输出,原报错预期需修订。原备注中的精确 ACL 错误码未由 GEIR 拒绝结果证明;这 24 条单独核对,不记为精度通过。

  • 原失败场景 NPU 10.968 μs,A100 GPU torch.compile 复合实现 14.0672 μs;GPU/NPU×0.9728=1.247682,满足 ≥0.1。

  • TTK 5bc080fe42c25b2a11654e514de10ac08771cb02,全程 --pc=1;构建 jobs=2。完整证据位于任务 validation/round3_full_20261104_103821。本地 CI 已通过,受测版本与明细见下方补充。

本地CI补充:受测提交7e5d8c6a889c3098d70a687092377e61df68d718,主仓master快照1566185bf41ab3fc1c2cdda13aeff45fcbcb95f3。

  • PR原先已有1个提交,无需重复squash;已保留备份,核对整理前后完整diff和tree一致,rebase后相对主线仍为1个提交。
  • 通过指定/workspace/CI-skills/ci_local_check.sh入口,pre-commit、隔离OAT、链接、示例SKIP契约和Check_Example检查均通过。
  • 仅对bn3d_training_reduce执行定向检查,构建参数--soc=ascend950、--soc=ascend910b的package和kernelUT目标构建均通过;-j2。该算子没有独立kernelUT用例,本次不宣称执行了kernelUT数值用例。
  • 算子实现、构建文件、测例及已核对的共享依赖与此前TTK精度受测提交6e2e4ab33230b06f9b5306cf398aa7f63ca98055一致;主线仅在算子目录新增README_en.md,已单独核对,不属于本PR改动。本轮未重复执行数值回归,也未修改README/API文档或示例。
  • PR远端HEAD已核对与本地CI受测SHA一致。compile触发评论及线上流水线已确认;最终线上通过状态尚未核实。

文档更新

无接口文档或示例变更。

类型标签

AI/Agent生成声明

likedislike
Pull Request has been successfully merged by CANN-robot.
(Thank you for raoliang_sac's contribution.)
raoliang_sacraoliang_sacCollaborators
18 days ago Created pull request, commit 2fd2e9f6
atomgit-bot
atomgit-bot
18 days ago Comments:

变更摘要

本 PR 修复 FP16 普通 full-R 多槽归约在槽间合并时丢失新舍入低位的问题。改动集中在 bn3d_training_reduce_dense_channel.h:在槽间合并的策略判定中,把原先仅针对 FP16 sub-R 的条件扩展为同时覆盖非 C0_PACKED 的普通 FP16 full-R 路径,使其改用带 TwoSum 补偿的 FoldAccSlots<false> 合并;C0 full-R 与 split-reduce 仍沿用原有折叠策略,Host tiling、UB 分配、golden 与精度阈值均不变。

主要改动

  • 扩展槽间补偿合并的触发条件:合并判定由 isSubR_ != 0 改为 isSubR_ != 0 || !C0_PACKED,使普通 FP16 full-R 的多 tile 累加槽合并同样进入 FoldAccSlots<false> 分支,而非默认的 FoldAccSlots。
  • 保留槽间相加新产生的舍入误差:通过走 TwoSum 补偿合并路径,把槽间主值相加所产生的新舍入误差同步并入 comp,避免旧实现仅做 Σacc − Σcomp 在临界舍入点丢失 1 ULP,从而修复 BN3DTrainingReduce_L1_149 的 sum 偏差与三方 RMSE ratio 超阈值问题。
  • 同步更新相关注释:将“FP16 sub-R 的多个搬运块”相关说明改为“FP16 sub-R 与普通 full-R 多 tile 的槽间合并均需保留新产生的舍入低位”,并说明普通 FP16 full-R 保留该误差、C0 full-R 和 split-reduce 保持原策略。

🧩 本次 Review 已使用自定义 Skill:

likedislike
Incorrect?
atomgit-bot
atomgit-bot
18 days ago Comments:

代码审查

✅ 未发现问题


🧩 本次 Review 已使用自定义 Skill:

likedislike
Incorrect?
CANN-robotCANN-robotCollaborators
18 days ago Added label: cann-cla/yes
CANN-robot
CANN-robotCollaborators
18 days ago Comments:

Thanks for your pull-request.
The full list of commands accepted by me can be found at here.
You can get sig-info at here.
You can self-configure the PR merge rules for this repository. For more details, please refer to Here.
For more, you also can visit HICANN.


PR Approval Progress

✅ Congratulations! All modules have met the lgtm and approve requirements.

Module Approval Details

module lgtm status approve status
norm ✅ 钱泽洪, 陈展熹 (2/2) ✅ 钱泽洪, 陈展熹 (2/1)

💡 Tip:

  • Committer can comment /approve or /lgtm
  • Commenting /approve implies both code review (lgtm) and intent to merge (approve)

CLA Signature Pass

raoliang_sac, thanks for your pull request. All authors of the commits have signed the CLA. 👍

likedislike
raoliang_sacraoliang_sacCollaborators
18 days ago Modify description of pull request
6 event messages are collapsed here。 See more
CANN-robotCANN-robotCollaborators
18 days ago Added label: cann-cla/yes
raoliang_sac
raoliang_sacCollaborators
18 days ago Comments:

compile

likedislike
raoliang_sacraoliang_sacCollaborators
18 days ago Pre merge succeeded (commit_id: 4c4346e92dfa5fef98ad12310c7d7d2c3534c27a)
CANN-robot
CANN-robotCollaborators
18 days ago Comments:
🚀 CI 流水线已启动
📋 执行详情: 点击查看流水线
likedislike
CANN-robotCANN-robotCollaborators
18 days ago Added label: ci-pipeline-running
raoliang_sacraoliang_sacCollaborators
18 days ago Modify description of pull request
陈展熹
陈展熹Collaborators
18 days ago Comments:

/approve

likedislike
CANN-robotCANN-robotCollaborators
18 days ago Added label: approved
CANN-robotCANN-robotCollaborators
18 days ago Removed label: ci-pipeline-running
CANN-robotCANN-robotCollaborators
18 days ago Added label: ci-pipeline-passed
qianzehong
qianzehongCollaborators
18 days ago Comments:

/lgtm
/approve

likedislike
CANN-robotCANN-robotCollaborators
18 days ago Added label: lgtm
CANN-robotCANN-robotCollaborators
18 days ago Closed associated issue
CANN-robotCANN-robotCollaborators
18 days ago Merged pull request