Merged
fix(moe_gating_top_k): sigmoid除法改为默认模式,对齐aclnnSigmoid舍入口径 #14971
fix(moe_gating_top_k): sigmoid除法改为默认模式,对齐aclnnSigmoid舍入口径 #14971
Merged
CPZhouAdd on 10 days ago
CPZhou
CPZhouCollaborators
10 days ago

描述

sigmoid除法改为默认模式,对齐aclnnSigmoid舍入口径

关联的Issue

#6944

测试

已通过冒烟测试

文档更新

不涉及

类型标签

likedislike
Pull Request has been successfully merged by CANN-robot.
(Thank you for CPZhou's contribution.)
CPZhouCPZhouCollaborators
10 days ago Created pull request, commit f48224dd
atomgit-bot
atomgit-bot
10 days ago Comments:

变更摘要

本 PR 针对 moe_gating_top_k 算子在 arch35 上的 regbase 实现,将 Sigmoid 计算中的 1 / (1 + exp(-x)) 除法由显式指定的 DivSpecificMode(MaskMergeMode::ZEROING 且开启特定模式标志)改为不指定模式的默认 Reg::Div,以对齐 aclnnSigmoid 的舍入口径,使算子间数值结果保持一致。

主要改动

  • 统一 Sigmoid 除法模式: 在 moe_gating_top_k_regbase.h 的 SigmoidWithBias 与 SigmoidWithoutBias 中,Reg::Div 不再使用 <float, &mode> 模板参数,改为调用默认模式的 Reg::Div,同时删除文件内的 mode 常量定义,使带偏置与不带偏置两条路径的 Sigmoid 结果与 aclnnSigmoid 舍入行为对齐。
  • E/K 全载分支同步调整: 在 moe_gating_top_k_e_k_fullload_regbase.h 的 ComputeX 中,删除常量 EK_DIV_MODE 并将 Sigmoid 除法改为默认 Reg::Div,使全载场景的归一化输出口径与其他分支一致。
  • 无分组分支同步调整: 在 moe_gating_top_k_without_group_regbase.h 中删除常量 WG_DIV_MODE,ComputeNormSigmoid 与 ProcessSingleExpertSigmoid(单专家回退路径)的 Sigmoid 除法均改为默认 Reg::Div,保证无分组实现各路径与其他实现数值一致。

🧩 本次 Review 已使用自定义 Skill:

likedislike
Incorrect?
CANN-robotCANN-robotCollaborators
10 days ago Added label: cann-cla/yes
CANN-robot
CANN-robotCollaborators
10 days ago Comments:

Thanks for your pull-request.
The full list of commands accepted by me can be found at here.
You can get sig-info at here.
You can self-configure the PR merge rules for this repository. For more details, please refer to Here.
For more, you also can visit HICANN.


PR Approval Progress

✅ Congratulations! All modules have met the lgtm and approve requirements.

Module Approval Details

module lgtm status approve status
moe ✅ cpy_123456, guoqiuhao (2/2) ✅ cpy_123456 (1/1)

💡 Tip:

  • Committer can comment /approve or /lgtm
  • Commenting /approve implies both code review (lgtm) and intent to merge (approve)

CLA Signature Pass

ZhouChuping, thanks for your pull request. All authors of the commits have signed the CLA. 👍

likedislike
CANN-robotCANN-robotCollaborators
10 days ago Set cpy_123456,cheng_zhiwei,juyangokok,monologue815,chen-linxin4,yu-xinjie62,jiang-lirui,Allan_Yu,songkai111,liudan12,Wei_NaChuan,cc-z,kdy18482276080,xuchengyang,mabing1118,guoqiuhao,Julius47 as approver
CANN-robotCANN-robotCollaborators
10 days ago Set cpy_123456,cheng_zhiwei,juyangokok,monologue815,chen-linxin4,yu-xinjie62,jiang-lirui,Allan_Yu,songkai111,liudan12,Wei_NaChuan,cc-z,kdy18482276080,xuchengyang,mabing1118,guoqiuhao,Julius47 as approver
CPZhou
CPZhouCollaborators
10 days ago Comments:

/compile

likedislike
CPZhouCPZhouCollaborators
10 days ago Pre merge succeeded (commit_id: b95b4d3599b443e09627dc7fb5173c73b651b722)
CANN-robot
CANN-robotCollaborators
10 days ago Comments:
🚀 CI 流水线已启动
📋 执行详情: 点击查看流水线
likedislike
CANN-robotCANN-robotCollaborators
10 days ago Added label: ci-pipeline-running
CANN-robotCANN-robotCollaborators
10 days ago Removed label: ci-pipeline-running
CANN-robotCANN-robotCollaborators
10 days ago Added label: ci-pipeline-passed
CPZhouCPZhouCollaborators
10 days ago Modify description of pull request
GuoqhCollaborators
9 days ago Comments:

/lgtm

likedislike
cpy_123456Collaborators9 days agoStart code review2
moe/moe_gating_top_k/op_kernel/arch35/moe_gating_top_k_e_k_fullload_regbase.h
@@ -164,3 +162,3 @@
164162 Reg::Exp(vreg2, vreg1, preg0);
165163 Reg::Adds(vreg3, vreg2, static_cast<float>(1), preg0);
166- Reg::Div<float, &EK_DIV_MODE>(vregSigmoidResult, vregOne, vreg3, preg0);
164+ Reg::Div(vregSigmoidResult, vregOne, vreg3, preg0);
cpy_1234569 days agoComments:

其余gatingTopK & v2算子是否涉及

likedislike
CPZhou
CPZhouCollaborators
9 days ago Comments:

moeGatingTopkSoftmax moeGatingTopkSoftmaxV2均不涉及

cpy_123456Collaborators
9 days ago Comments:

/approve
/lgtm

likedislike
CANN-robotCANN-robotCollaborators
9 days ago Added label: lgtmapproved
CANN-robot
CANN-robotCollaborators
9 days ago Comments:

The MR can not be merged, because of CodeReview discussion not resolved

If you want to solve this problem, you can click here to do it in the FAQs.

likedislike
CPZhouCPZhouCollaborators
9 days ago Resolved final issue
CANN-robotCANN-robotCollaborators
9 days ago Closed associated issue
CANN-robotCANN-robotCollaborators
9 days ago Merged pull request