[Bug]: shmeme仓dispatch_gmm_combine算子发生竞争误报-MindStudio-Sanitizer-AtomGit

已关闭

[Bug]: shmeme仓dispatch_gmm_combine算子发生竞争误报 #125

duandingshan创建于 6月9日关闭于 6月17日

duandingshan成员

6月9日创建

在提交新问题之前，请确保您已经在社区中搜索过相关问题，并使用了社区中提供的资源/工具后，仍未找到满意的解决方式。

⚠️ 安全信息提醒：请仔细检查提供的文本内容，确保其不包含敏感数据信息，包括但不限于：

API 令牌或密钥
密码或身份验证凭证
私有网址或接口地址
个人或机密数据
...

在分享配置信息或代码示例时，请将敏感信息脱敏处理，或使用 <TOKEN> 等占位符替代原有内容。

环境信息

例如：
- 操作系统
- 昇腾硬件信息
- CANN软件版本
- 安装的对应软件版本

🐛 问题描述

【问题描述】
shmeme仓dispatch_gmm_combine算子发生竞争误报
【环境信息】
ubuntu
【重现步骤】
1、source cann包
2、克隆仓git clone --depth 1 -b release/v1.3.0 https://gitcode.com/cann/shmem
3、mssanitizer -t racecheck ${可执行文件}
【预期结果】
算子展示正确，调用栈展示正确
【实际结果】
未展示调用栈，且异常kernel名字展示错误

欢迎加入社区，感谢您对社区的贡献 🎉!

D duandingshan成员

6月9日关联了里程碑：MindStudio 26.1.0

D duandingshan成员

6月9日添加了label：bug

went_code成员

6月9日评论：

👋 您好，欢迎向 MindStudio-Sanitizer 提交 Issue！
我们已收到您的反馈，感谢你对开源社区的支持。🎉

📅 处理时效 维护团队将在工作日 24 小时内查看并回复您的问题。
🔍 自助排查（推荐优先查看） 在等待回复期间，您可以先查阅以下资源，多数问题可快速解决：

MindStudio-Sanitizer官方文档
历史 Issue 中相似问题的解决方案

💡 **为了更快定位问题，请您确保 Issue 包含：

清晰的问题描述
可复现的操作步骤
相关日志、截图或环境信息
我们会尽快跟进，感谢您的理解与配合！

Goldfish_of_Siris成员

6月9日将 cycoe 设为负责人

Goldfish_of_Siris成员

6月9日添加了label：triaged

C cycoe成员

6月9日关联了pull request：[bugfix] 修复多个进程使用一张卡时算子信息上下文管理出错的问题

C cycoe成员

6月9日关联了pull request：[sync][bugfix] 修复多个进程使用一张卡时算子信息上下文管理出错的问题

cycoe成员

6月9日评论：

问题已收到，修复中

cycoe成员

6月15日评论：

问题已修复

duandingshan成员

6月17日评论：

验证通过：
r*, unsigned char*, unsigned char*, unsigned char*, unsigned char*, unsigned char*, unsigned char*, CoCTiling, long, optiling::MoeInitRoutingQuantV2TilingData, unsigned char*)"
[mssanitizer] Start racecheck sanitizer on kernel "DispatchGMM(unsigned long, Catlass::GemmCoord, unsigned char*, unsigned char*, unsigned char*, unsigned char*, unsigned char*, unsigned char*, unsigned char*, unsigned char*, unsigned char*, unsigned char*, unsigned char*, unsigned char*, unsigned char*, CoCTiling, long, optiling::MoeInitRoutingQuantV2TilingData, unsigned char*)"

test finished
[SUCCESS] demo run success in pe_id 1
[SUCCESS] demo run success in pe_id 0
====== ERROR: Potential WAR hazard detected at UB in "DispatchGMM(unsigned long, Catlass::GemmCoord, unsigned char*, unsigned char*, unsigned char*, unsigned char*, unsigned char*, unsigned char*, unsigned char*, unsigned char*, unsigned char*, unsigned char*, unsigned char*, unsigned char*, unsigned char*, CoCTiling, long, optiling::MoeInitRoutingQuantV2TilingData, unsigned char*)":
====== PIPE_V Read at WAR()+0x12820 in block 0 (aiv) on device 1 at pc current 0x262e0c (serialNo:296)
====== #0 /home/dds/B020/Ascend/cann-8.5.T109/compiler/tikcpp/tikcfw/interface/../../impl/basic_api/dav_c220/kernel_operator_vec_reduce_impl.h:283:5
====== #1 /home/dds/B020/Ascend/cann-8.5.T109/compiler/tikcpp/tikcfw/interface/../../impl/basic_api/dav_c220/kernel_operator_vec_reduce_impl.h:380:13
====== #2 /home/dds/B020/Ascend/cann-8.5.T109/compiler/tikcpp/tikcfw/interface/../../impl/basic_api/dav_c220/kernel_operator_vec_reduce_impl.h:702:9
====== #3 /home/dds/B020/Ascend/cann-8.5.T109/compiler/tikcpp/tikcfw/interface/../../impl/basic_api/dav_c220/kernel_operator_vec_reduce_impl.h:784:17
====== #4 /home/dds/B020/Ascend/cann-8.5.T109/compiler/tikcpp/tikcfw/interface/../../impl/basic_api/dav_c220/kernel_operator_vec_reduce_impl.h:809:9
====== #5 /home/dds/B020/Ascend/cann-8.5.T109/compiler/tikcpp/tikcfw/interface/../../impl/basic_api/kernel_operator_vec_reduce_intf_impl.h:923:5
====== #6 /home/dds/shmem/examples/dispatch_gmm_combine/include/moe_init_routing_quant_v2/moe_v2_fullload_dynamic_quant.h:262:5
====== #7 /home/dds/shmem/examples/dispatch_gmm_combine/include/moe_init_routing_quant_v2/moe_v2_fullload_dynamic_quant.h:316:9
====== #8 /home/dds/shmem/examples/dispatch_gmm_combine/include/moe_init_routing_quant_v2/moe_v2_fullload_dynamic_quant.h:485:13
====== #9 /home/dds/shmem/examples/dispatch_gmm_combine/include/moe_init_routing_quant_v2/moe_init_routing_quant_v2.h:62:12
====== #10 /home/dds/shmem/examples/dispatch_gmm_combine/include/dispatch_gmm_combine.h:608:9
====== #11 /home/dds/shmem/examples/dispatch_gmm_combine/include/dispatch_gmm_combine.h:239:9
====== #12 /home/dds/shmem/examples/dispatch_gmm_combine/main.cpp:257:9
====== #13 /home/dds/shmem/examples/dispatch_gmm_combine/main.cpp:275:8
====== PIPE_V Write at WAR()+0x12838 in block 0 (aiv) on device 1 at pc current 0x262f2c (serialNo:297)
====== #0 /home/dds/B020/Ascend/cann-8.5.T109/compiler/tikcpp/tikcfw/interface/../../impl/basic_api/dav_c220/kernel_operator_vec_reduce_impl.h:283:5
====== #1 /home/dds/B020/Ascend/cann-8.5.T109/compiler/tikcpp/tikcfw/interface/../../impl/basic_api/dav_c220/kernel_operator_vec_reduce_impl.h:380:13
====== #2 /home/dds/B020/Ascend/cann-8.5.T109/compiler/tikcpp/tikcfw/interface/../../impl/basic_api/dav_c220/kernel_operator_vec_reduce_impl.h:708:9
====== #3 /home/dds/B020/Ascend/cann-8.5.T109/compiler/tikcpp/tikcfw/interface/../../impl/basic_api/dav_c220/kernel_operator_vec_reduce_impl.h:784:17
====== #4 /home/dds/B020/Ascend/cann-8.5.T109/compiler/tikcpp/tikcfw/interface/../../impl/basic_api/dav_c220/kernel_operator_vec_reduce_impl.h:809:9
====== #5 /home/dds/B020/Ascend/cann-8.5.T109/compiler/tikcpp/tikcfw/interface/../../impl/basic_api/kernel_operator_vec_reduce_intf_impl.h:923:5
====== #6 /home/dds/shmem/examples/dispatch_gmm_combine/include/moe_init_routing_quant_v2/moe_v2_fullload_dynamic_quant.h:262:5
====== #7 /home/dds/shmem/examples/dispatch_gmm_combine/include/moe_init_routing_quant_v2/moe_v2_fullload_dynamic_quant.h:316:9
====== #8 /home/dds/shmem/examples/dispatch_gmm_combine/include/moe_init_routing_quant_v2/moe_v2_fullload_dynamic_quant.h:485:13
====== #9 /home/dds/shmem/examples/dispatch_gmm_combine/include/moe_init_routing_quant_v2/moe_init_routing_quant_v2.h:62:12
====== #10 /home/dds/shmem/examples/dispatch_gmm_combine/include/dispatch_gmm_combine.h:608:9
====== #11 /home/dds/shmem/examples/dispatch_gmm_combine/include/dispatch_gmm_combine.h:239:9
====== #12 /home/dds/shmem/examples/dispatch_gmm_combine/main.cpp:257:9
====== #13 /home/dds/shmem/examples/dispatch_gmm_combine/main.cpp:275:8

D duandingshan成员

6月17日 issue状态由 TODO 改变为 DONE

D duandingshan成员

6月17日关闭了 issue

ascend-robot成员

6月17日添加了label：resolved