已关闭
稀疏卷积报错,算子索引支持问题 #82
gcw_nPQeAUjL创建于  2025年11月28日关闭于  2025年11月28日
gcw_nPQeAUjL
2025年11月28日 创建

代码内部:
conv=SubMConv3d(
in_channels,
embed_channels,
kernel_size=5,
padding=1,
bias=False,
indice_key="stem",
)
sparse_conv_feat = conv(input.sparse_conv_feat)
代码运行:
[W1128 10:30:15.161753880 compiler_depend.ts:487] Warning: NPU warning, error code is 507035[Error]:
[Error]: The vector core execution is abnormal.
Rectify the fault based on the error information in the ascend log.
EZ9999: Inner Error!
EZ9999: [PID: 275560] 2025-11-28-10:30:15.627.092 The error from device(chipId:6, dieId:0), serial number is 3, there is an aivec error exception, core id is 0, error code = 0, dump info: pc start: 0x12c0c0168000, current: 0x12c0c016b800, vec error info: 0x211750af04, mte error info: 0xd7f0e9027f, ifu error info: 0x2f3205e0fe7c0, ccu error info: 0x11a91975568000fe, cube error info: 0, biu error info: 0, aic error mask: 0x6500020bd00028c, para base: 0x12c100441400.[FUNC:ProcessStarsCoreErrorInfo][FILE:device_error_proc.cc][LINE:1434]
TraceBack (most recent call last):
The extend info: errcode:(0, 0x4000, 0) errorStr: CCU instruction address check error. fixp_error0 info: 0xe9027f, fixp_error1 info: 0xd7, fsmId:0, tslot:1, thread:0, ctxid:0, blk:3, sublk:0, subErrType:4.[FUNC:ProcessStarsCoreErrorInfo][FILE:device_error_proc.cc][LINE:1446]
The error from device(chipId:6, dieId:0), serial number is 3, there is an aivec error exception, core id is 1, error code = 0, dump info: pc start: 0x12c0c0168000, current: 0x12c0c016b800, vec error info: 0x5213504756, mte error info: 0x67e09392e, ifu error info: 0x4bf6717928900, ccu error info: 0x339a851f568000fe, cube error info: 0, biu error info: 0, aic error mask: 0x6500020bd00028c, para base: 0x12c100441400.[FUNC:ProcessStarsCoreErrorInfo][FILE:device_error_proc.cc][LINE:1434]
The extend info: errcode:(0, 0x4000, 0) errorStr: CCU instruction address check error. fixp_error0 info: 0xe09392e, fixp_error1 info: 0x6, fsmId:0, tslot:1, thread:0, ctxid:0, blk:4, sublk:0, subErrType:4.[FUNC:ProcessStarsCoreErrorInfo][FILE:device_error_proc.cc][LINE:1446]
The error from device(chipId:6, dieId:0), serial number is 3, there is an aivec error exception, core id is 2, error code = 0, dump info: pc start: 0x12c0c0168000, current: 0x12c0c016b800, vec error info: 0x2f0ba51345, mte error info: 0xacbfbff874, ifu error info: 0x15a42072e4c80, ccu error info: 0x96718009568000fe, cube error info: 0, biu error info: 0, aic error mask: 0x6500020bd00028c, para base: 0x12c100441400.[FUNC:ProcessStarsCoreErrorInfo][FILE:device_error_proc.cc][LINE:1434]
The extend info: errcode:(0, 0x4000, 0) errorStr: CCU instruction address check error. fixp_error0 info: 0xfbff874, fixp_error1 info: 0xac, fsmId:0, tslot:1, thread:0, ctxid:0, blk:5, sublk:0, subErrType:4.[FUNC:ProcessStarsCoreErrorInfo][FILE:device_error_proc.cc][LINE:1446]...

疑问:
是否不支持“stem”索引

likedislike
HHHyyyjjjjj成员
2025年11月28日 评论:

请提供下算子输入的tensor shape,和使用的mx_driving版本

likedislike
HHHyyyjjjjj成员
2025年11月28日 评论:

仅支持输入和输出featuremap相同,k=5,padding必须等于2,k=3,padding必须等于1

likedislike
gcw_nPQeAUjL
2025年11月28日 评论:

驱动:25.0.rc1.1
cann:8.1.RC1
torch:2.4.0
torch_npu: 2.4.0.post2

in_channels = 512
embed_channels = 512
输入tensor shape, (512, 512)

将padding修改为2后,还是会报这个错:
[Error]: The vector core execution is abnormal.
Rectify the fault based on the error information in the ascend log.
EZ9999: Inner Error!
EZ9999: [PID: 651183] 2025-11-28-11:21:18.664.502 The error from device(chipId:6, dieId:0), serial number is 6, there is an aivec error exception, core id is 13, error code = 0, dump info: pc start: 0x12c0c0168000, current: 0x12c0c016b800, vec error info: 0x610bc27744, mte error info: 0xba5fc3008f, ifu error info: 0x6cf72ffe3c340, ccu error info: 0xd66510bc568000fe, cube error info: 0, biu error info: 0, aic error mask: 0x6500020bd00028c, para base: 0x12c100441400.[FUNC:ProcessStarsCoreErrorInfo][FILE:device_error_proc.cc][LINE:1434]
TraceBack (most recent call last):
The extend info: errcode:(0, 0x4000, 0) errorStr: CCU instruction address check error. fixp_error0 info: 0xfc3008f, fixp_error1 info: 0xba, fsmId:0, tslot:2, thread:0, ctxid:0, blk:0, sublk:0, subErrType:4.[FUNC:ProcessStarsCoreErrorInfo][FILE:device_error_proc.cc][LINE:1446]
The error from device(chipId:6, dieId:0), serial number is 6, there is an aivec error exception, core id is 14, error code = 0, dump info: pc start: 0x12c0c0168000, current: 0x12c0c016b800, vec error info: 0x6e16d7061c, mte error info: 0xc3fbf5999f, ifu error info: 0x65c63fab20380, ccu error info: 0x31ecfd4d568000fe, cube error info: 0, biu error info: 0, aic error mask: 0x6500020bd00028c, para base: 0x12c100441400.[FUNC:ProcessStarsCoreErrorInfo][FILE:device_error_proc.cc][LINE:1434]
The extend info: errcode:(0, 0x4000, 0) errorStr: CCU instruction address check error. fixp_error0 info: 0xbf5999f, fixp_error1 info: 0xc3, fsmId:0, tslot:2, thread:0, ctxid:0, blk:1, sublk:0, subErrType:4.[FUNC:ProcessStarsCoreErrorInfo][FILE:device_error_proc.cc][LINE:1446]
The error from device(chipId:6, dieId:0), serial number is 6, there is an aivec error exception, core id is 15, error code = 0, dump info: pc start: 0x12c0c0168000, current: 0x12c0c016b800, vec error info: 0xb1120520ce, mte error info: 0x7ebc440177, ifu error info: 0x6dfc21ebc3f00, ccu error info: 0xe362043f568000fe, cube error info: 0, biu error info: 0, aic error mask: 0x6500020bd00028c, para base: 0x12c100441400.[FUNC:ProcessStarsCoreErrorInfo][FILE:device_error_proc.cc][LINE:1434]
The extend info: errcode:(0, 0x4000, 0) errorStr: CCU instruction address check error. fixp_error0 info: 0xc440177, fixp_error1 info: 0x7e, fsmId:0, tslot:2, thread:0, ctxid:0, blk:2, sublk:0, subErrType:4.[FUNC:ProcessStarsCoreErrorInfo][FILE:device_error_proc.cc][LINE:1446]
The error from device(chipId:6, dieId:0), serial number is 6, there is an aivec error exception, core id is 21, error code = 0, dump info: pc start: 0x12c0c0168000, current: 0x12c0c016b800, vec error info: 0x61065fb186, mte error info: 0x1fc70f7fbc, ifu error info: 0x1837a858c6240, ccu error info: 0xfa610335568000fe, cube error info: 0, biu error info: 0, aic error mask: 0x6500020bd00028c, para base: 0x12c100441400.[FUNC:ProcessStarsCoreErrorInfo][FILE:device_error_proc.cc][LINE:1434]
The extend info: errcode:(0, 0x4000, 0) errorStr: CCU instruction address check error. fixp_error0 info: 0x70f7fbc, fixp_error1 info: 0x1f, fsmId:0, tslot:2, thread:0, ctxid:0, blk:4, sublk:0, subErrType:4.[FUNC:ProcessStarsCoreErrorInfo][FILE:device_error_proc.cc][LINE:1446]
The error from device(chipId:6, dieId:0), serial number is 6, there is an aivec error exception, core id is 20, error code = 0, dump info: pc start: 0x12c0c0168000, current: 0x12c0c016b800, vec error info: 0x2115457700, mte error info: 0xb0792fabff, ifu error info: 0x45bd80003bcc0, ccu error info: 0x90680f7f568000fe, cube error info: 0, biu error info: 0, aic error mask: 0x6500020bd00028c, para base: 0x12c100441400.[FUNC:ProcessStarsCoreErrorInfo][FILE:device_error_proc.cc][LINE:1434]
The extend info: errcode:(0, 0x4000, 0) errorStr: CCU instruction address check error. fixp_error0 info: 0x92fabff, fixp_error1 info: 0xb0, fsmId:1, tslot:2, thread:0, ctxid:0, blk:3, sublk:0, subErrType:4.[FUNC:ProcessStarsCoreErrorInfo][FILE:device_error_proc.cc][LINE:1446]
The error from device(chipId:6, dieId:0), serial number is 6, there is an aivec error exception, core id is 22, error code = 0, dump info: pc start: 0x12c0c0168000, current: 0x12c0c016b800, vec error info: 0x52053816c1, mte error info: 0xb0f00e88eb, ifu error info: 0x4b93cbb0a25c0, ccu error info: 0x36008420568000fe, cube error info: 0, biu error info: 0, aic error mask: 0x6500020bd00028c, para base: 0x12c100441400.[FUNC:ProcessStarsCoreErrorInfo][FILE:device_error_proc.cc][LINE:1434]
The extend info: errcode:(0, 0x4000, 0) errorStr: CCU instruction address check error. fixp_error0 info: 0xe88eb, fixp_error1 info: 0xb0, fsmId:1, tslot:2, thread:0, ctxid:0, blk:5, sublk:0, subErrType:4.[FUNC:ProcessStarsCoreErrorInfo][FILE:device_error_proc.cc][LINE:1446]
The error from device(chipId:6, dieId:0), serial number is 6, there is an aivec error exception, core id is 23, error code = 0, dump info: pc start: 0x12c0c0168000, current: 0x12c0c016b800, vec error info: 0x450fb6cf6a, mte error info: 0x1f075fb2e9, ifu error info: 0x7960679bc3040, ccu error info: 0x733f8ec568000fe, cube error info: 0, biu error info: 0, aic error mask: 0x6500020bd00028c, para base: 0x12c100441400.[FUNC:ProcessStarsCoreErrorInfo][FILE:device_error_proc.cc][LINE:1434]
The extend info: errcode:(0, 0x4000, 0) errorStr: CCU instruction address check error. fixp_error0 info: 0x75fb2e9, fixp_error1 info: 0x1f, fsmId:1, tslot:2, thread:0, ctxid:0, blk:6, sublk:0, subErrType:4.[FUNC:ProcessStarsCoreErrorInfo][FILE:device_error_proc.cc][LINE:1446]
The error from device(chipId:6, dieId:0), serial number is 6, there is an aivec error exception, core id is 24, error code = 0, dump info: pc start: 0x12c0c0168000, current: 0x12c0c016ab00, vec error info: 0x310180fed1, mte error info: 0xbd7f3ffa37, ifu error info: 0x760573fbe4b40, ccu error info: 0x6b563da9550000bb, cube error info: 0, biu error info: 0, aic error mask: 0x6500020bd00028c, para base: 0x12c100441400.[FUNC:ProcessStarsCoreErrorInfo][FILE:device_error_proc.cc][LINE:1434]
The extend info: errcode:(0, 0x4000, 0) errorStr: CCU instruction address check error. fixp_error0 info: 0xf3ffa37, fixp_error1 info: 0xbd, fsmId:1, tslot:2, thread:0, ctxid:0, blk:7, sublk:0, subErrType:4.[FUNC:ProcessStarsCoreErrorInfo][FILE:device_error_proc.cc][LINE:1446]
Kernel task happen error, retCode=0x31, [vector core exception].[FUNC:PreCheckTaskErr][FILE:davinci_kernel_task.cc][LINE:1366]
AIV Kernel happen error, retCode=0x31.[FUNC:GetError][FILE:stream.cc][LINE:1119]
rtDeviceSynchronizeWithTimeout execute failed, reason=[vector core exception][FUNC:FuncErrorReason][FILE:error_message_manage.cc][LINE:53]
wait for compute device to finish failed, runtime result = 507035.[FUNC:ReportCallError][FILE:log_inner.cpp][LINE:161]
(function npuSynchronizeUsedDevices)
[W1128 11:21:18.202857690 compiler_depend.ts:122] Warning: NPU warning, error code is 507035[Error]:
[Error]: The vector core execution is abnormal.
Rectify the fault based on the error information in the ascend log.
EZ9999: Inner Error!
EZ9999: [PID: 651183] 2025-11-28-11:21:18.674.292 Aicore kernel execute failed, device_id=6, stream_id=2, report_stream_id=2, task_id=2309, flip_num=0, fault kernel_name=SubmSparseConv3d_12e0858d31008a5bdaff9c7d98885d55_0, fault kernel info ext=none, program id=77, hash=4468565224395071664.[FUNC:GetError][FILE:stream.cc][LINE:1119]
TraceBack (most recent call last):
[AIC_INFO] after execute:args print end[FUNC:GetError][FILE:stream.cc][LINE:1119]
rtDeviceSynchronizeWithTimeout execute failed, reason=[vector core exception][FUNC:FuncErrorReason][FILE:error_mess

likedislike
gcw_nPQeAUjL
2025年11月28日 评论:

拉取的sdk代码偏老,通过拉取最新的就能解决了

likedislike
Ggcw_nPQeAUjL
2025年11月28日 issue状态由 TODO 改变为 DONE
Ggcw_nPQeAUjL
2025年11月28日 关闭了 issue