Thanks for sending an issue! Please fill in the following template to help quickly solve your problem.

Describe the current behavior / 问题描述 (Mandatory / 必填)

1、v1.5.0版本，社区报，执行kv_shuffle用例coredumo

2、报错：
bash scripts/run.sh 2
terminate called after throwing an instance of 'std::__ios_failure'
what(): basic_filebuf::xsgetn error reading the file: Bad address
scripts/run.sh: line 34: 1947335 Aborted (core dumped) ${APP}

shmem报错：
2026-07-10 19:40:44.626741 info 1949900 pid[1949900] [SHM_SHMEM shmemi_bootstrap.cpp:281] ACLSHMEMX_INIT_WITH_DEFAULT
2026-07-10 19:40:44.626805 info 1949900 pid[1949900] [SHM_SHMEM shmemi_bootstrap.cpp:293] Default bootstrap will use attr ip_port directly: tcp://127.0.0.1:27010
2026-07-10 19:40:44.627253 info 1949900 pid[1949900] [SHM_SHMEM shmemi_bootstrap.cpp:328] Calling plugin_init for aclshmem_bootstrap_config_store.so, use_attr_ipport=1, mype=0, npes=2
2026-07-10 19:40:44.627270 info 1949900 pid[1949900] [SHM_SHMEM store_factory.cpp:215] tls is not enabled.
2026-07-10 19:40:44.627403 info 1949908 pid[1949900] [SHM_SHMEM acc_tcp_link_delay_cleanup.h:114] AccDelay cleanup thread thread started
2026-07-10 19:40:44.627535 info 1949909 pid[1949900] [SHM_SHMEM acc_tcp_worker.cpp:152] Worker [name AccWrk, index 0, cpu -1, thread-priority 0, poll-timeout-ms 500] progress thread started
2026-07-10 19:40:44.627645 info 1949910 pid[1949900] [SHM_SHMEM acc_tcp_worker.cpp:152] Worker [name AccWrk, index 1, cpu -1, thread-priority 0, poll-timeout-ms 500] progress thread started
2026-07-10 19:40:44.627683 info 1949900 pid[1949900] [SHM_SHMEM store_net_utils.h:86] getaddrinfo '127.0.0.1' candidate: IPv4 127.0.0.1
2026-07-10 19:40:44.627697 info 1949900 pid[1949900] [SHM_SHMEM acc_tcp_listener.cpp:126] create socket success tmp_fd = 13 listen fd = 0
2026-07-10 19:40:44.627711 info 1949900 pid[1949900] [SHM_SHMEM acc_tcp_listener.cpp:154] bind ip port success13
2026-07-10 19:40:44.627793 info 1949911 pid[1949900] [SHM_SHMEM acc_tcp_listener.cpp:230] Acc listener accept thread for 127.0.0.1:27010 start ...
2026-07-10 19:40:44.628059 info 1949914 pid[1949900] [SHM_SHMEM acc_tcp_link_delay_cleanup.h:114] AccDelay cleanup thread thread started
2026-07-10 19:40:44.628138 info 1949915 pid[1949900] [SHM_SHMEM acc_tcp_worker.cpp:152] Worker [name AccWrk, index 0, cpu -1, thread-priority 0, poll-timeout-ms 500] progress thread started
2026-07-10 19:40:44.628247 info 1949916 pid[1949900] [SHM_SHMEM acc_tcp_worker.cpp:152] Worker [name AccWrk, index 1, cpu -1, thread-priority 0, poll-timeout-ms 500] progress thread started
2026-07-10 19:40:44.628322 info 1949900 pid[1949900] [SHM_SHMEM acc_tcp_server_default.cpp:558] Trying to connect to 127.0.0.1:27010
2026-07-10 19:40:44.628431 info 1949911 pid[1949900] [SHM_SHMEM acc_tcp_listener.cpp:322] Connected from 127.0.0.1:57366 successfully, ssl disable
2026-07-10 19:40:44.628452 info 1949911 pid[1949900] [SHM_SHMEM store_tcp_config_server.cpp:173] new link connected, linkId: 1, rank: 0
2026-07-10 19:40:44.628489 info 1949900 pid[1949900] [SHM_SHMEM acc_tcp_server_default.cpp:737] Connect to 127.0.0.1:27010 successfully, with ssl disable
2026-07-10 19:40:44.628494 info 1949900 pid[1949900] [SHM_SHMEM shmemi_bootstrap_config_store.cpp:346] pe 0: bootstrap plugin initialized successfully
2026-07-10 19:40:44.628496 info 1949900 pid[1949900] [SHM_SHMEM shmemi_bootstrap.cpp:330] plugin_init returned status=0 for aclshmem_bootstrap_config_store.so
2026-07-10 19:40:44.628583 error 1949900 pid[1949900] [SHM_SHMEM shmem_init.cpp:143] return shmem error: 107002 - aclrtCreateStream(&stream) failed.
2026-07-10 19:40:44.628584 error 1949900 pid[1949900] [SHM_SHMEM shmem_init.cpp:489] return shmem error: 107002 - aclshmemi_state_init_attr(attributes) failed.

Environment / 环境信息 (Mandatory / 必填)

A5 PCIE

Steps to reproduce the issue / 重现步骤 (Mandatory / 必填)

如上

Describe the expected behavior / 预期结果 (Mandatory / 必填)

无报错

Related log / screenshot / 日志 / 截图 (Mandatory / 必填)

(llm_datadist_py39) root@huawei:/npu/mwx1103108/0708/log# grep -rsn ERROR /root/ascend/log/
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:19:[ERROR] ASCENDCL(1947336,kv_shuffle):2026-07-10-19:40:46.664.180 [memory.cpp:623]1947336 aclrtMemcpyImpl:[Check][dst]param must not be null.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:23:[ERROR] ASCENDCL(1947336,kv_shuffle):2026-07-10-19:40:46.666.536 [memory.cpp:623]1947336 aclrtMemcpyImpl:[Check][dst]param must not be null.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:34:[ERROR] ASCENDCL(1947336,kv_shuffle):2026-07-10-19:40:46.666.696 [memory.cpp:647]1947336 aclrtMemsetImpl:[Check][devPtr]param must not be null.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:192:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.669.734 [task_recycle_cqrpt_base.cc:68]1948149 PrintTaskErrorMsg:Task run failed, stream_id=61, pos=0, task_sn=2, sqe_type=1(aiv), errType=0x1(task exception), sqSwStatus=0.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:198:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.331 [device_error_proc_c.cc:630]1948149 ProcessCoreErrorClass:comm_flag=2
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:199:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.376 [device_error_proc_c.cc:597]1948149 SetDeviceFaultTypeByAixErrClass:hardware local error, stream_id=61, task_id=0, errorCode=0x701002d.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:200:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.383 [device_error_proc_c.cc:690]1948149 AddExceptionRegInfo:add error register: core_id=0, stream_id=61, task_sn=2
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:201:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.393 [device_error_proc_c.cc:741]1948149 ProcessDavidStarsCoreErrorInfo:The error from device(chipId:0, dieId:0), serial number is 155, there is an aivec error exception, core id is 0, error code = 264, dump info: pc start: 0x120041000000, current: 0x1200410000a0, sc error info: 0xffffffffffff, su error info: 0xbe01404007de0000,0x42401430180017ff, mte error info: 0xaffe555c0005df97, vec error info: 0x9f7ffb3b007ef6fe, cube error info: 0, l1 error info: 0, aic error mask: 0x395856, para base: 0x120000200000, mte error: 0.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:203:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.433 [device_error_proc_c.cc:744]1948149 ProcessDavidStarsCoreErrorInfo:The extend info: errcode:(264) errorStr: The address for scalar to access GM is invalid subErrType: 0x4.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:205:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.440 [device_error_proc_c.cc:690]1948149 AddExceptionRegInfo:add error register: core_id=1, stream_id=61, task_sn=2
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:206:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.442 [device_error_proc_c.cc:741]1948149 ProcessDavidStarsCoreErrorInfo:The error from device(chipId:0, dieId:0), serial number is 155, there is an aivec error exception, core id is 1, error code = 264, dump info: pc start: 0x120041000000, current: 0x1200410000a0, sc error info: 0xffffffffffff, su error info: 0xbe0140401aa80000,0xfcf1145f180017ff, mte error info: 0x9a9fed6f00037f75, vec error info: 0xfdb5c95d001ef76c, cube error info: 0, l1 error info: 0, aic error mask: 0x395856, para base: 0x120000200000, mte error: 0.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:208:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.451 [device_error_proc_c.cc:744]1948149 ProcessDavidStarsCoreErrorInfo:The extend info: errcode:(264) errorStr: The address for scalar to access GM is invalid subErrType: 0x4.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:210:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.456 [device_error_proc_c.cc:690]1948149 AddExceptionRegInfo:add error register: core_id=2, stream_id=61, task_sn=2
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:211:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.458 [device_error_proc_c.cc:741]1948149 ProcessDavidStarsCoreErrorInfo:The error from device(chipId:0, dieId:0), serial number is 155, there is an aivec error exception, core id is 2, error code = 264, dump info: pc start: 0x120041000000, current: 0x1200410000a0, sc error info: 0xffffffffffff, su error info: 0xbe0140403e3e0000,0xedd9e43d180017ff, mte error info: 0x26512, vec error info: 0x6eedf53d003cdefc, cube error info: 0, l1 error info: 0, aic error mask: 0x395856, para base: 0x120000200000, mte error: 0.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:213:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.464 [device_error_proc_c.cc:744]1948149 ProcessDavidStarsCoreErrorInfo:The extend info: errcode:(264) errorStr: The address for scalar to access GM is invalid subErrType: 0x4.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:215:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.468 [device_error_proc_c.cc:690]1948149 AddExceptionRegInfo:add error register: core_id=3, stream_id=61, task_sn=2
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:216:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.470 [device_error_proc_c.cc:741]1948149 ProcessDavidStarsCoreErrorInfo:The error from device(chipId:0, dieId:0), serial number is 155, there is an aivec error exception, core id is 3, error code = 264, dump info: pc start: 0x120041000000, current: 0x1200410000a0, sc error info: 0xffffffffffff, su error info: 0xbe0140403e9a0000,0x3fff7ced180017ff, mte error info: 0x26512, vec error info: 0x3feb9fde0050d9c7, cube error info: 0, l1 error info: 0, aic error mask: 0x395856, para base: 0x120000200000, mte error: 0.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:218:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.476 [device_error_proc_c.cc:744]1948149 ProcessDavidStarsCoreErrorInfo:The extend info: errcode:(264) errorStr: The address for scalar to access GM is invalid subErrType: 0x4.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:220:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.480 [device_error_proc_c.cc:690]1948149 AddExceptionRegInfo:add error register: core_id=4, stream_id=61, task_sn=2
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:221:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.482 [device_error_proc_c.cc:741]1948149 ProcessDavidStarsCoreErrorInfo:The error from device(chipId:0, dieId:0), serial number is 155, there is an aivec error exception, core id is 4, error code = 264, dump info: pc start: 0x120041000000, current: 0x1200410000a0, sc error info: 0xffffffffffff, su error info: 0xbe0140403b380000,0xddffcd5c180017ff, mte error info: 0x26512, vec error info: 0x379fa9f70077d075, cube error info: 0, l1 error info: 0, aic error mask: 0x395856, para base: 0x120000200000, mte error: 0.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:223:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.489 [device_error_proc_c.cc:744]1948149 ProcessDavidStarsCoreErrorInfo:The extend info: errcode:(264) errorStr: The address for scalar to access GM is invalid subErrType: 0x4.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:225:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.494 [device_error_proc_c.cc:690]1948149 AddExceptionRegInfo:add error register: core_id=5, stream_id=61, task_sn=2
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:226:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.495 [device_error_proc_c.cc:741]1948149 ProcessDavidStarsCoreErrorInfo:The error from device(chipId:0, dieId:0), serial number is 155, there is an aivec error exception, core id is 5, error code = 264, dump info: pc start: 0x120041000000, current: 0x1200410000a0, sc error info: 0xffffffffffff, su error info: 0xbe0140402a0c0000,0xc565c4ff180017ff, mte error info: 0x26512, vec error info: 0x2370efdf0072fe4f, cube error info: 0, l1 error info: 0, aic error mask: 0x395856, para base: 0x120000200000, mte error: 0.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:228:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.501 [device_error_proc_c.cc:744]1948149 ProcessDavidStarsCoreErrorInfo:The extend info: errcode:(264) errorStr: The address for scalar to access GM is invalid subErrType: 0x4.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:230:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.505 [device_error_proc_c.cc:690]1948149 AddExceptionRegInfo:add error register: core_id=6, stream_id=61, task_sn=2
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:231:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.507 [device_error_proc_c.cc:741]1948149 ProcessDavidStarsCoreErrorInfo:The error from device(chipId:0, dieId:0), serial number is 155, there is an aivec error exception, core id is 6, error code = 264, dump info: pc start: 0x120041000000, current: 0x1200410000a0, sc error info: 0xffffffffffff, su error info: 0xbe0140402bec0000,0xfc7fecf5180017ff, mte error info: 0x26512, vec error info: 0xf570c2f7007f5fbd, cube error info: 0, l1 error info: 0, aic error mask: 0x395856, para base: 0x120000200000, mte error: 0.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:233:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.512 [device_error_proc_c.cc:744]1948149 ProcessDavidStarsCoreErrorInfo:The extend info: errcode:(264) errorStr: The address for scalar to access GM is invalid subErrType: 0x4.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:235:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.517 [device_error_proc_c.cc:690]1948149 AddExceptionRegInfo:add error register: core_id=7, stream_id=61, task_sn=2
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:236:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.518 [device_error_proc_c.cc:741]1948149 ProcessDavidStarsCoreErrorInfo:The error from device(chipId:0, dieId:0), serial number is 155, there is an aivec error exception, core id is 7, error code = 264, dump info: pc start: 0x120041000000, current: 0x1200410000a0, sc error info: 0xffffffffffff, su error info: 0xbe0140403b180000,0xfff1d7d1180017ff, mte error info: 0x26512, vec error info: 0xffb7e6cf001f1ff7, cube error info: 0, l1 error info: 0, aic error mask: 0x395856, para base: 0x120000200000, mte error: 0.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:238:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.525 [device_error_proc_c.cc:744]1948149 ProcessDavidStarsCoreErrorInfo:The extend info: errcode:(264) errorStr: The address for scalar to access GM is invalid subErrType: 0x4.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:240:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.530 [device_error_proc_c.cc:690]1948149 AddExceptionRegInfo:add error register: core_id=36, stream_id=61, task_sn=2
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:241:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.532 [device_error_proc_c.cc:741]1948149 ProcessDavidStarsCoreErrorInfo:The error from device(chipId:0, dieId:0), serial number is 155, there is an aivec error exception, core id is 36, error code = 264, dump info: pc start: 0x120041000000, current: 0x1200410000a0, sc error info: 0xffffffffffff, su error info: 0xbe01404004fc0000,0x541fc7bd180017ff, mte error info: 0xe84fedff00053f7b, vec error info: 0x7fff95ff00743fdf, cube error info: 0, l1 error info: 0, aic error mask: 0x395856, para base: 0x120000200000, mte error: 0.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:243:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.538 [device_error_proc_c.cc:744]1948149 ProcessDavidStarsCoreErrorInfo:The extend info: errcode:(264) errorStr: The address for scalar to access GM is invalid subErrType: 0x4.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:245:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.542 [device_error_proc_c.cc:690]1948149 AddExceptionRegInfo:add error register: core_id=37, stream_id=61, task_sn=2
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:246:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.544 [device_error_proc_c.cc:741]1948149 ProcessDavidStarsCoreErrorInfo:The error from device(chipId:0, dieId:0), serial number is 155, there is an aivec error exception, core id is 37, error code = 264, dump info: pc start: 0x120041000000, current: 0x1200410000a0, sc error info: 0xffffffffffff, su error info: 0xbe01404037780000,0x6b4372f5180017ff, mte error info: 0xfeef5ec30007ebf6, vec error info: 0x3b374d6e004f749f, cube error info: 0, l1 error info: 0, aic error mask: 0x395856, para base: 0x120000200000, mte error: 0.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:248:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.549 [device_error_proc_c.cc:744]1948149 ProcessDavidStarsCoreErrorInfo:The extend info: errcode:(264) errorStr: The address for scalar to access GM is invalid subErrType: 0x4.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:250:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.554 [device_error_proc_c.cc:690]1948149 AddExceptionRegInfo:add error register: core_id=38, stream_id=61, task_sn=2
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:251:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.556 [device_error_proc_c.cc:741]1948149 ProcessDavidStarsCoreErrorInfo:The error from device(chipId:0, dieId:0), serial number is 155, there is an aivec error exception, core id is 38, error code = 264, dump info: pc start: 0x120041000000, current: 0x1200410000a0, sc error info: 0xffffffffffff, su error info: 0xbe0140403bfe0000,0x7708f46d180017ff, mte error info: 0xb1f39ef50002ffff, vec error info: 0x37f7fffe0035bddf, cube error info: 0, l1 error info: 0, aic error mask: 0x395856, para base: 0x120000200000, mte error: 0.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:253:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.562 [device_error_proc_c.cc:744]1948149 ProcessDavidStarsCoreErrorInfo:The extend info: errcode:(264) errorStr: The address for scalar to access GM is invalid subErrType: 0x4.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:255:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.567 [device_error_proc_c.cc:690]1948149 AddExceptionRegInfo:add error register: core_id=39, stream_id=61, task_sn=2
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:256:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.569 [device_error_proc_c.cc:741]1948149 ProcessDavidStarsCoreErrorInfo:The error from device(chipId:0, dieId:0), serial number is 155, there is an aivec error exception, core id is 39, error code = 264, dump info: pc start: 0x120041000000, current: 0x1200410000a0, sc error info: 0xffffffffffff, su error info: 0xbe0140401ffc0000,0x13cfe631180017ff, mte error info: 0xfea1bdc20005defd, vec error info: 0xff15dffd006df168, cube error info: 0, l1 error info: 0, aic error mask: 0x395856, para base: 0x120000200000, mte error: 0.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:258:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.574 [device_error_proc_c.cc:744]1948149 ProcessDavidStarsCoreErrorInfo:The extend info: errcode:(264) errorStr: The address for scalar to access GM is invalid subErrType: 0x4.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:260:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.578 [device_error_proc_c.cc:690]1948149 AddExceptionRegInfo:add error register: core_id=40, stream_id=61, task_sn=2
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:261:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.580 [device_error_proc_c.cc:741]1948149 ProcessDavidStarsCoreErrorInfo:The error from device(chipId:0, dieId:0), serial number is 155, there is an aivec error exception, core id is 40, error code = 264, dump info: pc start: 0x120041000000, current: 0x1200410000a0, sc error info: 0xffffffffffff, su error info: 0xbe01404021be0000,0xed1ff87a180017ff, mte error info: 0x1ef635fe000795be, vec error info: 0x7535cfdd0000eefd, cube error info: 0, l1 error info: 0, aic error mask: 0x395856, para base: 0x120000200000, mte error: 0.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:263:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.586 [device_error_proc_c.cc:744]1948149 ProcessDavidStarsCoreErrorInfo:The extend info: errcode:(264) errorStr: The address for scalar to access GM is invalid subErrType: 0x4.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:265:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.590 [device_error_proc_c.cc:690]1948149 AddExceptionRegInfo:add error register: core_id=41, stream_id=61, task_sn=2
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:266:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.592 [device_error_proc_c.cc:741]1948149 ProcessDavidStarsCoreErrorInfo:The error from device(chipId:0, dieId:0), serial number is 155, there is an aivec error exception, core id is 41, error code = 264, dump info: pc start: 0x120041000000, current: 0x1200410000a0, sc error info: 0xffffffffffff, su error info: 0xbe01404037c00000,0x1e54035b180017ff, mte error info: 0xb4e1017e0004fbbb, vec error info: 0xd67ebb7c007cff57, cube error info: 0, l1 error info: 0, aic error mask: 0x395856, para base: 0x120000200000, mte error: 0.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:268:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.598 [device_error_proc_c.cc:744]1948149 ProcessDavidStarsCoreErrorInfo:The extend info: errcode:(264) errorStr: The address for scalar to access GM is invalid subErrType: 0x4.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:270:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.603 [device_error_proc_c.cc:690]1948149 AddExceptionRegInfo:add error register: core_id=42, stream_id=61, task_sn=2
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:271:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.605 [device_error_proc_c.cc:741]1948149 ProcessDavidStarsCoreErrorInfo:The error from device(chipId:0, dieId:0), serial number is 155, there is an aivec error exception, core id is 42, error code = 264, dump info: pc start: 0x120041000000, current: 0x1200410000a0, sc error info: 0xffffffffffff, su error info: 0xbe0140401f760000,0xdfeffcde180017ff, mte error info: 0x3e98fdf80007f8ee, vec error info: 0xefd9f9c003dfffd, cube error info: 0, l1 error info: 0, aic error mask: 0x395856, para base: 0x120000200000, mte error: 0.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:273:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.610 [device_error_proc_c.cc:744]1948149 ProcessDavidStarsCoreErrorInfo:The extend info: errcode:(264) errorStr: The address for scalar to access GM is invalid subErrType: 0x4.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:275:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.615 [device_error_proc_c.cc:690]1948149 AddExceptionRegInfo:add error register: core_id=43, stream_id=61, task_sn=2
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:276:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.616 [device_error_proc_c.cc:741]1948149 ProcessDavidStarsCoreErrorInfo:The error from device(chipId:0, dieId:0), serial number is 155, there is an aivec error exception, core id is 43, error code = 264, dump info: pc start: 0x120041000000, current: 0x1200410000a0, sc error info: 0xffffffffffff, su error info: 0xbe014040037c0000,0xd97ff76d180017ff, mte error info: 0xb7e29ff80005f3fd, vec error info: 0xc2323ebd0079df88, cube error info: 0, l1 error info: 0, aic error mask: 0x395856, para base: 0x120000200000, mte error: 0.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:278:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.622 [device_error_proc_c.cc:744]1948149 ProcessDavidStarsCoreErrorInfo:The extend info: errcode:(264) errorStr: The address for scalar to access GM is invalid subErrType: 0x4.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:283:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.645 [davinci_kernel_task_v200_base.cc:257]1948149 StarsV2SetStarsResultForDavinciTask:AIV Kernel happen error, retCode=0x31.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:284:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.649 [davinci_kernel_task.cc:1264]1948149 PreCheckTaskErr:An error occurred in the kernel task, retCode=0x31, [vector core exception].
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:287:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.702 [davinci_kernel_task.cc:1078]1948149 GetArgsInfo:[AIC_INFO] args(0 to 10) after execute:0, 0, 0, 0x120000016000, 0x7c62d4270128, 0x200, 0, 0, 0x8, 0x80, 0x80.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:288:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.706 [davinci_kernel_task.cc:1082]1948149 GetArgsInfo:tilingKey = 0, print 1 Times totalLen=(11*8), argsSize=92, schemMode=0, blockDim=16
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:289:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.683.711 [davinci_kernel_task.cc:1182]1948149 PrintErrorInfoForDavinciTask:[AIC_INFO] after execute:args print end
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:293:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.684.733 [davinci_kernel_task.cc:1212]1948149 PrintErrorInfoForDavinciTask:[DFX_INFO]AI Core kernel execution failed, device_id=1, stream_id=61, report_stream_id=61, task_id=0, flip_num=0, fault kernel_name=ShmemKVShuffle, fault kernel info ext=none, program id=0, hash=15171196078140488690.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:316:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.685.084 [task_recycle_cqrpt_base.cc:68]1948149 PrintTaskErrorMsg:Task run failed, stream_id=61, pos=1, task_sn=3, sqe_type=1(aiv), errType=0x1(task exception), sqSwStatus=0.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:323:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.700.630 [device_error_proc_c.cc:630]1948149 ProcessCoreErrorClass:comm_flag=2
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:324:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.700.652 [device_error_proc_c.cc:597]1948149 SetDeviceFaultTypeByAixErrClass:hardware local error, stream_id=61, task_id=1, errorCode=0x701002d.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:325:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.700.655 [device_error_proc_c.cc:690]1948149 AddExceptionRegInfo:add error register: core_id=0, stream_id=61, task_sn=3
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:326:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.700.659 [device_error_proc_c.cc:741]1948149 ProcessDavidStarsCoreErrorInfo:The error from device(chipId:0, dieId:0), serial number is 156, there is an aivec error exception, core id is 0, error code = 264, dump info: pc start: 0x120041000000, current: 0x1200410000a0, sc error info: 0xffffffffffff, su error info: 0xbe01404007de0000,0x42401430180017ff, mte error info: 0xaffe555c0005df97, vec error info: 0x9f7ffb3b007ef6fe, cube error info: 0, l1 error info: 0, aic error mask: 0x395856, para base: 0x120000200400, mte error: 0.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:328:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.700.676 [device_error_proc_c.cc:744]1948149 ProcessDavidStarsCoreErrorInfo:The extend info: errcode:(264) errorStr: The address for scalar to access GM is invalid subErrType: 0x4.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:330:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.700.682 [device_error_proc_c.cc:690]1948149 AddExceptionRegInfo:add error register: core_id=1, stream_id=61, task_sn=3
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:331:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.700.684 [device_error_proc_c.cc:741]1948149 ProcessDavidStarsCoreErrorInfo:The error from device(chipId:0, dieId:0), serial number is 156, there is an aivec error exception, core id is 1, error code = 264, dump info: pc start: 0x120041000000, current: 0x1200410000a0, sc error info: 0xffffffffffff, su error info: 0xbe0140401aa80000,0xfcf1145f180017ff, mte error info: 0x9a9fed6f00037f75, vec error info: 0xfdb5c95d001ef76c, cube error info: 0, l1 error info: 0, aic error mask: 0x395856, para base: 0x120000200400, mte error: 0.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:333:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.700.690 [device_error_proc_c.cc:744]1948149 ProcessDavidStarsCoreErrorInfo:The extend info: errcode:(264) errorStr: The address for scalar to access GM is invalid subErrType: 0x4.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:335:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.700.695 [device_error_proc_c.cc:690]1948149 AddExceptionRegInfo:add error register: core_id=2, stream_id=61, task_sn=3
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:336:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.700.697 [device_error_proc_c.cc:741]1948149 ProcessDavidStarsCoreErrorInfo:The error from device(chipId:0, dieId:0), serial number is 156, there is an aivec error exception, core id is 2, error code = 264, dump info: pc start: 0x120041000000, current: 0x1200410000a0, sc error info: 0xffffffffffff, su error info: 0xbe0140403e3e0000,0xedd9e43d180017ff, mte error info: 0x26512, vec error info: 0x6eedf53d003cdefc, cube error info: 0, l1 error info: 0, aic error mask: 0x395856, para base: 0x120000200400, mte error: 0.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:338:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.700.702 [device_error_proc_c.cc:744]1948149 ProcessDavidStarsCoreErrorInfo:The extend info: errcode:(264) errorStr: The address for scalar to access GM is invalid subErrType: 0x4.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:340:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.700.707 [device_error_proc_c.cc:690]1948149 AddExceptionRegInfo:add error register: core_id=3, stream_id=61, task_sn=3
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:341:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.700.709 [device_error_proc_c.cc:741]1948149 ProcessDavidStarsCoreErrorInfo:The error from device(chipId:0, dieId:0), serial number is 156, there is an aivec error exception, core id is 3, error code = 264, dump info: pc start: 0x120041000000, current: 0x1200410000a0, sc error info: 0xffffffffffff, su error info: 0xbe0140403e9a0000,0x3fff7ced180017ff, mte error info: 0x26512, vec error info: 0x3feb9fde0050d9c7, cube error info: 0, l1 error info: 0, aic error mask: 0x395856, para base: 0x120000200400, mte error: 0.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:343:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.700.715 [device_error_proc_c.cc:744]1948149 ProcessDavidStarsCoreErrorInfo:The extend info: errcode:(264) errorStr: The address for scalar to access GM is invalid subErrType: 0x4.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:345:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.700.719 [device_error_proc_c.cc:690]1948149 AddExceptionRegInfo:add error register: core_id=4, stream_id=61, task_sn=3
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:346:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.700.721 [device_error_proc_c.cc:741]1948149 ProcessDavidStarsCoreErrorInfo:The error from device(chipId:0, dieId:0), serial number is 156, there is an aivec error exception, core id is 4, error code = 264, dump info: pc start: 0x120041000000, current: 0x1200410000a0, sc error info: 0xffffffffffff, su error info: 0xbe0140403b380000,0xddffcd5c180017ff, mte error info: 0x26512, vec error info: 0x379fa9f70077d075, cube error info: 0, l1 error info: 0, aic error mask: 0x395856, para base: 0x120000200400, mte error: 0.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:348:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.700.727 [device_error_proc_c.cc:744]1948149 ProcessDavidStarsCoreErrorInfo:The extend info: errcode:(264) errorStr: The address for scalar to access GM is invalid subErrType: 0x4.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:350:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.700.731 [device_error_proc_c.cc:690]1948149 AddExceptionRegInfo:add error register: core_id=5, stream_id=61, task_sn=3
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:351:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.700.733 [device_error_proc_c.cc:741]1948149 ProcessDavidStarsCoreErrorInfo:The error from device(chipId:0, dieId:0), serial number is 156, there is an aivec error exception, core id is 5, error code = 264, dump info: pc start: 0x120041000000, current: 0x1200410000a0, sc error info: 0xffffffffffff, su error info: 0xbe0140402a0c0000,0xc565c4ff180017ff, mte error info: 0x26512, vec error info: 0x2370efdf0072fe4f, cube error info: 0, l1 error info: 0, aic error mask: 0x395856, para base: 0x120000200400, mte error: 0.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:353:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.700.738 [device_error_proc_c.cc:744]1948149 ProcessDavidStarsCoreErrorInfo:The extend info: errcode:(264) errorStr: The address for scalar to access GM is invalid subErrType: 0x4.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:355:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.700.743 [device_error_proc_c.cc:690]1948149 AddExceptionRegInfo:add error register: core_id=6, stream_id=61, task_sn=3
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:356:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.700.745 [device_error_proc_c.cc:741]1948149 ProcessDavidStarsCoreErrorInfo:The error from device(chipId:0, dieId:0), serial number is 156, there is an aivec error exception, core id is 6, error code = 264, dump info: pc start: 0x120041000000, current: 0x1200410000a0, sc error info: 0xffffffffffff, su error info: 0xbe0140402bec0000,0xfc7fecf5180017ff, mte error info: 0x26512, vec error info: 0xf570c2f7007f5fbd, cube error info: 0, l1 error info: 0, aic error mask: 0x395856, para base: 0x120000200400, mte error: 0.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:358:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.700.751 [device_error_proc_c.cc:744]1948149 ProcessDavidStarsCoreErrorInfo:The extend info: errcode:(264) errorStr: The address for scalar to access GM is invalid subErrType: 0x4.
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:360:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.700.755 [device_error_proc_c.cc:690]1948149 AddExceptionRegInfo:add error register: core_id=7, stream_id=61, task_sn=3
/root/ascend/log/debug/plog/plog-1947336_20260710194046786.log:361:[ERROR] RUNTIME(1947336,kv_shuffle):2026-07-10-19:40:46.700.757 [device_error_proc_c.cc:741]1948149 ProcessDavidStarsCoreErrorInfo:The error from device(chipId:0, dieId:0), serial number is 156, there is an aivec error exception, core id is 7, error code = 264, dump info: pc start: 0x120041000000, current: 0x1200410000a0, sc error info: 0xffffffffffff, su error info: 0xbe0140403b180000,0xfff1d7d1180017ff, mte error info: 0x26512, vec error info: 0xffb7e6cf001f1ff7, cube error info: 0, l1 error info: 0, aic error mask: 0x395856, para base: 0x120000200400, mte error: 0.

Special notes for this issue/备注 (Optional / 选填)