已关闭
CCU kernel cache reuses handles across instances #48
KaranocaVe创建于  28 天前关闭于  5 天前
KaranocaVe
KaranocaVe
28 天前 创建

CCU kernel cache reuses handles across instances

At source commit 79de7e489db8db78f8fe3b616682099a888a204d, the process-wide
KernelHandleCache uses only binary_cache_tag as its key. A cached
CcuKernelHandle is created by HcommCcuKernelRegister for a particular
ccu_ins and die, so the same tag on another instance or die incorrectly
reuses a handle from the first registration and skips registration for the new
configuration.

The regression test confirms the mismatch: with one shared tag, the second
instance launch keeps the first instance's handle, and a different die also
reuses that handle. Repeated launches for the same instance and die should
continue to reuse their registration.

The fix keys cached handles by binary tag, CCU instance identity, and physical
die mask. The change is limited to the host launch cache and its unit test.

Validation

Validation

Fixed source and scope

  • Repository: cann/asc-comm
  • Source commit: 79de7e489db8db78f8fe3b616682099a888a204d
  • Changed files: src/ccu/ccu_host_launch.cpp and
    tests/ut/ccu/test_ccu_host_launch.cpp
  • Host toolchain: CMake 3.22.1 and GCC 11.4.0

Baseline regression

Added TestCcuHostLaunch.BinaryCacheTagIsScopedToCcuInstance to the fixed
source without changing the implementation, then built and ran
asccomm_ut_ccu_host_launch. The baseline result was 11/12 passed. The new
test failed because the second instance did not cause another registration:
the observed registration instance remained 0x11 instead of 0x22, and the
launch reused the first handle. The different-die check likewise reused the
first die's handle.

Fixed result

After applying fix.patch, the same build target and test executable passed
all 12 tests:

cmake --build <build-dir> --target asccomm_ut_ccu_host_launch --parallel 2
<build-dir>/asccomm_ut_ccu_host_launch --gtest_brief=1

The summary was [ PASSED ] 12 tests. CTest did not register this target, so
the test executable was run directly. No NPU or device operation was used.

Patch and checklist review

  • git apply --check --whitespace=error-all fix.patch passed against a clean
    checkout of the fixed source commit; git apply --stat lists only the two
    files above.
  • clang-format --dry-run --Werror passed for both changed C++ files using
    clang-format 18.1.8, matching the target repository's configured version.
  • git diff --check passed for the source diff.

The result establishes the host-side cache-key bug and its regression fix. It
does not claim runtime verification on CCU hardware.

Additional checks

  • The repository-pinned pre-commit checks passed for both changed C++ files,
    including clang-format 18.1.8, OAT, codespell, and file-integrity checks.
  • The host unit suite was not rerun because the CANN toolkit is unavailable.
    The fixed-source validation above records all 12
    host unit tests passing.

关联 PR: https://gitcode.com/cann/asc-comm/merge_requests/234

likedislike
KaranocaVeKaranocaVe
28 天前 关联了pull request:[Bugfix] Scope cached CCU kernel handles by instance and die
KaranocaVeKaranocaVe
28 天前 修改了issue 的描述
KaranocaVe
KaranocaVe
27 天前 评论:

/assign

likedislike
CANN-robotCANN-robot成员
27 天前 将 KaranocaVe 设为负责人
cellur_z
cellur_z成员
9 天前 评论:

Thanks for the detailed report. Your analysis is correct: the current cache key only contains binary_cache_tag, so a handle can be incorrectly reused across CCU instances and dies.

We will review MR !234 as soon as possible.

likedislike
cellur_zcellur_z成员
9 天前 将 zhao_lulu 设为负责人
cellur_z
cellur_z成员
5 天前 评论:

您的代码已合入,感谢您对本社区的贡献,目前将关闭本issue,如果有后续问题,欢迎您再来提问讨论~

likedislike
cellur_zcellur_z成员
5 天前 issue状态由 进行中 改变为 已完成
cellur_zcellur_z成员
5 天前 关闭了 issue
CANN-robotCANN-robot成员
5 天前 添加了label:resolved
KaranocaVe
KaranocaVe
3 天前 评论:

@zhao_lulu 您好,这个 Issue 对应的修复已经合入:https://gitcode.com/cann/asc-comm/merge_requests/234

开发者计划反馈,Issue 需要有 Accepted 标签才能计入积分,仅关闭或标记 resolved 还不够。目前这条 Issue 尚未标记 Accepted,麻烦您结合问题和已合入的修复复核一下;确认符合受理条件后,帮忙补上 Accepted 标签。

感谢您的支持!

likedislike