Merged
[CANNBot]:dequant_rope_quant_kvcache新增支持ascend950 ascendc实现 #10481
[CANNBot]:dequant_rope_quant_kvcache新增支持ascend950 ascendc实现 #10481
Merged
yulianjieAdd on Aug 24
yulianjie
Aug 24

描述

本 PR 为 DequantRopeQuantKvcache 新增 Ascend 950 AscendC 平台支持,并补充空 Tensor 场景下的 Tiling 处理。算子接口及计算语义保持不变,主要改动如下:

  1. 在算子定义中注册 AICore().AddConfig("ascend950"),启用 Ascend 950 平台配置。
  2. 新增 Ascend 950 binary 配置,覆盖以下四类 Kernel:
    • DequantRopeQuantKvcache_f16_f16_f16
    • DequantRopeQuantKvcache_bf16_bf16_bf16
    • DequantRopeQuantKvcache_int32_bf16_bf16
    • DequantRopeQuantKvcache_int32_fp16_bf16
  3. 新增 Ascend 950 simplified key 配置,默认 key 为 0。
  4. 针对 __NPU_ARCH__ == 3510 且 taskNum == 0 的空任务场景,将 block dim 设置为 0,保存空任务 Tiling 数据并正常返回,避免无有效任务时启动 Kernel。
  5. 对涉及文件按照仓库规范完成格式整理。

本次共变更 4 个文件,新增 1082 行、删除 152 行,对应提交:

5fc0c904b708ef4398c90c4c296556e3191ba6ce

关联的Issue

关联 Issue #4523:https://gitcode.com/cann/ops-transformer/issues/4523

测试

  • GitCode 最新 CI 流水线通过,PR 当前包含 ci-pipeline-passed 标签。
  • API 检查通过,PR 当前包含 api-check-pass 标签。
  • 使用 TTK kernel 直调模式按照 L0 标准执行固定 200 条用例:
    • 精度检查:200/200 PASS
    • 内存越界检查:200/200 PASS
    • Torch XPU 对比:200/200 PASS
  • 补充执行量化边界场景 32 条:32/32 PASS。
  • 补充执行 page/二维尾块场景 9 条:9/9 PASS。
  • 本轮验证聚焦 L0 精度、功能和内存安全,性能结果不作为本 PR 验收项。

文档更新

本 PR 不涉及算子接口语义或接口文档变更;新增 Ascend 950 binary 配置和 simplified key 配置。

类型标签

likedislike
Pull Request has been successfully merged by CANN-robot.
(Thank you for yulianjie's contribution.)
Yyulianjie
Aug 24 Created pull request, commit ec41d845
atomgit-bot
atomgit-bot
Aug 24 Comments:

变更摘要

本 PR 为 DequantRopeQuantKvcache 算子新增 ascend950 芯片支持。主要包含三方面改动:在 op_host 侧通过 AICore().AddConfig("ascend950") 注册新平台的算子配置;新增 ascend950 目录下的二进制 kernel 配置文件(dequant_rope_quant_kvcache_binary.json)与 simplified_key 模式配置文件(dequant_rope_quant_kvcache_simplified_key.ini);并在 TilingDequantRopeQuantKvcache 中针对 __NPU_ARCH__ == 3510 增加空 tensor(taskNum == 0)场景的 tiling 处理。

主要改动

  • 注册 ascend950 平台配置:在 dequant_rope_quant_kvcache_def.cpp 的 AICore().AddConfig() 调用链中新增 AddConfig("ascend950"),使算子可在该平台上编译运行。
  • 新增 ascend950 二进制配置:新增 op_host/config/ascend950/dequant_rope_quant_kvcache_binary.json,定义 4 种 bin_filename 组合(DequantRopeQuantKvcache_f16_f16_f16、_bf16_bf16_bf16、_int32_bf16_bf16、_int32_fp16_bf16)各自的输入/输出张量(如 x、cos、sin、k_cache、v_cache、scale_k、scale_v 及可选的 offset_k、bias 等)与属性(size_splits、quant_mode、layout、kv_output、cache_mode),全部采用 ND 格式与 FormatAgnostic 匹配模式。
  • 新增 simplified_key 配置:新增 dequant_rope_quant_kvcache_simplified_key.ini,为 DequantRopeQuantKvcache 配置 default=0,用于指定 opc 工具编译二进制 kernel 时的 --simplified_key_mode 取值。
  • 空 tensor 场景 tiling 处理:在 dequant_rope_quant_kvcache_tiling.cpp 中,当 __NPU_ARCH__ == 3510 且 taskNum 为 0 时,将 block dim 置为 0、保存空的 tiling 数据并设置最小 workspace,直接返回成功,避免空 tensor 场景下的异常。
likedislike
Incorrect?
CANN-robot
CANN-robotCollaborators
Aug 24 Comments:

Hi @lianjieyu, welcome to submitting your first PR to ops-transformer!

PR Merge Steps

1. CLA Signing

If the current PR label includes cann-cla/yes, it means you have signed the CLA and can proceed to the next step. If the label includes cann-cla/no, please sign the CLA first. If you have any questions, please refer to the FAQ.

2. CI Check

Please comment /compile to trigger the CI pipeline check. If the CI run is successful, the PR will be tagged with ci-pipeline-passed and you can proceed to the next step. If the CI run fails, the PR will be tagged with ci-pipeline-failed, please check the CI logs to fix the issues in the PR. If you have any questions, please refer to the FAQ.

3. Code Review

After CI passes, please refer to the PR Approval Progress and proactively @ the committers in the table to review the code. After approval, committers will comment /lgtm and /approve. Once the lgtm and approved labels are successfully added, the PR will be merged automatically.

likedislike
CANN-robotCANN-robotCollaborators
Aug 24 Added label: cann-cla/yes
CANN-robot
CANN-robotCollaborators
Aug 24 Comments:

Thanks for your pull-request.
The full list of commands accepted by me can be found at here.
You can get sig-info at here.
You can self-configure the PR merge rules for this repository. For more details, please refer to Here.
For more, you also can visit HICANN.


PR Approval Progress

✅ Congratulations! All modules have met the lgtm and approve requirements.

Module Approval Details

module lgtm status approve status
*/*/README.md ✅ 陈娇, 马兵 (2/2) ✅ 陈娇 (1/1)
*/*/op_host/*_def.cpp ✅ 王永光, 马兵 (2/2) ✅ 王永光 (1/1)
posembedding ✅ 宋恺, 马兵, 於欣洁 (3/2) ✅ 於欣洁, 宋恺 (2/1)

💡 Tip:

  • Committer can comment /approve or /lgtm
  • Commenting /approve implies both code review (lgtm) and intent to merge (approve)

CLA Signature Pass

lianjieyu, thanks for your pull request. All authors of the commits have signed the CLA. 👍

likedislike
76 messages are collapsed here。 See more
於欣洁Collaborators
Aug 25 Comments:

/lgtm
/approve

likedislike
mabingCollaborators
Aug 26 Comments:

/lgtm

likedislike
CANN-robotCANN-robotCollaborators
Aug 26 Added label: lgtm
CANN-robotCANN-robotCollaborators
Aug 26 Closed associated issue
CANN-robotCANN-robotCollaborators
Aug 26 Merged pull request