Merged
[SectionStreamK] 支持S1G不合轴和KV不同Dim #10563
[SectionStreamK] 支持S1G不合轴和KV不同Dim #10563
Merged
FiguraDogeAdd on Aug 25
FiguraDogeCollaborators
Aug 25

描述

  1. Param添加参数OutputLayout表示S1G是否合轴,默认合轴BN2_S1G。
    修改输出参数名BN2为BN,S1G为M,修改涉及使用到SectionStreamK的METADATA算子。
    METADATA算子显式设置分核OutputLayout
  2. BaseInfo修改原参数headDim为headDimQk,添加参数headDimV。
    headDim_设置到headDimQk核headDimV上。修改涉及使用到SectionStreamK的METADATA算子。

关联的Issue

Issue #4559
Issue #4559
Issue #4418

测试

文档更新

类型标签

likedislike
Pull Request has been successfully merged by CANN-robot.
(Thank you for FiguraDoge's contribution.)
FFiguraDogeCollaborators
Aug 25 Created pull request, commit 4607b633
atomgit-bot
atomgit-bot
Aug 25 Comments:

变更摘要

本 PR 为 SectionStreamK 分核算法新增 S1G 不合轴输出布局支持,并支持 QK/KV 头维度不同的场景。核心改动包括:在 load_balance_common.h 中新增 OutputLayout 枚举(BN2_S1G 合轴、BN1_S1 不合轴)并在 GeneralBalanceParam 增加 outputLayout 参数(默认 BN2_S1G);将 BaseInfo 的 GetHeadDim()/headDim 拆分为 GetHeadDimQk()/GetHeadDimV() 与 headDimQk/headDimV;SectionStreamKImpl 根据输出布局计算 M 轴大小、BN 索引及 section 切分点,并将分核结果字段由 BN2/GS1 统一更名(bN2End→bNEnd、gS1End→mEnd、bN2Idx→bNIdx、gS1Idx→mIdx);同时同步修改 flash_attn_metadata、quant_flash_attn_metadata、stem_indexer_metadata 三个 METADATA 算子,更新元数据索引常量命名(如 FA_BN2_START_INDEX→FA_BN_START_INDEX、FD_BN2_IDX_INDEX→FD_BN_IDX_INDEX),并显式设置 param.outputLayout = OutputLayout::BN2_S1G。

主要改动

  • 新增 OutputLayout 输出布局参数: 在 load_balance_common.h 新增枚举 OutputLayout(BN2_S1G=0 默认合轴、BN1_S1 不合轴),并在 GeneralBalanceParam 中增加 outputLayout 字段,用于控制 S1G 是否合轴及对应的分核维度计算。
  • 拆分 QK/V 头维度接口: 在 base_info.h 中将 GetHeadDim() 拆分为 GetHeadDimQk()/GetHeadDimV(),字段 headDim 拆分为 headDimQk/headDimV(默认 64),使 KV 与 QK 可以采用不同的 head dim。
  • 分核算法按输出布局计算与映射: SectionStreamKImpl 新增 GetHeadNum、GetMSize、ToOutputLayoutBnIdx 辅助函数,据此计算 M 轴大小、BN 索引并转换 section 切分点;CalcS2Range 中 M 轴到 S1 token 的映射区分合轴(BN2_S1G)与不合轴(BN1_S1)两种排布方式;CalcGridInfoSection 中 S2 相关 token 开销按 headDimQk + headDimV 计算。
  • 分核结果与元数据字段统一更名: section_stream_k_def.h/section_stream_k_impl.h/section_stream_k.h 中 BN2→BN、GS1/S1G→M 全面更名(如 sectionBn2Idx→sectionBnIdx、S1GCache→MCache、bN2CostOfEachBatch→bNCostOfEachBatch),FA/FD 结果结构体字段同步调整。
  • METADATA 算子适配: flash_attn_metadata、quant_flash_attn_metadata、stem_indexer_metadata 的 AICPU kernel 及示例程序同步更新:元数据索引常量更名、baseInfo.headDimQk/headDimV 赋值,并在 InitLoadBalanceParams/ParamsInit/GenerateSectionStreamKParam 中显式设置 param.outputLayout = load_balance::OutputLayout::BN2_S1G。
likedislike
Incorrect?
CANN-robotCANN-robotCollaborators
Aug 25 Added label: cann-cla/yes
CANN-robot
CANN-robotCollaborators
Aug 25 Comments:

Thanks for your pull-request.
The full list of commands accepted by me can be found at here.
You can get sig-info at here.
You can self-configure the PR merge rules for this repository. For more details, please refer to Here.
For more, you also can visit HICANN.


PR Approval Progress

✅ Congratulations! All modules have met the lgtm and approve requirements.

Module Approval Details

module lgtm status approve status
attention ✅ 蒋礼锐, tang-hao-hw-gitcode, shen_weiling, shasha_an (4/2) ✅ shen_weiling, 蒋礼锐, tang-hao-hw-gitcode (3/1)
attention/common/op_kernel/load_balance ✅ 蒋礼锐, shasha_an (2/2) ✅ 蒋礼锐 (1/1)
experimental/attention ✅ 蒋礼锐, tang-hao-hw-gitcode, shen_weiling, shasha_an (4/2) ✅ 蒋礼锐, tang-hao-hw-gitcode, shen_weiling (3/1)

💡 Tip:

  • Committer can comment /approve or /lgtm
  • Commenting /approve implies both code review (lgtm) and intent to merge (approve)

CLA Signature Pass

ChengjiaWu, thanks for your pull request. All authors of the commits have signed the CLA. 👍

likedislike
FFiguraDogeCollaborators
Aug 25 Modify description of pull request
43 messages are collapsed here。 See more
CANN-robotCANN-robotCollaborators
Aug 26 Added label: lgtm
CANN-robot
CANN-robotCollaborators
Aug 26 Comments:

The MR can not be merged, because of CodeReview discussion not resolved

If you want to solve this problem, you can click here to do it in the FAQs.

likedislike
FFiguraDogeCollaborators
Aug 26 Resolved final issue
CANN-robotCANN-robotCollaborators
Aug 26 Closed associated issue
CANN-robotCANN-robotCollaborators
Aug 26 Merged pull request