Pull Request has been successfully merged by CANN-robot.
(Thank you for FiguraDoge's contribution.)变更摘要
本 PR 为 SectionStreamK 分核算法新增 S1G 不合轴输出布局支持,并支持 QK/KV 头维度不同的场景。核心改动包括:在 load_balance_common.h 中新增 OutputLayout 枚举(BN2_S1G 合轴、BN1_S1 不合轴)并在 GeneralBalanceParam 增加 outputLayout 参数(默认 BN2_S1G);将 BaseInfo 的 GetHeadDim()/headDim 拆分为 GetHeadDimQk()/GetHeadDimV() 与 headDimQk/headDimV;SectionStreamKImpl 根据输出布局计算 M 轴大小、BN 索引及 section 切分点,并将分核结果字段由 BN2/GS1 统一更名(bN2End→bNEnd、gS1End→mEnd、bN2Idx→bNIdx、gS1Idx→mIdx);同时同步修改 flash_attn_metadata、quant_flash_attn_metadata、stem_indexer_metadata 三个 METADATA 算子,更新元数据索引常量命名(如 FA_BN2_START_INDEX→FA_BN_START_INDEX、FD_BN2_IDX_INDEX→FD_BN_IDX_INDEX),并显式设置 param.outputLayout = OutputLayout::BN2_S1G。
主要改动
- 新增
OutputLayout输出布局参数: 在load_balance_common.h新增枚举OutputLayout(BN2_S1G=0 默认合轴、BN1_S1不合轴),并在GeneralBalanceParam中增加outputLayout字段,用于控制 S1G 是否合轴及对应的分核维度计算。 - 拆分 QK/V 头维度接口: 在
base_info.h中将GetHeadDim()拆分为GetHeadDimQk()/GetHeadDimV(),字段headDim拆分为headDimQk/headDimV(默认 64),使 KV 与 QK 可以采用不同的 head dim。 - 分核算法按输出布局计算与映射:
SectionStreamKImpl新增GetHeadNum、GetMSize、ToOutputLayoutBnIdx辅助函数,据此计算 M 轴大小、BN 索引并转换 section 切分点;CalcS2Range中 M 轴到 S1 token 的映射区分合轴(BN2_S1G)与不合轴(BN1_S1)两种排布方式;CalcGridInfoSection中 S2 相关 token 开销按headDimQk + headDimV计算。 - 分核结果与元数据字段统一更名:
section_stream_k_def.h/section_stream_k_impl.h/section_stream_k.h中 BN2→BN、GS1/S1G→M 全面更名(如sectionBn2Idx→sectionBnIdx、S1GCache→MCache、bN2CostOfEachBatch→bNCostOfEachBatch),FA/FD 结果结构体字段同步调整。 - METADATA 算子适配:
flash_attn_metadata、quant_flash_attn_metadata、stem_indexer_metadata的 AICPU kernel 及示例程序同步更新:元数据索引常量更名、baseInfo.headDimQk/headDimV赋值,并在InitLoadBalanceParams/ParamsInit/GenerateSectionStreamKParam中显式设置param.outputLayout = load_balance::OutputLayout::BN2_S1G。


Thanks for your pull-request.
The full list of commands accepted by me can be found at here.
You can get sig-info at here.
You can self-configure the PR merge rules for this repository. For more details, please refer to Here.
For more, you also can visit HICANN.
PR Approval Progress
✅ Congratulations! All modules have met the lgtm and approve requirements.
Module Approval Details
| module | lgtm status | approve status |
|---|---|---|
| attention | ✅ 蒋礼锐, tang-hao-hw-gitcode, shen_weiling, shasha_an (4/2) | ✅ shen_weiling, 蒋礼锐, tang-hao-hw-gitcode (3/1) |
| attention/common/op_kernel/load_balance | ✅ 蒋礼锐, shasha_an (2/2) | ✅ 蒋礼锐 (1/1) |
| experimental/attention | ✅ 蒋礼锐, tang-hao-hw-gitcode, shen_weiling, shasha_an (4/2) | ✅ 蒋礼锐, tang-hao-hw-gitcode, shen_weiling (3/1) |
💡 Tip:
- Committer can comment
/approveor/lgtm- Commenting
/approveimplies both code review (lgtm) and intent to merge (approve)
CLA Signature Pass
ChengjiaWu, thanks for your pull request. All authors of the commits have signed the CLA. 👍


The MR can not be merged, because of CodeReview discussion not resolved
If you want to solve this problem, you can click here to do it in the FAQs.


描述
修改输出参数名BN2为BN,S1G为M,修改涉及使用到SectionStreamK的METADATA算子。
METADATA算子显式设置分核OutputLayout
headDim_设置到headDimQk核headDimV上。修改涉及使用到SectionStreamK的METADATA算子。
关联的Issue
Issue #4559
Issue #4559
Issue #4418
测试
文档更新
类型标签