Closed
[Requirement|需求建议]: 负载均衡SectionStreamK支持Qk和V的dim不同 #4560
FiguraDogeAdd on  Aug 25Closed at  Aug 26
FiguraDogeCollaborators
Aug 25 Create

Thanks for sending an requirement! Please fill in the following template to help quickly solve your problem.

Backgroud(背景信息)

部分量化算子的Qk,V使用的dim值不同。此需要负载均衡支持此类输入。

Benefit / Necessity (价值/作用)

更准确的计算L2开销。

Design(设计方案)

  1. IBaseInfo纯接口类
    删除原有GetHeadDim()
    添加GetHeadDimQk()接口
    添加GetHeadDimV()接口

  2. BaseInfo实现类
    删除headDim字段
    添加headDimQk字段
    添加headDimV字段

  3. SectionStreamKImpl算法实现
    计算L2大小时,通过GetHeadDimQk()核GetHeadDimV()获取正确的dim大小。
    KV不再默认一致。

likedislike
FFiguraDogeCollaborators
Aug 25 Added label: requirement
FFiguraDogeCollaborators
Aug 25 Modify description of issue
weihao18Collaborators
Aug 25 Comments:

/assign @ChengjiaWu

likedislike
CANN-robotCANN-robotCollaborators
Aug 25 Assign ChengjiaWu as responsible person
CANN-robotCANN-robotCollaborators
Aug 26 Close issue
CANN-robotCANN-robotCollaborators
Aug 26 Added label: resolved