Closed
[Requirement|需求建议]: 负载均衡SectionStreamK支持Qk和V的dim不同 #4560
FiguraDogeAdd on Aug 25Closed at Aug 26
Aug 25 Added label: requirement
Aug 25 Modify description of issue
weihao18
Aug 25 Comments:
Aug 25 Comments:
/assign @ChengjiaWu


Aug 25 Assign ChengjiaWu as responsible person
Aug 26 Close issue
Aug 26 Added label: resolved
Thanks for sending an requirement! Please fill in the following template to help quickly solve your problem.
Backgroud(背景信息)
部分量化算子的Qk,V使用的dim值不同。此需要负载均衡支持此类输入。
Benefit / Necessity (价值/作用)
更准确的计算L2开销。
Design(设计方案)
IBaseInfo纯接口类
删除原有GetHeadDim()
添加GetHeadDimQk()接口
添加GetHeadDimV()接口
BaseInfo实现类
删除headDim字段
添加headDimQk字段
添加headDimV字段
SectionStreamKImpl算法实现
计算L2大小时,通过GetHeadDimQk()核GetHeadDimV()获取正确的dim大小。
KV不再默认一致。