Thanks for sending an requirement! Please fill in the following template to help quickly solve your problem.
quant_flash_attn接口支持FP8全量化
建quant_flash_attn接口支持FP8全量化
将fused_infer_attention_v2接口的fp8全量化适配到quant_flash_attn接口
/assign @wangke293
Thanks for sending an requirement! Please fill in the following template to help quickly solve your problem.
Backgroud（背景信息）
quant_flash_attn接口支持FP8全量化
Benefit / Necessity （价值/作用）
建quant_flash_attn接口支持FP8全量化
Design（设计方案）
将fused_infer_attention_v2接口的fp8全量化适配到quant_flash_attn接口