Thanks for sending an requirement! Please fill in the following template to help quickly solve your problem.
当前部分新增模型使用变体的SwiGLU公式，在silu后面进行clamp计算，与当前的DequantSwigluQuant算子公式有差异，需要进行支持。 def forward_native(x， limit) -> torch.Tensor: gate, up = x.chunk(2, dim=-1) gate = F.silu(gate) gate = gate.clamp(max=limit) up = up.clamp(min=-limit, max=limit) return gate * up
HW
/open
Thanks for sending an requirement! Please fill in the following template to help quickly solve your problem.
Backgroud（背景信息）
当前部分新增模型使用变体的SwiGLU公式，在silu后面进行clamp计算，与当前的DequantSwigluQuant算子公式有差异，需要进行支持。
def forward_native(x， limit) -> torch.Tensor:
gate, up = x.chunk(2, dim=-1)
gate = F.silu(gate)
gate = gate.clamp(max=limit)
up = up.clamp(min=-limit, max=limit)
return gate * up
Origin（信息来源）
HW
Benefit / Necessity （价值/作用）
Design（设计方案）