FilesLast commitLast update
11 days ago
4 months ago
4 months ago
3 days ago
4 months ago
26 days ago
11 months ago
1 month ago
17 days ago
README

SwiGlu

English | 简体中文

Applicable Products

Product Support
Ascend 950PR&950DT products √
Atlas A3 products √
Atlas A2 products √
Atlas 200I/500 A2 inference products ×
Atlas inference products √
Atlas training products ×
kirin X90 processor series products √
Kirin 9030 processor series products √

Function

  • Description: the Swish gated linear unit activation function, which implements the SwiGlu computation of x.

  • Formula:

    outi = SwiGlu(xi)=Swish(Ai)*Bi

    Here, Ai is the first half of the tensor after xi is split in half along the specified dim dimension, and Bi is the second half of the tensor after xi is split in half along the specified dim dimension.

Parameters

Parameter name Input/Output/Attribute Description Data type Data format
x Input The input x in the formula. BFLOAT16, FLOAT16, FLOAT32 ND
dim Attribute
  • The dimension index to split. The corresponding axis of x is split in half.
  • The value range is [-x.dim(), x.dim()-1].
INT64 -
out Output The outi in the formula. BFLOAT16, FLOAT16, FLOAT32 ND
  • For Atlas inference products, the supported data types are FLOAT16 and FLOAT32.
  • For kirin X90 processor series products and Kirin 9030 processor series products, BFLOAT16 is not supported.

Constraints

None

Invocation Description

Invocation method Invocation sample Description
aclnn invocation test_aclnn_swi_glu Invokes the SwiGlu operator through the aclnnSwiGlu API.
Graph-mode invocation - Invokes the SwiGlu operator through graph construction using the operator IR.