FilesLast commitLast update
6 days ago
3 months ago
10 months ago
18 days ago
16 days ago
6 months ago
10 months ago
25 days ago
11 days ago
README

AddLora

English | 简体中文

Product Support Status

Product Supported
Ascend 950PR&950DT products ×
Atlas A3 products ×
Atlas A2 products √
Atlas 200I/500 A2 inference products ×
Atlas inference products √
Atlas training products ×

Function Description

  • Operator function:

    Multiplies the input x by the corresponding weightA and weightB respectively based on the input index indices, then accumulates the results to the input y and outputs the result.

  • Computation formula:

    Given the input tensor x whose last dimension has a length of 2d, the AddLora function performs the following computation:

    1. Rearranges x based on the indices in indices, so that the x values corresponding to the same group of weights are placed together.

    2. Iterates over each LoRa group, and performs matrix multiplication between the corresponding x and weightA:

      Z1=xi⋅weightA[i,layerIdx,:,:]Z1 = x_{i} \cdot weightA[i, layerIdx, :, :]

    3. Performs matrix multiplication between the obtained Z1 and weightB:

      Z2=Z1⋅weightB[i,layerIdx,:,:]×scaleZ2 = Z1 \cdot weightB[i, layerIdx, :, :] \times scale

    4. Finally accumulates the Z2 output to y:

      out=y[:,yOffset:yOffset+ySliceSize]+Z2\text{out} = y[:, yOffset: yOffset+ySliceSize] + Z2

Parameter Description

Parameter Name Input/Output/Attribute Description Data Type Data Format
y Input The input y in the formula. FLOAT16 ND
x Input The input x in the formula. FLOAT16 ND
weightB Input The input weightB in the formula. FLOAT16 ND, NZ
indices Input The input indices in the formula. INT32 ND
weightAOptional Input The input weightA in the formula. FLOAT16 ND, NZ
layerIdx Attribute
  • Indicates the layer index.
  • The value must be less than the second dimension L of weightB.
INT -
scale Attribute The scaling factor. FLOAT -
yOffset Attribute
  • The offset when y is updated.
  • The value must be less than the second dimension H3 of y.
INT -
ySliceSize Attribute
  • The range when y is updated.
  • The value must be less than or equal to the second dimension H3 of y.
INT -
out Output The out in the formula. FLOAT16 ND
  • Atlas A2 products: The data formats of weightB and weightAOptional support ND.

Constraint Description

None

Invocation Description

Invocation Method Invocation Sample Description
aclnn invocation test_aclnn_add_lora Invokes the AddLora operator through the aclnnAddLora API.
Graph-mode invocation test_geir_add_lora Invokes the AddLora operator by building a graph through the operator IR.