FilesLast commitLast update
23 days ago
25 days ago
25 days ago
2 months ago
2 months ago
25 days ago
3 months ago
29 days ago
17 days ago
README

Sparse4to2QuantMatmul

English | 简体中文

Applicable Products

Product Support
Atlas A3 products √
Atlas A2 products √

Function

  • Operator function: Perform sparse 4:2 quantized matrix multiplication.

  • Formula:

    • Atlas A2 products, Atlas A3 products:

      out=x@sparseWeight∗sparseWeightScale∗xScale+biasout = x@sparseWeight * sparseWeightScale * xScale + bias

Parameters

Parameter name Input/Output/Attribute Description Data type Data format
x Input Left matrix in the matrix multiplication, corresponding to x in the formula. INT8 ND
sparseWeight Input Sparse right matrix in the matrix multiplication. It is compressed and corresponds to sparseWeight in the formula. INT8 FRACTAL_NZ
index Input Index matrix generated during the compression of the right matrix in the matrix multiplication. UINT8 ND
xScale Input Dequantization parameter corresponding to input x, corresponding to xScale in the formula. FLOAT32 ND
sparseWeightScale Input Dequantization parameter corresponding to input sparseWeight, corresponding to sparseWeightScale in the formula. FLOAT32 ND
bias Input Bias accumulated after the matrix multiplication, corresponding to bias in the formula. BFLOAT16 ND
y Output Computation result of the matrix multiplication. BFLOAT16 ND

Constraints

  • Empty tensors are not supported.
  • Non-contiguous tensors are not supported.

Invocation Description

Invocation method Sample code Description
aclnn API test_aclnn_sparse4to2quant_matmul Invoke the Sparse4to2QuantMatmul operator through
aclnnSparse4to2QuantMatmulWeightNz
and other APIs.