FilesLast commitLast update
10 days ago
2 months ago
24 days ago
22 days ago
22 days ago
8 days ago
8 days ago
5 days ago
2 months ago
1 month ago
17 days ago
README

AscendQuantV2

English | 简体中文

Applicable Products

Product Support
Ascend 950PR&950DT products √
Atlas A3 products √
Atlas A2 products √
Atlas 200I/500 A2 inference products ×
Atlas inference products √
Atlas training products ×
kirin X90 processor series products √
kirin 9030 processor series products √

Function

  • Operator function: performs a quantization operation on the input x. You can set axis to specify the axes that scale and offset correspond to. The shapes of scale and offset must be equal to the axes of x specified by axis or be 1. Currently, axis supports setting the last two dimensions.
  • Formula:
    • When sqrt_mode is false, the formula is:

      y=round((x∗scale)+offset)y = round((x * scale) + offset)

    • When sqrt_mode is true, the formula is:

      y=round((x∗scale∗scale)+offset)y = round((x * scale * scale) + offset)

Parameters

Parameter name Input/Output/Attribute Description Data type Data format
x Input The input on which quantization is performed, corresponding to `x` in the formula. FLOAT16, FLOAT32, BFLOAT16 ND
scale Input The scale value in quantization, corresponding to `scale` in the formula. FLOAT16, FLOAT32, BFLOAT16 ND
offset Optional input The offset value in dequantization, corresponding to `offset` in the formula. FLOAT16, FLOAT32, BFLOAT16 ND
sqrt_mode Optional attribute
  • Specifies the logic for scale participating in the computation, corresponding to `sqrtMode` in the formula.
  • The default value is false.
BOOL -
round_mode Optional attribute
  • Specifies the conversion method for casting to the output data type. Supported values: round, ceil, trunc, and floor.
  • The default value is "round".
STRING -
dst_type Optional attribute
  • Specifies the data type of the output.
  • The default value is DT_INT8.
INT -
axis Optional attribute
  • Specifies the dimension of x that scale and offset correspond to.
  • The default value is -1.
INT -
y Output Represents the result output `y` of AscendQuantV2, corresponding to `y` in the formula. The shape is consistent with the shape of the input `x`. INT8, INT4, HIFLOAT8, FLOAT8_E5M2, FLOAT8_E4M3FN ND
  • Atlas inference products:

    • Data type:
      • The input tensor data type does not support BFLOAT16.
      • The output tensor data type supports only INT8.
    • round_mode: supported values are round, ceil, trunc, and floor.
    • dst_type: supports the value 2, indicating INT8.
    • axis: supports only the last dimension of x (assuming the number of dimensions of the input x is xDimNum, the value of axis is -1 or xDimNum-1).
  • Atlas A2 products and Atlas A3 products:

    • The output tensor data type supports only INT8 and INT4.
    • dst_type: supports the values 2 and 29, indicating INT8 and INT4, respectively.
    • axis: supports the last two dimensions of x (assuming the number of dimensions of the input x is xDimNum, the value range of axis is [-2, -1] or [xDimNum-2, xDimNum-1]).
  • Ascend 950PR&950DT products:

    • round_mode: when dst_type is FLOAT8_E5M2 or FLOAT8_E4M3FN, only round is supported. When dst_type is HIFLOAT8, round and hybrid are supported. When dst_type is another type, round, ceil, trunc, and floor are supported.
    • axis: supports the last two dimensions of x (assuming the number of dimensions of the input x is xDimNum, the value range of axis is [-2, -1] or [xDimNum-2, xDimNum-1]).
  • Kirin X90/kirin 9030 processor series products: x, scale, and offset do not support BFLOAT16. The data type of y does not support INT4, HIFLOAT8, FLOAT8_E5M2, or FLOAT8_E4M3FN.

Constraints

None

Invocation Description

Invocation method Sample Code Description
aclnn API test_aclnn_ascend_quant Invokes the AscendQuantV2 operator through the aclnnAscendQuant API.
aclnn API test_aclnn_ascend_quant_v3 Invokes the AscendQuantV2 operator through the aclnnAscendQuantV3 API.
Graph mode - Invokes the AscendQuantV2 operator through graph construction using the operator IR.