FilesLast commitLast update
10 days ago
4 months ago
24 days ago
4 months ago
21 days ago
4 months ago
21 days ago
11 months ago
1 month ago
17 days ago
README

GroupQuant

English | 简体中文

Applicable Products

Product Support
Ascend 950PR&950DT products √
Atlas A3 products √
Atlas A2 products √
Atlas 200I/500 A2 inference products ×
Atlas inference products ×
Atlas training products ×
kirin X90 processor series products √
kirin 9030 processor series products √

Function

  • Operator function: Performs the grouped quantization operation on the input x.

  • Formula:

    y=round((x∗scale)+offsetOptional)y = round((x * scale) + offsetOptional)

Parameters

Parameter name Input/Output/Attribute Description Data type Data format
x Input Input on which quantization needs to be performed, which corresponds to `x` in the formula. The shape supports 2 dimensions. FLOAT32, FLOAT16, BFLOAT16 ND
scale Input Scale value in the quantization, which corresponds to `scale` in the formula. The shape supports 2 dimensions, and the second dimension equals the second dimension of the x shape. FLOAT32, FLOAT16, BFLOAT16 ND
group_index Input GroupIndex value in the quantization. The shape supports 1 dimension, and the dimension equals the first dimension of the scale shape. INT32, INT64 ND
offset Optional input Offset value in the quantization, which corresponds to `offsetOptional` in the formula. The data type is the same as that of `scale`. `offsetOptional` is 1 number. FLOAT32, FLOAT16, BFLOAT16 ND
dst_type Optional attribute
  • Specifies the data type of the output. Supported values are 2 and 29, which indicate INT8 and INT4 respectively.
  • Default value: DT_INT8.
INT -
y Output Output after grouped quantization, which corresponds to y in the formula. The shape is the same as the shape of `x`. INT4, INT8 ND
  • Kirin X90/kirin 9030 processor series products: BFLOAT16 is not supported.

Constraints

  • The data type of the input scale is the same as that of the input offset.
  • If the attribute dst_type is 29 (INT4), the last dimension of the shape of the output y must be divisible by 2.
  • The input group_index must be a non-decreasing sequence, and the maximum value must equal the size of the first dimension of the shape of the input x.
  • The shape of the input offset currently supports only [1, ] or [].

Invocation Description

Invocation method Sample code Description
aclnn API test_aclnn_group_quant Calls the GroupQuant operator through the aclnnGroupQuant API.
Graph mode - Calls the GroupQuant operator through graph construction with the operator IR.