FilesLast commitLast update
10 days ago
4 months ago
2 months ago
4 months ago
23 days ago
4 months ago
23 days ago
23 days ago
1 month ago
18 days ago
README

GroupedQuantMax

English | 简体中文

Applicable Products

Product Support
Ascend 950PR&950DT products √
Atlas A3 products ×
Atlas A2 products ×
Atlas 200I/500 A2 inference products ×
Atlas inference products ×
Atlas training products ×

Function

  • Operator function: Perform grouped quantization on input x based on the input scale, and compute the maximum absolute value amax of input x group by group.

  • Formula:

    group_listgroup\_list splits xx along dim-0 into num_groups groups, and group_list[g]group\_list[g] is the cumulative size of the first g+1g+1 groups along dim-0 (monotonically increasing, group_list[−1]=x.shape[0]group\_list[-1]=x.shape[0]). For each group (g = 0, 1, ..., num_groups - 1):

    x_g=x[group_list[g−1]:group_list[g],...]x\_g = x[group\_list[g-1] : group\_list[g], ...]

    y=Cast(x_g×scale[g],dst_type,round_mode)y = \text{Cast}(x\_g \times scale[g], \text{dst\_type}, \text{round\_mode})

    amax[g]=max⁡(∣x_g∣)amax[g] = \max(|x\_g|)

    Here xx denotes the input tensor, and x_gx\_g denotes the g-th group of data of the input tensor (the lower bound is 0 when g=0). scalescale denotes the quantization scaling factor, amax[g]amax[g] denotes the maximum absolute value of the group of the input tensor, dst_type\text{dst\_type} specifies the output data type, and round_mode\text{round\_mode} specifies the rounding mode.

Parameters

Parameter name Input/Output/Attribute Description Data type Data format
x Input Input tensor to be quantized, corresponding to x in the formula. Any shape of 2D to 8D is supported. FLOAT, FLOAT16, BFLOAT16 ND
scale Input Quantization scaling factor, corresponding to scale in the formula. The shape is fixed to [num_groups]. FLOAT ND
group_list Input Offset of each group of input x along axis 0 (cumsum mode), corresponding to group_list in the formula. The shape is fixed to [num_groups]. INT64 ND
y Output Quantized output tensor, corresponding to y in the formula. The shape is the same as that of x, and the data type is determined by dst_type. HIFLOAT8, FLOAT8_E5M2, FLOAT8_E4M3 ND
amax Output Maximum absolute value of x, corresponding to amax in the formula. The shape is fixed to [num_groups], and the data type is the same as that of x. FLOAT, FLOAT16, BFLOAT16 ND
round_mode Attribute The value must be a valid rounding mode supported by the Cast API. string -
dst_type Attribute Specifies the output quantization data type. 34=HIFLOAT8, 35=FLOAT8_E5M2, and 36=FLOAT8_E4M3. int -

Constraints

  • Matching relationship between round_mode and dst_type:
    • When dst_type is 35 or 36, round_mode must be "rint".
    • When dst_type is 34, round_mode must be "round" or "hybrid".
  • The range of num_groups currently supports only 1-384.

Invocation Description

Invocation method Invocation sample Description
aclnn invocation test_aclnn_grouped_quant_max Invoke the GroupedQuantMax operator through the aclnnGroupedQuantMax API.
Graph-mode invocation - Invoke the GroupedQuantMax operator through graph construction using the operator IR.