FilesLast commitLast update
10 days ago
2 months ago
4 days ago
4 days ago
4 days ago
4 days ago
9 days ago
1 month ago
17 days ago
README

DynamicMxQuant

English | 简体中文

Applicable Products

Product Support
Ascend 950PR&950DT products √
Atlas A3 products ×
Atlas A2 products ×
Atlas 200I/500 A2 inference products ×
Atlas inference products ×
Atlas training products ×

Function

  • Operator function: along the given axis, the operator computes the quantization scale mxscale corresponding to each group of blocksize elements, divides each element of the group by mxscale, converts the result to the corresponding dst_type according to round_mode, and obtains the quantized result y. When dst_type is FLOAT8_E4M3FN or FLOAT8_E5M2, the value of scale_alg specifies the different algorithms for computing mxscale.

Parameters

Parameter name Input/Output/Attribute Description Data type Data format
x Input Data to be quantized. FLOAT16, BFLOAT16 ND
axis Input Axis where the quantization occurs INT64 ND
round_mode Optional attribute Mode of the data conversion STRING ND
dst_type Input Specifies the type of y after the data conversion INT64 ND
blocksize Input Number of elements per quantization INT64 ND
scale_alg Input Computation method of mxscale INT64 ND
y Output Corresponding result after the input x is quantized FLOAT4_E2M1, FLOAT4_E1M2, FLOAT8_E4M3FN, FLOAT8_E5M2 ND
mxscale Output Quantization scale corresponding to each group FLOAT8_E8M0 ND

Constraints

  • The shape constraints of x and mxscale are as follows:
    • rank(mxscale) = rank(x) + 1.
    • axis_change = axis if axis >= 0 else axis + rank(x).
    • mxscale.shape[axis_change] = (ceil(x.shape[axis] / blocksize) + 2 - 1) / 2.
    • mxscale.shape[-1] = 2.
    • The other dimensions are consistent with the input x.

Invocation Description

Invocation method Invocation sample Description
aclnn invocation test_aclnn_dynamic_mx_quant Invokes the DynamicMxQuant operator through the aclnnDynamicMxQuant API.