FilesLast commitLast update
19 days ago
1 month ago
1 month ago
22 days ago
1 month ago
18 days ago
22 days ago
1 month ago
17 days ago
README

WtsARQ

English | 简体中文

Applicable Products

Product Support
Ascend 950PR&950DT products √
Atlas A3 products √
Atlas A2 products √
Atlas 200I/500 A2 inference products √
Atlas inference products ×
Atlas training products ×

Function

WtsARQ (Weights Adaptive Range Quantization) is used in the AMCT quantization-aware training (QAT) scenario. Based on the input weight w and its quantization range lower bound w_min and upper bound w_max, it adaptively computes the quantization scale, performs round-to-nearest (half-to-even) fake quantization of w to the int8 grid, dequantizes the result back to floating point, and outputs y. The operator is delivered in GE IR graph mode and has no aclnn interface. Frontends such as TensorFlow call it after being mapped to the same IR through framework plugins.

Computation process (perform range correction first, and then compute the scale):

  1. Range correction: wmin=min(wmin,0)w_{min} = min(w_{min}, 0), wmax=max(wmax,0)w_{max} = max(w_{max}, 0)
  2. offset_flag=false (default, asymmetric abs-max): scale=max(∣wmin∣/128, wmax/127)scale = max(|w_{min}| / 128,\ w_{max} / 127)
  3. offset_flag=true (symmetric with offset): scale=wmax/255−wmin/255scale = w_{max}/255 - w_{min}/255, and compute offset=−rint(wmin/scale)−128offset = -rint(w_{min}/scale) - 128
  4. Division-by-zero protection: when scale<1.1920929×10−7scale < 1.1920929 \times 10^{-7} (float32 eps), take scale=1.0scale = 1.0
  5. Fake quantization (rint is round-to-nearest half-to-even): q=clamp(rint(w/scale)+offset, −128, 127)−offsetq = clamp(rint(w/scale) + offset,\ -128,\ 127) - offset, and offset=0offset = 0 when offset_flag=false
  6. Dequantization: y=q×scaley = q \times scale

Parameters

Parameter name Input/Output/Attribute Description Data type Data format
w Input The weight tensor to be fake-quantized. It is the computation object of the range correction and the quantization formulas. float16, float32 ND
w_min Input The lower bound of the weight range. It participates in the computation with restricted broadcast: w_min = min(w_min, 0) is executed first. float16, float32 ND
w_max Input The upper bound of the weight range. It participates in the computation with restricted broadcast: w_max = max(w_max, 0) is executed first. float16, float32 ND
y Output The weight after fake quantization and dequantization. Its shape and dtype are the same as w. float16, float32 ND
num_bits Optional attribute The quantization bit width. Currently only 8 is supported. The default value is 8. int64 -
offset_flag Optional attribute Specifies whether to use the symmetric quantization branch with offset. The default value is false. When it is true, the scale is computed as w_max/255 - w_min/255. bool -

Constraints

  • Both the inputs and the outputs are in ND format. Non-ND storage formats are rejected in the tiling stage.
  • The dtypes of w, w_min, w_max, and y must be consistent, and only float16 or float32 is supported.
  • The rank of w supports 0~8. rank=0 (scalar) is processed as a single-element tensor.
  • The ranks of w_min and w_max must equal the rank of w, and their shapes must be exactly the same. Each dimension must equal the corresponding dimension of w or be 1 (restricted broadcast).
  • The total number of elements of w must not exceed 2^31. For an empty tensor (with a zero-length dimension), the output is also empty, and no data is written back.
  • In dynamic shape scenarios, InferShape supports unknown dimension -1 and unknown rank -2. Illicit dimension relationships that can be statically determined are rejected, and the tiling stage requires all dimensions to be concrete.
  • num_bits supports only 8, and other values are rejected.

Invocation Description

Invocation method Invocation sample Description
Graph-mode invocation test_geir_wts_arq Invoke the WtsARQ operator by constructing a graph through the operator IR.