FilesLast commitLast update
6 days ago
3 months ago
6 days ago
6 days ago
6 days ago
19 days ago
6 days ago
6 days ago
README

DataCompare

English | 简体中文

Product Support Status

Product Supported
Ascend 950PR&950DT products √
Atlas A3 products √
Atlas A2 products √
Atlas 200I/500 A2 inference products √
Atlas inference products √
Atlas training products √

Function Description

  • Operator function: compares the two input tensors x1 and x2, which have the same shape and dtype, element by element, counts the total number of elements whose differences exceed the tolerance range (All Reduce), and outputs a FLOAT scalar indicating the total number of mismatched elements.

  • Computation formula:

    Element-wise mismatch judgment:

    diffi=∣x1i−x2i∣diff_i = |x1_i - x2_i|

    thresholdi=atol+rtol×∣x2i∣threshold_i = atol + rtol \times |x2_i|

    mismatchi={1.0,diffi>thresholdi0.0,diffi≤thresholdimismatch_i = \begin{cases} 1.0, & diff_i > threshold_i \\ 0.0, & diff_i \leq threshold_i \end{cases}

    Reduction output (total number of mismatched elements):

    output=∑imismatchi(dtype: FLOAT)output = \sum_{i} mismatch_i \quad (\text{dtype: FLOAT})

  • Output 0: all elements are within the tolerance range, and the two tensors are considered matched.

  • Output > 0: elements with differences exceeding the tolerance exist. A larger value indicates a greater difference.

Parameter Description

Parameter Name Input/Output/Attribute Description Data Type Data Format
x1 Input The first input tensor, which corresponds to x1 in the formula. Its shape and dtype must be exactly the same as those of x2, and broadcasting is not supported. Tensors of 0 to 8 dimensions and empty Tensors are supported. FLOAT, FLOAT16, BFLOAT16, INT8, UINT8, INT32 ND
x2 Input The second input tensor, which corresponds to x2 in the formula. Its shape and dtype must be exactly the same as those of x1, and broadcasting is not supported. Tensors of 0 to 8 dimensions and empty Tensors are supported. FLOAT, FLOAT16, BFLOAT16, INT8, UINT8, INT32 ND
num Output The total number of mismatched elements, which corresponds to output in the formula. Fixed as a FLOAT scalar (0D tensor). FLOAT ND
atol Attribute
  • Absolute tolerance, which corresponds to atol in the formula.
  • Must be >= 0.
  • The default value is 1e-5.
FLOAT -
rtol Attribute
  • Relative tolerance, which corresponds to rtol in the formula.
  • Must be >= 0.
  • The default value is 1e-3.
FLOAT -

Constraint Description

  • The shapes of x1 and x2 must be exactly the same, and broadcasting is not supported.
  • The dtypes of x1 and x2 must be exactly the same, and mixed dtypes are not supported.
  • Only the ND format is supported for x1, x2, and the output num; non-ND format inputs are rejected by validation.
  • For input types other than FLOAT (FLOAT16/BFLOAT16/INT8/UINT8/INT32), the operator first casts them to FLOAT (fp32) internally and then performs the entire comparison chain; both the comparison threshold and the difference are computed in fp32 precision.
  • The value range of atol/rtol is [0,1×1038][0, 1\times10^{38}], and out-of-range inputs are rejected by validation.
  • The reduction method is All Reduce (reduction over all axes), and users cannot specify the axis.
  • Dynamic shape/dynamic rank inputs are supported: graphs may be built with shapes containing unknown dims (-1); at execution time the shape must have been materialized, the operator performs the comparison based on the total number of elements after materialization, the output is always a scalar, and the semantics are identical to those of static shapes.
  • The output is fixed as a FLOAT scalar.
  • Deterministic implementation: the same input is guaranteed to produce the same output.
  • When the total number of input elements exceeds 2242^{24} (about 16M), the count may be inaccurate due to the limitation of the exact representation range of FLOAT.

Invocation Description

Invocation Method Sample Code Description
GE graph mode test_geir_data_compare Invokes the DataCompare operator by building a graph through the operator IR.