FilesLast commitLast update
18 days ago
24 days ago
12 days ago
24 days ago
24 days ago
24 days ago
17 days ago
24 days ago
16 days ago
16 days ago
README

ReduceStdWithMean

English | 简体中文

Product Support Status

Product Supported
Ascend 950PR&950DT products √
Atlas A3 products √
Atlas A2 products √
Atlas 200I/500 A2 inference products √
Atlas inference products √
Atlas training products √

For Ascend 950PR/950DT, the kernel implementation is provided by this custom operator package (arch35). The other products are supported by the operator library built in CANN. Ascend 950PR/950DT, Atlas A3, and Atlas A2 support float16 / float32 / bfloat16. The other products support float16 / float32.

Function Description

Given the externally pre-computed mean mean, computes the standard deviation (invert=false) of the input x along the specified dimension dim or the reciprocal of the standard deviation (invert=true). It is a key sub-step for computing the statistics of the BatchNorm/LayerNorm/GroupNorm normalization layers, and is the underlying implementation of PyTorch torch.std/torch.std_mean/torch.batch_norm_stats on the NPU.

Standard deviation mode (invert=false, with correction taking effect):

y=∑j=0N−1(xj−meanj)2max⁡(0,  N−correction)y = \sqrt{\frac{\sum_{j=0}^{N-1}(x_j - mean_j)^2}{\max(0,\; N - correction)}}

Reciprocal-of-standard-deviation mode (invert=true, always dividing by N and adding epsilon, ignoring correction):

y=11N∑j=0N−1(xj−meanj)2+ϵy = \frac{1}{\sqrt{\dfrac{1}{N}\sum_{j=0}^{N-1}(x_j - mean_j)^2 + \epsilon}}

Here, N=∏d∈dimshape[d]N=\prod_{d \in dim}\text{shape}[d] is the total number of elements in the reduction dimensions, correctioncorrection is the Bessel correction factor (0=population, 1=sample), and ϵ\epsilon is a small value for numerical stability.

ReduceStdWithMean is an internal L0 operator of CANN. It is called in combination by the three upper-layer aclnn APIs aclnnStd, aclnnStdMeanCorrection, and aclnnBatchNormStats, and does not directly expose an independent aclnn API.

Upper-layer API Function correction invert eps
aclnnStd Standalone standard deviation Passed in by the user false —
aclnnStdMeanCorrection Outputs (std, mean) at the same time Passed in by the user false 0.001 (fixed)
aclnnBatchNormStats Outputs (mean, invstd) 0 (fixed) true Passed in by the user

Parameter Description

Parameter Name Input/Output/Attribute Description Data Type Data Format
x Input Tensor for computing the standard deviation. FLOAT16, FLOAT, BFLOAT16 ND
mean Input Externally pre-computed mean (already broadcast to the shape of x). Its dtype is the same as that of x. Same as x ND
y Output Output tensor. Its dtype is the same as that of x, and its shape is derived from dim + keepdim. Same as x ND
dim Optional attribute Reduction dimensions. The value range is [-rank(x), rank(x)-1]. Empty indicates all dimensions. The default value is {}. ListInt -
unbiased Optional attribute Bessel correction (legacy field). The default value is true. Bool -
keepdim Optional attribute Specifies whether to retain the reduced dimension as size 1. The default value is false. Bool -
invert Optional attribute If true, 1/sqrt(var+eps) is returned. If false, sqrt(var) is returned. The default value is false. Bool -
epsilon Optional attribute Term for numerical stability (used only when invert=true). It must be a non-negative floating-point number. The default value is 0.001. Float -
correction Optional attribute Correction factor: 0=population, 1=sample. The default value is 1. Int -

Constraint Description

  • The data types of x and mean must be the same, and only float16 / float32 / bfloat16 are supported. The output dtype is the same as that of x.
  • The number of shape dimensions of x does not exceed 8.
  • rank=0 scalar inputs are supported (x/mean is 0-D). If dim is empty, all dimensions are reduced and the output is a scalar (when keepdim=true, the output is 1-D with size 1).
  • The shape of mean must be exactly the same as that of x (mean must have been broadcast to the shape of x).
  • The value range of dim is [-rank(x), rank(x)-1]. Empty indicates reduction over all dimensions. dim cannot contain duplicate axes (duplicates are checked after negative axes are normalized).
  • The shape of the output y must be the same as the result derived from dim + keepdim.
  • correction supports only non-negative integers. When invert=true, correction is ignored and the division by N is always performed.
  • epsilon supports only non-negative floating-point numbers (the default value is 0.001). An error is returned for epsilon<0 in the tiling phase.
  • unbiased is a legacy field. If the correction attribute has a value, correction prevails. If it has no value, the value is converted as follows: unbiased=true→correction=1 and unbiased=false→correction=0.
  • An empty tensor (N=0) outputs NaN. When N<=correction (invert=false), +Inf or NaN is output (+Inf for general inputs, and 0×Inf=NaN for inputs whose elements are all equal).
  • When the input contains dynamic markers (-1/-2), the input is processed as a dynamic shape, and the specific semantics are defined by the downstream framework.

Invocation Description

Invocation Method Sample Code Description
GE graph-mode invocation test_geir_reduce_std_with_mean.cpp Invokes the operator by building a graph through GE IR. For the operator IR prototype, see op_graph/reduce_std_with_mean_proto.h
aclnn API test_aclnn_std_mean_correction.cpp Invokes the ReduceStdWithMean operator through the aclnnStdMeanCorrection API.
aclnn API test_aclnn_batch_norm_stats.cpp Invokes the ReduceStdWithMean operator through the aclnnBatchNormStats API.
aclnn API test_aclnn_std.cpp Invokes the ReduceStdWithMean operator through the aclnnStd API (the implementation of this API is in math/reduce_std_v2 in the same repository).