Broadcast Relationship
Broadcast Concept
Broadcast describes how operators handle tensors (or arrays) of different shapes during computation. In most cases, tensors (or arrays) of different shapes are allowed to automatically expand their shapes during element-wise operations so that their dimensions become compatible with each other. Typically, the smaller tensor (or array) is "broadcast" to match the larger tensor (or array).
Currently, many CANN operator API parameters support broadcast in their shapes, which can improve computational efficiency and reduce memory usage (especially in large-scale data scenarios). For a more detailed introduction to broadcast techniques, refer to the NumPy official website.
Broadcast Rules
When performing broadcast computation, the following rules generally apply:
-
Rule 1: If the number of dimensions differs between arrays, all arrays are aligned to the array with the longest shape. Dimensions that are insufficient are padded with 1 on the left until the number of dimensions is equal.
NOTE:
- Example 1: The number of dimensions is the number of dimensions in the shape of a tensor (or array). For example, x.shape=(1,1,2,4) has a dimension count of 4.
- Example 2: For example, to compute a+b, where a.shape=(2, 2, 3) and b.shape=(2, 3), array b is broadcast to b.shape=(1, 2, 3).
-
Rule 2: If the number of dimensions is equal between arrays and a certain dimension of one array is 1, that array with the dimension of 1 will be stretched to match the corresponding dimension shape of the other array.
NOTE: In this use case, broadcasting along one dimension is sufficient. For example, to compute a+b, where a.shape=(1, 3) and b.shape=(3, 1), the two arrays are broadcast to a.shape=(3, 3) and b.shape=(3, 3).
-
Rule 3: Broadcasting fails with an error if dimension correspondence cannot be determined from the API convention or explicitly specified broadcast axes, or if any pair of aligned dimensions has unequal lengths and neither length is 1.
NOTE: For a.shape=(2,2,2) and b.shape=(2,2), broadcasting along axis 0, 1, or 2 corresponds to padding b to (1,2,2), (2,1,2), or (2,2,1), respectively, with different element mappings. If the API has no default alignment convention, the broadcast axis must be specified explicitly. With right alignment under Rule 1, b becomes (1,2,2) and broadcasting succeeds. For example, aligning a.shape=(2,2,2) and b.shape=(2,3) under Rule 1 leaves incompatible final dimensions of lengths 2 and 3, so broadcasting fails with an error.
Based on the above rules, the broadcast process generally first expands dimensions according to Rule 1, and then stretches shapes according to Rule 2. A specific example is as follows:
Suppose a.shape=(2,2,3). The values are as follows:
[[[1 2 3],[4 5 6]],
[[1 2 3],[4 5 6]]]
Suppose b.shape=(2,3). The values are as follows:
[[1 2 3],
[-1 -2 -3]]
After expanding dimensions according to Rule 1, b.shape=(1,2,3). The values are as follows:
[[[1 2 3],
[-1 -2 -3]]]
After stretching the shape according to Rule 2, b.shape=(2,2,3). The values are as follows:
[[[1 2 3],[-1 -2 -3]],
[[1 2 3],[-1 -2 -3]]]
The actual result of a+b is as follows:
[[[2 4 6],[3 3 3]],
[[2 4 6],[3 3 3]]]
Constraints
When the data types or the deduced data types of two inputs a and b that satisfy the broadcast relationship fall within COMPLEX64, COMPLEX128, DOUBLE, INT16, UINT16, or UINT64, the following condition must be met in addition to the broadcast rules described above. Otherwise, the broadcast fails and the operator execution reports an error. Condition: After merging consecutive axes that require broadcasting and consecutive axes that do not require broadcasting, the total number of resulting dimensions must be less than 6. Example:
- When a.shape=(5, 1, 5, 1, 5, 1) and b.shape=(5, 5, 5, 5, 5, 5), there are no axes to merge, and the final number of dimensions is 6. The broadcast reports an error.
- When a.shape=(5, 1, 5, 5, 1, 1) and b.shape=(5, 5, 5, 5, 5, 5), the 2nd and 3rd dimensions do not require broadcasting, while the 4th and 5th dimensions both require broadcasting. After merging consecutive axes respectively, the total number of dimensions is 4. The broadcast succeeds.