已合并
修改公共文档的专项问题 #185
caiwenwen创建于 6 天前
修改公共文档的专项问题 #185
已合并
共 11 个文件变更+55-91
| @@ -47,9 +47,8 @@ Calculate a+b, actual result is: | |||
| 47 | 47 | ||
| 48 | When the data types of two inputs a and b that satisfy the broadcast relationship, or the deduced data types, are among COMPLEX64, COMPLEX128, DOUBLE, INT16, UINT16, or UINT64, in addition to satisfying the above broadcast rules, the following conditions must also be met, otherwise the broadcast will fail and cause the operator execution to report an error. | 48 | When the data types of two inputs a and b that satisfy the broadcast relationship, or the deduced data types, are among COMPLEX64, COMPLEX128, DOUBLE, INT16, UINT16, or UINT64, in addition to satisfying the above broadcast rules, the following conditions must also be met, otherwise the broadcast will fail and cause the operator execution to report an error. |
| 49 | 49 | ||
| 50 | -Condition: The merged dimension of consecutive axes that need broadcasting and consecutive axes that do not need broadcasting must be less than 6. | 50 | +- Condition: After merging consecutive axes that require broadcasting and consecutive axes that do not require broadcasting, the resulting number of segments must be less than 6. |
| 51 | - | 51 | +- Meaning: Mark each axis of the shapes involved in broadcasting one by one as requiring broadcasting or not requiring broadcasting (see the broadcasting rules above). Merge adjacent axes that have the same mark into one segment (consecutive axes that require broadcasting form one segment, and consecutive axes that do not require broadcasting form one segment). The total number of merged segments must be less than 6 (that is, no more than 5 segments); otherwise, broadcasting fails. |
| 52 | -Examples: | 52 | +- Example (indices start from 0): |
| 53 | - | 53 | + - When a.shape=(5, 1, 5, 1, 5, 1) and b.shape=(5, 5, 5, 5, 5, 5): axes 0, 2, and 4 do not require broadcasting, axes 1, 3, and 5 require broadcasting, and no adjacent axes with the same mark can be merged. The final number of dimensions is 6 segments, and broadcasting reports an error. |
| 54 | -- When a.shape=(5, 1, 5, 1, 5, 1) and b.shape=(5, 5, 5, 5, 5, 5), there are no axes that need to be merged, the final dimension is 6, and the broadcast reports an error. | 54 | + - When a.shape=(5, 1, 5, 5, 1, 1) and b.shape=(5, 5, 5, 5, 5, 5): axis 0 does not require broadcasting, axis 1 requires broadcasting, axes 2 and 3 do not require broadcasting (merged into one segment), and axes 4 and 5 require broadcasting (merged into one segment). After merging, the number of dimensions is 4, and broadcasting succeeds. |
| 55 | -- When a.shape=(5, 1, 5, 5, 1, 1) and b.shape=(5, 5, 5, 5, 5, 5), broadcasting is not needed in dimensions 2 and 3, and broadcasting is needed in dimensions 4 and 5. They are merged separately and continuously, and the merged dimension is 4, so the broadcast succeeds. | ||
| @@ -2,14 +2,14 @@ | |||
| 2 | 2 | ||
| 3 | When the **output aclTensor data type** of an API (such as aclnnAdd, aclnnMul, etc.) is inconsistent with the **calculation type after input data type promotion**, the API internally converts the calculation result to the data type corresponding to the output type. | 3 | When the **output aclTensor data type** of an API (such as aclnnAdd, aclnnMul, etc.) is inconsistent with the **calculation type after input data type promotion**, the API internally converts the calculation result to the data type corresponding to the output type. |
| 4 | 4 | ||
| 5 | +The data types that can be converted to and from each other include: | ||
| 6 | + | ||
| 7 | +- Floating-point types: ACL\_FLOAT16, ACL\_FLOAT, ACL\_DOUBLE, ACL\_BF16. | ||
| 8 | +- Integer types: ACL\_INT8, ACL\_UINT8, ACL\_INT16, ACL\_UINT16, ACL\_INT32, ACL\_UINT32, ACL\_INT64, ACL\_UINT64. | ||
| 9 | +- Complex types: ACL\_COMPLEX32, ACL\_COMPLEX64, ACL\_COMPLEX128. | ||
| 5 | Data type conversion must satisfy the following rules. Conversions that do not satisfy the rules cannot be performed, and parameter validation will fail when calling the API. | 10 | Data type conversion must satisfy the following rules. Conversions that do not satisfy the rules cannot be performed, and parameter validation will fail when calling the API. |
| 6 | - | 11 | +- Conversions between integer types are supported, as well as conversions to floating-point and complex types. |
| 7 | - - Floating-point types: ACL\_FLOAT16, ACL\_FLOAT, ACL\_DOUBLE, ACL\_BF16. | 12 | +- Conversions between floating-point types are supported, as well as conversions to integer and complex types. |
| 8 | - - Integer types: ACL\_INT8, ACL\_UINT8, ACL\_INT16, ACL\_UINT16, ACL\_INT32, ACL\_UINT32, ACL\_INT64, ACL\_UINT64. | 13 | +- Conversions between complex types are supported, as well as conversions to floating-point types. Among them, ACL\_COMPLEX32 currently supports conversion only with the ACL\_FLOAT16 floating-point type, and does not support conversion with other types. |
| 9 | - - Complex types: ACL\_COMPLEX64, ACL\_COMPLEX128. | 14 | +- BOOL supports conversion to integer, floating-point, and complex types. |
| 10 | - - Conversions between integer types are supported, as well as conversions to floating-point and complex types. | ||
| 11 | - - Conversions between floating-point types are supported, as well as conversions to complex types. | ||
| 12 | - - Conversions between complex types are supported. | ||
| 13 | - - BOOL supports conversion to integer, floating-point, and complex types. | ||
| 14 | - | ||
| 15 | Except for the above scenarios, other conversion scenarios are not supported. | 15 | Except for the above scenarios, other conversion scenarios are not supported. |
| @@ -56,4 +56,4 @@ Quantization modes (also known as quantization granularity) refer to the use of | |||
| 56 | - Pertensor-perchannel-pergroup quantization mode (abbreviated as T-CG quantization mode) | 56 | - Pertensor-perchannel-pergroup quantization mode (abbreviated as T-CG quantization mode) |
| 57 | - Perblock-perblock quantization mode (abbreviated as B-B quantization mode) | 57 | - Perblock-perblock quantization mode (abbreviated as B-B quantization mode) |
| 58 | - Pseudo-quantization: Generally refers to the mode in which only the weight matrix is quantized, including the perchannel quantization mode (abbreviated as C quantization mode). | 58 | - Pseudo-quantization: Generally refers to the mode in which only the weight matrix is quantized, including the perchannel quantization mode (abbreviated as C quantization mode). |
| 59 | -- MX quantization: Essentially Microscaling quantization, which maintains model precision at extremely low bits (such as 1 bit) by dynamically adjusting scaling factors. Here it refers to the pergroup-pergroup quantization mode (abbreviated as G-G quantization mode), which is a special case where the quantization parameter type is FLOAT8_E8M0 and the group size is 32. | 59 | +- MX quantization: Essentially Microscaling quantization, which maintains model precision at extremely low bits (such as 1 bit) by dynamically adjusting scaling factors. Here it refers to the pergroup-pergroup quantization mode (abbreviated as G-G quantization mode), which is a special case where the quantization parameter type is FLOAT8_E8M0 and the group size is 32. |
| @@ -31,4 +31,4 @@ The operator interface list is as follows: | |||
| 31 | | [aclnnObfuscationSetup](../../reliability/obfuscation_setup/docs/aclnnObfuscationSetup.md) | Completes the resource initialization and release of the PMCC model obfuscation engine. |Default deterministic implementation| - | | 31 | | [aclnnObfuscationSetup](../../reliability/obfuscation_setup/docs/aclnnObfuscationSetup.md) | Completes the resource initialization and release of the PMCC model obfuscation engine. |Default deterministic implementation| - | |
| 32 | | [aclnnObfuscationSetupV2](../../reliability/obfuscation_setup/docs/aclnnObfuscationSetupV2.md) | Completes the resource initialization and release of the PMCC model obfuscation engine. |Default deterministic implementation| - | | 32 | | [aclnnObfuscationSetupV2](../../reliability/obfuscation_setup/docs/aclnnObfuscationSetupV2.md) | Completes the resource initialization and release of the PMCC model obfuscation engine. |Default deterministic implementation| - | |
| 33 | | [aclnnCrypto](../../reliability/crypto/docs/aclnnCrypto.md) | Calls the aicpu encryption/decryption operator and executes according to the input parameters. |Default deterministic implementation| - | | 33 | | [aclnnCrypto](../../reliability/crypto/docs/aclnnCrypto.md) | Calls the aicpu encryption/decryption operator and executes according to the input parameters. |Default deterministic implementation| - | |
| 34 | -| [aclnnMatmulAbftVerify](../../reliability/matmul_abft_verify/docs/MatmulAbftVerify_en.md) | Performs V-ABFT fault detection on a precomputed matrix multiplication result and outputs per-row detection results. |Default deterministic implementation| - | | 34 | +| [aclnnMatmulAbftVerify](../../reliability/matmul_abft_verify/docs/aclnnMatmulAbftVerify_en.md) | Performs V-ABFT fault detection on a precomputed matrix multiplication result and outputs per-row detection results. |Default deterministic implementation| - | |
| @@ -1,59 +0,0 @@ | |||
| 1 | -# Quantization Introduction | ||
| 2 | - | ||
| 3 | -Quantization is widely used in deep learning models, especially during inference. Through quantization, models can run more efficiently on hardware, reducing computational resource consumption and accelerating the inference process, while also reducing the model's storage requirements. | ||
| 4 | - | ||
| 5 | -CANN operator quantization refers to the computational process of converting the input tensors of matrix (cube) operators such as Matmul in neural networks from high-bit to low-bit representation, while generating the corresponding quantization parameter scale. After the low-bit cube computation is complete, the quantization parameter scale can convert the low-bit values back to high-bit values, thereby ensuring the correctness of the overall computation results (the effect is approximately equivalent to directly using high-bit computation) and effectively improving computational efficiency. | ||
| 6 | - | ||
| 7 | -- Static quantization: Uses predetermined quantization parameters for quantization. In inference scenarios, the quantization of weights generally uses static quantization, which provides better operator performance. | ||
| 8 | -- Dynamic quantization: Uses input data to compute quantization parameters online for quantization. In inference scenarios, the quantization of activations generally uses dynamic quantization, which better adapts to data variations and provides higher precision. In training scenarios, dynamic quantization is also generally used to improve quantization precision. Note that dynamic quantization results in slightly worse operator performance because quantization parameters are generated online. | ||
| 9 | - | ||
| 10 | -## Quantization Modes | ||
| 11 | - | ||
| 12 | -Quantization modes (also known as quantization granularity) refer to the use of different quantization computation levels for different input tensors of an operator. Common quantization computation modes include: | ||
| 13 | - | ||
| 14 | -> Description: | ||
| 15 | -> | ||
| 16 | -> - The m, n, and k variables represent the sizes of different axes for tensor computation. | ||
| 17 | -> - The left matrix and right matrix refer to the two input tensors used for matrix multiplication computation in cube operators. Generally, the left matrix represents the activation and the right matrix represents the weight. Interpret and use them according to the actual situation. | ||
| 18 | - | ||
| 19 | -- Pertensor quantization (abbreviated as T quantization): The quantization target can be either the left matrix or the right matrix. Each tensor shares the same quantization parameter. | ||
| 20 | - | ||
| 21 | - Assuming the left matrix shape is (m, k) and the right matrix shape is (k, n), where k is the reduce axis, the shape of the generated quantization parameter is (1, ). | ||
| 22 | - | ||
| 23 | -  | ||
| 24 | - | ||
| 25 | -- Perchannel quantization (abbreviated as C quantization): The quantization target is the right matrix. Each channel uses an independent quantization parameter. | ||
| 26 | - | ||
| 27 | - Assuming the right matrix shape is (k, n), where k is the reduce axis, the shape of the generated quantization parameter is (n, ). | ||
| 28 | - | ||
| 29 | -  | ||
| 30 | - | ||
| 31 | -- Pertoken quantization (abbreviated as K quantization): The quantization target is the left matrix. Each token uses an independent quantization parameter. | ||
| 32 | - | ||
| 33 | - Assuming the left matrix shape is (m, k), where k is the reduce axis, the shape of the generated quantization parameter is (m, ). | ||
| 34 | - | ||
| 35 | -  | ||
| 36 | - | ||
| 37 | -- Pergroup quantization (abbreviated as G quantization): The quantization target can be either the left matrix or the right matrix. Data is grouped on the reduce axis, and each group uses an independent quantization parameter. | ||
| 38 | - - Assuming the left matrix shape is (m, k), where k is the reduce axis, data is grouped on the k axis with a group size of gs, and the shape of the generated quantization parameter is (m, k/gs). | ||
| 39 | - - Assuming the right matrix shape is (k, n), where k is the reduce axis, data is grouped on the k axis with a group size of gs, and the shape of the generated quantization parameter is (k/gs, n). | ||
| 40 | - | ||
| 41 | -  | ||
| 42 | - | ||
| 43 | -- Perblock quantization (abbreviated as B quantization): The quantization target can be either the left matrix or the right matrix. Data is divided into blocks on all axes, and each block uses an independent quantization parameter. | ||
| 44 | - | ||
| 45 | - - Assuming the left matrix shape is (m, k), where k is the reduce axis, data is grouped on the m and k axes by (bs, bs) blocks, where bs is the block size. The shape of the generated quantization parameter is (m/bs, k/bs). | ||
| 46 | - - Assuming the right matrix shape is (k, n), where k is the reduce axis, data is grouped on the k and n axes by (bs, bs) blocks, where bs is the block size. The shape of the generated quantization parameter is (k/bs, n/bs). | ||
| 47 | - | ||
| 48 | -  | ||
| 49 | - | ||
| 50 | -## Common Combined Quantization | ||
| 51 | - | ||
| 52 | -- Full quantization: Generally refers to the mode in which both the left and right matrices are quantized, including: | ||
| 53 | - - Pertensor-perchannel quantization mode (abbreviated as T-C quantization mode) | ||
| 54 | - - Pertoken-perchannel quantization mode (abbreviated as K-C quantization mode) | ||
| 55 | - - Pergroup-perblock quantization mode (abbreviated as G-B quantization mode) | ||
| 56 | - - Pertensor-perchannel-pergroup quantization mode (abbreviated as T-CG quantization mode) | ||
| 57 | - - Perblock-perblock quantization mode (abbreviated as B-B quantization mode) | ||
| 58 | -- Pseudo-quantization: Generally refers to the mode in which only the weight matrix is quantized, including the perchannel quantization mode (abbreviated as C quantization mode). | ||
| 59 | -- MX quantization: Essentially Microscaling quantization, which maintains model precision at extremely low bits (such as 1 bit) by dynamically adjusting scaling factors. Here it refers to the pergroup-pergroup quantization mode (abbreviated as G-G quantization mode), which is a special case where the quantization parameter type is FLOAT8_E8M0 and the group size is 32. | ||
| @@ -46,8 +46,12 @@ broadcast（广播）描述了算子在运算期间如何处理不同形状的 | |||
| 46 | ## 限制 | 46 | ## 限制 |
| 47 | 47 | ||
| 48 | 当满足broadcast关系的两个输入a和b的数据类型或推导后的数据类型在COMPLEX64、COMPLEX128、DOUBLE、INT16、UINT16、UINT64中时，除了满足上述广播规则，还需满足如下条件，否则广播会失败，导致算子执行报错。 | 48 | 当满足broadcast关系的两个输入a和b的数据类型或推导后的数据类型在COMPLEX64、COMPLEX128、DOUBLE、INT16、UINT16、UINT64中时，除了满足上述广播规则，还需满足如下条件，否则广播会失败，导致算子执行报错。 |
| 49 | -条件：连续的需要广播的轴和连续的不需要广播的轴合并之后的维度要求小于6。 | ||
| 50 | -举例： | ||
| 51 | 49 | ||
| 52 | -- 当a.shape=\(5, 1, 5, 1, 5, 1\)，b.shape=\(5, 5, 5, 5, 5, 5\)，没有需要合并的轴，最后维度为6，广播报错。 | 50 | +- 条件：连续的需要广播的轴和连续的不需要广播的轴合并之后的维度要求小于6。 |
| 53 | -- 当a.shape=\(5, 1, 5, 5, 1, 1\)，b.shape=\(5, 5, 5, 5, 5, 5\)，在第2和3维都不需要广播，4和5维都需要广播，分别连续合并，合并后的维度为4，广播成功。 | 51 | + |
| 52 | +- 含义：将参与广播的shape各轴逐个标记为“需要广播”或者“不需要广播”（广播规则如上）。相邻且标记相同的轴合并为一段（连续需要广播的轴合一段、连续不需要广播的轴合一段），合并后的总段数必须少于6（即不超过5段），否则广播失败。 | ||
| 53 | + | ||
| 54 | +- 举例（下标从0开始）： | ||
| 55 | + | ||
| 56 | + - 当a.shape=\(5, 1, 5, 1, 5, 1\)，b.shape=\(5, 5, 5, 5, 5, 5\)：轴0、2、4不需要广播，轴1、3、5需要广播，无相邻状态轴可合并，最后维度共6段，广播报错。 | ||
| 57 | + - 当a.shape=\(5, 1, 5, 5, 1, 1\)，b.shape=\(5, 5, 5, 5, 5, 5\)：轴0不需要广播，轴1需要广播，轴2、3维都不需要广播（合一段），轴4、5需要广播（合一段），合并后的维度为4，广播成功。 | ||
| @@ -2,14 +2,17 @@ | |||
| 2 | 2 | ||
| 3 | 当一个API（如aclnnAdd、aclnnMul等）**输出的aclTensor数据类型**与**输入的数据类型**推导后的计算类型不一致时，API内部就会将计算结果转换成输出类型对应的数据类型。 | 3 | 当一个API（如aclnnAdd、aclnnMul等）**输出的aclTensor数据类型**与**输入的数据类型**推导后的计算类型不一致时，API内部就会将计算结果转换成输出类型对应的数据类型。 |
| 4 | 4 | ||
| 5 | -数据类型转换需要满足以下规则，不满足规则的将不能进行转换，调用API时会出现参数校验失败。 | 5 | +互转换的数据类型包括： |
| 6 | 6 | ||
| 7 | - 浮点类型：ACL\_FLOAT16、ACL\_FLOAT、ACL\_DOUBLE、ACL\_BF16。 | 7 | - 浮点类型：ACL\_FLOAT16、ACL\_FLOAT、ACL\_DOUBLE、ACL\_BF16。 |
| 8 | - 整数类型：ACL\_INT8、ACL\_UINT8、ACL\_INT16、ACL\_UINT16、ACL\_INT32、ACL\_UINT32、ACL\_INT64、ACL\_UINT64。 | 8 | - 整数类型：ACL\_INT8、ACL\_UINT8、ACL\_INT16、ACL\_UINT16、ACL\_INT32、ACL\_UINT32、ACL\_INT64、ACL\_UINT64。 |
| 9 | -- 复数类型：ACL\_COMPLEX64、ACL\_COMPLEX128。 | 9 | +- 复数类型：ACL\_COMPLEX32、ACL\_COMPLEX64、ACL\_COMPLEX128。 |
| 10 | + | ||
| 11 | +转换需要满足以下规则，不满足规则的将不能进行转换，调用API时会出现参数校验失败。 | ||
| 12 | + | ||
| 10 | - 整数类型间可以转换，也支持往浮点、复数类型转换。 | 13 | - 整数类型间可以转换，也支持往浮点、复数类型转换。 |
| 11 | -- 浮点类型间可以转换，也支持往复数类型转换。 | 14 | +- 浮点类型间可以转换，也支持往整型、复数类型转换。 |
| 12 | -- 复数类型间可以转换。 | 15 | +- 复数类型间可以转换，也支持往浮点类型转换。其中ACL\_COMPLEX32当前仅支持与ACL\_FLOAT16浮点类型转换，不支持与其他类型转换。 |
| 13 | - BOOL支持往整数、浮点、复数类型转换。 | 16 | - BOOL支持往整数、浮点、复数类型转换。 |
| 14 | 17 | ||
| 15 | 除了以上场景，其他场景的转换均不支持。 | 18 | 除了以上场景，其他场景的转换均不支持。 |
| @@ -49,7 +49,7 @@ CANN算子量化是指对神经网络中Matmul等矩阵（cube）类算子的输 | |||
| 49 | 49 | ||
| 50 | ## 常见组合量化 | 50 | ## 常见组合量化 |
| 51 | 51 | ||
| 52 | -- 全量化：一般是指对左、右矩阵均进行量化的模式，包括 | 52 | +- 全量化：一般是指对左、右矩阵均进行量化的模式，包括： |
| 53 | - pertensor-perchannel量化模式（简称T-C量化模式） | 53 | - pertensor-perchannel量化模式（简称T-C量化模式） |
| 54 | - pertoken-perchannel量化模式（简称K-C量化模式） | 54 | - pertoken-perchannel量化模式（简称K-C量化模式） |
| 55 | - pergroup-perblock量化模式（简称G-B量化模式） | 55 | - pergroup-perblock量化模式（简称G-B量化模式） |
| @@ -31,4 +31,4 @@ | |||
| 31 | | [aclnnObfuscationSetup](../../reliability/obfuscation_setup/docs/aclnnObfuscationSetup.md) | 完成PMCC模型混淆引擎的资源初始化和释放。 |默认确定性实现| - | | 31 | | [aclnnObfuscationSetup](../../reliability/obfuscation_setup/docs/aclnnObfuscationSetup.md) | 完成PMCC模型混淆引擎的资源初始化和释放。 |默认确定性实现| - | |
| 32 | | [aclnnObfuscationSetupV2](../../reliability/obfuscation_setup/docs/aclnnObfuscationSetupV2.md) | 完成PMCC模型混淆引擎的资源初始化和释放。 |默认确定性实现| - | | 32 | | [aclnnObfuscationSetupV2](../../reliability/obfuscation_setup/docs/aclnnObfuscationSetupV2.md) | 完成PMCC模型混淆引擎的资源初始化和释放。 |默认确定性实现| - | |
| 33 | | [aclnnCrypto](../../reliability/crypto/docs/aclnnCrypto.md) | 调用aicpu加解密算子，按输入参数执行。 |默认确定性实现| - | | 33 | | [aclnnCrypto](../../reliability/crypto/docs/aclnnCrypto.md) | 调用aicpu加解密算子，按输入参数执行。 |默认确定性实现| - | |
| 34 | -| [aclnnMatmulAbftVerify](../../reliability/matmul_abft_verify/docs/MatmulAbftVerify.md) | 基于V-ABFT对预先计算的矩阵乘结果进行容错检测，并输出逐行检测结果。 |默认确定性实现| - | | 34 | +| [aclnnMatmulAbftVerify](../../reliability/matmul_abft_verify/docs/aclnnMatmulAbftVerify.md) | 基于V-ABFT对预先计算的矩阵乘结果进行容错检测，并输出逐行检测结果。 |默认确定性实现| - | |