已合并
docs: add batch consistency and determinism intro #100
docs: add batch consistency and determinism intro #100
已合并
LiuZonggu创建于 9月14日
共 16 个文件变更+514-262
MREADME.md+18-4
@@ -1,10 +1,22 @@
1+<div align="center">
2+ 
1# ops-ras3# ops-ras
2 4 
5+[![CANNcommunity](https://img.shields.io/badge/docs-社区全量算子-brightgreen.svg?style=flat)](https://www.hiascend.com/document/redirect/CannCommunityOplist)
6+[![repo](https://img.shields.io/badge/docs-项目文档-green.svg?style=flat)](docs)
7+[![license](https://img.shields.io/badge/license-CANN_Open_2.0-blue.svg)](LICENSE)
8+[![contributing](https://img.shields.io/badge/贡献指南-teal)](CONTRIBUTING.md)
9+[![SIG](https://img.shields.io/badge/SIG-ops_basic-yellow)](https://gitcode.com/cann/community/blob/master/CANN/sigs/ops-basic/README.md)
10+ 
3简体中文 | [English](./README_en.md)11简体中文 | [English](./README_en.md)
4 12 
13+</div>
14+ 
15+***
16+ 
5## 🔥Latest News17## 🔥Latest News
6 18 
7-- [2026/07] ops-ras项目首次上线。19+- \[2026/07] ops-ras项目首次上线。
8 20 
9## 🚀概述21## 🚀概述
10 22 
@@ -23,14 +35,14 @@ ops-ras是[CANN](https://hiascend.com/software/cann)(Compute Architecture for
23 35 
24## ⬇️源码下载36## ⬇️源码下载
25 37 
26-环境准备好后,下载与CANN版本配套的分支源码,通用命令如下,\$\{tag\_version\}替换为分支标签名。以9.0.0分支源码下载为例:38+环境准备好后,下载与CANN版本配套的分支源码,通用命令如下,\${tag\_version}替换为分支标签名。以9.0.0分支源码下载为例:
27 39 
28```bash40```bash
29# 通用命令:git clone -b ${tag_version} https://gitcode.com/cann/ops-ras.git41# 通用命令:git clone -b ${tag_version} https://gitcode.com/cann/ops-ras.git
30git clone -b 9.0.0 https://gitcode.com/cann/ops-ras.git42git clone -b 9.0.0 https://gitcode.com/cann/ops-ras.git
31```43```
32 44 
33-> 说明:若环境中已存在配套分支源码,**可跳过本步骤**,例如CANNLab默认已提供最新最新版CANN对应的源码。45+> 说明:若环境中已存在配套分支源码,**可跳过本步骤**,例如CANNLab默认已提供最新版CANN对应的源码。
34 46 
35## 📖学习教程47## 📖学习教程
36 48 
@@ -44,8 +56,10 @@ git clone -b 9.0.0 https://gitcode.com/cann/ops-ras.git
44- [安全声明](SECURITY.md)56- [安全声明](SECURITY.md)
45- [许可证](LICENSE)57- [许可证](LICENSE)
46- [所属SIG](https://gitcode.com/cann/community/tree/master/CANN/sigs/ops-basic)58- [所属SIG](https://gitcode.com/cann/community/tree/master/CANN/sigs/ops-basic)
59+- [committer列表](https://gitcode.com/cann/community/blob/master/CANN/sigs/ops-basic/README.md#committer%E5%88%97%E8%A1%A8)
60+ 
61+***
47 62 
48------
49PS:本项目功能和文档正在持续更新和完善中,欢迎您关注最新版本。63PS:本项目功能和文档正在持续更新和完善中,欢迎您关注最新版本。
50 64 
51- **问题反馈**:通过GitCode[【Issues】](https://gitcode.com/cann/ops-ras/issues)提交问题。65- **问题反馈**:通过GitCode[【Issues】](https://gitcode.com/cann/ops-ras/issues)提交问题。
@@ -1,7 +1,19 @@
1+<div align="center">
2+ 
1# ops-ras3# ops-ras
2 4 
5+[![CANNcommunity](https://img.shields.io/badge/docs-Community_Operators-brightgreen.svg?style=flat)](https://www.hiascend.com/document/redirect/CannCommunityOplist)
6+[![repo](https://img.shields.io/badge/docs-Project_Docs-green.svg?style=flat)](docs)
7+[![license](https://img.shields.io/badge/license-CANN_Open_2.0-blue.svg)](LICENSE)
8+[![contributing](https://img.shields.io/badge/Contributing-teal)](CONTRIBUTING.md)
9+[![SIG](https://img.shields.io/badge/SIG-ops_basic-yellow)](https://gitcode.com/cann/community/blob/master/CANN/sigs/ops-basic/README.md)
10+ 
3English | [简体中文](./README.md)11English | [简体中文](./README.md)
4 12 
13+</div>
14+ 
15+---
16+ 
5## 🔥Latest News17## 🔥Latest News
6 18 
7- [2026/07] The ops-ras project was first released.19- [2026/07] The ops-ras project was first released.
@@ -44,6 +56,7 @@ git clone -b 9.0.0 https://gitcode.com/cann/ops-ras.git
44- [Security Statement](SECURITY.md)56- [Security Statement](SECURITY.md)
45- [License](LICENSE)57- [License](LICENSE)
46- [Affiliated SIG](https://gitcode.com/cann/community/tree/master/CANN/sigs/ops-basic)58- [Affiliated SIG](https://gitcode.com/cann/community/tree/master/CANN/sigs/ops-basic)
59+- [Committer List](https://gitcode.com/cann/community/blob/master/CANN/sigs/ops-basic/README.md#committer%E5%88%97%E8%A1%A8)
47 60 
48-----61-----
49PS: The functions and documentation of this project are being continuously updated and improved. We recommend that you follow the latest version.62PS: The functions and documentation of this project are being continuously updated and improved. We recommend that you follow the latest version.
@@ -10,3 +10,5 @@
10 - [Conversion Relationship](./conversion_relationship.md)10 - [Conversion Relationship](./conversion_relationship.md)
11 - [Quant more Introduction](./quant_more_introduction.md)11 - [Quant more Introduction](./quant_more_introduction.md)
12 - [Sparse Mode Introduction](./sparse_mode_introduction.md)12 - [Sparse Mode Introduction](./sparse_mode_introduction.md)
13+ - [Deterministic Computation](./determinism_compute.md)
14+ - [Batch Consistency](./batch_consistency.md)
@@ -0,0 +1,29 @@
1+# Batch Consistency
2+ 
3+## Introduction
4+ 
5+During application development, some operators may produce result deviations for the same Token across different batch sizes or at different positions within the same batch, in pursuit of higher performance.
6+Currently, for some operators, under the same runtime environment conditions, the computation process can be configured to adopt a Batch consistency algorithm, so that the computation results remain completely consistent no matter how the inputs are combined.
7+**Batch consistency algorithm**: For a given Token, the output must be bit-wise completely consistent, regardless of the position of the Token within the batch, the batch size, or which other Tokens are batched together with it.
8+ 
9+## Notes
10+ 
11+- It is generally recommended NOT to enable Batch consistency computation, because after Batch consistency computation is enabled for an operator, the relevant operators suffer a certain degree of performance degradation, and the single-run performance of the model may decrease. However, in scenarios such as experiments, debugging, and regression testing, where identical results across multiple runs are required to locate problems and experiment with algorithms, Batch consistency computation can improve efficiency.
12+ 
13+- The current configuration is a process-level switch.
14+ 
15+- **Version constraints**: TorchNPU version 26.2.0 or later, and CANN version 9.2.0 or later.
16+ 
17+## Usage
18+ 
19+Currently, the mainstream invocation methods for CANN operators are aclnn APIs and PyTorch APIs (torch_extension). Some operator APIs are Batch-consistent by default, while others are non-Batch-consistent by default. For operators with non-Batch-consistent implementations, some can be manually configured to enable Batch consistency.
20+ 
21+- **Invoking aclnn APIs**
22+ 
23+ In this scenario, enable Batch consistency through the "Runtime Configuration > aclrtSetSysParamOpt" interface (process-level) in the [Runtime API](https://hiascend.com/document/redirect/CannCommunityRuntimeApi). Specifically, set `ACL_OPT_DETERMINISTIC=3` to enable Batch consistency computation.
24+ 
25+- **Invoking PyTorch APIs**
26+ 
27+ In this scenario, enable Batch consistency computation through the `torch_npu.npu.set_deterministic_level` interface in the [TorchNPU Custom APIs](https://www.hiascend.com/document/detail/zh/Pytorch/latest/apiref/customapi/docs/zh/custom_APIs/overview.md).
28+ 
29+For operator APIs of different frameworks, refer to the specific aclnn API or PyTorch API documentation for their default Batch consistency policies.
@@ -0,0 +1,27 @@
1+# Deterministic Computation
2+ 
3+## Introduction
4+ 
5+During application development, the results of multiple runs of the same operator may not be completely consistent due to differences such as CANN versions or NPU models. Even under the same conditions with the same random seed, the results of multiple NPU executions may differ. This difference usually originates from asynchronous multi-threaded execution in the operator implementation, which causes the floating-point accumulation order to change.
6+ 
7+In certain special scenarios (such as experiments, debugging, and regression testing), operators need to produce identical outputs across multiple runs to facilitate problem locating or algorithm analysis. This solution is commonly referred to as "deterministic computation".
8+ 
9+## Notes
10+ 
11+- It is generally recommended NOT to enable deterministic computation, because deterministic computation of the same operator is usually slower than non-deterministic computation, which may degrade the single-run performance of the model. However, in scenarios such as experiments, debugging, and regression testing, where consistent results across multiple runs are required to locate problems or analyze algorithms, deterministic computation can improve efficiency.
12+ 
13+- Thread description: The deterministic state can be set only once in the same thread. If it is set multiple times, the last effective setting takes precedence. An effective setting means that after the deterministic state is set, an operator task is actually dispatched. If the state is set but no operator task is dispatched, the deterministic variable is enabled but not delivered to the operator, and the operator is not executed. Setting determinism multiple times in one thread is not recommended for now. This issue exists regardless of whether the binary is enabled or disabled, and will be resolved in a future version.
14+ 
15+## Usage
16+ 
17+Currently, the mainstream invocation methods for CANN operators are aclnn APIs and PyTorch APIs (torch_extension). Some operator APIs are deterministic by default, while others are non-deterministic by default. For operators with non-deterministic implementations, some can be manually configured to enable deterministic computation.
18+ 
19+- **Invoking aclnn APIs**
20+ 
21+ In this scenario, enable deterministic computation through the "Runtime Configuration > aclrtSetSysParamOpt" interface (process-level) in the [Runtime API](https://hiascend.com/document/redirect/CannCommunityRuntimeApi). Specifically, set `ACL_OPT_DETERMINISTIC=1` to enable deterministic computation.
22+ 
23+- **Invoking PyTorch APIs**
24+ 
25+ In this scenario, enable deterministic computation through the native PyTorch switch (`torch.use_deterministic_algorithms`).
26+ 
27+For operator APIs of different frameworks, refer to the specific aclnn API or PyTorch API documentation for their default deterministic computation policies.
@@ -38,12 +38,12 @@ The following uses [add_examples](../../../examples/add_example/) as an example
38 38 
39```bash39```bash
40# Note: Enter the project root directory and execute the following compilation command. The command is for reference only. For details, refer to the operator invocation instructions.40# Note: Enter the project root directory and execute the following compilation command. The command is for reference only. For details, refer to the operator invocation instructions.
41-bash build.sh --pkg --soc=Ascend950 --vendor_name=custom --ops=add_example41+bash build.sh --pkg --soc=ascend950 --vendor_name=custom --ops=add_example
42# Install the custom operator package42# Install the custom operator package
43./build_out/cann-ops-ras-${vendor_name}_linux-${arch}.run43./build_out/cann-ops-ras-${vendor_name}_linux-${arch}.run
44```44```
45 45 
46-* Complete the compilation of test_aclnn_add_example.cpp by following [aclnn Invocation](../invocation/op_invocation.md#aclnn-invocation), and generate the executable file test_aclnn_add_example.46+* Complete the compilation of test_aclnn_add_example.cpp by following [aclnn Invocation](../invocation/quick_op_invocation.md), and generate the executable file test_aclnn_add_example.
47 47 
48## Execute Simulation Command48## Execute Simulation Command
49 49 
@@ -51,27 +51,20 @@ bash build.sh --pkg --soc=Ascend950 --vendor_name=custom --ops=add_example
51cannsim record ./test_aclnn_add_example -s Ascend950 --gen-report51cannsim record ./test_aclnn_add_example -s Ascend950 --gen-report
52```52```
53 53 
54-The simulation tool execution log files are in the examples/add_example/examples/build/bin/cannsim_* directory. The execution log file is:54+The simulation tool execution log files are in the examples/add_example/examples/build/bin/cannsim_* directory. The execution log file is cannsim.log.
55- 
56-```bash
57-cannsim.log
58-```
59 55 
60From the simulation tool log file, you can see the print information in the sample:56From the simulation tool log file, you can see the print information in the sample:
61 57 
62```bash58```bash
63-add_example first input[0] is: 1.000000, second input[0] is: 1.000000, result[0] is: 2.00000059+mean result[0] is: 2.000000
64-add_example first input[1] is: 1.000000, second input[1] is: 1.000000, result[1] is: 2.00000060+mean result[1] is: 2.000000
65-add_example first input[2] is: 1.000000, second input[2] is: 1.000000, result[2] is: 2.00000061+mean result[2] is: 2.000000
66-add_example first input[3] is: 1.000000, second input[3] is: 1.000000, result[3] is: 2.00000062+...
67-add_example first input[4] is: 1.000000, second input[4] is: 1.000000, result[4] is: 2.000000
68-add_example first input[5] is: 1.000000, second input[5] is: 1.000000, result[5] is: 2.000000
69-add_example first input[6] is: 1.000000, second input[6] is: 1.000000, result[6] is: 2.000000
70```63```
71 64 
72## View Performance Pipeline65## View Performance Pipeline
73 66 
74-The simulation performance pipeline files are in the `examples/add_example/examples/build/bin/cannsim_*/report` directory of this project. The pipeline-related file is:67+The simulation performance pipeline files are in the `examples/add_example/examples/build/bin/cannsim_*/report/results/kernel_*/core_*` directory of this project. The pipeline-related file is:
75 68 
76```bash69```bash
77trace_core0.json70trace_core0.json
@@ -87,7 +80,7 @@ Execute the application in the simulation environment.
87 80 
88## Command Format81## Command Format
89 82 
90-cannsim record [options] user_app --user-options83+cannsim record [options] user_app
91 84 
92## Parameter Description85## Parameter Description
93 86 
@@ -95,22 +88,23 @@ Table 1 Simulation Execution Parameter Description
95 88 
96|Parameter|Required/Optional|Description|89|Parameter|Required/Optional|Description|
97| --- | --- | --- |90| --- | --- | --- |
98-|-s or --soc-version [options] parameter | Required | Specify the target chip version for simulation (for example: Ascend950).|91+|-s or --soc-version | Required | Specify the target chip version for simulation (for example: Ascend950).|
99-|-o or --output [options] parameter | Optional| The path where the generated files are stored. It can be configured as an absolute path or a relative path, and the user executing the tool must have read-write permissions. If the path is not specified, data is saved in the current directory by default.|92+|-o or --output | Optional | The path where the generated files are stored. It can be configured as an absolute path or a relative path, and the user executing the tool must have read-write permissions. If the path is not specified, data is saved in the current directory by default.|
100-|-g or --gen-report [options] parameter | Optional | Enable automatic analysis after simulation completion and generate an analysis report. By default, automatic analysis is not enabled.|93+|-g or --gen-report | Optional | Enable automatic analysis after simulation completion and generate an analysis report. By default, automatic analysis is not enabled.|
101-|user_app|Required|Operator executable file.|94+|-u or --user-option | Optional | User-defined operator parameters, which are passed to the operator program as command-line options.|
102-|--user-options|Optional|Running parameters of the operator executable file.|95+|-n or --core-id | Optional | AI Cores for which logs are enabled during simulation. The format is the same as that of report -n: 'all', '0-2,12-14', '5'. All cores are enabled by default. When -g is used and this parameter is not specified, it falls back to core 0.|
96+|user_app|Required|Operator program or command to be run (for example, ./app, python train.py, bash run.sh).|
103 97 
104## Usage Example98## Usage Example
105 99 
1061. Complete operator development and compilation.1001. Complete operator development and compilation.
1072. Execute the simulation command. Refer to the following usage examples:1012. Execute the simulation command. Refer to the following usage examples:
108 102 
109- ```text103+ ```bash
110- Method 1: Enable simulation and save the output to the ./output directory. /path/to/app is the operator program.104+ # Method 1: Enable simulation and save the output to the ./output directory. /path/to/app is the operator program.
111 $ cannsim record /path/to/app -o ./output -s Ascend950105 $ cannsim record /path/to/app -o ./output -s Ascend950
112 106 
113- Method 2: Enable simulation and generate a report for subsequent performance analysis.107+ # Method 2: Enable simulation and generate a report for subsequent performance analysis.
114 $ cannsim record /path/to/app -o ./output -s Ascend950 --gen-report108 $ cannsim record /path/to/app -o ./output -s Ascend950 --gen-report
115 ```109 ```
116 110 
@@ -118,12 +112,33 @@ Table 1 Simulation Execution Parameter Description
118 112 
119 ```text113 ```text
120 ├─cannsim_{timestamp}_${user_app}114 ├─cannsim_{timestamp}_${user_app}
121- ├── cannsim.log115+ ├── log
116+ │ ├── AIC_0_0_0_0_ChiWrap.log0
117+ │ ├── ccum_0_0_2.txt0
118+ │ ├── hha_0_0_0_states.log
119+ │ ├── L2Buf_0_0_0_0.txt0
120+ │ ├── L2cache_stats.log
121+ │ ├── lpddr_state0.log
122+ │ ├── ManyRing_0_0_0_0DatPerf.txt0
123+ │ ├── sdmam_0_0_0_debug.log0
124+ │ ├── sllc_chip_0_die_0_1_0_0_perf.log
125+ │ ├── STARS_ChiWrapper_0_1.log0
126+ │ ├── Tg_Log_File_0_0.txt
127+ │ ├── UB_0_0_2_0_ChiWrap.log0
128+ │ └── ub_log
129+ │ ├── UB_0_0_2_BA_0_statis.log
130+ ├── log_ca
131+ │ ├── core0.cubecore0_su_perf_summary_log
132+ │ ├── core0_summary_log
133+ │ ├── core0.wrapper_log.dump
134+ │ ├── mcu_log.dump
135+ │ ├── stars_log0_0.dump
136+ │ └── stars_log0_1.dump
122 ```137 ```
123 138 
124-4. You can obtain the operator execution results and compare the accuracy. The results are displayed in cannsim.log. An example is as follows:139+4. You can obtain the operator execution results and compare the accuracy. An example is as follows:
125 140 
126- The following output is only an example of the AscendC single-operator direct invocation accuracy comparison result. It may vary slightly depending on the version. Please refer to the actual output.141+ The following output is only an example of the Ascend C single-operator direct invocation accuracy comparison result. It may vary slightly depending on the version. Please refer to the actual output.
127 142 
128 ```bash143 ```bash
129 INFO:root:[INFO] compare data case[ case001]144 INFO:root:[INFO] compare data case[ case001]
@@ -150,9 +165,10 @@ Table 1 Simulation Result Analysis Parameter Description
150 165 
151|Parameter | Required/Optional | Description|166|Parameter | Required/Optional | Description|
152| --- | --- | --- |167| --- | --- | --- |
153-|-e or --export [options] parameter | Required | The original result file directory. It must be specified as the result directory generated after simulation execution, pointing to the cannsim_{timestamp}_${user_app} level. It can be configured as an absolute path or a relative path, and the tool execution user must have read-write permissions.|168+|-e or --export | Required | Simulation execution result directory, specified at the cannsim_{timestamp}_${user_app} level. It can be configured as an absolute path or a relative path, and the execution user must have read-write permissions.|
154-|-o or --output [options] parameter | Optional | The analysis result output directory. It can be configured as an absolute path or a relative path, and the execution user must have read-write permissions. If the path is not specified, data is saved in the current directory by default. If the generated result file has the same name as an existing file, the existing file is overwritten.|169+|-o or --output | Optional | Instruction pipeline output directory. It can be configured as an absolute path or a relative path, and the execution user must have read-write permissions. If no path is specified, it defaults to the export directory.|
155-|-n or --core-id [options] parameter | Optional | Specify the core ID for generating the instruction pipeline. If not specified, the pipeline for core 0 is generated by default. The configuration format is as follows: To generate pipelines for all cores, configure 'all'. To specify a core ID range, for example: '0-1'. To specify a single core ID, for example: '5'.|170+|-n or --core-id | Optional | Specify the core ID for generating the instruction pipeline. Supported formats: 'all', '0-2,12-14', '5'. If not specified, the instruction pipeline for core 0 is generated by default.|
171+|-f or --object-file | Optional | Device object file path, used to assist in report generation.|
156 172 
157## Usage Example173## Usage Example
158 174 
@@ -169,7 +185,7 @@ Table 1 Simulation Result Analysis Parameter Description
169 185 
1703. After the command execution completes, the corresponding pipeline files are generated in the output configured directory. The file format is JSON. The output result example is as follows:1863. After the command execution completes, the corresponding pipeline files are generated in the output configured directory. The file format is JSON. The output result example is as follows:
171 187 
172- ```bash188+ ```text
173 trace_core0.json189 trace_core0.json
174 trace_core1.json190 trace_core1.json
175 ...191 ...
@@ -189,7 +205,7 @@ Table 1 Simulation Result Analysis Parameter Description
189 |MTE1|Data transfer pipeline; data transfer direction: L1 ->{L0A/L0B, UBUF}.|205 |MTE1|Data transfer pipeline; data transfer direction: L1 ->{L0A/L0B, UBUF}.|
190 |MTE2|Data transfer pipeline; data transfer direction: {DDR/GM, L2} ->{L1, L0A/B, UBUF}.|206 |MTE2|Data transfer pipeline; data transfer direction: {DDR/GM, L2} ->{L1, L0A/B, UBUF}.|
191 |MTE3|Data transfer pipeline; data transfer direction: UBUF -> {DDR/GM, L2, L1}, L1->{DDR/L2}.|207 |MTE3|Data transfer pipeline; data transfer direction: UBUF -> {DDR/GM, L2, L1}, L1->{DDR/L2}.|
192- |FIXP|Data transfer pipeline; data transfer direction: FIXPIPE L0C -> OUT/L1.|208+ |FIXP|Data transfer pipeline; data transfer direction: FIXPIPE L0C -> OUT/L1. (Only the Atlas A2 training series products / Atlas A2 inference series products support the display.)|
193 |FLOWCTRL|Control flow instruction.|209 |FLOWCTRL|Control flow instruction.|
194 |ICACHELOAD|View ICache misses.|210 |ICACHELOAD|View ICache misses.|
195 211 
@@ -234,16 +250,28 @@ None
234 250 
235## Output Description251## Output Description
236 252 
237-```bash253+```text
238-usage: cannsim [-h] {record,report} ...254+Usage: cannsim [OPTIONS] COMMAND [ARGS]...
239 255 
240Command-line tool for performance simulation analysis on Ascend hardware.256Command-line tool for performance simulation analysis on Ascend hardware.
257+The simulation emulates real Ascend hardware behavior—including compute
258+units, memory hierarchy, and scheduling—enabling accurate performance
259+modeling without physical devices.
241 260 
242-positional arguments:261+Examples:
243- {record,report} Available commands262+$ cannsim record ./app -s Ascend910B -o ./output
244- record Run user application in AscendOps simulation environment263+$ cannsim report -e ./output/sim -o ./output/trace.json
245- report Generate performance analysis reports
246 264 
247-options:265+Note:
248- -h, --help show this help message and exit266+- Input app must be a valid AscendOps-built executable.
267+- Output directories are auto-created.
268+- `trace.json` is compatible with Chrome tracing tools.
269+- Advanced reporting features (e.g., HTML, diagrams) are planned but not yet available.
270+ 
271+Options:
272+--help Show this message and exit.
273+ 
274+Commands:
275+record Run user application in AscendOps simulation environment.
276+report Command-line interface for generating performance reports.
249```277```
@@ -1,190 +1,181 @@
1# Environment Deployment1# Environment Deployment
2 2 
3-Before performing [operator invocation](../invocation/quick_op_invocation.md) or [operator development](../develop/aicore_develop_guide.md) based on this project, complete the basic environment setup by following the steps below.3+Before performing the operations in the [Learning Tutorials](../../../README_en.md#learning-tutorials), please complete the basic environment setup and source code download by following the steps below, and ensure that the NPU driver, firmware, and CANN software (`Ascend-cann-toolkit` and `Ascend-cann-ops`) have been installed.
4 4 
5-Note that the meanings of compilation and runtime scenarios mentioned in this document are as follows. Install as needed:5+## Environment Installation
6 6 
7-- Compilation scenario: For scenarios where only compilation without running this project is required, you only need to install the prerequisite dependencies and the CANN toolkit package.7+This project provides multiple ways to set up the Ascend environment. Choose as needed.
8-- Runtime scenario: For scenarios where this project is run (compilation and running or pure running), in addition to installing the prerequisite dependencies and the CANN toolkit package, you also need to install the driver and firmware, and the CANN ops package.
9 8 
10-## Prerequisites9+> **Note**: The compilation and runtime scenarios mentioned in this document are defined as follows. Choose based on your actual situation.
10+>
11+> - Compilation: For scenarios where only this project is compiled without running, you only need to install the CANN toolkit package.
12+> - Runtime: For scenarios where this project is run (compilation and running, or pure running), you need to install the driver and firmware, the CANN toolkit package, and the CANN ops package.
11 13 
12-Before using this project, ensure that the following basic dependencies, NPU driver, and firmware are installed.14+| Installation Method | Description | Applicable Scenario |
15+| ----- | ------ | ------ |
16+| CANNLab | One-stop development platform that provides an online Ascend environment that can be run directly, without manual installation.<br>Currently, single-node computing power is available, and the **latest CANN package is installed by default**. | Suitable for developers without Ascend devices.|
17+| Docker | Docker images are an efficient deployment method with CANN packages and required dependencies pre-integrated.<br>Currently applicable to Atlas A2 and A3 series products. Supported OSs: ubuntu22.04 and openeuler24.03. The **latest CANN package is installed by default**. |Suitable for developers who have Ascend devices and need to quickly set up an environment.|
18+| Manual Installation | Manually install CANN packages and basic dependencies, with high flexibility. |Suitable for developers who have Ascend devices and want to manually install CANN packages or experience the latest master branch capabilities.|
13 19 
14-1. **Install Dependencies**20+### Method 1: CANNLab
15 21 
16- The dependencies used for source code compilation of this project are as follows. Please note the version requirements.22+For developers without Ascend devices, you can directly use the CANNLab cloud development environment, that is, the "**one-stop development platform**". This platform provides an online Ascend environment that can be run directly. The environment has the required drivers, firmware, software packages, and dependencies installed, so manual installation is not required.
17 23 
18- - python >= 3.7.0 (recommended version <= 3.10)24+> **Note**: The environment installs the latest CANN package by default. When downloading the source code, ensure that it matches the software version. For more information about the development platform, refer to the [CANNLab Guide](https://gitcode.com/cann/cann-learning-hub/blob/master/docs/CANNLab_env_experience_guide.md).
19- - gcc >= 7.3.0
20- - cmake >= 3.16.0
21- - pigz (optional, installing it can improve packaging speed, recommended version >= 2.4)
22- - dos2unix
23- - gawk
24- - make
25 25 
26- The above dependency packages can be installed through the install\_deps.sh script in the project root directory. The command is as follows. If you encounter an unsupported system, refer to the file to adapt it yourself.26+1. Enter the open source project and click the "`CANNLab`" button. Log in with a certified Huawei Cloud account. If you have not registered or certified, please register and certify as prompted.
27 27 
28- ```bash28+ <img src="../figures/cloudIDE.png" alt="Cloud Platform" width="750px" height="85px">
29- bash install_deps.sh
30- ```
31 29 
32-2. **Install Driver and Firmware (Runtime Dependency)**30+2. Create an NPU environment and configure specifications as prompted. After the cloud development environment is started, click "`Connect > WebIDE`" to enter the one-stop development platform. By default, the resources of this open source project are stored in the `/mnt/workspace/gitCode` directory.
33- 
34- When running operators, you must install the driver and firmware. If you are only compiling operators, you can skip this operation.
35- 
36- Click [download link](https://www.hiascend.com/hardware/firmware-drivers/community) to obtain the corresponding `Ascend-hdk-<chip_type>-npu-driver_<version>_linux-<arch>.run` and `Ascend-hdk-<chip_type>-npu-firmware_<version>.run` packages according to the actual product model and environment architecture.
37- 
38- For installation instructions, refer to [CANN Software Installation Guide](https://www.hiascend.com/document/detail/en/CANNCommunityEdition/latest/softwareinst/instg/instg_0000.html?OS=openEuler&InstallType=netyum).
39- 
40-## Environment Preparation (Choose One of Three)
41- 
42-This project provides multiple ways to deploy CANN packages. Choose as needed.
43- 
44-- WebIDE and Docker environment: Provides minimal environment setup, **default installation of the latest commercial release CANN software package** (currently CANN 8.5.0).
45-- Manual installation of CANN package: If you want to experience manual installation of CANN package or experience the latest master branch capabilities, manual installation is recommended.
46- 
47-### Using WebIDE Environment
48- 
49-For users without an environment, you can directly use the WebIDE development platform, that is, the "**Operator One-stop Development Platform**". This platform provides an online Ascend environment that can be run directly. The environment has installed the necessary software packages, and no manual installation is required. For more information about the development platform, refer to [LINK](https://gitcode.com/org/cann/discussions/54).
50- 
51-1. Enter the open source project and click the "`Cloud Development`" button. Log in with a certified Huawei Cloud account. If you have not registered or certified, please register and certify according to the page prompts.
52- 
53- <img src="../figures/cloudIDE.png" alt="Cloud Platform" width="750px" height="90px">
54- 
55-2. Create and start the cloud development environment according to the page prompts. Click "`Connect > WebIDE`" to enter the operator one-stop development platform. The resources of the open source project are in the `/mnt/workspace` directory by default.
56 31 
57 <img src="../figures/webIDE.png" alt="Cloud Platform" width="1000px" height="150px">32 <img src="../figures/webIDE.png" alt="Cloud Platform" width="1000px" height="150px">
58 33 
59-### Using Docker Deployment34+### Method 2: Docker Deployment
60 35 
61-> **Note:**36+For developers with Ascend devices who want to quickly set up an Ascend environment, Docker image deployment is recommended.
37+ 
38+> **Note**:
62>39>
63-> - Docker image is an efficient deployment method. Currently, it is only applicable to Atlas A2 series products and only adapted to the Ubuntu operating system.40+> - The image file is relatively large and takes some time to download. Please wait patiently. For options of docker commands, run `docker --help`.
64-> - The image file is relatively large, and downloading takes some time. Please wait patiently.41+> - The environment installs the latest CANN package by default. When downloading the source code, ensure that it matches the software version.
65 42 
66-#### 1. Download Image43+1. **Install the driver (runtime dependency)**
67 44 
68-1. Log in to the host machine as the root user. Ensure that the Docker engine (version 1.11.2 or above) is installed on the host machine.45+ The driver is a runtime dependency and can be skipped if you only compile operators. Run `npu-smi info` to check whether NPU information is displayed. If no, install the driver by referring to [CANN Quick Installation](https://www.hiascend.com/en/cann/download):
69-2. Pull the image with the CANN software package and operator-development dependencies pre-integrated from the [Ascend Image Repository](https://www.hiascend.com/developer/ascendhub/detail/17da20d1c2b6493cb38765adeba85884). The command is as follows. Choose according to the actual architecture:46+ 
47+ - Step 1: On the page, select your product series, CPU architecture, and operating system, and select **Online Installation (Yum)** as the installation mode.
48+ - Step 2: Follow the instructions on the page to complete the three procedures: **Configuring the user group**, **Installing dependencies and configuring the source**, and **Installing the NPU driver**.
49+ - Step 3: Run `npu-smi info`. If NPU device information is properly displayed, the driver is successfully installed.
50+ 
51+2. **Download the image**
52+ 
53+ - Step 1: Log in to the host as the root user. Ensure that Docker Engine (v1.11.2 or later) has been installed on the host. Run `docker --version` to check the Docker version. If Docker is not installed, refer to the [Docker official installation guide](https://docs.docker.com/engine/install/).
54+ - Step 2: Pull the image with the CANN software packages and dependencies required for operator development pre-integrated from the [Ascend image repository](https://www.hiascend.com/developer/ascendhub/detail/17da20d1c2b6493cb38765adeba85884).
55+ 
56+ The following is an example. Replace the CANN version, chip series, operating system, and Python version as required. For supported values of each field, see the Ascend image repository page.
70 57 
71 ```bash58 ```bash
72- # Example: Pull ARM architecture CANN development image59+ # Use cann:9.1.0-beta.1 as an example
73- docker pull --platform=arm64 swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:8.5.0-910b-ubuntu22.04-py3.10-ops60+ docker pull swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:9.1.0-beta.1-910b-ubuntu22.04-py3.12-devel
74- # Example: Pull X86 architecture CANN development image
75- docker pull --platform=amd64 swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:8.5.0-910b-ubuntu22.04-py3.10-ops
76 ```61 ```
77 62 
78-#### 2. Run Docker63+ > **Note**: The image tag format is `<CANN version>-<chip series>-<OS>-<Python version>-devel`. Images with the `-devel` suffix are operator development images, which contain the dependencies for operator development and compilation.
79 64 
80-After pulling the image, you need to start the container with specific parameters so that the container can access the host's Ascend device.65+3. **Run Docker**
81 66 
82-```bash67+ After pulling the image, start the container with specific parameters so that the container can access the Ascend devices on the host.
83-docker run --name cann_container --device /dev/davinci0 --device /dev/davinci_manager --device /dev/devmm_svm --device /dev/hisi_hdc -v /usr/local/dcmi:/usr/local/dcmi -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi -v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ -v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info -v /etc/ascend_install.info:/etc/ascend_install.info -it swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:8.5.0-910b-ubuntu22.04-py3.10-ops bash
84-```
85- 
86-| Parameter | Description | Notes |
87-| :--- | :--- | :--- |
88-| `--name cann_container` | Specifies a name for the container for easy management. | Can be customized. |
89-| `--device /dev/davinci0` | Core: Maps the host's NPU device card to the container. Multiple NPU device cards can be specified. | Must be adjusted according to the actual situation: `davinci0` corresponds to the 0th NPU card in the system. Please execute the `npu-smi info` command on the host first, and modify this number according to the device number displayed in the output (such as `NPU 0`, `NPU 1`).|
90-| `--device /dev/davinci_manager` | Maps the NPU device management interface. | |
91-| `--device /dev/devmm_svm` | Maps the device memory management interface. | |
92-| `--device /dev/hisi_hdc` | Maps the communication interface between host and device. | |
93-| `-v /usr/local/dcmi:/usr/local/dcmi` | Mounts the device container management interface (DCMI) related tools and libraries. | |
94-| `-v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi` | Mounts the `npu-smi` tool. | Enables running this command directly in the container to query NPU status and performance information.|
95-| `-v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/` | Key mount: Maps the host's NPU driver library to the container. | |
96-| `-v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info` | Mounts the driver version information file. | |
97-| `-v /etc/ascend_install.info:/etc/ascend_install.info` | Mounts the CANN software installation information file. | |
98-| `-it` | Combination parameter of `-i` (interactive) and `-t` (allocate pseudo terminal). | |
99-| `swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:8.5.0-910b-ubuntu22.04-py3.10-ops` | Specifies the Docker image to run. | Please ensure that this image name and tag are exactly the same as the image you pulled through `docker pull`. |
100-| `bash` | The command executed immediately after the container starts. | |
101- 
102-### Manual Installation of CANN Package
103- 
104-#### 1. Download Software Package
105- 
106-Obtain `Ascend-cann-toolkit_${cann_version}_linux-${arch}.run` and `Ascend-cann-${soc_name}-ops_${cann_version}_linux-${arch}.run` according to the following scenarios.
107- 
108-- Scenario 1: If you want to experience the **officially released CANN package** capabilities, visit the [CANN Official Download Center](https://www.hiascend.com/en/cann/download?versionId=731&ids=d806%2Ch0502%2Ch0601%2Ch0702), select the corresponding version of the CANN software package (only CANN 8.5.0 and later versions are supported). For installation instructions, refer to [CANN Software Installation Guide](https://www.hiascend.com/document/detail/en/CANNCommunityEdition/latest/softwareinst/instg/instg_0000.html?OS=openEuler&InstallType=netyum).
109- 
110-- Scenario 2: If you want to experience the **latest master branch capabilities**, click [download link](https://ascend.devcloud.huaweicloud.com/artifactory/cann-run-release/software/master) to obtain.
111- 
112-Note that the product model and environment architecture must correspond to the actual environment. In addition, the ops package is a runtime dependency. If you are only compiling operators, you can skip installing this package.
113- 
114-#### 2. Install Software Package
115- 
116-1. **Install Community CANN Toolkit Package**
117 68 
118 ```bash69 ```bash
119- # Ensure the installation package has executable permission70+ docker run --name cann_container --device /dev/davinci0 --device /dev/davinci_manager --device /dev/devmm_svm --device /dev/hisi_hdc -v /usr/local/dcmi:/usr/local/dcmi -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi -v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ -v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info -v /etc/ascend_install.info:/etc/ascend_install.info -it swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:9.1.0-beta.1-910b-ubuntu22.04-py3.12-devel bash
120- chmod +x Ascend-cann-toolkit_${cann_version}_linux-${arch}.run
121- # Installation command
122- ./Ascend-cann-toolkit_${cann_version}_linux-${arch}.run --install --force --install-path=${install_path}
123 ```71 ```
124 72 
125- - $\{cann\_version\}: Represents the CANN package version number.73+ > **Note**: `--name` specifies the container name. Replace it with a recognizable custom name. If the name is already in use, the container fails to start.
126- - $\{arch\}: Represents the CPU architecture, such as aarch64, x86_64.
127- - $\{install\_path\}: Represents the specified installation path. The default installation is in the `/usr/local/Ascend` directory.
128 74 
129-2. **Install Community CANN Ops Package (Runtime Dependency)**75+ | Parameter | Description | Notes |
76+ | :--- | :--- | :--- |
77+ | `--name cann_container` | Specifies a name for the container for easy management. | Can be customized. |
78+ | `--device /dev/davinci0` | Key configuration: maps the NPU device card of the host to the container. Multiple NPU device cards can be mapped. | Must be adjusted based on the actual situation: `davinci0` corresponds to the 0th NPU card in the system. Run the `npu-smi info` command on the host first, and modify the number based on the device number displayed in the output (such as `NPU 0` and `NPU 1`).|
79+ | `--device /dev/davinci_manager` | Maps the NPU device management interface. | - |
80+ | `--device /dev/devmm_svm` | Maps the device memory management interface. | - |
81+ | `--device /dev/hisi_hdc` | Maps the communication interface between the host and the device. | - |
82+ | `-v /usr/local/dcmi:/usr/local/dcmi` | Mounts the tools and libraries related to the device container management interface (DCMI). | - |
83+ | `-v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi` | Mounts the `npu-smi` tool. | Enables you to run this command in the container to query NPU status and performance information.|
84+ | `-v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/` | Key mount: maps the NPU driver library of the host to the container. | - |
85+ | `-v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info` | Mounts the driver version information file. | - |
86+ | `-v /etc/ascend_install.info:/etc/ascend_install.info` | Mounts the CANN software installation information file. | - |
87+ | `-it` | Combination of `-i` (interactive) and `-t` (allocating a pseudo terminal). | - |
88+ | `swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:9.1.0-beta.1-910b-ubuntu22.04-py3.12-devel` | Specifies the Docker image to run. | Ensure that the image name and tag are exactly the same as those of the image you pulled using `docker pull`. |
89+ | `bash` | Command executed immediately after the container starts. | - |
130 90 
131- When running operators, you must install this package. If you are only compiling operators, you can skip this operation.91+### Method 3: Manual Installation
92+ 
93+For developers with Ascend devices who want to manually set up an Ascend environment, refer to the following steps.
94+ 
95+#### Installing Software
96+ 
97+- **Scenario 1: Experiencing the latest master capabilities or developing based on the CANN weekly version**
98+ 
99+ Refer to [CANN Quick Installation](https://www.hiascend.com/en/cann/download), select `Weekly version`, download the corresponding package based on the product series, CPU architecture, and operating system, and complete the installation by running the commands provided on the page.
100+ 
101+- **Scenario 2: Experiencing branch capabilities or developing based on the CANN stable version**
102+ 
103+ Refer to [CANN Quick Installation](https://www.hiascend.com/en/cann/download), select `Stable version` (only CANN 8.5.0 and later versions are supported), download the corresponding package based on the product series, CPU architecture, and operating system, and complete the installation by running the commands provided on the page.
104+ 
105+#### Installing Basic Dependencies
106+ 
107+The basic dependencies of this project are as follows. Ensure that the version requirements are met.
108+ 
109+- python >= 3.7.0 (recommended version <= 3.10)
110+- gcc >= 7.3.0
111+- cmake >= 3.16.0
112+- pigz (optional, installing it can improve packaging speed, recommended version >= 2.4)
113+- dos2unix
114+- gawk
115+- make
116+- patch
117+- googletest (required only when executing UT, recommended version [release-1.11.0](https://github.com/google/googletest/releases/tag/release-1.11.0))
118+ 
119+The preceding dependencies can be installed at once using the project script. The procedure is as follows:
120+ 
121+1. Download the source code.
122+ 
123+ Download the branch source code matching the CANN version. The command is as follows. Replace `${tag_version}` with the branch tag name.
132 124 
133 ```bash125 ```bash
134- # Ensure the installation package has executable permission126+ git clone -b ${tag_version} https://gitcode.com/cann/ops-ras.git
135- chmod +x Ascend-cann-${soc_name}-ops_${cann_version}_linux-${arch}.run
136- # Installation command
137- ./Ascend-cann-${soc_name}-ops_${cann_version}_linux-${arch}.run --install --install-path=${install_path}
138 ```127 ```
139 128 
140- - $\{soc\_name\}: Represents the NPU model name.129+2. Install the dependencies.
141- - $\{install\_path\}: Represents the specified installation path. It needs to be installed in the same path as the toolkit package. The default installation is in the `/usr/local/Ascend` directory.130+ 
131+ First, run `install_deps.sh` in the project root directory to install the preceding dependencies at once. The command is as follows. If your system is not supported, refer to the file and adapt it yourself.
132+ 
133+ ```bash
134+ bash install_deps.sh
135+ ```
136+ 
137+ Then, run `requirements.txt` in the project root directory to install the third-party Python library dependencies. The command is as follows.
138+ 
139+ ```bash
140+ pip3 install -r requirements.txt
141+ ```
142 142 
143## Environment Verification143## Environment Verification
144 144 
145-After installing the CANN package, verify that the environment and driver are normal.145+After the CANN packages are installed, verify that the environment and driver are normal.
146 146 
147-- **Check NPU Device**:147+- **Check the NPU device**
148 148 
149 ```bash149 ```bash
150- # Run npu-smi. If device information is displayed normally, the driver is normal150+ # Run npu-smi. If device information is properly displayed, the driver is normal
151 npu-smi info151 npu-smi info
152 ```152 ```
153 153 
154-- **Check CANN Installation**:154+- **Check the CANN version**
155 155 
156 ```bash156 ```bash
157- # View CANN Toolkit version information (default path installation)157+ # View the CANN toolkit package version information (default installation path)
158- cat /usr/local/Ascend/ascend-toolkit/latest/opp/version.info158+ # Docker and manual installation scenarios:
159+ cat /usr/local/Ascend/cann/${arch}-linux/ascend_toolkit_install.info
160+ # CANNLab scenario:
161+ cat /home/developer/Ascend/cann/${arch}-linux/ascend_toolkit_install.info
162+ 
163+ # View the CANN ops package version information (default installation path)
164+ # Docker and manual installation scenarios:
165+ cat /usr/local/Ascend/cann/${arch}-linux/ascend_ops_install.info
166+ # CANNLab scenario:
167+ cat /home/developer/Ascend/cann/${arch}-linux/ascend_ops_install.info
159 ```168 ```
160 169 
170+ `${arch}` indicates the current architecture, which can be queried using `uname -m`, for example, aarch64 or x86_64.
171+ 
161## Environment Variable Configuration172## Environment Variable Configuration
162 173 
163-Choose the appropriate command to make the environment variables effective as needed.174+Select the appropriate command as needed to make the environment variables take effect.
164 175 
165```bash176```bash
166-# Default path installation, taking root user as an example (for non-root users, replace /usr/local with ${HOME})177+# Default installation path, taking the root user as an example (for non-root users, replace /usr/local with ${HOME})
167source /usr/local/Ascend/cann/set_env.sh178source /usr/local/Ascend/cann/set_env.sh
168-# Specified path installation179+# Specified installation path
169# source ${install_path}/cann/set_env.sh180# source ${install_path}/cann/set_env.sh
170```181```
171- 
172-## Source Code Download
173- 
174-Download the project source code through the following command, and install other dependencies. Replace $\{tag\_version\} with the branch tag name. The matching relationship between this source code repository and the CANN version can be found in the [release repository](https://gitcode.com/cann/release-management).
175- 
176-```bash
177-# Download the corresponding branch source code of the project
178-git clone -b ${tag_version} https://gitcode.com/cann/ops-ras.git
179-# Install root directory requirements.txt dependencies
180-cd ops-ras
181-pip3 install -r requirements.txt
182-```
183- 
184-> [!NOTE] Note
185-> When using the HTTPS protocol on the gitcode platform, you need to configure and use a personal access token instead of the login password for cloning, pushing, and other operations.
186- 
187-If your compilation environment cannot access the network and cannot download the code through the `git` command, you need to download the source code in a networked environment and manually upload it to the target environment.
188- 
189-- In a networked environment, enter [this project homepage](https://gitcode.com/cann/ops-ras), and complete the source code download through the `Download ZIP` or `clone` button according to the instructions.
190-- Connect to the offline environment and upload the source code to your specified directory. If you downloaded a source code compressed package, you also need to decompress it.
@@ -17,12 +17,9 @@ Here, `${INSTALL_DIR}` represents the path where CANN is installed, and `${ops_p
17 17 
18## Interface List18## Interface List
19 19 
20-> **Determinism Description**:20+> [!NOTE]
21>21>
22-> - Configuration description: Due to factors such as different CANN or NPU models, it may not be possible to guarantee that multiple runs of the same operator produce consistent results. Under the same conditions (platform, device, version number, and other randomness parameters), some operator interfaces can enable deterministic algorithms through `aclrtCtxSetSysParamOpt` (see [Runtime API](https://hiascend.com/document/redirect/CannCommunityCppApi)) to make multiple runs produce consistent results.22+> - Operator feature introduction: Before invoking APIs, please first learn the basic knowledge of operators, including **deterministic algorithms**, **Batch consistency**, and **common quantization modes**. For details, see [Basic Operator Concepts](context/basic_concept.md).
23-> - Performance description: The same operator using deterministic computation is typically slower than non-deterministic computation, so the single-run performance of the model may degrade. However, in scenarios such as experimentation, debugging, and troubleshooting that require consistent results across multiple runs to locate issues, deterministic computation can improve efficiency.
24-> - Thread description: The deterministic state can only be set once in the same thread. If set multiple times, the last effective setting takes precedence. An effective setting means that after setting the deterministic state, an operator task has actually been dispatched. If only the setting is made without dispatching an operator, the deterministic variable is enabled but not dispatched to the operator, so the operator is not executed.
25-> Solution: It is not recommended to set determinism multiple times in one thread. This issue exists in both binary-enabled and binary-disabled cases and will be resolved in a future version.
26> - Symbol description: The "-" symbol in the table indicates that the interface does not currently support the product in that column.23> - Symbol description: The "-" symbol in the table indicates that the interface does not currently support the product in that column.
27 24 
28The operator interface list is as follows:25The operator interface list is as follows:
@@ -10,3 +10,5 @@
10- [互转换关系](./conversion_relationship.md)10- [互转换关系](./conversion_relationship.md)
11- [量化介绍](./quant_mode_introduction.md)11- [量化介绍](./quant_mode_introduction.md)
12- [sparse模式介绍](./sparse_mode_introduction.md)12- [sparse模式介绍](./sparse_mode_introduction.md)
13+- [确定性计算](./determinism_compute.md)
14+- [Batch一致性](./batch_consistency.md)
@@ -0,0 +1,29 @@
1+# Batch一致性
2+ 
3+## 简介
4+ 
5+在应用开发过程中,部分算子为了追求较高的性能,针对同一个Token在不同批次大小或者同一批次不同位置场景中,可能存在计算结果偏差。
6+当前,针对部分算子在满足相同的运行环境条件下,可以通过配置计算过程采用Batch一致性算法,来使得无论如何组合输入,计算结果保持完全一致。
7+**batch一致性算法**:对于给定的一个Token,无论该Token在批次内所处的位置、批次大小、或是和哪些其他Token一同被批处理,输出结果必须逐比特完全一致。
8+ 
9+## 注意事项
10+ 
11+- 通常建议不开启Batch一致性计算,因为同一个算子开启Batch一致性计算后相关算子存在一定的性能劣化,因此模型的单次运行性能可能会下降。但在实验、调试和回归测试等需要保证多次运行结果相同来定位问题和实验算法的场景,Batch一致性计算可以提升效率。
12+ 
13+- 当前配置为进程级开关配置。
14+ 
15+- **版本约束**:TorchNPU版本大于等于26.2.0,CANN版本大于等于9.2.0。
16+ 
17+## 使用方法
18+ 
19+目前CANN算子的主流调用方式为aclnn API或PyTorch API(torch_extension)。部分算子API默认Batch一致性实现,部分算子API默认非Batch一致性实现。对于非Batch一致性实现的算子,部分可通过手动配置开启Batch一致性。
20+ 
21+- **调用aclnn API**
22+ 
23+ 该场景下,通过[《Runtime运行时API》](https://hiascend.com/document/redirect/CannCommunityRuntimeApi)中“运行时配置>aclrtSetSysParamOpt”接口(进程级)配置开启Batch一致性。具体通过设置`ACL_OPT_DETERMINISTIC=3`开启Batch一致性计算。
24+ 
25+- **调用PyTorch API**
26+ 
27+ 该场景下,通过[《TorchNPU自定义API》](https://www.hiascend.com/document/detail/zh/Pytorch/latest/apiref/customapi/docs/zh/custom_APIs/overview.md)中`torch_npu.npu.set_deterministic_level`接口开启Batch一致性计算。
28+ 
29+对于不同框架的算子API,其默认的Batch一致性计算实现策略请以具体的aclnn API文档或PyTorch API文档描述为准。
@@ -0,0 +1,27 @@
1+# 确定性计算
2+ 
3+## 简介
4+ 
5+在应用开发过程中,由于CANN版本或NPU型号等差异,同一算子多次运行的结果可能不完全一致;即使在相同条件下使用相同随机种子,多次NPU执行结果也可能不同。该差异通常源于算子实现中存在异步多线程执行,导致浮点数累加顺序变化。
6+ 
7+在某些特殊场景下(如实验、调试和回归测试),需要算子多次运行产生相同输出,以便定位问题或进行算法分析,这一方案通常称为“确定性计算”。
8+ 
9+## 注意事项
10+ 
11+- 通常建议不开启确定性计算,因为同一算子采用确定性计算一般比非确定性计算执行更慢,可能导致模型单次运行性能下降。但在实验、调试和回归测试等需要保证多次运行结果一致以定位问题或分析算法的场景中,确定性计算可以提升效率。
12+ 
13+- 线程说明:同一线程中只能设置一次确定性状态,多次设置以最后一次有效设置为准。有效设置是指设置确定性状态后,真正执行了一次算子任务下发。如果仅设置而未下发算子任务,则确定性变量虽已开启但未下发给算子,算子不会执行。暂不推荐在一个线程中多次设置确定性。该问题在二进制开启和关闭情况下均存在,将在后续版本中解决。
14+ 
15+## 使用方法
16+ 
17+目前CANN算子的主流调用方式为aclnn API或PyTorch API(torch_extension)。部分算子API默认确定性实现,部分算子API默认非确定性实现。对于非确定性实现的算子,部分可通过手动配置开启确定性计算。
18+ 
19+- **调用aclnn API**
20+ 
21+ 该场景下,通过[《Runtime运行时API》](https://hiascend.com/document/redirect/CannCommunityRuntimeApi)中“运行时配置>aclrtSetSysParamOpt”接口(进程级)配置开启确定性计算。具体通过设置`ACL_OPT_DETERMINISTIC=1`开启确定性计算。
22+ 
23+- **调用PyTorch API**
24+ 
25+ 该场景下,通过PyTorch原生开关(`torch.use_deterministic_algorithms`)开启确定性计算。
26+ 
27+对于不同框架的算子API,其默认的确定性计算实现策略请以具体的aclnn API文档或PyTorch API文档描述为准。
@@ -21,12 +21,12 @@ CANN Simulator是一款面向算子开发场景的SoC级芯片仿真工具,用
21* 本工具为开发工具,不建议在生产环境使用。21* 本工具为开发工具,不建议在生产环境使用。
22* 工具的仿真功能仅支持单卡场景,无法仿真多卡环境,代码中只能设置为0卡。若修改可见卡号,将导致仿真失败。22* 工具的仿真功能仅支持单卡场景,无法仿真多卡环境,代码中只能设置为0卡。若修改可见卡号,将导致仿真失败。
23* 仿真环境仅支持AI Core计算类算子(不支持MC2和HCCL类型的算子)。23* 仿真环境仅支持AI Core计算类算子(不支持MC2和HCCL类型的算子)。
24-CANN Simulator工具目前处于尝鲜版本阶段,仅支持Ascend950PR芯片,建议仿真器运行环境配置为16核CPU和32GB以上内存。24+* CANN Simulator工具目前处于尝鲜版本阶段,仅支持Ascend950PR芯片,建议仿真器运行环境配置为16核CPU和32GB以上内存。
25* 目前不支持arm环境仿真。25* 目前不支持arm环境仿真。
26 26 
27## 环境准备27## 环境准备
28 28 
29-CANN Simulator集成在CANN toolkit包里,参考[环境部署](../install/quick_install.md)中的软件包安装 -> 安装最新版CANN toolkit包章节29+CANN Simulator集成在CANN toolkit包里,参考[环境部署](../install/quick_install.md)完成软件包的安装
30 30 
31# 快速开始31# 快速开始
32 32 
@@ -38,7 +38,7 @@ CANN Simulator集成在CANN toolkit包里,参考[环境部署](../install/quic
38 38 
39```bash39```bash
40# 说明:进入项目根目录,执行如下编译命令,命令仅供参考,详细可以查看算子调用的说明。40# 说明:进入项目根目录,执行如下编译命令,命令仅供参考,详细可以查看算子调用的说明。
41-bash build.sh --pkg --soc=Ascend950 --vendor_name=custom --ops=add_example41+bash build.sh --pkg --soc=ascend950 --vendor_name=custom --ops=add_example
42# 安装自定义算子包42# 安装自定义算子包
43./build_out/cann-ops-ras-${vendor_name}_linux-${arch}.run43./build_out/cann-ops-ras-${vendor_name}_linux-${arch}.run
44```44```
@@ -64,7 +64,7 @@ mean result[2] is: 2.000000
64 64 
65## 查看性能流水65## 查看性能流水
66 66 
67-仿真性能流水文件在本项目`examples/add_example/examples/build/bin/cannsim_*/report`目录,流水相关文件为:67+仿真性能流水文件在本项目`examples/add_example/examples/build/bin/cannsim_*/report/results/kernel_*/core_*`目录,流水相关文件为:
68 68 
69```bash69```bash
70trace_core0.json70trace_core0.json
@@ -72,7 +72,7 @@ trace_core0.json
72 72 
73在Chrome浏览器中输入“chrome://tracing”地址,并将生成的指令流水图文件(trace_core0.json)拖到空白处打开,具体参数介绍参考“仿真结果解析”章节。73在Chrome浏览器中输入“chrome://tracing”地址,并将生成的指令流水图文件(trace_core0.json)拖到空白处打开,具体参数介绍参考“仿真结果解析”章节。
74 74 
75-# 仿真执行75+# 仿真执行说明
76 76 
77## 命令功能77## 命令功能
78 78 
@@ -80,7 +80,7 @@ trace_core0.json
80 80 
81## 命令格式81## 命令格式
82 82 
83-cannsim record [options] user_app --user_options83+cannsim record [options] user_app
84 84 
85## 参数说明85## 参数说明
86 86 
@@ -88,11 +88,11 @@ cannsim record [options] user_app --user_options
88 88 
89|参数|可选/必选|说明|89|参数|可选/必选|说明|
90| --- | --- | --- |90| --- | --- | --- |
91-|-s \<value\> 或 --soc_version \<value\> [options]参数 | 必选 | 指定模拟目标芯片版本(如:Ascend950)。|91+|-s或 --soc-version | 必选 | 指定模拟目标芯片版本(如:Ascend950)。|
92-|-o \<value\> 或 --output \<value\> [options]参数 | 可选| 生成文件所在路径,可配置为绝对路径或者相对路径,并且执行工具的用户需要具有读写权限。如果未指定路径,则默认在当前目录下保存数据。|92+|-o或 --output | 可选 | 生成文件所在路径,可配置为绝对路径或者相对路径,并且执行工具的用户需要具有读写权限。如果未指定路径,则默认在当前目录下保存数据。|
93-|-g或 --gen-report[options]参数 | 可选 | 启用仿真完成后是否进行自动解析,并生成分析报告。默认不自动解析。|93+|-g或 --gen-report | 可选 | 启用仿真完成后是否进行自动解析,并生成分析报告。默认不自动解析。|
94-|-n或 --core-id | 可选 | 仿真期间启用日志的AI Core,格式同report -n:'all'、'0-2,12-14'、'5'。默认全开;配合 -g且未指定时回退到core 0。|
95|-u或 --user-option | 可选 | 用户自定义算子参数,以命令行选项形式传递给算子程序。|94|-u或 --user-option | 可选 | 用户自定义算子参数,以命令行选项形式传递给算子程序。|
95+|-n或 --core-id | 可选 | 仿真期间启用日志的AI Core,格式同report -n:'all'、'0-2,12-14'、'5'。默认全开;配合 -g且未指定时回退到core 0。|
96|user_app|必选|待运行的算子程序或命令(如 ./app, python train.py, bash run.sh)。|96|user_app|必选|待运行的算子程序或命令(如 ./app, python train.py, bash run.sh)。|
97 97 
98## 使用示例98## 使用示例
@@ -101,10 +101,10 @@ cannsim record [options] user_app --user_options
1012. 执行仿真命令,可参考以下使用示例1012. 执行仿真命令,可参考以下使用示例
102 102 
103 ```bash103 ```bash
104- 方式一:启用仿真,并将输出保存至 ./output目录,/path/to/app为算子程序104+ # 方式一:启用仿真,并将输出保存至 ./output目录,/path/to/app为算子程序
105 $ cannsim record /path/to/app -o ./output -s Ascend950105 $ cannsim record /path/to/app -o ./output -s Ascend950
106 106 
107- 方式二:启用仿真并生成报告,用于后续性能分析107+ # 方式二:启用仿真并生成报告,用于后续性能分析
108 $ cannsim record /path/to/app -o ./output -s Ascend950 --gen-report108 $ cannsim record /path/to/app -o ./output -s Ascend950 --gen-report
109 ```109 ```
110 110 
@@ -138,7 +138,7 @@ cannsim record [options] user_app --user_options
138 138 
1394. 用户可以获取算子执行结果,并进行精度的对比,结果示例如下1394. 用户可以获取算子执行结果,并进行精度的对比,结果示例如下
140 140 
141- 以下输出仅为AscendC单算子直调精度比较结果举例,因版本不同略有差异,请以实际输出为准。141+ 以下输出仅为Ascend C单算子直调精度比较结果举例,因版本不同略有差异,请以实际输出为准。
142 142 
143 ```bash143 ```bash
144 INFO:root:[INFO] compare data case[ case001]144 INFO:root:[INFO] compare data case[ case001]
@@ -149,7 +149,7 @@ cannsim record [options] user_app --user_options
149 149 
1505. 查看算子指令流水图,参考仿真结果解析。1505. 查看算子指令流水图,参考仿真结果解析。
151 151 
152-# 仿真结果解析152+# 仿真结果解析说明
153 153 
154## 命令功能154## 命令功能
155 155 
@@ -165,13 +165,14 @@ cannsim report [options]
165 165 
166|参数 | 可选/必选 | 说明|166|参数 | 可选/必选 | 说明|
167| --- | --- | --- |167| --- | --- | --- |
168-|-e \<value\> 或 --export \<value\> [options]参数 | 必选 | 原始结果文件目录,需指定为仿真执行后生成的结果目录,指定到cannsim_{timestamp}_${user_app}层,可配置为绝对路径或者相对路径,并且工具执行用户具有可读写权限。|168+|-e或 --export | 必选 | 仿真执行结果目录,指定到cannsim_{timestamp}_${user_app}层,可配置为绝对路径或者相对路径,且执行用户需具有读写权限。|
169-|-o或 --output [options]参数 | 可选 | 解析结果输出目录,可配置为绝对路径或者相对路径,且执行用户需具有读写权限。若未指定路径,默认在当前目录下保存数据。如果生成的结果文件与现有文件同名,则会覆盖原有文件。|169+|-o或 --output | 可选 | 指令流水图输出目录,可配置为绝对路径或者相对路径,且执行用户需具有读写权限。若未指定路径,默认与export目录相同。|
170-|-n或 --core-id [options]参数 | 可选 | 指定生成指令流水的核ID,不指定默认生成0核的指令流水。配置的格式如下:生成所有核的流水,配置为‘all’。指定核ID的范围,如:‘0-1’。指定单核ID,如‘5’。|170+|-n或 --core-id | 可选 | 指定生成指令流水的核ID,支持格式:'all'、'0-2,12-14'、'5'。不指定默认生成0核的指令流水。|
171+|-f或 --object-file | 可选 | 设备对象文件路径,用于辅助生成报告。|
171 172 
172## 使用示例173## 使用示例
173 174 
174-1. 参考仿真执行执行算子仿真,对比输出示例,确保对应的结果执行正确。175+1. 参考仿真执行,执行算子仿真,对比输出示例,确保对应的结果执行正确。
1752. 执行仿真结果解析命令,可参考以下执行用例。1762. 执行仿真结果解析命令,可参考以下执行用例。
176 177 
177 ```bash178 ```bash
@@ -179,7 +180,7 @@ cannsim report [options]
179 cannsim report -e /path/to/cannsim_{timestamp}_${user_app}180 cannsim report -e /path/to/cannsim_{timestamp}_${user_app}
180 181 
181 在指定目录下生成核0、核1、核11、核12的性能分析报告182 在指定目录下生成核0、核1、核11、核12的性能分析报告
182- cannsim report -e /path/to/cannsim_{timestamp}_${user_app} -o /path/to/report -n ‘0-1, 11-12’183+ cannsim report -e /path/to/cannsim_{timestamp}_${user_app} -o /path/to/report -n '0-1, 11-12'
183 ```184 ```
184 185 
1853. 命令执行完后,会在output配置的目录下生成对应的流水文件,文件格式为json格式,输出结果示例如下:1863. 命令执行完后,会在output配置的目录下生成对应的流水文件,文件格式为json格式,输出结果示例如下:
@@ -191,7 +192,7 @@ cannsim report [options]
191 ```192 ```
192 193 
1934. 仿真结果查看1944. 仿真结果查看
194- 在Chrome浏览器中输入“chrome://tracing”地址,并将通过生成指令流水图文件(trace.json)拖到空白处打开,键盘上输入快捷键(W:放大,S:缩小,A:左移,D:右移)可进行查看。195+ 在Chrome浏览器中输入“chrome://tracing”地址,并将生成的指令流水图文件(trace.json)拖到空白处打开,键盘上输入快捷键(W:放大,S:缩小,A:左移,D:右移)可进行查看。
195 ![指令流水图](../figures/指令流水图.png)196 ![指令流水图](../figures/指令流水图.png)
196 197 
197 表2 关键字段说明198 表2 关键字段说明
@@ -1,86 +1,181 @@
1# 环境部署1# 环境部署
2 2 
3-## 前提条件3+在学习各类[学习教程](../../../README.md#学习教程)操作之前,请您先参考下面步骤完成基础环境搭建和源码下载,确保已安装NPU驱动、固件和CANN软件(`Ascend-cann-toolkit`和`Ascend-cann-ops`)等。
4 4 
5-使用本项目前,请确保如下基础依赖、NPU驱动和固件已安装。5+## 环境安装
6 6 
7-1. **安装依赖**7+本项目提供多种搭建昇腾环境的方式,请按需选择。
8 8 
9- 本项目源码编译用到的依赖如下,请注意版本要求。9+> **说明**:本文提到的编译态和运行态含义如下,请根据实际情况选择。
10+>
11+> - 编译态:针对仅编译本项目不运行的场景,只需安装CANN toolkit包。
12+> - 运行态:针对运行本项目的场景(编译运行或纯运行),需安装驱动与固件、CANN toolkit包、CANN ops包。
10 13 
11- - python >= 3.7.014+| 安装方式 | 使用说明 | 使用场景 |
12- - gcc >= 7.3.015+| ----- | ------ | ------ |
13- - cmake >= 3.16.016+| CANNLab | 一站式开发平台,提供在线直接运行的昇腾环境,无需手动安装。<br>当前可提供单机算力,**默认安装最新版本CANN包**。 | 适用于没有昇腾设备的开发者。|
14- - pigz(可选,安装后可提升打包速度,建议版本 >= 2.4)17+| Docker | Docker镜像是一种高效部署方式,已预集成CANN包和必备依赖。<br>当前适用于Atlas A2、A3系列产品,OS支持ubuntu22.04、openeuler24.03。**默认安装最新版本CANN包**。 |适用于有昇腾设备,需要快速搭建环境的开发者。|
15- - dos2unix18+| 手动安装 | 手动安装CANN包和基础依赖,灵活性高。 |适用于有昇腾设备,想体验手动安装CANN包或体验最新master分支能力的开发者。|
16- - gawk
17- - googletest(仅执行UT时依赖,建议版本 [release-1.11.0](https://github.com/google/googletest/releases/tag/release-1.11.0))
18 19 
19- 上述依赖包可通过项目根目录install\_deps.sh安装,命令如下,若遇到不支持系统,请参考该文件自行适配。20+### 方式1:CANNLab
20 21 
21- ```bash22+对于无昇腾设备的开发者,可直接使用CANNLab云开发环境,即“**一站式开发平台**”,该平台为您提供在线可直接运行的昇腾环境,环境中已安装必备的驱动、固件、软件包和依赖,无需手动安装。
22- bash install_deps.sh
23- ```
24 23 
25-2. **安装驱动与固件(运行态依赖)**24+> **说明**:环境默认安装最新版本CANN包,源码下载时注意与软件配套。更多关于开发平台的介绍请参考[CANNLab指导](https://gitcode.com/cann/cann-learning-hub/blob/master/docs/CANNLab_env_experience_guide.md)。
26 25 
27- 运行算子时必须安装驱动与固件,若仅编译算子,可跳过本操作。26+1. 进入开源项目,单击“`CANNLab`”按钮,使用已认证过的华为云账号登录。若未注册或认证,请根据页面提示进行注册和认证。
28 27 
29- 单击[下载链接](https://www.hiascend.com/hardware/firmware-drivers/community),根据实际产品型号和环境架构,获取对应的`Ascend-hdk-<chip_type>-npu-driver_<version>_linux-<arch>.run`、`Ascend-hdk-<chip_type>-npu-firmware_<version>.run`包。28+ <img src="../figures/cloudIDE.png" alt="云平台" width="750px" height="85px">
30 29 
31- 安装指导详见《[CANN 软件安装指南](https://www.hiascend.com/document/redirect/CannCommunityInstSoftware)》。30+2. 根据页面提示创建NPU环境并配置规格,启动云开发环境后,单击“`连接 > WebIDE`”进入一站式开发平台。当前开源项目资源默认在`/mnt/workspace/gitCode`目录。
32 31 
33-## 软件包安装32+ <img src="../figures/webIDE.png" alt="云平台" width="1000px" height="150px">
34 33 
35-1. **安装最新版CANN toolkit包**34+### 方式2:Docker部署
36 35 
37- 根据实际产品型号和环境架构,获取对应的`Ascend-cann-toolkit_${cann_version}_linux-${arch}.run`包。36+对于有昇腾设备的开发者,若您想快速搭建昇腾环境,可使用Docker镜像部署。
38- - Atlas A2/A3系列产品:单击[下载链接](https://mirrors.huaweicloud.com/artifactory/cann-run/software)获取软件包。37+ 
39- - Ascend 950PR/Ascend 950DT产品:单击[下载链接](https://mirrors.huaweicloud.com/artifactory/cann-run/software)获取软件包。38+> **说明**:
39+>
40+> - 镜像文件比较大,下载需要一定时间,请您耐心等待。关于docker命令的选项介绍可通过`docker --help`查询。
41+> - 环境默认安装最新版本CANN包,源码下载时注意与软件配套。
42+ 
43+1. **安装驱动(运行态依赖)**
44+ 
45+ 驱动是运行态依赖,若仅编译算子,可以不安装。使用`npu-smi info`检查是否有NPU相关信息,若没有,请参考《[CANN快速安装](https://www.hiascend.com/cann/download)》完成驱动安装:
46+ 
47+ - 步骤1:在页面中选择自己的产品系列、CPU架构、操作系统,安装方式选择“在线安装(Yum)”。
48+ - 步骤2:按照页面指引依次完成“配置用户属组”、“安装依赖&配源”、“安装NPU驱动”三个环节。
49+ - 步骤3:执行`npu-smi info`,若能正常显示NPU设备信息,则说明驱动安装成功。
50+ 
51+2. **下载镜像**
52+ 
53+ - 步骤1:以root用户登录宿主机。确保宿主机已安装Docker引擎(版本1.11.2及以上),使用`docker --version`检查Docker版本,若没有,请参考[Docker官方安装指南](https://docs.docker.com/engine/install/)。
54+ - 步骤2:从[昇腾镜像仓库](https://www.hiascend.com/developer/ascendhub/detail/17da20d1c2b6493cb38765adeba85884)拉取已预集成CANN软件包及算子开发所需依赖的镜像。
55+ 
56+ 示例如下,请自行替换CANN版本号、芯片系列、操作系统、python版本等信息,各字段支持的取值可在上述昇腾镜像仓库页面查询。
40 57 
41 ```bash58 ```bash
42- # 确保安装包具有可执行权限59+ # 以cann:9.1.0-beta.1版本为例
43- chmod +x Ascend-cann-toolkit_${cann_version}_linux-${arch}.run60+ docker pull swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:9.1.0-beta.1-910b-ubuntu22.04-py3.12-devel
44- # 安装命令
45- ./Ascend-cann-toolkit_${cann_version}_linux-${arch}.run --install --force --install-path=${install_path}
46 ```61 ```
47 62 
48- - \$\{cann\_version\}:表示CANN包版本号。63+ > **说明**:镜像标签格式为`<CANN版本>-<芯片系列>-<操作系统>-<Python版本>-devel`。带`-devel`后缀的镜像为算子开发镜像,内含算子开发编译依赖。
49- - \$\{arch\}:表示CPU架构,如aarch64、x86_64。
50- - \$\{install\_path\}:表示指定安装路径,默认安装在`/usr/local/Ascend`目录。
51 64 
52-2. **安装最新版CANN ops包(运行态依赖)**65+3. **运行Docker**
53 66 
54- 运行算子时必须安装本包,若仅编译算子,可跳过本操作。67+ 拉取镜像后,需要以特定参数启动容器,以便容器内能访问宿主的昇腾设备。
55- 
56- 根据实际产品型号和环境架构,获取对应的`Ascend-cann-${soc_name}-ops_${cann_version}_linux-${arch}.run`包。
57- - Atlas A2/A3系列产品:单击[下载链接](https://mirrors.huaweicloud.com/artifactory/cann-run/software)获取软件包。
58- - Ascend 950PR/Ascend 950DT产品:单击[下载链接](https://mirrors.huaweicloud.com/artifactory/cann-run/software)获取软件包。
59 68 
60 ```bash69 ```bash
61- # 确保安装包具有可执行权限70+ docker run --name cann_container --device /dev/davinci0 --device /dev/davinci_manager --device /dev/devmm_svm --device /dev/hisi_hdc -v /usr/local/dcmi:/usr/local/dcmi -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi -v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ -v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info -v /etc/ascend_install.info:/etc/ascend_install.info -it swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:9.1.0-beta.1-910b-ubuntu22.04-py3.12-devel bash
62- chmod +x Ascend-cann-${soc_name}-ops_${cann_version}_linux-${arch}.run
63- # 安装命令
64- ./Ascend-cann-${soc_name}-ops_${cann_version}_linux-${arch}.run --install --install-path=${install_path}
65 ```71 ```
66 72 
67- - \$\{soc\_name\}:表示NPU型号名称,即\$\{soc\_version\}删除“ascend”后剩余的内容。73+ > **说明**:`--name`用于指定容器名称,请替换为便于识别的自定义名称;若该名称已被占用,容器会启动失败。
68- - \$\{install\_path\}:表示指定安装路径,需要与toolkit包安装在相同路径,默认安装在`/usr/local/Ascend`目录。74+ 
75+ | 参数 | 说明 | 注意事项 |
76+ | :--- | :--- | :--- |
77+ | `--name cann_container` | 为容器指定名称,便于管理。 | 可自定义。 |
78+ | `--device /dev/davinci0` | 核心:将宿主机的NPU设备卡映射到容器内,可指定映射多张NPU设备卡。 | 必须根据实际情况调整:`davinci0`对应系统中的第0张NPU卡。请先在宿主机执行`npu-smi info`命令,根据输出显示的设备号(如`NPU 0`, `NPU 1`)来修改此编号。|
79+ | `--device /dev/davinci_manager` | 映射NPU设备管理接口。 | - |
80+ | `--device /dev/devmm_svm` | 映射设备内存管理接口。 | - |
81+ | `--device /dev/hisi_hdc` | 映射主机与设备间的通信接口。 | - |
82+ | `-v /usr/local/dcmi:/usr/local/dcmi` | 挂载设备容器管理接口(DCMI)相关工具和库。 | - |
83+ | `-v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi` | 挂载`npu-smi`工具。 | 使容器内可以直接运行此命令来查询NPU状态和性能信息。|
84+ | `-v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/` | 关键挂载:将宿主机的NPU驱动库映射到容器内。 | - |
85+ | `-v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info` | 挂载驱动版本信息文件。 | - |
86+ | `-v /etc/ascend_install.info:/etc/ascend_install.info` | 挂载CANN软件安装信息文件。 | - |
87+ | `-it` | `-i`(交互式)和`-t`(分配伪终端)的组合参数。 | - |
88+ | `swr.cn-south-1.myhuaweicloud.com/ascendhub/cann:9.1.0-beta.1-910b-ubuntu22.04-py3.12-devel` | 指定要运行的Docker镜像。 | 请确保此镜像名和标签(tag)与您通过`docker pull`拉取的镜像完全一致。 |
89+ | `bash` | 容器启动后立即执行的命令。 | - |
90+ 
91+### 方式3:手动安装
92+ 
93+对于有昇腾设备的开发者,若您想手动搭建昇腾环境,请参考下述步骤。
94+ 
95+#### 安装软件
96+ 
97+- **场景1:基于CANN weekly版本体验最新master能力或开发**
98+ 
99+ 请参考《[CANN快速安装](https://www.hiascend.com/cann/download?versionId=794&ids=d803%2Ch0501%2Ch0601%2Ch0703&currentTab=1)》,选择`Weekly版本`,并根据产品系列、CPU架构、操作系统等下载对应包,最后参考页面提供的命令完成安装。
100+ 
101+- **场景2:基于CANN 稳定版本体验分支能力或开发**
102+ 
103+ 请参考《[CANN快速安装](https://www.hiascend.com/cann/download)》,选择`稳定版本`(仅支持CANN 8.5.0及后续版本),并根据产品系列、CPU架构、操作系统等下载对应包,最后参考页面提供的命令完成安装。
104+ 
105+#### 安装基础依赖
106+ 
107+本项目基础依赖如下,注意满足版本号要求。
108+ 
109+- python >= 3.7.0(建议版本 <= 3.10)
110+- gcc >= 7.3.0
111+- cmake >= 3.16.0
112+- pigz(可选,安装后可提升打包速度,建议版本 >= 2.4)
113+- dos2unix
114+- gawk
115+- make
116+- patch
117+- googletest(仅执行UT时依赖,建议版本 [release-1.11.0](https://github.com/google/googletest/releases/tag/release-1.11.0))
118+ 
119+上述依赖可通过项目脚本一键安装,操作步骤如下:
120+ 
121+1. 下载源码。
122+ 
123+ 下载与CANN版本配套的分支源码,命令如下,`${tag_version}`替换为分支标签名。
124+ 
125+ ```bash
126+ git clone -b ${tag_version} https://gitcode.com/cann/ops-ras.git
127+ ```
128+ 
129+2. 安装依赖。
130+ 
131+ 先通过项目根目录`install_deps.sh`一键安装上述依赖,命令如下,若遇到不支持的系统,请参考该文件自行适配。
132+ 
133+ ```bash
134+ bash install_deps.sh
135+ ```
136+ 
137+ 再通过项目根目录`requirements.txt`继续安装python三方库依赖,命令如下。
138+ 
139+ ```bash
140+ pip3 install -r requirements.txt
141+ ```
142+ 
143+## 环境验证
144+ 
145+安装完CANN包后,需验证环境和驱动是否正常。
146+ 
147+- **检查NPU设备**
148+ 
149+ ```bash
150+ # 运行npu-smi,若能正常显示设备信息,则驱动正常
151+ npu-smi info
152+ ```
153+ 
154+- **检查CANN版本**
155+ 
156+ ```bash
157+ # 查看CANN toolkit包版本信息(默认路径安装)
158+ # docker和手动安装场景:
159+ cat /usr/local/Ascend/cann/${arch}-linux/ascend_toolkit_install.info
160+ # CANNLab场景:
161+ cat /home/developer/Ascend/cann/${arch}-linux/ascend_toolkit_install.info
162+ 
163+ # 查看CANN ops包版本信息(默认路径安装)
164+ # docker和手动安装场景:
165+ cat /usr/local/Ascend/cann/${arch}-linux/ascend_ops_install.info
166+ # CANNLab场景:
167+ cat /home/developer/Ascend/cann/${arch}-linux/ascend_ops_install.info
168+ ```
169+ 
170+ 其中`${arch}`可通过`uname -m`查询当前架构,如aarch64、x86_64。
69 171 
70## 环境变量配置172## 环境变量配置
71 173 
174+按需选择合适的命令使环境变量生效。
175+ 
72```bash176```bash
73# 默认路径安装,以root用户为例(非root用户,将/usr/local替换为${HOME})177# 默认路径安装,以root用户为例(非root用户,将/usr/local替换为${HOME})
74source /usr/local/Ascend/cann/set_env.sh178source /usr/local/Ascend/cann/set_env.sh
75# 指定路径安装179# 指定路径安装
76# source ${install_path}/cann/set_env.sh180# source ${install_path}/cann/set_env.sh
77```181```
78- 
79-## 源码下载
80- 
81-```bash
82-# 下载项目源码,以master分支为例
83-git clone https://gitcode.com/cann/ops-ras.git
84-# 安装根目录requirements.txt依赖
85-pip3 install -r requirements.txt
86-```
@@ -17,13 +17,10 @@
17 17 
18## 接口列表18## 接口列表
19 19 
20-> **确定性简介**:20+> [!NOTE]
21>21>
22-> - 配置说明:因CANN或NPU型号不同等原因,可能无法保证同一个算子多次运行结果一致。在相同条件下(平台、设备、版本号和其他随机性参数等),部分算子接口可通过`aclrtCtxSetSysParamOpt`(参见[《Runtime运行时API》](https://hiascend.com/document/redirect/CannCommunityCppApi))开启确定性算法,使多次运行结果一致。22+> - 算子特性介绍:调用API前,请先学习算子相关基础知识,包括**确定性算法**、**Batch一致性**、**常见量化模式**等,具体介绍参见[算子基本概念](context/basic_concept.md)。
23-> - 性能说明:同一个算子采用确定性计算通常比非确定性慢,因此模型单次运行性能可能会下降。但在实验、调试调测等需要保证多次运行结果相同来定位问题的场景,确定性计算可以提升效率。23+> - 符号说明:表格中“-”符号表示该接口暂不支持当前列产品。
24-> - 线程说明:同一线程中只能设置一次确定性状态,多次设置以最后一次有效设置为准。有效设置是指设置确定性状态后,真正执行了一次算子任务下发。如果仅设置,没有算子下发,只能是确定性变量开启但未下发给算子,因此不执行算子。
25-> 解决方案:暂不推荐一个线程多次设置确定性。该问题在二进制开启和关闭情况下均存在,在后续版本中会解决该问题。
26-> - 符号说明:表中“- ”符号表示该接口暂不支持当前列产品。
27 24 
28算子接口列表如下:25算子接口列表如下:
29 26