已合并
docs: sync flash attention performance references #885
bxr创建于 4月30日
docs: sync flash attention performance references #885
已合并
共 3 个文件变更+35-3
| @@ -130,11 +130,27 @@ This repository also demonstrates how standard tile operations can be mapped to | |||
| 130 | 130 | ||
| 131 | ### Flash Attention | 131 | ### Flash Attention |
| 132 | 132 | ||
| 133 | -- Reference implementation: `kernels/manual/common/flash_atten/` | 133 | +- Reference implementations: `kernels/manual/common/flash_atten/` for A2/A3 and `kernels/manual/a5/flash_atten/` for A5 |
| 134 | - Detailed analysis and tuning notes: [Flash Attention Operator Implementation](kernels/manual/common/flash_atten/README.md) | 134 | - Detailed analysis and tuning notes: [Flash Attention Operator Implementation](kernels/manual/common/flash_atten/README.md) |
| 135 | +- A5 build guide, with A5 performance numbers still pending: [Flash Attention Performance Kernel (A5)](kernels/manual/a5/flash_atten/README.md) | ||
| 135 | - S0: query sequence length (number of rows in Q/O) | 136 | - S0: query sequence length (number of rows in Q/O) |
| 136 | - S1: key/value sequence length (number of rows in K/V) | 137 | - S1: key/value sequence length (number of rows in K/V) |
| 137 | 138 | ||
| 139 | +Ascend 910B2 (A2/A3) multi-core comparison, using `torch_npu` as the baseline: | ||
| 140 | + | ||
| 141 | +| Sequence length | PTO time (us) | torch_npu time (us) | PTO TFLOPS | torch_npu TFLOPS | Speedup | | ||
| 142 | +| --- | --- | --- | --- | --- | --- | | ||
| 143 | +| 1024 | 20.960 | 58.461 | 25.61 | 9.18 | 2.79x | | ||
| 144 | +| 2048 | 32.461 | 70.801 | 66.16 | 30.33 | 2.18x | | ||
| 145 | +| 4096 | 88.902 | 118.302 | 96.62 | 72.61 | 1.33x | | ||
| 146 | +| 8192 | 292.626 | 353.147 | 117.42 | 97.30 | 1.21x | | ||
| 147 | +| 16384 | 909.058 | 1118.462 | 151.19 | 122.88 | 1.23x | | ||
| 148 | +| 32768 | 3262.645 | 3646.173 | 168.50 | 150.78 | 1.12x | | ||
| 149 | + | ||
| 150 | + | ||
| 151 | + | ||
| 152 | +Additional A2/A3 normalized reference: | ||
| 153 | + | ||
| 138 |  | 154 |  |
| 139 | 155 | ||
| 140 | ### Communication Instruction Bandwidth | 156 | ### Communication Instruction Bandwidth |
| @@ -130,11 +130,27 @@ python3 tests/script/run_st.py -r sim -v a3 -t tadd -g TADDTest.case_float_64x64 | |||
| 130 | 130 | ||
| 131 | ### Flash Attention | 131 | ### Flash Attention |
| 132 | 132 | ||
| 133 | -- 参考实现：`kernels/manual/common/flash_atten/` | 133 | +- 参考实现：`kernels/manual/common/flash_atten/` 用于 A2/A3，`kernels/manual/a5/flash_atten/` 用于 A5 |
| 134 | -- 详细分析与调参说明：[Flash Attention 算子实现](kernels/manual/common/flash_atten/README_zh.md) | 134 | +- 详细分析与调优说明：[Flash Attention 算子实现](kernels/manual/common/flash_atten/README_zh.md) |
| 135 | +- A5 构建说明（A5 性能数据仍待补充）：[Flash Attention Performance Kernel (A5)](kernels/manual/a5/flash_atten/README.md) | ||
| 135 | - S0：query 序列长度（Q/O 的行数） | 136 | - S0：query 序列长度（Q/O 的行数） |
| 136 | - S1：key/value 序列长度（K/V 的行数） | 137 | - S1：key/value 序列长度（K/V 的行数） |
| 137 | 138 | ||
| 139 | +Ascend 910B2（A2/A3）多核对比，基线为 `torch_npu`： | ||
 | |||
| 140 | + | ||
| 141 | +| 序列长度 | PTO 时间 (us) | torch_npu 时间 (us) | PTO TFLOPS | torch_npu TFLOPS | 加速比 | | ||
| 142 | +| --- | --- | --- | --- | --- | --- | | ||
| 143 | +| 1024 | 20.960 | 58.461 | 25.61 | 9.18 | 2.79x | | ||
| 144 | +| 2048 | 32.461 | 70.801 | 66.16 | 30.33 | 2.18x | | ||
| 145 | +| 4096 | 88.902 | 118.302 | 96.62 | 72.61 | 1.33x | | ||
| 146 | +| 8192 | 292.626 | 353.147 | 117.42 | 97.30 | 1.21x | | ||
| 147 | +| 16384 | 909.058 | 1118.462 | 151.19 | 122.88 | 1.23x | | ||
| 148 | +| 32768 | 3262.645 | 3646.173 | 168.50 | 150.78 | 1.12x | | ||
| 149 | + | ||
| 150 | + | ||
| 151 | + | ||
| 152 | +附：A2/A3 归一化性能参考： | ||
| 153 | + | ||
| 138 |  | 154 |  |
| 139 | 155 | ||
| 140 | ### 通信指令带宽 | 156 | ### 通信指令带宽 |
please delete (A2/A3) becuase we already specified the testing platform is 910B2.