已合并
【PR】: 补齐tensorflow相关的autofuse融合样例 #1765
zzq创建于 8月18日
【PR】: 补齐tensorflow相关的autofuse融合样例 #1765
已合并
共 17 个文件变更+540-158
| @@ -1,4 +1,4 @@ | |||
| 1 | -# PyTorch Inductor 场景用例演示 | 1 | +# PyTorch 场景用例演示 |
| 2 | 2 | ||
| 3 | ## 功能描述 | 3 | ## 功能描述 |
| 4 | 4 | ||
| @@ -6,9 +6,9 @@ | |||
| 6 | 6 | ||
| 7 | 当前包含以下三个用例： | 7 | 当前包含以下三个用例： |
| 8 | 8 | ||
| 9 | -- `add + ge`：将加法和比较算子融合为一个算子； | 9 | +* `add + ge`：将加法和比较算子融合为一个算子； |
| 10 | -- `mul + reducesum`：将乘法和求和归约算子融合为一个算子； | 10 | +* `mul + reducesum`：将乘法和求和归约算子融合为一个算子； |
| 11 | -- `gather + add`：构造索引取数和逐元素加法图模式 | 11 | +* `gather + add`：构造索引取数和逐元素加法图模式 |
| 12 | 12 | ||
| 13 | 注：当前暂不支持gather融合能力，等待[ issue175 ](https://gitcode.com/cann/graph-autofusion/issues/175)这个issue完成后gather可以和add进行融合。 | 13 | 注：当前暂不支持gather融合能力，等待[ issue175 ](https://gitcode.com/cann/graph-autofusion/issues/175)这个issue完成后gather可以和add进行融合。 |
| 14 | 14 | ||
| @@ -37,17 +37,17 @@ pytorch | |||
| 37 | ## 前置说明 | 37 | ## 前置说明 |
| 38 | 38 | ||
| 39 | 运行本用例前，请先认真阅读[ PyTorch环境安装说明 ](../../../docs/env_install/pytorch/env_pytorch.md)。需完成以下步骤： | 39 | 运行本用例前，请先认真阅读[ PyTorch环境安装说明 ](../../../docs/env_install/pytorch/env_pytorch.md)。需完成以下步骤： |
| 40 | + | ||
| 40 | 1. CANN 包版本要求为 `9.0.0` 及以上，通过 [CANN 快速安装](https://www.hiascend.com/cann/download?versionId=745&ids=d802%2Ch0501%2Ch0602%2Ch0701) 正确安装 toolkit 和 ops 包，可以参考[ 安装指导 ](../../../docs/zh/quick_install.md)。 | 41 | 1. CANN 包版本要求为 `9.0.0` 及以上，通过 [CANN 快速安装](https://www.hiascend.com/cann/download?versionId=745&ids=d802%2Ch0501%2Ch0602%2Ch0701) 正确安装 toolkit 和 ops 包，可以参考[ 安装指导 ](../../../docs/zh/quick_install.md)。 |
| 41 | 2. `torch_npu` 版本要求为 `2.9.0` 及以上，可以根据 [环境快速安装脚本](../../../scripts/env_install/pytorch/setup_torch_npu_daily.sh) 快速安装python环境和 `torch_npu` 。 | 42 | 2. `torch_npu` 版本要求为 `2.9.0` 及以上，可以根据 [环境快速安装脚本](../../../scripts/env_install/pytorch/setup_torch_npu_daily.sh) 快速安装python环境和 `torch_npu` 。 |
| 42 | 43 | ||
| 43 | ## 设置环境变量 | 44 | ## 设置环境变量 |
| 44 | 45 | ||
| 46 | +执行用例前，请确保当前终端已进入安装有 PyTorch 和 `torch_npu` 的 Python 环境。Python 环境的安装和激活方式请参考[ PyTorch环境安装说明 ](../../../docs/env_install/pytorch/env_pytorch.md)。 | ||
| 47 | + | ||
| 45 | 每次新开终端后，执行： | 48 | 每次新开终端后，执行： |
| 46 | 49 | ||
| 47 | ```bash | 50 | ```bash |
| 48 | -# 环境激活 | ||
| 49 | -source /mnt/workspace/env/venv/torch210_daily/bin/activate | ||
| 50 | - | ||
| 51 | # CANN 包安装路径根据实际安装位置确定。 | 51 | # CANN 包安装路径根据实际安装位置确定。 |
| 52 | export CANN_INSTALL_PATH=/home/developer/Ascend | 52 | export CANN_INSTALL_PATH=/home/developer/Ascend |
| 53 | 53 | ||
| @@ -60,26 +60,11 @@ export ASCEND_DEVICE_ID=0 | |||
| 60 | 60 | ||
| 61 | ## 执行用例 | 61 | ## 执行用例 |
| 62 | 62 | ||
| 63 | -### add + ge 融合 | 63 | +当前包含以下三个用例，可根据实际场景参考对应文档完成用例执行： |
| 64 | 64 | ||
| 65 | -```bash | 65 | +* [add + ge 融合用例](./af_pointwise/README.md) |
| 66 | -cd af_pointwise | 66 | +* [mul + reducesum 融合用例](./af_reduce/README.md) |
| 67 | -python af_add_ge.py | 67 | +* [gather + add 图模式用例](./af_gather/README.md) |
| 68 | -``` | ||
| 69 | - | ||
| 70 | -### mul + reducesum 融合 | ||
| 71 | - | ||
| 72 | -```bash | ||
| 73 | -cd af_reduce | ||
| 74 | -python af_mul_reducesum.py | ||
| 75 | -``` | ||
| 76 | - | ||
| 77 | -### gather + add 图模式 | ||
| 78 | - | ||
| 79 | -```bash | ||
| 80 | -cd af_gather | ||
| 81 | -python af_gather_add.py | ||
| 82 | -``` | ||
| 83 | 68 | ||
| 84 | ## 预期执行结果 | 69 | ## 预期执行结果 |
| 85 | 70 | ||
| @@ -101,6 +86,6 @@ op_summary_时间戳.csv | |||
| 101 | 86 | ||
| 102 | ## 参考 | 87 | ## 参考 |
| 103 | 88 | ||
| 104 | -- [Autofuse 简介与快速上手](../../README.md) | 89 | +* [Autofuse 简介与快速上手](../../README.md) |
| 105 | -- [Profiling 性能分析工具指南](https://hiascend.com/document/redirect/CannCommunityToolProfiling) | 90 | +* [Profiling 性能分析工具指南](https://hiascend.com/document/redirect/CannCommunityToolProfiling) |
| 106 | -- [精度调试工具指南](https://hiascend.com/document/redirect/CannCommunityToolAccucacy) | 91 | +* [精度调试工具指南](https://hiascend.com/document/redirect/CannCommunityToolAccucacy) |
| @@ -1,4 +1,4 @@ | |||
| 1 | -# PyTorch Inductor Examples | 1 | +# PyTorch Scenario Examples |
| 2 | 2 | ||
| 3 | ## Description | 3 | ## Description |
| 4 | 4 | ||
| @@ -44,12 +44,11 @@ Before running these examples, carefully read the [PyTorch Environment Installat | |||
| 44 | 44 | ||
| 45 | ## Setting Environment Variables | 45 | ## Setting Environment Variables |
| 46 | 46 | ||
| 47 | +Before running the examples, ensure that the current terminal is using a Python environment with PyTorch and `torch_npu` installed. For details about environment installation and activation, see the [PyTorch Environment Installation Guide](../../../docs/env_install/pytorch/env_pytorch.md). | ||
| 48 | + | ||
| 47 | Run the following commands whenever you open a new terminal: | 49 | Run the following commands whenever you open a new terminal: |
| 48 | 50 | ||
| 49 | ```bash | 51 | ```bash |
| 50 | -# Activate the Python environment. | ||
| 51 | -source /mnt/workspace/env/venv/torch210_daily/bin/activate | ||
| 52 | - | ||
| 53 | # Set the CANN installation path based on the actual installation location. | 52 | # Set the CANN installation path based on the actual installation location. |
| 54 | export CANN_INSTALL_PATH=/home/developer/Ascend | 53 | export CANN_INSTALL_PATH=/home/developer/Ascend |
| 55 | 54 | ||
| @@ -62,26 +61,11 @@ export ASCEND_DEVICE_ID=0 | |||
| 62 | 61 | ||
| 63 | ## Running the Examples | 62 | ## Running the Examples |
| 64 | 63 | ||
| 65 | -### add + ge Fusion | 64 | +The following three examples are provided. Refer to the corresponding document based on your use case: |
| 66 | 65 | ||
| 67 | -```bash | 66 | +* [add + ge Fusion Example](./af_pointwise/README_en.md) |
| 68 | -cd af_pointwise | 67 | +* [mul + reducesum Fusion Example](./af_reduce/README_en.md) |
| 69 | -python af_add_ge.py | 68 | +* [gather + add Graph Pattern Example](./af_gather/README_en.md) |
| 70 | -``` | ||
| 71 | - | ||
| 72 | -### mul + reducesum Fusion | ||
| 73 | - | ||
| 74 | -```bash | ||
| 75 | -cd af_reduce | ||
| 76 | -python af_mul_reducesum.py | ||
| 77 | -``` | ||
| 78 | - | ||
| 79 | -### gather + add Graph Pattern | ||
| 80 | - | ||
| 81 | -```bash | ||
| 82 | -cd af_gather | ||
| 83 | -python af_gather_add.py | ||
| 84 | -``` | ||
| 85 | 69 | ||
| 86 | ## Expected Results | 70 | ## Expected Results |
| 87 | 71 | ||
| @@ -2,17 +2,30 @@ | |||
| 2 | 2 | ||
| 3 | ## 功能描述 | 3 | ## 功能描述 |
| 4 | 4 | ||
| 5 | -使用 autofuse 完成 TensorFlow 网络下的算子融合。通过 GE（Graph Engine）中的 Autofuse fusion pass 自动识别可融合算子并完成融合。 | 5 | +使用 AutoFuse 完成 TensorFlow 网络下的算子融合。通过 GE（Graph Engine）中的 AutoFuse fusion pass 自动识别可融合算子并完成融合。 |
| 6 | 6 | ||
| 7 | ## 目录结构 | 7 | ## 目录结构 |
| 8 | 8 | ||
| 9 | ```text | 9 | ```text |
| 10 | -├── README.md # 本文档 | 10 | +├── README.md |
| 11 | -├── README_en.md # 英文文档 | 11 | +├── README_en.md |
| 12 | -└── af_tf_eleandele/ # elementwise 类型算子融合的样例 | 12 | +├── common/ |
| 13 | - ├── README.md # 样例说明 | 13 | +│ ├── __init__.py |
| 14 | - ├── README_en.md # 英文样例说明 | 14 | +│ ├── config.py # 公共配置 |
| 15 | - └── test_abs_relu_exp.py # 通过 autofuse 完成 abs + relu + exp 三个 elementwise 算子的融合 | 15 | +│ ├── profiling_utils.py # Profiling 公共工具 |
| 16 | +│ └── tf_runner.py # TF1/TF2 公共运行框架 | ||
| 17 | +├── af_tf_eleandele/ | ||
| 18 | +│ ├── README.md | ||
| 19 | +│ ├── README_en.md | ||
| 20 | +│ └── test_abs_relu_exp.py # abs + relu + exp 融合样例 | ||
| 21 | +├── af_tf_eleandbroadcast/ | ||
| 22 | +│ ├── README.md | ||
| 23 | +│ ├── README_en.md | ||
| 24 | +│ └── test_abs_add_relu.py # abs + add + relu 融合样例 | ||
| 25 | +└── af_tf_eleandreduce/ | ||
| 26 | + ├── README.md | ||
| 27 | + ├── README_en.md | ||
| 28 | + └── test_abs_reducesum.py # abs + reduce_sum 融合样例 | ||
| 16 | ``` | 29 | ``` |
| 17 | 30 | ||
| 18 | ## 前置说明 | 31 | ## 前置说明 |
| @@ -58,21 +71,19 @@ export AUTOFUSE_FLAGS="--enable_autofuse=true" | |||
| 58 | 71 | ||
| 59 | ## 执行用例 | 72 | ## 执行用例 |
| 60 | 73 | ||
| 61 | -```bash | 74 | +当前包含以下三个用例，可根据实际场景参考对应文档执行： |
| 62 | -# TF 1.15 环境 | ||
| 63 | -python3 autofuse/examples/tensorflow/af_tf_eleandele/test_abs_relu_exp.py --mode tf1 | ||
| 64 | 75 | ||
| 65 | -# TF 2.6.5 环境（兼容模式） | 76 | +- [Elementwise + Elementwise](./af_tf_eleandele/README.md)：`abs + relu + exp` |
| 66 | -python3 autofuse/examples/tensorflow/af_tf_eleandele/test_abs_relu_exp.py --mode tf2-compat | 77 | +- [Elementwise + Broadcast](./af_tf_eleandbroadcast/README.md)：`abs + add + relu` |
| 67 | -``` | 78 | +- [Elementwise + Reduce](./af_tf_eleandreduce/README.md)：`abs + reduce_sum` |
| 68 | 79 | ||
| 69 | ## 预期执行结果 | 80 | ## 预期执行结果 |
| 70 | 81 | ||
| 71 | -脚本执行 100 步推理，无报错表示用例执行成功；是否发生融合，需要通过 Dump 图或 Profiling 进一步确认。 | 82 | +各用例脚本均执行 100 步推理，无报错表示用例执行成功；是否发生融合，需要通过 Dump 图或 Profiling 进一步确认。 |
| 72 | 83 | ||
| 73 | ## 参考 | 84 | ## 参考 |
| 74 | 85 | ||
| 75 | -- [Autofuse 简介](../../README.md) | 86 | +- [AutoFuse 简介](../../README.md) |
| 76 | - [环境编译部署](../../../docs/env_install/tensorflow/env_tf.md) | 87 | - [环境编译部署](../../../docs/env_install/tensorflow/env_tf.md) |
| 77 | - [aarch64 架构 TF 源码编译](../../../docs/env_install/tensorflow/build_tf_aarch64.md) | 88 | - [aarch64 架构 TF 源码编译](../../../docs/env_install/tensorflow/build_tf_aarch64.md) |
| 78 | - [精度调试工具指南](https://hiascend.com/document/redirect/CannCommunityToolAccucacy) | 89 | - [精度调试工具指南](https://hiascend.com/document/redirect/CannCommunityToolAccucacy) |
| @@ -1,18 +1,31 @@ | |||
| 1 | # TensorFlow Scenario Examples | 1 | # TensorFlow Scenario Examples |
| 2 | 2 | ||
| 3 | -## Function Description | 3 | +## Description |
| 4 | 4 | ||
| 5 | Use AutoFuse to perform operator fusion in TensorFlow networks. The AutoFuse fusion pass in GE (Graph Engine) automatically identifies operators that can be fused and completes the fusion. | 5 | Use AutoFuse to perform operator fusion in TensorFlow networks. The AutoFuse fusion pass in GE (Graph Engine) automatically identifies operators that can be fused and completes the fusion. |
| 6 | 6 | ||
| 7 | ## Directory Structure | 7 | ## Directory Structure |
| 8 | 8 | ||
| 9 | ```text | 9 | ```text |
| 10 | -├── README.md # Chinese documentation | 10 | +├── README.md |
| 11 | -├── README_en.md # English documentation | 11 | +├── README_en.md |
| 12 | -└── af_tf_eleandele/ # Example of elementwise operator fusion | 12 | +├── common/ |
| 13 | - ├── README.md # Chinese example description | 13 | +│ ├── __init__.py |
| 14 | - ├── README_en.md # English example description | 14 | +│ ├── config.py # Common configuration |
| 15 | - └── test_abs_relu_exp.py # Uses AutoFuse to fuse the abs, relu, and exp elementwise operators | 15 | +│ ├── profiling_utils.py # Common Profiling utilities |
| 16 | +│ └── tf_runner.py # Common TF1/TF2 runtime framework | ||
| 17 | +├── af_tf_eleandele/ | ||
| 18 | +│ ├── README.md | ||
| 19 | +│ ├── README_en.md | ||
| 20 | +│ └── test_abs_relu_exp.py # abs + relu + exp fusion example | ||
| 21 | +├── af_tf_eleandbroadcast/ | ||
| 22 | +│ ├── README.md | ||
| 23 | +│ ├── README_en.md | ||
| 24 | +│ └── test_abs_add_relu.py # abs + add + relu fusion example | ||
| 25 | +└── af_tf_eleandreduce/ | ||
| 26 | + ├── README.md | ||
| 27 | + ├── README_en.md | ||
| 28 | + └── test_abs_reducesum.py # abs + reduce_sum fusion example | ||
| 16 | ``` | 29 | ``` |
| 17 | 30 | ||
| 18 | ## Prerequisites | 31 | ## Prerequisites |
| @@ -58,19 +71,17 @@ export ASCEND_DEVICE_ID=0 | |||
| 58 | export AUTOFUSE_FLAGS="--enable_autofuse=true" | 71 | export AUTOFUSE_FLAGS="--enable_autofuse=true" |
| 59 | ``` | 72 | ``` |
| 60 | 73 | ||
| 61 | -## Run the Example | 74 | +## Run the Examples |
| 62 | 75 | ||
| 63 | -```bash | 76 | +The following three examples are available. Refer to the corresponding document based on the scenario you want to run: |
| 64 | -# TensorFlow 1.15 environment | ||
| 65 | -python3 autofuse/examples/tensorflow/af_tf_eleandele/test_abs_relu_exp.py --mode tf1 | ||
| 66 | 77 | ||
| 67 | -# TensorFlow 2.6.5 environment in compatibility mode | 78 | +- [Elementwise + Elementwise](./af_tf_eleandele/README_en.md): `abs + relu + exp` |
| 68 | -python3 autofuse/examples/tensorflow/af_tf_eleandele/test_abs_relu_exp.py --mode tf2-compat | 79 | +- [Elementwise + Broadcast](./af_tf_eleandbroadcast/README_en.md): `abs + add + relu` |
| 69 | -``` | 80 | +- [Elementwise + Reduce](./af_tf_eleandreduce/README_en.md): `abs + reduce_sum` |
| 70 | 81 | ||
| 71 | ## Expected Result | 82 | ## Expected Result |
| 72 | 83 | ||
| 73 | -The script performs 100 inference steps. If no errors occur, the sample is considered to have run successfully. Whether operator fusion has occurred should be further verified using graph dumps or profiling data. | 84 | +Each example performs 100 inference steps. If no errors occur, the example has executed successfully. Use graph dump files or Profiling data to confirm whether fusion takes effect. |
| 74 | 85 | ||
| 75 | ## References | 86 | ## References |
| 76 | 87 | ||
| @@ -0,0 +1,38 @@ | |||
| 1 | +# Elementwise + Broadcast 融合样例（abs + add + relu） | ||
| 2 | + | ||
| 3 | +## 用例功能 | ||
| 4 | + | ||
| 5 | +AutoFuse 融合 `abs + add + relu` 三个算子。其中 `data1` 的 shape 为 `[128, 192]`，`data2` 的 shape 为 `[192]`，`add` 计算时会将 `data2` Broadcast 到 `[128, 192]`，用于验证 Elementwise 与 Broadcast 类算子的融合能力。 | ||
| 6 | + | ||
| 7 | +脚本通过 `--mode` 参数选择 TensorFlow 版本： | ||
| 8 | + | ||
| 9 | +| 模式 | TF 版本 | NPU 接入方式 | 图 API | | ||
| 10 | +|------|---------|-------------|--------| | ||
| 11 | +| `tf1` | TF 1.15.0 | `npu_bridge`（import 副作用注册） | `tf.placeholder` + `Session` + `NpuOptimizer` | | ||
| 12 | +| `tf2-compat` | TF 2.6.5 | `npu_device.compat.enable_v1()` | `tf.compat.v1.placeholder` + `tf.compat.v1.Session` | | ||
| 13 | + | ||
| 14 | +## 执行命令 | ||
| 15 | + | ||
| 16 | +先在 graph-autofusion 仓库根目录激活对应 TensorFlow 环境，并确保已开启 AutoFuse： | ||
| 17 | + | ||
| 18 | +```bash | ||
| 19 | +# TF 1.15 环境 | ||
| 20 | +source scripts/env_install/tensorflow/env/activate_tf1.sh | ||
| 21 | +export AUTOFUSE_FLAGS="--enable_autofuse=true" | ||
| 22 | +python3 autofuse/examples/tensorflow/af_tf_eleandbroadcast/test_abs_add_relu.py --mode tf1 | ||
| 23 | +``` | ||
| 24 | + | ||
| 25 | +或： | ||
| 26 | + | ||
| 27 | +```bash | ||
| 28 | +# TF 2.6.5 环境（兼容模式） | ||
| 29 | +source scripts/env_install/tensorflow/env/activate_tf2.sh | ||
| 30 | +export AUTOFUSE_FLAGS="--enable_autofuse=true" | ||
| 31 | +python3 autofuse/examples/tensorflow/af_tf_eleandbroadcast/test_abs_add_relu.py --mode tf2-compat | ||
| 32 | +``` | ||
| 33 | + | ||
| 34 | +## 预期执行结果 | ||
| 35 | + | ||
| 36 | +脚本构造 `abs → add(Broadcast) → relu` 计算图，并在 NPU 上执行 100 步推理。脚本执行无报错表示用例执行成功；是否发生融合，需要通过 Dump 图或 Profiling 进一步确认。 | ||
| 37 | + | ||
| 38 | +脚本已内置 Profiling 配置。按照上述命令执行后，在 `./profiling/PROF_*/mindstudio_profiler_output/op_summary_*.csv` 中查看算子执行情况。融合生效时，可以观察到包含 `Abs`、`Add`、`Relu` 的 AutoFuse 融合 Kernel（通常类似 `autofuse_pointwise_0_Abs_Add_Relu`），且不再出现对应的独立 `Abs`、`Add`、`Relu` Kernel。 | ||
| @@ -0,0 +1,38 @@ | |||
| 1 | +# Elementwise + Broadcast Fusion Example (abs + add + relu) | ||
| 2 | + | ||
| 3 | +## Function Description | ||
| 4 | + | ||
| 5 | +AutoFuse fuses the `abs + add + relu` operators. `data1` has shape `[128, 192]`, while `data2` has shape `[192]`. During the `add` operation, `data2` is broadcast to `[128, 192]`. This example verifies fusion between Elementwise and Broadcast operators. | ||
| 6 | + | ||
| 7 | +The script uses the `--mode` parameter to select the TensorFlow version: | ||
| 8 | + | ||
| 9 | +| Mode | TensorFlow Version | NPU Integration Method | Graph API | | ||
| 10 | +|------|--------------------|------------------------|-----------| | ||
| 11 | +| `tf1` | TensorFlow 1.15.0 | `npu_bridge` (registered through import side effects) | `tf.placeholder` + `Session` + `NpuOptimizer` | | ||
| 12 | +| `tf2-compat` | TensorFlow 2.6.5 | `npu_device.compat.enable_v1()` | `tf.compat.v1.placeholder` + `tf.compat.v1.Session` | | ||
| 13 | + | ||
| 14 | +## Execution Commands | ||
| 15 | + | ||
| 16 | +Activate the corresponding TensorFlow environment from the graph-autofusion repository root and make sure AutoFuse is enabled: | ||
| 17 | + | ||
| 18 | +```bash | ||
| 19 | +# TensorFlow 1.15 environment | ||
| 20 | +source scripts/env_install/tensorflow/env/activate_tf1.sh | ||
| 21 | +export AUTOFUSE_FLAGS="--enable_autofuse=true" | ||
| 22 | +python3 autofuse/examples/tensorflow/af_tf_eleandbroadcast/test_abs_add_relu.py --mode tf1 | ||
| 23 | +``` | ||
| 24 | + | ||
| 25 | +Or: | ||
| 26 | + | ||
| 27 | +```bash | ||
| 28 | +# TensorFlow 2.6.5 environment (compatibility mode) | ||
| 29 | +source scripts/env_install/tensorflow/env/activate_tf2.sh | ||
| 30 | +export AUTOFUSE_FLAGS="--enable_autofuse=true" | ||
| 31 | +python3 autofuse/examples/tensorflow/af_tf_eleandbroadcast/test_abs_add_relu.py --mode tf2-compat | ||
| 32 | +``` | ||
| 33 | + | ||
| 34 | +## Expected Result | ||
| 35 | + | ||
| 36 | +The script constructs an `abs → add(Broadcast) → relu` computation graph and performs 100 inference steps on the NPU. If the script finishes without errors, the example has executed successfully. Use graph dump files or Profiling data to confirm whether fusion takes effect. | ||
| 37 | + | ||
| 38 | +Profiling is already configured in the script. After running the commands above, check `./profiling/PROF_*/mindstudio_profiler_output/op_summary_*.csv`. When fusion takes effect, an AutoFuse fused Kernel containing `Abs`, `Add`, and `Relu` can be observed, typically with a name similar to `autofuse_pointwise_0_Abs_Add_Relu`, while the corresponding standalone `Abs`, `Add`, and `Relu` Kernels no longer appear. | ||
| @@ -0,0 +1,53 @@ | |||
| 1 | +#!/usr/bin/env python3 | ||
| 2 | +# -*- coding: utf-8 -*- | ||
| 3 | +# ----------------------------------------------------------------------------------------------------------- | ||
| 4 | +# Copyright (c) 2026 Huawei Technologies Co., Ltd. | ||
| 5 | +# This program is free software, you can redistribute it and/or modify it under the terms and conditions of | ||
| 6 | +# CANN Open Software License Agreement Version 2.0 (the "License"). | ||
| 7 | +# Please refer to the License for details. You may not use this file except in compliance with the License. | ||
| 8 | +# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, | ||
| 9 | +# INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. | ||
| 10 | +# See LICENSE in the root of the software repository for the full text of the License. | ||
| 11 | +# ----------------------------------------------------------------------------------------------------------- | ||
| 12 | +# | ||
| 13 | +# TensorFlow 场景 AutoFuse 示例（abs -> add(broadcast) -> relu）。 | ||
| 14 | +# 支持 TF1（npu_bridge）和 TF2 兼容模式（npu_device.compat），通过 --mode 选择。 | ||
| 15 | +# | ||
| 16 | +# 用法： | ||
| 17 | +# TF1 环境： python3 test_abs_add_relu.py --mode tf1 | ||
| 18 | +# TF2 环境： python3 test_abs_add_relu.py --mode tf2-compat | ||
| 19 | +# | ||
| 20 | + | ||
| 21 | +import os | ||
| 22 | +import sys | ||
| 23 | + | ||
| 24 | +import numpy as np | ||
| 25 | +import tensorflow as tf | ||
| 26 | + | ||
| 27 | +# 将 tensorflow 示例根目录加入模块搜索路径，便于直接运行当前脚本时复用 common 中的公共能力。 | ||
| 28 | +_TENSORFLOW_EXAMPLE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) | ||
| 29 | +sys.path.insert(0, _TENSORFLOW_EXAMPLE_DIR) | ||
| 30 | + | ||
| 31 | +from common.tf_runner import run_example # noqa: E402 | ||
| 32 | + | ||
| 33 | + | ||
| 34 | +def build_model(placeholder_fn): | ||
| 35 | + """构建 abs -> add(broadcast) -> relu 计算图及对应输入数据。""" | ||
| 36 | + data1 = placeholder_fn(tf.float16, shape=[128, 192]) | ||
| 37 | + data2 = placeholder_fn(tf.float16, shape=[192]) | ||
| 38 | + | ||
| 39 | + input_data1 = np.random.uniform(-1.0, 1.0, size=(128, 192)).astype(np.float16) | ||
| 40 | + input_data2 = np.random.uniform(-1.0, 1.0, size=(192,)).astype(np.float16) | ||
| 41 | + | ||
| 42 | + abs_0 = tf.abs(data1) | ||
| 43 | + add_0 = tf.add(abs_0, data2) | ||
| 44 | + relu_0 = tf.nn.relu(add_0) | ||
| 45 | + | ||
| 46 | + return relu_0, {data1: input_data1, data2: input_data2} | ||
| 47 | + | ||
| 48 | + | ||
| 49 | +if __name__ == "__main__": | ||
| 50 | + run_example( | ||
| 51 | + build_model, | ||
| 52 | + description="AutoFuse abs-add(broadcast)-relu 示例", | ||
| 53 | + ) | ||
| @@ -1,16 +1,14 @@ | |||
| 1 | -# elementwise + elementwise 融合样例（abs + relu + exp） | 1 | +# Elementwise + Elementwise 融合样例（abs + relu + exp） |
朱 | |||
| 2 | 2 | ||
| 3 | ## 用例功能 | 3 | ## 用例功能 |
| 4 | 4 | ||
| 5 | -autofuse 融合 `abs + relu + exp` 三个 elementwise 算子。脚本通过 `--mode` 参数选择 TensorFlow 版本： | 5 | +AutoFuse 融合 `abs + relu + exp` 三个 Elementwise 算子。脚本通过 `--mode` 参数选择 TensorFlow 版本： |
| 6 | 6 | ||
| 7 | | 模式 | TF 版本 | NPU 接入方式 | 图 API | | 7 | | 模式 | TF 版本 | NPU 接入方式 | 图 API | |
| 8 | |------|---------|-------------|--------| | 8 | |------|---------|-------------|--------| |
| 9 | | `tf1` | TF 1.15.0 | `npu_bridge`（import 副作用注册） | `tf.placeholder` + `Session` + `NpuOptimizer` | | 9 | | `tf1` | TF 1.15.0 | `npu_bridge`（import 副作用注册） | `tf.placeholder` + `Session` + `NpuOptimizer` | |
| 10 | | `tf2-compat` | TF 2.6.5 | `npu_device.compat.enable_v1()` | `tf.compat.v1.placeholder` + `tf.compat.v1.Session` | | 10 | | `tf2-compat` | TF 2.6.5 | `npu_device.compat.enable_v1()` | `tf.compat.v1.placeholder` + `tf.compat.v1.Session` | |
| 11 | 11 | ||
| 12 | -。 | ||
| 13 | - | ||
| 14 | ## 执行命令 | 12 | ## 执行命令 |
| 15 | 13 | ||
| 16 | 以下命令均在 graph-autofusion 仓库根目录执行。 | 14 | 以下命令均在 graph-autofusion 仓库根目录执行。 |
| @@ -18,15 +16,17 @@ autofuse 融合 `abs + relu + exp` 三个 elementwise 算子。脚本通过 `--m | |||
| 18 | ```bash | 16 | ```bash |
| 19 | # TF 1.15 环境 | 17 | # TF 1.15 环境 |
| 20 | source scripts/env_install/tensorflow/env/activate_tf1.sh | 18 | source scripts/env_install/tensorflow/env/activate_tf1.sh |
| 19 | +export AUTOFUSE_FLAGS="--enable_autofuse=true" | ||
| 21 | python3 autofuse/examples/tensorflow/af_tf_eleandele/test_abs_relu_exp.py --mode tf1 | 20 | python3 autofuse/examples/tensorflow/af_tf_eleandele/test_abs_relu_exp.py --mode tf1 |
| 22 | 21 | ||
| 23 | # TF 2.6.5 环境（兼容模式） | 22 | # TF 2.6.5 环境（兼容模式） |
| 24 | source scripts/env_install/tensorflow/env/activate_tf2.sh | 23 | source scripts/env_install/tensorflow/env/activate_tf2.sh |
| 24 | +export AUTOFUSE_FLAGS="--enable_autofuse=true" | ||
| 25 | python3 autofuse/examples/tensorflow/af_tf_eleandele/test_abs_relu_exp.py --mode tf2-compat | 25 | python3 autofuse/examples/tensorflow/af_tf_eleandele/test_abs_relu_exp.py --mode tf2-compat |
| 26 | ``` | 26 | ``` |
| 27 | 27 | ||
| 28 | ## 预期执行结果 | 28 | ## 预期执行结果 |
| 29 | 29 | ||
| 30 | -脚本构造 `abs → relu → exp` 计算图，在 NPU 上执行 100 步推理，无报错即表示融合成功。三个算子被融合为一个 `AscBackend` 类型的融合算子 `autofuse_pointwise_0_Abs_Relu_Exp`，在 NPU 上以单个 kernel 执行。 | 30 | +脚本构造 `abs → relu → exp` 计算图，并在 NPU 上执行 100 步推理。脚本执行无报错表示用例执行成功；是否发生融合，需要通过 Dump 图或 Profiling 进一步确认。 |
| 31 | 31 | ||
| 32 | -如需查看融合效果，可开启 profiling（脚本已内置 profiling 配置），执行完成后在仓库根目录的 `./profiling` 目录下查看 `PROF_*/mindstudio_profiler_output/op_summary_*.csv`，此时 Profiling 中存在 autofuse_pointwise_0_Abs_Relu_Exp，且不再出现独立的 Abs、Relu、Exp Kernel，表示三个算子已融合为一个融合算子。 | 32 | +脚本已内置 Profiling 配置。按照上述命令执行后，在 `./profiling/PROF_*/mindstudio_profiler_output/op_summary_*.csv` 中查看算子执行情况。融合生效时，可以观察到包含 `Abs`、`Relu`、`Exp` 的 AutoFuse 融合 Kernel（通常类似 `autofuse_pointwise_0_Abs_Relu_Exp`），且不再出现对应的独立 `Abs`、`Relu`、`Exp` Kernel。 |
| @@ -16,15 +16,17 @@ Run all the following commands from the graph-autofusion repository root. | |||
| 16 | ```bash | 16 | ```bash |
| 17 | # TensorFlow 1.15 environment | 17 | # TensorFlow 1.15 environment |
| 18 | source scripts/env_install/tensorflow/env/activate_tf1.sh | 18 | source scripts/env_install/tensorflow/env/activate_tf1.sh |
| 19 | +export AUTOFUSE_FLAGS="--enable_autofuse=true" | ||
| 19 | python3 autofuse/examples/tensorflow/af_tf_eleandele/test_abs_relu_exp.py --mode tf1 | 20 | python3 autofuse/examples/tensorflow/af_tf_eleandele/test_abs_relu_exp.py --mode tf1 |
| 20 | 21 | ||
| 21 | # TensorFlow 2.6.5 environment (compatibility mode) | 22 | # TensorFlow 2.6.5 environment (compatibility mode) |
| 22 | source scripts/env_install/tensorflow/env/activate_tf2.sh | 23 | source scripts/env_install/tensorflow/env/activate_tf2.sh |
| 24 | +export AUTOFUSE_FLAGS="--enable_autofuse=true" | ||
| 23 | python3 autofuse/examples/tensorflow/af_tf_eleandele/test_abs_relu_exp.py --mode tf2-compat | 25 | python3 autofuse/examples/tensorflow/af_tf_eleandele/test_abs_relu_exp.py --mode tf2-compat |
| 24 | ``` | 26 | ``` |
| 25 | 27 | ||
| 26 | ## Expected Result | 28 | ## Expected Result |
| 27 | 29 | ||
| 28 | -The script constructs an `abs → relu → exp` computation graph and performs 100 inference steps on the NPU. If no error is reported, the fusion is successful. The three operators are fused into an `AscBackend`-type fused operator named `autofuse_pointwise_0_Abs_Relu_Exp`, which is executed as a single Kernel on the NPU. | 30 | +The script constructs an `abs → relu → exp` computation graph and performs 100 inference steps on the NPU. If the script finishes without errors, the example has executed successfully. Use graph dump files or Profiling data to confirm whether fusion takes effect. |
| 29 | 31 | ||
| 30 | -To view the fusion result, enable profiling (the script already includes the profiling configuration). After execution, check `PROF_*/mindstudio_profiler_output/op_summary_*.csv` under the `./profiling` directory in the repository root. If `autofuse_pointwise_0_Abs_Relu_Exp` appears in the profiling data and the standalone `Abs`, `Relu`, and `Exp` kernels no longer appear, it indicates that the three operators have been fused into a single fused operator. | 32 | +Profiling is already configured in the script. After running the commands above, check `./profiling/PROF_*/mindstudio_profiler_output/op_summary_*.csv`. When fusion takes effect, an AutoFuse fused Kernel containing `Abs`, `Relu`, and `Exp` can be observed, typically with a name similar to `autofuse_pointwise_0_Abs_Relu_Exp`, while the corresponding standalone `Abs`, `Relu`, and `Exp` Kernels no longer appear. |
| @@ -10,7 +10,7 @@ | |||
| 10 | # See LICENSE in the root of the software repository for the full text of the License. | 10 | # See LICENSE in the root of the software repository for the full text of the License. |
| 11 | # ----------------------------------------------------------------------------------------------------------- | 11 | # ----------------------------------------------------------------------------------------------------------- |
| 12 | # | 12 | # |
| 13 | -# TensorFlow 场景 autofuse 示例（abs -> relu -> exp 逐元素融合）。 | 13 | +# TensorFlow 场景 AutoFuse 示例（abs -> relu -> exp 逐元素融合）。 |
| 14 | # 支持 TF1（npu_bridge）和 TF2 兼容模式（npu_device.compat），通过 --mode 选择。 | 14 | # 支持 TF1（npu_bridge）和 TF2 兼容模式（npu_device.compat），通过 --mode 选择。 |
| 15 | # | 15 | # |
| 16 | # 用法： | 16 | # 用法： |
| @@ -18,86 +18,33 @@ | |||
| 18 | # TF2 环境： python3 test_abs_relu_exp.py --mode tf2-compat | 18 | # TF2 环境： python3 test_abs_relu_exp.py --mode tf2-compat |
| 19 | # | 19 | # |
| 20 | 20 | ||
| 21 | -import argparse | ||
| 22 | -import glob | ||
| 23 | import os | 21 | import os |
| 24 | -import subprocess | 22 | +import sys |
| 25 | - | ||
| 26 | 23 | ||
| 27 | import numpy as np | 24 | import numpy as np |
| 28 | import tensorflow as tf | 25 | import tensorflow as tf |
| 29 | 26 | ||
| 30 | -_PROFILING_DIR = os.path.abspath("./profiling") | 27 | +# 将 tensorflow 示例根目录加入模块搜索路径，便于直接运行当前脚本时复用 common 中的公共能力。 |
| 31 | -_PROFILES_JSON = ( | 28 | +_TENSORFLOW_EXAMPLE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) |
| 32 | - '{"output":"%s","training_trace":"on","task_time":"on",' | 29 | +sys.path.insert(0, _TENSORFLOW_EXAMPLE_DIR) |
| 33 | - '"hccl":"on","aicpu":"on","aic_metrics":"PipeUtilization","msproftx":"off"}' | 30 | + |
| 34 | -) % _PROFILING_DIR | 31 | +from common.tf_runner import run_example # noqa: E402 |
| 35 | 32 | ||
| 36 | 33 | ||
| 37 | -def configure_npu(sess_config): | 34 | +def build_model(placeholder_fn): |
| 38 | - """在已有 sess_config 上配置 NpuOptimizer（推理模式 + profiling）。""" | 35 | + """构建 abs -> relu -> exp 计算图及对应输入数据。""" |
| 39 | - custom_op = sess_config.graph_options.rewrite_options.custom_optimizers.add() | ||
| 40 | - custom_op.name = "NpuOptimizer" | ||
| 41 | - custom_op.parameter_map["use_off_line"].b = True | ||
| 42 | - custom_op.parameter_map["graph_run_mode"].i = 0 | ||
| 43 | - custom_op.parameter_map["profiling_mode"].b = True | ||
| 44 | - custom_op.parameter_map["profiling_options"].s = tf.compat.as_bytes(_PROFILES_JSON) | ||
| 45 | - return sess_config | ||
| 46 | - | ||
| 47 | - | ||
| 48 | -def get_profile_dirs(): | ||
| 49 | - """获取当前已有的 PROF_* 目录。""" | ||
| 50 | - profile_pattern = os.path.join(_PROFILING_DIR, "PROF_*") | ||
| 51 | - return set(glob.glob(profile_pattern)) | ||
| 52 | - | ||
| 53 | - | ||
| 54 | -def export_new_profiling(profile_dirs_before): | ||
| 55 | - """解析本次执行新生成的 Profiling 数据。""" | ||
| 56 | - for profile_dir in sorted(get_profile_dirs() - profile_dirs_before): | ||
| 57 | - subprocess.run( | ||
| 58 | - ["msprof", "--export=on", "--output={}".format(profile_dir)], | ||
| 59 | - check=True, | ||
| 60 | - ) | ||
| 61 | - | ||
| 62 | - | ||
| 63 | -def run_model(placeholder_fn, configproto_fn): | ||
| 64 | - """构建 abs->relu->exp 模型并在 NPU 上执行 100 步。""" | ||
| 65 | - profile_dirs_before = get_profile_dirs() | ||
| 66 | data1 = placeholder_fn(tf.float16, shape=[128, 192]) | 36 | data1 = placeholder_fn(tf.float16, shape=[128, 192]) |
| 67 | input_data = np.random.rand(128, 192).astype(np.float16) | 37 | input_data = np.random.rand(128, 192).astype(np.float16) |
| 38 | + | ||
| 68 | abs_0 = tf.abs(data1) | 39 | abs_0 = tf.abs(data1) |
| 69 | relu_0 = tf.nn.relu(abs_0) | 40 | relu_0 = tf.nn.relu(abs_0) |
| 70 | exp_0 = tf.exp(relu_0) | 41 | exp_0 = tf.exp(relu_0) |
| 71 | - sess_config = configproto_fn(allow_soft_placement=True, log_device_placement=False) | ||
| 72 | - configure_npu(sess_config) | ||
| 73 | - feed_dict = {data1: input_data} | ||
| 74 | - step = 100 | ||
| 75 | - with tf.compat.v1.Session(config=sess_config) as sess: | ||
| 76 | - for _ in range(step): | ||
| 77 | - sess.run(exp_0, feed_dict=feed_dict) | ||
| 78 | - export_new_profiling(profile_dirs_before) | ||
| 79 | 42 | ||
| 43 | + return exp_0, {data1: input_data} | ||
| 80 | 44 | ||
| 81 | -def run_tf1(): | ||
| 82 | - """TF1 场景：通过 npu_bridge 注册 NPU 算子。""" | ||
| 83 | - import npu_bridge | ||
| 84 | - | ||
| 85 | - _ = npu_bridge # 通过import副作用注册NPU算子 | ||
| 86 | - run_model(tf.placeholder, tf.ConfigProto) | ||
| 87 | - | ||
| 88 | - | ||
| 89 | -def run_tf2_compat(): | ||
| 90 | - """TF2 兼容场景：通过 npu_device.compat 切到 v1 行为。""" | ||
| 91 | - import npu_device | ||
| 92 | - import npu_device.compat | ||
| 93 | - | ||
| 94 | - npu_device.compat.enable_v1() | ||
| 95 | - run_model(tf.compat.v1.placeholder, tf.compat.v1.ConfigProto) | ||
| 96 | - | ||
| 97 | - | ||
| 98 | -_MODES = {"tf1": run_tf1, "tf2-compat": run_tf2_compat} | ||
| 99 | 45 | ||
| 100 | if __name__ == "__main__": | 46 | if __name__ == "__main__": |
| 101 | - parser = argparse.ArgumentParser(description="autofuse abs-relu-exp 示例") | 47 | + run_example( |
| 102 | - parser.add_argument("--mode", choices=list(_MODES), required=True) | 48 | + build_model, |
| 103 | - _MODES[parser.parse_args().mode]() | 49 | + description="AutoFuse abs-relu-exp 示例", |
| 50 | + ) | ||
| @@ -0,0 +1,38 @@ | |||
| 1 | +# Elementwise + Reduce 融合样例（abs + reduce_sum） | ||
| 2 | + | ||
| 3 | +## 用例功能 | ||
| 4 | + | ||
| 5 | +AutoFuse 融合 `abs + reduce_sum` 两个算子。输入 shape 为 `[128, 192]`，先执行 Elementwise 类型的 `abs`，再沿 `axis=1` 执行 Reduce 类型的 `reduce_sum`，用于验证 Elementwise 与 Reduce 类算子的融合能力。 | ||
| 6 | + | ||
| 7 | +脚本通过 `--mode` 参数选择 TensorFlow 版本： | ||
| 8 | + | ||
| 9 | +| 模式 | TF 版本 | NPU 接入方式 | 图 API | | ||
| 10 | +|------|---------|-------------|--------| | ||
| 11 | +| `tf1` | TF 1.15.0 | `npu_bridge`（import 副作用注册） | `tf.placeholder` + `Session` + `NpuOptimizer` | | ||
| 12 | +| `tf2-compat` | TF 2.6.5 | `npu_device.compat.enable_v1()` | `tf.compat.v1.placeholder` + `tf.compat.v1.Session` | | ||
| 13 | + | ||
| 14 | +## 执行命令 | ||
| 15 | + | ||
| 16 | +Reduce 融合默认不使能。先在 graph-autofusion 仓库根目录激活对应 TensorFlow 环境，再显式开启 Reduce 融合： | ||
| 17 | + | ||
| 18 | +```bash | ||
| 19 | +# TF 1.15 环境 | ||
| 20 | +source scripts/env_install/tensorflow/env/activate_tf1.sh | ||
| 21 | +export AUTOFUSE_FLAGS="--enable_autofuse=true;--autofuse_enable_pass=reduce" | ||
| 22 | +python3 autofuse/examples/tensorflow/af_tf_eleandreduce/test_abs_reducesum.py --mode tf1 | ||
| 23 | +``` | ||
| 24 | + | ||
| 25 | +或： | ||
| 26 | + | ||
| 27 | +```bash | ||
| 28 | +# TF 2.6.5 环境（兼容模式） | ||
| 29 | +source scripts/env_install/tensorflow/env/activate_tf2.sh | ||
| 30 | +export AUTOFUSE_FLAGS="--enable_autofuse=true;--autofuse_enable_pass=reduce" | ||
| 31 | +python3 autofuse/examples/tensorflow/af_tf_eleandreduce/test_abs_reducesum.py --mode tf2-compat | ||
| 32 | +``` | ||
| 33 | + | ||
| 34 | +## 预期执行结果 | ||
| 35 | + | ||
| 36 | +脚本构造 `abs → reduce_sum` 计算图，并在 NPU 上执行 100 步推理。脚本执行无报错表示用例执行成功；是否发生融合，需要通过 Dump 图或 Profiling 进一步确认。 | ||
| 37 | + | ||
| 38 | +脚本已内置 Profiling 配置。按照上述命令执行后，在 `./profiling/PROF_*/mindstudio_profiler_output/op_summary_*.csv` 中查看算子执行情况。融合生效时，可以观察到 `autofuse_reduce_` 前缀的融合 Kernel，其中包含 `Abs` 和 ReduceSum 计算，且不再出现对应的独立 `Abs`、`ReduceSum` Kernel。不同 CANN 版本生成的融合 Kernel 名称可能略有差异。 | ||
| @@ -0,0 +1,38 @@ | |||
| 1 | +# Elementwise + Reduce Fusion Example (abs + reduce_sum) | ||
| 2 | + | ||
| 3 | +## Function Description | ||
| 4 | + | ||
| 5 | +AutoFuse fuses the `abs + reduce_sum` operators. The input shape is `[128, 192]`. The graph first performs the Elementwise `abs` operation and then performs the Reduce `reduce_sum` operation along `axis=1`. This example verifies fusion between Elementwise and Reduce operators. | ||
| 6 | + | ||
| 7 | +The script uses the `--mode` parameter to select the TensorFlow version: | ||
| 8 | + | ||
| 9 | +| Mode | TensorFlow Version | NPU Integration Method | Graph API | | ||
| 10 | +|------|--------------------|------------------------|-----------| | ||
| 11 | +| `tf1` | TensorFlow 1.15.0 | `npu_bridge` (registered through import side effects) | `tf.placeholder` + `Session` + `NpuOptimizer` | | ||
| 12 | +| `tf2-compat` | TensorFlow 2.6.5 | `npu_device.compat.enable_v1()` | `tf.compat.v1.placeholder` + `tf.compat.v1.Session` | | ||
| 13 | + | ||
| 14 | +## Execution Commands | ||
| 15 | + | ||
| 16 | +Reduce fusion is disabled by default. Activate the corresponding TensorFlow environment from the graph-autofusion repository root, and then explicitly enable Reduce fusion: | ||
| 17 | + | ||
| 18 | +```bash | ||
| 19 | +# TensorFlow 1.15 environment | ||
| 20 | +source scripts/env_install/tensorflow/env/activate_tf1.sh | ||
| 21 | +export AUTOFUSE_FLAGS="--enable_autofuse=true;--autofuse_enable_pass=reduce" | ||
| 22 | +python3 autofuse/examples/tensorflow/af_tf_eleandreduce/test_abs_reducesum.py --mode tf1 | ||
| 23 | +``` | ||
| 24 | + | ||
| 25 | +Or: | ||
| 26 | + | ||
| 27 | +```bash | ||
| 28 | +# TensorFlow 2.6.5 environment (compatibility mode) | ||
| 29 | +source scripts/env_install/tensorflow/env/activate_tf2.sh | ||
| 30 | +export AUTOFUSE_FLAGS="--enable_autofuse=true;--autofuse_enable_pass=reduce" | ||
| 31 | +python3 autofuse/examples/tensorflow/af_tf_eleandreduce/test_abs_reducesum.py --mode tf2-compat | ||
| 32 | +``` | ||
| 33 | + | ||
| 34 | +## Expected Result | ||
| 35 | + | ||
| 36 | +The script constructs an `abs → reduce_sum` computation graph and performs 100 inference steps on the NPU. If the script finishes without errors, the example has executed successfully. Use graph dump files or Profiling data to confirm whether fusion takes effect. | ||
| 37 | + | ||
| 38 | +Profiling is already configured in the script. After running the commands above, check `./profiling/PROF_*/mindstudio_profiler_output/op_summary_*.csv`. When fusion takes effect, a fused Kernel with the `autofuse_reduce_` prefix can be observed. The fused Kernel contains the `Abs` and ReduceSum computations, while the corresponding standalone `Abs` and `ReduceSum` Kernels no longer appear. The exact fused Kernel name may vary slightly between CANN versions. | ||
| @@ -0,0 +1,49 @@ | |||
| 1 | +#!/usr/bin/env python3 | ||
| 2 | +# -*- coding: utf-8 -*- | ||
| 3 | +# ----------------------------------------------------------------------------------------------------------- | ||
| 4 | +# Copyright (c) 2026 Huawei Technologies Co., Ltd. | ||
| 5 | +# This program is free software, you can redistribute it and/or modify it under the terms and conditions of | ||
| 6 | +# CANN Open Software License Agreement Version 2.0 (the "License"). | ||
| 7 | +# Please refer to the License for details. You may not use this file except in compliance with the License. | ||
| 8 | +# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, | ||
| 9 | +# INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. | ||
| 10 | +# See LICENSE in the root of the software repository for the full text of the License. | ||
| 11 | +# ----------------------------------------------------------------------------------------------------------- | ||
| 12 | +# | ||
| 13 | +# TensorFlow 场景 AutoFuse 示例（abs -> reduce_sum）。 | ||
| 14 | +# 支持 TF1（npu_bridge）和 TF2 兼容模式（npu_device.compat），通过 --mode 选择。 | ||
| 15 | +# | ||
| 16 | +# 用法： | ||
| 17 | +# TF1 环境： python3 test_abs_reducesum.py --mode tf1 | ||
| 18 | +# TF2 环境： python3 test_abs_reducesum.py --mode tf2-compat | ||
| 19 | +# | ||
| 20 | + | ||
| 21 | +import os | ||
| 22 | +import sys | ||
| 23 | + | ||
| 24 | +import numpy as np | ||
| 25 | +import tensorflow as tf | ||
| 26 | + | ||
| 27 | +# 将 tensorflow 示例根目录加入模块搜索路径，便于直接运行当前脚本时复用 common 中的公共能力。 | ||
| 28 | +_TENSORFLOW_EXAMPLE_DIR = os.path.dirname(os.path.dirname(os.path.abspath(__file__))) | ||
| 29 | +sys.path.insert(0, _TENSORFLOW_EXAMPLE_DIR) | ||
| 30 | + | ||
| 31 | +from common.tf_runner import run_example # noqa: E402 | ||
| 32 | + | ||
| 33 | + | ||
| 34 | +def build_model(placeholder_fn): | ||
| 35 | + """构建 abs -> reduce_sum 计算图及对应输入数据。""" | ||
| 36 | + data1 = placeholder_fn(tf.float16, shape=[128, 192]) | ||
| 37 | + input_data = np.random.uniform(-1.0, 1.0, size=(128, 192)).astype(np.float16) | ||
| 38 | + | ||
| 39 | + abs_0 = tf.abs(data1) | ||
| 40 | + reduce_sum_0 = tf.reduce_sum(abs_0, axis=1) | ||
| 41 | + | ||
| 42 | + return reduce_sum_0, {data1: input_data} | ||
| 43 | + | ||
| 44 | + | ||
| 45 | +if __name__ == "__main__": | ||
| 46 | + run_example( | ||
| 47 | + build_model, | ||
| 48 | + description="AutoFuse abs-reduce_sum 示例", | ||
| 49 | + ) | ||
| @@ -0,0 +1,10 @@ | |||
| 1 | +# ----------------------------------------------------------------------------------------------------------- | ||
| 2 | +# Copyright (c) 2026 Huawei Technologies Co., Ltd. | ||
| 3 | +# This program is free software, you can redistribute it and/or modify it under the terms and conditions of | ||
| 4 | +# CANN Open Software License Agreement Version 2.0 (the "License"). | ||
| 5 | +# Please refer to the License for details. You may not use this file except in compliance with the License. | ||
| 6 | +# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, | ||
| 7 | +# INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. | ||
| 8 | +# See LICENSE in the root of the software repository for the full text of the License. | ||
| 9 | +# ----------------------------------------------------------------------------------------------------------- | ||
| 10 | +"""TensorFlow AutoFuse 示例公共模块。""" | ||
| @@ -0,0 +1,48 @@ | |||
| 1 | +#!/usr/bin/env python3 | ||
| 2 | +# -*- coding: utf-8 -*- | ||
| 3 | +# ----------------------------------------------------------------------------------------------------------- | ||
| 4 | +# Copyright (c) 2026 Huawei Technologies Co., Ltd. | ||
| 5 | +# This program is free software, you can redistribute it and/or modify it under the terms and conditions of | ||
| 6 | +# CANN Open Software License Agreement Version 2.0 (the "License"). | ||
| 7 | +# Please refer to the License for details. You may not use this file except in compliance with the License. | ||
| 8 | +# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, | ||
| 9 | +# INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. | ||
| 10 | +# See LICENSE in the root of the software repository for the full text of the License. | ||
| 11 | +# ----------------------------------------------------------------------------------------------------------- | ||
| 12 | +"""TensorFlow AutoFuse 示例的公共配置。""" | ||
| 13 | + | ||
| 14 | +import os | ||
| 15 | + | ||
| 16 | +# Profiling 数据输出目录。保持与原有用例一致，默认输出到当前工作目录下的 profiling 目录。 | ||
| 17 | +PROFILING_DIR = os.path.abspath("./profiling") | ||
| 18 | + | ||
| 19 | +# 每个示例执行的推理步数。增加步数可获得更稳定的 Profiling 数据，但会增加运行时间。 | ||
| 20 | +RUN_STEPS = 100 | ||
| 21 | + | ||
| 22 | +# Session 允许 TensorFlow 在目标设备无法执行某个算子时进行设备回退。 | ||
| 23 | +ALLOW_SOFT_PLACEMENT = True | ||
| 24 | + | ||
| 25 | +# 是否在日志中打印每个算子的实际设备放置信息。默认关闭，避免产生大量日志。 | ||
| 26 | +LOG_DEVICE_PLACEMENT = False | ||
| 27 | + | ||
| 28 | +# NpuOptimizer 的离线编译开关。True 表示使用离线编译方式生成并执行 NPU 图。 | ||
| 29 | +USE_OFF_LINE = True | ||
| 30 | + | ||
| 31 | +# NpuOptimizer 图运行模式。0 表示推理场景。 | ||
| 32 | +GRAPH_RUN_MODE = 0 | ||
| 33 | + | ||
| 34 | +# 是否开启 NPU Profiling。开启后会在 PROFILING_DIR 下生成性能采集数据。 | ||
| 35 | +PROFILING_MODE = True | ||
| 36 | + | ||
| 37 | +# NPU Profiling 采集配置： | ||
| 38 | +# - output：Profiling 数据输出目录。 | ||
| 39 | +# - training_trace：采集迭代轨迹信息。 | ||
| 40 | +# - task_time：采集 Task 执行时间。 | ||
| 41 | +# - hccl：采集 HCCL 通信信息。 | ||
| 42 | +# - aicpu：采集 AI CPU 算子信息。 | ||
| 43 | +# - aic_metrics：采集 AI Core 指标，此处使用 PipeUtilization。 | ||
| 44 | +# - msproftx：是否采集 msproftx 标记信息，本示例关闭。 | ||
| 45 | +PROFILING_OPTIONS = ( | ||
| 46 | + '{"output":"%s","training_trace":"on","task_time":"on",' | ||
| 47 | + '"hccl":"on","aicpu":"on","aic_metrics":"PipeUtilization","msproftx":"off"}' | ||
| 48 | +) % PROFILING_DIR | ||
| @@ -0,0 +1,34 @@ | |||
| 1 | +#!/usr/bin/env python3 | ||
| 2 | +# -*- coding: utf-8 -*- | ||
| 3 | +# ----------------------------------------------------------------------------------------------------------- | ||
| 4 | +# Copyright (c) 2026 Huawei Technologies Co., Ltd. | ||
| 5 | +# This program is free software, you can redistribute it and/or modify it under the terms and conditions of | ||
| 6 | +# CANN Open Software License Agreement Version 2.0 (the "License"). | ||
| 7 | +# Please refer to the License for details. You may not use this file except in compliance with the License. | ||
| 8 | +# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, | ||
| 9 | +# INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. | ||
| 10 | +# See LICENSE in the root of the software repository for the full text of the License. | ||
| 11 | +# ----------------------------------------------------------------------------------------------------------- | ||
| 12 | +"""TensorFlow AutoFuse 示例的 Profiling 公共工具。""" | ||
| 13 | + | ||
| 14 | +import glob | ||
| 15 | +import os | ||
| 16 | +import subprocess | ||
| 17 | + | ||
| 18 | +from .config import PROFILING_DIR | ||
| 19 | + | ||
| 20 | + | ||
| 21 | +def get_profile_dirs(): | ||
| 22 | + """获取 Profiling 输出目录下当前已经存在的 PROF_* 目录。""" | ||
| 23 | + profile_pattern = os.path.join(PROFILING_DIR, "PROF_*") | ||
| 24 | + return set(glob.glob(profile_pattern)) | ||
| 25 | + | ||
| 26 | + | ||
| 27 | +def export_new_profiling(profile_dirs_before): | ||
| 28 | + """使用 msprof 导出本次执行过程中新增的 Profiling 数据。""" | ||
| 29 | + profile_dirs_after = get_profile_dirs() | ||
| 30 | + for profile_dir in sorted(profile_dirs_after - profile_dirs_before): | ||
| 31 | + subprocess.run( | ||
| 32 | + ["msprof", "--export=on", "--output={}".format(profile_dir)], | ||
| 33 | + check=True, | ||
| 34 | + ) | ||
| @@ -0,0 +1,96 @@ | |||
| 1 | +#!/usr/bin/env python3 | ||
| 2 | +# -*- coding: utf-8 -*- | ||
| 3 | +# ----------------------------------------------------------------------------------------------------------- | ||
| 4 | +# Copyright (c) 2026 Huawei Technologies Co., Ltd. | ||
| 5 | +# This program is free software, you can redistribute it and/or modify it under the terms and conditions of | ||
| 6 | +# CANN Open Software License Agreement Version 2.0 (the "License"). | ||
| 7 | +# Please refer to the License for details. You may not use this file except in compliance with the License. | ||
| 8 | +# THIS SOFTWARE IS PROVIDED ON AN "AS IS" BASIS, WITHOUT WARRANTIES OF ANY KIND, EITHER EXPRESS OR IMPLIED, | ||
| 9 | +# INCLUDING BUT NOT LIMITED TO NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR A PARTICULAR PURPOSE. | ||
| 10 | +# See LICENSE in the root of the software repository for the full text of the License. | ||
| 11 | +# ----------------------------------------------------------------------------------------------------------- | ||
| 12 | +"""TensorFlow AutoFuse 示例的公共运行框架。""" | ||
| 13 | + | ||
| 14 | +import argparse | ||
| 15 | + | ||
| 16 | +import tensorflow as tf | ||
| 17 | + | ||
| 18 | +from .config import ( | ||
| 19 | + ALLOW_SOFT_PLACEMENT, | ||
| 20 | + GRAPH_RUN_MODE, | ||
| 21 | + LOG_DEVICE_PLACEMENT, | ||
| 22 | + PROFILING_MODE, | ||
| 23 | + PROFILING_OPTIONS, | ||
| 24 | + RUN_STEPS, | ||
| 25 | + USE_OFF_LINE, | ||
| 26 | +) | ||
| 27 | +from .profiling_utils import export_new_profiling, get_profile_dirs | ||
| 28 | + | ||
| 29 | + | ||
| 30 | +def configure_npu(sess_config): | ||
| 31 | + """为 TensorFlow Session 配置 NpuOptimizer 和 Profiling 参数。""" | ||
| 32 | + custom_op = sess_config.graph_options.rewrite_options.custom_optimizers.add() | ||
| 33 | + custom_op.name = "NpuOptimizer" | ||
| 34 | + | ||
| 35 | + # 使用离线编译模式执行 NPU 图。 | ||
| 36 | + custom_op.parameter_map["use_off_line"].b = USE_OFF_LINE | ||
| 37 | + | ||
| 38 | + # graph_run_mode=0 表示推理模式。 | ||
| 39 | + custom_op.parameter_map["graph_run_mode"].i = GRAPH_RUN_MODE | ||
| 40 | + | ||
| 41 | + # 开启 Profiling，并将公共 Profiling 配置传递给 NpuOptimizer。 | ||
| 42 | + custom_op.parameter_map["profiling_mode"].b = PROFILING_MODE | ||
| 43 | + custom_op.parameter_map["profiling_options"].s = tf.compat.as_bytes( | ||
| 44 | + PROFILING_OPTIONS | ||
| 45 | + ) | ||
| 46 | + return sess_config | ||
| 47 | + | ||
| 48 | + | ||
| 49 | +def run_model(build_model, placeholder_fn, configproto_fn): | ||
| 50 | + """构建用例模型，并使用统一的 Session/NPU 配置执行推理。""" | ||
| 51 | + profile_dirs_before = get_profile_dirs() | ||
| 52 | + | ||
| 53 | + # 各用例只负责定义模型和输入数据，并返回待执行 Tensor 与 feed_dict。 | ||
| 54 | + output_tensor, feed_dict = build_model(placeholder_fn) | ||
| 55 | + | ||
| 56 | + sess_config = configproto_fn( | ||
| 57 | + allow_soft_placement=ALLOW_SOFT_PLACEMENT, | ||
| 58 | + log_device_placement=LOG_DEVICE_PLACEMENT, | ||
| 59 | + ) | ||
| 60 | + configure_npu(sess_config) | ||
| 61 | + | ||
| 62 | + with tf.compat.v1.Session(config=sess_config) as sess: | ||
| 63 | + for _ in range(RUN_STEPS): | ||
| 64 | + sess.run(output_tensor, feed_dict=feed_dict) | ||
| 65 | + | ||
| 66 | + export_new_profiling(profile_dirs_before) | ||
| 67 | + | ||
| 68 | + | ||
| 69 | +def run_tf1(build_model): | ||
| 70 | + """使用 TF1 + npu_bridge 方式运行用例。""" | ||
| 71 | + import npu_bridge | ||
| 72 | + | ||
| 73 | + _ = npu_bridge # 通过 import 副作用注册 NPU 算子。 | ||
| 74 | + run_model(build_model, tf.placeholder, tf.ConfigProto) | ||
| 75 | + | ||
| 76 | + | ||
| 77 | +def run_tf2_compat(build_model): | ||
| 78 | + """使用 TF2 的 v1 兼容模式 + npu_device 运行用例。""" | ||
| 79 | + import npu_device | ||
| 80 | + import npu_device.compat | ||
| 81 | + | ||
| 82 | + _ = npu_device # 保留 npu_device 导入，确保 NPU 相关能力完成注册。 | ||
| 83 | + npu_device.compat.enable_v1() | ||
| 84 | + run_model(build_model, tf.compat.v1.placeholder, tf.compat.v1.ConfigProto) | ||
| 85 | + | ||
| 86 | + | ||
| 87 | +def run_example(build_model, description): | ||
| 88 | + """解析运行模式，并调用对应 TensorFlow 运行入口。""" | ||
| 89 | + parser = argparse.ArgumentParser(description=description) | ||
| 90 | + parser.add_argument("--mode", choices=["tf1", "tf2-compat"], required=True) | ||
| 91 | + mode = parser.parse_args().mode | ||
| 92 | + | ||
| 93 | + if mode == "tf1": | ||
| 94 | + run_tf1(build_model) | ||
| 95 | + else: | ||
| 96 | + run_tf2_compat(build_model) | ||
这个目录怎么光有文档说明，没有对应用例