已合并
add chapter01 to master br #786
wangmingming创建于 20 天前
add chapter01 to master br #786
已合并
共 12 个文件变更+868-0
| @@ -0,0 +1,59 @@ | |||
| 1 | +{ | ||
| 2 | + "cells": [ | ||
| 3 | + { | ||
| 4 | + "cell_type": "markdown", | ||
| 5 | + "id": "md00", | ||
| 6 | + "metadata": {}, | ||
| 7 | + "source": [ | ||
| 8 | + "# AutoFuse 基础介绍\n", | ||
| 9 | + "\n", | ||
| 10 | + "本章帮助你建立 AutoFuse 的整体认知并掌握基础使能方式,从\"什么是 AutoFuse\"开始,逐步介绍其核心价值、接入路线、融合过程及开启方式等。\n", | ||
| 11 | + "\n", | ||
| 12 | + "---\n", | ||
| 13 | + "\n", | ||
| 14 | + "## 前置要求\n", | ||
| 15 | + "\n", | ||
| 16 | + "为了充分掌握本章内容,你应已具备以下能力:\n", | ||
| 17 | + "\n", | ||
| 18 | + "- 熟悉 CANN、昇腾 NPU 的基础概念,了解深度学习模型由算子组成的基本结构。\n", | ||
| 19 | + "- 理解 Tensor、shape、dtype 等深度学习核心概念。\n", | ||
| 20 | + "- 了解昇腾环境中模型经图编译后执行的基本过程。\n", | ||
| 21 | + "\n", | ||
| 22 | + "---\n", | ||
| 23 | + "\n", | ||
| 24 | + "## 章节目标\n", | ||
| 25 | + "\n", | ||
| 26 | + "完成本章后,你将能够:\n", | ||
| 27 | + "\n", | ||
| 28 | + "- 说明 AutoFuse 解决的核心问题,理解其缓解 Memory Bound 场景的原理。\n", | ||
| 29 | + "- 区分 AutoFuse 与业务代码直接调用 API 的差异,理解其在图编译流程中的作用。\n", | ||
| 30 | + "- 了解 AutoFuse 的两条接入路线及融合过程的整体脉络。\n", | ||
| 31 | + "- 掌握 `AUTOFUSE_FLAGS` 配置方法,完成 AutoFuse 基础使能。\n", | ||
| 32 | + "\n", | ||
| 33 | + "## 章节内容\n", | ||
| 34 | + "\n", | ||
| 35 | + "* [1.1 章节介绍](01.01_chapter_intro.ipynb)\n", | ||
| 36 | + "* [1.2 AutoFuse简介](01.02_autofuse_introduction.ipynb)\n", | ||
| 37 | + "* [1.3 AutoFuse使能基础](01.03_enable_autofusion.ipynb)\n", | ||
| 38 | + "* [1.4 章节练习](01.04_chapter_practice.ipynb)\n", | ||
| 39 | + "\n", | ||
| 40 | + "---\n", | ||
| 41 | + "\n", | ||
| 42 | + "本章课程请从 [AutoFuse简介](01.02_autofuse_introduction.ipynb) 开始学习。\n" | ||
| 43 | + ] | ||
| 44 | + } | ||
| 45 | + ], | ||
| 46 | + "metadata": { | ||
| 47 | + "kernelspec": { | ||
| 48 | + "display_name": "Python 3", | ||
| 49 | + "language": "python", | ||
| 50 | + "name": "python3" | ||
| 51 | + }, | ||
| 52 | + "language_info": { | ||
| 53 | + "name": "python", | ||
| 54 | + "pygments_lexer": "ipython3" | ||
| 55 | + } | ||
| 56 | + }, | ||
| 57 | + "nbformat": 4, | ||
| 58 | + "nbformat_minor": 5 | ||
| 59 | +} | ||
| @@ -0,0 +1,350 @@ | |||
| 1 | +{ | ||
| 2 | + "cells": [ | ||
| 3 | + { | ||
| 4 | + "cell_type": "markdown", | ||
| 5 | + "id": "md00", | ||
| 6 | + "metadata": {}, | ||
| 7 | + "source": [ | ||
| 8 | + "# AutoFuse简介\n", | ||
| 9 | + "\n", | ||
| 10 | + "本节围绕\"什么是 AutoFuse\"\"接入路线\"\"融合过程\"三条主线展开,并补充产品支持型号与使用边界。为了便于初学者理解,本节先说明通用术语与相关概念,再补充课程化解释,帮助你建立 AutoFuse 的整体认知。\n", | ||
| 11 | + "\n", | ||
| 12 | + "本节学习大纲如下:\n", | ||
| 13 | + "\n", | ||
| 14 | + "- 通用术语与相关概念\n", | ||
| 15 | + "- 什么是 AutoFuse\n", | ||
| 16 | + "- AutoFuse 接入路线\n", | ||
| 17 | + "- AutoFuse 融合过程\n", | ||
| 18 | + "- 产品支持型号与使用边界\n" | ||
| 19 | + ] | ||
| 20 | + }, | ||
| 21 | + { | ||
| 22 | + "cell_type": "markdown", | ||
| 23 | + "id": "md01", | ||
| 24 | + "metadata": {}, | ||
| 25 | + "source": [ | ||
| 26 | + "## 1. 通用术语与相关概念\n", | ||
| 27 | + "为便于理解后续内容,学习本课程前请先了解如下术语、缩略语及相关概念。\n", | ||
| 28 | + "\n", | ||
| 29 | + "<table align=\"left\">\n", | ||
| 30 | + " <thead>\n", | ||
| 31 | + " <tr>\n", | ||
| 32 | + " <th>术语</th>\n", | ||
| 33 | + " <th>说明</th>\n", | ||
| 34 | + " <th>在本课程中的理解方式</th>\n", | ||
| 35 | + " </tr>\n", | ||
| 36 | + " </thead>\n", | ||
| 37 | + " <tbody>\n", | ||
| 38 | + " <tr>\n", | ||
| 39 | + " <td>算子</td>\n", | ||
| 40 | + " <td>Operator,深度学习模型中执行具体计算的基本单元</td>\n", | ||
| 41 | + " <td>AutoFuse 识别和融合的基本对象</td>\n", | ||
| 42 | + " </tr>\n", | ||
| 43 | + " <tr>\n", | ||
| 44 | + " <td>融合算子</td>\n", | ||
| 45 | + " <td>Fused Operator,将多个相邻算子合并后形成的算子</td>\n", | ||
| 46 | + " <td>AutoFuse 融合后的计算形式</td>\n", | ||
| 47 | + " </tr>\n", | ||
| 48 | + " <tr>\n", | ||
| 49 | + " <td>NPU</td>\n", | ||
| 50 | + " <td>Neural Processing Unit,神经网络处理器</td>\n", | ||
| 51 | + " <td>昇腾 AI 处理器执行计算的硬件基础</td>\n", | ||
| 52 | + " </tr>\n", | ||
| 53 | + " <tr>\n", | ||
| 54 | + " <td>Host</td>\n", | ||
| 55 | + " <td>主机侧</td>\n", | ||
| 56 | + " <td>负责模型编译、准备参数和训练/推理进程管理</td>\n", | ||
| 57 | + " </tr>\n", | ||
| 58 | + " <tr>\n", | ||
| 59 | + " <td>Device</td>\n", | ||
| 60 | + " <td>昇腾设备侧</td>\n", | ||
| 61 | + " <td>负责模型内算子的调度和执行</td>\n", | ||
| 62 | + " </tr>\n", | ||
| 63 | + " <tr>\n", | ||
| 64 | + " <td>GM</td>\n", | ||
| 65 | + " <td>Global Memory,全局内存</td>\n", | ||
| 66 | + " <td>设备侧容量较大但访问代价更高的存储空间,作为算子间数据流转存储单元</td>\n", | ||
| 67 | + " </tr>\n", | ||
| 68 | + " <tr>\n", | ||
| 69 | + " <td>UB</td>\n", | ||
| 70 | + " <td>Unified Buffer,统一缓冲区</td>\n", | ||
| 71 | + " <td>AI Core 片上缓存空间,容量小但访问速度比 GM 快几十倍,作为算子内数据流转存储单元</td>\n", | ||
| 72 | + " </tr>\n", | ||
| 73 | + " <tr>\n", | ||
| 74 | + " <td>MTE</td>\n", | ||
| 75 | + " <td>Memory Transfer Engine,AI Core 中与数据搬运相关的能力</td>\n", | ||
| 76 | + " <td>搬运数据的硬件通道,用于在 GM 与 UB 间完成数据搬运</td>\n", | ||
| 77 | + " </tr>\n", | ||
| 78 | + " <tr>\n", | ||
| 79 | + " <td>Vector 计算</td>\n", | ||
| 80 | + " <td>对向量数据同时做相同运算的计算方式</td>\n", | ||
| 81 | + " <td>AutoFuse 常见收益场景之一,多个小 Vector 计算可能带来较多搬运开销</td>\n", | ||
| 82 | + " </tr>\n", | ||
| 83 | + " <tr>\n", | ||
| 84 | + " <td>Memory Bound</td>\n", | ||
| 85 | + " <td>性能主要受数据搬运限制,而不是受计算能力限制</td>\n", | ||
| 86 | + " <td>数据搬运耗时是性能瓶颈,而不是计算耗时</td>\n", | ||
| 87 | + " </tr>\n", | ||
| 88 | + " <tr>\n", | ||
| 89 | + " <td>图编译</td>\n", | ||
| 90 | + " <td>将模型图转换为昇腾可执行模型或编译产物的过程</td>\n", | ||
| 91 | + " <td>AutoFuse 主要发挥作用的阶段</td>\n", | ||
| 92 | + " </tr>\n", | ||
| 93 | + " <tr>\n", | ||
| 94 | + " <td>OM 模型</td>\n", | ||
| 95 | + " <td>Offline Model,昇腾离线模型文件</td>\n", | ||
| 96 | + " <td>通过 ATC 触发离线编译后常见的模型交付件</td>\n", | ||
| 97 | + " </tr>\n", | ||
| 98 | + " <tr>\n", | ||
| 99 | + " <td>GE</td>\n", | ||
| 100 | + " <td>Graph Engine,昇腾图引擎</td>\n", | ||
| 101 | + " <td>模型编译加速组件,使能 AutoFuse 的场景之一</td>\n", | ||
| 102 | + " </tr>\n", | ||
| 103 | + " <tr>\n", | ||
| 104 | + " <td>ATC</td>\n", | ||
| 105 | + " <td>Ascend Tensor Compiler,昇腾模型转换/离线编译工具</td>\n", | ||
| 106 | + " <td>常见的离线编译入口,底层会进入 GE 图编译相关流程</td>\n", | ||
| 107 | + " </tr>\n", | ||
| 108 | + " <tr>\n", | ||
| 109 | + " <td>IR</td>\n", | ||
| 110 | + " <td>Intermediate Representation,中间表示</td>\n", | ||
| 111 | + " <td>编译器内部用于描述计算逻辑的表达形式</td>\n", | ||
| 112 | + " </tr>\n", | ||
| 113 | + " <tr>\n", | ||
| 114 | + " <td>AscIR</td>\n", | ||
| 115 | + " <td>Ascend C IR,面向 Ascend C 语言建模的中间表示</td>\n", | ||
| 116 | + " <td>衔接前端图表示与后端算子代码的中间桥梁</td>\n", | ||
| 117 | + " </tr>\n", | ||
| 118 | + " <tr>\n", | ||
| 119 | + " <td>AutoFuse</td>\n", | ||
| 120 | + " <td>基于 Ascend C 的自动融合框架</td>\n", | ||
| 121 | + " <td>在图编译阶段识别可融合范围,并生成融合算子相关实现</td>\n", | ||
| 122 | + " </tr>\n", | ||
| 123 | + " <tr>\n", | ||
| 124 | + " <td>融合策略</td>\n", | ||
| 125 | + " <td>判断哪些结构适合融合的规则和求解过程</td>\n", | ||
| 126 | + " <td>避免把所有相邻算子都盲目合并</td>\n", | ||
| 127 | + " </tr>\n", | ||
| 128 | + " <tr>\n", | ||
| 129 | + " <td>FusedGraph</td>\n", | ||
| 130 | + " <td>表示一个融合范围的图结构</td>\n", | ||
| 131 | + " <td>AutoFuse 前端识别出的可融合算子集合</td>\n", | ||
| 132 | + " </tr>\n", | ||
| 133 | + " <tr>\n", | ||
| 134 | + " <td>AscBackend</td>\n", | ||
| 135 | + " <td>AutoFuse 融合后形成的 Ascend IR 算子节点</td>\n", | ||
| 136 | + " <td>承载融合子图结构,携带 AscGraph 属性信息</td>\n", | ||
| 137 | + " </tr>\n", | ||
| 138 | + " <tr>\n", | ||
| 139 | + " <td>AscGraph</td>\n", | ||
| 140 | + " <td>AscBackend 节点携带的子图对象</td>\n", | ||
| 141 | + " <td>内部包含多个 AscIR 节点,用于描述融合后的计算结构</td>\n", | ||
| 142 | + " </tr>\n", | ||
| 143 | + " <tr>\n", | ||
| 144 | + " <td>Schedule</td>\n", | ||
| 145 | + " <td>算子调度优化</td>\n", | ||
| 146 | + " <td>对融合后的计算逻辑进行组织优化,调整执行顺序、合并循环、复用缓存,在不改变结果的前提下提升性能</td>\n", | ||
| 147 | + " </tr>\n", | ||
| 148 | + " <tr>\n", | ||
| 149 | + " <td>Tiling</td>\n", | ||
| 150 | + " <td>将大 shape 的计算任务切分为适配硬件缓存大小的小块</td>\n", | ||
| 151 | + " <td>融合性能优化的关键技术</td>\n", | ||
| 152 | + " </tr>\n", | ||
| 153 | + " <tr>\n", | ||
| 154 | + " <td>Auto Tiling</td>\n", | ||
| 155 | + " <td>自动切分</td>\n", | ||
| 156 | + " <td>评估多种切分方案的性能,自动选出使 Kernel 执行性能最优的 Tiling 策略</td>\n", | ||
| 157 | + " </tr>\n", | ||
| 158 | + " <tr>\n", | ||
| 159 | + " <td>Lowering</td>\n", | ||
| 160 | + " <td>表达层级转换</td>\n", | ||
| 161 | + " <td>将较高层的图语义转换为后端更容易处理的 AscIR 表达</td>\n", | ||
| 162 | + " </tr>\n", | ||
| 163 | + " <tr>\n", | ||
| 164 | + " <td>Codegen</td>\n", | ||
| 165 | + " <td>Code Generation,代码生成</td>\n", | ||
| 166 | + " <td>解析调度结果,生成 Host 侧 Tiling 代码和 Device 侧 Kernel 代码</td>\n", | ||
| 167 | + " </tr>\n", | ||
| 168 | + " <tr>\n", | ||
| 169 | + " <td>Ascend C</td>\n", | ||
| 170 | + " <td>面向昇腾 AI 处理器的算子开发语言</td>\n", | ||
| 171 | + " <td>AutoFuse 后端生成融合算子代码时依赖的目标语言</td>\n", | ||
| 172 | + " </tr>\n", | ||
| 173 | + " <tr>\n", | ||
| 174 | + " <td>BiSheng Compiler</td>\n", | ||
| 175 | + " <td>毕昇编译器,昇腾 AI 软件栈中的编译器组件</td>\n", | ||
| 176 | + " <td>AutoFuse 生成的融合 kernel 源码通过毕昇编译器编译为 NPU 可执行的二进制产物</td>\n", | ||
| 177 | + " </tr>\n", | ||
| 178 | + " <tr>\n", | ||
| 179 | + " <td>Kernel</td>\n", | ||
| 180 | + " <td>在 NPU 上实际执行计算的代码片段</td>\n", | ||
| 181 | + " <td>AutoFuse 后端生成的融合计算单元</td>\n", | ||
| 182 | + " </tr>\n", | ||
| 183 | + " <tr>\n", | ||
| 184 | + " <td>Dynamic Shape</td>\n", | ||
| 185 | + " <td>动态 shape</td>\n", | ||
| 186 | + " <td>模型输入或中间张量的某些维度在编译时不能完全固定</td>\n", | ||
| 187 | + " </tr>\n", | ||
| 188 | + " <tr>\n", | ||
| 189 | + " <td>Dynamo</td>\n", | ||
| 190 | + " <td>PyTorch Dynamo,PyTorch 生态中的动态图捕获与图生成组件</td>\n", | ||
| 191 | + " <td>可理解为 PyTorch 编译链路的前端入口,负责捕获 Python 模型执行并生成可交给后续编译器处理的图表示</td>\n", | ||
| 192 | + " </tr>\n", | ||
| 193 | + " <tr>\n", | ||
| 194 | + " <td>PyTorch Inductor</td>\n", | ||
| 195 | + " <td>PyTorch 生态中的编译优化组件</td>\n", | ||
| 196 | + " <td>使能 AutoFuse 的另一条重要对接路线</td>\n", | ||
| 197 | + " </tr>\n", | ||
| 198 | + " </tbody>\n", | ||
| 199 | + "</table>\n", | ||
| 200 | + "<div style=\"clear:left\"></div>\n" | ||
| 201 | + ] | ||
| 202 | + }, | ||
| 203 | + { | ||
| 204 | + "cell_type": "markdown", | ||
| 205 | + "id": "md02", | ||
| 206 | + "metadata": {}, | ||
| 207 | + "source": [ | ||
| 208 | + "## 2. 什么是 AutoFuse\n", | ||
| 209 | + "\n", | ||
| 210 | + "AutoFuse 是基于 Ascend C 的自动融合框架,支持自动融合范围识别、自动算子代码生成、Auto Tiling 优化及 Dynamic Shape 等特性。\n", | ||
| 211 | + "\n", | ||
| 212 | + "在算法网络中,由于存在大量 Vector 计算,各个 Vector 计算之间会产生大量内存搬运,导致 Memory Bound 问题。AutoFuse 通过自动将多个算子融合为一个算子,减少网络中的算子数量和内存搬运,从而缓解 Memory Bound 问题,释放昇腾算力,提升模型执行性能。\n", | ||
| 213 | + "\n", | ||
| 214 | + "<div style=\"text-align:left\">\n", | ||
| 215 | + "<img src=\"./images/autofuse_benefit_principle.png\" alt=\"AutoFuse 收益原理\" width=\"40%\">\n", | ||
| 216 | + "</div>\n", | ||
| 217 | + "\n", | ||
| 218 | + "结合课程视角,可以这样理解:AutoFuse 并不是让计算本身消失,而是将多个满足条件的小算子组织成一个融合算子,让中间结果尽量在片上缓存中流转,减少重复搬运和多次调度。从收益机制看,自动融合理论上在 MTE 搬运、Dynamic Shape 调度开销方面都会有一定收益;对于小 shape、MTE Bound 的推荐网络,一般更容易获得正收益。" | ||
| 219 | + ] | ||
| 220 | + }, | ||
| 221 | + { | ||
| 222 | + "cell_type": "markdown", | ||
| 223 | + "id": "md03", | ||
| 224 | + "metadata": {}, | ||
| 225 | + "source": [ | ||
| 226 | + "## 3. AutoFuse 接入路线\n", | ||
| 227 | + "\n", | ||
| 228 | + "AutoFuse 自动融合方案基于昇腾 NPU 底层统一的 Ascend C IR 与代码生成能力,目前提供了两条接入路径。\n", | ||
| 229 | + "\n", | ||
| 230 | + "<div style=\"text-align:left\">\n", | ||
| 231 | + "<img src=\"./images/autofuse_technical_route.png\" alt=\"AutoFuse 接入路线\" width=\"40%\">\n", | ||
| 232 | + "</div>\n", | ||
| 233 | + "第一条路径基于昇腾自研 GE 框架,注重 NPU 亲和性。该路径包含三部分核心能力:\n", | ||
| 234 | + "\n", | ||
| 235 | + "- Ascend IR 的符号化 shape 推导,通过变量符号表达动态变化的 shape,从而在编译时基于符号化的 shape 进行代码生成。\n", | ||
| 236 | + "- Ascend IR 到 AscIR 的 Lowering 实现,使用低层级的 AscIR 表达 Ascend IR 的计算逻辑,确定融合结构。\n", | ||
| 237 | + "- 融合策略,结合 AscIR 的特点与约束,进行融合结构间的循环轴合并,获得融合最优解。\n", | ||
| 238 | + "\n", | ||
| 239 | + "第二条路径对接 PyTorch Inductor,聚焦生态支持,复用 Inductor 的融合能力,并将 Inductor IR 表达的融合结构转换为 Ascend C IR 图进行代码生成。算子融合方式与第一条路径类似,当前路径功能已支持,但是还未正式商用。" | ||
| 240 | + ] | ||
| 241 | + }, | ||
| 242 | + { | ||
| 243 | + "cell_type": "markdown", | ||
| 244 | + "id": "md04", | ||
| 245 | + "metadata": {}, | ||
| 246 | + "source": [ | ||
| 247 | + "## 4. AutoFuse 融合过程\n", | ||
| 248 | + "\n", | ||
| 249 | + "AutoFuse 自动融合的实现过程可以划分为两部分:\n", | ||
| 250 | + "\n", | ||
| 251 | + "- 自动确定融合范围。\n", | ||
| 252 | + "- 根据融合范围自动生成融合 Kernel 执行源码及 Tiling 计算源码。\n", | ||
| 253 | + "\n", | ||
| 254 | + "前者称为自动融合前端,后者称为自动融合后端(对应上图中的公共底层能力)。前端主要根据规则或配置判断哪些算子能够融合,并确定一个融合算子的融合范围;融合范围使用 FusedGraph 表达。\n", | ||
| 255 | + "\n", | ||
| 256 | + "<div style=\"text-align:left\">\n", | ||
| 257 | + "<img src=\"./images/fusedgraph_structure.png\" alt=\"FusedGraph\" width=\"28%\">\n", | ||
| 258 | + "</div>\n", | ||
| 259 | + "\n", | ||
| 260 | + "FusedGraph 内部包含一个或多个 AscBackend 节点。一个 AscBackend 节点携带一个 AscGraph 属性,一个 AscGraph 内包含多个 AscIR 节点。\n", | ||
| 261 | + "\n", | ||
| 262 | + "<div style=\"text-align:left\">\n", | ||
| 263 | + "<img src=\"./images/ascgraph_structure.png\" alt=\"AscBackend 对应的 AscGraph\" width=\"45%\">\n", | ||
| 264 | + "</div>\n", | ||
| 265 | + "\n", | ||
| 266 | + "后端接收到 FusedGraph 后,根据融合范围自动生成融合 Kernel 执行源码及 Tiling 计算源码。这里先帮助读者建立整体流程认知,自动融合原理会在后续章节体现。" | ||
| 267 | + ] | ||
| 268 | + }, | ||
| 269 | + { | ||
| 270 | + "cell_type": "markdown", | ||
| 271 | + "id": "md05", | ||
| 272 | + "metadata": {}, | ||
| 273 | + "source": [ | ||
| 274 | + "## 5. 产品支持型号与使用边界\n", | ||
| 275 | + "\n", | ||
| 276 | + "AutoFuse 自动融合特性当前支持以下产品型号:\n", | ||
| 277 | + "\n", | ||
| 278 | + "- Atlas 350 加速卡\n", | ||
| 279 | + "- Atlas A3 训练系列产品 / Atlas A3 推理系列产品\n", | ||
| 280 | + "- Atlas A2 训练系列产品 / Atlas A2 推理系列产品\n", | ||
| 281 | + "\n", | ||
| 282 | + "除了产品型号,还需要关注 AutoFuse 的使用边界:\n", | ||
| 283 | + "\n", | ||
| 284 | + "- **开启方式**:通过 `AUTOFUSE_FLAGS` 在图编译阶段开启;基础配置方式会在下一节说明。\n", | ||
| 285 | + "- **触发方式**:AutoFuse 依赖图编译流程触发,使用者无需在业务代码中直接调用其接口。\n", | ||
| 286 | + " - 基于 GE 框架的图编译路径,由 GE/ATC 编译流程在满足条件时自动调用 AutoFuse 融合流程。\n", | ||
| 287 | + " - 对接 PyTorch Inductor 的编译路径(通过 Dynamo 捕获模型图并交由 Inductor 编译),由 PyTorch 编译入口触发 AutoFuse 融合流程。\n", | ||
| 288 | + "- **收益不确定性**:是否获得性能收益需要结合模型结构、shape、算子类型和实际性能分析判断,无法保证所有模型开启后都有提升。" | ||
| 289 | + ] | ||
| 290 | + }, | ||
| 291 | + { | ||
| 292 | + "cell_type": "markdown", | ||
| 293 | + "id": "md07", | ||
| 294 | + "metadata": {}, | ||
| 295 | + "source": [ | ||
| 296 | + "## 6. 课后练习\n", | ||
| 297 | + "\n", | ||
| 298 | + "本节介绍了 AutoFuse 的基本定义、接入路线、融合过程和使用边界,请根据学习内容完成以下题目进行自测。\n", | ||
| 299 | + "\n", | ||
| 300 | + "1. (判断题)AutoFuse 主要在图编译阶段发挥作用,通过自动识别融合范围并生成融合算子相关实现,减少算子数量和内存搬运。\n", | ||
| 301 | + "\n", | ||
| 302 | + "2. (判断题)只要开启 AutoFuse,所有模型都一定能获得性能提升。\n", | ||
| 303 | + "\n", | ||
| 304 | + "3. (单选题)AutoFuse 缓解 Memory Bound 问题的主要方式是什么?\n", | ||
| 305 | + " A. 将所有计算转移到 CPU 上执行\n", | ||
| 306 | + " B. 将多个满足条件的算子融合为一个算子,减少中间数据搬运和调度开销\n", | ||
| 307 | + " C. 删除模型中的所有小算子\n", | ||
| 308 | + " D. 只改变模型文件名,不改变编译过程\n", | ||
| 309 | + "\n", | ||
| 310 | + "4. (单选题)如何理解 AutoFuse 的两条接入路线?\n", | ||
| 311 | + " A. 两条路线都体现 AutoFuse 自动融合的关键思想,课程从原理出发理解整体链路\n", | ||
| 312 | + " B. 只有基于 GE 框架的路径属于自动融合,另一条路径只是生态适配,不涉及融合\n", | ||
| 313 | + " C. PyTorch Inductor 路径已经作为商用交付能力,使用者必须优先选择该路径\n", | ||
| 314 | + " D. 必须由业务代码直接调用 AutoFuse 内部后端接口,否则无法触发自动融合\n", | ||
| 315 | + "\n", | ||
| 316 | + "5. (多选题)从普通使用者视角看,学习 AutoFuse 时需要重点关注哪些内容?\n", | ||
| 317 | + " A. 如何通过 `AUTOFUSE_FLAGS` 开启相关能力\n", | ||
| 318 | + " B. AutoFuse 依赖 GE 图编译流程生效,通过 ATC 触发离线编译生成 OM 是常见场景,在线 GE 编译路径满足条件时也可能生效\n", | ||
| 319 | + " C. 需要结合编译日志、融合产物、性能数据和精度对比判断融合效果\n", | ||
| 320 | + " D. 必须由业务代码直接调用 AutoFuse 内部后端接口,否则编译链路无法触发自动融合\n", | ||
| 321 | + "\n", | ||
| 322 | + "**执行以下代码获取答案。**\n" | ||
| 323 | + ] | ||
| 324 | + }, | ||
| 325 | + { | ||
| 326 | + "cell_type": "code", | ||
| 327 | + "execution_count": null, | ||
| 328 | + "id": "code00", | ||
| 329 | + "metadata": {}, | ||
| 330 | + "outputs": [], | ||
| 331 | + "source": [ | ||
| 332 | + "!cat ./answer/01.02_answer.txt" | ||
| 333 | + ] | ||
| 334 | + } | ||
| 335 | + ], | ||
| 336 | + "metadata": { | ||
| 337 | + "kernelspec": { | ||
| 338 | + "display_name": "Python 3", | ||
| 339 | + "language": "python", | ||
| 340 | + "name": "python3" | ||
| 341 | + }, | ||
| 342 | + "language_info": { | ||
| 343 | + "name": "python", | ||
| 344 | + "pygments_lexer": "ipython3", | ||
| 345 | + "version": "3.12.9" | ||
| 346 | + } | ||
| 347 | + }, | ||
| 348 | + "nbformat": 4, | ||
| 349 | + "nbformat_minor": 5 | ||
| 350 | +} | ||
| @@ -0,0 +1,265 @@ | |||
| 1 | +{ | ||
| 2 | + "cells": [ | ||
| 3 | + { | ||
| 4 | + "cell_type": "markdown", | ||
| 5 | + "id": "md00", | ||
| 6 | + "metadata": {}, | ||
| 7 | + "source": [ | ||
| 8 | + "# AutoFuse使能基础\n", | ||
| 9 | + "\n", | ||
| 10 | + "上一节介绍了 AutoFuse 的基本概念、接入路线和融合过程等,本节面向使用者说明如何开启 AutoFuse 的基础能力。这里的\"使能\"可以理解为:在执行模型图编译前,通过环境变量打开 AutoFuse 功能入口,并按需声明基础融合能力。学完本节后,你应能知道基础开关怎么打开、如何验证配置是否生效,以及哪些融合能力需要额外声明。更深入的融合原理、实践与问题定位会在后续章节展开。\n", | ||
| 11 | + "\n", | ||
| 12 | + "本节学习大纲如下:\n", | ||
| 13 | + "\n", | ||
| 14 | + "- 使用前提\n", | ||
| 15 | + "- AutoFuse 使能配置\n", | ||
| 16 | + "- 常见误区\n" | ||
| 17 | + ] | ||
| 18 | + }, | ||
| 19 | + { | ||
| 20 | + "cell_type": "markdown", | ||
| 21 | + "id": "md01", | ||
| 22 | + "metadata": {}, | ||
| 23 | + "source": [ | ||
| 24 | + "## 1. 使用前提\n", | ||
| 25 | + "\n", | ||
| 26 | + "使用 AutoFuse 前提条件如下:\n", | ||
| 27 | + "\n", | ||
| 28 | + "<table align=\"left\">\n", | ||
| 29 | + "<thead><tr><th>条件</th><th>要求</th><th>说明</th></tr></thead>\n", | ||
| 30 | + "<tbody>\n", | ||
| 31 | + "<tr><td>安装软件包</td><td>准备带有 AI 处理器的硬件环境,并安装驱动、固件和 CANN 软件包</td><td>具体安装步骤请参见 <a href='https://www.hiascend.com/document/detail/zh/CANNCommunityEdition/910beta1/softwareinst/instg/instg_0000.html?OS=openEuler&InstallType=netyum'>《CANN 软件安装》</a></td></tr>\n", | ||
| 32 | + "<tr><td>GCC 版本</td><td>9.5.0 及以上</td><td>建议使用 9.5.0 版本</td></tr>\n", | ||
| 33 | + "<tr><td>CMake 版本</td><td>3.20.0 及以上</td><td>建议使用 3.20.0 版本</td></tr>\n", | ||
| 34 | + "<tr><td>CANN 环境变量</td><td>安装 CANN 软件后,使用 CANN 运行用户登录环境,并执行 `source ${INSTALL_DIR}/set_env.sh`</td><td>`${INSTALL_DIR}` 请替换为 CANN 软件安装后的文件存储路径;以 root 用户安装为例,默认路径为 /usr/local/Ascend/cann</td></tr>\n", | ||
| 35 | + "</tbody>\n", | ||
| 36 | + "</table>\n", | ||
| 37 | + "<div style=\"clear:left\"></div>\n", | ||
| 38 | + "\n", | ||
| 39 | + "完成上述准备后,再在同一个编译环境中配置 AutoFuse 基础开关。\n" | ||
| 40 | + ] | ||
| 41 | + }, | ||
| 42 | + { | ||
| 43 | + "cell_type": "markdown", | ||
| 44 | + "id": "md02", | ||
| 45 | + "metadata": {}, | ||
| 46 | + "source": [ | ||
| 47 | + "## 2. AutoFuse 使能配置\n", | ||
| 48 | + "\n", | ||
| 49 | + "AutoFuse 的使能方式与对接路线密切相关。使用 GE 图编译路线时,需要在模型图编译前通过环境变量 <code>AUTOFUSE_FLAGS</code> 开启自动融合;使用 PyTorch Inductor 对接路线时,当前无需额外配置环境变量,只需在 Python 脚本中 <code>import inductor_npu_ext</code> 即可使能 AutoFuse。下面主要介绍 GE 路线的配置方式。\n", | ||
| 50 | + "\n", | ||
| 51 | + "### 2.1 最小开启配置\n", | ||
| 52 | + "\n", | ||
| 53 | + "AutoFuse 的基础使能入口是环境变量 <code>AUTOFUSE_FLAGS</code>,最小只需一行命令即可开启:\n", | ||
| 54 | + "\n", | ||
| 55 | + "```bash\n", | ||
| 56 | + "export AUTOFUSE_FLAGS=\"--enable_autofuse=true\"\n", | ||
| 57 | + "```\n", | ||
| 58 | + "\n", | ||
| 59 | + "<table align=\"left\">\n", | ||
| 60 | + " <thead>\n", | ||
| 61 | + " <tr>\n", | ||
| 62 | + " <th>配置项</th>\n", | ||
| 63 | + " <th>含义</th>\n", | ||
| 64 | + " </tr>\n", | ||
| 65 | + " </thead>\n", | ||
| 66 | + " <tbody>\n", | ||
| 67 | + " <tr>\n", | ||
| 68 | + " <td><code>--enable_autofuse=true</code></td>\n", | ||
| 69 | + " <td>打开 AutoFuse 总开关,允许图编译流程执行自动融合</td>\n", | ||
| 70 | + " </tr>\n", | ||
| 71 | + " <tr>\n", | ||
| 72 | + " <td><code>--enable_autofuse=false</code></td>\n", | ||
| 73 | + " <td>关闭 AutoFuse 总开关,不执行自动融合</td>\n", | ||
| 74 | + " </tr>\n", | ||
| 75 | + " </tbody>\n", | ||
| 76 | + "</table>\n", | ||
| 77 | + "<div style=\"clear:left\"></div>\n", | ||
| 78 | + "\n", | ||
| 79 | + "设置后需要在 **同一个 shell 环境** 中执行模型编译命令。环境变量不会自动影响已经启动的其他进程。\n", | ||
| 80 | + "\n", | ||
| 81 | + "\n", | ||
| 82 | + "### 2.2 配置格式说明\n", | ||
| 83 | + "\n", | ||
| 84 | + "如需追加其他配置项,请注意:\n", | ||
| 85 | + "\n", | ||
| 86 | + "- 多个配置项之间用 **英文分号** <code>;</code> 分隔\n", | ||
| 87 | + "- 全部写在 **同一个** <code>AUTOFUSE_FLAGS</code> 字符串中\n", | ||
| 88 | + "\n", | ||
| 89 | + "```bash\n", | ||
| 90 | + "# 格式示意\n", | ||
| 91 | + "export AUTOFUSE_FLAGS=\"配置项1;配置项2;配置项3\"\n", | ||
| 92 | + "```\n", | ||
| 93 | + "\n", | ||
| 94 | + "### 2.3 扩展开关配置\n", | ||
| 95 | + "\n", | ||
| 96 | + "AutoFuse 框架目前支持 Elemwise、Broadcast、Reduce、Concat 等 4 类算子的融合。开启 <code>--enable_autofuse=true</code> 后:\n", | ||
| 97 | + "\n", | ||
| 98 | + "- **Elemwise、Broadcast** 等基础融合能力默认可用,无需额外配置\n", | ||
| 99 | + "- **Reduce、Concat** 等融合能力默认不开启,需通过 <code>--autofuse_enable_pass</code> 显式声明\n", | ||
| 100 | + "\n", | ||
| 101 | + "例如同时开启 Reduce 和 Concat:\n", | ||
| 102 | + "\n", | ||
| 103 | + "```bash\n", | ||
| 104 | + "export AUTOFUSE_FLAGS=\"--enable_autofuse=true;--autofuse_enable_pass=reduce,concat\"\n", | ||
| 105 | + "```\n", | ||
| 106 | + "\n", | ||
| 107 | + "<table align=\"left\">\n", | ||
| 108 | + " <thead>\n", | ||
| 109 | + " <tr>\n", | ||
| 110 | + " <th>配置项</th>\n", | ||
| 111 | + " <th>含义</th>\n", | ||
| 112 | + " </tr>\n", | ||
| 113 | + " </thead>\n", | ||
| 114 | + " <tbody>\n", | ||
| 115 | + " <tr>\n", | ||
| 116 | + " <td><code>--autofuse_enable_pass=reduce,concat</code></td>\n", | ||
| 117 | + " <td>额外开启指定融合 pass,多种类型用英文逗号分隔</td>\n", | ||
| 118 | + " </tr>\n", | ||
| 119 | + " </tbody>\n", | ||
| 120 | + "</table>\n", | ||
| 121 | + "<div style=\"clear:left\"></div>\n", | ||
| 122 | + "\n", | ||
| 123 | + "各融合类型的默认开关状态如下:\n", | ||
| 124 | + "\n", | ||
| 125 | + "<table align=\"left\">\n", | ||
| 126 | + " <thead>\n", | ||
| 127 | + " <tr>\n", | ||
| 128 | + " <th>融合类型</th>\n", | ||
| 129 | + " <th>含义</th>\n", | ||
| 130 | + " <th>默认状态</th>\n", | ||
| 131 | + " </tr>\n", | ||
| 132 | + " </thead>\n", | ||
| 133 | + " <tbody>\n", | ||
| 134 | + " <tr>\n", | ||
| 135 | + " <td>Elemwise</td>\n", | ||
| 136 | + " <td>逐元素计算,如 Add、Mul、Abs</td>\n", | ||
| 137 | + " <td>✅ 开启 AutoFuse 后默认支持</td>\n", | ||
| 138 | + " </tr>\n", | ||
| 139 | + " <tr>\n", | ||
| 140 | + " <td>Broadcast</td>\n", | ||
| 141 | + " <td>广播计算,从尾对齐,长度1的维度自动扩展</td>\n", | ||
| 142 | + " <td>✅ 开启 AutoFuse 后默认支持</td>\n", | ||
| 143 | + " </tr>\n", | ||
| 144 | + " <tr>\n", | ||
| 145 | + " <td>Reduce</td>\n", | ||
| 146 | + " <td>规约计算,如 Sum、Max、Mean</td>\n", | ||
| 147 | + " <td>❌ 需显式开启</td>\n", | ||
| 148 | + " </tr>\n", | ||
| 149 | + " <tr>\n", | ||
| 150 | + " <td>Concat</td>\n", | ||
| 151 | + " <td>拼接计算,按指定维度拼接 Tensor</td>\n", | ||
| 152 | + " <td>❌ 需显式开启</td>\n", | ||
| 153 | + " </tr>\n", | ||
| 154 | + " </tbody>\n", | ||
| 155 | + "</table>\n", | ||
| 156 | + "<div style=\"clear:left\"></div>\n", | ||
| 157 | + "\n", | ||
| 158 | + "\n", | ||
| 159 | + "### 2.4 配置检查\n", | ||
| 160 | + "\n", | ||
| 161 | + "配置完成后,可通过以下命令确认当前 shell 中是否已经设置 <code>AUTOFUSE_FLAGS</code>:\n", | ||
| 162 | + "\n", | ||
| 163 | + "```bash\n", | ||
| 164 | + "echo $AUTOFUSE_FLAGS\n", | ||
| 165 | + "```\n", | ||
| 166 | + "\n", | ||
| 167 | + "如果输出为空,说明当前 shell 未设置该变量;如果能看到 <code>--enable_autofuse=true</code> 等配置内容,说明基础开关已设置到当前环境。需要注意,该检查只能确认环境变量是否设置,最终是否产生融合还需结合模型图结构、算子类型、编译日志和性能分析判断。\n" | ||
| 168 | + ] | ||
| 169 | + }, | ||
| 170 | + { | ||
| 171 | + "cell_type": "markdown", | ||
| 172 | + "id": "md05", | ||
| 173 | + "metadata": {}, | ||
| 174 | + "source": [ | ||
| 175 | + "## 3. 常见误区\n", | ||
| 176 | + "\n", | ||
| 177 | + "<table align=\"left\">\n", | ||
| 178 | + " <thead>\n", | ||
| 179 | + " <tr>\n", | ||
| 180 | + " <th>误区</th>\n", | ||
| 181 | + " <th>正确认知</th>\n", | ||
| 182 | + " </tr>\n", | ||
| 183 | + " </thead>\n", | ||
| 184 | + " <tbody>\n", | ||
| 185 | + " <tr>\n", | ||
| 186 | + " <td>只设置 <code>AUTOFUSE_DFX_FLAGS</code> 就能开启 AutoFuse</td>\n", | ||
| 187 | + " <td><code>AUTOFUSE_DFX_FLAGS</code> 是进阶定位配置,基础开启依赖 <code>AUTOFUSE_FLAGS=\"--enable_autofuse=true\"</code></td>\n", | ||
| 188 | + " </tr>\n", | ||
| 189 | + " <tr>\n", | ||
| 190 | + " <td>设置环境变量后所有模型一定收益</td>\n", | ||
| 191 | + " <td>收益需要结合实际模型或分析执行结果判断</td>\n", | ||
| 192 | + " </tr>\n", | ||
| 193 | + " <tr>\n", | ||
| 194 | + " <td>Reduce、Concat 只要打开总开关就一定融合</td>\n", | ||
| 195 | + " <td>默认不使能的融合能力,需要显式开启</td>\n", | ||
| 196 | + " </tr>\n", | ||
| 197 | + " <tr>\n", | ||
| 198 | + " <td>AutoFuse 是业务代码里直接调用的 API</td>\n", | ||
| 199 | + " <td>它内嵌在 GE 等组件的图编译流程中,用户只需通过环境变量开启即可</td>\n", | ||
| 200 | + " </tr>\n", | ||
| 201 | + " </tbody>\n", | ||
| 202 | + "</table>\n", | ||
| 203 | + "<div style=\"clear:left\"></div>\n" | ||
| 204 | + ] | ||
| 205 | + }, | ||
| 206 | + { | ||
| 207 | + "cell_type": "markdown", | ||
| 208 | + "id": "md07", | ||
| 209 | + "metadata": {}, | ||
| 210 | + "source": [ | ||
| 211 | + "## 4. 课后练习\n", | ||
| 212 | + "\n", | ||
| 213 | + "本节介绍了 AutoFuse 的基础使能方式,请根据学习内容完成以下题目进行自测。\n", | ||
| 214 | + "\n", | ||
| 215 | + "1. (判断题)AutoFuse 的基础使能入口是 <code>AUTOFUSE_FLAGS</code>,最小配置可以写为 <code>--enable_autofuse=true</code>。\n", | ||
| 216 | + "\n", | ||
| 217 | + "2. (判断题)<code>AUTOFUSE_DFX_FLAGS</code> 是开启 AutoFuse 的必需配置,不配置该环境变量时 AutoFuse 一定无法开启。\n", | ||
| 218 | + "\n", | ||
| 219 | + "3. (单选题)如果需要额外开启 Reduce 和 Concat 融合能力,以下哪种配置更符合课程中的说明?\n", | ||
| 220 | + " A. <code>export AUTOFUSE_FLAGS=\"--enable_autofuse=true;--autofuse_enable_pass=reduce,concat\"</code>\n", | ||
| 221 | + " B. <code>export AUTOFUSE_FLAGS=\"--enable_autofuse=false\"</code>\n", | ||
| 222 | + " C. <code>export AUTOFUSE_DFX_FLAGS=\"--debug_dir=/tmp\"</code>\n", | ||
| 223 | + " D. <code>export AUTOFUSE_FLAGS=\"--autofuse_disable_pass=reduce,concat\"</code>\n", | ||
| 224 | + "\n", | ||
| 225 | + "4. (单选题)设置 <code>AUTOFUSE_FLAGS</code> 后,为什么仍建议确认当前 shell 中的环境变量?\n", | ||
| 226 | + " A. 环境变量只对当前 shell 或其子进程生效\n", | ||
| 227 | + " B. 环境变量会自动修改所有历史终端\n", | ||
| 228 | + " C. 环境变量设置后会自动持久化到系统中,无需重新配置\n", | ||
| 229 | + " D. 环境变量对所有已打开的终端都自动生效,只需设置一次即可\n", | ||
| 230 | + "\n", | ||
| 231 | + "5. (多选题)从普通使用者视角看,配置 AutoFuse 基础开关时需要注意哪些事项?\n", | ||
| 232 | + " A. 先开启 <code>--enable_autofuse=true</code>\n", | ||
| 233 | + " B. Reduce、Concat 默认不使能,需要时可通过 <code>--autofuse_enable_pass</code> 显式开启\n", | ||
| 234 | + " C. 开启 AutoFuse 后仍要结合模型和 profiling 判断是否收益\n", | ||
| 235 | + " D. 只设置 <code>AUTOFUSE_DFX_FLAGS</code> 就能替代基础开关\n", | ||
| 236 | + "\n", | ||
| 237 | + "**执行以下代码获取答案。**" | ||
| 238 | + ] | ||
| 239 | + }, | ||
| 240 | + { | ||
| 241 | + "cell_type": "code", | ||
| 242 | + "execution_count": null, | ||
| 243 | + "id": "code00", | ||
| 244 | + "metadata": {}, | ||
| 245 | + "outputs": [], | ||
| 246 | + "source": [ | ||
| 247 | + "!cat ./answer/01.03_answer.txt" | ||
| 248 | + ] | ||
| 249 | + } | ||
| 250 | + ], | ||
| 251 | + "metadata": { | ||
| 252 | + "kernelspec": { | ||
| 253 | + "display_name": "Python 3", | ||
| 254 | + "language": "python", | ||
| 255 | + "name": "python3" | ||
| 256 | + }, | ||
| 257 | + "language_info": { | ||
| 258 | + "name": "python", | ||
| 259 | + "pygments_lexer": "ipython3", | ||
| 260 | + "version": "3.12.9" | ||
| 261 | + } | ||
| 262 | + }, | ||
| 263 | + "nbformat": 4, | ||
| 264 | + "nbformat_minor": 5 | ||
| 265 | +} | ||
| @@ -0,0 +1,73 @@ | |||
| 1 | +{ | ||
| 2 | + "cells": [ | ||
| 3 | + { | ||
| 4 | + "cell_type": "markdown", | ||
| 5 | + "id": "md00", | ||
| 6 | + "metadata": {}, | ||
| 7 | + "source": [ | ||
| 8 | + "# 章节练习\n", | ||
| 9 | + "\n", | ||
| 10 | + "本章节练习用于综合检查 AutoFuse 基础介绍相关内容的学习效果,重点考察你是否能够把 AutoFuse 的定位、收益来源和基础使能方式串联起来。\n", | ||
| 11 | + "\n", | ||
| 12 | + "1. (判断题)AutoFuse 属于图编译流程中的自动融合能力,普通使用者通常通过环境变量和模型编译入口间接使用,而不是把它当作一个运行时业务 API 直接调用。\n", | ||
| 13 | + "\n", | ||
| 14 | + "2. (判断题)如果模型中存在 Reduce 或 Concat 结构,只要设置 `--enable_autofuse=true`,Reduce 和 Concat 融合能力就一定已经开启。\n", | ||
| 15 | + "\n", | ||
| 16 | + "3. (单选题)结合 FusedGraph 结构图,以下哪种描述更符合 AutoFuse 前端确定融合范围后的表达方式?\n", | ||
| 17 | + " A. FusedGraph 用来表达一个融合范围,内部可以包含一个或多个 AscBackend 节点\n", | ||
| 18 | + " B. FusedGraph 是用户手写的业务模型代码,编译阶段不会再转换它\n", | ||
| 19 | + " C. FusedGraph 只表示单个原始算子,不能表达多个算子组成的融合范围\n", | ||
| 20 | + " D. FusedGraph 只用于运行时调度,与图编译阶段的自动融合无关\n", | ||
| 21 | + "\n", | ||
| 22 | + "4. (单选题)关于 AutoFuse 的两条接入路线,以下哪种理解更符合本章节介绍?\n", | ||
| 23 | + " A. GE 框架路径注重图模式融合,PyTorch Inductor 路径注重生态对接,二者最终都会对接 Ascend C IR 与代码生成等底层能力\n", | ||
| 24 | + " B. GE 框架路径只负责产品型号检测,不参与融合范围识别和图表达转换\n", | ||
| 25 | + " C. PyTorch Inductor 路径完全绕过 Ascend C IR,因此无法复用 AutoFuse 的后端生成能力\n", | ||
| 26 | + " D. 两条接入路线互相排斥,学习 AutoFuse 时只能选择其中一条理解\n", | ||
| 27 | + "\n", | ||
| 28 | + "5. (多选题)普通使用者希望验证 AutoFuse 基础开关是否设置到当前环境,应重点检查哪些信息?\n", | ||
| 29 | + " A. CANN 环境变量是否已经正确加载\n", | ||
| 30 | + " B. `AUTOFUSE_FLAGS` 是否在当前执行环境中设置\n", | ||
| 31 | + " C. 模型是否通过 ATC 等可触发 GE 图编译的入口生成编译产物\n", | ||
| 32 | + " D. 模型推理精度是否满足预期\n", | ||
| 33 | + "\n", | ||
| 34 | + "6. (多选题)如果开启 AutoFuse 后没有明显性能收益,以下哪些分析方向更合理?\n", | ||
| 35 | + " A. 检查模型瓶颈是否主要来自 Memory Bound,而不是其他大计算算子\n", | ||
| 36 | + " B. 检查模型中的算子类型、shape 和融合范围是否满足融合条件\n", | ||
| 37 | + " C. 使用 profiling、编译日志或 DFX 信息对比开启前后的差异\n", | ||
| 38 | + " D. 直接认定 AutoFuse 功能失效,因为开启后必须提升性能\n", | ||
| 39 | + "\n", | ||
| 40 | + "7. (多选题)关于 AutoFuse 基础使能方式,以下说法正确的是哪些?\n", | ||
| 41 | + " A. `AUTOFUSE_FLAGS` 用于控制 AutoFuse 是否开启以及启用哪些基础融合能力\n", | ||
| 42 | + " B. `AUTOFUSE_DFX_FLAGS` 属于进阶定位和调优配置,不是基础使能必需项\n", | ||
| 43 | + " C. Reduce、Concat 默认不使能,需要按场景通过 pass 显式开启\n", | ||
| 44 | + " D. 开启 AutoFuse 后可以默认认为所有模型都有明显性能收益\n", | ||
| 45 | + "\n", | ||
| 46 | + "**执行以下代码获取答案。**" | ||
| 47 | + ] | ||
| 48 | + }, | ||
| 49 | + { | ||
| 50 | + "cell_type": "code", | ||
| 51 | + "execution_count": null, | ||
| 52 | + "id": "code00", | ||
| 53 | + "metadata": {}, | ||
| 54 | + "outputs": [], | ||
| 55 | + "source": [ | ||
| 56 | + "!cat ./answer/01.04_answer.txt" | ||
| 57 | + ] | ||
| 58 | + } | ||
| 59 | + ], | ||
| 60 | + "metadata": { | ||
| 61 | + "kernelspec": { | ||
| 62 | + "display_name": "Python 3", | ||
| 63 | + "language": "python", | ||
| 64 | + "name": "python3" | ||
| 65 | + }, | ||
| 66 | + "language_info": { | ||
| 67 | + "name": "python", | ||
| 68 | + "pygments_lexer": "ipython3" | ||
| 69 | + } | ||
| 70 | + }, | ||
| 71 | + "nbformat": 4, | ||
| 72 | + "nbformat_minor": 5 | ||
| 73 | +} | ||
| @@ -0,0 +1,14 @@ | |||
| 1 | +1. √ | ||
| 2 | +解析:AutoFuse 主要在图编译阶段发挥作用,根据模型图结构自动识别可融合范围,并生成融合算子相关实现,从而减少算子数量和中间数据搬运。 | ||
| 3 | + | ||
| 4 | +2. × | ||
| 5 | +解析:AutoFuse 是否带来性能收益取决于模型结构、shape、算子类型、内存搬运瓶颈和实际性能分析结果,并不保证所有模型都提升。 | ||
| 6 | + | ||
| 7 | +3. B | ||
| 8 | +解析:AutoFuse 的核心收益来源之一是将多个满足条件的小算子融合为一个算子,减少中间结果反复搬运和多次调度开销,从而缓解 Memory Bound 问题。 | ||
| 9 | + | ||
| 10 | +4. A | ||
| 11 | +解析:AutoFuse 包含基于 GE 框架和对接 PyTorch Inductor 的两条路线,两条路线都体现了自动融合的关键思想。当前课程从自动融合原理出发理解整体链路;PyTorch Inductor 对接路线已具备功能支持,但尚未作为商用交付能力。 | ||
| 12 | + | ||
| 13 | +5. A、B、C | ||
| 14 | +解析:普通使用者通常通过 `AUTOFUSE_FLAGS` 在图编译阶段开启能力。AutoFuse 依赖 GE 图编译流程生效,通过 ATC 触发离线编译生成 OM 是常见场景;如果在线场景同样进入支持 AutoFuse 的 GE 图编译路径,也可能触发自动融合。验证效果时需要结合编译日志、融合产物、性能数据和精度对比判断;普通业务代码不需要直接调用 AutoFuse 内部后端接口,相关能力通常由 GE 图编译、ATC 离线编译或对接 PyTorch Inductor 的编译链路在满足条件时触发。 | ||
| @@ -0,0 +1,14 @@ | |||
| 1 | +1. √ | ||
| 2 | +解析:AutoFuse 的基础使能入口是 `AUTOFUSE_FLAGS`,最小开启配置可以写为 `--enable_autofuse=true`。 | ||
| 3 | + | ||
| 4 | +2. × | ||
| 5 | +解析:进阶定位或调优配置用于辅助分析和优化,不是开启 AutoFuse 的必需配置。基础使能主要依赖 `AUTOFUSE_FLAGS`。 | ||
| 6 | + | ||
| 7 | +3. A | ||
| 8 | +解析:Reduce、Concat 默认不使能,需要额外开启时可通过 `--autofuse_enable_pass=reduce,concat` 显式配置,同时需要先开启 `--enable_autofuse=true`。 | ||
| 9 | + | ||
| 10 | +4. A | ||
| 11 | +解析:环境变量只对当前 shell 或其子进程生效,因此需要确认编译命令和环境变量处在同一个执行环境中。 | ||
| 12 | + | ||
| 13 | +5. A、B、C | ||
| 14 | +解析:普通使用者应先开启整体 AutoFuse 能力,再按场景开启 Reduce、Concat 等扩展融合能力;开启后仍需要结合实际模型和后续性能分析判断是否收益。进阶定位或调优配置不能替代基础开关。 | ||
| @@ -0,0 +1,20 @@ | |||
| 1 | +1. √ | ||
| 2 | +解析:AutoFuse 是图编译流程中的自动融合能力。普通使用者通常通过环境变量和模型编译入口间接使用,不需要把内部后端 API 当作运行时业务接口直接调用。 | ||
| 3 | + | ||
| 4 | +2. × | ||
| 5 | +解析:Reduce、Concat 默认不使能。即使设置了 `--enable_autofuse=true`,如需开启 Reduce 或 Concat 融合能力,仍需要通过 `--autofuse_enable_pass=reduce,concat` 等配置显式开启。 | ||
| 6 | + | ||
| 7 | +3. A | ||
| 8 | +解析:FusedGraph 用于表达 AutoFuse 前端识别出的一个融合范围,内部可以包含一个或多个 AscBackend 节点。它不是用户手写的业务模型代码,也不是只表示单个原始算子的运行时调度对象。 | ||
| 9 | + | ||
| 10 | +4. A | ||
| 11 | +解析:AutoFuse 包含基于 GE 框架的图模式融合路径,也包含对接 PyTorch Inductor 的生态融合路径。GE 路径更侧重昇腾自研图编译体系中的融合能力,PyTorch Inductor 路径更侧重与 PyTorch 编译生态对接;两条路径都会围绕 Ascend C IR、Schedule、Codegen 等底层能力完成后续表达转换和代码生成。 | ||
| 12 | + | ||
| 13 | +5. A、B、C | ||
| 14 | +解析:验证 AutoFuse 基础开关是否设置到当前环境,应检查 CANN 环境变量是否加载、`AUTOFUSE_FLAGS` 是否在当前执行环境中设置,以及模型是否经过 ATC 等可触发 GE 图编译的入口。模型推理精度是否满足预期属于精度验证范畴,与开关配置确认不直接相关。 | ||
| 15 | + | ||
| 16 | +6. A、B、C | ||
| 17 | +解析:没有明显性能收益时,应结合瓶颈类型、算子类型、shape、融合范围、profiling、编译日志和 DFX 信息分析。AutoFuse 不保证所有模型开启后都提升性能。 | ||
| 18 | + | ||
| 19 | +7. A、B、C | ||
| 20 | +解析:`AUTOFUSE_FLAGS` 用于功能控制;`AUTOFUSE_DFX_FLAGS` 更适合用于进阶验证、定位和调优辅助;Reduce、Concat 默认不使能,需要按场景通过 pass 显式开启。开启 AutoFuse 后不能默认认为所有模型都有明显性能收益。 | ||
| @@ -0,0 +1,73 @@ | |||
| 1 | +# AutoFuse 自动融合开发系列教程 | ||
| 2 | + | ||
| 3 | +本教程面向希望理解并掌握昇腾 AutoFuse 自动融合能力的开发者,基于 CANN 官方资料整理。课程设计兼顾普通使用者与进阶开发者需求:从零开始建立认知,逐步深入到融合原理与实战应用。 | ||
| 4 | + | ||
| 5 | +## 教程结构 | ||
| 6 | + | ||
| 7 | +教程按章节组织,每个章节包含以下内容: | ||
| 8 | + | ||
| 9 | +- **Notebooks**:涵盖课程知识点、可执行示例与练习题,适用于自主学习或讲师引导式教学。可在 gitcode 提供的轻量级 Notebook 环境中直接运行,也可自行搭建 JupyterLab 在本地执行。 | ||
| 10 | +- **answer**:提供课后习题与章节练习的参考答案,便于自测与验证。 | ||
| 11 | +- **images**:存放章节所需的图示资源,Notebook 中通过相对路径引用。 | ||
| 12 | + | ||
| 13 | +> **注意事项** | ||
| 14 | +> | ||
| 15 | +> - AutoFuse 自动融合特性仅支持 Atlas 350 加速卡、Atlas A2 训练/推理系列产品、Atlas A3 训练/推理系列产品。 | ||
| 16 | +> - AutoFuse 的使能方式取决于对接路线。采用 GE 图编译路线时,需在模型图编译前通过环境变量 `AUTOFUSE_FLAGS` 开启自动融合;采用 PyTorch Inductor 对接路线时,当前无需额外配置环境变量,只需在 Python 脚本中导入 `inductor_npu_ext`。使能后,AutoFuse 会在编译阶段自动识别可融合算子模式、生成融合内核并完成相关优化,无需用户手工编写融合代码。 | ||
| 17 | + | ||
| 18 | +## 软硬件配套说明 | ||
| 19 | + | ||
| 20 | +| 项目 | 要求 | | ||
| 21 | +| --------- | ------------------------------------------------------------------------ | | ||
| 22 | +| 支持硬件 | Atlas 350 加速卡、Atlas A2 训练/推理系列产品、Atlas A3 训练/推理系列产品 | | ||
| 23 | +| CANN 版本 | 9.0.0 及以上 | | ||
| 24 | +| Python | 3.11 | | ||
| 25 | + | ||
| 26 | +## 在线体验环境 | ||
| 27 | + | ||
| 28 | +本教程支持以下在线体验环境: | ||
| 29 | + | ||
| 30 | +| 体验环境 | 镜像模板 / 版本 | Python 内核 | 说明 | | ||
| 31 | +| ----------------------------------- | ------------------------ | -------------- | ------------------------------------------------------------------------------------------------------------------------------------------------- | | ||
| 32 | +| cann-learning-hub 在线体验 notebook | cann_9.0.0_py3.11-A2-arm | Python 3.11.15 | 各 Notebook 表格中的"在线体验"链接可直接打开运行 | | ||
| 33 | +| CANNLab 云开发环境 | cann_9.0.0_py3.11-A2-arm | Python 3.11.4 | 参考[CANNLab 环境体验指南](https://gitcode.com/cann/cann-learning-hub/blob/master/docs/CANNLab_env_experience_guide.md)创建CANNLab环境运行notebook | | ||
| 34 | + | ||
| 35 | +> **注意:** 如在本地环境离线体验,需自行安装配套的 CANN 软件,具体请参考 [CANN 安装指南](https://www.hiascend.com/document/detail/zh/CANNCommunityEdition/600alpha003/softwareinstall/instg/atlasdeploy_03_0001.html)。 | ||
| 36 | + | ||
| 37 | +## 课程目录 | ||
| 38 | + | ||
| 39 | +### 第一章 AutoFuse 基础介绍 | ||
| 40 | + | ||
| 41 | +帮助建立 AutoFuse 的整体认知,掌握基础使能方式 | ||
| 42 | + | ||
| 43 | +| Notebook | 链接 | 状态 | | ||
| 44 | +| -------------------- | -------------- | --------- | | ||
| 45 | +| 1.1 章节介绍 | 在线体验建设中 | ✅ 已发布 | | ||
| 46 | +| 1.2 AutoFuse简介 | 在线体验建设中 | ✅ 已发布 | | ||
| 47 | +| 1.3 AutoFuse使能基础 | 在线体验建设中 | ✅ 已发布 | | ||
| 48 | +| 1.4 章节练习 | 在线体验建设中 | ✅ 已发布 | | ||
| 49 | + | ||
| 50 | +### 第二章 AutoFuse 自动融合原理 | ||
| 51 | + | ||
| 52 | +深入理解融合条件判断与策略求解的核心机制 | ||
| 53 | + | ||
| 54 | +| Notebook | 链接 | 状态 | | ||
| 55 | +| ---------------- | -------------- | ---------- | | ||
| 56 | +| 2.1 章节介绍 | 在线体验建设中 | ⏳ 未发布 | | ||
| 57 | +| 2.2 融合条件判断 | 在线体验建设中 | ⏳ 未发布 | | ||
| 58 | +| 2.3 融合策略求解 | 在线体验建设中 | ⏳ 未发布 | | ||
| 59 | +| 2.4 章节练习 | 在线体验建设中 | ⏳ 未发布 | | ||
| 60 | + | ||
| 61 | +### 第三章 AutoFuse 项目实践与问题定位 | ||
| 62 | + | ||
| 63 | +面向实际项目开发,掌握高级配置与问题排查方法 | ||
| 64 | + | ||
| 65 | +| Notebook | 链接 | 状态 | | ||
| 66 | +| -------------------------- | -------------- | --------- | | ||
| 67 | +| 3.1 章节介绍 | 在线体验建设中 | ⏳ 未发布 | | ||
| 68 | +| 3.2 AutoFuse使能进阶 | 在线体验建设中 | ⏳ 未发布 | | ||
| 69 | +| 3.3 对接PyTorch项目实践 | 在线体验建设中 | ⏳ 未发布 | | ||
| 70 | +| 3.4 对接TensorFlow项目实践 | 在线体验建设中 | ⏳ 未发布 | | ||
| 71 | +| 3.5 性能分析方法 | 在线体验建设中 | ⏳ 未发布 | | ||
| 72 | +| 3.6 问题定位方法 | 在线体验建设中 | ⏳ 未发布 | | ||
| 73 | +| 3.7 章节练习 | 在线体验建设中 | ⏳ 未发布 | | ||