已合并
skills: 新增三个 CANN NPU 性能优化 skill (multistream-optimize / npu-perfanalysis / perf-breakdown) #302
unstain创建于 6月10日
skills: 新增三个 CANN NPU 性能优化 skill (multistream-optimize / npu-perfanalysis / perf-breakdown) #302
已合并
共 29 个文件变更+10852-0
| @@ -0,0 +1,168 @@ | |||
| 1 | +--- | ||
| 2 | +name: cann-multistream-optimize | ||
| 3 | +description: NPU 多流整网优化技能。用于分析和实施模型的多流优化、双流、stream overlap、控核与 TorchAir 多流改造。先做整网模块 DAG 与模块间并行性分析,再做每个模块的算子级分析、开发调试和验收。触发场景包括:多流优化、双流、stream overlap、控核、整网 DAG、模块拆解、NPU 多流、TorchAir 多流、limit_core_num、多流改造没收益调试。 | ||
| 4 | +user-invocable: true | ||
| 5 | +--- | ||
| 6 | + | ||
| 7 | +# NPU 多流整网优化技能 | ||
| 8 | + | ||
| 9 | +面向本仓库的 NPU 多流整网优化任务,按“分析 → 开发调试 → 验收”推进。优先参考仓库内已有案例、代码模式和官方 API 索引。 | ||
| 10 | + | ||
| 11 | +## 概述 | ||
| 12 | + | ||
| 13 | +本技能解决的是**整网多流优化**。 | ||
| 14 | + | ||
| 15 | +工作主线固定如下: | ||
| 16 | + | ||
| 17 | +1. 先拆整网模块,画模块级 DAG,判断**模块与模块**哪些能并行。 | ||
| 18 | +2. 再对**每个模块**拆算子,画模块内算子 DAG,判断模块内哪些算子能并行。 | ||
| 19 | +3. 在依赖关系明确后再做多流开发、调试和验收。 | ||
| 20 | + | ||
| 21 | +分析和实施结果统一落到: | ||
| 22 | + | ||
| 23 | +`multi-stream-analysis/<network_or_case_name>.md` | ||
| 24 | + | ||
| 25 | +报告结构使用 [`references/report-template.md`](references/report-template.md)。 | ||
| 26 | + | ||
| 27 | +## 重要原则 | ||
| 28 | + | ||
| 29 | +- **先整网后局部**:先回答整网主路径和模块间并行性,再进入模块内算子。 | ||
| 30 | +- **模块级目标明确**:模块拆解的目标是判断**模块与模块**哪些可并行。 | ||
| 31 | +- **每个模块都要下钻**:第二层要对每个模块都补齐算子清单、算子 DAG 和模块内并行性结论。 | ||
| 32 | +- **同步必须显式**:多流改造必须有清晰的 `event` / `wait` / `wait_stream` / `wait_tensor` 关系。 | ||
| 33 | +- **开关必须可关闭**:多流路径要保留 enable 开关和原始回退路径,方便调试与验收。 | ||
| 34 | +- **先证明正确,再追性能**:先验证依赖、功能和精度,再看 overlap 和时延收益。 | ||
| 35 | +- **overlap 不等于收益**:出现拖尾、资源争抢、host bound、shape 劣化时,要继续评估控核、和图模式限制。 | ||
| 36 | +- **不要混抄案例**:必须先确定当前模型的执行模式,再选一套主 API 路径,不要把 eager 和 graph 风格混着套。 | ||
| 37 | +- **调试优化最优编排**:优化点分流编排不唯一,调试时要多尝试不同的编排,选择性能最优方案。 | ||
| 38 | + | ||
| 39 | +## 工作流程 | ||
| 40 | + | ||
| 41 | +### 第一步:分析 | ||
| 42 | + | ||
| 43 | +1. 先读取 [`references/report-template.md`](references/report-template.md),在 `docs/common/multi-stream-analysis/` 下创建或更新结果文件。 | ||
| 44 | +2. 按 [`references/module-decomposition-spec.md`](references/module-decomposition-spec.md) 完成整网模块拆解: | ||
| 45 | + - 确定分析阶段是 `prefill` 还是 `decode` | ||
| 46 | + - 产出整网模块清单、依赖清单、Mermaid 模块 DAG | ||
| 47 | + - 输出模块级并行性结论,统一使用“主串行链 / 可并行组 / 待验证组 / 建议流分组” | ||
| 48 | +3. 对**每个模块**继续做算子级拆解: | ||
| 49 | + - 产出模块内算子清单、依赖清单、Mermaid 算子 DAG | ||
| 50 | + - 输出模块内并行性结论,仍使用分组式描述 | ||
| 51 | +4. 选案例时按 [`references/case-routing.md`](references/case-routing.md) 进入最接近的模式。 | ||
| 52 | + | ||
| 53 | +### 第二步:开发 | ||
| 54 | + | ||
| 55 | +1. 先判断当前代码属于哪种执行模式: | ||
| 56 | + - eager / patch 风格 | ||
| 57 | + - graph / TorchAir 风格 | ||
| 58 | +2. 只选一套主 API 路径实现第一版 overlap: | ||
| 59 | + - eager / patch:`torch.npu.Stream()`、`record_event()`、`wait_event()`、`wait_stream()` | ||
| 60 | + - graph / TorchAir:`npu_stream_switch`、`npu_tagged_event_*`、`npu_wait_tensor` | ||
| 61 | +3. 优先做**最小可验证 overlap**: | ||
| 62 | + - 只改一个明确的并行点 | ||
| 63 | + - 先补同步,再扩大并行窗口 | ||
| 64 | + - 先保留原路径和 enable 开关 | ||
| 65 | +4. 若 overlap 已成立但仍有明显拖尾,再评估: | ||
| 66 | + - `limit_core_num` | ||
| 67 | + - `set_stream_limit` / `get_stream_limit` | ||
| 68 | + - `npu_prefetch` | ||
| 69 | +5. 所有开发决策回写结果 md:写清流分组、同步点、开关、回退路径和预期收益。 | ||
| 70 | + | ||
| 71 | +### 第三步:调试 | ||
| 72 | + | ||
| 73 | +如果多流编排方式不唯一,多尝试不同的编排方式,选择性能最优的方案。调试时按下面 4 类问题分开处理,不要混在一起排查: | ||
| 74 | + | ||
| 75 | +1. **依赖错误 / 同步错误** | ||
| 76 | + - 重点检查事件记录、等待顺序、跨流汇合点、共享状态写入次序 | ||
| 77 | + - 典型现象:读到未完成结果、死等、结果偶发错误 | ||
| 78 | +2. **精度或功能异常** | ||
| 79 | + - 先对比优化前基线,再按 `prefill/decode → 模块 → 算子` 缩小范围 | ||
| 80 | + - 判断是否是多流引入的状态时序问题,而不是算子本身问题 | ||
| 81 | +3. **性能无收益** | ||
| 82 | + - 重点看 shape 变化、task 数量增加、host bound、带宽争抢、流间资源抢占、拖尾 | ||
| 83 | + - 需要时继续评估控核、预取、superkernel 或缩小 overlap 范围 | ||
| 84 | +4. **图模式 / runtime 限制** | ||
| 85 | + - 重点看 graph break、图模式 API 约束、stream 语义差异、运行时不支持 | ||
| 86 | + - graph 场景优先按 TorchAir 路径排查,不要回退成 eager 思路硬套 | ||
| 87 | + | ||
| 88 | +### 第四步:验收 | ||
| 89 | + | ||
| 90 | +验收必须至少覆盖下面 4 类,并把结果回写报告: | ||
| 91 | + | ||
| 92 | +1. **功能验收** | ||
| 93 | + - 多流 enable 前后输出一致 | ||
| 94 | + - 开关关闭后原路径可正常运行 | ||
| 95 | +2. **同步验收** | ||
| 96 | + - 汇合点前后无缺失等待 | ||
| 97 | + - 共享状态、KVCache、通信结果没有读写乱序 | ||
| 98 | +3. **性能验收** | ||
| 99 | + - 记录优化前后关键时延、吞吐或单步耗时,确认确实有 overlap 和时延改善 | ||
| 100 | + - 确认优化点之外的算子或模块耗时没有劣化 | ||
| 101 | + - 如果无收益,要明确是实现问题还是场景不适合 | ||
| 102 | +4. **Profile 验收** | ||
| 103 | + - 确认确实存在 overlap,而不是逻辑上分流但执行上仍串行 | ||
| 104 | + - 确认关键拖尾是否已缩短,是否出现新的空洞或资源争抢 | ||
| 105 | + | ||
| 106 | +## API 选型规则 | ||
| 107 | + | ||
| 108 | +### eager / patch 风格 | ||
| 109 | + | ||
| 110 | +优先使用: | ||
| 111 | + | ||
| 112 | +- `torch.npu.Stream()` | ||
| 113 | +- `record_event()` | ||
| 114 | +- `wait_event()` | ||
| 115 | +- `wait_stream()` | ||
| 116 | + | ||
| 117 | +适用场景: | ||
| 118 | + | ||
| 119 | +- patch 形态改造 | ||
| 120 | +- 现有代码已经显式使用 `torch.npu.current_stream()` | ||
| 121 | +- runtime 不走 TorchAir 图内多流表达 | ||
| 122 | + | ||
| 123 | +### graph / TorchAir 风格 | ||
| 124 | + | ||
| 125 | +优先使用: | ||
| 126 | + | ||
| 127 | +- `npu_stream_switch` | ||
| 128 | +- `npu_wait_tensor` | ||
| 129 | +- `npu_record_tagged_stream` | ||
| 130 | +- `npu_tagged_event_wait` | ||
| 131 | + | ||
| 132 | +适用场景: | ||
| 133 | + | ||
| 134 | +- `ge_graph` / TorchAir 多流表达 | ||
| 135 | +- 代码已经显式使用 scope / tagged event 风格 | ||
| 136 | +- 需要把同步关系表达在图内 | ||
| 137 | + | ||
| 138 | +### 拖尾与资源问题 | ||
| 139 | + | ||
| 140 | +出现“已 overlap 但收益不稳定”时,再评估: | ||
| 141 | + | ||
| 142 | +- `limit_core_num` | ||
| 143 | +- `torch_npu.set_stream_limit` | ||
| 144 | +- `torch_npu.get_stream_limit` | ||
| 145 | +- `torch_npu.npu_prefetch` | ||
| 146 | + | ||
| 147 | +使用顺序: | ||
| 148 | + | ||
| 149 | +1. 先确认依赖和 overlap 正确 | ||
| 150 | +2. 再看是否存在拖尾或资源争抢 | ||
| 151 | +3. 最后再引入控核、stream limit 或预取 | ||
| 152 | + | ||
| 153 | +详细路由见 [`references/api-routing.md`](references/api-routing.md)。 | ||
| 154 | + | ||
| 155 | +## 参考文档索引 | ||
| 156 | + | ||
| 157 | +按需读取。 | ||
| 158 | + | ||
| 159 | +- **整网模块拆解规范**: | ||
| 160 | + [`references/module-decomposition-spec.md`](references/module-decomposition-spec.md) | ||
| 161 | +- **案例选型路由**: | ||
| 162 | + [`references/case-routing.md`](references/case-routing.md) | ||
| 163 | +- **官方 API 索引**: | ||
| 164 | + [`references/official-docs-latest.md`](references/official-docs-latest.md) | ||
| 165 | +- **API 选型路由**: | ||
| 166 | + [`references/api-routing.md`](references/api-routing.md) | ||
| 167 | +- **固定报告模板**: | ||
| 168 | + [`references/report-template.md`](references/report-template.md) | ||
| @@ -0,0 +1,153 @@ | |||
| 1 | +# <network_or_case_name> 多流分析 | ||
| 2 | + | ||
| 3 | +## 1. 分析范围 | ||
| 4 | + | ||
| 5 | +- 模型/网络: | ||
| 6 | +- 阶段:`prefill` / `decode` | ||
| 7 | +- 分析对象: | ||
| 8 | +- 说明: | ||
| 9 | + | ||
| 10 | +## 2. 整网模块清单 | ||
| 11 | + | ||
| 12 | +| module_id | module_name | module_type | inputs | outputs | side_effect | resource_hint | | ||
| 13 | +| --- | --- | --- | --- | --- | --- | --- | | ||
| 14 | + | ||
| 15 | +## 3. 整网模块依赖清单 | ||
| 16 | + | ||
| 17 | +| from | to | dependency_type | reason | | ||
| 18 | +| --- | --- | --- | --- | | ||
| 19 | + | ||
| 20 | +## 4. 整网模块 DAG | ||
| 21 | + | ||
| 22 | +```mermaid | ||
| 23 | +flowchart LR | ||
| 24 | +``` | ||
| 25 | + | ||
| 26 | +## 5. 模块级并行性结论 | ||
| 27 | + | ||
| 28 | +### 5.1 主串行链 | ||
| 29 | + | ||
| 30 | +- | ||
| 31 | + | ||
| 32 | +### 5.2 可并行模块组 | ||
| 33 | + | ||
| 34 | +- 组 A: | ||
| 35 | +- 组 B: | ||
| 36 | + | ||
| 37 | +### 5.3 待验证模块组 | ||
| 38 | + | ||
| 39 | +- | ||
| 40 | + | ||
| 41 | +### 5.4 建议流分组 | ||
| 42 | + | ||
| 43 | +- `Stream0` / `Main Path`: | ||
| 44 | +- `Stream1` / `Side Path`: | ||
| 45 | +- `Stream2` / `Comm Path`: | ||
| 46 | + | ||
| 47 | +## 6. 模块拆解结论 | ||
| 48 | + | ||
| 49 | +- 哪些模块必须串行: | ||
| 50 | +- 哪些模块可以并行: | ||
| 51 | +- 哪些模块并行性待验证: | ||
| 52 | +- 推荐优先检查的多流切入点: | ||
| 53 | + | ||
| 54 | +## 7. 模块内算子拆解 | ||
| 55 | + | ||
| 56 | +### 7.1 <module_name> | ||
| 57 | + | ||
| 58 | +#### 算子清单 | ||
| 59 | + | ||
| 60 | +| op_id | op_name | op_type | inputs | outputs | side_effect | resource_hint | | ||
| 61 | +| --- | --- | --- | --- | --- | --- | --- | | ||
| 62 | + | ||
| 63 | +#### 算子依赖清单 | ||
| 64 | + | ||
| 65 | +| from | to | dependency_type | reason | | ||
| 66 | +| --- | --- | --- | --- | | ||
| 67 | + | ||
| 68 | +#### 算子 DAG | ||
| 69 | + | ||
| 70 | +```mermaid | ||
| 71 | +flowchart LR | ||
| 72 | +``` | ||
| 73 | + | ||
| 74 | +#### 算子并行性结论 | ||
| 75 | + | ||
| 76 | +##### 模块内主串行链 | ||
| 77 | + | ||
| 78 | +- | ||
| 79 | + | ||
| 80 | +##### 模块内可并行算子组 | ||
| 81 | + | ||
| 82 | +- 组 A: | ||
| 83 | +- 组 B: | ||
| 84 | + | ||
| 85 | +##### 模块内待验证算子组 | ||
| 86 | + | ||
| 87 | +- | ||
| 88 | + | ||
| 89 | +##### 模块内建议流分组 | ||
| 90 | + | ||
| 91 | +- `Stream0` / `Main Path`: | ||
| 92 | +- `Stream1` / `Side Path`: | ||
| 93 | +- `Stream2` / `Comm Path`: | ||
| 94 | + | ||
| 95 | +#### 模块内结论 | ||
| 96 | + | ||
| 97 | +- 模块内必须串行的部分: | ||
| 98 | +- 模块内可以并行的部分: | ||
| 99 | +- 模块内待验证的部分: | ||
| 100 | +- 可能的流切换点 / 同步点: | ||
| 101 | + | ||
| 102 | +### 7.2 <module_name> | ||
| 103 | + | ||
| 104 | +#### 算子清单 | ||
| 105 | + | ||
| 106 | +| op_id | op_name | op_type | inputs | outputs | side_effect | resource_hint | | ||
| 107 | +| --- | --- | --- | --- | --- | --- | --- | | ||
| 108 | + | ||
| 109 | +#### 算子依赖清单 | ||
| 110 | + | ||
| 111 | +| from | to | dependency_type | reason | | ||
| 112 | +| --- | --- | --- | --- | | ||
| 113 | + | ||
| 114 | +#### 算子 DAG | ||
| 115 | + | ||
| 116 | +```mermaid | ||
| 117 | +flowchart LR | ||
| 118 | +``` | ||
| 119 | + | ||
| 120 | +#### 算子并行性结论 | ||
| 121 | + | ||
| 122 | +##### 模块内主串行链 | ||
| 123 | + | ||
| 124 | +- | ||
| 125 | + | ||
| 126 | +##### 模块内可并行算子组 | ||
| 127 | + | ||
| 128 | +- 组 A: | ||
| 129 | +- 组 B: | ||
| 130 | + | ||
| 131 | +##### 模块内待验证算子组 | ||
| 132 | + | ||
| 133 | +- | ||
| 134 | + | ||
| 135 | +##### 模块内建议流分组 | ||
| 136 | + | ||
| 137 | +- `Stream0` / `Main Path`: | ||
| 138 | +- `Stream1` / `Side Path`: | ||
| 139 | +- `Stream2` / `Comm Path`: | ||
| 140 | + | ||
| 141 | +#### 模块内结论 | ||
| 142 | + | ||
| 143 | +- 模块内必须串行的部分: | ||
| 144 | +- 模块内可以并行的部分: | ||
| 145 | +- 模块内待验证的部分: | ||
| 146 | +- 可能的流切换点 / 同步点: | ||
| 147 | + | ||
| 148 | +## 8. 最终结论 | ||
| 149 | + | ||
| 150 | +- 整网最适合做多流优化的位置: | ||
| 151 | +- 模块级并行的主结论: | ||
| 152 | +- 算子级并行的主结论: | ||
| 153 | +- 后续建议: | ||
| @@ -0,0 +1,36 @@ | |||
| 1 | +# 多流与控核 API 路由 | ||
| 2 | + | ||
| 3 | +本文件用于把“执行模式 / 问题类型”映射到推荐 API。官方文档版本以 [`official-docs-latest.md`](./official-docs-latest.md) 为准。 | ||
| 4 | + | ||
| 5 | +## 先判执行模式 | ||
| 6 | + | ||
| 7 | +| 当前场景 | 推荐 API 风格 | 首选 API | 官方文档入口 | | ||
| 8 | +| --- | --- | --- | --- | | ||
| 9 | +| eager / patch 改造 | 显式流对象 | `torch.npu.Stream()`、`record_event()`、`wait_event()`、`wait_stream()` | 优先参考仓库案例;图内表达总说明见[图内多流表达功能(Ascend IR)](https://www.hiascend.com/document/detail/zh/Pytorch/730/modthirdparty/torchairuseguide/torchair_00033.html) | | ||
| 10 | +| `ge_graph` / TorchAir 图内多流 | TorchAir scope / tagged event | `npu_stream_switch`、`npu_wait_tensor`、`npu_record_tagged_stream`、`npu_tagged_event_wait` | [npu_stream_switch](https://www.hiascend.com/document/detail/zh/Pytorch/720/modthirdparty/torchairuseguide/torchair_00101.html), [npu_wait_tensor](https://www.hiascend.com/document/detail/zh/Pytorch/730/modthirdparty/torchairuseguide/torchair_00114.html), [npu_record_tagged_stream](https://www.hiascend.com/document/detail/zh/Pytorch/720/modthirdparty/torchairuseguide/torchair_00099.html), [npu_tagged_event_wait](https://www.hiascend.com/document/detail/zh/Pytorch/730/modthirdparty/torchairuseguide/torchair_00108.html) | | ||
| 11 | +| `aclgraph` / 图模式入口判断 | 图模式总说明 | 先看图模式入口和多流总说明,再选具体 API | [PyTorch 图模式使用(TorchAir)入口](https://www.hiascend.com/document/detail/zh/Pytorch/730/index/index.html), [图内多流表达功能(aclgraph)](https://www.hiascend.com/document/detail/zh/Pytorch/730/modthirdparty/torchairuseguide/torchair_00024.html) | | ||
| 12 | + | ||
| 13 | +## 再判问题类型 | ||
| 14 | + | ||
| 15 | +| 问题类型 | 推荐 API | 什么时候用 | 注意事项 | 官方文档入口 | | ||
| 16 | +| --- | --- | --- | --- | --- | | ||
| 17 | +| 需要把一段计算切到副流 | `torch.npu.Stream()` 或 `npu_stream_switch` | 已确认两段路径没有直接 `data` 依赖,只在后面汇合 | 先明确汇合点,再决定是否要 event 或 wait | [npu_stream_switch](https://www.hiascend.com/document/detail/zh/Pytorch/720/modthirdparty/torchairuseguide/torchair_00101.html) | | ||
| 18 | +| 需要显式记录和等待事件 | `record_event()` / `wait_event()`;或 `npu_record_tagged_stream` / `npu_tagged_event_wait` | 两条流之间存在控制依赖,但后继不直接吃前驱输出 tensor | 图模式里优先用 tagged event 风格;eager/patch 里优先用 stream event | [npu_record_tagged_stream](https://www.hiascend.com/document/detail/zh/Pytorch/720/modthirdparty/torchairuseguide/torchair_00099.html), [npu_tagged_event_wait](https://www.hiascend.com/document/detail/zh/Pytorch/730/modthirdparty/torchairuseguide/torchair_00108.html) | | ||
| 19 | +| 图内表达 tensor 等待关系 | `npu_wait_tensor` | 需要把依赖表达在图内,而不是只在 Python 侧做 wait | 优先用于 TorchAir 图模式,不要硬套到显式 stream 风格 | [npu_wait_tensor](https://www.hiascend.com/document/detail/zh/Pytorch/730/modthirdparty/torchairuseguide/torchair_00114.html) | | ||
| 20 | +| overlap 已成立但一条流明显拖尾 | `limit_core_num` | 已经看到两条流资源争抢,或一条流长期占满 Core | 先确认拖尾来自资源争抢,而不是依赖或 shape 问题 | [limit_core_num](https://www.hiascend.com/document/detail/zh/Pytorch/710/modthirdparty/torchairuseguide/torchair_00091.html) | | ||
| 21 | +| 需要进一步查看或设置 stream 资源限制 | `torch_npu.get_stream_limit` / `torch_npu.set_stream_limit` | 已进入控核或 stream 资源调优阶段 | 这不是第一手多流 API,通常在资源调优阶段再用 | [torch_npu.get_stream_limit](https://www.hiascend.com/document/detail/zh/Pytorch/720/apiref/torchnpuCustomsapi/context/torch_npu-get_stream_limit.md), [torch_npu.set_stream_limit](https://www.hiascend.com/document/detail/zh/Pytorch/730/apiref/torchnpuCustomsapi/docs/context/torch_npu-set_stream_limit.md) | | ||
| 22 | +| 需要扩大计算窗口,掩盖权重搬运 | `torch_npu.npu_prefetch` | overlap 正确,但仍有访存或带宽空洞可被前序轻算子掩盖 | 只在前序算子不明显抢带宽时使用;常和多流 + 控核联动 | [torch_npu.npu_prefetch](https://www.hiascend.com/document/detail/zh/Pytorch/700/apiref/apilist/ptaoplist_000530.html) | | ||
| 23 | + | ||
| 24 | +## 推荐决策顺序 | ||
| 25 | + | ||
| 26 | +1. 先确定当前是 eager / patch 还是 graph / TorchAir。 | ||
| 27 | +2. 先选一套主 API 路径,不要混着写。 | ||
| 28 | +3. 先把依赖和同步做对,再确认是否真的有 overlap。 | ||
| 29 | +4. 只有在 overlap 正确但拖尾明显时,才进入控核、stream limit、预取调优。 | ||
| 30 | + | ||
| 31 | +## 常见误区 | ||
| 32 | + | ||
| 33 | +- 不要在 eager 路径里照搬 TorchAir 的 tagged event 风格。 | ||
| 34 | +- 不要把 `limit_core_num` 当成默认步骤;它只解决资源分配问题,不解决依赖错误。 | ||
| 35 | +- 不要用 `npu_prefetch` 掩盖一个本来就不该并行的链路;先证明链路没有错误依赖。 | ||
| 36 | +- `npu_tagged_event_record` 的独立官方页面不稳定时,优先看总说明页和仓库案例代码,不要自己猜语义。 | ||
| @@ -0,0 +1,32 @@ | |||
| 1 | +# 多流案例路由 | ||
| 2 | + | ||
| 3 | +本文件按**优化模式**路由仓库内多流案例。 | ||
| 4 | + | ||
| 5 | +> 下表"参考源码"列为 `cann-recipes-infer` 模型仓中的代表实现路径(相对本仓库为外部路径, | ||
| 6 | +> 仅作定位参考,不作为可点击链接)。 | ||
| 7 | + | ||
| 8 | +## 快速选型表 | ||
| 9 | + | ||
| 10 | +| 优化模式 | 参考源码 | 什么时候优先选 | 典型风险 | | ||
| 11 | +| --- | --- | --- | --- | | ||
| 12 | +| MoE shared expert 双流 | `models/deepseek-v3.2-exp/models/modeling_deepseek.py`、`models/glm-5/models/modeling_glm.py` | 路由专家和共享专家结果在后面汇合,且 decode shape 稳定 | 同步点放错会导致 merge 读到未完成结果;共享专家过重时 overlap 不一定有收益 | | ||
| 13 | +| Indexer / Prolog 多流 | `models/glm-5/models/indexer.py` | Attention 前处理链路可拆成两段或多段,等待在后面汇合 | 这类优化常常是前处理子链 overlap,不是完整大模块并行,边界容易拆错 | | ||
| 14 | +| KVCache offload 异步流 | `models/deepseek-v3.2-exp/models/offload_cache.py`、`models/glm-5/models/offload_cache.py` | 设备内存紧张,需要把搬运从主计算流剥离 | 主流和搬运流的状态一致性最重要;异步搬运可能掩盖不了 H2D/D2H 带宽瓶颈 | | ||
| 15 | +| Prefill micro-batch 双流 + event | `models/deepseek-r1/models/modeling_deepseek.py` | prefill 同时有明显计算和通信,且切 micro-batch 后 shape 线性度还可以 | 最容易引入 host bound、shape 劣化和事件编排错误 | | ||
| 16 | +| 多流 + 控核 | `models/longcat-flash/models/modeling_longcat_flash.py` | 已有 overlap,但一条流明显拖尾或资源被另一条流吃满 | 控核值不是通用常量;多流、控核、预取和图模式往往是耦合设计 | | ||
| 17 | +| AFD 通信/计算 overlap | 以分离式部署链路为主 | 分离式部署或通信链路成了主要瓶颈 | 重点不在本地双算子并行,而在通信和本地计算 overlap;等待链容易拖尾 | | ||
| 18 | +| Patch 形态多流 | `models/qwen3-next/patches/stage1/0003-feat-moe-multi-stream.patch` | 优化不能直接落到模型仓,需要嵌进 runtime 或 patch | patch 代码更依赖现有生命周期,`wait_stream()` 顺序错了会直接破坏运行时逻辑 | | ||
| 19 | +| 多模态变体 | `models/hunyuan-image-3.0/adaptor_patches/hunyuan.py` | 共享流在模块初始化阶段就作为能力注入,而不是在前向里临时加 scope | 多流能力进入模块构造期后,不能只复制一段前向代码;初始化和分布式上下文要一起看 | | ||
| 20 | + | ||
| 21 | +## 使用顺序 | ||
| 22 | + | ||
| 23 | +1. 先确定当前优化属于哪一类模式。 | ||
| 24 | +2. 读对应代表代码或补丁,确认模块边界、依赖关系、实际 API 风格和 enable 开关设计。 | ||
| 25 | +3. 如果一个实现同时落在多类模式里,先选主模式,再把其他能力当补充手段。 | ||
| 26 | + | ||
| 27 | +## 常见误用 | ||
| 28 | + | ||
| 29 | +- 不要把 `MoE shared expert 双流` 和 `Prefill micro-batch 双流` 当成同一种流水,它们的同步粒度完全不同。 | ||
| 30 | +- 不要看到有两条流就默认需要 `limit_core_num`,控核只在资源争抢和拖尾明显时再引入。 | ||
| 31 | +- 不要把 `KVCache offload` 这类搬运流当成计算流优化,它优先关注的是状态一致性和带宽掩盖。 | ||
| 32 | +- 不要直接拼接多个案例的代码片段,必须先按当前执行模式选一套主路径。 | ||
| @@ -0,0 +1,355 @@ | |||
| 1 | +# AI Agent 整网模块拆解规范 | ||
| 2 | + | ||
| 3 | +本文用于指导 AI agent 分析多流优化。流程分两层: | ||
| 4 | + | ||
| 5 | +1. **整网模块拆解** | ||
| 6 | + 目标:把整网拆成模块,画出模块级 DAG,判断模块与模块哪些能并行。 | ||
| 7 | +2. **每个模块的算子级拆解** | ||
| 8 | + 目标:对每个模块继续拆成算子,画出模块内算子 DAG,判断模块内哪些算子能并行。 | ||
| 9 | + | ||
| 10 | +当前文档同时约束这两层,但第一层优先。 | ||
| 11 | + | ||
| 12 | +## 总目标 | ||
| 13 | + | ||
| 14 | +agent 必须完成下面 4 件事: | ||
| 15 | + | ||
| 16 | +- 把整网拆成一组模块 | ||
| 17 | +- 画出整网模块 DAG | ||
| 18 | +- 判断模块与模块哪些能并行 | ||
| 19 | +- 对每个模块继续拆成算子并判断模块内算子并行性 | ||
| 20 | + | ||
| 21 | +## 第一层输出物 | ||
| 22 | + | ||
| 23 | +第一层分析完成后,必须产出: | ||
| 24 | + | ||
| 25 | +1. 整网模块清单 | ||
| 26 | +2. 整网模块依赖清单 | ||
| 27 | +3. 整网模块 Mermaid DAG | ||
| 28 | +4. 模块级并行性结论 | ||
| 29 | + | ||
| 30 | +第二层分析完成后,必须补充: | ||
| 31 | + | ||
| 32 | +1. 每个模块的算子清单 | ||
| 33 | +2. 每个模块的算子级依赖清单 | ||
| 34 | +3. 每个模块的算子级 Mermaid DAG | ||
| 35 | +4. 每个模块内部的算子并行性结论 | ||
| 36 | + | ||
| 37 | +## 固定产物地址与模板 | ||
| 38 | + | ||
| 39 | +分析结果统一输出为一个 Markdown 文件,放在: | ||
| 40 | + | ||
| 41 | +`docs/common/multi-stream-analysis/<network_or_case_name>.md` | ||
| 42 | + | ||
| 43 | +命名规则: | ||
| 44 | + | ||
| 45 | +- 使用小写英文、短横线连接 | ||
| 46 | +- 名字描述“整网分析对象”,不要只写某个局部函数名 | ||
| 47 | +- 示例: | ||
| 48 | + - `docs/common/multi-stream-analysis/deepseek-r1-decode.md` | ||
| 49 | + - `docs/common/multi-stream-analysis/longcat-flash-decode.md` | ||
| 50 | + - `docs/common/multi-stream-analysis/hunyuanimage3-moe-path.md` | ||
| 51 | + | ||
| 52 | +模板文件固定放在: | ||
| 53 | + | ||
| 54 | +`./analysis-template.md` | ||
| 55 | + | ||
| 56 | +agent 输出时必须: | ||
| 57 | + | ||
| 58 | +1. 基于该模板生成结果 | ||
| 59 | +2. 保留模板的一级、二级标题结构 | ||
| 60 | +3. 先写整网模块分析,再写每个模块的算子分析 | ||
| 61 | +4. Mermaid 图直接内嵌在结果文件中 | ||
| 62 | + | ||
| 63 | +## 一、整网模块拆解规则 | ||
| 64 | + | ||
| 65 | +### 1. 拆的是整网,不是局部函数 | ||
| 66 | + | ||
| 67 | +第一步必须从**整网执行路径**出发,而不是直接抓某个局部热点函数。 | ||
| 68 | + | ||
| 69 | +agent 要先回答这 3 个问题: | ||
| 70 | + | ||
| 71 | +- 整网主路径从哪里开始,到哪里结束 | ||
| 72 | +- 当前分析的是 `prefill` 还是 `decode` | ||
| 73 | +- 整网里有哪些一级模块 | ||
| 74 | + | ||
| 75 | +如果这一步还没完成,就不能直接进入某个局部模块内部。 | ||
| 76 | + | ||
| 77 | +### 2. 一级模块定义 | ||
| 78 | + | ||
| 79 | +一级模块必须同时满足下面 4 条: | ||
| 80 | + | ||
| 81 | +1. 语义完整 | ||
| 82 | + 例如 `Embedding`、`Attention Main Path`、`Router Path`、`Shared Expert`、`Dispatch`、`Expert Compute`、`Combine`、`KVCache Offload`、`LM Head` | ||
| 83 | + | ||
| 84 | +2. 输入输出明确 | ||
| 85 | + 能说清该模块吃什么、产什么、是否写共享状态 | ||
| 86 | + | ||
| 87 | +3. 可独立讨论调度 | ||
| 88 | + 可以单独判断“是否可能和别的模块并行” | ||
| 89 | + | ||
| 90 | +4. 足够大 | ||
| 91 | + 不能只是局部 reshape、cast、view 或某个融合算子内部的小步骤 | ||
| 92 | + | ||
| 93 | +### 3. 一级模块优先沿这些边界拆 | ||
| 94 | + | ||
| 95 | +- 子网络边界:`Attention`、`MoE`、`MLP`、`LM Head` | ||
| 96 | +- 通信边界:`all_to_all`、`send/recv`、`all_gather`、`reduce_scatter` | ||
| 97 | +- 状态边界:`KVCache update`、`offload`、`reload` | ||
| 98 | +- 同步边界:`record_event / wait_event`、`wait_stream` | ||
| 99 | +- 资源边界:明显偏计算、偏通信、偏搬运的切换点 | ||
| 100 | + | ||
| 101 | +### 4. 第一层不要过早拆成算子 | ||
| 102 | + | ||
| 103 | +第一层不要把下面内容单独当模块: | ||
| 104 | + | ||
| 105 | +- reshape / cast / view | ||
| 106 | +- 融合算子内部步骤 | ||
| 107 | +- 没有调度意义的局部前后处理 | ||
| 108 | + | ||
| 109 | +第一层的目标只是把整网骨架立起来,并回答模块与模块之间的并行关系。 | ||
| 110 | + | ||
| 111 | +## 二、整网模块 DAG 规则 | ||
| 112 | + | ||
| 113 | +### 1. 节点 | ||
| 114 | + | ||
| 115 | +- 一个节点对应一个一级模块 | ||
| 116 | +- 节点名要体现模块语义,不要用代码行名 | ||
| 117 | +- 汇合点可以单独画成控制节点,例如 `merge_shared_router` | ||
| 118 | + | ||
| 119 | +### 2. 边类型 | ||
| 120 | + | ||
| 121 | +每条边必须标注依赖类型,只允许下面 4 类: | ||
| 122 | + | ||
| 123 | +- `data` | ||
| 124 | + 后继模块直接消费前驱模块输出 | ||
| 125 | +- `state` | ||
| 126 | + 后继模块依赖前驱写完 cache、buffer 或共享状态 | ||
| 127 | +- `event` | ||
| 128 | + 后继模块依赖前驱的事件、同步或 stream wait | ||
| 129 | +- `collective_order` | ||
| 130 | + 通信顺序固定,不能随意重排 | ||
| 131 | + | ||
| 132 | +### 3. 共同输入不是依赖 | ||
| 133 | + | ||
| 134 | +如果两个模块都使用同一个输入,例如 `hidden_states` 同时进入 `router_path` 和 `shared_expert`,不要把它们之间画成依赖边,而是补一个共同上游节点。 | ||
| 135 | + | ||
| 136 | +正确画法: | ||
| 137 | + | ||
| 138 | +```mermaid | ||
| 139 | +flowchart LR | ||
| 140 | + X["hidden_states"] | ||
| 141 | + A["router_path"] | ||
| 142 | + B["shared_expert"] | ||
| 143 | + M["merge"] | ||
| 144 | + | ||
| 145 | + X --> A | ||
| 146 | + X --> B | ||
| 147 | + A --> M | ||
| 148 | + B --> M | ||
| 149 | +``` | ||
| 150 | + | ||
| 151 | +### 4. Mermaid 画法 | ||
| 152 | + | ||
| 153 | +- 统一使用 `flowchart LR` | ||
| 154 | +- 如果还没决定流归属,用 `Main Path / Side Path / Comm Path` | ||
| 155 | +- 如果代码里已经明确是 `Stream0 / Stream1`,才用流名做 `subgraph` | ||
| 156 | + | ||
| 157 | +边规则: | ||
| 158 | + | ||
| 159 | +- `-->`:`data` 或 `state` | ||
| 160 | +- `-.->`:`event` | ||
| 161 | + | ||
| 162 | +## 三、模块级并行性判断 | ||
| 163 | + | ||
| 164 | +### 判为 `serial` | ||
| 165 | + | ||
| 166 | +满足任一条件就必须串行: | ||
| 167 | + | ||
| 168 | +- 存在 `data` 依赖 | ||
| 169 | +- 存在共享可写状态冲突 | ||
| 170 | +- 通信顺序固定,不能插入 | ||
| 171 | +- 其中一个只是另一个内部步骤 | ||
| 172 | + | ||
| 173 | +### 判为 `parallel_candidate` | ||
| 174 | + | ||
| 175 | +同时满足下面条件时,可以判为模块级并行候选: | ||
| 176 | + | ||
| 177 | +- 没有 `data` 依赖 | ||
| 178 | +- 没有明确共享写冲突 | ||
| 179 | +- 两者结果在后面的汇合点才相遇 | ||
| 180 | +- 通信顺序没有把它们绑定成单链条 | ||
| 181 | + | ||
| 182 | +### 判为 `parallel_pending_validation` | ||
| 183 | + | ||
| 184 | +逻辑上可并行,但下面因素还没确认: | ||
| 185 | + | ||
| 186 | +- 资源是否冲突 | ||
| 187 | +- shape 是否太小 | ||
| 188 | +- 图模式或 runtime 是否有限制 | ||
| 189 | +- 是否会引入额外 clone / buffer / host 开销 | ||
| 190 | + | ||
| 191 | +## 四、算子级拆解规则 | ||
| 192 | + | ||
| 193 | +第二层不是只分析部分模块,而是**每个模块都要继续拆成算子**。 | ||
| 194 | + | ||
| 195 | +### 1. 算子级拆解目标 | ||
| 196 | + | ||
| 197 | +对每个模块都要回答: | ||
| 198 | + | ||
| 199 | +- 模块内部有哪些关键算子或算子组 | ||
| 200 | +- 算子与算子之间的 DAG 是什么 | ||
| 201 | +- 模块内部哪些算子能并行,哪些必须串行 | ||
| 202 | + | ||
| 203 | +### 2. 算子级拆解原则 | ||
| 204 | + | ||
| 205 | +- 仍然优先沿计算、通信、同步、状态边界拆 | ||
| 206 | +- 一个算子节点可以是单算子,也可以是没有必要继续拆的算子组 | ||
| 207 | +- 模块内的共同输入、汇合点、同步点仍然要单独标清 | ||
| 208 | + | ||
| 209 | +### 3. 算子级输出 | ||
| 210 | + | ||
| 211 | +每个模块都要补齐: | ||
| 212 | + | ||
| 213 | +- 模块内算子清单 | ||
| 214 | +- 算子级依赖 | ||
| 215 | +- 算子级 DAG | ||
| 216 | +- 模块内算子并行性结论 | ||
| 217 | + | ||
| 218 | +## 五、推荐输出格式 | ||
| 219 | + | ||
| 220 | +### 1. 整网模块清单 | ||
| 221 | + | ||
| 222 | +| module_id | module_name | module_type | inputs | outputs | side_effect | resource_hint | | ||
| 223 | +| --- | --- | --- | --- | --- | --- | --- | | ||
| 224 | + | ||
| 225 | +### 2. 整网模块依赖清单 | ||
| 226 | + | ||
| 227 | +| from | to | dependency_type | reason | | ||
| 228 | +| --- | --- | --- | --- | | ||
| 229 | + | ||
| 230 | +### 3. 整网模块 DAG | ||
| 231 | + | ||
| 232 | +直接输出 Mermaid。 | ||
| 233 | + | ||
| 234 | +### 4. 模块级并行性结论 | ||
| 235 | + | ||
| 236 | +模块级并行性结论**不要**写成 `module_a / module_b` 的二元表,因为整网分析往往涉及: | ||
| 237 | + | ||
| 238 | +- 多个模块同时并行 | ||
| 239 | +- 一个流里串行一组模块,另一个流里并行另一组模块 | ||
| 240 | +- 多个分叉点和多个汇合点 | ||
| 241 | + | ||
| 242 | +因此,模块级并行性结论必须直接按“分组和结构”描述,至少包含下面 4 部分: | ||
| 243 | + | ||
| 244 | +1. `主串行链` | ||
| 245 | + 说明哪些模块构成当前主路径,不能打断。 | ||
| 246 | + | ||
| 247 | +2. `可并行模块组` | ||
| 248 | + 说明哪些模块可以作为一组与主路径或其他组并行。 | ||
| 249 | + | ||
| 250 | +3. `待验证模块组` | ||
| 251 | + 说明逻辑可并行,但资源或运行时约束还要验证的模块组。 | ||
| 252 | + | ||
| 253 | +4. `建议流分组` | ||
| 254 | + 用 `Stream0 / Stream1 / Stream2` 或 `Main Path / Side Path / Comm Path` 的方式描述推荐分组,不要求现在就和代码中的真实流一一对应。 | ||
| 255 | + | ||
| 256 | +推荐写法示例: | ||
| 257 | + | ||
| 258 | +- 主串行链:`embedding -> attention_main -> merge -> lm_head` | ||
| 259 | +- 可并行模块组: | ||
| 260 | + - 组 A:`router_path -> dispatch -> combine` | ||
| 261 | + - 组 B:`shared_expert` | ||
| 262 | +- 待验证模块组: | ||
| 263 | + - 组 C:`kvcache_reload -> indexer_prolog` | ||
| 264 | +- 建议流分组: | ||
| 265 | + - `Stream0`:`attention_main -> merge -> lm_head` | ||
| 266 | + - `Stream1`:`shared_expert` | ||
| 267 | + - `Stream2`:`router_path -> dispatch -> combine` | ||
| 268 | + | ||
| 269 | +### 5. 每个模块的算子级结果 | ||
| 270 | + | ||
| 271 | +对每个模块都要补一组结果: | ||
| 272 | + | ||
| 273 | +- 算子清单 | ||
| 274 | +- 算子依赖清单 | ||
| 275 | +- 算子级 Mermaid DAG | ||
| 276 | +- 模块内算子并行性结论 | ||
| 277 | + | ||
| 278 | +算子级并行性结论也**不要**写成 `op_a / op_b` 的二元表。原因和模块级相同:模块内部同样可能是多组算子并行、多个汇合点、多个同步点。 | ||
| 279 | + | ||
| 280 | +因此,每个模块内部的算子并行性结论也必须按“分组和结构”描述,至少包含: | ||
| 281 | + | ||
| 282 | +1. `模块内主串行链` | ||
| 283 | +2. `模块内可并行算子组` | ||
| 284 | +3. `模块内待验证算子组` | ||
| 285 | +4. `模块内建议流分组` | ||
| 286 | + | ||
| 287 | +推荐写法示例: | ||
| 288 | + | ||
| 289 | +- 模块内主串行链:`qkv_prepare -> attention_score -> merge_out` | ||
| 290 | +- 模块内可并行算子组: | ||
| 291 | + - 组 A:`shared_expert_gate -> shared_expert_down_proj` | ||
| 292 | + - 组 B:`router_topk -> dispatch` | ||
| 293 | +- 模块内待验证算子组: | ||
| 294 | + - 组 C:`kvcache_reload -> indexer_prolog` | ||
| 295 | +- 模块内建议流分组: | ||
| 296 | + - `Stream0`:`qkv_prepare -> attention_score -> merge_out` | ||
| 297 | + - `Stream1`:`shared_expert_gate -> shared_expert_down_proj` | ||
| 298 | + - `Stream2`:`router_topk -> dispatch` | ||
| 299 | + | ||
| 300 | +实际写文件时,必须按 `./analysis-template.md` 的结构落盘。 | ||
| 301 | + | ||
| 302 | +## 六、最小示例 | ||
| 303 | + | ||
| 304 | +下面示例只演示第一层:如何把整网中的一段 MoE 路径整理成模块级局部骨架。 | ||
| 305 | + | ||
| 306 | +### 模块清单 | ||
| 307 | + | ||
| 308 | +| module_id | module_name | module_type | inputs | outputs | side_effect | resource_hint | | ||
| 309 | +| --- | --- | --- | --- | --- | --- | --- | | ||
| 310 | +| router_path | Router Path | compute | hidden_states | routed_hidden_states | 无 | compute + comm | | ||
| 311 | +| shared_expert | Shared Expert | compute | hidden_states | shared_hidden_states | 无 | compute | | ||
| 312 | +| merge_shared_router | Merge Shared Router | control | routed_hidden_states, shared_hidden_states | hidden_states | 无 | light compute | | ||
| 313 | + | ||
| 314 | +### 依赖清单 | ||
| 315 | + | ||
| 316 | +| from | to | dependency_type | reason | | ||
| 317 | +| --- | --- | --- | --- | | ||
| 318 | +| router_path | merge_shared_router | data | merge 需要 routed_hidden_states | | ||
| 319 | +| shared_expert | merge_shared_router | data | merge 需要 shared_hidden_states | | ||
| 320 | + | ||
| 321 | +### Mermaid DAG | ||
| 322 | + | ||
| 323 | +```mermaid | ||
| 324 | +flowchart LR | ||
| 325 | + X["hidden_states"] | ||
| 326 | + | ||
| 327 | + subgraph P0["Main Path"] | ||
| 328 | + A["router_path"] | ||
| 329 | + M["merge_shared_router"] | ||
| 330 | + A --> M | ||
| 331 | + end | ||
| 332 | + | ||
| 333 | + subgraph P1["Side Path"] | ||
| 334 | + B["shared_expert"] | ||
| 335 | + end | ||
| 336 | + | ||
| 337 | + X --> A | ||
| 338 | + X --> B | ||
| 339 | + B --> M | ||
| 340 | +``` | ||
| 341 | + | ||
| 342 | +### 模块级并行性结论 | ||
| 343 | + | ||
| 344 | +- 主串行链:`router_path -> merge_shared_router` | ||
| 345 | +- 可并行模块组: | ||
| 346 | + - 组 A:`shared_expert` | ||
| 347 | +- 待验证模块组: | ||
| 348 | + - 无 | ||
| 349 | +- 建议流分组: | ||
| 350 | + - `Main Path`:`router_path -> merge_shared_router` | ||
| 351 | + - `Side Path`:`shared_expert` | ||
| 352 | + | ||
| 353 | +## 关键词 | ||
| 354 | + | ||
| 355 | +`whole-graph decomposition` `module dag` `module parallelism` `operator dag` | ||
| @@ -0,0 +1,120 @@ | |||
| 1 | +# 多流与控核官方文档索引(最新可检索版本) | ||
| 2 | + | ||
| 3 | +本文整理当前官方站点中与多流、控核以及本仓库案例中实际使用到的相关 API 对应的**最新可检索版本**文档。 | ||
| 4 | + | ||
| 5 | +说明: | ||
| 6 | + | ||
| 7 | +- 这里的“最新”指当前官方站点可稳定检索到的最新页面版本。 | ||
| 8 | +- 不同 API 的最新页面版本不完全一致,不能强行统一到同一个版本号。 | ||
| 9 | +- 如果某个 API 没有检索到更高版本的独立页面,就保留当前能确认的最新官方页面。 | ||
| 10 | + | ||
| 11 | +## 1. 总说明文档 | ||
| 12 | + | ||
| 13 | +### 多流总说明 | ||
| 14 | + | ||
| 15 | +- [图内多流表达功能(Ascend IR)- 7.3.0](https://www.hiascend.com/document/detail/zh/Pytorch/730/modthirdparty/torchairuseguide/torchair_00033.html) | ||
| 16 | +- [图内多流表达功能(aclgraph)- 7.3.0](https://www.hiascend.com/document/detail/zh/Pytorch/730/modthirdparty/torchairuseguide/torchair_00024.html) | ||
| 17 | + | ||
| 18 | +### 图模式入口 | ||
| 19 | + | ||
| 20 | +- [PyTorch 图模式使用(TorchAir)入口 - 7.3.0](https://www.hiascend.com/document/detail/zh/Pytorch/730/index/index.html) | ||
| 21 | + | ||
| 22 | +## 2. 多流相关 API | ||
| 23 | + | ||
| 24 | +### TorchAir scope / ops | ||
| 25 | + | ||
| 26 | +- `torchair.scope.npu_stream_switch` | ||
| 27 | + - [npu_stream_switch - 7.2.0](https://www.hiascend.com/document/detail/zh/Pytorch/720/modthirdparty/torchairuseguide/torchair_00101.html) | ||
| 28 | + | ||
| 29 | +- `torchair.scope.npu_wait_tensor` | ||
| 30 | + - [npu_wait_tensor - 7.3.0](https://www.hiascend.com/document/detail/zh/Pytorch/730/modthirdparty/torchairuseguide/torchair_00114.html) | ||
| 31 | + | ||
| 32 | +- `torchair.ops.npu_record_tagged_stream` | ||
| 33 | + - [npu_record_tagged_stream - 7.2.0](https://www.hiascend.com/document/detail/zh/Pytorch/720/modthirdparty/torchairuseguide/torchair_00099.html) | ||
| 34 | + | ||
| 35 | +- `torchair.ops.npu_tagged_event_wait` | ||
| 36 | + - [npu_tagged_event_wait - 7.3.0](https://www.hiascend.com/document/detail/zh/Pytorch/730/modthirdparty/torchairuseguide/torchair_00108.html) | ||
| 37 | + | ||
| 38 | +- `torchair.ops.wait` | ||
| 39 | + - [wait - 7.3.0](https://www.hiascend.com/document/detail/zh/Pytorch/730/modthirdparty/torchairuseguide/torchair_00111.html) | ||
| 40 | + | ||
| 41 | +### 说明 | ||
| 42 | + | ||
| 43 | +- `npu_tagged_event_record` 这次整理中没有稳定检索到比总说明页更适合直接引用的更高版本独立页面。 | ||
| 44 | +- 如果后续仓库文档要引用该能力,建议优先引用“图内多流表达功能”总说明,再辅以本仓库案例代码。 | ||
| 45 | + | ||
| 46 | +## 3. 控核相关 API | ||
| 47 | + | ||
| 48 | +### TorchAir 控核接口 | ||
| 49 | + | ||
| 50 | +- `torchair.scope.limit_core_num` | ||
| 51 | + - [limit_core_num - 7.1.0](https://www.hiascend.com/document/detail/zh/Pytorch/710/modthirdparty/torchairuseguide/torchair_00091.html) | ||
| 52 | + | ||
| 53 | +### torch_npu stream 资源限制接口 | ||
| 54 | + | ||
| 55 | +- `torch_npu.set_stream_limit` | ||
| 56 | + - [torch_npu.set_stream_limit - 7.3.0](https://www.hiascend.com/document/detail/zh/Pytorch/730/apiref/torchnpuCustomsapi/docs/context/torch_npu-set_stream_limit.md) | ||
| 57 | + | ||
| 58 | +- `torch_npu.get_stream_limit` | ||
| 59 | + - [torch_npu.get_stream_limit - 7.2.0](https://www.hiascend.com/document/detail/zh/Pytorch/720/apiref/torchnpuCustomsapi/context/torch_npu-get_stream_limit.md) | ||
| 60 | + | ||
| 61 | +## 4. 与案例联动的补充 API | ||
| 62 | + | ||
| 63 | +### 预取 | ||
| 64 | + | ||
| 65 | +- `torch_npu.npu_prefetch` | ||
| 66 | + - [torch_npu.npu_prefetch - 7.0.0](https://www.hiascend.com/document/detail/zh/Pytorch/700/apiref/apilist/ptaoplist_000530.html) | ||
| 67 | + | ||
| 68 | +说明: | ||
| 69 | + | ||
| 70 | +- `npu_prefetch` 不是多流 API,但在 `LongCat-Flash` 的“多流 + 控核 + 预取”联动里是关键补充能力。 | ||
| 71 | + | ||
| 72 | +## 5. 与仓库案例的对应关系 | ||
| 73 | + | ||
| 74 | +### DeepSeek-V3.2-Exp / DeepSeek-R1 / Kimi-K2 / GLM-5 | ||
| 75 | + | ||
| 76 | +重点参考: | ||
| 77 | + | ||
| 78 | +- [图内多流表达功能(Ascend IR)- 7.3.0](https://www.hiascend.com/document/detail/zh/Pytorch/730/modthirdparty/torchairuseguide/torchair_00033.html) | ||
| 79 | +- [npu_stream_switch - 7.2.0](https://www.hiascend.com/document/detail/zh/Pytorch/720/modthirdparty/torchairuseguide/torchair_00101.html) | ||
| 80 | +- [npu_wait_tensor - 7.3.0](https://www.hiascend.com/document/detail/zh/Pytorch/730/modthirdparty/torchairuseguide/torchair_00114.html) | ||
| 81 | +- [npu_record_tagged_stream - 7.2.0](https://www.hiascend.com/document/detail/zh/Pytorch/720/modthirdparty/torchairuseguide/torchair_00099.html) | ||
| 82 | +- [npu_tagged_event_wait - 7.3.0](https://www.hiascend.com/document/detail/zh/Pytorch/730/modthirdparty/torchairuseguide/torchair_00108.html) | ||
| 83 | + | ||
| 84 | +### LongCat-Flash | ||
| 85 | + | ||
| 86 | +重点参考: | ||
| 87 | + | ||
| 88 | +- [图内多流表达功能(Ascend IR)- 7.3.0](https://www.hiascend.com/document/detail/zh/Pytorch/730/modthirdparty/torchairuseguide/torchair_00033.html) | ||
| 89 | +- [limit_core_num - 7.1.0](https://www.hiascend.com/document/detail/zh/Pytorch/710/modthirdparty/torchairuseguide/torchair_00091.html) | ||
| 90 | +- [torch_npu.set_stream_limit - 7.3.0](https://www.hiascend.com/document/detail/zh/Pytorch/730/apiref/torchnpuCustomsapi/docs/context/torch_npu-set_stream_limit.md) | ||
| 91 | +- [torch_npu.get_stream_limit - 7.2.0](https://www.hiascend.com/document/detail/zh/Pytorch/720/apiref/torchnpuCustomsapi/context/torch_npu-get_stream_limit.md) | ||
| 92 | +- [torch_npu.npu_prefetch - 7.0.0](https://www.hiascend.com/document/detail/zh/Pytorch/700/apiref/apilist/ptaoplist_000530.html) | ||
| 93 | + | ||
| 94 | +### Prefill Micro-Batch 双流流水 | ||
| 95 | + | ||
| 96 | +重点参考: | ||
| 97 | + | ||
| 98 | +- [图内多流表达功能(Ascend IR)- 7.3.0](https://www.hiascend.com/document/detail/zh/Pytorch/730/modthirdparty/torchairuseguide/torchair_00033.html) | ||
| 99 | +- [图内多流表达功能(aclgraph)- 7.3.0](https://www.hiascend.com/document/detail/zh/Pytorch/730/modthirdparty/torchairuseguide/torchair_00024.html) | ||
| 100 | + | ||
| 101 | +## 6. 版本结论 | ||
| 102 | + | ||
| 103 | +当前这批能力里,能确认到的最新可检索版本大致如下: | ||
| 104 | + | ||
| 105 | +- 优先使用 `7.3.0` | ||
| 106 | + - 图内多流表达功能 | ||
| 107 | + - aclgraph 多流表达功能 | ||
| 108 | + - `npu_wait_tensor` | ||
| 109 | + - `npu_tagged_event_wait` | ||
| 110 | + - `torch_npu.set_stream_limit` | ||
| 111 | + - TorchAir 图模式总入口 | ||
| 112 | + | ||
| 113 | +- 当前检索到的最新页面不是 `7.3.0` | ||
| 114 | + - `npu_stream_switch`:7.2.0 | ||
| 115 | + - `npu_record_tagged_stream`:7.2.0 | ||
| 116 | + - `torch_npu.get_stream_limit`:7.2.0 | ||
| 117 | + - `limit_core_num`:7.1.0 | ||
| 118 | + - `torch_npu.npu_prefetch`:7.0.0 | ||
| 119 | + | ||
| 120 | +因此,在仓库文档中应采用“**每个接口分别取最新页面**”的策略,而不是统一写成同一个版本号。 | ||
| @@ -0,0 +1,191 @@ | |||
| 1 | +# <network_or_case_name> 多流优化报告 | ||
| 2 | + | ||
| 3 | +> 结果文件固定放在 `docs/common/multi-stream-analysis/<network_or_case_name>.md`。 | ||
| 4 | +> 如果任务当前只做到分析,开发、调试、验收部分可先写 `N/A`,但章节必须保留。 | ||
| 5 | + | ||
| 6 | +## 1. 分析范围 | ||
| 7 | + | ||
| 8 | +- 模型/网络: | ||
| 9 | +- 阶段:`prefill` / `decode` | ||
| 10 | +- 分析对象: | ||
| 11 | +- 代码入口: | ||
| 12 | +- 当前执行模式:`eager` / `patch` / `ge_graph` / `aclgraph` / 其他 | ||
| 13 | +- 说明: | ||
| 14 | + | ||
| 15 | +## 2. 目标优化点 | ||
| 16 | + | ||
| 17 | +- 当前希望获得的 overlap: | ||
| 18 | +- 预期并行对象: | ||
| 19 | +- 预期收益: | ||
| 20 | +- 明确不在本轮处理的范围: | ||
| 21 | + | ||
| 22 | +## 3. 整网模块清单 | ||
| 23 | + | ||
| 24 | +| module_id | module_name | module_type | inputs | outputs | side_effect | resource_hint | | ||
| 25 | +| --- | --- | --- | --- | --- | --- | --- | | ||
| 26 | + | ||
| 27 | +## 4. 整网模块依赖清单 | ||
| 28 | + | ||
| 29 | +| from | to | dependency_type | reason | | ||
| 30 | +| --- | --- | --- | --- | | ||
| 31 | + | ||
| 32 | +## 5. 整网模块 DAG | ||
| 33 | + | ||
| 34 | +```mermaid | ||
| 35 | +flowchart LR | ||
| 36 | +``` | ||
| 37 | + | ||
| 38 | +## 6. 模块级并行性结论 | ||
| 39 | + | ||
| 40 | +### 6.1 主串行链 | ||
| 41 | + | ||
| 42 | +- | ||
| 43 | + | ||
| 44 | +### 6.2 可并行模块组 | ||
| 45 | + | ||
| 46 | +- 组 A: | ||
| 47 | +- 组 B: | ||
| 48 | + | ||
| 49 | +### 6.3 待验证模块组 | ||
| 50 | + | ||
| 51 | +- | ||
| 52 | + | ||
| 53 | +### 6.4 建议流分组 | ||
| 54 | + | ||
| 55 | +- `Stream0` / `Main Path`: | ||
| 56 | +- `Stream1` / `Side Path`: | ||
| 57 | +- `Stream2` / `Comm Path`: | ||
| 58 | + | ||
| 59 | +## 7. 模块拆解结论 | ||
| 60 | + | ||
| 61 | +- 哪些模块必须串行: | ||
| 62 | +- 哪些模块可以并行: | ||
| 63 | +- 哪些模块并行性待验证: | ||
| 64 | +- 推荐优先实现的多流切入点: | ||
| 65 | + | ||
| 66 | +## 8. 每个模块的算子级拆解 | ||
| 67 | + | ||
| 68 | +> 为每个模块复制下面的块,不要只写部分模块。 | ||
| 69 | + | ||
| 70 | +### 8.x <module_name> | ||
| 71 | + | ||
| 72 | +#### 算子清单 | ||
| 73 | + | ||
| 74 | +| op_id | op_name | op_type | inputs | outputs | side_effect | resource_hint | | ||
| 75 | +| --- | --- | --- | --- | --- | --- | --- | | ||
| 76 | + | ||
| 77 | +#### 算子依赖清单 | ||
| 78 | + | ||
| 79 | +| from | to | dependency_type | reason | | ||
| 80 | +| --- | --- | --- | --- | | ||
| 81 | + | ||
| 82 | +#### 算子 DAG | ||
| 83 | + | ||
| 84 | +```mermaid | ||
| 85 | +flowchart LR | ||
| 86 | +``` | ||
| 87 | + | ||
| 88 | +#### 算子并行性结论 | ||
| 89 | + | ||
| 90 | +##### 模块内主串行链 | ||
| 91 | + | ||
| 92 | +- | ||
| 93 | + | ||
| 94 | +##### 模块内可并行算子组 | ||
| 95 | + | ||
| 96 | +- 组 A: | ||
| 97 | +- 组 B: | ||
| 98 | + | ||
| 99 | +##### 模块内待验证算子组 | ||
| 100 | + | ||
| 101 | +- | ||
| 102 | + | ||
| 103 | +##### 模块内建议流分组 | ||
| 104 | + | ||
| 105 | +- `Stream0` / `Main Path`: | ||
| 106 | +- `Stream1` / `Side Path`: | ||
| 107 | +- `Stream2` / `Comm Path`: | ||
| 108 | + | ||
| 109 | +#### 模块内结论 | ||
| 110 | + | ||
| 111 | +- 模块内必须串行的部分: | ||
| 112 | +- 模块内可以并行的部分: | ||
| 113 | +- 模块内待验证的部分: | ||
| 114 | +- 可能的流切换点 / 同步点: | ||
| 115 | + | ||
| 116 | +## 9. 实施记录 | ||
| 117 | + | ||
| 118 | +### 9.1 方案选择 | ||
| 119 | + | ||
| 120 | +- 采用的主 API 路径: | ||
| 121 | +- enable 开关: | ||
| 122 | +- 保留的回退路径: | ||
| 123 | +- 关键同步设计: | ||
| 124 | +- 是否引入控核 / stream limit / prefetch: | ||
| 125 | + | ||
| 126 | +### 9.2 关键改动 | ||
| 127 | + | ||
| 128 | +| 状态 | 内容 | 文件 | | ||
| 129 | +| --- | --- | --- | | ||
| 130 | + | ||
| 131 | +### 9.3 当前实现结论 | ||
| 132 | + | ||
| 133 | +- 当前已实现的 overlap: | ||
| 134 | +- 当前仍未覆盖的并行点: | ||
| 135 | +- 当前代码中的主要风险: | ||
| 136 | + | ||
| 137 | +## 10. 调试记录 | ||
| 138 | + | ||
| 139 | +### 10.1 依赖 / 同步问题 | ||
| 140 | + | ||
| 141 | +- | ||
| 142 | + | ||
| 143 | +### 10.2 精度 / 功能问题 | ||
| 144 | + | ||
| 145 | +- | ||
| 146 | + | ||
| 147 | +### 10.3 性能无收益问题 | ||
| 148 | + | ||
| 149 | +- | ||
| 150 | + | ||
| 151 | +### 10.4 图模式 / runtime 限制 | ||
| 152 | + | ||
| 153 | +- | ||
| 154 | + | ||
| 155 | +### 10.5 当前调试结论 | ||
| 156 | + | ||
| 157 | +- 已解决: | ||
| 158 | +- 未解决: | ||
| 159 | +- 下一步优先级: | ||
| 160 | + | ||
| 161 | +## 11. 验收结果 | ||
| 162 | + | ||
| 163 | +### 11.1 功能验收 | ||
| 164 | + | ||
| 165 | +- 是否通过: | ||
| 166 | +- 结果摘要: | ||
| 167 | + | ||
| 168 | +### 11.2 同步验收 | ||
| 169 | + | ||
| 170 | +- 是否通过: | ||
| 171 | +- 结果摘要: | ||
| 172 | + | ||
| 173 | +### 11.3 性能验收 | ||
| 174 | + | ||
| 175 | +- 优化前: | ||
| 176 | +- 优化后: | ||
| 177 | +- 结论: | ||
| 178 | + | ||
| 179 | +### 11.4 Profile 验收 | ||
| 180 | + | ||
| 181 | +- 是否观察到 overlap: | ||
| 182 | +- 是否仍有明显拖尾: | ||
| 183 | +- 关键结论: | ||
| 184 | + | ||
| 185 | +## 12. 最终结论 | ||
| 186 | + | ||
| 187 | +- 整网最适合做多流优化的位置: | ||
| 188 | +- 模块级并行的主结论: | ||
| 189 | +- 算子级并行的主结论: | ||
| 190 | +- 当前实现是否建议保留: | ||
| 191 | +- 后续建议: | ||
| @@ -0,0 +1,616 @@ | |||
| 1 | +--- | ||
| 2 | +name: cann-npu-perfanalysis | ||
| 3 | +description: > | ||
| 4 | + 分析 Ascend NPU profiling 数据,覆盖 8 个性能维度(迭代效率、算子热点、硬件利用率/MFU、 | ||
| 5 | + 通信效率、设备空泡、等待锚点、层级结构、多卡均衡),并给出 Host/Device Bound 总判定、 | ||
| 6 | + Device 内 compute/memory/communication/latency bound 分类、算子级 compute/memory bound、 | ||
| 7 | + wait pollution / total_cost 双排名、AICPU 暴露程度,输出有主次逻辑的诊断理由与优化建议, | ||
| 8 | + 生成 Markdown + HTML 双格式报告。 | ||
| 9 | + 触发场景:NPU 性能分析、step time 分析、算子热点、通信瓶颈、MFU 计算、 | ||
| 10 | + Host Dispatch 瓶颈、host bound、device bound、compute bound、memory bound、communication bound、 | ||
| 11 | + Free Time 高、wait anchor、device bubble、kernel gap、wait pollution、total cost、 | ||
| 12 | + cube utilization、AICPU 暴露、AICPU masked、多卡负载不均、快慢卡、AllReduce 开销、 | ||
| 13 | + overlap 比例、ASCEND_PROFILER_OUTPUT、kernel_details.csv、op_statistic.csv、 | ||
| 14 | + step_trace_time.csv、communication.json、profiling report。 | ||
| 15 | +--- | ||
| 16 | + | ||
| 17 | +# npu-perf-analysis | ||
| 18 | + | ||
| 19 | +分析 Ascend NPU Profiler 输出,覆盖 8 个性能维度,输出 Host/Device Bound 判定、瓶颈理由、优化建议与双格式报告(Markdown + HTML)。 | ||
| 20 | + | ||
| 21 | +--- | ||
| 22 | + | ||
| 23 | +## 输出组织原则(主次逻辑) | ||
| 24 | + | ||
| 25 | +报告必须按以下逻辑组织,不得把所有指标平铺: | ||
| 26 | + | ||
| 27 | +1. **先给结论**:执行摘要必须先回答“主要瓶颈是什么、优先级是多少、为什么”。 | ||
| 28 | +2. **再给 Bound 总判定**:明确输出 `HOST_BOUND` / `DEVICE_COMPUTE_BOUND` / `DEVICE_MEMORY_BOUND` / `DEVICE_COMMUNICATION_BOUND` / `DEVICE_LATENCY_BOUND` / `MIXED_BOUND` / `INSUFFICIENT_EVIDENCE`。 | ||
| 29 | +3. **再给证据链**:每个诊断必须包含 `事实指标 → 阈值对比 → 判定理由 → 置信度`。 | ||
| 30 | +4. **最后给行动建议**:建议必须按 P0-P3 排序,且说明“针对哪个瓶颈、改什么、预期影响哪个指标”。 | ||
| 31 | +5. **避免指标堆砌**:详细表格放在各维度章节;摘要只放 Top 3 发现和主因/次因。 | ||
| 32 | + | ||
| 33 | +--- | ||
| 34 | + | ||
| 35 | +## 参考文件加载指引 | ||
| 36 | + | ||
| 37 | +| 参考文件 | 何时加载 | | ||
| 38 | +|---|---| | ||
| 39 | +| `references/data-schema.md` | Phase 0 开始时,了解各 CSV/JSON 字段含义 | | ||
| 40 | +| `references/metrics-formulas.md` | Phase 1-2 计算指标时 | | ||
| 41 | +| `references/thresholds.md` | Phase 1-3 判定瓶颈优先级时 | | ||
| 42 | +| `references/hardware-specs.md` | Phase 2B 计算 MFU 时 | | ||
| 43 | + | ||
| 44 | +--- | ||
| 45 | + | ||
| 46 | +## 数据层次结构 | ||
| 47 | + | ||
| 48 | +``` | ||
| 49 | +ASCEND_PROFILER_OUTPUT/ | ||
| 50 | +├── step_trace_time.csv → Phase 1(迭代效率)+ Phase 2C(通信概况) | ||
| 51 | +├── op_statistic.csv → Phase 2A(算子热点) | ||
| 52 | +├── kernel_details.csv → Phase 2B/2D/2E/2F(MFU / 空泡 / 等待锚点 / 层级结构) | ||
| 53 | +├── communication.json → Phase 2C(通信带宽) | ||
| 54 | +├── communication_matrix.json → Phase 2H(多卡均衡,通常为空) | ||
| 55 | +├── trace_view.json → Phase 2G(Host evidence / wait pollution,可选) | ||
| 56 | +├── operator_details.csv → Phase 2G(Host-Device 算子归因,可选) | ||
| 57 | +└── OPPROF_*/ → Phase 2I(msprof op PMU 算子级 bound,可选) | ||
| 58 | +``` | ||
| 59 | + | ||
| 60 | +--- | ||
| 61 | + | ||
| 62 | +## Phase 0:数据预检(Data Inventory & Validation) | ||
| 63 | + | ||
| 64 | +**目标**:建立本次分析的能力矩阵,后续各维度按矩阵决定执行或跳过。 | ||
| 65 | + | ||
| 66 | +**步骤:** | ||
| 67 | + | ||
| 68 | +1. **定位 ASCEND_PROFILER_OUTPUT** | ||
| 69 | + - 若用户给定路径直接包含 `ASCEND_PROFILER_OUTPUT/`,使用该路径。 | ||
| 70 | + - 否则,递归扫描子目录(最多 2 层),找到第一个 `ASCEND_PROFILER_OUTPUT/` 目录。 | ||
| 71 | + - 在报告中记录"实际数据路径"(`actual_path`)。 | ||
| 72 | + | ||
| 73 | +2. **文件清点**:列出存在的文件,建立能力矩阵: | ||
| 74 | + | ||
| 75 | + | 文件 | 缺失时的降级策略 | | ||
| 76 | + |---|---| | ||
| 77 | + | `step_trace_time.csv` | 跳过 Phase 1;无法判定迭代效率 | | ||
| 78 | + | `op_statistic.csv` | 跳过 Phase 2A;无法统计算子热点 | | ||
| 79 | + | `kernel_details.csv` | 跳过 Phase 2B/2D/2E/2F | | ||
| 80 | + | `communication.json` | Phase 2C 仅用 step_trace_time 的 overlap 列 | | ||
| 81 | + | `communication_matrix.json` | 跳过 Phase 2H 带宽矩阵部分 | | ||
| 82 | + | `trace_view.json` | Phase 2G 仅做指标级 Host/Device 判定,无法做 host event overlap 软归因 | | ||
| 83 | + | `operator_details.csv` | Phase 2G 不输出 PyTorch 算子 Host/Device 归因 | | ||
| 84 | + | `OPPROF_*/*.csv` | Phase 2I 仅用 `kernel_details.csv` V2 做算子级 bound,无法用 PMU 细分 GM/UB/Bank Conflict | | ||
| 85 | + | ||
| 86 | +3. **Schema 版本检测**(`kernel_details.csv`): | ||
| 87 | + - 读取表头,若存在 `cube_utilization(%)` 列 → **V2 Schema**;否则 → **V1 Schema**。 | ||
| 88 | + - V1 Schema 下 MFU 改用 `aic_cube_fops / Duration_us / 1e6` 估算,置信度低,报告中须注明。 | ||
| 89 | + | ||
| 90 | +4. **设备与步数统计**(`step_trace_time.csv`): | ||
| 91 | + - 统计 `Device_id` 的唯一值 → 单卡 / 多卡模式。 | ||
| 92 | + - 统计 `Step` 的唯一值 → 步数列表。 | ||
| 93 | + | ||
| 94 | +5. **芯片型号检测**(优先级从高到低): | ||
| 95 | + - 用户在问题中直接指定 → 使用用户值。 | ||
| 96 | + - `profiler_metadata.json` 或 `profiler_info_N.json` 中记录的设备信息 → 解析。 | ||
| 97 | + - `kernel_details.csv` 的 `Block Dim` 最大值(910B3 通常为 64,910B4 通常为 32)→ 推断。 | ||
| 98 | + - 以上均不可用 → 默认 **Ascend 910B3(294.91 TFLOPs/s BF16)**,并在报告中标注。 | ||
| 99 | + | ||
| 100 | +--- | ||
| 101 | + | ||
| 102 | +## Phase 1:迭代效率(Dimension 1) | ||
| 103 | + | ||
| 104 | +**数据源**:`step_trace_time.csv` | ||
| 105 | + | ||
| 106 | +**计算以下每步指标**(公式见 `references/metrics-formulas.md`): | ||
| 107 | + | ||
| 108 | +``` | ||
| 109 | +computing_ratio = Computing / Stage | ||
| 110 | +comm_not_overlap_ratio = Communication(Not Overlapped) / Stage | ||
| 111 | +free_ratio = Free / Stage | ||
| 112 | +overlap_ratio = Overlapped / Communication (若 Communication=0 则跳过) | ||
| 113 | +``` | ||
| 114 | + | ||
| 115 | +**预热步检测规则**(依次检查): | ||
| 116 | +1. 若第一步的 `comm_not_overlap_ratio` 比后续步骤均值高出 **> 20 个百分点** → 标注为"疑似预热步,不参与均值统计"。 | ||
| 117 | +2. 若第一步的 `free_ratio` 或 `computing_ratio` 与后续步骤均值偏差 **> 20 个百分点** → 同上标注。 | ||
| 118 | +3. 仅有 1 步时 → 注明"单步采集,无法进行预热步判断"。 | ||
| 119 | + | ||
| 120 | +**正常步均值**:排除预热步后,计算各指标的算术均值。 | ||
| 121 | + | ||
| 122 | +**瓶颈判定**(阈值见 `references/thresholds.md`): | ||
| 123 | + | ||
| 124 | +| 指标 | 警告 | 严重 | 优先级规则 | | ||
| 125 | +|---|---|---|---| | ||
| 126 | +| free_ratio | > 10% | > 10% | 超过严重阈值 2 倍 → P0;超过严重阈值 → P1;超过警告线 → P3 | | ||
| 127 | +| comm_not_overlap_ratio | > 15% | > 30% | 超过严重阈值 2 倍 → P0;超过 → P1;超过警告线 → P2 | | ||
| 128 | +| overlap_ratio | < 50% 良好;< 20% 差 | — | < 20% 且 CommNO > 15% → P2 | | ||
| 129 | + | ||
| 130 | +**多卡模式**:额外计算每设备的 Stage 均值和 `variance_ratio = (max_Stage - min_Stage) / avg_Stage`。 | ||
| 131 | + | ||
| 132 | +--- | ||
| 133 | + | ||
| 134 | +## Phase 2:深度分析(Dimensions 2–8 + Bound 扩展) | ||
| 135 | + | ||
| 136 | +### 2A. 算子热点(Dimension 2) | ||
| 137 | + | ||
| 138 | +**数据源**:`op_statistic.csv` | ||
| 139 | + | ||
| 140 | +1. 按 `Total Time(us)` 降序排列,取 Top-10。 | ||
| 141 | +2. 热点标注:`Ratio(%) > 20` → `hotspot`;`> 10%` → `watch`;其余 → `normal`。 | ||
| 142 | +3. 若出现 MoE 专属算子(MoeGatingTopK / GroupedMatmul / DispatchFFNCombine / MoeInitRouting 系列),标注为 `moe_normal`(MoE 架构正常开销,不是瓶颈)。 | ||
| 143 | +4. 若出现 `Core Type = AI_CPU`,标注为 `aicpu`,并在报告中警告 AICPU 暴露风险。 | ||
| 144 | +5. 按 `Core Type` 分组,计算各组的 Total Time 占比。 | ||
| 145 | +6. 若 `kernel_details.csv` 存在,同时生成两类热点视图: | ||
| 146 | + - **duration_hotspots**:按 `Duration_us` 聚合,代表真实设备执行耗时。 | ||
| 147 | + - **total_cost_hotspots**:按 `Duration_us + Wait_Time_us` 聚合,代表时间线可见总成本。 | ||
| 148 | + - 若某算子 `total_cost` 排名高但 `wait_ratio > 0.95` 或 wait 与通信窗口重叠,标记 `WAIT_ANCHOR_FALSE_HOTSPOT` 或 `WAIT_POLLUTION_RISK`,不得作为真实计算热点。 | ||
| 149 | + | ||
| 150 | +**重要背景说明**:在 `free_ratio` 极高(> 50%)的情况下,op_statistic 的 `Total Time` 仅代表设备有效工作期间的算子耗时,不是 Stage 总时间的占比。高 `Ratio(%)` 不等于该算子是瓶颈,首要问题仍是减少 Free Time。 | ||
| 151 | + | ||
| 152 | +### 2B. 硬件利用率 / MFU(Dimension 3) | ||
| 153 | + | ||
| 154 | +**数据源**:`kernel_details.csv` | ||
| 155 | + | ||
| 156 | +**V2 Schema**(存在 `cube_utilization(%)` 列): | ||
| 157 | +- 对 MatMul / GroupedMatmul / FusedInferAttentionScore kernel,解析 `Input Shapes`: | ||
| 158 | + - MatMul:`M,K;K,N` 或 `M,K;N,K`(转置,FLOPs 相同)→ `FLOPs = 2 × M × K × N` | ||
| 159 | + - FIA:`FLOPs = 2 × q_batch × q_heads × q_seq × kv_seq × (q_dim + kv_dim)` × 0.5(因果掩码) | ||
| 160 | +- `MFU = (FLOPs / Duration_us / 1e6) / Peak_TFLOPs_s`(Peak 见 `references/hardware-specs.md`) | ||
| 161 | +- 直接读取 `cube_utilization(%)` 作为 Cube 流水线内部效率。 | ||
| 162 | +- `aic_mte2_ratio > 0.8` → 该 kernel Memory Bound(权重加载是瓶颈)。 | ||
| 163 | +- `aic_mac_ratio > 0.8` → 该 kernel Compute Bound(Cube 饱和)。 | ||
| 164 | +- `aic_fixpipe_ratio > 0.3` → 该 kernel FixPipe Bound。 | ||
| 165 | +- `aiv_scalar_ratio > 0.8` → 该 kernel Scalar / Latency Bound。 | ||
| 166 | +- `aiv_mte2_ratio > 0.5` 或 `aiv_mte3_ratio > 0.5` → 该 Vector kernel Memory Bound 风险。 | ||
| 167 | + | ||
| 168 | +**V1 Schema**: | ||
| 169 | +- `Achieved_TFLOPs_s = aic_cube_fops / Duration_us / 1e6`,`MFU = Achieved_TFLOPs_s / Peak`,置信度低,须在报告中注明。 | ||
| 170 | + | ||
| 171 | +**特殊情况**:Decode 阶段 M=1 的矩阵乘法,MFU 天然极低(< 5%),**不标注为异常**。 | ||
| 172 | + | ||
| 173 | +### 2C. 通信效率(Dimension 4) | ||
| 174 | + | ||
| 175 | +**数据源**:`communication.json` + `step_trace_time.csv` | ||
| 176 | + | ||
| 177 | +1. 从 `communication.json` 读取 `collective` 下各操作的 `Elapse Time`、`Transit Size`、`Bandwidth`。 | ||
| 178 | +2. 按操作名前缀归类:`HcomAllReduce_*` → AllReduce,`HcomAllGather_*` → AllGather,`HcomReduceScatter_*` → ReduceScatter。 | ||
| 179 | +3. 若所有传输介质的 `Bandwidth(GB/s)` 均为 0 → **单 Rank 采集**,仅报告 Elapse Time,注明"无法获取跨 Rank 带宽"。 | ||
| 180 | +4. 跨引用 `step_trace_time.csv` 的 `overlap_ratio`(Phase 1 已算)。 | ||
| 181 | +5. `Communication = 0`(step_trace_time)且无通信数据 → 跳过本维度,注明"无通信操作"。 | ||
| 182 | + | ||
| 183 | +### 2D. 设备空泡(Dimension 5) | ||
| 184 | + | ||
| 185 | +**数据源**:`kernel_details.csv` | ||
| 186 | + | ||
| 187 | +对每步(Step): | ||
| 188 | +1. 收集该步所有 kernel 的时间区间 `[Start_Time_us, Start_Time_us + Duration_us]`。 | ||
| 189 | +2. 合并重叠区间 → `busy_union`。 | ||
| 190 | +3. 步窗口 = 从该步第一个 kernel 的 Start 到 `Stage_us` 结束。 | ||
| 191 | +4. 计算:`prelaunch_gap`(步窗口开始到第一个 kernel)、`internal_bubble_total`(区间间隙之和)、`underfeed_ratio = (prelaunch_gap + internal_bubble_total) / stage_us`。 | ||
| 192 | +5. 阈值:`underfeed_ratio > 20%` → 严重(见 `references/thresholds.md`)。 | ||
| 193 | + | ||
| 194 | +**注意**:设备空泡与 step_trace_time 的 `Free Time` 粒度不同,不要混淆(Free Time 是 Host 调度视角,空泡是 Device 时间轴视角)。 | ||
| 195 | + | ||
| 196 | +### 2E. 等待锚点假热点(Dimension 6) | ||
| 197 | + | ||
| 198 | +**数据源**:`kernel_details.csv` | ||
| 199 | + | ||
| 200 | +1. 对每个 kernel 计算:`wait_ratio = Wait_Time_us / (Duration_us + Wait_Time_us)`。 | ||
| 201 | +2. 判定等待锚点:`wait_ratio > 0.95` **且** `Duration_us < 10.0`。 | ||
| 202 | +3. 按 `total_cost = Duration_us + Wait_Time_us` 降序列出所有等待锚点。 | ||
| 203 | +4. 对每个等待锚点,找出同 Stream 内时间上紧邻的**前一个 kernel** 和**后一个 kernel**,作为上下文信息。 | ||
| 204 | + | ||
| 205 | +### 2F. 层级结构(Dimension 7) | ||
| 206 | + | ||
| 207 | +**数据源**:`kernel_details.csv` | ||
| 208 | + | ||
| 209 | +1. **MoE 检测**:若出现 `MoeGatingTopK` / `MoeGatingTopKSoftmax` / `MoeComputeExpertTokens` / `DispatchFFNCombine` / `MoeInitRoutingV3` / `GroupedMatmul` 中的两个以上 → 判定为 MoE 模型。 | ||
| 210 | +2. **层数估算**:`FusedInferAttentionScore` 的 count ÷ 步数内出现频率 → 估算 Transformer 层数。 | ||
| 211 | +3. **推理阶段**:FIA Input Shapes 中 Q 维度 seq_len=1 → Decode 阶段;seq_len > 1 → Prefill 阶段。 | ||
| 212 | +4. `kernel_details.csv` 行数 < 500 → 跳过层级分析,注明"数据过稀疏"。 | ||
| 213 | + | ||
| 214 | +### 2G. Host/Device Bound 总判定(新增核心能力) | ||
| 215 | + | ||
| 216 | +**数据源**:`step_trace_time.csv` + `kernel_details.csv` + 可选 `trace_view.json` / `operator_details.csv` | ||
| 217 | + | ||
| 218 | +按以下顺序判定整体主瓶颈: | ||
| 219 | + | ||
| 220 | +1. **Host Bound 候选**: | ||
| 221 | + - `free_ratio > 10%` 或 `underfeed_ratio > 20%`。 | ||
| 222 | + - 若 `trace_view.json` 存在,扫描 bubble 窗口内 host events: | ||
| 223 | + - sync/H2D marker 覆盖率 ≥ 20% → `possible_sync_or_h2d` | ||
| 224 | + - comm marker 覆盖率 ≥ 20% → `possible_comm_wait` | ||
| 225 | + - host event 可见但无 sync/comm 主导 → `possible_host_launch_lag` | ||
| 226 | + - host event 覆盖率 < 5% 且 bubble 高 → `possible_untraced_host_blocking` | ||
| 227 | + - 输出 `HOST_BOUND` 或 `HOST_ORIGINATED_RISK`,并说明证据是否充足。 | ||
| 228 | + | ||
| 229 | +2. **Device Bound 候选**: | ||
| 230 | + - `free_ratio <= 10%` 且 `underfeed_ratio <= 20%`,但 `computing_ratio`、kernel busy 或通信未重叠占比高。 | ||
| 231 | + - 继续细分: | ||
| 232 | + - `DEVICE_COMPUTE_BOUND`:MFU / cube / vector 利用率高,或 `aic_mac_ratio > 0.8`。 | ||
| 233 | + - `DEVICE_MEMORY_BOUND`:`aic_mte2_ratio > 0.8`,或 Vector MTE / GM-UB 带宽证据占主导。 | ||
| 234 | + - `DEVICE_COMMUNICATION_BOUND`:`comm_not_overlap_ratio > 30%` 或通信 wait / bandwidth 异常。 | ||
| 235 | + - `DEVICE_LATENCY_BOUND`:compute 与 memory 利用率都低,小 kernel 多、scalar 占比高、launch/同步碎片明显。 | ||
| 236 | + | ||
| 237 | +3. **Mixed / Insufficient Evidence**: | ||
| 238 | + - Host 与 Device 证据同时超过阈值 → `MIXED_BOUND`,按贡献排序写主因/次因。 | ||
| 239 | + - 关键文件缺失或证据冲突 → `INSUFFICIENT_EVIDENCE`,必须说明缺哪些数据,不得强行归因。 | ||
| 240 | + | ||
| 241 | +### 2H. 多卡均衡(Dimension 8) | ||
| 242 | + | ||
| 243 | +**数据源**:`step_trace_time.csv` + 可选 `communication_matrix.json` | ||
| 244 | + | ||
| 245 | +若存在多个 `Device_id`,计算 per-device Stage / Computing / Free / CommNO 均值和 `variance_ratio`。 | ||
| 246 | +若仅单设备数据,跳过并注明"仅单卡数据"。 | ||
| 247 | + | ||
| 248 | +### 2I. 算子级 Compute/Memory Bound(新增核心能力) | ||
| 249 | + | ||
| 250 | +**数据源**:优先 `OPPROF_*/ArithmeticUtilization.csv` + `Memory.csv` + `ResourceConflictRatio.csv`,否则退化为 `kernel_details.csv` V2。 | ||
| 251 | + | ||
| 252 | +1. **PMU 数据存在时**: | ||
| 253 | + - 计算 `AI = (aic_cube_fops + aiv_vector_fops) / (GM_to_UB_datas + UB_to_GM_datas)`。 | ||
| 254 | + - 若 AI 低于硬件平衡点或经验阈值(无硬件带宽时用 50 FLOPs/Byte)→ `OP_MEMORY_BOUND`。 | ||
| 255 | + - 若 AI 高于硬件平衡点且 Cube/Vector 利用率高 → `OP_COMPUTE_BOUND`。 | ||
| 256 | + - 若 `GM_to_UB_bw_usage_rate < 30%` 或 `UB_to_GM_bw_usage_rate < 30%`,标记低带宽利用率。 | ||
| 257 | + - 若 `aiv_vec_bankgroup_cflt_ratio > 0.1` 或 `aiv_vec_bank_cflt_ratio > 0.1`,标记 `BANK_CONFLICT_RISK`。 | ||
| 258 | + | ||
| 259 | +2. **仅 kernel_details V2 时**: | ||
| 260 | + - `aic_mte2_ratio > 0.8` → `OP_MEMORY_BOUND` | ||
| 261 | + - `aic_mac_ratio > 0.8` → `OP_COMPUTE_BOUND` | ||
| 262 | + - `aiv_scalar_ratio > 0.8` → `OP_SCALAR_BOUND` | ||
| 263 | + - `aic_fixpipe_ratio > 0.3` → `OP_FIXPIPE_BOUND` | ||
| 264 | + | ||
| 265 | +3. **仅 op_statistic 时**: | ||
| 266 | + - 只输出热点,不输出 compute/memory bound;标注"证据不足"。 | ||
| 267 | + | ||
| 268 | +--- | ||
| 269 | + | ||
| 270 | +## Phase 3:瓶颈诊断(Bottleneck Synthesis) | ||
| 271 | + | ||
| 272 | +综合 Phase 1-2 的所有发现,先输出整体 Bound 判定,再按优先级输出瓶颈列表: | ||
| 273 | + | ||
| 274 | +| 优先级 | 判定条件 | | ||
| 275 | +|---|---| | ||
| 276 | +| **P0** | 任一指标超过其严重阈值 2 倍 | | ||
| 277 | +| **P1** | 任一指标超过严重阈值但 < 2 倍 | | ||
| 278 | +| **P2** | 多个指标处于警告范围;或单个次要指标略超严重阈值 | | ||
| 279 | +| **P3** | 单个指标超过警告线,其余正常 | | ||
| 280 | + | ||
| 281 | +- 对每个瓶颈,写明:标签(如 `HOST_DISPATCH_BOTTLENECK`)、指标数值、与阈值的对比。 | ||
| 282 | +- 对每个瓶颈,必须写明:**诊断理由**(为什么这个指标说明该瓶颈)和**反证/降级条件**(哪些数据缺失或冲突)。 | ||
| 283 | +- 预热步异常**不计入**正常步瓶颈(用括号单独说明)。 | ||
| 284 | +- 等待锚点的高 `total_cost` **不作为算子热点瓶颈**(须明确区分假热点)。 | ||
| 285 | +- 若 Host/Device Bound 与单项指标结论冲突,以证据链说明主因/次因,不得只给单标签。 | ||
| 286 | + | ||
| 287 | +--- | ||
| 288 | + | ||
| 289 | +## Phase 4:报告生成 | ||
| 290 | + | ||
| 291 | +Phase 4 分为三步,其中步骤 A 由 Agent 完成,步骤 B-C 由脚本完成。 | ||
| 292 | + | ||
| 293 | +### 步骤 A:生成 Markdown 报告(report.md) | ||
| 294 | + | ||
| 295 | +按以下固定章节顺序输出(共 14 章),用 Write 工具写入 `report.md`: | ||
| 296 | + | ||
| 297 | +``` | ||
| 298 | +一、分析上下文 — 路径、芯片、Schema、步数、设备数、文件清单、数据质量 | ||
| 299 | +二、执行摘要 — Top 3 发现、主因/次因、P0-P3 优先级 | ||
| 300 | +三、整体 Bound 判定 — Host/Device Bound、Device 子类、理由与置信度 | ||
| 301 | +四、维度 1:迭代效率 — 逐步时间拆分表格 + 瓶颈判定 | ||
| 302 | +五、维度 2:算子热点 — duration 热点 + total_cost 热点 + 核心类型分布 | ||
| 303 | +六、维度 3:硬件利用率/MFU — 代表性 kernel MFU + cube_utilization | ||
| 304 | +七、算子级 Bound 分析 — compute/memory/scalar/fixpipe/bank conflict | ||
| 305 | +八、维度 4:通信效率 — 集合通信类型 + 带宽数据 + wait pollution | ||
| 306 | +九、维度 5:设备空泡 — underfeed_ratio + host evidence 来源分析 | ||
| 307 | +十、维度 6:等待锚点 — 假热点列表 + 前后 kernel 上下文 | ||
| 308 | +十一、维度 7:层级结构 — 模型类型 + 层数 + MoE 特征 | ||
| 309 | +十二、维度 8:多卡均衡 — per-device Stage 对比(或跳过说明) | ||
| 310 | +十三、瓶颈诊断 — P0-P3 排列,含事实、理由、反证、置信度 | ||
| 311 | +十四、优化建议 — 按优先级,每条含具体措施 + 预期收益 | ||
| 312 | +``` | ||
| 313 | + | ||
| 314 | +被跳过的维度用一行说明原因(文件缺失 / 数据不足 / 单卡等)。 | ||
| 315 | + | ||
| 316 | +### 步骤 B:写出 analysis_data.json | ||
| 317 | + | ||
| 318 | +将本次分析的所有计算结果整理为结构化 JSON,写入 `analysis_data.json`。JSON Schema 见下方。 | ||
| 319 | + | ||
| 320 | +**此步骤由 Agent 完成**:Agent 将 Phase 1-3 计算出的所有指标按 Schema 整理。 | ||
| 321 | + | ||
| 322 | +### 步骤 C:运行 HTML 生成器(脚本完成) | ||
| 323 | + | ||
| 324 | +```bash | ||
| 325 | +python3 "$(dirname "$(find . -name SKILL.md | grep npu-perf-analysis | head -1)")/references/generate_html.py" \ | ||
| 326 | + analysis_data.json report.html | ||
| 327 | +``` | ||
| 328 | + | ||
| 329 | +HTML 生成器支持多种主题风格,参考 cann-perf-breakdown: | ||
| 330 | + | ||
| 331 | +```bash | ||
| 332 | +python3 references/generate_html.py analysis_data.json report.html --theme dracula | ||
| 333 | +python3 references/generate_html.py analysis_data.json report.html --theme vscode-dark | ||
| 334 | +python3 references/generate_html.py analysis_data.json report.html --theme github-light | ||
| 335 | +python3 references/generate_html.py analysis_data.json report.html --theme solarized-light | ||
| 336 | +``` | ||
| 337 | + | ||
| 338 | +可选主题:`dracula`(默认)、`vscode-dark`、`one-dark`、`github-light`、`solarized-light`。 | ||
| 339 | + | ||
| 340 | +若无法定位 SKILL.md 路径,也可直接用绝对路径: | ||
| 341 | + | ||
| 342 | +```bash | ||
| 343 | +python3 /path/to/npu-perf-analysis/references/generate_html.py analysis_data.json report.html | ||
| 344 | +``` | ||
| 345 | + | ||
| 346 | +脚本读取 `analysis_data.json`,渲染带导航栏、进度条、严重度徽章、可折叠章节的单文件 HTML 报告,写入 `report.html`。 | ||
| 347 | + | ||
| 348 | +--- | ||
| 349 | + | ||
| 350 | +## analysis_data.json Schema | ||
| 351 | + | ||
| 352 | +```json | ||
| 353 | +{ | ||
| 354 | + "meta": { | ||
| 355 | + "data_path": "用户输入的路径", | ||
| 356 | + "actual_path": "实际找到的 ASCEND_PROFILER_OUTPUT 路径", | ||
| 357 | + "chip": "Ascend 910B3", | ||
| 358 | + "peak_tflops": 294.91, | ||
| 359 | + "schema_version": "V2", | ||
| 360 | + "steps": [3, 4, 5], | ||
| 361 | + "steps_desc": "Step 3、4、5(Step 3 为预热步)", | ||
| 362 | + "devices": [0], | ||
| 363 | + "devices_desc": "Device_id=0(单设备)", | ||
| 364 | + "files_present": ["kernel_details.csv", "op_statistic.csv", "..."], | ||
| 365 | + "generated_at": "2026-05-07", | ||
| 366 | + "quality_notes": "数据质量说明..." | ||
| 367 | + }, | ||
| 368 | + "iteration_efficiency": { | ||
| 369 | + "steps": [ | ||
| 370 | + { | ||
| 371 | + "step": 3, | ||
| 372 | + "computing_us": 11841.0, | ||
| 373 | + "comm_no_us": 17543.0, | ||
| 374 | + "overlapped_us": 5.8, | ||
| 375 | + "communication_us": 17549.0, | ||
| 376 | + "free_us": 4642.0, | ||
| 377 | + "stage_us": 34027.0, | ||
| 378 | + "computing_ratio": 0.348, | ||
| 379 | + "comm_no_ratio": 0.516, | ||
| 380 | + "free_ratio": 0.136, | ||
| 381 | + "overlap_ratio": 0.00033, | ||
| 382 | + "is_warmup": true, | ||
| 383 | + "warmup_reason": "CommNO 占比 51.6%,比后续步骤(8-10%)高出 40+ 个百分点" | ||
| 384 | + } | ||
| 385 | + ], | ||
| 386 | + "warmup_steps": [3], | ||
| 387 | + "normal_steps": [4, 5], | ||
| 388 | + "avg": { | ||
| 389 | + "computing_ratio": 0.704, | ||
| 390 | + "comm_no_ratio": 0.087, | ||
| 391 | + "free_ratio": 0.209, | ||
| 392 | + "overlap_ratio": 0.004, | ||
| 393 | + "stage_us": 16839.5 | ||
| 394 | + }, | ||
| 395 | + "bottleneck": { | ||
| 396 | + "label": "HOST_DISPATCH_MILD", | ||
| 397 | + "priority": "P3", | ||
| 398 | + "evidence": "free_ratio=20.9%(正常步均值),超过警告线 10%,未超过严重阈值 10% 的 2 倍" | ||
| 399 | + } | ||
| 400 | + }, | ||
| 401 | + "bound_classification": { | ||
| 402 | + "overall_bound": "HOST_BOUND", | ||
| 403 | + "device_bound_type": null, | ||
| 404 | + "primary_bottleneck": "Host Dispatch", | ||
| 405 | + "secondary_bottlenecks": ["WAIT_POLLUTION_RISK"], | ||
| 406 | + "priority": "P0", | ||
| 407 | + "confidence": "高", | ||
| 408 | + "facts": [ | ||
| 409 | + "free_ratio=82.0%,超过严重阈值 10% 的 8.2 倍", | ||
| 410 | + "underfeed_ratio=73.0%,超过严重阈值 20%" | ||
| 411 | + ], | ||
| 412 | + "reasoning": "Stage 主要由 Free/underfeed 构成,设备没有持续被 AI Core/HCCL 喂满,因此主瓶颈是 Host 侧下发或同步等待,而不是单个 MatMul kernel 算力不足。", | ||
| 413 | + "counter_evidence": "未提供 trace_view.json,无法进一步区分 Python 调度、H2D 同步或未采样 Host 阻塞。", | ||
| 414 | + "host_evidence": { | ||
| 415 | + "available": false, | ||
| 416 | + "sync_or_h2d_overlap_ratio": null, | ||
| 417 | + "comm_marker_overlap_ratio": null, | ||
| 418 | + "host_visible_coverage_ratio": null, | ||
| 419 | + "soft_labels": ["possible_untraced_host_blocking"] | ||
| 420 | + } | ||
| 421 | + }, | ||
| 422 | + "operator_hotspots": { | ||
| 423 | + "top_ops": [ | ||
| 424 | + { | ||
| 425 | + "rank": 1, | ||
| 426 | + "name": "MatMul", | ||
| 427 | + "core_type": "AI_CORE", | ||
| 428 | + "count": 96, | ||
| 429 | + "total_us": 4674.0, | ||
| 430 | + "avg_us": 48.7, | ||
| 431 | + "max_us": 125.3, | ||
| 432 | + "ratio": 48.5, | ||
| 433 | + "flag": "hotspot" | ||
| 434 | + } | ||
| 435 | + ], | ||
| 436 | + "core_type_breakdown": { | ||
| 437 | + "AI_CORE": 48.5, | ||
| 438 | + "AI_VECTOR_CORE": 35.3, | ||
| 439 | + "MIX_AIC": 16.3, | ||
| 440 | + "AI_CPU": 0.0 | ||
| 441 | + }, | ||
| 442 | + "has_aicpu": false, | ||
| 443 | + "note": "可选背景说明文字" | ||
| 444 | + }, | ||
| 445 | + "hardware_utilization": { | ||
| 446 | + "skipped": false, | ||
| 447 | + "skip_reason": null, | ||
| 448 | + "schema_version": "V2", | ||
| 449 | + "representative_kernels": [ | ||
| 450 | + { | ||
| 451 | + "name": "MatMul", | ||
| 452 | + "input_shapes": "1,4096;4096,4096", | ||
| 453 | + "flops": 33554432.0, | ||
| 454 | + "duration_us": 48.7, | ||
| 455 | + "mfu": 0.023, | ||
| 456 | + "cube_utilization": 74.5, | ||
| 457 | + "memory_bound": false, | ||
| 458 | + "verdict": "✅ 正常(Decode M=1)" | ||
| 459 | + } | ||
| 460 | + ], | ||
| 461 | + "avg_cube_utilization": 74.5, | ||
| 462 | + "avg_mfu": 0.023, | ||
| 463 | + "note": "Decode 阶段 M=1,MFU 天然极低,不标注为异常。" | ||
| 464 | + }, | ||
| 465 | + "operator_bound_analysis": { | ||
| 466 | + "skipped": false, | ||
| 467 | + "source": "kernel_details_v2", | ||
| 468 | + "summary": { | ||
| 469 | + "compute_bound_count": 12, | ||
| 470 | + "memory_bound_count": 4, | ||
| 471 | + "scalar_bound_count": 1, | ||
| 472 | + "fixpipe_bound_count": 0, | ||
| 473 | + "bank_conflict_count": 0 | ||
| 474 | + }, | ||
| 475 | + "top_operators": [ | ||
| 476 | + { | ||
| 477 | + "name": "MatMul", | ||
| 478 | + "core_type": "AI_CORE", | ||
| 479 | + "duration_us": 48.7, | ||
| 480 | + "bound_type": "OP_COMPUTE_BOUND", | ||
| 481 | + "evidence": "aic_mac_ratio=0.84 > 0.8,Cube 单元占主导", | ||
| 482 | + "confidence": "中", | ||
| 483 | + "recommendation_hint": "优先检查 shape/batch 是否能提高 MFU;若 Decode M=1,不建议以提升 MFU 为主目标。" | ||
| 484 | + } | ||
| 485 | + ], | ||
| 486 | + "pmu_notes": "未发现 OPPROF_* msprof op 输出,未计算 GM/UB 带宽与 Bank Conflict。" | ||
| 487 | + }, | ||
| 488 | + "communication_efficiency": { | ||
| 489 | + "skipped": false, | ||
| 490 | + "skip_reason": null, | ||
| 491 | + "overlap_ratio_avg": 0.004, | ||
| 492 | + "collectives": { | ||
| 493 | + "AllReduce": 32, | ||
| 494 | + "AllGather": 1, | ||
| 495 | + "ReduceScatter": 0, | ||
| 496 | + "Broadcast": 0 | ||
| 497 | + }, | ||
| 498 | + "bandwidth": { | ||
| 499 | + "RDMA": 0.0, | ||
| 500 | + "HCCS": 0.0, | ||
| 501 | + "PCIE": 0.0 | ||
| 502 | + }, | ||
| 503 | + "single_rank": true, | ||
| 504 | + "max_elapse_ms": 16.3, | ||
| 505 | + "note": "communication.json 中所有传输介质带宽数据为 0,判断为单 Rank 采集,无法获取跨 Rank 通信带宽。" | ||
| 506 | + }, | ||
| 507 | + "device_bubbles": { | ||
| 508 | + "skipped": false, | ||
| 509 | + "skip_reason": null, | ||
| 510 | + "underfeed_ratio": 0.21, | ||
| 511 | + "prelaunch_gap_ms": 0.8, | ||
| 512 | + "internal_bubble_ms": 2.5, | ||
| 513 | + "note": "主要来源为 Host 下发间隙(prelaunch + inter-kernel gaps)。" | ||
| 514 | + }, | ||
| 515 | + "wait_anchors": [ | ||
| 516 | + { | ||
| 517 | + "name": "AivKernel", | ||
| 518 | + "duration_us": 6.0, | ||
| 519 | + "wait_us": 16293.0, | ||
| 520 | + "wait_ratio": 0.9996, | ||
| 521 | + "total_cost_us": 16299.0, | ||
| 522 | + "stream_id": "43", | ||
| 523 | + "prev_kernel": "HcomAllReduce_xxxxx", | ||
| 524 | + "next_kernel": "MatMul_xxxxx" | ||
| 525 | + } | ||
| 526 | + ], | ||
| 527 | + "layer_structure": { | ||
| 528 | + "skipped": false, | ||
| 529 | + "skip_reason": null, | ||
| 530 | + "model_type": "MoE", | ||
| 531 | + "num_layers": 32, | ||
| 532 | + "has_moe": true, | ||
| 533 | + "moe_ops_detected": ["MoeGatingTopKSoftmax", "MoeComputeExpertTokens", "GroupedMatmul"], | ||
| 534 | + "inference_phase": "Decode", | ||
| 535 | + "note": "32 层 MoE Transformer,Decode 阶段单 token 生成。" | ||
| 536 | + }, | ||
| 537 | + "multi_card": { | ||
| 538 | + "skipped": true, | ||
| 539 | + "skip_reason": "仅有 Device_id=0 的单设备数据,无法进行多卡负载均衡分析", | ||
| 540 | + "devices": [0], | ||
| 541 | + "variance_ratio": null | ||
| 542 | + }, | ||
| 543 | + "bottleneck_diagnosis": [ | ||
| 544 | + { | ||
| 545 | + "priority": "P3", | ||
| 546 | + "label": "HOST_DISPATCH_MILD", | ||
| 547 | + "dimension": "迭代效率", | ||
| 548 | + "evidence": "free_ratio=20.9%(Step 4-5 均值),超过警告线 10%,未超过严重阈值 10% 的 2 倍", | ||
| 549 | + "reasoning": "Free Time 计入 Stage 且设备无计算/通信执行,说明设备侧等待 Host 下发或同步解除。", | ||
| 550 | + "counter_evidence": "无 trace_view.json,无法唯一定位 Python 调度、H2D 同步或外部阻塞。", | ||
| 551 | + "confidence": "中" | ||
| 552 | + } | ||
| 553 | + ], | ||
| 554 | + "recommendations": [ | ||
| 555 | + { | ||
| 556 | + "priority": "P3", | ||
| 557 | + "title": "减少小算子下发次数", | ||
| 558 | + "action": "合并 Add、Mul、Cast 等高频小算子为融合算子,减少 Host-Device 往返", | ||
| 559 | + "benefit": "Free Time 从 21% 降至 <15%" | ||
| 560 | + } | ||
| 561 | + ], | ||
| 562 | + "not_recommended": [ | ||
| 563 | + "MatMul 内核效率优化(Decode M=1 低 MFU 属正常)", | ||
| 564 | + "通信策略优化(CommNO=8.7% 在正常范围内)" | ||
| 565 | + ] | ||
| 566 | +} | ||
| 567 | +``` | ||
| 568 | + | ||
| 569 | +--- | ||
| 570 | + | ||
| 571 | +## NEVER 列表(禁止行为) | ||
| 572 | + | ||
| 573 | +- **NEVER** 将 `step_trace_time.csv` 的 `Free Time` 与 kernel 级设备空泡混同——两者粒度、来源、修复方式均不同。 | ||
| 574 | +- **NEVER** 仅凭高 `Ratio(%)` 就判断某算子是性能瓶颈,必须先检查该 kernel 的 `wait_ratio`(可能是等待锚点假热点)。 | ||
| 575 | +- **NEVER** 仅凭 `op_statistic.csv` 判断算子 compute/memory bound;没有 `kernel_details.csv` V2 或 msprof op PMU 时只能说"热点",不能说"compute bound/memory bound"。 | ||
| 576 | +- **NEVER** 把 Host Bound 说成确定根因(如 Python/GIL/H2D)而不给 host event 证据;证据不足时必须输出 soft label 和缺失数据。 | ||
| 577 | +- **NEVER** 忽略 `total_cost` 与 `duration` 的差异;total_cost 高且 wait 高时必须检查等待锚点和 wait pollution。 | ||
| 578 | +- **NEVER** 对 `Communication=0` 的数据集输出任何通信优化建议(如 AllReduce overlap、减少通信量)。 | ||
| 579 | +- **NEVER** 为 Decode 单 token(M=1)场景的低 MFU 标注异常或给出"提升 MFU"的建议。 | ||
| 580 | +- **NEVER** 将预热步(Step 1 或 CommNO 异常偏高步)的数据纳入正常步均值。 | ||
| 581 | +- **NEVER** 报告跨 Rank 带宽为 0 时不加说明——必须注明"单 Rank 采集,带宽数据不可用"。 | ||
| 582 | +- **NEVER** 对仅有单设备数据的场景进行多卡均衡分析。 | ||
| 583 | +- **NEVER** 仅报告一种粒度的指标——必须同时输出 step 级效率(step_trace_time)和 op 级热点(op_statistic)。 | ||
| 584 | +- **NEVER** 对 MoE 模型的 GroupedMatmul、MoeGatingTopK 等算子建议"减少计算量"——这是 MoE 架构的正常开销。 | ||
| 585 | + | ||
| 586 | +--- | ||
| 587 | + | ||
| 588 | +## 优雅降级(Graceful Degradation) | ||
| 589 | + | ||
| 590 | +| 缺失场景 | 降级行为 | | ||
| 591 | +|---|---| | ||
| 592 | +| 无 `kernel_details.csv` | 跳过维度 3/5/6/7,仅凭 op_statistic 做算子热点 | | ||
| 593 | +| V1 Schema | 低置信度 MFU(基于 aic_cube_fops),报告中注明 | | ||
| 594 | +| 无 `communication.json` | Phase 2C 仅报告 step_trace_time 的 overlap_ratio | | ||
| 595 | +| 无 `trace_view.json` | Host/Device Bound 仍可基于 Free/underfeed 判定,但 Host 根因只输出 soft label,置信度下降 | | ||
| 596 | +| 无 `OPPROF_*` | 算子级 bound 退化为 `kernel_details.csv` V2;若也缺失 V2 字段,则只输出热点不输出 compute/memory bound | | ||
| 597 | +| 单 Device_id | 跳过维度 8,注明"仅单卡数据" | | ||
| 598 | +| 单步采集 | 注明"无法预热步判断和步间周期性分析" | | ||
| 599 | +| FIA Input Shapes 为空 | 跳过 FIA MFU 计算,仅报告 cube_utilization | | ||
| 600 | +| 所有带宽为 0 | 注明"单 Rank 采集",仅报告 Elapse Time | | ||
| 601 | +| 嵌套目录 | Phase 0 自动扫描,在 actual_path 中记录完整路径 | | ||
| 602 | +| `kernel_details.csv` 行数 < 500 | 跳过维度 7(层级结构),注明"数据过稀疏" | | ||
| 603 | + | ||
| 604 | +--- | ||
| 605 | + | ||
| 606 | +## 评测用例参考 | ||
| 607 | + | ||
| 608 | +评测用例集见 `eval/eval-cases.md`,包含 5 个标准场景: | ||
| 609 | + | ||
| 610 | +| 用例 | 数据集 | 核心验证点 | | ||
| 611 | +|---|---|---| | ||
| 612 | +| 1 | gemma | 预热步识别(Step 3 CommNO=51.6%),MoE 特征,等待锚点 AivKernel | | ||
| 613 | +| 2 | qwen7b | P0 Host Dispatch(free_ratio=82%),不推荐通信优化 | | ||
| 614 | +| 3 | ds3.2 | 单设备声明(Device_id=3),预热步(Step 10 CommNO=68.7%),双瓶颈 | | ||
| 615 | +| 4 | longcat | 嵌套目录自动发现,P0 通信瓶颈(74.1%),单 Rank 带宽声明 | | ||
| 616 | +| 5 | gemma | 等待锚点专项(AivKernel wait_ratio=99.96%),假热点降级 | | ||
| @@ -0,0 +1,209 @@ | |||
| 1 | +# npu-perf-analysis Skill 评测用例集 | ||
| 2 | + | ||
| 3 | +本文件包含 5 个标准评测场景,用于验证 `npu-perf-analysis` Skill 的分析正确性。每个用例定义了输入条件、预期输出结构和可验证的关键断言。 | ||
| 4 | + | ||
| 5 | +--- | ||
| 6 | + | ||
| 7 | +## 用例 1:MoE 模型预热步识别 + 等待锚点检测 | ||
| 8 | + | ||
| 9 | +**场景描述** | ||
| 10 | +分析 Gemma MoE 推理模型的 profiling 数据,需正确识别预热步、MoE 模型特征,以及等待锚点假热点。 | ||
| 11 | + | ||
| 12 | +**输入** | ||
| 13 | +``` | ||
| 14 | +数据路径:prof-data/gemma/ASCEND_PROFILER_OUTPUT/ | ||
| 15 | +用户问题:帮我分析这份 NPU profiling 数据,看看有没有性能问题。 | ||
| 16 | +``` | ||
| 17 | + | ||
| 18 | +**预期输出结构** | ||
| 19 | +- 包含"分析上下文"章节:列出文件清单、Schema 版本(V2)、步骤数(3)、设备数(1) | ||
| 20 | +- 包含"迭代效率"章节:含 Step 3/4/5 的时间拆分表格 | ||
| 21 | +- 包含"算子热点"章节:Top-10 算子表格 | ||
| 22 | +- 包含"等待锚点"章节:列出被标记的 kernel | ||
| 23 | +- 包含"瓶颈诊断"章节:包含 P0–P3 分级 | ||
| 24 | +- 包含"优化建议"章节 | ||
| 25 | +- 同时生成 Markdown 报告和 HTML 报告 | ||
| 26 | + | ||
| 27 | +**关键断言** | ||
| 28 | + | ||
| 29 | +**断言 1.1** — 预热步标注 | ||
| 30 | +Step 3 的 `CommNO=51.6%` 与 Step 4-5 的 `CommNO≈8-10%` 差距超 20%,报告必须将 Step 3 标注为"疑似预热步,不具代表性",且不参与正常步均值计算。 | ||
| 31 | + | ||
| 32 | +**断言 1.2** — 正常步瓶颈判定 | ||
| 33 | +基于 Step 4-5 的平均数据: | ||
| 34 | +- `computing_ratio ≈ 70%`,`free_ratio ≈ 21%`(超过 10% 警告线) | ||
| 35 | +- 报告须注明"Free Time 偏高(约 21%),可能存在轻微 Host Dispatch 延迟",但因未超过严重阈值 2 倍,不应判定 P0 | ||
| 36 | + | ||
| 37 | +**断言 1.3** — MoE 模型特征检测 | ||
| 38 | +算子热点章节须出现以下 MoE 特征算子,并说明其为 MoE 架构的正常计算开销: | ||
| 39 | +- `MoeGatingTopKSoftmax` / `MoeComputeExpertTokens` / `GroupedMatmul` / `MoeInitRouting` | ||
| 40 | + | ||
| 41 | +**断言 1.4** — 等待锚点检测 | ||
| 42 | +等待锚点章节须包含至少 1 条: | ||
| 43 | +- `AivKernel`:`wait_ratio ≈ 99.96%`,`Duration ≈ 6μs`,`Wait ≈ 16293μs` | ||
| 44 | +- 明确标注"假热点,按 total_cost 排名靠前,但实际计算时间极短,真实原因在上游" | ||
| 45 | + | ||
| 46 | +**断言 1.5** — 不错误推荐 | ||
| 47 | +报告的优化建议中,**不应**出现"优化 MatMul 算子实现"或"减少通信量"等明显不对症的建议。 | ||
| 48 | + | ||
| 49 | +--- | ||
| 50 | + | ||
| 51 | +## 用例 2:Host Dispatch 严重瓶颈识别 | ||
| 52 | + | ||
| 53 | +**场景描述** | ||
| 54 | +分析 Qwen-7B Dense 模型的 profiling 数据,该数据集的特征是 Free Time 极高(约 82%)、无通信操作。 | ||
| 55 | + | ||
| 56 | +**输入** | ||
| 57 | +``` | ||
| 58 | +数据路径:prof-data/qwen7b/ASCEND_PROFILER_OUTPUT/ | ||
| 59 | +用户问题:为什么这个模型训练很慢?效率低在哪里? | ||
| 60 | +``` | ||
| 61 | + | ||
| 62 | +**预期输出结构** | ||
| 63 | +- 维度 4(通信效率):明确标注"跳过:无通信数据(CommNO=0,Overlap=0)" | ||
| 64 | +- 维度 8(多卡均衡):明确标注"跳过:仅有单卡数据(Device_id=0)" | ||
| 65 | +- 包含 P0 瓶颈诊断 | ||
| 66 | +- 优化建议至少包含一条针对 Host Dispatch 的具体措施 | ||
| 67 | + | ||
| 68 | +**关键断言** | ||
| 69 | + | ||
| 70 | +**断言 2.1** — P0 Host Dispatch 瓶颈 | ||
| 71 | +Step 2-4 的 `free_ratio` 均约 81-83%,报告须输出: | ||
| 72 | +- 优先级:**P0** | ||
| 73 | +- 标签:`HOST_DISPATCH_BOTTLENECK` | ||
| 74 | +- 证据:`free_ratio = 82%(Step 2)/ 81%(Step 3)/ 81%(Step 4)`,超过严重阈值 10% 的 8 倍 | ||
| 75 | + | ||
| 76 | +**断言 2.2** — 不推荐通信优化 | ||
| 77 | +优化建议中,**绝对不能**出现任何关于"通信"、"AllReduce"、"overlap"的优化建议(该数据集 Communication 全为 0)。 | ||
| 78 | + | ||
| 79 | +**断言 2.3** — 首要建议指向 Host 侧 | ||
| 80 | +P0 的优化建议须包含以下方向之一: | ||
| 81 | +- 减少算子下发次数(算子融合、图编译) | ||
| 82 | +- 检查 PyTorch eager 模式小算子碎片化 | ||
| 83 | +- 使用 torch.compile 或 mindspore 图模式 | ||
| 84 | + | ||
| 85 | +**断言 2.4** — 算子热点正确列出 | ||
| 86 | +`MatMulV2` 以 `Ratio=66.2%` 排名第一,但报告须区分"绝对热点"与"相对重要性":在 free_ratio=82% 的情况下,设备实际有效执行时间仅约 18%,MatMul 的真实影响被放大;首要问题是 Host Dispatch,而非 MatMul 本身效率。 | ||
| 87 | + | ||
| 88 | +--- | ||
| 89 | + | ||
| 90 | +## 用例 3:多步采集 + 单设备 + 预热步识别 | ||
| 91 | + | ||
| 92 | +**场景描述** | ||
| 93 | +分析 DeepSpeed 3.2 的 profiling 数据(Device_id=3,采集了 10 步),需正确识别预热步,并在单设备数据中诊断瓶颈。 | ||
| 94 | + | ||
| 95 | +**输入** | ||
| 96 | +``` | ||
| 97 | +数据路径:prof-data/ds3.2/ASCEND_PROFILER_OUTPUT/ | ||
| 98 | +用户问题:分析一下这张卡的训练性能,有没有瓶颈? | ||
| 99 | +``` | ||
| 100 | + | ||
| 101 | +**预期输出结构** | ||
| 102 | +- 分析上下文须注明:Device_id=3,步骤数=10(Step 10-19) | ||
| 103 | +- 维度 8(多卡均衡):明确标注"跳过:仅有 Device_id=3 的单设备数据,无法进行多卡对比" | ||
| 104 | +- 迭代效率章节含 10 步数据,且 Step 10 被标注为预热步 | ||
| 105 | + | ||
| 106 | +**关键断言** | ||
| 107 | + | ||
| 108 | +**断言 3.1** — 单设备声明 | ||
| 109 | +报告中必须出现:"本次采集仅包含 Device_id=3 的数据,无法进行多卡负载均衡分析"。 | ||
| 110 | + | ||
| 111 | +**断言 3.2** — 预热步标注 | ||
| 112 | +Step 10 的 `CommNO=68.7%` 与后续步骤(6-13%)差距极大,须标注为"预热步,不参与均值统计"。 | ||
| 113 | + | ||
| 114 | +**断言 3.3** — 正常步瓶颈双诊断 | ||
| 115 | +基于 Step 11-19(排除 Step 10): | ||
| 116 | +- `avg_free_ratio ≈ 34%` → **P1 Host Dispatch 瓶颈**(严重阈值 10%,34% 超过 3 倍) | ||
| 117 | +- `avg_comm_not_overlap_ratio ≈ 9.7%` → 接近警告线(15%),判定为 **P2 或观察级**(须引用具体均值) | ||
| 118 | + | ||
| 119 | +**断言 3.4** — 通信类型识别 | ||
| 120 | +从 `communication.json`(如存在集合通信数据)或算子名推断,报告须识别出 AllReduce / AllGather / ReduceScatter 三种通信操作类型(ds3.2 是 ZeRO-3 训练框架)。 | ||
| 121 | + | ||
| 122 | +--- | ||
| 123 | + | ||
| 124 | +## 用例 4:通信严重瓶颈 + 嵌套目录处理 | ||
| 125 | + | ||
| 126 | +**场景描述** | ||
| 127 | +分析 Longcat 长文本 MoE 推理模型,数据目录为嵌套结构(ASCEND_PROFILER_OUTPUT 在子目录内),且通信占总时间 74%,属于严重通信瓶颈。 | ||
| 128 | + | ||
| 129 | +**输入** | ||
| 130 | +``` | ||
| 131 | +数据路径:prof-data/longcat/ | ||
| 132 | +用户问题:这个模型的性能瓶颈在哪里? | ||
| 133 | +``` | ||
| 134 | + | ||
| 135 | +**预期输出结构** | ||
| 136 | +- 分析上下文:注明实际找到的 ASCEND_PROFILER_OUTPUT 子路径 | ||
| 137 | +- 维度 1(迭代效率):含 Step 5 的数据 | ||
| 138 | +- 维度 4(通信效率):含通信时间分布,并注明带宽数据不可用 | ||
| 139 | +- 维度 8(多卡均衡):标注"单步单设备,跳过" | ||
| 140 | + | ||
| 141 | +**关键断言** | ||
| 142 | + | ||
| 143 | +**断言 4.1** — 嵌套目录自动发现 | ||
| 144 | +Phase 0 须自动扫描子目录并找到: | ||
| 145 | +`liteserver-b9ea-smoke-0_2930608_20260421212924990_ascend_pt/ASCEND_PROFILER_OUTPUT/` | ||
| 146 | +报告中须注明完整的实际路径。 | ||
| 147 | + | ||
| 148 | +**断言 4.2** — P0 通信瓶颈 | ||
| 149 | +Step 5 的 `comm_not_overlap_ratio = 74.1%`(严重阈值 30% 的 2.5 倍),报告须输出: | ||
| 150 | +- 优先级:**P0** | ||
| 151 | +- 标签:`COMMUNICATION_BOTTLENECK` | ||
| 152 | +- 证据:CommNO=75467μs,Stage=101889μs,比例=74.1% | ||
| 153 | + | ||
| 154 | +**断言 4.3** — 单步声明 | ||
| 155 | +报告须包含:"仅采集到 1 个 Step(Step 5),无法进行预热步判断和步间周期性分析"。 | ||
| 156 | + | ||
| 157 | +**断言 4.4** — 单 Rank 带宽声明 | ||
| 158 | +通信效率章节须包含:"`communication.json` 中所有传输介质带宽数据为 0,判断为单 Rank 采集,无法获取跨 Rank 通信带宽。仅报告通信时长信息。" | ||
| 159 | + | ||
| 160 | +**断言 4.5** — MoE 特征识别 | ||
| 161 | +层级结构章节须识别出 MoE 算子特征(`FusedInferAttentionScore`、`GroupedMatmul`、`MoeInitRoutingV3`),并注明这是一个 MoE 推理模型。 | ||
| 162 | + | ||
| 163 | +--- | ||
| 164 | + | ||
| 165 | +## 用例 5:等待锚点假热点专项分析 | ||
| 166 | + | ||
| 167 | +**场景描述** | ||
| 168 | +使用 Gemma 数据集,专项测试等待锚点检测能力:识别出那些按 `total_cost` 排名靠前但实际上是假热点的 kernel,并正确区分真实热点。 | ||
| 169 | + | ||
| 170 | +**输入** | ||
| 171 | +``` | ||
| 172 | +数据路径:prof-data/gemma/ASCEND_PROFILER_OUTPUT/ | ||
| 173 | +用户问题:帮我找出 NPU 上最耗时的算子,并判断是否存在假热点。 | ||
| 174 | +``` | ||
| 175 | + | ||
| 176 | +**预期输出结构** | ||
| 177 | +- 维度 2(算子热点):基于 `op_statistic.csv` 的 Top-10 表格 | ||
| 178 | +- 维度 6(等待锚点):专门的等待锚点列表,包含 wait_ratio 和上下文 | ||
| 179 | +- 瓶颈诊断:对热点算子的定性说明,区分假热点和真实热点 | ||
| 180 | + | ||
| 181 | +**关键断言** | ||
| 182 | + | ||
| 183 | +**断言 5.1** — 等待锚点被检测到 | ||
| 184 | +等待锚点章节须列出至少 1 条满足条件的 kernel: | ||
| 185 | +- **AivKernel**:`wait_ratio ≈ 99.96%`,`Duration ≈ 6μs`,`Wait ≈ 16293μs` | ||
| 186 | +- 或 **UpdateModelParam_static_bin**:`wait_ratio ≈ 99.7%`,`Duration ≈ 7.8μs`,`Wait ≈ 2748μs` | ||
| 187 | + | ||
| 188 | +**断言 5.2** — 假热点降级 | ||
| 189 | +等待锚点 kernel 在按 `total_cost`(Duration+Wait)排名时可能排名靠前,但报告须明确说明: | ||
| 190 | +"上述 kernel 按 total_cost 排名靠前,但 wait_ratio > 95% 且 Duration < 10μs,判定为等待锚点假热点。其真实计算耗时极短,不是瓶颈所在;实际问题是上游操作导致设备空转。" | ||
| 191 | + | ||
| 192 | +**断言 5.3** — 真实热点正确识别 | ||
| 193 | +在算子热点章节,`MatMul`(`Ratio=48.5%`,`Avg=24.4μs`)被标注为**真实热点**(高 Duration、高总时间、无异常 wait_ratio),与等待锚点有明确区分。 | ||
| 194 | + | ||
| 195 | +**断言 5.4** — 上下文信息提供 | ||
| 196 | +等待锚点的报告须包含该 kernel 前后紧邻的 kernel 名称(用于人工排查上游问题),不能只输出 kernel 名称而无上下文。 | ||
| 197 | + | ||
| 198 | +--- | ||
| 199 | + | ||
| 200 | +## 评测评分标准 | ||
| 201 | + | ||
| 202 | +| 断言类型 | 权重 | 说明 | | ||
| 203 | +|---|---|---| | ||
| 204 | +| 数值精度(±5%以内) | 高 | 计算出的比例/指标与实际数据吻合 | | ||
| 205 | +| 严重度分级正确性 | 高 | P0/P1/P2/P3 的判定符合阈值规则 | | ||
| 206 | +| 不错误推荐 | 高 | 无不对症的优化建议(如对零通信数据推荐通信优化) | | ||
| 207 | +| 缺失数据处理 | 中 | 正确声明跳过的维度及原因 | | ||
| 208 | +| 术语使用准确性 | 中 | 等待锚点/假热点/预热步等术语使用正确 | | ||
| 209 | +| 报告完整性 | 低 | 12 个章节均存在(或有合理跳过说明) | | ||
| @@ -0,0 +1,301 @@ | |||
| 1 | +# Profiling 数据文件字段说明 | ||
| 2 | + | ||
| 3 | +> **使用时机**:执行任何 CSV / JSON 读取之前必须先读本文件,了解各列的语义和计量单位,避免误用字段。 | ||
| 4 | + | ||
| 5 | +--- | ||
| 6 | + | ||
| 7 | +## 一、step_trace_time.csv(11 列) | ||
| 8 | + | ||
| 9 | +记录每个训练/推理迭代的时间拆分,是迭代效率分析的主要来源。 | ||
| 10 | + | ||
| 11 | +| 列名 | 单位 | 含义 | | ||
| 12 | +|---|---|---| | ||
| 13 | +| `Device_id` | — | NPU 设备编号(多卡时有多行) | | ||
| 14 | +| `Step` | — | 迭代编号(从 3 起通常含预热步) | | ||
| 15 | +| `Computing` | μs | 设备真实执行计算 kernel 的时间(AI Core / AI CPU / AICPU 均含,不含通信重叠部分) | | ||
| 16 | +| `Communication(Not Overlapped)` | μs | **未被计算遮蔽**的通信时间(真实通信代价,直接计入迭代耗时) | | ||
| 17 | +| `Overlapped` | μs | 与计算**并发执行**的通信时间(已被遮蔽,不额外占用迭代时间) | | ||
| 18 | +| `Communication` | μs | 总通信时间 = `Communication(Not Overlapped)` + `Overlapped` | | ||
| 19 | +| `Free` | μs | NPU 处于空闲的时间,原因是 Host 侧未及时下发 kernel(Host Dispatch 瓶颈指标) | | ||
| 20 | +| `Stage` | μs | 迭代总耗时(wall time),约等于 `Computing + Communication(Not Overlapped) + Free + Bubble + Preparing` | | ||
| 21 | +| `Bubble` | μs | 流水线气泡(Pipeline Parallelism 场景下),非 PP 场景为 0 | | ||
| 22 | +| `Communication(Not Overlapped and Exclude Receive)` | μs | 排除点对点 Receive 后的未重叠通信时间(用于更精确的集合通信分析) | | ||
| 23 | +| `Preparing` | μs | 迭代开始前的准备/同步时间 | | ||
| 24 | + | ||
| 25 | +**关键语义区分:** | ||
| 26 | +- `Free Time` ≠ `device bubble`:前者是 step 级聚合(Host 未及时下发),后者是 kernel 级间隙(两个 kernel 之间的设备空闲);成因不同,优化路径不同 | ||
| 27 | +- `Communication(Not Overlapped)` 是真正的通信瓶颈指标,`Communication`(总通信)包含了已被遮蔽的部分,不能直接用来判断通信代价 | ||
| 28 | +- `Stage` 是实际耗时,可用于多卡负载均衡对比 | ||
| 29 | + | ||
| 30 | +--- | ||
| 31 | + | ||
| 32 | +## 二、op_statistic.csv(9 列) | ||
| 33 | + | ||
| 34 | +记录整个采集窗口内各算子类型的聚合统计,**不是 per-step**。 | ||
| 35 | + | ||
| 36 | +| 列名 | 单位 | 含义 | | ||
| 37 | +|---|---|---| | ||
| 38 | +| `Device_id` | — | NPU 设备编号 | | ||
| 39 | +| `OP Type` | — | 算子类型名称(如 MatMul、RmsNorm、GroupedMatmul) | | ||
| 40 | +| `Core Type` | — | 执行核心类型:`AI_CORE`(Cube 核)/ `AI_VECTOR_CORE`(Vector 核)/ `MIX_AIC`(混合 AI Core)/ `MIX_AIV`(混合 AI Vector)/ `AI_CPU`(CPU 侧回退)/ `COMMUNICATION`(HCCL 通信) | | ||
| 41 | +| `Count` | 次 | 整个采集窗口内该算子类型的执行次数 | | ||
| 42 | +| `Total Time(us)` | μs | 所有执行的 Duration 之和(不含 Wait Time) | | ||
| 43 | +| `Min Time(us)` | μs | 单次最短 Duration | | ||
| 44 | +| `Avg Time(us)` | μs | 平均 Duration | | ||
| 45 | +| `Max Time(us)` | μs | 单次最长 Duration(Max/Avg >> 1 说明存在异常慢执行) | | ||
| 46 | +| `Ratio(%)` | % | 该算子类型占**所有算子 Total Time 之和**的比例(注意:不是占 Stage 的比例) | | ||
| 47 | + | ||
| 48 | +**注意:** `Ratio(%)` 是 kernel 维度的时间占比,由于多 stream 并发,`sum(Total Time)` 可能远大于实际 Stage 耗时。不要直接用 `Ratio(%)` 推断对 wall time 的贡献,需结合 `step_trace_time.csv` 交叉验证。 | ||
| 49 | + | ||
| 50 | +**AI_CPU 出现即为告警**:正常情况下算子应运行在 AI_CORE 或 AI_VECTOR_CORE,若 `Core Type = AI_CPU` 说明存在 AICPU 回退,会暴露在设备执行时间线上。 | ||
| 51 | + | ||
| 52 | +--- | ||
| 53 | + | ||
| 54 | +## 三、kernel_details.csv(两套 Schema 版本) | ||
| 55 | + | ||
| 56 | +最细粒度的 kernel 级别 profiling 数据,每行对应一次 kernel 调用。 | ||
| 57 | + | ||
| 58 | +### 版本检测规则 | ||
| 59 | +读取 CSV 头部: | ||
| 60 | +- **存在 `cube_utilization(%)` 列** → **V2 Schema**(支持详细流水线利用率分析) | ||
| 61 | +- **不存在该列** → **V1 Schema**(仅支持基础 MFU 估算) | ||
| 62 | + | ||
| 63 | +### 公共列(V1 和 V2 均有) | ||
| 64 | + | ||
| 65 | +| 列名 | 单位 | 含义 | | ||
| 66 | +|---|---|---| | ||
| 67 | +| `Step Id` | — | 所属迭代编号(对应 step_trace_time 的 Step 列) | | ||
| 68 | +| `Device_id` | — | NPU 设备编号 | | ||
| 69 | +| `Model ID` | — | 图编号 | | ||
| 70 | +| `Task ID` | — | 任务 ID | | ||
| 71 | +| `Stream ID` | — | 所属 stream(多 stream 并发时用于分组) | | ||
| 72 | +| `Name` | — | kernel 名称(包含算子类型信息,如 `MatMul`、`HcomAllReduce`) | | ||
| 73 | +| `Type` | — | kernel 类别(如 `AI_CORE`、`AI_CPU`、`HCCL`) | | ||
| 74 | +| `OP State` | — | 算子状态:`dynamic`(动态 shape)/ `static`(静态 shape) | | ||
| 75 | +| `Accelerator Core` | — | 执行核心类型(`AI_CORE` / `AI_VECTOR_CORE` / `MIX_AIC` / `MIX_AIV`) | | ||
| 76 | +| `Start Time(us)` | μs | kernel 在设备上的绝对开始时间戳(用于构造时间轴、计算空泡) | | ||
| 77 | +| `Duration(us)` | μs | kernel 在设备上的实际执行时长(不含等待) | | ||
| 78 | +| `Wait Time(us)` | μs | kernel 在队列中等待(未执行)的时间;若 wait 远大于 duration,可能是等待锚点 | | ||
| 79 | +| `Input Shapes` | — | 输入张量 shape,格式如 `"M,K;K,N"` 或为空(shape 未记录) | | ||
| 80 | +| `Output Shapes` | — | 输出张量 shape | | ||
| 81 | +| `Input Data Types` | — | 输入数据类型(如 `float16;float16`) | | ||
| 82 | +| `Input Formats` | — | 输入内存格式(如 `ND;ND`) | | ||
| 83 | +| `Block Dim` | — | AI Core 块数(并行度) | | ||
| 84 | +| `HF32 Eligible` | — | 是否可使用 HF32 | | ||
| 85 | + | ||
| 86 | +### V2 专有列(AI_CORE 类 kernel) | ||
| 87 | + | ||
| 88 | +| 列名 | 单位 | 含义 | | ||
| 89 | +|---|---|---| | ||
| 90 | +| `aicore_time(us)` | μs | AI Core 有效执行时间 | | ||
| 91 | +| `aic_total_cycles` | 周期 | AI Core 总周期数 | | ||
| 92 | +| `aic_mac_time(us)` | μs | Cube(矩阵乘)单元执行时间 | | ||
| 93 | +| `aic_mac_ratio` | 0–1 | MAC 时间占 aicore_time 的比例;高 → Compute Bound | | ||
| 94 | +| `aic_scalar_time(us)` | μs | 标量单元执行时间 | | ||
| 95 | +| `aic_scalar_ratio` | 0–1 | 标量时间占比 | | ||
| 96 | +| `aic_mte1_time(us)` | μs | MTE1(L1→L0 数据搬移)时间 | | ||
| 97 | +| `aic_mte1_ratio` | 0–1 | MTE1 时间占比 | | ||
| 98 | +| `aic_mte2_time(us)` | μs | MTE2(L2/HBM→L1 读取)时间 | | ||
| 99 | +| `aic_mte2_ratio` | 0–1 | MTE2 时间占比;高 → Memory Bound(权重读取瓶颈) | | ||
| 100 | +| `aic_fixpipe_time(us)` | μs | FixPipe(后处理:格式转换、激活)时间 | | ||
| 101 | +| `aic_fixpipe_ratio` | 0–1 | FixPipe 时间占比 | | ||
| 102 | +| `aic_icache_miss_rate` | % | 指令缓存缺失率 | | ||
| 103 | +| `cube_utilization(%)` | % | Cube 单元有效执行时间占 kernel 总时间的比例(流水线效率,非 MFU) | | ||
| 104 | + | ||
| 105 | +### V2 专有列(AI_VECTOR_CORE 类 kernel) | ||
| 106 | + | ||
| 107 | +| 列名 | 单位 | 含义 | | ||
| 108 | +|---|---|---| | ||
| 109 | +| `aiv_time(us)` | μs | Vector Core 有效执行时间 | | ||
| 110 | +| `aiv_total_cycles` | 周期 | Vector Core 总周期数 | | ||
| 111 | +| `aiv_vec_time(us)` | μs | 向量计算时间 | | ||
| 112 | +| `aiv_vec_ratio` | 0–1 | 向量计算占比 | | ||
| 113 | +| `aiv_scalar_time(us)` | μs | 标量计算时间 | | ||
| 114 | +| `aiv_scalar_ratio` | 0–1 | 标量计算占比 | | ||
| 115 | +| `aiv_mte2_time(us)` | μs | GM→UB(全局内存到统一缓冲区)读取时间 | | ||
| 116 | +| `aiv_mte2_ratio` | 0–1 | 内存读取占比 | | ||
| 117 | +| `aiv_mte3_time(us)` | μs | UB→GM(统一缓冲区到全局内存)写入时间 | | ||
| 118 | +| `aiv_mte3_ratio` | 0–1 | 内存写入占比 | | ||
| 119 | +| `aiv_icache_miss_rate` | % | Vector Core 指令缓存缺失率 | | ||
| 120 | + | ||
| 121 | +### V1 专有列 | ||
| 122 | + | ||
| 123 | +| 列名 | 含义 | | ||
| 124 | +|---|---| | ||
| 125 | +| `aic_mac_fp16_ratio` | FP16 MAC 占比(V1 的粗粒度版本) | | ||
| 126 | +| `aic_mac_int8_ratio` | INT8 MAC 占比 | | ||
| 127 | +| `aic_cube_fops` | AI Core Cube 单元完成的浮点运算数(用于 V1 MFU 估算) | | ||
| 128 | +| `aiv_vec_fp32_ratio` | Vector Core FP32 计算占比 | | ||
| 129 | +| `aiv_vec_fp16_ratio` | Vector Core FP16 计算占比 | | ||
| 130 | +| `aiv_vector_fops` | Vector Core 完成的浮点运算数 | | ||
| 131 | + | ||
| 132 | +**注意**:V1 schema 下 MFU 只能用 `aic_cube_fops / Duration_us` 估算,置信度低于 V2,需在报告中明确标注。 | ||
| 133 | + | ||
| 134 | +--- | ||
| 135 | + | ||
| 136 | +## 四、operator_details.csv(9 列) | ||
| 137 | + | ||
| 138 | +记录 PyTorch 算子维度的 Host 侧与设备侧时间拆分,用于 Host-Device 时间归因。 | ||
| 139 | + | ||
| 140 | +| 列名 | 单位 | 含义 | | ||
| 141 | +|---|---|---| | ||
| 142 | +| `Name` | — | PyTorch 算子名称(如 `aten::matmul`、`aten::_to_copy`) | | ||
| 143 | +| `Input Shapes` | — | 输入张量 shape(可为空) | | ||
| 144 | +| `Call Stack` | — | Python 调用栈(用于定位代码行) | | ||
| 145 | +| `Host Self Duration(us)` | μs | 在 Host CPU 上自身执行时间(不含子调用) | | ||
| 146 | +| `Host Total Duration(us)` | μs | 在 Host CPU 上含子调用的总执行时间 | | ||
| 147 | +| `Device Self Duration(us)` | μs | 在 NPU 设备上自身执行时间 | | ||
| 148 | +| `Device Total Duration(us)` | μs | 在 NPU 设备上含子调用的总执行时间 | | ||
| 149 | +| `Device Self Duration With AICore(us)` | μs | 仅 AI Core 部分的设备自身时间 | | ||
| 150 | +| `Device Total Duration With AICore(us)` | μs | 仅 AI Core 部分的设备总时间 | | ||
| 151 | + | ||
| 152 | +**使用场景**:当 `Host Self Duration` 显著高于 `Device Self Duration` 时,说明该算子存在 Host Bound 风险(Host 计算未被设备执行隐藏)。`Device Duration = 0` 的行通常为纯 Host 侧操作(内存分配、复制等)。 | ||
| 153 | + | ||
| 154 | +--- | ||
| 155 | + | ||
| 156 | +## 五、communication.json(嵌套 JSON) | ||
| 157 | + | ||
| 158 | +记录每个 step 内所有集合通信操作的详细时序和带宽信息。 | ||
| 159 | + | ||
| 160 | +### 顶层结构 | ||
| 161 | +```json | ||
| 162 | +{ | ||
| 163 | + "step3": { | ||
| 164 | + "p2p": {}, | ||
| 165 | + "collective": { | ||
| 166 | + "<op_name>": { ... } | ||
| 167 | + } | ||
| 168 | + }, | ||
| 169 | + "step4": { ... } | ||
| 170 | +} | ||
| 171 | +``` | ||
| 172 | + | ||
| 173 | +### 操作名格式 | ||
| 174 | +``` | ||
| 175 | +HcomAllReduce_<hash>_<sequence>@<group_id> | ||
| 176 | +HcomAllGather_<hash>_<sequence>@<group_id> | ||
| 177 | +HcomReduceScatter_<hash>_<sequence>@<group_id> | ||
| 178 | +hcom_allReduce_<hash>_<sequence>@<group_id> | ||
| 179 | +``` | ||
| 180 | +从前缀提取操作类型:`HcomAllReduce` → AllReduce,`HcomAllGather` → AllGather,`HcomReduceScatter` → ReduceScatter。 | ||
| 181 | + | ||
| 182 | +### 每个操作的内部结构 | ||
| 183 | + | ||
| 184 | +**`Communication Time Info`**(时间信息): | ||
| 185 | + | ||
| 186 | +| 字段 | 单位 | 含义 | | ||
| 187 | +|---|---|---| | ||
| 188 | +| `Start Timestamp(us)` | μs | 通信操作开始时间戳 | | ||
| 189 | +| `Elapse Time(ms)` | ms | 通信操作总耗时(wall time) | | ||
| 190 | +| `Transit Time(ms)` | ms | 数据在网络中实际传输的时间 | | ||
| 191 | +| `Wait Time(ms)` | ms | 等待其他 Rank 就绪的时间 | | ||
| 192 | +| `Synchronization Time(ms)` | ms | 同步栅栏时间 | | ||
| 193 | +| `Idle Time(ms)` | ms | 通信期间的空闲时间 | | ||
| 194 | +| `Wait Time Ratio` | 0–1 | Wait Time / Elapse Time | | ||
| 195 | +| `Synchronization Time Ratio` | 0–1 | Sync Time / Elapse Time | | ||
| 196 | + | ||
| 197 | +**`Communication Bandwidth Info`**(带宽信息,按传输介质分组): | ||
| 198 | + | ||
| 199 | +各介质(RDMA / HCCS / PCIE / SDMA / SIO)均包含: | ||
| 200 | + | ||
| 201 | +| 字段 | 单位 | 含义 | | ||
| 202 | +|---|---|---| | ||
| 203 | +| `Transit Size(MB)` | MB | 通过该介质传输的数据量 | | ||
| 204 | +| `Transit Time(ms)` | ms | 该介质的传输时间 | | ||
| 205 | +| `Bandwidth(GB/s)` | GB/s | 实测带宽(= Transit Size / Transit Time) | | ||
| 206 | +| `Large Packet Ratio` | 0–1 | 大包占比(小包多则带宽效率低) | | ||
| 207 | + | ||
| 208 | +**重要提示**:若所有介质的 `Transit Size(MB)` 均为 0 或 `Bandwidth(GB/s)` 均为 0,说明这是**单 Rank 采集**,无跨 Rank 通信数据,此时只能报告 `Elapse Time`,不应报告带宽数字。 | ||
| 209 | + | ||
| 210 | +--- | ||
| 211 | + | ||
| 212 | +## 六、communication_matrix.json | ||
| 213 | + | ||
| 214 | +结构与 `communication.json` 相同(按 step 嵌套),记录 Rank 对之间的流量矩阵。 | ||
| 215 | + | ||
| 216 | +- **单 Rank 采集时**:`collective` 和 `p2p` 均为空对象 `{}` | ||
| 217 | +- **多 Rank 采集时**:包含各 Rank 对的通信数据(可用于识别热点链路和负载不均) | ||
| 218 | + | ||
| 219 | +--- | ||
| 220 | + | ||
| 221 | +## 七、trace_view.json(可选) | ||
| 222 | + | ||
| 223 | +Chrome Trace Format,记录 Host 与 Device 的时间线事件。用于 Host/Device Bound 软归因和 wait pollution 交叉验证。 | ||
| 224 | + | ||
| 225 | +| 字段 | 单位 | 含义 | | ||
| 226 | +|---|---|---| | ||
| 227 | +| `ph` | — | 事件类型,常见 `X` 为完整事件 | | ||
| 228 | +| `ts` | μs | 事件开始时间戳 | | ||
| 229 | +| `dur` | μs | 事件持续时间 | | ||
| 230 | +| `name` | — | 事件名称,如 `ProfilerStep#N`、`aten::to`、`aclrtMemcpy`、`HcomAllReduce` | | ||
| 231 | +| `cat` | — | 事件类别,如 `cpu_op`、`python_function`、`user_annotation`、`kernel`、`communication`、`AscendCL` | | ||
| 232 | +| `pid` / `tid` | — | 进程 / 线程 / stream 标识 | | ||
| 233 | +| `args` | — | 附加信息,可能包含 Call Stack、Input Shapes 等 | | ||
| 234 | + | ||
| 235 | +**Host evidence 分类:** | ||
| 236 | + | ||
| 237 | +| 事件族 | 匹配示例 | 诊断含义 | | ||
| 238 | +|---|---|---| | ||
| 239 | +| sync/H2D | `aten::to`、`aten::_to_copy`、`aclrtMemcpy*`、`aclrtSynchronize*`、`HostToDevice` | 可能是同步或 Host↔Device 拷贝导致设备空闲 | | ||
| 240 | +| communication marker | `c10d`、`Hccl`、`hcom`、`StreamWaitEvent`、`Notify_Wait` | 可能是通信等待或同步 | | ||
| 241 | +| host launch | `cpu_op`、`python_function`、`AscendCL@*` | 可能是 Host 下发延迟、Python 调度或 runtime API 开销 | | ||
| 242 | + | ||
| 243 | +若 `trace_view.json` 缺失,不得给出确定 Host 根因,只能基于 `free_ratio` / `underfeed_ratio` 给出 Host-originated risk 或 soft label。 | ||
| 244 | + | ||
| 245 | +--- | ||
| 246 | + | ||
| 247 | +## 八、msprof op 输出(OPPROF_*,可选) | ||
| 248 | + | ||
| 249 | +`msprof op --kernel-name=<kernel>` 会生成 `OPPROF_<timestamp>/`,用于单 kernel PMU 级分析。 | ||
| 250 | + | ||
| 251 | +| 文件 | 用途 | | ||
| 252 | +|---|---| | ||
| 253 | +| `OpBasicInfo.csv` | 算子名称、类型、`Task Duration(us)`、`Block Dim` | | ||
| 254 | +| `ArithmeticUtilization.csv` | Cube/Vector FLOPs、Cube/Vector 占比 | | ||
| 255 | +| `Memory.csv` | GM→UB、UB→GM 数据量、带宽、带宽利用率 | | ||
| 256 | +| `MemoryL0.csv` / `MemoryUB.csv` | L0/UB 读写带宽细节 | | ||
| 257 | +| `PipeUtilization.csv` | 计算/搬运流水线占比 | | ||
| 258 | +| `ResourceConflictRatio.csv` | UB Bank Group / Bank conflict / 资源冲突率 | | ||
| 259 | +| `L2Cache.csv` | L2 Cache 命中率 | | ||
| 260 | + | ||
| 261 | +关键字段: | ||
| 262 | + | ||
| 263 | +| 文件 | 字段 | 含义 | | ||
| 264 | +|---|---|---| | ||
| 265 | +| `ArithmeticUtilization.csv` | `aic_cube_fops`、`aiv_vector_fops` | FLOPs,用于算术强度 | | ||
| 266 | +| `ArithmeticUtilization.csv` | `aic_cube_ratio`、`aiv_vec_ratio` | Cube / Vector 计算占比 | | ||
| 267 | +| `Memory.csv` | `GM_to_UB_datas(KB)`、`UB_to_GM_datas(KB)` | GM↔UB 数据量 | | ||
| 268 | +| `Memory.csv` | `GM_to_UB_bw_usage_rate(%)`、`UB_to_GM_bw_usage_rate(%)` | 带宽利用率 | | ||
| 269 | +| `ResourceConflictRatio.csv` | `aiv_vec_bankgroup_cflt_ratio`、`aiv_vec_bank_cflt_ratio` | Bank conflict 风险 | | ||
| 270 | + | ||
| 271 | +**优先级**:若存在 msprof op 输出,算子级 compute/memory bound 以 PMU 数据为高置信证据;否则退化为 `kernel_details.csv` V2 的 `aic_*` / `aiv_*` ratio。 | ||
| 272 | + | ||
| 273 | +--- | ||
| 274 | + | ||
| 275 | +## 九、目录结构说明 | ||
| 276 | + | ||
| 277 | +### 标准结构(gemma / qwen7b / ds3.2) | ||
| 278 | +``` | ||
| 279 | +<model>/ASCEND_PROFILER_OUTPUT/ | ||
| 280 | +├── kernel_details.csv | ||
| 281 | +├── op_statistic.csv | ||
| 282 | +├── operator_details.csv | ||
| 283 | +├── step_trace_time.csv | ||
| 284 | +├── communication.json | ||
| 285 | +├── communication_matrix.json | ||
| 286 | +├── trace_view.json (大文件,本 Skill 仅按需使用) | ||
| 287 | +├── api_statistic.csv | ||
| 288 | +└── *.db (SQLite 分析数据库,可选) | ||
| 289 | +``` | ||
| 290 | + | ||
| 291 | +### 嵌套结构(longcat 风格) | ||
| 292 | +``` | ||
| 293 | +<run_id>/ | ||
| 294 | +├── ASCEND_PROFILER_OUTPUT/ ← Phase 0 自动扫描子目录找到此处 | ||
| 295 | +│ ├── kernel_details.csv | ||
| 296 | +│ └── ... | ||
| 297 | +├── FRAMEWORK/ | ||
| 298 | +├── logs/ | ||
| 299 | +└── PROF_*/ | ||
| 300 | +``` | ||
| 301 | +Phase 0 发现嵌套结构时,应扫描 1–2 级子目录寻找 `ASCEND_PROFILER_OUTPUT`,找到后以该目录为分析根目录,并在报告中注明实际路径。 | ||
| @@ -0,0 +1,770 @@ | |||
| 1 | +#!/usr/bin/env python3 | ||
| 2 | +# -*- coding: utf-8 -*- | ||
| 3 | +# ---------------------------------------------------------------------------- | ||
| 4 | +# Copyright (c) 2025 Huawei Technologies Co., Ltd. | ||
| 5 | +# | ||
| 6 | +# Licensed under the Apache License, Version 2.0 (the "License"); | ||
| 7 | +# you may not use this file except in compliance with the License. | ||
| 8 | +# You may obtain a copy of the License at | ||
| 9 | +# | ||
| 10 | +# http://www.apache.org/licenses/LICENSE-2.0 | ||
| 11 | +# | ||
| 12 | +# Unless required by applicable law or agreed to in writing, software | ||
| 13 | +# distributed under the License is distributed on an "AS IS" BASIS, | ||
| 14 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| 15 | +# See the License for the specific language governing permissions and | ||
| 16 | +# limitations under the License. | ||
| 17 | +# ---------------------------------------------------------------------------- | ||
| 18 | +""" | ||
| 19 | +NPU 性能分析 HTML 报告生成器 | ||
| 20 | + | ||
| 21 | +用法: | ||
| 22 | + python3 generate_html.py <analysis_data.json> <output.html> | ||
| 23 | + | ||
| 24 | +analysis_data.json 由 Claude 在完成分析后写出,本脚本负责将其渲染为美观的 HTML 报告。 | ||
| 25 | +""" | ||
| 26 | + | ||
| 27 | +import json | ||
| 28 | +import html | ||
| 29 | +import logging | ||
| 30 | +import sys | ||
| 31 | +import argparse | ||
| 32 | +from datetime import datetime, timezone | ||
| 33 | + | ||
| 34 | +logger = logging.getLogger(__name__) | ||
| 35 | + | ||
| 36 | +# ── 优先级颜色映射 ──────────────────────────────────────────────────────────── | ||
| 37 | +PRIORITY_COLOR = { | ||
| 38 | + 'P0': ('#e74c3c', '#fff'), | ||
| 39 | + 'P1': ('#e67e22', '#fff'), | ||
| 40 | + 'P2': ('#f1c40f', '#333'), | ||
| 41 | + 'P3': ('#3498db', '#fff'), | ||
| 42 | + 'OK': ('#2ecc71', '#333'), | ||
| 43 | +} | ||
| 44 | +CARD_BORDER = { | ||
| 45 | + 'P0': '#e74c3c', 'P1': '#e67e22', 'P2': '#f1c40f', | ||
| 46 | + 'P3': '#3498db', 'OK': '#2ecc71', 'INFO': '#95a5a6', | ||
| 47 | +} | ||
| 48 | + | ||
| 49 | + | ||
| 50 | +def badge(label, priority=None): | ||
| 51 | + if priority and priority in PRIORITY_COLOR: | ||
| 52 | + bg, fg = PRIORITY_COLOR[priority] | ||
| 53 | + else: | ||
| 54 | + bg, fg = '#95a5a6', '#fff' | ||
| 55 | + return (f'<span style="background:{bg};color:{fg};padding:2px 8px;' | ||
| 56 | + f'border-radius:12px;font-size:11px;font-weight:700">{html.escape(str(label))}</span>') | ||
| 57 | + | ||
| 58 | + | ||
| 59 | +def progress_bar(value_pct, warn=15, severe=30, higher_is_bad=True, width_scale=1.8): | ||
| 60 | + v = float(value_pct) | ||
| 61 | + if higher_is_bad: | ||
| 62 | + color = '#e74c3c' if v >= severe else '#e67e22' if v >= warn else '#2ecc71' | ||
| 63 | + else: | ||
| 64 | + color = '#2ecc71' if v >= warn else '#e67e22' if v >= severe else '#e74c3c' | ||
| 65 | + w = max(4, int(min(v, 100) * width_scale)) | ||
| 66 | + return (f'<div style="display:flex;align-items:center;gap:8px">' | ||
| 67 | + f'<div style="height:10px;width:{w}px;background:{color};border-radius:5px;min-width:4px"></div>' | ||
| 68 | + f'<span style="font-size:12px;color:var(--text-secondary)">{v:.1f}%</span></div>') | ||
| 69 | + | ||
| 70 | + | ||
| 71 | +def cell_style(value, warn, severe, higher_is_bad=True, is_pct=False): | ||
| 72 | + v = float(value) * (100 if is_pct else 1) | ||
| 73 | + if higher_is_bad: | ||
| 74 | + if v >= severe: | ||
| 75 | + return 'background:#fdecea;color:#e74c3c;font-weight:600' | ||
| 76 | + if v >= warn: | ||
| 77 | + return 'background:#fff8e1;color:#e67e22;font-weight:600' | ||
| 78 | + return 'background:#f0faf4;color:#27ae60' | ||
| 79 | + else: | ||
| 80 | + if v <= severe: | ||
| 81 | + return 'background:#fdecea;color:#e74c3c;font-weight:600' | ||
| 82 | + if v <= warn: | ||
| 83 | + return 'background:#fff8e1;color:#e67e22;font-weight:600' | ||
| 84 | + return 'background:#f0faf4;color:#27ae60' | ||
| 85 | + | ||
| 86 | + | ||
| 87 | +def fmt_us(v): | ||
| 88 | + v = float(v) | ||
| 89 | + if v >= 1000: | ||
| 90 | + return f'{v:,.0f}' | ||
| 91 | + return f'{v:.1f}' | ||
| 92 | + | ||
| 93 | + | ||
| 94 | +def section(anchor, title, body, open_=False): | ||
| 95 | + open_attr = ' open' if open_ else '' | ||
| 96 | + return f''' | ||
| 97 | +<section id="{anchor}"> | ||
| 98 | +<details{open_attr}> | ||
| 99 | +<summary>{html.escape(title)}</summary> | ||
| 100 | +<div class="sb">{body}</div> | ||
| 101 | +</details> | ||
| 102 | +</section>''' | ||
| 103 | + | ||
| 104 | + | ||
| 105 | +def note(text, kind='info'): | ||
| 106 | + colors = {'info': ('var(--note-info-bg)', 'var(--note-info-border)'), | ||
| 107 | + 'warn': ('var(--note-warn-bg)', 'var(--note-warn-border)'), | ||
| 108 | + 'alert': ('var(--note-alert-bg)', 'var(--note-alert-border)')} | ||
| 109 | + bg, border = colors.get(kind, colors['info']) | ||
| 110 | + return (f'<div style="background:{bg};border-left:4px solid {border};' | ||
| 111 | + f'padding:10px 14px;margin:10px 0;border-radius:0 6px 6px 0;' | ||
| 112 | + f'font-size:13px;line-height:1.6">{text}</div>') | ||
| 113 | + | ||
| 114 | + | ||
| 115 | +def skip_div(reason): | ||
| 116 | + return f'<div style="padding:12px 0;color:#95a5a6;font-style:italic">⏭️ {html.escape(reason)}</div>' | ||
| 117 | + | ||
| 118 | + | ||
| 119 | +# ── 各章节渲染 ──────────────────────────────────────────────────────────────── | ||
| 120 | + | ||
| 121 | +def render_meta(d): | ||
| 122 | + m = d.get('meta', {}) | ||
| 123 | + rows = [ | ||
| 124 | + ('输入路径', html.escape(m.get('data_path', '—'))), | ||
| 125 | + ('实际数据路径', f'<code>{html.escape(m.get("actual_path", m.get("data_path","—")))}</code>'), | ||
| 126 | + ('芯片型号', html.escape(m.get('chip', 'Ascend 910B3'))), | ||
| 127 | + ('Schema 版本', f'<strong>{html.escape(m.get("schema_version","—"))}</strong>'), | ||
| 128 | + ('采集步数', html.escape(str(m.get('steps_desc', ', '.join(str(s) for s in m.get('steps', [])))))), | ||
| 129 | + ('设备数', html.escape(str(m.get('devices_desc', ', '.join(str(x) for x in m.get('devices', [])))))), | ||
| 130 | + ('文件清单', html.escape(', '.join(m.get('files_present', [])))), | ||
| 131 | + ('数据质量说明', html.escape(m.get('quality_notes', '—'))), | ||
| 132 | + ] | ||
| 133 | + trs = ''.join(f'<tr><td style="width:140px;font-weight:600;color:var(--text-primary)">' | ||
| 134 | + f'{k}</td><td>{v}</td></tr>' for k, v in rows) | ||
| 135 | + return f'<table>{trs}</table>' | ||
| 136 | + | ||
| 137 | + | ||
| 138 | +def render_summary(d): | ||
| 139 | + diag = d.get('bottleneck_diagnosis', []) | ||
| 140 | + # Summary cards (top 3) | ||
| 141 | + cards_html = '' | ||
| 142 | + if not diag: | ||
| 143 | + cards_html += _card('OK', '整体健康', '无已知瓶颈', '所有指标在正常范围内') | ||
| 144 | + else: | ||
| 145 | + for item in diag[:3]: | ||
| 146 | + p = item.get('priority', 'P3') | ||
| 147 | + lbl = item.get('label', '') | ||
| 148 | + ev = item.get('evidence', '')[:80] | ||
| 149 | + cards_html += _card(p, lbl, '', ev) | ||
| 150 | + # Table | ||
| 151 | + trs = '' | ||
| 152 | + for item in diag: | ||
| 153 | + p = item.get('priority', 'P3') | ||
| 154 | + trs += (f'<tr><td>{badge(p, p)}</td><td>{html.escape(item.get("dimension",""))}</td>' | ||
| 155 | + f'<td>{html.escape(item.get("evidence","")[:120])}</td></tr>') | ||
| 156 | + if not trs: | ||
| 157 | + trs = '<tr><td colspan="3" style="color:#27ae60">✅ 无显著瓶颈</td></tr>' | ||
| 158 | + table = f'<table><tr><th>优先级</th><th>维度</th><th>核心发现</th></tr>{trs}</table>' | ||
| 159 | + return f'<div style="display:flex;gap:14px;flex-wrap:wrap;margin-bottom:16px">{cards_html}</div>{table}' | ||
| 160 | + | ||
| 161 | + | ||
| 162 | +def _card(priority, title, subtitle, desc): | ||
| 163 | + border = CARD_BORDER.get(priority, '#ccc') | ||
| 164 | + bdg = badge(priority, priority) | ||
| 165 | + return (f'<div class="summary-card" style="border-left-color:{border}">' | ||
| 166 | + f'{bdg}<div class="card-subtitle">{html.escape(str(subtitle))}</div>' | ||
| 167 | + f'<div class="card-title">{html.escape(str(title))}</div>' | ||
| 168 | + f'<div class="card-desc">{html.escape(str(desc))}</div></div>') | ||
| 169 | + | ||
| 170 | + | ||
| 171 | +def render_bound_classification(d): | ||
| 172 | + bc = d.get('bound_classification', {}) | ||
| 173 | + if not bc: | ||
| 174 | + return skip_div('bound_classification 数据缺失;旧版 analysis_data.json 可忽略本章节') | ||
| 175 | + priority = bc.get('priority', 'P3') | ||
| 176 | + overall = bc.get('overall_bound', 'INSUFFICIENT_EVIDENCE') | ||
| 177 | + device_type = bc.get('device_bound_type') or '—' | ||
| 178 | + primary = bc.get('primary_bottleneck', '—') | ||
| 179 | + secondary = ', '.join(str(x) for x in bc.get('secondary_bottlenecks', [])) or '—' | ||
| 180 | + confidence = bc.get('confidence', '—') | ||
| 181 | + facts = bc.get('facts', []) | ||
| 182 | + facts_html = ''.join(f'<li>{html.escape(str(x))}</li>' for x in facts) or '<li>—</li>' | ||
| 183 | + reasoning = html.escape(str(bc.get('reasoning', '—'))) | ||
| 184 | + counter = html.escape(str(bc.get('counter_evidence', '—'))) | ||
| 185 | + rows = [ | ||
| 186 | + ('整体 Bound', f'<code>{html.escape(str(overall))}</code> {badge(priority, priority)}'), | ||
| 187 | + ('Device 子类', f'<code>{html.escape(str(device_type))}</code>'), | ||
| 188 | + ('主瓶颈', html.escape(str(primary))), | ||
| 189 | + ('次要瓶颈', html.escape(str(secondary))), | ||
| 190 | + ('置信度', html.escape(str(confidence))), | ||
| 191 | + ] | ||
| 192 | + trs = ''.join(f'<tr><td style="width:130px;font-weight:600;color:var(--text-primary)">' | ||
| 193 | + f'{k}</td><td>{v}</td></tr>' for k, v in rows) | ||
| 194 | + host = bc.get('host_evidence', {}) | ||
| 195 | + host_html = '' | ||
| 196 | + if host: | ||
| 197 | + labels = ', '.join(str(x) for x in host.get('soft_labels', [])) or '—' | ||
| 198 | + host_rows = [ | ||
| 199 | + ('Host evidence', '可用' if host.get('available') else '不可用'), | ||
| 200 | + ('sync/H2D overlap', _fmt_ratio_or_dash(host.get('sync_or_h2d_overlap_ratio'))), | ||
| 201 | + ('comm marker overlap', _fmt_ratio_or_dash(host.get('comm_marker_overlap_ratio'))), | ||
| 202 | + ('host visible coverage', _fmt_ratio_or_dash(host.get('host_visible_coverage_ratio'))), | ||
| 203 | + ('soft labels', html.escape(labels)), | ||
| 204 | + ] | ||
| 205 | + host_trs = ''.join(f'<tr><td>{k}</td><td>{v}</td></tr>' for k, v in host_rows) | ||
| 206 | + host_html = f'<h3>Host Evidence</h3><table>{host_trs}</table>' | ||
| 207 | + return ( | ||
| 208 | + f'<table>{trs}</table>' | ||
| 209 | + f'<h3>事实指标</h3><ul class="facts-list">{facts_html}</ul>' | ||
| 210 | + f'{note("<strong>诊断理由:</strong>" + reasoning, "info")}' | ||
| 211 | + f'{note("<strong>反证 / 降级条件:</strong>" + counter, "warn")}' | ||
| 212 | + f'{host_html}' | ||
| 213 | + ) | ||
| 214 | + | ||
| 215 | + | ||
| 216 | +def _fmt_ratio_or_dash(v): | ||
| 217 | + if v is None: | ||
| 218 | + return '—' | ||
| 219 | + try: | ||
| 220 | + return f'{float(v) * 100:.1f}%' | ||
| 221 | + except (TypeError, ValueError): | ||
| 222 | + return html.escape(str(v)) | ||
| 223 | + | ||
| 224 | + | ||
| 225 | +def _iter_step_rows(steps, warmup_steps): | ||
| 226 | + """渲染迭代效率每步表格行。""" | ||
| 227 | + trs = '' | ||
| 228 | + for s in steps: | ||
| 229 | + is_w = s.get('step') in warmup_steps or s.get('is_warmup', False) | ||
| 230 | + row_style = 'background:#fff3cd' if is_w else '' | ||
| 231 | + free_r = s.get('free_ratio', 0) * 100 | ||
| 232 | + comm_r = s.get('comm_no_ratio', 0) * 100 | ||
| 233 | + comp_r = s.get('computing_ratio', 0) * 100 | ||
| 234 | + over_r = s.get('overlap_ratio', 0) * 100 | ||
| 235 | + free_st = cell_style(free_r, 10, 20) | ||
| 236 | + comm_st = cell_style(comm_r, 15, 30) | ||
| 237 | + comp_st = cell_style(comp_r, 50, 30, higher_is_bad=False) | ||
| 238 | + remark = '⚠️ 预热步' if is_w else '正常步' | ||
| 239 | + trs += (f'<tr style="{row_style}">' | ||
| 240 | + f'<td>{s.get("step")}</td>' | ||
| 241 | + f'<td>{fmt_us(s.get("stage_us",0))}</td>' | ||
| 242 | + f'<td style="{comp_st}">{comp_r:.1f}%</td>' | ||
| 243 | + f'<td style="{comm_st}">{comm_r:.1f}%</td>' | ||
| 244 | + f'<td style="{free_st}">{free_r:.1f}%</td>' | ||
| 245 | + f'<td>{over_r:.2f}%</td>' | ||
| 246 | + f'<td>{remark}</td></tr>') | ||
| 247 | + return trs | ||
| 248 | + | ||
| 249 | + | ||
| 250 | +def render_iter_efficiency(d): | ||
| 251 | + ie = d.get('iteration_efficiency', {}) | ||
| 252 | + if not ie: | ||
| 253 | + return skip_div('iteration_efficiency 数据缺失') | ||
| 254 | + steps = ie.get('steps', []) | ||
| 255 | + warmup_steps = set(ie.get('warmup_steps', [])) | ||
| 256 | + # Step table | ||
| 257 | + header = ('<tr><th>Step</th><th>Stage (μs)</th><th>Computing%</th><th>CommNO%</th>' | ||
| 258 | + '<th>Free%</th><th>Overlap%</th><th>备注</th></tr>') | ||
| 259 | + trs = _iter_step_rows(steps, warmup_steps) | ||
| 260 | + step_table = f'<table>{header}{trs}</table>' | ||
| 261 | + # Averages | ||
| 262 | + avg = ie.get('avg', {}) | ||
| 263 | + avg_free = avg.get('free_ratio', 0) * 100 | ||
| 264 | + avg_comm = avg.get('comm_no_ratio', 0) * 100 | ||
| 265 | + avg_comp = avg.get('computing_ratio', 0) * 100 | ||
| 266 | + avg_stage = avg.get('stage_us', 0) | ||
| 267 | + norm_steps = ie.get('normal_steps', []) | ||
| 268 | + avg_note = '' | ||
| 269 | + if norm_steps: | ||
| 270 | + avg_note = note( | ||
| 271 | + f'正常步均值(Step {", ".join(str(s) for s in norm_steps)}):' | ||
| 272 | + f'Computing={avg_comp:.1f}%,CommNO={avg_comm:.1f}%,Free={avg_free:.1f}%,Stage≈{fmt_us(avg_stage)} μs', | ||
| 273 | + 'info' | ||
| 274 | + ) | ||
| 275 | + # Progress bars | ||
| 276 | + bars = f'''<table> | ||
| 277 | +<tr><th>指标</th><th>均值</th><th>可视化</th><th>判定</th></tr> | ||
| 278 | +<tr><td>Computing</td><td>{avg_comp:.1f}%</td> | ||
| 279 | + <td>{progress_bar(avg_comp,30,50,higher_is_bad=False,width_scale=1.4)}</td> | ||
| 280 | + <td>{"✅ 正常" if avg_comp>=50 else "⚠️ 偏低"}</td></tr> | ||
| 281 | +<tr><td>CommNO</td><td>{avg_comm:.1f}%</td><td>{progress_bar(avg_comm,15,30,width_scale=2)}</td> | ||
| 282 | + <td>{"🔴 严重" if avg_comm>=30 else "⚠️ 警告" if avg_comm>=15 else "✅ 正常"}</td></tr> | ||
| 283 | +<tr><td>Free</td><td>{avg_free:.1f}%</td><td>{progress_bar(avg_free,10,20,width_scale=2)}</td> | ||
| 284 | + <td>{"🔴 严重" if avg_free>=20 else "⚠️ 警告" if avg_free>=10 else "✅ 正常"}</td></tr> | ||
| 285 | +</table>''' | ||
| 286 | + # Bottleneck | ||
| 287 | + bn = ie.get('bottleneck', {}) | ||
| 288 | + bn_html = '' | ||
| 289 | + if bn and bn.get('label'): | ||
| 290 | + p = bn.get('priority', 'P3') | ||
| 291 | + bg_colors = {'P0': '#fef5f5', 'P1': '#fef9f0', 'P2': '#fefdf0', 'P3': '#f0f8ff'} | ||
| 292 | + bg = bg_colors.get(p, '#f0f8ff') | ||
| 293 | + bn_html = (f'<div style="background:{bg};border-radius:8px;padding:12px 16px;margin:10px 0">' | ||
| 294 | + f'<strong>{badge(p, p)} {html.escape(bn.get("label",""))}</strong>' | ||
| 295 | + f'<div style="margin-top:6px;font-size:13px;color:#555">{html.escape(bn.get("evidence",""))}</div>' | ||
| 296 | + f'</div>') | ||
| 297 | + return (f'<h3 style="margin:16px 0 8px;font-size:14px">每步时间拆分</h3>{step_table}{avg_note}' | ||
| 298 | + f'<h3 style="margin:16px 0 8px;font-size:14px">指标可视化</h3>{bars}{bn_html}') | ||
| 299 | + | ||
| 300 | + | ||
| 301 | +def render_op_hotspots(d): | ||
| 302 | + oh = d.get('operator_hotspots', {}) | ||
| 303 | + if not oh: | ||
| 304 | + return skip_div('operator_hotspots 数据缺失') | ||
| 305 | + ops = oh.get('top_ops', []) | ||
| 306 | + flag_map = { | ||
| 307 | + 'hotspot': badge('🔴 热点', 'P0'), | ||
| 308 | + 'watch': badge('⚠️ 关注', 'P2'), | ||
| 309 | + 'normal': badge('✅ 正常', 'OK'), | ||
| 310 | + 'moe_normal': badge('✅ MoE正常', 'OK'), | ||
| 311 | + 'aicpu': badge('🚨 AICPU', 'P0'), | ||
| 312 | + } | ||
| 313 | + header = ('<tr><th>#</th><th>算子类型</th><th>核心类型</th><th>次数</th><th>Total (μs)</th>' | ||
| 314 | + '<th>Avg (μs)</th><th>Ratio%</th><th>标注</th></tr>') | ||
| 315 | + trs = '' | ||
| 316 | + for op in ops: | ||
| 317 | + ratio = float(op.get('ratio', 0)) | ||
| 318 | + ratio_st = cell_style(ratio, 10, 20) | ||
| 319 | + flag_html = flag_map.get(op.get('flag', 'normal'), badge('✅', 'OK')) | ||
| 320 | + trs += (f'<tr><td>{op.get("rank")}</td>' | ||
| 321 | + f'<td><code>{html.escape(op.get("name",""))}</code></td>' | ||
| 322 | + f'<td>{html.escape(op.get("core_type",""))}</td>' | ||
| 323 | + f'<td>{op.get("count")}</td>' | ||
| 324 | + f'<td>{fmt_us(op.get("total_us",0))}</td>' | ||
| 325 | + f'<td>{fmt_us(op.get("avg_us",0))}</td>' | ||
| 326 | + f'<td style="{ratio_st}">{ratio:.2f}%</td>' | ||
| 327 | + f'<td>{flag_html}</td></tr>') | ||
| 328 | + table = f'<table>{header}{trs}</table>' | ||
| 329 | + # Core type breakdown | ||
| 330 | + ct = oh.get('core_type_breakdown', {}) | ||
| 331 | + ct_rows = '' | ||
| 332 | + for name, pct in ct.items(): | ||
| 333 | + ct_rows += (f'<tr><td>{html.escape(name)}</td>' | ||
| 334 | + f'<td>{progress_bar(float(pct),50,80,higher_is_bad=False,width_scale=1.5)}</td></tr>') | ||
| 335 | + ct_table = f'<h3 style="margin:16px 0 8px;font-size:14px">核心类型分布</h3><table>{ct_rows}</table>' if ct_rows else '' | ||
| 336 | + extra_note = note(oh['note']) if oh.get('note') else '' | ||
| 337 | + return f'<h3 style="margin:0 0 8px;font-size:14px">Top-10 算子</h3>{table}{ct_table}{extra_note}' | ||
| 338 | + | ||
| 339 | + | ||
| 340 | +def render_hw_util(d): | ||
| 341 | + hw = d.get('hardware_utilization', {}) | ||
| 342 | + if not hw or hw.get('skipped'): | ||
| 343 | + return skip_div(hw.get('skip_reason', '硬件利用率数据不可用')) | ||
| 344 | + kernels = hw.get('representative_kernels', []) | ||
| 345 | + header = '<tr><th>算子</th><th>Input Shapes</th><th>MFU</th><th>cube_utilization%</th><th>判定</th></tr>' | ||
| 346 | + trs = '' | ||
| 347 | + for k in kernels: | ||
| 348 | + mfu = k.get('mfu') | ||
| 349 | + cu = k.get('cube_utilization') | ||
| 350 | + mfu_str = f'{float(mfu)*100:.1f}%' if mfu is not None else '—' | ||
| 351 | + cu_str = f'{float(cu):.1f}%' if cu is not None else '—' | ||
| 352 | + cu_st = cell_style(float(cu) if cu else 50, 40, 20, higher_is_bad=False) if cu else '' | ||
| 353 | + verdict = k.get('verdict', '') | ||
| 354 | + trs += (f'<tr><td><code>{html.escape(k.get("name",""))}</code></td>' | ||
| 355 | + f'<td style="font-size:11px">{html.escape(k.get("input_shapes","")[:50])}</td>' | ||
| 356 | + f'<td>{mfu_str}</td>' | ||
| 357 | + f'<td style="{cu_st}">{cu_str}</td>' | ||
| 358 | + f'<td>{html.escape(verdict)}</td></tr>') | ||
| 359 | + table = f'<table>{header}{trs}</table>' if trs else '' | ||
| 360 | + extra = note(hw['note']) if hw.get('note') else '' | ||
| 361 | + avg_cu = hw.get('avg_cube_utilization') | ||
| 362 | + avg_cu_html = '' | ||
| 363 | + if avg_cu is not None: | ||
| 364 | + avg_cu_html = (f'<p style="margin:8px 0;font-size:13px">MatMul 平均 cube_utilization:' | ||
| 365 | + f'<strong>{float(avg_cu):.1f}%</strong> ' | ||
| 366 | + f'{badge("✅ 健康", "OK") if float(avg_cu)>=40 else badge("⚠️ 偏低","P2")}</p>') | ||
| 367 | + return f'{table}{avg_cu_html}{extra}' | ||
| 368 | + | ||
| 369 | + | ||
| 370 | +def render_operator_bound(d): | ||
| 371 | + oba = d.get('operator_bound_analysis', {}) | ||
| 372 | + if not oba: | ||
| 373 | + return skip_div('operator_bound_analysis 数据缺失;旧版 analysis_data.json 可忽略本章节') | ||
| 374 | + if oba.get('skipped'): | ||
| 375 | + return skip_div(oba.get('skip_reason', '算子级 Bound 分析被跳过')) | ||
| 376 | + src = html.escape(str(oba.get('source', '—'))) | ||
| 377 | + summary = oba.get('summary', {}) | ||
| 378 | + summary_rows = '' | ||
| 379 | + for key, label in [ | ||
| 380 | + ('compute_bound_count', 'Compute Bound'), | ||
| 381 | + ('memory_bound_count', 'Memory Bound'), | ||
| 382 | + ('scalar_bound_count', 'Scalar / Latency Bound'), | ||
| 383 | + ('fixpipe_bound_count', 'FixPipe Bound'), | ||
| 384 | + ('bank_conflict_count', 'Bank Conflict'), | ||
| 385 | + ]: | ||
| 386 | + if key in summary: | ||
| 387 | + summary_rows += f'<tr><td>{label}</td><td>{summary.get(key)}</td></tr>' | ||
| 388 | + summary_html = f'<p style="margin:6px 0 12px;color:var(--text-secondary)">数据源:<code>{src}</code></p>' | ||
| 389 | + if summary_rows: | ||
| 390 | + summary_html += f'<table><tr><th>分类</th><th>数量</th></tr>{summary_rows}</table>' | ||
| 391 | + | ||
| 392 | + ops = oba.get('top_operators', []) | ||
| 393 | + rows = '' | ||
| 394 | + for op in ops: | ||
| 395 | + bt = op.get('bound_type', 'UNKNOWN') | ||
| 396 | + p = _bound_priority(bt) | ||
| 397 | + rows += ( | ||
| 398 | + f'<tr><td><code>{html.escape(str(op.get("name","")))}</code></td>' | ||
| 399 | + f'<td>{html.escape(str(op.get("core_type","—")))}</td>' | ||
| 400 | + f'<td>{fmt_us(op.get("duration_us", 0))}</td>' | ||
| 401 | + f'<td>{badge(bt, p)}</td>' | ||
| 402 | + f'<td style="font-size:12px">{html.escape(str(op.get("evidence","")))}</td>' | ||
| 403 | + f'<td>{html.escape(str(op.get("confidence","")))}</td>' | ||
| 404 | + f'<td style="font-size:12px">{html.escape(str(op.get("recommendation_hint","")))}</td></tr>' | ||
| 405 | + ) | ||
| 406 | + op_table = '' | ||
| 407 | + if rows: | ||
| 408 | + op_table = ('<h3>Top 算子 Bound</h3><table><tr><th>算子</th><th>Core</th>' | ||
| 409 | + '<th>Duration(μs)</th><th>Bound</th><th>证据</th><th>置信度</th>' | ||
| 410 | + '<th>建议提示</th></tr>' + rows + '</table>') | ||
| 411 | + pmu_notes = note(html.escape(str(oba.get('pmu_notes'))), 'warn') if oba.get('pmu_notes') else '' | ||
| 412 | + return summary_html + op_table + pmu_notes | ||
| 413 | + | ||
| 414 | + | ||
| 415 | +def _bound_priority(bound_type): | ||
| 416 | + if bound_type in ('OP_MEMORY_BOUND', 'DEVICE_MEMORY_BOUND', 'OP_COMPUTE_BOUND', 'DEVICE_COMPUTE_BOUND'): | ||
| 417 | + return 'P1' | ||
| 418 | + if bound_type in ('OP_SCALAR_BOUND', 'DEVICE_LATENCY_BOUND', 'OP_FIXPIPE_BOUND', 'BANK_CONFLICT_RISK'): | ||
| 419 | + return 'P2' | ||
| 420 | + return 'P3' | ||
| 421 | + | ||
| 422 | + | ||
| 423 | +def render_comm(d): | ||
| 424 | + ce = d.get('communication_efficiency', {}) | ||
| 425 | + if not ce or ce.get('skipped'): | ||
| 426 | + return skip_div(ce.get('skip_reason', '无通信数据')) | ||
| 427 | + rows = [] | ||
| 428 | + or_ = ce.get('overlap_ratio_avg', 0) * 100 | ||
| 429 | + or_st = cell_style(or_, 20, 50, higher_is_bad=False) | ||
| 430 | + rows.append(('overlap_ratio(均值)', f'<span style="{or_st}">{or_:.2f}%</span>', '目标 > 50%')) | ||
| 431 | + coll = ce.get('collectives', {}) | ||
| 432 | + coll_str = ', '.join(f'{k}×{v}' for k, v in coll.items() if v) | ||
| 433 | + if coll_str: | ||
| 434 | + rows.append(('集合通信类型', coll_str, '')) | ||
| 435 | + bw = ce.get('bandwidth', {}) | ||
| 436 | + for medium, bw_val in bw.items(): | ||
| 437 | + if bw_val is not None: | ||
| 438 | + bw_st = '' | ||
| 439 | + if medium == 'RDMA' and float(bw_val) > 0: | ||
| 440 | + bw_st = cell_style(float(bw_val), 0.5, 0.5, higher_is_bad=False) | ||
| 441 | + rows.append((f'{medium} 带宽', f'<span style="{bw_st}">{float(bw_val):.2f} GB/s</span>', '')) | ||
| 442 | + single = ce.get('single_rank', False) | ||
| 443 | + if single: | ||
| 444 | + rows.append(('带宽数据', '⚠️ 单 Rank 采集,所有带宽数据为 0,不可用', '')) | ||
| 445 | + trs = ''.join(f'<tr><td style="font-weight:600">{k}</td><td>{v}</td>' | ||
| 446 | + f'<td style="color:#95a5a6">{r}</td></tr>' for k, v, r in rows) | ||
| 447 | + table = f'<table><tr><th>指标</th><th>数值</th><th>说明</th></tr>{trs}</table>' | ||
| 448 | + extra = note(ce['note']) if ce.get('note') else '' | ||
| 449 | + return f'{table}{extra}' | ||
| 450 | + | ||
| 451 | + | ||
| 452 | +def render_bubbles(d): | ||
| 453 | + db = d.get('device_bubbles', {}) | ||
| 454 | + if not db or db.get('skipped'): | ||
| 455 | + return skip_div(db.get('skip_reason', '设备空泡数据不可用') if db else '设备空泡数据缺失') | ||
| 456 | + ur = float(db.get('underfeed_ratio', 0)) * 100 | ||
| 457 | + ur_st = cell_style(ur, 5, 20) | ||
| 458 | + rows = [ | ||
| 459 | + ('underfeed_ratio', f'<span style="{ur_st}">{ur:.1f}%</span>', '严重 > 20%'), | ||
| 460 | + ('prelaunch_gap', f'{db.get("prelaunch_gap_ms", 0):.2f} ms', '严重 > 5ms'), | ||
| 461 | + ('internal_bubble_total', f'{db.get("internal_bubble_ms", 0):.2f} ms', ''), | ||
| 462 | + ] | ||
| 463 | + trs = ''.join(f'<tr><td style="font-weight:600">{k}</td><td>{v}</td>' | ||
| 464 | + f'<td style="color:#95a5a6">{r}</td></tr>' for k, v, r in rows) | ||
| 465 | + extra = note(db['note']) if db.get('note') else '' | ||
| 466 | + return f'<table><tr><th>指标</th><th>估算值</th><th>说明</th></tr>{trs}</table>{extra}' | ||
| 467 | + | ||
| 468 | + | ||
| 469 | +def render_wait_anchors(d): | ||
| 470 | + wa = d.get('wait_anchors', []) | ||
| 471 | + if not wa: | ||
| 472 | + return '<div style="color:#27ae60;padding:8px 0">✅ 未检测到等待锚点假热点。</div>' | ||
| 473 | + header = ('<tr><th>Kernel</th><th>Duration(μs)</th><th>Wait(μs)</th><th>wait_ratio</th>' | ||
| 474 | + '<th>total_cost(μs)</th><th>前序Kernel</th><th>判定</th></tr>') | ||
| 475 | + trs = '' | ||
| 476 | + for k in wa: | ||
| 477 | + wr = float(k.get('wait_ratio', 0)) * 100 | ||
| 478 | + wr_st = cell_style(wr, 50, 95) | ||
| 479 | + trs += (f'<tr><td><code>{html.escape(k.get("name",""))}</code></td>' | ||
| 480 | + f'<td style="color:#27ae60">{fmt_us(k.get("duration_us",0))}</td>' | ||
| 481 | + f'<td style="color:#e74c3c">{fmt_us(k.get("wait_us",0))}</td>' | ||
| 482 | + f'<td style="{wr_st}">{wr:.1f}%</td>' | ||
| 483 | + f'<td>{fmt_us(k.get("total_cost_us",0))}</td>' | ||
| 484 | + f'<td style="font-size:11px;color:#95a5a6">{html.escape(k.get("prev_kernel","")[:40])}</td>' | ||
| 485 | + f'<td>{badge("🔴 假热点","P0")}</td></tr>') | ||
| 486 | + desc = note( | ||
| 487 | + '以上 kernel 按 total_cost 排名靠前,但 wait_ratio > 95% 且 Duration < 10μs,判定为<strong>等待锚点假热点</strong>。' | ||
| 488 | + '真实计算耗时极短,不是性能瓶颈;实际问题是上游操作(通常为通信)导致设备等待。', | ||
| 489 | + 'warn' | ||
| 490 | + ) | ||
| 491 | + return f'<table>{header}{trs}</table>{desc}' | ||
| 492 | + | ||
| 493 | + | ||
| 494 | +def render_layer_struct(d): | ||
| 495 | + ls = d.get('layer_structure', {}) | ||
| 496 | + if not ls or ls.get('skipped'): | ||
| 497 | + return skip_div(ls.get('skip_reason', '层级结构数据不可用') if ls else '层级结构数据缺失') | ||
| 498 | + rows = [ | ||
| 499 | + ('模型类型', f'<strong>{html.escape(ls.get("model_type","Unknown"))}</strong>'), | ||
| 500 | + ('推理阶段', html.escape(ls.get('inference_phase', '—'))), | ||
| 501 | + ('层数(估算)', str(ls.get('num_layers', '—'))), | ||
| 502 | + ('MoE 特征算子', html.escape(', '.join(ls.get('moe_ops_detected', [])) or '未检测到')), | ||
| 503 | + ] | ||
| 504 | + trs = ''.join(f'<tr><td style="font-weight:600">{k}</td><td>{v}</td></tr>' for k, v in rows) | ||
| 505 | + extra = note(ls['note']) if ls.get('note') else '' | ||
| 506 | + return f'<table>{trs}</table>{extra}' | ||
| 507 | + | ||
| 508 | + | ||
| 509 | +def render_multi_card(d): | ||
| 510 | + mc = d.get('multi_card', {}) | ||
| 511 | + if not mc or mc.get('skipped'): | ||
| 512 | + return skip_div(mc.get('skip_reason', '仅单设备数据,无法进行多卡分析') if mc else '多卡数据不可用') | ||
| 513 | + vr = mc.get('variance_ratio') | ||
| 514 | + if vr is not None: | ||
| 515 | + vr_pct = float(vr) * 100 | ||
| 516 | + vr_st = cell_style(vr_pct, 10, 20) | ||
| 517 | + return f'<p>Stage variance_ratio = <span style="{vr_st}">{vr_pct:.1f}%</span>(阈值:警告 10%,严重 20%)</p>' | ||
| 518 | + return '<p>多卡数据存在,但 variance_ratio 未计算。</p>' | ||
| 519 | + | ||
| 520 | + | ||
| 521 | +def render_diagnosis(d): | ||
| 522 | + diag = d.get('bottleneck_diagnosis', []) | ||
| 523 | + if not diag: | ||
| 524 | + return '<div style="color:#27ae60;padding:8px 0">✅ 无已知性能瓶颈。</div>' | ||
| 525 | + header = '<tr><th>优先级</th><th>标签</th><th>维度</th><th>指标证据</th><th>诊断理由</th><th>反证/降级</th><th>置信度</th></tr>' | ||
| 526 | + trs = '' | ||
| 527 | + for item in diag: | ||
| 528 | + p = item.get('priority', 'P3') | ||
| 529 | + trs += (f'<tr><td>{badge(p, p)}</td>' | ||
| 530 | + f'<td><code>{html.escape(item.get("label",""))}</code></td>' | ||
| 531 | + f'<td>{html.escape(item.get("dimension",""))}</td>' | ||
| 532 | + f'<td style="font-size:12px">{html.escape(item.get("evidence",""))}</td>' | ||
| 533 | + f'<td style="font-size:12px">{html.escape(item.get("reasoning",""))}</td>' | ||
| 534 | + f'<td style="font-size:12px;color:var(--text-secondary)">' | ||
| 535 | + f'{html.escape(item.get("counter_evidence",""))}</td>' | ||
| 536 | + f'<td>{html.escape(item.get("confidence",""))}</td></tr>') | ||
| 537 | + return f'<table>{header}{trs}</table>' | ||
| 538 | + | ||
| 539 | + | ||
| 540 | +def render_recommendations(d): | ||
| 541 | + recs = d.get('recommendations', []) | ||
| 542 | + not_rec = d.get('not_recommended', []) | ||
| 543 | + if not recs: | ||
| 544 | + return '<div style="color:#95a5a6">暂无具体优化建议。</div>' | ||
| 545 | + header = '<tr><th>优先级</th><th>建议</th><th>具体措施</th><th>预期收益</th></tr>' | ||
| 546 | + trs = '' | ||
| 547 | + for r in recs: | ||
| 548 | + p = r.get('priority', 'P3') | ||
| 549 | + trs += (f'<tr><td>{badge(p, p)}</td>' | ||
| 550 | + f'<td><strong>{html.escape(r.get("title",""))}</strong></td>' | ||
| 551 | + f'<td style="font-size:12px">{html.escape(r.get("action",""))}</td>' | ||
| 552 | + f'<td style="font-size:12px;color:#27ae60">{html.escape(r.get("benefit",""))}</td></tr>') | ||
| 553 | + table = f'<table>{header}{trs}</table>' | ||
| 554 | + not_rec_html = '' | ||
| 555 | + if not_rec: | ||
| 556 | + items = '、'.join(html.escape(x) for x in not_rec) | ||
| 557 | + not_rec_html = note(f'<strong>不建议优先优化</strong>:{items}', 'warn') | ||
| 558 | + return f'{table}{not_rec_html}' | ||
| 559 | + | ||
| 560 | + | ||
| 561 | +# ── CSS ─────────────────────────────────────────────────────────────────────── | ||
| 562 | +CSS = ''' | ||
| 563 | +*{margin:0;padding:0;box-sizing:border-box} | ||
| 564 | +:root{ | ||
| 565 | + --bg-primary:#282a36;--bg-secondary:#21222c;--bg-tertiary:#343746;--bg-hover:#44475a; | ||
| 566 | + --text-primary:#f8f8f2;--text-secondary:#aab0d4;--accent-blue:#8be9fd;--accent-green:#50fa7b;--accent-orange:#ffb86c; | ||
| 567 | + --border-color:#44475a;--button-bg:#bd93f9;--button-hover:#ff79c6;--nav-bg:#1d1e26;--nav-text:#d8dcff; | ||
| 568 | + --table-head:#343746;--table-row-alt:#2e3040;--table-hover:#3b3f55;--code-bg:#3b3f55;--card-bg:#343746; | ||
| 569 | + --note-info-bg:rgba(139,233,253,.12);--note-info-border:#8be9fd; | ||
| 570 | + --note-warn-bg:rgba(255,184,108,.14);--note-warn-border:#ffb86c; | ||
| 571 | + --note-alert-bg:rgba(255,85,85,.14);--note-alert-border:#ff5555; | ||
| 572 | +} | ||
| 573 | +[data-theme="vscode-dark"]{ | ||
| 574 | +--bg-primary:#1e1e1e;--bg-secondary:#252526;--bg-tertiary:#2d2d2d;--bg-hover:#333333; | ||
| 575 | +--text-primary:#d4d4d4;--text-secondary:#9a9a9a;--accent-blue:#4fc1ff;--accent-green:#6a9955;--accent-orange:#ce9178; | ||
| 576 | +--border-color:#3c3c3c;--button-bg:#0e639c;--button-hover:#1177bb;--nav-bg:#181818;--nav-text:#cccccc; | ||
| 577 | +--table-head:#2d2d2d;--table-row-alt:#252526;--table-hover:#303030;--code-bg:#333333;--card-bg:#252526} | ||
| 578 | +[data-theme="one-dark"]{ | ||
| 579 | +--bg-primary:#282c34;--bg-secondary:#21252b;--bg-tertiary:#2c313a;--bg-hover:#353b45; | ||
| 580 | +--text-primary:#abb2bf;--text-secondary:#7f8794;--accent-blue:#61afef;--accent-green:#98c379;--accent-orange:#d19a66; | ||
| 581 | +--border-color:#181a1f;--button-bg:#4d78cc;--button-hover:#528bff;--nav-bg:#1f2329;--nav-text:#abb2bf; | ||
| 582 | +--table-head:#2c313a;--table-row-alt:#252a31;--table-hover:#333944;--code-bg:#353b45;--card-bg:#2c313a} | ||
| 583 | +[data-theme="github-light"]{ | ||
| 584 | +--bg-primary:#ffffff;--bg-secondary:#f6f8fa;--bg-tertiary:#ffffff;--bg-hover:#eaeef2; | ||
| 585 | +--text-primary:#1f2328;--text-secondary:#656d76;--accent-blue:#0969da;--accent-green:#1a7f37;--accent-orange:#bc4c00; | ||
| 586 | +--border-color:#d1d9e0;--button-bg:#0969da;--button-hover:#0550ae;--nav-bg:#24292f;--nav-text:#d1d9e0; | ||
| 587 | +--table-head:#f6f8fa;--table-row-alt:#f6f8fa;--table-hover:#eef4ff;--code-bg:#f6f8fa;--card-bg:#ffffff; | ||
| 588 | +--note-info-bg:#ddf4ff;--note-info-border:#0969da;--note-warn-bg:#fff8c5;--note-warn-border:#bc4c00; | ||
| 589 | +--note-alert-bg:#ffebe9;--note-alert-border:#cf222e} | ||
| 590 | +[data-theme="solarized-light"]{ | ||
| 591 | +--bg-primary:#fdf6e3;--bg-secondary:#eee8d5;--bg-tertiary:#fff8e8;--bg-hover:#e0dac7; | ||
| 592 | +--text-primary:#073642;--text-secondary:#657b83;--accent-blue:#268bd2;--accent-green:#859900;--accent-orange:#cb4b16; | ||
| 593 | +--border-color:#d3cbb7;--button-bg:#268bd2;--button-hover:#1a6da8;--nav-bg:#073642;--nav-text:#eee8d5; | ||
| 594 | +--table-head:#eee8d5;--table-row-alt:#f4edda;--table-hover:#e8e1cc;--code-bg:#eee8d5;--card-bg:#fff8e8; | ||
| 595 | +--note-info-bg:#e4f2f7;--note-info-border:#268bd2;--note-warn-bg:#faedcf;--note-warn-border:#cb4b16; | ||
| 596 | +--note-alert-bg:#f7dfd8;--note-alert-border:#dc322f} | ||
| 597 | +body{font-family:"PingFang SC","Microsoft YaHei",sans-serif;font-size:14px; | ||
| 598 | +background:var(--bg-primary);color:var(--text-primary)} | ||
| 599 | +code{background:var(--code-bg);color:var(--text-primary);padding:1px 5px; | ||
| 600 | +border-radius:3px;font-size:12px;font-family:monospace} | ||
| 601 | +nav{position:fixed;top:0;left:0;width:230px;height:100vh;background:var(--nav-bg); | ||
| 602 | +color:var(--nav-text);overflow-y:auto;z-index:100;padding:16px 0} | ||
| 603 | +nav h2{font-size:12px;color:var(--text-secondary);padding:0 14px 10px; | ||
| 604 | +border-bottom:1px solid var(--border-color);letter-spacing:1px} | ||
| 605 | +nav ul{list-style:none;margin-top:6px} | ||
| 606 | +nav ul li a{display:block;padding:6px 14px;color:var(--nav-text);font-size:12px; | ||
| 607 | +transition:all .2s;border-left:3px solid transparent;text-decoration:none} | ||
| 608 | +nav ul li a:hover,nav ul li a.active{color:#fff;background:rgba(255,255,255,.08);border-left-color:var(--accent-blue)} | ||
| 609 | +main{margin-left:230px;padding:20px 28px;max-width:1180px} | ||
| 610 | +h1{font-size:22px;font-weight:700;color:var(--text-primary);margin:20px 0 4px} | ||
| 611 | +.sub{color:var(--text-secondary);font-size:12px;margin-bottom:14px} | ||
| 612 | +.controls{display:flex;gap:10px;align-items:center;background:var(--bg-secondary); | ||
| 613 | +border:1px solid var(--border-color);border-radius:8px;padding:10px 12px; | ||
| 614 | +margin-bottom:14px;color:var(--text-secondary);font-size:12px} | ||
| 615 | +.controls select{background:var(--bg-tertiary);color:var(--text-primary); | ||
| 616 | +border:1px solid var(--border-color);border-radius:4px;padding:5px 8px} | ||
| 617 | +details{background:var(--bg-tertiary);border:1px solid var(--border-color); | ||
| 618 | +border-radius:8px;margin-bottom:12px;box-shadow:0 1px 6px rgba(0,0,0,.12);overflow:hidden} | ||
| 619 | +details[open]{box-shadow:0 2px 10px rgba(0,0,0,.10)} | ||
| 620 | +summary{padding:13px 18px;cursor:pointer;font-size:14px;font-weight:600; | ||
| 621 | +color:var(--text-primary);list-style:none;user-select:none;display:flex;align-items:center;gap:6px} | ||
| 622 | +summary::-webkit-details-marker{display:none} | ||
| 623 | +summary::before{content:"▶";font-size:9px;color:var(--text-secondary);transition:transform .2s;min-width:11px} | ||
| 624 | +details[open]>summary::before{transform:rotate(90deg)} | ||
| 625 | +.sb{padding:4px 18px 18px} | ||
| 626 | +table{width:100%;border-collapse:collapse;font-size:13px;margin:6px 0} | ||
| 627 | +th{background:var(--table-head);text-align:left;padding:8px 10px;font-weight:600; | ||
| 628 | +color:var(--text-primary);border-bottom:1px solid var(--border-color)} | ||
| 629 | +td{padding:7px 10px;border-bottom:1px solid var(--border-color);vertical-align:middle;color:var(--text-primary)} | ||
| 630 | +tr:nth-child(even) td{background:var(--table-row-alt)} | ||
| 631 | +tr:hover td{background:var(--table-hover)!important} | ||
| 632 | +section{scroll-margin-top:20px} | ||
| 633 | +footer{margin-top:36px;padding:14px 0;border-top:1px solid var(--border-color); | ||
| 634 | +color:var(--text-secondary);font-size:11px} | ||
| 635 | +h3{font-size:13px;font-weight:600;color:var(--text-primary);margin:14px 0 8px} | ||
| 636 | +.summary-card{flex:1;min-width:220px;background:var(--card-bg);border-radius:8px; | ||
| 637 | +padding:14px 18px;box-shadow:0 2px 8px rgba(0,0,0,.12);border-left:5px solid var(--border-color)} | ||
| 638 | +.card-subtitle{font-size:12px;color:var(--text-secondary);margin:6px 0 2px} | ||
| 639 | +.card-title{font-size:17px;font-weight:700;color:var(--text-primary)} | ||
| 640 | +.card-desc{font-size:12px;color:var(--text-secondary);margin-top:4px} | ||
| 641 | +.facts-list{margin:8px 0 8px 20px;color:var(--text-primary);line-height:1.7} | ||
| 642 | +''' | ||
| 643 | + | ||
| 644 | +NAV_ITEMS = [ | ||
| 645 | + ('ctx', '一、分析上下文'), | ||
| 646 | + ('summary', '二、执行摘要'), | ||
| 647 | + ('bound', '三、整体 Bound'), | ||
| 648 | + ('dim1', '四、迭代效率'), | ||
| 649 | + ('dim2', '五、算子热点'), | ||
| 650 | + ('dim3', '六、硬件利用率'), | ||
| 651 | + ('opbound', '七、算子 Bound'), | ||
| 652 | + ('dim4', '八、通信效率'), | ||
| 653 | + ('dim5', '九、设备空泡'), | ||
| 654 | + ('dim6', '十、等待锚点'), | ||
| 655 | + ('dim7', '十一、层级结构'), | ||
| 656 | + ('dim8', '十二、多卡均衡'), | ||
| 657 | + ('diag', '十三、瓶颈诊断'), | ||
| 658 | + ('opt', '十四、优化建议'), | ||
| 659 | +] | ||
| 660 | + | ||
| 661 | + | ||
| 662 | +THEMES = ['dracula', 'vscode-dark', 'one-dark', 'github-light', 'solarized-light'] | ||
| 663 | + | ||
| 664 | + | ||
| 665 | +def _build_sections(data): | ||
| 666 | + """按顺序渲染全部分析维度 section 并拼接。""" | ||
| 667 | + return ( | ||
| 668 | + section('ctx', '一、分析上下文', render_meta(data), open_=True) + | ||
| 669 | + section('summary', '二、执行摘要', render_summary(data), open_=True) + | ||
| 670 | + section('bound', '三、整体 Bound 判定', render_bound_classification(data), open_=True) + | ||
| 671 | + section('dim1', '四、维度 1:迭代效率', render_iter_efficiency(data), open_=True) + | ||
| 672 | + section('dim2', '五、维度 2:算子热点', render_op_hotspots(data), open_=True) + | ||
| 673 | + section('dim3', '六、维度 3:硬件利用率 / MFU', render_hw_util(data)) + | ||
| 674 | + section('opbound', '七、算子级 Bound 分析', render_operator_bound(data), open_=True) + | ||
| 675 | + section('dim4', '八、维度 4:通信效率', render_comm(data)) + | ||
| 676 | + section('dim5', '九、维度 5:设备空泡', render_bubbles(data)) + | ||
| 677 | + section('dim6', '十、维度 6:等待锚点假热点', render_wait_anchors(data)) + | ||
| 678 | + section('dim7', '十一、维度 7:层级结构', render_layer_struct(data)) + | ||
| 679 | + section('dim8', '十二、维度 8:多卡负载均衡', render_multi_card(data)) + | ||
| 680 | + section('diag', '十三、瓶颈诊断', render_diagnosis(data), open_=True) + | ||
| 681 | + section('opt', '十四、优化建议', render_recommendations(data), open_=True) | ||
| 682 | + ) | ||
| 683 | + | ||
| 684 | + | ||
| 685 | +_NAV_SCRIPT = '''<script> | ||
| 686 | +const secs=document.querySelectorAll('section[id]'); | ||
| 687 | +const links=document.querySelectorAll('nav a'); | ||
| 688 | +window.addEventListener('scroll',()=>{ | ||
| 689 | + let cur=''; | ||
| 690 | + secs.forEach(s=>{if(window.scrollY+80>=s.offsetTop)cur=s.id}); | ||
| 691 | + links.forEach(a=>a.classList.toggle('active',a.getAttribute('href')==='#'+cur)); | ||
| 692 | +}); | ||
| 693 | +links.forEach(a=>a.addEventListener('click',()=>{ | ||
| 694 | + const id=a.getAttribute('href').slice(1); | ||
| 695 | + document.getElementById(id)?.querySelector('details') && | ||
| 696 | + (document.getElementById(id).querySelector('details').open=true); | ||
| 697 | +})); | ||
| 698 | +const themeSelect=document.getElementById('theme-select'); | ||
| 699 | +const savedTheme=localStorage.getItem('npu-perf-theme'); | ||
| 700 | +if(savedTheme && @@THEMES@@.includes(savedTheme)){ | ||
| 701 | + document.body.dataset.theme=savedTheme; | ||
| 702 | + themeSelect.value=savedTheme; | ||
| 703 | +} | ||
| 704 | +themeSelect.addEventListener('change',()=>{ | ||
| 705 | + document.body.dataset.theme=themeSelect.value; | ||
| 706 | + localStorage.setItem('npu-perf-theme',themeSelect.value); | ||
| 707 | +}); | ||
| 708 | +</script>''' | ||
| 709 | + | ||
| 710 | + | ||
| 711 | +def generate(data: dict, theme: str = 'dracula') -> str: | ||
| 712 | + if theme not in THEMES: | ||
| 713 | + theme = 'dracula' | ||
| 714 | + m = data.get('meta', {}) | ||
| 715 | + chip = m.get('chip', 'Ascend 910B3') | ||
| 716 | + schema = m.get('schema_version', '—') | ||
| 717 | + generated_at = m.get('generated_at', datetime.now(timezone.utc).astimezone().strftime('%Y-%m-%d')) | ||
| 718 | + subtitle = f"{html.escape(m.get('data_path',''))} · {html.escape(chip)} · {schema} Schema · {generated_at}" | ||
| 719 | + | ||
| 720 | + nav_links = '\n'.join(f'<li><a href="#{a}">{t}</a></li>' for a, t in NAV_ITEMS) | ||
| 721 | + | ||
| 722 | + sections_html = _build_sections(data) | ||
| 723 | + options = ''.join(f'<option value="{t}"{" selected" if t == theme else ""}>{t}</option>' for t in THEMES) | ||
| 724 | + nav_script = _NAV_SCRIPT.replace('@@THEMES@@', repr(THEMES)) | ||
| 725 | + | ||
| 726 | + return f'''<!DOCTYPE html> | ||
| 727 | +<html lang="zh-CN"> | ||
| 728 | +<head> | ||
| 729 | +<meta charset="UTF-8"> | ||
| 730 | +<meta name="viewport" content="width=device-width,initial-scale=1"> | ||
| 731 | +<title>NPU 性能分析报告</title> | ||
| 732 | +<style>{CSS}</style> | ||
| 733 | +</head> | ||
| 734 | +<body data-theme="{theme}"> | ||
| 735 | +<nav> | ||
| 736 | + <h2>NPU 分析导航</h2> | ||
| 737 | + <ul>{nav_links}</ul> | ||
| 738 | +</nav> | ||
| 739 | +<main> | ||
| 740 | + <h1>NPU 性能分析报告</h1> | ||
| 741 | + <div class="sub">{subtitle}</div> | ||
| 742 | + <div class="controls"><label>主题风格 <select id="theme-select">{options}</select></label></div> | ||
| 743 | + {sections_html} | ||
| 744 | + <footer>生成时间:{generated_at} | 数据路径:{html.escape(m.get("actual_path", m.get("data_path","")))} | ||
| 745 | + | Schema:{schema} | 芯片:{html.escape(chip)}</footer> | ||
| 746 | +</main> | ||
| 747 | +{nav_script} | ||
| 748 | +</body> | ||
| 749 | +</html>''' | ||
| 750 | + | ||
| 751 | + | ||
| 752 | +def main(): | ||
| 753 | + logging.basicConfig(level=logging.INFO, format='%(message)s', stream=sys.stdout) | ||
| 754 | + parser = argparse.ArgumentParser(description='NPU 性能分析 HTML 报告生成器') | ||
| 755 | + parser.add_argument('analysis_data_json', help='analysis_data.json 路径') | ||
| 756 | + parser.add_argument('output_html', help='输出 HTML 路径') | ||
| 757 | + parser.add_argument('--theme', choices=THEMES, default='dracula', | ||
| 758 | + help='HTML 主题风格,默认 dracula') | ||
| 759 | + args = parser.parse_args() | ||
| 760 | + json_path, html_path = args.analysis_data_json, args.output_html | ||
| 761 | + with open(json_path, encoding='utf-8') as f: | ||
| 762 | + data = json.load(f) | ||
| 763 | + html_content = generate(data, args.theme) | ||
| 764 | + with open(html_path, 'w', encoding='utf-8') as f: | ||
| 765 | + f.write(html_content) | ||
| 766 | + logger.info('HTML 报告已生成:%s', html_path) | ||
| 767 | + | ||
| 768 | + | ||
| 769 | +if __name__ == '__main__': | ||
| 770 | + main() | ||
| @@ -0,0 +1,95 @@ | |||
| 1 | +# 昇腾 NPU 硬件规格参考 | ||
| 2 | + | ||
| 3 | +> **使用时机**:Phase 2 计算 MFU 时需要从本文件查取 Peak TFLOPS;计算通信带宽效率时参考各传输介质理论峰值。 | ||
| 4 | + | ||
| 5 | +--- | ||
| 6 | + | ||
| 7 | +## 一、昇腾 910B 系列算力规格 | ||
| 8 | + | ||
| 9 | +| 芯片型号 | BF16/FP16 峰值算力 (TFLOPs/s) | INT8 峰值算力 (TOPs/s) | HBM 带宽 | 片内 HCCS 带宽 | | ||
| 10 | +|---|---|---|---|---| | ||
| 11 | +| Ascend 910B1 | ~378.88 | ~757.76 | ~900 GB/s | ~400+ GB/s | | ||
| 12 | +| Ascend 910B2 | ~353.89 | ~707.78 | ~900 GB/s | ~400+ GB/s | | ||
| 13 | +| **Ascend 910B3** | **~294.91** | ~589.82 | ~800 GB/s | ~400+ GB/s | | ||
| 14 | +| Ascend 910B4 | ~270.00 | ~540.00 | ~800 GB/s | ~392 GB/s | | ||
| 15 | + | ||
| 16 | +**默认假设**:若 profiling 数据中未说明芯片型号,使用 **Ascend 910B3(294.91 TFLOPs/s BF16)** 作为 Peak,并在报告中明确标注"【默认 910B3,请确认实际芯片型号】"。 | ||
| 17 | + | ||
| 18 | +### 芯片型号检测方法(优先级从高到低) | ||
| 19 | + | ||
| 20 | +1. **用户直接指定**:用户在问题中提及芯片型号 | ||
| 21 | +2. **profiler_metadata.json**:longcat 风格采集目录中的 `profiler_metadata.json` 或 `profiler_info_N.json` 可能包含设备信息 | ||
| 22 | +3. **kernel_details.csv 推断**:根据 `Block Dim` 最大值(910B3 通常为 64,910B4 通常为 32)推断 | ||
| 23 | +4. **默认值**:使用 910B3 | ||
| 24 | + | ||
| 25 | +--- | ||
| 26 | + | ||
| 27 | +## 二、AI Core 流水线单元说明(V2 Schema) | ||
| 28 | + | ||
| 29 | +昇腾 AI Core 包含多个并行执行单元,`kernel_details.csv` V2 Schema 中的 `aic_*_ratio` 字段反映各单元在 kernel 执行期间的时间占比。 | ||
| 30 | + | ||
| 31 | +| 单元 | 字段 | 功能 | 瓶颈判断 | | ||
| 32 | +|---|---|---|---| | ||
| 33 | +| **MAC/Cube** | `aic_mac_ratio` | 矩阵乘法计算单元(执行 GEMM / Conv) | > 0.8 → Compute Bound | | ||
| 34 | +| **MTE2** | `aic_mte2_ratio` | 数据读取引擎(L2/HBM → L1 Cache) | > 0.8 → Memory Bound(权重加载是瓶颈) | | ||
| 35 | +| **MTE1** | `aic_mte1_ratio` | 数据搬移(L1 → L0,提供 MAC 单元操作数) | 高 → L1 至 L0 搬移是瓶颈 | | ||
| 36 | +| **FixPipe** | `aic_fixpipe_ratio` | 后处理(量化、激活函数、格式转换) | > 0.3 → 后处理是瓶颈 | | ||
| 37 | +| **Scalar** | `aic_scalar_ratio` | 标量运算(地址计算、控制流) | 高 → 标量操作过多,可能 shape 推断复杂 | | ||
| 38 | + | ||
| 39 | +各比例之和约等于 1.0(允许小误差)。 | ||
| 40 | + | ||
| 41 | +### 典型 kernel 流水线模式 | ||
| 42 | + | ||
| 43 | +| 场景 | 典型比例模式 | | ||
| 44 | +|---|---| | ||
| 45 | +| GEMM(大 batch,高效执行) | aic_mac_ratio ≈ 0.7–0.85,aic_mte2_ratio ≈ 0.1–0.2 | | ||
| 46 | +| GEMM(权重加载瓶颈) | aic_mac_ratio ≈ 0.3–0.4,aic_mte2_ratio ≈ 0.6–0.7 | | ||
| 47 | +| RmsNorm / LayerNorm | aic_mac_ratio ≈ 0,aic_vec_ratio(AIV)≈ 0.6–0.8 | | ||
| 48 | +| 小算子(Mul、Add、Cast) | 所有比例均低,Duration 极短,Wait 可能较高 | | ||
| 49 | + | ||
| 50 | +--- | ||
| 51 | + | ||
| 52 | +## 三、AI Vector Core(AIV)流水线单元 | ||
| 53 | + | ||
| 54 | +| 单元 | 字段 | 功能 | | ||
| 55 | +|---|---|---| | ||
| 56 | +| **Vector** | `aiv_vec_ratio` | 向量计算(Element-wise 操作) | | ||
| 57 | +| **MTE2** | `aiv_mte2_ratio` | GM → UB(全局内存到统一缓冲区)读取 | | ||
| 58 | +| **MTE3** | `aiv_mte3_ratio` | UB → GM 写入 | | ||
| 59 | +| **Scalar** | `aiv_scalar_ratio` | 标量控制流 | | ||
| 60 | + | ||
| 61 | +--- | ||
| 62 | + | ||
| 63 | +## 四、HCCL 通信传输介质参数 | ||
| 64 | + | ||
| 65 | +| 介质 | 典型场景 | 理论峰值带宽 | 备注 | | ||
| 66 | +|---|---|---|---| | ||
| 67 | +| **HCCS** | 同机 NPU-NPU 互联 | ~400+ GB/s / 端口 | 昇腾高速片间总线,用于同节点 Tensor Parallelism | | ||
| 68 | +| **RDMA** | 跨节点通信 | ~200–400 Gbps(取决于 IB/RoCE 配置) | 用于节点间 AllReduce / AllGather | | ||
| 69 | +| **SDMA** | 同节点 DMA 传输 | ~数十 GB/s | 节点内辅助传输,带宽低于 HCCS | | ||
| 70 | +| **PCIE** | Host-NPU 传输 | ~64 GB/s(PCIe Gen4 ×16) | 用于 Host-Device 数据迁移 | | ||
| 71 | +| **SIO** | 信号 I/O | 取决于配置 | 辅助通道 | | ||
| 72 | + | ||
| 73 | +### 通信带宽健康判断 | ||
| 74 | + | ||
| 75 | +| 介质 | 实测带宽 | 判断 | | ||
| 76 | +|---|---|---| | ||
| 77 | +| RDMA | > 1.5 GB/s | 正常 | | ||
| 78 | +| RDMA | 0.5–1.5 GB/s | 警告(小包问题或网络拥塞) | | ||
| 79 | +| RDMA | < 0.5 GB/s | 严重(严重小包或链路故障) | | ||
| 80 | +| HCCS | 接近峰值 400 GB/s | 正常 | | ||
| 81 | +| HCCS | < 50 GB/s | 警告(数据量过小或配置问题) | | ||
| 82 | + | ||
| 83 | +--- | ||
| 84 | + | ||
| 85 | +## 五、常见模型 MFU 参考值 | ||
| 86 | + | ||
| 87 | +| 场景 | 典型 MFU 范围 | 说明 | | ||
| 88 | +|---|---|---| | ||
| 89 | +| 大规模预训练(大 batch,大矩阵) | 40%–65% | AI Core Cube 充分饱和 | | ||
| 90 | +| SFT 微调(中等 batch) | 25%–45% | 稍有碎片,仍较高效 | | ||
| 91 | +| 推理 Prefill(长 prompt) | 20%–50% | 取决于 batch size 和 seq len | | ||
| 92 | +| 推理 Decode(batch=1,单 token) | < 5% | 正常,矩阵 M=1 无法充分利用 Cube | | ||
| 93 | +| 推理 Decode(大 batch,连续 batching) | 15%–40% | batch 越大,MFU 越高 | | ||
| 94 | + | ||
| 95 | +> **低 MFU 不等于有问题**:需结合场景和 batch size 综合判断。 | ||
| @@ -0,0 +1,453 @@ | |||
| 1 | +# 性能指标计算公式 | ||
| 2 | + | ||
| 3 | +> **使用时机**:执行 Phase 1–3 中任何指标计算之前读取本文件。所有公式均基于 `data-schema.md` 中定义的字段。 | ||
| 4 | + | ||
| 5 | +--- | ||
| 6 | + | ||
| 7 | +## 一、迭代效率(Dimension 1) | ||
| 8 | + | ||
| 9 | +数据来源:`step_trace_time.csv` | ||
| 10 | + | ||
| 11 | +### 1.1 基础比例指标 | ||
| 12 | + | ||
| 13 | +``` | ||
| 14 | +computing_ratio = Computing / Stage | ||
| 15 | +comm_not_overlap_ratio = Communication(Not Overlapped) / Stage | ||
| 16 | +free_ratio = Free / Stage | ||
| 17 | +bubble_ratio = Bubble / Stage | ||
| 18 | +preparing_ratio = Preparing / Stage | ||
| 19 | + | ||
| 20 | +# 通信遮蔽率:越高说明通信被计算遮蔽的比例越大(越好) | ||
| 21 | +overlap_ratio = Overlapped / Communication # 若 Communication == 0,则 overlap_ratio = 0 | ||
| 22 | +``` | ||
| 23 | + | ||
| 24 | +**预热步检测**(同一 device 相邻步之间对比): | ||
| 25 | +``` | ||
| 26 | +若 |step_N.free_ratio - avg(step_N+1..end.free_ratio)| > 0.20,则 step_N 为疑似预热步 | ||
| 27 | +``` | ||
| 28 | + | ||
| 29 | +### 1.2 多卡负载均衡(Dimension 8) | ||
| 30 | + | ||
| 31 | +同一 Step,跨多个 Device_id 对比 `Stage`: | ||
| 32 | +``` | ||
| 33 | +max_stage = max(Stage for all Device_id) | ||
| 34 | +min_stage = min(Stage for all Device_id) | ||
| 35 | +avg_stage = mean(Stage for all Device_id) | ||
| 36 | +variance_ratio = (max_stage - min_stage) / avg_stage | ||
| 37 | +``` | ||
| 38 | + | ||
| 39 | +若某 Device 的 `Free` 显著高于平均值(> avg_free × 2),则判定为**Host Dispatch 慢卡**(表面上是快卡,实为 Host 侧拖慢)。 | ||
| 40 | + | ||
| 41 | +--- | ||
| 42 | + | ||
| 43 | +## 二、设备空泡(Dimension 5) | ||
| 44 | + | ||
| 45 | +数据来源:`kernel_details.csv` | ||
| 46 | + | ||
| 47 | +### 2.1 时间轴构建 | ||
| 48 | + | ||
| 49 | +``` | ||
| 50 | +# 对每个 Step,收集该 Step 内所有 kernel 的时间区间 | ||
| 51 | +device_intervals = [] | ||
| 52 | +for each kernel row with Step_Id == target_step: | ||
| 53 | + s = Start_Time_us | ||
| 54 | + e = Start_Time_us + Duration_us | ||
| 55 | + if Duration_us > 0: | ||
| 56 | + device_intervals.append((s, e)) | ||
| 57 | +``` | ||
| 58 | + | ||
| 59 | +### 2.2 区间合并(去重多 Stream 重叠) | ||
| 60 | + | ||
| 61 | +```python | ||
| 62 | +def merge_intervals(intervals): | ||
| 63 | + if not intervals: | ||
| 64 | + return [] | ||
| 65 | + sorted_ivs = sorted(intervals, key=lambda x: x[0]) | ||
| 66 | + merged = [sorted_ivs[0]] | ||
| 67 | + for s, e in sorted_ivs[1:]: | ||
| 68 | + if s <= merged[-1][1] + 1: # 1μs 容差,合并相邻区间 | ||
| 69 | + merged[-1] = (merged[-1][0], max(merged[-1][1], e)) | ||
| 70 | + else: | ||
| 71 | + merged.append((s, e)) | ||
| 72 | + return merged | ||
| 73 | +``` | ||
| 74 | + | ||
| 75 | +### 2.3 空泡指标计算 | ||
| 76 | + | ||
| 77 | +``` | ||
| 78 | +merged_segments = merge_intervals(device_intervals) | ||
| 79 | +busy_union = sum(seg[1] - seg[0] for seg in merged_segments) | ||
| 80 | + | ||
| 81 | +# Step 窗口:以该 Step 第一个 kernel 的 Start Time 为起点,Stage(μs) 为窗口长度 | ||
| 82 | +step_start = min(Start_Time_us for kernels in this step) | ||
| 83 | +step_end = step_start + Stage_us # Stage 取自 step_trace_time.csv | ||
| 84 | + | ||
| 85 | +prelaunch_gap = merged_segments[0][0] - step_start # 首个 kernel 开始前的空闲 | ||
| 86 | +tail_gap = step_end - merged_segments[-1][1] # 最后一个 kernel 结束后的空闲 | ||
| 87 | +internal_bubble_total = sum(merged_segments[i+1][0] - merged_segments[i][1] | ||
| 88 | + for i in range(len(merged_segments) - 1)) # kernel 之间的间隙之和 | ||
| 89 | +largest_internal_bubble = max gap between consecutive merged segments (若有) | ||
| 90 | + | ||
| 91 | +underfeed_ratio = (prelaunch_gap + tail_gap + internal_bubble_total) / Stage_us | ||
| 92 | +``` | ||
| 93 | + | ||
| 94 | +**Step 窗口替代方案**(当 `Step Id` 列缺失时):用 `step_trace_time.csv` 中相邻 Step 的时间戳差估算窗口,或以全局采集区间为单一伪步。 | ||
| 95 | + | ||
| 96 | +--- | ||
| 97 | + | ||
| 98 | +## 三、等待锚点检测(Dimension 6) | ||
| 99 | + | ||
| 100 | +数据来源:`kernel_details.csv` | ||
| 101 | + | ||
| 102 | +``` | ||
| 103 | +wait_ratio = Wait_Time_us / (Duration_us + Wait_Time_us) | ||
| 104 | + | ||
| 105 | +is_wait_anchor = (wait_ratio > 0.95) AND (Duration_us < 10.0) | ||
| 106 | +``` | ||
| 107 | + | ||
| 108 | +**解释**:等待锚点是指 `wait_ratio` 接近 1 但 `Duration` 极短的 kernel。它在按 `total_cost = Duration + Wait` 排名时看起来很"贵",但实际上自身几乎不执行计算——真正的问题是它前面的操作让设备长时间空转。 | ||
| 109 | + | ||
| 110 | +**误判防范**:若该 kernel 的 `is_wait_anchor == True`,在热点排名中明确降级并标注"假热点,非真实计算瓶颈",同时列出其前后紧邻的 kernel 作为上下文。 | ||
| 111 | + | ||
| 112 | +--- | ||
| 113 | + | ||
| 114 | +## 四、硬件利用率 / MFU(Dimension 3) | ||
| 115 | + | ||
| 116 | +数据来源:`kernel_details.csv` | ||
| 117 | + | ||
| 118 | +### 4.1 MFU 计算(V2 Schema,MatMul 类 kernel) | ||
| 119 | + | ||
| 120 | +``` | ||
| 121 | +# Input Shapes 解析:格式为 "M,K;K,N" 或 "M,K;N,K"(转置无关,FLOPs 相同) | ||
| 122 | +parse_matmul_shapes(input_shapes): | ||
| 123 | + tensors = input_shapes.split(";") | ||
| 124 | + A_shape = [int(x) for x in tensors[0].split(",")] | ||
| 125 | + B_shape = [int(x) for x in tensors[1].split(",")] | ||
| 126 | + M, K = A_shape[0], A_shape[1] | ||
| 127 | + N = B_shape[1] if B_shape[0] == K else B_shape[0] # 兼容转置 | ||
| 128 | + return M, K, N | ||
| 129 | + | ||
| 130 | +FLOPs = 2 × M × K × N | ||
| 131 | +Duration_s = Duration_us × 1e-6 | ||
| 132 | +Achieved_TFLOPs = FLOPs / Duration_s / 1e12 | ||
| 133 | +MFU = Achieved_TFLOPs / Peak_TFLOPs # Peak 取自 hardware-specs.md | ||
| 134 | +``` | ||
| 135 | + | ||
| 136 | +**批量 MatMul**(Input Shapes = "B,M,K;B,K,N"): | ||
| 137 | +``` | ||
| 138 | +FLOPs = 2 × B × M × K × N | ||
| 139 | +``` | ||
| 140 | + | ||
| 141 | +### 4.2 MFU 计算(V2 Schema,FusedInferAttentionScore / FlashAttention) | ||
| 142 | + | ||
| 143 | +``` | ||
| 144 | +# BNSD layout: batch × heads × seq × head_dim | ||
| 145 | +# BSH layout: batch × seq × (heads × head_dim) | ||
| 146 | +# 均支持,从 Input Shapes 推断 | ||
| 147 | + | ||
| 148 | +FLOPs = 2 × q_batch × q_heads × q_seq × kv_seq × (q_head_dim + kv_head_dim) | ||
| 149 | + | ||
| 150 | +# 若为因果掩码(causal mask): | ||
| 151 | +FLOPs = FLOPs × 0.5 | ||
| 152 | + | ||
| 153 | +MFU = (FLOPs / Duration_s / 1e12) / Peak_TFLOPs | ||
| 154 | +``` | ||
| 155 | + | ||
| 156 | +**注意**:batch=1、单 token 推理(decode)场景下,因 q_seq=1,MatMul MFU 天然极低(< 5%),**不应将此视为硬件故障**,须在报告中注明"推理 decode 单 token,绝对 MFU 低属正常"。 | ||
| 157 | + | ||
| 158 | +### 4.3 MFU 估算(V1 Schema,低置信度) | ||
| 159 | + | ||
| 160 | +``` | ||
| 161 | +Achieved_TFLOPs = aic_cube_fops / Duration_us / 1e6 # aic_cube_fops 单位为 FLOPs | ||
| 162 | +MFU = Achieved_TFLOPs / Peak_TFLOPs | ||
| 163 | +``` | ||
| 164 | +V1 估算在报告中须标注"【V1 估算,置信度较低】"。 | ||
| 165 | + | ||
| 166 | +### 4.4 cube_utilization(%) 的语义 | ||
| 167 | + | ||
| 168 | +`cube_utilization(%)` = Cube 单元有效执行时间 / kernel 总时间,衡量**流水线效率**(kernel 内部 Cube 有多忙),**不等于 MFU**(MFU 衡量相对硬件峰值的算力利用率)。两者都应报告,含义不同: | ||
| 169 | +- `cube_utilization` 低 → kernel 内部流水线有气泡(可能 MTE 成为瓶颈) | ||
| 170 | +- `MFU` 低 → 算力远低于硬件峰值(可能 shape 太小、batch 不足) | ||
| 171 | + | ||
| 172 | +### 4.5 瓶颈类型判定(AI Core kernel) | ||
| 173 | + | ||
| 174 | +``` | ||
| 175 | +if aic_mte2_ratio > 0.8: → Memory Bound(HBM/L2 读取是主要瓶颈) | ||
| 176 | +elif aic_mac_ratio > 0.8: → Compute Bound(Cube 单元是瓶颈) | ||
| 177 | +elif aic_fixpipe_ratio > 0.3: → FixPipe Bound(后处理或激活函数是瓶颈) | ||
| 178 | +else: → Latency Bound / Mixed(带宽和算力利用率均低) | ||
| 179 | +``` | ||
| 180 | + | ||
| 181 | +--- | ||
| 182 | + | ||
| 183 | +## 五、通信效率(Dimension 4) | ||
| 184 | + | ||
| 185 | +数据来源:`communication.json` + `step_trace_time.csv` | ||
| 186 | + | ||
| 187 | +### 5.1 遮蔽率(step 级) | ||
| 188 | + | ||
| 189 | +``` | ||
| 190 | +overlap_ratio = Overlapped / Communication # 来自 step_trace_time.csv | ||
| 191 | +comm_efficiency_index = 1 - comm_not_overlap_ratio # 越高越好 | ||
| 192 | +``` | ||
| 193 | + | ||
| 194 | +### 5.2 各集合通信操作汇总 | ||
| 195 | + | ||
| 196 | +从 `communication.json` 按操作类型(AllReduce / AllGather / ReduceScatter)汇总: | ||
| 197 | +``` | ||
| 198 | +对每个 step 中的每个 collective op: | ||
| 199 | + type = extract_type(op_name) # 从前缀解析 | ||
| 200 | + total_elapse[type] += Elapse Time(ms) | ||
| 201 | + total_wait[type] += Wait Time(ms) | ||
| 202 | + total_size[type] += sum(Transit Size(MB) across all transports) | ||
| 203 | +``` | ||
| 204 | + | ||
| 205 | +### 5.3 带宽计算 | ||
| 206 | + | ||
| 207 | +``` | ||
| 208 | +for each transport in [RDMA, HCCS, PCIE, SDMA, SIO]: | ||
| 209 | + if Transit_Size_MB > 0 and Transit_Time_ms > 0: | ||
| 210 | + bandwidth_GBps = Transit_Size_MB / 1024 / (Transit_Time_ms / 1000) | ||
| 211 | + else: | ||
| 212 | + bandwidth_GBps = "N/A(单 Rank 采集或无数据)" | ||
| 213 | +``` | ||
| 214 | + | ||
| 215 | +--- | ||
| 216 | + | ||
| 217 | +## 六、算子层级结构(Dimension 7) | ||
| 218 | + | ||
| 219 | +数据来源:`kernel_details.csv` | ||
| 220 | + | ||
| 221 | +### 6.1 结构边界检测 | ||
| 222 | + | ||
| 223 | +以 `FusedInferAttentionScore`(FIA)kernel 作为层边界标记(对 Transformer 类模型): | ||
| 224 | +``` | ||
| 225 | +FIA_kernels = [row for row if "FusedInferAttentionScore" in row.Name] | ||
| 226 | +sorted by Start_Time_us | ||
| 227 | + | ||
| 228 | +layer_count = len(FIA_kernels) / FIA_per_layer # 通常 FIA_per_layer = 1(单卡) | ||
| 229 | +``` | ||
| 230 | + | ||
| 231 | +若无 FIA,退回使用重复 kernel 名称模式检测:连续出现相同 kernel 名序列作为层重复单元。 | ||
| 232 | + | ||
| 233 | +### 6.2 模型类型判定 | ||
| 234 | + | ||
| 235 | +``` | ||
| 236 | +if any("MoeGatingTopK" in name or "DispatchFFNCombine" in name | ||
| 237 | + or "MoeComputeExpertTokens" in name) for name in all_kernel_names: | ||
| 238 | + model_type = "MoE(混合专家模型)" | ||
| 239 | +elif any("GroupedMatmul" in name): | ||
| 240 | + model_type = "MoE(GroupedMatmul 路由)或稀疏模型" | ||
| 241 | +else: | ||
| 242 | + model_type = "Dense(稠密模型)" | ||
| 243 | +``` | ||
| 244 | + | ||
| 245 | +### 6.3 单层耗时估算 | ||
| 246 | + | ||
| 247 | +``` | ||
| 248 | +for each FIA-delimited layer interval [FIA_i.start, FIA_{i+1}.start): | ||
| 249 | + layer_kernels = kernels with Start_Time in this interval | ||
| 250 | + layer_wall_ms = (FIA_{i+1}.start - FIA_i.start) / 1000 | ||
| 251 | + layer_kernel_sum_ms = sum(Duration_us for kernel in layer_kernels) / 1000 | ||
| 252 | + layer_busy_union_ms = compute busy union for layer_kernels | ||
| 253 | +``` | ||
| 254 | + | ||
| 255 | +--- | ||
| 256 | + | ||
| 257 | +## 七、AICPU 曝露率(辅助指标) | ||
| 258 | + | ||
| 259 | +数据来源:`kernel_details.csv` | ||
| 260 | + | ||
| 261 | +``` | ||
| 262 | +AI_CPU_kernels = [row for row if row.Accelerator_Core == "AI_CPU" | ||
| 263 | + or row.Type == "AI_CPU"] | ||
| 264 | + | ||
| 265 | +for each AI_CPU_kernel: | ||
| 266 | + # 检查是否与同时段的 AI_CORE kernel 并发(同 step,时间重叠) | ||
| 267 | + concurrent_core_kernels = [k for k in same_step_kernels | ||
| 268 | + if k.Accelerator_Core in ["AI_CORE", "MIX_AIC"] | ||
| 269 | + and intervals_overlap(k, ai_cpu_kernel)] | ||
| 270 | + if concurrent_core_kernels: | ||
| 271 | + status = "MASKED(被 AI Core 遮蔽,不影响性能)" | ||
| 272 | + else: | ||
| 273 | + status = "EXPOSED(暴露在关键路径,直接增加时延)" | ||
| 274 | +``` | ||
| 275 | + | ||
| 276 | +### 7.1 masked_ratio 精确计算 | ||
| 277 | + | ||
| 278 | +``` | ||
| 279 | +ai_cpu_interval = [start, start + duration] | ||
| 280 | +overlap_with_ai_core = sum(overlap(ai_cpu_interval, core_interval) | ||
| 281 | + for core_interval in merged_ai_core_intervals_same_step) | ||
| 282 | +masked_ratio = overlap_with_ai_core / duration | ||
| 283 | + | ||
| 284 | +if masked_ratio >= 0.9: | ||
| 285 | + aicpu_status = "AICPU_MASKED_BUT_UNDESIRABLE" | ||
| 286 | +elif masked_ratio >= 0.2: | ||
| 287 | + aicpu_status = "AICPU_PARTIALLY_EXPOSED" | ||
| 288 | +else: | ||
| 289 | + aicpu_status = "AICPU_EXPOSED_NOT_ALLOWED" | ||
| 290 | +``` | ||
| 291 | + | ||
| 292 | +--- | ||
| 293 | + | ||
| 294 | +## 八、Host / Device Bound 分类 | ||
| 295 | + | ||
| 296 | +数据来源:`step_trace_time.csv` + `kernel_details.csv` + 可选 `trace_view.json` | ||
| 297 | + | ||
| 298 | +### 8.1 Host-originated risk | ||
| 299 | + | ||
| 300 | +``` | ||
| 301 | +host_risk_score = 0 | ||
| 302 | +if avg_free_ratio > 0.10: | ||
| 303 | + host_risk_score += 2 | ||
| 304 | +if underfeed_ratio > 0.20: | ||
| 305 | + host_risk_score += 2 | ||
| 306 | +if recurring_bubble_pattern: | ||
| 307 | + host_risk_score += 1 | ||
| 308 | +if host_event_overlap_available and max(sync_overlap, comm_overlap, host_visible_coverage) >= 0.20: | ||
| 309 | + host_risk_score += 1 | ||
| 310 | +``` | ||
| 311 | + | ||
| 312 | +判定: | ||
| 313 | + | ||
| 314 | +``` | ||
| 315 | +if host_risk_score >= 4: | ||
| 316 | + overall_bound = "HOST_BOUND" | ||
| 317 | +elif host_risk_score >= 2: | ||
| 318 | + overall_bound = "HOST_ORIGINATED_RISK" | ||
| 319 | +``` | ||
| 320 | + | ||
| 321 | +### 8.2 trace_view Host event overlap | ||
| 322 | + | ||
| 323 | +对每个 bubble window `[gap_start, gap_end]`: | ||
| 324 | + | ||
| 325 | +``` | ||
| 326 | +gap_duration = gap_end - gap_start | ||
| 327 | +host_visible_coverage_ratio = union_overlap(host_events, gap_window) / gap_duration | ||
| 328 | +sync_or_h2d_overlap_ratio = union_overlap(sync_or_h2d_events, gap_window) / gap_duration | ||
| 329 | +comm_marker_overlap_ratio = union_overlap(comm_marker_events, gap_window) / gap_duration | ||
| 330 | + | ||
| 331 | +if sync_or_h2d_overlap_ratio >= 0.20: | ||
| 332 | + soft_label = "possible_sync_or_h2d" | ||
| 333 | +elif comm_marker_overlap_ratio >= 0.20: | ||
| 334 | + soft_label = "possible_comm_wait" | ||
| 335 | +elif host_visible_coverage_ratio >= 0.10: | ||
| 336 | + soft_label = "possible_host_launch_lag" | ||
| 337 | +elif host_visible_coverage_ratio < 0.05: | ||
| 338 | + soft_label = "possible_untraced_host_blocking" | ||
| 339 | +else: | ||
| 340 | + soft_label = "insufficient_evidence" | ||
| 341 | +``` | ||
| 342 | + | ||
| 343 | +### 8.3 Device 子类判定 | ||
| 344 | + | ||
| 345 | +``` | ||
| 346 | +if comm_not_overlap_ratio > 0.30: | ||
| 347 | + device_bound_type = "DEVICE_COMMUNICATION_BOUND" | ||
| 348 | +elif any(kernel.aic_mte2_ratio > 0.8 or kernel.aiv_mte2_ratio > 0.5 or kernel.aiv_mte3_ratio > 0.5): | ||
| 349 | + device_bound_type = "DEVICE_MEMORY_BOUND" | ||
| 350 | +elif any(kernel.aic_mac_ratio > 0.8 or kernel.aiv_vec_ratio > 0.5): | ||
| 351 | + device_bound_type = "DEVICE_COMPUTE_BOUND" | ||
| 352 | +elif many_small_kernels or any(kernel.aiv_scalar_ratio > 0.8): | ||
| 353 | + device_bound_type = "DEVICE_LATENCY_BOUND" | ||
| 354 | +else: | ||
| 355 | + device_bound_type = "DEVICE_BOUND_UNCLASSIFIED" | ||
| 356 | +``` | ||
| 357 | + | ||
| 358 | +### 8.4 Mixed Bound | ||
| 359 | + | ||
| 360 | +``` | ||
| 361 | +if host_risk_score >= 2 and device_bound_type in [ | ||
| 362 | + "DEVICE_COMPUTE_BOUND", "DEVICE_MEMORY_BOUND", "DEVICE_COMMUNICATION_BOUND", "DEVICE_LATENCY_BOUND" | ||
| 363 | +]: | ||
| 364 | + overall_bound = "MIXED_BOUND" | ||
| 365 | + primary = contributor with largest Stage share / severity multiple | ||
| 366 | + secondary = remaining contributors | ||
| 367 | +``` | ||
| 368 | + | ||
| 369 | +--- | ||
| 370 | + | ||
| 371 | +## 九、算子级 Compute / Memory Bound | ||
| 372 | + | ||
| 373 | +### 9.1 msprof op PMU 算术强度 | ||
| 374 | + | ||
| 375 | +数据来源:`OPPROF_*/ArithmeticUtilization.csv` + `Memory.csv` | ||
| 376 | + | ||
| 377 | +``` | ||
| 378 | +total_flops = sum(aic_cube_fops) + sum(aiv_vector_fops) | ||
| 379 | +total_bytes = (sum(GM_to_UB_datas(KB)) + sum(UB_to_GM_datas(KB))) * 1024 | ||
| 380 | +arithmetic_intensity = total_flops / total_bytes | ||
| 381 | + | ||
| 382 | +# 若没有芯片理论内存带宽,使用经验阈值 50 FLOPs/Byte | ||
| 383 | +balance_point = peak_tflops * 1e12 / peak_memory_bandwidth_Bps | ||
| 384 | +threshold = balance_point if peak_memory_bandwidth_Bps is known else 50 | ||
| 385 | + | ||
| 386 | +if arithmetic_intensity < threshold: | ||
| 387 | + op_bound = "OP_MEMORY_BOUND" | ||
| 388 | +else: | ||
| 389 | + op_bound = "OP_COMPUTE_BOUND" | ||
| 390 | +``` | ||
| 391 | + | ||
| 392 | +### 9.2 带宽与冲突辅助判断 | ||
| 393 | + | ||
| 394 | +``` | ||
| 395 | +gm_to_ub_util = mean(GM_to_UB_bw_usage_rate(%)) | ||
| 396 | +ub_to_gm_util = mean(UB_to_GM_bw_usage_rate(%)) | ||
| 397 | + | ||
| 398 | +if gm_to_ub_util < 30 or ub_to_gm_util < 30: | ||
| 399 | + add_flag("LOW_BANDWIDTH_UTILIZATION") | ||
| 400 | + | ||
| 401 | +if aiv_vec_bankgroup_cflt_ratio > 0.10 or aiv_vec_bank_cflt_ratio > 0.10: | ||
| 402 | + add_flag("BANK_CONFLICT_RISK") | ||
| 403 | +``` | ||
| 404 | + | ||
| 405 | +### 9.3 kernel_details V2 退化规则 | ||
| 406 | + | ||
| 407 | +``` | ||
| 408 | +if aic_mte2_ratio > 0.8: | ||
| 409 | + op_bound = "OP_MEMORY_BOUND" | ||
| 410 | +elif aic_mac_ratio > 0.8: | ||
| 411 | + op_bound = "OP_COMPUTE_BOUND" | ||
| 412 | +elif aiv_scalar_ratio > 0.8: | ||
| 413 | + op_bound = "OP_SCALAR_BOUND" | ||
| 414 | +elif aic_fixpipe_ratio > 0.3: | ||
| 415 | + op_bound = "OP_FIXPIPE_BOUND" | ||
| 416 | +else: | ||
| 417 | + op_bound = "OP_MIXED_OR_UNKNOWN" | ||
| 418 | +``` | ||
| 419 | + | ||
| 420 | +--- | ||
| 421 | + | ||
| 422 | +## 十、Wait Pollution / 双热点排名 | ||
| 423 | + | ||
| 424 | +数据来源:`kernel_details.csv` + 可选 `communication.json` / `trace_view.json` | ||
| 425 | + | ||
| 426 | +``` | ||
| 427 | +duration_hotspot_score = sum(Duration_us by op/kernel name) | ||
| 428 | +total_cost_score = sum(Duration_us + Wait_Time_us by op/kernel name) | ||
| 429 | +wait_ratio = Wait_Time_us / (Duration_us + Wait_Time_us) | ||
| 430 | +``` | ||
| 431 | + | ||
| 432 | +若 `total_cost_score` 高但 `duration_hotspot_score` 低: | ||
| 433 | + | ||
| 434 | +``` | ||
| 435 | +if wait_ratio > 0.95 and Duration_us < 10: | ||
| 436 | + flag = "WAIT_ANCHOR_FALSE_HOTSPOT" | ||
| 437 | +elif wait_interval overlaps communication_window: | ||
| 438 | + flag = "WAIT_POLLUTION_RISK" | ||
| 439 | +else: | ||
| 440 | + flag = "WAIT_DOMINATED_HOTSPOT" | ||
| 441 | +``` | ||
| 442 | + | ||
| 443 | +报告中必须同时输出 `duration_hotspots` 与 `total_cost_hotspots`,并解释二者差异。 | ||
| 444 | + | ||
| 445 | +--- | ||
| 446 | + | ||
| 447 | +## 注意事项 | ||
| 448 | + | ||
| 449 | +1. **多 Stream 并发**:`kernel_sum_ms`(Duration 之和)可能远大于 `wall_ms`(wall time),差值体现多 Stream 的并发收益。不要直接用 `kernel_sum_ms` 推断单 stream 耗时。 | ||
| 450 | + | ||
| 451 | +2. **Step Id 缺失时**:按时间戳将 kernel 分配到对应 step 窗口(以 step_trace_time.csv 的 Stage 推算窗口边界)。 | ||
| 452 | + | ||
| 453 | +3. **单步采集**:当 step_trace_time.csv 只有一行时,无法计算步间方差或识别预热步,须在报告中注明"仅单步采集,预热步识别不可用,周期性分析不可用"。 | ||
| @@ -0,0 +1,123 @@ | |||
| 1 | +# 性能判定阈值表 | ||
| 2 | + | ||
| 3 | +> **使用时机**:Phase 3 瓶颈诊断时,对照本表将各维度指标转换为严重度等级(正常 / 警告 / 严重)和优先级(P0–P3)。 | ||
| 4 | + | ||
| 5 | +--- | ||
| 6 | + | ||
| 7 | +## 一、主判定阈值表 | ||
| 8 | + | ||
| 9 | +| 维度 | 指标 | 正常(✅) | 警告(⚠️) | 严重(🔴) | 说明 | | ||
| 10 | +|---|---|---|---|---|---| | ||
| 11 | +| **迭代效率** | `free_ratio` | < 5% | 5%–10% | > 10% | Host Dispatch 瓶颈指标 | | ||
| 12 | +| **迭代效率** | `comm_not_overlap_ratio` | < 15% | 15%–30% | > 30% | 真实通信代价指标 | | ||
| 13 | +| **迭代效率** | `overlap_ratio` | > 50% | 20%–50% | < 20% | 通信遮蔽效率,越高越好 | | ||
| 14 | +| **迭代效率** | `bubble_ratio` | < 2% | 2%–5% | > 5% | 流水线气泡(PP 场景) | | ||
| 15 | +| **算子热点** | 单一算子 `Ratio(%)` | < 10% | 10%–20% | > 20% | 热点算子判定 | | ||
| 16 | +| **算子热点** | AI_CPU 核心类型出现 | 无 | 少量 | 大量 | AICPU 回退风险 | | ||
| 17 | +| **硬件利用率** | MatMul MFU | > 40% | 20%–40% | < 20% | 注:推理 decode batch=1 天然低 MFU,不作为严重告警 | | ||
| 18 | +| **硬件利用率** | `cube_utilization(%)` | > 70% | 40%–70% | < 40% | 流水线内部 Cube 利用率 | | ||
| 19 | +| **硬件利用率** | `aic_mte2_ratio` | < 0.5 | 0.5–0.8 | > 0.8 | 权重读取占主导 → Memory Bound | | ||
| 20 | +| **硬件利用率** | `aic_mac_ratio` | > 0.7 | 0.4–0.7 | < 0.4 | Cube 计算占比低 → 非 Compute Bound | | ||
| 21 | +| **通信效率** | RDMA 带宽 | > 1.5 GB/s | 0.5–1.5 GB/s | < 0.5 GB/s | 小包问题 / 网络拥塞 | | ||
| 22 | +| **通信效率** | Wait Time Ratio | < 0.2 | 0.2–0.5 | > 0.5 | 通信中等待其他 Rank 时间占比 | | ||
| 23 | +| **设备空泡** | `underfeed_ratio` | < 5% | 5%–20% | > 20% | 设备整体空闲占比 | | ||
| 24 | +| **设备空泡** | `prelaunch_gap` | < 1ms | 1ms–5ms | > 5ms | 第一个 kernel 下发前的空闲 | | ||
| 25 | +| **设备空泡** | `largest_internal_bubble` | < 1ms 且 < 10%×Stage | 1ms 或 10%–15%×Stage | > 1ms 且 > 15%×Stage | 最大内部间隙 | | ||
| 26 | +| **等待锚点** | `wait_ratio` | < 0.5 | 0.5–0.95 | > 0.95 且 Duration < 10μs | 假热点判定条件 | | ||
| 27 | +| **多卡均衡** | `variance_ratio` | < 10% | 10%–20% | > 20% | 最大-最小 Stage 差 / 平均 Stage | | ||
| 28 | +| **多卡均衡** | 某卡 `Free` 显著偏高 | < avg×1.5 | avg×1.5–2 | > avg×2 | Host Dispatch 慢卡(假快卡) | | ||
| 29 | +| **Bound 总判定** | `host_risk_score` | < 2 | 2–3 | ≥ 4 | Host-originated risk 综合分 | | ||
| 30 | +| **Host evidence** | `sync_or_h2d_overlap_ratio` | < 10% | 10%–20% | ≥ 20% | bubble 与同步/H2D 标记重叠 | | ||
| 31 | +| **Host evidence** | `comm_marker_overlap_ratio` | < 10% | 10%–20% | ≥ 20% | bubble 与通信等待标记重叠 | | ||
| 32 | +| **算子级 Bound** | Arithmetic Intensity | ≥ balance point | 接近 balance point | < balance point | 低于硬件平衡点 → Memory Bound | | ||
| 33 | +| **算子级 Bound** | GM/UB 带宽利用率 | > 50% | 30%–50% | < 30% | 低带宽利用率,检查访存模式 | | ||
| 34 | +| **算子级 Bound** | `aiv_scalar_ratio` | < 0.5 | 0.5–0.8 | > 0.8 | Scalar / Latency Bound 风险 | | ||
| 35 | +| **算子级 Bound** | `aiv_mte2_ratio` / `aiv_mte3_ratio` | < 0.3 | 0.3–0.5 | > 0.5 | Vector kernel 访存占主导 | | ||
| 36 | +| **算子级 Bound** | Bank conflict ratio | < 5% | 5%–10% | > 10% | UB Bank conflict 风险 | | ||
| 37 | + | ||
| 38 | +--- | ||
| 39 | + | ||
| 40 | +## 二、严重度标签(来自 ascend-profiling-anomaly 规则库) | ||
| 41 | + | ||
| 42 | +| 标签 | 判定条件 | | ||
| 43 | +|---|---| | ||
| 44 | +| `DEVICE_IDLE_GAP_HEAVY` | `underfeed_ratio ≥ 0.30` 或 `largest_internal_bubble ≥ max(1ms, 10% × Stage)` | | ||
| 45 | +| `PRELAUNCH_GAP_HEAVY` | `prelaunch_gap ≥ max(1ms, 10% × Stage)` | | ||
| 46 | +| `TAIL_GAP_HEAVY` | `tail_gap ≥ max(1ms, 10% × Stage)` | | ||
| 47 | +| `INTERNAL_BUBBLE_HEAVY` | `internal_bubble_total ≥ 20% × Stage` | | ||
| 48 | +| `WAIT_ANCHOR_FALSE_HOTSPOT` | `wait_ratio > 0.95 AND Duration_us < 10.0` | | ||
| 49 | +| `WAIT_POLLUTION_RISK` | 高 wait kernel 的 wait 区间与通信窗口重叠 | | ||
| 50 | +| `AICPU_EXPOSED_RISK` | 存在 AI_CPU kernel 且无并发 AI_CORE 覆盖 | | ||
| 51 | +| `AICPU_MASKED_BUT_UNDESIRABLE` | `masked_ratio >= 0.9` | | ||
| 52 | +| `AICPU_PARTIALLY_EXPOSED` | `0.2 <= masked_ratio < 0.9` | | ||
| 53 | +| `AICPU_EXPOSED_NOT_ALLOWED` | `masked_ratio < 0.2` | | ||
| 54 | +| `COMM_SYNC_RISK` | `Wait Time Ratio > 0.5`(通信中大量时间等待 Rank 同步) | | ||
| 55 | +| `HOST_DISPATCH_BOTTLENECK` | `free_ratio > 0.10` | | ||
| 56 | +| `COMMUNICATION_BOTTLENECK` | `comm_not_overlap_ratio > 0.30` | | ||
| 57 | +| `LOAD_IMBALANCE` | 多卡 `variance_ratio > 0.20` | | ||
| 58 | +| `HOST_BOUND` | `free_ratio > 0.10` 或 `underfeed_ratio > 0.20`,且 Host-originated risk 为主因 | | ||
| 59 | +| `DEVICE_COMPUTE_BOUND` | `aic_mac_ratio > 0.8` 或算术强度高且计算利用率高 | | ||
| 60 | +| `DEVICE_MEMORY_BOUND` | `aic_mte2_ratio > 0.8` 或算术强度低于平衡点 | | ||
| 61 | +| `DEVICE_COMMUNICATION_BOUND` | `comm_not_overlap_ratio > 0.30` 或通信 wait/bandwidth 异常 | | ||
| 62 | +| `DEVICE_LATENCY_BOUND` | 计算/访存利用率均低,小 kernel 多或 `aiv_scalar_ratio > 0.8` | | ||
| 63 | +| `MIXED_BOUND` | Host 与 Device 子类同时超过阈值 | | ||
| 64 | + | ||
| 65 | +--- | ||
| 66 | + | ||
| 67 | +## 三、优先级分类规则(P0–P3) | ||
| 68 | + | ||
| 69 | +| 优先级 | 判定条件 | 含义 | | ||
| 70 | +|---|---|---| | ||
| 71 | +| **P0** | 任意单一指标超过严重阈值 **2 倍以上** | 最高优先级,需立即处理;例:free_ratio = 82%(严重阈值 10% 的 8 倍) | | ||
| 72 | +| **P1** | 任意单一指标落入严重范围(未达到 2 倍) | 高优先级;例:comm_not_overlap_ratio = 35% | | ||
| 73 | +| **P2** | 多个指标同时处于警告范围,互相叠加 | 中优先级;例:free_ratio 15% + overlap_ratio 25% 同时出现 | | ||
| 74 | +| **P3** | 单一指标处于警告范围,其他指标正常 | 低优先级,观察跟踪;例:单一算子 Ratio 17% | | ||
| 75 | + | ||
| 76 | +**归因语言规范**: | ||
| 77 | +- 测量到的事实 → 陈述句("free_ratio = 82%,超过严重阈值 10% 的 8 倍") | ||
| 78 | +- 推断的原因 → 用 `possible_xxx`("可能由于 Python eager 模式小算子碎片化下发") | ||
| 79 | +- 证据不足时 → 明确说明"归因证据不足,建议重采集 host stack 数据" | ||
| 80 | +- Bound 诊断必须写清楚 **事实 → 阈值 → 判定理由 → 置信度 → 缺失证据**。 | ||
| 81 | +- 没有 `kernel_details.csv` V2 或 msprof op PMU 时,不得输出算子 compute/memory bound,只能输出热点和"证据不足"。 | ||
| 82 | + | ||
| 83 | +--- | ||
| 84 | + | ||
| 85 | +## 四、Bound 分类决策表 | ||
| 86 | + | ||
| 87 | +| 主判定 | 必要证据 | 排除/降级条件 | | ||
| 88 | +|---|---|---| | ||
| 89 | +| `HOST_BOUND` | `free_ratio > 10%` 或 `underfeed_ratio > 20%`,且 Stage 主要被 Free/空泡占用 | 若通信未重叠 >30% 或 PMU 显示设备 kernel 饱和,应降为 `MIXED_BOUND` | | ||
| 90 | +| `DEVICE_COMPUTE_BOUND` | Host 空闲不高;`aic_mac_ratio > 0.8`、MFU/利用率高或 AI 高 | Decode M=1 低 MFU 不作为异常;若 MTE 高则改为 memory | | ||
| 91 | +| `DEVICE_MEMORY_BOUND` | Host 空闲不高;`aic_mte2_ratio > 0.8`、Vector MTE 高、AI 低或带宽利用异常 | 若带宽数据缺失,只能中/低置信 | | ||
| 92 | +| `DEVICE_COMMUNICATION_BOUND` | `comm_not_overlap_ratio > 30%` 或通信 wait ratio >0.5,overlap 低 | `Communication=0` 时禁止输出通信优化建议 | | ||
| 93 | +| `DEVICE_LATENCY_BOUND` | 小 kernel 多、scalar 占比高、计算和带宽利用率均低 | 若 Free 很高,优先判 Host Bound | | ||
| 94 | +| `MIXED_BOUND` | 两类以上证据同时超过阈值 | 必须排序主因/次因 | | ||
| 95 | + | ||
| 96 | +--- | ||
| 97 | + | ||
| 98 | +## 五、预热步识别规则 | ||
| 99 | + | ||
| 100 | +| 场景 | 判定方式 | | ||
| 101 | +|---|---| | ||
| 102 | +| 第一个 Step 的 `comm_not_overlap_ratio` 比后续步高 **> 20%** | 标记为"疑似预热步(初始 AllReduce 含初始化开销)" | | ||
| 103 | +| 第一个 Step 的 `Computing` 比后续步低 **> 20%** | 标记为"疑似预热步(编译/JIT 预热未完成)" | | ||
| 104 | +| 仅有单步 | 无法判断是否为预热步,注明"单步采集,建议多步验证" | | ||
| 105 | + | ||
| 106 | +预热步**不参与**迭代效率均值计算,须单独列出并注明。 | ||
| 107 | + | ||
| 108 | +--- | ||
| 109 | + | ||
| 110 | +## 六、特殊场景处理说明 | ||
| 111 | + | ||
| 112 | +### 推理 Decode 单 Token(batch=1) | ||
| 113 | +- **表现**:MatMul MFU < 5%,这是正常现象,**不判定为 P0/P1** | ||
| 114 | +- **原因**:单 token 时矩阵 M=1,计算量极小,无法充分利用 AI Core | ||
| 115 | +- **处理**:报告中注明"推理 decode 单 token,绝对 MFU 低属正常;建议关注相对效率(cube_utilization)和通信占比" | ||
| 116 | + | ||
| 117 | +### 单 Rank 采集(通信带宽全为 0) | ||
| 118 | +- **表现**:`communication.json` 中所有 `Transit Size(MB) = 0` | ||
| 119 | +- **处理**:仅报告 `Elapse Time`,不输出带宽数字,注明"单 Rank 采集,无跨 Rank 通信带宽数据" | ||
| 120 | + | ||
| 121 | +### MoE 模型特征识别 | ||
| 122 | +- **正常 MoE 特征**:GroupedMatmul、MoeGatingTopKSoftmax、MoeComputeExpertTokens、DispatchFFNCombine、MoeInitRoutingV3 等 MoE 专属算子出现属预期行为 | ||
| 123 | +- **不应**将 GroupedMatmul 的高 Ratio(%) 单独标为热点,需结合模型架构说明其为 MoE Expert 计算的正常开销 | ||
| @@ -0,0 +1,223 @@ | |||
| 1 | +--- | ||
| 2 | +name: cann-perf-breakdown | ||
| 3 | +description: | | ||
| 4 | + NPU 性能数据拆解技能。将 kernel_details.csv 中的性能数据按模型结构进行拆解分析。 | ||
| 5 | + 触发场景:分析 kernel_details.csv、拆解性能数据到模型层级、分析大模型性能瓶颈、仅模型代码做架构拆解、仅性能数据做诊断(委托 cann-npu-perfanalysis)。 | ||
| 6 | +--- | ||
| 7 | + | ||
| 8 | +# NPU 性能数据拆解技能 | ||
| 9 | + | ||
| 10 | +将 NPU profiling 输出的 `kernel_details.csv` 按 Transformer 模型结构进行层级拆解,生成结构化的 JSON 和 Markdown/HTML 分析报告。 | ||
| 11 | + | ||
| 12 | +--- | ||
| 13 | + | ||
| 14 | +## 适用场景 | ||
| 15 | + | ||
| 16 | +| 场景 | 说明 | | ||
| 17 | +|------|------| | ||
| 18 | +| 大模型性能分析 | 分析各类 Transformer 模型 | | ||
| 19 | +| 层级耗时拆解 | 拆解 Embedding/Block/Head 各层耗时 | | ||
| 20 | +| Block 内部分析 | 分析 Attention、MLP、Norm 子层 | | ||
| 21 | +| 架构特征识别 | 识别 MLA、MoE、GQA 等架构 | | ||
| 22 | +| 仅模型代码 | 仅做结构拆解,不绑定具体性能数据(多分支用 `branches` 表达) | | ||
| 23 | +| 仅性能数据 | 委托给 `cann-npu-perfanalysis` sibling skill 做 8 维诊断 | | ||
| 24 | + | ||
| 25 | +--- | ||
| 26 | + | ||
| 27 | +## 入口分派(Mode 判定) | ||
| 28 | + | ||
| 29 | +启动时检查工作目录,按以下优先级判定模式: | ||
| 30 | + | ||
| 31 | +| 条件 | 模式 | 行为 | | ||
| 32 | +|---|---|---| | ||
| 33 | +| 有模型源码(`*modeling*.py` 等) **且** 有 `kernel_details.csv` 或 `raw_ops*.json` | **Mode A** | 完整 5 步流程,输出 `analysis_config.json` + 报告 + 指标 | | ||
| 34 | +| 仅模型源码 | **Mode B** | 仅做结构拆解,输出 `model_structure.json`(schema 沿用 analysis_config.json,`op_indices=[]`,可加 `branches` 字段)。详见 `references/mode_b_branches.md` | | ||
| 35 | +| 仅性能数据(csv 或 `ASCEND_PROFILER_OUTPUT/`) | **Mode C** | 委托 `cann-npu-perfanalysis` sibling skill。详见 `references/mode_c_delegate.md` | | ||
| 36 | + | ||
| 37 | +> Mode B 与 Mode C 不进入 Step 3/4/5;Mode A 走完整 5 步。 | ||
| 38 | + | ||
| 39 | +--- | ||
| 40 | + | ||
| 41 | +## 工作流(Mode A) | ||
| 42 | + | ||
| 43 | +``` | ||
| 44 | +Step 1: analyze_kernels.py | ||
| 45 | + → raw_ops.json + raw_ops_details.json + raw_ops.compact.json | ||
| 46 | + ↓ | ||
| 47 | +Step 2: AI 拆解(投喂 raw_ops.compact.json + 模型源码) | ||
| 48 | + → analysis_config.json(首版) | ||
| 49 | + ↓ | ||
| 50 | +Step 3: Review(脚本检查 → AI 仅看 issue 列表 → 修正循环) | ||
| 51 | + → analysis_config.json(终版) | ||
| 52 | + ↓ | ||
| 53 | +Step 4: generate_report.py(默认 MD + HTML) | ||
| 54 | + → {prefix}_report.md / {prefix}_report.html | ||
| 55 | + ↓ | ||
| 56 | +Step 5: compute_metrics.py | ||
| 57 | + → metrics_report.md | ||
| 58 | +``` | ||
| 59 | + | ||
| 60 | +--- | ||
| 61 | + | ||
| 62 | +### Step 1: 提取单 Step 数据 | ||
| 63 | + | ||
| 64 | +先枚举所有 step 并选择非 warmup 的代表 step。未显式传 `-s` 时, | ||
| 65 | +`analyze_kernels.py` 会按 kernel_count + kernel 类型分布寻找稳定 step 组; | ||
| 66 | +若该组最早 step 的 kernel_sum 相比后续 step 中位数是明显离群值,则跳过该 | ||
| 67 | +warmup/outlier step,并选择后续 step 中最接近中位数的一步。若最早 step 不离群, | ||
| 68 | +保留最早稳定 step。需要强制复现某个 step 时再显式传 `-s`。 | ||
| 69 | + | ||
| 70 | +```bash | ||
| 71 | +python scripts/analyze_kernels.py \ | ||
| 72 | + -f kernel_details.csv \ | ||
| 73 | + -o outputs/raw_ops.json \ | ||
| 74 | + -d outputs/raw_ops_details.json \ | ||
| 75 | + -m outputs/steps_summary.md \ | ||
| 76 | + --compact-out outputs/raw_ops.compact.json | ||
| 77 | +``` | ||
| 78 | + | ||
| 79 | +**作用**:从 `kernel_details.csv` 提取单个 step 数据,避免后续分析 token 消耗过大。 | ||
| 80 | + | ||
| 81 | +| 输出 | 用途 | | ||
| 82 | +|---|---| | ||
| 83 | +| `steps_summary.md` | 所有 step 的 kernel_count、kernel 类型和耗时概览,含自动选步原因 | | ||
| 84 | +| `raw_ops.json` | 自动选择或 `-s` 指定的单 Step kernel 概要,enrich/校验脚本用 | | ||
| 85 | +| `raw_ops_details.json` | 单 Step kernel 详情(含 CSV 全部字段),Step 4/5 报告与指标用 | | ||
| 86 | +| `raw_ops.compact.json` | Step 2 投喂给 AI 的精简视图(删除 `start_time_us`/`duration_us`,连续相同算子折叠)| | ||
| 87 | + | ||
| 88 | +每个 operator 含 `org_index`,表示其在 `kernel_details.csv` 中的 0-based 行号。 | ||
| 89 | + | ||
| 90 | +**可选**: | ||
| 91 | + | ||
| 92 | +```bash | ||
| 93 | +python scripts/segment_layers.py -r outputs/raw_ops.json -o outputs/op_segments.json | ||
| 94 | +``` | ||
| 95 | + | ||
| 96 | +为 Step 2 生成 layer 边界**候选**(基于最长重复子段)。AI 可作为定位参考,最终边界以源码语义为准。 | ||
| 97 | + | ||
| 98 | +**参考**:`references/kernel_data_guide.md` | ||
| 99 | + | ||
| 100 | +--- | ||
| 101 | + | ||
| 102 | +### Step 2: 拆解模型结构 | ||
| 103 | + | ||
| 104 | +按 `references/structure_analysis_guide.md` §D.1 执行: | ||
| 105 | + | ||
| 106 | +1. 选择代表性 step 与 decoder layer 实例 | ||
| 107 | +2. 阅读模型源码,提取模块层级和稳定函数边界 | ||
| 108 | +3. 在 op 序列中定位 decoder layer 边界(可参考 `op_segments.json`) | ||
| 109 | +4. 逐层映射 op 到源码模块或函数语义 | ||
| 110 | +5. 输出 `analysis_config.json` | ||
| 111 | +6. 运行 enrich: | ||
| 112 | + | ||
| 113 | +```bash | ||
| 114 | +python scripts/analyze_kernels.py --enrich \ | ||
| 115 | + -c outputs/analysis_config.json \ | ||
| 116 | + -r outputs/raw_ops.json | ||
| 117 | +``` | ||
| 118 | + | ||
| 119 | +字段规范、节点来源规则、命名规则、边界规则、shape_semantic 必填范围(11 类算子 + 统一维度符号 B/T/H/D/...)等**全部细节**见 `references/structure_analysis_guide.md`。SKILL.md 不再重述以避免分歧。 | ||
| 120 | + | ||
| 121 | +--- | ||
| 122 | + | ||
| 123 | +### Step 3: Review 拆解结果 | ||
| 124 | + | ||
| 125 | +按 `references/structure_analysis_guide.md` §D.2 执行: | ||
| 126 | + | ||
| 127 | +```bash | ||
| 128 | +python scripts/check_structure.py -c outputs/analysis_config.json --json > outputs/issues.json | ||
| 129 | +python scripts/validate_shapes.py -c outputs/analysis_config.json --fail-fast --json >> outputs/issues.json | ||
| 130 | +python scripts/check_op_coverage.py -c outputs/analysis_config.json -r outputs/raw_ops.json --json >> outputs/issues.json | ||
| 131 | +``` | ||
| 132 | + | ||
| 133 | +- `issues.json` 为空 → 跳过 AI review,直接进入 Step 4 | ||
| 134 | +- 否则拉起 review subagent,**仅投喂** issue 命中的节点片段、对应源码切片(按 `code_ref` 稀疏读)、对应 raw_ops 切片;subagent 修正后回到本步重跑,迭代上限 3 次 | ||
| 135 | + | ||
| 136 | +--- | ||
| 137 | + | ||
| 138 | +### Step 4: 生成报告 | ||
| 139 | + | ||
| 140 | +```bash | ||
| 141 | +python scripts/generate_report.py \ | ||
| 142 | + -r outputs/raw_ops_details.json \ | ||
| 143 | + -c outputs/analysis_config.json \ | ||
| 144 | + -o outputs/{prefix}_report.md \ | ||
| 145 | + --html -d 3 | ||
| 146 | +``` | ||
| 147 | + | ||
| 148 | +| 输出 | 说明 | | ||
| 149 | +|---|---| | ||
| 150 | +| `{prefix}_report.md` | Markdown 分析报告 | | ||
| 151 | +| `{prefix}_report.html` | HTML 分析报告(默认产出) | | ||
| 152 | + | ||
| 153 | +--- | ||
| 154 | + | ||
| 155 | +### Step 5: 计算性能指标 | ||
| 156 | + | ||
| 157 | +```bash | ||
| 158 | +python scripts/compute_metrics.py \ | ||
| 159 | + -r outputs/raw_ops_details.json \ | ||
| 160 | + -c outputs/analysis_config.json \ | ||
| 161 | + -o outputs/metrics_report.md \ | ||
| 162 | + -d 3 | ||
| 163 | +``` | ||
| 164 | + | ||
| 165 | +**四维基础指标**: | ||
| 166 | + | ||
| 167 | +| 指标 | 定义 | 含义 | | ||
| 168 | +|------|------|------| | ||
| 169 | +| `wall_ms` | 最后 kernel 结束 - 首个 kernel 开始 | 实际墙上时钟耗时(含间隙) | | ||
| 170 | +| `busy_union_ms` | 合并后的设备忙碌时间 | 设备实际利用率(去重叠) | | ||
| 171 | +| `kernel_sum_ms` | 所有 kernel 时长的算术和 | 总计算量(忽略重叠) | | ||
| 172 | +| `total_cost_ms` | Σ(duration + wait) | 完整成本(含等待) | | ||
| 173 | + | ||
| 174 | +**衍生指标**:`并行度 = kernel_sum_ms / wall_ms`、`bubble_ms = wall_ms - busy_union_ms`、`占比% = wall_ms / step_wall × 100`。 | ||
| 175 | + | ||
| 176 | +**诊断规则**: | ||
| 177 | + | ||
| 178 | +| 条件 | 阈值 | 诊断结论 | | ||
| 179 | +|------|------|----------| | ||
| 180 | +| kernel_sum > wall | > 1.5× | 高流并行度(多流重叠执行) | | ||
| 181 | +| kernel_sum > wall | > 1.2× | 中等流并行 | | ||
| 182 | +| wall > busy_union | > 1.5× | 存在间隙气泡 | | ||
| 183 | +| total_cost > kernel_sum | > 1.3× | 等待时间显著,检查 wait-anchor 热点 | | ||
| 184 | +| busy_union / wall | 80%~95% | 利用率良好 | | ||
| 185 | +| busy_union / wall | < 80% | 利用率偏低 | | ||
| 186 | +| busy_union ≈ wall ≈ kernel_sum | 偏差 < 10% | 干净顺序执行 | | ||
| 187 | + | ||
| 188 | +**适用范围**:decoder layer 及子节点、stages、runtime_auxiliary 及子节点。 | ||
| 189 | + | ||
| 190 | +--- | ||
| 191 | + | ||
| 192 | +## 输出文件 | ||
| 193 | + | ||
| 194 | +| 文件 | 说明 | | ||
| 195 | +|------|------| | ||
| 196 | +| `raw_ops.json` | 单 Step kernel 概要(脚本用) | | ||
| 197 | +| `raw_ops_details.json` | 单 Step kernel 详情(Step 4/5 用) | | ||
| 198 | +| `raw_ops.compact.json` | Step 2 投喂 AI 的精简视图 | | ||
| 199 | +| `op_segments.json`(可选) | layer 边界候选 | | ||
| 200 | +| `analysis_config.json` | 拆解配置(Mode A 终版)| | ||
| 201 | +| `model_structure.json` | 仅结构(Mode B)| | ||
| 202 | +| `issues.json` | Step 3 检查结果 | | ||
| 203 | +| `{prefix}_report.md` / `.html` | 分析报告 | | ||
| 204 | +| `metrics_report.md` | 性能指标分析报告 | | ||
| 205 | + | ||
| 206 | +--- | ||
| 207 | + | ||
| 208 | +## 参考资源 | ||
| 209 | + | ||
| 210 | +| 文件 | 何时查阅 | | ||
| 211 | +|------|----------| | ||
| 212 | +| `references/structure_analysis_guide.md` | Step 2 拆解 + Step 3 Review 全部规则细节 | | ||
| 213 | +| `references/kernel_data_guide.md` | Step 1 — `kernel_details.csv` 字段说明 | | ||
| 214 | +| `references/mode_b_branches.md` | Mode B 多分支表达约定 | | ||
| 215 | +| `references/mode_c_delegate.md` | Mode C 委托 `cann-npu-perfanalysis` 模板 | | ||
| 216 | +| `scripts/analyze_kernels.py` | Step 1 提取与 Step 2 enrich | | ||
| 217 | +| `scripts/segment_layers.py` | layer 边界候选 | | ||
| 218 | +| `scripts/check_structure.py` | Step 3 树结构良构性 | | ||
| 219 | +| `scripts/validate_shapes.py` | Step 3 shape_semantic 一致性 | | ||
| 220 | +| `scripts/check_op_coverage.py` | Step 3 op 全覆盖 | | ||
| 221 | +| `scripts/regression_check.py` | 与 baseline 做结构回归 | | ||
| 222 | +| `scripts/generate_report.py` | Step 4 报告生成 | | ||
| 223 | +| `scripts/compute_metrics.py` | Step 5 指标计算 | | ||
| @@ -0,0 +1,140 @@ | |||
| 1 | +# NPU性能数据说明 | ||
| 2 | + | ||
| 3 | +## kernel_details.csv | ||
| 4 | + | ||
| 5 | +This is the most granular device-side data. Each row is one kernel invocation on the NPU. | ||
| 6 | + | ||
| 7 | +--- | ||
| 8 | + | ||
| 9 | +## 字段列表 | ||
| 10 | + | ||
| 11 | +### 基础标识 | ||
| 12 | + | ||
| 13 | +| Column | Type | Unit | Description | | ||
| 14 | +|---|---|---|---| | ||
| 15 | +| `Step Id` | integer | — | Training/inference step identifier | | ||
| 16 | +| `Device_id` | integer | — | NPU device identifier | | ||
| 17 | +| `Model ID` | integer | — | Model identifier | | ||
| 18 | +| `Task ID` | integer | — | Task identifier | | ||
| 19 | +| `Stream ID` | integer | — | Device stream this kernel ran on | | ||
| 20 | +| `Name` | string | — | Kernel or task name, e.g. `MatMul`, `Add`, `HcomAllReduce` | | ||
| 21 | +| `Type` | string | — | Kernel type name | | ||
| 22 | + | ||
| 23 | +### 执行状态 | ||
| 24 | + | ||
| 25 | +| Column | Type | Unit | Description | | ||
| 26 | +|---|---|---|---| | ||
| 27 | +| `OP State` | string | — | Operation execution state | | ||
| 28 | +| `Accelerator Core` | string | — | Core type used: `AiCore`, `AiCpu`, `AiVector`, `MixAic`, etc. | | ||
| 29 | + | ||
| 30 | +### 时间信息 | ||
| 31 | + | ||
| 32 | +| Column | Type | Unit | Description | | ||
| 33 | +|---|---|---|---| | ||
| 34 | +| `Start Time(us)` | float | μs | Absolute start timestamp on device clock | | ||
| 35 | +| `Duration(us)` | float | μs | Kernel execution time (device busy) | | ||
| 36 | +| `Wait Time(us)` | float | μs | Time kernel spent waiting before execution started | | ||
| 37 | + | ||
| 38 | +### 并行参数 | ||
| 39 | + | ||
| 40 | +| Column | Type | Unit | Description | | ||
| 41 | +|---|---|---|---| | ||
| 42 | +| `Block Dim` / `Block Num` | integer | — | Parallelism dimension | | ||
| 43 | +| `Mix Block Dim` / `Mix Block Num` | integer | — | Mixed parallelism dimension | | ||
| 44 | +| `HF32 Eligible` | string | — | HF32 eligibility flag | | ||
| 45 | + | ||
| 46 | +### 张量信息(可选) | ||
| 47 | + | ||
| 48 | +以下字段取决于 `record_shapes` 配置,可能为空: | ||
| 49 | + | ||
| 50 | +| Column | Type | Unit | Description | | ||
| 51 | +|---|---|---|---| | ||
| 52 | +| `Input Shapes` | string | — | e.g. `"[2048,4096];[4096,4096]"` | | ||
| 53 | +| `Output Shapes` | string | — | e.g. `"[2048,4096]"` | | ||
| 54 | +| `Input Data Types` | string | — | e.g. `"FLOAT16;FLOAT16"` | | ||
| 55 | +| `Output Data Types` | string | — | e.g. `"FLOAT16"` | | ||
| 56 | +| `Input Formats` | string | — | e.g. `"ND;ND"` | | ||
| 57 | +| `Output Formats` | string | — | e.g. `"ND"` | | ||
| 58 | + | ||
| 59 | +### 上下文 | ||
| 60 | + | ||
| 61 | +| Column | Type | Unit | Description | | ||
| 62 | +|---|---|---|---| | ||
| 63 | +| `Context ID` | integer | — | Context identifier | | ||
| 64 | + | ||
| 65 | +### AI Core 指标 | ||
| 66 | + | ||
| 67 | +| Column | Type | Unit | Description | | ||
| 68 | +|---|---|---|---| | ||
| 69 | +| `aicore_time(us)` | float | μs | AI Core execution time | | ||
| 70 | +| `aic_total_cycles` | integer | — | AI Core total cycles | | ||
| 71 | + | ||
| 72 | +**AI Core 详细指标(可选,取决于 profiling 配置)**: | ||
| 73 | + | ||
| 74 | +| Column | Type | Unit | Description | | ||
| 75 | +|---|---|---|---| | ||
| 76 | +| `aic_mac_time(us)` | float | μs | MAC unit time | | ||
| 77 | +| `aic_mac_ratio` | float | — | MAC time ratio (0-1) | | ||
| 78 | +| `aic_scalar_time(us)` | float | μs | Scalar unit time | | ||
| 79 | +| `aic_scalar_ratio` | float | — | Scalar time ratio (0-1) | | ||
| 80 | +| `aic_mte1_time(us)` | float | μs | MTE1 (memory transfer engine 1) time | | ||
| 81 | +| `aic_mte1_ratio` | float | — | MTE1 time ratio (0-1) | | ||
| 82 | +| `aic_mte2_time(us)` | float | μs | MTE2 time | | ||
| 83 | +| `aic_mte2_ratio` | float | — | MTE2 time ratio (0-1) | | ||
| 84 | +| `aic_fixpipe_time(us)` | float | μs | FixPipe unit time | | ||
| 85 | +| `aic_fixpipe_ratio` | float | — | FixPipe time ratio (0-1) | | ||
| 86 | +| `aic_icache_miss_rate` | float | — | AI Core ICache miss rate | | ||
| 87 | +| `cube_utilization(%)` | float | % | Cube utilization percentage | | ||
| 88 | + | ||
| 89 | +**AI Core FLOPs**: | ||
| 90 | + | ||
| 91 | +| Column | Type | Unit | Description | | ||
| 92 | +|---|---|---|---| | ||
| 93 | +| `aic_mac_fp16_ratio` | float | — | FP16 MAC ratio | | ||
| 94 | +| `aic_mac_int8_ratio` | float | — | INT8 MAC ratio | | ||
| 95 | +| `aic_cube_fops` | integer | — | Cube FLOPs count | | ||
| 96 | + | ||
| 97 | +### AI Vector 指标 | ||
| 98 | + | ||
| 99 | +| Column | Type | Unit | Description | | ||
| 100 | +|---|---|---|---| | ||
| 101 | +| `aiv_time(us)` | float | μs | AI Vector execution time | | ||
| 102 | +| `aiv_total_cycles` | integer | — | AI Vector total cycles | | ||
| 103 | + | ||
| 104 | +**AI Vector 详细指标(可选,取决于 profiling 配置)**: | ||
| 105 | + | ||
| 106 | +| Column | Type | Unit | Description | | ||
| 107 | +|---|---|---|---| | ||
| 108 | +| `aiv_vec_time(us)` | float | μs | Vector unit time | | ||
| 109 | +| `aiv_vec_ratio` | float | — | Vector time ratio (0-1) | | ||
| 110 | +| `aiv_scalar_time(us)` | float | μs | AI Vector scalar unit time | | ||
| 111 | +| `aiv_scalar_ratio` | float | — | AI Vector scalar time ratio (0-1) | | ||
| 112 | +| `aiv_mte2_time(us)` | float | μs | AI Vector MTE2 time | | ||
| 113 | +| `aiv_mte2_ratio` | float | — | AI Vector MTE2 time ratio (0-1) | | ||
| 114 | +| `aiv_mte3_time(us)` | float | μs | AI Vector MTE3 time | | ||
| 115 | +| `aiv_mte3_ratio` | float | — | AI Vector MTE3 time ratio (0-1) | | ||
| 116 | +| `aiv_icache_miss_rate` | float | — | AI Vector ICache miss rate | | ||
| 117 | + | ||
| 118 | +**AI Vector FLOPs**: | ||
| 119 | + | ||
| 120 | +| Column | Type | Unit | Description | | ||
| 121 | +|---|---|---|---| | ||
| 122 | +| `aiv_vec_fp32_ratio` | float | — | FP32 vector ratio | | ||
| 123 | +| `aiv_vec_fp16_ratio` | float | — | FP16 vector ratio | | ||
| 124 | +| `aiv_vec_int32_ratio` | float | — | INT32 vector ratio | | ||
| 125 | +| `aiv_vec_misc_ratio` | float | — | Misc vector ratio | | ||
| 126 | +| `aiv_vector_fops` | integer | — | Vector FLOPs count | | ||
| 127 | + | ||
| 128 | +--- | ||
| 129 | + | ||
| 130 | +## Task Type 取值 | ||
| 131 | + | ||
| 132 | +| Task Type | Description | | ||
| 133 | +|-----------|-------------| | ||
| 134 | +| `AI_CORE` | AI Core kernel | | ||
| 135 | +| `AI_CPU` | AI CPU kernel | | ||
| 136 | +| `HCCL` | Collective communication | | ||
| 137 | +| `MIX_AIC` | Mixed AI Core | | ||
| 138 | +| `MIX_AIV` | Mixed AI Vector | | ||
| 139 | +| `FFTS_PLUS` | FFTS Plus | | ||
| 140 | +| `DVPP` | Digital Vision Pre-Processing | | ||
| @@ -0,0 +1,146 @@ | |||
| 1 | +# Mode B:仅模型代码的结构拆解 | ||
| 2 | + | ||
| 3 | +当工作目录中**只有模型源码而无性能数据**时,本 skill 进入 Mode B:仅做静态结构拆解,输出 `model_structure.json`。 | ||
| 4 | + | ||
| 5 | +--- | ||
| 6 | + | ||
| 7 | +## 使用条件 | ||
| 8 | + | ||
| 9 | +- 存在模型源码(`*modeling*.py` 或类似入口) | ||
| 10 | +- 不存在 `kernel_details.csv`、`raw_ops*.json`、`ASCEND_PROFILER_OUTPUT/` | ||
| 11 | + | ||
| 12 | +> 若两类输入同时存在 → Mode A;仅有性能数据 → Mode C(见 `mode_c_delegate.md`)。 | ||
| 13 | + | ||
| 14 | +--- | ||
| 15 | + | ||
| 16 | +## 输出文件 | ||
| 17 | + | ||
| 18 | +`outputs/model_structure.json`,schema 沿用 `analysis_config.json`(见 `structure_analysis_guide.md` §C),但有以下差异: | ||
| 19 | + | ||
| 20 | +| 字段 | Mode A | Mode B | | ||
| 21 | +|---|---|---| | ||
| 22 | +| 顶层 `mode` | 不存在 | `"structure_only"` | | ||
| 23 | +| 叶节点 `op_indices` | 必有 | **必为 `[]`**(无性能数据可绑定) | | ||
| 24 | +| 节点 `kernels` 数组 | 11 类算子必填 `shape_semantic` | **不填写**(无 raw_ops 可核对) | | ||
| 25 | +| 节点 `branches` | 不存在 | **可选**:表达静态分析无法消歧的分支 | | ||
| 26 | +| `representative_step` | step ID(数字) | `null` | | ||
| 27 | + | ||
| 28 | +--- | ||
| 29 | + | ||
| 30 | +## branches 字段:表达多分支拓扑 | ||
| 31 | + | ||
| 32 | +源码中存在条件分支但**无法仅靠静态分析判定哪条分支会执行**时(典型:MoE-vs-dense layer、sliding-vs-full attention、是否启用 MTP),在节点上加 `branches` 字段,把每条可能的分支作为一个完整子树列出。 | ||
| 33 | + | ||
| 34 | +### Schema | ||
| 35 | + | ||
| 36 | +```json | ||
| 37 | +{ | ||
| 38 | + "name": "<branch_parent_name>", | ||
| 39 | + "semantic": "<分支语义说明>", | ||
| 40 | + "code_ref": "<源码位置>", | ||
| 41 | + "branches": [ | ||
| 42 | + { | ||
| 43 | + "condition": "<判断条件,源码原文或等价 Python 表达式>", | ||
| 44 | + "name": "<本分支名(建议直接用源码类名/函数名)>", | ||
| 45 | + "semantic": "<本分支说明>", | ||
| 46 | + "code_ref": "<本分支起止行>", | ||
| 47 | + "children": [...] | ||
| 48 | + }, | ||
| 49 | + ... | ||
| 50 | + ] | ||
| 51 | +} | ||
| 52 | +``` | ||
| 53 | + | ||
| 54 | +### 示例 | ||
| 55 | + | ||
| 56 | +#### 例 1:MoE vs dense decoder layer | ||
| 57 | + | ||
| 58 | +```json | ||
| 59 | +{ | ||
| 60 | + "name": "decoder_layer_variants", | ||
| 61 | + "semantic": "Decoder layer with MoE-vs-dense topology determined by config.use_moe", | ||
| 62 | + "code_ref": "modeling.py:200-350", | ||
| 63 | + "branches": [ | ||
| 64 | + { | ||
| 65 | + "condition": "config.use_moe == True", | ||
| 66 | + "name": "Gemma4DecoderLayer_moe", | ||
| 67 | + "semantic": "MoE decoder layer with topK expert routing", | ||
| 68 | + "code_ref": "modeling.py:200-280", | ||
| 69 | + "children": [ | ||
| 70 | + {"name": "input_layernorm", "code_ref": "modeling.py:220"}, | ||
| 71 | + {"name": "self_attn", "code_ref": "modeling.py:225-260"}, | ||
| 72 | + {"name": "moe_block", "code_ref": "modeling.py:265-280", "children": [ | ||
| 73 | + {"name": "gate", "code_ref": "modeling.py:266"}, | ||
| 74 | + {"name": "experts", "code_ref": "modeling.py:268-278"} | ||
| 75 | + ]} | ||
| 76 | + ] | ||
| 77 | + }, | ||
| 78 | + { | ||
| 79 | + "condition": "config.use_moe == False", | ||
| 80 | + "name": "Gemma4DecoderLayer_dense", | ||
| 81 | + "semantic": "Dense decoder layer with standard MLP", | ||
| 82 | + "code_ref": "modeling.py:285-350", | ||
| 83 | + "children": [ | ||
| 84 | + {"name": "input_layernorm", "code_ref": "modeling.py:295"}, | ||
| 85 | + {"name": "self_attn", "code_ref": "modeling.py:300-330"}, | ||
| 86 | + {"name": "mlp", "code_ref": "modeling.py:335-345"} | ||
| 87 | + ] | ||
| 88 | + } | ||
| 89 | + ] | ||
| 90 | +} | ||
| 91 | +``` | ||
| 92 | + | ||
| 93 | +#### 例 2:sliding window vs full attention | ||
| 94 | + | ||
| 95 | +```json | ||
| 96 | +{ | ||
| 97 | + "name": "self_attn_variants", | ||
| 98 | + "code_ref": "modeling.py:120-180", | ||
| 99 | + "branches": [ | ||
| 100 | + {"condition": "layer_idx in config.sliding_window_layers", "name": "self_attn_sliding", ...}, | ||
| 101 | + {"condition": "otherwise", "name": "self_attn_full", ...} | ||
| 102 | + ] | ||
| 103 | +} | ||
| 104 | +``` | ||
| 105 | + | ||
| 106 | +--- | ||
| 107 | + | ||
| 108 | +## 拆解流程 | ||
| 109 | + | ||
| 110 | +1. **读取所有模型源码**(不只是入口文件,包含被引用的子 module 文件) | ||
| 111 | +2. 识别 `ForCausalLM` 顶层类与其 `forward` 主调用链 | ||
| 112 | +3. 按 `references/structure_analysis_guide.md` §A.1(节点来源)/§A.2(命名)/§A.3(边界)规则提取结构树 | ||
| 113 | +4. 遇到条件分支: | ||
| 114 | + - **若静态可判定**(如 `config.num_experts > 0` 且 config 已知)→ 选定分支,正常拆树 | ||
| 115 | + - **若静态不可判定** → 用 `branches` 字段保留全部可能性 | ||
| 116 | +5. 不绑定 `op_indices`(留空数组) | ||
| 117 | +6. 不填 `kernels` 与 `shape_semantic`(无 raw_ops 可核对) | ||
| 118 | +7. 输出 `outputs/model_structure.json` | ||
| 119 | +8. 运行结构良构性检查(`shape_semantic` 与 `op_coverage` 检查跳过): | ||
| 120 | + | ||
| 121 | +```bash | ||
| 122 | +python scripts/check_structure.py -c outputs/model_structure.json --mode B --json > outputs/issues.json | ||
| 123 | +``` | ||
| 124 | + | ||
| 125 | +--- | ||
| 126 | + | ||
| 127 | +## 与 Mode A baseline 的等价性 | ||
| 128 | + | ||
| 129 | +若同模型已有 Mode A 的 `analysis_config.json`,可校验 Mode B 输出是 Mode A 树的"骨架超集": | ||
| 130 | + | ||
| 131 | +```bash | ||
| 132 | +python scripts/regression_check.py --mode B \ | ||
| 133 | + --baseline run_1/<model>/outputs/analysis_config.json \ | ||
| 134 | + --new outputs/model_structure.json | ||
| 135 | +``` | ||
| 136 | + | ||
| 137 | +期望: | ||
| 138 | +- 所有 Mode A 节点路径都在 Mode B 中存在 | ||
| 139 | +- Mode B 可多出 `branches` 节点(合理) | ||
| 140 | +- 不应少于 Mode A | ||
| 141 | + | ||
| 142 | +--- | ||
| 143 | + | ||
| 144 | +## 不会输出的内容 | ||
| 145 | + | ||
| 146 | +Mode B 不进入 Step 3 Review、Step 4 报告、Step 5 指标——这些都依赖 op_indices 与 raw_ops。 | ||
| @@ -0,0 +1,102 @@ | |||
| 1 | +# Mode C:仅性能数据,委托给 cann-npu-perfanalysis | ||
| 2 | + | ||
| 3 | +当工作目录中**只有性能数据而无模型源码**时,本 skill 进入 Mode C:委托给 sibling skill `cann-npu-perfanalysis` 做 8 维性能诊断(不做模型结构拆解)。 | ||
| 4 | + | ||
| 5 | +> sibling skill 仓库:https://gitcode.com/jinyingqi/npu-perf-analysis | ||
| 6 | + | ||
| 7 | +--- | ||
| 8 | + | ||
| 9 | +## 使用条件 | ||
| 10 | + | ||
| 11 | +- 存在 `kernel_details.csv` 或 `ASCEND_PROFILER_OUTPUT/` 目录 | ||
| 12 | +- 不存在模型源码(`*modeling*.py` 或类似入口) | ||
| 13 | + | ||
| 14 | +> 模型源码若也存在 → Mode A;仅有源码 → Mode B(见 `mode_b_branches.md`)。 | ||
| 15 | + | ||
| 16 | +--- | ||
| 17 | + | ||
| 18 | +## 委托方式 | ||
| 19 | + | ||
| 20 | +Mode C 不复用本 skill 的拆解流程。**spawn 一个 general-purpose subagent**,让其加载 `cann-npu-perfanalysis` 的 SKILL.md + references 后执行。 | ||
| 21 | + | ||
| 22 | +### 步骤 | ||
| 23 | + | ||
| 24 | +#### 1. 准备 sibling skill 本地副本 | ||
| 25 | + | ||
| 26 | +首次运行时 clone(约 < 5 MB): | ||
| 27 | + | ||
| 28 | +```bash | ||
| 29 | +mkdir -p .skills_cache | ||
| 30 | +git clone --depth 1 https://gitcode.com/jinyingqi/npu-perf-analysis .skills_cache/npu-perf-analysis | ||
| 31 | +``` | ||
| 32 | + | ||
| 33 | +后续运行如目录已存在则跳过。可选 `git -C .skills_cache/npu-perf-analysis pull` 拉取更新。 | ||
| 34 | + | ||
| 35 | +#### 2. 拉起 subagent | ||
| 36 | + | ||
| 37 | +使用 Agent 工具,subagent_type=`general-purpose`,prompt 模板: | ||
| 38 | + | ||
| 39 | +``` | ||
| 40 | +你将作为 cann-npu-perfanalysis 技能的执行 agent,对以下 NPU profiling 数据做 8 维性能诊断。 | ||
| 41 | + | ||
| 42 | +技能定义: | ||
| 43 | +- SKILL.md: .skills_cache/npu-perf-analysis/SKILL.md | ||
| 44 | +- 参考资料目录: .skills_cache/npu-perf-analysis/references/ | ||
| 45 | + - data-schema.md # CSV/JSON column dictionary | ||
| 46 | + - metrics-formulas.md # Phase 1-2 公式 | ||
| 47 | + - thresholds.md # P0-P3 阈值 | ||
| 48 | + - hardware-specs.md # 各芯片峰值 TFLOPs | ||
| 49 | + | ||
| 50 | +输入数据: | ||
| 51 | +- profile 目录: <ASCEND_PROFILER_OUTPUT_DIR> | ||
| 52 | + (含 kernel_details.csv、step_trace_time.csv、op_statistic.csv、communication.json、communication_matrix.json) | ||
| 53 | + | ||
| 54 | +执行要求: | ||
| 55 | +1. 严格按 SKILL.md 的 Phase 0–4 流程执行(解析 → 计算指标 → 诊断瓶颈 → 输出) | ||
| 56 | +2. 自动检测 V1/V2 schema(看是否有 cube_utilization(%) 列) | ||
| 57 | +3. 缺失文件时跳过对应维度(dim 3/5/6/7),rows < 500 时跳过 dim 7 | ||
| 58 | +4. 严格遵循 SKILL.md 的 NEVER 列表 | ||
| 59 | + | ||
| 60 | +输出文件(写入调用方 outputs/ 目录): | ||
| 61 | +- analysis_data.json # 8 维结构化诊断数据,schema 见 SKILL.md 第 255-424 行 | ||
| 62 | +- report.md # 人读报告,含 P0-P3 优先级瓶颈 | ||
| 63 | +- report.html # 用 .skills_cache/npu-perf-analysis/references/generate_html.py 渲染 | ||
| 64 | + | ||
| 65 | +返回:三文件路径 + 一句话顶层结论。 | ||
| 66 | +``` | ||
| 67 | + | ||
| 68 | +#### 3. 处理 subagent 返回 | ||
| 69 | + | ||
| 70 | +- 若成功:把 `outputs/analysis_data.json` / `report.md` / `report.html` 列入 Mode C 产物 | ||
| 71 | +- 若失败(如 profile 目录缺关键文件):把 subagent 的错误信息直接呈现给用户,不再 fallback | ||
| 72 | + | ||
| 73 | +--- | ||
| 74 | + | ||
| 75 | +## 与本 skill 输出的关系 | ||
| 76 | + | ||
| 77 | +`cann-npu-perfanalysis` **不做模型结构拆解**——它输出的是性能诊断(迭代效率、算子热点、硬件利用率/MFU、通信效率、设备空泡、等待锚点、层级结构粗判、多卡均衡)。 | ||
| 78 | + | ||
| 79 | +| 维度 | 本 skill (Mode A) | cann-npu-perfanalysis (Mode C) | | ||
| 80 | +|---|---|---| | ||
| 81 | +| 模型结构树 | ✅ 完整拆解到子模块 | ❌ 只做 MoE-yes/no、layer 数粗判 | | ||
| 82 | +| op→源码归属 | ✅ 精准对齐 | ❌ | | ||
| 83 | +| 性能瓶颈诊断 | ❌(仅给四维指标) | ✅ P0–P3 | | ||
| 84 | +| 硬件利用率 / MFU | ❌ | ✅ | | ||
| 85 | +| 通信效率分析 | ❌ | ✅ | | ||
| 86 | +| 多卡均衡分析 | ❌ | ✅ | | ||
| 87 | + | ||
| 88 | +互补关系。**Mode C 与 Mode A 的输出 schema 不同,不做强 diff 校验**。 | ||
| 89 | + | ||
| 90 | +--- | ||
| 91 | + | ||
| 92 | +## 缓存策略 | ||
| 93 | + | ||
| 94 | +- `.skills_cache/npu-perf-analysis/` 提交到 `.gitignore` | ||
| 95 | +- 首次 clone 失败(网络问题)时给出明确错误:`Mode C 需要先 clone npu-perf-analysis:git clone https://gitcode.com/jinyingqi/npu-perf-analysis .skills_cache/npu-perf-analysis` | ||
| 96 | +- 不在 SKILL 启动时自动 clone,只在判定为 Mode C 时按需触发 | ||
| 97 | + | ||
| 98 | +--- | ||
| 99 | + | ||
| 100 | +## 不会输出的内容 | ||
| 101 | + | ||
| 102 | +Mode C 不产出 `analysis_config.json`、`raw_ops*.json`、`{prefix}_report.md`、`metrics_report.md`——这些是本 skill Mode A 的产物。Mode C 的全部产物来自 sibling skill。 | ||
| @@ -0,0 +1,411 @@ | |||
| 1 | +# 结构分析指南 | ||
| 2 | + | ||
| 3 | +本文档定义如何从模型源码与 `raw_ops.json` 推导出 `analysis_config.json`。目标是**优先**产出正确、清晰、稳定的模型架构表示,为按层、按模块做耗时分析提供可靠输入。**拆分以源码语义为准,kernel 序列只用于定位和验证。** | ||
| 4 | + | ||
| 5 | +文档分为四部分: | ||
| 6 | + | ||
| 7 | +- **A. 模型 → 结构树**:从源码读出节点、命名、边界、歧义消解 | ||
| 8 | +- **B. kernel → 节点**:把 raw_ops 的算子精确归属到节点,含 shape_semantic 必填范围(**单源**) | ||
| 9 | +- **C. 输出格式**:`analysis_config.json` 的 schema 与字段规范 | ||
| 10 | +- **D. 工作流速览**:Step 2 拆解 + Step 3 Review 的执行步骤 | ||
| 11 | + | ||
| 12 | +--- | ||
| 13 | + | ||
| 14 | +## A. 模型 → 结构树 | ||
| 15 | + | ||
| 16 | +### A.1 节点来源规则 | ||
| 17 | + | ||
| 18 | +树中每个节点**只允许**来自以下四类来源之一。**任何不属于以上四类的节点都不允许出现在树中。** 不能为了追求更细粒度临时造新节点。若同一模块定义代码存在多个不同调用实例,要算作多个节点,不能合并为一个。 | ||
| 19 | + | ||
| 20 | +| 类型 | 含义 | op 覆盖范围 | | ||
| 21 | +|---|---|---| | ||
| 22 | +| **1. 源码中的 torch module** | 在执行路径中被显式调用的 `nn.Module`。最主要的节点来源 | 该模块 forward 产生的全部 op(含其内部细节 Cast/Reshape/Quant 等) | | ||
| 23 | +| **2. 源码中的稳定函数封装语义** | 没有独立模块属性名但被稳定封装为单独函数,且函数职责清楚、边界稳定。**只能作为已有模块或 stage 的内部补充拆分**,不能取代更高层的源码模块边界 | 该函数封装范围的全部 op(含其内部细节) | | ||
| 24 | +| **3. 独立通信阶段** | `all_gather` / `reduce_scatter` / `all_to_all` / `all_reduce` 等通信操作;**仅当**它们在源码中是独立阶段、前后边界清楚、且不自然属于某个已有模块内部时才保留。判断标准见 [B.2](#b2-通信操作的处理) | 单次 hcom 调用(`HcomAllGather` / `HcomReduceScatter` / `HcomAllToAll` / `hcom_allReduce`)及其紧邻的必要桥接实现细节 | | ||
| 25 | +| **4. 明确的典型模型结构语义** | 对应一个明确的"主 kernel",其余全是围绕它的实现细节 op(Cast/Reshape/DynamicQuant/DequantX)。允许使用补充语义名(如 `activation`、`dispatch`、`combine`),也允许使用源码里的函数名或 kernel 语义名 | 主 kernel + 围绕的实现细节 op | | ||
| 26 | + | ||
| 27 | +类型 2 使用条件: | ||
| 28 | +- 该函数在主执行路径上被明确调用 | ||
| 29 | +- 它描述的是稳定语义阶段,而不是一次性的实现细节整理 | ||
| 30 | +- 节点命名直接使用源码中的函数名 | ||
| 31 | + | ||
| 32 | +### A.2 命名规则 | ||
| 33 | + | ||
| 34 | +1. **优先使用源码中的属性名**:`self.xxx = SomeModule(...)` 中的 `xxx` 就是首选节点名。 | ||
| 35 | +2. **其次使用源码中的函数名**:当需要在模块内部继续划分,而该阶段没有独立模块属性名时,可以直接使用源码中的函数名。 | ||
| 36 | +3. **补充名称最后使用**:只有源码中既没有合适模块名,也没有合适函数名时,才使用允许的补充语义名。 | ||
| 37 | +4. **节点名必须覆盖其实际语义范围**:如果当前 trace 只能稳定定位整体边界,则应使用更宽的名字,而不是用过窄的名字覆盖更大的语义范围。 | ||
| 38 | +5. **`layer_types` 只描述 decoder layer 类型**:名称使用源码中 decoder layer 的类名,若模型有多种 decoder layer 或存在 mtp,`layer_types` 会有多个。其余不属于 decoder layer 的步骤,应放入 `stages`。不属于模型架构主干的运行时逻辑应放入 `runtime_auxiliary`。 | ||
| 39 | +6. **`layer_types` 命名必须使用源码类名原文**:若源码里同一模型中存在多种有结构差异的 decoder layer 时,在类名后附加区分后缀。**不允许**使用简写或省略类名。若需要附加区分后缀,后缀统一使用**全小写短语**。 | ||
| 40 | + | ||
| 41 | +### A.3 边界划分规则 | ||
| 42 | + | ||
| 43 | +#### A.3.1 layer 边界优先按源码中的 decoder layer 结构确定 | ||
| 44 | + | ||
| 45 | +先根据源码分析 decoder layer 的计算顺序,应先在 op 序列中识别这一重复模式,再区分不同的 decoder layer(若存在不同 decoder layer)。需要注意,即使完全相同的 decoder layer,算子序列可能因为并行执行,执行顺序不会完全相同。 | ||
| 46 | + | ||
| 47 | +#### A.3.2 节点内部允许按函数 / nn.module 或语义继续拆分 | ||
| 48 | + | ||
| 49 | +如果模块内部源码存在稳定的函数封装边界,且拆分符合模型结构惯例(可以拆出经典模型结构)可以继续拆分。但不要把边界不稳定的连续零散小算子拆成节点。节点的边界要对应语义清晰的代码和算子序列以及稳定的边界。 | ||
| 50 | + | ||
| 51 | +**禁止**把一个节点内部的调用阶段提升为平级兄弟节点,要么判断是否能作为子节点,要么并入。 | ||
| 52 | + | ||
| 53 | +#### A.3.3 代表性 step 中重复出现的结构阶段不展开成多份 | ||
| 54 | + | ||
| 55 | +如果同一源码阶段、模块或函数封装在代表性 step 中重复出现,且这些实例语义一致、内部结构同构,应像 decoder layer 一样按"类型 + 索引"表达,**只保留一份结构定义**,并用索引字段标明重复实例;不要按迭代号或出现顺序展开成多份近乎相同的副本。**这条规则同时适用于 `stages`、`layer_types` 和 `runtime_auxiliary` 三处**。 | ||
| 56 | + | ||
| 57 | +**部分同构场景的处理模板**:当一组重复阶段中前 N-1 次结构完全同构、只有最后一次尾部略短(典型:spec decoding 的 N 次迭代,前 N-1 次含"采样 + 为下一轮设置",最后一次只有"采样"没有下一轮设置),**必须**按以下方式折叠: | ||
| 58 | + | ||
| 59 | +- 结构同构的部分折叠为**一条**,用 `instance_indices=[0, 1, ..., N-2]` 标明覆盖的实例(`op_indices` 给出任一同构实例的代表性拆分)。 | ||
| 60 | +- 非同构的尾部另起一条,单独命名(例如 `xxx_final_iter_tail`),不使用 `instance_indices`。 | ||
| 61 | + | ||
| 62 | +**op 数渐增(accumulation 类)场景的折叠**:若每次迭代 op 数不同是因为**累积操作**(如历史 spec_tokens 的 `ConcatD` 随迭代变长),这**不构成"非同构"**;应按"kernel 类型序列 + 可变长 ConcatD"视为结构同构的实例,**仍必须用 `instance_indices=[0, 1, ..., N-1]` 折叠**为单条。`op_indices` 给出任一实例的代表性拆分,不同实例间 op 数量的些微差异(±1-2 ops 的 ConcatD 变长)不作为拆分理由。 | ||
| 63 | + | ||
| 64 | +#### A.3.4 不要求为了完整覆盖而把 runtime 逻辑强塞进模型模块 | ||
| 65 | + | ||
| 66 | +步骤二的首要目标是产出清晰的模型架构。像 token 选择、spec token 验证、输入更新这类不属于模型主干的逻辑,不应并入 `lm_head`、`embed_tokens` 等模块,应放入 `runtime_auxiliary`。 | ||
| 67 | + | ||
| 68 | +#### A.3.5 确保完全理解计算过程和对应的算子 | ||
| 69 | + | ||
| 70 | +遇到并行复杂交错的模块,算子划分要确保完全理解计算过程,可依据 `stream_id`、`input_shapes`、`output_shapes` 辅助划分(详见 [B.4](#b4-辅助证据)),算子划分到模块需要有依据,能和源码、计算过程对应。 | ||
| 71 | + | ||
| 72 | +#### A.3.6 多次迭代执行的模型包装层的建模方式(如 MTP / spec decoding) | ||
| 73 | + | ||
| 74 | +若源码中存在"对模型主干做多次迭代"的包装层(典型例子:MTP / spec decoding 头,外层 loop 调用一个内部包含 decoder layer 的 module N 次),按以下方式建模(**不允许**二义解释): | ||
| 75 | + | ||
| 76 | +- 该包装层内部如果包含**真实的 decoder layer 类**,则该内部 decoder layer **必须**进入 `layer_types`,其 `layer_indices` 覆盖该 decoder layer 在代表性 step 中出现的所有 layer_idx(多次迭代会产生多个 layer_idx 实例)。**不允许**把该内部 decoder layer 内联进 `stages` 里。 | ||
| 77 | +- 该包装层的 **scaffold**(例如 enorm / hnorm / eh_proj / shared_head_norm / embed_tokens / lm_head 这类围绕 decoder layer 的辅助模块)放入 `stages`,每次迭代结构同构则用 `stage_indices` 折叠为一份定义。 | ||
| 78 | +- 迭代之间、迭代之后的运行时辅助逻辑(graph setup、token sampling、verify、param update 等)放入 `runtime_auxiliary`,结构同构部分用 `instance_indices` 折叠。 | ||
| 79 | + | ||
| 80 | +### A.4 歧义消解顺序 | ||
| 81 | + | ||
| 82 | +先根据计算过程、源码及 [B.4 辅助证据](#b4-辅助证据)确定算子对应的具体计算和节点归属。遇到拿不准的 op 归属时,按以下**固定优先级**处理: | ||
| 83 | + | ||
| 84 | +1. 先判断它是否属于模型架构主干,还是属于运行时辅助逻辑。 | ||
| 85 | +2. 如果属于模型架构主干,优先并入最近的源码模块。 | ||
| 86 | +3. 如果源码中没有合适模块,但存在稳定函数封装,使用函数名。 | ||
| 87 | +4. 如果是独立通信阶段,保留为通信节点。 | ||
| 88 | +5. 如果以上都不适用,再使用允许的补充语义名。 | ||
| 89 | +6. 如果仍然不确定,继续向上合并到父节点。 | ||
| 90 | +7. 如果明确不属于模型架构主干,则放入 `runtime_auxiliary`。 | ||
| 91 | + | ||
| 92 | +不能为了覆盖某些 op 而临时创建新节点名,也不能根据单次运行的局部 kernel 排布改动树结构。 | ||
| 93 | + | ||
| 94 | +--- | ||
| 95 | + | ||
| 96 | +## B. kernel → 节点 | ||
| 97 | + | ||
| 98 | +### B.1 实现细节 op 的处理 | ||
| 99 | + | ||
| 100 | +以下 kernel 类型通常属于实现细节,**默认不作为独立树节点**: | ||
| 101 | + | ||
| 102 | +`Cast`、`Concat`、`Transpose`、`Reshape`、`DynamicQuant`、`Dequant*`、`ScatterNdUpdate`、`Split`、`RotaryMul`、`AivKernel`(非明确对应某个源码模块或函数边界时) | ||
| 103 | + | ||
| 104 | +处理方式:**并入最近的源码模块节点或函数语义节点。** | ||
| 105 | + | ||
| 106 | +### B.2 通信操作的处理 | ||
| 107 | + | ||
| 108 | +通信 op(`HcomAllGather`、`HcomReduceScatter`、`HcomAllToAll`、`hcom_allReduce` 以及伴随通信的 `AivKernel`)按以下规则处理: | ||
| 109 | + | ||
| 110 | +1. **如果通信是某个模块或函数内部实现的一部分**,并入该模块或函数,不单独拆出。 | ||
| 111 | +2. **如果通信是模块间独立的数据搬运阶段**,可以保留为树节点。例如 embedding 后的 `all_gather/reduce_scatter`、`forward_lm_head` 前的 `all_gather`。 | ||
| 112 | +3. **判断标准以源码为准**:看通信调用发生在谁的 `forward` 或辅助函数内部。如果发生在某个模块或函数内部,就并入该节点。 | ||
| 113 | +4. **伴随通信的 `AivKernel` 不单独命名**:应并入最近的通信节点或所属模块节点。 | ||
| 114 | + | ||
| 115 | +### B.3 op 映射规则 | ||
| 116 | + | ||
| 117 | +要把 op 精准对应到模型拆解的节点,**不允许有模糊或猜测**。 | ||
| 118 | + | ||
| 119 | +1. 确保对单个节点的计算过程和起始边界完全理解。 | ||
| 120 | +2. 把具体算子和源码的每一个计算对齐,需要结合 [B.4 辅助证据](#b4-辅助证据)和源码判断。 | ||
| 121 | +3. 完成对齐后,检验节点的计算流程和对应算子的对应是否有多余或缺漏,需要结合 [B.4 辅助证据](#b4-辅助证据)和源码判断。如果存在问题,需要重新分析并做对应,直到没有 gap。 | ||
| 122 | + | ||
| 123 | +### B.4 辅助证据 | ||
| 124 | + | ||
| 125 | +`raw_ops.json` 中的 `stream_id`、`input_shapes`、`output_shapes` 是把算子对应到具体源码和计算过程的重要辅助证据。 | ||
| 126 | + | ||
| 127 | +**`stream_id` 的用途**: | ||
| 128 | +- 用于识别多 stream 并行执行。 | ||
| 129 | +- 用于判断某些节点是否可以使用非连续 `op_indices` 收拢到同一语义阶段。 | ||
| 130 | + | ||
| 131 | +**shape 的用途**: | ||
| 132 | +- 用于辅助区分同名 op 在不同阶段的语义位置。 | ||
| 133 | +- 用于判断某个 `MatMul`、`GroupedMatmul`、`QuantBatchMatmul` 等同名算子更可能属哪个节点。 | ||
| 134 | +- 用于核对前后节点是否真的存在维度变化边界。 | ||
| 135 | + | ||
| 136 | +### B.5 shape_semantic 必填范围(单源) | ||
| 137 | + | ||
| 138 | +`kernels` 数组的条目中,对以下算子**必须**提供 `shape_semantic`,用模型维度符号标注 shape 含义: | ||
| 139 | + | ||
| 140 | +| 类别 | 算子 | | ||
| 141 | +|---|---| | ||
| 142 | +| **MatMul 类** | `MatMul` / `MatMulV2` / `QuantBatchMatmulV3` / `GroupedMatmul` / `GemmEx` / `BatchMatMul`(及变体) | | ||
| 143 | +| **Attention 类** | `FlashAttentionScore` / `FusedInferAttentionScore` / `KvQuantSparseFlashAttention` | | ||
| 144 | +| **通信类** | `HcomAllGather` / `HcomReduceScatter` / `HcomAllToAll` / `hcom_allReduce` | | ||
| 145 | +| **Norm 类** | `RmsNorm` / `LayerNormV3` / `InplaceAddRmsNorm` / `AddRmsNorm*` / `AddRmsNormDynamicQuant` | | ||
| 146 | +| **Fused 计算类** | `MlaPrologV3` / `DequantSwigluQuant` / `LightningIndexerQuant` / `MoeGatingTopKHash` | | ||
| 147 | +| **旋转位置编码** | `RotaryMul` | | ||
| 148 | +| **KV cache 更新** | `ScatterNdUpdate`、KV cache 拼接用的 `ConcatV2` / `ConcatD` | | ||
| 149 | +| **残差加法** | `Add`(明确为残差连接时) | | ||
| 150 | +| **Embedding/Gather** | `GatherV2` / `GatherV3` | | ||
| 151 | +| **MoE 调度** | `MoeDistributeDispatchV2` / `MoeDistributeCombineV2` | | ||
| 152 | + | ||
| 153 | +**免填**:`Cast`、`Reshape`、`Transpose`(纯格式转换)、`DynamicQuant`/`Dequant*`(量化辅助)。 | ||
| 154 | + | ||
| 155 | +**统一维度符号**:`B`(Batch)、`T`(Time/SeqLen,统一用 T 不用 S)、`H`(NumHeads)、`D`(HeadDim)、`hidden`(hidden_size)、`ffn`(intermediate_size)、`E`(num_experts)、`topK`(experts per token)、`q_rank`(q_lora_rank)、`kv_rank`(kv_lora_rank)。 | ||
| 156 | + | ||
| 157 | +**示例**: | ||
| 158 | +- `[B*T, hidden] @ [hidden, q_rank]` | ||
| 159 | +- `[B, H, T, D] × [B, H, D, T] → [B, H, T, T]` | ||
| 160 | +- `[B*T, hidden] → [B*T, hidden]`(Norm) | ||
| 161 | +- `[B*T, H, kv_rank] → AllGather → [B*T, H, kv_rank*tp]`(通信) | ||
| 162 | + | ||
| 163 | +#### B.5.1 shape_semantic 正确性规则(填写前必须逐一核对) | ||
| 164 | + | ||
| 165 | +1. **先看实际 shape**:填写前先查看该 kernel 在 `raw_ops.json` 中的 `input_shapes` 和 `output_shapes`,以实际 shape 为准,**绝不凭印象或架构直觉猜测**。 | ||
| 166 | +2. **`→` 左侧描述输入,右侧描述输出**:`→` 左边只写主要输入张量的语义维度(通常是 input[0]),右边写所有关键输出张量;多输出用逗号分隔。 | ||
| 167 | +3. **命名维度必须与实际数值一致**:若写 `H_q=128`,则实际输出 tensor 中必须存在维度 128;若写 `kv_rank=512`,则实际 shape 中必须存在 512。**不得**写 config 字段名之外的新符号而不标注数值(如直接写 `H_idx` 而不写 `H_idx=64`)。 | ||
| 168 | +4. **fused kernel 须追踪每个关键输出**:对 `MlaPrologV3`、`LightningIndexerQuant`、`AddRmsNormDynamicQuant` 等多输出 fused kernel,需对照 raw_ops 的每条 output shape,识别其含义后在 shape_semantic 右侧列出所有关键输出。**不得**将 Q 输出标成 K,**不得**混用 `H_idx`/`H_q`。 | ||
| 169 | +5. **吸收(absorbed)注意**:MLA 推理时 `W_kv_b` 已被吸收进 Q,`MlaPrologV3` 输出的是 `Q_nope_abs[B*T, H_q, kv_rank]`,维度是 `kv_rank`(512)而非 `D_nope`(128)。 | ||
| 170 | +6. **验证脚本**:写完 analysis_config.json 并运行 enrich 后,执行 `python scripts/validate_shapes.py -c outputs/analysis_config.json` 确认无 ERROR,再进行下一步。 | ||
| 171 | + | ||
| 172 | +--- | ||
| 173 | + | ||
| 174 | +## C. 输出格式 | ||
| 175 | + | ||
| 176 | +### C.1 顶层 schema 与示例 | ||
| 177 | + | ||
| 178 | +```json | ||
| 179 | +{ | ||
| 180 | + "model_name": "model-name", | ||
| 181 | + "representative_step": 1, | ||
| 182 | + "notes": "简要说明网络结构组成、层数、并行方式和特殊执行路径", | ||
| 183 | + "stages": { | ||
| 184 | + "preprocessing": { | ||
| 185 | + "name": "preprocessing", | ||
| 186 | + "children": [ | ||
| 187 | + {"name": "stage_module_a", "op_indices": [0, 1, 2]} | ||
| 188 | + ] | ||
| 189 | + }, | ||
| 190 | + "repeated_stage_type_x": { | ||
| 191 | + "name": "repeated_stage_type_x", | ||
| 192 | + "stage_indices": [0, 1], | ||
| 193 | + "children": [ | ||
| 194 | + {"name": "stage_module_c", "op_indices": [50, 51]} | ||
| 195 | + ] | ||
| 196 | + }, | ||
| 197 | + "postprocessing": { | ||
| 198 | + "name": "postprocessing", | ||
| 199 | + "children": [ | ||
| 200 | + {"name": "stage_module_b", "op_indices": [100]}, | ||
| 201 | + {"name": "stage_function_c", "op_indices": [101, 102, 103]} | ||
| 202 | + ] | ||
| 203 | + } | ||
| 204 | + }, | ||
| 205 | + "layer_types": { | ||
| 206 | + "layer_type_a": {"layer_indices": [0]}, | ||
| 207 | + "layer_type_b": {"layer_indices": [1, 2]}, | ||
| 208 | + "layer_type_c": {"layer_indices": [3, 4, 5]} | ||
| 209 | + }, | ||
| 210 | + "layer_structure": { | ||
| 211 | + "layer_type_a": { | ||
| 212 | + "name": "layer_type_a", | ||
| 213 | + "semantic": "Decoder layer with self-attention and MLP", | ||
| 214 | + "code_ref": "modeling.py:200-350", | ||
| 215 | + "children": [ | ||
| 216 | + {"name": "submodule_1", "semantic": "LayerNorm before attention", "code_ref": "modeling.py:220", "op_indices": [10], "kernels": [{"index": 10, "semantic": "RMSNorm kernel", "code_ref": "modeling.py:222"}]}, | ||
| 217 | + { | ||
| 218 | + "name": "submodule_2", | ||
| 219 | + "children": [ | ||
| 220 | + { | ||
| 221 | + "name": "function_1", | ||
| 222 | + "children": [ | ||
| 223 | + {"name": "substage_1", "op_indices": [11]}, | ||
| 224 | + {"name": "substage_2", "op_indices": [12, 13, 14]} | ||
| 225 | + ] | ||
| 226 | + }, | ||
| 227 | + {"name": "substage_3", "op_indices": [15]}, | ||
| 228 | + { | ||
| 229 | + "name": "function_2", | ||
| 230 | + "children": [ | ||
| 231 | + {"name": "submodule_3", "op_indices": [16, 17]} | ||
| 232 | + ] | ||
| 233 | + } | ||
| 234 | + ] | ||
| 235 | + }, | ||
| 236 | + {"name": "submodule_4", "op_indices": [18]}, | ||
| 237 | + { | ||
| 238 | + "name": "submodule_5", | ||
| 239 | + "children": [ | ||
| 240 | + {"name": "substage_4", "op_indices": [19]}, | ||
| 241 | + {"name": "substage_5", "op_indices": [20]}, | ||
| 242 | + {"name": "substage_6", "op_indices": [21]} | ||
| 243 | + ] | ||
| 244 | + } | ||
| 245 | + ] | ||
| 246 | + } | ||
| 247 | + }, | ||
| 248 | + "runtime_auxiliary": [ | ||
| 249 | + { | ||
| 250 | + "name": "runtime_helper_a", | ||
| 251 | + "op_indices": [200, 201, 202] | ||
| 252 | + }, | ||
| 253 | + { | ||
| 254 | + "name": "runtime_helper_b", | ||
| 255 | + "instance_indices": [0, 1, 2], | ||
| 256 | + "op_indices": [210, 211] | ||
| 257 | + } | ||
| 258 | + ] | ||
| 259 | +} | ||
| 260 | +``` | ||
| 261 | + | ||
| 262 | +### C.2 顶层字段说明 | ||
| 263 | + | ||
| 264 | +| 字段 | 说明 | | ||
| 265 | +|---|---| | ||
| 266 | +| `model_name` | 模型名称 | | ||
| 267 | +| `representative_step` | 选定的代表性 step ID | | ||
| 268 | +| `notes` | 网络结构组成、层数、并行方式、特殊执行路径 | | ||
| 269 | +| `stages` | decoder layer 之外、但属于模型架构主干的阶段。重复出现的同类阶段只保留一份定义,用 `stage_indices` 标明实例 | | ||
| 270 | +| `layer_types` | 只列 decoder layer 类型及其层号范围 | | ||
| 271 | +| `layer_structure` | 每种 decoder layer 类型对应一棵代表性结构树。`op_indices` 表示该类型在代表性 step 中的参考拆分,不要求逐一展开所有重复实例 | | ||
| 272 | +| `runtime_auxiliary` | 不属于模型架构主干、但在代表性 step 中真实存在的运行时逻辑(token 选择、验证、输入更新、调度辅助逻辑)。重复出现的同类逻辑只保留一份定义,用 `instance_indices` 标明实例 | | ||
| 273 | + | ||
| 274 | +### C.3 节点字段说明 | ||
| 275 | + | ||
| 276 | +| 字段 | 必选 | 说明 | | ||
| 277 | +|---|---|---| | ||
| 278 | +| `name` | 必选 | 节点名(按 [A.2 命名规则](#a2-命名规则)) | | ||
| 279 | +| `semantic` | **必选**(`Cast`/`Reshape` 除外) | 节点或 kernel 的语义说明,如 "Attention QKV projection"。所有有意义的算子(包括 `Add` 残差、`Mul` 门控、`Concat` KV 缓存拼接、RMSNorm 各步骤等)均需填写,清晰说明其在模型中的计算作用 | | ||
| 280 | +| `code_ref` | **必选**(除非无法判断) | 源码位置引用,格式如 `filename.py:line` 或 `filename.py:start-end` | | ||
| 281 | +| `op_indices` | 叶节点必有 | 该节点覆盖的 op 索引列表,**允许非连续** | | ||
| 282 | +| `children` | 中间节点必有 | 子节点数组;中间节点必要时也可同时带 `op_indices` 表示不属于任何子节点的额外 op | | ||
| 283 | +| `kernels` | 可选 | 节点下的 kernel 语义信息数组,每个元素含 `index`、`semantic`、`shape_semantic`、`code_ref` | | ||
| 284 | +| `stage_indices` / `instance_indices` | 重复阶段必有 | 该结构阶段或辅助逻辑在代表性 step 中重复出现的实例索引。存在时 `op_indices` 只需给出该类型一份代表性拆分 | | ||
| 285 | + | ||
| 286 | +### C.4 树结构规则 | ||
| 287 | + | ||
| 288 | +- 每个节点包含 `name`。 | ||
| 289 | +- 叶子节点包含 `op_indices`,允许非连续。 | ||
| 290 | +- 中间节点包含 `children`,必要时也可以同时带 `op_indices` 表示不属于任何子节点的额外 op。 | ||
| 291 | +- 节点和 kernel **必选**包含 `semantic`(语义说明)和 `code_ref`(源码位置)。 | ||
| 292 | +- 同一代表性 step 中重复出现的同构阶段,只保留一份类型定义,并使用 `stage_indices` 或 `instance_indices` 标明重复实例,**不**按迭代展开成多份相同子树。 | ||
| 293 | +- **不**要求为了"全覆盖"把 runtime 辅助逻辑强塞进模块树;这类内容应进入 `runtime_auxiliary`。 | ||
| 294 | +- **不能**静默遗漏已经识别出的重要阶段。主干结构和 runtime 辅助逻辑都应在配置中有明确归属。 | ||
| 295 | +- `kernels` 数组对 [B.5](#b5-shape_semantic-必填范围单源) 列出的算子**必须**填写 `shape_semantic`。 | ||
| 296 | + | ||
| 297 | +--- | ||
| 298 | + | ||
| 299 | +## D. 工作流速览 | ||
| 300 | + | ||
| 301 | +按以下固定顺序执行,每一步的输入依赖前一步的输出。 | ||
| 302 | + | ||
| 303 | +### D.1 Step 2:拆解模型结构 | ||
| 304 | + | ||
| 305 | +#### D.1.1 选择代表性 step 和 decoder layer 实例 | ||
| 306 | + | ||
| 307 | +从 `steps_summary.md`、`raw_ops.compact.json`(或 `raw_ops.json`)中选一个稳定的 decode step(跳过 warmup)。选择标准: | ||
| 308 | + | ||
| 309 | +- 如果只有一个 step,不用选择直接使用。 | ||
| 310 | +- 先按 kernel_count + kernel 类型分布分组,选择主流稳定分组;若存在多种 op 数量的 step,选择包含最多 decoder layer 的那种作为代表。 | ||
| 311 | +- 对同一稳定分组,检查最早 step 的 kernel_sum 是否相对后续 step 中位数明显离群;若离群,把最早 step 视为 warmup/outlier 并跳过。 | ||
| 312 | +- 若最早 step 不离群,优先选择最早稳定 step;若跳过 warmup/outlier,则选择后续 step 中 kernel_sum 最接近中位数的一步。 | ||
| 313 | +- 在 `notes` 中明示代表 step 及选择原因;需要复现历史报告或特定 step 时,可以显式传 `-s`。 | ||
| 314 | + | ||
| 315 | +**选择代表性 decoder layer 实例(同 layer_type 内多实例时)**: | ||
| 316 | + | ||
| 317 | +同一 `layer_type` 在代表性 step 中会出现多个实例,它们内部结构一致,但首层常常存在融合差异(例如 layer 0 的 `input_layernorm` 是独立 `RmsNorm`,layer 1+ 融合上一层 residual 成 `InplaceAddRmsNorm`)。选择规则: | ||
| 318 | + | ||
| 319 | +- 默认选**第一个完整实例**(通常 layer 0)作为 representative 结构。 | ||
| 320 | +- **例外(必须遵守)**:若首层因跨层 fusion 差异导致 op 数量或关键 kernel 类型与后续同类实例不同(典型:layer 0 的 `input_layernorm` 是独立 `RmsNorm`、后续层是 `InplaceAddRmsNorm` / `AddRmsNormX` 等融合版),**必须**选第二个实例(通常 layer 1)作为 representative,以使 representative 结构能一致套用在所有其余同类实例上。 | ||
| 321 | +- 选定后在 `notes` 中明示选了哪一个实例、以及 layer 0 与后续实例的已知差异,便于步骤三的 stats 脚本识别。 | ||
| 322 | +- 若存在结构不同的多种 decoder layer,都要选出作为独立 layer_type。 | ||
| 323 | +- 若存在 mtp,mtp decoder layer 要提取出来成为一个独立 layer_type。**不得**与主模型的 decoder layer 合并。 | ||
| 324 | + | ||
| 325 | +#### D.1.2 阅读模型源码,提取模块层级和稳定函数边界 | ||
| 326 | + | ||
| 327 | +阅读 modeling 源码文件,从最外层 `ForCausalLM` 到最内层子模块,提取完整的执行语义。重点关注: | ||
| 328 | + | ||
| 329 | +- `__init__` 中注册了哪些子模块(`self.xxx = ...`)。 | ||
| 330 | +- `forward` 中实际调用了哪些模块,调用顺序是什么。 | ||
| 331 | +- 除 `nn.Module` 之外,是否存在稳定的函数封装边界。 | ||
| 332 | + | ||
| 333 | +形成模型结构树,节点来源参考 [A.1 节点来源规则](#a1-节点来源规则)。 | ||
| 334 | + | ||
| 335 | +#### D.1.3 在 op 序列中定位 decoder layer 边界 | ||
| 336 | + | ||
| 337 | +使用结构锚点在代表性 step 的 op 列表中划分 decoder layer 边界。`layer_types` 只描述 decoder layer 类型;decoder layer 之外的模型结构应归入 `stages`,不属于模型架构主干的运行时逻辑应归入 `runtime_auxiliary`。 | ||
| 338 | + | ||
| 339 | +> 提示:若有 `outputs/op_segments.json`(由 `scripts/segment_layers.py` 生成),可作为 layer 边界**候选起点**;最终边界仍以源码语义为准。 | ||
| 340 | + | ||
| 341 | +#### D.1.4 逐层映射 op 到源码模块或函数语义 | ||
| 342 | + | ||
| 343 | +对每一层,按源码中的调用顺序,将 op 序列拆分到各个节点中。节点优先按 `torch module` 划分;当某段逻辑并非独立模块,但在源码中被稳定地封装为单独函数且语义明确时,可以按函数边界或其他稳定模型结构语义划分。把算子映射到对应的节点(参考 [B.3 op 映射规则](#b3-op-映射规则))。每个叶子节点覆盖的 op 用 `op_indices` 表示,允许非连续。 | ||
| 344 | + | ||
| 345 | +#### D.1.5 输出 `analysis_config.json` | ||
| 346 | + | ||
| 347 | +将结构分析结果写入配置文件。格式见 [C. 输出格式](#c-输出格式)。 | ||
| 348 | + | ||
| 349 | +#### D.1.6 运行 enrich 命令 | ||
| 350 | + | ||
| 351 | +```bash | ||
| 352 | +python scripts/analyze_kernels.py --enrich \ | ||
| 353 | + -c outputs/analysis_config.json \ | ||
| 354 | + -r outputs/raw_ops.json | ||
| 355 | +``` | ||
| 356 | + | ||
| 357 | +enrich 命令为每个叶节点的 `op_indices` 追加 `op_data` 字段(含 `index`、`org_index`、`name`、`duration_us`、`stream_id`、`task_type`、`input_shapes`、`output_shapes`、`shape_raw`),并将节点 `kernels` 数组中已有的 `semantic`、`shape_semantic`、`code_ref` 合并进对应算子的 `op_data` 条目。原有字段不变。 | ||
| 358 | + | ||
| 359 | +### D.2 Step 3:Review 拆解结果 | ||
| 360 | + | ||
| 361 | +Step 3 用脚本暴露问题、AI 仅修正定位过的节点。**不再裸读全量源码与 raw_ops**。 | ||
| 362 | + | ||
| 363 | +#### D.2.1 运行确定性脚本检查 | ||
| 364 | + | ||
| 365 | +```bash | ||
| 366 | +python scripts/check_structure.py -c outputs/analysis_config.json --json > outputs/issues.json | ||
| 367 | +python scripts/validate_shapes.py -c outputs/analysis_config.json --fail-fast --json >> outputs/issues.json | ||
| 368 | +python scripts/check_op_coverage.py -c outputs/analysis_config.json -r outputs/raw_ops.json --json >> outputs/issues.json | ||
| 369 | +``` | ||
| 370 | + | ||
| 371 | +脚本规则与本指南直接对齐: | ||
| 372 | + | ||
| 373 | +| 脚本 | 检查 | | ||
| 374 | +|---|---| | ||
| 375 | +| `check_structure.py` | 树良构性(schema 完整、layer_types/layer_structure 匹配、必填字段、索引列表无重复、双归属检测) | | ||
| 376 | +| `validate_shapes.py` | shape_semantic 与实际 tensor shape 一致性 | | ||
| 377 | +| `check_op_coverage.py` | op 全覆盖、不重叠、shape_semantic 必填类算子全在 kernels 数组登记 | | ||
| 378 | + | ||
| 379 | +#### D.2.2 若 issues.json 为空 → 跳过 AI review | ||
| 380 | + | ||
| 381 | +直接进入 D.3。 | ||
| 382 | + | ||
| 383 | +#### D.2.3 否则拉起 review subagent | ||
| 384 | + | ||
| 385 | +subagent 与主 agent 使用相同模型与推理强度。**输入**: | ||
| 386 | + | ||
| 387 | +1. `issues.json` | ||
| 388 | +2. `analysis_config.json` 中 issue 命中的节点(按路径过滤后投喂,**不投喂全量配置**) | ||
| 389 | +3. 这些节点 `code_ref` 行号对应的源码切片(**稀疏读**:用 `Read` 工具的 `offset/limit` 参数只读相关行段) | ||
| 390 | +4. 这些节点 `op_indices` 对应的 raw_ops 切片 | ||
| 391 | + | ||
| 392 | +**任务**: | ||
| 393 | + | ||
| 394 | +``` | ||
| 395 | +依据 issues.json、源码切片、op 切片,对每一项 issue 验证算子归属与字段填写正确性。 | ||
| 396 | +若存在错误,直接修正 analysis_config.json 中对应位置;只允许在存在错误时修改。 | ||
| 397 | +要精准对齐源码,结合 ops 的 shape 和 stream_id 判断。 | ||
| 398 | +不能有任何冗余或遗漏。 | ||
| 399 | +``` | ||
| 400 | + | ||
| 401 | +**输出**:修正后的 `analysis_config.json` + review 结论。 | ||
| 402 | + | ||
| 403 | +#### D.2.4 重新 enrich + 校验 | ||
| 404 | + | ||
| 405 | +修正后重新运行 D.1.6 enrich 与 D.2.1 三脚本,循环执行直至 issues 列表为空或迭代上限(默认 3 次)。 | ||
| 406 | + | ||
| 407 | +### D.3 与下游的衔接 | ||
| 408 | + | ||
| 409 | +- 通过 D.2.4 后,`analysis_config.json` 进入 Step 4(generate_report.py)与 Step 5(compute_metrics.py),脚本调用见 SKILL.md。 | ||
| 410 | +- 若是 Mode B(仅模型源码),输出文件名为 `model_structure.json`,op_indices 留空,可加 `branches` 字段表达多分支;不进入 Step 4/5。详见 `references/mode_b_branches.md`。 | ||
| 411 | +- 若是 Mode C(仅性能数据),不走本流程,委托给 cann-npu-perfanalysis sibling skill;详见 `references/mode_c_delegate.md`。 | ||
| @@ -0,0 +1,1712 @@ | |||
| 1 | +#!/usr/bin/env python3 | ||
| 2 | +# -*- coding: utf-8 -*- | ||
| 3 | +# ---------------------------------------------------------------------------- | ||
| 4 | +# Copyright (c) 2025 Huawei Technologies Co., Ltd. | ||
| 5 | +# | ||
| 6 | +# Licensed under the Apache License, Version 2.0 (the "License"); | ||
| 7 | +# you may not use this file except in compliance with the License. | ||
| 8 | +# You may obtain a copy of the License at | ||
| 9 | +# | ||
| 10 | +# http://www.apache.org/licenses/LICENSE-2.0 | ||
| 11 | +# | ||
| 12 | +# Unless required by applicable law or agreed to in writing, software | ||
| 13 | +# distributed under the License is distributed on an "AS IS" BASIS, | ||
| 14 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| 15 | +# See the License for the specific language governing permissions and | ||
| 16 | +# limitations under the License. | ||
| 17 | +# ---------------------------------------------------------------------------- | ||
| 18 | +"""generate_report.py 的静态 HTML 资源:CSS 样式与 JS 模板常量。 | ||
| 19 | + | ||
| 20 | +从 generate_report.py 分离,避免单文件过大;内容逐字节保持不变。 | ||
| 21 | +""" | ||
| 22 | +HTML_CSS = ''' | ||
| 23 | + <style> | ||
| 24 | + :root { | ||
| 25 | + --bg-primary: #1e1e1e; | ||
| 26 | + --bg-secondary: #252526; | ||
| 27 | + --bg-tertiary: #2d2d2d; | ||
| 28 | + --bg-hover: #2d2d2d; | ||
| 29 | + --text-primary: #d4d4d4; | ||
| 30 | + --text-secondary: #808080; | ||
| 31 | + --accent-blue: #4fc1ff; | ||
| 32 | + --accent-green: #6a9955; | ||
| 33 | + --accent-orange: #ce9178; | ||
| 34 | + --border-color: #3c3c3c; | ||
| 35 | + --button-bg: #0e639c; | ||
| 36 | + --button-hover: #1177bb; | ||
| 37 | + --tooltip-bg: #1e1e2e; | ||
| 38 | + --tooltip-border: #45475a; | ||
| 39 | + } | ||
| 40 | + [data-theme="dracula"] { | ||
| 41 | + --bg-primary: #282a36; | ||
| 42 | + --bg-secondary: #21222c; | ||
| 43 | + --bg-tertiary: #343746; | ||
| 44 | + --bg-hover: #44475a; | ||
| 45 | + --text-primary: #f8f8f2; | ||
| 46 | + --text-secondary: #6272a4; | ||
| 47 | + --accent-blue: #8be9fd; | ||
| 48 | + --accent-green: #50fa7b; | ||
| 49 | + --accent-orange: #ffb86c; | ||
| 50 | + --border-color: #44475a; | ||
| 51 | + --button-bg: #bd93f9; | ||
| 52 | + --button-hover: #ff79c6; | ||
| 53 | + --tooltip-bg: #1d1e26; | ||
| 54 | + --tooltip-border: #6272a4; | ||
| 55 | + } | ||
| 56 | + [data-theme="one-dark"] { | ||
| 57 | + --bg-primary: #282c34; | ||
| 58 | + --bg-secondary: #21252b; | ||
| 59 | + --bg-tertiary: #2c313a; | ||
| 60 | + --bg-hover: #2c313a; | ||
| 61 | + --text-primary: #abb2bf; | ||
| 62 | + --text-secondary: #5c6370; | ||
| 63 | + --accent-blue: #61afef; | ||
| 64 | + --accent-green: #98c379; | ||
| 65 | + --accent-orange: #d19a66; | ||
| 66 | + --border-color: #181a1f; | ||
| 67 | + --button-bg: #4d78cc; | ||
| 68 | + --button-hover: #528bff; | ||
| 69 | + --tooltip-bg: #21252b; | ||
| 70 | + --tooltip-border: #5c6370; | ||
| 71 | + } | ||
| 72 | + [data-theme="github-light"] { | ||
| 73 | + --bg-primary: #ffffff; | ||
| 74 | + --bg-secondary: #f6f8fa; | ||
| 75 | + --bg-tertiary: #f0f2f5; | ||
| 76 | + --bg-hover: #eaeef2; | ||
| 77 | + --text-primary: #1f2328; | ||
| 78 | + --text-secondary: #656d76; | ||
| 79 | + --accent-blue: #0969da; | ||
| 80 | + --accent-green: #1a7f37; | ||
| 81 | + --accent-orange: #bc4c00; | ||
| 82 | + --border-color: #d1d9e0; | ||
| 83 | + --button-bg: #0969da; | ||
| 84 | + --button-hover: #0550ae; | ||
| 85 | + --tooltip-bg: #f6f8fa; | ||
| 86 | + --tooltip-border: #d1d9e0; | ||
| 87 | + } | ||
| 88 | + [data-theme="solarized-light"] { | ||
| 89 | + --bg-primary: #fdf6e3; | ||
| 90 | + --bg-secondary: #eee8d5; | ||
| 91 | + --bg-tertiary: #e8e1cc; | ||
| 92 | + --bg-hover: #e0dac7; | ||
| 93 | + --text-primary: #073642; | ||
| 94 | + --text-secondary: #839496; | ||
| 95 | + --accent-blue: #268bd2; | ||
| 96 | + --accent-green: #859900; | ||
| 97 | + --accent-orange: #cb4b16; | ||
| 98 | + --border-color: #d3cbb7; | ||
| 99 | + --button-bg: #268bd2; | ||
| 100 | + --button-hover: #1a6da8; | ||
| 101 | + --tooltip-bg: #eee8d5; | ||
| 102 | + --tooltip-border: #839496; | ||
| 103 | + } | ||
| 104 | + * { margin: 0; padding: 0; box-sizing: border-box; } | ||
| 105 | + body { | ||
| 106 | + font-family: 'Segoe UI', 'SF Pro Display', -apple-system, sans-serif; | ||
| 107 | + background: var(--bg-primary); | ||
| 108 | + color: var(--text-primary); | ||
| 109 | + line-height: 1.6; | ||
| 110 | + padding: 40px; | ||
| 111 | + max-width: 1600px; | ||
| 112 | + margin: 0 auto; | ||
| 113 | + } | ||
| 114 | + h1 { font-size: 28px; margin-bottom: 20px; color: var(--text-primary); | ||
| 115 | + border-bottom: 2px solid var(--border-color); padding-bottom: 15px; } | ||
| 116 | + .metadata { | ||
| 117 | + background: var(--bg-tertiary); | ||
| 118 | + padding: 15px 20px; | ||
| 119 | + border-radius: 8px; | ||
| 120 | + margin-bottom: 20px; | ||
| 121 | + display: flex; | ||
| 122 | + flex-direction: column; | ||
| 123 | + gap: 12px; | ||
| 124 | + } | ||
| 125 | + .meta-row { display: flex; flex-wrap: wrap; gap: 25px; } | ||
| 126 | + .meta-item { display: flex; gap: 8px; align-items: center; } | ||
| 127 | + .meta-label { color: var(--text-secondary); font-size: 13px; } | ||
| 128 | + .meta-value { color: var(--text-primary); font-family: 'Consolas', monospace; font-size: 13px; } | ||
| 129 | + .meta-datasources { display: flex; flex-direction: column; gap: 4px; margin-left: 20px; } | ||
| 130 | + .meta-datasource-item { | ||
| 131 | + display: flex; | ||
| 132 | + gap: 12px; | ||
| 133 | + align-items: baseline; | ||
| 134 | + color: var(--text-primary); | ||
| 135 | + font-family: 'Consolas', monospace; | ||
| 136 | + font-size: 12px; | ||
| 137 | + } | ||
| 138 | + .meta-datasource-item .step-badge { | ||
| 139 | + background: var(--button-bg); | ||
| 140 | + color: #fff; | ||
| 141 | + padding: 1px 6px; | ||
| 142 | + border-radius: 3px; | ||
| 143 | + font-size: 11px; | ||
| 144 | + } | ||
| 145 | + .kernel-fields-config { | ||
| 146 | + position: relative; | ||
| 147 | + display: inline-block; | ||
| 148 | + } | ||
| 149 | + .kernel-fields-panel { | ||
| 150 | + position: absolute; | ||
| 151 | + top: 100%; | ||
| 152 | + left: 0; | ||
| 153 | + background: var(--bg-tertiary); | ||
| 154 | + border: 1px solid var(--border-color); | ||
| 155 | + border-radius: 6px; | ||
| 156 | + padding: 12px; | ||
| 157 | + min-width: 200px; | ||
| 158 | + z-index: 1000; | ||
| 159 | + display: none; | ||
| 160 | + box-shadow: 0 4px 12px rgba(0,0,0,0.3); | ||
| 161 | + margin-top: 4px; | ||
| 162 | + } | ||
| 163 | + .kernel-fields-panel.visible { | ||
| 164 | + display: block; | ||
| 165 | + } | ||
| 166 | + .kernel-fields-panel label { | ||
| 167 | + display: flex; | ||
| 168 | + align-items: center; | ||
| 169 | + gap: 8px; | ||
| 170 | + padding: 4px 0; | ||
| 171 | + cursor: pointer; | ||
| 172 | + font-size: 13px; | ||
| 173 | + color: var(--text-primary); | ||
| 174 | + } | ||
| 175 | + .kernel-fields-panel input[type="checkbox"] { | ||
| 176 | + cursor: pointer; | ||
| 177 | + } | ||
| 178 | + .kernel-fields-actions { | ||
| 179 | + display: flex; | ||
| 180 | + gap: 8px; | ||
| 181 | + margin-top: 8px; | ||
| 182 | + padding-top: 8px; | ||
| 183 | + border-top: 1px solid var(--border-color); | ||
| 184 | + } | ||
| 185 | + .kernel-fields-actions button { | ||
| 186 | + flex: 1; | ||
| 187 | + padding: 4px 8px; | ||
| 188 | + font-size: 11px; | ||
| 189 | + background: var(--bg-secondary); | ||
| 190 | + border: 1px solid var(--border-color); | ||
| 191 | + color: var(--text-primary); | ||
| 192 | + border-radius: 3px; | ||
| 193 | + cursor: pointer; | ||
| 194 | + } | ||
| 195 | + .kernel-fields-actions button:hover { | ||
| 196 | + background: var(--bg-hover); | ||
| 197 | + } | ||
| 198 | + .controls { | ||
| 199 | + background: var(--bg-tertiary); | ||
| 200 | + padding: 15px 20px; | ||
| 201 | + border-radius: 8px; | ||
| 202 | + margin-bottom: 20px; | ||
| 203 | + display: flex; | ||
| 204 | + align-items: center; | ||
| 205 | + gap: 15px; | ||
| 206 | + flex-wrap: wrap; | ||
| 207 | + } | ||
| 208 | + .controls label { display: flex; align-items: center; gap: 8px; color: var(--text-secondary); font-size: 13px; } | ||
| 209 | + .controls select { | ||
| 210 | + background: var(--bg-secondary); | ||
| 211 | + color: var(--text-primary); | ||
| 212 | + border: 1px solid var(--border-color); | ||
| 213 | + padding: 6px 12px; | ||
| 214 | + border-radius: 4px; | ||
| 215 | + font-size: 13px; | ||
| 216 | + cursor: pointer; | ||
| 217 | + } | ||
| 218 | + .controls button { | ||
| 219 | + background: var(--button-bg); | ||
| 220 | + color: #ffffff; | ||
| 221 | + border: none; | ||
| 222 | + padding: 8px 16px; | ||
| 223 | + border-radius: 4px; | ||
| 224 | + cursor: pointer; | ||
| 225 | + font-size: 13px; | ||
| 226 | + transition: background 0.2s; | ||
| 227 | + } | ||
| 228 | + .controls button:hover { background: var(--button-hover); } | ||
| 229 | + .tree { background: var(--bg-secondary); border-radius: 8px; padding: 20px; position: relative; } | ||
| 230 | + .tree-root { list-style: none; } | ||
| 231 | + .tree-children { list-style: none; padding-left: 24px; overflow: hidden; transition: max-height 0.25s ease-out; } | ||
| 232 | + .tree-node.collapsed > .tree-children { max-height: 0; } | ||
| 233 | + .tree-node.expanded > .tree-children { max-height: 50000px; } | ||
| 234 | + .node-header { | ||
| 235 | + display: flex; | ||
| 236 | + align-items: center; | ||
| 237 | + padding: 5px 10px; | ||
| 238 | + border-radius: 4px; | ||
| 239 | + cursor: pointer; | ||
| 240 | + transition: background 0.15s; | ||
| 241 | + flex-wrap: wrap; | ||
| 242 | + } | ||
| 243 | + .node-header:hover { background: var(--bg-hover); } | ||
| 244 | + .node-toggle { | ||
| 245 | + width: 16px; | ||
| 246 | + font-size: 10px; | ||
| 247 | + color: var(--text-secondary); | ||
| 248 | + text-align: center; | ||
| 249 | + flex-shrink: 0; | ||
| 250 | + transition: transform 0.2s; | ||
| 251 | + } | ||
| 252 | + .tree-node.collapsed > .node-header .node-toggle { transform: rotate(0deg); } | ||
| 253 | + .tree-node.expanded > .node-header .node-toggle { transform: rotate(90deg); } | ||
| 254 | + .tree-node.leaf > .node-header .node-toggle { visibility: hidden; } | ||
| 255 | + .tree-node.kernel-only > .node-header .node-toggle { visibility: hidden; } | ||
| 256 | + .node-name { | ||
| 257 | + flex: 1; | ||
| 258 | + margin-left: 8px; | ||
| 259 | + font-family: 'Consolas', 'Monaco', monospace; | ||
| 260 | + font-size: 14px; | ||
| 261 | + min-width: 150px; | ||
| 262 | + } | ||
| 263 | + .node-semantic { | ||
| 264 | + color: var(--accent-orange); | ||
| 265 | + font-size: 11px; | ||
| 266 | + font-style: italic; | ||
| 267 | + font-weight: normal; | ||
| 268 | + margin-left: 6px; | ||
| 269 | + } | ||
| 270 | + .node-semantic-wrapper { | ||
| 271 | + display: inline-flex; | ||
| 272 | + align-items: center; | ||
| 273 | + color: var(--accent-orange); | ||
| 274 | + font-size: 11px; | ||
| 275 | + font-style: italic; | ||
| 276 | + margin-left: 6px; | ||
| 277 | + } | ||
| 278 | + .node-semantic-truncated { | ||
| 279 | + overflow: hidden; | ||
| 280 | + text-overflow: ellipsis; | ||
| 281 | + white-space: nowrap; | ||
| 282 | + max-width: 400px; | ||
| 283 | + } | ||
| 284 | + .node-semantic-full { | ||
| 285 | + display: none; | ||
| 286 | + } | ||
| 287 | + .node-semantic-wrapper.expanded .node-semantic-truncated { | ||
| 288 | + display: none; | ||
| 289 | + } | ||
| 290 | + .node-semantic-wrapper.expanded .node-semantic-full { | ||
| 291 | + display: inline; | ||
| 292 | + white-space: normal; | ||
| 293 | + } | ||
| 294 | + .semantic-expand-btn { | ||
| 295 | + display: inline-flex; | ||
| 296 | + align-items: center; | ||
| 297 | + justify-content: center; | ||
| 298 | + cursor: pointer; | ||
| 299 | + margin-left: 4px; | ||
| 300 | + flex-shrink: 0; | ||
| 301 | + user-select: none; | ||
| 302 | + width: 18px; | ||
| 303 | + height: 18px; | ||
| 304 | + border: none; | ||
| 305 | + border-radius: 3px; | ||
| 306 | + background: var(--accent-orange); | ||
| 307 | + padding: 0; | ||
| 308 | + transition: background 0.15s; | ||
| 309 | + } | ||
| 310 | + .semantic-expand-btn svg { | ||
| 311 | + width: 12px; | ||
| 312 | + height: 12px; | ||
| 313 | + fill: var(--bg-primary); | ||
| 314 | + transition: transform 0.2s; | ||
| 315 | + } | ||
| 316 | + .node-semantic-wrapper.expanded .semantic-expand-btn { | ||
| 317 | + background: var(--accent-blue); | ||
| 318 | + } | ||
| 319 | + .node-semantic-wrapper.expanded .semantic-expand-btn svg { | ||
| 320 | + transform: rotate(180deg); | ||
| 321 | + } | ||
| 322 | + .semantic-expand-btn:hover { | ||
| 323 | + opacity: 0.85; | ||
| 324 | + } | ||
| 325 | + .kernel-semantic { | ||
| 326 | + color: var(--accent-orange); | ||
| 327 | + font-size: 11px; | ||
| 328 | + font-style: italic; | ||
| 329 | + margin-left: 4px; | ||
| 330 | + } | ||
| 331 | + .kernel-count, .stream-count { | ||
| 332 | + color: var(--text-secondary); | ||
| 333 | + font-size: 11px; | ||
| 334 | + margin-left: 4px; | ||
| 335 | + } | ||
| 336 | + .tree-node.hidden-kernel { | ||
| 337 | + display: none; | ||
| 338 | + } | ||
| 339 | + .kernel-meta { | ||
| 340 | + color: var(--text-secondary); | ||
| 341 | + font-size: 11px; | ||
| 342 | + margin-left: 28px; | ||
| 343 | + width: calc(100% - 28px); | ||
| 344 | + display: grid; | ||
| 345 | + grid-template-columns: repeat(3, 1fr); | ||
| 346 | + gap: 2px 16px; | ||
| 347 | + font-family: 'Consolas', monospace; | ||
| 348 | + } | ||
| 349 | + .kernel-meta-item { | ||
| 350 | + display: flex; | ||
| 351 | + align-items: flex-start; | ||
| 352 | + gap: 4px; | ||
| 353 | + min-width: 0; | ||
| 354 | + } | ||
| 355 | + .kernel-meta-label { | ||
| 356 | + color: var(--text-secondary); | ||
| 357 | + flex-shrink: 0; | ||
| 358 | + } | ||
| 359 | + .kernel-meta-value { | ||
| 360 | + color: var(--text-primary); | ||
| 361 | + word-break: break-all; | ||
| 362 | + overflow-wrap: break-word; | ||
| 363 | + } | ||
| 364 | + .kernel-meta-item.shape-semantic-tip { | ||
| 365 | + position: relative; | ||
| 366 | + cursor: help; | ||
| 367 | + } | ||
| 368 | + .kernel-meta-item.shape-semantic-tip::after { | ||
| 369 | + content: attr(data-shape-semantic); | ||
| 370 | + position: absolute; | ||
| 371 | + bottom: calc(100% + 5px); | ||
| 372 | + left: 0; | ||
| 373 | + background: rgba(20,22,34,0.97); | ||
| 374 | + color: #e8c77a; | ||
| 375 | + border: 1px solid rgba(232,199,122,0.35); | ||
| 376 | + padding: 5px 9px; | ||
| 377 | + border-radius: 5px; | ||
| 378 | + font-size: 11px; | ||
| 379 | + font-family: 'Consolas', monospace; | ||
| 380 | + white-space: normal; | ||
| 381 | + max-width: 480px; | ||
| 382 | + min-width: 180px; | ||
| 383 | + pointer-events: none; | ||
| 384 | + opacity: 0; | ||
| 385 | + transition: opacity 0.15s; | ||
| 386 | + z-index: 300; | ||
| 387 | + box-shadow: 0 4px 12px rgba(0,0,0,0.5); | ||
| 388 | + line-height: 1.5; | ||
| 389 | + } | ||
| 390 | + .kernel-meta-item.shape-semantic-tip:hover::after { | ||
| 391 | + opacity: 1; | ||
| 392 | + } | ||
| 393 | + .node-duration { | ||
| 394 | + text-align: right; | ||
| 395 | + min-width: 200px; | ||
| 396 | + font-family: 'Consolas', 'Monaco', monospace; | ||
| 397 | + font-size: 12px; | ||
| 398 | + display: flex; | ||
| 399 | + justify-content: flex-end; | ||
| 400 | + gap: 6px; | ||
| 401 | + } | ||
| 402 | + .duration-us { color: var(--accent-green); } | ||
| 403 | + .duration-ms { color: var(--accent-orange); font-size: 11px; } | ||
| 404 | + .duration-pct { color: var(--accent-blue); font-weight: bold; min-width: 65px; } | ||
| 405 | + .tree-node[data-type="kernel"] .node-name { | ||
| 406 | + color: var(--accent-blue); | ||
| 407 | + font-style: italic; | ||
| 408 | + font-size: 13px; | ||
| 409 | + max-width: 200px; | ||
| 410 | + overflow: hidden; | ||
| 411 | + text-overflow: ellipsis; | ||
| 412 | + white-space: nowrap; | ||
| 413 | + } | ||
| 414 | + .tree-node[data-type="kernel"] .node-duration { | ||
| 415 | + margin-left: auto; | ||
| 416 | + } | ||
| 417 | + .kernel-info-btn { | ||
| 418 | + display: inline-flex; | ||
| 419 | + align-items: center; | ||
| 420 | + justify-content: center; | ||
| 421 | + width: 18px; | ||
| 422 | + height: 18px; | ||
| 423 | + background: var(--accent-blue); | ||
| 424 | + color: #fff; | ||
| 425 | + border-radius: 50%; | ||
| 426 | + cursor: pointer; | ||
| 427 | + margin-left: 4px; | ||
| 428 | + flex-shrink: 0; | ||
| 429 | + user-select: none; | ||
| 430 | + border-bottom: none; | ||
| 431 | + padding: 0; | ||
| 432 | + border: none; | ||
| 433 | + } | ||
| 434 | + .kernel-info-btn svg { | ||
| 435 | + width: 12px; | ||
| 436 | + height: 12px; | ||
| 437 | + fill: #fff; | ||
| 438 | + } | ||
| 439 | + .kernel-info-btn:hover { | ||
| 440 | + opacity: 0.8; | ||
| 441 | + } | ||
| 442 | + .node-header[title] { cursor: help; } | ||
| 443 | + .node-spacer { width: 16px; flex-shrink: 0; } | ||
| 444 | + .tree-node.leaf > .node-header { cursor: pointer; } | ||
| 445 | + | ||
| 446 | + /* Kernel Tooltip Styles */ | ||
| 447 | + .kernel-tooltip { | ||
| 448 | + position: fixed; | ||
| 449 | + background: var(--tooltip-bg); | ||
| 450 | + border: 1px solid var(--tooltip-border); | ||
| 451 | + border-radius: 8px; | ||
| 452 | + padding: 12px 16px; | ||
| 453 | + max-width: 1200px; | ||
| 454 | + max-height: 80vh; | ||
| 455 | + overflow-y: auto; | ||
| 456 | + z-index: 10000; | ||
| 457 | + box-shadow: 0 8px 32px rgba(0,0,0,0.4); | ||
| 458 | + font-size: 12px; | ||
| 459 | + display: none; | ||
| 460 | + } | ||
| 461 | + .kernel-tooltip.visible { display: block; } | ||
| 462 | + .kernel-tooltip-header { | ||
| 463 | + font-weight: bold; | ||
| 464 | + font-size: 14px; | ||
| 465 | + margin-bottom: 8px; | ||
| 466 | + padding-bottom: 8px; | ||
| 467 | + border-bottom: 1px solid var(--tooltip-border); | ||
| 468 | + color: var(--accent-blue); | ||
| 469 | + word-break: break-all; | ||
| 470 | + } | ||
| 471 | + .kernel-tooltip-semantic { | ||
| 472 | + background: rgba(79, 193, 255, 0.1); | ||
| 473 | + border-left: 3px solid var(--accent-blue); | ||
| 474 | + padding: 6px 10px; | ||
| 475 | + margin-bottom: 10px; | ||
| 476 | + font-style: italic; | ||
| 477 | + color: var(--text-secondary); | ||
| 478 | + } | ||
| 479 | + .kernel-tooltip-container { | ||
| 480 | + width: 100%; | ||
| 481 | + } | ||
| 482 | + .kernel-tooltip-row { | ||
| 483 | + display: flex; | ||
| 484 | + gap: 16px; | ||
| 485 | + margin-bottom: 8px; | ||
| 486 | + } | ||
| 487 | + .kernel-tooltip-row:last-child { | ||
| 488 | + margin-bottom: 0; | ||
| 489 | + } | ||
| 490 | + .kernel-tooltip-group { | ||
| 491 | + flex: 1; | ||
| 492 | + min-width: 160px; | ||
| 493 | + } | ||
| 494 | + .kernel-tooltip-group-grid { | ||
| 495 | + display: grid; | ||
| 496 | + grid-template-columns: auto 1fr; | ||
| 497 | + gap: 2px 8px; | ||
| 498 | + } | ||
| 499 | + .kernel-tooltip-label { | ||
| 500 | + font-weight: 500; | ||
| 501 | + color: var(--text-secondary); | ||
| 502 | + white-space: nowrap; | ||
| 503 | + font-size: 11px; | ||
| 504 | + } | ||
| 505 | + .kernel-tooltip-value { | ||
| 506 | + color: var(--text-primary); | ||
| 507 | + word-break: break-all; | ||
| 508 | + font-size: 11px; | ||
| 509 | + } | ||
| 510 | + .kernel-tooltip-section { | ||
| 511 | + font-weight: bold; | ||
| 512 | + color: var(--accent-orange); | ||
| 513 | + padding: 6px 0 4px 0; | ||
| 514 | + font-size: 12px; | ||
| 515 | + border-bottom: 1px solid var(--tooltip-border); | ||
| 516 | + margin-bottom: 4px; | ||
| 517 | + } | ||
| 518 | + .kernel-tooltip-full { | ||
| 519 | + grid-column: 1 / -1; | ||
| 520 | + display: flex; | ||
| 521 | + gap: 8px; | ||
| 522 | + } | ||
| 523 | + .kernel-tooltip-full .kernel-tooltip-label { | ||
| 524 | + flex-shrink: 0; | ||
| 525 | + } | ||
| 526 | + | ||
| 527 | + /* Timeline Styles - Module Schematic View */ | ||
| 528 | +.timeline-section { | ||
| 529 | + background: var(--bg-secondary); | ||
| 530 | + border-radius: 8px; | ||
| 531 | + padding: 20px; | ||
| 532 | + margin-top: 20px; | ||
| 533 | + } | ||
| 534 | + .timeline-header { | ||
| 535 | + display: flex; | ||
| 536 | + justify-content: space-between; | ||
| 537 | + align-items: center; | ||
| 538 | + margin-bottom: 15px; | ||
| 539 | + } | ||
| 540 | + .timeline-header h2 { | ||
| 541 | + font-size: 18px; | ||
| 542 | + color: var(--text-primary); | ||
| 543 | + margin: 0; | ||
| 544 | + } | ||
| 545 | + .timeline-hint { | ||
| 546 | + font-size: 12px; | ||
| 547 | + color: var(--text-secondary); | ||
| 548 | + } | ||
| 549 | + .timeline-container { | ||
| 550 | + position: relative; | ||
| 551 | + overflow-x: auto; | ||
| 552 | + overflow-y: visible; | ||
| 553 | + } | ||
| 554 | + .timeline-overview { | ||
| 555 | + margin: 16px 0; | ||
| 556 | + } | ||
| 557 | + .timeline-overview-title { | ||
| 558 | + font-size: 14px; | ||
| 559 | + color: var(--text-primary); | ||
| 560 | + margin-bottom: 12px; | ||
| 561 | + font-weight: 500; | ||
| 562 | + } | ||
| 563 | + .timeline-overview-list { | ||
| 564 | + border: 1px solid var(--border-color); | ||
| 565 | + border-radius: 6px; | ||
| 566 | + padding: 8px; | ||
| 567 | + background: var(--bg-tertiary); | ||
| 568 | + } | ||
| 569 | + .timeline-node-item { | ||
| 570 | + display: flex; | ||
| 571 | + align-items: center; | ||
| 572 | + padding: 10px 12px; | ||
| 573 | + margin: 4px 0; | ||
| 574 | + background: var(--bg-secondary); | ||
| 575 | + border-radius: 6px; | ||
| 576 | + cursor: pointer; | ||
| 577 | + transition: background 0.15s; | ||
| 578 | + } | ||
| 579 | + .timeline-node-item:hover { | ||
| 580 | + background: var(--bg-hover); | ||
| 581 | + } | ||
| 582 | + .timeline-node-item.has-children:hover { | ||
| 583 | + background: rgba(79, 193, 255, 0.1); | ||
| 584 | + } | ||
| 585 | + .node-expand-icon { | ||
| 586 | + width: 20px; | ||
| 587 | + font-size: 12px; | ||
| 588 | + color: var(--text-secondary); | ||
| 589 | + text-align: center; | ||
| 590 | + } | ||
| 591 | + .timeline-node-item.has-children .node-expand-icon { | ||
| 592 | + color: var(--accent-blue); | ||
| 593 | + } | ||
| 594 | + .node-name { | ||
| 595 | + flex: 1; | ||
| 596 | + font-weight: 500; | ||
| 597 | + color: var(--text-primary); | ||
| 598 | + } | ||
| 599 | + .node-streams { | ||
| 600 | + font-size: 11px; | ||
| 601 | + color: var(--text-secondary); | ||
| 602 | + margin: 0 12px; | ||
| 603 | + white-space: nowrap; | ||
| 604 | + } | ||
| 605 | + .node-duration { | ||
| 606 | + font-size: 11px; | ||
| 607 | + color: var(--accent-green); | ||
| 608 | + margin-right: 12px; | ||
| 609 | + white-space: nowrap; | ||
| 610 | + } | ||
| 611 | + .node-kernels { | ||
| 612 | + font-size: 11px; | ||
| 613 | + color: var(--text-secondary); | ||
| 614 | + white-space: nowrap; | ||
| 615 | + } | ||
| 616 | + .timeline-expand-area { | ||
| 617 | + margin: 16px 0; | ||
| 618 | + padding: 16px; | ||
| 619 | + background: var(--bg-tertiary); | ||
| 620 | + border-radius: 8px; | ||
| 621 | + border: 2px solid var(--accent-blue); | ||
| 622 | + } | ||
| 623 | + .expand-header { | ||
| 624 | + display: flex; | ||
| 625 | + align-items: center; | ||
| 626 | + margin-bottom: 16px; | ||
| 627 | + } | ||
| 628 | + .expand-close { | ||
| 629 | + background: var(--bg-secondary); | ||
| 630 | + border: 1px solid var(--border-color); | ||
| 631 | + border-radius: 4px; | ||
| 632 | + padding: 4px 12px; | ||
| 633 | + color: var(--text-secondary); | ||
| 634 | + cursor: pointer; | ||
| 635 | + font-size: 12px; | ||
| 636 | + margin-right: 12px; | ||
| 637 | + } | ||
| 638 | + .expand-close:hover { | ||
| 639 | + background: var(--bg-hover); | ||
| 640 | + color: var(--text-primary); | ||
| 641 | + } | ||
| 642 | + .expand-breadcrumb { | ||
| 643 | + display: flex; | ||
| 644 | + align-items: center; | ||
| 645 | + gap: 4px; | ||
| 646 | + flex: 1; | ||
| 647 | + min-width: 0; | ||
| 648 | + overflow-x: auto; | ||
| 649 | + } | ||
| 650 | + .breadcrumb-item { | ||
| 651 | + font-size: 12px; | ||
| 652 | + color: var(--accent-blue); | ||
| 653 | + cursor: pointer; | ||
| 654 | + white-space: nowrap; | ||
| 655 | + padding: 2px 6px; | ||
| 656 | + border-radius: 3px; | ||
| 657 | + transition: background 0.15s; | ||
| 658 | + } | ||
| 659 | + .breadcrumb-item:hover { | ||
| 660 | + background: rgba(79, 193, 255, 0.1); | ||
| 661 | + } | ||
| 662 | + .breadcrumb-item.current { | ||
| 663 | + color: var(--text-primary); | ||
| 664 | + font-weight: 600; | ||
| 665 | + cursor: default; | ||
| 666 | + } | ||
| 667 | + .breadcrumb-item.current:hover { | ||
| 668 | + background: transparent; | ||
| 669 | + } | ||
| 670 | + .breadcrumb-sep { | ||
| 671 | + color: var(--text-secondary); | ||
| 672 | + font-size: 10px; | ||
| 673 | + flex-shrink: 0; | ||
| 674 | + } | ||
| 675 | + .expand-title { | ||
| 676 | + font-weight: bold; | ||
| 677 | + color: var(--accent-blue); | ||
| 678 | + font-size: 14px; | ||
| 679 | + } | ||
| 680 | + .expand-gantt { | ||
| 681 | + margin: 16px 0; | ||
| 682 | + } | ||
| 683 | + .gantt-header { | ||
| 684 | + font-size: 13px; | ||
| 685 | + color: var(--text-primary); | ||
| 686 | + margin-bottom: 8px; | ||
| 687 | + } | ||
| 688 | + .gantt-hint { | ||
| 689 | + font-size: 11px; | ||
| 690 | + color: var(--text-secondary); | ||
| 691 | + } | ||
| 692 | + .gantt-container { | ||
| 693 | + background: var(--bg-secondary); | ||
| 694 | + border-radius: 6px; | ||
| 695 | + padding: 12px; | ||
| 696 | + border: 1px solid var(--border-color); | ||
| 697 | + } | ||
| 698 | + .gantt-stream-group { | ||
| 699 | + margin: 8px 0; | ||
| 700 | + } | ||
| 701 | + .gantt-stream-row { | ||
| 702 | + display: flex; | ||
| 703 | + align-items: center; | ||
| 704 | + min-height: 32px; | ||
| 705 | + } | ||
| 706 | + .gantt-stream-label { | ||
| 707 | + width: 90px; | ||
| 708 | + font-size: 11px; | ||
| 709 | + color: var(--text-secondary); | ||
| 710 | + flex-shrink: 0; | ||
| 711 | + } | ||
| 712 | + .gantt-bars { | ||
| 713 | + flex: 1; | ||
| 714 | + position: relative; | ||
| 715 | + height: 28px; | ||
| 716 | + background: rgba(0,0,0,0.15); | ||
| 717 | + border-radius: 4px; | ||
| 718 | + overflow: visible; | ||
| 719 | + } | ||
| 720 | + .gantt-bar { | ||
| 721 | + position: absolute; | ||
| 722 | + height: 20px; | ||
| 723 | + top: 4px; | ||
| 724 | + border-radius: 3px; | ||
| 725 | + cursor: pointer; | ||
| 726 | + transition: opacity 0.15s, box-shadow 0.15s; | ||
| 727 | + } | ||
| 728 | + .gantt-bar:hover { | ||
| 729 | + opacity: 0.85; | ||
| 730 | + z-index: 10; | ||
| 731 | + box-shadow: 0 0 8px rgba(255,255,255,0.4); | ||
| 732 | + } | ||
| 733 | + .gantt-bar-secondary { | ||
| 734 | + height: 16px; | ||
| 735 | + top: 6px; | ||
| 736 | + } | ||
| 737 | + .gantt-bar-arrow { | ||
| 738 | + position: absolute; | ||
| 739 | + bottom: -5px; | ||
| 740 | + left: 50%; | ||
| 741 | + transform: translateX(-50%); | ||
| 742 | + width: 0; | ||
| 743 | + height: 0; | ||
| 744 | + border-left: 4px solid transparent; | ||
| 745 | + border-right: 4px solid transparent; | ||
| 746 | + border-top: 5px solid; | ||
| 747 | + } | ||
| 748 | + .gantt-labels-row { | ||
| 749 | + position: relative; | ||
| 750 | + min-height: 22px; | ||
| 751 | + margin-left: 90px; | ||
| 752 | + margin-top: 2px; | ||
| 753 | + } | ||
| 754 | + .gantt-label { | ||
| 755 | + position: absolute; | ||
| 756 | + font-size: 10px; | ||
| 757 | + white-space: nowrap; | ||
| 758 | + transform: translateX(-50%); | ||
| 759 | + padding: 1px 5px; | ||
| 760 | + border-radius: 3px; | ||
| 761 | + background: var(--bg-tertiary); | ||
| 762 | + } | ||
| 763 | + .gantt-label::before { | ||
| 764 | + content: ''; | ||
| 765 | + position: absolute; | ||
| 766 | + top: -4px; | ||
| 767 | + left: 50%; | ||
| 768 | + transform: translateX(-50%); | ||
| 769 | + width: 0; | ||
| 770 | + height: 0; | ||
| 771 | + border-left: 3px solid transparent; | ||
| 772 | + border-right: 3px solid transparent; | ||
| 773 | + border-bottom: 4px solid var(--bg-tertiary); | ||
| 774 | + } | ||
| 775 | + .gantt-time-axis { | ||
| 776 | + display: flex; | ||
| 777 | + justify-content: space-between; | ||
| 778 | + margin-top: 8px; | ||
| 779 | + padding-top: 8px; | ||
| 780 | + border-top: 1px solid var(--border-color); | ||
| 781 | + font-size: 10px; | ||
| 782 | + color: var(--text-secondary); | ||
| 783 | + } | ||
| 784 | + .expand-tree { | ||
| 785 | + margin-top: 16px; | ||
| 786 | + } | ||
| 787 | + .expand-tree-title { | ||
| 788 | + font-size: 13px; | ||
| 789 | + color: var(--text-primary); | ||
| 790 | + margin-bottom: 8px; | ||
| 791 | + } | ||
| 792 | + .tree-hint { | ||
| 793 | + font-size: 11px; | ||
| 794 | + color: var(--text-secondary); | ||
| 795 | + } | ||
| 796 | + .expand-tree-content { | ||
| 797 | + background: var(--bg-secondary); | ||
| 798 | + border-radius: 6px; | ||
| 799 | + padding: 8px 12px; | ||
| 800 | + border: 1px solid var(--border-color); | ||
| 801 | + } | ||
| 802 | + .tree-item { | ||
| 803 | + display: flex; | ||
| 804 | + align-items: center; | ||
| 805 | + padding: 6px 8px; | ||
| 806 | + margin: 2px 0; | ||
| 807 | + border-radius: 4px; | ||
| 808 | + cursor: default; | ||
| 809 | + transition: background 0.15s; | ||
| 810 | + } | ||
| 811 | + .tree-item.clickable { | ||
| 812 | + cursor: pointer; | ||
| 813 | + } | ||
| 814 | + .tree-item.clickable:hover { | ||
| 815 | + background: rgba(79, 193, 255, 0.1); | ||
| 816 | + } | ||
| 817 | + .tree-icon { | ||
| 818 | + width: 20px; | ||
| 819 | + font-size: 12px; | ||
| 820 | + color: var(--text-secondary); | ||
| 821 | + text-align: center; | ||
| 822 | + flex-shrink: 0; | ||
| 823 | + } | ||
| 824 | + .tree-item.clickable .tree-icon { | ||
| 825 | + color: var(--accent-blue); | ||
| 826 | + } | ||
| 827 | + .tree-name { | ||
| 828 | + flex: 1; | ||
| 829 | + font-size: 12px; | ||
| 830 | + color: var(--text-primary); | ||
| 831 | + } | ||
| 832 | + .tree-streams { | ||
| 833 | + font-size: 10px; | ||
| 834 | + color: var(--text-secondary); | ||
| 835 | + margin: 0 8px; | ||
| 836 | + white-space: nowrap; | ||
| 837 | + } | ||
| 838 | + .tree-duration { | ||
| 839 | + font-size: 10px; | ||
| 840 | + color: var(--accent-green); | ||
| 841 | + white-space: nowrap; | ||
| 842 | + } | ||
| 843 | + .timeline-tooltip { | ||
| 844 | + position: fixed; | ||
| 845 | + background: var(--tooltip-bg); | ||
| 846 | + border: 1px solid var(--tooltip-border); | ||
| 847 | + border-radius: 6px; | ||
| 848 | + padding: 10px 12px; | ||
| 849 | + font-size: 12px; | ||
| 850 | + z-index: 10000; | ||
| 851 | + box-shadow: 0 4px 12px rgba(0,0,0,0.4); | ||
| 852 | + max-width: 400px; | ||
| 853 | + display: none; | ||
| 854 | + } | ||
| 855 | + .timeline-tooltip.visible { | ||
| 856 | + display: block; | ||
| 857 | + } | ||
| 858 | + .timeline-tooltip-title { | ||
| 859 | + font-weight: bold; | ||
| 860 | + color: var(--accent-blue); | ||
| 861 | + margin-bottom: 6px; | ||
| 862 | + } | ||
| 863 | + .timeline-tooltip-row { | ||
| 864 | + display: flex; | ||
| 865 | + justify-content: space-between; | ||
| 866 | + gap: 20px; | ||
| 867 | + margin-bottom: 3px; | ||
| 868 | + color: var(--text-primary); | ||
| 869 | + } | ||
| 870 | + .timeline-tooltip-label { | ||
| 871 | + color: var(--text-secondary); | ||
| 872 | + } | ||
| 873 | + .timeline-tooltip-streams { | ||
| 874 | + margin-top: 6px; | ||
| 875 | + padding-top: 6px; | ||
| 876 | + border-top: 1px solid var(--tooltip-border); | ||
| 877 | + font-size: 11px; | ||
| 878 | + } | ||
| 879 | + .timeline-detail-panel { | ||
| 880 | + margin-top: 15px; | ||
| 881 | + background: var(--bg-tertiary); | ||
| 882 | + border-radius: 6px; | ||
| 883 | + padding: 15px; | ||
| 884 | + display: none; | ||
| 885 | + } | ||
| 886 | + .timeline-detail-panel.visible { | ||
| 887 | + display: block; | ||
| 888 | + } | ||
| 889 | + .timeline-detail-header { | ||
| 890 | + display: flex; | ||
| 891 | + justify-content: space-between; | ||
| 892 | + align-items: center; | ||
| 893 | + margin-bottom: 10px; | ||
| 894 | + } | ||
| 895 | + .timeline-detail-title { | ||
| 896 | + font-weight: bold; | ||
| 897 | + color: var(--accent-blue); | ||
| 898 | + } | ||
| 899 | + .timeline-detail-close { | ||
| 900 | + background: transparent; | ||
| 901 | + border: none; | ||
| 902 | + color: var(--text-secondary); | ||
| 903 | + cursor: pointer; | ||
| 904 | + font-size: 18px; | ||
| 905 | + line-height: 1; | ||
| 906 | + } | ||
| 907 | + .timeline-detail-close:hover { | ||
| 908 | + color: var(--text-primary); | ||
| 909 | + } | ||
| 910 | + .timeline-detail-ops { | ||
| 911 | + background: var(--bg-secondary); | ||
| 912 | + border-radius: 4px; | ||
| 913 | + padding: 12px; | ||
| 914 | + } | ||
| 915 | + .timeline-stream-row { | ||
| 916 | + display: flex; | ||
| 917 | + flex-wrap: wrap; | ||
| 918 | + align-items: center; | ||
| 919 | + padding: 8px 0; | ||
| 920 | + border-bottom: 1px solid var(--border-color); | ||
| 921 | + gap: 6px; | ||
| 922 | + } | ||
| 923 | + .timeline-stream-row:last-child { | ||
| 924 | + border-bottom: none; | ||
| 925 | + } | ||
| 926 | + .timeline-stream-label { | ||
| 927 | + width: 100px; | ||
| 928 | + flex-shrink: 0; | ||
| 929 | + font-size: 11px; | ||
| 930 | + font-weight: 500; | ||
| 931 | + padding: 4px 8px; | ||
| 932 | + border-radius: 3px; | ||
| 933 | + text-align: center; | ||
| 934 | + } | ||
| 935 | + .timeline-stream-ops { | ||
| 936 | + flex: 1; | ||
| 937 | + position: relative; | ||
| 938 | + height: 26px; | ||
| 939 | + background: rgba(0,0,0,0.12); | ||
| 940 | + border-radius: 4px; | ||
| 941 | + } | ||
| 942 | + .timeline-detail-bar { | ||
| 943 | + position: absolute; | ||
| 944 | + height: 22px; | ||
| 945 | + top: 2px; | ||
| 946 | + border-radius: 3px; | ||
| 947 | + font-size: 9px; | ||
| 948 | + color: #fff; | ||
| 949 | + display: flex; | ||
| 950 | + align-items: center; | ||
| 951 | + justify-content: center; | ||
| 952 | + overflow: hidden; | ||
| 953 | + cursor: pointer; | ||
| 954 | + transition: opacity 0.15s, box-shadow 0.15s; | ||
| 955 | + } | ||
| 956 | + .timeline-detail-bar:hover { | ||
| 957 | + opacity: 0.85; | ||
| 958 | + z-index: 10; | ||
| 959 | + box-shadow: 0 0 6px rgba(255,255,255,0.3); | ||
| 960 | + } | ||
| 961 | + .covered-bar { | ||
| 962 | + background-image: repeating-linear-gradient( | ||
| 963 | + 45deg, transparent, transparent 3px, | ||
| 964 | + rgba(255,255,255,0.18) 3px, rgba(255,255,255,0.18) 6px | ||
| 965 | + ) !important; | ||
| 966 | + opacity: 0.55 !important; | ||
| 967 | + } | ||
| 968 | + .covered-badge { | ||
| 969 | + font-size: 10px; | ||
| 970 | + background: rgba(255,120,100,0.25); | ||
| 971 | + color: #ff7864; | ||
| 972 | + border: 1px solid rgba(255,120,100,0.45); | ||
| 973 | + border-radius: 3px; | ||
| 974 | + padding: 0 4px; | ||
| 975 | + margin-left: 5px; | ||
| 976 | + vertical-align: middle; | ||
| 977 | + white-space: nowrap; | ||
| 978 | + } | ||
| 979 | + </style> | ||
| 980 | + | ||
| 981 | + ''' | ||
| 982 | + | ||
| 983 | + | ||
| 984 | +JS_TEMPLATE = """ | ||
| 985 | + <script> | ||
| 986 | + let currentTooltip = null; | ||
| 987 | + const tooltipData = @@TOOLTIP_JSON@@; | ||
| 988 | + const defaultFields = @@FIELDS_JSON@@; | ||
| 989 | + const tlData = @@TL_JSON@@; | ||
| 990 | + | ||
| 991 | + function showTooltip(el) { | ||
| 992 | + const idx = el.dataset.index; | ||
| 993 | + | ||
| 994 | + if (currentTooltip && currentTooltip.classList.contains('visible') && | ||
| 995 | + currentTooltip.dataset.currentIdx === idx) { | ||
| 996 | + hideTooltip(); | ||
| 997 | + return; | ||
| 998 | + } | ||
| 999 | + | ||
| 1000 | + if (!tooltipData[idx]) return; | ||
| 1001 | + | ||
| 1002 | + if (!currentTooltip) { | ||
| 1003 | + currentTooltip = document.createElement('div'); | ||
| 1004 | + currentTooltip.className = 'kernel-tooltip'; | ||
| 1005 | + document.body.appendChild(currentTooltip); | ||
| 1006 | + } | ||
| 1007 | + | ||
| 1008 | + currentTooltip.innerHTML = tooltipData[idx]; | ||
| 1009 | + currentTooltip.classList.add('visible'); | ||
| 1010 | + currentTooltip.dataset.currentIdx = idx; | ||
| 1011 | + currentTooltip.style.left = ''; | ||
| 1012 | + currentTooltip.style.top = ''; | ||
| 1013 | + currentTooltip.style.right = ''; | ||
| 1014 | + | ||
| 1015 | + const rect = el.getBoundingClientRect(); | ||
| 1016 | + currentTooltip.style.position = 'fixed'; | ||
| 1017 | + | ||
| 1018 | + const tooltipEl = currentTooltip; | ||
| 1019 | + const tw = tooltipEl.offsetWidth; | ||
| 1020 | + const th = tooltipEl.offsetHeight; | ||
| 1021 | + const vw = window.innerWidth; | ||
| 1022 | + const vh = window.innerHeight; | ||
| 1023 | + | ||
| 1024 | + let left = rect.right + 12; | ||
| 1025 | + let top = rect.top; | ||
| 1026 | + | ||
| 1027 | + if (left + tw > vw - 10) { | ||
| 1028 | + left = Math.max(10, vw - tw - 10); | ||
| 1029 | + } | ||
| 1030 | + if (top + th > vh - 10) { | ||
| 1031 | + top = Math.max(10, vh - th - 10); | ||
| 1032 | + } | ||
| 1033 | + if (top < 10) top = 10; | ||
| 1034 | + | ||
| 1035 | + currentTooltip.style.left = left + 'px'; | ||
| 1036 | + currentTooltip.style.top = top + 'px'; | ||
| 1037 | + } | ||
| 1038 | + | ||
| 1039 | + function hideTooltip() { | ||
| 1040 | + if (currentTooltip) { | ||
| 1041 | + currentTooltip.classList.remove('visible'); | ||
| 1042 | + } | ||
| 1043 | + } | ||
| 1044 | + | ||
| 1045 | + function applyTheme(theme) { | ||
| 1046 | + document.documentElement.setAttribute('data-theme', theme); | ||
| 1047 | + localStorage.setItem('report-theme', theme); | ||
| 1048 | + document.getElementById('theme-select').value = theme; | ||
| 1049 | + } | ||
| 1050 | + | ||
| 1051 | + function initTheme() { | ||
| 1052 | + const saved = localStorage.getItem('report-theme') || '@@DEFAULT_THEME@@'; | ||
| 1053 | + applyTheme(saved); | ||
| 1054 | + } | ||
| 1055 | + | ||
| 1056 | + function toggleKernels(visible) { | ||
| 1057 | + document.querySelectorAll('.tree-node[data-type="kernel"]').forEach(node => { | ||
| 1058 | + if (visible) { | ||
| 1059 | + node.classList.remove('hidden-kernel'); | ||
| 1060 | + } else { | ||
| 1061 | + node.classList.add('hidden-kernel'); | ||
| 1062 | + } | ||
| 1063 | + }); | ||
| 1064 | + if (visible) { | ||
| 1065 | + document.querySelectorAll('.tree-node[data-type="kernel"]').forEach(kernel => { | ||
| 1066 | + let parent = kernel.parentElement; | ||
| 1067 | + while (parent) { | ||
| 1068 | + if (parent.classList.contains('tree-node') && parent.classList.contains('collapsed')) { | ||
| 1069 | + parent.classList.remove('collapsed'); | ||
| 1070 | + parent.classList.add('expanded'); | ||
| 1071 | + } | ||
| 1072 | + parent = parent.parentElement; | ||
| 1073 | + } | ||
| 1074 | + }); | ||
| 1075 | + } | ||
| 1076 | + } | ||
| 1077 | + | ||
| 1078 | + function updateKernelMetaFields() { | ||
| 1079 | + const panel = document.getElementById('kernel-fields-panel'); | ||
| 1080 | + const checked = Array.from(panel.querySelectorAll('input:checked')).map(cb => cb.value); | ||
| 1081 | + document.querySelectorAll('.kernel-meta').forEach(meta => { | ||
| 1082 | + meta.querySelectorAll('.kernel-meta-item').forEach(item => { | ||
| 1083 | + const field = item.dataset.field; | ||
| 1084 | + item.style.display = checked.includes(field) ? 'flex' : 'none'; | ||
| 1085 | + }); | ||
| 1086 | + }); | ||
| 1087 | + } | ||
| 1088 | + | ||
| 1089 | + /* ========== Timeline Logic ========== */ | ||
| 1090 | + let tlTooltip = null; | ||
| 1091 | + let currentExpandedNode = null; | ||
| 1092 | + let navHistory = []; | ||
| 1093 | + | ||
| 1094 | + function formatUs(us) { | ||
| 1095 | + if (us >= 1000) return (us / 1000).toFixed(1) + 'ms'; | ||
| 1096 | + return us.toFixed(1) + 'us'; | ||
| 1097 | + } | ||
| 1098 | + | ||
| 1099 | + function initTimelineTooltip() { | ||
| 1100 | + if (!tlTooltip) { | ||
| 1101 | + tlTooltip = document.createElement('div'); | ||
| 1102 | + tlTooltip.className = 'timeline-tooltip'; | ||
| 1103 | + document.body.appendChild(tlTooltip); | ||
| 1104 | + } | ||
| 1105 | + } | ||
| 1106 | + | ||
| 1107 | + function showTimelineTooltip(el, barInfo) { | ||
| 1108 | + initTimelineTooltip(); | ||
| 1109 | + let streamInfo = ''; | ||
| 1110 | + if (barInfo.streams && barInfo.streams.length > 0) { | ||
| 1111 | + streamInfo = '<div class="timeline-tooltip-streams">' + | ||
| 1112 | + '<span class="timeline-tooltip-label">Streams:</span> ' + | ||
| 1113 | + barInfo.streams.map(s => 'Stream ' + s).join(', ') + '</div>'; | ||
| 1114 | + } | ||
| 1115 | + const countSuffix = barInfo.multiplier > 1 ? ' (×' + barInfo.multiplier + ')' : ''; | ||
| 1116 | + const hasChildrenHint = barInfo.has_children ? '点击展开子节点' : '点击查看算子详情'; | ||
| 1117 | + tlTooltip.innerHTML = | ||
| 1118 | + '<div class="timeline-tooltip-title">' + barInfo.name + countSuffix + '</div>' + | ||
| 1119 | + '<div class="timeline-tooltip-row"><span class="timeline-tooltip-label">时间:</span> ' + | ||
| 1120 | + formatUs(barInfo.duration) + '</div>' + | ||
| 1121 | + '<div class="timeline-tooltip-row"><span class="timeline-tooltip-label">Kernels:</span> ' + | ||
| 1122 | + barInfo.kernel_count + '</div>' + | ||
| 1123 | + streamInfo + | ||
| 1124 | + '<div style="margin-top:6px;color:var(--text-secondary);font-size:11px">' + hasChildrenHint + '</div>'; | ||
| 1125 | + tlTooltip.classList.add('visible'); | ||
| 1126 | + const rect = el.getBoundingClientRect(); | ||
| 1127 | + const vw = window.innerWidth; | ||
| 1128 | + const vh = window.innerHeight; | ||
| 1129 | + let left = rect.right + 10; | ||
| 1130 | + let top = rect.top; | ||
| 1131 | + if (left + 320 > vw) left = Math.max(10, rect.left - 330); | ||
| 1132 | + if (top + 180 > vh) top = Math.max(10, vh - 190); | ||
| 1133 | + if (top < 10) top = 10; | ||
| 1134 | + tlTooltip.style.left = left + 'px'; | ||
| 1135 | + tlTooltip.style.top = top + 'px'; | ||
| 1136 | + } | ||
| 1137 | + | ||
| 1138 | + function hideTimelineTooltip() { | ||
| 1139 | + if (tlTooltip) tlTooltip.classList.remove('visible'); | ||
| 1140 | + } | ||
| 1141 | + | ||
| 1142 | + function showOpsDetail(nodeIndex) { | ||
| 1143 | + if (!tlData.bar_data || !tlData.bar_data[nodeIndex]) return; | ||
| 1144 | + const bar = tlData.bar_data[nodeIndex]; | ||
| 1145 | + const panel = document.getElementById('timeline-detail-panel'); | ||
| 1146 | + const title = document.getElementById('timeline-detail-title'); | ||
| 1147 | + const opsContainer = document.getElementById('timeline-detail-ops'); | ||
| 1148 | + | ||
| 1149 | + const countSuffix = bar.multiplier > 1 ? ' (×' + bar.multiplier + ')' : ''; | ||
| 1150 | + title.textContent = bar.name + countSuffix + ' - 算子列表'; | ||
| 1151 | + | ||
| 1152 | + const allOps = []; | ||
| 1153 | + if (bar.per_stream_ops) { | ||
| 1154 | + for (const sid in bar.per_stream_ops) { | ||
| 1155 | + for (const op of bar.per_stream_ops[sid]) { | ||
| 1156 | + allOps.push(op); | ||
| 1157 | + } | ||
| 1158 | + } | ||
| 1159 | + } | ||
| 1160 | + if (allOps.length === 0) { | ||
| 1161 | + opsContainer.innerHTML = '<div style="padding:10px;color:var(--text-secondary)">无算子数据</div>'; | ||
| 1162 | + panel.classList.add('visible'); | ||
| 1163 | + return; | ||
| 1164 | + } | ||
| 1165 | + | ||
| 1166 | + const timeMin = Math.min(...allOps.map(o => o.start)); | ||
| 1167 | + const timeMax = Math.max(...allOps.map(o => o.end)); | ||
| 1168 | + const timeDuration = Math.max(1, timeMax - timeMin); | ||
| 1169 | + | ||
| 1170 | + const streamColors = {}; | ||
| 1171 | + const palette = ['#4fc1ff', '#6a9955', '#ce9178', '#c586c0', '#569cd6', '#dcdcaa', '#e06c75']; | ||
| 1172 | + let ci = 0; | ||
| 1173 | + const sortedSids = Object.keys(bar.per_stream_ops).sort((a, b) => Number(a) - Number(b)); | ||
| 1174 | + for (const sid of sortedSids) { | ||
| 1175 | + streamColors[sid] = palette[ci % palette.length]; | ||
| 1176 | + ci++; | ||
| 1177 | + } | ||
| 1178 | + | ||
| 1179 | + // Determine dominant stream (most ops) | ||
| 1180 | + const dominantSid = sortedSids.reduce((a, b) => | ||
| 1181 | + (bar.per_stream_ops[a].length >= bar.per_stream_ops[b].length ? a : b), sortedSids[0]); | ||
| 1182 | + const dominantOps = (bar.per_stream_ops[dominantSid] || []).map(o => ({ | ||
| 1183 | + start: o.start, end: o.end | ||
| 1184 | + })); | ||
| 1185 | + function isCoveredByDominant(op) { | ||
| 1186 | + return dominantOps.some(d => d.start < op.end && d.end > op.start); | ||
| 1187 | + } | ||
| 1188 | + | ||
| 1189 | + let html = ''; | ||
| 1190 | + for (const sid of sortedSids) { | ||
| 1191 | + const ops = bar.per_stream_ops[sid].sort((a, b) => a.start - b.start); | ||
| 1192 | + const color = streamColors[sid]; | ||
| 1193 | + const isAux = sid !== dominantSid; | ||
| 1194 | + const coveredCount = isAux ? ops.filter(op => isCoveredByDominant(op)).length : 0; | ||
| 1195 | + const coveredBadge = (isAux && coveredCount > 0) | ||
| 1196 | + ? '<span class="covered-badge">' + coveredCount + ' covered</span>' : ''; | ||
| 1197 | + html += '<div class="timeline-stream-row">'; | ||
| 1198 | + html += '<span class="timeline-stream-label" style="background:' + color + | ||
| 1199 | + '22;color:' + color + ';border:1px solid ' + color + '44">Stream ' + sid + | ||
| 1200 | + ' (' + ops.length + ')' + coveredBadge + '</span>'; | ||
| 1201 | + html += '<div class="timeline-stream-ops">'; | ||
| 1202 | + ops.forEach((op, opIndex) => { | ||
| 1203 | + const leftPct = ((op.start - timeMin) / timeDuration) * 100; | ||
| 1204 | + const widthPct = Math.max(1, (op.duration / timeDuration) * 100); | ||
| 1205 | + const shortName = op.name.length > 10 ? op.name.substring(0, 8) + '..' : op.name; | ||
| 1206 | + const covered = isAux && isCoveredByDominant(op); | ||
| 1207 | + const coveredTip = covered ? ('\\n⚠ covered by stream ' + dominantSid) : ''; | ||
| 1208 | + const tipText = op.name + '\\n历时: ' + formatUs(op.duration) + | ||
| 1209 | + '\\nStart: ' + formatUs(op.start) + coveredTip; | ||
| 1210 | + const showText = widthPct > 6; | ||
| 1211 | + // Alternating colors: even = full opacity, odd = semi-transparent + top border | ||
| 1212 | + const isEven = opIndex % 2 === 0; | ||
| 1213 | + const barAlpha = isEven ? 'ee' : '88'; | ||
| 1214 | + const borderExtra = isEven ? '' : ';border-top:2px solid ' + color + 'cc'; | ||
| 1215 | + const coveredClass = covered ? ' covered-bar' : ''; | ||
| 1216 | + html += '<span class="timeline-detail-bar' + coveredClass + '" style="left:' + | ||
| 1217 | + leftPct.toFixed(1) + '%;width:' + widthPct.toFixed(1) + '%;background:' + | ||
| 1218 | + color + barAlpha + borderExtra + '" title="' + tipText + '">' + | ||
| 1219 | + (showText ? shortName : '') + '</span>'; | ||
| 1220 | + }); | ||
| 1221 | + html += '</div></div>'; | ||
| 1222 | + } | ||
| 1223 | + opsContainer.innerHTML = html; | ||
| 1224 | + panel.classList.add('visible'); | ||
| 1225 | + panel.scrollIntoView({ behavior: 'smooth', block: 'nearest' }); | ||
| 1226 | + } | ||
| 1227 | + | ||
| 1228 | + function renderGantt(parentNode, children) { | ||
| 1229 | + const container = document.getElementById('gantt-container'); | ||
| 1230 | + if (!container || !children || children.length === 0) { | ||
| 1231 | + container.innerHTML = '<div style="padding:10px;color:var(--text-secondary)">无子节点数据</div>'; | ||
| 1232 | + return; | ||
| 1233 | + } | ||
| 1234 | + | ||
| 1235 | + const parentStart = parentNode.start; | ||
| 1236 | + const parentDuration = parentNode.duration || 1; | ||
| 1237 | + const allStreams = new Set(); | ||
| 1238 | + children.forEach(c => c.streams.forEach(s => allStreams.add(s))); | ||
| 1239 | + const sortedStreams = Array.from(allStreams).sort(); | ||
| 1240 | + | ||
| 1241 | + let html = ''; | ||
| 1242 | + sortedStreams.forEach(stream => { | ||
| 1243 | + const barsOnStream = children.filter(c => c.streams.includes(stream)); | ||
| 1244 | + if (barsOnStream.length === 0) return; | ||
| 1245 | + | ||
| 1246 | + html += '<div class="gantt-stream-group">'; | ||
| 1247 | + html += '<div class="gantt-stream-row">'; | ||
| 1248 | + html += '<span class="gantt-stream-label">Stream ' + stream + '</span>'; | ||
| 1249 | + html += '<div class="gantt-bars">'; | ||
| 1250 | + | ||
| 1251 | + barsOnStream.forEach(child => { | ||
| 1252 | + const leftPct = Math.max(0, (child.start - parentStart) / parentDuration * 100); | ||
| 1253 | + const widthPct = Math.max(1.5, child.duration / parentDuration * 100); | ||
| 1254 | + const rightPct = Math.min(100, leftPct + widthPct); | ||
| 1255 | + const actualWidth = rightPct - leftPct; | ||
| 1256 | + const centerPct = leftPct + actualWidth / 2; | ||
| 1257 | + const isDominant = child.dominant_stream === stream; | ||
| 1258 | + const opacity = isDominant ? '1' : '0.4'; | ||
| 1259 | + const borderStyle = isDominant ? '' : ';border:1px dashed ' + child.color; | ||
| 1260 | + | ||
| 1261 | + html += '<div class="gantt-bar' + (isDominant ? '' : ' gantt-bar-secondary') + | ||
| 1262 | + '" style="left:' + leftPct.toFixed(1) + '%;width:' + actualWidth.toFixed(1) + | ||
| 1263 | + '%;background:' + child.color + ';opacity:' + opacity + borderStyle + | ||
| 1264 | + '" data-node-index="' + child.node_index + '" data-center="' + centerPct.toFixed(1) + | ||
| 1265 | + '" data-color="' + child.color + '">' + | ||
| 1266 | + (isDominant ? '<div class="gantt-bar-arrow" style="border-top-color:' + | ||
| 1267 | + child.color + '"></div>' : '') + | ||
| 1268 | + '</div>'; | ||
| 1269 | + }); | ||
| 1270 | + | ||
| 1271 | + html += '</div></div>'; | ||
| 1272 | + | ||
| 1273 | + html += '<div class="gantt-labels-row">'; | ||
| 1274 | + barsOnStream.forEach(child => { | ||
| 1275 | + const leftPct = Math.max(0, (child.start - parentStart) / parentDuration * 100); | ||
| 1276 | + const widthPct = Math.max(1.5, child.duration / parentDuration * 100); | ||
| 1277 | + const centerPct = Math.min(98, Math.max(2, leftPct + widthPct / 2)); | ||
| 1278 | + const isDominant = child.dominant_stream === stream; | ||
| 1279 | + const shortName = child.name.length > 12 ? child.name.substring(0, 10) + '..' : child.name; | ||
| 1280 | + | ||
| 1281 | + if (!isDominant) return; | ||
| 1282 | + html += '<div class="gantt-label" style="left:' + centerPct.toFixed(1) + | ||
| 1283 | + '%;color:' + child.color + ';border-color:' + child.color + | ||
| 1284 | + '66" data-center="' + centerPct.toFixed(1) + '">' + | ||
| 1285 | + shortName + '</div>'; | ||
| 1286 | + }); | ||
| 1287 | + html += '</div></div>'; | ||
| 1288 | + }); | ||
| 1289 | + | ||
| 1290 | + const timeMarkers = [0, 0.25, 0.5, 0.75, 1].map(r => formatUs(parentStart + parentDuration * r)); | ||
| 1291 | + html += '<div class="gantt-time-axis">' + | ||
| 1292 | + '<span>' + timeMarkers[0] + '</span>' + | ||
| 1293 | + '<span>' + timeMarkers[1] + '</span>' + | ||
| 1294 | + '<span>' + timeMarkers[2] + '</span>' + | ||
| 1295 | + '<span>' + timeMarkers[3] + '</span>' + | ||
| 1296 | + '<span>' + timeMarkers[4] + '</span></div>'; | ||
| 1297 | + | ||
| 1298 | + container.innerHTML = html; | ||
| 1299 | + | ||
| 1300 | + layoutLabels(container); | ||
| 1301 | + | ||
| 1302 | + container.querySelectorAll('.gantt-bar').forEach(bar => { | ||
| 1303 | + const ni = parseInt(bar.dataset.nodeIndex); | ||
| 1304 | + const info = tlData.bar_data ? tlData.bar_data[ni] : null; | ||
| 1305 | + if (!info) return; | ||
| 1306 | + bar.addEventListener('mouseenter', () => showTimelineTooltip(bar, info)); | ||
| 1307 | + bar.addEventListener('mouseleave', () => setTimeout(() => { | ||
| 1308 | + if (!tlTooltip || !tlTooltip.matches(':hover')) hideTimelineTooltip(); }, 150)); | ||
| 1309 | + bar.addEventListener('click', (e) => { e.stopPropagation(); hideTimelineTooltip(); showOpsDetail(ni); }); | ||
| 1310 | + }); | ||
| 1311 | + } | ||
| 1312 | + | ||
| 1313 | + function layoutLabels(container) { | ||
| 1314 | + container.querySelectorAll('.gantt-labels-row').forEach(row => { | ||
| 1315 | + const labels = Array.from(row.querySelectorAll('.gantt-label')); | ||
| 1316 | + if (labels.length === 0) return; | ||
| 1317 | + | ||
| 1318 | + labels.sort((a, b) => parseFloat(a.dataset.center) - parseFloat(b.dataset.center)); | ||
| 1319 | + | ||
| 1320 | + const minGap = 10; | ||
| 1321 | + for (let i = 1; i < labels.length; i++) { | ||
| 1322 | + const prev = labels[i - 1]; | ||
| 1323 | + const curr = labels[i]; | ||
| 1324 | + const prevLeft = parseFloat(prev.style.left); | ||
| 1325 | + let currLeft = parseFloat(curr.style.left); | ||
| 1326 | + | ||
| 1327 | + const prevWidth = prev.offsetWidth; | ||
| 1328 | + const currWidth = curr.offsetWidth; | ||
| 1329 | + const threshold = (prevWidth + currWidth) / 2 / row.offsetWidth * 100 + minGap; | ||
| 1330 | + | ||
| 1331 | + if (currLeft - prevLeft < threshold) { | ||
| 1332 | + currLeft = prevLeft + threshold; | ||
| 1333 | + if (currLeft > 98) currLeft = 98; | ||
| 1334 | + curr.style.left = currLeft.toFixed(1) + '%'; | ||
| 1335 | + } | ||
| 1336 | + } | ||
| 1337 | + }); | ||
| 1338 | + } | ||
| 1339 | + | ||
| 1340 | + function renderTree(parentNode, children) { | ||
| 1341 | + const container = document.getElementById('tree-content'); | ||
| 1342 | + if (!container || !children || children.length === 0) { | ||
| 1343 | + container.innerHTML = '<div style="padding:10px;color:var(--text-secondary)">无子节点</div>'; | ||
| 1344 | + return; | ||
| 1345 | + } | ||
| 1346 | + | ||
| 1347 | + const baseDepth = parentNode.depth; | ||
| 1348 | + let html = ''; | ||
| 1349 | + | ||
| 1350 | + function renderNode(node, depth) { | ||
| 1351 | + const indent = (depth - baseDepth) * 20; | ||
| 1352 | + const hasChildren = node.has_children; | ||
| 1353 | + const icon = hasChildren ? '▶' : '─'; | ||
| 1354 | + const clickableClass = hasChildren ? 'clickable' : ''; | ||
| 1355 | + const streamsStr = node.streams.slice(0, 3).join(',') + (node.streams.length > 3 ? '..' : ''); | ||
| 1356 | + | ||
| 1357 | + html += '<div class="tree-item ' + clickableClass + '" data-node-index="' + | ||
| 1358 | + node.node_index + '" style="padding-left:' + indent + 'px">'; | ||
| 1359 | + html += '<span class="tree-icon">' + icon + '</span>'; | ||
| 1360 | + html += '<span class="tree-name">' + node.name + '</span>'; | ||
| 1361 | + html += '<span class="tree-streams">[' + streamsStr + ']</span>'; | ||
| 1362 | + html += '<span class="tree-duration">' + formatUs(node.duration) + '</span>'; | ||
| 1363 | + html += '</div>'; | ||
| 1364 | + } | ||
| 1365 | + | ||
| 1366 | + children.forEach(child => renderNode(child, child.depth)); | ||
| 1367 | + container.innerHTML = html; | ||
| 1368 | + | ||
| 1369 | + container.querySelectorAll('.tree-item.clickable').forEach(item => { | ||
| 1370 | + const ni = parseInt(item.dataset.nodeIndex); | ||
| 1371 | + item.addEventListener('click', (e) => { | ||
| 1372 | + e.stopPropagation(); | ||
| 1373 | + showNodeChildren(ni); | ||
| 1374 | + }); | ||
| 1375 | + item.addEventListener('mouseenter', () => { | ||
| 1376 | + const info = tlData.bar_data ? tlData.bar_data[ni] : null; | ||
| 1377 | + if (info) showTimelineTooltip(item, info); | ||
| 1378 | + }); | ||
| 1379 | + item.addEventListener('mouseleave', () => setTimeout(() => { | ||
| 1380 | + if (!tlTooltip || !tlTooltip.matches(':hover')) hideTimelineTooltip(); }, 150)); | ||
| 1381 | + }); | ||
| 1382 | + | ||
| 1383 | + container.querySelectorAll('.tree-item:not(.clickable)').forEach(item => { | ||
| 1384 | + const ni = parseInt(item.dataset.nodeIndex); | ||
| 1385 | + item.addEventListener('click', (e) => { | ||
| 1386 | + e.stopPropagation(); | ||
| 1387 | + showOpsDetail(ni); | ||
| 1388 | + }); | ||
| 1389 | + }); | ||
| 1390 | + } | ||
| 1391 | + | ||
| 1392 | + function showNodeChildren(nodeIndex, addToHistory = true) { | ||
| 1393 | + if (!tlData.bar_data || !tlData.bar_data[nodeIndex]) return; | ||
| 1394 | + const node = tlData.bar_data[nodeIndex]; | ||
| 1395 | + const childrenIndices = node.children_indices || []; | ||
| 1396 | + | ||
| 1397 | + if (childrenIndices.length === 0) { | ||
| 1398 | + showOpsDetail(nodeIndex); | ||
| 1399 | + return; | ||
| 1400 | + } | ||
| 1401 | + | ||
| 1402 | + if (addToHistory && currentExpandedNode !== null && currentExpandedNode !== nodeIndex) { | ||
| 1403 | + navHistory.push(currentExpandedNode); | ||
| 1404 | + } | ||
| 1405 | + | ||
| 1406 | + const children = childrenIndices.map(i => tlData.bar_data[i]).filter(c => c); | ||
| 1407 | + | ||
| 1408 | + const expandArea = document.getElementById('timeline-expand-area'); | ||
| 1409 | + const title = document.getElementById('expand-title'); | ||
| 1410 | + | ||
| 1411 | + const countSuffix = node.multiplier > 1 ? ' (×' + node.multiplier + ')' : ''; | ||
| 1412 | + title.textContent = node.name + countSuffix + ' - 子节点多流时序'; | ||
| 1413 | + | ||
| 1414 | + renderGantt(node, children); | ||
| 1415 | + renderTree(node, children); | ||
| 1416 | + renderBreadcrumb(nodeIndex); | ||
| 1417 | + | ||
| 1418 | + currentExpandedNode = nodeIndex; | ||
| 1419 | + expandArea.style.display = 'block'; | ||
| 1420 | + expandArea.scrollIntoView({ behavior: 'smooth', block: 'start' }); | ||
| 1421 | + } | ||
| 1422 | + | ||
| 1423 | + function renderBreadcrumb(currentNodeIndex) { | ||
| 1424 | + const bcContainer = document.getElementById('expand-breadcrumb'); | ||
| 1425 | + if (!bcContainer) return; | ||
| 1426 | + | ||
| 1427 | + const pathIndices = []; | ||
| 1428 | + let idx = currentNodeIndex; | ||
| 1429 | + while (idx >= 0 && tlData.bar_data[idx]) { | ||
| 1430 | + pathIndices.unshift(idx); | ||
| 1431 | + idx = tlData.bar_data[idx].parent_index; | ||
| 1432 | + } | ||
| 1433 | + | ||
| 1434 | + let html = ''; | ||
| 1435 | + pathIndices.forEach((ni, i) => { | ||
| 1436 | + const info = tlData.bar_data[ni]; | ||
| 1437 | + const isCurrent = (ni === currentNodeIndex); | ||
| 1438 | + const countSuffix = info.multiplier > 1 ? ' ×' + info.multiplier : ''; | ||
| 1439 | + const label = info.name + countSuffix; | ||
| 1440 | + | ||
| 1441 | + if (i > 0) { | ||
| 1442 | + html += '<span class="breadcrumb-sep">›</span>'; | ||
| 1443 | + } | ||
| 1444 | + html += '<span class="breadcrumb-item' + (isCurrent ? ' current' : '') + | ||
| 1445 | + '" data-node-index="' + ni + '">' + label + '</span>'; | ||
| 1446 | + }); | ||
| 1447 | + | ||
| 1448 | + bcContainer.innerHTML = html; | ||
| 1449 | + | ||
| 1450 | + bcContainer.querySelectorAll('.breadcrumb-item:not(.current)').forEach(item => { | ||
| 1451 | + const ni = parseInt(item.dataset.nodeIndex); | ||
| 1452 | + item.addEventListener('click', (e) => { | ||
| 1453 | + e.stopPropagation(); | ||
| 1454 | + const targetIdx = navHistory.indexOf(ni); | ||
| 1455 | + if (targetIdx >= 0) { | ||
| 1456 | + navHistory = navHistory.slice(0, targetIdx); | ||
| 1457 | + } | ||
| 1458 | + showNodeChildren(ni, false); | ||
| 1459 | + }); | ||
| 1460 | + }); | ||
| 1461 | + } | ||
| 1462 | + | ||
| 1463 | + function goBackToParent() { | ||
| 1464 | + if (navHistory.length === 0) { | ||
| 1465 | + hideExpandArea(); | ||
| 1466 | + return; | ||
| 1467 | + } | ||
| 1468 | + const prevNodeIndex = navHistory.pop(); | ||
| 1469 | + showNodeChildren(prevNodeIndex, false); | ||
| 1470 | + } | ||
| 1471 | + | ||
| 1472 | + function hideExpandArea() { | ||
| 1473 | + const expandArea = document.getElementById('timeline-expand-area'); | ||
| 1474 | + if (expandArea) { | ||
| 1475 | + expandArea.style.display = 'none'; | ||
| 1476 | + currentExpandedNode = null; | ||
| 1477 | + navHistory = []; | ||
| 1478 | + } | ||
| 1479 | + } | ||
| 1480 | + | ||
| 1481 | + document.addEventListener('DOMContentLoaded', function() { | ||
| 1482 | + let kernelsVisible = true; | ||
| 1483 | + const kernelBtn = document.getElementById('toggle-kernels-btn'); | ||
| 1484 | + kernelBtn.addEventListener('click', () => { | ||
| 1485 | + kernelsVisible = !kernelsVisible; | ||
| 1486 | + toggleKernels(kernelsVisible); | ||
| 1487 | + kernelBtn.textContent = kernelsVisible ? | ||
| 1488 | + '\u9690\u85cfKernel\u5e8f\u5217' : '\u663e\u793aKernel\u5e8f\u5217'; | ||
| 1489 | + }); | ||
| 1490 | + | ||
| 1491 | + let kernelMetaVisible = true; | ||
| 1492 | + const kernelMetaBtn = document.getElementById('toggle-kernel-meta-btn'); | ||
| 1493 | + kernelMetaBtn.addEventListener('click', () => { | ||
| 1494 | + kernelMetaVisible = !kernelMetaVisible; | ||
| 1495 | + document.querySelectorAll('.kernel-meta').forEach(el => { | ||
| 1496 | + el.style.display = kernelMetaVisible ? 'grid' : 'none'; | ||
| 1497 | + }); | ||
| 1498 | + kernelMetaBtn.textContent = kernelMetaVisible ? | ||
| 1499 | + '\u9690\u85cfKernel\u4fe1\u606f' : '\u663e\u793aKernel\u4fe1\u606f'; | ||
| 1500 | + }); | ||
| 1501 | + | ||
| 1502 | + let allExpanded = false; | ||
| 1503 | + const expandBtn = document.getElementById('toggle-expand-btn'); | ||
| 1504 | + const depthSelect = document.getElementById('depth-select'); | ||
| 1505 | + function applyDepthExpansion(maxDepth) { | ||
| 1506 | + document.querySelectorAll('.tree-node').forEach(node => { | ||
| 1507 | + const depth = parseInt(node.dataset.depth); | ||
| 1508 | + if (!node.classList.contains('leaf') && !node.classList.contains('kernel-only')) { | ||
| 1509 | + if (depth < maxDepth) { | ||
| 1510 | + node.classList.remove('collapsed'); | ||
| 1511 | + node.classList.add('expanded'); | ||
| 1512 | + } else { | ||
| 1513 | + node.classList.remove('expanded'); | ||
| 1514 | + node.classList.add('collapsed'); | ||
| 1515 | + } | ||
| 1516 | + } | ||
| 1517 | + }); | ||
| 1518 | + } | ||
| 1519 | + expandBtn.addEventListener('click', () => { | ||
| 1520 | + allExpanded = !allExpanded; | ||
| 1521 | + if (allExpanded) { | ||
| 1522 | + applyDepthExpansion(parseInt(depthSelect.value)); | ||
| 1523 | + expandBtn.textContent = '\u5168\u90e8\u6536\u8d77'; | ||
| 1524 | + } else { | ||
| 1525 | + document.querySelectorAll('.tree-node:not(.leaf)').forEach(node => { | ||
| 1526 | + node.classList.remove('expanded'); | ||
| 1527 | + node.classList.add('collapsed'); | ||
| 1528 | + }); | ||
| 1529 | + expandBtn.textContent = '\u5168\u90e8\u5c55\u5f00'; | ||
| 1530 | + } | ||
| 1531 | + }); | ||
| 1532 | + | ||
| 1533 | + depthSelect.addEventListener('change', () => { | ||
| 1534 | + applyDepthExpansion(parseInt(depthSelect.value)); | ||
| 1535 | + if (allExpanded) { | ||
| 1536 | + expandBtn.textContent = '\u5168\u90e8\u6536\u8d77'; | ||
| 1537 | + } | ||
| 1538 | + }); | ||
| 1539 | + | ||
| 1540 | + let auxiliaryVisible = false; | ||
| 1541 | + const auxiliaryBtn = document.getElementById('toggle-auxiliary-btn'); | ||
| 1542 | + document.querySelectorAll('.tree-node[data-category="auxiliary"]').forEach(node => { | ||
| 1543 | + node.style.display = 'none'; | ||
| 1544 | + }); | ||
| 1545 | + document.querySelectorAll('.timeline-node-item[data-category="auxiliary"]').forEach(node => { | ||
| 1546 | + node.style.display = 'none'; | ||
| 1547 | + }); | ||
| 1548 | + auxiliaryBtn.addEventListener('click', () => { | ||
| 1549 | + auxiliaryVisible = !auxiliaryVisible; | ||
| 1550 | + document.querySelectorAll('.tree-node[data-category="auxiliary"]').forEach(node => { | ||
| 1551 | + node.style.display = auxiliaryVisible ? '' : 'none'; | ||
| 1552 | + }); | ||
| 1553 | + document.querySelectorAll('.timeline-node-item[data-category="auxiliary"]').forEach(node => { | ||
| 1554 | + node.style.display = auxiliaryVisible ? '' : 'none'; | ||
| 1555 | + }); | ||
| 1556 | + auxiliaryBtn.textContent = auxiliaryVisible ? | ||
| 1557 | + '\u9690\u85cf\u8f85\u52a9\u5c42\u6b21' : '\u663e\u793a\u8f85\u52a9\u5c42\u6b21'; | ||
| 1558 | + }); | ||
| 1559 | + | ||
| 1560 | + const fieldsPanel = document.getElementById('kernel-fields-panel'); | ||
| 1561 | + const fieldsBtn = document.getElementById('kernel-fields-btn'); | ||
| 1562 | + | ||
| 1563 | + fieldsPanel.querySelectorAll('input[type="checkbox"]').forEach(cb => { | ||
| 1564 | + cb.checked = defaultFields.includes(cb.value); | ||
| 1565 | + }); | ||
| 1566 | + | ||
| 1567 | + fieldsBtn.addEventListener('click', (e) => { | ||
| 1568 | + e.stopPropagation(); | ||
| 1569 | + fieldsPanel.classList.toggle('visible'); | ||
| 1570 | + }); | ||
| 1571 | + | ||
| 1572 | + document.addEventListener('click', (e) => { | ||
| 1573 | + if (!e.target.closest('.kernel-fields-config')) { | ||
| 1574 | + fieldsPanel.classList.remove('visible'); | ||
| 1575 | + } | ||
| 1576 | + }); | ||
| 1577 | + | ||
| 1578 | + fieldsPanel.querySelectorAll('input[type="checkbox"]').forEach(cb => { | ||
| 1579 | + cb.addEventListener('change', updateKernelMetaFields); | ||
| 1580 | + }); | ||
| 1581 | + | ||
| 1582 | + document.getElementById('fields-select-all').addEventListener('click', (e) => { | ||
| 1583 | + e.stopPropagation(); | ||
| 1584 | + fieldsPanel.querySelectorAll('input[type="checkbox"]').forEach(cb => cb.checked = true); | ||
| 1585 | + updateKernelMetaFields(); | ||
| 1586 | + }); | ||
| 1587 | + | ||
| 1588 | + document.getElementById('fields-select-none').addEventListener('click', (e) => { | ||
| 1589 | + e.stopPropagation(); | ||
| 1590 | + fieldsPanel.querySelectorAll('input[type="checkbox"]').forEach(cb => cb.checked = false); | ||
| 1591 | + updateKernelMetaFields(); | ||
| 1592 | + }); | ||
| 1593 | + | ||
| 1594 | + document.getElementById('fields-reset').addEventListener('click', (e) => { | ||
| 1595 | + e.stopPropagation(); | ||
| 1596 | + fieldsPanel.querySelectorAll('input[type="checkbox"]').forEach(cb => { | ||
| 1597 | + cb.checked = defaultFields.includes(cb.value); | ||
| 1598 | + }); | ||
| 1599 | + updateKernelMetaFields(); | ||
| 1600 | + }); | ||
| 1601 | + | ||
| 1602 | + let semanticAllExpanded = false; | ||
| 1603 | + const semanticBtn = document.getElementById('toggle-semantic-btn'); | ||
| 1604 | + | ||
| 1605 | + semanticBtn.addEventListener('click', () => { | ||
| 1606 | + semanticAllExpanded = !semanticAllExpanded; | ||
| 1607 | + document.querySelectorAll('.node-semantic-wrapper').forEach(w => { | ||
| 1608 | + if (semanticAllExpanded) { | ||
| 1609 | + w.classList.add('expanded'); | ||
| 1610 | + } else { | ||
| 1611 | + const full = w.dataset.full || ''; | ||
| 1612 | + if (full.length > 120) { | ||
| 1613 | + w.classList.remove('expanded'); | ||
| 1614 | + } | ||
| 1615 | + } | ||
| 1616 | + }); | ||
| 1617 | + semanticBtn.textContent = semanticAllExpanded ? | ||
| 1618 | + '\u6536\u8d77\u5168\u90e8\u8bed\u4e49' : '\u5c55\u5f00\u5168\u90e8\u8bed\u4e49'; | ||
| 1619 | + }); | ||
| 1620 | + | ||
| 1621 | + document.getElementById('theme-select').addEventListener('change', function() { | ||
| 1622 | + applyTheme(this.value); | ||
| 1623 | + }); | ||
| 1624 | + | ||
| 1625 | + document.addEventListener('click', function(e) { | ||
| 1626 | + const btn = e.target.closest('.semantic-expand-btn'); | ||
| 1627 | + if (!btn) return; | ||
| 1628 | + e.stopPropagation(); | ||
| 1629 | + const wrapper = btn.closest('.node-semantic-wrapper'); | ||
| 1630 | + if (wrapper) { | ||
| 1631 | + wrapper.classList.toggle('expanded'); | ||
| 1632 | + } | ||
| 1633 | + }); | ||
| 1634 | + | ||
| 1635 | + document.querySelectorAll('.tree-node:not(.leaf)').forEach(node => { | ||
| 1636 | + const header = node.querySelector('.node-header'); | ||
| 1637 | + header.addEventListener('click', (e) => { | ||
| 1638 | + if (e.target.closest('[data-type="kernel"]')) return; | ||
| 1639 | + if (e.target.closest('.semantic-expand-btn')) return; | ||
| 1640 | + if (e.target.closest('.node-semantic-wrapper')) return; | ||
| 1641 | + if (e.target.closest('.kernel-info-btn')) return; | ||
| 1642 | + node.classList.toggle('collapsed'); | ||
| 1643 | + node.classList.toggle('expanded'); | ||
| 1644 | + }); | ||
| 1645 | + }); | ||
| 1646 | + | ||
| 1647 | + document.addEventListener('click', (e) => { | ||
| 1648 | + if (!e.target.closest('.tree-node[data-type="kernel"]') && | ||
| 1649 | + !e.target.closest('.kernel-tooltip') && | ||
| 1650 | + !e.target.closest('.kernel-info-btn')) { | ||
| 1651 | + hideTooltip(); | ||
| 1652 | + } | ||
| 1653 | + }); | ||
| 1654 | + | ||
| 1655 | + document.querySelectorAll('.kernel-info-btn').forEach(btn => { | ||
| 1656 | + btn.addEventListener('click', (e) => { | ||
| 1657 | + e.stopPropagation(); | ||
| 1658 | + showTooltip(btn); | ||
| 1659 | + }); | ||
| 1660 | + }); | ||
| 1661 | + | ||
| 1662 | + if (currentTooltip) { | ||
| 1663 | + currentTooltip.addEventListener('mouseleave', hideTooltip); | ||
| 1664 | + } | ||
| 1665 | + | ||
| 1666 | + initTheme(); | ||
| 1667 | + | ||
| 1668 | +/* ========== Timeline Event Listeners ========== */ | ||
| 1669 | + document.querySelectorAll('.timeline-node-item').forEach(item => { | ||
| 1670 | + const ni = parseInt(item.dataset.nodeIndex); | ||
| 1671 | + const info = tlData.bar_data ? tlData.bar_data[ni] : null; | ||
| 1672 | + if (!info) return; | ||
| 1673 | + | ||
| 1674 | + item.addEventListener('mouseenter', () => showTimelineTooltip(item, info)); | ||
| 1675 | + item.addEventListener('mouseleave', () => setTimeout(() => { | ||
| 1676 | + if (!tlTooltip || !tlTooltip.matches(':hover')) hideTimelineTooltip(); }, 150)); | ||
| 1677 | + item.addEventListener('click', (e) => { | ||
| 1678 | + e.stopPropagation(); | ||
| 1679 | + hideTimelineTooltip(); | ||
| 1680 | + if (info.has_children) { | ||
| 1681 | + showNodeChildren(ni); | ||
| 1682 | + } else { | ||
| 1683 | + showOpsDetail(ni); | ||
| 1684 | + } | ||
| 1685 | + }); | ||
| 1686 | + }); | ||
| 1687 | + | ||
| 1688 | + if (tlTooltip) { | ||
| 1689 | + tlTooltip.addEventListener('mouseleave', hideTimelineTooltip); | ||
| 1690 | + } | ||
| 1691 | + | ||
| 1692 | + const expandCloseBtn = document.getElementById('expand-close-btn'); | ||
| 1693 | + if (expandCloseBtn) { | ||
| 1694 | + expandCloseBtn.addEventListener('click', hideExpandArea); | ||
| 1695 | + } | ||
| 1696 | + | ||
| 1697 | + const detailCloseBtn = document.getElementById('timeline-detail-close'); | ||
| 1698 | + if (detailCloseBtn) { | ||
| 1699 | + detailCloseBtn.addEventListener('click', () => { | ||
| 1700 | + document.getElementById('timeline-detail-panel').classList.remove('visible'); | ||
| 1701 | + }); | ||
| 1702 | + } | ||
| 1703 | + | ||
| 1704 | + document.addEventListener('click', (e) => { | ||
| 1705 | + if (!e.target.closest('.timeline-node-item') && !e.target.closest('.timeline-tooltip') && | ||
| 1706 | + !e.target.closest('.timeline-expand-area') && !e.target.closest('.timeline-detail-panel')) { | ||
| 1707 | + hideTimelineTooltip(); | ||
| 1708 | + } | ||
| 1709 | + }); | ||
| 1710 | + }); | ||
| 1711 | + </script> | ||
| 1712 | + """ | ||
| @@ -0,0 +1,59 @@ | |||
| 1 | +#!/usr/bin/env python3 | ||
| 2 | +# -*- coding: utf-8 -*- | ||
| 3 | +# ---------------------------------------------------------------------------- | ||
| 4 | +# Copyright (c) 2025 Huawei Technologies Co., Ltd. | ||
| 5 | +# | ||
| 6 | +# Licensed under the Apache License, Version 2.0 (the "License"); | ||
| 7 | +# you may not use this file except in compliance with the License. | ||
| 8 | +# You may obtain a copy of the License at | ||
| 9 | +# | ||
| 10 | +# http://www.apache.org/licenses/LICENSE-2.0 | ||
| 11 | +# | ||
| 12 | +# Unless required by applicable law or agreed to in writing, software | ||
| 13 | +# distributed under the License is distributed on an "AS IS" BASIS, | ||
| 14 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| 15 | +# See the License for the specific language governing permissions and | ||
| 16 | +# limitations under the License. | ||
| 17 | +# ---------------------------------------------------------------------------- | ||
| 18 | +"""perf-breakdown 脚本共享工具函数。 | ||
| 19 | + | ||
| 20 | +各脚本以 `python scripts/<name>.py` 形式从 skill 根目录调用,scripts 目录位于 | ||
| 21 | +sys.path[0],故可直接 `from _common import ...`。 | ||
| 22 | +""" | ||
| 23 | +import json | ||
| 24 | +from pathlib import Path | ||
| 25 | + | ||
| 26 | + | ||
| 27 | +def validate_file_exists(filepath: str) -> Path: | ||
| 28 | + path = Path(filepath) | ||
| 29 | + if not path.exists(): | ||
| 30 | + raise FileNotFoundError(f"文件不存在: {filepath}") | ||
| 31 | + return path | ||
| 32 | + | ||
| 33 | + | ||
| 34 | +def load_json(filepath: Path) -> dict: | ||
| 35 | + try: | ||
| 36 | + with open(filepath, 'r', encoding='utf-8') as f: | ||
| 37 | + return json.load(f) | ||
| 38 | + except json.JSONDecodeError as e: | ||
| 39 | + raise ValueError(f"JSON 格式错误: {filepath}: {e}") from e | ||
| 40 | + | ||
| 41 | + | ||
| 42 | +# 始终必填 shape_semantic 的算子(被 check_op_coverage / check_structure / | ||
| 43 | +# regression_check 共用,避免三处重复定义)。 | ||
| 44 | +SHAPE_SEMANTIC_ALWAYS_REQUIRED = { | ||
| 45 | + 'MatMul', 'MatMulV2', 'QuantBatchMatmulV3', 'GroupedMatmul', 'GemmEx', 'BatchMatMul', | ||
| 46 | + 'FlashAttentionScore', 'FusedInferAttentionScore', 'KvQuantSparseFlashAttention', | ||
| 47 | + 'HcomAllGather', 'HcomReduceScatter', 'HcomAllToAll', 'hcom_allReduce', 'HcomAllReduce', | ||
| 48 | + 'RmsNorm', 'LayerNormV3', 'InplaceAddRmsNorm', 'AddRmsNormDynamicQuant', | ||
| 49 | + 'MlaPrologV3', 'DequantSwigluQuant', 'LightningIndexerQuant', 'MoeGatingTopKHash', | ||
| 50 | + 'RotaryMul', | ||
| 51 | + 'GatherV2', 'GatherV3', | ||
| 52 | + 'MoeDistributeDispatchV2', 'MoeDistributeCombineV2', | ||
| 53 | +} | ||
| 54 | + | ||
| 55 | + | ||
| 56 | +def is_shape_always_required(name: str) -> bool: | ||
| 57 | + """算子是否始终必填 shape_semantic(含 AddRmsNorm 前缀系列)。""" | ||
| 58 | + return name in SHAPE_SEMANTIC_ALWAYS_REQUIRED or name.startswith('AddRmsNorm') | ||
| 59 | + | ||
| @@ -0,0 +1,782 @@ | |||
| 1 | +#!/usr/bin/env python3 | ||
| 2 | +# -*- coding: utf-8 -*- | ||
| 3 | +# ---------------------------------------------------------------------------- | ||
| 4 | +# Copyright (c) 2025 Huawei Technologies Co., Ltd. | ||
| 5 | +# | ||
| 6 | +# Licensed under the Apache License, Version 2.0 (the "License"); | ||
| 7 | +# you may not use this file except in compliance with the License. | ||
| 8 | +# You may obtain a copy of the License at | ||
| 9 | +# | ||
| 10 | +# http://www.apache.org/licenses/LICENSE-2.0 | ||
| 11 | +# | ||
| 12 | +# Unless required by applicable law or agreed to in writing, software | ||
| 13 | +# distributed under the License is distributed on an "AS IS" BASIS, | ||
| 14 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| 15 | +# See the License for the specific language governing permissions and | ||
| 16 | +# limitations under the License. | ||
| 17 | +# ---------------------------------------------------------------------------- | ||
| 18 | +""" | ||
| 19 | +分析 kernel_details.csv,输出结构化 JSON 统计信息 | ||
| 20 | + | ||
| 21 | +用法: | ||
| 22 | + python analyze_kernels.py [-f FILE] [-s STEP] [-o OUTPUT] [-d DETAILS] [-m MD] | ||
| 23 | + | ||
| 24 | +选项: | ||
| 25 | + -f, --file FILE 指定 CSV 文件路径 (默认: kernel_details.csv) | ||
| 26 | + -s, --step STEP 指定要输出详情的 step ID (默认: 自动选择非 warmup 代表 step) | ||
| 27 | + -o, --output FILE 输出 operators JSON 文件路径 (默认: kernels.json) | ||
| 28 | + -d, --details FILE 输出详细 operators JSON 文件路径 (包含 CSV 全部字段) | ||
| 29 | + -m, --markdown FILE 输出统计摘要 Markdown 文件路径 (不指定则不生成) | ||
| 30 | + -h, --help 显示帮助信息 | ||
| 31 | +""" | ||
| 32 | +import csv | ||
| 33 | +import json | ||
| 34 | +import sys | ||
| 35 | +import logging | ||
| 36 | +import argparse | ||
| 37 | +import os | ||
| 38 | +from collections import defaultdict | ||
| 39 | + | ||
| 40 | +logger = logging.getLogger(__name__) | ||
| 41 | + | ||
| 42 | +REQUIRED_COLUMNS = ['Step Id', 'Name', 'Duration(us)', 'Start Time(us)', 'Stream ID'] | ||
| 43 | + | ||
| 44 | +IGNORED_COLUMNS = {'Step Id'} | ||
| 45 | + | ||
| 46 | +METRIC_COLUMN_PATTERNS = [ | ||
| 47 | + 'duration', 'time', 'cycles', 'ratio', 'utilization', | ||
| 48 | + 'fops', 'rate', 'miss', 'count', 'num', 'dim', 'id' | ||
| 49 | +] | ||
| 50 | + | ||
| 51 | +VALUE_COLUMN_PATTERNS = [ | ||
| 52 | + 'name', 'type', 'state', 'core', 'shapes', 'formats', | ||
| 53 | + 'eligible', 'formats', 'context' | ||
| 54 | +] | ||
| 55 | + | ||
| 56 | +FLOAT_SUFFIXES = ['(us)', '(%)'] | ||
| 57 | + | ||
| 58 | + | ||
| 59 | +def is_metric_column(col_name): | ||
| 60 | + col_lower = col_name.lower() | ||
| 61 | + for pattern in METRIC_COLUMN_PATTERNS: | ||
| 62 | + if pattern in col_lower: | ||
| 63 | + return True | ||
| 64 | + return False | ||
| 65 | + | ||
| 66 | + | ||
| 67 | +def is_value_column(col_name): | ||
| 68 | + col_lower = col_name.lower() | ||
| 69 | + for pattern in VALUE_COLUMN_PATTERNS: | ||
| 70 | + if pattern in col_lower: | ||
| 71 | + return True | ||
| 72 | + return False | ||
| 73 | + | ||
| 74 | + | ||
| 75 | +def parse_column_value(value, col_name, json_key=None): | ||
| 76 | + value = value.strip().rstrip('\t') | ||
| 77 | + if not value or value == 'N/A': | ||
| 78 | + return None | ||
| 79 | + | ||
| 80 | + col_lower = col_name.lower() | ||
| 81 | + | ||
| 82 | + for suffix in FLOAT_SUFFIXES: | ||
| 83 | + if col_lower.endswith(suffix.lower()) or suffix.lower() in col_lower: | ||
| 84 | + try: | ||
| 85 | + return float(value) | ||
| 86 | + except (ValueError, AttributeError): | ||
| 87 | + return value | ||
| 88 | + | ||
| 89 | + if is_metric_column(col_name): | ||
| 90 | + try: | ||
| 91 | + if '.' in value: | ||
| 92 | + return float(value) | ||
| 93 | + return int(value) | ||
| 94 | + except (ValueError, AttributeError): | ||
| 95 | + return value | ||
| 96 | + | ||
| 97 | + try: | ||
| 98 | + if '.' in value: | ||
| 99 | + fval = float(value) | ||
| 100 | + return fval if fval != int(fval) else int(fval) | ||
| 101 | + return int(value) | ||
| 102 | + except (ValueError, AttributeError): | ||
| 103 | + pass | ||
| 104 | + | ||
| 105 | + return value | ||
| 106 | + | ||
| 107 | + | ||
| 108 | +def csv_col_name_to_json_key(col_name): | ||
| 109 | + key = col_name.lower() | ||
| 110 | + key = key.replace('(us)', '_us') | ||
| 111 | + key = key.replace('(%)', '_pct') | ||
| 112 | + key = key.replace(' ', '_') | ||
| 113 | + key = key.replace('(', '') | ||
| 114 | + key = key.replace(')', '') | ||
| 115 | + return key | ||
| 116 | + | ||
| 117 | + | ||
| 118 | +class ValidationError(Exception): | ||
| 119 | + pass | ||
| 120 | + | ||
| 121 | + | ||
| 122 | +class ConsistencyError(Exception): | ||
| 123 | + pass | ||
| 124 | + | ||
| 125 | + | ||
| 126 | +def _step_sort_key(step_id): | ||
| 127 | + return int(step_id) if str(step_id).isdigit() else str(step_id) | ||
| 128 | + | ||
| 129 | + | ||
| 130 | +def _kernel_signature(summary): | ||
| 131 | + return ( | ||
| 132 | + summary['kernel_count'], | ||
| 133 | + tuple(sorted(summary['kernel_types'].items())), | ||
| 134 | + ) | ||
| 135 | + | ||
| 136 | + | ||
| 137 | +def _median(values): | ||
| 138 | + """有序中位数。""" | ||
| 139 | + s = sorted(values) | ||
| 140 | + mid = len(s) // 2 | ||
| 141 | + return s[mid] if len(s) % 2 else (s[mid - 1] + s[mid]) / 2 | ||
| 142 | + | ||
| 143 | + | ||
| 144 | +def _should_skip_warmup(ordered): | ||
| 145 | + """判断最早步是否为 warmup/离群,返回 (skip, later_median)。""" | ||
| 146 | + later = ordered[1:] | ||
| 147 | + if not later: | ||
| 148 | + return False, None | ||
| 149 | + later_median = _median([x['total_duration_us'] for x in later]) | ||
| 150 | + if later_median > 0: | ||
| 151 | + ratio = ordered[0]['total_duration_us'] / later_median | ||
| 152 | + return (ratio > 1.5 or ratio < 0.67), later_median | ||
| 153 | + return False, later_median | ||
| 154 | + | ||
| 155 | + | ||
| 156 | +def _repr_step_reason(selected, ordered, skip_earliest, later_median): | ||
| 157 | + """构建代表步选择理由文本。""" | ||
| 158 | + if skip_earliest: | ||
| 159 | + return (f'auto-selected non-warmup step {selected["step_id"]}: ' | ||
| 160 | + f'largest stable kernel signature group size={len(ordered)}, ' | ||
| 161 | + f'skipped earliest warmup/outlier candidate={ordered[0]["step_id"]}, ' | ||
| 162 | + f'duration_us={selected["total_duration_us"]}, ' | ||
| 163 | + f'later_median_us={round(later_median, 1)}') | ||
| 164 | + return (f'auto-selected stable step {selected["step_id"]}: ' | ||
| 165 | + f'largest stable kernel signature group size={len(ordered)}, ' | ||
| 166 | + f'earliest candidate is not a duration outlier, ' | ||
| 167 | + f'duration_us={selected["total_duration_us"]}') | ||
| 168 | + | ||
| 169 | + | ||
| 170 | +def choose_representative_step(steps_summary): | ||
| 171 | + """选择稳定的非预热代表步(调用方未指定 -s 时)。 | ||
| 172 | + | ||
| 173 | + 策略:按 kernel_count + 类型分布分组,取最大组;若最早步 kernel_sum 明显 | ||
| 174 | + 离群则视为 warmup 跳过;在候选中选 duration 最接近中位数者,保持确定性。 | ||
| 175 | + """ | ||
| 176 | + if not steps_summary: | ||
| 177 | + return None, 'no steps available' | ||
| 178 | + if len(steps_summary) == 1: | ||
| 179 | + return steps_summary[0]['step_id'], 'single step available' | ||
| 180 | + | ||
| 181 | + groups = defaultdict(list) | ||
| 182 | + for item in steps_summary: | ||
| 183 | + groups[_kernel_signature(item)].append(item) | ||
| 184 | + | ||
| 185 | + def group_rank(items): | ||
| 186 | + first_key = _step_sort_key(items[0]['step_id']) | ||
| 187 | + first_rank = -first_key if isinstance(first_key, int) else 0 | ||
| 188 | + return (len(items), items[0]['kernel_count'], first_rank) | ||
| 189 | + | ||
| 190 | + selected_group = max(groups.values(), key=group_rank) | ||
| 191 | + ordered = sorted(selected_group, key=lambda x: _step_sort_key(x['step_id'])) | ||
| 192 | + skip_earliest, later_median = _should_skip_warmup(ordered) | ||
| 193 | + | ||
| 194 | + candidates = ordered[1:] if skip_earliest else [ordered[0]] | ||
| 195 | + median = _median([x['total_duration_us'] for x in candidates]) | ||
| 196 | + selected = min( | ||
| 197 | + candidates, | ||
| 198 | + key=lambda x: (abs(x['total_duration_us'] - median), _step_sort_key(x['step_id'])) | ||
| 199 | + ) | ||
| 200 | + reason = _repr_step_reason(selected, ordered, skip_earliest, later_median) | ||
| 201 | + return selected['step_id'], reason | ||
| 202 | + | ||
| 203 | + | ||
| 204 | +def validate_file(file_path): | ||
| 205 | + if not os.path.exists(file_path): | ||
| 206 | + raise ValidationError(f'错误: 文件不存在: {file_path}') | ||
| 207 | + if not os.path.isfile(file_path): | ||
| 208 | + raise ValidationError(f'错误: 路径不是文件: {file_path}') | ||
| 209 | + if not os.access(file_path, os.R_OK): | ||
| 210 | + raise ValidationError(f'错误: 文件不可读: {file_path}') | ||
| 211 | + | ||
| 212 | + | ||
| 213 | +def validate_csv_structure(fieldnames): | ||
| 214 | + missing = [col for col in REQUIRED_COLUMNS if col not in fieldnames] | ||
| 215 | + if missing: | ||
| 216 | + raise ValidationError( | ||
| 217 | + f'错误: CSV 文件缺少必需列\n' | ||
| 218 | + f'缺少的列: {", ".join(missing)}\n' | ||
| 219 | + f'当前的列: {", ".join(fieldnames)}' | ||
| 220 | + ) | ||
| 221 | + | ||
| 222 | + | ||
| 223 | +def get_safe_value(row, key, default=''): | ||
| 224 | + return row.get(key, default).strip().rstrip('\t') | ||
| 225 | + | ||
| 226 | + | ||
| 227 | +def parse_float(value): | ||
| 228 | + try: | ||
| 229 | + return float(value.strip().rstrip('\t')) | ||
| 230 | + except (ValueError, AttributeError): | ||
| 231 | + return 0.0 | ||
| 232 | + | ||
| 233 | + | ||
| 234 | +def extract_kernel_type(name): | ||
| 235 | + if not name: | ||
| 236 | + return 'Unknown' | ||
| 237 | + | ||
| 238 | + if '/' in name: | ||
| 239 | + return name | ||
| 240 | + | ||
| 241 | + if '_' not in name: | ||
| 242 | + return name | ||
| 243 | + | ||
| 244 | + parts = name.split('_') | ||
| 245 | + | ||
| 246 | + while parts and parts[-1].isdigit(): | ||
| 247 | + parts.pop() | ||
| 248 | + | ||
| 249 | + if not parts: | ||
| 250 | + return name | ||
| 251 | + | ||
| 252 | + last_part = parts[-1] | ||
| 253 | + if last_part and last_part[0].isupper() and not last_part.isdigit(): | ||
| 254 | + return last_part | ||
| 255 | + | ||
| 256 | + return '_'.join([p for p in parts if p]) | ||
| 257 | + | ||
| 258 | + | ||
| 259 | +def parse_args(): | ||
| 260 | + parser = argparse.ArgumentParser( | ||
| 261 | + description='分析 kernel_details.csv,输出结构化 JSON 统计信息', | ||
| 262 | + formatter_class=argparse.RawDescriptionHelpFormatter | ||
| 263 | + ) | ||
| 264 | + parser.add_argument('-f', '--file', default='kernel_details.csv', | ||
| 265 | + help='指定 CSV 文件路径 (默认: kernel_details.csv)') | ||
| 266 | + parser.add_argument('-s', '--step', type=int, default=None, | ||
| 267 | + help='指定要输出详情的 step ID (默认: 自动选择第一个 step)') | ||
| 268 | + parser.add_argument('-o', '--output', default='kernels.json', | ||
| 269 | + help='输出 operators JSON 文件路径 (默认: kernels.json)') | ||
| 270 | + parser.add_argument('-d', '--details', default=None, | ||
| 271 | + help='输出详细 operators JSON 文件路径 (包含 CSV 全部字段)') | ||
| 272 | + parser.add_argument('-m', '--markdown', default=None, | ||
| 273 | + help='输出统计摘要 Markdown 文件路径 (不指定则不生成)') | ||
| 274 | + parser.add_argument('--compact-out', default=None, dest='compact_out', | ||
| 275 | + help='输出 Step 2 投喂用的精简 JSON 路径 ' | ||
| 276 | + '(删除 start_time_us/duration_us,连续相同算子折叠)') | ||
| 277 | + parser.add_argument('--enrich', action='store_true', default=False, | ||
| 278 | + help='enrich 模式:将 raw_ops.json 数据内嵌到 analysis_config.json') | ||
| 279 | + parser.add_argument('-c', '--config', default=None, | ||
| 280 | + help='[enrich 模式] analysis_config.json 路径') | ||
| 281 | + parser.add_argument('-r', '--raw-ops', default=None, dest='raw_ops', | ||
| 282 | + help='[enrich 模式] raw_ops.json 路径') | ||
| 283 | + return parser.parse_args() | ||
| 284 | + | ||
| 285 | + | ||
| 286 | +def _raise_kernel_mismatch(first_types, first_step, s): | ||
| 287 | + """构建并抛出两个 step 间 kernel 分布不一致的错误。""" | ||
| 288 | + diff_kernels = [] | ||
| 289 | + all_keys = set(first_types.keys()) | set(s['kernel_types'].keys()) | ||
| 290 | + for k in sorted(all_keys): | ||
| 291 | + v1 = first_types.get(k, 0) | ||
| 292 | + v2 = s['kernel_types'].get(k, 0) | ||
| 293 | + if v1 != v2: | ||
| 294 | + diff_kernels.append(f' {k}: Step {first_step}={v1}, Step {s["step_id"]}={v2}') | ||
| 295 | + raise ConsistencyError( | ||
| 296 | + '错误: 各 Step 的 Kernel 分布不一致\n' | ||
| 297 | + '差异:\n' + '\n'.join(diff_kernels[:10]) + | ||
| 298 | + (f'\n ... 还有 {len(diff_kernels) - 10} 个差异' if len(diff_kernels) > 10 else '') | ||
| 299 | + ) | ||
| 300 | + | ||
| 301 | + | ||
| 302 | +def check_consistency(steps_summary): | ||
| 303 | + if len(steps_summary) <= 1: | ||
| 304 | + return | ||
| 305 | + | ||
| 306 | + first_types = None | ||
| 307 | + first_step = None | ||
| 308 | + | ||
| 309 | + for s in steps_summary: | ||
| 310 | + if first_types is None: | ||
| 311 | + first_types = s['kernel_types'] | ||
| 312 | + first_step = s['step_id'] | ||
| 313 | + elif s['kernel_types'] != first_types: | ||
| 314 | + _raise_kernel_mismatch(first_types, first_step, s) | ||
| 315 | + | ||
| 316 | + | ||
| 317 | +def _parse_kernel_row(row, row_num, all_columns, step_kernels_len): | ||
| 318 | + """解析单行 CSV,返回 (step_id, op_info, detail_info);step_id 为空返回 None。""" | ||
| 319 | + step_id = get_safe_value(row, 'Step Id') | ||
| 320 | + if not step_id: | ||
| 321 | + return None | ||
| 322 | + | ||
| 323 | + name = get_safe_value(row, 'Name') | ||
| 324 | + kernel_type = extract_kernel_type(name) | ||
| 325 | + duration = parse_float(get_safe_value(row, 'Duration(us)')) | ||
| 326 | + | ||
| 327 | + op_info = { | ||
| 328 | + 'index': step_kernels_len, | ||
| 329 | + 'org_index': row_num - 2, | ||
| 330 | + 'original_name': name, | ||
| 331 | + 'normalized_name': kernel_type, | ||
| 332 | + 'duration_us': duration, | ||
| 333 | + 'start_time_us': parse_float(get_safe_value(row, 'Start Time(us)')), | ||
| 334 | + 'stream_id': get_safe_value(row, 'Stream ID'), | ||
| 335 | + 'task_type': get_safe_value(row, 'Type') if 'Type' in row else '', | ||
| 336 | + 'input_shapes': get_safe_value(row, 'Input Shapes').strip('"') if 'Input Shapes' in row else '', | ||
| 337 | + 'output_shapes': (get_safe_value(row, 'Output Shapes').strip('"') | ||
| 338 | + if 'Output Shapes' in row else '') | ||
| 339 | + } | ||
| 340 | + | ||
| 341 | + detail_info = {'index': op_info['index'], 'org_index': op_info['org_index']} | ||
| 342 | + for col in all_columns: | ||
| 343 | + if col in IGNORED_COLUMNS: | ||
| 344 | + continue | ||
| 345 | + json_key = csv_col_name_to_json_key(col) | ||
| 346 | + raw_value = get_safe_value(row, col) | ||
| 347 | + if raw_value: | ||
| 348 | + detail_info[json_key] = parse_column_value(raw_value, col, json_key) | ||
| 349 | + if json_key in ('start_time_us', 'duration_us'): | ||
| 350 | + detail_info[f'{json_key}_raw'] = raw_value | ||
| 351 | + for key in ('input_shapes', 'output_shapes'): | ||
| 352 | + if key in detail_info and isinstance(detail_info[key], str): | ||
| 353 | + detail_info[key] = detail_info[key].strip('"') | ||
| 354 | + return step_id, op_info, detail_info, kernel_type, duration | ||
| 355 | + | ||
| 356 | + | ||
| 357 | +def _consume_csv_row(row, row_num, all_columns, steps_data): | ||
| 358 | + """解析并累加单行 CSV 到 steps_data;成功计入返回 True,跳过返回 False。""" | ||
| 359 | + try: | ||
| 360 | + step_id = get_safe_value(row, 'Step Id') | ||
| 361 | + if not step_id: | ||
| 362 | + return False | ||
| 363 | + parsed = _parse_kernel_row(row, row_num, all_columns, len(steps_data[step_id]['kernels'])) | ||
| 364 | + except Exception as e: | ||
| 365 | + logger.warning('警告: 第 %s 行数据解析失败: %s', row_num, e) | ||
| 366 | + return False | ||
| 367 | + if parsed is None: | ||
| 368 | + return False | ||
| 369 | + step_id, op_info, detail_info, kernel_type, duration = parsed | ||
| 370 | + sd = steps_data[step_id] | ||
| 371 | + sd['kernels'].append(op_info) | ||
| 372 | + sd['kernels_details'].append(detail_info) | ||
| 373 | + sd['total_duration'] += duration | ||
| 374 | + sd['kernel_types'][kernel_type] += 1 | ||
| 375 | + sd['all_columns'] = all_columns | ||
| 376 | + return True | ||
| 377 | + | ||
| 378 | + | ||
| 379 | +def _read_csv_rows(csv_file, steps_data): | ||
| 380 | + """打开 CSV,逐行消费,返回 (all_columns, row_count)。""" | ||
| 381 | + with open(csv_file, 'r', encoding='utf-8') as f: | ||
| 382 | + reader = csv.DictReader(f) | ||
| 383 | + if reader.fieldnames is None: | ||
| 384 | + raise ValidationError('错误: CSV 文件为空或格式不正确') | ||
| 385 | + validate_csv_structure(reader.fieldnames) | ||
| 386 | + all_columns = [col for col in reader.fieldnames if col not in IGNORED_COLUMNS] | ||
| 387 | + row_count = 0 | ||
| 388 | + for row_num, row in enumerate(reader, start=2): | ||
| 389 | + if _consume_csv_row(row, row_num, all_columns, steps_data): | ||
| 390 | + row_count += 1 | ||
| 391 | + return all_columns, row_count | ||
| 392 | + | ||
| 393 | + | ||
| 394 | +def _parse_csv(csv_file, steps_data): | ||
| 395 | + """读取 CSV,填充 steps_data,返回 all_columns。""" | ||
| 396 | + try: | ||
| 397 | + all_columns, row_count = _read_csv_rows(csv_file, steps_data) | ||
| 398 | + if row_count == 0: | ||
| 399 | + raise ValidationError('错误: CSV 文件没有有效数据行') | ||
| 400 | + except ValidationError: | ||
| 401 | + raise | ||
| 402 | + except Exception as e: | ||
| 403 | + raise ValidationError(f'错误: 解析 CSV 文件失败: {e}') from e | ||
| 404 | + return all_columns | ||
| 405 | + | ||
| 406 | + | ||
| 407 | +def _step_summary_entry(step_id, data): | ||
| 408 | + """构建单个 step 的 summary 条目。""" | ||
| 409 | + return { | ||
| 410 | + 'step_id': step_id, | ||
| 411 | + 'total_duration_us': round(data['total_duration'], 1), | ||
| 412 | + 'kernel_count': len(data['kernels']), | ||
| 413 | + 'kernel_types_count': len(data['kernel_types']), | ||
| 414 | + 'kernel_types': dict(sorted(data['kernel_types'].items(), key=lambda x: -x[1])) | ||
| 415 | + } | ||
| 416 | + | ||
| 417 | + | ||
| 418 | +def _selected_step_payload(data, str_step_id, reason, *, with_details): | ||
| 419 | + """构建 selected_step_operators / _details 的公共负载。""" | ||
| 420 | + payload = { | ||
| 421 | + 'step_id': str_step_id, | ||
| 422 | + 'selection_reason': reason, | ||
| 423 | + 'total_duration_us': round(data['total_duration'], 1), | ||
| 424 | + 'kernel_count': len(data['kernels']), | ||
| 425 | + 'kernel_types_count': len(data['kernel_types']), | ||
| 426 | + 'kernel_types': dict(sorted(data['kernel_types'].items(), key=lambda x: -x[1])), | ||
| 427 | + } | ||
| 428 | + if with_details: | ||
| 429 | + payload['csv_columns'] = data['all_columns'] | ||
| 430 | + payload['operators'] = data['kernels_details'] | ||
| 431 | + else: | ||
| 432 | + payload['operators'] = data['kernels'] | ||
| 433 | + return payload | ||
| 434 | + | ||
| 435 | + | ||
| 436 | +def analyze_kernels(csv_file, detail_step_id=None): | ||
| 437 | + validate_file(csv_file) | ||
| 438 | + | ||
| 439 | + steps_data = defaultdict(lambda: { | ||
| 440 | + 'kernels': [], | ||
| 441 | + 'kernels_details': [], | ||
| 442 | + 'total_duration': 0.0, | ||
| 443 | + 'kernel_types': defaultdict(int), | ||
| 444 | + 'all_columns': [] | ||
| 445 | + }) | ||
| 446 | + | ||
| 447 | + all_columns = _parse_csv(csv_file, steps_data) | ||
| 448 | + sorted_steps = sorted(steps_data.keys(), key=lambda x: int(x) if x.isdigit() else x) | ||
| 449 | + | ||
| 450 | + result = { | ||
| 451 | + 'step_count': len(steps_data), | ||
| 452 | + 'steps_summary': [_step_summary_entry(sid, steps_data[sid]) for sid in sorted_steps], | ||
| 453 | + 'csv_columns': all_columns | ||
| 454 | + } | ||
| 455 | + | ||
| 456 | + check_consistency(result['steps_summary']) | ||
| 457 | + | ||
| 458 | + if detail_step_id is None: | ||
| 459 | + detail_step_id, selection_reason = choose_representative_step(result['steps_summary']) | ||
| 460 | + result['selected_step_reason'] = selection_reason | ||
| 461 | + else: | ||
| 462 | + result['selected_step_reason'] = f'user-specified step {detail_step_id}' | ||
| 463 | + | ||
| 464 | + if detail_step_id is not None: | ||
| 465 | + str_step_id = str(detail_step_id) | ||
| 466 | + if str_step_id not in steps_data: | ||
| 467 | + raise ValidationError( | ||
| 468 | + f'错误: Step {detail_step_id} 不存在\n' | ||
| 469 | + f'可用的 Step: {", ".join(sorted_steps)}' | ||
| 470 | + ) | ||
| 471 | + data = steps_data[str_step_id] | ||
| 472 | + reason = result.get('selected_step_reason', '') | ||
| 473 | + result['selected_step_operators'] = _selected_step_payload( | ||
| 474 | + data, str_step_id, reason, with_details=False) | ||
| 475 | + result['selected_step_operators_details'] = _selected_step_payload( | ||
| 476 | + data, str_step_id, reason, with_details=True) | ||
| 477 | + | ||
| 478 | + return result, sorted_steps | ||
| 479 | + | ||
| 480 | + | ||
| 481 | +def _op_signature(op): | ||
| 482 | + """compact 折叠的判同签名。""" | ||
| 483 | + return ( | ||
| 484 | + op.get('normalized_name', ''), | ||
| 485 | + op.get('stream_id', ''), | ||
| 486 | + op.get('task_type', ''), | ||
| 487 | + op.get('input_shapes', ''), | ||
| 488 | + op.get('output_shapes', ''), | ||
| 489 | + ) | ||
| 490 | + | ||
| 491 | + | ||
| 492 | +def _fold_operators(src_ops, keep_fields): | ||
| 493 | + """将连续相同签名(≥3)的 op 折叠为 repeat 块,其余原样保留。""" | ||
| 494 | + compact_ops = [] | ||
| 495 | + i, n = 0, len(src_ops) | ||
| 496 | + while i < n: | ||
| 497 | + sig = _op_signature(src_ops[i]) | ||
| 498 | + j = i + 1 | ||
| 499 | + while j < n and _op_signature(src_ops[j]) == sig: | ||
| 500 | + j += 1 | ||
| 501 | + run_len = j - i | ||
| 502 | + if run_len >= 3: | ||
| 503 | + head, tail = src_ops[i], src_ops[j - 1] | ||
| 504 | + compact_ops.append({ | ||
| 505 | + 'repeat': True, | ||
| 506 | + 'normalized_name': head.get('normalized_name', ''), | ||
| 507 | + 'stream_id': head.get('stream_id', ''), | ||
| 508 | + 'task_type': head.get('task_type', ''), | ||
| 509 | + 'input_shapes': head.get('input_shapes', ''), | ||
| 510 | + 'output_shapes': head.get('output_shapes', ''), | ||
| 511 | + 'first_index': head.get('index'), | ||
| 512 | + 'last_index': tail.get('index'), | ||
| 513 | + 'first_org_index': head.get('org_index'), | ||
| 514 | + 'count': run_len, | ||
| 515 | + }) | ||
| 516 | + else: | ||
| 517 | + for k in range(i, j): | ||
| 518 | + op = src_ops[k] | ||
| 519 | + compact_ops.append({f: op.get(f) for f in keep_fields if f in op}) | ||
| 520 | + i = j | ||
| 521 | + return compact_ops | ||
| 522 | + | ||
| 523 | + | ||
| 524 | +def build_compact_view(operators_block: dict) -> dict: | ||
| 525 | + """将 selected_step_operators 视图压缩为 Step 2 投喂的精简版本。 | ||
| 526 | + | ||
| 527 | + 删除时间字段,仅保留 index/org_index/normalized_name/stream_id/task_type/ | ||
| 528 | + input_shapes/output_shapes;连续相同块折叠为 {repeat:true,...,count}。 | ||
| 529 | + """ | ||
| 530 | + keep_fields = ('index', 'org_index', 'normalized_name', 'stream_id', | ||
| 531 | + 'task_type', 'input_shapes', 'output_shapes') | ||
| 532 | + compact_ops = _fold_operators(operators_block.get('operators', []), keep_fields) | ||
| 533 | + return { | ||
| 534 | + 'step_id': operators_block.get('step_id'), | ||
| 535 | + 'kernel_count': operators_block.get('kernel_count'), | ||
| 536 | + 'kernel_types_count': operators_block.get('kernel_types_count'), | ||
| 537 | + 'kernel_types': operators_block.get('kernel_types', {}), | ||
| 538 | + 'compact_operator_count': len(compact_ops), | ||
| 539 | + 'note': ('compact view for Step 2: timing fields removed; ' | ||
| 540 | + 'consecutive identical ops folded as {repeat:true,...,count}'), | ||
| 541 | + 'operators': compact_ops, | ||
| 542 | + } | ||
| 543 | + | ||
| 544 | + | ||
| 545 | +def generate_markdown(result, csv_file): | ||
| 546 | + lines = [] | ||
| 547 | + lines.append('# Kernel Analysis Summary') | ||
| 548 | + lines.append('') | ||
| 549 | + lines.append(f'**Source File:** `{os.path.basename(csv_file)}`') | ||
| 550 | + lines.append('') | ||
| 551 | + lines.append('---') | ||
| 552 | + lines.append('') | ||
| 553 | + | ||
| 554 | + lines.append('## Overview') | ||
| 555 | + lines.append('') | ||
| 556 | + lines.append('| Step ID | Kernel Count | Kernel Types | Total Duration (us) |') | ||
| 557 | + lines.append('|---------|--------------|--------------|---------------------|') | ||
| 558 | + for s in result['steps_summary']: | ||
| 559 | + lines.append(f'| {s["step_id"]} | {s["kernel_count"]} | {s["kernel_types_count"]} | {s["total_duration_us"]} |') | ||
| 560 | + lines.append('') | ||
| 561 | + if result.get('selected_step_reason'): | ||
| 562 | + lines.append(f'**Selected step:** {result.get("selected_step_reason")}') | ||
| 563 | + lines.append('') | ||
| 564 | + | ||
| 565 | + lines.append('---') | ||
| 566 | + lines.append('') | ||
| 567 | + lines.append('## Kernel Types Distribution') | ||
| 568 | + lines.append('') | ||
| 569 | + lines.append(f'### Steps {", ".join([s["step_id"] for s in result["steps_summary"]])}') | ||
| 570 | + lines.append('') | ||
| 571 | + lines.append('| Kernel Name | Count |') | ||
| 572 | + lines.append('|-------------|-------|') | ||
| 573 | + | ||
| 574 | + first_types = result['steps_summary'][0]['kernel_types'] | ||
| 575 | + for ktype, count in first_types.items(): | ||
| 576 | + lines.append(f'| {ktype} | {count} |') | ||
| 577 | + lines.append('') | ||
| 578 | + | ||
| 579 | + return '\n'.join(lines) | ||
| 580 | + | ||
| 581 | + | ||
| 582 | +def _is_empty_value(v) -> bool: | ||
| 583 | + return v is None or v == '' or str(v).strip().upper() == 'N/A' | ||
| 584 | + | ||
| 585 | + | ||
| 586 | +def _format_shape_pair(input_shapes: str, output_shapes: str) -> str: | ||
| 587 | + ins = str(input_shapes).strip('"') if not _is_empty_value(input_shapes) else '' | ||
| 588 | + outs = str(output_shapes).strip('"') if not _is_empty_value(output_shapes) else '' | ||
| 589 | + parts = [] | ||
| 590 | + if ins: | ||
| 591 | + parts.append(f'[{ins}]') | ||
| 592 | + if outs: | ||
| 593 | + parts.append(f'[{outs}]') | ||
| 594 | + return '→'.join(parts) if parts else '' | ||
| 595 | + | ||
| 596 | + | ||
| 597 | +def _build_op_entry(idx, op, kernel_sem): | ||
| 598 | + """根据 op + 语义信息构建单个 op_data 条目。""" | ||
| 599 | + entry = { | ||
| 600 | + 'index': op.get('index', idx), | ||
| 601 | + 'org_index': op.get('org_index', -1), | ||
| 602 | + 'name': op.get('normalized_name') or op.get('name', 'Unknown'), | ||
| 603 | + 'duration_us': op.get('duration_us', 0), | ||
| 604 | + 'stream_id': op.get('stream_id', ''), | ||
| 605 | + 'task_type': op.get('task_type', ''), | ||
| 606 | + 'input_shapes': op.get('input_shapes', ''), | ||
| 607 | + 'output_shapes': op.get('output_shapes', ''), | ||
| 608 | + } | ||
| 609 | + shape_sem = _format_shape_pair(entry['input_shapes'], entry['output_shapes']) | ||
| 610 | + if shape_sem: | ||
| 611 | + entry['shape_raw'] = shape_sem | ||
| 612 | + sem_info = kernel_sem.get(idx, {}) | ||
| 613 | + for field in ('semantic', 'shape_semantic', 'code_ref'): | ||
| 614 | + if field in sem_info: | ||
| 615 | + entry[field] = sem_info[field] | ||
| 616 | + entry = {k: v for k, v in entry.items() if not _is_empty_value(v) and v != -1} | ||
| 617 | + if 'index' not in entry: | ||
| 618 | + entry['index'] = idx | ||
| 619 | + return entry | ||
| 620 | + | ||
| 621 | + | ||
| 622 | +def _enrich_op_data(node, op_dict, kernel_sem): | ||
| 623 | + """为含 op_indices 的节点构建 op_data 列表。""" | ||
| 624 | + op_data = [] | ||
| 625 | + for idx in node['op_indices']: | ||
| 626 | + op = op_dict.get(idx) | ||
| 627 | + if op is None: | ||
| 628 | + continue | ||
| 629 | + op_data.append(_build_op_entry(idx, op, kernel_sem)) | ||
| 630 | + node['op_data'] = op_data | ||
| 631 | + | ||
| 632 | + | ||
| 633 | +def _enrich_kernels(node, op_dict): | ||
| 634 | + """为 kernels[] 节点补全缺失字段。""" | ||
| 635 | + for ks in node['kernels']: | ||
| 636 | + idx = ks.get('index') | ||
| 637 | + op = op_dict.get(idx) if idx is not None else None | ||
| 638 | + if not op: | ||
| 639 | + continue | ||
| 640 | + ks.setdefault('name', op.get('normalized_name') or op.get('name', 'Unknown')) | ||
| 641 | + ks.setdefault('org_index', op.get('org_index', -1)) | ||
| 642 | + ks.setdefault('duration_us', op.get('duration_us', 0)) | ||
| 643 | + ks.setdefault('input_shapes', op.get('input_shapes', '')) | ||
| 644 | + ks.setdefault('output_shapes', op.get('output_shapes', '')) | ||
| 645 | + if 'shape_raw' not in ks: | ||
| 646 | + shape_sem = _format_shape_pair(op.get('input_shapes', ''), op.get('output_shapes', '')) | ||
| 647 | + if shape_sem: | ||
| 648 | + ks['shape_raw'] = shape_sem | ||
| 649 | + | ||
| 650 | + | ||
| 651 | +def _kernel_semantic_entry(ks): | ||
| 652 | + """从 kernel spec 提取非空 semantic/shape_semantic/code_ref 字段。""" | ||
| 653 | + entry = {} | ||
| 654 | + for field in ('semantic', 'shape_semantic', 'code_ref'): | ||
| 655 | + val = ks.get(field, '') | ||
| 656 | + if val: | ||
| 657 | + entry[field] = val | ||
| 658 | + return entry | ||
| 659 | + | ||
| 660 | + | ||
| 661 | +def _collect_kernels_semantics(node: dict) -> dict: | ||
| 662 | + """收集节点下各 kernel 的语义信息,按 index 索引。""" | ||
| 663 | + result = {} | ||
| 664 | + for ks in node.get('kernels', []): | ||
| 665 | + idx = ks.get('index') | ||
| 666 | + if idx is None: | ||
| 667 | + continue | ||
| 668 | + entry = _kernel_semantic_entry(ks) | ||
| 669 | + if entry: | ||
| 670 | + result[idx] = entry | ||
| 671 | + return result | ||
| 672 | + | ||
| 673 | + | ||
| 674 | +def enrich_analysis_config(config: dict, operators: list) -> dict: | ||
| 675 | + import copy | ||
| 676 | + config = copy.deepcopy(config) | ||
| 677 | + op_dict = {op['index']: op for op in operators} | ||
| 678 | + | ||
| 679 | + def enrich_node(node: dict) -> dict: | ||
| 680 | + kernel_sem = _collect_kernels_semantics(node) | ||
| 681 | + | ||
| 682 | + if 'op_indices' in node: | ||
| 683 | + _enrich_op_data(node, op_dict, kernel_sem) | ||
| 684 | + | ||
| 685 | + if 'kernels' in node: | ||
| 686 | + _enrich_kernels(node, op_dict) | ||
| 687 | + | ||
| 688 | + for child in node.get('children', []): | ||
| 689 | + enrich_node(child) | ||
| 690 | + | ||
| 691 | + return node | ||
| 692 | + | ||
| 693 | + for stage_name, stage_info in config.get('stages', {}).items(): | ||
| 694 | + config['stages'][stage_name] = enrich_node(stage_info) | ||
| 695 | + | ||
| 696 | + for layer_type, structure in config.get('layer_structure', {}).items(): | ||
| 697 | + config['layer_structure'][layer_type] = enrich_node(structure) | ||
| 698 | + | ||
| 699 | + for i, aux in enumerate(config.get('runtime_auxiliary', [])): | ||
| 700 | + config['runtime_auxiliary'][i] = enrich_node(aux) | ||
| 701 | + | ||
| 702 | + return config | ||
| 703 | + | ||
| 704 | + | ||
| 705 | +def enrich_main(args): | ||
| 706 | + config_path = args.config | ||
| 707 | + raw_ops_path = args.raw_ops | ||
| 708 | + output_path = args.output if args.output != 'kernels.json' else config_path | ||
| 709 | + | ||
| 710 | + if not os.path.exists(config_path): | ||
| 711 | + raise ValidationError(f'错误: 配置文件不存在: {config_path}') | ||
| 712 | + if not os.path.exists(raw_ops_path): | ||
| 713 | + raise ValidationError(f'错误: raw_ops 文件不存在: {raw_ops_path}') | ||
| 714 | + | ||
| 715 | + with open(config_path, 'r', encoding='utf-8') as f: | ||
| 716 | + config = json.load(f) | ||
| 717 | + with open(raw_ops_path, 'r', encoding='utf-8') as f: | ||
| 718 | + raw_ops = json.load(f) | ||
| 719 | + | ||
| 720 | + operators = raw_ops.get('operators', []) | ||
| 721 | + enriched = enrich_analysis_config(config, operators) | ||
| 722 | + | ||
| 723 | + with open(output_path, 'w', encoding='utf-8') as f: | ||
| 724 | + json.dump(enriched, f, indent=2, ensure_ascii=False) | ||
| 725 | + logger.info('Enriched config 已保存到: %s', output_path) | ||
| 726 | + | ||
| 727 | + | ||
| 728 | +def main(): | ||
| 729 | + logging.basicConfig(level=logging.INFO, format='%(message)s', stream=sys.stdout) | ||
| 730 | + args = parse_args() | ||
| 731 | + if getattr(args, 'enrich', False): | ||
| 732 | + try: | ||
| 733 | + enrich_main(args) | ||
| 734 | + except ValidationError as e: | ||
| 735 | + logger.error(str(e)) | ||
| 736 | + sys.exit(1) | ||
| 737 | + return | ||
| 738 | + | ||
| 739 | + try: | ||
| 740 | + result, sorted_steps = analyze_kernels(args.file, args.step) | ||
| 741 | + | ||
| 742 | + if 'selected_step_operators' in result: | ||
| 743 | + operators_json = result['selected_step_operators'] | ||
| 744 | + with open(args.output, 'w', encoding='utf-8') as f: | ||
| 745 | + json.dump(operators_json, f, indent=2, ensure_ascii=False) | ||
| 746 | + logger.info('Operators JSON 已保存到: %s', args.output) | ||
| 747 | + | ||
| 748 | + if args.details and 'selected_step_operators_details' in result: | ||
| 749 | + details_json = result['selected_step_operators_details'] | ||
| 750 | + with open(args.details, 'w', encoding='utf-8') as f: | ||
| 751 | + json.dump(details_json, f, indent=2, ensure_ascii=False) | ||
| 752 | + logger.info('Details JSON 已保存到: %s', args.details) | ||
| 753 | + | ||
| 754 | + if args.compact_out and 'selected_step_operators' in result: | ||
| 755 | + compact_json = build_compact_view(result['selected_step_operators']) | ||
| 756 | + with open(args.compact_out, 'w', encoding='utf-8') as f: | ||
| 757 | + json.dump(compact_json, f, indent=2, ensure_ascii=False) | ||
| 758 | + src_n = compact_json['kernel_count'] | ||
| 759 | + cmp_n = compact_json['compact_operator_count'] | ||
| 760 | + saved = (1 - cmp_n / src_n) * 100 if src_n else 0 | ||
| 761 | + logger.info('Compact JSON 已保存到: %s (原 %s 算子 → %s 条目,折叠率 %.1f%%)', | ||
| 762 | + args.compact_out, src_n, cmp_n, saved) | ||
| 763 | + | ||
| 764 | + if args.markdown: | ||
| 765 | + markdown_content = generate_markdown(result, args.file) | ||
| 766 | + with open(args.markdown, 'w', encoding='utf-8') as f: | ||
| 767 | + f.write(markdown_content) | ||
| 768 | + logger.info('Summary Markdown 已保存到: %s', args.markdown) | ||
| 769 | + | ||
| 770 | + except ValidationError as e: | ||
| 771 | + logger.error(str(e)) | ||
| 772 | + sys.exit(1) | ||
| 773 | + except ConsistencyError as e: | ||
| 774 | + logger.error(str(e)) | ||
| 775 | + sys.exit(1) | ||
| 776 | + except Exception as e: | ||
| 777 | + logger.error('错误: 未知错误: %s', e) | ||
| 778 | + sys.exit(1) | ||
| 779 | + | ||
| 780 | + | ||
| 781 | +if __name__ == '__main__': | ||
| 782 | + main() | ||
| @@ -0,0 +1,213 @@ | |||
| 1 | +#!/usr/bin/env python3 | ||
| 2 | +# -*- coding: utf-8 -*- | ||
| 3 | +# ---------------------------------------------------------------------------- | ||
| 4 | +# Copyright (c) 2025 Huawei Technologies Co., Ltd. | ||
| 5 | +# | ||
| 6 | +# Licensed under the Apache License, Version 2.0 (the "License"); | ||
| 7 | +# you may not use this file except in compliance with the License. | ||
| 8 | +# You may obtain a copy of the License at | ||
| 9 | +# | ||
| 10 | +# http://www.apache.org/licenses/LICENSE-2.0 | ||
| 11 | +# | ||
| 12 | +# Unless required by applicable law or agreed to in writing, software | ||
| 13 | +# distributed under the License is distributed on an "AS IS" BASIS, | ||
| 14 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| 15 | +# See the License for the specific language governing permissions and | ||
| 16 | +# limitations under the License. | ||
| 17 | +# ---------------------------------------------------------------------------- | ||
| 18 | +""" | ||
| 19 | +Step 3 Review: 校验 raw_ops 中的算子在 analysis_config 中的覆盖关系 (C1–C3)。 | ||
| 20 | + | ||
| 21 | +C1: raw_ops 全部 op 是否被 layer_structure ∪ stages ∪ runtime_auxiliary 覆盖 | ||
| 22 | +C2: 同一 op_index 是否出现在多个节点 op_indices 中(错误,与 check_structure S9 warning 互补—— | ||
| 23 | + 本脚本视为 ERROR,因为同时还要检查未覆盖) | ||
| 24 | +C3: 11 类必填 shape_semantic 的 kernel 中,哪些尚未在 analysis_config 的 kernels 数组里登记 | ||
| 25 | +""" | ||
| 26 | +import logging | ||
| 27 | +import argparse | ||
| 28 | +import json | ||
| 29 | +import os | ||
| 30 | +import sys | ||
| 31 | + | ||
| 32 | +from _common import is_shape_always_required | ||
| 33 | + | ||
| 34 | +logger = logging.getLogger(__name__) | ||
| 35 | + | ||
| 36 | + | ||
| 37 | +class Issue(dict): | ||
| 38 | + def __init__(self, code, severity, path, message): | ||
| 39 | + super().__init__(id=code, severity=severity, node_path=path, message=message) | ||
| 40 | + | ||
| 41 | + | ||
| 42 | +def collect_leaf_op_paths(config): | ||
| 43 | + out = {} # op_index -> [leaf_path] | ||
| 44 | + kernels_registered = {} # op_index -> kernel name (if registered with shape_semantic info) | ||
| 45 | + | ||
| 46 | + def walk(node, path): | ||
| 47 | + if not isinstance(node, dict): | ||
| 48 | + return | ||
| 49 | + for idx in node.get('op_indices', []) or []: | ||
| 50 | + out.setdefault(idx, []).append(path) | ||
| 51 | + for ks in node.get('kernels', []) or []: | ||
| 52 | + kn = ks.get('name', '') or '' | ||
| 53 | + kn = kn.split('/')[-1] if '/' in kn else kn | ||
| 54 | + idx = ks.get('index') | ||
| 55 | + if idx is not None: | ||
| 56 | + kernels_registered[idx] = { | ||
| 57 | + 'name': kn, | ||
| 58 | + 'has_shape_semantic': bool(ks.get('shape_semantic')), | ||
| 59 | + 'path': path, | ||
| 60 | + } | ||
| 61 | + for child in node.get('children', []) or []: | ||
| 62 | + cname = child.get('name', '?') | ||
| 63 | + walk(child, f'{path}/{cname}') | ||
| 64 | + | ||
| 65 | + for sname, sinfo in (config.get('stages') or {}).items(): | ||
| 66 | + walk(sinfo, f'stages/{sname}') | ||
| 67 | + for ltype, lstruct in (config.get('layer_structure') or {}).items(): | ||
| 68 | + walk(lstruct, f'layer_structure/{ltype}') | ||
| 69 | + for i, aux in enumerate(config.get('runtime_auxiliary') or []): | ||
| 70 | + walk(aux, f'runtime_auxiliary[{i}]') | ||
| 71 | + | ||
| 72 | + return out, kernels_registered | ||
| 73 | + | ||
| 74 | + | ||
| 75 | +def _collect_section_op_indices(config_section): | ||
| 76 | + """递归收集一段树结构下所有 op_indices 的集合。""" | ||
| 77 | + ops = set() | ||
| 78 | + | ||
| 79 | + def walk(node): | ||
| 80 | + if not isinstance(node, dict): | ||
| 81 | + return | ||
| 82 | + for idx in node.get('op_indices', []) or []: | ||
| 83 | + ops.add(idx) | ||
| 84 | + for child in node.get('children', []) or []: | ||
| 85 | + walk(child) | ||
| 86 | + | ||
| 87 | + if isinstance(config_section, dict): | ||
| 88 | + walk(config_section) | ||
| 89 | + elif isinstance(config_section, list): | ||
| 90 | + for item in config_section: | ||
| 91 | + walk(item) | ||
| 92 | + return ops | ||
| 93 | + | ||
| 94 | + | ||
| 95 | +def _check_c1_count(config, n_ops, issues): | ||
| 96 | + """C1: 计数校验(代表性 layer 展开后)。返回 (total_accounted, gap)。""" | ||
| 97 | + stages_ops = set() | ||
| 98 | + for _sname, sinfo in (config.get('stages') or {}).items(): | ||
| 99 | + stages_ops |= _collect_section_op_indices(sinfo) | ||
| 100 | + | ||
| 101 | + runtime_ops = set() | ||
| 102 | + for aux in (config.get('runtime_auxiliary') or []): | ||
| 103 | + runtime_ops |= _collect_section_op_indices(aux) | ||
| 104 | + | ||
| 105 | + layer_total_accounted = 0 | ||
| 106 | + per_layer_type_info = [] | ||
| 107 | + for ltype, lstruct in (config.get('layer_structure') or {}).items(): | ||
| 108 | + rep_ops = _collect_section_op_indices(lstruct) | ||
| 109 | + n_instances = len((config.get('layer_types') or {}).get(ltype, {}).get('layer_indices', [])) | ||
| 110 | + accounted = len(rep_ops) * max(n_instances, 1) | ||
| 111 | + layer_total_accounted += accounted | ||
| 112 | + per_layer_type_info.append({ | ||
| 113 | + 'layer_type': ltype, | ||
| 114 | + 'rep_ops_count': len(rep_ops), | ||
| 115 | + 'n_instances': n_instances, | ||
| 116 | + 'accounted': accounted, | ||
| 117 | + }) | ||
| 118 | + | ||
| 119 | + total_accounted = len(stages_ops) + len(runtime_ops) + layer_total_accounted | ||
| 120 | + gap = n_ops - total_accounted | ||
| 121 | + gap_pct = round(100 * gap / max(n_ops, 1), 2) | ||
| 122 | + if abs(gap_pct) > 5: | ||
| 123 | + issues.append(Issue('C1', 'warning', '<global>', | ||
| 124 | + f'op count gap: raw_ops={n_ops}, accounted={total_accounted}, ' | ||
| 125 | + f'gap={gap} ({gap_pct}%). 详细: {per_layer_type_info}')) | ||
| 126 | + return total_accounted, gap | ||
| 127 | + | ||
| 128 | + | ||
| 129 | +def _check_c2_overlap(op_to_paths, issues): | ||
| 130 | + """C2: 节点间 op_indices 重叠。""" | ||
| 131 | + for idx, paths in op_to_paths.items(): | ||
| 132 | + if len(paths) <= 1: | ||
| 133 | + continue | ||
| 134 | + sections = set(p.split('/')[0] for p in paths) | ||
| 135 | + if len(sections) == 1: | ||
| 136 | + issues.append(Issue('C2', 'error', f'op_index={idx}', | ||
| 137 | + f'op_index={idx} 在同 section 多叶节点出现: {paths}')) | ||
| 138 | + else: | ||
| 139 | + issues.append(Issue('C2', 'warning', f'op_index={idx}', | ||
| 140 | + f'op_index={idx} 跨 section 出现: {paths}(确认是否合理)')) | ||
| 141 | + | ||
| 142 | + | ||
| 143 | +def _check_c3_shape_semantic(kernels_registered, issues): | ||
| 144 | + """C3: kernels[] 已登记的算子缺 shape_semantic。""" | ||
| 145 | + for idx, info in kernels_registered.items(): | ||
| 146 | + kn = info.get('name', '') | ||
| 147 | + if is_shape_always_required(kn) and not info.get('has_shape_semantic'): | ||
| 148 | + issues.append(Issue('C3', 'error', f'{info.get("path", "?")}/kernels[index={idx}]', | ||
| 149 | + f'{kn} 必填 shape_semantic 但未提供')) | ||
| 150 | + | ||
| 151 | + | ||
| 152 | +def check_coverage(config, raw_ops): | ||
| 153 | + """coverage 校验(C1 计数 / C2 重叠 / C3 shape_semantic)。 | ||
| 154 | + | ||
| 155 | + analysis_config.json 采用"代表性 layer + layer_indices"模式, | ||
| 156 | + layer_structure 只列代表层的 op_indices,故用计数校验而非逐 op 覆盖判定。 | ||
| 157 | + """ | ||
| 158 | + issues = [] | ||
| 159 | + op_to_paths, kernels_registered = collect_leaf_op_paths(config) | ||
| 160 | + n_ops = len(raw_ops.get('operators', [])) | ||
| 161 | + | ||
| 162 | + total_accounted, gap = _check_c1_count(config, n_ops, issues) | ||
| 163 | + _check_c2_overlap(op_to_paths, issues) | ||
| 164 | + _check_c3_shape_semantic(kernels_registered, issues) | ||
| 165 | + | ||
| 166 | + return issues, n_ops, total_accounted, gap | ||
| 167 | + | ||
| 168 | + | ||
| 169 | +def main(): | ||
| 170 | + logging.basicConfig(level=logging.INFO, format='%(message)s', stream=sys.stdout) | ||
| 171 | + parser = argparse.ArgumentParser(description='Step 3 Review: op coverage check') | ||
| 172 | + parser.add_argument('-c', '--config', required=True, help='analysis_config.json 路径') | ||
| 173 | + parser.add_argument('-r', '--raw-ops', dest='raw_ops', required=True, help='raw_ops.json 路径') | ||
| 174 | + parser.add_argument('--json', action='store_true', help='以 JSON 输出') | ||
| 175 | + args = parser.parse_args() | ||
| 176 | + | ||
| 177 | + if not os.path.exists(args.config) or not os.path.exists(args.raw_ops): | ||
| 178 | + logger.error('错误: 输入文件不存在') | ||
| 179 | + sys.exit(1) | ||
| 180 | + | ||
| 181 | + with open(args.config, 'r', encoding='utf-8') as f: | ||
| 182 | + config = json.load(f) | ||
| 183 | + with open(args.raw_ops, 'r', encoding='utf-8') as f: | ||
| 184 | + raw_ops = json.load(f) | ||
| 185 | + | ||
| 186 | + issues, n_ops, n_accounted, gap = check_coverage(config, raw_ops) | ||
| 187 | + errors = [i for i in issues if i['severity'] == 'error'] | ||
| 188 | + | ||
| 189 | + if args.json: | ||
| 190 | + out = { | ||
| 191 | + 'script': 'check_op_coverage.py', | ||
| 192 | + 'config': args.config, | ||
| 193 | + 'raw_ops': args.raw_ops, | ||
| 194 | + 'total_ops': n_ops, | ||
| 195 | + 'accounted_ops': n_accounted, | ||
| 196 | + 'gap': gap, | ||
| 197 | + 'gap_pct': round(100 * gap / max(n_ops, 1), 2), | ||
| 198 | + 'error_count': len(errors), | ||
| 199 | + 'issues': issues, | ||
| 200 | + } | ||
| 201 | + logger.info(json.dumps(out, indent=2, ensure_ascii=False)) | ||
| 202 | + else: | ||
| 203 | + for it in issues: | ||
| 204 | + sev = it['severity'].upper() | ||
| 205 | + logger.info('[%s] %s @ %s: %s', sev, it["id"], it["node_path"], it["message"]) | ||
| 206 | + logger.info('\n汇总: ops 总数=%d, 折算覆盖=%d, gap=%d (%s%%), errors=%d', | ||
| 207 | + n_ops, n_accounted, gap, round(100 * gap / max(n_ops, 1), 2), len(errors)) | ||
| 208 | + | ||
| 209 | + sys.exit(1 if errors else 0) | ||
| 210 | + | ||
| 211 | + | ||
| 212 | +if __name__ == '__main__': | ||
| 213 | + main() | ||
| @@ -0,0 +1,287 @@ | |||
| 1 | +#!/usr/bin/env python3 | ||
| 2 | +# -*- coding: utf-8 -*- | ||
| 3 | +# ---------------------------------------------------------------------------- | ||
| 4 | +# Copyright (c) 2025 Huawei Technologies Co., Ltd. | ||
| 5 | +# | ||
| 6 | +# Licensed under the Apache License, Version 2.0 (the "License"); | ||
| 7 | +# you may not use this file except in compliance with the License. | ||
| 8 | +# You may obtain a copy of the License at | ||
| 9 | +# | ||
| 10 | +# http://www.apache.org/licenses/LICENSE-2.0 | ||
| 11 | +# | ||
| 12 | +# Unless required by applicable law or agreed to in writing, software | ||
| 13 | +# distributed under the License is distributed on an "AS IS" BASIS, | ||
| 14 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| 15 | +# See the License for the specific language governing permissions and | ||
| 16 | +# limitations under the License. | ||
| 17 | +# ---------------------------------------------------------------------------- | ||
| 18 | +""" | ||
| 19 | +Step 3 Review: 校验 analysis_config.json 的结构良构性 (S1–S9)。 | ||
| 20 | + | ||
| 21 | +输出格式与 check_op_coverage.py、validate_shapes.py 统一: | ||
| 22 | + 普通模式:可读文本 + 错误码退出 | ||
| 23 | + --json : 追加单行 JSON 到 stdout,schema 见 schema_doc() | ||
| 24 | +""" | ||
| 25 | +import logging | ||
| 26 | +import argparse | ||
| 27 | +import json | ||
| 28 | +import os | ||
| 29 | +import sys | ||
| 30 | + | ||
| 31 | +from _common import is_shape_always_required | ||
| 32 | + | ||
| 33 | +logger = logging.getLogger(__name__) | ||
| 34 | + | ||
| 35 | + | ||
| 36 | +# 始终必填 shape_semantic 的算子(与 references/structure_analysis_guide.md §B.5 单源) | ||
| 37 | +# 这些算子无歧义,每个实例都必须填 shape_semantic | ||
| 38 | +# 仅在已登记到 kernels[] 时才校验 shape_semantic 的算子(语义上下文相关) | ||
| 39 | +# - Add 仅在残差连接语义时 | ||
| 40 | +# - ConcatV2/ConcatD 仅在 KV cache 拼接时 | ||
| 41 | +# - ScatterNdUpdate 仅在 KV cache 更新时 | ||
| 42 | +# 静态脚本无法判定上下文,故仅当 AI 已主动登记时才校验字段存在 | ||
| 43 | +SHAPE_SEMANTIC_IF_REGISTERED = { | ||
| 44 | + 'Add', 'ConcatV2', 'ConcatD', 'ScatterNdUpdate', | ||
| 45 | +} | ||
| 46 | + | ||
| 47 | +# semantic 免填的算子 | ||
| 48 | +SEMANTIC_OPTIONAL = {'Cast', 'Reshape'} | ||
| 49 | + | ||
| 50 | + | ||
| 51 | +def is_shape_contextual(name: str) -> bool: | ||
| 52 | + return name in SHAPE_SEMANTIC_IF_REGISTERED | ||
| 53 | + | ||
| 54 | + | ||
| 55 | +class Issue(dict): | ||
| 56 | + def __init__(self, code, severity, path, message): | ||
| 57 | + super().__init__(id=code, severity=severity, node_path=path, message=message) | ||
| 58 | + | ||
| 59 | + | ||
| 60 | +def walk_tree(node, path, callback): | ||
| 61 | + if not isinstance(node, dict): | ||
| 62 | + return | ||
| 63 | + callback(node, path) | ||
| 64 | + for child in node.get('children', []) or []: | ||
| 65 | + cname = child.get('name', '?') | ||
| 66 | + walk_tree(child, f'{path}/{cname}', callback) | ||
| 67 | + | ||
| 68 | + | ||
| 69 | +def collect_leaf_op_indices(config): | ||
| 70 | + """返回 [(op_index, leaf_path)] 列表""" | ||
| 71 | + out = [] | ||
| 72 | + | ||
| 73 | + def visit(node, path): | ||
| 74 | + # 叶子或带 op_indices 的中间节点 | ||
| 75 | + for idx in node.get('op_indices', []) or []: | ||
| 76 | + out.append((idx, path)) | ||
| 77 | + | ||
| 78 | + for sname, sinfo in (config.get('stages') or {}).items(): | ||
| 79 | + walk_tree(sinfo, f'stages/{sname}', visit) | ||
| 80 | + for ltype, lstruct in (config.get('layer_structure') or {}).items(): | ||
| 81 | + walk_tree(lstruct, f'layer_structure/{ltype}', visit) | ||
| 82 | + for i, aux in enumerate(config.get('runtime_auxiliary') or []): | ||
| 83 | + walk_tree(aux, f'runtime_auxiliary[{i}]', visit) | ||
| 84 | + return out | ||
| 85 | + | ||
| 86 | + | ||
| 87 | +def walk_all_sections(config, visit): | ||
| 88 | + """对 stages / layer_structure / runtime_auxiliary 三段树统一遍历。""" | ||
| 89 | + for sname, sinfo in (config.get('stages') or {}).items(): | ||
| 90 | + walk_tree(sinfo, f'stages/{sname}', visit) | ||
| 91 | + for ltype, lstruct in (config.get('layer_structure') or {}).items(): | ||
| 92 | + walk_tree(lstruct, f'layer_structure/{ltype}', visit) | ||
| 93 | + for i, aux in enumerate(config.get('runtime_auxiliary') or []): | ||
| 94 | + walk_tree(aux, f'runtime_auxiliary[{i}]', visit) | ||
| 95 | + | ||
| 96 | + | ||
| 97 | +def _check_s1_schema(config, issues): | ||
| 98 | + """S1: 树节点 schema 完整。""" | ||
| 99 | + def visit(node, path): | ||
| 100 | + if 'name' not in node: | ||
| 101 | + issues.append(Issue('S1', 'error', path, '节点缺少 name 字段')) | ||
| 102 | + is_leaf = ('children' not in node or not node['children']) | ||
| 103 | + if is_leaf and 'op_indices' not in node and 'branches' not in node: | ||
| 104 | + issues.append(Issue('S1', 'error', path, '叶节点缺少 op_indices')) | ||
| 105 | + walk_all_sections(config, visit) | ||
| 106 | + | ||
| 107 | + | ||
| 108 | +def _check_s2_layer_match(config, issues): | ||
| 109 | + """S2: layer_types ↔ layer_structure 匹配。""" | ||
| 110 | + lt_keys = set((config.get('layer_types') or {}).keys()) | ||
| 111 | + ls_keys = set((config.get('layer_structure') or {}).keys()) | ||
| 112 | + for k in lt_keys - ls_keys: | ||
| 113 | + issues.append(Issue('S2', 'error', f'layer_types/{k}', | ||
| 114 | + 'layer_types 中存在但 layer_structure 中缺失')) | ||
| 115 | + for k in ls_keys - lt_keys: | ||
| 116 | + issues.append(Issue('S2', 'error', f'layer_structure/{k}', | ||
| 117 | + 'layer_structure 中存在但 layer_types 中缺失')) | ||
| 118 | + | ||
| 119 | + | ||
| 120 | +def _check_s3_semantic(config, issues): | ||
| 121 | + """S3: semantic 必填(除 SEMANTIC_OPTIONAL kernel)。""" | ||
| 122 | + def visit(node, path): | ||
| 123 | + if not node.get('semantic'): | ||
| 124 | + issues.append(Issue('S3', 'error', path, '节点缺少 semantic')) | ||
| 125 | + for ks in node.get('kernels', []) or []: | ||
| 126 | + kn = ks.get('name', '') or '' | ||
| 127 | + kn_norm = kn.split('/')[-1] if '/' in kn else kn | ||
| 128 | + if kn_norm in SEMANTIC_OPTIONAL: | ||
| 129 | + continue | ||
| 130 | + if not ks.get('semantic'): | ||
| 131 | + issues.append(Issue('S3', 'error', | ||
| 132 | + f'{path}/kernels[index={ks.get("index")}]', | ||
| 133 | + f'kernel {kn_norm} 缺少 semantic')) | ||
| 134 | + walk_all_sections(config, visit) | ||
| 135 | + | ||
| 136 | + | ||
| 137 | +def _check_s4_code_ref(config, issues): | ||
| 138 | + """S4: code_ref 必填。""" | ||
| 139 | + def visit(node, path): | ||
| 140 | + if not node.get('code_ref') and 'branches' not in node: | ||
| 141 | + issues.append(Issue('S4', 'warning', path, '节点缺少 code_ref')) | ||
| 142 | + walk_all_sections(config, visit) | ||
| 143 | + | ||
| 144 | + | ||
| 145 | +def _check_s5_shape_semantic(config, issues): | ||
| 146 | + """S5: shape_semantic 字段(始终必填类 error,上下文必填类 warning)。""" | ||
| 147 | + def visit(node, path): | ||
| 148 | + for ks in node.get('kernels', []) or []: | ||
| 149 | + kn = ks.get('name', '') or '' | ||
| 150 | + kn_norm = kn.split('/')[-1] if '/' in kn else kn | ||
| 151 | + if is_shape_always_required(kn_norm) or is_shape_contextual(kn_norm): | ||
| 152 | + if not ks.get('shape_semantic'): | ||
| 153 | + sev = 'error' if is_shape_always_required(kn_norm) else 'warning' | ||
| 154 | + issues.append(Issue('S5', sev, | ||
| 155 | + f'{path}/kernels[index={ks.get("index")}]', | ||
| 156 | + f'{kn_norm} 已登记 kernels[] 但缺 shape_semantic')) | ||
| 157 | + walk_all_sections(config, visit) | ||
| 158 | + | ||
| 159 | + | ||
| 160 | +def _check_s6_indices_unique(config, issues): | ||
| 161 | + """S6: layer_indices / stage_indices / instance_indices 列表无重复值。""" | ||
| 162 | + for ltype, info in (config.get('layer_types') or {}).items(): | ||
| 163 | + idx = info.get('layer_indices', []) | ||
| 164 | + if len(idx) != len(set(idx)): | ||
| 165 | + issues.append(Issue('S6', 'error', f'layer_types/{ltype}', 'layer_indices 含重复值')) | ||
| 166 | + | ||
| 167 | + def visit(node, path): | ||
| 168 | + for fld in ('stage_indices', 'instance_indices'): | ||
| 169 | + v = node.get(fld) | ||
| 170 | + if v and len(v) != len(set(v)): | ||
| 171 | + issues.append(Issue('S6', 'error', path, f'{fld} 含重复值')) | ||
| 172 | + for sname, sinfo in (config.get('stages') or {}).items(): | ||
| 173 | + walk_tree(sinfo, f'stages/{sname}', visit) | ||
| 174 | + for i, aux in enumerate(config.get('runtime_auxiliary') or []): | ||
| 175 | + walk_tree(aux, f'runtime_auxiliary[{i}]', visit) | ||
| 176 | + | ||
| 177 | + | ||
| 178 | +def _check_s7_instances(config, issues): | ||
| 179 | + """S7: 每个 layer_type 至少 1 个实例。""" | ||
| 180 | + for ltype, info in (config.get('layer_types') or {}).items(): | ||
| 181 | + if not info.get('layer_indices'): | ||
| 182 | + issues.append(Issue('S7', 'error', f'layer_types/{ltype}', | ||
| 183 | + f'layer_type {ltype} 没有 layer_indices')) | ||
| 184 | + | ||
| 185 | + | ||
| 186 | +def _check_s8_no_dup_aux(config, issues): | ||
| 187 | + """S8: runtime_auxiliary 节点不应出现在 layer_structure 子树中。""" | ||
| 188 | + aux_names = set() | ||
| 189 | + for aux in config.get('runtime_auxiliary') or []: | ||
| 190 | + if isinstance(aux, dict) and aux.get('name'): | ||
| 191 | + aux_names.add(aux['name']) | ||
| 192 | + | ||
| 193 | + def visit(node, path): | ||
| 194 | + if node.get('name') in aux_names: | ||
| 195 | + issues.append(Issue('S8', 'warning', path, | ||
| 196 | + f'节点名 {node["name"]} 同时出现在 runtime_auxiliary')) | ||
| 197 | + for ltype, lstruct in (config.get('layer_structure') or {}).items(): | ||
| 198 | + walk_tree(lstruct, f'layer_structure/{ltype}', visit) | ||
| 199 | + | ||
| 200 | + | ||
| 201 | +def _check_s9_op_overlap(config, issues): | ||
| 202 | + """S9: 同 op_index 不在多个叶节点同时出现。""" | ||
| 203 | + op_to_paths = {} | ||
| 204 | + for op_idx, leaf_path in collect_leaf_op_indices(config): | ||
| 205 | + op_to_paths.setdefault(op_idx, []).append(leaf_path) | ||
| 206 | + for op_idx, paths in op_to_paths.items(): | ||
| 207 | + if len(paths) > 1: | ||
| 208 | + issues.append(Issue('S9', 'warning', f'op_index={op_idx}', | ||
| 209 | + f'op_index {op_idx} 在多个叶节点出现: {paths}')) | ||
| 210 | + | ||
| 211 | + | ||
| 212 | +def check_structure(config): | ||
| 213 | + issues = [] | ||
| 214 | + _check_s1_schema(config, issues) | ||
| 215 | + _check_s2_layer_match(config, issues) | ||
| 216 | + _check_s3_semantic(config, issues) | ||
| 217 | + _check_s4_code_ref(config, issues) | ||
| 218 | + _check_s5_shape_semantic(config, issues) | ||
| 219 | + _check_s6_indices_unique(config, issues) | ||
| 220 | + _check_s7_instances(config, issues) | ||
| 221 | + _check_s8_no_dup_aux(config, issues) | ||
| 222 | + _check_s9_op_overlap(config, issues) | ||
| 223 | + return issues | ||
| 224 | + | ||
| 225 | + | ||
| 226 | +def schema_doc(): | ||
| 227 | + return { | ||
| 228 | + 'script': 'check_structure.py', | ||
| 229 | + 'rules': { | ||
| 230 | + 'S1': '树节点 schema 完整:name 必有;叶节点必有 op_indices;中间节点必有 children', | ||
| 231 | + 'S2': 'layer_types 与 layer_structure 中存在的 key 集合一致', | ||
| 232 | + 'S3': '节点 semantic 必填(kernel 中 Cast/Reshape 除外)', | ||
| 233 | + 'S4': '节点 code_ref 推荐填写(warning)', | ||
| 234 | + 'S5': '11 类算子必有 shape_semantic 字段', | ||
| 235 | + 'S6': 'layer_indices/stage_indices/instance_indices 无重复值', | ||
| 236 | + 'S7': '每个 layer_type 至少有 1 个 layer_indices 实例', | ||
| 237 | + 'S8': 'runtime_auxiliary 节点不在 layer_structure 子树中重名(warning)', | ||
| 238 | + 'S9': '同 op_index 不在多个叶节点同时出现(warning)', | ||
| 239 | + } | ||
| 240 | + } | ||
| 241 | + | ||
| 242 | + | ||
| 243 | +def main(): | ||
| 244 | + logging.basicConfig(level=logging.INFO, format='%(message)s', stream=sys.stdout) | ||
| 245 | + parser = argparse.ArgumentParser(description='Step 3 Review: structure well-formedness') | ||
| 246 | + parser.add_argument('-c', '--config', required=True, help='analysis_config.json 路径') | ||
| 247 | + parser.add_argument('--json', action='store_true', help='以 JSON 输出(追加到 stdout)') | ||
| 248 | + parser.add_argument('--mode', default='A', choices=['A', 'B'], | ||
| 249 | + help='Mode A 严格校验,Mode B 跳过 op_indices/kernels 必填检查') | ||
| 250 | + args = parser.parse_args() | ||
| 251 | + | ||
| 252 | + if not os.path.exists(args.config): | ||
| 253 | + logger.error('错误: 文件不存在: %s', args.config) | ||
| 254 | + sys.exit(1) | ||
| 255 | + with open(args.config, 'r', encoding='utf-8') as f: | ||
| 256 | + config = json.load(f) | ||
| 257 | + | ||
| 258 | + issues = check_structure(config) | ||
| 259 | + | ||
| 260 | + if args.mode == 'B': | ||
| 261 | + # Mode B 不要求 op_indices/kernels(无 raw_ops 可绑定) | ||
| 262 | + issues = [i for i in issues if i['id'] not in ('S1', 'S5', 'S9')] | ||
| 263 | + | ||
| 264 | + errors = [i for i in issues if i['severity'] == 'error'] | ||
| 265 | + warnings = [i for i in issues if i['severity'] == 'warning'] | ||
| 266 | + | ||
| 267 | + if args.json: | ||
| 268 | + out = { | ||
| 269 | + 'script': 'check_structure.py', | ||
| 270 | + 'config': args.config, | ||
| 271 | + 'mode': args.mode, | ||
| 272 | + 'error_count': len(errors), | ||
| 273 | + 'warning_count': len(warnings), | ||
| 274 | + 'issues': issues, | ||
| 275 | + } | ||
| 276 | + logger.info(json.dumps(out, indent=2, ensure_ascii=False)) | ||
| 277 | + else: | ||
| 278 | + for sev_label, items in [('ERROR', errors), ('WARNING', warnings)]: | ||
| 279 | + for it in items: | ||
| 280 | + logger.info('[%s] %s @ %s: %s', sev_label, it["id"], it["node_path"], it["message"]) | ||
| 281 | + logger.info('\n汇总: errors=%d, warnings=%d', len(errors), len(warnings)) | ||
| 282 | + | ||
| 283 | + sys.exit(1 if errors else 0) | ||
| 284 | + | ||
| 285 | + | ||
| 286 | +if __name__ == '__main__': | ||
| 287 | + main() | ||
| @@ -0,0 +1,475 @@ | |||
| 1 | +#!/usr/bin/env python3 | ||
| 2 | +# -*- coding: utf-8 -*- | ||
| 3 | +# ---------------------------------------------------------------------------- | ||
| 4 | +# Copyright (c) 2025 Huawei Technologies Co., Ltd. | ||
| 5 | +# | ||
| 6 | +# Licensed under the Apache License, Version 2.0 (the "License"); | ||
| 7 | +# you may not use this file except in compliance with the License. | ||
| 8 | +# You may obtain a copy of the License at | ||
| 9 | +# | ||
| 10 | +# http://www.apache.org/licenses/LICENSE-2.0 | ||
| 11 | +# | ||
| 12 | +# Unless required by applicable law or agreed to in writing, software | ||
| 13 | +# distributed under the License is distributed on an "AS IS" BASIS, | ||
| 14 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| 15 | +# See the License for the specific language governing permissions and | ||
| 16 | +# limitations under the License. | ||
| 17 | +# ---------------------------------------------------------------------------- | ||
| 18 | +""" | ||
| 19 | +NPU 性能指标计算脚本 - Step 4 | ||
| 20 | + | ||
| 21 | +计算各节点的四维性能指标及衍生指标: | ||
| 22 | +- wall_ms: 实际墙上时钟耗时(含间隙) | ||
| 23 | +- busy_union_ms: 设备实际忙碌时间(合并去重叠) | ||
| 24 | +- kernel_sum_ms: 所有 kernel 时长算术和 | ||
| 25 | +- total_cost_ms: 总成本(duration + wait) | ||
| 26 | +- parallelism: 并行度(kernel_sum_ms / wall_ms) | ||
| 27 | +- bubble_ms: 气泡时间(wall_ms - busy_union_ms) | ||
| 28 | +- ratio_pct: 占比(wall_ms / step总wall_ms × 100) | ||
| 29 | + | ||
| 30 | +用法: | ||
| 31 | + python compute_metrics.py -r raw_ops_details.json -c analysis_config.json -o metrics_report.md | ||
| 32 | +""" | ||
| 33 | +import logging | ||
| 34 | +import argparse | ||
| 35 | +import sys | ||
| 36 | +from dataclasses import dataclass | ||
| 37 | +from typing import List, Tuple | ||
| 38 | + | ||
| 39 | +from _common import validate_file_exists, load_json | ||
| 40 | + | ||
| 41 | +logger = logging.getLogger(__name__) | ||
| 42 | + | ||
| 43 | + | ||
| 44 | + | ||
| 45 | +class TreeWalkCtx: | ||
| 46 | + """collect_tree_nodes 递归过程中保持不变的上下文。""" | ||
| 47 | + operators: List[dict] | ||
| 48 | + max_depth: int | ||
| 49 | + | ||
| 50 | + | ||
| 51 | + | ||
| 52 | +def merge_intervals(intervals: List[Tuple[float, float]]) -> float: | ||
| 53 | + """ | ||
| 54 | + 合并重叠区间,返回总长度 | ||
| 55 | + intervals: [(start, end), ...] | ||
| 56 | + """ | ||
| 57 | + if not intervals: | ||
| 58 | + return 0.0 | ||
| 59 | + | ||
| 60 | + sorted_intervals = sorted(intervals, key=lambda x: x[0]) | ||
| 61 | + merged = [sorted_intervals[0]] | ||
| 62 | + | ||
| 63 | + for start, end in sorted_intervals[1:]: | ||
| 64 | + last_start, last_end = merged[-1] | ||
| 65 | + if start <= last_end: | ||
| 66 | + merged[-1] = (last_start, max(last_end, end)) | ||
| 67 | + else: | ||
| 68 | + merged.append((start, end)) | ||
| 69 | + | ||
| 70 | + return sum(end - start for start, end in merged) | ||
| 71 | + | ||
| 72 | + | ||
| 73 | +def _accumulate_op_timing(ops): | ||
| 74 | + """遍历 ops 累加时间信息,返回 (wall_us, intervals, duration_sum, total_cost)。""" | ||
| 75 | + start_times, end_times, intervals = [], [], [] | ||
| 76 | + duration_sum = total_cost = 0.0 | ||
| 77 | + for op in ops: | ||
| 78 | + start = op.get('start_time_us', 0) | ||
| 79 | + duration = op.get('duration_us', 0) | ||
| 80 | + wait = op.get('wait_time_us', 0) or 0 | ||
| 81 | + start_times.append(start) | ||
| 82 | + end_times.append(start + duration) | ||
| 83 | + intervals.append((start, start + duration)) | ||
| 84 | + duration_sum += duration | ||
| 85 | + total_cost += duration + wait | ||
| 86 | + wall_us = max(end_times) - min(start_times) if start_times else 0 | ||
| 87 | + return wall_us, intervals, duration_sum, total_cost | ||
| 88 | + | ||
| 89 | + | ||
| 90 | +def compute_node_metrics(op_indices: List[int], operators: List[dict], multiplier: int = 1) -> dict: | ||
| 91 | + """计算单个节点的四维指标及衍生指标。""" | ||
| 92 | + empty_result = { | ||
| 93 | + 'wall_ms': 0.0, 'busy_union_ms': 0.0, 'kernel_sum_ms': 0.0, 'total_cost_ms': 0.0, | ||
| 94 | + 'parallelism': '-', 'bubble_ms': 0.0, 'kernel_count': 0, 'diagnosis': '无算子' | ||
| 95 | + } | ||
| 96 | + if not op_indices: | ||
| 97 | + return empty_result | ||
| 98 | + | ||
| 99 | + op_dict = {op['index']: op for op in operators} | ||
| 100 | + ops = [op_dict[i] for i in op_indices if i in op_dict] | ||
| 101 | + if not ops: | ||
| 102 | + return empty_result | ||
| 103 | + | ||
| 104 | + wall_us, intervals, duration_sum, total_cost = _accumulate_op_timing(ops) | ||
| 105 | + busy_union_us = merge_intervals(intervals) | ||
| 106 | + | ||
| 107 | + wall_ms = wall_us / 1000 * multiplier | ||
| 108 | + busy_union_ms = busy_union_us / 1000 * multiplier | ||
| 109 | + kernel_sum_ms = duration_sum / 1000 * multiplier | ||
| 110 | + total_cost_ms = total_cost / 1000 * multiplier | ||
| 111 | + | ||
| 112 | + parallelism = f"{kernel_sum_ms / wall_ms:.1f}×" if wall_ms > 0.001 else '-' | ||
| 113 | + bubble_ms = round(wall_ms - busy_union_ms, 3) | ||
| 114 | + diagnosis = generate_diagnosis(wall_ms, busy_union_ms, kernel_sum_ms, total_cost_ms) | ||
| 115 | + | ||
| 116 | + return { | ||
| 117 | + 'wall_ms': round(wall_ms, 3), | ||
| 118 | + 'busy_union_ms': round(busy_union_ms, 3), | ||
| 119 | + 'kernel_sum_ms': round(kernel_sum_ms, 3), | ||
| 120 | + 'total_cost_ms': round(total_cost_ms, 3), | ||
| 121 | + 'parallelism': parallelism, | ||
| 122 | + 'bubble_ms': bubble_ms, | ||
| 123 | + 'kernel_count': len(ops) * multiplier, | ||
| 124 | + 'diagnosis': diagnosis | ||
| 125 | + } | ||
| 126 | + | ||
| 127 | + | ||
| 128 | +_DIAG_EPS = 0.001 | ||
| 129 | + | ||
| 130 | + | ||
| 131 | +def _diag_parallelism(wall, kernel_sum): | ||
| 132 | + if kernel_sum > wall * 1.5: | ||
| 133 | + ratio = kernel_sum / wall if wall > _DIAG_EPS else 0 | ||
| 134 | + return f"高流并行度(kernel_sum/wall={ratio:.1f}×)" | ||
| 135 | + if kernel_sum > wall * 1.2: | ||
| 136 | + ratio = kernel_sum / wall if wall > _DIAG_EPS else 0 | ||
| 137 | + return f"中等流并行(kernel_sum/wall={ratio:.1f}×)" | ||
| 138 | + return None | ||
| 139 | + | ||
| 140 | + | ||
| 141 | +def _diag_bubble(wall, busy_union): | ||
| 142 | + if wall > busy_union * 1.5: | ||
| 143 | + gap_ratio = (wall - busy_union) / wall * 100 if wall > _DIAG_EPS else 0 | ||
| 144 | + return f"存在间隙气泡(gap={gap_ratio:.0f}%)" | ||
| 145 | + return None | ||
| 146 | + | ||
| 147 | + | ||
| 148 | +def _diag_wait(kernel_sum, total_cost): | ||
| 149 | + wait_time = total_cost - kernel_sum | ||
| 150 | + if wait_time > 0.1 and kernel_sum > _DIAG_EPS and total_cost > kernel_sum * 1.3: | ||
| 151 | + wait_ratio = wait_time / kernel_sum * 100 | ||
| 152 | + if wait_ratio < 1000: | ||
| 153 | + return f"等待时间显著(wait占比{wait_ratio:.0f}%)" | ||
| 154 | + return "等待时间显著(kernel极短但wait长)" | ||
| 155 | + if wait_time > kernel_sum * 2 and kernel_sum < 0.1: | ||
| 156 | + return "等待时间显著(kernel极短但wait长)" | ||
| 157 | + return None | ||
| 158 | + | ||
| 159 | + | ||
| 160 | +def _diag_utilization(wall, busy_union): | ||
| 161 | + if busy_union > 0 and wall > 0: | ||
| 162 | + utilization = busy_union / wall * 100 | ||
| 163 | + if utilization > 95: | ||
| 164 | + return None | ||
| 165 | + if utilization > 80: | ||
| 166 | + return f"利用率良好({utilization:.0f}%)" | ||
| 167 | + return f"利用率偏低({utilization:.0f}%)" | ||
| 168 | + return None | ||
| 169 | + | ||
| 170 | + | ||
| 171 | +def generate_diagnosis(wall: float, busy_union: float, kernel_sum: float, total_cost: float) -> str: | ||
| 172 | + """根据四指标生成诊断结论(中文)。规则见各 _diag_* 子函数。""" | ||
| 173 | + if wall == 0 and kernel_sum == 0: | ||
| 174 | + return "无数据" | ||
| 175 | + | ||
| 176 | + candidate_diagnoses = ( | ||
| 177 | + _diag_parallelism(wall, kernel_sum), | ||
| 178 | + _diag_bubble(wall, busy_union), | ||
| 179 | + _diag_wait(kernel_sum, total_cost), | ||
| 180 | + _diag_utilization(wall, busy_union), | ||
| 181 | + ) | ||
| 182 | + diagnoses = [d for d in candidate_diagnoses if d] | ||
| 183 | + | ||
| 184 | + if not diagnoses: | ||
| 185 | + if abs(wall - busy_union) < wall * 0.1 and abs(kernel_sum - wall) < wall * 0.1: | ||
| 186 | + return "干净顺序执行" | ||
| 187 | + return "正常执行" | ||
| 188 | + | ||
| 189 | + return ";".join(diagnoses) | ||
| 190 | + | ||
| 191 | + | ||
| 192 | +def collect_all_op_indices(node: dict) -> List[int]: | ||
| 193 | + """ | ||
| 194 | + 递归收集节点及其子节点的所有 op_indices | ||
| 195 | + """ | ||
| 196 | + indices = list(node.get('op_indices', [])) | ||
| 197 | + for child in node.get('children', []): | ||
| 198 | + indices.extend(collect_all_op_indices(child)) | ||
| 199 | + return indices | ||
| 200 | + | ||
| 201 | + | ||
| 202 | +def collect_tree_nodes(node: dict, ctx: TreeWalkCtx, multiplier: int, | ||
| 203 | + current_depth: int, parent_path: str = "") -> List[dict]: | ||
| 204 | + """ | ||
| 205 | + 递归收集树中所有节点及其指标 | ||
| 206 | + """ | ||
| 207 | + results = [] | ||
| 208 | + name = node.get('name', 'Unknown') | ||
| 209 | + path = f"{parent_path}/{name}" if parent_path else name | ||
| 210 | + | ||
| 211 | + all_op_indices = collect_all_op_indices(node) | ||
| 212 | + metrics = compute_node_metrics(all_op_indices, ctx.operators, multiplier) | ||
| 213 | + | ||
| 214 | + results.append({ | ||
| 215 | + 'name': name, | ||
| 216 | + 'path': path, | ||
| 217 | + 'depth': current_depth, | ||
| 218 | + 'multiplier': multiplier, | ||
| 219 | + **metrics | ||
| 220 | + }) | ||
| 221 | + | ||
| 222 | + if current_depth < ctx.max_depth: | ||
| 223 | + children = node.get('children', []) | ||
| 224 | + for child in children: | ||
| 225 | + child_results = collect_tree_nodes( | ||
| 226 | + child, ctx, 1, current_depth + 1, path | ||
| 227 | + ) | ||
| 228 | + results.extend(child_results) | ||
| 229 | + | ||
| 230 | + return results | ||
| 231 | + | ||
| 232 | + | ||
| 233 | +def categorize_nodes(nodes: List[dict], config: dict) -> Tuple[List[dict], List[dict]]: | ||
| 234 | + """ | ||
| 235 | + 将节点分为主要模块(layer_structure)和辅助模块(stages + runtime_auxiliary) | ||
| 236 | + """ | ||
| 237 | + layer_type_names = set(config.get('layer_types', {}).keys()) | ||
| 238 | + stage_names = set() | ||
| 239 | + for stage_name in config.get('stages', {}).keys(): | ||
| 240 | + stage_names.add(stage_name) | ||
| 241 | + stage_info = config.get('stages', {}).get(stage_name, {}) | ||
| 242 | + stage_names.add(stage_info.get('name', stage_name)) | ||
| 243 | + | ||
| 244 | + main_nodes = [] | ||
| 245 | + aux_nodes = [] | ||
| 246 | + | ||
| 247 | + for node in nodes: | ||
| 248 | + path = node.get('path', '') | ||
| 249 | + name = node.get('name', '') | ||
| 250 | + | ||
| 251 | + is_main = any(lt in path or lt == name for lt in layer_type_names) | ||
| 252 | + is_aux = any(s in path or s == name for s in stage_names) | ||
| 253 | + | ||
| 254 | + if 'runtime_auxiliary' in path or 'runtime' in name.lower(): | ||
| 255 | + aux_nodes.append(node) | ||
| 256 | + elif is_main: | ||
| 257 | + main_nodes.append(node) | ||
| 258 | + elif is_aux: | ||
| 259 | + aux_nodes.append(node) | ||
| 260 | + elif node.get('depth', 0) <= 1: | ||
| 261 | + aux_nodes.append(node) | ||
| 262 | + | ||
| 263 | + return main_nodes, aux_nodes | ||
| 264 | + | ||
| 265 | + | ||
| 266 | +def compute_step_wall_ms(operators: List[dict]) -> float: | ||
| 267 | + """ | ||
| 268 | + 计算整个 step 的 wall_ms(所有 operators 的时间跨度) | ||
| 269 | + """ | ||
| 270 | + if not operators: | ||
| 271 | + return 0.0 | ||
| 272 | + min_start = min(op.get('start_time_us', 0) for op in operators) | ||
| 273 | + max_end = max(op.get('start_time_us', 0) + op.get('duration_us', 0) for op in operators) | ||
| 274 | + return (max_end - min_start) / 1000 | ||
| 275 | + | ||
| 276 | + | ||
| 277 | +def generate_metrics_table(nodes: List[dict], title: str, total_wall_ms: float) -> str: | ||
| 278 | + """ | ||
| 279 | + 生成 Markdown 指标表格 | ||
| 280 | + """ | ||
| 281 | + if not nodes: | ||
| 282 | + return f"### {title}\n\n无数据\n" | ||
| 283 | + | ||
| 284 | + lines = [ | ||
| 285 | + f"### {title}", | ||
| 286 | + "", | ||
| 287 | + "| 节点 | wall_ms | busy_union_ms | kernel_sum_ms | total_cost_ms | 并行度 | bubble_ms | 占比% | kernel数 | 诊断结论 |", | ||
| 288 | + ("|------|---------|---------------|---------------|---------------|" | ||
| 289 | + "--------|-----------|-------|----------|----------|") | ||
| 290 | + ] | ||
| 291 | + | ||
| 292 | + for node in nodes: | ||
| 293 | + name = node.get('name', 'Unknown') | ||
| 294 | + depth = node.get('depth', 0) | ||
| 295 | + multiplier = node.get('multiplier', 1) | ||
| 296 | + | ||
| 297 | + indent = " " * (depth - 1) | ||
| 298 | + display_name = f"{indent}{name}" | ||
| 299 | + if multiplier > 1 and depth <= 1: | ||
| 300 | + display_name = f"{display_name} ×{multiplier}" | ||
| 301 | + | ||
| 302 | + wall = node.get('wall_ms', 0) | ||
| 303 | + busy = node.get('busy_union_ms', 0) | ||
| 304 | + ksum = node.get('kernel_sum_ms', 0) | ||
| 305 | + tcost = node.get('total_cost_ms', 0) | ||
| 306 | + parallelism = node.get('parallelism', '-') | ||
| 307 | + bubble = node.get('bubble_ms', 0) | ||
| 308 | + ratio_pct = f"{wall / total_wall_ms * 100:.1f}" if total_wall_ms > 0.001 else '-' | ||
| 309 | + kcount = node.get('kernel_count', 0) | ||
| 310 | + diag = node.get('diagnosis', '') | ||
| 311 | + | ||
| 312 | + lines.append( | ||
| 313 | + f"| {display_name} | {wall:.3f} | {busy:.3f} | {ksum:.3f} | {tcost:.3f} " | ||
| 314 | + f"| {parallelism} | {bubble:.3f} | {ratio_pct} | {kcount} | {diag} |" | ||
| 315 | + ) | ||
| 316 | + | ||
| 317 | + lines.append("") | ||
| 318 | + return "\n".join(lines) | ||
| 319 | + | ||
| 320 | + | ||
| 321 | +_METRICS_DOC_LINES = [ | ||
| 322 | + "## 指标说明", | ||
| 323 | + "", | ||
| 324 | + "### 四维基础指标", | ||
| 325 | + "", | ||
| 326 | + "| 指标 | 定义 | 含义 |", | ||
| 327 | + "|------|------|------|", | ||
| 328 | + "| wall_ms | 最后kernel结束 - 首个kernel开始 | 实际墙上时钟耗时(含间隙) |", | ||
| 329 | + "| busy_union_ms | 合并后的设备忙碌时间 | 设备实际利用率(去重叠) |", | ||
| 330 | + "| kernel_sum_ms | 所有kernel时长的算术和 | 总计算量(忽略重叠) |", | ||
| 331 | + "| total_cost_ms | Σ(duration + wait) | 完整成本(含等待) |", | ||
| 332 | + "", | ||
| 333 | + "### 衍生指标", | ||
| 334 | + "", | ||
| 335 | + "| 指标 | 计算 | 含义 |", | ||
| 336 | + "|------|------|------|", | ||
| 337 | + "| 并行度 | kernel_sum_ms / wall_ms | 多流并行执行倍数(>1×表示有并行) |", | ||
| 338 | + "| bubble_ms | wall_ms - busy_union_ms | 设备空闲气泡时间 |", | ||
| 339 | + "| 占比% | wall_ms / step总wall_ms × 100 | 占整个 step 墙上时钟的比例 |", | ||
| 340 | + "", | ||
| 341 | + "## 诊断规则", | ||
| 342 | + "", | ||
| 343 | + "| 条件 | 阈值 | 诊断结论 |", | ||
| 344 | + "|------|------|----------|", | ||
| 345 | + "| kernel_sum > wall | > 1.5× | 高流并行度(多流重叠执行) |", | ||
| 346 | + "| kernel_sum > wall | > 1.2× | 中等流并行 |", | ||
| 347 | + "| wall > busy_union | > 1.5× | 存在间隙气泡 |", | ||
| 348 | + "| total_cost > kernel_sum | > 1.3× | 等待时间显著,检查 wait-anchor 热点 |", | ||
| 349 | + "| busy_union / wall | 80%~95% | 利用率良好 |", | ||
| 350 | + "| busy_union / wall | < 80% | 利用率偏低 |", | ||
| 351 | + "| busy_union ≈ wall ≈ kernel_sum | 偏差 < 10% | 干净顺序执行 |", | ||
| 352 | + "| 其他 | — | 正常执行 |", | ||
| 353 | + "", | ||
| 354 | + "---", | ||
| 355 | + "", | ||
| 356 | +] | ||
| 357 | + | ||
| 358 | + | ||
| 359 | +def _collect_metric_nodes(config, walk_ctx): | ||
| 360 | + """从 stages / layer_structure / runtime_auxiliary 收集全部带指标的节点。""" | ||
| 361 | + all_nodes = [] | ||
| 362 | + for _sname, sinfo in config.get('stages', {}).items(): | ||
| 363 | + stage_indices = sinfo.get('stage_indices', [0]) | ||
| 364 | + all_nodes.extend(collect_tree_nodes(sinfo, walk_ctx, | ||
| 365 | + len(stage_indices) if stage_indices else 1, 1)) | ||
| 366 | + | ||
| 367 | + layer_types = config.get('layer_types', {}) | ||
| 368 | + layer_structure = config.get('layer_structure', {}) | ||
| 369 | + for layer_type in layer_types.keys(): | ||
| 370 | + structure = layer_structure.get(layer_type, {}) | ||
| 371 | + if not structure: | ||
| 372 | + continue | ||
| 373 | + layer_indices = layer_types.get(layer_type, {}).get('layer_indices', []) | ||
| 374 | + all_nodes.extend(collect_tree_nodes(structure, walk_ctx, len(layer_indices), 1)) | ||
| 375 | + | ||
| 376 | + for aux in config.get('runtime_auxiliary', []): | ||
| 377 | + instance_indices = aux.get('instance_indices', [0]) | ||
| 378 | + all_nodes.extend(collect_tree_nodes(aux, walk_ctx, | ||
| 379 | + len(instance_indices) if instance_indices else 1, 1)) | ||
| 380 | + return all_nodes | ||
| 381 | + | ||
| 382 | + | ||
| 383 | +def _dedupe_and_sort_nodes(nodes): | ||
| 384 | + """按 (path, depth) 去重并排序。""" | ||
| 385 | + seen = set() | ||
| 386 | + unique = [] | ||
| 387 | + for n in nodes: | ||
| 388 | + key = (n.get('path', ''), n.get('depth', 0)) | ||
| 389 | + if key not in seen: | ||
| 390 | + seen.add(key) | ||
| 391 | + unique.append(n) | ||
| 392 | + return sorted(unique, key=lambda x: (x.get('depth', 0), x.get('path', ''))) | ||
| 393 | + | ||
| 394 | + | ||
| 395 | +def generate_metrics_report(raw_ops: dict, config: dict, operators: List[dict], | ||
| 396 | + max_depth: int = 3) -> str: | ||
| 397 | + """生成完整的指标报告。""" | ||
| 398 | + model_name = config.get('model_name', 'Model') | ||
| 399 | + step_id = raw_ops.get('step_id', 'N/A') | ||
| 400 | + total_kernels = raw_ops.get('kernel_count', len(operators)) | ||
| 401 | + total_wall_ms = compute_step_wall_ms(operators) | ||
| 402 | + | ||
| 403 | + walk_ctx = TreeWalkCtx(operators=operators, max_depth=max_depth) | ||
| 404 | + all_nodes = _collect_metric_nodes(config, walk_ctx) | ||
| 405 | + main_nodes, aux_nodes = categorize_nodes(all_nodes, config) | ||
| 406 | + main_nodes = _dedupe_and_sort_nodes(main_nodes) | ||
| 407 | + aux_nodes = _dedupe_and_sort_nodes(aux_nodes) | ||
| 408 | + | ||
| 409 | + lines = [ | ||
| 410 | + f"# {model_name} 性能指标分析", | ||
| 411 | + "", | ||
| 412 | + f"**Step**: {step_id} | **总Kernel数**: {total_kernels} | **Step wall_ms**: {total_wall_ms:.3f}", | ||
| 413 | + "", | ||
| 414 | + ] | ||
| 415 | + lines.extend(_METRICS_DOC_LINES) | ||
| 416 | + | ||
| 417 | + if aux_nodes: | ||
| 418 | + lines.append(generate_metrics_table(aux_nodes, "辅助模块指标(stages / runtime)", total_wall_ms)) | ||
| 419 | + if main_nodes: | ||
| 420 | + lines.append(generate_metrics_table(main_nodes, "主要模块指标(decoder layers)", total_wall_ms)) | ||
| 421 | + | ||
| 422 | + return "\n".join(lines) | ||
| 423 | + | ||
| 424 | + | ||
| 425 | +def main(): | ||
| 426 | + logging.basicConfig(level=logging.INFO, format='%(message)s', stream=sys.stdout) | ||
| 427 | + parser = argparse.ArgumentParser( | ||
| 428 | + description='NPU 性能指标计算 - Step 4', | ||
| 429 | + formatter_class=argparse.RawDescriptionHelpFormatter, | ||
| 430 | + epilog=''' | ||
| 431 | +示例: | ||
| 432 | + %(prog)s -r raw_ops_details.json -c analysis_config.json -o metrics_report.md | ||
| 433 | + %(prog)s -r raw_ops_details.json -c analysis_config.json -o metrics_report.md -d 4 | ||
| 434 | + ''' | ||
| 435 | + ) | ||
| 436 | + | ||
| 437 | + parser.add_argument('-r', '--raw-ops', required=True, metavar='FILE', | ||
| 438 | + help='raw_ops_details.json 文件路径') | ||
| 439 | + parser.add_argument('-c', '--config', required=True, metavar='FILE', | ||
| 440 | + help='analysis_config.json 文件路径') | ||
| 441 | + parser.add_argument('-o', '--output', metavar='FILE', | ||
| 442 | + help='输出 Markdown 文件路径(默认打印到标准输出)') | ||
| 443 | + parser.add_argument('-d', '--depth', type=int, default=3, | ||
| 444 | + help='指标计算深度(默认: 3)') | ||
| 445 | + | ||
| 446 | + args = parser.parse_args() | ||
| 447 | + | ||
| 448 | + try: | ||
| 449 | + raw_ops_file = validate_file_exists(args.raw_ops) | ||
| 450 | + config_file = validate_file_exists(args.config) | ||
| 451 | + | ||
| 452 | + raw_ops = load_json(raw_ops_file) | ||
| 453 | + config = load_json(config_file) | ||
| 454 | + | ||
| 455 | + operators = raw_ops.get('operators', []) | ||
| 456 | + | ||
| 457 | + report = generate_metrics_report(raw_ops, config, operators, args.depth) | ||
| 458 | + | ||
| 459 | + if args.output: | ||
| 460 | + with open(args.output, 'w', encoding='utf-8') as f: | ||
| 461 | + f.write(report) | ||
| 462 | + logger.info("指标报告已生成: %s", args.output) | ||
| 463 | + else: | ||
| 464 | + logger.info(report) | ||
| 465 | + | ||
| 466 | + except (FileNotFoundError, ValueError) as e: | ||
| 467 | + logger.error("错误: %s", e) | ||
| 468 | + sys.exit(1) | ||
| 469 | + except Exception as e: | ||
| 470 | + logger.error("未知错误: %s", e) | ||
| 471 | + sys.exit(1) | ||
| 472 | + | ||
| 473 | + | ||
| 474 | +if __name__ == '__main__': | ||
| 475 | + main() | ||
| @@ -0,0 +1,1844 @@ | |||
| 1 | +#!/usr/bin/env python3 | ||
| 2 | +# -*- coding: utf-8 -*- | ||
| 3 | +# ---------------------------------------------------------------------------- | ||
| 4 | +# Copyright (c) 2025 Huawei Technologies Co., Ltd. | ||
| 5 | +# | ||
| 6 | +# Licensed under the Apache License, Version 2.0 (the "License"); | ||
| 7 | +# you may not use this file except in compliance with the License. | ||
| 8 | +# You may obtain a copy of the License at | ||
| 9 | +# | ||
| 10 | +# http://www.apache.org/licenses/LICENSE-2.0 | ||
| 11 | +# | ||
| 12 | +# Unless required by applicable law or agreed to in writing, software | ||
| 13 | +# distributed under the License is distributed on an "AS IS" BASIS, | ||
| 14 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| 15 | +# See the License for the specific language governing permissions and | ||
| 16 | +# limitations under the License. | ||
| 17 | +# ---------------------------------------------------------------------------- | ||
| 18 | +"""NPU 性能拆解报告生成脚本""" | ||
| 19 | + | ||
| 20 | +import logging | ||
| 21 | +import argparse | ||
| 22 | +import sys | ||
| 23 | +from dataclasses import dataclass | ||
| 24 | +from pathlib import Path | ||
| 25 | +from typing import List, Tuple | ||
| 26 | + | ||
| 27 | +from _common import validate_file_exists, load_json | ||
| 28 | +from _assets import HTML_CSS, JS_TEMPLATE | ||
| 29 | + | ||
| 30 | +logger = logging.getLogger(__name__) | ||
| 31 | + | ||
| 32 | + | ||
| 33 | + | ||
| 34 | +class HtmlRenderCtx: | ||
| 35 | + """HTML 树渲染过程中保持不变的上下文(降低单函数参数个数)。""" | ||
| 36 | + operators: list | ||
| 37 | + total_duration: float | ||
| 38 | + max_depth: int | ||
| 39 | + kernel_semantics: dict = None | ||
| 40 | + kernel_display_fields: list = None | ||
| 41 | + | ||
| 42 | + | ||
| 43 | + | ||
| 44 | +class KernelRenderCtx: | ||
| 45 | + """单个 kernel 叶节点渲染所需的上下文(降低 helper 参数个数)。""" | ||
| 46 | + ctx: 'HtmlRenderCtx' | ||
| 47 | + multiplier: int | ||
| 48 | + current_depth: int | ||
| 49 | + display_fields: list | ||
| 50 | + | ||
| 51 | + | ||
| 52 | + | ||
| 53 | +class NodeHeaderInfo: | ||
| 54 | + """模块节点头部计算所需的输入。""" | ||
| 55 | + time_info: dict | ||
| 56 | + kernel_count: int | ||
| 57 | + multiplier: int | ||
| 58 | + current_depth: int | ||
| 59 | + max_depth: int | ||
| 60 | + is_leaf: bool | ||
| 61 | + is_kernel_only: bool | ||
| 62 | + | ||
| 63 | + | ||
| 64 | + | ||
| 65 | +class NodeHeaderView: | ||
| 66 | + """模块节点头部渲染所需的展示字段。""" | ||
| 67 | + name: str | ||
| 68 | + count_str: str | ||
| 69 | + node_class: str | ||
| 70 | + current_depth: int | ||
| 71 | + data_category: str | ||
| 72 | + kernel_count_str: str | ||
| 73 | + stream_count_str: str | ||
| 74 | + duration_hover: str | ||
| 75 | + duration: float | ||
| 76 | + percentage: str | ||
| 77 | + | ||
| 78 | + | ||
| 79 | + | ||
| 80 | +class ReportMeta: | ||
| 81 | + """HTML 报告 metadata 区所需字段。""" | ||
| 82 | + generate_time: str | ||
| 83 | + total_duration: float | ||
| 84 | + step_id: object | ||
| 85 | + kernel_count: int | ||
| 86 | + raw_ops_path: str | ||
| 87 | + config_path: str | ||
| 88 | + architecture_desc: str | ||
| 89 | + | ||
| 90 | + | ||
| 91 | + | ||
| 92 | +class TimelineNodeState: | ||
| 93 | + """generate_timeline_data.process_node 的递归状态。""" | ||
| 94 | + multiplier: int = 1 | ||
| 95 | + depth: int = 0 | ||
| 96 | + parent_index: int = -1 | ||
| 97 | + is_auxiliary: bool = False | ||
| 98 | + | ||
| 99 | + | ||
| 100 | + | ||
| 101 | +class _TimelineBuildCtx: | ||
| 102 | + """generate_timeline_data 递归构建过程中共享的可变状态。""" | ||
| 103 | + op_dict: dict | ||
| 104 | + timeline_nodes: list | ||
| 105 | + node_color_map: dict | ||
| 106 | + | ||
| 107 | + def assign_color(self, path: str) -> str: | ||
| 108 | + parts = path.split('/') | ||
| 109 | + key = parts[-1] if len(parts) > 1 else parts[0] | ||
| 110 | + if key not in self.node_color_map: | ||
| 111 | + self.node_color_map[key] = TIMELINE_PALETTE[len(self.node_color_map) % len(TIMELINE_PALETTE)] | ||
| 112 | + return self.node_color_map[key] | ||
| 113 | + | ||
| 114 | + | ||
| 115 | +def _timeline_process_node(node, parent_path, ctx, state=None): | ||
| 116 | + """递归构建单个 timeline 节点;返回其在 timeline_nodes 中的索引,无算子返回 -1。""" | ||
| 117 | + if state is None: | ||
| 118 | + state = TimelineNodeState() | ||
| 119 | + name = node.get('name', 'Unknown') | ||
| 120 | + path = f"{parent_path}/{name}" if parent_path else name | ||
| 121 | + color = ctx.assign_color(path) | ||
| 122 | + | ||
| 123 | + all_ops_indices = _collect_node_op_indices_all(node) | ||
| 124 | + all_ops = [ctx.op_dict[i] for i in all_ops_indices if i in ctx.op_dict] | ||
| 125 | + if not all_ops: | ||
| 126 | + return -1 | ||
| 127 | + | ||
| 128 | + stream_set = {str(op.get('stream_id', '0')) for op in all_ops} | ||
| 129 | + min_start = min(op.get('start_time_us', 0) for op in all_ops) | ||
| 130 | + max_end = max(op.get('start_time_us', 0) + op.get('duration_us', 0) for op in all_ops) | ||
| 131 | + per_stream_ops = _build_per_stream_ops(all_ops_indices, ctx.op_dict) | ||
| 132 | + current_index = len(ctx.timeline_nodes) | ||
| 133 | + stream_counts = {sid: len(ops) for sid, ops in per_stream_ops.items()} | ||
| 134 | + dominant_stream = max(stream_counts.items(), key=lambda x: x[1])[0] if stream_counts else '0' | ||
| 135 | + | ||
| 136 | + ctx.timeline_nodes.append({ | ||
| 137 | + 'name': name, 'path': path, 'color': color, 'multiplier': state.multiplier, | ||
| 138 | + 'start': min_start, 'end': max_end, 'duration': max_end - min_start, | ||
| 139 | + 'kernel_count': len(all_ops), 'streams': sorted(stream_set), | ||
| 140 | + 'dominant_stream': dominant_stream, 'per_stream_ops': per_stream_ops, | ||
| 141 | + 'op_indices': all_ops_indices, 'depth': state.depth, | ||
| 142 | + 'parent_index': state.parent_index, 'children_indices': [], | ||
| 143 | + 'has_children': False, 'category': 'auxiliary' if state.is_auxiliary else '', | ||
| 144 | + }) | ||
| 145 | + | ||
| 146 | + child_indices = [] | ||
| 147 | + for child in node.get('children', []): | ||
| 148 | + child_state = TimelineNodeState(multiplier=state.multiplier, depth=state.depth + 1, | ||
| 149 | + parent_index=current_index, is_auxiliary=state.is_auxiliary) | ||
| 150 | + child_idx = _timeline_process_node(child, path, ctx, child_state) | ||
| 151 | + if child_idx >= 0: | ||
| 152 | + child_indices.append(child_idx) | ||
| 153 | + ctx.timeline_nodes[current_index]['children_indices'] = child_indices | ||
| 154 | + ctx.timeline_nodes[current_index]['has_children'] = len(child_indices) > 0 | ||
| 155 | + return current_index | ||
| 156 | + | ||
| 157 | + | ||
| 158 | + | ||
| 159 | +class ReportOptions: | ||
| 160 | + """报告生成选项(降低 generate_report / generate_html_report 参数个数)。""" | ||
| 161 | + output_path: str = None | ||
| 162 | + depth: int = 3 | ||
| 163 | + html: bool = False | ||
| 164 | + html_output: str = None | ||
| 165 | + theme: str = 'vscode-dark' | ||
| 166 | + kernel_display_fields: list = None | ||
| 167 | + raw_ops_path: str = '' | ||
| 168 | + config_path: str = '' | ||
| 169 | + | ||
| 170 | + | ||
| 171 | +def validate_raw_ops(data: dict) -> None: | ||
| 172 | + required = ['step_id', 'total_duration_us', 'kernel_count', 'operators'] | ||
| 173 | + missing = [k for k in required if k not in data] | ||
| 174 | + if missing: | ||
| 175 | + raise ValueError(f"raw_ops.json 缺少必要字段: {missing}") | ||
| 176 | + | ||
| 177 | + for i, op in enumerate(data.get('operators', [])): | ||
| 178 | + op_required = ['index', 'duration_us'] | ||
| 179 | + op_missing = [k for k in op_required if k not in op] | ||
| 180 | + if op_missing: | ||
| 181 | + raise ValueError(f"operators[{i}] 缺少必要字段: {op_missing}") | ||
| 182 | + | ||
| 183 | + | ||
| 184 | +def validate_analysis_config(data: dict) -> None: | ||
| 185 | + required = ['model_name', 'layer_types', 'layer_structure'] | ||
| 186 | + missing = [k for k in required if k not in data] | ||
| 187 | + if missing: | ||
| 188 | + raise ValueError(f"analysis_config.json 缺少必要字段: {missing}") | ||
| 189 | + | ||
| 190 | + | ||
| 191 | +def get_duration_by_indices(operators: list, indices: list) -> float: | ||
| 192 | + total = 0.0 | ||
| 193 | + op_dict = {op['index']: op for op in operators} | ||
| 194 | + for idx in indices: | ||
| 195 | + if idx in op_dict: | ||
| 196 | + total += op_dict[idx].get('duration_us', 0) | ||
| 197 | + return total | ||
| 198 | + | ||
| 199 | + | ||
| 200 | +def get_kernels_by_indices(operators: list, indices: list) -> list: | ||
| 201 | + op_dict = {op['index']: op for op in operators} | ||
| 202 | + kernels = [] | ||
| 203 | + for idx in indices: | ||
| 204 | + if idx in op_dict: | ||
| 205 | + op = op_dict[idx] | ||
| 206 | + name = op.get('normalized_name') or op.get('type') or op.get('name', 'Unknown') | ||
| 207 | + kernels.append(name) | ||
| 208 | + return kernels | ||
| 209 | + | ||
| 210 | + | ||
| 211 | +def get_kernel_details_by_indices(operators: list, indices: list) -> list: | ||
| 212 | + op_dict = {op['index']: op for op in operators} | ||
| 213 | + kernels = [] | ||
| 214 | + for idx in indices: | ||
| 215 | + if idx in op_dict: | ||
| 216 | + op = op_dict[idx] | ||
| 217 | + name = op.get('normalized_name') or op.get('type') or op.get('name', 'Unknown') | ||
| 218 | + kernels.append({ | ||
| 219 | + 'name': name, | ||
| 220 | + 'duration': op.get('duration_us', 0), | ||
| 221 | + 'index': idx | ||
| 222 | + }) | ||
| 223 | + kernels.sort(key=lambda x: -x['duration']) | ||
| 224 | + return kernels | ||
| 225 | + | ||
| 226 | + | ||
| 227 | +def get_kernel_full_details_by_indices(operators: list, indices: list) -> list: | ||
| 228 | + op_dict = {op['index']: op for op in operators} | ||
| 229 | + kernels = [] | ||
| 230 | + for idx in indices: | ||
| 231 | + if idx in op_dict: | ||
| 232 | + op = op_dict[idx] | ||
| 233 | + entry = { | ||
| 234 | + 'index': idx, | ||
| 235 | + 'duration': op.get('duration_us', 0), | ||
| 236 | + 'duration_raw': op.get('duration_us_raw', str(op.get('duration_us', 0))), | ||
| 237 | + 'start_time': op.get('start_time_us', 0), | ||
| 238 | + 'start_time_raw': op.get('start_time_us_raw', str(op.get('start_time_us', 0))), | ||
| 239 | + 'stream_id': op.get('stream_id', ''), | ||
| 240 | + 'all_fields': dict(op) | ||
| 241 | + } | ||
| 242 | + if 'normalized_name' in op: | ||
| 243 | + entry['name'] = op.get('normalized_name', 'Unknown') | ||
| 244 | + entry['original_name'] = op.get('original_name', '') | ||
| 245 | + entry['task_type'] = op.get('task_type', '') | ||
| 246 | + entry['input_shapes'] = op.get('input_shapes', '') | ||
| 247 | + entry['output_shapes'] = op.get('output_shapes', '') | ||
| 248 | + else: | ||
| 249 | + entry['name'] = op.get('type', op.get('name', 'Unknown')) | ||
| 250 | + entry['original_name'] = op.get('name', '') | ||
| 251 | + entry['task_type'] = op.get('type', '') | ||
| 252 | + entry['input_shapes'] = op.get('input_shapes', '') | ||
| 253 | + entry['output_shapes'] = op.get('output_shapes', '') | ||
| 254 | + entry['input_data_types'] = op.get('input_data_types', '') | ||
| 255 | + entry['output_data_types'] = op.get('output_data_types', '') | ||
| 256 | + entry['input_formats'] = op.get('input_formats', '') | ||
| 257 | + entry['output_formats'] = op.get('output_formats', '') | ||
| 258 | + kernels.append(entry) | ||
| 259 | + kernels.sort(key=lambda x: x['start_time']) | ||
| 260 | + return kernels | ||
| 261 | + | ||
| 262 | + | ||
| 263 | +def collect_kernel_semantics(config: dict) -> dict: | ||
| 264 | + semantics = {} | ||
| 265 | + | ||
| 266 | + def _collect_from_node(node: dict, parent_path: str = ''): | ||
| 267 | + name = node.get('name', '') | ||
| 268 | + path = f"{parent_path}/{name}" if parent_path else name | ||
| 269 | + | ||
| 270 | + for ks in node.get('kernels', []): | ||
| 271 | + idx = ks.get('index') | ||
| 272 | + if idx is not None: | ||
| 273 | + semantics[idx] = { | ||
| 274 | + 'semantic': ks.get('semantic', '') or ks.get('comment', ''), | ||
| 275 | + 'shape_semantic': ks.get('shape_semantic', ''), | ||
| 276 | + 'code_ref': ks.get('code_ref', ''), | ||
| 277 | + 'path': path | ||
| 278 | + } | ||
| 279 | + | ||
| 280 | + for child in node.get('children', []): | ||
| 281 | + _collect_from_node(child, path) | ||
| 282 | + | ||
| 283 | + for _stage_name, stage_info in config.get('stages', {}).items(): | ||
| 284 | + _collect_from_node(stage_info) | ||
| 285 | + | ||
| 286 | + for _layer_type, structure in config.get('layer_structure', {}).items(): | ||
| 287 | + _collect_from_node(structure) | ||
| 288 | + | ||
| 289 | + for aux in config.get('runtime_auxiliary', []): | ||
| 290 | + _collect_from_node(aux) | ||
| 291 | + | ||
| 292 | + return semantics | ||
| 293 | + | ||
| 294 | + | ||
| 295 | +def format_field_display(key: str, value) -> str: | ||
| 296 | + if value is None or value == '' or value == 'N/A': | ||
| 297 | + return '' | ||
| 298 | + | ||
| 299 | + if isinstance(value, float): | ||
| 300 | + if '_ratio' in key: | ||
| 301 | + return f'{value * 100:.1f}%' | ||
| 302 | + if '_time' in key or key == 'duration_us' or key == 'start_time_us': | ||
| 303 | + return f'{value:.3f} us' | ||
| 304 | + if key.endswith('_pct') or key == 'cube_utilization_pct': | ||
| 305 | + return f'{value:.1f}%' | ||
| 306 | + return f'{value:.3f}' | ||
| 307 | + | ||
| 308 | + if isinstance(value, int): | ||
| 309 | + return str(value) | ||
| 310 | + | ||
| 311 | + s = str(value) | ||
| 312 | + if s.startswith('"') and s.endswith('"'): | ||
| 313 | + s = s[1:-1] | ||
| 314 | + return s | ||
| 315 | + | ||
| 316 | + | ||
| 317 | +FIELD_DISPLAY_NAMES = { | ||
| 318 | + 'name': 'Name', | ||
| 319 | + 'type': 'Type', | ||
| 320 | + 'duration_us': 'Duration', | ||
| 321 | + 'start_time_us': 'Start Time', | ||
| 322 | + 'stream_id': 'Stream', | ||
| 323 | + 'device_id': 'Device', | ||
| 324 | + 'task_id': 'Task ID', | ||
| 325 | + 'op_state': 'OP State', | ||
| 326 | + 'accelerator_core': 'Accelerator Core', | ||
| 327 | + 'wait_time_us': 'Wait Time', | ||
| 328 | + 'block_dim': 'Block Dim', | ||
| 329 | + 'mix_block_dim': 'Mix Block Dim', | ||
| 330 | + 'hf32_eligible': 'HF32 Eligible', | ||
| 331 | + 'input_shapes': 'Input Shapes', | ||
| 332 | + 'input_data_types': 'Input Types', | ||
| 333 | + 'input_formats': 'Input Formats', | ||
| 334 | + 'output_shapes': 'Output Shapes', | ||
| 335 | + 'output_data_types': 'Output Types', | ||
| 336 | + 'output_formats': 'Output Formats', | ||
| 337 | + 'context_id': 'Context ID', | ||
| 338 | + 'aicore_time_us': 'AI Core Time', | ||
| 339 | + 'aic_total_cycles': 'AI Core Cycles', | ||
| 340 | + 'aic_mac_time_us': 'MAC Time', | ||
| 341 | + 'aic_mac_ratio': 'MAC Ratio', | ||
| 342 | + 'aic_scalar_time_us': 'Scalar Time', | ||
| 343 | + 'aic_scalar_ratio': 'Scalar Ratio', | ||
| 344 | + 'aic_mte1_time_us': 'MTE1 Time', | ||
| 345 | + 'aic_mte1_ratio': 'MTE1 Ratio', | ||
| 346 | + 'aic_mte2_time_us': 'MTE2 Time', | ||
| 347 | + 'aic_mte2_ratio': 'MTE2 Ratio', | ||
| 348 | + 'aic_fixpipe_time_us': 'FixPipe Time', | ||
| 349 | + 'aic_fixpipe_ratio': 'FixPipe Ratio', | ||
| 350 | + 'aic_icache_miss_rate': 'AI Core ICache Miss', | ||
| 351 | + 'aiv_time_us': 'AI Vector Time', | ||
| 352 | + 'aiv_total_cycles': 'AI Vector Cycles', | ||
| 353 | + 'aiv_vec_time_us': 'Vector Time', | ||
| 354 | + 'aiv_vec_ratio': 'Vector Ratio', | ||
| 355 | + 'aiv_scalar_time_us': 'AIV Scalar Time', | ||
| 356 | + 'aiv_scalar_ratio': 'AIV Scalar Ratio', | ||
| 357 | + 'aiv_mte2_time_us': 'AIV MTE2 Time', | ||
| 358 | + 'aiv_mte2_ratio': 'AIV MTE2 Ratio', | ||
| 359 | + 'aiv_mte3_time_us': 'AIV MTE3 Time', | ||
| 360 | + 'aiv_mte3_ratio': 'AIV MTE3 Ratio', | ||
| 361 | + 'aiv_icache_miss_rate': 'AIV ICache Miss', | ||
| 362 | + 'cube_utilization_pct': 'Cube Utilization', | ||
| 363 | + 'aic_mac_fp16_ratio': 'MAC FP16 Ratio', | ||
| 364 | + 'aic_mac_int8_ratio': 'MAC INT8 Ratio', | ||
| 365 | + 'aic_cube_fops': 'Cube FLOPs', | ||
| 366 | + 'aiv_vec_fp32_ratio': 'Vec FP32 Ratio', | ||
| 367 | + 'aiv_vec_fp16_ratio': 'Vec FP16 Ratio', | ||
| 368 | + 'aiv_vec_int32_ratio': 'Vec INT32 Ratio', | ||
| 369 | + 'aiv_vec_misc_ratio': 'Vec Misc Ratio', | ||
| 370 | + 'aiv_vector_fops': 'Vector FLOPs', | ||
| 371 | + 'model_id': 'Model ID', | ||
| 372 | +} | ||
| 373 | + | ||
| 374 | +TOOLTIP_FIELD_ORDER = [ | ||
| 375 | + 'name', 'type', 'op_state', 'accelerator_core', | ||
| 376 | + 'duration_us', 'start_time_us', 'wait_time_us', | ||
| 377 | + 'stream_id', 'device_id', 'task_id', 'block_dim', 'mix_block_dim', | ||
| 378 | + 'input_shapes', 'input_data_types', 'input_formats', | ||
| 379 | + 'output_shapes', 'output_data_types', 'output_formats', | ||
| 380 | + 'hf32_eligible', 'context_id', | ||
| 381 | + 'aicore_time_us', 'aic_total_cycles', | ||
| 382 | + 'aic_mac_time_us', 'aic_mac_ratio', | ||
| 383 | + 'aic_scalar_time_us', 'aic_scalar_ratio', | ||
| 384 | + 'aic_mte1_time_us', 'aic_mte1_ratio', | ||
| 385 | + 'aic_mte2_time_us', 'aic_mte2_ratio', | ||
| 386 | + 'aic_fixpipe_time_us', 'aic_fixpipe_ratio', | ||
| 387 | + 'aic_icache_miss_rate', | ||
| 388 | + 'cube_utilization_pct', | ||
| 389 | + 'aic_mac_fp16_ratio', 'aic_mac_int8_ratio', 'aic_cube_fops', | ||
| 390 | + 'aiv_time_us', 'aiv_total_cycles', | ||
| 391 | + 'aiv_vec_time_us', 'aiv_vec_ratio', | ||
| 392 | + 'aiv_scalar_time_us', 'aiv_scalar_ratio', | ||
| 393 | + 'aiv_mte2_time_us', 'aiv_mte2_ratio', | ||
| 394 | + 'aiv_mte3_time_us', 'aiv_mte3_ratio', | ||
| 395 | + 'aiv_icache_miss_rate', | ||
| 396 | + 'aiv_vec_fp32_ratio', 'aiv_vec_fp16_ratio', | ||
| 397 | + 'aiv_vec_int32_ratio', 'aiv_vec_misc_ratio', | ||
| 398 | + 'aiv_vector_fops', | ||
| 399 | +] | ||
| 400 | + | ||
| 401 | +DEFAULT_KERNEL_DISPLAY_FIELDS = [ | ||
| 402 | + 'stream_id', 'input_shapes', 'output_shapes' | ||
| 403 | +] | ||
| 404 | + | ||
| 405 | +KERNEL_FIELD_LABELS = { | ||
| 406 | + 'input_shapes': 'Input', | ||
| 407 | + 'output_shapes': 'Output', | ||
| 408 | + 'type': 'Type', | ||
| 409 | + 'stream_id': 'Stream', | ||
| 410 | + 'device_id': 'Device', | ||
| 411 | + 'task_id': 'Task ID', | ||
| 412 | + 'start_time_us': 'Start', | ||
| 413 | + 'duration_us': 'Duration', | ||
| 414 | + 'wait_time_us': 'Wait', | ||
| 415 | + 'op_state': 'State', | ||
| 416 | + 'accelerator_core': 'Acc Core', | ||
| 417 | + 'block_dim': 'Block Dim', | ||
| 418 | + 'input_data_types': 'In DType', | ||
| 419 | + 'output_data_types': 'Out DType', | ||
| 420 | + 'input_formats': 'In Fmt', | ||
| 421 | + 'output_formats': 'Out Fmt', | ||
| 422 | +} | ||
| 423 | + | ||
| 424 | +ALL_KERNEL_META_FIELDS = [ | ||
| 425 | + 'stream_id', 'input_shapes', 'output_shapes', | ||
| 426 | + 'start_time_us', 'duration_us', 'wait_time_us', | ||
| 427 | + 'device_id', 'task_id', 'type', 'op_state', | ||
| 428 | + 'accelerator_core', 'block_dim', | ||
| 429 | + 'input_data_types', 'output_data_types', 'input_formats', 'output_formats', | ||
| 430 | +] | ||
| 431 | + | ||
| 432 | + | ||
| 433 | +def get_kernel_field_value(kernel: dict, field_key: str, multiplier: int, total_duration: float) -> str: | ||
| 434 | + if field_key == 'duration_us': | ||
| 435 | + k_duration_raw = kernel.get('duration', 0) | ||
| 436 | + raw_str = kernel.get('duration_raw', '') | ||
| 437 | + if raw_str: | ||
| 438 | + return f'{raw_str} us' | ||
| 439 | + return f'{format_duration_us(k_duration_raw)} us' | ||
| 440 | + | ||
| 441 | + if field_key == 'start_time_us': | ||
| 442 | + raw_str = kernel.get('start_time_raw', '') | ||
| 443 | + if raw_str: | ||
| 444 | + return f'{raw_str} us' | ||
| 445 | + return f'{format_duration_us(kernel.get("start_time", 0))} us' | ||
| 446 | + | ||
| 447 | + if field_key == 'type': | ||
| 448 | + return kernel.get('task_type', '') or kernel.get('all_fields', {}).get('type', 'N/A') | ||
| 449 | + | ||
| 450 | + if field_key in ('input_shapes', 'output_shapes', 'input_data_types', | ||
| 451 | + 'output_data_types', 'input_formats', 'output_formats'): | ||
| 452 | + val = kernel.get(field_key) or kernel.get('all_fields', {}).get(field_key, 'N/A') | ||
| 453 | + if val and len(str(val)) > 50: | ||
| 454 | + return str(val)[:47] + '...' | ||
| 455 | + return str(val) if val else 'N/A' | ||
| 456 | + | ||
| 457 | + if field_key == 'stream_id': | ||
| 458 | + return str(kernel.get('stream_id', '') or kernel.get('all_fields', {}).get('stream_id', 'N/A')) | ||
| 459 | + | ||
| 460 | + all_fields = kernel.get('all_fields', {}) | ||
| 461 | + if field_key in all_fields: | ||
| 462 | + val = all_fields[field_key] | ||
| 463 | + raw_key = f'{field_key}_raw' | ||
| 464 | + if raw_key in all_fields: | ||
| 465 | + return str(all_fields[raw_key]) | ||
| 466 | + if val is None: | ||
| 467 | + return 'N/A' | ||
| 468 | + return str(format_field_display(field_key, val)) | ||
| 469 | + | ||
| 470 | + if field_key in kernel: | ||
| 471 | + val = kernel[field_key] | ||
| 472 | + if val is None: | ||
| 473 | + return 'N/A' | ||
| 474 | + return str(format_field_display(field_key, val)) | ||
| 475 | + | ||
| 476 | + return 'N/A' | ||
| 477 | + | ||
| 478 | + | ||
| 479 | +def collect_all_op_indices(node: dict) -> list: | ||
| 480 | + indices = list(node.get('op_indices', [])) | ||
| 481 | + for child in node.get('children', []): | ||
| 482 | + indices.extend(collect_all_op_indices(child)) | ||
| 483 | + return indices | ||
| 484 | + | ||
| 485 | + | ||
| 486 | +def count_all_kernels(node: dict, operators: list) -> int: | ||
| 487 | + return len(collect_all_op_indices(node)) | ||
| 488 | + | ||
| 489 | + | ||
| 490 | +def get_node_time_span_info(node: dict, operators: list) -> dict: | ||
| 491 | + all_op_indices = collect_all_op_indices(node) | ||
| 492 | + op_dict = {op['index']: op for op in operators} | ||
| 493 | + | ||
| 494 | + if not all_op_indices: | ||
| 495 | + return {'time_span': 0, 'streams': {}, 'stream_info': '', 'stream_hover': '', | ||
| 496 | + 'calculation': '', 'stream_count': 0} | ||
| 497 | + | ||
| 498 | + ops = [op_dict[i] for i in all_op_indices if i in op_dict] | ||
| 499 | + | ||
| 500 | + if not ops: | ||
| 501 | + return {'time_span': 0, 'streams': {}, 'stream_info': '', 'stream_hover': '', | ||
| 502 | + 'calculation': '', 'stream_count': 0} | ||
| 503 | + | ||
| 504 | + def get_op_name(op): | ||
| 505 | + return op.get('normalized_name') or op.get('type') or op.get('name', 'Unknown') | ||
| 506 | + | ||
| 507 | + earliest_op = min(ops, key=lambda op: op.get('start_time_us', 0)) | ||
| 508 | + min_start = earliest_op.get('start_time_us', 0) | ||
| 509 | + earliest_name = get_op_name(earliest_op) | ||
| 510 | + | ||
| 511 | + max_end = min_start | ||
| 512 | + latest_op = None | ||
| 513 | + for op in ops: | ||
| 514 | + end_time = op.get('start_time_us', 0) + op.get('duration_us', 0) | ||
| 515 | + if end_time > max_end: | ||
| 516 | + max_end = end_time | ||
| 517 | + latest_op = op | ||
| 518 | + | ||
| 519 | + latest_name = get_op_name(latest_op) if latest_op else 'Unknown' | ||
| 520 | + latest_start = latest_op.get('start_time_us', 0) if latest_op else 0 | ||
| 521 | + latest_duration = latest_op.get('duration_us', 0) if latest_op else 0 | ||
| 522 | + | ||
| 523 | + if len(ops) == 1: | ||
| 524 | + time_span = ops[0].get('duration_us', 0) | ||
| 525 | + else: | ||
| 526 | + time_span = round(max_end - min_start, 2) | ||
| 527 | + | ||
| 528 | + stream_counts = {} | ||
| 529 | + for op in ops: | ||
| 530 | + sid = op.get('stream_id', 'unknown') | ||
| 531 | + stream_counts[sid] = stream_counts.get(sid, 0) + 1 | ||
| 532 | + | ||
| 533 | + stream_count = len(stream_counts) | ||
| 534 | + stream_info = f"{stream_count} streams" | ||
| 535 | + stream_hover = ' '.join([f"Stream {sid}: {cnt}" | ||
| 536 | + for sid, cnt in sorted(stream_counts.items(), key=lambda x: -x[1])]) | ||
| 537 | + calculation = (f"{latest_name}({latest_start:.2f}+{latest_duration:.2f}) - " | ||
| 538 | + f"{earliest_name}({min_start:.2f}) = {time_span:.2f} us") | ||
| 539 | + | ||
| 540 | + return { | ||
| 541 | + 'time_span': time_span, | ||
| 542 | + 'streams': stream_counts, | ||
| 543 | + 'stream_info': stream_info, | ||
| 544 | + 'stream_hover': stream_hover, | ||
| 545 | + 'stream_count': stream_count, | ||
| 546 | + 'calculation': calculation, | ||
| 547 | + } | ||
| 548 | + | ||
| 549 | + | ||
| 550 | +def get_unique_kernels(kernels: list) -> list: | ||
| 551 | + seen = set() | ||
| 552 | + unique = [] | ||
| 553 | + for k in kernels: | ||
| 554 | + if k not in seen: | ||
| 555 | + seen.add(k) | ||
| 556 | + unique.append(k) | ||
| 557 | + return unique | ||
| 558 | + | ||
| 559 | + | ||
| 560 | +def format_duration_us(duration: float) -> str: | ||
| 561 | + return f"{duration:.2f}" | ||
| 562 | + | ||
| 563 | + | ||
| 564 | +def format_duration_ms(duration: float) -> str: | ||
| 565 | + return f"{duration / 1000:.2f}" | ||
| 566 | + | ||
| 567 | + | ||
| 568 | +def format_percentage(duration: float, total: float) -> str: | ||
| 569 | + if total == 0: | ||
| 570 | + return "0.00" | ||
| 571 | + return f"{duration / total * 100:.2f}" | ||
| 572 | + | ||
| 573 | + | ||
| 574 | +def get_node_total_duration(node: dict, operators: list) -> float: | ||
| 575 | + duration = get_duration_by_indices(operators, node.get('op_indices', [])) | ||
| 576 | + for child in node.get('children', []): | ||
| 577 | + duration += get_node_total_duration(child, operators) | ||
| 578 | + return duration | ||
| 579 | + | ||
| 580 | + | ||
| 581 | + | ||
| 582 | +class TimingCtx: | ||
| 583 | + """collect_timing_tree_lines 递归过程中保持不变的上下文。""" | ||
| 584 | + operators: list | ||
| 585 | + total_duration: float | ||
| 586 | + max_depth: int | ||
| 587 | + align_col: int | ||
| 588 | + | ||
| 589 | + | ||
| 590 | +def _timing_kernel_lines(op_indices, ctx, multiplier, new_indent): | ||
| 591 | + """渲染叶节点 kernel 明细行(时序文本树)。""" | ||
| 592 | + lines = [] | ||
| 593 | + kernel_details = get_kernel_details_by_indices(ctx.operators, op_indices) | ||
| 594 | + for i, k in enumerate(kernel_details): | ||
| 595 | + k_duration = k['duration'] * multiplier | ||
| 596 | + k_pct = format_percentage(k_duration, ctx.total_duration) | ||
| 597 | + k_dur_str = f"{format_duration_us(k_duration)} us ({format_duration_ms(k_duration)} ms, {k_pct}%)" | ||
| 598 | + k_prefix = new_indent + ("└── " if i == len(kernel_details) - 1 else "├── ") | ||
| 599 | + k_name_padded = k['name'].ljust(ctx.align_col - len(k_prefix)) | ||
| 600 | + lines.append(f"{k_prefix}{k_name_padded}{k_dur_str}") | ||
| 601 | + return lines | ||
| 602 | + | ||
| 603 | + | ||
| 604 | +def collect_timing_tree_lines(node: dict, ctx: TimingCtx, multiplier: int, | ||
| 605 | + pos: tuple, current_depth: int) -> List[str]: | ||
| 606 | + indent, is_last = pos | ||
| 607 | + align_col = ctx.align_col | ||
| 608 | + name = node.get('name', 'Unknown') | ||
| 609 | + op_indices = node.get('op_indices', []) | ||
| 610 | + children = node.get('children', []) | ||
| 611 | + | ||
| 612 | + duration = get_duration_by_indices(ctx.operators, op_indices) * multiplier | ||
| 613 | + for child in children: | ||
| 614 | + duration += get_node_total_duration(child, ctx.operators) * multiplier | ||
| 615 | + | ||
| 616 | + count_str = f" (*{multiplier}层)" if multiplier > 1 and current_depth == 1 else "" | ||
| 617 | + percentage = format_percentage(duration, ctx.total_duration) | ||
| 618 | + duration_str = f"{format_duration_us(duration)} us ({format_duration_ms(duration)} ms, {percentage}%)" | ||
| 619 | + | ||
| 620 | + lines = [] | ||
| 621 | + if current_depth == 0: | ||
| 622 | + lines.append(f"{(name + count_str).ljust(align_col)}{duration_str}") | ||
| 623 | + else: | ||
| 624 | + full_prefix = indent + ("└── " if is_last else "├── ") | ||
| 625 | + name_padded = (name + count_str).ljust(align_col - len(full_prefix)) | ||
| 626 | + lines.append(f"{full_prefix}{name_padded}{duration_str}") | ||
| 627 | + | ||
| 628 | + new_indent = indent + (" " if is_last else "│ ") | ||
| 629 | + | ||
| 630 | + if current_depth >= ctx.max_depth: | ||
| 631 | + lines.extend(_timing_kernel_lines(op_indices, ctx, multiplier, new_indent)) | ||
| 632 | + return lines | ||
| 633 | + | ||
| 634 | + if op_indices and not children: | ||
| 635 | + lines.extend(_timing_kernel_lines(op_indices, ctx, multiplier, new_indent)) | ||
| 636 | + | ||
| 637 | + for i, child in enumerate(children): | ||
| 638 | + is_child_last = (i == len(children) - 1) | ||
| 639 | + lines.extend(collect_timing_tree_lines(child, ctx, multiplier, | ||
| 640 | + (new_indent, is_child_last), | ||
| 641 | + current_depth + 1)) | ||
| 642 | + return lines | ||
| 643 | + | ||
| 644 | + | ||
| 645 | +def _collect_stage_modules(config, operators, order_start): | ||
| 646 | + """收集 stages 段的 module / kernel_module 条目。返回 (modules, kernels, next_order)。""" | ||
| 647 | + modules, kernel_modules = [], [] | ||
| 648 | + order = order_start | ||
| 649 | + for stage_name, stage_info in config.get('stages', {}).items(): | ||
| 650 | + stage_indices = stage_info.get('stage_indices', [0]) | ||
| 651 | + stage_count = len(stage_indices) if stage_indices else 1 | ||
| 652 | + stage_children = stage_info.get('children', []) | ||
| 653 | + stage_duration = get_duration_by_indices(operators, stage_info.get('op_indices', [])) | ||
| 654 | + stage_kernels = get_kernels_by_indices(operators, stage_info.get('op_indices', [])) | ||
| 655 | + for child in stage_children: | ||
| 656 | + child_duration, child_kernels = collect_node_stats(child, operators) | ||
| 657 | + stage_duration += child_duration | ||
| 658 | + stage_kernels.extend(child_kernels) | ||
| 659 | + modules.append({ | ||
| 660 | + 'name': stage_info.get('name', stage_name), 'level': 1, 'order': order, | ||
| 661 | + 'count': stage_count, 'kernels_per_pass': len(stage_kernels), | ||
| 662 | + 'total_kernels': len(stage_kernels) * stage_count, | ||
| 663 | + 'total_duration': stage_duration * stage_count, | ||
| 664 | + }) | ||
| 665 | + order += 1 | ||
| 666 | + for child in stage_children: | ||
| 667 | + cm, ck = collect_child_modules_with_kernels(child, operators, stage_count, 2, order) | ||
| 668 | + modules.extend(cm) | ||
| 669 | + kernel_modules.extend(ck) | ||
| 670 | + order += len(cm) | ||
| 671 | + return modules, kernel_modules, order | ||
| 672 | + | ||
| 673 | + | ||
| 674 | +def _collect_layer_modules(config, operators, order_start): | ||
| 675 | + """收集 layer_structure 段的 module / kernel_module 条目。""" | ||
| 676 | + modules, kernel_modules = [], [] | ||
| 677 | + order = order_start | ||
| 678 | + layer_types = config.get('layer_types', {}) | ||
| 679 | + layer_structure = config.get('layer_structure', {}) | ||
| 680 | + for layer_type, layer_info in layer_types.items(): | ||
| 681 | + structure = layer_structure.get(layer_type, {}) | ||
| 682 | + if not structure: | ||
| 683 | + continue | ||
| 684 | + layer_count = len(layer_info.get('layer_indices', [])) | ||
| 685 | + layer_duration = get_node_total_duration(structure, operators) | ||
| 686 | + layer_kernels = collect_all_kernels(structure, operators) | ||
| 687 | + modules.append({ | ||
| 688 | + 'name': structure.get('name', layer_type), 'level': 1, 'order': order, | ||
| 689 | + 'count': layer_count, 'kernels_per_pass': len(layer_kernels), | ||
| 690 | + 'total_kernels': len(layer_kernels) * layer_count, | ||
| 691 | + 'total_duration': layer_duration * layer_count, | ||
| 692 | + }) | ||
| 693 | + order += 1 | ||
| 694 | + cm, ck = collect_child_modules_with_kernels(structure, operators, layer_count, 2, order) | ||
| 695 | + modules.extend(cm) | ||
| 696 | + kernel_modules.extend(ck) | ||
| 697 | + order += len(cm) | ||
| 698 | + return modules, kernel_modules, order | ||
| 699 | + | ||
| 700 | + | ||
| 701 | +def _collect_aux_modules(config, operators, order_start): | ||
| 702 | + """收集 runtime_auxiliary 段的 module 条目。""" | ||
| 703 | + modules = [] | ||
| 704 | + order = order_start | ||
| 705 | + for aux in config.get('runtime_auxiliary', []) or []: | ||
| 706 | + aux_duration = get_duration_by_indices(operators, aux.get('op_indices', [])) | ||
| 707 | + aux_kernels = get_kernels_by_indices(operators, aux.get('op_indices', [])) | ||
| 708 | + modules.append({ | ||
| 709 | + 'name': aux.get('name', 'runtime_aux'), 'level': 1, 'order': order, | ||
| 710 | + 'count': 1, 'kernels_per_pass': len(aux_kernels), | ||
| 711 | + 'total_kernels': len(aux_kernels), 'total_duration': aux_duration, | ||
| 712 | + }) | ||
| 713 | + order += 1 | ||
| 714 | + return modules, order | ||
| 715 | + | ||
| 716 | + | ||
| 717 | +def collect_all_modules(config: dict, operators: list) -> Tuple[list, list]: | ||
| 718 | + modules = [{ | ||
| 719 | + 'name': config.get('model_name', 'Model'), 'level': 0, 'order': 0, 'count': 1, | ||
| 720 | + 'kernels_per_pass': len(operators), 'total_kernels': len(operators), | ||
| 721 | + 'total_duration': sum(op.get('duration_us', 0) for op in operators), | ||
| 722 | + }] | ||
| 723 | + kernel_modules = [] | ||
| 724 | + | ||
| 725 | + stage_m, stage_k, order = _collect_stage_modules(config, operators, 1) | ||
| 726 | + modules.extend(stage_m) | ||
| 727 | + kernel_modules.extend(stage_k) | ||
| 728 | + | ||
| 729 | + layer_m, layer_k, order = _collect_layer_modules(config, operators, order) | ||
| 730 | + modules.extend(layer_m) | ||
| 731 | + kernel_modules.extend(layer_k) | ||
| 732 | + | ||
| 733 | + aux_m, _ = _collect_aux_modules(config, operators, order) | ||
| 734 | + modules.extend(aux_m) | ||
| 735 | + | ||
| 736 | + return modules, kernel_modules | ||
| 737 | + | ||
| 738 | + | ||
| 739 | +def collect_node_stats(node: dict, operators: list) -> tuple: | ||
| 740 | + duration = get_duration_by_indices(operators, node.get('op_indices', [])) | ||
| 741 | + kernels = get_kernels_by_indices(operators, node.get('op_indices', [])) | ||
| 742 | + | ||
| 743 | + for child in node.get('children', []): | ||
| 744 | + child_duration, child_kernels = collect_node_stats(child, operators) | ||
| 745 | + duration += child_duration | ||
| 746 | + kernels.extend(child_kernels) | ||
| 747 | + | ||
| 748 | + return duration, kernels | ||
| 749 | + | ||
| 750 | + | ||
| 751 | +def collect_all_kernels(node: dict, operators: list) -> list: | ||
| 752 | + kernels = get_kernels_by_indices(operators, node.get('op_indices', [])) | ||
| 753 | + for child in node.get('children', []): | ||
| 754 | + kernels.extend(collect_all_kernels(child, operators)) | ||
| 755 | + return kernels | ||
| 756 | + | ||
| 757 | + | ||
| 758 | +def collect_child_modules(node: dict, operators: list, multiplier: int) -> list: | ||
| 759 | + modules = [] | ||
| 760 | + | ||
| 761 | + op_indices = node.get('op_indices', []) | ||
| 762 | + if op_indices: | ||
| 763 | + duration = get_duration_by_indices(operators, op_indices) | ||
| 764 | + kernels = get_kernels_by_indices(operators, op_indices) | ||
| 765 | + modules.append({ | ||
| 766 | + 'name': node.get('name', 'Unknown'), | ||
| 767 | + 'count': multiplier, | ||
| 768 | + 'kernels_per_pass': len(kernels), | ||
| 769 | + 'total_kernels': len(kernels) * multiplier, | ||
| 770 | + 'total_duration': duration * multiplier, | ||
| 771 | + }) | ||
| 772 | + | ||
| 773 | + for child in node.get('children', []): | ||
| 774 | + modules.extend(collect_child_modules(child, operators, multiplier)) | ||
| 775 | + | ||
| 776 | + return modules | ||
| 777 | + | ||
| 778 | + | ||
| 779 | +def collect_child_modules_with_kernels(node: dict, operators: list, multiplier: int, | ||
| 780 | + level: int, start_order: int) -> Tuple[list, list]: | ||
| 781 | + modules = [] | ||
| 782 | + kernel_modules = [] | ||
| 783 | + order = start_order | ||
| 784 | + | ||
| 785 | + children = node.get('children', []) | ||
| 786 | + for _i, child in enumerate(children): | ||
| 787 | + child_name = child.get('name', 'Unknown') | ||
| 788 | + child_op_indices = child.get('op_indices', []) | ||
| 789 | + child_children = child.get('children', []) | ||
| 790 | + | ||
| 791 | + if child_op_indices and not child_children: | ||
| 792 | + kernel_details = get_kernel_details_by_indices(operators, child_op_indices) | ||
| 793 | + for kd in kernel_details: | ||
| 794 | + kernel_modules.append({ | ||
| 795 | + 'module_name': child_name, | ||
| 796 | + 'kernel_name': kd['name'], | ||
| 797 | + 'level': level + 1, | ||
| 798 | + 'duration': kd['duration'] * multiplier, | ||
| 799 | + 'count': multiplier, | ||
| 800 | + }) | ||
| 801 | + | ||
| 802 | + child_duration = get_duration_by_indices(operators, child_op_indices) | ||
| 803 | + for sub_child in child_children: | ||
| 804 | + child_duration += get_node_total_duration(sub_child, operators) | ||
| 805 | + | ||
| 806 | + child_kernels = get_kernels_by_indices(operators, child_op_indices) | ||
| 807 | + for sub_child in child_children: | ||
| 808 | + _, sub_kernels = collect_node_stats(sub_child, operators) | ||
| 809 | + child_kernels.extend(sub_kernels) | ||
| 810 | + | ||
| 811 | + modules.append({ | ||
| 812 | + 'name': child_name, | ||
| 813 | + 'level': level, | ||
| 814 | + 'order': order, | ||
| 815 | + 'count': multiplier, | ||
| 816 | + 'kernels_per_pass': len(child_kernels), | ||
| 817 | + 'total_kernels': len(child_kernels) * multiplier, | ||
| 818 | + 'total_duration': child_duration * multiplier, | ||
| 819 | + }) | ||
| 820 | + order += 1 | ||
| 821 | + | ||
| 822 | + sub_modules, sub_kernels = collect_child_modules_with_kernels(child, operators, multiplier, level + 1, order) | ||
| 823 | + modules.extend(sub_modules) | ||
| 824 | + kernel_modules.extend(sub_kernels) | ||
| 825 | + order += len(sub_modules) | ||
| 826 | + | ||
| 827 | + return modules, kernel_modules | ||
| 828 | + | ||
| 829 | + | ||
| 830 | +def _build_timing_temp_lines(config, model_name, model_duration, total_duration): | ||
| 831 | + """构建时序树的临时行列表(模型行 + stage/layer/aux 节点元组)。""" | ||
| 832 | + percentage = format_percentage(model_duration, total_duration) | ||
| 833 | + temp_lines = [(model_name, | ||
| 834 | + f"{format_duration_us(model_duration)} us " | ||
| 835 | + f"({format_duration_ms(model_duration)} ms, {percentage}%)")] | ||
| 836 | + | ||
| 837 | + for stage_name, stage_info in config.get('stages', {}).items(): | ||
| 838 | + stage_indices = stage_info.get('stage_indices', [0]) | ||
| 839 | + stage_count = len(stage_indices) if stage_indices else 1 | ||
| 840 | + temp_lines.append((stage_info.get('name', stage_name), stage_count, stage_info, False, 'stage')) | ||
| 841 | + | ||
| 842 | + layer_types = config.get('layer_types', {}) | ||
| 843 | + layer_structure = config.get('layer_structure', {}) | ||
| 844 | + layer_items = [(lt, li, layer_structure[lt]) for lt, li in layer_types.items() | ||
| 845 | + if layer_structure.get(lt)] | ||
| 846 | + for idx, (layer_type, layer_info, structure) in enumerate(layer_items): | ||
| 847 | + layer_count = len(layer_info.get('layer_indices', [])) | ||
| 848 | + is_last_layer = (idx == len(layer_items) - 1) | ||
| 849 | + temp_lines.append((structure.get('name', layer_type), layer_count, structure, is_last_layer, 'layer')) | ||
| 850 | + | ||
| 851 | + for aux in config.get('runtime_auxiliary', []): | ||
| 852 | + temp_lines.append((aux.get('name', 'runtime_aux'), 1, aux, False, 'aux')) | ||
| 853 | + return temp_lines | ||
| 854 | + | ||
| 855 | + | ||
| 856 | +def _timing_align_col(temp_lines): | ||
| 857 | + """计算时序树名称列对齐宽度。""" | ||
| 858 | + max_name_len = 0 | ||
| 859 | + for item in temp_lines: | ||
| 860 | + if len(item) == 2: | ||
| 861 | + max_name_len = max(max_name_len, len(item[0])) | ||
| 862 | + else: | ||
| 863 | + name, count, _node, _is_last, item_type = item | ||
| 864 | + count_str = f" (*{count}层)" if count > 1 and item_type in ('layer',) else "" | ||
| 865 | + instance_str = f" (*{count}实例)" if count > 1 and item_type in ('stage', 'aux') else "" | ||
| 866 | + max_name_len = max(max_name_len, len(name) + len(count_str) + len(instance_str)) | ||
| 867 | + return max(max_name_len + 2, 40) | ||
| 868 | + | ||
| 869 | + | ||
| 870 | +def generate_analysis_section(config: dict, operators: list, total_duration: float, max_depth: int = 4) -> str: | ||
| 871 | + model_name = config.get('model_name', 'Model') | ||
| 872 | + model_duration = sum(op.get('duration_us', 0) for op in operators) | ||
| 873 | + | ||
| 874 | + temp_lines = _build_timing_temp_lines(config, model_name, model_duration, total_duration) | ||
| 875 | + align_col = _timing_align_col(temp_lines) | ||
| 876 | + | ||
| 877 | + timing_ctx = TimingCtx(operators=operators, total_duration=total_duration, | ||
| 878 | + max_depth=max_depth, align_col=align_col) | ||
| 879 | + lines = ["## 模型性能分析", "", "```text"] | ||
| 880 | + | ||
| 881 | + for item_idx, item in enumerate(temp_lines): | ||
| 882 | + if len(item) == 2: | ||
| 883 | + name_padded = item[0].ljust(align_col) | ||
| 884 | + lines.append(f"{name_padded}{item[1]}") | ||
| 885 | + else: | ||
| 886 | + name, count, node, _is_last, item_type = item | ||
| 887 | + is_last = (item_idx == len(temp_lines) - 1) | ||
| 888 | + lines.extend(collect_timing_tree_lines(node, timing_ctx, count, ("", is_last), 1)) | ||
| 889 | + | ||
| 890 | + lines.append("```") | ||
| 891 | + lines.append("") | ||
| 892 | + return '\n'.join(lines) | ||
| 893 | + | ||
| 894 | + | ||
| 895 | + | ||
| 896 | + | ||
| 897 | +def get_html_css() -> str: | ||
| 898 | + return HTML_CSS | ||
| 899 | + | ||
| 900 | + | ||
| 901 | +TIMELINE_PALETTE = [ | ||
| 902 | + '#4fc1ff', '#6a9955', '#ce9178', '#c586c0', '#569cd6', | ||
| 903 | + '#dcdcaa', '#e06c75', '#61afef', '#98c379', '#d19a66', | ||
| 904 | + '#c678dd', '#e5c07b', '#56b6c2', '#be5046', '#7ec699', | ||
| 905 | + '#f99157', '#cc99cc', '#99cc99', '#6699cc', '#f2777a', | ||
| 906 | +] | ||
| 907 | + | ||
| 908 | + | ||
| 909 | +def _collect_node_op_indices_all(node: dict) -> list: | ||
| 910 | + indices = list(node.get('op_indices', [])) | ||
| 911 | + for child in node.get('children', []): | ||
| 912 | + indices.extend(_collect_node_op_indices_all(child)) | ||
| 913 | + return indices | ||
| 914 | + | ||
| 915 | + | ||
| 916 | +def _compute_node_time_range(node: dict, operators: list) -> dict: | ||
| 917 | + all_indices = _collect_node_op_indices_all(node) | ||
| 918 | + op_dict = {op['index']: op for op in operators} | ||
| 919 | + ops = [op_dict[i] for i in all_indices if i in op_dict] | ||
| 920 | + if not ops: | ||
| 921 | + return {'start': 0, 'end': 0, 'duration': 0, 'streams': {}, 'kernel_count': 0} | ||
| 922 | + min_start = min(op.get('start_time_us', 0) for op in ops) | ||
| 923 | + max_end = max(op.get('start_time_us', 0) + op.get('duration_us', 0) for op in ops) | ||
| 924 | + stream_counts = {} | ||
| 925 | + for op in ops: | ||
| 926 | + sid = str(op.get('stream_id', 'unknown')) | ||
| 927 | + stream_counts[sid] = stream_counts.get(sid, 0) + 1 | ||
| 928 | + return { | ||
| 929 | + 'start': min_start, | ||
| 930 | + 'end': max_end, | ||
| 931 | + 'duration': max_end - min_start, | ||
| 932 | + 'streams': stream_counts, | ||
| 933 | + 'kernel_count': len(ops), | ||
| 934 | + } | ||
| 935 | + | ||
| 936 | + | ||
| 937 | +def _build_per_stream_ops(all_ops_indices, op_dict): | ||
| 938 | + """按 stream_id 分组并按 start 排序 op 列表。""" | ||
| 939 | + per_stream_ops = {} | ||
| 940 | + for i in all_ops_indices: | ||
| 941 | + op = op_dict.get(i) | ||
| 942 | + if not op: | ||
| 943 | + continue | ||
| 944 | + sid = str(op.get('stream_id', '0')) | ||
| 945 | + per_stream_ops.setdefault(sid, []).append({ | ||
| 946 | + 'index': op['index'], | ||
| 947 | + 'name': op.get('normalized_name') or op.get('type') or op.get('name', 'Unknown'), | ||
| 948 | + 'original_name': op.get('original_name', op.get('name', '')), | ||
| 949 | + 'start': op.get('start_time_us', 0), | ||
| 950 | + 'end': op.get('start_time_us', 0) + op.get('duration_us', 0), | ||
| 951 | + 'duration': op.get('duration_us', 0), | ||
| 952 | + 'wait_time': op.get('wait_time_us', 0), | ||
| 953 | + 'stream_id': sid, | ||
| 954 | + }) | ||
| 955 | + for sid in per_stream_ops: | ||
| 956 | + per_stream_ops[sid].sort(key=lambda x: x['start']) | ||
| 957 | + return per_stream_ops | ||
| 958 | + | ||
| 959 | + | ||
| 960 | +def generate_timeline_data(config: dict, operators: list, total_duration: float) -> list: | ||
| 961 | + ctx = _TimelineBuildCtx( | ||
| 962 | + op_dict={op['index']: op for op in operators}, | ||
| 963 | + timeline_nodes=[], | ||
| 964 | + node_color_map={}, | ||
| 965 | + ) | ||
| 966 | + model_name = config.get('model_name', 'Model') | ||
| 967 | + | ||
| 968 | + for _stage_name, stage_info in config.get('stages', {}).items(): | ||
| 969 | + stage_indices = stage_info.get('stage_indices', [0]) | ||
| 970 | + stage_count = len(stage_indices) if stage_indices else 1 | ||
| 971 | + _timeline_process_node(stage_info, model_name, ctx, | ||
| 972 | + TimelineNodeState(multiplier=stage_count, is_auxiliary=True)) | ||
| 973 | + | ||
| 974 | + layer_types = config.get('layer_types', {}) | ||
| 975 | + layer_structure = config.get('layer_structure', {}) | ||
| 976 | + for layer_type, layer_info in layer_types.items(): | ||
| 977 | + structure = layer_structure.get(layer_type, {}) | ||
| 978 | + if not structure: | ||
| 979 | + continue | ||
| 980 | + layer_count = len(layer_info.get('layer_indices', [])) | ||
| 981 | + _timeline_process_node(structure, model_name, ctx, | ||
| 982 | + TimelineNodeState(multiplier=layer_count)) | ||
| 983 | + | ||
| 984 | + for aux in config.get('runtime_auxiliary', []): | ||
| 985 | + _timeline_process_node(aux, model_name, ctx, | ||
| 986 | + TimelineNodeState(multiplier=1, is_auxiliary=True)) | ||
| 987 | + | ||
| 988 | + return ctx.timeline_nodes | ||
| 989 | + | ||
| 990 | + | ||
| 991 | +def _timeline_static_frame(): | ||
| 992 | + """时序图的静态框架 HTML(展开区 + 详情面板 + section 收尾)。""" | ||
| 993 | + return [ | ||
| 994 | + '<div class="timeline-expand-area" id="timeline-expand-area" style="display:none">', | ||
| 995 | + '<div class="expand-header">', | ||
| 996 | + '<button class="expand-close" id="expand-close-btn">收起</button>', | ||
| 997 | + '<div class="expand-breadcrumb" id="expand-breadcrumb"></div>', | ||
| 998 | + '<span class="expand-title" id="expand-title">-</span>', | ||
| 999 | + '</div>', | ||
| 1000 | + '<div class="expand-gantt" id="expand-gantt">', | ||
| 1001 | + '<div class="gantt-header">子节点多流时序图 <span class="gantt-hint">(按时间比例显示)</span></div>', | ||
| 1002 | + '<div class="gantt-container" id="gantt-container"></div>', | ||
| 1003 | + '</div>', | ||
| 1004 | + '<div class="expand-tree" id="expand-tree">', | ||
| 1005 | + '<div class="expand-tree-title">子节点结构 <span class="tree-hint">(点击有子节点的项继续展开)</span></div>', | ||
| 1006 | + '<div class="expand-tree-content" id="tree-content"></div>', | ||
| 1007 | + '</div>', | ||
| 1008 | + '</div>', | ||
| 1009 | + '</div>', | ||
| 1010 | + '<div class="timeline-detail-panel" id="timeline-detail-panel">', | ||
| 1011 | + '<div class="timeline-detail-header">', | ||
| 1012 | + '<span class="timeline-detail-title" id="timeline-detail-title">-</span>', | ||
| 1013 | + '<button class="timeline-detail-close" id="timeline-detail-close">×</button>', | ||
| 1014 | + '</div>', | ||
| 1015 | + '<div class="timeline-detail-ops" id="timeline-detail-ops"></div>', | ||
| 1016 | + '</div>', | ||
| 1017 | + '</section>', | ||
| 1018 | + ] | ||
| 1019 | + | ||
| 1020 | + | ||
| 1021 | +def _build_timeline_bar_data(timeline_nodes): | ||
| 1022 | + """构建供前端 JS 渲染甘特条的 bar_data 列表。""" | ||
| 1023 | + bar_data_json = [] | ||
| 1024 | + for ni, node in enumerate(timeline_nodes): | ||
| 1025 | + bar_data_json.append({ | ||
| 1026 | + 'name': node['name'], | ||
| 1027 | + 'path': node['path'], | ||
| 1028 | + 'start': node['start'], | ||
| 1029 | + 'end': node['end'], | ||
| 1030 | + 'duration': node['duration'], | ||
| 1031 | + 'color': node['color'], | ||
| 1032 | + 'multiplier': node.get('multiplier', 1), | ||
| 1033 | + 'kernel_count': node['kernel_count'], | ||
| 1034 | + 'streams': node['streams'], | ||
| 1035 | + 'dominant_stream': node.get('dominant_stream', node['streams'][0] if node['streams'] else '0'), | ||
| 1036 | + 'node_index': ni, | ||
| 1037 | + 'per_stream_ops': node['per_stream_ops'], | ||
| 1038 | + 'depth': node.get('depth', 0), | ||
| 1039 | + 'parent_index': node.get('parent_index', -1), | ||
| 1040 | + 'children_indices': node.get('children_indices', []), | ||
| 1041 | + 'has_children': node.get('has_children', False), | ||
| 1042 | + 'category': node.get('category', ''), | ||
| 1043 | + }) | ||
| 1044 | + return bar_data_json | ||
| 1045 | + | ||
| 1046 | + | ||
| 1047 | +def _fmt_timeline_dur(us: float) -> str: | ||
| 1048 | + if us >= 1000: | ||
| 1049 | + return f"{us/1000:.1f}ms" | ||
| 1050 | + return f"{us:.1f}us" | ||
| 1051 | + | ||
| 1052 | + | ||
| 1053 | +def _timeline_overview_item(ni, node): | ||
| 1054 | + """渲染时序图顶层节点概览中的单个 item。""" | ||
| 1055 | + name = node['name'] | ||
| 1056 | + multiplier = node.get('multiplier', 1) | ||
| 1057 | + streams = node.get('streams', []) | ||
| 1058 | + count_suffix = f" ×{multiplier}" if multiplier > 1 else "" | ||
| 1059 | + streams_str = ','.join(streams[:4]) + ('...' if len(streams) > 4 else '') | ||
| 1060 | + duration_str = _fmt_timeline_dur(node.get('duration', 0)) | ||
| 1061 | + has_children = node.get('has_children', False) | ||
| 1062 | + expand_icon = '▶' if has_children else '─' | ||
| 1063 | + expand_class = 'has-children' if has_children else '' | ||
| 1064 | + color = node.get('color', '#4fc1ff') | ||
| 1065 | + category = node.get('category', '') | ||
| 1066 | + category_attr = f' data-category="{category}"' if category else '' | ||
| 1067 | + return ( | ||
| 1068 | + f'<div class="timeline-node-item {expand_class}" data-node-index="{ni}"{category_attr}>' | ||
| 1069 | + f'<span class="node-expand-icon">{expand_icon}</span>' | ||
| 1070 | + f'<span class="node-name" style="border-left:3px solid {color};padding-left:6px">' | ||
| 1071 | + f'{name}{count_suffix}</span>' | ||
| 1072 | + f'<span class="node-streams">Streams: [{streams_str}]</span>' | ||
| 1073 | + f'<span class="node-duration">{duration_str}</span>' | ||
| 1074 | + f'<span class="node-kernels">{node.get("kernel_count", 0)} kernels</span>' | ||
| 1075 | + f'</div>' | ||
| 1076 | + ) | ||
| 1077 | + | ||
| 1078 | + | ||
| 1079 | +def generate_timeline_html(config: dict, operators: list, total_duration: float, max_depth: int) -> tuple: | ||
| 1080 | + timeline_nodes = generate_timeline_data(config, operators, total_duration) | ||
| 1081 | + | ||
| 1082 | + if not operators or not timeline_nodes: | ||
| 1083 | + return ('<section id="timeline" class="timeline-section"><h2>多流时序图</h2>' | ||
| 1084 | + '<p style="color:var(--text-secondary)">无可用的时序数据</p></section>'), {} | ||
| 1085 | + | ||
| 1086 | + top_level_nodes = [i for i, node in enumerate(timeline_nodes) if node.get('depth', 0) == 0] | ||
| 1087 | + | ||
| 1088 | + html_parts = [ | ||
| 1089 | + '<section id="timeline" class="timeline-section">', | ||
| 1090 | + '<div class="timeline-header">', | ||
| 1091 | + '<h2>多流时序图</h2>', | ||
| 1092 | + '<span class="timeline-hint">点击节点展开子节点多流时序,悬停查看时间信息</span>', | ||
| 1093 | + '</div>', | ||
| 1094 | + '<div class="timeline-container">', | ||
| 1095 | + '<div class="timeline-overview">', | ||
| 1096 | + '<div class="timeline-overview-title">顶层节点概览</div>', | ||
| 1097 | + '<div class="timeline-overview-list">', | ||
| 1098 | + ] | ||
| 1099 | + | ||
| 1100 | + for ni, node in enumerate(timeline_nodes): | ||
| 1101 | + if node.get('depth', 0) != 0: | ||
| 1102 | + continue | ||
| 1103 | + html_parts.append(_timeline_overview_item(ni, node)) | ||
| 1104 | + | ||
| 1105 | + html_parts.append('</div>') | ||
| 1106 | + html_parts.append('</div>') | ||
| 1107 | + | ||
| 1108 | + html_parts.extend(_timeline_static_frame()) | ||
| 1109 | + | ||
| 1110 | + bar_data_json = _build_timeline_bar_data(timeline_nodes) | ||
| 1111 | + | ||
| 1112 | + return '\n'.join(html_parts), { | ||
| 1113 | + 'top_level_nodes': top_level_nodes, | ||
| 1114 | + 'bar_data': bar_data_json, | ||
| 1115 | + } | ||
| 1116 | + | ||
| 1117 | + | ||
| 1118 | +def get_html_js(default_theme: str, tooltip_data: dict = None, display_fields: list = None, | ||
| 1119 | + timeline_data: dict = None) -> str: | ||
| 1120 | + import json as _json | ||
| 1121 | + tooltip_json = _json.dumps(tooltip_data or {}, ensure_ascii=False) | ||
| 1122 | + fields_json = _json.dumps(display_fields or DEFAULT_KERNEL_DISPLAY_FIELDS, ensure_ascii=False) | ||
| 1123 | + tl_json = _json.dumps(timeline_data or {}, ensure_ascii=False) | ||
| 1124 | + return (JS_TEMPLATE | ||
| 1125 | + .replace('@@TOOLTIP_JSON@@', tooltip_json) | ||
| 1126 | + .replace('@@FIELDS_JSON@@', fields_json) | ||
| 1127 | + .replace('@@TL_JSON@@', tl_json) | ||
| 1128 | + .replace('@@DEFAULT_THEME@@', default_theme)) | ||
| 1129 | + | ||
| 1130 | + | ||
| 1131 | + | ||
| 1132 | + | ||
| 1133 | +_TOOLTIP_LONG_FIELDS = {'input_shapes', 'output_shapes', 'input_data_types', | ||
| 1134 | + 'output_data_types', 'input_formats', 'output_formats'} | ||
| 1135 | + | ||
| 1136 | + | ||
| 1137 | +def _tooltip_field_val(all_fields, key): | ||
| 1138 | + """取字段显示值(优先 *_raw),无值返回 None。""" | ||
| 1139 | + raw_key = f'{key}_raw' | ||
| 1140 | + if raw_key in all_fields: | ||
| 1141 | + val = all_fields[raw_key] | ||
| 1142 | + if val is not None and val != '' and val != 'N/A': | ||
| 1143 | + suffix = ' us' if key.endswith('_us') else '' | ||
| 1144 | + return f'{val}{suffix}' | ||
| 1145 | + return None | ||
| 1146 | + val = all_fields.get(key) | ||
| 1147 | + if val is not None and val != '' and val != 'N/A': | ||
| 1148 | + return format_field_display(key, val) | ||
| 1149 | + return None | ||
| 1150 | + | ||
| 1151 | + | ||
| 1152 | +def _render_tooltip_group(all_fields, title, fields): | ||
| 1153 | + """渲染 tooltip 中的一个字段分组;无内容返回 None。""" | ||
| 1154 | + has_content = any( | ||
| 1155 | + all_fields.get(key) not in (None, '', 'N/A') for key, _ in fields) | ||
| 1156 | + if not has_content: | ||
| 1157 | + return None | ||
| 1158 | + parts = [f'<div class="kernel-tooltip-section">{title}</div>', | ||
| 1159 | + '<div class="kernel-tooltip-group-grid">'] | ||
| 1160 | + for key, label in fields: | ||
| 1161 | + display_val = _tooltip_field_val(all_fields, key) | ||
| 1162 | + if display_val is None: | ||
| 1163 | + continue | ||
| 1164 | + if key in _TOOLTIP_LONG_FIELDS: | ||
| 1165 | + parts.append(f'<div class="kernel-tooltip-full">' | ||
| 1166 | + f'<span class="kernel-tooltip-label">{label}:</span>' | ||
| 1167 | + f'<span class="kernel-tooltip-value">{display_val}</span></div>') | ||
| 1168 | + else: | ||
| 1169 | + parts.append(f'<div class="kernel-tooltip-label">{label}</div>') | ||
| 1170 | + parts.append(f'<div class="kernel-tooltip-value">{display_val}</div>') | ||
| 1171 | + parts.append('</div>') | ||
| 1172 | + return ''.join(parts) | ||
| 1173 | + | ||
| 1174 | + | ||
| 1175 | +_TOOLTIP_GROUP_ROWS = [ | ||
| 1176 | + [ | ||
| 1177 | + ('Basic Info & Timing', [ | ||
| 1178 | + ('type', 'Type'), | ||
| 1179 | + ('op_state', 'OP State'), | ||
| 1180 | + ('accelerator_core', 'Acc Core'), | ||
| 1181 | + ('duration_us', 'Duration'), | ||
| 1182 | + ('start_time_us', 'Start'), | ||
| 1183 | + ('wait_time_us', 'Wait'), | ||
| 1184 | + ]), | ||
| 1185 | + ('Stream/Device', [ | ||
| 1186 | + ('stream_id', 'Stream'), | ||
| 1187 | + ('device_id', 'Device'), | ||
| 1188 | + ('task_id', 'Task ID'), | ||
| 1189 | + ('block_dim', 'Block Dim'), | ||
| 1190 | + ]), | ||
| 1191 | + ('Tensor Info', [ | ||
| 1192 | + ('input_shapes', 'Input Shapes'), | ||
| 1193 | + ('input_data_types', 'Input DType'), | ||
| 1194 | + ('input_formats', 'Input Fmt'), | ||
| 1195 | + ('output_shapes', 'Output Shapes'), | ||
| 1196 | + ('output_data_types', 'Output DType'), | ||
| 1197 | + ('output_formats', 'Output Fmt'), | ||
| 1198 | + ]), | ||
| 1199 | + ], | ||
| 1200 | + [ | ||
| 1201 | + ('AI Core Metrics', [ | ||
| 1202 | + ('aicore_time_us', 'AI Core Time'), | ||
| 1203 | + ('aic_mac_time_us', 'MAC Time'), | ||
| 1204 | + ('aic_mac_ratio', 'MAC Ratio'), | ||
| 1205 | + ('aic_scalar_time_us', 'Scalar Time'), | ||
| 1206 | + ('aic_mte1_time_us', 'MTE1 Time'), | ||
| 1207 | + ('aic_mte2_time_us', 'MTE2 Time'), | ||
| 1208 | + ('aic_fixpipe_time_us', 'FixPipe Time'), | ||
| 1209 | + ('aic_icache_miss_rate', 'ICache Miss'), | ||
| 1210 | + ('cube_utilization_pct', 'Cube Util'), | ||
| 1211 | + ]), | ||
| 1212 | + ('AI Vector Metrics', [ | ||
| 1213 | + ('aiv_time_us', 'AI Vector Time'), | ||
| 1214 | + ('aiv_vec_time_us', 'Vector Time'), | ||
| 1215 | + ('aiv_vec_ratio', 'Vec Ratio'), | ||
| 1216 | + ('aiv_scalar_time_us', 'Scalar Time'), | ||
| 1217 | + ('aiv_mte2_time_us', 'MTE2 Time'), | ||
| 1218 | + ('aiv_mte3_time_us', 'MTE3 Time'), | ||
| 1219 | + ('aiv_icache_miss_rate', 'ICache Miss'), | ||
| 1220 | + ]), | ||
| 1221 | + ('Others', [ | ||
| 1222 | + ('mix_block_dim', 'Mix Block Dim'), | ||
| 1223 | + ('hf32_eligible', 'HF32 Eligible'), | ||
| 1224 | + ('context_id', 'Context ID'), | ||
| 1225 | + ('aic_total_cycles', 'AI Core Cycles'), | ||
| 1226 | + ('aic_scalar_ratio', 'Scalar Ratio'), | ||
| 1227 | + ('aic_mte1_ratio', 'MTE1 Ratio'), | ||
| 1228 | + ('aic_mte2_ratio', 'MTE2 Ratio'), | ||
| 1229 | + ('aic_fixpipe_ratio', 'FixPipe Ratio'), | ||
| 1230 | + ('aic_mac_fp16_ratio', 'MAC FP16'), | ||
| 1231 | + ('aic_mac_int8_ratio', 'MAC INT8'), | ||
| 1232 | + ('aic_cube_fops', 'Cube FLOPs'), | ||
| 1233 | + ('aiv_total_cycles', 'AIV Cycles'), | ||
| 1234 | + ('aiv_scalar_ratio', 'AIV Scalar Ratio'), | ||
| 1235 | + ('aiv_mte2_ratio', 'AIV MTE2 Ratio'), | ||
| 1236 | + ('aiv_mte3_ratio', 'AIV MTE3 Ratio'), | ||
| 1237 | + ('aiv_vec_fp32_ratio', 'Vec FP32'), | ||
| 1238 | + ('aiv_vec_fp16_ratio', 'Vec FP16'), | ||
| 1239 | + ('aiv_vec_int32_ratio', 'Vec INT32'), | ||
| 1240 | + ('aiv_vec_misc_ratio', 'Vec Misc'), | ||
| 1241 | + ('aiv_vector_fops', 'Vector FLOPs'), | ||
| 1242 | + ]), | ||
| 1243 | + ], | ||
| 1244 | +] | ||
| 1245 | + | ||
| 1246 | + | ||
| 1247 | +def generate_kernel_tooltip_html(kernel: dict, semantic_info: dict = None) -> str: | ||
| 1248 | + all_fields = kernel.get('all_fields', {}) | ||
| 1249 | + idx = kernel.get('index', '') | ||
| 1250 | + | ||
| 1251 | + parts = [f'<div class="kernel-tooltip-header">{kernel.get("name", "Unknown")}</div>'] | ||
| 1252 | + | ||
| 1253 | + if semantic_info: | ||
| 1254 | + semantic = semantic_info.get('semantic', '') | ||
| 1255 | + shape_semantic = semantic_info.get('shape_semantic', '') | ||
| 1256 | + path = semantic_info.get('path', '') | ||
| 1257 | + code_ref = semantic_info.get('code_ref', '') | ||
| 1258 | + if semantic or shape_semantic: | ||
| 1259 | + code_ref_html = f'<br/><small>Code: {code_ref}</small>' if code_ref else '' | ||
| 1260 | + shape_html = f'<br/><small class="kernel-shape-semantic">{shape_semantic}</small>' if shape_semantic else '' | ||
| 1261 | + parts.append(f'<div class="kernel-tooltip-semantic">{semantic}{shape_html}' | ||
| 1262 | + f'<br/><small>Path: {path}</small>{code_ref_html}</div>') | ||
| 1263 | + | ||
| 1264 | + parts.append('<div class="kernel-tooltip-container">') | ||
| 1265 | + idx_display = all_fields.get('index', idx) | ||
| 1266 | + parts.append('<div class="kernel-tooltip-label">Index:</div>') | ||
| 1267 | + parts.append(f'<div class="kernel-tooltip-value" style="margin-bottom:8px">{idx_display}</div>') | ||
| 1268 | + | ||
| 1269 | + | ||
| 1270 | + for row in _TOOLTIP_GROUP_ROWS: | ||
| 1271 | + row_parts = [] | ||
| 1272 | + for group_title, fields in row: | ||
| 1273 | + group_html = _render_tooltip_group(all_fields, group_title, fields) | ||
| 1274 | + if group_html: | ||
| 1275 | + row_parts.append(f'<div class="kernel-tooltip-group">{group_html}</div>') | ||
| 1276 | + if row_parts: | ||
| 1277 | + parts.append('<div class="kernel-tooltip-row">') | ||
| 1278 | + parts.extend(row_parts) | ||
| 1279 | + parts.append('</div>') | ||
| 1280 | + | ||
| 1281 | + parts.append('</div>') | ||
| 1282 | + return ''.join(parts) | ||
| 1283 | + | ||
| 1284 | + | ||
| 1285 | +def _render_kernel_meta_field(k, field_key, krc, k_shape_semantic): | ||
| 1286 | + """渲染单个 kernel-meta 字段的 span(含 shape_semantic hover)。""" | ||
| 1287 | + label = KERNEL_FIELD_LABELS.get(field_key, field_key) | ||
| 1288 | + value = get_kernel_field_value(k, field_key, krc.multiplier, krc.ctx.total_duration) | ||
| 1289 | + is_default = field_key in krc.display_fields | ||
| 1290 | + style_attr = '' if is_default else ' style="display:none"' | ||
| 1291 | + tip_cls = '' | ||
| 1292 | + tip_attr = '' | ||
| 1293 | + if field_key in ('input_shapes', 'output_shapes') and k_shape_semantic: | ||
| 1294 | + tip_cls = ' shape-semantic-tip' | ||
| 1295 | + if '→' in k_shape_semantic: | ||
| 1296 | + _parts = k_shape_semantic.split('→', 1) | ||
| 1297 | + _tip = _parts[0].strip() if field_key == 'input_shapes' else _parts[1].strip() | ||
| 1298 | + else: | ||
| 1299 | + _tip = k_shape_semantic | ||
| 1300 | + tip_attr = f' data-shape-semantic="{_tip}"' | ||
| 1301 | + return (f'<span class="kernel-meta-item{tip_cls}" data-field="{field_key}"{style_attr}{tip_attr}>' | ||
| 1302 | + f'<span class="kernel-meta-label">{label}:</span>' | ||
| 1303 | + f'<span class="kernel-meta-value">{value}</span></span>') | ||
| 1304 | + | ||
| 1305 | + | ||
| 1306 | +def _kernel_semantic_html(semantic_info): | ||
| 1307 | + """构建 kernel 名称后的黄色 semantic 包裹块。""" | ||
| 1308 | + k_semantic = semantic_info.get('semantic', '') if semantic_info else '' | ||
| 1309 | + if not k_semantic: | ||
| 1310 | + return '' | ||
| 1311 | + k_expand_btn = ('<span class="semantic-expand-btn"><svg viewBox="0 0 24 24">' | ||
| 1312 | + '<path d="M7 10l5 5 5-5z"/></svg></span>') | ||
| 1313 | + k_display = k_semantic[:57] + '...' if len(k_semantic) > 60 else k_semantic | ||
| 1314 | + return (f' <span class="node-semantic-wrapper" data-full="{k_semantic}">' | ||
| 1315 | + f'<span class="node-semantic-truncated">{k_display}</span>' | ||
| 1316 | + f'<span class="node-semantic-full">{k_semantic}</span>' | ||
| 1317 | + f'{k_expand_btn}</span>') | ||
| 1318 | + | ||
| 1319 | + | ||
| 1320 | +_KERNEL_INFO_SVG = ( | ||
| 1321 | + '<svg viewBox="0 0 24 24">' | ||
| 1322 | + '<path d="M15.5 14h-.79l-.28-.27A6.471 6.471 0 0 0 16 9.5 6.5 6.5 0 1 0 9.5 16' | ||
| 1323 | + 'c1.61 0 3.09-.59 4.23-1.57l.27.28v.79l5 4.99L20.49 19l-4.99-5zm-6 0' | ||
| 1324 | + 'C7.01 14 5 11.99 5 9.5S7.01 5 9.5 5 14 7.01 14 9.5 11.99 14 9.5 14z"/></svg>') | ||
| 1325 | + | ||
| 1326 | + | ||
| 1327 | +def _render_one_kernel(k, ki, total_kernels, krc): | ||
| 1328 | + """渲染单个 kernel 叶节点,返回 (html_lines, (idx, tooltip_html))。""" | ||
| 1329 | + ctx = krc.ctx | ||
| 1330 | + multiplier = krc.multiplier | ||
| 1331 | + current_depth = krc.current_depth | ||
| 1332 | + total_duration = ctx.total_duration | ||
| 1333 | + kernel_semantics = ctx.kernel_semantics | ||
| 1334 | + k_duration_total = k['duration'] * multiplier | ||
| 1335 | + k_pct = format_percentage(k_duration_total, total_duration) | ||
| 1336 | + idx = k.get('index', 0) | ||
| 1337 | + semantic_info = kernel_semantics.get(idx) if kernel_semantics else None | ||
| 1338 | + tooltip_html = generate_kernel_tooltip_html(k, semantic_info) | ||
| 1339 | + kernel_seq = f"[{ki+1}/{total_kernels}]" | ||
| 1340 | + kernel_name = k.get('name', 'Unknown') | ||
| 1341 | + k_shape_semantic = semantic_info.get('shape_semantic', '') if semantic_info else '' | ||
| 1342 | + kernel_semantic_html = _kernel_semantic_html(semantic_info) | ||
| 1343 | + | ||
| 1344 | + lines = [ | ||
| 1345 | + f'<li class="tree-node leaf" data-depth="{current_depth + 1}" data-type="kernel">', | ||
| 1346 | + '<div class="node-header">', | ||
| 1347 | + '<span class="node-spacer"></span>', | ||
| 1348 | + f'<span class="node-name">{kernel_seq} {kernel_name}</span>{kernel_semantic_html}', | ||
| 1349 | + f'<span class="kernel-info-btn" data-index="{idx}">{_KERNEL_INFO_SVG}</span>', | ||
| 1350 | + '<span class="node-duration">', | ||
| 1351 | + f'<span class="duration-us">{format_duration_us(k_duration_total)} us</span>', | ||
| 1352 | + f'<span class="duration-pct">({k_pct}%)</span>', | ||
| 1353 | + '</span>', | ||
| 1354 | + '</div>', | ||
| 1355 | + '<div class="kernel-meta">', | ||
| 1356 | + ] | ||
| 1357 | + for field_key in ALL_KERNEL_META_FIELDS: | ||
| 1358 | + lines.append(_render_kernel_meta_field(k, field_key, krc, k_shape_semantic)) | ||
| 1359 | + lines.append('</div>') | ||
| 1360 | + lines.append('</li>') | ||
| 1361 | + return lines, (idx, tooltip_html) | ||
| 1362 | + | ||
| 1363 | + | ||
| 1364 | +def _node_semantic_suffix(node): | ||
| 1365 | + """构建节点名后的 semantic | [code_ref] 展开块;无则返回 ''。""" | ||
| 1366 | + semantic_parts = [] | ||
| 1367 | + node_semantic = node.get('semantic', '') or node.get('comment', '') | ||
| 1368 | + if node_semantic: | ||
| 1369 | + semantic_parts.append(node_semantic) | ||
| 1370 | + if node.get('code_ref'): | ||
| 1371 | + semantic_parts.append(f'[{node["code_ref"]}]') | ||
| 1372 | + if not semantic_parts: | ||
| 1373 | + return '' | ||
| 1374 | + full_text = ' | '.join(semantic_parts) | ||
| 1375 | + expand_btn = ('<span class="semantic-expand-btn"><svg viewBox="0 0 24 24">' | ||
| 1376 | + '<path d="M7 10l5 5 5-5z"/></svg></span>') | ||
| 1377 | + display_text = full_text[:117] + '...' if len(full_text) > 120 else full_text | ||
| 1378 | + return (f' <span class="node-semantic-wrapper" data-full="{full_text}">' | ||
| 1379 | + f'<span class="node-semantic-truncated">{display_text}</span>' | ||
| 1380 | + f'<span class="node-semantic-full">{full_text}</span>' | ||
| 1381 | + f'{expand_btn}</span>') | ||
| 1382 | + | ||
| 1383 | + | ||
| 1384 | +def _node_header_attrs(info: NodeHeaderInfo): | ||
| 1385 | + """计算模块节点头部的派生属性:(kernel_count_str, stream_count_str, duration_hover, node_class)。""" | ||
| 1386 | + time_info = info.time_info | ||
| 1387 | + kernel_count = info.kernel_count | ||
| 1388 | + if not info.is_leaf and kernel_count > 0: | ||
| 1389 | + kernel_count_str = f'<span class="kernel-count">({kernel_count} kernels)</span>' | ||
| 1390 | + stream_count_str = (f'<span class="stream-count" title="{time_info["stream_hover"]}">' | ||
| 1391 | + f'({time_info["stream_info"]})</span>') | ||
| 1392 | + else: | ||
| 1393 | + kernel_count_str = '' | ||
| 1394 | + stream_count_str = '' | ||
| 1395 | + | ||
| 1396 | + duration_hover = time_info['calculation'] if kernel_count > 0 else '' | ||
| 1397 | + if duration_hover and info.multiplier > 1: | ||
| 1398 | + duration_hover = f"{duration_hover} × {info.multiplier} = {time_info['time_span'] * info.multiplier:.2f} us" | ||
| 1399 | + | ||
| 1400 | + node_class = 'expanded' if info.current_depth < info.max_depth else 'collapsed' | ||
| 1401 | + if info.is_leaf: | ||
| 1402 | + node_class = 'leaf' | ||
| 1403 | + elif info.is_kernel_only: | ||
| 1404 | + node_class += ' kernel-only' | ||
| 1405 | + return kernel_count_str, stream_count_str, duration_hover, node_class | ||
| 1406 | + | ||
| 1407 | + | ||
| 1408 | +def _module_node_header_html(node, hdr: NodeHeaderView): | ||
| 1409 | + """渲染模块节点的头部 HTML 行列表(<li> 开标签 + node-header)。""" | ||
| 1410 | + category_attr = f' data-category="{hdr.data_category}"' if hdr.data_category else '' | ||
| 1411 | + semantic_suffix = _node_semantic_suffix(node) | ||
| 1412 | + duration_title = f' title="{hdr.duration_hover}"' if hdr.duration_hover else '' | ||
| 1413 | + return [ | ||
| 1414 | + f'<li class="tree-node {hdr.node_class}" data-depth="{hdr.current_depth}" data-type="module"{category_attr}>', | ||
| 1415 | + '<div class="node-header">', | ||
| 1416 | + '<span class="node-toggle">▶</span>', | ||
| 1417 | + f'<span class="node-name">{hdr.name}{hdr.count_str} ' | ||
| 1418 | + f'{hdr.kernel_count_str}{hdr.stream_count_str}{semantic_suffix}</span>', | ||
| 1419 | + f'<span class="node-duration"{duration_title}>', | ||
| 1420 | + f'<span class="duration-us">{format_duration_us(hdr.duration)} us</span>', | ||
| 1421 | + f'<span class="duration-ms">({format_duration_ms(hdr.duration)} ms)</span>', | ||
| 1422 | + f'<span class="duration-pct">({hdr.percentage}%)</span>', | ||
| 1423 | + '</span>', | ||
| 1424 | + '</div>', | ||
| 1425 | + ] | ||
| 1426 | + | ||
| 1427 | + | ||
| 1428 | +def render_html_tree_node(node: dict, ctx: HtmlRenderCtx, multiplier: int, | ||
| 1429 | + current_depth: int, data_category: str = '') -> tuple: | ||
| 1430 | + operators = ctx.operators | ||
| 1431 | + display_fields = ctx.kernel_display_fields or DEFAULT_KERNEL_DISPLAY_FIELDS | ||
| 1432 | + name = node.get('name', 'Unknown') | ||
| 1433 | + op_indices = node.get('op_indices', []) | ||
| 1434 | + children = node.get('children', []) | ||
| 1435 | + | ||
| 1436 | + has_kernels = bool(op_indices) | ||
| 1437 | + has_children = bool(children) | ||
| 1438 | + is_leaf = not has_kernels and not has_children | ||
| 1439 | + is_kernel_only = has_kernels and not has_children | ||
| 1440 | + | ||
| 1441 | + time_info = get_node_time_span_info(node, operators) | ||
| 1442 | + duration = get_node_total_duration(node, operators) * multiplier | ||
| 1443 | + kernel_count = count_all_kernels(node, operators) | ||
| 1444 | + percentage = format_percentage(duration, ctx.total_duration) | ||
| 1445 | + count_str = f" (*{multiplier}层)" if multiplier > 1 and current_depth == 1 else "" | ||
| 1446 | + | ||
| 1447 | + kernel_count_str, stream_count_str, duration_hover, node_class = _node_header_attrs( | ||
| 1448 | + NodeHeaderInfo(time_info=time_info, kernel_count=kernel_count, multiplier=multiplier, | ||
| 1449 | + current_depth=current_depth, max_depth=ctx.max_depth, | ||
| 1450 | + is_leaf=is_leaf, is_kernel_only=is_kernel_only)) | ||
| 1451 | + | ||
| 1452 | + html_parts = _module_node_header_html(node, NodeHeaderView( | ||
| 1453 | + name=name, count_str=count_str, node_class=node_class, current_depth=current_depth, | ||
| 1454 | + data_category=data_category, kernel_count_str=kernel_count_str, | ||
| 1455 | + stream_count_str=stream_count_str, duration_hover=duration_hover, | ||
| 1456 | + duration=duration, percentage=percentage)) | ||
| 1457 | + | ||
| 1458 | + tooltip_parts = [] | ||
| 1459 | + | ||
| 1460 | + if has_kernels or has_children: | ||
| 1461 | + html_parts.append('<ul class="tree-children">') | ||
| 1462 | + | ||
| 1463 | + if has_children: | ||
| 1464 | + for child in children: | ||
| 1465 | + child_html, child_tooltips = render_html_tree_node( | ||
| 1466 | + child, ctx, multiplier, current_depth + 1) | ||
| 1467 | + html_parts.append(child_html) | ||
| 1468 | + tooltip_parts.extend(child_tooltips) | ||
| 1469 | + | ||
| 1470 | + if has_kernels: | ||
| 1471 | + kernels = get_kernel_full_details_by_indices(operators, op_indices) | ||
| 1472 | + total_kernels = len(kernels) | ||
| 1473 | + krc = KernelRenderCtx(ctx=ctx, multiplier=multiplier, | ||
| 1474 | + current_depth=current_depth, display_fields=display_fields) | ||
| 1475 | + for ki, k in enumerate(kernels): | ||
| 1476 | + lines, tip = _render_one_kernel(k, ki, total_kernels, krc) | ||
| 1477 | + tooltip_parts.append(tip) | ||
| 1478 | + html_parts.extend(lines) | ||
| 1479 | + | ||
| 1480 | + html_parts.append('</ul>') | ||
| 1481 | + | ||
| 1482 | + html_parts.append('</li>') | ||
| 1483 | + return '\n'.join(html_parts), tooltip_parts | ||
| 1484 | + | ||
| 1485 | + | ||
| 1486 | +def _compute_model_span(operators): | ||
| 1487 | + """计算模型整体 duration 与时间跨度说明文本,返回 (model_duration, calculation)。""" | ||
| 1488 | + if not operators: | ||
| 1489 | + return 0, '' | ||
| 1490 | + model_duration = sum(op.get('duration_us', 0) for op in operators) | ||
| 1491 | + earliest_op = min(operators, key=lambda op: op.get('start_time_us', 0)) | ||
| 1492 | + latest_op = max(operators, key=lambda op: op.get('start_time_us', 0) + op.get('duration_us', 0)) | ||
| 1493 | + min_start = earliest_op.get('start_time_us', 0) | ||
| 1494 | + max_end = latest_op.get('start_time_us', 0) + latest_op.get('duration_us', 0) | ||
| 1495 | + earliest_name = earliest_op.get('normalized_name') or earliest_op.get('type', 'Unknown') | ||
| 1496 | + latest_name = latest_op.get('normalized_name') or latest_op.get('type', 'Unknown') | ||
| 1497 | + latest_start = latest_op.get('start_time_us', 0) | ||
| 1498 | + latest_duration = latest_op.get('duration_us', 0) | ||
| 1499 | + time_span = max_end - min_start | ||
| 1500 | + calculation = (f"{latest_name}({latest_start:.2f}+{latest_duration:.2f}) - " | ||
| 1501 | + f"{earliest_name}({min_start:.2f}) = {time_span:.2f} us") | ||
| 1502 | + return model_duration, calculation | ||
| 1503 | + | ||
| 1504 | + | ||
| 1505 | +def _tree_root_header(display_name, model_duration, percentage, duration_hover): | ||
| 1506 | + """渲染分析树根节点(模型层)的头部 HTML 片段。""" | ||
| 1507 | + duration_title = f' title="{duration_hover}"' if duration_hover else '' | ||
| 1508 | + return [ | ||
| 1509 | + '<li class="tree-node expanded" data-depth="0" data-type="module">', | ||
| 1510 | + '<div class="node-header">', | ||
| 1511 | + '<span class="node-toggle">▶</span>', | ||
| 1512 | + f'<span class="node-name">{display_name}</span>', | ||
| 1513 | + f'<span class="node-duration"{duration_title}>', | ||
| 1514 | + f'<span class="duration-us">{format_duration_us(model_duration)} us</span>', | ||
| 1515 | + f'<span class="duration-ms">({format_duration_ms(model_duration)} ms)</span>', | ||
| 1516 | + f'<span class="duration-pct">({percentage}%)</span>', | ||
| 1517 | + '</span>', | ||
| 1518 | + '</div>', | ||
| 1519 | + '<ul class="tree-children">', | ||
| 1520 | + ] | ||
| 1521 | + | ||
| 1522 | + | ||
| 1523 | +def generate_html_tree_section(config: dict, operators: list, total_duration: float, | ||
| 1524 | + max_depth: int, kernel_display_fields: list = None) -> tuple: | ||
| 1525 | + model_name = config.get('model_name', 'Model') | ||
| 1526 | + display_name, _ = _extract_model_display_name(model_name) | ||
| 1527 | + kernel_semantics = collect_kernel_semantics(config) | ||
| 1528 | + display_fields = kernel_display_fields or DEFAULT_KERNEL_DISPLAY_FIELDS | ||
| 1529 | + | ||
| 1530 | + model_duration, calculation = _compute_model_span(operators) | ||
| 1531 | + percentage = format_percentage(model_duration, total_duration) | ||
| 1532 | + | ||
| 1533 | + stages = config.get('stages', {}) | ||
| 1534 | + layer_types = config.get('layer_types', {}) | ||
| 1535 | + layer_structure = config.get('layer_structure', {}) | ||
| 1536 | + runtime_aux = config.get('runtime_auxiliary', []) | ||
| 1537 | + | ||
| 1538 | + html_parts = ['<section id="analysis">', '<div class="tree">', '<ul class="tree-root">'] | ||
| 1539 | + html_parts.extend(_tree_root_header(display_name, model_duration, percentage, | ||
| 1540 | + calculation if calculation else '')) | ||
| 1541 | + | ||
| 1542 | + all_tooltips = [] | ||
| 1543 | + tree_ctx = HtmlRenderCtx(operators=operators, total_duration=total_duration, | ||
| 1544 | + max_depth=max_depth, kernel_semantics=kernel_semantics, | ||
| 1545 | + kernel_display_fields=display_fields) | ||
| 1546 | + | ||
| 1547 | + for _stage_name, stage_info in stages.items(): | ||
| 1548 | + stage_indices = stage_info.get('stage_indices', [0]) | ||
| 1549 | + stage_count = len(stage_indices) if stage_indices else 1 | ||
| 1550 | + node_html, node_tooltips = render_html_tree_node( | ||
| 1551 | + stage_info, tree_ctx, stage_count, 1, data_category='auxiliary') | ||
| 1552 | + html_parts.append(node_html) | ||
| 1553 | + all_tooltips.extend(node_tooltips) | ||
| 1554 | + | ||
| 1555 | + for layer_type, layer_info in layer_types.items(): | ||
| 1556 | + structure = layer_structure.get(layer_type, {}) | ||
| 1557 | + if not structure: | ||
| 1558 | + continue | ||
| 1559 | + layer_count = len(layer_info.get('layer_indices', [])) | ||
| 1560 | + node_html, node_tooltips = render_html_tree_node(structure, tree_ctx, layer_count, 1) | ||
| 1561 | + html_parts.append(node_html) | ||
| 1562 | + all_tooltips.extend(node_tooltips) | ||
| 1563 | + | ||
| 1564 | + for aux in runtime_aux: | ||
| 1565 | + node_html, node_tooltips = render_html_tree_node( | ||
| 1566 | + aux, tree_ctx, 1, 1, data_category='auxiliary') | ||
| 1567 | + html_parts.append(node_html) | ||
| 1568 | + all_tooltips.extend(node_tooltips) | ||
| 1569 | + | ||
| 1570 | + html_parts.extend(['</ul>', '</li>', '</ul>', '</div>', '</section>']) | ||
| 1571 | + | ||
| 1572 | + return '\n'.join(html_parts), dict(all_tooltips) | ||
| 1573 | + | ||
| 1574 | + | ||
| 1575 | +def _extract_model_display_name(model_name: str) -> tuple: | ||
| 1576 | + base_name = model_name | ||
| 1577 | + architecture_desc = '' | ||
| 1578 | + paren_start = model_name.find('(') | ||
| 1579 | + if paren_start >= 0: | ||
| 1580 | + paren_end = model_name.rfind(')') | ||
| 1581 | + if paren_end > paren_start: | ||
| 1582 | + architecture_desc = model_name[paren_start + 1:paren_end].strip() | ||
| 1583 | + base_name = model_name[:paren_start].strip() | ||
| 1584 | + return base_name, architecture_desc | ||
| 1585 | + | ||
| 1586 | + | ||
| 1587 | +def _html_metadata_section(meta: ReportMeta): | ||
| 1588 | + """报告 metadata 区 HTML 片段列表。""" | ||
| 1589 | + return [ | ||
| 1590 | + '<div class="metadata">', | ||
| 1591 | + '<div class="meta-row">', | ||
| 1592 | + (f'<div class="meta-item"><span class="meta-label">生成时间:</span>' | ||
| 1593 | + f'<span class="meta-value">{meta.generate_time}</span></div>'), | ||
| 1594 | + (f'<div class="meta-item"><span class="meta-label">总耗时:</span>' | ||
| 1595 | + f'<span class="meta-value">{format_duration_us(meta.total_duration)} us ' | ||
| 1596 | + f'({format_duration_ms(meta.total_duration)} ms)</span></div>'), | ||
| 1597 | + f'<div class="meta-item"><span class="meta-label">Step:</span>' | ||
| 1598 | + f'<span class="meta-value">{meta.step_id}</span></div>', | ||
| 1599 | + (f'<div class="meta-item"><span class="meta-label">Kernel数量:</span>' | ||
| 1600 | + f'<span class="meta-value">{meta.kernel_count}</span></div>'), | ||
| 1601 | + '</div>', | ||
| 1602 | + '<div class="meta-row">', | ||
| 1603 | + '<div class="meta-item" style="flex-direction: column; align-items: flex-start; gap: 4px;">', | ||
| 1604 | + '<span class="meta-label">数据源:</span>', | ||
| 1605 | + '<div class="meta-datasources">', | ||
| 1606 | + f'<div class="meta-datasource-item">{meta.raw_ops_path}</div>', | ||
| 1607 | + f'<div class="meta-datasource-item">{meta.config_path}</div>', | ||
| 1608 | + '</div>', | ||
| 1609 | + '</div>', | ||
| 1610 | + '</div>', | ||
| 1611 | + '<div class="meta-row">', | ||
| 1612 | + (f'<div class="meta-item"><span class="meta-label">模型简介:</span>' | ||
| 1613 | + f'<span class="meta-value">{meta.architecture_desc}</span></div>' | ||
| 1614 | + if meta.architecture_desc else ''), | ||
| 1615 | + '</div>', | ||
| 1616 | + '</div>', | ||
| 1617 | + ] | ||
| 1618 | + | ||
| 1619 | + | ||
| 1620 | +_KERNEL_FIELD_CHECKBOXES = [ | ||
| 1621 | + ('stream_id', 'Stream', True), ('input_shapes', 'Input Shapes', True), | ||
| 1622 | + ('output_shapes', 'Output Shapes', True), ('start_time_us', 'Start Time', False), | ||
| 1623 | + ('duration_us', 'Duration', False), ('wait_time_us', 'Wait Time', False), | ||
| 1624 | + ('device_id', 'Device', False), ('task_id', 'Task ID', False), | ||
| 1625 | + ('type', 'Type', False), ('op_state', 'OP State', False), | ||
| 1626 | + ('accelerator_core', 'Acc Core', False), ('block_dim', 'Block Dim', False), | ||
| 1627 | + ('input_data_types', 'In DType', False), ('output_data_types', 'Out DType', False), | ||
| 1628 | + ('input_formats', 'In Fmt', False), ('output_formats', 'Out Fmt', False), | ||
| 1629 | +] | ||
| 1630 | + | ||
| 1631 | + | ||
| 1632 | +def _html_controls_section(): | ||
| 1633 | + """报告 controls 工具栏 HTML 片段列表(全静态)。""" | ||
| 1634 | + parts = [ | ||
| 1635 | + '<div class="controls">', | ||
| 1636 | + '<button id="toggle-expand-btn">全部展开</button>', | ||
| 1637 | + '<select id="depth-select">', | ||
| 1638 | + '<option value="1">1层</option>', | ||
| 1639 | + '<option value="2">2层</option>', | ||
| 1640 | + '<option value="3" selected>3层</option>', | ||
| 1641 | + '<option value="4">4层</option>', | ||
| 1642 | + '<option value="999">全部</option>', | ||
| 1643 | + '</select>', | ||
| 1644 | + '<button id="toggle-auxiliary-btn">显示辅助层次</button>', | ||
| 1645 | + '<button id="toggle-kernels-btn">隐藏Kernel序列</button>', | ||
| 1646 | + '<button id="toggle-kernel-meta-btn">隐藏Kernel信息</button>', | ||
| 1647 | + '<div class="kernel-fields-config">', | ||
| 1648 | + '<button id="kernel-fields-btn">⚙Kernel显示配置</button>', | ||
| 1649 | + '<div class="kernel-fields-panel" id="kernel-fields-panel">', | ||
| 1650 | + ] | ||
| 1651 | + for val, label, checked in _KERNEL_FIELD_CHECKBOXES: | ||
| 1652 | + chk = ' checked' if checked else '' | ||
| 1653 | + parts.append(f'<label><input type="checkbox" value="{val}"{chk}> {label}</label>') | ||
| 1654 | + parts += [ | ||
| 1655 | + '<div class="kernel-fields-actions">', | ||
| 1656 | + '<button id="fields-select-all">全选</button>', | ||
| 1657 | + '<button id="fields-select-none">全不选</button>', | ||
| 1658 | + '<button id="fields-reset">默认</button>', | ||
| 1659 | + '</div>', | ||
| 1660 | + '</div>', | ||
| 1661 | + '</div>', | ||
| 1662 | + '<button id="toggle-semantic-btn">展开全部语义</button>', | ||
| 1663 | + '<label>主题风格: <select id="theme-select">', | ||
| 1664 | + '<option value="dracula" selected>Dracula</option>', | ||
| 1665 | + '<option value="vscode-dark">VS Code Dark</option>', | ||
| 1666 | + '<option value="one-dark">One Dark Pro</option>', | ||
| 1667 | + '<option value="github-light">GitHub Light</option>', | ||
| 1668 | + '<option value="solarized-light">Solarized Light</option>', | ||
| 1669 | + '</select></label>', | ||
| 1670 | + '</div>', | ||
| 1671 | + ] | ||
| 1672 | + return parts | ||
| 1673 | + | ||
| 1674 | + | ||
| 1675 | +def generate_html_report(raw_ops: dict, config: dict, operators: list, total_duration: float, | ||
| 1676 | + opts: ReportOptions) -> str: | ||
| 1677 | + import datetime | ||
| 1678 | + depth = opts.depth | ||
| 1679 | + theme = opts.theme | ||
| 1680 | + kernel_display_fields = opts.kernel_display_fields | ||
| 1681 | + raw_ops_path = opts.raw_ops_path | ||
| 1682 | + config_path = opts.config_path | ||
| 1683 | + model_name = config.get('model_name', 'Model') | ||
| 1684 | + display_name, architecture_desc = _extract_model_display_name(model_name) | ||
| 1685 | + generate_time = datetime.datetime.now(datetime.timezone.utc).astimezone().strftime('%Y-%m-%d %H:%M:%S') | ||
| 1686 | + kernel_count = raw_ops.get('kernel_count', len(operators)) | ||
| 1687 | + step_id = raw_ops.get('step_id', 'N/A') | ||
| 1688 | + | ||
| 1689 | + display_fields = kernel_display_fields or DEFAULT_KERNEL_DISPLAY_FIELDS | ||
| 1690 | + tree_html, tooltip_data = generate_html_tree_section( | ||
| 1691 | + config, operators, total_duration, depth, kernel_display_fields) | ||
| 1692 | + timeline_html, timeline_data = generate_timeline_html(config, operators, total_duration, depth) | ||
| 1693 | + | ||
| 1694 | + html_parts = [ | ||
| 1695 | + '<!DOCTYPE html>', | ||
| 1696 | + '<html lang="zh-CN">', | ||
| 1697 | + '<head>', | ||
| 1698 | + '<meta charset="UTF-8">', | ||
| 1699 | + '<meta name="viewport" content="width=device-width, initial-scale=1.0">', | ||
| 1700 | + f'<title>{display_name} 性能分析</title>', | ||
| 1701 | + get_html_css(), | ||
| 1702 | + '</head>', | ||
| 1703 | + '<body>', | ||
| 1704 | + f'<h1>{display_name} 性能分析</h1>', | ||
| 1705 | + ] | ||
| 1706 | + html_parts += _html_metadata_section(ReportMeta( | ||
| 1707 | + generate_time=generate_time, total_duration=total_duration, step_id=step_id, | ||
| 1708 | + kernel_count=kernel_count, raw_ops_path=raw_ops_path, config_path=config_path, | ||
| 1709 | + architecture_desc=architecture_desc)) | ||
| 1710 | + html_parts += _html_controls_section() | ||
| 1711 | + html_parts += [ | ||
| 1712 | + tree_html, | ||
| 1713 | + timeline_html, | ||
| 1714 | + get_html_js(theme, tooltip_data, display_fields, timeline_data), | ||
| 1715 | + '</body>', | ||
| 1716 | + '</html>' | ||
| 1717 | + ] | ||
| 1718 | + | ||
| 1719 | + return '\n'.join(html_parts) | ||
| 1720 | + | ||
| 1721 | + | ||
| 1722 | +def generate_report(raw_ops_path: str, config_path: str, opts: ReportOptions = None) -> str: | ||
| 1723 | + if opts is None: | ||
| 1724 | + opts = ReportOptions() | ||
| 1725 | + output_path = opts.output_path | ||
| 1726 | + depth = opts.depth | ||
| 1727 | + html = opts.html | ||
| 1728 | + html_output = opts.html_output | ||
| 1729 | + raw_ops_file = validate_file_exists(raw_ops_path) | ||
| 1730 | + config_file = validate_file_exists(config_path) | ||
| 1731 | + | ||
| 1732 | + raw_ops = load_json(raw_ops_file) | ||
| 1733 | + config = load_json(config_file) | ||
| 1734 | + | ||
| 1735 | + validate_raw_ops(raw_ops) | ||
| 1736 | + validate_analysis_config(config) | ||
| 1737 | + | ||
| 1738 | + operators = raw_ops.get('operators', []) | ||
| 1739 | + total_duration = raw_ops.get('total_duration_us', 1) | ||
| 1740 | + | ||
| 1741 | + report = None | ||
| 1742 | + | ||
| 1743 | + if output_path: | ||
| 1744 | + report_parts = [] | ||
| 1745 | + report_parts.append(f"# {config.get('model_name', 'Model')} 性能拆解报告") | ||
| 1746 | + report_parts.append("") | ||
| 1747 | + report_parts.append(generate_analysis_section(config, operators, total_duration, depth)) | ||
| 1748 | + report = '\n'.join(report_parts) | ||
| 1749 | + with open(output_path, 'w', encoding='utf-8') as f: | ||
| 1750 | + f.write(report) | ||
| 1751 | + logger.info("Markdown报告已生成: %s", output_path) | ||
| 1752 | + | ||
| 1753 | + if html: | ||
| 1754 | + html_opts = ReportOptions(output_path=output_path, depth=depth, html=html, | ||
| 1755 | + html_output=html_output, theme=opts.theme, | ||
| 1756 | + kernel_display_fields=opts.kernel_display_fields, | ||
| 1757 | + raw_ops_path=raw_ops_path, config_path=config_path) | ||
| 1758 | + html_content = generate_html_report(raw_ops, config, operators, total_duration, html_opts) | ||
| 1759 | + if html_output: | ||
| 1760 | + html_path = html_output | ||
| 1761 | + elif output_path: | ||
| 1762 | + html_path = str(Path(output_path).with_suffix('.html')) | ||
| 1763 | + else: | ||
| 1764 | + html_path = 'report.html' | ||
| 1765 | + with open(html_path, 'w', encoding='utf-8') as f: | ||
| 1766 | + f.write(html_content) | ||
| 1767 | + logger.info("HTML报告已生成: %s", html_path) | ||
| 1768 | + | ||
| 1769 | + if not output_path and not html: | ||
| 1770 | + report_parts = [] | ||
| 1771 | + report_parts.append(f"# {config.get('model_name', 'Model')} 性能拆解报告") | ||
| 1772 | + report_parts.append("") | ||
| 1773 | + report_parts.append(generate_analysis_section(config, operators, total_duration, depth)) | ||
| 1774 | + report = '\n'.join(report_parts) | ||
| 1775 | + | ||
| 1776 | + return report | ||
| 1777 | + | ||
| 1778 | + | ||
| 1779 | +def _build_arg_parser(): | ||
| 1780 | + parser = argparse.ArgumentParser( | ||
| 1781 | + description='NPU 性能拆解报告生成脚本', | ||
| 1782 | + formatter_class=argparse.RawDescriptionHelpFormatter, | ||
| 1783 | + epilog=''' | ||
| 1784 | +示例: | ||
| 1785 | + %(prog)s raw_ops.json analysis_config.json | ||
| 1786 | + %(prog)s raw_ops.json analysis_config.json -o report.md | ||
| 1787 | + %(prog)s -r raw_ops.json -c analysis_config.json -o report.md | ||
| 1788 | + %(prog)s raw_ops.json analysis_config.json -d 3 -o report.md | ||
| 1789 | + %(prog)s raw_ops.json analysis_config.json --html | ||
| 1790 | + %(prog)s raw_ops.json analysis_config.json -o report.md --html | ||
| 1791 | + ''' | ||
| 1792 | + ) | ||
| 1793 | + parser.add_argument('raw_ops', nargs='?', help='算子序列JSON文件路径') | ||
| 1794 | + parser.add_argument('config', nargs='?', help='模型层次拆解JSON文件路径') | ||
| 1795 | + parser.add_argument('-r', '--raw-ops', dest='raw_ops_opt', metavar='FILE', | ||
| 1796 | + help='算子序列JSON文件路径') | ||
| 1797 | + parser.add_argument('-c', '--config', dest='config_opt', metavar='FILE', | ||
| 1798 | + help='模型层次拆解JSON文件路径') | ||
| 1799 | + parser.add_argument('-o', '--output', metavar='FILE', | ||
| 1800 | + help='Markdown输出报告文件路径 (默认打印到标准输出)') | ||
| 1801 | + parser.add_argument('-d', '--depth', type=int, default=3, | ||
| 1802 | + help='树状结构展示深度 (默认: 3)') | ||
| 1803 | + parser.add_argument('--html', action='store_true', help='生成HTML格式报告') | ||
| 1804 | + parser.add_argument('--html-output', metavar='FILE', | ||
| 1805 | + help='HTML输出文件路径 (默认: 与-o同名但后缀为.html)') | ||
| 1806 | + parser.add_argument('--theme', choices=['dracula', 'vscode-dark', 'one-dark', 'github-light', 'solarized-light'], | ||
| 1807 | + default='dracula', help='HTML报告主题风格 (默认: dracula)') | ||
| 1808 | + parser.add_argument('--kernel-fields', metavar='FIELDS', | ||
| 1809 | + help=('kernel默认显示字段(逗号分隔), 如: input_shapes,' | ||
| 1810 | + 'output_shapes,type,stream_id,start_time_us,duration_us')) | ||
| 1811 | + return parser | ||
| 1812 | + | ||
| 1813 | + | ||
| 1814 | +def main(): | ||
| 1815 | + logging.basicConfig(level=logging.INFO, format='%(message)s', stream=sys.stdout) | ||
| 1816 | + args = _build_arg_parser().parse_args() | ||
| 1817 | + | ||
| 1818 | + raw_ops_path = args.raw_ops_opt or args.raw_ops | ||
| 1819 | + config_path = args.config_opt or args.config | ||
| 1820 | + if not raw_ops_path or not config_path: | ||
| 1821 | + _build_arg_parser().print_help() | ||
| 1822 | + sys.exit(1) | ||
| 1823 | + | ||
| 1824 | + kernel_display_fields = None | ||
| 1825 | + if args.kernel_fields: | ||
| 1826 | + kernel_display_fields = [f.strip() for f in args.kernel_fields.split(',') if f.strip()] | ||
| 1827 | + | ||
| 1828 | + try: | ||
| 1829 | + opts = ReportOptions(output_path=args.output, depth=args.depth, | ||
| 1830 | + html=args.html, html_output=args.html_output, | ||
| 1831 | + theme=args.theme, kernel_display_fields=kernel_display_fields) | ||
| 1832 | + report = generate_report(raw_ops_path, config_path, opts) | ||
| 1833 | + if not args.output and not args.html: | ||
| 1834 | + logger.info(report) | ||
| 1835 | + except (FileNotFoundError, ValueError) as e: | ||
| 1836 | + logger.error("错误: %s", e) | ||
| 1837 | + sys.exit(1) | ||
| 1838 | + except Exception as e: | ||
| 1839 | + logger.error("未知错误: %s", e) | ||
| 1840 | + sys.exit(1) | ||
| 1841 | + | ||
| 1842 | + | ||
| 1843 | +if __name__ == '__main__': | ||
| 1844 | + main() | ||
| @@ -0,0 +1,286 @@ | |||
| 1 | +#!/usr/bin/env python3 | ||
| 2 | +# -*- coding: utf-8 -*- | ||
| 3 | +# ---------------------------------------------------------------------------- | ||
| 4 | +# Copyright (c) 2025 Huawei Technologies Co., Ltd. | ||
| 5 | +# | ||
| 6 | +# Licensed under the Apache License, Version 2.0 (the "License"); | ||
| 7 | +# you may not use this file except in compliance with the License. | ||
| 8 | +# You may obtain a copy of the License at | ||
| 9 | +# | ||
| 10 | +# http://www.apache.org/licenses/LICENSE-2.0 | ||
| 11 | +# | ||
| 12 | +# Unless required by applicable law or agreed to in writing, software | ||
| 13 | +# distributed under the License is distributed on an "AS IS" BASIS, | ||
| 14 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| 15 | +# See the License for the specific language governing permissions and | ||
| 16 | +# limitations under the License. | ||
| 17 | +# ---------------------------------------------------------------------------- | ||
| 18 | +""" | ||
| 19 | +Step 3 / P8: 与 baseline 做结构等价性回归 (L1–L8)。 | ||
| 20 | + | ||
| 21 | +L1: top-level keys 集合相等 | ||
| 22 | +L2: layer_types 集合 + layer_indices 列表相等 | ||
| 23 | +L3: stages keys 与 stage_indices 集合相等 | ||
| 24 | +L4: runtime_auxiliary 名字集合相等 | ||
| 25 | +L5: 每个 layer_type 的叶节点路径集合 Jaccard ≥ 0.95 | ||
| 26 | +L6: 每个叶节点的 op_indices 集合 Jaccard ≥ 0.90 | ||
| 27 | +L7: op 总覆盖(union of op_indices)相等 | ||
| 28 | +L8: 11 类算子的 shape_semantic 字段存在性 100% | ||
| 29 | +""" | ||
| 30 | +import logging | ||
| 31 | +import argparse | ||
| 32 | +import json | ||
| 33 | +import os | ||
| 34 | +import sys | ||
| 35 | + | ||
| 36 | +from _common import is_shape_always_required as is_shape_required | ||
| 37 | + | ||
| 38 | +logger = logging.getLogger(__name__) | ||
| 39 | + | ||
| 40 | + | ||
| 41 | +def jaccard(a: set, b: set) -> float: | ||
| 42 | + if not a and not b: | ||
| 43 | + return 1.0 | ||
| 44 | + return len(a & b) / max(len(a | b), 1) | ||
| 45 | + | ||
| 46 | + | ||
| 47 | +def collect_leaves(layer_struct): | ||
| 48 | + """返回 {leaf_path: set(op_indices)}""" | ||
| 49 | + out = {} | ||
| 50 | + | ||
| 51 | + def walk(node, path): | ||
| 52 | + if not isinstance(node, dict): | ||
| 53 | + return | ||
| 54 | + if 'op_indices' in node and node.get('op_indices') is not None: | ||
| 55 | + out[path] = set(node['op_indices']) | ||
| 56 | + for child in node.get('children', []) or []: | ||
| 57 | + cname = child.get('name', '?') | ||
| 58 | + walk(child, f'{path}/{cname}') | ||
| 59 | + | ||
| 60 | + walk(layer_struct, layer_struct.get('name', 'root')) | ||
| 61 | + return out | ||
| 62 | + | ||
| 63 | + | ||
| 64 | +def collect_all_op_indices(config): | ||
| 65 | + out = set() | ||
| 66 | + | ||
| 67 | + def walk(node): | ||
| 68 | + if not isinstance(node, dict): | ||
| 69 | + return | ||
| 70 | + for idx in node.get('op_indices', []) or []: | ||
| 71 | + out.add(idx) | ||
| 72 | + for child in node.get('children', []) or []: | ||
| 73 | + walk(child) | ||
| 74 | + | ||
| 75 | + for s in (config.get('stages') or {}).values(): | ||
| 76 | + walk(s) | ||
| 77 | + for ls in (config.get('layer_structure') or {}).values(): | ||
| 78 | + walk(ls) | ||
| 79 | + for aux in (config.get('runtime_auxiliary') or []): | ||
| 80 | + walk(aux) | ||
| 81 | + return out | ||
| 82 | + | ||
| 83 | + | ||
| 84 | +def collect_kernels_with_shape(config): | ||
| 85 | + """返回 {(path, op_index): (kernel_name, has_shape_semantic)}""" | ||
| 86 | + out = {} | ||
| 87 | + | ||
| 88 | + def walk(node, path): | ||
| 89 | + if not isinstance(node, dict): | ||
| 90 | + return | ||
| 91 | + for ks in node.get('kernels', []) or []: | ||
| 92 | + kn = ks.get('name', '') or '' | ||
| 93 | + kn = kn.split('/')[-1] if '/' in kn else kn | ||
| 94 | + idx = ks.get('index') | ||
| 95 | + if idx is not None: | ||
| 96 | + out[(path, idx)] = (kn, bool(ks.get('shape_semantic'))) | ||
| 97 | + for child in node.get('children', []) or []: | ||
| 98 | + walk(child, f'{path}/{child.get("name", "?")}') | ||
| 99 | + | ||
| 100 | + for sname, s in (config.get('stages') or {}).items(): | ||
| 101 | + walk(s, f'stages/{sname}') | ||
| 102 | + for ltype, ls in (config.get('layer_structure') or {}).items(): | ||
| 103 | + walk(ls, f'layer_structure/{ltype}') | ||
| 104 | + for i, aux in enumerate(config.get('runtime_auxiliary') or []): | ||
| 105 | + walk(aux, f'runtime_auxiliary[{i}]') | ||
| 106 | + return out | ||
| 107 | + | ||
| 108 | + | ||
| 109 | +def _reg_l1(baseline, new): | ||
| 110 | + bl_keys, new_keys = set(baseline.keys()), set(new.keys()) | ||
| 111 | + if bl_keys == new_keys: | ||
| 112 | + return {'id': 'L1', 'pass': True, 'detail': f'top-level keys match: {sorted(bl_keys)}'} | ||
| 113 | + return {'id': 'L1', 'pass': False, | ||
| 114 | + 'detail': f'baseline only: {sorted(bl_keys - new_keys)}, ' | ||
| 115 | + f'new only: {sorted(new_keys - bl_keys)}'} | ||
| 116 | + | ||
| 117 | + | ||
| 118 | +def _reg_l2(bl_lt, new_lt, bl_set, new_set): | ||
| 119 | + l2_pass = (bl_set == new_set) and all( | ||
| 120 | + sorted(bl_lt[k].get('layer_indices', [])) == sorted(new_lt[k].get('layer_indices', [])) | ||
| 121 | + for k in bl_set) | ||
| 122 | + return {'id': 'L2', 'pass': l2_pass, | ||
| 123 | + 'detail': f'baseline layer_types={sorted(bl_set)}, new={sorted(new_set)}'} | ||
| 124 | + | ||
| 125 | + | ||
| 126 | +def _reg_l3(baseline, new): | ||
| 127 | + bl_st = set((baseline.get('stages') or {}).keys()) | ||
| 128 | + new_st = set((new.get('stages') or {}).keys()) | ||
| 129 | + return {'id': 'L3', 'pass': bl_st == new_st, | ||
| 130 | + 'detail': f'stages baseline={sorted(bl_st)}, new={sorted(new_st)}'} | ||
| 131 | + | ||
| 132 | + | ||
| 133 | +def _reg_l4(baseline, new): | ||
| 134 | + def names(aux_list): | ||
| 135 | + return {a.get('name') for a in (aux_list or []) if isinstance(a, dict) and a.get('name')} | ||
| 136 | + bl_aux = names(baseline.get('runtime_auxiliary')) | ||
| 137 | + new_aux = names(new.get('runtime_auxiliary')) | ||
| 138 | + return {'id': 'L4', 'pass': bl_aux == new_aux, | ||
| 139 | + 'detail': f'baseline aux={sorted(bl_aux)}, new={sorted(new_aux)}'} | ||
| 140 | + | ||
| 141 | + | ||
| 142 | +def _reg_l5(baseline, new, common_types, l5_threshold): | ||
| 143 | + l5_results = [] | ||
| 144 | + for k in common_types: | ||
| 145 | + bl_paths = set(collect_leaves(baseline['layer_structure'].get(k, {})).keys()) | ||
| 146 | + new_paths = set(collect_leaves(new['layer_structure'].get(k, {})).keys()) | ||
| 147 | + l5_results.append({ | ||
| 148 | + 'layer_type': k, | ||
| 149 | + 'jaccard': round(jaccard(bl_paths, new_paths), 3), | ||
| 150 | + 'baseline_only': sorted(list(bl_paths - new_paths))[:10], | ||
| 151 | + 'new_only': sorted(list(new_paths - bl_paths))[:10], | ||
| 152 | + }) | ||
| 153 | + l5_pass = all(r['jaccard'] >= l5_threshold for r in l5_results) if l5_results else True | ||
| 154 | + return {'id': 'L5', 'pass': l5_pass, 'threshold': l5_threshold, 'detail': l5_results} | ||
| 155 | + | ||
| 156 | + | ||
| 157 | +def _reg_l6(baseline, new, common_types, l6_threshold): | ||
| 158 | + l6_results = [] | ||
| 159 | + for k in common_types: | ||
| 160 | + bl_leaves = collect_leaves(baseline['layer_structure'].get(k, {})) | ||
| 161 | + new_leaves = collect_leaves(new['layer_structure'].get(k, {})) | ||
| 162 | + for path in set(bl_leaves) & set(new_leaves): | ||
| 163 | + j = jaccard(bl_leaves[path], new_leaves[path]) | ||
| 164 | + if j < l6_threshold: | ||
| 165 | + l6_results.append({ | ||
| 166 | + 'layer_type': k, 'leaf': path, 'jaccard': round(j, 3), | ||
| 167 | + 'baseline_only': sorted(list(bl_leaves[path] - new_leaves[path]))[:5], | ||
| 168 | + 'new_only': sorted(list(new_leaves[path] - bl_leaves[path]))[:5], | ||
| 169 | + }) | ||
| 170 | + return {'id': 'L6', 'pass': not l6_results, 'threshold': l6_threshold, | ||
| 171 | + 'mismatches': l6_results[:30], 'mismatch_count': len(l6_results)} | ||
| 172 | + | ||
| 173 | + | ||
| 174 | +def _reg_l7(baseline, new): | ||
| 175 | + bl_ops = collect_all_op_indices(baseline) | ||
| 176 | + new_ops = collect_all_op_indices(new) | ||
| 177 | + return {'id': 'L7', 'pass': bl_ops == new_ops, | ||
| 178 | + 'detail': { | ||
| 179 | + 'baseline_count': len(bl_ops), 'new_count': len(new_ops), | ||
| 180 | + 'missing_in_new': sorted(list(bl_ops - new_ops))[:20], | ||
| 181 | + 'extra_in_new': sorted(list(new_ops - bl_ops))[:20], | ||
| 182 | + }} | ||
| 183 | + | ||
| 184 | + | ||
| 185 | +def _reg_l8(new): | ||
| 186 | + new_kernels = collect_kernels_with_shape(new) | ||
| 187 | + missing = [(p, idx, kn) for (p, idx), (kn, ok) in new_kernels.items() | ||
| 188 | + if is_shape_required(kn) and not ok] | ||
| 189 | + return {'id': 'L8', 'pass': not missing, | ||
| 190 | + 'missing_count': len(missing), 'missing_examples': missing[:10]} | ||
| 191 | + | ||
| 192 | + | ||
| 193 | +def check_regression(baseline, new, l5_threshold=0.95, l6_threshold=0.90): | ||
| 194 | + bl_lt = baseline.get('layer_types') or {} | ||
| 195 | + new_lt = new.get('layer_types') or {} | ||
| 196 | + bl_set, new_set = set(bl_lt.keys()), set(new_lt.keys()) | ||
| 197 | + common_types = bl_set & new_set | ||
| 198 | + return [ | ||
| 199 | + _reg_l1(baseline, new), | ||
| 200 | + _reg_l2(bl_lt, new_lt, bl_set, new_set), | ||
| 201 | + _reg_l3(baseline, new), | ||
| 202 | + _reg_l4(baseline, new), | ||
| 203 | + _reg_l5(baseline, new, common_types, l5_threshold), | ||
| 204 | + _reg_l6(baseline, new, common_types, l6_threshold), | ||
| 205 | + _reg_l7(baseline, new), | ||
| 206 | + _reg_l8(new), | ||
| 207 | + ] | ||
| 208 | + | ||
| 209 | + | ||
| 210 | +def _emit_findings(args, findings, hard_fails, soft_fails): | ||
| 211 | + """按 --json / 文本模式输出 findings。""" | ||
| 212 | + if args.json: | ||
| 213 | + logger.info(json.dumps({ | ||
| 214 | + 'script': 'regression_check.py', | ||
| 215 | + 'baseline': args.baseline, | ||
| 216 | + 'new': args.new, | ||
| 217 | + 'mode': args.mode, | ||
| 218 | + 'hard_fails': len(hard_fails), | ||
| 219 | + 'soft_fails': len(soft_fails), | ||
| 220 | + 'findings': findings, | ||
| 221 | + }, indent=2, ensure_ascii=False)) | ||
| 222 | + return | ||
| 223 | + for f in findings: | ||
| 224 | + mark = '✓' if f['pass'] else '✗' | ||
| 225 | + logger.info('[%s] %s', mark, f["id"]) | ||
| 226 | + if not f['pass']: | ||
| 227 | + logger.info(' detail: %s', json.dumps(f.get("detail", f), ensure_ascii=False)[:500]) | ||
| 228 | + logger.info('\n汇总: hard_fails=%d, soft_fails=%d', len(hard_fails), len(soft_fails)) | ||
| 229 | + | ||
| 230 | + | ||
| 231 | +def _write_regression_report(report_path, args, findings): | ||
| 232 | + """写 Markdown 回归报告。""" | ||
| 233 | + lines = [f'# Regression Check: {os.path.basename(args.new)} vs baseline\n', | ||
| 234 | + f'- Baseline: `{args.baseline}`', | ||
| 235 | + f'- New: `{args.new}`', | ||
| 236 | + f'- Mode: {args.mode}\n', | ||
| 237 | + '| 项 | 通过 | 详情 |', | ||
| 238 | + '|---|---|---|'] | ||
| 239 | + for f in findings: | ||
| 240 | + mark = '✓' if f['pass'] else '✗' | ||
| 241 | + detail = json.dumps(f.get('detail', f), ensure_ascii=False)[:200] | ||
| 242 | + lines.append(f'| {f["id"]} | {mark} | `{detail}` |') | ||
| 243 | + with open(report_path, 'w', encoding='utf-8') as fp: | ||
| 244 | + fp.write('\n'.join(lines) + '\n') | ||
| 245 | + logger.info('\nReport 已保存到: %s', report_path) | ||
| 246 | + | ||
| 247 | + | ||
| 248 | +def main(): | ||
| 249 | + logging.basicConfig(level=logging.INFO, format='%(message)s', stream=sys.stdout) | ||
| 250 | + parser = argparse.ArgumentParser(description='Regression check L1–L8 vs baseline') | ||
| 251 | + parser.add_argument('--baseline', required=True, help='baseline analysis_config.json') | ||
| 252 | + parser.add_argument('--new', required=True, help='new analysis_config.json') | ||
| 253 | + parser.add_argument('--mode', default='A', choices=['A', 'B'], | ||
| 254 | + help='Mode B 跳过 L6/L7/L8(无 op_indices/kernels)') | ||
| 255 | + parser.add_argument('--report', help='输出 Markdown 报告路径') | ||
| 256 | + parser.add_argument('--json', action='store_true', help='以 JSON 输出 stdout') | ||
| 257 | + parser.add_argument('--l5', type=float, default=0.95, help='L5 Jaccard 阈值') | ||
| 258 | + parser.add_argument('--l6', type=float, default=0.90, help='L6 Jaccard 阈值') | ||
| 259 | + args = parser.parse_args() | ||
| 260 | + | ||
| 261 | + for p in (args.baseline, args.new): | ||
| 262 | + if not os.path.exists(p): | ||
| 263 | + logger.error('错误: 文件不存在: %s', p) | ||
| 264 | + sys.exit(1) | ||
| 265 | + | ||
| 266 | + with open(args.baseline, 'r', encoding='utf-8') as f: | ||
| 267 | + baseline = json.load(f) | ||
| 268 | + with open(args.new, 'r', encoding='utf-8') as f: | ||
| 269 | + new = json.load(f) | ||
| 270 | + | ||
| 271 | + findings = check_regression(baseline, new, args.l5, args.l6) | ||
| 272 | + if args.mode == 'B': | ||
| 273 | + findings = [f for f in findings if f['id'] not in ('L6', 'L7', 'L8')] | ||
| 274 | + | ||
| 275 | + hard_fails = [f for f in findings if not f['pass'] and f['id'] in ('L1', 'L2', 'L3', 'L4', 'L7')] | ||
| 276 | + soft_fails = [f for f in findings if not f['pass'] and f['id'] in ('L5', 'L6', 'L8')] | ||
| 277 | + | ||
| 278 | + _emit_findings(args, findings, hard_fails, soft_fails) | ||
| 279 | + if args.report: | ||
| 280 | + _write_regression_report(args.report, args, findings) | ||
| 281 | + | ||
| 282 | + sys.exit(1 if hard_fails else 0) | ||
| 283 | + | ||
| 284 | + | ||
| 285 | +if __name__ == '__main__': | ||
| 286 | + main() | ||
| @@ -0,0 +1,192 @@ | |||
| 1 | +#!/usr/bin/env python3 | ||
| 2 | +# -*- coding: utf-8 -*- | ||
| 3 | +# ---------------------------------------------------------------------------- | ||
| 4 | +# Copyright (c) 2025 Huawei Technologies Co., Ltd. | ||
| 5 | +# | ||
| 6 | +# Licensed under the Apache License, Version 2.0 (the "License"); | ||
| 7 | +# you may not use this file except in compliance with the License. | ||
| 8 | +# You may obtain a copy of the License at | ||
| 9 | +# | ||
| 10 | +# http://www.apache.org/licenses/LICENSE-2.0 | ||
| 11 | +# | ||
| 12 | +# Unless required by applicable law or agreed to in writing, software | ||
| 13 | +# distributed under the License is distributed on an "AS IS" BASIS, | ||
| 14 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| 15 | +# See the License for the specific language governing permissions and | ||
| 16 | +# limitations under the License. | ||
| 17 | +# ---------------------------------------------------------------------------- | ||
| 18 | +""" | ||
| 19 | +基于 raw_ops.json 的 normalized_name 序列做粗粒度 layer 边界候选检测。 | ||
| 20 | + | ||
| 21 | +启发式:找出"周期性"出现的 marker kernel(典型如 InplaceAddRmsNorm / FlashAttentionScore), | ||
| 22 | +其相邻出现间隔的离散系数(CV = std / mean)越小、出现次数越多,越像 layer 边界标记。 | ||
| 23 | + | ||
| 24 | +输出 outputs/op_segments.json,格式: | ||
| 25 | +{ | ||
| 26 | + "best_marker": <kernel_name>, | ||
| 27 | + "layer_length_estimate": <int>, | ||
| 28 | + "confidence": <float, 0–1>, | ||
| 29 | + "boundaries": [op_index, op_index, ...], # 每个 layer 的起始索引 | ||
| 30 | + "ranges": [{"layer_idx": 0, "op_range": [start, end], "marker_index": <int>}, ...], | ||
| 31 | + "candidates": [{ "kernel": ..., "count": ..., "mean": ..., "cv": ... }, ...] | ||
| 32 | +} | ||
| 33 | + | ||
| 34 | +confidence < 0.5 时不输出 boundaries(视为不可靠候选)。 | ||
| 35 | +""" | ||
| 36 | +import logging | ||
| 37 | +import argparse | ||
| 38 | +import json | ||
| 39 | +import math | ||
| 40 | +import os | ||
| 41 | +import sys | ||
| 42 | +from collections import defaultdict | ||
| 43 | +logger = logging.getLogger(__name__) | ||
| 44 | + | ||
| 45 | + | ||
| 46 | + | ||
| 47 | +def detect_periodic_markers(operators, min_occurrences=4, max_cv=0.4): | ||
| 48 | + """ | ||
| 49 | + 返回按 cv 升序排列的 marker 候选列表。 | ||
| 50 | + 每条:(kind, positions, mean, cv) | ||
| 51 | + """ | ||
| 52 | + positions_by_kind = defaultdict(list) | ||
| 53 | + for i, op in enumerate(operators): | ||
| 54 | + # compact 视图中的 repeat 块需展开 | ||
| 55 | + if op.get('repeat'): | ||
| 56 | + kind = op.get('normalized_name', '') | ||
| 57 | + count = op.get('count', 0) | ||
| 58 | + first = op.get('first_index', i) | ||
| 59 | + for k in range(count): | ||
| 60 | + positions_by_kind[kind].append(first + k) | ||
| 61 | + else: | ||
| 62 | + kind = op.get('normalized_name', '') | ||
| 63 | + positions_by_kind[kind].append(op.get('index', i)) | ||
| 64 | + | ||
| 65 | + candidates = [] | ||
| 66 | + for kind, positions in positions_by_kind.items(): | ||
| 67 | + if len(positions) < min_occurrences: | ||
| 68 | + continue | ||
| 69 | + positions.sort() | ||
| 70 | + intervals = [positions[i + 1] - positions[i] for i in range(len(positions) - 1)] | ||
| 71 | + if not intervals: | ||
| 72 | + continue | ||
| 73 | + mean = sum(intervals) / len(intervals) | ||
| 74 | + if mean <= 0: | ||
| 75 | + continue | ||
| 76 | + var = sum((x - mean) ** 2 for x in intervals) / len(intervals) | ||
| 77 | + cv = math.sqrt(var) / mean | ||
| 78 | + if cv > max_cv: | ||
| 79 | + continue | ||
| 80 | + candidates.append({ | ||
| 81 | + 'kernel': kind, | ||
| 82 | + 'count': len(positions), | ||
| 83 | + 'mean': mean, | ||
| 84 | + 'cv': cv, | ||
| 85 | + 'positions': positions, | ||
| 86 | + }) | ||
| 87 | + | ||
| 88 | + candidates.sort(key=lambda c: (c['cv'], -c['count'])) | ||
| 89 | + return candidates | ||
| 90 | + | ||
| 91 | + | ||
| 92 | +def confidence_from(cv, count): | ||
| 93 | + """Confidence = (1 - cv) * saturation(count). 落在 [0, 1].""" | ||
| 94 | + cv_term = max(0.0, 1.0 - cv / 0.5) # cv=0 → 1; cv≥0.5 → 0 | ||
| 95 | + count_term = min(1.0, count / 16.0) # count<4 已被过滤;16+ 满分 | ||
| 96 | + return round(cv_term * count_term, 3) | ||
| 97 | + | ||
| 98 | + | ||
| 99 | +def build_segments(operators, candidate): | ||
| 100 | + positions = candidate['positions'] | ||
| 101 | + layer_length = int(round(candidate['mean'])) | ||
| 102 | + ranges = [] | ||
| 103 | + for i, pos in enumerate(positions): | ||
| 104 | + if i + 1 < len(positions): | ||
| 105 | + end = positions[i + 1] - 1 | ||
| 106 | + else: | ||
| 107 | + end = pos + layer_length - 1 | ||
| 108 | + # 起点偏移到 marker 之前一段(典型 marker 在 layer 中部,回退 ~30% layer 长度) | ||
| 109 | + start = max(0, pos - layer_length // 3) | ||
| 110 | + ranges.append({ | ||
| 111 | + 'layer_idx': i, | ||
| 112 | + 'op_range': [start, end], | ||
| 113 | + 'marker_index': pos, | ||
| 114 | + }) | ||
| 115 | + return ranges | ||
| 116 | + | ||
| 117 | + | ||
| 118 | +def _build_segments_result(operators, candidates, min_confidence): | ||
| 119 | + """根据周期 marker 候选构建 op_segments 结果 dict。""" | ||
| 120 | + if not candidates: | ||
| 121 | + return { | ||
| 122 | + 'best_marker': None, | ||
| 123 | + 'layer_length_estimate': None, | ||
| 124 | + 'confidence': 0.0, | ||
| 125 | + 'boundaries': [], | ||
| 126 | + 'ranges': [], | ||
| 127 | + 'candidates': [], | ||
| 128 | + 'note': '未检测到周期性 marker,AI 须按 structure_analysis_guide §A.3.1 全手动定位 layer 边界', | ||
| 129 | + } | ||
| 130 | + best = candidates[0] | ||
| 131 | + confidence = confidence_from(best['cv'], best['count']) | ||
| 132 | + note_tail = ('boundaries 可作 layer 候选起点,最终边界以源码语义为准。' | ||
| 133 | + if confidence >= min_confidence | ||
| 134 | + else f'confidence < {min_confidence},仅供参考,AI 应回退到全手动定位。') | ||
| 135 | + return { | ||
| 136 | + 'best_marker': best['kernel'], | ||
| 137 | + 'layer_length_estimate': int(round(best['mean'])), | ||
| 138 | + 'confidence': confidence, | ||
| 139 | + 'boundaries': best['positions'] if confidence >= min_confidence else [], | ||
| 140 | + 'ranges': build_segments(operators, best) if confidence >= min_confidence else [], | ||
| 141 | + 'candidates': [ | ||
| 142 | + {'kernel': c['kernel'], 'count': c['count'], | ||
| 143 | + 'mean': round(c['mean'], 2), 'cv': round(c['cv'], 3)} | ||
| 144 | + for c in candidates[:5] | ||
| 145 | + ], | ||
| 146 | + 'note': (f'最佳 marker={best["kernel"]},{best["count"]} 次出现,' | ||
| 147 | + f'平均间隔 {best["mean"]:.1f}, CV={best["cv"]:.3f}, confidence={confidence}. ' | ||
| 148 | + + note_tail), | ||
| 149 | + } | ||
| 150 | + | ||
| 151 | + | ||
| 152 | +def main(): | ||
| 153 | + logging.basicConfig(level=logging.INFO, format='%(message)s', stream=sys.stdout) | ||
| 154 | + parser = argparse.ArgumentParser(description='Layer 边界候选检测') | ||
| 155 | + parser.add_argument('-r', '--raw-ops', dest='raw_ops', required=True, | ||
| 156 | + help='raw_ops.json 路径(或 raw_ops.compact.json)') | ||
| 157 | + parser.add_argument('-o', '--output', default='outputs/op_segments.json', | ||
| 158 | + help='输出 op_segments.json 路径') | ||
| 159 | + parser.add_argument('--min-occurrences', type=int, default=4, | ||
| 160 | + help='marker 候选最少出现次数 (default: 4)') | ||
| 161 | + parser.add_argument('--max-cv', type=float, default=0.4, | ||
| 162 | + help='marker 候选最大允许 CV (default: 0.4)') | ||
| 163 | + parser.add_argument('--min-confidence', type=float, default=0.5, | ||
| 164 | + help='最低 confidence 阈值;低于该值不输出 boundaries') | ||
| 165 | + args = parser.parse_args() | ||
| 166 | + | ||
| 167 | + if not os.path.exists(args.raw_ops): | ||
| 168 | + logger.error('错误: 文件不存在: %s', args.raw_ops) | ||
| 169 | + sys.exit(1) | ||
| 170 | + | ||
| 171 | + with open(args.raw_ops, 'r', encoding='utf-8') as f: | ||
| 172 | + raw = json.load(f) | ||
| 173 | + operators = raw.get('operators', []) | ||
| 174 | + if not operators: | ||
| 175 | + logger.error('错误: raw_ops 无 operators') | ||
| 176 | + sys.exit(1) | ||
| 177 | + | ||
| 178 | + candidates = detect_periodic_markers( | ||
| 179 | + operators, min_occurrences=args.min_occurrences, max_cv=args.max_cv) | ||
| 180 | + result = _build_segments_result(operators, candidates, args.min_confidence) | ||
| 181 | + | ||
| 182 | + os.makedirs(os.path.dirname(args.output) or '.', exist_ok=True) | ||
| 183 | + with open(args.output, 'w', encoding='utf-8') as f: | ||
| 184 | + json.dump(result, f, indent=2, ensure_ascii=False) | ||
| 185 | + logger.info('op_segments 已保存到: %s', args.output) | ||
| 186 | + logger.info(' best_marker: %s', result["best_marker"]) | ||
| 187 | + logger.info(' confidence: %s', result["confidence"]) | ||
| 188 | + logger.info(' layer count: %s', len(result["boundaries"])) | ||
| 189 | + | ||
| 190 | + | ||
| 191 | +if __name__ == '__main__': | ||
| 192 | + main() | ||
| @@ -0,0 +1,358 @@ | |||
| 1 | +#!/usr/bin/env python3 | ||
| 2 | +# -*- coding: utf-8 -*- | ||
| 3 | +# ---------------------------------------------------------------------------- | ||
| 4 | +# Copyright (c) 2025 Huawei Technologies Co., Ltd. | ||
| 5 | +# | ||
| 6 | +# Licensed under the Apache License, Version 2.0 (the "License"); | ||
| 7 | +# you may not use this file except in compliance with the License. | ||
| 8 | +# You may obtain a copy of the License at | ||
| 9 | +# | ||
| 10 | +# http://www.apache.org/licenses/LICENSE-2.0 | ||
| 11 | +# | ||
| 12 | +# Unless required by applicable law or agreed to in writing, software | ||
| 13 | +# distributed under the License is distributed on an "AS IS" BASIS, | ||
| 14 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| 15 | +# See the License for the specific language governing permissions and | ||
| 16 | +# limitations under the License. | ||
| 17 | +# ---------------------------------------------------------------------------- | ||
| 18 | +""" | ||
| 19 | +validate_shapes.py — 检查 analysis_config.json 中 shape_semantic 与实际 tensor shape 的一致性。 | ||
| 20 | + | ||
| 21 | +使用方式: | ||
| 22 | + python scripts/validate_shapes.py -c outputs/analysis_config.json | ||
| 23 | + python scripts/validate_shapes.py -c outputs/analysis_config.json --strict # ERROR 也报 WARNING | ||
| 24 | +""" | ||
| 25 | + | ||
| 26 | +import logging | ||
| 27 | +import json | ||
| 28 | +import re | ||
| 29 | +import sys | ||
| 30 | +import argparse | ||
| 31 | +from pathlib import Path | ||
| 32 | +logger = logging.getLogger(__name__) | ||
| 33 | + | ||
| 34 | + | ||
| 35 | + | ||
| 36 | +# --------------------------------------------------------------------------- | ||
| 37 | +# Symbol table: 从 config 字段映射到数值 | ||
| 38 | +# --------------------------------------------------------------------------- | ||
| 39 | +SYMBOL_MAP = { | ||
| 40 | + 'hidden': 'hidden_size', | ||
| 41 | + 'ffn': 'intermediate_size', | ||
| 42 | + 'moe_ffn': 'moe_intermediate_size', | ||
| 43 | + 'H_q': 'num_attention_heads', | ||
| 44 | + 'H': 'num_attention_heads', | ||
| 45 | + 'num_heads': 'num_attention_heads', | ||
| 46 | + 'D_rope': 'qk_rope_head_dim', | ||
| 47 | + 'D_nope': 'qk_nope_head_dim', | ||
| 48 | + 'D': 'v_head_dim', | ||
| 49 | + 'v_head_dim': 'v_head_dim', | ||
| 50 | + 'q_rank': 'q_lora_rank', | ||
| 51 | + 'kv_rank': 'kv_lora_rank', | ||
| 52 | + 'E': 'n_routed_experts', | ||
| 53 | + 'topK': 'num_experts_per_tok', | ||
| 54 | + 'vocab': 'vocab_size', | ||
| 55 | + 'H_idx': 'index_n_heads', | ||
| 56 | + 'D_idx': 'index_head_dim', | ||
| 57 | + 'index_topk': 'index_topk', | ||
| 58 | + 'H_k': 'num_key_value_heads', | ||
| 59 | +} | ||
| 60 | + | ||
| 61 | + | ||
| 62 | +def build_symbol_table(config: dict) -> dict: | ||
| 63 | + cfg = config.get('config', {}) | ||
| 64 | + table = {} | ||
| 65 | + for sym, field in SYMBOL_MAP.items(): | ||
| 66 | + v = cfg.get(field) | ||
| 67 | + if v is not None: | ||
| 68 | + table[sym] = int(v) | ||
| 69 | + return table | ||
| 70 | + | ||
| 71 | + | ||
| 72 | +# --------------------------------------------------------------------------- | ||
| 73 | +# Shape parsing helpers | ||
| 74 | +# --------------------------------------------------------------------------- | ||
| 75 | + | ||
| 76 | +def parse_shapes(shape_str: str) -> list[list[int]]: | ||
| 77 | + """Parse semicolon-separated shape string into list of dim lists. | ||
| 78 | + E.g. "4,7168;96,448,16,16" → [[4,7168],[96,448,16,16]] | ||
| 79 | + """ | ||
| 80 | + if not shape_str or str(shape_str).upper() == 'N/A': | ||
| 81 | + return [] | ||
| 82 | + result = [] | ||
| 83 | + for part in str(shape_str).split(';'): | ||
| 84 | + part = part.strip().strip('"') | ||
| 85 | + if not part: | ||
| 86 | + continue | ||
| 87 | + try: | ||
| 88 | + dims = [int(x) for x in part.split(',') if x.strip()] | ||
| 89 | + if dims: | ||
| 90 | + result.append(dims) | ||
| 91 | + except ValueError: | ||
| 92 | + pass | ||
| 93 | + return result | ||
| 94 | + | ||
| 95 | + | ||
| 96 | +def all_dims_set(shapes: list[list[int]]) -> set[int]: | ||
| 97 | + """Flat set of all dimension values across all tensors.""" | ||
| 98 | + s = set() | ||
| 99 | + for dims in shapes: | ||
| 100 | + s.update(dims) | ||
| 101 | + return s | ||
| 102 | + | ||
| 103 | + | ||
| 104 | +# --------------------------------------------------------------------------- | ||
| 105 | +# shape_semantic extraction | ||
| 106 | +# --------------------------------------------------------------------------- | ||
| 107 | + | ||
| 108 | +def extract_bracket_tokens(shape_sem: str) -> list[str]: | ||
| 109 | + """Extract token strings from [...] brackets in shape_semantic.""" | ||
| 110 | + tokens = [] | ||
| 111 | + for m in re.finditer(r'\[([^\]]+)\]', shape_sem): | ||
| 112 | + for tok in m.group(1).split(','): | ||
| 113 | + tokens.append(tok.strip()) | ||
| 114 | + return tokens | ||
| 115 | + | ||
| 116 | + | ||
| 117 | +def extract_explicit_values(shape_sem: str) -> dict[str, int]: | ||
| 118 | + """Extract 'sym=N' patterns from inside [...] brackets only. | ||
| 119 | + Patterns outside brackets (e.g. parallelism annotations like "EP=64 ranks") are ignored. | ||
| 120 | + """ | ||
| 121 | + result = {} | ||
| 122 | + for bracket_m in re.finditer(r'\[([^\]]+)\]', shape_sem): | ||
| 123 | + inner = bracket_m.group(1) | ||
| 124 | + for m in re.finditer(r'([A-Za-z_][A-Za-z0-9_*]*)=(\d+)', inner): | ||
| 125 | + sym, val = m.group(1), int(m.group(2)) | ||
| 126 | + result[sym] = val | ||
| 127 | + return result | ||
| 128 | + | ||
| 129 | + | ||
| 130 | +def _literal_dims_in_bracket(inner: str, trivial: set) -> list[int]: | ||
| 131 | + """从单个 [...] 内提取非平凡数字字面量。""" | ||
| 132 | + nums = [] | ||
| 133 | + for tok in inner.split(','): | ||
| 134 | + tok = tok.strip() | ||
| 135 | + if re.fullmatch(r'\d+', tok) and int(tok) not in trivial: | ||
| 136 | + nums.append(int(tok)) | ||
| 137 | + return nums | ||
| 138 | + | ||
| 139 | + | ||
| 140 | +def extract_literal_dims(shape_sem: str) -> list[int]: | ||
| 141 | + """Extract standalone numeric literals inside [...] dimension brackets. | ||
| 142 | + Skips 1 (trivial broadcast scalar) and 2 (trivial pairing). | ||
| 143 | + """ | ||
| 144 | + trivial = {1, 2} | ||
| 145 | + nums = [] | ||
| 146 | + for m in re.finditer(r'\[([^\]]+)\]', shape_sem): | ||
| 147 | + nums.extend(_literal_dims_in_bracket(m.group(1), trivial)) | ||
| 148 | + return nums | ||
| 149 | + | ||
| 150 | + | ||
| 151 | +# --------------------------------------------------------------------------- | ||
| 152 | +# Per-kernel validation | ||
| 153 | +# --------------------------------------------------------------------------- | ||
| 154 | + | ||
| 155 | +def validate_kernel(kernel: dict, op_data: dict, symbol_table: dict, strict: bool) -> list[tuple]: | ||
| 156 | + """Return list of (level, message) issues for one kernel.""" | ||
| 157 | + issues = [] | ||
| 158 | + shape_sem = kernel.get('shape_semantic', '') | ||
| 159 | + if not shape_sem: | ||
| 160 | + return issues | ||
| 161 | + | ||
| 162 | + idx = kernel.get('index', '?') | ||
| 163 | + name = op_data.get('name') or kernel.get('name', '?') | ||
| 164 | + | ||
| 165 | + in_shapes = parse_shapes(op_data.get('input_shapes', '')) | ||
| 166 | + out_shapes = parse_shapes(op_data.get('output_shapes', '')) | ||
| 167 | + in_dims = all_dims_set(in_shapes) | ||
| 168 | + out_dims = all_dims_set(out_shapes) | ||
| 169 | + all_dims = in_dims | out_dims | ||
| 170 | + | ||
| 171 | + # 1. Named value cross-check: sym=N in shape_semantic vs config | ||
| 172 | + explicit = extract_explicit_values(shape_sem) | ||
| 173 | + for sym, val in explicit.items(): | ||
| 174 | + expected = symbol_table.get(sym) | ||
| 175 | + if expected is not None and expected != val: | ||
| 176 | + issues.append(('ERROR', | ||
| 177 | + f'[{idx}] {name}: shape_semantic 写 {sym}={val},但 config 中 {sym}={expected}')) | ||
| 178 | + # Named value should appear somewhere in actual dims | ||
| 179 | + if val not in all_dims: | ||
| 180 | + issues.append(('WARNING', | ||
| 181 | + f'[{idx}] {name}: shape_semantic 中 {sym}={val} 不出现在实际 tensor dims {sorted(all_dims)} ' | ||
| 182 | + f'(in={op_data.get("input_shapes","")[:50]}, out={op_data.get("output_shapes","")[:50]})')) | ||
| 183 | + | ||
| 184 | + # 2. Resolve symbolic tokens, check literal dims inside [...] | ||
| 185 | + literal_dims = extract_literal_dims(shape_sem) | ||
| 186 | + for num in set(literal_dims): | ||
| 187 | + if num not in all_dims: | ||
| 188 | + issues.append(('WARNING', | ||
| 189 | + f'[{idx}] {name}: shape_semantic 中出现字面量 {num},但不存在于实际 dims {sorted(all_dims)} ' | ||
| 190 | + f'(in={op_data.get("input_shapes","")[:50]}, out={op_data.get("output_shapes","")[:50]})')) | ||
| 191 | + | ||
| 192 | + # 3. Arrow-split consistency: left of → should relate to inputs, right to outputs | ||
| 193 | + arrow = '→' | ||
| 194 | + if arrow in shape_sem: | ||
| 195 | + left, right = shape_sem.split(arrow, 1) | ||
| 196 | + left_lits = [n for n in extract_literal_dims(f'[{left}]') if n > 1] | ||
| 197 | + right_lits = [n for n in extract_literal_dims(f'[{right}]') if n > 1] | ||
| 198 | + | ||
| 199 | + # Left-side literals should appear in actual input dims (or at least in all_dims) | ||
| 200 | + for num in left_lits: | ||
| 201 | + if in_dims and num not in in_dims and num not in all_dims: | ||
| 202 | + level = 'WARNING' if not strict else 'WARNING' | ||
| 203 | + issues.append((level, | ||
| 204 | + f'[{idx}] {name}: shape_semantic 输入侧 (→左) 包含 {num},' | ||
| 205 | + f'但实际输入 dims={sorted(in_dims)}')) | ||
| 206 | + | ||
| 207 | + # Right-side literals should appear in actual output dims | ||
| 208 | + for num in right_lits: | ||
| 209 | + if out_dims and num not in out_dims and num not in all_dims: | ||
| 210 | + issues.append(('WARNING', | ||
| 211 | + f'[{idx}] {name}: shape_semantic 输出侧 (→右) 包含 {num},' | ||
| 212 | + f'但实际输出 dims={sorted(out_dims)}')) | ||
| 213 | + | ||
| 214 | + return issues | ||
| 215 | + | ||
| 216 | + | ||
| 217 | +# --------------------------------------------------------------------------- | ||
| 218 | +# Config traversal | ||
| 219 | +# --------------------------------------------------------------------------- | ||
| 220 | + | ||
| 221 | +def collect_kernels_with_op_data(config: dict) -> list[tuple]: | ||
| 222 | + """Yield (kernel_entry, op_data_dict) for all kernels that have shape_semantic.""" | ||
| 223 | + pairs = [] | ||
| 224 | + | ||
| 225 | + def visit(node: dict): | ||
| 226 | + if not isinstance(node, dict): | ||
| 227 | + return | ||
| 228 | + kernels = node.get('kernels', []) | ||
| 229 | + op_data_list = node.get('op_data', []) | ||
| 230 | + op_data_map = {od.get('index'): od for od in op_data_list} | ||
| 231 | + | ||
| 232 | + for k in kernels: | ||
| 233 | + if k.get('shape_semantic'): | ||
| 234 | + od = op_data_map.get(k.get('index'), {}) | ||
| 235 | + pairs.append((k, od)) | ||
| 236 | + | ||
| 237 | + for child in node.get('children', []): | ||
| 238 | + visit(child) | ||
| 239 | + | ||
| 240 | + for stage in config.get('stages', {}).values(): | ||
| 241 | + visit(stage) | ||
| 242 | + for struct in config.get('layer_structure', {}).values(): | ||
| 243 | + visit(struct) | ||
| 244 | + for aux in config.get('runtime_auxiliary', []): | ||
| 245 | + visit(aux) | ||
| 246 | + | ||
| 247 | + return pairs | ||
| 248 | + | ||
| 249 | + | ||
| 250 | +# --------------------------------------------------------------------------- | ||
| 251 | +# Main | ||
| 252 | +# --------------------------------------------------------------------------- | ||
| 253 | + | ||
| 254 | +def _collect_issues(pairs, symbol_table, strict, fail_fast): | ||
| 255 | + """对所有 kernel 跑校验,返回 all_issues。""" | ||
| 256 | + all_issues = [] | ||
| 257 | + for kernel, op_data in pairs: | ||
| 258 | + issues = validate_kernel(kernel, op_data, symbol_table, strict) | ||
| 259 | + all_issues.extend(issues) | ||
| 260 | + if fail_fast and any(lvl == 'ERROR' for lvl, _ in issues): | ||
| 261 | + break | ||
| 262 | + return all_issues | ||
| 263 | + | ||
| 264 | + | ||
| 265 | +def _emit_json_result(config_path, pairs, all_issues, fail_fast): | ||
| 266 | + """以 JSON 形式输出校验结果。""" | ||
| 267 | + errors = [m for lvl, m in all_issues if lvl == 'ERROR'] | ||
| 268 | + warnings = [m for lvl, m in all_issues if lvl == 'WARNING'] | ||
| 269 | + formatted_issues = [{ | ||
| 270 | + 'id': 'V1', | ||
| 271 | + 'severity': 'error' if level == 'ERROR' else 'warning', | ||
| 272 | + 'node_path': '<kernel>', | ||
| 273 | + 'message': msg, | ||
| 274 | + } for level, msg in all_issues] | ||
| 275 | + logger.info(json.dumps({ | ||
| 276 | + 'script': 'validate_shapes.py', | ||
| 277 | + 'config': config_path, | ||
| 278 | + 'kernels_checked': len(pairs), | ||
| 279 | + 'error_count': len(errors), | ||
| 280 | + 'warning_count': len(warnings), | ||
| 281 | + 'fail_fast': fail_fast, | ||
| 282 | + 'issues': formatted_issues, | ||
| 283 | + }, ensure_ascii=False, indent=2)) | ||
| 284 | + | ||
| 285 | + | ||
| 286 | +def _emit_text_result(pairs, errors, warnings, fail_fast): | ||
| 287 | + """以可读文本形式输出校验结果。""" | ||
| 288 | + if errors: | ||
| 289 | + logger.info('=== ERROR (%d) ===', len(errors)) | ||
| 290 | + for _, m in errors: | ||
| 291 | + logger.info(' [ERROR] %s', m) | ||
| 292 | + logger.info('') | ||
| 293 | + if warnings: | ||
| 294 | + logger.info('=== WARNING (%d) ===', len(warnings)) | ||
| 295 | + for _, m in warnings: | ||
| 296 | + logger.info(' [WARN] %s', m) | ||
| 297 | + logger.info('') | ||
| 298 | + logger.info('共检查 %d 个 kernel,%d 个 ERROR,%d 个 WARNING。%s', | ||
| 299 | + len(pairs), len(errors), len(warnings), | ||
| 300 | + ' (fail-fast)' if fail_fast else '') | ||
| 301 | + | ||
| 302 | + | ||
| 303 | +def run_validation(config_path: str, strict: bool = False, | ||
| 304 | + fail_fast: bool = False, json_out: bool = False) -> int: | ||
| 305 | + config = json.loads(Path(config_path).read_text()) | ||
| 306 | + symbol_table = build_symbol_table(config) | ||
| 307 | + | ||
| 308 | + if not json_out: | ||
| 309 | + logger.info('模型: %s', config.get("model_name", "?")) | ||
| 310 | + logger.info('符号表: %s', symbol_table) | ||
| 311 | + logger.info('') | ||
| 312 | + | ||
| 313 | + pairs = collect_kernels_with_op_data(config) | ||
| 314 | + if not pairs: | ||
| 315 | + msg = '未找到带 shape_semantic 的 kernel(是否已运行 --enrich?)' | ||
| 316 | + if json_out: | ||
| 317 | + logger.info(json.dumps({ | ||
| 318 | + 'script': 'validate_shapes.py', | ||
| 319 | + 'config': config_path, | ||
| 320 | + 'error_count': 1, | ||
| 321 | + 'issues': [{'id': 'V0', 'severity': 'error', 'node_path': '<global>', 'message': msg}], | ||
| 322 | + }, ensure_ascii=False, indent=2)) | ||
| 323 | + else: | ||
| 324 | + logger.info(msg) | ||
| 325 | + return 1 | ||
| 326 | + | ||
| 327 | + all_issues = _collect_issues(pairs, symbol_table, strict, fail_fast) | ||
| 328 | + errors = [(lvl, m) for lvl, m in all_issues if lvl == 'ERROR'] | ||
| 329 | + warnings = [(lvl, m) for lvl, m in all_issues if lvl == 'WARNING'] | ||
| 330 | + | ||
| 331 | + if json_out: | ||
| 332 | + _emit_json_result(config_path, pairs, all_issues, fail_fast) | ||
| 333 | + return 1 if errors else 0 | ||
| 334 | + | ||
| 335 | + if not all_issues: | ||
| 336 | + logger.info('✓ 全部 %d 个 shape_semantic 校验通过,无问题。', len(pairs)) | ||
| 337 | + return 0 | ||
| 338 | + | ||
| 339 | + _emit_text_result(pairs, errors, warnings, fail_fast) | ||
| 340 | + return 1 if errors else 0 | ||
| 341 | + | ||
| 342 | + | ||
| 343 | +def main(): | ||
| 344 | + logging.basicConfig(level=logging.INFO, format='%(message)s', stream=sys.stdout) | ||
| 345 | + parser = argparse.ArgumentParser(description='校验 analysis_config.json 中 shape_semantic 的一致性') | ||
| 346 | + parser.add_argument('-c', '--config', required=True, help='analysis_config.json 路径') | ||
| 347 | + parser.add_argument('--strict', action='store_true', help='把 WARNING 也视为失败') | ||
| 348 | + parser.add_argument('--fail-fast', action='store_true', dest='fail_fast', | ||
| 349 | + help='遇到首个 ERROR 即退出') | ||
| 350 | + parser.add_argument('--json', action='store_true', dest='json_out', | ||
| 351 | + help='以 JSON 输出结果(与 check_structure / check_op_coverage 一致)') | ||
| 352 | + args = parser.parse_args() | ||
| 353 | + sys.exit(run_validation(args.config, strict=args.strict, | ||
| 354 | + fail_fast=args.fail_fast, json_out=args.json_out)) | ||
| 355 | + | ||
| 356 | + | ||
| 357 | +if __name__ == '__main__': | ||
| 358 | + main() | ||