已合并
feat: update performance breakdown skill #515
unstain创建于 15 天前
feat: update performance breakdown skill #515
已合并
共 56 个文件变更+14159-4832
| @@ -1,13 +1,14 @@ | |||
| 1 | --- | 1 | --- |
| 2 | name: cann-perf-breakdown | 2 | name: cann-perf-breakdown |
| 3 | description: | | 3 | description: | |
| 4 | - NPU 性能数据拆解技能。将 kernel_details.csv 中的性能数据按模型结构进行拆解分析。 | 4 | + NPU 性能数据拆解技能。以模型源码为主证据拆解结构，再把 kernel_details.csv 的性能数据挂到该结构上。 |
| 5 | 触发场景：分析 kernel_details.csv、拆解性能数据到模型层级、分析大模型性能瓶颈、仅模型代码做架构拆解、仅性能数据做诊断（委托 cann-npu-perfanalysis）。 | 5 | 触发场景：分析 kernel_details.csv、拆解性能数据到模型层级、分析大模型性能瓶颈、仅模型代码做架构拆解、仅性能数据做诊断（委托 cann-npu-perfanalysis）。 |
| 6 | + 适用于任意 Transformer 模型族，不假定任何特定模型的模块名或 kernel 名。 | ||
| 6 | --- | 7 | --- |
| 7 | 8 | ||
| 8 | # NPU 性能数据拆解技能 | 9 | # NPU 性能数据拆解技能 |
| 9 | 10 | ||
| 10 | -将 NPU profiling 输出的 `kernel_details.csv` 按 Transformer 模型结构进行层级拆解，生成结构化的 JSON 和 Markdown/HTML 分析报告。 | 11 | +将 NPU profiling 输出的 `kernel_details.csv` 按 Transformer 模型结构进行层级拆解，生成可验证的结构化 JSON 与性能指标。 |
| 11 | 12 | ||
| 12 | --- | 13 | --- |
| 13 | 14 | ||
| @@ -20,7 +21,7 @@ description: | | |||
| 20 | | Block 内部分析 | 分析 Attention、MLP、Norm 子层 | | 21 | | Block 内部分析 | 分析 Attention、MLP、Norm 子层 | |
| 21 | | 架构特征识别 | 识别 MLA、MoE、GQA 等架构 | | 22 | | 架构特征识别 | 识别 MLA、MoE、GQA 等架构 | |
| 22 | | 仅模型代码 | 仅做结构拆解，不绑定具体性能数据（多分支用 `branches` 表达） | | 23 | | 仅模型代码 | 仅做结构拆解，不绑定具体性能数据（多分支用 `branches` 表达） | |
| 23 | -| 仅性能数据 | 委托给 `cann-npu-perfanalysis` sibling skill 做 8 维诊断 | | 24 | +| 仅性能数据 | 委托给仓库内的 `cann-npu-perfanalysis` skill 做 8 维诊断 | |
| 24 | 25 | ||
| 25 | --- | 26 | --- |
| 26 | 27 | ||
| @@ -30,31 +31,121 @@ description: | | |||
| 30 | 31 | ||
| 31 | | 条件 | 模式 | 行为 | | 32 | | 条件 | 模式 | 行为 | |
| 32 | |---|---|---| | 33 | |---|---|---| |
| 33 | -| 有模型源码（`*modeling*.py` 等） **且** 有 `kernel_details.csv` 或 `raw_ops*.json` | **Mode A** | 完整 5 步流程，输出 `analysis_config.json` + 报告 + 指标 | | 34 | +| 有模型源码（`*modeling*.py` 等） **且** 有 `kernel_details.csv` 或 `raw_ops*.json` | **Mode A** | 完整 11 步评分闭环（schema v2），输出 manifest + config + semantic review + validation + score；达标后才输出指标并进入 Stage 2 | |
| 34 | -| 仅模型源码 | **Mode B** | 仅做结构拆解，输出 `model_structure.json`（schema 沿用 analysis_config.json，`op_indices=[]`，可加 `branches` 字段）。详见 `references/mode_b_branches.md` | | 35 | +| 仅模型源码 | **Mode B** | 提取并**校验**架构（`model_manifest.json`）后输出结构树 `model_structure.json`（v2，`op_indices=[]`，可加 `branches`）。**不是**空 op tree，须过架构校验。详见 `references/mode_b_branches.md` | |
| 35 | -| 仅性能数据（csv 或 `ASCEND_PROFILER_OUTPUT/`） | **Mode C** | 委托 `cann-npu-perfanalysis` sibling skill。详见 `references/mode_c_delegate.md` | | 36 | +| 仅性能数据（csv 或 `ASCEND_PROFILER_OUTPUT/`） | **Mode C** | 委托仓库内的 `cann-npu-perfanalysis` skill。详见 `references/mode_c_delegate.md` | |
| 36 | 37 | ||
| 37 | -> Mode B 与 Mode C 不进入 Step 3/4/5；Mode A 走完整 5 步。 | 38 | +> Mode C 不进入结构拆解；Mode A 走完整 11 步；Mode B 至少走 Step 1-3（架构提取+校验+结构树）。 |
| 39 | + | ||
| 40 | +**核心区分（schema v2，详见 `references/structure_analysis_guide.md` §C/§E）**： | ||
| 41 | + | ||
| 42 | +- **学习到的模型层**（`architecture.layer_groups` / `prediction_modules`）≠ **运行时调用**（`trace_instances`）。 | ||
| 43 | +- MTP/spec decoding：外层循环重复调用**同一个**学习到的 decoder layer，记为“1 learned layer + N invocations”，**禁止**写成 N 个模型层或伪层号 `6,7,8`。 | ||
| 44 | +- **`children` 只表达包含关系**，相邻不等于有数据流边。残差、并行支路、skip 一律走变量传递，必须在 `branches` 里显式声明；未声明的边在下游就不存在，**下游禁止猜测补边**。 | ||
| 45 | +- **性能数据只覆盖采集到的范围**：未被采集的层不得外推指标。采集范围可选地记在 `trace_scope`（`full_model` / `rank_local` / `pipeline_stage_local` / `unknown`），无证据时留空或写 `unknown`，不得声称 pipeline rank。 | ||
| 46 | + | ||
| 47 | +**覆盖四分类（严格模式 unmapped 必须为 0）**： | ||
| 48 | + | ||
| 49 | +- `mapped_model_ops`：模型模块算子（trace_instances + stages + structures 叶子）。 | ||
| 50 | +- `mapped_runtime_ops`：运行时辅助算子（runtime_auxiliary）。 | ||
| 51 | +- `excluded_profiler_ops`：**仅**纯 profiler/bookkeeping，`reason_code` 用有限枚举 + `evidence`；主计算算子（MatMul/Attention/Norm/MoE/通信/Gather/KV cache/采样）禁止 excluded。 | ||
| 52 | +- `unmapped_ops`：归属未知 = 映射未完成，**严格校验必然失败**（填 reason 不算完成）。 | ||
| 53 | +- 探索模式 `--allow-unmapped` 状态为 `exploratory`，绝不为 `passed`，报告显著标注未验证。 | ||
| 54 | +- 每个代表 step 的全部 op 必须落入前三类之一；映射规程见 `references/ai_mapping_protocol.md`。 | ||
| 55 | +- **100% Kernel 覆盖不等于语义正确**：Q/K/V 分支、残差、层边界和尾部阶段还必须通过 `semantic_review.json` 的源码/Trace 审查。 | ||
| 38 | 56 | ||
| 39 | --- | 57 | --- |
| 40 | 58 | ||
| 41 | -## 工作流（Mode A） | 59 | +## 工作流（Mode A，11 步评分闭环） |
| 42 | 60 | ||
| 43 | ``` | 61 | ``` |
| 44 | -Step 1: analyze_kernels.py | 62 | +Step 1: 发现输入并判定模式 |
| 63 | + ↓ | ||
| 64 | +Step 2: extract_model_manifest.py（AST 静态提取全局架构真值） | ||
| 65 | + → model_manifest.json | ||
| 66 | + ↓ | ||
| 67 | +Step 3: validate_architecture.py（校验全局架构，无 manifest 时以 config 自洽为准） | ||
| 68 | + ↓ | ||
| 69 | +Step 4: analyze_kernels.py（提取代表 profiling step） | ||
| 45 | → raw_ops.json + raw_ops_details.json + raw_ops.compact.json | 70 | → raw_ops.json + raw_ops_details.json + raw_ops.compact.json |
| 46 | ↓ | 71 | ↓ |
| 47 | -Step 2: AI 拆解（投喂 raw_ops.compact.json + 模型源码） | 72 | +Step 4b: device_freq.py（AI Core 实测频率；trace 声明值 × 逐 kernel 反推值交叉验证） |
| 48 | - → analysis_config.json（首版） | 73 | + → device_freq.json |
| 49 | ↓ | 74 | ↓ |
| 50 | -Step 3: Review（脚本检查 → AI 仅看 issue 列表 → 修正循环） | 75 | +Step 5: extract_dataflow.py（AST 解析 forward()，得到确定性数据流真值） |
| 51 | - → analysis_config.json（终版） | 76 | + → dataflow_source.json（modules / calls / edges / branches / forks / merges |
| 77 | + / variants / unsupported） | ||
| 52 | ↓ | 78 | ↓ |
| 53 | -Step 4: generate_report.py（默认 MD + HTML） | 79 | +Step 6: AI 拆解（按 references/ai_mapping_protocol.md 执行）+ map_trace_instances.py |
| 54 | - → {prefix}_report.md / {prefix}_report.html | 80 | + 把代表 step 的【全部】op 映射到 model / runtime_auxiliary / 严格允许的 excluded |
| 81 | + → analysis_config.json（v2：architecture + trace_instances + structures） | ||
| 55 | ↓ | 82 | ↓ |
| 56 | -Step 5: compute_metrics.py | 83 | +Step 7: 生成可复用的代表结构树（structures，每类 layer_group 一棵；子模块 op 归属 |
| 57 | - → metrics_report.md | 84 | + 由 AI 按 ai_mapping_protocol.md 完成，check_sublayers.py 校验一致性） |
| 85 | + ↓ | ||
| 86 | +Step 8: AI 按 semantic_review_protocol.md 完整审查源码与代表 Trace | ||
| 87 | + → semantic_review.json（绑定 config/raw_ops/manifest SHA256） | ||
| 88 | + ↓ | ||
| 89 | +Step 9: run_validation.py（统一 schema/structure/architecture/dataflow/coverage/semantic 校验） | ||
| 90 | + → validation_report.json（要求 status 恰为 passed；`passed_with_warnings` 不算通过） | ||
| 91 | + ↓ | ||
| 92 | +Step 10: score_breakdown.py（固定 100 分名义量表 + 可运行分母 + 分项门槛 + 硬性否决） | ||
| 93 | + → breakdown_score.json（要求 convertible=true、可运行正确率 >= 95%、 | ||
| 94 | + 【全部可运行核心】分项达最低比例、hard_gates passed） | ||
| 95 | + ↓ 未达标：读取 iteration_request.json，基于历史最佳配置定向修正，回到 Step 8 | ||
| 96 | +Step 11: compute_metrics.py（仅在评分通过后） | ||
| 97 | + → metrics_report.md + metrics_findings.json（咨询性质） | ||
| 98 | +``` | ||
| 99 | + | ||
| 100 | +评分细则和停止条件见 `references/breakdown_scoring.md`。正式流程必须循环 Step 8-10，直至语义审查、校验和评分全部通过；配置变化后旧审查因 SHA256 不匹配自动失效。 | ||
| 101 | + | ||
| 102 | +**停止条件（只有两个）**： | ||
| 103 | + | ||
| 104 | +1. `breakdown_score.convertible == true`：所有实际可运行的检查正确率 `>= 95%`，全部可运行核心维度达到原最低比例（架构 `22/25`、数据流与分支 `27/30`、层与子模块边界 `18/20`），Kernel 精确覆盖和证据门禁通过，hard_gates 全过。状态按证据层级为 `verified`、`verified_unbound_scalars` 或 `structure_unverified`；`passed` 仅为旧产物兼容值。 | ||
| 105 | +2. 评估轮次达到 `--max-iterations`（默认 **10**）仍未达标 → `blocked_max_iterations`。 | ||
| 106 | + | ||
| 107 | +> `score` 是固定 100 分名义量表上的原始分，`runnable_max` 是当前输入实际能运行的检查分母。缺少 checkpoint/source snapshot 时，不可绑定的 scalar 检查退出分母并降低证据结论上限，而不是把正确拆解永久判失败；是否可进入下游必须读 `convertible`，不得重新写死 `score >= 95`。 | ||
| 108 | + | ||
| 109 | +默认关闭"连续无提升早停"（`--stall-limit 0`），循环会跑满 10 轮；需要早停再显式传 `--stall-limit N`。 | ||
| 110 | + | ||
| 111 | +达到 10 轮上限仍未达标时，**必须**输出根因分析：当前分数、每个未达标维度的分差、卡住的语义检查项、以及缺失的证据类型；禁止通过降低门槛、删除主计算 Kernel 或扩大 excluded 来提分。 | ||
| 112 | + | ||
| 113 | +> 架构阶段（Step 2-3）**必须**完整扫描源码；稀疏源码读只用于 Step 8 之后的 issue 修复。 | ||
| 114 | +> | ||
| 115 | +> **通用能力 vs 验证数据**：正式流程对**每个具体 trace** 用 `scripts/run_breakdown.py` | ||
| 116 | +> 驱动确定性步骤 + AI 按 `references/ai_mapping_protocol.md` 生成映射（op 数/顺序随 | ||
| 117 | +> batch/seq/next_n/并行而变）。Skill 包内**不保留**模型源码、真实 profiling 数据或测试 | ||
| 118 | +> fixture；验证数据由调用方或仓库测试体系从外部提供。 | ||
| 119 | + | ||
| 120 | +**语义审查与统一校验命令（替代旧的三条 `>>` 拼接）**： | ||
| 121 | + | ||
| 122 | +```bash | ||
| 123 | +python scripts/prepare_semantic_review.py \ | ||
| 124 | + -c outputs/analysis_config.json \ | ||
| 125 | + -r outputs/raw_ops.json \ | ||
| 126 | + -m outputs/model_manifest.json \ | ||
| 127 | + --source-dir /path/to/model-source \ | ||
| 128 | + -o outputs/semantic_review_request.json | ||
| 129 | +# AI 完整阅读源码与代表 Trace 后，按请求生成 outputs/semantic_review.json | ||
| 130 | + | ||
| 131 | +python scripts/run_validation.py \ | ||
| 132 | + -c outputs/analysis_config.json \ | ||
| 133 | + -r outputs/raw_ops.json \ | ||
| 134 | + -m outputs/model_manifest.json \ | ||
| 135 | + --source-dir /path/to/model-source \ | ||
| 136 | + --semantic-review outputs/semantic_review.json \ | ||
| 137 | + -o outputs/validation_report.json | ||
| 138 | +# status != passed 时阻断正式报告；--allow-warnings 仅用于校验分诊， | ||
| 139 | +# 会产生 passed_with_warnings，仍不属于正式通过 | ||
| 140 | + | ||
| 141 | +python scripts/score_breakdown.py \ | ||
| 142 | + -v outputs/validation_report.json \ | ||
| 143 | + -c outputs/analysis_config.json \ | ||
| 144 | + -r outputs/raw_ops.json \ | ||
| 145 | + -m outputs/model_manifest.json \ | ||
| 146 | + --semantic-review outputs/semantic_review.json \ | ||
| 147 | + -o outputs/breakdown_score.json | ||
| 148 | +# convertible != true 时读取 iteration_request.json 定向修正，禁止生成正式报告 | ||
| 58 | ``` | 149 | ``` |
| 59 | 150 | ||
| 60 | --- | 151 | --- |
| @@ -84,9 +175,56 @@ python scripts/analyze_kernels.py \ | |||
| 84 | | `raw_ops.json` | 自动选择或 `-s` 指定的单 Step kernel 概要，enrich/校验脚本用 | | 175 | | `raw_ops.json` | 自动选择或 `-s` 指定的单 Step kernel 概要，enrich/校验脚本用 | |
| 85 | | `raw_ops_details.json` | 单 Step kernel 详情（含 CSV 全部字段），Step 4/5 报告与指标用 | | 176 | | `raw_ops_details.json` | 单 Step kernel 详情（含 CSV 全部字段），Step 4/5 报告与指标用 | |
| 86 | | `raw_ops.compact.json` | Step 2 投喂给 AI 的精简视图（删除 `start_time_us`/`duration_us`，连续相同算子折叠）| | 177 | | `raw_ops.compact.json` | Step 2 投喂给 AI 的精简视图（删除 `start_time_us`/`duration_us`，连续相同算子折叠）| |
| 178 | +| `device_freq.json` | AI Core 实测频率与交叉验证结果，下游所有 cycle 派生指标的分母 | | ||
| 87 | 179 | ||
| 88 | 每个 operator 含 `org_index`，表示其在 `kernel_details.csv` 中的 0-based 行号。 | 180 | 每个 operator 含 `org_index`，表示其在 `kernel_details.csv` 中的 0-based 行号。 |
| 89 | 181 | ||
| 182 | +### AI Core 频率（Step 4b） | ||
| 183 | + | ||
| 184 | +```bash | ||
| 185 | +python scripts/device_freq.py \ | ||
| 186 | + -d outputs/raw_ops_details.json \ | ||
| 187 | + --trace ASCEND_PROFILER_OUTPUT/trace_view.json \ | ||
| 188 | + -o outputs/device_freq.json | ||
| 189 | +``` | ||
| 190 | + | ||
| 191 | +两个彼此独立的来源，分别上报，不合并： | ||
| 192 | + | ||
| 193 | +- `declared`：trace_view.json 里的 `AI Core Freq` 计数器事件。**一次采集通常只有 | ||
| 194 | + 两个采样点**，所以它是标称值，不是频率曲线。 | ||
| 195 | +- `derived`：逐 kernel 用 `cycles / time / cores` 反推，样本密集，而且它才是下游 | ||
| 196 | + cycle 派生指标真正依赖的那个值。两者冲突时以 `derived` 为准，并在 | ||
| 197 | + `cross_check.agreement` 里显式报出 mismatch，绝不取平均掩盖分歧。 | ||
| 198 | + | ||
| 199 | +**核数除数是关键**。`aic_total_cycles` 是该 kernel 占用的所有 core 的累加，所以只除 | ||
| 200 | +时间得到的是 cores × clock —— 24 核 kernel 会读成约 44 GHz。AIV 计数器在 `MIX_AIC` | ||
| 201 | +kernel 上要优先用 `Mix Block Dim`（向量段与 cube 段的核数不同），用 `Block Dim` 会 | ||
| 202 | +正好差 2 倍。 | ||
| 203 | + | ||
| 204 | +采集里没有计数器不算错误：频率与全部派生字段一律为 `null`，UI 显示不可用。**绝不用 | ||
| 205 | +假设频率替代**，那会静默缩放整套指标。DS3.2 实测两个来源一致为 1850 MHz，spread | ||
| 206 | +0.18%，无降频。 | ||
| 207 | + | ||
| 208 | +### MoE 专家计数（Step 2 产出，供 stage 2 使用） | ||
| 209 | + | ||
| 210 | +MoE 模型的 `model_manifest.json` 会带三项 fact，各自带 `source_ref` 指向配置源码行： | ||
| 211 | + | ||
| 212 | +| fact | DS3.2 实测 | 用途 | | ||
| 213 | +|---|---|---| | ||
| 214 | +| `n_routed_experts` | 256 | 路由专家总数 | | ||
| 215 | +| `n_shared_experts` | 1 | 共享专家数；**不被 EP 分片**，每个 rank 都有副本 | | ||
| 216 | +| `num_experts_per_tok` | 8 | top-k 路由宽度 | | ||
| 217 | + | ||
| 218 | +这三项是**声明值**，来自模型源码 AST，不是从 kernel shape 推断的。**绝不从 | ||
| 219 | +`GroupedMatmul` 的权重 shape 反推专家总数** —— 那个首维只是本 rank 的分片 | ||
| 220 | +（DS3.2 上是 16），当成总数会把模型规模少报一个 EP 倍数。 | ||
| 221 | + | ||
| 222 | +专家清单（`<model-id>_expert_inventory.json`，逐个列出全部 257 个专家及其 | ||
| 223 | +`data_state`）由 **stage 2 的 `build_expert_inventory.py`** 生成，不在本 skill： | ||
| 224 | +它需要 `kernel_attribution.json` 判定常驻专家数，也需要 `perf_data.json` 取共享专家 | ||
| 225 | +的实测耗时，两者都是 stage 2 的产物。本 skill 只负责把上面三项声明值连同 source_ref | ||
| 226 | +准确提取出来。细节见 stage 2 的 SKILL.md「MoE expert inventory」一节。 | ||
| 227 | + | ||
| 90 | **可选**： | 228 | **可选**： |
| 91 | 229 | ||
| 92 | ```bash | 230 | ```bash |
| @@ -116,49 +254,49 @@ python scripts/analyze_kernels.py --enrich \ | |||
| 116 | -r outputs/raw_ops.json | 254 | -r outputs/raw_ops.json |
| 117 | ``` | 255 | ``` |
| 118 | 256 | ||
| 119 | -字段规范、节点来源规则、命名规则、边界规则、shape_semantic 必填范围（11 类算子 + 统一维度符号 B/T/H/D/...）等**全部细节**见 `references/structure_analysis_guide.md`。SKILL.md 不再重述以避免分歧。 | 257 | +字段规范、节点来源规则、命名规则、边界规则、**显式数据流边的写法**等**全部细节**见 `references/structure_analysis_guide.md`。SKILL.md 不再重述以避免分歧。 |
| 258 | + | ||
| 259 | +`shape_semantic` 是可选注解，不进入正式门禁：它是叠加在 profiler 维度上的解释，缺失并不说明拆解错误。 | ||
| 120 | 260 | ||
| 121 | --- | 261 | --- |
| 122 | 262 | ||
| 123 | -### Step 3: Review 拆解结果 | 263 | +### Step 3: 校验、评分并迭代拆解 |
| 124 | 264 | ||
| 125 | 按 `references/structure_analysis_guide.md` §D.2 执行： | 265 | 按 `references/structure_analysis_guide.md` §D.2 执行： |
| 126 | 266 | ||
| 127 | -```bash | 267 | +统一入口输出**单一合法 JSON**（不要再用 `>>` 拼接多个 JSON 文档到 `issues.json`）： |
| 128 | -python scripts/check_structure.py -c outputs/analysis_config.json --json > outputs/issues.json | ||
| 129 | -python scripts/validate_shapes.py -c outputs/analysis_config.json --fail-fast --json >> outputs/issues.json | ||
| 130 | -python scripts/check_op_coverage.py -c outputs/analysis_config.json -r outputs/raw_ops.json --json >> outputs/issues.json | ||
| 131 | -``` | ||
| 132 | - | ||
| 133 | -- `issues.json` 为空 → 跳过 AI review，直接进入 Step 4 | ||
| 134 | -- 否则拉起 review subagent，**仅投喂** issue 命中的节点片段、对应源码切片（按 `code_ref` 稀疏读）、对应 raw_ops 切片；subagent 修正后回到本步重跑，迭代上限 3 次 | ||
| 135 | - | ||
| 136 | ---- | ||
| 137 | - | ||
| 138 | -### Step 4: 生成报告 | ||
| 139 | 268 | ||
| 140 | ```bash | 269 | ```bash |
| 141 | -python scripts/generate_report.py \ | 270 | +python scripts/run_validation.py \ |
| 142 | - -r outputs/raw_ops_details.json \ | ||
| 143 | -c outputs/analysis_config.json \ | 271 | -c outputs/analysis_config.json \ |
| 144 | - -o outputs/{prefix}_report.md \ | 272 | + -r outputs/raw_ops.json \ |
| 145 | - --html -d 3 | 273 | + -m outputs/model_manifest.json \ |
| 274 | + --source-dir /path/to/model-source \ | ||
| 275 | + --semantic-review outputs/semantic_review.json \ | ||
| 276 | + -o outputs/validation_report.json | ||
| 146 | ``` | 277 | ``` |
| 147 | 278 | ||
| 148 | -| 输出 | 说明 | | 279 | +- 校验前必须按 `references/semantic_review_protocol.md` 生成与当前输入哈希绑定的 `semantic_review.json`；九项源码/Trace 语义检查一个不少 |
| 149 | -|---|---| | 280 | +- 校验后必须运行 `score_breakdown.py`；仅 semantic review 与 validation 通过、且 score 输出 `convertible=true`（可运行正确率与分项比例达标、hard gates 全过）才生成指标并进入 `cann-perf-breakdown-to-ui-json` |
| 150 | -| `{prefix}_report.md` | Markdown 分析报告 | | 281 | +- 未达标时读取 `iteration_request.json`，编辑 `base_config_for_revision` 指向的历史最佳候选副本，只修正 `failed_dimensions`、`blocking_issues` 和 `validation_issues` 命中的内容；不得修改 `immutable_best_snapshot` |
| 151 | -| `{prefix}_report.html` | HTML 分析报告（默认产出） | | 282 | +- 修正后重新 enrich、重新语义审查、校验、评分；旧 review 不得复用；每轮配置和结果保存在 `iterations/`，直至通过 |
| 283 | +- `run_breakdown.py` 只负责生成请求和确定性门禁，不会自行调用 AI 改配置；执行本 Skill 的 agent 必须消费 `ai_mapping_request.json` / `semantic_review_request.json` / `iteration_request.json` 并继续循环，不能在 `needs_iteration` 时提前结束 | ||
| 284 | +- 默认最多评估 10 轮；每轮读 `iteration_request.json` 的 `remaining_iterations` 决定是否继续。默认不早停；只有显式传 `--stall-limit N` 时才会在连续 N 轮无提升后标记 `blocked_no_progress` | ||
| 285 | +- 完整量表、硬性否决项和防刷分规则见 `references/breakdown_scoring.md` | ||
| 286 | +- 单独调试某一维度时可直接运行 `check_structure.py` / `check_dataflow.py` / `check_op_coverage.py`（各自 `--json` 输出单一 JSON，不要 `>>` 追加） | ||
| 287 | +- `validate_shapes.py` 与 `run_validation.py --with-shapes` 仅供排查，不属于正式流程 | ||
| 288 | +- `check_manifest_trace.py`（MT1）默认只作 `info`：一次采集可能只覆盖单个 step 或单个 rank，trace 是辅助证据，不能反驳源码读出的结构；确需阻断时显式传 `--fail-on-trace-mismatch` | ||
| 152 | 289 | ||
| 153 | --- | 290 | --- |
| 154 | 291 | ||
| 155 | -### Step 5: 计算性能指标 | 292 | +### Step 4: 计算性能指标 |
| 156 | 293 | ||
| 157 | ```bash | 294 | ```bash |
| 158 | python scripts/compute_metrics.py \ | 295 | python scripts/compute_metrics.py \ |
| 159 | -r outputs/raw_ops_details.json \ | 296 | -r outputs/raw_ops_details.json \ |
| 160 | -c outputs/analysis_config.json \ | 297 | -c outputs/analysis_config.json \ |
| 161 | -o outputs/metrics_report.md \ | 298 | -o outputs/metrics_report.md \ |
| 299 | + --findings-out outputs/metrics_findings.json \ | ||
| 162 | -d 3 | 300 | -d 3 |
| 163 | ``` | 301 | ``` |
| 164 | 302 | ||
| @@ -187,6 +325,16 @@ python scripts/compute_metrics.py \ | |||
| 187 | 325 | ||
| 188 | **适用范围**：decoder layer 及子节点、stages、runtime_auxiliary 及子节点。 | 326 | **适用范围**：decoder layer 及子节点、stages、runtime_auxiliary 及子节点。 |
| 189 | 327 | ||
| 328 | +**诊断建议（咨询性质）**： | ||
| 329 | + | ||
| 330 | +`--findings-out` 输出 `metrics_findings.json`，把上表的诊断结论结构化为 `findings[{code, severity, metrics, text, advice_l1, next_data, not_applicable}]`，并按 `references/diagnosis_advice.md` 挂上「下一步该看什么数据」的建议。`metrics_report.md` 末尾同时生成按 `code` 聚类的建议段落。 | ||
| 331 | + | ||
| 332 | +- `code` 取有限枚举：`STREAM_PARALLEL_HIGH` / `STREAM_PARALLEL_MID` / `GAP_BUBBLE` / `WAIT_DOMINANT` / `UTIL_GOOD` / `UTIL_LOW` / `CLEAN_SEQUENTIAL` / `NORMAL` / `NO_DATA`。 | ||
| 333 | +- **建议不进任何门禁**：不参与 `validation_report`、`breakdown_score`、`hard_gates`，不影响上文的两个停止条件，不参与迭代循环。文档中 `advisory_only: true` 是契约的一部分，下游读到该文件必须视其为咨询信息。 | ||
| 334 | +- 建议只回答「下一步看什么数据」，**不断言根因**；指标本身不足以证明根因。 | ||
| 335 | +- `metric_scope == "aggregate"` 的节点（多 invocation 合并）其 gap/利用率/占比只描述该组总体，**不得外推到单实例**；此类建议自动带 `[聚合口径]` 前缀。 | ||
| 336 | +- 建议正文的唯一来源是 `references/diagnosis_advice.md`，改建议只改该文档、不改代码；该文档缺失时静默降级为「无建议」，不影响指标产出。 | ||
| 337 | + | ||
| 190 | --- | 338 | --- |
| 191 | 339 | ||
| 192 | ## 输出文件 | 340 | ## 输出文件 |
| @@ -196,12 +344,20 @@ python scripts/compute_metrics.py \ | |||
| 196 | | `raw_ops.json` | 单 Step kernel 概要（脚本用） | | 344 | | `raw_ops.json` | 单 Step kernel 概要（脚本用） | |
| 197 | | `raw_ops_details.json` | 单 Step kernel 详情（Step 4/5 用） | | 345 | | `raw_ops_details.json` | 单 Step kernel 详情（Step 4/5 用） | |
| 198 | | `raw_ops.compact.json` | Step 2 投喂 AI 的精简视图 | | 346 | | `raw_ops.compact.json` | Step 2 投喂 AI 的精简视图 | |
| 347 | +| `device_freq.json` | Step 4b AI Core 实测频率与交叉验证；下游所有 cycle 派生指标的分母 | | ||
| 348 | +| `model_manifest.json` | Step 2 架构真值；MoE 模型含 `n_routed_experts` / `n_shared_experts` / `num_experts_per_tok` 三项 fact（各带 source_ref）| | ||
| 349 | +| `dataflow_source.json` | Step 5 从 `forward()` 提取的数据流真值；AI 映射与语义审查都必须引用其边 ID | | ||
| 199 | | `op_segments.json`（可选） | layer 边界候选 | | 350 | | `op_segments.json`（可选） | layer 边界候选 | |
| 200 | | `analysis_config.json` | 拆解配置（Mode A 终版）| | 351 | | `analysis_config.json` | 拆解配置（Mode A 终版）| |
| 201 | | `model_structure.json` | 仅结构（Mode B）| | 352 | | `model_structure.json` | 仅结构（Mode B）| |
| 202 | -| `issues.json` | Step 3 检查结果 | | 353 | +| `semantic_review_request.json` / `semantic_review.json` | Step 8 的审查请求与源码/Trace 语义审查结论；绑定三项输入 SHA256 | |
| 203 | -| `{prefix}_report.md` / `.html` | 分析报告 | | 354 | +| `validation_report.json` | Step 9 统一校验结果（单一 JSON，`status`/`error_count`/`checks`/`issues`）| |
| 204 | -| `metrics_report.md` | 性能指标分析报告 | | 355 | +| `breakdown_score.json` | Step 10 的 100 分量化评分、分项结果、硬性否决项与修正动作 | |
| 356 | +| `iteration_history.json` | 每轮分数、是否提升及历史最佳配置快照 | | ||
| 357 | +| `iteration_request.json` | 未达标时下一轮的定向修正输入；通过后不再使用 | | ||
| 358 | +| `iterations/` | 每一轮 config、validation 和 score 的不可混淆快照 | | ||
| 359 | +| `metrics_report.md` | 性能指标分析报告（含按 code 聚类的诊断建议段落） | | ||
| 360 | +| `metrics_findings.json` | 结构化诊断 + L1 建议，`advisory_only: true`；**不进任何门禁** | | ||
| 205 | 361 | ||
| 206 | --- | 362 | --- |
| 207 | 363 | ||
| @@ -213,11 +369,30 @@ python scripts/compute_metrics.py \ | |||
| 213 | | `references/kernel_data_guide.md` | Step 1 — `kernel_details.csv` 字段说明 | | 369 | | `references/kernel_data_guide.md` | Step 1 — `kernel_details.csv` 字段说明 | |
| 214 | | `references/mode_b_branches.md` | Mode B 多分支表达约定 | | 370 | | `references/mode_b_branches.md` | Mode B 多分支表达约定 | |
| 215 | | `references/mode_c_delegate.md` | Mode C 委托 `cann-npu-perfanalysis` 模板 | | 371 | | `references/mode_c_delegate.md` | Mode C 委托 `cann-npu-perfanalysis` 模板 | |
| 216 | -| `scripts/analyze_kernels.py` | Step 1 提取与 Step 2 enrich | | 372 | +| `references/diagnosis_advice.md` | 诊断码 → L1 建议对照表；建议正文的唯一来源，咨询性质不进门禁 | |
| 373 | +| `scripts/extract_model_manifest.py` | Step 2 架构真值提取 → `model_manifest.json`（取 modeling 实际 import 的 config 类；可达时以 checkpoint `config.json` 覆盖默认参数，不可达则降级为 `low` 并记 gap） | | ||
| 374 | +| `scripts/validate_architecture.py` | Step 3 全局架构校验（层号完整/互斥、Dense/MoE、MTP、partial trace）；manifest 为低置信度时 A1/A4 降为 warning，避免门禁强制执行错误层数 | | ||
| 375 | +| `scripts/check_manifest_trace.py` | manifest 层数 × trace per-layer kernel 数交叉校验（MT1），并反推 trace 支持的层数 | | ||
| 376 | +| `scripts/analyze_kernels.py` | Step 4 提取与 enrich | | ||
| 377 | +| `scripts/device_freq.py` | Step 4b AI Core 实测频率 → `device_freq.json`（trace 声明值 × 逐 kernel 反推值交叉验证）| | ||
| 378 | +| `scripts/extract_dataflow.py` | **Step 5 数据流真值提取**：AST 解析 `forward()` 得到调用顺序、残差汇合（含 fused add-norm 与 `+=`）、并行 fork、config-gated 变体；data-dependent 分支记为 `unsupported` 而不猜测 | | ||
| 379 | +| `scripts/check_dataflow.py` | **D1-D7 校验**：配置声明的边必须与源码一致（漏声明残差 / 方向反了 / 绕过了源码没调用的节点 / 并行支路被串行化 / 未声明的数据依赖分支）；无源码时不出结论 | | ||
| 380 | +| `scripts/detect_trace_scope.py` | 可选：trace scope / 并行归属检测。`trace_scope` 已非必填字段，仅在需要标注采集范围时使用 | | ||
| 217 | | `scripts/segment_layers.py` | layer 边界候选 | | 381 | | `scripts/segment_layers.py` | layer 边界候选 | |
| 218 | -| `scripts/check_structure.py` | Step 3 树结构良构性 | | 382 | +| `references/ai_mapping_protocol.md` | **Step 6 AI 映射规程 + 提示词模板（全 op 归属，禁止 unmapped 冒充）** | |
| 219 | -| `scripts/validate_shapes.py` | Step 3 shape_semantic 一致性 | | 383 | +| `references/semantic_review_protocol.md` | **Step 8 源码/Trace 语义审查：Q/K/V、残差、层边界、尾部与 runtime** | |
| 220 | -| `scripts/check_op_coverage.py` | Step 3 op 全覆盖 | | 384 | +| `references/breakdown_scoring.md` | **Step 10 固定评分体系、硬性否决项、迭代与停止规则** | |
| 221 | -| `scripts/regression_check.py` | 与 baseline 做结构回归 | | 385 | +| `scripts/map_trace_instances.py` | Step 6 观测 invocation → `trace_instances` 精确映射 | |
| 222 | -| `scripts/generate_report.py` | Step 4 报告生成 | | 386 | +| `scripts/check_structure.py` | 树结构良构性（v1/v2） | |
| 223 | -| `scripts/compute_metrics.py` | Step 5 指标计算 | | 387 | +| `scripts/validate_shapes.py` | 可选排查：shape_semantic 一致性。**不在正式门禁内**，需 `run_validation.py --with-shapes` 才运行 | |
| 388 | +| `scripts/check_op_coverage.py` | 精确 op 覆盖（union，非数量外推） | | ||
| 389 | +| `scripts/check_sublayers.py` | 代表结构树子模块一致性（父=子 union、无重叠、主节点有真实 op、模板⊆代表实例） | | ||
| 390 | +| `scripts/prepare_semantic_review.py` / `validate_semantic_review.py` | 生成带哈希的审查请求，并确定性校验证据与输入一致性 | | ||
| 391 | +| `scripts/run_breakdown.py` | **通用 forward-eval 驱动**：缺映射时停在 `awaiting_ai_mapping`，缺失或过期审查时停在 `awaiting_semantic_review`；不伪造 passed | | ||
| 392 | +| `scripts/run_validation.py` | Step 9 统一校验入口，输出单一 JSON | | ||
| 393 | +| `scripts/score_breakdown.py` | Step 10 确定性评分入口，输出 breakdown_score.json | | ||
| 394 | +| `scripts/migrate_config.py` | v1 → v2 迁移（标记 `legacy_unverified`） | | ||
| 395 | +| `scripts/regression_check.py` | 结构回归 + 对 manifest 的语义架构回归（MA1-MA3） | | ||
| 396 | +| `scripts/compute_metrics.py` | Step 11 指标计算 + 结构化诊断/建议（`--findings-out`） | | ||
| 397 | +| `schemas/*.schema.json` | model_manifest / analysis_config_v2 / semantic_review / validation_report / breakdown_score 的严格 schema | | ||
| 398 | +| `adapters/*.py` | 模型族架构提取差异封装（deepseek/gemma/qwen/longcat） | | ||
| @@ -0,0 +1,310 @@ | |||
| 1 | +--- | ||
| 2 | +name: cann-perf-breakdown | ||
| 3 | +description: | | ||
| 4 | + Break down NPU performance data. Use model source as primary evidence to derive structure, then attach kernel_details.csv performance data to that structure. | ||
| 5 | + Trigger for kernel_details.csv analysis, model-level performance breakdown, LLM bottleneck analysis, source-only architecture extraction, or performance-only diagnosis delegated to cann-npu-perfanalysis. | ||
| 6 | + Applies to any Transformer family and assumes no fixed module or kernel names. | ||
| 7 | +--- | ||
| 8 | + | ||
| 9 | +# NPU Performance Breakdown | ||
| 10 | + | ||
| 11 | +Break down NPU profiling `kernel_details.csv` into a Transformer hierarchy and produce verifiable structured JSON and performance metrics. | ||
| 12 | + | ||
| 13 | +--- | ||
| 14 | + | ||
| 15 | +## Supported Scenarios | ||
| 16 | + | ||
| 17 | +| Scenario | Description | | ||
| 18 | +|---|---| | ||
| 19 | +| LLM performance analysis | Analyze any Transformer family | | ||
| 20 | +| Hierarchical timing | Attribute time to Embedding, Blocks, Head, and other stages | | ||
| 21 | +| Block internals | Analyze Attention, MLP, Norm, and nested submodules | | ||
| 22 | +| Architecture detection | Identify MLA, MoE, GQA, and similar features | | ||
| 23 | +| Source only | Produce a validated structure without binding performance data; express branches with `branches` | | ||
| 24 | +| Performance data only | Delegate eight-dimensional diagnosis to the repository's `cann-npu-perfanalysis` skill | | ||
| 25 | + | ||
| 26 | +## Entry Dispatch | ||
| 27 | + | ||
| 28 | +Inspect the working directory and select the first matching mode: | ||
| 29 | + | ||
| 30 | +| Condition | Mode | Action | | ||
| 31 | +|---|---|---| | ||
| 32 | +| Model source such as `*modeling*.py` plus `kernel_details.csv` or `raw_ops*.json` | **Mode A** | Run the complete schema-v2 11-step scoring loop; emit metrics and enter Stage 2 only after acceptance | | ||
| 33 | +| Model source only | **Mode B** | Extract and validate `model_manifest.json`, then emit non-empty v2 `model_structure.json` with `op_indices=[]` and optional `branches`; see `references/mode_b_branches.md` | | ||
| 34 | +| Performance data only | **Mode C** | Delegate to `cann-npu-perfanalysis`; see `references/mode_c_delegate_en.md` | | ||
| 35 | + | ||
| 36 | +Mode C does not enter structure breakdown. Mode A runs all 11 steps. Mode B runs at least architecture extraction, validation, and structure-tree generation. | ||
| 37 | + | ||
| 38 | +### Schema-v2 distinctions | ||
| 39 | + | ||
| 40 | +- Learned model layers in `architecture.layer_groups` and `prediction_modules` are not runtime calls in `trace_instances`. | ||
| 41 | +- In MTP/speculative decoding, an outer loop calling the same learned decoder layer N times means one learned layer plus N invocations. Never create N learned layers or synthetic indices such as 6, 7, and 8. | ||
| 42 | +- `children` expresses containment only. Residuals, parallel paths, and skips must be declared explicitly in `branches`; downstream consumers must not infer edges from child order. | ||
| 43 | +- Performance exists only for captured scope. Never extrapolate metrics to uncaptured layers. Optional `trace_scope` may be `full_model`, `rank_local`, `pipeline_stage_local`, or `unknown`; without evidence, omit it or use `unknown`. | ||
| 44 | + | ||
| 45 | +### Four-way coverage classification | ||
| 46 | + | ||
| 47 | +- `mapped_model_ops`: model-module operators owned through trace instances, stages, and structure leaves. | ||
| 48 | +- `mapped_runtime_ops`: runtime support operators in `runtime_auxiliary`. | ||
| 49 | +- `excluded_profiler_ops`: pure profiler/bookkeeping only, with an allowed `reason_code` and `evidence`. Never exclude main computation such as MatMul, Attention, Norm, MoE, communication, Gather, KV cache, or sampling. | ||
| 50 | +- `unmapped_ops`: unknown ownership means mapping is incomplete and strict validation fails. A `reason` does not count as coverage. | ||
| 51 | + | ||
| 52 | +`--allow-unmapped` is exploratory and never yields formal `passed`. Every representative-step operator must belong to one of the first three classes. Follow `references/ai_mapping_protocol_en.md`. | ||
| 53 | + | ||
| 54 | +One hundred percent kernel coverage does not prove semantic correctness. Source/trace semantic review must also validate Q/K/V branches, residual paths, layer boundaries, and tail stages. | ||
| 55 | + | ||
| 56 | +## Mode A: 11-Step Scoring Loop | ||
| 57 | + | ||
| 58 | +```text | ||
| 59 | +Step 1 Discover inputs and select a mode | ||
| 60 | +Step 2 extract_model_manifest.py -> model_manifest.json | ||
| 61 | +Step 3 validate_architecture.py | ||
| 62 | +Step 4 analyze_kernels.py -> raw_ops.json, raw_ops_details.json, raw_ops.compact.json | ||
| 63 | +Step 4b device_freq.py -> device_freq.json | ||
| 64 | +Step 5 extract_dataflow.py -> dataflow_source.json | ||
| 65 | +Step 6 AI mapping per ai_mapping_protocol_en.md plus map_trace_instances.py | ||
| 66 | + -> schema-v2 analysis_config.json | ||
| 67 | +Step 7 Build one reusable structures tree per layer group and validate sublayers | ||
| 68 | +Step 8 Complete source/trace semantic review -> semantic_review.json | ||
| 69 | +Step 9 run_validation.py -> validation_report.json with status exactly passed | ||
| 70 | +Step 10 score_breakdown.py -> breakdown_score.json with convertible=true | ||
| 71 | + Otherwise revise the historical best candidate from iteration_request.json and return to Step 8 | ||
| 72 | +Step 11 compute_metrics.py, only after acceptance | ||
| 73 | + -> metrics_report.md and advisory metrics_findings.json | ||
| 74 | +``` | ||
| 75 | + | ||
| 76 | +See `references/breakdown_scoring_en.md` for scoring and stop rules. Any configuration change invalidates the old semantic-review SHA256 binding. | ||
| 77 | + | ||
| 78 | +### Stop conditions | ||
| 79 | + | ||
| 80 | +Only two conditions stop the loop: | ||
| 81 | + | ||
| 82 | +1. `breakdown_score.convertible == true`: runnable correctness is at least 95%; every runnable core dimension reaches its original ratio (architecture 22/25, dataflow and branches 27/30, layer/submodule boundaries 18/20); exact coverage and evidence pass; and every hard gate passes. Evidence-qualified status is `verified`, `verified_unbound_scalars`, or `structure_unverified`. `passed` is legacy compatibility only. | ||
| 83 | +2. `--max-iterations`, default 10, is exhausted without acceptance; status is `blocked_max_iterations`. | ||
| 84 | + | ||
| 85 | +`score` is a raw value on a nominal 100-point scale. `runnable_max` is the denominator supported by available inputs. Missing checkpoint/source evidence removes an unanswerable scalar check from the denominator and lowers the evidence ceiling rather than permanently failing a correct structure. Downstream eligibility must use `convertible`, not a new hard-coded `score >= 95` test. | ||
| 86 | + | ||
| 87 | +Early stop is disabled by default with `--stall-limit 0`. Enable it explicitly with `--stall-limit N`. | ||
| 88 | + | ||
| 89 | +At the iteration limit, report the current score, each failed dimension's deficit, blocked semantic checks, and missing evidence. Never improve a score by lowering gates, deleting main kernels, or expanding exclusions. | ||
| 90 | + | ||
| 91 | +Architecture extraction in Steps 2-3 must scan the complete source. Sparse source reads are allowed only for issue repair after Step 8. The skill package must not retain model source, real profiling captures, or test fixtures; callers and repository tests provide validation inputs externally. | ||
| 92 | + | ||
| 93 | +### Semantic review and unified validation | ||
| 94 | + | ||
| 95 | +```bash | ||
| 96 | +python scripts/prepare_semantic_review.py \ | ||
| 97 | + -c outputs/analysis_config.json \ | ||
| 98 | + -r outputs/raw_ops.json \ | ||
| 99 | + -m outputs/model_manifest.json \ | ||
| 100 | + --source-dir /path/to/model-source \ | ||
| 101 | + -o outputs/semantic_review_request.json | ||
| 102 | +# Read source and the representative trace completely, then create outputs/semantic_review.json. | ||
| 103 | + | ||
| 104 | +python scripts/run_validation.py \ | ||
| 105 | + -c outputs/analysis_config.json \ | ||
| 106 | + -r outputs/raw_ops.json \ | ||
| 107 | + -m outputs/model_manifest.json \ | ||
| 108 | + --source-dir /path/to/model-source \ | ||
| 109 | + --semantic-review outputs/semantic_review.json \ | ||
| 110 | + -o outputs/validation_report.json | ||
| 111 | +# status other than passed blocks formal reporting. --allow-warnings is triage only. | ||
| 112 | + | ||
| 113 | +python scripts/score_breakdown.py \ | ||
| 114 | + -v outputs/validation_report.json \ | ||
| 115 | + -c outputs/analysis_config.json \ | ||
| 116 | + -r outputs/raw_ops.json \ | ||
| 117 | + -m outputs/model_manifest.json \ | ||
| 118 | + --semantic-review outputs/semantic_review.json \ | ||
| 119 | + -o outputs/breakdown_score.json | ||
| 120 | +# If convertible is not true, consume iteration_request.json and continue. Do not report formally. | ||
| 121 | +``` | ||
| 122 | + | ||
| 123 | +## Step 1: Extract One Profiling Step | ||
| 124 | + | ||
| 125 | +Enumerate steps and choose a stable non-warmup representative. Without `-s`, `analyze_kernels.py` groups by kernel count and kernel-type distribution. It skips the earliest step only when its kernel sum is a clear outlier against the median of later steps, then selects the closest later step. Use `-s` only to reproduce a specific step. | ||
| 126 | + | ||
| 127 | +```bash | ||
| 128 | +python scripts/analyze_kernels.py \ | ||
| 129 | + -f kernel_details.csv \ | ||
| 130 | + -o outputs/raw_ops.json \ | ||
| 131 | + -d outputs/raw_ops_details.json \ | ||
| 132 | + -m outputs/steps_summary.md \ | ||
| 133 | + --compact-out outputs/raw_ops.compact.json | ||
| 134 | +``` | ||
| 135 | + | ||
| 136 | +| Output | Purpose | | ||
| 137 | +|---|---| | ||
| 138 | +| `steps_summary.md` | Per-step count, kernel types, duration, and selection reason | | ||
| 139 | +| `raw_ops.json` | Selected-step operator summary for enrichment and validation | | ||
| 140 | +| `raw_ops_details.json` | Selected-step records with all CSV columns | | ||
| 141 | +| `raw_ops.compact.json` | Folded AI input without timing fields | | ||
| 142 | +| `device_freq.json` | Measured AI Core frequency and cross-check used by cycle-derived metrics | | ||
| 143 | + | ||
| 144 | +Each operator includes `org_index`, its zero-based row in `kernel_details.csv`. | ||
| 145 | + | ||
| 146 | +### AI Core frequency | ||
| 147 | + | ||
| 148 | +```bash | ||
| 149 | +python scripts/device_freq.py \ | ||
| 150 | + -d outputs/raw_ops_details.json \ | ||
| 151 | + --trace ASCEND_PROFILER_OUTPUT/trace_view.json \ | ||
| 152 | + -o outputs/device_freq.json | ||
| 153 | +``` | ||
| 154 | + | ||
| 155 | +Report two independent sources without averaging: | ||
| 156 | + | ||
| 157 | +- `declared`: sparse `AI Core Freq` events in `trace_view.json`, normally a nominal value rather than a curve. | ||
| 158 | +- `derived`: per-kernel `cycles / time / cores`, the value used by downstream cycle-derived metrics. On conflict, use `derived` and record a mismatch in `cross_check.agreement`. | ||
| 159 | + | ||
| 160 | +Core count is essential because `aic_total_cycles` is summed over all participating cores. For AIV counters on `MIX_AIC`, prefer `Mix Block Dim`, since vector and cube core counts differ. | ||
| 161 | + | ||
| 162 | +If counters are absent, frequency and every derived field are `null`; never substitute an assumed frequency. | ||
| 163 | + | ||
| 164 | +### MoE expert facts | ||
| 165 | + | ||
| 166 | +For MoE, `model_manifest.json` contains `n_routed_experts`, `n_shared_experts`, and `num_experts_per_tok`, each with source evidence. These are AST-derived declarations, not kernel-shape inference. Never infer total experts from the first dimension of GroupedMatmul weights; that is often only the local EP shard. | ||
| 167 | + | ||
| 168 | +Stage 2 `build_expert_inventory.py` generates the per-expert inventory because it needs Stage 2 kernel attribution and measured shared-expert timing. This skill only extracts the three declarations and their references. | ||
| 169 | + | ||
| 170 | +Optionally generate candidate layer boundaries: | ||
| 171 | + | ||
| 172 | +```bash | ||
| 173 | +python scripts/segment_layers.py -r outputs/raw_ops.json -o outputs/op_segments.json | ||
| 174 | +``` | ||
| 175 | + | ||
| 176 | +Candidates are hints only; source semantics determine final boundaries. See `references/kernel_data_guide_en.md`. | ||
| 177 | + | ||
| 178 | +## Step 2: Break Down Model Structure | ||
| 179 | + | ||
| 180 | +Follow `references/structure_analysis_guide_en.md` D.1: | ||
| 181 | + | ||
| 182 | +1. Select the representative step and decoder-layer instance. | ||
| 183 | +2. Read model source and derive modules and stable function boundaries. | ||
| 184 | +3. Locate decoder boundaries in the operator sequence. | ||
| 185 | +4. Map every operator to source-backed module/function semantics. | ||
| 186 | +5. Write `analysis_config.json`. | ||
| 187 | +6. Enrich it: | ||
| 188 | + | ||
| 189 | +```bash | ||
| 190 | +python scripts/analyze_kernels.py --enrich \ | ||
| 191 | + -c outputs/analysis_config.json \ | ||
| 192 | + -r outputs/raw_ops.json | ||
| 193 | +``` | ||
| 194 | + | ||
| 195 | +The guide is authoritative for sources, names, boundaries, and explicit dataflow edges. `shape_semantic` is optional and is not a formal gate. | ||
| 196 | + | ||
| 197 | +## Step 3: Validate, Score, and Iterate | ||
| 198 | + | ||
| 199 | +Follow `references/structure_analysis_guide_en.md` D.2. Use the unified entry point, which emits one valid JSON document; never append multiple JSON objects with `>>`. | ||
| 200 | + | ||
| 201 | +```bash | ||
| 202 | +python scripts/run_validation.py \ | ||
| 203 | + -c outputs/analysis_config.json \ | ||
| 204 | + -r outputs/raw_ops.json \ | ||
| 205 | + -m outputs/model_manifest.json \ | ||
| 206 | + --source-dir /path/to/model-source \ | ||
| 207 | + --semantic-review outputs/semantic_review.json \ | ||
| 208 | + -o outputs/validation_report.json | ||
| 209 | +``` | ||
| 210 | + | ||
| 211 | +- Create a current, hash-bound nine-item semantic review before validation. | ||
| 212 | +- Run scoring afterward. Generate metrics and enter `cann-perf-breakdown-to-ui-json` only when semantic review and validation pass and `convertible=true`. | ||
| 213 | +- Otherwise edit the best candidate referenced by `base_config_for_revision`, correcting only failed dimensions, blocking issues, and validation issues. Never edit `immutable_best_snapshot`. | ||
| 214 | +- Re-enrich, rereview, revalidate, and rescore after every edit. Do not reuse an old review. | ||
| 215 | +- `run_breakdown.py` creates requests and deterministic gates but does not invoke AI. The executing agent must consume mapping, semantic-review, and iteration requests until a stop condition. | ||
| 216 | +- Evaluate at most 10 rounds by default. Read `remaining_iterations`; use early stop only when explicitly configured. | ||
| 217 | +- Run individual checks only for diagnosis. Each `--json` command emits one JSON object. | ||
| 218 | +- Shape validation is optional diagnostic work. MT1 is `info` by default because a partial capture cannot refute source-derived architecture. | ||
| 219 | + | ||
| 220 | +## Step 4: Compute Performance Metrics | ||
| 221 | + | ||
| 222 | +```bash | ||
| 223 | +python scripts/compute_metrics.py \ | ||
| 224 | + -r outputs/raw_ops_details.json \ | ||
| 225 | + -c outputs/analysis_config.json \ | ||
| 226 | + -o outputs/metrics_report.md \ | ||
| 227 | + --findings-out outputs/metrics_findings.json \ | ||
| 228 | + -d 3 | ||
| 229 | +``` | ||
| 230 | + | ||
| 231 | +| Metric | Definition | Meaning | | ||
| 232 | +|---|---|---| | ||
| 233 | +| `wall_ms` | Last kernel end minus first kernel start | Wall-clock duration including gaps | | ||
| 234 | +| `busy_union_ms` | Union of device-busy intervals | Actual non-overlapping busy time | | ||
| 235 | +| `kernel_sum_ms` | Arithmetic sum of kernel durations | Total work ignoring overlap | | ||
| 236 | +| `total_cost_ms` | Sum of duration plus wait | Full cost including waits | | ||
| 237 | + | ||
| 238 | +Derived values include parallelism (`kernel_sum_ms / wall_ms`), `bubble_ms`, and wall share of the step. | ||
| 239 | + | ||
| 240 | +| Condition | Threshold | Diagnosis | | ||
| 241 | +|---|---|---| | ||
| 242 | +| kernel sum / wall | > 1.5 | High multi-stream overlap | | ||
| 243 | +| kernel sum / wall | > 1.2 | Moderate overlap | | ||
| 244 | +| wall / busy union | > 1.5 | Gap bubble | | ||
| 245 | +| total cost / kernel sum | > 1.3 | Significant waiting; inspect wait anchors | | ||
| 246 | +| busy union / wall | 80%-95% | Good utilization | | ||
| 247 | +| busy union / wall | < 80% | Low utilization | | ||
| 248 | +| busy union, wall, kernel sum | within 10% | Clean sequential execution | | ||
| 249 | + | ||
| 250 | +Apply metrics to decoder layers and subnodes, stages, and runtime auxiliary nodes. | ||
| 251 | + | ||
| 252 | +`metrics_findings.json` is advisory only. Codes are `STREAM_PARALLEL_HIGH`, `STREAM_PARALLEL_MID`, `GAP_BUBBLE`, `WAIT_DOMINANT`, `UTIL_GOOD`, `UTIL_LOW`, `CLEAN_SEQUENTIAL`, `NORMAL`, and `NO_DATA`. Findings never affect validation, score, hard gates, stop conditions, or iteration. They say what data to inspect next and must not assert root cause. Aggregate-scope advice cannot be extrapolated to one instance. Runtime advice text is loaded from `references/diagnosis_advice.md`; `references/diagnosis_advice_en.md` is its read-only English translation. | ||
| 253 | + | ||
| 254 | +## Outputs | ||
| 255 | + | ||
| 256 | +| File | Description | | ||
| 257 | +|---|---| | ||
| 258 | +| `raw_ops.json` | Selected-step operator summary | | ||
| 259 | +| `raw_ops_details.json` | Selected-step full records | | ||
| 260 | +| `raw_ops.compact.json` | Folded AI view | | ||
| 261 | +| `device_freq.json` | Measured and cross-checked AI Core frequency | | ||
| 262 | +| `model_manifest.json` | Source-derived architecture truth and MoE facts | | ||
| 263 | +| `dataflow_source.json` | Source-derived `forward()` dataflow truth | | ||
| 264 | +| `op_segments.json` | Optional boundary candidates | | ||
| 265 | +| `analysis_config.json` | Final Mode A breakdown | | ||
| 266 | +| `model_structure.json` | Mode B structure | | ||
| 267 | +| `semantic_review_request.json`, `semantic_review.json` | Hash-bound review request and conclusions | | ||
| 268 | +| `validation_report.json` | Unified Step 9 validation | | ||
| 269 | +| `breakdown_score.json` | Step 10 score, dimensions, hard gates, and actions | | ||
| 270 | +| `iteration_history.json` | Round history and best snapshots | | ||
| 271 | +| `iteration_request.json` | Targeted next-round request | | ||
| 272 | +| `iterations/` | Immutable round snapshots | | ||
| 273 | +| `metrics_report.md` | Performance report with advisory diagnoses | | ||
| 274 | +| `metrics_findings.json` | Structured advisory findings with `advisory_only: true` | | ||
| 275 | + | ||
| 276 | +## References | ||
| 277 | + | ||
| 278 | +| File | Use | | ||
| 279 | +|---|---| | ||
| 280 | +| `references/structure_analysis_guide_en.md` | All Step 2 and Step 3 structure rules | | ||
| 281 | +| `references/kernel_data_guide_en.md` | `kernel_details.csv` fields | | ||
| 282 | +| `references/mode_b_branches.md` | Mode B branch representation | | ||
| 283 | +| `references/mode_c_delegate_en.md` | Mode C delegation | | ||
| 284 | +| `references/diagnosis_advice_en.md` | English translation of the runtime diagnostic-advice source | | ||
| 285 | +| `references/ai_mapping_protocol_en.md` | Mandatory full-operator mapping protocol | | ||
| 286 | +| `references/semantic_review_protocol_en.md` | Required source/trace semantic review | | ||
| 287 | +| `references/breakdown_scoring_en.md` | Scoring, hard gates, iteration, and stopping | | ||
| 288 | +| `scripts/extract_model_manifest.py` | Architecture extraction | | ||
| 289 | +| `scripts/validate_architecture.py` | Global architecture validation | | ||
| 290 | +| `scripts/check_manifest_trace.py` | Manifest/trace cross-check | | ||
| 291 | +| `scripts/analyze_kernels.py` | Step extraction and enrichment | | ||
| 292 | +| `scripts/device_freq.py` | Frequency derivation and cross-check | | ||
| 293 | +| `scripts/extract_dataflow.py` | AST-derived dataflow truth | | ||
| 294 | +| `scripts/check_dataflow.py` | D1-D7 source/config checks | | ||
| 295 | +| `scripts/detect_trace_scope.py` | Optional capture-scope detection | | ||
| 296 | +| `scripts/segment_layers.py` | Boundary candidates | | ||
| 297 | +| `scripts/map_trace_instances.py` | Invocation mapping | | ||
| 298 | +| `scripts/check_structure.py` | Tree well-formedness | | ||
| 299 | +| `scripts/validate_shapes.py` | Optional shape diagnostics | | ||
| 300 | +| `scripts/check_op_coverage.py` | Exact union coverage | | ||
| 301 | +| `scripts/check_sublayers.py` | Representative subtree consistency | | ||
| 302 | +| `scripts/prepare_semantic_review.py`, `scripts/validate_semantic_review.py` | Review preparation and deterministic validation | | ||
| 303 | +| `scripts/run_breakdown.py` | Forward-evaluation driver and request gates | | ||
| 304 | +| `scripts/run_validation.py` | Unified validation | | ||
| 305 | +| `scripts/score_breakdown.py` | Deterministic scoring | | ||
| 306 | +| `scripts/migrate_config.py` | v1-to-v2 migration with `legacy_unverified` | | ||
| 307 | +| `scripts/regression_check.py` | Structural and semantic architecture regression | | ||
| 308 | +| `scripts/compute_metrics.py` | Metrics and advisory findings | | ||
| 309 | +| `schemas/*.schema.json` | Strict artifact schemas | | ||
| 310 | +| `adapters/*.py` | Family-specific architecture extraction | | ||
| @@ -0,0 +1,120 @@ | |||
| 1 | +# Copyright (c) 2026 Huawei Technologies Co., Ltd. | ||
| 2 | +# | ||
| 3 | +# Licensed under the Apache License, Version 2.0 (the "License"); | ||
| 4 | +# you may not use this file except in compliance with the License. | ||
| 5 | +# You may obtain a copy of the License at | ||
| 6 | +# | ||
| 7 | +# http://www.apache.org/licenses/LICENSE-2.0 | ||
| 8 | +# | ||
| 9 | +# Unless required by applicable law or agreed to in writing, software | ||
| 10 | +# distributed under the License is distributed on an "AS IS" BASIS, | ||
| 11 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| 12 | +# See the License for the specific language governing permissions and | ||
| 13 | +# limitations under the License. | ||
| 14 | +# | ||
| 15 | +"""Model-family adapters for architecture extraction. | ||
| 16 | + | ||
| 17 | +Adapters ONLY encapsulate static-extraction differences between model families. | ||
| 18 | +They must never hardcode architecture numbers without a source_ref: every value | ||
| 19 | +an adapter returns is read from the model's own configuration/modeling source via | ||
| 20 | +AST, and carries the file:line it came from. | ||
| 21 | + | ||
| 22 | +Selection is by evidence (class names / config keys found in the source), never by | ||
| 23 | +a caller-supplied model name string alone, and never by position in this list. | ||
| 24 | +""" | ||
| 25 | +from .base import BaseAdapter, Fact, GENERIC | ||
| 26 | +from .deepseek import DeepseekAdapter | ||
| 27 | +from .gemma import GemmaAdapter | ||
| 28 | +from .qwen import QwenAdapter | ||
| 29 | +from .longcat import LongcatAdapter | ||
| 30 | + | ||
| 31 | +#: Registration order carries NO meaning for selection -- see `resolve_adapter`. It only | ||
| 32 | +#: fixes the order ambiguous candidates are reported in, so the error message is stable. | ||
| 33 | +ADAPTERS = [ | ||
| 34 | + DeepseekAdapter(), | ||
| 35 | + LongcatAdapter(), | ||
| 36 | + GemmaAdapter(), | ||
| 37 | + QwenAdapter(), | ||
| 38 | +] | ||
| 39 | + | ||
| 40 | +CONFIDENCE_RANK = {'high': 3, 'medium': 2, 'low': 1} | ||
| 41 | + | ||
| 42 | + | ||
| 43 | +class AmbiguousAdapterError(Exception): | ||
| 44 | + """Two or more families claim this source with equal, top-tier confidence. | ||
| 45 | + | ||
| 46 | + Resolving this by list order is what makes a misread architecture invisible: the | ||
| 47 | + manifest would name one family, carry that family's layer-count and MoE-boundary key | ||
| 48 | + spellings, and look exactly as authoritative as a correct one. The source genuinely | ||
| 49 | + matches both signatures -- that is a fact the caller has to see and settle, not | ||
| 50 | + something this module may decide. | ||
| 51 | + """ | ||
| 52 | + | ||
| 53 | + def __init__(self, candidates): | ||
| 54 | + self.candidates = candidates | ||
| 55 | + names = ', '.join(f"{c['adapter']}({c['confidence']})" for c in candidates) | ||
| 56 | + super().__init__( | ||
| 57 | + f'适配器选择歧义：{names} 同时以最高置信度匹配同一份源码。' | ||
| 58 | + f'不按注册顺序裁决——请补充区分性证据（config key / class 名）' | ||
| 59 | + f'或显式指定适配器。理由：' | ||
| 60 | + + ' | '.join(f"{c['adapter']}: {'; '.join(c['reasons']) or '未给出理由'}" | ||
| 61 | + for c in candidates)) | ||
| 62 | + | ||
| 63 | + | ||
| 64 | +def _normalise(result): | ||
| 65 | + """Read a `matches()` return value as (confidence, reasons) or None. | ||
| 66 | + | ||
| 67 | + A bare `True` is still accepted so adapters written against the original protocol keep | ||
| 68 | + working; it is read as 'medium' because a boolean carries no strength information and | ||
| 69 | + claiming 'high' on its behalf would let it win a tier it never asserted. | ||
| 70 | + """ | ||
| 71 | + if result is None or result is False: | ||
| 72 | + return None | ||
| 73 | + if result is True: | ||
| 74 | + return 'medium', [] | ||
| 75 | + try: | ||
| 76 | + confidence, reasons = result | ||
| 77 | + except (TypeError, ValueError): | ||
| 78 | + return 'medium', [] | ||
| 79 | + if confidence not in CONFIDENCE_RANK: | ||
| 80 | + confidence = 'medium' | ||
| 81 | + return confidence, list(reasons or []) | ||
| 82 | + | ||
| 83 | + | ||
| 84 | +def resolve_adapter(evidence: dict): | ||
| 85 | + """Return a selection record: {adapter, confidence, reasons, candidates, ambiguous}. | ||
| 86 | + | ||
| 87 | + Every adapter is consulted; the highest confidence tier wins. A single candidate in | ||
| 88 | + that tier is the answer. Two or more raise `AmbiguousAdapterError` -- see there for | ||
| 89 | + why this is not resolved silently. No match at all falls back to the generic base, | ||
| 90 | + which asserts no family knowledge. | ||
| 91 | + """ | ||
| 92 | + candidates = [] | ||
| 93 | + for adapter in ADAPTERS: | ||
| 94 | + verdict = _normalise(adapter.matches(evidence)) | ||
| 95 | + if verdict is None: | ||
| 96 | + continue | ||
| 97 | + confidence, reasons = verdict | ||
| 98 | + candidates.append({'adapter': adapter.name, 'confidence': confidence, | ||
| 99 | + 'reasons': reasons, '_obj': adapter}) | ||
| 100 | + | ||
| 101 | + if not candidates: | ||
| 102 | + return {'adapter': GENERIC, 'name': GENERIC.name, 'confidence': 'unknown', | ||
| 103 | + 'reasons': ['没有任何族签名匹配；使用不含族知识的 generic 基类'], | ||
| 104 | + 'candidates': [], 'ambiguous': False} | ||
| 105 | + | ||
| 106 | + top = max(CONFIDENCE_RANK[c['confidence']] for c in candidates) | ||
| 107 | + winners = [c for c in candidates if CONFIDENCE_RANK[c['confidence']] == top] | ||
| 108 | + reported = [{k: v for k, v in c.items() if k != '_obj'} for c in candidates] | ||
| 109 | + if len(winners) > 1: | ||
| 110 | + raise AmbiguousAdapterError([{k: v for k, v in w.items() if k != '_obj'} | ||
| 111 | + for w in winners]) | ||
| 112 | + winner = winners[0] | ||
| 113 | + return {'adapter': winner['_obj'], 'name': winner['adapter'], | ||
| 114 | + 'confidence': winner['confidence'], 'reasons': winner['reasons'], | ||
| 115 | + 'candidates': reported, 'ambiguous': False} | ||
| 116 | + | ||
| 117 | + | ||
| 118 | +def select_adapter(evidence: dict): | ||
| 119 | + """The adapter for this evidence. Raises `AmbiguousAdapterError` on a tie.""" | ||
| 120 | + return resolve_adapter(evidence)['adapter'] | ||
| @@ -0,0 +1,749 @@ | |||
| 1 | +# Copyright (c) 2026 Huawei Technologies Co., Ltd. | ||
| 2 | +# Licensed under the Apache License, Version 2.0 (the "License"); | ||
| 3 | +# you may not use this file except in compliance with the License. | ||
| 4 | +# You may obtain a copy of the License at | ||
| 5 | +# | ||
| 6 | +# http://www.apache.org/licenses/LICENSE-2.0 | ||
| 7 | +# | ||
| 8 | +# Unless required by applicable law or agreed to in writing, software | ||
| 9 | +# distributed under the License is distributed on an "AS IS" BASIS, | ||
| 10 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| 11 | +# See the License for the specific language governing permissions and | ||
| 12 | +# limitations under the License. | ||
| 13 | +# | ||
| 14 | +"""Base adapter + generic AST extraction toolkit. | ||
| 15 | + | ||
| 16 | +No model code is imported or executed (models may require torch_npu). Everything | ||
| 17 | +here is pure `ast` static analysis over the source text. | ||
| 18 | +""" | ||
| 19 | +import ast | ||
| 20 | +import os | ||
| 21 | +from dataclasses import dataclass, field | ||
| 22 | + | ||
| 23 | + | ||
| 24 | + | ||
| 25 | +class Fact: | ||
| 26 | + key: str | ||
| 27 | + value: object | ||
| 28 | + source_ref: str # "file.py:line" or "unknown" | ||
| 29 | + method: str # extraction_method enum value | ||
| 30 | + confidence: str # high | medium | low | unknown | ||
| 31 | + | ||
| 32 | + def as_dict(self): | ||
| 33 | + return { | ||
| 34 | + 'key': self.key, | ||
| 35 | + 'value': self.value, | ||
| 36 | + 'source_ref': self.source_ref, | ||
| 37 | + 'method': self.method, | ||
| 38 | + 'confidence': self.confidence, | ||
| 39 | + } | ||
| 40 | + | ||
| 41 | + | ||
| 42 | +UNKNOWN = 'unknown' | ||
| 43 | + | ||
| 44 | + | ||
| 45 | + | ||
| 46 | +class ExtractionContext: | ||
| 47 | + config_tree: object | ||
| 48 | + modeling_tree: object | ||
| 49 | + config_defaults: dict | ||
| 50 | + base_dir: str | ||
| 51 | + config_path: str | ||
| 52 | + modeling_path: str | ||
| 53 | + | ||
| 54 | + def ref(self, lineno, path=None): | ||
| 55 | + return rel_ref(path or self.config_path, lineno, self.base_dir) | ||
| 56 | + | ||
| 57 | + | ||
| 58 | + | ||
| 59 | +class ExtractionState: | ||
| 60 | + context: ExtractionContext | ||
| 61 | + facts: list = field(default_factory=list) | ||
| 62 | + gaps: list = field(default_factory=list) | ||
| 63 | + num_main: object = UNKNOWN | ||
| 64 | + pred_count: int = 0 | ||
| 65 | + pred_key: str = None | ||
| 66 | + first_k_dense: int = None | ||
| 67 | + has_moe: bool = False | ||
| 68 | + moe_key_unbound: bool = False | ||
| 69 | + base_class_name: str = 'DecoderLayer' | ||
| 70 | + construct_ref: str = None | ||
| 71 | + layer_groups: list = field(default_factory=list) | ||
| 72 | + prediction_modules: list = field(default_factory=list) | ||
| 73 | + | ||
| 74 | + | ||
| 75 | +def rel_ref(path, lineno, base=None, end=None): | ||
| 76 | + """Format a source_ref, made relative to `base` if possible.""" | ||
| 77 | + p = path | ||
| 78 | + if base: | ||
| 79 | + try: | ||
| 80 | + p = os.path.relpath(path, base) | ||
| 81 | + except ValueError: | ||
| 82 | + p = path | ||
| 83 | + if end and end != lineno: | ||
| 84 | + return f'{p}:{lineno}-{end}' | ||
| 85 | + return f'{p}:{lineno}' | ||
| 86 | + | ||
| 87 | + | ||
| 88 | +def _config_names_from_import(node): | ||
| 89 | + if not isinstance(node, ast.ImportFrom): | ||
| 90 | + return set() | ||
| 91 | + if 'config' not in (node.module or '').lower(): | ||
| 92 | + return set() | ||
| 93 | + return {alias.name for alias in node.names if alias.name.endswith('Config')} | ||
| 94 | + | ||
| 95 | + | ||
| 96 | +def config_classes_imported_by(modeling_tree): | ||
| 97 | + """Names imported from a sibling config module by the modeling source. | ||
| 98 | + | ||
| 99 | + LongCat's config file defines both LongcatFlashConfig (num_layers=61, the full | ||
| 100 | + Flash model) and LongcatFlashNgramConfig (num_layers=28), and the modeling file | ||
| 101 | + imports ONLY the latter. Picking by source order reads a class the model never | ||
| 102 | + instantiates, so the layer count is wrong before any other analysis begins. | ||
| 103 | + """ | ||
| 104 | + if modeling_tree is None: | ||
| 105 | + return set() | ||
| 106 | + names = set() | ||
| 107 | + for node in ast.walk(modeling_tree): | ||
| 108 | + names.update(_config_names_from_import(node)) | ||
| 109 | + return names | ||
| 110 | + | ||
| 111 | + | ||
| 112 | +def _config_candidate(node): | ||
| 113 | + if not isinstance(node, ast.ClassDef) or not node.name.endswith('Config'): | ||
| 114 | + return None | ||
| 115 | + init_fn = next((item for item in node.body | ||
| 116 | + if isinstance(item, ast.FunctionDef) and item.name == '__init__'), None) | ||
| 117 | + return node, init_fn | ||
| 118 | + | ||
| 119 | + | ||
| 120 | +def _most_derived_config(candidates): | ||
| 121 | + base_names = set() | ||
| 122 | + for candidate, _init_fn in candidates: | ||
| 123 | + base_names.update(base.id for base in candidate.bases if isinstance(base, ast.Name)) | ||
| 124 | + derived = [candidate for candidate in candidates if candidate[0].name not in base_names] | ||
| 125 | + return (derived or candidates)[0] | ||
| 126 | + | ||
| 127 | + | ||
| 128 | +def find_config_class(tree, modeling_tree=None): | ||
| 129 | + """Return (ClassDef, __init__) for the config class the model actually uses. | ||
| 130 | + | ||
| 131 | + A config module may define several *Config classes for different model variants. | ||
| 132 | + The one the modeling source imports is the authoritative one; source order is not | ||
| 133 | + evidence of anything. Falls back to the first class with an __init__ when there is | ||
| 134 | + no import to go by, which is the single-class case. | ||
| 135 | + """ | ||
| 136 | + candidates = [] | ||
| 137 | + for node in ast.walk(tree): | ||
| 138 | + candidate = _config_candidate(node) | ||
| 139 | + if candidate is not None: | ||
| 140 | + candidates.append(candidate) | ||
| 141 | + if not candidates: | ||
| 142 | + return None, None | ||
| 143 | + | ||
| 144 | + imported = config_classes_imported_by(modeling_tree) | ||
| 145 | + with_init = [c for c in candidates if c[1] is not None] | ||
| 146 | + preferred = [c for c in with_init if c[0].name in imported] | ||
| 147 | + if preferred: | ||
| 148 | + # A subclass overrides its base's defaults, so the narrowest imported class wins. | ||
| 149 | + return _most_derived_config(preferred) | ||
| 150 | + return with_init[0] if with_init else candidates[0] | ||
| 151 | + | ||
| 152 | + | ||
| 153 | +def extract_init_defaults(init_fn): | ||
| 154 | + """Return {arg_name: (literal_value, lineno)} for keyword defaults in an __init__.""" | ||
| 155 | + out = {} | ||
| 156 | + args = init_fn.args | ||
| 157 | + defaults = args.defaults | ||
| 158 | + posargs = args.args | ||
| 159 | + # defaults align to the tail of posargs | ||
| 160 | + if defaults: | ||
| 161 | + offset = len(posargs) - len(defaults) | ||
| 162 | + for i, default in enumerate(defaults): | ||
| 163 | + name = posargs[offset + i].arg | ||
| 164 | + val = literal_or_none(default) | ||
| 165 | + if val is not None or isinstance(default, ast.Constant): | ||
| 166 | + out[name] = (val, getattr(default, 'lineno', init_fn.lineno)) | ||
| 167 | + # kwonly | ||
| 168 | + for kw, default in zip(args.kwonlyargs, args.kw_defaults): | ||
| 169 | + if default is not None: | ||
| 170 | + out[kw.arg] = (literal_or_none(default), getattr(default, 'lineno', init_fn.lineno)) | ||
| 171 | + return out | ||
| 172 | + | ||
| 173 | + | ||
| 174 | +def literal_or_none(node): | ||
| 175 | + """Best-effort literal evaluation; returns None if not a simple literal.""" | ||
| 176 | + try: | ||
| 177 | + return ast.literal_eval(node) | ||
| 178 | + except (ValueError, TypeError, SyntaxError): | ||
| 179 | + return None | ||
| 180 | + | ||
| 181 | + | ||
| 182 | +def find_range_calls_over_config(tree, attr_names): | ||
| 183 | + """ | ||
| 184 | + Find `for <var> in range(... config.<attr> ...)` or `range(self.<attr>)` loops | ||
| 185 | + used to build ModuleList/ModuleDict of decoder layers. | ||
| 186 | + Returns list of dicts: {attr, lineno, is_moduledict}. | ||
| 187 | + """ | ||
| 188 | + results = [] | ||
| 189 | + for node in ast.walk(tree): | ||
| 190 | + if isinstance(node, ast.comprehension): | ||
| 191 | + rng = node.iter | ||
| 192 | + attr = _range_attr(rng, attr_names) | ||
| 193 | + if attr: | ||
| 194 | + results.append({'attr': attr, 'lineno': getattr(rng, 'lineno', 0)}) | ||
| 195 | + if isinstance(node, ast.For): | ||
| 196 | + attr = _range_attr(node.iter, attr_names) | ||
| 197 | + if attr: | ||
| 198 | + results.append({'attr': attr, 'lineno': getattr(node.iter, 'lineno', 0)}) | ||
| 199 | + return results | ||
| 200 | + | ||
| 201 | + | ||
| 202 | +def _range_attr(call, attr_names): | ||
| 203 | + """If call is range(<attr>) where attr resolves to one of attr_names, return the attr name.""" | ||
| 204 | + if not isinstance(call, ast.Call): | ||
| 205 | + return None | ||
| 206 | + if not (isinstance(call.func, ast.Name) and call.func.id == 'range'): | ||
| 207 | + return None | ||
| 208 | + for arg in call.args: | ||
| 209 | + for sub in ast.walk(arg): | ||
| 210 | + if isinstance(sub, ast.Attribute) and sub.attr in attr_names: | ||
| 211 | + return sub.attr | ||
| 212 | + return None | ||
| 213 | + | ||
| 214 | + | ||
| 215 | +def find_decoder_layer_classes(tree): | ||
| 216 | + """Return heuristic layer class names, DecoderLayer-suffixed prioritized over Block. | ||
| 217 | + | ||
| 218 | + A 'Block' can also match unrelated helper classes (e.g. Gemma4SparseMoeBlock), | ||
| 219 | + so DecoderLayer wins when both exist. Blocks named like MoE/MLP/Attention/Norm | ||
| 220 | + helpers are excluded from the Block fallback. | ||
| 221 | + """ | ||
| 222 | + decoder_layers = [] | ||
| 223 | + blocks = [] | ||
| 224 | + helper_names = ('Moe', 'MLP', 'Mlp', 'Attention', 'Norm', 'Embed', 'Expert', 'Router', 'Gate') | ||
| 225 | + for node in ast.walk(tree): | ||
| 226 | + if isinstance(node, ast.ClassDef): | ||
| 227 | + if node.name.endswith('DecoderLayer'): | ||
| 228 | + decoder_layers.append((node.name, node.lineno)) | ||
| 229 | + elif node.name.endswith('Block') and not any(h in node.name for h in helper_names): | ||
| 230 | + blocks.append((node.name, node.lineno)) | ||
| 231 | + return decoder_layers if decoder_layers else blocks | ||
| 232 | + | ||
| 233 | + | ||
| 234 | +def _callee_name(call): | ||
| 235 | + """Dotted name of a call's callee, e.g. `nn.ModuleList` -> 'nn.ModuleList'.""" | ||
| 236 | + func = call.func | ||
| 237 | + parts = [] | ||
| 238 | + while isinstance(func, ast.Attribute): | ||
| 239 | + parts.append(func.attr) | ||
| 240 | + func = func.value | ||
| 241 | + if isinstance(func, ast.Name): | ||
| 242 | + parts.append(func.id) | ||
| 243 | + return '.'.join(reversed(parts)) if parts else UNKNOWN | ||
| 244 | + | ||
| 245 | + | ||
| 246 | +def _assignment_value(node, target_attr): | ||
| 247 | + if not isinstance(node, ast.Assign): | ||
| 248 | + return None | ||
| 249 | + if not any(isinstance(target, ast.Attribute) and target.attr == target_attr | ||
| 250 | + for target in node.targets): | ||
| 251 | + return None | ||
| 252 | + try: | ||
| 253 | + source = ast.unparse(node.value) | ||
| 254 | + except Exception: | ||
| 255 | + source = None | ||
| 256 | + return source, node.lineno | ||
| 257 | + | ||
| 258 | + | ||
| 259 | +def find_predicate_assignment(tree, target_attr): | ||
| 260 | + """ | ||
| 261 | + Find `self.<target_attr> = <expr>` and return (source_text_of_expr, lineno) if the | ||
| 262 | + expression references known predicate config keys. Returns (None, None) if absent. | ||
| 263 | + """ | ||
| 264 | + for node in ast.walk(tree): | ||
| 265 | + assignment = _assignment_value(node, target_attr) | ||
| 266 | + if assignment is not None: | ||
| 267 | + return assignment | ||
| 268 | + return None, None | ||
| 269 | + | ||
| 270 | + | ||
| 271 | +def _scan_source_evidence(source, evidence): | ||
| 272 | + if not source: | ||
| 273 | + return | ||
| 274 | + try: | ||
| 275 | + tree = ast.parse(source) | ||
| 276 | + except SyntaxError: | ||
| 277 | + return | ||
| 278 | + for node in ast.walk(tree): | ||
| 279 | + if isinstance(node, ast.ClassDef): | ||
| 280 | + evidence['class_names'].add(node.name) | ||
| 281 | + if isinstance(node, ast.FunctionDef) and node.name == '__init__': | ||
| 282 | + evidence['config_keys'].update(arg.arg for arg in node.args.args) | ||
| 283 | + evidence['config_keys'].update(arg.arg for arg in node.args.kwonlyargs) | ||
| 284 | + | ||
| 285 | + | ||
| 286 | +def scan_evidence(config_src, modeling_src): | ||
| 287 | + """Cheap evidence scan (class names + config keys) for adapter selection.""" | ||
| 288 | + ev = {'class_names': set(), 'config_keys': set()} | ||
| 289 | + for source in (config_src, modeling_src): | ||
| 290 | + _scan_source_evidence(source, ev) | ||
| 291 | + return ev | ||
| 292 | + | ||
| 293 | + | ||
| 294 | +def _assignment_components(statement, modeling_path, base_dir): | ||
| 295 | + if not isinstance(statement, ast.Assign) or not isinstance(statement.value, ast.Call): | ||
| 296 | + return [] | ||
| 297 | + components = [] | ||
| 298 | + for target in statement.targets: | ||
| 299 | + is_self_attr = (isinstance(target, ast.Attribute) | ||
| 300 | + and isinstance(target.value, ast.Name) | ||
| 301 | + and target.value.id == 'self') | ||
| 302 | + if not is_self_attr: | ||
| 303 | + continue | ||
| 304 | + components.append({ | ||
| 305 | + 'attr': target.attr, | ||
| 306 | + 'constructor': _callee_name(statement.value), | ||
| 307 | + 'source_ref': rel_ref(modeling_path or UNKNOWN, statement.lineno, base_dir), | ||
| 308 | + }) | ||
| 309 | + return components | ||
| 310 | + | ||
| 311 | + | ||
| 312 | +def _class_components(node, modeling_path, base_dir): | ||
| 313 | + init_fn = next((item for item in node.body | ||
| 314 | + if isinstance(item, ast.FunctionDef) and item.name == '__init__'), None) | ||
| 315 | + if init_fn is None: | ||
| 316 | + return None | ||
| 317 | + components = [] | ||
| 318 | + for statement in ast.walk(init_fn): | ||
| 319 | + components.extend(_assignment_components(statement, modeling_path, base_dir)) | ||
| 320 | + return { | ||
| 321 | + 'class_name': node.name, | ||
| 322 | + 'source_ref': rel_ref(modeling_path or UNKNOWN, node.lineno, base_dir), | ||
| 323 | + 'components': components, | ||
| 324 | + } | ||
| 325 | + | ||
| 326 | + | ||
| 327 | +def _role_hint_candidate(lowered, role, hints): | ||
| 328 | + best = None | ||
| 329 | + for hint in hints: | ||
| 330 | + index = lowered.rfind(hint) | ||
| 331 | + if index < 0: | ||
| 332 | + continue | ||
| 333 | + candidate = (index + len(hint), len(hint), role, hint) | ||
| 334 | + if best is None or candidate[:2] > best[:2]: | ||
| 335 | + best = candidate | ||
| 336 | + return best | ||
| 337 | + | ||
| 338 | + | ||
| 339 | +def _best_role_hint(attribute, role_hints): | ||
| 340 | + best = None | ||
| 341 | + lowered = attribute.lower() | ||
| 342 | + for role, hints in (role_hints or {}).items(): | ||
| 343 | + candidate = _role_hint_candidate(lowered, role, hints) | ||
| 344 | + if candidate is not None and (best is None or candidate[:2] > best[:2]): | ||
| 345 | + best = candidate | ||
| 346 | + return None if best is None else (best[2], best[3]) | ||
| 347 | + | ||
| 348 | + | ||
| 349 | +def _first_integer_default(context, keys): | ||
| 350 | + for key in keys: | ||
| 351 | + if key not in context.config_defaults: | ||
| 352 | + continue | ||
| 353 | + value, lineno = context.config_defaults[key] | ||
| 354 | + if isinstance(value, int): | ||
| 355 | + return key, value, lineno | ||
| 356 | + return None | ||
| 357 | + | ||
| 358 | + | ||
| 359 | +def _append_default_fact(state, found, confidence='high'): | ||
| 360 | + if found is None: | ||
| 361 | + return | ||
| 362 | + key, value, lineno = found | ||
| 363 | + state.facts.append(Fact(key, value, state.context.ref(lineno), | ||
| 364 | + 'ast_default_arg', confidence)) | ||
| 365 | + | ||
| 366 | + | ||
| 367 | +def _extract_layer_counts(adapter, state): | ||
| 368 | + main = _first_integer_default(state.context, adapter.main_layer_count_keys) | ||
| 369 | + if main is None: | ||
| 370 | + state.gaps.append('num_main_layers: 无法从 config 默认参数静态解析') | ||
| 371 | + else: | ||
| 372 | + _key, state.num_main, _line = main | ||
| 373 | + _append_default_fact(state, main) | ||
| 374 | + | ||
| 375 | + prediction = _first_integer_default(state.context, adapter.prediction_count_keys) | ||
| 376 | + if prediction is not None: | ||
| 377 | + state.pred_key, state.pred_count, _line = prediction | ||
| 378 | + _append_default_fact(state, prediction) | ||
| 379 | + | ||
| 380 | + dense = _first_integer_default(state.context, adapter.dense_boundary_keys) | ||
| 381 | + if dense is not None: | ||
| 382 | + _key, state.first_k_dense, _line = dense | ||
| 383 | + _append_default_fact(state, dense) | ||
| 384 | + | ||
| 385 | + | ||
| 386 | +def _extract_moe_presence(adapter, state): | ||
| 387 | + defaults = state.context.config_defaults | ||
| 388 | + for key in adapter.moe_expert_keys: | ||
| 389 | + if key not in defaults: | ||
| 390 | + continue | ||
| 391 | + value, lineno = defaults[key] | ||
| 392 | + if isinstance(value, int) and value > 0: | ||
| 393 | + state.has_moe = True | ||
| 394 | + _append_default_fact(state, (key, value, lineno)) | ||
| 395 | + return | ||
| 396 | + if value is None or value is False: | ||
| 397 | + state.moe_key_unbound = True | ||
| 398 | + _append_default_fact(state, (key, value, lineno), 'low') | ||
| 399 | + state.gaps.append( | ||
| 400 | + f'{key}: MoE 配置键存在但默认值为 {value}，无法静态判定是否启用 MoE。' | ||
| 401 | + f'该层组分类降为 low confidence；需由 checkpoint config.json 或 ' | ||
| 402 | + f'trace 算子（MoeGatingTopK* / GroupedMatmul）判定') | ||
| 403 | + return | ||
| 404 | + | ||
| 405 | + | ||
| 406 | +def _append_first_positive_fact(adapter, state, keys): | ||
| 407 | + defaults = state.context.config_defaults | ||
| 408 | + for key in keys: | ||
| 409 | + if key not in defaults: | ||
| 410 | + continue | ||
| 411 | + value, lineno = defaults[key] | ||
| 412 | + if isinstance(value, int) and value > 0: | ||
| 413 | + _append_default_fact(state, (key, value, lineno)) | ||
| 414 | + return | ||
| 415 | + | ||
| 416 | + | ||
| 417 | +def _extract_moe_dimensions(adapter, state): | ||
| 418 | + if not state.has_moe: | ||
| 419 | + return | ||
| 420 | + _append_first_positive_fact(adapter, state, adapter.shared_expert_keys) | ||
| 421 | + _append_first_positive_fact(adapter, state, adapter.experts_per_token_keys) | ||
| 422 | + | ||
| 423 | + | ||
| 424 | +def _extract_layer_construction(adapter, state): | ||
| 425 | + context = state.context | ||
| 426 | + layer_classes = (find_decoder_layer_classes(context.modeling_tree) | ||
| 427 | + if context.modeling_tree else []) | ||
| 428 | + range_calls = (find_range_calls_over_config( | ||
| 429 | + context.modeling_tree, set(adapter.main_layer_count_keys)) | ||
| 430 | + if context.modeling_tree else []) | ||
| 431 | + state.base_class_name = layer_classes[0][0] if layer_classes else 'DecoderLayer' | ||
| 432 | + if not range_calls: | ||
| 433 | + state.gaps.append('main decoder ModuleList range(...) 构造未静态定位') | ||
| 434 | + return | ||
| 435 | + state.construct_ref = rel_ref(context.modeling_path, range_calls[0]['lineno'], | ||
| 436 | + context.base_dir) | ||
| 437 | + state.facts.append(Fact('main_layer_construction', | ||
| 438 | + f"range({range_calls[0]['attr']})", state.construct_ref, | ||
| 439 | + 'ast_range_expr', 'high')) | ||
| 440 | + | ||
| 441 | + | ||
| 442 | +class BaseAdapter: | ||
| 443 | + """Generic architecture extractor. Family adapters subclass and override hooks. | ||
| 444 | + | ||
| 445 | + Every family-specific constant belongs here rather than in a validator or a reference | ||
| 446 | + doc. A checker that knows a model family's module names cannot abstain for a family it | ||
| 447 | + has never seen, so the family knowledge is confined to these subclasses and the rest of | ||
| 448 | + the pipeline compares graphs. | ||
| 449 | + | ||
| 450 | + The class attributes below default to EMPTY on purpose in the generic base: an unknown | ||
| 451 | + family has no known config keys, and inheriting DeepSeek's names would make a Qwen model | ||
| 452 | + silently read as having a dense/MoE boundary it does not have. `main_layer_count_keys` | ||
| 453 | + is the one exception -- those four spellings cover essentially every HF-style config, and | ||
| 454 | + without any of them the generic adapter could not extract a layer count at all. | ||
| 455 | + """ | ||
| 456 | + | ||
| 457 | + name = 'generic' | ||
| 458 | + | ||
| 459 | + # config keys this family uses for the main layer count (first match wins) | ||
| 460 | + main_layer_count_keys = ('num_hidden_layers', 'num_layers', 'n_layer', 'num_layer') | ||
| 461 | + # config keys for prediction/MTP module counts. Empty in the base: a model with no | ||
| 462 | + # prediction module must not be searched for one. | ||
| 463 | + prediction_count_keys = () | ||
| 464 | + # dense/moe boundary predicate keys. Empty in the base for the same reason. | ||
| 465 | + dense_boundary_keys = () | ||
| 466 | + moe_expert_keys = () | ||
| 467 | + # Shared/always-on experts, which expert parallelism does not shard: every rank holds a | ||
| 468 | + # copy. Empty in the base for the same reason as above -- inheriting DeepSeek's | ||
| 469 | + # `n_shared_experts` would make a family without shared experts appear to have one. | ||
| 470 | + shared_expert_keys = () | ||
| 471 | + # Experts activated per token (top-k routing). Empty in the base likewise. | ||
| 472 | + experts_per_token_keys = () | ||
| 473 | + | ||
| 474 | + #: Capabilities this family is known to have, asserted only when the evidence scan | ||
| 475 | + #: confirms them. Validators gate family-specific rules on these, so a Dense model is | ||
| 476 | + #: never asked to have MoE branches. See `capabilities()`. | ||
| 477 | + capability_keys = {} | ||
| 478 | + | ||
| 479 | + #: capability_id -> class-name substrings that evidence it. A second evidence channel | ||
| 480 | + #: for capabilities that live in a module rather than a config value: DeepSeek V3.2's | ||
| 481 | + #: sparse indexer is declared by `DeepseekIndexerAttention` existing, while its | ||
| 482 | + #: `index_topk` sits in a separate `DeepseekV3IndexConfig` the main config never | ||
| 483 | + #: inherits -- so a config-value-only rule would miss it. Class names are matched | ||
| 484 | + #: against the modeling source, which is the module actually being built. | ||
| 485 | + capability_class_hints = {} | ||
| 486 | + | ||
| 487 | + #: Known kernel-name aliases per semantic role, as CANDIDATE HINTS for op mapping -- | ||
| 488 | + #: never as a required table. The same semantic has different kernel names across | ||
| 489 | + #: backends, op-library versions and quantisation modes, so a fixed list would fail the | ||
| 490 | + #: next version of the same model. | ||
| 491 | + kernel_anchors = {} | ||
| 492 | + | ||
| 493 | + #: Dataflow shapes this family must exhibit, checked against the AST-derived graph. | ||
| 494 | + #: Each entry: {'id', 'requires', 'reason'}. `requires` names the capability that gates | ||
| 495 | + #: the rule (None = ungated). These are DECLARATIVE only -- an adapter states the | ||
| 496 | + #: constraint and the checker evaluates it; an adapter never edits report output. | ||
| 497 | + #: Empty means "assert nothing". | ||
| 498 | + dataflow_invariants = () | ||
| 499 | + | ||
| 500 | + #: Constructs this family is known to defeat static extraction on, as a list of | ||
| 501 | + #: {'id', 'reason'}. Recorded so the gap is visible in the manifest rather than | ||
| 502 | + #: surfacing later as an unexplained checker abstention. | ||
| 503 | + known_deviations = () | ||
| 504 | + | ||
| 505 | + | ||
| 506 | + def matches(evidence): | ||
| 507 | + """Return False, or a (confidence, reasons) pair. | ||
| 508 | + | ||
| 509 | + Returning a bare bool is still supported for backwards compatibility and is read as | ||
| 510 | + `('medium', [])`. Confidence matters because selection by list order silently | ||
| 511 | + resolves ambiguity: two families whose signatures both fire is a fact about the | ||
| 512 | + source that the caller must see, not something to settle by whoever is listed first. | ||
| 513 | + """ | ||
| 514 | + return False | ||
| 515 | + | ||
| 516 | + #: Semantic role -> submodule-attribute-name substrings that indicate it. Used by | ||
| 517 | + #: `infer_roles` to propose a `semantic` for a node. These are CANDIDATES for a human or | ||
| 518 | + #: AI mapper to confirm against the source, never an authority: a name is a hint about | ||
| 519 | + #: intent, and a model is free to call its router `gate` or its MLP `feed_forward`. | ||
| 520 | + role_hints = { | ||
| 521 | + 'attention': ('self_attn', 'attention', 'attn'), | ||
| 522 | + 'mlp': ('mlp', 'feed_forward', 'ffn'), | ||
| 523 | + 'router_gating': ('gate', 'router'), | ||
| 524 | + 'experts': ('experts', 'expert'), | ||
| 525 | + 'shared_expert': ('shared_expert', 'shared_experts'), | ||
| 526 | + 'moe': ('moe_block', 'sparse_moe', 'moe'), | ||
| 527 | + 'norm': ('layernorm', 'layer_norm', 'rmsnorm', 'norm', 'ln_'), | ||
| 528 | + 'embedding': ('embed_tokens', 'embedding', 'wte'), | ||
| 529 | + } | ||
| 530 | + | ||
| 531 | + | ||
| 532 | + def extract_components(modeling_tree, config_defaults, base_dir=None, | ||
| 533 | + modeling_path=None): | ||
| 534 | + """Candidate component definitions: the submodules each layer class declares. | ||
| 535 | + | ||
| 536 | + Returns a list of {'class_name', 'source_ref', 'components': [{'attr', 'count', | ||
| 537 | + 'source_ref'}]}. This is a proposal derived from `__init__` assignments, not a | ||
| 538 | + decomposition -- containment and dataflow are decided against `forward()` (see | ||
| 539 | + extract_dataflow.py), because `__init__` order says nothing about execution order | ||
| 540 | + and a declared submodule may never be called. | ||
| 541 | + """ | ||
| 542 | + if modeling_tree is None: | ||
| 543 | + return [] | ||
| 544 | + wanted = {name for name, _line in find_decoder_layer_classes(modeling_tree)} | ||
| 545 | + out = [] | ||
| 546 | + for node in ast.walk(modeling_tree): | ||
| 547 | + if not isinstance(node, ast.ClassDef) or node.name not in wanted: | ||
| 548 | + continue | ||
| 549 | + component_set = _class_components(node, modeling_path, base_dir) | ||
| 550 | + if component_set is not None: | ||
| 551 | + out.append(component_set) | ||
| 552 | + return out | ||
| 553 | + | ||
| 554 | + | ||
| 555 | + def build_layer_groups(state): | ||
| 556 | + num_main = state.num_main | ||
| 557 | + first_k_dense = state.first_k_dense | ||
| 558 | + has_moe = state.has_moe | ||
| 559 | + base_class_name = state.base_class_name | ||
| 560 | + context = state.context | ||
| 561 | + if num_main == UNKNOWN: | ||
| 562 | + return [] | ||
| 563 | + # dense/moe split | ||
| 564 | + if has_moe and first_k_dense is not None: | ||
| 565 | + pred_src, pred_line = (None, None) | ||
| 566 | + if context.modeling_tree is not None: | ||
| 567 | + pred_src, pred_line = find_predicate_assignment(context.modeling_tree, 'is_moe') | ||
| 568 | + groups = [] | ||
| 569 | + if first_k_dense > 0: | ||
| 570 | + groups.append({ | ||
| 571 | + 'type': f'{base_class_name}_dense', | ||
| 572 | + 'classification': 'dense', | ||
| 573 | + 'model_layer_indices': list(range(0, first_k_dense)), | ||
| 574 | + 'predicate': f'layer_idx < first_k_dense_replace ({first_k_dense})', | ||
| 575 | + 'source_ref': context.ref(pred_line, context.modeling_path) if pred_line else UNKNOWN, | ||
| 576 | + 'confidence': 'high' if pred_line else 'medium', | ||
| 577 | + }) | ||
| 578 | + groups.append({ | ||
| 579 | + 'type': f'{base_class_name}_moe', | ||
| 580 | + 'classification': 'moe', | ||
| 581 | + 'model_layer_range': [first_k_dense, num_main - 1], | ||
| 582 | + 'predicate': f'layer_idx >= first_k_dense_replace ({first_k_dense})', | ||
| 583 | + 'source_ref': context.ref(pred_line, context.modeling_path) if pred_line else UNKNOWN, | ||
| 584 | + 'confidence': 'high' if pred_line else 'medium', | ||
| 585 | + }) | ||
| 586 | + return groups | ||
| 587 | + # all-moe | ||
| 588 | + if has_moe and first_k_dense is None: | ||
| 589 | + return [{ | ||
| 590 | + 'type': f'{base_class_name}_moe', | ||
| 591 | + 'classification': 'moe', | ||
| 592 | + 'model_layer_range': [0, num_main - 1], | ||
| 593 | + 'predicate': 'all layers MoE (no dense boundary key found)', | ||
| 594 | + 'source_ref': state.construct_ref or UNKNOWN, | ||
| 595 | + 'confidence': 'medium', | ||
| 596 | + }] | ||
| 597 | + # dense-only: the group spans every layer, so the ModuleList construction site is | ||
| 598 | + # exactly the evidence for it — no per-layer predicate needs locating. | ||
| 599 | + # | ||
| 600 | + # `construct_ref` proves how many layers there are; it says nothing about whether | ||
| 601 | + # they are dense. When an MoE key was found but carried no bound value, this branch | ||
| 602 | + # is a fallback rather than a finding, so it must not inherit high confidence -- | ||
| 603 | + # otherwise A3 enforces `dense` against trace evidence that says otherwise. | ||
| 604 | + confidence = 'high' if state.construct_ref else 'medium' | ||
| 605 | + if state.moe_key_unbound: | ||
| 606 | + confidence = 'low' | ||
| 607 | + return [{ | ||
| 608 | + 'type': base_class_name, | ||
| 609 | + 'classification': 'dense', | ||
| 610 | + 'model_layer_range': [0, num_main - 1], | ||
| 611 | + 'predicate': None, | ||
| 612 | + 'source_ref': state.construct_ref or UNKNOWN, | ||
| 613 | + 'confidence': confidence, | ||
| 614 | + }] | ||
| 615 | + | ||
| 616 | + | ||
| 617 | + def build_prediction_modules(state): | ||
| 618 | + num_main = state.num_main | ||
| 619 | + pred_count = state.pred_count | ||
| 620 | + pred_key = state.pred_key | ||
| 621 | + base_class_name = state.base_class_name | ||
| 622 | + context = state.context | ||
| 623 | + if not pred_count: | ||
| 624 | + return [] | ||
| 625 | + if num_main == UNKNOWN: | ||
| 626 | + state.gaps.append('prediction module 层号无法确定（num_main_layers unknown）') | ||
| 627 | + return [{ | ||
| 628 | + 'type': f'{base_class_name}_mtp', | ||
| 629 | + 'learned_module_count': pred_count, | ||
| 630 | + 'model_layer_indices': [], | ||
| 631 | + 'count_config_key': pred_key, | ||
| 632 | + 'source_ref': UNKNOWN, | ||
| 633 | + 'confidence': 'low', | ||
| 634 | + }] | ||
| 635 | + # MTP layers are appended after the main layers: indices [num_main .. num_main+pred_count-1] | ||
| 636 | + indices = list(range(num_main, num_main + pred_count)) | ||
| 637 | + # locate the MTP construction (ModuleDict keyed str(mtp_start + i)) | ||
| 638 | + mtp_ref = UNKNOWN | ||
| 639 | + if context.modeling_tree is not None: | ||
| 640 | + for node in ast.walk(context.modeling_tree): | ||
| 641 | + if isinstance(node, ast.ClassDef) and 'MTP' in node.name.upper(): | ||
| 642 | + mtp_ref = context.ref(node.lineno, context.modeling_path) | ||
| 643 | + break | ||
| 644 | + return [{ | ||
| 645 | + 'type': f'{base_class_name}_mtp', | ||
| 646 | + 'learned_module_count': pred_count, | ||
| 647 | + 'model_layer_indices': indices, | ||
| 648 | + 'count_config_key': pred_key, | ||
| 649 | + 'source_ref': mtp_ref, | ||
| 650 | + 'confidence': 'high' if mtp_ref != UNKNOWN else 'medium', | ||
| 651 | + }] | ||
| 652 | + | ||
| 653 | + | ||
| 654 | + def refine(state): | ||
| 655 | + """Hook for family-specific post-processing. Default: no-op.""" | ||
| 656 | + return | ||
| 657 | + | ||
| 658 | + def deviations(self): | ||
| 659 | + """Known unsupported/dynamic constructs for this family. Declarative, not fatal.""" | ||
| 660 | + return [dict(item) for item in (self.known_deviations or ())] | ||
| 661 | + | ||
| 662 | + def capabilities(self, evidence, config_defaults): | ||
| 663 | + """Capabilities this source actually evidences, as a list of dicts. | ||
| 664 | + | ||
| 665 | + Evidence is a key RESOLVED TO A TRUTHY VALUE, not merely a key that exists. Gemma 4 | ||
| 666 | + declares `num_experts=None, enable_moe_block=False` in its config `__init__`, so | ||
| 667 | + name-presence alone would assert `moe` for a deployment that has the expert block | ||
| 668 | + switched off -- and asserting a capability turns on the validation rules that | ||
| 669 | + require the matching fork/join (check_dataflow D7). A key resolving to | ||
| 670 | + None/False/0 is the source saying the feature is off; a key absent entirely is | ||
| 671 | + unknown. Neither is a reason to claim it. | ||
| 672 | + | ||
| 673 | + `config_defaults` is the authority because `extract_model_manifest` has already | ||
| 674 | + merged the checkpoint config.json over the Python defaults, so a deployment that | ||
| 675 | + does enable experts is read from its deployed values. | ||
| 676 | + """ | ||
| 677 | + found = [] | ||
| 678 | + defaults = config_defaults or {} | ||
| 679 | + for capability_id, required in (self.capability_keys or {}).items(): | ||
| 680 | + hits = [] | ||
| 681 | + for key in required: | ||
| 682 | + if key not in defaults: | ||
| 683 | + continue | ||
| 684 | + value = defaults[key][0] if isinstance(defaults[key], tuple) else defaults[key] | ||
| 685 | + if value: | ||
| 686 | + hits.append(key) | ||
| 687 | + if hits: | ||
| 688 | + line = defaults[hits[0]][1] if isinstance(defaults[hits[0]], tuple) else 1 | ||
| 689 | + found.append({'id': capability_id, 'evidence_keys': sorted(hits), | ||
| 690 | + 'source_ref': f'config:{hits[0]}:{line}'}) | ||
| 691 | + | ||
| 692 | + seen = {item['id'] for item in found} | ||
| 693 | + classes = set(evidence.get('class_names') or ()) | ||
| 694 | + for capability_id, hints in (self.capability_class_hints or {}).items(): | ||
| 695 | + if capability_id in seen: | ||
| 696 | + continue | ||
| 697 | + matched = sorted(name for name in classes | ||
| 698 | + if any(hint in name for hint in hints)) | ||
| 699 | + if matched: | ||
| 700 | + found.append({'id': capability_id, 'evidence_keys': matched[:4], | ||
| 701 | + 'source_ref': f'class:{matched[0]}'}) | ||
| 702 | + return found | ||
| 703 | + | ||
| 704 | + def infer_roles(self, modeling_tree, base_dir=None, modeling_path=None): | ||
| 705 | + """Propose semantic roles for declared submodules, each with a source_ref. | ||
| 706 | + | ||
| 707 | + Only names that match a `role_hints` entry are proposed, and every proposal carries | ||
| 708 | + `confidence: 'low'` plus the hint that fired. A name is evidence of intent, not of | ||
| 709 | + behaviour -- what a module does is decided by its `forward()`. Anything unmatched is | ||
| 710 | + returned under 'unresolved' rather than assigned a default role, so a mapper sees | ||
| 711 | + what still needs deciding instead of inheriting a guess. | ||
| 712 | + """ | ||
| 713 | + roles, unresolved = [], [] | ||
| 714 | + for component_set in self.extract_components(modeling_tree, {}, base_dir, | ||
| 715 | + modeling_path): | ||
| 716 | + for component in component_set['components']: | ||
| 717 | + attr = component['attr'] | ||
| 718 | + # The rightmost match is the compound name's head noun; length breaks ties. | ||
| 719 | + best = _best_role_hint(attr, self.role_hints) | ||
| 720 | + entry = {'class_name': component_set['class_name'], 'attr': attr, | ||
| 721 | + 'source_ref': component['source_ref']} | ||
| 722 | + if best: | ||
| 723 | + roles.append(dict(entry, role=best[0], confidence='low', | ||
| 724 | + basis=f'attr 名含 `{best[1]}` → role_hints[{best[0]}]')) | ||
| 725 | + else: | ||
| 726 | + unresolved.append(entry) | ||
| 727 | + return {'roles': roles, 'unresolved': unresolved} | ||
| 728 | + | ||
| 729 | + # ---- extraction API ------------------------------------------------- | ||
| 730 | + | ||
| 731 | + def extract(self, context): | ||
| 732 | + """ | ||
| 733 | + Return (facts:list[Fact], layer_groups:list[dict], prediction_modules:list[dict], | ||
| 734 | + num_main_layers, evidence_gaps:list[str]). | ||
| 735 | + Generic implementation; adapters may override or post-process via `refine`. | ||
| 736 | + """ | ||
| 737 | + state = ExtractionState(context) | ||
| 738 | + _extract_layer_counts(self, state) | ||
| 739 | + _extract_moe_presence(self, state) | ||
| 740 | + _extract_moe_dimensions(self, state) | ||
| 741 | + _extract_layer_construction(self, state) | ||
| 742 | + state.layer_groups = self.build_layer_groups(state) | ||
| 743 | + state.prediction_modules = self.build_prediction_modules(state) | ||
| 744 | + self.refine(state) | ||
| 745 | + return (state.facts, state.layer_groups, state.prediction_modules, | ||
| 746 | + state.num_main, state.gaps) | ||
| 747 | + | ||
| 748 | + | ||
| 749 | +GENERIC = BaseAdapter() | ||
| @@ -0,0 +1,94 @@ | |||
| 1 | +# Copyright (c) 2026 Huawei Technologies Co., Ltd. | ||
| 2 | +# Licensed under the Apache License, Version 2.0 (the "License"); | ||
| 3 | +# you may not use this file except in compliance with the License. | ||
| 4 | +# You may obtain a copy of the License at | ||
| 5 | +# | ||
| 6 | +# http://www.apache.org/licenses/LICENSE-2.0 | ||
| 7 | +# | ||
| 8 | +# Unless required by applicable law or agreed to in writing, software | ||
| 9 | +# distributed under the License is distributed on an "AS IS" BASIS, | ||
| 10 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| 11 | +# See the License for the specific language governing permissions and | ||
| 12 | +# limitations under the License. | ||
| 13 | +# | ||
| 14 | +"""DeepSeek V3 / V3.2 adapter. | ||
| 15 | + | ||
| 16 | +Facts still come from the model's own source via AST — this adapter only knows | ||
| 17 | +*where* to look (config key names, is_moe predicate, MTP ModuleDict shape). | ||
| 18 | +""" | ||
| 19 | +from .base import BaseAdapter | ||
| 20 | + | ||
| 21 | + | ||
| 22 | +class DeepseekAdapter(BaseAdapter): | ||
| 23 | + name = 'deepseek' | ||
| 24 | + main_layer_count_keys = ('num_hidden_layers',) | ||
| 25 | + prediction_count_keys = ('num_nextn_predict_layers',) | ||
| 26 | + dense_boundary_keys = ('first_k_dense_replace',) | ||
| 27 | + moe_expert_keys = ('n_routed_experts',) | ||
| 28 | + shared_expert_keys = ('n_shared_experts',) | ||
| 29 | + experts_per_token_keys = ('num_experts_per_tok',) | ||
| 30 | + | ||
| 31 | + #: A capability is asserted only when one of its keys is actually present in this | ||
| 32 | + #: source. `n_shared_experts` is what makes the shared-expert parallel path real; the | ||
| 33 | + #: V3.2 sparse indexer shows up as index_topk/index_n_heads. | ||
| 34 | + capability_keys = { | ||
| 35 | + 'moe': ('n_routed_experts', 'first_k_dense_replace'), | ||
| 36 | + 'shared_expert': ('n_shared_experts',), | ||
| 37 | + 'mtp': ('num_nextn_predict_layers',), | ||
| 38 | + 'mla': ('kv_lora_rank', 'q_lora_rank'), | ||
| 39 | + 'sparse_index_attention': ('index_topk', 'index_n_heads'), | ||
| 40 | + } | ||
| 41 | + | ||
| 42 | + #: V3.2's `index_topk`/`index_n_heads` live in a separate `DeepseekV3IndexConfig` that | ||
| 43 | + #: the main `DeepseekV3Config` does not inherit, so the config-value channel above never | ||
| 44 | + #: sees them. The indexer module existing in the modeling source is the evidence. | ||
| 45 | + capability_class_hints = { | ||
| 46 | + 'sparse_index_attention': ('IndexerAttention', 'Indexer'), | ||
| 47 | + } | ||
| 48 | + | ||
| 49 | + #: CANDIDATE HINTS for op mapping, never a required table: the same semantic carries | ||
| 50 | + #: different kernel names across op-library versions and quantisation modes. The | ||
| 51 | + #: mapper's anchor must still come from this model's own op sequence. | ||
| 52 | + kernel_anchors = { | ||
| 53 | + 'attention_core': ('FlashAttentionScore', 'PromptFlashAttention', | ||
| 54 | + 'IncreFlashAttention', 'MLAProlog'), | ||
| 55 | + 'router_gating': ('MoeGatingTopKSoftmax', 'TopKSoftmax'), | ||
| 56 | + 'expert_dispatch': ('MoeInitRouting', 'MoeDistributeDispatch'), | ||
| 57 | + 'expert_combine': ('MoeFinalizeRouting', 'MoeDistributeCombine'), | ||
| 58 | + 'grouped_matmul': ('GroupedMatmul', 'GroupedMatMul'), | ||
| 59 | + 'fused_add_norm': ('AddRmsNorm', 'AddRMSNorm'), | ||
| 60 | + } | ||
| 61 | + | ||
| 62 | + #: Dataflow shapes the family must exhibit, gated on the capability being evidenced. | ||
| 63 | + #: These are checked against the AST-derived graph, not asserted from the name. | ||
| 64 | + dataflow_invariants = ( | ||
| 65 | + {'id': 'shared_expert_parallel', | ||
| 66 | + 'requires': 'shared_expert', | ||
| 67 | + 'reason': '共享专家与路由专家读同一份 MoE 输入，在 combine 点汇合；' | ||
| 68 | + '写成串行链会把并行支路的耗时算进主链'}, | ||
| 69 | + ) | ||
| 70 | + | ||
| 71 | + known_deviations = ( | ||
| 72 | + {'id': 'quantisation_gated_arms', | ||
| 73 | + 'reason': 'w8a8/量化分支由部署配置在运行前固定，属 config-gated：' | ||
| 74 | + '应作为 variants 提取并绑定 execution profile，而非 data-dependent'}, | ||
| 75 | + {'id': 'index_config_not_inherited', | ||
| 76 | + 'reason': 'index_topk/index_n_heads 定义在 DeepseekV3IndexConfig，' | ||
| 77 | + '主 config 不继承；sparse indexer 由 modeling 源码的类存在性判定'}, | ||
| 78 | + ) | ||
| 79 | + | ||
| 80 | + def matches(self, evidence): | ||
| 81 | + classes = evidence.get('class_names', set()) | ||
| 82 | + keys = evidence.get('config_keys', set()) | ||
| 83 | + reasons = [] | ||
| 84 | + named = sorted(c for c in classes if 'Deepseek' in c or 'DeepSeek' in c) | ||
| 85 | + if named: | ||
| 86 | + reasons.append(f'class 名含 Deepseek: {named[:3]}') | ||
| 87 | + signature = {'first_k_dense_replace', 'num_nextn_predict_layers'} | ||
| 88 | + if signature <= keys: | ||
| 89 | + reasons.append('config 同时含 first_k_dense_replace 与 num_nextn_predict_layers') | ||
| 90 | + if not reasons: | ||
| 91 | + return False | ||
| 92 | + # A class name is the family declaring itself; a key signature only says the config | ||
| 93 | + # is shaped like this family's, which a derivative could also satisfy. | ||
| 94 | + return ('high' if named else 'medium'), reasons | ||
| @@ -0,0 +1,91 @@ | |||
| 1 | +# Copyright (c) 2026 Huawei Technologies Co., Ltd. | ||
| 2 | +# Licensed under the Apache License, Version 2.0 (the "License"); | ||
| 3 | +# you may not use this file except in compliance with the License. | ||
| 4 | +# You may obtain a copy of the License at | ||
| 5 | +# | ||
| 6 | +# http://www.apache.org/licenses/LICENSE-2.0 | ||
| 7 | +# | ||
| 8 | +# Unless required by applicable law or agreed to in writing, software | ||
| 9 | +# distributed under the License is distributed on an "AS IS" BASIS, | ||
| 10 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| 11 | +# See the License for the specific language governing permissions and | ||
| 12 | +# limitations under the License. | ||
| 13 | +# | ||
| 14 | +"""Gemma 4 adapter. | ||
| 15 | + | ||
| 16 | +No MTP. Sliding/full attention follows a `layer_types` pattern built in the config | ||
| 17 | +__init__; classifying individual layers requires evaluating that runtime-built list, so | ||
| 18 | +when it cannot be resolved to literal indices the per-layer sliding/full split is left as | ||
| 19 | +an evidence gap rather than guessed. | ||
| 20 | + | ||
| 21 | +Gemma 4 is NOT unconditionally dense. The 26B-A4B variant carries a | ||
| 22 | +`Gemma4SparseMoeBlock` in every decoder layer, gated by `enable_moe_block`, alongside the | ||
| 23 | +dense `Gemma4MLP` -- the two coexist per layer rather than splitting the stack into dense | ||
| 24 | +and MoE ranges, which is why `dense_boundary_keys` is still empty while `moe_expert_keys` | ||
| 25 | +is not. Declaring the family dense would drop the expert path from every layer of a | ||
| 26 | +deployment that has the flag on. | ||
| 27 | +""" | ||
| 28 | +import ast | ||
| 29 | +from .base import BaseAdapter, Fact | ||
| 30 | + | ||
| 31 | + | ||
| 32 | +class GemmaAdapter(BaseAdapter): | ||
| 33 | + name = 'gemma' | ||
| 34 | + main_layer_count_keys = ('num_hidden_layers',) | ||
| 35 | + prediction_count_keys = () # no MTP | ||
| 36 | + #: Empty NOT because the family is dense, but because there is no dense/MoE *boundary*: | ||
| 37 | + #: the expert block sits in every layer next to the dense MLP (see module docstring). | ||
| 38 | + dense_boundary_keys = () | ||
| 39 | + moe_expert_keys = ('num_experts',) | ||
| 40 | + | ||
| 41 | + #: `enable_moe_block` defaults to False and `num_experts` to None, so on the Python | ||
| 42 | + #: defaults alone this asserts nothing — the capability appears only once a checkpoint | ||
| 43 | + #: config.json supplies real values. That is the intended behaviour: absent evidence | ||
| 44 | + #: is unknown, not "dense". | ||
| 45 | + capability_keys = { | ||
| 46 | + 'sliding_window_attention': ('sliding_window', 'layer_types'), | ||
| 47 | + 'moe': ('num_experts', 'enable_moe_block', 'moe_intermediate_size'), | ||
| 48 | + } | ||
| 49 | + | ||
| 50 | + #: Candidate hints only. | ||
| 51 | + kernel_anchors = { | ||
| 52 | + 'attention_core': ('FlashAttentionScore', 'PromptFlashAttention', | ||
| 53 | + 'IncreFlashAttention'), | ||
| 54 | + 'fused_add_norm': ('AddRmsNorm', 'AddRMSNorm'), | ||
| 55 | + } | ||
| 56 | + | ||
| 57 | + known_deviations = ( | ||
| 58 | + {'id': 'layer_types_runtime_built', | ||
| 59 | + 'reason': 'sliding/full 的逐层分类由 config.__init__ 在运行期构建列表，' | ||
| 60 | + 'AST 不展开为具体层号；保持 dense 单组而不猜测'}, | ||
| 61 | + {'id': 'enable_moe_block_gated', | ||
| 62 | + 'reason': 'MoE 块由 enable_moe_block 开关控制且与 dense MLP 同层共存；' | ||
| 63 | + '实际是否走专家路径取决于部署配置，不能从 Python 默认值推断'}, | ||
| 64 | + ) | ||
| 65 | + | ||
| 66 | + def matches(self, evidence): | ||
| 67 | + classes = evidence.get('class_names', set()) | ||
| 68 | + keys = evidence.get('config_keys', set()) | ||
| 69 | + reasons = [] | ||
| 70 | + named = sorted(c for c in classes if 'Gemma' in c) | ||
| 71 | + if named: | ||
| 72 | + reasons.append(f'class 名含 Gemma: {named[:3]}') | ||
| 73 | + if ('sliding_window' in keys and 'num_hidden_layers' in keys | ||
| 74 | + and 'first_k_dense_replace' not in keys): | ||
| 75 | + reasons.append('config 含 sliding_window + num_hidden_layers 且无 ' | ||
| 76 | + 'first_k_dense_replace') | ||
| 77 | + if not reasons: | ||
| 78 | + return False | ||
| 79 | + return ('high' if named else 'medium'), reasons | ||
| 80 | + | ||
| 81 | + def refine(self, state): | ||
| 82 | + context = state.context | ||
| 83 | + if 'sliding_window' in context.config_defaults: | ||
| 84 | + val, lineno = context.config_defaults['sliding_window'] | ||
| 85 | + state.facts.append(Fact('sliding_window', val, | ||
| 86 | + context.ref(lineno), 'ast_default_arg', 'high')) | ||
| 87 | + # The sliding/full per-layer split is built at runtime (list comprehension over | ||
| 88 | + # num_hidden_layers with a modulo pattern). We do not evaluate it statically to | ||
| 89 | + # concrete indices; record the gap so it is never silently guessed. | ||
| 90 | + state.gaps.append('Gemma sliding/full per-layer 分类由 config.__init__ 运行时构建，' | ||
| 91 | + '未静态展开为具体层号（保持 dense 单组）') | ||
| @@ -0,0 +1,79 @@ | |||
| 1 | +# Copyright (c) 2026 Huawei Technologies Co., Ltd. | ||
| 2 | +# Licensed under the Apache License, Version 2.0 (the "License"); | ||
| 3 | +# you may not use this file except in compliance with the License. | ||
| 4 | +# You may obtain a copy of the License at | ||
| 5 | +# | ||
| 6 | +# http://www.apache.org/licenses/LICENSE-2.0 | ||
| 7 | +# | ||
| 8 | +# Unless required by applicable law or agreed to in writing, software | ||
| 9 | +# distributed under the License is distributed on an "AS IS" BASIS, | ||
| 10 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| 11 | +# See the License for the specific language governing permissions and | ||
| 12 | +# limitations under the License. | ||
| 13 | +# | ||
| 14 | +"""LongCat Flash (Lite) adapter. | ||
| 15 | + | ||
| 16 | +LongCat uses `num_layers` (with a `num_hidden_layers` property alias) and is | ||
| 17 | +all-MoE (no first_k_dense boundary). It also carries num_nextn_predict_layers. | ||
| 18 | +""" | ||
| 19 | +from .base import BaseAdapter, Fact, rel_ref | ||
| 20 | + | ||
| 21 | + | ||
| 22 | +class LongcatAdapter(BaseAdapter): | ||
| 23 | + name = 'longcat' | ||
| 24 | + main_layer_count_keys = ('num_layers', 'num_hidden_layers') | ||
| 25 | + prediction_count_keys = ('num_nextn_predict_layers',) | ||
| 26 | + dense_boundary_keys = ('first_k_dense_replace',) # absent -> all-MoE | ||
| 27 | + moe_expert_keys = ('n_routed_experts',) | ||
| 28 | + | ||
| 29 | + capability_keys = { | ||
| 30 | + 'moe': ('n_routed_experts',), | ||
| 31 | + 'zero_expert': ('zero_expert_num', 'n_zero_experts'), | ||
| 32 | + 'mtp': ('num_nextn_predict_layers',), | ||
| 33 | + } | ||
| 34 | + | ||
| 35 | + #: Candidate hints only — resolved against this model's own op sequence. | ||
| 36 | + kernel_anchors = { | ||
| 37 | + 'attention_core': ('FlashAttentionScore', 'PromptFlashAttention', | ||
| 38 | + 'IncreFlashAttention'), | ||
| 39 | + 'router_gating': ('MoeGatingTopKSoftmax', 'TopKSoftmax'), | ||
| 40 | + 'expert_dispatch': ('MoeInitRouting', 'MoeDistributeDispatch'), | ||
| 41 | + 'expert_combine': ('MoeFinalizeRouting', 'MoeDistributeCombine'), | ||
| 42 | + 'grouped_matmul': ('GroupedMatmul', 'GroupedMatMul'), | ||
| 43 | + 'fused_add_norm': ('AddRmsNorm', 'AddRMSNorm'), | ||
| 44 | + } | ||
| 45 | + | ||
| 46 | + #: LongCat's layer runs two attention blocks around the MoE, so a single-attention | ||
| 47 | + #: template silently halves the attention cost. Checked against the derived graph. | ||
| 48 | + dataflow_invariants = ( | ||
| 49 | + {'id': 'dual_attention_per_layer', | ||
| 50 | + 'requires': None, | ||
| 51 | + 'kind': 'min_call_occurrences', | ||
| 52 | + 'match_any': ['attention', 'attn'], | ||
| 53 | + 'min_occurrences': 2, | ||
| 54 | + 'reason': 'LongCat 单层内有两个 attention 块；只声明一个会让 attention 耗时腰斩'}, | ||
| 55 | + ) | ||
| 56 | + | ||
| 57 | + known_deviations = ( | ||
| 58 | + {'id': 'zero_expert_routing_data_dependent', | ||
| 59 | + 'reason': 'zero-expert 是否被选中取决于运行期 router 输出，' | ||
| 60 | + '静态分析只能确认该支路存在，不能确认它在某次采集中被走过'}, | ||
| 61 | + {'id': 'multiple_config_classes', | ||
| 62 | + 'reason': 'config 模块同时定义 LongcatFlashConfig 与 LongcatFlashNgramConfig，' | ||
| 63 | + '两者层数/专家数不同；以 modeling 源码实际 import 的那个为准'}, | ||
| 64 | + ) | ||
| 65 | + | ||
| 66 | + def matches(self, evidence): | ||
| 67 | + classes = evidence.get('class_names', set()) | ||
| 68 | + keys = evidence.get('config_keys', set()) | ||
| 69 | + reasons = [] | ||
| 70 | + named = sorted(c for c in classes if 'Longcat' in c or 'LongCat' in c) | ||
| 71 | + if named: | ||
| 72 | + reasons.append(f'class 名含 LongCat: {named[:3]}') | ||
| 73 | + # Signature: num_layers + experts, and no dense boundary (LongCat is all-MoE). | ||
| 74 | + if ('num_layers' in keys and 'n_routed_experts' in keys | ||
| 75 | + and 'first_k_dense_replace' not in keys): | ||
| 76 | + reasons.append('config 含 num_layers + n_routed_experts 且无 first_k_dense_replace') | ||
| 77 | + if not reasons: | ||
| 78 | + return False | ||
| 79 | + return ('high' if named else 'medium'), reasons | ||
| @@ -0,0 +1,47 @@ | |||
| 1 | +# Copyright (c) 2026 Huawei Technologies Co., Ltd. | ||
| 2 | +# Licensed under the Apache License, Version 2.0 (the "License"); | ||
| 3 | +# you may not use this file except in compliance with the License. | ||
| 4 | +# You may obtain a copy of the License at | ||
| 5 | +# | ||
| 6 | +# http://www.apache.org/licenses/LICENSE-2.0 | ||
| 7 | +# | ||
| 8 | +# Unless required by applicable law or agreed to in writing, software | ||
| 9 | +# distributed under the License is distributed on an "AS IS" BASIS, | ||
| 10 | +# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| 11 | +# See the License for the specific language governing permissions and | ||
| 12 | +# limitations under the License. | ||
| 13 | +# | ||
| 14 | +"""Qwen (7B) adapter. | ||
| 15 | + | ||
| 16 | +Qwen7B is a plain dense transformer: num_hidden_layers, QWenBlock, no MoE, no MTP. | ||
| 17 | +""" | ||
| 18 | +from .base import BaseAdapter | ||
| 19 | + | ||
| 20 | + | ||
| 21 | +class QwenAdapter(BaseAdapter): | ||
| 22 | + name = 'qwen' | ||
| 23 | + main_layer_count_keys = ('num_hidden_layers',) | ||
| 24 | + prediction_count_keys = () | ||
| 25 | + dense_boundary_keys = () | ||
| 26 | + moe_expert_keys = ('n_routed_experts', 'num_experts') # future Qwen-MoE; absent in 7B | ||
| 27 | + | ||
| 28 | + #: 7B evidences none of these; they fire only for a Qwen variant that actually has | ||
| 29 | + #: the keys. Declaring `moe` unconditionally would make a dense model fail D7. | ||
| 30 | + capability_keys = { | ||
| 31 | + 'moe': ('n_routed_experts', 'num_experts'), | ||
| 32 | + 'shared_expert': ('shared_expert_intermediate_size',), | ||
| 33 | + } | ||
| 34 | + | ||
| 35 | + #: Candidate hints only. | ||
| 36 | + kernel_anchors = { | ||
| 37 | + 'attention_core': ('FlashAttentionScore', 'PromptFlashAttention', | ||
| 38 | + 'IncreFlashAttention'), | ||
| 39 | + 'fused_add_norm': ('AddRmsNorm', 'AddRMSNorm'), | ||
| 40 | + } | ||
| 41 | + | ||
| 42 | + def matches(self, evidence): | ||
| 43 | + classes = evidence.get('class_names', set()) | ||
| 44 | + named = sorted(c for c in classes if c.startswith('QWen') or c.startswith('Qwen')) | ||
| 45 | + if not named: | ||
| 46 | + return False | ||
| 47 | + return 'high', [f'class 名以 QWen/Qwen 开头: {named[:3]}'] | ||
| @@ -0,0 +1,4 @@ | |||
| 1 | +interface: | ||
| 2 | + display_name: "CANN 性能拆解" | ||
| 3 | + short_description: "从模型源码与 NPU Profiling 生成可验证的结构化性能拆解" | ||
| 4 | + default_prompt: "Use $cann-perf-breakdown to produce a validated NPU performance breakdown from model source and profiling data." | ||
| @@ -0,0 +1,165 @@ | |||
| 1 | +# Step 6 — AI 算子映射协议（可执行） | ||
| 2 | + | ||
| 3 | +本文件规定 Mode A **Step 6** 中 AI 如何把一个代表性 profiling step 的**全部**算子精确映射到模型结构，产出 schema v2 的 `analysis_config.json`。脚本负责校验，但语义映射由 AI 完成——本协议是该映射的强制操作规程。 | ||
| 4 | + | ||
| 5 | +> 核心红线：**代表性 step 的每一个 op 都必须获得明确归属**（model / runtime_auxiliary / 严格允许的 excluded）。**禁止**用一个包含几百个索引的 `unmapped_ops` 节点冒充完成。`unmapped_ops` 非空 → 严格校验必然失败。 | ||
| 6 | + | ||
| 7 | +--- | ||
| 8 | + | ||
| 9 | +## 1. 必备输入（缺一不可） | ||
| 10 | + | ||
| 11 | +开始映射前必须先获得并阅读： | ||
| 12 | + | ||
| 13 | +| 输入 | 来源 | 用途 | | ||
| 14 | +|---|---|---| | ||
| 15 | +| `model_manifest.json` | Step 2 `extract_model_manifest.py` | 全局架构真值：主层数、Dense/MoE 层号、learned MTP 层号、source_ref | | ||
| 16 | +| `dataflow_source.json` | Step 5 `extract_dataflow.py` | **数据流真值**：`forward()` 的调用顺序、残差汇合（`merges`）、并行 fork（`forks`）、config-gated 变体（`variants`）、无法静态判定的分支（`unsupported`）。声明 `branches` 时必须引用这里的证据 | | ||
| 17 | +| `raw_ops.compact.json` | Step 4 | 精简 op 序列（折叠重复），用于快速识别阶段边界 | | ||
| 18 | +| `raw_ops.json` | Step 4 | 完整 op 序列（index/name/stream/shape），映射与边界定位的权威依据 | | ||
| 19 | +| `op_segments.json`（可选） | `segment_layers.py` | layer 边界**候选**，仅作起点，最终以源码语义为准 | | ||
| 20 | +| 模型源码 | `models/<model>/` | decoder layer / MTP wrapper / lm_head / embedding 的 forward 语义 | | ||
| 21 | + | ||
| 22 | +--- | ||
| 23 | + | ||
| 24 | +## 2. 映射步骤（固定顺序） | ||
| 25 | + | ||
| 26 | +### 2.1 先识别完整执行阶段 | ||
| 27 | + | ||
| 28 | +在完整 op 序列上，用稳定 anchor 切出宏观阶段边界，先粗后细。 | ||
| 29 | + | ||
| 30 | +**anchor 必须来自本次模型的源码与 manifest，不得套用固定 kernel 名表**。同一语义在不同模型族、不同后端版本下 kernel 名完全不同（融合与否、量化与否、算子库版本），把某个模型族的名字写成通用规则，就是把适配器该做的事写死在协议里。正确做法是：先从 `dataflow_source.json` 读出该模块 `forward()` 的调用顺序，再在 op 序列里找与之对应的稳定重复段；`adapters/<family>.py` 的 `kernel_anchors` 可提供该族的已知别名作为**候选提示**。 | ||
| 31 | + | ||
| 32 | +按语义（而非名字）寻找这些边界： | ||
| 33 | + | ||
| 34 | +- **attention 起点**：每个 decoder invocation 的第一个注意力主 kernel。 | ||
| 35 | +- **layer 尾部**：残差归一（可能与下一层入口 norm 融合成一个 kernel，见 §2.5.1）。 | ||
| 36 | +- **MoE 标志**：门控 / 路由 / 分组 GEMM / dispatch-combine 通信。 | ||
| 37 | +- **lm_head / logits**：末尾的投影 + 词表维通信/Cast。 | ||
| 38 | +- **采样 / verify**：argmax 及 spec token 拼接。 | ||
| 39 | +- **runtime bookkeeping**：参数更新、warmup 的集合通信。 | ||
| 40 | + | ||
| 41 | +用 `stream_id` 辅助区分并行阶段（详见 `structure_analysis_guide.md` §B.4）。 | ||
| 42 | + | ||
| 43 | +### 2.2 再识别每个 decoder / MTP invocation | ||
| 44 | + | ||
| 45 | +对每个观测到的 invocation 建一条 `trace_instances[]` 记录： | ||
| 46 | + | ||
| 47 | +- 保存**真实** `op_range`（连续）或 `op_indices`（非连续），覆盖该次调用的全部 op。 | ||
| 48 | +- `model_layer_index`：能从源码/manifest 证明就填真实层号；**无法证明**时填 `"unknown"`，但 `layer_group_type` 与算子归属仍必须完成。 | ||
| 49 | +- `invocation_index`：同一模型层的第几次调用（MTP 迭代 0/1/2）。 | ||
| 50 | +- **MTP/spec decoding**：同一个 learned 层被调用 N 次 → N 条 instance，`model_layer_index` **全部相同**，用 `representative_instance_id` 指向模板实例。**禁止**写成 N 个模型层或伪层号。 | ||
| 51 | + | ||
| 52 | +### 2.3 代表结构树只复用定义，不替代实例映射 | ||
| 53 | + | ||
| 54 | +`structures[<layer_group_type>]` 存一棵代表性子结构树（用于报告体积压缩）。它**不**承担覆盖：真正的 op 归属来自 `trace_instances` 的 op_range/op_indices。代表树的叶子可留空 op_indices（报告用 representative 实例 op 计时）。 | ||
| 55 | + | ||
| 56 | +### 2.4 阶段与 runtime 的归属 | ||
| 57 | + | ||
| 58 | +- decoder layer 之外、属于模型主干的阶段（embedding、final norm、lm_head、MTP scaffold/output）→ `stages`，重复用 `stage_indices` 折叠。 | ||
| 59 | +- 不属于模型主干的运行时逻辑（token verify、sampling、输入更新、graph/step init、每次 MTP 迭代的 gather/all-gather 脚手架）→ `runtime_auxiliary`，重复用 `instance_indices` 折叠。 | ||
| 60 | +- 实现细节 op（Cast/Reshape/Transpose、动态量化与反量化）并入最近的真实模块，不单独成节点（§B.1）。 | ||
| 61 | + | ||
| 62 | +### 2.5 excluded 仅限极少数 profiler/bookkeeping | ||
| 63 | + | ||
| 64 | +只有真正不含模型数学、纯 profiler/设备记账的 op 才能进 `excluded_profiler_ops`，且必须用有限 `reason_code` 枚举 + `evidence`： | ||
| 65 | + | ||
| 66 | +`profiler_marker` / `stream_sync_placeholder` / `cross_step_bookkeeping` / `device_param_update` / `empty_shape_noop` | ||
| 67 | + | ||
| 68 | +**禁止**把 MatMul / Attention / Norm / MoE / 通信 / Gather / KV cache / 采样等主计算算子放入 excluded（脚本 C6 会阻断）。 | ||
| 69 | + | ||
| 70 | +### 2.5.1 融合 add-norm 链的三段归属策略 | ||
| 71 | + | ||
| 72 | +许多实现的 decoder layer 尾部那个 add-norm kernel，物理上是**下一层的 input_layernorm**（跨层 | ||
| 73 | +fusion：本层 residual + 下一层入口归一化融合成一个 kernel）。是否存在这种融合，从 | ||
| 74 | +`dataflow_source.json` 的 `merges[].kind == "fused_in_call"` 判断，不要靠 kernel 名猜。 | ||
| 75 | + | ||
| 76 | +融合把整个主干串成一条 norm 链，链上每个 norm 都把**上一次调用**遗留的 residual 加进来。链有两个 | ||
| 77 | +端点，端点的行为和链内不同，必须分三段判定： | ||
| 78 | + | ||
| 79 | +1. **链头**（首层入口）：没有上一次调用，`past_residual is None`，融合退化成一个**独立的 norm | ||
| 80 | + op**。它属于首层自己，不是任何"上一层的尾部"。这一段会让首层 op 数比其他层多一个，所以代表模板 | ||
| 81 | + 应取一个"典型"层而非首层，首层差异在 `instance.note` 标注。 | ||
| 82 | +2. **链内**（相邻两层之间）：按约定**归入当前层尾部**（命名如 `input_layernorm_next`），使每个 | ||
| 83 | + invocation 的 op 集合连续、不跨界，也不与下一 invocation 的注意力起点重复计数。相应地，该层的 | ||
| 84 | + 注意力残差有一端落在上一次调用里，对应的 `branches[]` 要写成绕回式（见 §2.5.2），下游才会识 | ||
| 85 | + 别为跨 invocation 的 carry。 | ||
| 86 | +3. **链尾**（最后一次被观测到的层之后）：没有下一层，residual 由**外层模块自己调用的 final norm / | ||
| 87 | + shared_head_norm** 吸收。这个 norm **必须**登记为 `stages` 里的独立阶段（与 §2.4 一致）， | ||
| 88 | + **不得**当作"下一层的 input_layernorm" 挂在最后一层尾部——那一层根本不存在。同时，最后一层的 | ||
| 89 | + 代表模板不能复用链内模板：链内模板带 `input_layernorm_next` 子节点，套在链尾层上会让报告显示一 | ||
| 90 | + 个不存在的层尾节点。若 schema 不支持实例级结构覆写，就为它建一个单独的 structure（如 | ||
| 91 | + `<Layer>_final`）。它跨出了 structure 边界的那条残差边，写在顶层 `dataflow.edges`（§2.5.2）。 | ||
| 92 | + | ||
| 93 | +**判定依据是源码调用点 + 后继拓扑，不是 kernel 名、shape 或 stream。** 链尾 norm 与链内 norm 在 | ||
| 94 | +trace 里通常完全同形：同 kernel、同 shape、同 stream，且 invocation 的 op 数也不变（少了一个层尾 | ||
| 95 | +norm，但外层 norm 补上），因此 **SL6 这类"op 数与模板不符"的确定性检查抓不到链尾错误**（它只抓得到 | ||
| 96 | +链头，因为链头确实多一个 op）。唯一能区分的是后继：链内 norm 后面紧跟下一层的注意力起点，链尾 norm | ||
| 97 | +后面紧跟 lm_head。这一段只能由语义审查（`tail_stages_correct`）把关。 | ||
| 98 | + | ||
| 99 | +### 2.5.2 显式声明数据流边（残差 / 并行 / skip） | ||
| 100 | + | ||
| 101 | +`children` **只表达包含关系**：相邻两个 child 不构成数据流边，下游（Skill 2 建图、Skill 3 渲染）**禁止**从顺序推导连接。残差、并行支路和 skip 全部活在变量传递里（`hidden, residual = norm(hidden, residual)`），在 `children` 顺序上不留痕迹，所以**没有在 `branches` 里声明的边，在下游就等于不存在**。 | ||
| 102 | + | ||
| 103 | +对每个 structure，逐条对照 `dataflow_source.json` 里该模块的 `merges` 和 `forks`： | ||
| 104 | + | ||
| 105 | +- 每个 `merges[]`（含 `kind: fused_in_call` 的融合 add-norm 与 `kind: in_place_add` 的 `+=`）都必须对应一条 `branches[]`。融合形式没有独立的 Add 算子，只看 op 序列是看不出来的——这正是最容易整层丢失残差的地方。 | ||
| 106 | +- 每个 `forks[]`（一个值被 2 个以上消费者读取，包括直接读 `forward()` 入参的情形，如共享专家与路由并列）必须声明为 `kind: parallel` 的分支；若把它们写成相邻 children 且不给 branches，下游会把并行支路渲染成串行链。 | ||
| 107 | +- `inputs` 是分叉点，`output` 是汇合点，两者之间的兄弟节点就是被绕过的部分。**方向不能反**：起点取在主路径上（两端相邻、中间没有被绕过的节点）会被 D2 判为方向错误。 | ||
| 108 | +- 一端落在**上一次调用**的残差（融合 add-norm 的典型形态）应写成绕回式（`inputs` 位置在 `output` 之后），下游据此识别为跨 invocation 的 carry 而不是层内环。把它写成正向反而会复现 G7 要抓的反向残差缺陷。 | ||
| 109 | +- `variants`（config-gated 分支，如量化模式/TP 规模）选中哪一支由部署决定：在 `execution_profiles` 里说明本次采集对应的 profile。 | ||
| 110 | +- `unsupported`（依赖运行期数据的分支）无法静态判定：必须在 `deviations[]` 里显式声明本次走的是哪一支及理由，否则 D5 阻断。 | ||
| 111 | + | ||
| 112 | +每条 `branches[]` 都要带 `source_ref`（或 `code_ref`）指向源码行。`check_dataflow.py` 会把这些声明与 AST 重新推导的图逐条比对，D1-D7 任一 error 都会阻断正式流程。 | ||
| 113 | + | ||
| 114 | +### 2.6 收敛条件 | ||
| 115 | + | ||
| 116 | +`model_mapped + runtime_mapped + excluded == total_ops`，`unmapped == 0`，`duplicate == 0`，`out_of_range == 0`。任一不满足则继续映射，不得提交。 | ||
| 117 | + | ||
| 118 | +--- | ||
| 119 | + | ||
| 120 | +## 3. Step 6 提示词模板（可直接投喂给映射 subagent） | ||
| 121 | + | ||
| 122 | +```text | ||
| 123 | +你是 NPU 性能拆解的算子映射专家。目标：把代表性 step 的全部 <TOTAL> 个算子精确映射到本模型的结构，产出 schema v2 的 analysis_config.json。 | ||
| 124 | + | ||
| 125 | +输入（均已提供，必须全部使用）： | ||
| 126 | +- model_manifest.json：<粘贴或路径> # 全局架构真值：主层数、Dense/MoE 层号范围、learned 预测层号、source_ref、capabilities | ||
| 127 | +- dataflow_source.json：<路径> # forward() 的数据流真值：calls/merges/forks/variants/unsupported | ||
| 128 | +- raw_ops.json：<路径> # 完整 op 序列（index/normalized_name/stream_id/input_shapes/output_shapes） | ||
| 129 | +- raw_ops.compact.json：<路径> # 折叠视图，用于先看宏观结构 | ||
| 130 | +- op_segments.json：<路径，可选> # 边界候选，仅参考 | ||
| 131 | +- 模型源码切片：<按 code_ref 提供 decoder/预测模块/lm_head/embedding forward> | ||
| 132 | + | ||
| 133 | +硬性要求： | ||
| 134 | +1. anchor 只能来自本模型源码与 manifest：先读 dataflow_source.json 得到该模块 forward() 的调用顺序，再在 op 序列里找对应的稳定重复段。禁止套用其他模型族的固定 kernel 名表。 | ||
| 135 | +2. 每个 decoder/预测模块 invocation 写一条 trace_instances[]，保存真实 op_range/op_indices，覆盖该次调用全部 op。 | ||
| 136 | +3. 同一个 learned 层被外层循环调用 N 次（MTP/spec decoding）= 1 个 learned 层 + N 条 invocation（model_layer_index 相同、invocation_index 递增），禁止写成 N 个模型层或伪层号。 | ||
| 137 | +4. model_layer_index 无法证明时填 "unknown"，但 layer_group_type、op 归属仍要完成。 | ||
| 138 | +5. embedding/final_norm/lm_head/预测模块 scaffold 与 output → stages（重复用 stage_indices）。 | ||
| 139 | +6. token verify/sampling/输入更新/step init/每次迭代脚手架 → runtime_auxiliary（重复用 instance_indices）。 | ||
| 140 | +7. 仅纯 profiler/bookkeeping 可进 excluded_profiler_ops，必须带 reason_code 枚举 + evidence；主计算算子禁止 excluded。 | ||
| 141 | +8. 全部算子必须落入 model/runtime/excluded 之一；unmapped_ops 必须为空。 | ||
| 142 | +9. **数据流边必须显式声明**：dataflow_source.json 里每个 merges[]（含 fused_in_call / in_place_add）对应一条 branches[]；每个 forks[] 声明为 kind: parallel。children 只表达包含关系，未声明的边下游不存在。每条 branches[] 带 source_ref 指向源码行。 | ||
| 143 | +10. unsupported[]（依赖运行期数据的分支）必须在 deviations[] 里声明本次走哪一支及理由；variants[]（config-gated）在 execution_profiles 里说明本次对应的 profile。 | ||
| 144 | +11. 完成后自检：sum(model,runtime,excluded)==<TOTAL> 且 unmapped==0 且无 duplicate/out-of-range；merges 数量 == branches 覆盖的汇合点数量。 | ||
| 145 | + | ||
| 146 | +输出：完整 analysis_config.json（schema v2）。随后必须按 `references/semantic_review_protocol.md` 生成并完成 `semantic_review.json`，再把它传给 `run_validation.py` 和 `score_breakdown.py`。只有 semantic review、validation 与 score 都为 passed 才算完成；unmapped>0、duplicate>0、语义审查失败、总分或核心分项未达标一律继续映射。 | ||
| 147 | +``` | ||
| 148 | + | ||
| 149 | +--- | ||
| 150 | + | ||
| 151 | +## 4. 与校验的闭环 | ||
| 152 | + | ||
| 153 | +映射产出后先完成 `semantic_review.json`（Step 8），再运行 `run_validation.py`（Step 9）和 `score_breakdown.py`（Step 10）。若任一状态不是 `passed`： | ||
| 154 | + | ||
| 155 | +- `coverage.unmapped > 0` → 回到 §2.2/§2.4，把 unmapped 索引逐个归属到真实模块或 runtime。 | ||
| 156 | +- `coverage.duplicate > 0` → 两个 owner 争抢同一 op，收紧边界。 | ||
| 157 | +- `C6` → 有主计算算子被误放 excluded，移回模型节点。 | ||
| 158 | +- `architecture` A1–A9 → 架构与 manifest 不符，回到 Step 2/3。 | ||
| 159 | +- `dataflow` D1–D7 → 声明的边与源码不符，回到 §2.5.2 逐条对照 `dataflow_source.json`：D1 漏声明残差、D2 分支方向反了、D3 绕过的是源码没调用的节点、D4 并行支路被写成串行链、D5 未声明运行期分支、D6 源码调用的子模块在结构里没有位置、D7 manifest 声明的 capability 找不到对应 fork/join。 | ||
| 160 | +- `semantic_review` → 按源码逐项修复 Q/K/V 分支、残差、layer 边界、final norm/tail/runtime；不得用 100% Kernel 覆盖代替语义证据。 | ||
| 161 | +- 读取 `iteration_request.json`，从其中的 `base_config_for_revision` 开始，只修复失败分项与阻断问题，然后重新 enrich、校验和评分。 | ||
| 162 | +- 配置修改后必须重新运行 `prepare_semantic_review.py` 并生成新 review；旧 SHA256 review 不得复用。 | ||
| 163 | +- 不得降低 95 分总门槛或任何分项最低分，不得用 excluded 隐藏主计算 Kernel，不得删除本应存在的 `branches` 来消除 D2/D3 告警。 | ||
| 164 | + | ||
| 165 | +探索期可临时用 `--allow-unmapped` 观察分布，但结果状态为 `exploratory`，**不是** `passed`，且报告会显著标注“未验证”，不可作为正式结果。 | ||
| @@ -0,0 +1,150 @@ | |||
| 1 | +# Step 6: Executable AI Operator-Mapping Protocol | ||
| 2 | + | ||
| 3 | +This document defines how AI maps **every** operator in one representative profiling step to model structure in Mode A Step 6 and produces schema-v2 `analysis_config.json`. Scripts validate the result, but AI performs semantic attribution. This protocol is mandatory. | ||
| 4 | + | ||
| 5 | +> Hard rule: **Every operator in the representative step must have an explicit owner**: model, runtime auxiliary, or a strictly allowed exclusion. Never hide hundreds of indices in one `unmapped_ops` node. Non-empty `unmapped_ops` always fails strict validation. | ||
| 6 | + | ||
| 7 | +--- | ||
| 8 | + | ||
| 9 | +## 1. Required Inputs | ||
| 10 | + | ||
| 11 | +Obtain and read every input before mapping: | ||
| 12 | + | ||
| 13 | +| Input | Source | Purpose | | ||
| 14 | +|---|---|---| | ||
| 15 | +| `model_manifest.json` | Step 2 `extract_model_manifest.py` | Architecture truth: main-layer count, Dense/MoE indices, learned MTP indices, and `source_ref` | | ||
| 16 | +| `dataflow_source.json` | Step 5 `extract_dataflow.py` | **Dataflow truth** from `forward()`: call order, residual `merges`, parallel `forks`, config-gated `variants`, and statically undecidable `unsupported` branches. Cite it for every branch declaration. | | ||
| 17 | +| `raw_ops.compact.json` | Step 4 | Folded operator sequence for locating coarse stage boundaries | | ||
| 18 | +| `raw_ops.json` | Step 4 | Authoritative complete operator sequence with index, name, stream, and shape | | ||
| 19 | +| `op_segments.json` (optional) | `segment_layers.py` | Candidate layer boundaries only; source semantics remain authoritative | | ||
| 20 | +| Model source | `models/<model>/` | Forward semantics for decoder layers, MTP wrappers, lm_head, and embedding | | ||
| 21 | + | ||
| 22 | +--- | ||
| 23 | + | ||
| 24 | +## 2. Mapping Procedure | ||
| 25 | + | ||
| 26 | +Follow this order exactly. | ||
| 27 | + | ||
| 28 | +### 2.1 Identify complete execution stages first | ||
| 29 | + | ||
| 30 | +Use stable anchors to split the full operator sequence into coarse stages before refining it. | ||
| 31 | + | ||
| 32 | +**Anchors must come from this model's source and manifest. Never apply a fixed kernel-name list.** Kernel names for the same semantics vary by model family, fusion, quantization, backend version, and operator library. First read the module call order from `dataflow_source.json`, then find its stable repeated patterns in the trace. Family-specific `adapters/<family>.py` `kernel_anchors` are candidate hints only. | ||
| 33 | + | ||
| 34 | +Find semantic boundaries rather than literal names: | ||
| 35 | + | ||
| 36 | +- **Attention start:** first main attention kernel in each decoder invocation. | ||
| 37 | +- **Layer tail:** residual normalization, possibly fused with the next layer's input norm; see 2.5.1. | ||
| 38 | +- **MoE markers:** gating, routing, grouped GEMM, and dispatch/combine communication. | ||
| 39 | +- **lm_head/logits:** terminal projection plus vocabulary-dimension communication or cast. | ||
| 40 | +- **Sampling/verification:** argmax and speculative-token assembly. | ||
| 41 | +- **Runtime bookkeeping:** parameter updates and warmup collectives. | ||
| 42 | + | ||
| 43 | +Use `stream_id` to distinguish parallel stages; see `structure_analysis_guide_en.md` B.4. | ||
| 44 | + | ||
| 45 | +### 2.2 Identify each decoder and MTP invocation | ||
| 46 | + | ||
| 47 | +Create one `trace_instances[]` record for every observed invocation: | ||
| 48 | + | ||
| 49 | +- Store the real continuous `op_range` or non-contiguous `op_indices`, covering every operator in that invocation. | ||
| 50 | +- Set `model_layer_index` to the real index only when source or manifest proves it. Otherwise use `"unknown"`, while still completing `layer_group_type` and operator ownership. | ||
| 51 | +- Set `invocation_index` to the invocation number for that model layer, such as MTP iteration 0, 1, or 2. | ||
| 52 | +- For MTP/speculative decoding, one learned layer invoked N times means N instances with the **same** `model_layer_index`; point them to the template with `representative_instance_id`. Never invent N model layers or synthetic layer indices. | ||
| 53 | + | ||
| 54 | +### 2.3 Reuse representative structure definitions without replacing instance mapping | ||
| 55 | + | ||
| 56 | +`structures[<layer_group_type>]` stores one representative subtree to reduce report size. It does not provide coverage. Real ownership comes from each `trace_instances` `op_range` or `op_indices`. Representative-tree leaves may omit indices; reporting uses the representative instance for timing. | ||
| 57 | + | ||
| 58 | +### 2.4 Assign stages and runtime operators | ||
| 59 | + | ||
| 60 | +- Model-backbone stages outside decoder layers, including embedding, final norm, lm_head, and MTP scaffold/output, belong in `stages`; fold repetitions with `stage_indices`. | ||
| 61 | +- Non-backbone runtime logic, including token verification, sampling, input updates, graph/step initialization, and per-MTP-iteration gather/all-gather scaffolding, belongs in `runtime_auxiliary`; fold repetitions with `instance_indices`. | ||
| 62 | +- Merge implementation operators such as Cast, Reshape, Transpose, dynamic quantization, and dequantization into the nearest real module. Do not create standalone structural nodes for them. | ||
| 63 | + | ||
| 64 | +### 2.5 Restrict exclusions to profiler and bookkeeping operations | ||
| 65 | + | ||
| 66 | +Only operations with no model mathematics and used solely for profiling or device bookkeeping may enter `excluded_profiler_ops`. Each must use an allowed `reason_code` and `evidence`: | ||
| 67 | + | ||
| 68 | +`profiler_marker`, `stream_sync_placeholder`, `cross_step_bookkeeping`, `device_param_update`, or `empty_shape_noop`. | ||
| 69 | + | ||
| 70 | +Never exclude main computation such as MatMul, Attention, Norm, MoE, communication, Gather, KV cache, or sampling. C6 blocks these exclusions. | ||
| 71 | + | ||
| 72 | +### 2.5.1 Three-part ownership for fused add-norm chains | ||
| 73 | + | ||
| 74 | +In many implementations, the add-norm kernel at the end of one decoder layer is physically the **next layer's input_layernorm**, fused across layers. Determine this from `dataflow_source.json` `merges[].kind == "fused_in_call"`, not from kernel names. | ||
| 75 | + | ||
| 76 | +Fusion links the backbone into a norm chain in which each norm adds the residual left by the previous invocation. Treat its endpoints differently from its interior: | ||
| 77 | + | ||
| 78 | +1. **Chain head:** The first layer has no previous invocation and `past_residual is None`, so fusion degenerates to an independent norm. It belongs to the first layer, not a previous layer tail. The first layer therefore has one extra operator; choose a typical later layer as the representative and record the first-layer difference in `instance.note`. | ||
| 79 | +2. **Chain interior:** By convention, assign the fused norm to the current layer tail, with a name such as `input_layernorm_next`. This keeps each invocation's operators contiguous and avoids double-counting the next attention start. One attention-residual endpoint then lies in the previous invocation; declare its `branches[]` edge in wraparound form as described in 2.5.2 so downstream consumers recognize a cross-invocation carry. | ||
| 80 | +3. **Chain tail:** After the last observed layer, no next layer exists. An outer-module final norm or `shared_head_norm` consumes the residual. Register it as a separate `stages` entry. Never attach it to the last layer as a nonexistent next input norm. The last layer cannot reuse an interior template containing `input_layernorm_next`; create a separate structure such as `<Layer>_final` if instance-level override is unavailable. Put the cross-structure residual edge in top-level `dataflow.edges`. | ||
| 81 | + | ||
| 82 | +Use source call sites and successor topology, not kernel name, shape, or stream. Tail and interior norms often look identical in the trace, and the invocation operator count can remain unchanged because an outer norm replaces the interior tail norm. Deterministic checks such as SL6 cannot detect this tail error. The distinguishing successor is another attention start for an interior norm and lm_head for the tail norm. `tail_stages_correct` semantic review must enforce it. | ||
| 83 | + | ||
| 84 | +### 2.5.2 Declare dataflow edges explicitly | ||
| 85 | + | ||
| 86 | +`children` expresses containment only. Adjacent children do not imply an edge, and downstream graph construction and rendering must not infer one from order. Residuals, parallel branches, and skips exist in variable flow and are absent unless declared in `branches`. | ||
| 87 | + | ||
| 88 | +For every structure, compare against the module's `merges` and `forks` in `dataflow_source.json`: | ||
| 89 | + | ||
| 90 | +- Every `merges[]`, including `fused_in_call` add-norm and `in_place_add` `+=`, requires a matching `branches[]`. A fused form has no standalone Add kernel, so operator order alone cannot reveal it. | ||
| 91 | +- Every `forks[]`, including multiple consumers of a direct `forward()` input, requires a `kind: parallel` declaration. Adjacent children without a branch render incorrectly as a chain. | ||
| 92 | +- `inputs` are branch points and `output` is the merge. Siblings between them are bypassed. Do not reverse direction: a start on the main path with adjacent endpoints and no bypass is a D2 error. | ||
| 93 | +- For a residual endpoint in the previous invocation, use wraparound form with the input positioned after the output in child order. Downstream consumers then recognize a cross-invocation carry rather than an intra-layer cycle. | ||
| 94 | +- Select config-gated `variants`, such as quantization or TP scale, through `execution_profiles` for this capture. | ||
| 95 | +- Runtime-data-dependent `unsupported` branches are statically undecidable. Declare the selected path and reason in `deviations[]`; otherwise D5 blocks. | ||
| 96 | + | ||
| 97 | +Every `branches[]` item must contain `source_ref` or `code_ref`. `check_dataflow.py` re-derives the AST graph and compares every declaration; any D1-D7 error blocks formal completion. | ||
| 98 | + | ||
| 99 | +### 2.6 Convergence | ||
| 100 | + | ||
| 101 | +Require `model_mapped + runtime_mapped + excluded == total_ops`, with `unmapped == 0`, `duplicate == 0`, and `out_of_range == 0`. Continue mapping until all hold. | ||
| 102 | + | ||
| 103 | +--- | ||
| 104 | + | ||
| 105 | +## 3. Step 6 Prompt Template | ||
| 106 | + | ||
| 107 | +```text | ||
| 108 | +You are an expert in mapping NPU performance operators to model structure. Map all <TOTAL> operators in the representative step precisely and produce schema-v2 analysis_config.json. | ||
| 109 | + | ||
| 110 | +Required inputs: | ||
| 111 | +- model_manifest.json: <content or path> # architecture truth, layer groups, learned prediction indices, source references, capabilities | ||
| 112 | +- dataflow_source.json: <path> # forward calls, merges, forks, variants, unsupported branches | ||
| 113 | +- raw_ops.json: <path> # complete indexed operator sequence with names, streams, and shapes | ||
| 114 | +- raw_ops.compact.json: <path> # folded overview | ||
| 115 | +- op_segments.json: <optional path> # candidate boundaries only | ||
| 116 | +- model source slices: <decoder/prediction/lm_head/embedding forward slices selected by code_ref> | ||
| 117 | + | ||
| 118 | +Hard requirements: | ||
| 119 | +1. Derive anchors only from this model's source and manifest. Read forward call order first; never import a fixed kernel-name list from another family. | ||
| 120 | +2. Create one trace_instances[] item per decoder/prediction invocation with its real op_range/op_indices and complete coverage. | ||
| 121 | +3. N calls to one learned MTP/speculative layer mean one learned layer and N invocations with the same model_layer_index and increasing invocation_index. Never create synthetic layers. | ||
| 122 | +4. Use "unknown" when model_layer_index cannot be proved, but still complete layer_group_type and ownership. | ||
| 123 | +5. Put embedding, final norm, lm_head, and prediction scaffold/output in stages, folding repetitions with stage_indices. | ||
| 124 | +6. Put token verification, sampling, input updates, step initialization, and per-iteration scaffolding in runtime_auxiliary, folding repetitions with instance_indices. | ||
| 125 | +7. Only pure profiler/bookkeeping operations may be excluded, and each requires an allowed reason_code plus evidence. Never exclude main computation. | ||
| 126 | +8. Every operator must belong to model, runtime, or excluded. unmapped_ops must be empty. | ||
| 127 | +9. Declare all dataflow edges explicitly. Match every dataflow_source merge, including fused_in_call and in_place_add, with branches[] and every fork with kind: parallel. children is containment only. Include source_ref on every branch. | ||
| 128 | +10. Declare runtime-dependent unsupported paths in deviations and config-gated variants in execution_profiles. | ||
| 129 | +11. Verify sum(model,runtime,excluded)==<TOTAL>, unmapped==0, no duplicates or out-of-range indices, and every merge/fork has a declaration. | ||
| 130 | + | ||
| 131 | +Output complete schema-v2 analysis_config.json. Then follow references/semantic_review_protocol_en.md to produce semantic_review.json and pass it to run_validation.py and score_breakdown.py. Completion requires convertible score plus passed semantic review and validation. Continue mapping on any coverage, duplicate, semantic, total-score, or core-dimension failure. | ||
| 132 | +``` | ||
| 133 | + | ||
| 134 | +--- | ||
| 135 | + | ||
| 136 | +## 4. Validation Loop | ||
| 137 | + | ||
| 138 | +After mapping, complete `semantic_review.json` in Step 8, then run `run_validation.py` in Step 9 and `score_breakdown.py` in Step 10. If any status is not accepted: | ||
| 139 | + | ||
| 140 | +- `coverage.unmapped > 0`: return to 2.2/2.4 and assign every index to a real module or runtime node. | ||
| 141 | +- `coverage.duplicate > 0`: two owners overlap; tighten boundaries. | ||
| 142 | +- C6: a main-compute operator was excluded; restore it to a model node. | ||
| 143 | +- Architecture A1-A9: reconcile architecture with manifest and return to Steps 2/3. | ||
| 144 | +- Dataflow D1-D7: compare declarations against `dataflow_source.json`; D1 is a missing residual, D2 reversed direction, D3 bypassing a source-unrelated node, D4 parallel serialized as a chain, D5 an undeclared runtime branch, D6 a called source submodule absent from structure, and D7 a manifest capability without its fork/join. | ||
| 145 | +- Semantic review: correct Q/K/V topology, residuals, layer boundaries, final norm, tail, and runtime against source. Do not substitute 100% kernel coverage for semantic evidence. | ||
| 146 | +- Read `iteration_request.json`, start from `base_config_for_revision`, and fix only failed dimensions and hard gates before re-enrichment, validation, and scoring. | ||
| 147 | +- Regenerate the semantic review after every configuration change. Never reuse stale SHA256 bindings. | ||
| 148 | +- Never lower the 95-point total or dimension thresholds, hide main kernels in exclusions, or delete required branches merely to silence D2/D3. | ||
| 149 | + | ||
| 150 | +During exploration, `--allow-unmapped` may show distributions, but its status is `exploratory`, not `passed`, and its report is visibly unverified. It is not a formal result. | ||
| @@ -0,0 +1,111 @@ | |||
| 1 | +# 模型拆解评分与迭代协议 | ||
| 2 | + | ||
| 3 | +本协议用于 Mode A 的正式拆解。评分不是代替源码审查，而是把源码、trace 和统一校验器已经产出的证据转换为稳定、可比较的质量门槛。 | ||
| 4 | + | ||
| 5 | +## 1. 通过条件 | ||
| 6 | + | ||
| 7 | +一次拆解只有同时满足以下条件才可转换为正式下游报告： | ||
| 8 | + | ||
| 9 | +1. 可运行检查正确率 `score / runnable_max >= 95%`。`score` 仍位于固定 100 分名义量表，不能单独作为门禁；输入确实缺失而无法运行的检查只退出 `runnable_max`，并通过证据结论降级明确披露。 | ||
| 10 | +2. **每个可运行核心维度**都达到原最低比例： | ||
| 11 | + | ||
| 12 | + | 维度 | id | 满分 | 最低分 | | ||
| 13 | + |---|---|---:|---:| | ||
| 14 | + | 架构完整性 | `architecture_integrity` | 25 | 22 | | ||
| 15 | + | 数据流与分支正确性 | `dataflow_branch_correctness` | 30 | 27 | | ||
| 16 | + | Layer/子模块边界 | `layer_submodule_boundaries` | 20 | 18 | | ||
| 17 | + | Kernel 精确覆盖 | `kernel_exact_coverage` | 20 | 20 | | ||
| 18 | + | 证据与可追溯性 | `evidence_traceability` | 5 | 4 | | ||
| 19 | + | ||
| 20 | + **两道门必须都能独立生效**：核心分项最低比例和总正确率分别检查，某项被真实输入缺口判为不可运行时不得把它记为满分，也不得让它把正确拆解永久挡死。Kernel 覆盖与证据项始终可运行，仍按原门槛执行。 | ||
| 21 | + | ||
| 22 | + 原“Shape 与语义一致性”和“Trace 实例与 scope”两项已删除，其 20 分并入数据流（+10）与架构（+5）以及 Kernel 覆盖（+5）。删除原因：`shape_semantic` 缺失、以及一次采集只覆盖单个 step，描述的是**输入里有什么**，而不是**拆解对不对**——这两项给“不可能出错的证据”发分，既无法否证也无法区分好坏。取而代之的两项都可由源码机器校验。 | ||
| 23 | + | ||
| 24 | + 扣分粒度为每个 `warning` 扣该检查权重的 20%（`WARNING_STEP`）。此前 quality 只取 {1.0, 0.6, 0.0}，任一 warning 即扣 40%，使“27/30”这类分数门槛实际等价于“零 warning”——看似分级实为二元。现在单个 warning 可容忍，成堆 warning 不可。 | ||
| 25 | +3. 所有硬性否决项为零。 | ||
| 26 | +4. `semantic_review.json` 的九项检查和总状态必须全部为 `passed`，且输入 SHA256 未过期。 | ||
| 27 | + 唯一例外是 `source_model_identity` 上的证据缺口 `warning` finding：它报的是输入不含该标量， | ||
| 28 | + 而非语义结论有错，记为 `info` 不阻断（见 `semantic_review_protocol.md`「唯一例外」一节）。 | ||
| 29 | + 该情形的上限由下述 capture_tier 机制强制，不靠 validation 状态兼职。 | ||
| 30 | +5. `run_validation.py` 的状态必须为 `passed`；`passed_with_warnings` 和 `exploratory` 只能用于排查，不能生成正式报告。 | ||
| 31 | + 数据可用性类问题（`A1`/`MT1`/`MA1`/`SR_EVIDENCE_GAP_FINDING`）一律发 `info`：它们报的是输入缺少某个 | ||
| 32 | + 标量的证据，而不是拆解错误，因此不该把状态推离 `passed`。真正的上限由 capture_tier 施加——tier A | ||
| 33 | + 把 `source_model_identity` 的 8 分移出分母，`runnable_max` 92 低于 95，结论封顶 | ||
| 34 | + `verified_unbound_scalars`。若改用 `warning`，这类采集会被 `GATE_VALIDATION` 判成 | ||
| 35 | + `needs_iteration`，把"输入缺证据"误报成"拆解待修"。 | ||
| 36 | + | ||
| 37 | +最终 `status` 表达证据所能支持的结论：`verified`、`verified_unbound_scalars`、`structure_unverified`、`exploratory` 或 `needs_iteration`。前三者 `convertible=true`；`passed` 仅为 pre-tier 历史兼容值。门槛是 Skill 的质量基线，迭代时禁止降低。 | ||
| 38 | + | ||
| 39 | +## 2. 100 分量表 | ||
| 40 | + | ||
| 41 | +| 维度 | 分值 | 主要证据 | | ||
| 42 | +|---|---:|---| | ||
| 43 | +| 架构完整性 | 25 | architecture/regression + `source_model_identity`/`module_inventory_complete` | | ||
| 44 | +| 数据流与分支正确性 | 30 | **`check_dataflow.py` 的 D1-D7**（源码 `forward()` 与配置声明边的一致性）+ structure/sublayers + Q/K/V 分支与残差语义审查 | | ||
| 45 | +| Layer/子模块边界 | 20 | structure/coverage/sublayers + layer/tail/runtime/code-ref 语义审查 | | ||
| 46 | +| Kernel 精确覆盖 | 20 | model/runtime/excluded 对代表 step 的精确并集，要求 missing/duplicate/out-of-range/unmapped 全为零 | | ||
| 47 | +| 证据与可追溯性 | 5 | source_of_truth、code_ref、manifest source_ref 和 evidence_gaps | | ||
| 48 | + | ||
| 49 | +数据流成为最高权重维度（30），因为它是唯一既最容易出错、又能被源码确定性判定的部分：`forward()` 就是数据流图，声明与源码不符可以机器判定，不需要依赖 AI 自述。 | ||
| 50 | + | ||
| 51 | +`shape_semantic` 与 `trace_scope` 仍可作为可选注解存在于配置中，但不参与评分，也不进入 `run_validation.py` 的正式门禁。 | ||
| 52 | + | ||
| 53 | +`scripts/score_breakdown.py` 只从结构化产物计算分数，不接受自由填写总分。 | ||
| 54 | + | ||
| 55 | +## 3. 硬性否决项 | ||
| 56 | + | ||
| 57 | +下列任一项存在时，即使总分达到 95 也必须重新拆解： | ||
| 58 | + | ||
| 59 | +- unified validation 不是 `passed`。 | ||
| 60 | +- 缺少 `semantic_review.json`、审查失败，或 artifact SHA256 与当前 config/raw/manifest 不一致。 | ||
| 61 | +- `unmapped_ops` 非空，或 coverage 的 unmapped/missing/duplicate/out_of_range 非零。 | ||
| 62 | +- v1 迁移结果仍为 `legacy_unverified`。 | ||
| 63 | +- Mode A 缺少源码真值或结构节点没有可用 `code_ref`，却声称精确模型映射。 | ||
| 64 | +- 校验 issue 已发现学习层数、MTP invocation、Dense/MoE、Q/K/V、残差或 layer 边界冲突。 | ||
| 65 | +- `check_dataflow.py` 报出 error 级 D1/D2/D5：源码有残差汇合而配置未声明任何 `branches`（D1）、声明的分支方向反了且没有绕过任何节点（D2）、或源码存在依赖运行期数据的分支而配置未在 `deviations` 中显式声明所走分支（D5）。这三项都是「配置与源码直接矛盾」，不是覆盖率或注解问题。 | ||
| 66 | + | ||
| 67 | +> 无源码时 `check_dataflow` 不出结论（检查缺席，而非通过）。缺席不构成否决，但也不能充当已校验的证据——正式流程要求 Mode A 提供源码。 | ||
| 68 | + | ||
| 69 | +不得通过删除主计算 Kernel、扩大 `excluded_profiler_ops`、伪造 source_ref、合并本应独立的分支或降低阈值来消除失败项。 | ||
| 70 | + | ||
| 71 | +## 4. 闭环迭代 | ||
| 72 | + | ||
| 73 | +每一轮严格执行： | ||
| 74 | + | ||
| 75 | +1. 基于源码、manifest、raw_ops 和 `dataflow_source.json` 生成候选 `analysis_config.json`。 | ||
| 76 | +2. 运行 enrich，生成 `semantic_review_request.json`；AI 完整核对源码与 Trace 后填写 `semantic_review.json`。 | ||
| 77 | +3. 运行 `run_validation.py` 和 `score_breakdown.py`。 | ||
| 78 | +4. 若通过，才运行 report/metrics。 | ||
| 79 | +5. 若未通过，读取 `iteration_request.json`，只针对 `blocking_issues`、`failed_dimensions` 和 `required_actions` 修正配置，然后从第 2 步重新执行。 | ||
| 80 | + | ||
| 81 | +每次 `analysis_config.json` 发生变化都必须重新审查。旧 review 哈希失效时，驱动停在 `awaiting_semantic_review`，该候选不会作为低分轮次写入历史。 | ||
| 82 | + | ||
| 83 | +驱动本身不会调用 AI 修改候选。执行 Skill 的 agent 必须读取并消费 `ai_mapping_request.json`、`semantic_review_request.json` 和 `iteration_request.json`，持续完成映射、审查与定向修正；只有通过、达到停止条件或确实缺少新证据时才能结束。 | ||
| 84 | + | ||
| 85 | +`run_breakdown.py` 会将每轮输入与评分保存到 `iterations/`，并维护 `iteration_history.json`。下一轮编辑 `iteration_request.json.base_config_for_revision` 指向的候选副本；`immutable_best_snapshot` 只读、不得原地修改。候选分数下降时，不得把下降版本当作新的基线。 | ||
| 86 | + | ||
| 87 | +## 4.1 停止条件 | ||
| 88 | + | ||
| 89 | +只有两个终止条件： | ||
| 90 | + | ||
| 91 | +1. **达标停止**：`breakdown_score.convertible == true`（可运行正确率 `>= 95%` + 可运行核心维度达最低比例 + 硬性否决项为零 + validation/semantic review 均 `passed`）。结论状态说明证据层级，不得把 `verified_unbound_scalars` 的名义分数低于 95 误判为需要迭代。 | ||
| 92 | +2. **轮次上限停止**：评估轮次达到 `--max-iterations`（默认 **10**）仍未达标，状态为 `blocked_max_iterations`。 | ||
| 93 | + | ||
| 94 | +默认**不再**因"连续两轮无提升"提前退出：`--stall-limit` 默认为 `0`（关闭早停），循环会一直跑到达标或第 10 轮。需要早停时显式传 `--stall-limit N`（例如 `--stall-limit 3`），此时连续 N 轮既没超过历史最佳分数、也没减少阻断项/失败维度才会标记 `blocked_no_progress`。 | ||
| 95 | + | ||
| 96 | +分数相同时，阻断项或失败维度减少也算有效进展。每轮的 `iteration_request.json` 会给出 `remaining_iterations` 和 `consecutive_non_improving_rounds`，agent 必须据此继续迭代，不能在 `needs_iteration` 时提前结束，也不能用无限重复同一候选来假装迭代。 | ||
| 97 | + | ||
| 98 | +达到 `blocked_max_iterations` 时，必须在收尾输出中写明：当前分数、每个未达标维度的分差、以及**为什么拆解仍不正确**的根因（缺哪类证据、哪个语义检查过不去），而不是只报一个分数。 | ||
| 99 | + | ||
| 100 | +## 5. 输出约定 | ||
| 101 | + | ||
| 102 | +| 文件 | 含义 | | ||
| 103 | +|---|---| | ||
| 104 | +| `breakdown_score.json` | 当前轮总分、分项分数、硬性否决项和修正动作 | | ||
| 105 | +| `semantic_review_request.json` / `semantic_review.json` | 当前候选的审查任务与带证据结论 | | ||
| 106 | +| `iteration_history.json` | 所有轮次的分数、是否提升、历史最佳配置快照 | | ||
| 107 | +| `iteration_request.json` | 下一轮只需处理的失败维度、阻断问题与约束 | | ||
| 108 | +| `iterations/iteration_N_analysis_config.json` | 第 N 轮候选配置快照 | | ||
| 109 | +| `iterations/iteration_N_semantic_review.json` | 第 N 轮语义审查快照 | | ||
| 110 | +| `iterations/iteration_N_validation_report.json` | 第 N 轮统一校验结果快照 | | ||
| 111 | +| `iterations/iteration_N_breakdown_score.json` | 第 N 轮评分快照 | | ||
| @@ -0,0 +1,103 @@ | |||
| 1 | +# Model Breakdown Scoring and Iteration Protocol | ||
| 2 | + | ||
| 3 | +Use this protocol for formal Mode A breakdowns. Scoring does not replace source review. It converts evidence already produced by source inspection, trace analysis, and the unified validator into a stable and comparable quality gate. | ||
| 4 | + | ||
| 5 | +## 1. Acceptance Conditions | ||
| 6 | + | ||
| 7 | +A breakdown may be converted into a formal downstream report only when all conditions below hold: | ||
| 8 | + | ||
| 9 | +1. Runnable-check correctness is at least 95%: `score / runnable_max >= 95%`. `score` remains on a nominal 100-point scale and must not be used alone as the gate. Checks that genuinely cannot run because inputs are absent leave `runnable_max`; the evidence-qualified conclusion explicitly reports the downgrade. | ||
| 10 | +2. **Every runnable core dimension** reaches its original minimum ratio: | ||
| 11 | + | ||
| 12 | + | Dimension | id | Maximum | Minimum | | ||
| 13 | + |---|---|---:|---:| | ||
| 14 | + | Architecture integrity | `architecture_integrity` | 25 | 22 | | ||
| 15 | + | Dataflow and branch correctness | `dataflow_branch_correctness` | 30 | 27 | | ||
| 16 | + | Layer/submodule boundaries | `layer_submodule_boundaries` | 20 | 18 | | ||
| 17 | + | Exact kernel coverage | `kernel_exact_coverage` | 20 | 20 | | ||
| 18 | + | Evidence and traceability | `evidence_traceability` | 5 | 4 | | ||
| 19 | + | ||
| 20 | + Both gates must operate independently: enforce each core ratio and the total correctness ratio separately. When a dimension is genuinely unrunnable because its input is absent, do not award full points and do not let it permanently block an otherwise correct breakdown. Kernel coverage and evidence are always runnable and retain their original thresholds. | ||
| 21 | + | ||
| 22 | + The former Shape/Semantic Consistency and Trace Instance/Scope dimensions were removed. Their 20 points moved to dataflow (+10), architecture (+5), and kernel coverage (+5). Missing `shape_semantic` and a single-step capture describe what evidence is present, not whether the breakdown is correct. Those dimensions rewarded evidence that could not be falsified. Their replacements are machine-checkable against source. | ||
| 23 | + | ||
| 24 | + Each `warning` deducts 20% of that check's weight (`WARNING_STEP`). The old quality set `{1.0, 0.6, 0.0}` made one warning cost 40%, turning thresholds such as 27/30 into a hidden zero-warning rule. One warning is now tolerable; many warnings are not. | ||
| 25 | +3. No hard gate is present. | ||
| 26 | +4. All nine `semantic_review.json` checks and its overall status are `passed`, and artifact SHA256 values are current. The sole exception is a `warning` evidence-gap finding on `source_model_identity`: it reports a missing scalar in the inputs, not a semantic error, and becomes non-blocking `info`. See the sole-exception section in `semantic_review_protocol_en.md`. The capture-tier mechanism below enforces its conclusion ceiling. | ||
| 27 | +5. `run_validation.py` status is `passed`. `passed_with_warnings` and `exploratory` are diagnostic only and cannot produce a formal report. Data-availability findings (`A1`, `MT1`, `MA1`, `SR_EVIDENCE_GAP_FINDING`) use `info`, because they report absent scalar evidence rather than a breakdown defect. Capture tier enforces the real ceiling: tier A removes the eight `source_model_identity` points from the denominator, so `runnable_max` is 92, below 95, and the conclusion is capped at `verified_unbound_scalars`. Using `warning` would make `GATE_VALIDATION` return `needs_iteration`, incorrectly treating absent evidence as a fixable breakdown error. | ||
| 28 | + | ||
| 29 | +The final `status` states what the evidence supports: `verified`, `verified_unbound_scalars`, `structure_unverified`, `exploratory`, or `needs_iteration`. The first three have `convertible=true`; `passed` exists only for pre-tier compatibility. Never lower these quality thresholds during iteration. | ||
| 30 | + | ||
| 31 | +## 2. The 100-Point Scale | ||
| 32 | + | ||
| 33 | +| Dimension | Points | Primary evidence | | ||
| 34 | +|---|---:|---| | ||
| 35 | +| Architecture integrity | 25 | architecture/regression plus `source_model_identity` and `module_inventory_complete` | | ||
| 36 | +| Dataflow and branch correctness | 30 | `check_dataflow.py` D1-D7, structure/sublayers, and semantic review of Q/K/V and residuals | | ||
| 37 | +| Layer/submodule boundaries | 20 | structure/coverage/sublayers plus layer, tail, runtime, and code-reference review | | ||
| 38 | +| Exact kernel coverage | 20 | Exact union of model/runtime/excluded for the representative step; missing, duplicate, out-of-range, and unmapped must all be zero | | ||
| 39 | +| Evidence and traceability | 5 | `source_of_truth`, `code_ref`, manifest `source_ref`, and `evidence_gaps` | | ||
| 40 | + | ||
| 41 | +Dataflow has the highest weight because it is both error-prone and deterministically checkable. `forward()` is the dataflow graph; source/config contradictions do not depend on AI self-reporting. | ||
| 42 | + | ||
| 43 | +`shape_semantic` and `trace_scope` may remain as optional annotations, but they do not affect scoring or the formal `run_validation.py` gate. | ||
| 44 | + | ||
| 45 | +`scripts/score_breakdown.py` computes scores only from structured artifacts and does not accept a manually supplied total. | ||
| 46 | + | ||
| 47 | +## 3. Hard Gates | ||
| 48 | + | ||
| 49 | +Any item below requires another breakdown even when the score is at least 95: | ||
| 50 | + | ||
| 51 | +- Unified validation is not `passed`. | ||
| 52 | +- `semantic_review.json` is missing or failed, or its artifact SHA256 values do not match current config/raw/manifest files. | ||
| 53 | +- `unmapped_ops` is non-empty, or coverage has nonzero unmapped, missing, duplicate, or out-of-range counts. | ||
| 54 | +- A migrated v1 result remains `legacy_unverified`. | ||
| 55 | +- Mode A lacks source truth or usable node `code_ref` values while claiming exact model attribution. | ||
| 56 | +- Validation found a learned-layer count, MTP invocation, Dense/MoE, Q/K/V, residual, or layer-boundary conflict. | ||
| 57 | +- `check_dataflow.py` emits error-level D1, D2, or D5: a source residual merge has no declared `branches`, branch direction is reversed and bypasses nothing, or a runtime-data-dependent source branch lacks an explicit `deviations` selection. These are direct source/config contradictions, not coverage or annotation issues. | ||
| 58 | + | ||
| 59 | +Without source, `check_dataflow` abstains. Absence is not a hard failure, but it is not validation evidence; formal Mode A requires source. | ||
| 60 | + | ||
| 61 | +Never clear a failure by deleting main-compute kernels, expanding `excluded_profiler_ops`, fabricating `source_ref`, merging independent branches, or lowering thresholds. | ||
| 62 | + | ||
| 63 | +## 4. Closed-Loop Iteration | ||
| 64 | + | ||
| 65 | +Execute every round in this order: | ||
| 66 | + | ||
| 67 | +1. Generate a candidate `analysis_config.json` from source, manifest, raw operators, and `dataflow_source.json`. | ||
| 68 | +2. Run enrichment and generate `semantic_review_request.json`. After complete source and trace review, fill `semantic_review.json`. | ||
| 69 | +3. Run `run_validation.py` and `score_breakdown.py`. | ||
| 70 | +4. Run report and metrics generation only after acceptance. | ||
| 71 | +5. Otherwise read `iteration_request.json`, revise only `blocking_issues`, `failed_dimensions`, and `required_actions`, and restart at step 2. | ||
| 72 | + | ||
| 73 | +Every `analysis_config.json` change requires a new semantic review. When the old review hash is stale, the driver stops at `awaiting_semantic_review`; the candidate is not recorded as a low-scoring round. | ||
| 74 | + | ||
| 75 | +The driver does not call AI to edit candidates. The agent executing this skill must consume `ai_mapping_request.json`, `semantic_review_request.json`, and `iteration_request.json` and continue mapping, review, and targeted correction until acceptance, a stop condition, or a genuine lack of new evidence. | ||
| 76 | + | ||
| 77 | +`run_breakdown.py` stores each round in `iterations/` and maintains `iteration_history.json`. Edit the candidate referenced by `iteration_request.json.base_config_for_revision`. Treat `immutable_best_snapshot` as read-only. A lower-scoring candidate never becomes the new baseline. | ||
| 78 | + | ||
| 79 | +### 4.1 Stop Conditions | ||
| 80 | + | ||
| 81 | +Only two stop conditions exist: | ||
| 82 | + | ||
| 83 | +1. **Accepted:** `breakdown_score.convertible == true`, meaning runnable correctness is at least 95%, every runnable core dimension reaches its ratio, no hard gate exists, and validation and semantic review both pass. Respect the evidence-qualified status; do not misclassify a nominal score below 95 for `verified_unbound_scalars` as requiring iteration. | ||
| 84 | +2. **Iteration limit:** `--max-iterations` rounds, default **10**, have been evaluated without acceptance; status becomes `blocked_max_iterations`. | ||
| 85 | + | ||
| 86 | +The driver no longer stops after two non-improving rounds by default. `--stall-limit` defaults to `0`, disabling early stop. To enable it, pass `--stall-limit N`, such as 3. Only N consecutive rounds that neither exceed the historical best score nor reduce hard gates/failed dimensions produce `blocked_no_progress`. | ||
| 87 | + | ||
| 88 | +Reducing hard gates or failed dimensions counts as progress even when the score is equal. The agent must use `remaining_iterations` and `consecutive_non_improving_rounds` from `iteration_request.json`; do not stop at `needs_iteration` and do not repeat the same candidate indefinitely. | ||
| 89 | + | ||
| 90 | +At `blocked_max_iterations`, report the current score, the deficit for every failed dimension, and the root reason the breakdown remains incorrect, including missing evidence or the semantic check that cannot pass. Do not report only a score. | ||
| 91 | + | ||
| 92 | +## 5. Outputs | ||
| 93 | + | ||
| 94 | +| File | Meaning | | ||
| 95 | +|---|---| | ||
| 96 | +| `breakdown_score.json` | Current total, dimension scores, hard gates, and corrective actions | | ||
| 97 | +| `semantic_review_request.json` / `semantic_review.json` | Review task and evidence-backed conclusions for the current candidate | | ||
| 98 | +| `iteration_history.json` | All round scores, improvement status, and best-config snapshots | | ||
| 99 | +| `iteration_request.json` | Failed dimensions, hard gates, constraints, and next-round work | | ||
| 100 | +| `iterations/iteration_N_analysis_config.json` | Candidate snapshot for round N | | ||
| 101 | +| `iterations/iteration_N_semantic_review.json` | Semantic-review snapshot for round N | | ||
| 102 | +| `iterations/iteration_N_validation_report.json` | Unified-validation snapshot for round N | | ||
| 103 | +| `iterations/iteration_N_breakdown_score.json` | Score snapshot for round N | | ||
| @@ -0,0 +1,77 @@ | |||
| 1 | +# 诊断码与 L1 建议对照表 | ||
| 2 | + | ||
| 3 | +本文是 `compute_metrics.py` 的 `findings[].code` 与建议文本的**唯一来源**。修改建议只改本文，不改代码。 | ||
| 4 | + | ||
| 5 | +## 定位 | ||
| 6 | + | ||
| 7 | +- 建议是**纯咨询信息**。它不进 `validation_report.json`、不进 `breakdown_score.json`、不影响 hard_gates、不影响 SKILL.md 的两个停止条件、不参与迭代循环。 | ||
| 8 | +- 建议只回答「下一步该看什么数据」，**不断言根因**。指标本身不足以证明根因，断言根因会把猜测伪装成结论。 | ||
| 9 | +- 建议不是优化方案。是否值得优化、怎么优化，取决于业务目标和硬件配置，不在本 skill 的判断范围内。 | ||
| 10 | + | ||
| 11 | +## 聚合作用域的硬性约束 | ||
| 12 | + | ||
| 13 | +`metrics_findings.json` 每条带 `metric_scope`。当 `metric_scope == "aggregate"` 时（多 invocation 合并统计）： | ||
| 14 | + | ||
| 15 | +- `wall_ms` 是各实例 wall 之和乘以 multiplier，可能**超过** step 总 wall（已实测到 `占比%` 达 400.0%）。 | ||
| 16 | +- 该作用域的 `gap_pct` / `utilization_pct` 只描述「该组总体」，**不能**推出任何单实例结论。 | ||
| 17 | +- 因此聚合节点的建议**必须**先降到 `instance` 作用域复核，再谈动作。所有 `advice_l1` 文本在 aggregate 作用域下自动追加前缀 `[聚合口径]`。 | ||
| 18 | + | ||
| 19 | +这条与 Skill 3 `app.js` 的 tooltip 免责声明（「表示总量，不代表单层热点或异常等级」）是同一条线，不得只在一处生效。 | ||
| 20 | + | ||
| 21 | +## ADVICE_TABLE | ||
| 22 | + | ||
| 23 | +以下每个 `## code:` 段被 `load_advice_table()` 解析。`next_data` 是必需字段，`not_applicable` 是必需字段。 | ||
| 24 | + | ||
| 25 | +## code: GAP_BUBBLE | ||
| 26 | + | ||
| 27 | +- **advice**: 该节点 wall 显著大于设备忙碌时间，空档不在本节点的 kernel 内部。先确认空档落在哪里：按 `start_time_us` 排序本节点 op，算相邻 op 的 `start[i+1] - (start[i]+duration[i])`，取最大的几段。 | ||
| 28 | +- **next_data**: `raw_ops_details.json` 本节点 `op_indices` 的 `start_time_us` / `duration_us`；`ASCEND_PROFILER_OUTPUT/trace_view.json` 对应时间窗；host 侧下发间隔看 `api_statistic.csv`。 | ||
| 29 | +- **not_applicable**: 节点 kernel 数 ≤ 2 时 gap 比例无统计意义；`metric_scope == "aggregate"` 时该值是跨实例累加的假空档。 | ||
| 30 | + | ||
| 31 | +## code: WAIT_DOMINANT | ||
| 32 | + | ||
| 33 | +- **advice**: `total_cost - kernel_sum` 占比高，说明成本主要在等待而非计算。先分类等待来源：按 `wait_time` 降序取本节点 top-5 kernel，看它们的前驱算子类型。前驱是集合通信算子则属通信等待；前驱是计算算子而 wait 仍高则看是否跨流依赖；kernel_sum 极小而 wait 极大通常是 host 下发或图执行边界，不是设备侧问题。 | ||
| 34 | +- **next_data**: `raw_ops_details.json` 的 `Wait Time(us)` 字段；通信带宽看 `ASCEND_PROFILER_OUTPUT/communication.json` 与 `communication_matrix.json`；host 下发看 `api_statistic.csv`。 | ||
| 35 | +- **not_applicable**: `kernel_sum_ms < 0.1` 时 wait 占比会放大到几百甚至上千倍，比例数值不可用于横向比较，只能作为「该节点几乎不计算」的信号。 | ||
| 36 | + | ||
| 37 | +## code: UTIL_LOW | ||
| 38 | + | ||
| 39 | +- **advice**: 忙碌时间占 wall 不足 80%。先区分是「本节点自身有空档」还是「本节点被上游拖慢」：若同时命中 GAP_BUBBLE，按 GAP_BUBBLE 处理；若未命中 GAP_BUBBLE 而利用率仍低，说明 wall 被少数长 kernel 拉长，看 `op_ratio` 里占比最高的算子类型。 | ||
| 40 | +- **next_data**: 本节点 `op_ratio`（Skill 2 的 `ui_facts/*_perf_data.json` 已含该字段）；对应算子的 `aic_mac_ratio` / `aiv_vec_ratio` / `aic_mte2_ratio` 判断是计算受限还是访存受限。 | ||
| 41 | +- **not_applicable**: 聚合作用域；以及节点只含通信算子时（通信节点的「利用率」语义不同，等待是其正常状态）。 | ||
| 42 | + | ||
| 43 | +## code: STREAM_PARALLEL_HIGH | ||
| 44 | + | ||
| 45 | +- **advice**: kernel 时长算术和显著超过 wall，说明多流重叠执行。这通常是**好现象**，不需要动作。仅在需要归因单个算子耗时时注意：此时不能用 `kernel_sum` 占比代表墙上时钟占比，两者口径不同。 | ||
| 46 | +- **next_data**: 若要确认重叠关系，看 `raw_ops_details.json` 的 `Stream ID` 分布与时间区间交叠。 | ||
| 47 | +- **not_applicable**: 不适用于以 wall 为口径的耗时排序场景。 | ||
| 48 | + | ||
| 49 | +## code: STREAM_PARALLEL_MID | ||
| 50 | + | ||
| 51 | +- **advice**: 存在中等程度多流重叠。同 STREAM_PARALLEL_HIGH，无需动作，注意口径差异即可。 | ||
| 52 | +- **next_data**: 同 STREAM_PARALLEL_HIGH。 | ||
| 53 | +- **not_applicable**: 同 STREAM_PARALLEL_HIGH。 | ||
| 54 | + | ||
| 55 | +## code: UTIL_GOOD | ||
| 56 | + | ||
| 57 | +- **advice**: 利用率在 80%~95%，无需动作。 | ||
| 58 | +- **next_data**: 无。 | ||
| 59 | +- **not_applicable**: 聚合作用域下该值不代表单实例利用率。 | ||
| 60 | + | ||
| 61 | +## code: CLEAN_SEQUENTIAL | ||
| 62 | + | ||
| 63 | +- **advice**: 四项指标彼此接近，属干净顺序执行，无空档无显著等待，无需动作。 | ||
| 64 | +- **next_data**: 无。 | ||
| 65 | +- **not_applicable**: 无。 | ||
| 66 | + | ||
| 67 | +## code: NORMAL | ||
| 68 | + | ||
| 69 | +- **advice**: 未命中任何异常阈值，无需动作。 | ||
| 70 | +- **next_data**: 无。 | ||
| 71 | +- **not_applicable**: 无。 | ||
| 72 | + | ||
| 73 | +## code: NO_DATA | ||
| 74 | + | ||
| 75 | +- **advice**: 该节点在代表 step 内没有 kernel。若源码中该模块应当执行，说明拆解的 op 归属可能有缺口，回到 Step 6 检查该节点的 `op_indices`；若该模块本轮确实未执行（config-gated 未选中的分支），属正常。 | ||
| 76 | +- **next_data**: `analysis_config.json` 该节点的 `op_indices`；`kernel_attribution.json`。 | ||
| 77 | +- **not_applicable**: 无。 | ||
| @@ -0,0 +1,77 @@ | |||
| 1 | +# Diagnostic Codes and L1 Advice | ||
| 2 | + | ||
| 3 | +This document is a read-only English translation of `diagnosis_advice.md`. The runtime `compute_metrics.py` loader reads the Chinese file, which is the **single source of truth** for `findings[].code` values and advice text. Make runtime changes there, then synchronize this translation. | ||
| 4 | + | ||
| 5 | +## Scope | ||
| 6 | + | ||
| 7 | +- Advice is purely informational. It does not enter `validation_report.json` or `breakdown_score.json`, affect hard gates or the two SKILL.md stop conditions, or participate in iteration. | ||
| 8 | +- Advice answers only "what data should be inspected next." Metrics alone cannot prove a root cause, so do not present a hypothesis as a conclusion. | ||
| 9 | +- Advice is not an optimization prescription. Whether and how to optimize depends on business goals and hardware configuration and is outside this skill's scope. | ||
| 10 | + | ||
| 11 | +## Mandatory Aggregate-Scope Constraint | ||
| 12 | + | ||
| 13 | +Every `metrics_findings.json` item has `metric_scope`. When `metric_scope == "aggregate"`, multiple invocations have been combined: | ||
| 14 | + | ||
| 15 | +- `wall_ms` is the sum of instance walls multiplied by the multiplier and may exceed the total step wall. Ratios of 400% have been observed. | ||
| 16 | +- `gap_pct` and `utilization_pct` describe only the aggregate group and imply nothing about any individual instance. | ||
| 17 | +- Therefore, advice for an aggregate node must first require verification at `instance` scope. Every aggregate `advice_l1` string automatically receives the runtime prefix `[聚合口径]` (aggregate scope). | ||
| 18 | + | ||
| 19 | +This is the same rule as the Skill 3 `app.js` tooltip disclaimer: aggregate totals do not identify a single-layer hotspot or severity. Keep both locations consistent. | ||
| 20 | + | ||
| 21 | +## ADVICE_TABLE | ||
| 22 | + | ||
| 23 | +In the runtime source `diagnosis_advice.md`, `load_advice_table()` parses each matching `## code:` section. This translation mirrors those sections. Both `next_data` and `not_applicable` are required. | ||
| 24 | + | ||
| 25 | +## code: GAP_BUBBLE | ||
| 26 | + | ||
| 27 | +- **advice**: Node wall time is substantially greater than device busy time, so the gap is outside the node's kernels. Locate it by sorting the node's operators by `start_time_us`, calculating `start[i+1] - (start[i] + duration[i])`, and inspecting the largest intervals. | ||
| 28 | +- **next_data**: `start_time_us` and `duration_us` for this node's `op_indices` in `raw_ops_details.json`; the corresponding interval in `ASCEND_PROFILER_OUTPUT/trace_view.json`; host launch intervals in `api_statistic.csv`. | ||
| 29 | +- **not_applicable**: Gap ratios are not meaningful when the node has two or fewer kernels. With `metric_scope == "aggregate"`, the value is a false cross-instance accumulated gap. | ||
| 30 | + | ||
| 31 | +## code: WAIT_DOMINANT | ||
| 32 | + | ||
| 33 | +- **advice**: A large `total_cost - kernel_sum` share means waiting dominates computation. Classify the wait by sorting kernels by `wait_time` and inspecting the predecessors of the top five. A collective predecessor indicates communication wait; a compute predecessor with high wait suggests a cross-stream dependency; very small kernel sum with very large wait usually indicates host launch or graph boundaries rather than device computation. | ||
| 34 | +- **next_data**: `Wait Time(us)` in `raw_ops_details.json`; `ASCEND_PROFILER_OUTPUT/communication.json` and `communication_matrix.json` for communication bandwidth; `api_statistic.csv` for host launch. | ||
| 35 | +- **not_applicable**: When `kernel_sum_ms < 0.1`, the wait ratio can expand to hundreds or thousands of percent. Do not compare the ratio across nodes; use it only as a signal that the node performs almost no computation. | ||
| 36 | + | ||
| 37 | +## code: UTIL_LOW | ||
| 38 | + | ||
| 39 | +- **advice**: Busy time is less than 80% of wall time. Distinguish a gap inside the node from upstream delay. If `GAP_BUBBLE` also applies, follow that advice. Otherwise inspect the highest-share operator type in `op_ratio`, because a small number of long kernels are stretching wall time. | ||
| 40 | +- **next_data**: This node's `op_ratio` in Skill 2 `ui_facts/*_perf_data.json`; use the relevant operator's `aic_mac_ratio`, `aiv_vec_ratio`, and `aic_mte2_ratio` to distinguish compute and memory limits. | ||
| 41 | +- **not_applicable**: Aggregate scope, and communication-only nodes where waiting is normal and utilization has different semantics. | ||
| 42 | + | ||
| 43 | +## code: STREAM_PARALLEL_HIGH | ||
| 44 | + | ||
| 45 | +- **advice**: The arithmetic sum of kernel durations substantially exceeds wall time, indicating overlapping execution on multiple streams. This is usually beneficial and needs no action. Do not use the `kernel_sum` share as a wall-clock share when attributing individual operator time; the units differ. | ||
| 46 | +- **next_data**: Inspect `Stream ID` distribution and overlapping intervals in `raw_ops_details.json` when overlap must be confirmed. | ||
| 47 | +- **not_applicable**: Wall-time ranking scenarios. | ||
| 48 | + | ||
| 49 | +## code: STREAM_PARALLEL_MID | ||
| 50 | + | ||
| 51 | +- **advice**: Moderate multi-stream overlap exists. As with `STREAM_PARALLEL_HIGH`, no action is required; preserve the distinction between arithmetic kernel sum and wall time. | ||
| 52 | +- **next_data**: Same as `STREAM_PARALLEL_HIGH`. | ||
| 53 | +- **not_applicable**: Same as `STREAM_PARALLEL_HIGH`. | ||
| 54 | + | ||
| 55 | +## code: UTIL_GOOD | ||
| 56 | + | ||
| 57 | +- **advice**: Utilization is between 80% and 95%; no action is required. | ||
| 58 | +- **next_data**: None. | ||
| 59 | +- **not_applicable**: At aggregate scope this value does not represent per-instance utilization. | ||
| 60 | + | ||
| 61 | +## code: CLEAN_SEQUENTIAL | ||
| 62 | + | ||
| 63 | +- **advice**: The four metrics are close, indicating clean sequential execution without meaningful gaps or waits. No action is required. | ||
| 64 | +- **next_data**: None. | ||
| 65 | +- **not_applicable**: None. | ||
| 66 | + | ||
| 67 | +## code: NORMAL | ||
| 68 | + | ||
| 69 | +- **advice**: No diagnostic threshold was met; no action is required. | ||
| 70 | +- **next_data**: None. | ||
| 71 | +- **not_applicable**: None. | ||
| 72 | + | ||
| 73 | +## code: NO_DATA | ||
| 74 | + | ||
| 75 | +- **advice**: The node has no kernels in the representative step. If source says the module should execute, operator ownership may be incomplete; return to Step 6 and inspect its `op_indices`. If the module was not selected in this run because of a configuration gate, this is expected. | ||
| 76 | +- **next_data**: The node's `op_indices` in `analysis_config.json`; `kernel_attribution.json`. | ||
| 77 | +- **not_applicable**: None. | ||
| @@ -53,8 +53,8 @@ This is the most granular device-side data. Each row is one kernel invocation on | |||
| 53 | | `Output Shapes` | string | — | e.g. `"[2048,4096]"` | | 53 | | `Output Shapes` | string | — | e.g. `"[2048,4096]"` | |
| 54 | | `Input Data Types` | string | — | e.g. `"FLOAT16;FLOAT16"` | | 54 | | `Input Data Types` | string | — | e.g. `"FLOAT16;FLOAT16"` | |
| 55 | | `Output Data Types` | string | — | e.g. `"FLOAT16"` | | 55 | | `Output Data Types` | string | — | e.g. `"FLOAT16"` | |
| 56 | -| `Input Formats` | string | — | e.g. `"ND;ND"` | | 56 | +| `Input Formats` | string | — | e.g. `"ND;ND"` <!-- codespell:ignore --> | |
| 57 | -| `Output Formats` | string | — | e.g. `"ND"` | | 57 | +| `Output Formats` | string | — | e.g. `"ND"` <!-- codespell:ignore --> | |
| 58 | 58 | ||
| 59 | ### 上下文 | 59 | ### 上下文 |
| 60 | 60 | ||
| @@ -0,0 +1,140 @@ | |||
| 1 | +# NPU Performance Data Guide | ||
| 2 | + | ||
| 3 | +## kernel_details.csv | ||
| 4 | + | ||
| 5 | +This is the most granular device-side data. Each row is one kernel invocation on the NPU. | ||
| 6 | + | ||
| 7 | +--- | ||
| 8 | + | ||
| 9 | +## Columns | ||
| 10 | + | ||
| 11 | +### Basic identifiers | ||
| 12 | + | ||
| 13 | +| Column | Type | Unit | Description | | ||
| 14 | +|---|---|---|---| | ||
| 15 | +| `Step Id` | integer | - | Training/inference step identifier | | ||
| 16 | +| `Device_id` | integer | - | NPU device identifier | | ||
| 17 | +| `Model ID` | integer | - | Model identifier | | ||
| 18 | +| `Task ID` | integer | - | Task identifier | | ||
| 19 | +| `Stream ID` | integer | - | Device stream this kernel ran on | | ||
| 20 | +| `Name` | string | - | Kernel or task name, e.g. `MatMul`, `Add`, `HcomAllReduce` | | ||
| 21 | +| `Type` | string | - | Kernel type name | | ||
| 22 | + | ||
| 23 | +### Execution state | ||
| 24 | + | ||
| 25 | +| Column | Type | Unit | Description | | ||
| 26 | +|---|---|---|---| | ||
| 27 | +| `OP State` | string | - | Operation execution state | | ||
| 28 | +| `Accelerator Core` | string | - | Core type used: `AiCore`, `AiCpu`, `AiVector`, `MixAic`, etc. | | ||
| 29 | + | ||
| 30 | +### Timing | ||
| 31 | + | ||
| 32 | +| Column | Type | Unit | Description | | ||
| 33 | +|---|---|---|---| | ||
| 34 | +| `Start Time(us)` | float | us | Absolute start timestamp on the device clock | | ||
| 35 | +| `Duration(us)` | float | us | Kernel execution time (device busy time) | | ||
| 36 | +| `Wait Time(us)` | float | us | Time the kernel waited before execution started | | ||
| 37 | + | ||
| 38 | +### Parallelism parameters | ||
| 39 | + | ||
| 40 | +| Column | Type | Unit | Description | | ||
| 41 | +|---|---|---|---| | ||
| 42 | +| `Block Dim` / `Block Num` | integer | - | Parallelism dimension | | ||
| 43 | +| `Mix Block Dim` / `Mix Block Num` | integer | - | Mixed parallelism dimension | | ||
| 44 | +| `HF32 Eligible` | string | - | HF32 eligibility flag | | ||
| 45 | + | ||
| 46 | +### Tensor information (optional) | ||
| 47 | + | ||
| 48 | +These fields depend on the `record_shapes` setting and may be empty. | ||
| 49 | + | ||
| 50 | +| Column | Type | Unit | Description | | ||
| 51 | +|---|---|---|---| | ||
| 52 | +| `Input Shapes` | string | - | e.g. `"[2048,4096];[4096,4096]"` | | ||
| 53 | +| `Output Shapes` | string | - | e.g. `"[2048,4096]"` | | ||
| 54 | +| `Input Data Types` | string | - | e.g. `"FLOAT16;FLOAT16"` | | ||
| 55 | +| `Output Data Types` | string | - | e.g. `"FLOAT16"` | | ||
| 56 | +| `Input Formats` | string | - | e.g. `"ND;ND"` <!-- codespell:ignore --> | | ||
| 57 | +| `Output Formats` | string | - | e.g. `"ND"` <!-- codespell:ignore --> | | ||
| 58 | + | ||
| 59 | +### Context | ||
| 60 | + | ||
| 61 | +| Column | Type | Unit | Description | | ||
| 62 | +|---|---|---|---| | ||
| 63 | +| `Context ID` | integer | - | Context identifier | | ||
| 64 | + | ||
| 65 | +### AI Core metrics | ||
| 66 | + | ||
| 67 | +| Column | Type | Unit | Description | | ||
| 68 | +|---|---|---|---| | ||
| 69 | +| `aicore_time(us)` | float | us | AI Core execution time | | ||
| 70 | +| `aic_total_cycles` | integer | - | AI Core total cycles | | ||
| 71 | + | ||
| 72 | +**Detailed AI Core metrics (optional, depending on profiling settings):** | ||
| 73 | + | ||
| 74 | +| Column | Type | Unit | Description | | ||
| 75 | +|---|---|---|---| | ||
| 76 | +| `aic_mac_time(us)` | float | us | MAC unit time | | ||
| 77 | +| `aic_mac_ratio` | float | - | MAC time ratio (0-1) | | ||
| 78 | +| `aic_scalar_time(us)` | float | us | Scalar unit time | | ||
| 79 | +| `aic_scalar_ratio` | float | - | Scalar time ratio (0-1) | | ||
| 80 | +| `aic_mte1_time(us)` | float | us | MTE1 (memory transfer engine 1) time | | ||
| 81 | +| `aic_mte1_ratio` | float | - | MTE1 time ratio (0-1) | | ||
| 82 | +| `aic_mte2_time(us)` | float | us | MTE2 time | | ||
| 83 | +| `aic_mte2_ratio` | float | - | MTE2 time ratio (0-1) | | ||
| 84 | +| `aic_fixpipe_time(us)` | float | us | FixPipe unit time | | ||
| 85 | +| `aic_fixpipe_ratio` | float | - | FixPipe time ratio (0-1) | | ||
| 86 | +| `aic_icache_miss_rate` | float | - | AI Core ICache miss rate | | ||
| 87 | +| `cube_utilization(%)` | float | % | Cube utilization percentage | | ||
| 88 | + | ||
| 89 | +**AI Core FLOPs:** | ||
| 90 | + | ||
| 91 | +| Column | Type | Unit | Description | | ||
| 92 | +|---|---|---|---| | ||
| 93 | +| `aic_mac_fp16_ratio` | float | - | FP16 MAC ratio | | ||
| 94 | +| `aic_mac_int8_ratio` | float | - | INT8 MAC ratio | | ||
| 95 | +| `aic_cube_fops` | integer | - | Cube FLOPs count | | ||
| 96 | + | ||
| 97 | +### AI Vector metrics | ||
| 98 | + | ||
| 99 | +| Column | Type | Unit | Description | | ||
| 100 | +|---|---|---|---| | ||
| 101 | +| `aiv_time(us)` | float | us | AI Vector execution time | | ||
| 102 | +| `aiv_total_cycles` | integer | - | AI Vector total cycles | | ||
| 103 | + | ||
| 104 | +**Detailed AI Vector metrics (optional, depending on profiling settings):** | ||
| 105 | + | ||
| 106 | +| Column | Type | Unit | Description | | ||
| 107 | +|---|---|---|---| | ||
| 108 | +| `aiv_vec_time(us)` | float | us | Vector unit time | | ||
| 109 | +| `aiv_vec_ratio` | float | - | Vector time ratio (0-1) | | ||
| 110 | +| `aiv_scalar_time(us)` | float | us | AI Vector scalar unit time | | ||
| 111 | +| `aiv_scalar_ratio` | float | - | AI Vector scalar time ratio (0-1) | | ||
| 112 | +| `aiv_mte2_time(us)` | float | us | AI Vector MTE2 time | | ||
| 113 | +| `aiv_mte2_ratio` | float | - | AI Vector MTE2 time ratio (0-1) | | ||
| 114 | +| `aiv_mte3_time(us)` | float | us | AI Vector MTE3 time | | ||
| 115 | +| `aiv_mte3_ratio` | float | - | AI Vector MTE3 time ratio (0-1) | | ||
| 116 | +| `aiv_icache_miss_rate` | float | - | AI Vector ICache miss rate | | ||
| 117 | + | ||
| 118 | +**AI Vector FLOPs:** | ||
| 119 | + | ||
| 120 | +| Column | Type | Unit | Description | | ||
| 121 | +|---|---|---|---| | ||
| 122 | +| `aiv_vec_fp32_ratio` | float | - | FP32 vector ratio | | ||
| 123 | +| `aiv_vec_fp16_ratio` | float | - | FP16 vector ratio | | ||
| 124 | +| `aiv_vec_int32_ratio` | float | - | INT32 vector ratio | | ||
| 125 | +| `aiv_vec_misc_ratio` | float | - | Miscellaneous vector ratio | | ||
| 126 | +| `aiv_vector_fops` | integer | - | Vector FLOPs count | | ||
| 127 | + | ||
| 128 | +--- | ||
| 129 | + | ||
| 130 | +## Task Type Values | ||
| 131 | + | ||
| 132 | +| Task Type | Description | | ||
| 133 | +|---|---| | ||
| 134 | +| `AI_CORE` | AI Core kernel | | ||
| 135 | +| `AI_CPU` | AI CPU kernel | | ||
| 136 | +| `HCCL` | Collective communication | | ||
| 137 | +| `MIX_AIC` | Mixed AI Core | | ||
| 138 | +| `MIX_AIV` | Mixed AI Vector | | ||
| 139 | +| `FFTS_PLUS` | FFTS Plus | | ||
| 140 | +| `DVPP` | Digital Vision Pre-Processing | | ||
| @@ -1,8 +1,6 @@ | |||
| 1 | # Mode C：仅性能数据，委托给 cann-npu-perfanalysis | 1 | # Mode C：仅性能数据，委托给 cann-npu-perfanalysis |
| 2 | 2 | ||
| 3 | -当工作目录中**只有性能数据而无模型源码**时，本 skill 进入 Mode C：委托给 sibling skill `cann-npu-perfanalysis` 做 8 维性能诊断（不做模型结构拆解）。 | 3 | +当工作目录中**只有性能数据而无模型源码**时，本 skill 进入 Mode C：委托给同一仓库的 `cann-npu-perfanalysis` skill 做 8 维性能诊断（不做模型结构拆解）。 |
| 4 | - | ||
| 5 | -> sibling skill 仓库：https://gitcode.com/jinyingqi/npu-perf-analysis | ||
| 6 | 4 | ||
| 7 | --- | 5 | --- |
| 8 | 6 | ||
| @@ -17,20 +15,13 @@ | |||
| 17 | 15 | ||
| 18 | ## 委托方式 | 16 | ## 委托方式 |
| 19 | 17 | ||
| 20 | -Mode C 不复用本 skill 的拆解流程。**spawn 一个 general-purpose subagent**，让其加载 `cann-npu-perfanalysis` 的 SKILL.md + references 后执行。 | 18 | +Mode C 不复用本 skill 的拆解流程。加载 `skills/cann-npu-perfanalysis/SKILL.md` 及其 references 后执行；需要隔离上下文时可交给 general-purpose subagent。 |
| 21 | 19 | ||
| 22 | ### 步骤 | 20 | ### 步骤 |
| 23 | 21 | ||
| 24 | -#### 1. 准备 sibling skill 本地副本 | 22 | +#### 1. 定位仓库内 skill |
| 25 | 23 | ||
| 26 | -首次运行时 clone（约 < 5 MB）： | 24 | +确认当前仓库存在 `skills/cann-npu-perfanalysis/SKILL.md`。如果缺失，直接报告依赖不完整，不在运行期间下载外部代码。 |
| 27 | - | ||
| 28 | -```bash | ||
| 29 | -mkdir -p .skills_cache | ||
| 30 | -git clone --depth 1 https://gitcode.com/jinyingqi/npu-perf-analysis .skills_cache/npu-perf-analysis | ||
| 31 | -``` | ||
| 32 | - | ||
| 33 | -后续运行如目录已存在则跳过。可选 `git -C .skills_cache/npu-perf-analysis pull` 拉取更新。 | ||
| 34 | 25 | ||
| 35 | #### 2. 拉起 subagent | 26 | #### 2. 拉起 subagent |
| 36 | 27 | ||
| @@ -40,8 +31,8 @@ git clone --depth 1 https://gitcode.com/jinyingqi/npu-perf-analysis .skills_cach | |||
| 40 | 你将作为 cann-npu-perfanalysis 技能的执行 agent，对以下 NPU profiling 数据做 8 维性能诊断。 | 31 | 你将作为 cann-npu-perfanalysis 技能的执行 agent，对以下 NPU profiling 数据做 8 维性能诊断。 |
| 41 | 32 | ||
| 42 | 技能定义： | 33 | 技能定义： |
| 43 | -- SKILL.md: .skills_cache/npu-perf-analysis/SKILL.md | 34 | +- SKILL.md: skills/cann-npu-perfanalysis/SKILL.md |
| 44 | -- 参考资料目录: .skills_cache/npu-perf-analysis/references/ | 35 | +- 参考资料目录: skills/cann-npu-perfanalysis/references/ |
| 45 | - data-schema.md # CSV/JSON column dictionary | 36 | - data-schema.md # CSV/JSON column dictionary |
| 46 | - metrics-formulas.md # Phase 1-2 公式 | 37 | - metrics-formulas.md # Phase 1-2 公式 |
| 47 | - thresholds.md # P0-P3 阈值 | 38 | - thresholds.md # P0-P3 阈值 |
| @@ -60,7 +51,7 @@ git clone --depth 1 https://gitcode.com/jinyingqi/npu-perf-analysis .skills_cach | |||
| 60 | 输出文件（写入调用方 outputs/ 目录）： | 51 | 输出文件（写入调用方 outputs/ 目录）： |
| 61 | - analysis_data.json # 8 维结构化诊断数据，schema 见 SKILL.md 第 255-424 行 | 52 | - analysis_data.json # 8 维结构化诊断数据，schema 见 SKILL.md 第 255-424 行 |
| 62 | - report.md # 人读报告，含 P0-P3 优先级瓶颈 | 53 | - report.md # 人读报告，含 P0-P3 优先级瓶颈 |
| 63 | -- report.html # 用 .skills_cache/npu-perf-analysis/references/generate_html.py 渲染 | 54 | +- report.html # 用 skills/cann-npu-perfanalysis/references/generate_html.py 渲染 |
| 64 | 55 | ||
| 65 | 返回：三文件路径 + 一句话顶层结论。 | 56 | 返回：三文件路径 + 一句话顶层结论。 |
| 66 | ``` | 57 | ``` |
| @@ -89,11 +80,11 @@ git clone --depth 1 https://gitcode.com/jinyingqi/npu-perf-analysis .skills_cach | |||
| 89 | 80 | ||
| 90 | --- | 81 | --- |
| 91 | 82 | ||
| 92 | -## 缓存策略 | 83 | +## 依赖策略 |
| 93 | 84 | ||
| 94 | -- `.skills_cache/npu-perf-analysis/` 提交到 `.gitignore` | 85 | +- 只使用随 `oam-tools` 一起评审和发布的 `skills/cann-npu-perfanalysis/` |
| 95 | -- 首次 clone 失败（网络问题）时给出明确错误：`Mode C 需要先 clone npu-perf-analysis：git clone https://gitcode.com/jinyingqi/npu-perf-analysis .skills_cache/npu-perf-analysis` | 86 | +- 不在 skill 运行期间 clone、pull 或执行外部仓库代码 |
| 96 | -- 不在 SKILL 启动时自动 clone，只在判定为 Mode C 时按需触发 | 87 | +- 依赖缺失时给出明确错误：`Mode C 需要仓库内的 skills/cann-npu-perfanalysis/SKILL.md` |
| 97 | 88 | ||
| 98 | --- | 89 | --- |
| 99 | 90 | ||
| @@ -0,0 +1,92 @@ | |||
| 1 | +# Mode C: Performance Data Only, Delegate to cann-npu-perfanalysis | ||
| 2 | + | ||
| 3 | +When the working directory contains **performance data but no model source**, enter Mode C and delegate to the sibling `cann-npu-perfanalysis` skill for eight-dimensional performance diagnosis. Do not perform model-structure breakdown. | ||
| 4 | + | ||
| 5 | +--- | ||
| 6 | + | ||
| 7 | +## Preconditions | ||
| 8 | + | ||
| 9 | +- `kernel_details.csv` or an `ASCEND_PROFILER_OUTPUT/` directory exists. | ||
| 10 | +- Model source (`*modeling*.py` or an equivalent entry point) does not exist. | ||
| 11 | + | ||
| 12 | +If model source also exists, use Mode A. If only source exists, use Mode B; see `mode_b_branches.md`. | ||
| 13 | + | ||
| 14 | +--- | ||
| 15 | + | ||
| 16 | +## Delegation | ||
| 17 | + | ||
| 18 | +Do not reuse this skill's breakdown workflow in Mode C. Load `skills/cann-npu-perfanalysis/SKILL.md` and its references, then execute that skill. Use a general-purpose subagent when context isolation is required. | ||
| 19 | + | ||
| 20 | +### 1. Locate the sibling skill | ||
| 21 | + | ||
| 22 | +Confirm that `skills/cann-npu-perfanalysis/SKILL.md` exists in the current repository. If it is missing, report the incomplete dependency. Do not download external code at runtime. | ||
| 23 | + | ||
| 24 | +### 2. Start the subagent | ||
| 25 | + | ||
| 26 | +Use the Agent tool with `subagent_type=general-purpose` and the following prompt template: | ||
| 27 | + | ||
| 28 | +```text | ||
| 29 | +Act as the execution agent for the cann-npu-perfanalysis skill and perform an eight-dimensional diagnosis of the following NPU profiling data. | ||
| 30 | + | ||
| 31 | +Skill definition: | ||
| 32 | +- SKILL.md: skills/cann-npu-perfanalysis/SKILL.md | ||
| 33 | +- Reference directory: skills/cann-npu-perfanalysis/references/ | ||
| 34 | + - data-schema.md # CSV/JSON column dictionary | ||
| 35 | + - metrics-formulas.md # Phase 1-2 formulas | ||
| 36 | + - thresholds.md # P0-P3 thresholds | ||
| 37 | + - hardware-specs.md # Peak TFLOPs by chip | ||
| 38 | + | ||
| 39 | +Input data: | ||
| 40 | +- profile directory: <ASCEND_PROFILER_OUTPUT_DIR> | ||
| 41 | + (contains kernel_details.csv, step_trace_time.csv, op_statistic.csv, | ||
| 42 | + communication.json, and communication_matrix.json) | ||
| 43 | + | ||
| 44 | +Requirements: | ||
| 45 | +1. Follow the Phase 0-4 workflow in SKILL.md exactly: parse, calculate metrics, diagnose bottlenecks, and emit outputs. | ||
| 46 | +2. Detect the V1/V2 schema automatically by checking for the cube_utilization(%) column. | ||
| 47 | +3. Skip dimensions 3/5/6/7 when their files are missing; skip dimension 7 when rows < 500. | ||
| 48 | +4. Obey every item in the SKILL.md NEVER list. | ||
| 49 | + | ||
| 50 | +Write these files to the caller's outputs/ directory: | ||
| 51 | +- analysis_data.json # eight-dimensional structured diagnosis; schema in SKILL.md lines 255-424 | ||
| 52 | +- report.md # human-readable report with P0-P3 bottlenecks | ||
| 53 | +- report.html # rendered by skills/cann-npu-perfanalysis/references/generate_html.py | ||
| 54 | + | ||
| 55 | +Return the three file paths and one sentence with the top-level conclusion. | ||
| 56 | +``` | ||
| 57 | + | ||
| 58 | +### 3. Process the result | ||
| 59 | + | ||
| 60 | +- On success, list `outputs/analysis_data.json`, `report.md`, and `report.html` as Mode C artifacts. | ||
| 61 | +- On failure, such as a missing required profile file, present the subagent error directly and do not fall back. | ||
| 62 | + | ||
| 63 | +--- | ||
| 64 | + | ||
| 65 | +## Relationship to This Skill's Outputs | ||
| 66 | + | ||
| 67 | +`cann-npu-perfanalysis` does not break down model structure. It produces performance diagnosis across iteration efficiency, operator hotspots, hardware utilization/MFU, communication efficiency, device bubbles, wait anchors, coarse hierarchy inference, and multi-device balance. | ||
| 68 | + | ||
| 69 | +| Dimension | This skill (Mode A) | cann-npu-perfanalysis (Mode C) | | ||
| 70 | +|---|---|---| | ||
| 71 | +| Model structure tree | Complete submodule-level breakdown | Only coarse MoE and layer-count inference | | ||
| 72 | +| Operator-to-source attribution | Exact alignment | Not provided | | ||
| 73 | +| Performance bottleneck diagnosis | Four metrics only | P0-P3 diagnosis | | ||
| 74 | +| Hardware utilization / MFU | Not provided | Provided | | ||
| 75 | +| Communication analysis | Not provided | Provided | | ||
| 76 | +| Multi-device balance | Not provided | Provided | | ||
| 77 | + | ||
| 78 | +The modes are complementary. **Mode C and Mode A use different output schemas and must not be compared with a strict diff.** | ||
| 79 | + | ||
| 80 | +--- | ||
| 81 | + | ||
| 82 | +## Dependency Policy | ||
| 83 | + | ||
| 84 | +- Use only `skills/cann-npu-perfanalysis/` reviewed and shipped with `oam-tools`. | ||
| 85 | +- Do not clone, pull, or execute code from an external repository at skill runtime. | ||
| 86 | +- When the dependency is missing, report: `Mode C requires skills/cann-npu-perfanalysis/SKILL.md in this repository`. | ||
| 87 | + | ||
| 88 | +--- | ||
| 89 | + | ||
| 90 | +## Outputs Not Produced | ||
| 91 | + | ||
| 92 | +Mode C does not produce `analysis_config.json`, `raw_ops*.json`, `{prefix}_report.md`, or `metrics_report.md`. Those are Mode A artifacts. Every Mode C artifact comes from the sibling skill. | ||
| @@ -0,0 +1,71 @@ | |||
| 1 | +# 模型拆解语义审查协议 | ||
| 2 | + | ||
| 3 | +确定性脚本能证明 schema、Kernel 覆盖和集合边界正确，但不能仅凭集合判断模型语义正确。正式拆解必须增加 `semantic_review.json`，由执行 Skill 的 AI 在阅读模型源码、配置和代表 step 后填写；脚本负责验证证据、文件版本和结论是否自洽。 | ||
| 4 | + | ||
| 5 | +## 强制审查项 | ||
| 6 | + | ||
| 7 | +以下九项必须逐项出现，不能合并或省略： | ||
| 8 | + | ||
| 9 | +| ID | 必须确认的内容 | | ||
| 10 | +|---|---| | ||
| 11 | +| `source_model_identity` | manifest、配置和所读源码属于同一模型与变体 | | ||
| 12 | +| `module_inventory_complete` | embedding、所有 decoder 类型、final norm、lm_head、预测模块等均完整 | | ||
| 13 | +| `dataflow_edges_complete` | 结构中的计算先后和输入输出边均与 forward 一致 | | ||
| 14 | +| `branch_topology_correct` | Q/K/V 等并行分支独立，未因相邻或同名 Kernel 错误合并 | | ||
| 15 | +| `residual_paths_correct` | attention/MLP 残差均有正确的两路输入和汇合位置 | | ||
| 16 | +| `layer_boundaries_correct` | Norm、跨层 fused op、每个 invocation 均归入正确层，不跨层抢占 | | ||
| 17 | +| `tail_stages_correct` | final norm、lm_head、sampling/runtime 等尾部阶段归类正确 | | ||
| 18 | +| `runtime_nodes_observed` | runtime 节点确实在本次 Trace 中出现，未按源码臆造执行节点 | | ||
| 19 | +| `code_refs_resolve` | 关键结论使用的源码引用均真实存在且行号有效 | | ||
| 20 | + | ||
| 21 | +## 证据规则 | ||
| 22 | + | ||
| 23 | +- 每个 `passed` 项必须至少有一条 `evidence`。 | ||
| 24 | +- 每条证据必须有 `explanation`，并至少提供 `source_ref`、`config_path`、`op_indices` 之一。 | ||
| 25 | +- 分支、残差、层边界结论应同时使用源码引用与配置路径或 op 索引，不能只写自由文本。 | ||
| 26 | +- `source_evidence` 至少列出一个实际阅读过的 forward/config 源码片段。 | ||
| 27 | +- 无法确认时写 `unknown`；发现错误时写 `failed` 并加入 finding。禁止在证据不足时写 `passed`。 | ||
| 28 | +- 任一 `failed`/`unknown` 检查，或任一 `error` finding，都会阻断正式评分；`warning` finding 同样阻断， | ||
| 29 | + 唯一例外见下。 | ||
| 30 | + | ||
| 31 | +### 唯一例外：证据缺口 finding | ||
| 32 | + | ||
| 33 | +`source_model_identity` 上的 `warning` finding 报的是**输入不含该标量的证据**（典型情形：采集里没有 | ||
| 34 | +checkpoint `config.json`，层数只能取 Python 默认参数），而不是九项语义结论中的任何一项有错。按代码优先 | ||
| 35 | +规则，源码是架构真值、trace 不裁定标量，所以它记为证据缺口：验证器发出 `SR_EVIDENCE_GAP_FINDING` | ||
| 36 | +（severity `info`，与 `A1`/`MT1`/`MA1` 同组），不阻断，`detail.evidence_gap_findings` 列出其 id。 | ||
| 37 | + | ||
| 38 | +**这不是放宽门槛**：未绑定 checkpoint 的上限由 `score_breakdown.py` 的 capture_tier 强制——tier A 把 | ||
| 39 | +`source_model_identity` 的 8 分移出分母（`runnable_max` 92 < `MIN_TOTAL_SCORE` 95），结论因此封顶在 | ||
| 40 | +`verified_unbound_scalars`，拿不到 `verified`。把它发成 `warning` 反而会让顶层状态变成 | ||
| 41 | +`passed_with_warnings` 并被 `GATE_VALIDATION` 判为 `needs_iteration`，即"拆解有错待修"——与事实相反。 | ||
| 42 | + | ||
| 43 | +例外只覆盖这一个检查上的 `warning`：其他任何检查上的 `warning`，以及这个检查上的 `error`，照旧阻断。 | ||
| 44 | + | ||
| 45 | +## 文件绑定与迭代 | ||
| 46 | + | ||
| 47 | +`semantic_review.json.artifacts` 保存 `analysis_config.json`、`raw_ops.json`、`model_manifest.json` 的 SHA256。任何一个文件变化，旧审查立即失效。修改拆解配置后必须重新生成审查请求、重新阅读受影响源码和 Trace，再生成新的审查文件。 | ||
| 48 | + | ||
| 49 | +先生成请求： | ||
| 50 | + | ||
| 51 | +```bash | ||
| 52 | +python scripts/prepare_semantic_review.py \ | ||
| 53 | + -c outputs/analysis_config.json \ | ||
| 54 | + -r outputs/raw_ops.json \ | ||
| 55 | + -m outputs/model_manifest.json \ | ||
| 56 | + --source-dir models/<model> \ | ||
| 57 | + -o outputs/semantic_review_request.json | ||
| 58 | +``` | ||
| 59 | + | ||
| 60 | +AI 按请求中的 `review_template` 填写并保存为 `semantic_review.json`，然后验证： | ||
| 61 | + | ||
| 62 | +```bash | ||
| 63 | +python scripts/validate_semantic_review.py \ | ||
| 64 | + -s outputs/semantic_review.json \ | ||
| 65 | + -c outputs/analysis_config.json \ | ||
| 66 | + -r outputs/raw_ops.json \ | ||
| 67 | + -m outputs/model_manifest.json \ | ||
| 68 | + --source-dir models/<model> | ||
| 69 | +``` | ||
| 70 | + | ||
| 71 | +Kernel 100% 覆盖只表示每个 Kernel 有 owner，不表示 Q/K/V、残差或层边界正确；语义审查通过后才允许正式评分和报告生成。 | ||
| @@ -0,0 +1,64 @@ | |||
| 1 | +# Model Breakdown Semantic Review Protocol | ||
| 2 | + | ||
| 3 | +Deterministic scripts can prove schema validity, kernel coverage, and set boundaries, but those sets alone cannot prove semantic correctness. Every formal breakdown must include `semantic_review.json`, completed by the AI executing the skill after reading model source, configuration, and the representative step. Scripts validate evidence, artifact versions, and internal consistency. | ||
| 4 | + | ||
| 5 | +## Required Review Items | ||
| 6 | + | ||
| 7 | +All nine items must appear separately. Do not merge or omit them. | ||
| 8 | + | ||
| 9 | +| ID | Required conclusion | | ||
| 10 | +|---|---| | ||
| 11 | +| `source_model_identity` | The manifest, configuration, and reviewed source describe the same model and variant. | | ||
| 12 | +| `module_inventory_complete` | Embedding, every decoder type, final norm, lm_head, prediction modules, and other required modules are complete. | | ||
| 13 | +| `dataflow_edges_complete` | Computation order and input/output edges match `forward()`. | | ||
| 14 | +| `branch_topology_correct` | Parallel branches such as Q/K/V remain independent and were not merged because kernels were adjacent or shared names. | | ||
| 15 | +| `residual_paths_correct` | Attention and MLP residuals have the correct two inputs and merge locations. | | ||
| 16 | +| `layer_boundaries_correct` | Norms, cross-layer fused operations, and every invocation belong to the correct layer without cross-layer capture. | | ||
| 17 | +| `tail_stages_correct` | Final norm, lm_head, sampling, runtime, and other tail stages are classified correctly. | | ||
| 18 | +| `runtime_nodes_observed` | Runtime nodes actually occur in this trace and were not invented from source alone. | | ||
| 19 | +| `code_refs_resolve` | Source references supporting key conclusions exist and their line numbers resolve. | | ||
| 20 | + | ||
| 21 | +## Evidence Rules | ||
| 22 | + | ||
| 23 | +- Every `passed` item must contain at least one `evidence` entry. | ||
| 24 | +- Every evidence entry must include an `explanation` and at least one of `source_ref`, `config_path`, or `op_indices`. | ||
| 25 | +- Branch, residual, and layer-boundary conclusions should cite both source and a configuration path or operator indices. Free-form prose alone is insufficient. | ||
| 26 | +- `source_evidence` must list at least one forward/config source fragment that was actually read. | ||
| 27 | +- Use `unknown` when a conclusion cannot be established. Use `failed` and add a finding when an error is found. Never use `passed` without sufficient evidence. | ||
| 28 | +- Any `failed` or `unknown` check, and any `error` finding, blocks formal scoring. A `warning` finding also blocks, with the sole exception below. | ||
| 29 | + | ||
| 30 | +### Sole Exception: Evidence-Gap Findings | ||
| 31 | + | ||
| 32 | +A `warning` finding on `source_model_identity` can report that the inputs do not contain evidence for a scalar, rather than that any semantic conclusion is wrong. A typical case is a capture without checkpoint `config.json`, where layer count can only come from a Python default. Under the code-first rule, source is the architecture truth and the trace does not adjudicate scalars. Record this as an evidence gap: the validator emits `SR_EVIDENCE_GAP_FINDING` with `severity: info`, groups it with `A1`/`MT1`/`MA1`, does not block, and lists its id in `detail.evidence_gap_findings`. | ||
| 33 | + | ||
| 34 | +This does **not** lower the gate. The capture-tier mechanism in `score_breakdown.py` enforces the ceiling for an unbound checkpoint. Tier A removes the eight `source_model_identity` points from the denominator (`runnable_max` is 92, below `MIN_TOTAL_SCORE` 95), so the conclusion is capped at `verified_unbound_scalars` and cannot become `verified`. Emitting a normal warning would instead make the top-level status `passed_with_warnings`, causing `GATE_VALIDATION` to classify the candidate as `needs_iteration`, incorrectly turning missing input evidence into a breakdown defect. | ||
| 35 | + | ||
| 36 | +The exception covers only a `warning` on this one check. A warning on any other check, or an `error` on this check, still blocks. | ||
| 37 | + | ||
| 38 | +## Artifact Binding and Iteration | ||
| 39 | + | ||
| 40 | +`semantic_review.json.artifacts` stores SHA256 values for `analysis_config.json`, `raw_ops.json`, and `model_manifest.json`. A change to any file invalidates the previous review immediately. After changing the breakdown configuration, regenerate the request, reread the affected source and trace, and create a new review. | ||
| 41 | + | ||
| 42 | +Generate the request first: | ||
| 43 | + | ||
| 44 | +```bash | ||
| 45 | +python scripts/prepare_semantic_review.py \ | ||
| 46 | + -c outputs/analysis_config.json \ | ||
| 47 | + -r outputs/raw_ops.json \ | ||
| 48 | + -m outputs/model_manifest.json \ | ||
| 49 | + --source-dir models/<model> \ | ||
| 50 | + -o outputs/semantic_review_request.json | ||
| 51 | +``` | ||
| 52 | + | ||
| 53 | +Fill the request's `review_template`, save it as `semantic_review.json`, and validate it: | ||
| 54 | + | ||
| 55 | +```bash | ||
| 56 | +python scripts/validate_semantic_review.py \ | ||
| 57 | + -s outputs/semantic_review.json \ | ||
| 58 | + -c outputs/analysis_config.json \ | ||
| 59 | + -r outputs/raw_ops.json \ | ||
| 60 | + -m outputs/model_manifest.json \ | ||
| 61 | + --source-dir models/<model> | ||
| 62 | +``` | ||
| 63 | + | ||
| 64 | +One hundred percent kernel coverage proves only that each kernel has an owner. It does not prove Q/K/V topology, residual paths, or layer boundaries. Formal scoring and report generation are allowed only after semantic review passes. | ||