草稿
[WIP] Stage-5 knowledge base, pl-native engine skills, and cann-bench targeting for pypto-pro-op-orchestrator #585
hid22900163创建于 15 天前
[WIP] Stage-5 knowledge base, pl-native engine skills, and cann-bench targeting for pypto-pro-op-orchestrator #585
草稿
共 73 个文件变更+11062-231
| @@ -657,7 +657,7 @@ if pto_kernels is not None and pto_kernels.USE_ACL_GRAPH: | |||
| 657 | 657 | ||
| 658 | **⚠️ 单算子替换性能预期：** | 658 | **⚠️ 单算子替换性能预期：** |
| 659 | 659 | ||
| 660 | -当只替换单个独立算子（如 RMSNorm）而未做前后融合时，**PTO 模式通常比基线慢 2-3x**。原因：(1) JIT kernel 首次编译；（2）kernel launch + dtype cast 开销 > 原 CANN 融合 kernel；（3）单算子无法抵消调用开销。 | 660 | +当只替换单个独立算子（如 RMSNorm）而未做前后融合时，**PTO 模式通常比基线慢 2-3x**（实测数据见 `references/directory_structure.md`）。原因：(1) JIT kernel 首次编译；（2）kernel launch + dtype cast 开销 > 原 CANN 融合 kernel；（3）单算子无法抵消调用开销。 |
| 661 | 661 | ||
| 662 | **这属于正常现象**，不是 bug。RMSNorm 的收益来自与相邻算子的融合（如 pre-attn RMSNorm + QKV projection），单算子替换仅验证**路由逻辑正确性和精度一致性**。真正的性能提升见后续 round：融合 pre-attn、post-attn 等复合算子。 | 662 | **这属于正常现象**，不是 bug。RMSNorm 的收益来自与相邻算子的融合（如 pre-attn RMSNorm + QKV projection），单算子替换仅验证**路由逻辑正确性和精度一致性**。真正的性能提升见后续 round：融合 pre-attn、post-attn 等复合算子。 |
| 663 | 663 | ||
| @@ -185,7 +185,7 @@ README 末尾追加性能对比表格和复现命令，方便用户 run： | |||
| 185 | | baseline | `python3 scripts/ask_{model_name}.py --prompt "你好" --device <id> --output-length 50` | 7.4s | 3.1s | 9.7 tok/s | 7313 MB | | 185 | | baseline | `python3 scripts/ask_{model_name}.py --prompt "你好" --device <id> --output-length 50` | 7.4s | 3.1s | 9.7 tok/s | 7313 MB | |
| 186 | | pto | `python3 scripts/ask_{model_name}.py --prompt "你好" --device <id> --output-length 50 --use-pto` | 7.0s | 10.0s | 3.0 tok/s | 7313 MB | | 186 | | pto | `python3 scripts/ask_{model_name}.py --prompt "你好" --device <id> --output-length 50 --use-pto` | 7.0s | 10.0s | 3.0 tok/s | 7313 MB | |
| 187 | 187 | ||
| 188 | -> 单算子替换时 PTO 比基线慢 2-3x 属正常（JIT 首编 + kernel launch 开销），收益来自多算子融合。 | 188 | +> 单算子替换时 PTO 比基线慢 2-3x 属正常（上表为实测；JIT 首编 + kernel launch 开销），收益来自多算子融合。 |
| 189 | ``` | 189 | ``` |
| 190 | 190 | ||
| 191 | > 表格数据从 `--report-file` 输出的 JSON 中提取，命令与表格一一对应。如被替换算子名不同或使用 `--use-acl-graph`，应收录对应命令和数据行。 | 191 | > 表格数据从 `--report-file` 输出的 JSON 中提取，命令与表格一一对应。如被替换算子名不同或使用 `--use-acl-graph`，应收录对应命令和数据行。 |
| @@ -1,6 +1,6 @@ | |||
| 1 | --- | 1 | --- |
| 2 | name: pypto-pro-environment-check | 2 | name: pypto-pro-environment-check |
| 3 | -description: PyPTO-Pro 环境检测与反馈技能。当 PyPTO-Pro 工作流（Stage 1–4）任意阶段遇到疑似环境问题（软件/硬件）时统一加载：torch_npu / pypto_pro 导入失败、npu-smi 无响应、NPU 设备不可见或不可用、CANN 未配置、kernel 编译/运行超时疑似设备 hang 等。采用「事实验证优先、脚本诊断兜底」两步法 + 设备 hang 三段式评定，用于区分环境故障与算子故障；只执行有界检测并返回结构化证据，不安装依赖、不修改环境。触发词：环境检查、环境问题、环境验证、environment check、设备 hang、卡死、超时。 | 3 | +description: PyPTO-Pro 环境检测与反馈技能。当 PyPTO-Pro 工作流（Stage 1–5）任意阶段遇到疑似环境问题（软件/硬件）时统一加载：torch_npu / pypto_pro 导入失败、npu-smi 无响应、NPU 设备不可见或不可用、CANN 未配置、kernel 编译/运行超时疑似设备 hang 等。采用「事实验证优先、脚本诊断兜底」两步法 + 设备 hang 三段式评定，用于区分环境故障与算子故障；只执行有界检测并返回结构化证据，不安装依赖、不修改环境。触发词：环境检查、环境问题、环境验证、environment check、设备 hang、卡死、超时。 |
| 4 | --- | 4 | --- |
| 5 | 5 | ||
| 6 | # PyPTO-Pro 环境检测（pypto-pro-environment-check） | 6 | # PyPTO-Pro 环境检测（pypto-pro-environment-check） |
| @@ -16,12 +16,13 @@ description: 设计 PyPTO-Pro 算子的 tile 级执行方案。当 SPEC、Golden | |||
| 16 | ## 两条实现约束（设计阶段须落实） | 16 | ## 两条实现约束（设计阶段须落实） |
| 17 | 17 | ||
| 18 | > 完整定义与证据门槛见`$CANNBOT_CONFIG_ROOT/references/performance-constraints.md`。进入 R0 前必须读取；设计阶段须在 R3（地址分配）和 R1（API 映射）中落实： | 18 | > 完整定义与证据门槛见`$CANNBOT_CONFIG_ROOT/references/performance-constraints.md`。进入 R0 前必须读取；设计阶段须在 R3（地址分配）和 R1（API 映射）中落实： |
| 19 | -> 1. 所有需要 buffer 切换/轮转的 tile 一律用 `make_tile_group` + `auto_mutex`，`make_tile` 仅限单次使用 scratch tile。手动同步的适用范围见 `pypto-pro-material-explore` SKILL「实现选择规则」节。R3 落实 buffer 管理方式，R6 落实 cross_core 同步方案。 | 19 | +> 1. 所有需要 buffer 切换/轮转的 tile 一律用 `make_tile_group` + `auto_mutex`，`make_tile` 仅限单次使用 scratch tile。自动排序须显式传 `mutex_ids=`——只给 `depth=` 而无可用 `mutex_ids` 的 group 轮转正常但**不携带任何排序信息**，且 `make_tile_group` 对未知/拼错的关键字（如 `auto_mutex=`、`mutex_id=`）静默丢弃、无任何告警（源码阅读结论，CANN 9.2.0 安装态解析器，2026-08-25，非上机实测；见 [`sync-stitch.md`](../pypto-pro-op-kb/constraints/sync-stitch.md)）。手动同步的适用范围见 `pypto-pro-material-explore` SKILL「实现选择规则」节。R3 落实 buffer 管理方式，R6 落实 cross_core 同步方案。 |
| 20 | > 2. Vector 选择按该规范写入 DESIGN.md §1：已选 KB 模板明确要求当前步骤使用 `pl.*` 时按模板，否则使用 `vf.*`；本阶段不运行候选实验。 | 20 | > 2. Vector 选择按该规范写入 DESIGN.md §1：已选 KB 模板明确要求当前步骤使用 `pl.*` 时按模板，否则使用 `vf.*`；本阶段不运行候选实验。 |
| 21 | 21 | ||
| 22 | ## 知识库 | 22 | ## 知识库 |
| 23 | 23 | ||
| 24 | -知识来源与读取边界见下方「结构化 Binding 流程（architect）」。 | 24 | +知识来源与读取边界见下方「结构化 Binding 流程（architect）」；按 |
| 25 | +[`pypto-pro-op-kb/ROUTER.md`](../pypto-pro-op-kb/ROUTER.md) 每次只读取一个与当前决策相关的参考。 | ||
| 25 | [pattern selector](../pypto-pro-op-kb/patterns/pattern-index.md) 仅用于确认已选 pattern 的验证 | 26 | [pattern selector](../pypto-pro-op-kb/patterns/pattern-index.md) 仅用于确认已选 pattern 的验证 |
| 26 | 状态：`validated skeleton` 可作为代码起点，`conceptual only` 只能用于推导，不得据此扩选。 | 27 | 状态：`validated skeleton` 可作为代码起点，`conceptual only` 只能用于推导，不得据此扩选。 |
| 27 | 28 | ||
| @@ -32,6 +33,16 @@ description: 设计 PyPTO-Pro 算子的 tile 级执行方案。当 SPEC、Golden | |||
| 32 | > 不要凭页面描述照写。 | 33 | > 不要凭页面描述照写。 |
| 33 | 平台专属约束必须先探测目标平台。API 文档与当前环境的官方样例仍是签名和行为的权威来源。 | 34 | 平台专属约束必须先探测目标平台。API 文档与当前环境的官方样例仍是签名和行为的权威来源。 |
| 34 | 35 | ||
| 36 | +**在划分 Phase / Section 结构之前先查 pattern selector**：可复用的 pl 数据流骨架就在 | ||
| 37 | +[`pypto-pro-op-kb/patterns/`](../pypto-pro-op-kb/patterns/pattern-index.md)，选好再设计比设计完再对照便宜。 | ||
| 38 | + | ||
| 39 | +⚠️ **Pro 流程只允许单一 `@pl.jit` kernel**，所以除 `Validation` 列之外，还须看 pattern 页标题 | ||
| 40 | +下方是否带 `Pro: not legal` banner。带该 banner 的页（当前无此类页面）所描述的多 kernel 形态属于「知识 | ||
| 41 | +成立、Pro 设计非法」——产出这样的 DESIGN.md 只会在 `stage4-check` #5 被拦下并浪费一轮迭代；只取 | ||
| 42 | +其融合单 kernel 子形态（同一个 `@pl.jit` 内多个 section）。机器可读的清单是 | ||
| 43 | +[`pypto-pro-op-kb/topology-map.json`](../pypto-pro-op-kb/topology-map.json) 的 `target_gated`；pattern selector 本身只标注 | ||
| 44 | +validation 状态，不含合法性列，不要从中推读。 | ||
| 45 | + | ||
| 35 | ## 输入 | 46 | ## 输入 |
| 36 | 47 | ||
| 37 | | 来源 | 路径 | 用途 | | 48 | | 来源 | 路径 | 用途 | |
| @@ -163,7 +174,7 @@ EXPLORE_REPORT.md §3只用于提供候选映射。逐项核对API参考页后 | |||
| 163 | 174 | ||
| 164 | 1. 按 `target_memory` 把 R2 的 tile 分组，**每个内存空间各自从 `0x00000` 开始**连续排列地址，不重叠。UB/L1 首地址须 32 字节对齐；L0A/L0B/L0C 的对齐以对应 API 文档 / 官方指定算子为准。 | 175 | 1. 按 `target_memory` 把 R2 的 tile 分组，**每个内存空间各自从 `0x00000` 开始**连续排列地址，不重叠。UB/L1 首地址须 32 字节对齐；L0A/L0B/L0C 的对齐以对应 API 文档 / 官方指定算子为准。 |
| 165 | 2. 标注同地址不同 layout 的 tile 对（如有） | 176 | 2. 标注同地址不同 layout 的 tile 对（如有） |
| 166 | -3. 分配方式应使用 `make_tile_group` + `auto_mutex`：TileGroup提供多槽buffer，代码通过`next()`或显式下标选择槽位，`auto_mutex`根据mutex信息管理执行域内部的跨Pipe依赖（见上方「实现约束」）。 | 177 | +3. 分配方式应使用 `make_tile_group` + `auto_mutex`：TileGroup提供多槽buffer，代码通过`next()`或显式下标选择槽位，`auto_mutex`根据mutex信息管理执行域内部的跨Pipe依赖（见上方「实现约束」）。mutex 信息须以显式 `mutex_ids=` 传入：只给 `depth=` 而无可用 `mutex_ids` 的 group 轮转正常但不携带任何排序信息，拼错的关键字被静默丢弃（源码阅读结论，CANN 9.2.0，2026-08-25，非上机实测；见 [`sync-stitch.md`](../pypto-pro-op-kb/constraints/sync-stitch.md)）。 |
| 167 | 4. **逐空间**验证该空间上的 tile 总大小不超过其容量上限。容量值以 EXPLORE_REPORT §7 探测记录为准——§7 必含 UB 容量；含 cube 时须补探 L1/L0 各空间容量（§7 未记录则回退 material-explore 补测，不得在此臆测数值） | 178 | 4. **逐空间**验证该空间上的 tile 总大小不超过其容量上限。容量值以 EXPLORE_REPORT §7 探测记录为准——§7 必含 UB 容量；含 cube 时须补探 L1/L0 各空间容量（§7 未记录则回退 material-explore 补测，不得在此臆测数值） |
| 168 | 5. **double buffer 地址规划**：`make_tile_group` 的 buffer 数 > 1 时地址占用按倍数放大，须在地址表中显式反映（buffer 数、受影响 tile、是否需 PONG 地址）。buffer 数取值参照官方指定算子中相似算子的实际配置 | 179 | 5. **double buffer 地址规划**：`make_tile_group` 的 buffer 数 > 1 时地址占用按倍数放大，须在地址表中显式反映（buffer 数、受影响 tile、是否需 PONG 地址）。buffer 数取值参照官方指定算子中相似算子的实际配置 |
| 169 | 180 | ||
| @@ -192,6 +203,7 @@ EXPLORE_REPORT.md §3只用于提供候选映射。逐项核对API参考页后 | |||
| 192 | 203 | ||
| 193 | **输出**：填入模板 §4： | 204 | **输出**：填入模板 §4： |
| 194 | - 参考样例路径与可复用结构点 | 205 | - 参考样例路径与可复用结构点 |
| 206 | +- **工作区参考实现对照表（架构冻结前必填）**：对 PRO_MATERIAL_INDEX §B / EXPLORE_REPORT §4 及 dispatch/prompt 点名的**每个**工作区内同类参考实现各填一行——归约/遍历形态（如单遍寄存器累积 vs 多遍整 tile）、每元素字节预算（B/elem，按该参考的搬运量估算）、缓冲方案，以及本设计是否偏离该参考、偏离理由。任一点名参考的形态与本设计不同而无偏离理由，R8 不得通过；工作区无点名参考时填「无点名参考实现」 | ||
| 195 | - 本算子的Section代码结构、结果单元、跨Tile状态生命周期、动态循环上界、各Module内的循环嵌套和分核信息的获取位置 | 207 | - 本算子的Section代码结构、结果单元、跨Tile状态生命周期、动态循环上界、各Module内的循环嵌套和分核信息的获取位置 |
| 196 | - `is_fusion=true`时补充第一阶段每次交给下一阶段的数据范围、产生下一份数据的循环索引、`task_id`递增位置、交替阶段链、候选预加载轮数、逐阶段delay计算表，以及启动、稳定运行和末尾剩余阶段的执行时序；使用上下文缓冲时补充字段、深度和索引 | 208 | - `is_fusion=true`时补充第一阶段每次交给下一阶段的数据范围、产生下一份数据的循环索引、`task_id`递增位置、交替阶段链、候选预加载轮数、逐阶段delay计算表，以及启动、稳定运行和末尾剩余阶段的执行时序；使用上下文缓冲时补充字段、深度和索引 |
| 197 | - 引用`loop_design.md`，说明采用的Section和循环组织方式 | 209 | - 引用`loop_design.md`，说明采用的Section和循环组织方式 |
| @@ -309,6 +321,7 @@ EXPLORE_REPORT §4中的官方指定算子用于核对完整调用方式，不 | |||
| 309 | | | cross_core 同步方案是否正确（存在跨执行域或跨Block/subblock依赖时）：手动预加载流水的阶段延迟、上下文槽位、就绪/释放事件、初始释放事件、同步点和event_id是否参照权威文档与当前实现 | 回到 R4/R6 修正 | | 321 | | | cross_core 同步方案是否正确（存在跨执行域或跨Block/subblock依赖时）：手动预加载流水的阶段延迟、上下文槽位、就绪/释放事件、初始释放事件、同步点和event_id是否参照权威文档与当前实现 | 回到 R4/R6 修正 | |
| 310 | | **一致性** | R0-R7 各轮输出是否存在矛盾（如 API 需要的 tile 在 R2 中缺失） | 回溯到矛盾产生的轮次修正 | | 322 | | **一致性** | R0-R7 各轮输出是否存在矛盾（如 API 需要的 tile 在 R2 中缺失） | 回溯到矛盾产生的轮次修正 | |
| 311 | | | 证据链是否完整（每个决策都有来源） | 补充缺失的文档引用或官方指定算子路径 | | 323 | | | 证据链是否完整（每个决策都有来源） | 补充缺失的文档引用或官方指定算子路径 | |
| 324 | +| | 工作区参考实现对照表完整（每个点名参考一行：归约/遍历形态、B/elem、缓冲方案、是否偏离及理由；偏离项均有理由） | 回到 R4 补齐对照表 / 重审偏离决策 | | ||
| 312 | | | 每个内存空间（UB/L1/L0A/L0B/L0C）的 tile 总用量分别不超过各自容量上限（R3 逐空间验证，含 cube 时须查 L1/L0） | 回到 R3 重排地址 / R2 缩 tile | | 325 | | | 每个内存空间（UB/L1/L0A/L0B/L0C）的 tile 总用量分别不超过各自容量上限（R3 逐空间验证，含 cube 时须查 L1/L0） | 回到 R3 重排地址 / R2 缩 tile | |
| 313 | | **条件性检查** | 若 §6 填“不涉及cross_core”，确认不存在Cube↔Vector或跨Block/subblock的数据依赖 | 回到 R0 重新评估 | | 326 | | **条件性检查** | 若 §6 填“不涉及cross_core”，确认不存在Cube↔Vector或跨Block/subblock的数据依赖 | 回到 R0 重新评估 | |
| 314 | 327 | ||
| @@ -193,6 +193,14 @@ SCALE = 1.0 / sqrt({D_logical}) # 缩放因子（若算子有 scale 步骤） | |||
| 193 | - **可复用结构点**: {该样例中可复用的Section / 循环 / 分核信息获取方式} | 193 | - **可复用结构点**: {该样例中可复用的Section / 循环 / 分核信息获取方式} |
| 194 | - **补充参考**（如有）: {`../pypto-pro-material-explore/references/official_samples.md` 清单中其他参考样例路径及参考点} | 194 | - **补充参考**（如有）: {`../pypto-pro-material-explore/references/official_samples.md` 清单中其他参考样例路径及参考点} |
| 195 | 195 | ||
| 196 | +### 工作区参考实现对照表（架构冻结前必填） | ||
| 197 | + | ||
| 198 | +> 对 PRO_MATERIAL_INDEX §B / EXPLORE_REPORT §4 及 dispatch/prompt 点名的**每个**工作区内同类参考实现各填一行；无点名参考时填「无点名参考实现」。任一参考的形态与本设计不同而「偏离理由」为空，§9 评估不得通过。 | ||
| 199 | + | ||
| 200 | +| 参考实现（路径） | 归约/遍历形态 | 每元素字节预算（B/elem） | 缓冲方案 | 本设计是否偏离 | 偏离理由 | | ||
| 201 | +|---|---|---|---|---|---| | ||
| 202 | +| {路径} | {如 单遍寄存器累积 / 多遍整 tile 归约 / 逐行流式} | {估算值及口径} | {单缓冲 / double buffer / 槽位数} | {否 / 是} | {偏离时必填：证据或量化依据} | | ||
| 203 | + | ||
| 196 | ### 本算子结构说明 | 204 | ### 本算子结构说明 |
| 197 | 205 | ||
| 198 | - **Module / Section对应关系**: {每个Module放在哪个Section；相邻同域Module是否共用一个Section} | 206 | - **Module / Section对应关系**: {每个Module放在哪个Section；相邻同域Module是否共用一个Section} |
| @@ -205,6 +213,14 @@ SCALE = 1.0 / sqrt({D_logical}) # 缩放因子（若算子有 scale 步骤） | |||
| 205 | 213 | ||
| 206 | > 同步点见 §6（R6），尾块处理见 §7（R7）。 | 214 | > 同步点见 §6（R6），尾块处理见 §7（R7）。 |
| 207 | 215 | ||
| 216 | +### 小 shape 固定开销策略（case 锚点填充 fixed-overhead / launch cost 子格时必填） | ||
| 217 | + | ||
| 218 | +> **条件性**：仅当 case 锚点填充 one-tile-per-core（parity）regime 格的 **fixed-overhead / launch cost** 子格时必填；未填充则本节填“不涉及”。判定即该行的几何无关触发问题（原文见 [dominant-cost-attack-matrix](../../pypto-pro-op-kb/playbooks/dominant-cost-attack-matrix.md) 的 fixed-overhead / launch cost 行；regime 格见 [capacity-and-regime-method](../../pypto-pro-op-kb/references/capacity-and-regime-method.md)）：在最大合法并行度下，总工作量相对核数是否小到 `t_hw` 与该卡**实测**的单次启动固定成本相当，且被标记子格占 Σ R_i 的实质份额（R_i 可忽略的极小 case 不触发）。单次启动固定成本是卡特定值，任何 µs 字面量都不得替代实测；该卡未实测时子格按 **PROVISIONAL** 标记处理，arch-a5 恒定几何扫描即该行的首个探针——不得仅因扫描未做而填“不涉及”。几何（tile 尺寸、分核）变更后须回到本节重新判定。 | ||
| 219 | + | ||
| 220 | +- **触发算术**: {逐项引用：锚点 case 的 `t_hw`、最大合法并行度下的工作单元数与核数、被标记子格的 Σ R_i 份额；结论：填充 / 未填充 / PROVISIONAL（注明该卡单次启动成本未实测）} | ||
| 221 | +- **小 shape 策略**: {dispatch（host 侧按 tilingkey 分发）/ rung（编译期档位特化）/ straight-line（直线化）的选择及理由} | ||
| 222 | +- **规避的单次启动成本**: {所选策略避免的是哪一项 per-launch 固定成本，如设备侧运行时分支、逐行档位判定、标量序言} | ||
| 223 | + | ||
| 208 | ### CV 手动预加载流水设计（`is_fusion=true`时必填） | 224 | ### CV 手动预加载流水设计（`is_fusion=true`时必填） |
| 209 | 225 | ||
| 210 | > 依据[CV融合算子手动预加载流水设计](../references/cv_fusion_pipeline.md)。先确定第一阶段每次交给下一阶段的数据范围，以及哪些循环索引会产生下一份数据，再为这些数据分配连续编号，并展示稳定运行时Cube与Vector同时处理不同编号数据的时序。 | 226 | > 依据[CV融合算子手动预加载流水设计](../references/cv_fusion_pipeline.md)。先确定第一阶段每次交给下一阶段的数据范围，以及哪些循环索引会产生下一份数据，再为这些数据分配连续编号，并展示稳定运行时Cube与Vector同时处理不同编号数据的时序。 |
| @@ -345,6 +361,7 @@ pl.set_validshape(tile_a, [valid_m, valid_n]) # 运行时告知硬件 | |||
| 345 | |--------|------|------| | 361 | |--------|------|------| |
| 346 | | R0-R7 输出无矛盾 | ✅ / ❌ | {交叉验证} | | 362 | | R0-R7 输出无矛盾 | ✅ / ❌ | {交叉验证} | |
| 347 | | 所有决策有证据支撑 | ✅ / ❌ | {证据链检查} | | 363 | | 所有决策有证据支撑 | ✅ / ❌ | {证据链检查} | |
| 364 | +| 工作区参考实现对照表完整，偏离项均有理由（见 §4） | ✅ / ❌ | {回 R4 补齐对照表 / 重审偏离决策} | | ||
| 348 | | 各内存空间（UB/L1/L0A/L0B/L0C）tile 总用量分别不超各自容量上限（R3 逐空间验证，含 cube 时须查 L1/L0） | ✅ / ❌ | {回 R3 重排地址 / R2 缩 tile} | | 365 | | 各内存空间（UB/L1/L0A/L0B/L0C）tile 总用量分别不超各自容量上限（R3 逐空间验证，含 cube 时须查 L1/L0） | ✅ / ❌ | {回 R3 重排地址 / R2 缩 tile} | |
| 349 | | `tile_dims` 使用时已关注大 stride 对性能的影响 | ✅ / ❌ | {回 R2 调整布局} | | 366 | | `tile_dims` 使用时已关注大 stride 对性能的影响 | ✅ / ❌ | {回 R2 调整布局} | |
| 350 | | 条件性检查（若 §6 填“不涉及cross_core”，确认不存在Cube↔Vector或跨Block/subblock的数据依赖） | ✅ / ❌ | {R0 重新评估} | | 367 | | 条件性检查（若 §6 填“不涉及cross_core”，确认不存在Cube↔Vector或跨Block/subblock的数据依赖） | ✅ / ❌ | {R0 重新评估} | |
| @@ -17,6 +17,7 @@ description: 实现、调试并自验证 PyPTO-Pro 算子 kernel。用于按冻 | |||
| 17 | - `custom/<op>/EXPLORE_REPORT.md`：已核对的 API 约束和相似样例。 | 17 | - `custom/<op>/EXPLORE_REPORT.md`：已核对的 API 约束和相似样例。 |
| 18 | - `custom/<op>/PRO_MATERIAL_INDEX.md`：需要回看原文时的 API、官方样例和教程路径。 | 18 | - `custom/<op>/PRO_MATERIAL_INDEX.md`：需要回看原文时的 API、官方样例和教程路径。 |
| 19 | - flat 的 `custom/<op>/KB_SELECTION.json`，或 split 布局下全部 `custom/<op>/<class>/KB_SELECTION.json`，以及其中选中的所有参考。 | 19 | - flat 的 `custom/<op>/KB_SELECTION.json`，或 split 布局下全部 `custom/<op>/<class>/KB_SELECTION.json`，以及其中选中的所有参考。 |
| 20 | +- `custom/<op>/DESIGN_BINDINGS.json`：Stage 3 冻结的 Knowledge Bindings；其中每条 `obligation + applies` 活动 requirement 都是本阶段义务，逐条落实规则见下方「KB usage 规范（Coder/Verifier 共用）」。 | ||
| 20 | 21 | ||
| 21 | 按 dispatch 交付： | 22 | 按 dispatch 交付： |
| 22 | 23 | ||
| @@ -32,7 +33,7 @@ description: 实现、调试并自验证 PyPTO-Pro 算子 kernel。用于按冻 | |||
| 32 | 33 | ||
| 33 | 1. 一个交付文件只含一个 `@pl.jit` kernel，核心计算全部在该 kernel 内；wrapper 只启动一次 kernel，且启动不在 host 循环内。 | 34 | 1. 一个交付文件只含一个 `@pl.jit` kernel，核心计算全部在该 kernel 内；wrapper 只启动一次 kernel，且启动不在 host 循环内。 |
| 34 | 2. 严格执行 DESIGN.md 已冻结的 Module 边界、API 序列、tile 属性/地址、循环、同步、尾块和 `vector_selection`。本轮功能或精度测试表明设计有误时上报疑似 `design_violation`，不得静默改设计。 | 35 | 2. 严格执行 DESIGN.md 已冻结的 Module 边界、API 序列、tile 属性/地址、循环、同步、尾块和 `vector_selection`。本轮功能或精度测试表明设计有误时上报疑似 `design_violation`，不得静默改设计。 |
| 35 | -3. 轮转 tile 使用 `make_tile_group` + `auto_mutex`；`make_tile` 只用于不参与轮转的单次 scratch。不要在 `auto_mutex` 管理的 tile 上叠加手动 `sync_src`/`sync_dst`。 | 36 | +3. 轮转 tile 使用 `make_tile_group` + `auto_mutex`；`make_tile` 只用于不参与轮转的单次 scratch。跨算子依赖的轮转 group 必须显式传 `mutex_ids=`——只给 `depth=` 而无可用 `mutex_ids` 的 group 轮转正常但不携带任何排序信息，且 `make_tile_group` 静默丢弃未知/拼错的关键字（源码阅读结论，CANN 9.2.0，2026-08-25，非上机实测；见 [sync-stitch.md](../pypto-pro-op-kb/constraints/sync-stitch.md)）。不要在 `auto_mutex` 管理的 tile 上叠加手动 `sync_src`/`sync_dst`。 |
| 36 | 4. wrapper 只做参数检查、读取 KB 约束列明的只读元数据、纯 Python 整数推导、`torch.empty` 分配当前 wrapper 合同声明的输出和一次 kernel 启动；完整边界与迁移方式见 [wrapper-boundary.md](../pypto-pro-op-kb/constraints/wrapper-boundary.md)。DESIGN、usage、`deviated` 或 profile 均不能放宽该硬约束。 | 37 | 4. wrapper 只做参数检查、读取 KB 约束列明的只读元数据、纯 Python 整数推导、`torch.empty` 分配当前 wrapper 合同声明的输出和一次 kernel 启动；完整边界与迁移方式见 [wrapper-boundary.md](../pypto-pro-op-kb/constraints/wrapper-boundary.md)。DESIGN、usage、`deviated` 或 profile 均不能放宽该硬约束。 |
| 37 | 5. 测试通过 wrapper 调用 kernel；不得删改 DESIGN.md §8 的 case 来迁就实现，也不得把核心计算移到测试或 host 代码。 | 38 | 5. 测试通过 wrapper 调用 kernel；不得删改 DESIGN.md §8 的 case 来迁就实现，也不得把核心计算移到测试或 host 代码。 |
| 38 | 39 | ||
| @@ -52,6 +53,7 @@ description: 实现、调试并自验证 PyPTO-Pro 算子 kernel。用于按冻 | |||
| 52 | | [templates/fp32-chain-precision-fragments.py.tmpl](templates/fp32-chain-precision-fragments.py.tmpl) | fp32 链路的精度安全片段 | 需要与 CPU 参考逐位对齐时 | | 53 | | [templates/fp32-chain-precision-fragments.py.tmpl](templates/fp32-chain-precision-fragments.py.tmpl) | fp32 链路的精度安全片段 | 需要与 CPU 参考逐位对齐时 | |
| 53 | | [templates/kb-usage-template.json](templates/kb-usage-template.json) | `KB_USAGE.json` 字段骨架与单条记录示例 | 写入或核验 usage 时 | | 54 | | [templates/kb-usage-template.json](templates/kb-usage-template.json) | `KB_USAGE.json` 字段骨架与单条记录示例 | 写入或核验 usage 时 | |
| 54 | | [scripts/list_idle_chip_ids.sh](scripts/list_idle_chip_ids.sh) | 查找空闲 NPU chip | 运行前按需执行 | | 55 | | [scripts/list_idle_chip_ids.sh](scripts/list_idle_chip_ids.sh) | 查找空闲 NPU chip | 运行前按需执行 | |
| 56 | +| [pypto-pro-op-perf-tune](../pypto-pro-op-perf-tune/SKILL.md) | wrapper 动态反作弊检查：只检查 profile 中的 device op，不评价性能 | 需要验证 wrapper 未绕过 kernel 时 | | ||
| 55 | | [KB CONTRACT](../pypto-pro-op-kb/CONTRACT.md) | KB JSON 的基础字段、路径与状态词表 | 读取 selection 或写 usage 前 | | 57 | | [KB CONTRACT](../pypto-pro-op-kb/CONTRACT.md) | KB JSON 的基础字段、路径与状态词表 | 读取 selection 或写 usage 前 | |
| 56 | 58 | ||
| 57 | 参考之间是互补关系：DESIGN.md 决定“实现什么”，纯 Vector 模板或 Cube 官方样例提供“如何写”的主要起点，`KB_SELECTION.json` 已选参考补充必须落实的 pattern 和约束；三者不得相互替代。模板和样例不是 API 或性能事实源，使用时仍须核对目标版本 API 文档；与 DESIGN.md 或已选 KB 冲突的参考片段直接弃用，只有上游合同本身无法同时落实时才按根因分流，本 skill 不自行改合同。标为 conceptual 的片段不得直接复制成交付代码。 | 59 | 参考之间是互补关系：DESIGN.md 决定“实现什么”，纯 Vector 模板或 Cube 官方样例提供“如何写”的主要起点，`KB_SELECTION.json` 已选参考补充必须落实的 pattern 和约束；三者不得相互替代。模板和样例不是 API 或性能事实源，使用时仍须核对目标版本 API 文档；与 DESIGN.md 或已选 KB 冲突的参考片段直接弃用，只有上游合同本身无法同时落实时才按根因分流，本 skill 不自行改合同。标为 conceptual 的片段不得直接复制成交付代码。 |
| @@ -86,6 +88,8 @@ L1 的 `suffix_k` 是累积序号：1 → `1`，2 → `12`，3 → `123`。文 | |||
| 86 | - §8：至少 4 个目标测试 case； | 88 | - §8：至少 4 个目标测试 case； |
| 87 | - §10：完整数据流。 | 89 | - §10：完整数据流。 |
| 88 | 90 | ||
| 91 | +§5 的分核方案要有平台事实垫底。本次运行探测到的 `vector_core_num`、`core_num` 与 `torch.npu.get_device_name(...)` 设备名，必须在 **Stage 3 开始之前**就写进运行报告 `RUN_REPORT.md`（运行编排提供该报告时；本流程内的等价落点是 `custom/<op>/MEMORY.md`）：同代 A5 存在核数不同的部件，全部真实，判别依据是设备名而非 `soc_version`（见 [arch-a5.md](../pypto-pro-op-kb/constraints/arch-a5.md)），没有这条记录的设计等于在无事实依据下选定了启动宽度。编码前确认记录在案；不在案就先探测一次并补记（命令、原始输出、`date -u +%FT%TZ` 时间戳齐全）再继续，并在交付报告里把这个缺口告知编排器。该记录只是口径校验，不是取值来源：kernel 的启动宽度仍然只能在运行时从 `get_platform_info()` 读取，把探测到的数字抄成字面量与抄表格同罪。DESIGN.md §5 冻结的核数与探测值不一致时停止编码，按 `design_violation` 返回。 | ||
| 92 | + | ||
| 89 | L1 还要用 `module_interfaces.yaml` 核对当前 Module 的 `inputs`、`outputs`、`golden_steps` 和 section 类型。缺少关键合同，或上下游产物互相矛盾时停止编码并上报疑似 `design_violation`。 | 93 | L1 还要用 `module_interfaces.yaml` 核对当前 Module 的 `inputs`、`outputs`、`golden_steps` 和 section 类型。缺少关键合同，或上下游产物互相矛盾时停止编码并上报疑似 `design_violation`。 |
| 90 | 94 | ||
| 91 | ### 3. 核对全部 API | 95 | ### 3. 核对全部 API |
| @@ -150,6 +154,8 @@ TensorList（`is_list: true`）同样只能启动一次 kernel，且启动不得 | |||
| 150 | 154 | ||
| 151 | - **验证范围**：L1 Module 只检 DESIGN/Module 合同分配给当前 Module，或当前 staged 实际承载的 active 及相关 validation scope，不提前检后续 Module；L0/finalize 检全部 active 与 scope。 | 155 | - **验证范围**：L1 Module 只检 DESIGN/Module 合同分配给当前 Module，或当前 staged 实际承载的 active 及相关 validation scope，不提前检后续 Module；L0/finalize 检全部 active 与 scope。 |
| 152 | - **验证方法**：不得改变验证目标，输入就绪即执行；仅 L1 Module 可在原方法明确依赖尚未生成的最终 file/symbol、wrapper 或 final-only profile 时暂缓，并在本轮返回证据中记录四元组、原方法、缺失依赖和 `not_run_for_staged`。L0/finalize 不得暂缓。 | 156 | - **验证方法**：不得改变验证目标，输入就绪即执行；仅 L1 Module 可在原方法明确依赖尚未生成的最终 file/symbol、wrapper 或 final-only profile 时暂缓，并在本轮返回证据中记录四元组、原方法、缺失依赖和 `not_run_for_staged`。L0/finalize 不得暂缓。 |
| 157 | +- **谓词有效性**：任何用于证明落实的验证方法都必须能证伪被守卫的那个缺陷——**缺陷成立时也能通过的方法无效**。「该常量只出现一次」「常量旁有来源注释」不检查取值来源；「符号存在」「文件里出现了这个 API」不检查写法。方法要落在缺陷写法与正确写法真正分岔的位置：来源类义务查取值链能否逐跳回溯到那次运行时查询，写法类义务查调用节点的结构位置。冻结的 `verification_method` 本身属于此类时是 requirement 语义错误：上报疑似 `design_violation`，附四元组和一份能通过该方法的反例写法，不得照抄执行。 | ||
| 158 | +- **结构谓词（向量交付必检）**：凡交付含 `@pl.vector_function` / `section_vector` 的实现，无论 Binding 写了什么，还须逐条实跑两条结构谓词并留证：**P-A** 到达 `block_dim` 的核数必须能逐跳回溯到一次 `get_platform_info()` 调用（允许 `min(cores, tasks)` 类钳制；字面量只允许出现在 `getattr` 兜底位；不得用裸字面量 grep 代替——`LANES = 64` 一类是正确的无关字面量）；**P-B** 任何 `vf.update_mask(...)` 调用节点不得位于 `pl.range` 循环体内（形态与代价见 [vec-alignment-and-rotation.md](../pypto-pro-op-kb/constraints/vec-alignment-and-rotation.md) 的提升节）。命中即 FAIL。 | ||
| 153 | - **证据复用**：仅被检实现、验证输入、方法、检查内容和覆盖范围均未变化时复用；仍按四元组记录原方法、命令/输入、原始结果和结论。已存在或本轮写入的 usage 均用 `json.load` 预检；L0/finalize 的各 class usage 必须存在。 | 159 | - **证据复用**：仅被检实现、验证输入、方法、检查内容和覆盖范围均未变化时复用；仍按四元组记录原方法、命令/输入、原始结果和结论。已存在或本轮写入的 usage 均用 `json.load` 预检；L0/finalize 的各 class usage 必须存在。 |
| 154 | 160 | ||
| 155 | ### 7. 运行并调试 | 161 | ### 7. 运行并调试 |
| @@ -189,4 +195,5 @@ python custom/<op>/modules/test_<op>_module<suffix_k>.py | |||
| 189 | - wrapper 的完整 host 调用链符合实现合同 #4。 | 195 | - wrapper 的完整 host 调用链符合实现合同 #4。 |
| 190 | - DESIGN.md §8 全部 case 已实际运行并通过；测试数据、dtype、shape、value range 未被偷换。 | 196 | - DESIGN.md §8 全部 case 已实际运行并通过；测试数据、dtype、shape、value range 未被偷换。 |
| 191 | - dev-only import 位于函数内，交付态模块可安全导入。 | 197 | - dev-only import 位于函数内，交付态模块可安全导入。 |
| 198 | +- 启动宽度逐跳回溯到 `get_platform_info()`，字面量只出现在 getattr 兜底位；平台探测记录（`vector_core_num` + 设备名）在案。 | ||
| 192 | - 返回运行命令、逐四元组方法证据、原始结果、产物路径和分类 verdict；不声称 verifier PASS。若无需修改，明确说明并附本模式全部产物的检查证据，不得空返回或只给笼统结论。 | 199 | - 返回运行命令、逐四元组方法证据、原始结果、产物路径和分类 verdict；不声称 verifier PASS。若无需修改，明确说明并附本模式全部产物的检查证据，不得空返回或只给笼统结论。 |
| @@ -135,7 +135,7 @@ past two full cycles), plus M/N tails, multi-tile reuse and batch. Anything less | |||
| 135 | leaves a boundary the rotation never crossed. | 135 | leaves a boundary the rotation never crossed. |
| 136 | 136 | ||
| 137 | **Performance stays a per-operator profiler decision.** A double-buffered | 137 | **Performance stays a per-operator profiler decision.** A double-buffered |
| 138 | -rotation is worth roughly 1.2-1.7x on kernel-only medians for a mid-size | 138 | +rotation measures roughly 1.2-1.7x on kernel-only medians for a mid-size |
| 139 | geometry, but the spread across shapes is wider than the mean, and a wider K/N | 139 | geometry, but the spread across shapes is wider than the mean, and a wider K/N |
| 140 | tile is rejected whenever its *ranking* is unstable across shapes rather than | 140 | tile is rejected whenever its *ranking* is unstable across shapes rather than |
| 141 | when its mean is worse -- an unstable ranking means the next shape decides the | 141 | when its mean is worse -- an unstable ranking means the next shape decides the |
| @@ -85,6 +85,32 @@ pattern must describe a distinct design decision for the class; a merely similar | |||
| 85 | an unmet precondition, duplicate guidance or generic background reading is not selectable. | 85 | an unmet precondition, duplicate guidance or generic background reading is not selectable. |
| 86 | Constraints remain independent of pattern selection and are never suppressed. | 86 | Constraints remain independent of pattern selection and are never suppressed. |
| 87 | 87 | ||
| 88 | +## Property keys | ||
| 89 | + | ||
| 90 | +`topology-map.json`'s `contract.property_keys` is the enum a selection's `properties` block | ||
| 91 | +draws from, and `property_modifiers` routes on a subset of it. Every key is defined in prose | ||
| 92 | +here as well as in the map: `long_axis` in particular routes both a constraint and a pattern, so | ||
| 93 | +two planners reading the bare enum could disagree about when it holds and both be inside the | ||
| 94 | +contract. | ||
| 95 | + | ||
| 96 | +Each key below is stated as the predicate that makes it true, together with the boundary that | ||
| 97 | +decides it. For most keys that boundary is a **judgement call**; record the reasoning for a | ||
| 98 | +borderline call in the selection artifact rather than resolving it silently. | ||
| 99 | + | ||
| 100 | +| Key | Holds when | Boundary | | ||
| 101 | +|---|---|---| | ||
| 102 | +| `dtypes` | The set of input/output element types the class must serve. Always recorded. | None; read it off the declared contract. | | ||
| 103 | +| `ranks` | The set of input ranks the one kernel must serve. Always recorded. | None; read it off the declared contract. | | ||
| 104 | +| `unaligned_shapes` | Some declared shape leaves an axis that is not a whole multiple of the tile or burst granule the design uses on that axis. | Depends on the tile geometry, so it is only decidable **after** a geometry is proposed. Re-check it if the geometry changes. | | ||
| 105 | +| `tail_blocks` | Work items do not divide evenly across the chosen work-item count, so at least one item is partial and needs a runtime valid window. | Same dependency on geometry as `unaligned_shapes`; the two usually hold together and are not the same statement. | | ||
| 106 | +| `long_axis` | One axis **exceeds what a single legal tile can hold for its dtype**, so the kernel must traverse it in more than one tile and carry state across the traversal. It is the property counterpart of the `long-axis` topology: the topology says the *computation* is shaped that way, the property says *these declared shapes* reach it. | The threshold is the widest tile the UB budget allows for the narrowest dtype in `dtypes` — a computed number, not a constant. The operational test used by the routed pattern is `ceil(axis / MaxN_top) >= 2`, where `MaxN_top` is the top rung of the width ladder. An axis that fits one tile for fp32 and not for fp16 makes this **true**, because one kernel must serve both. | | ||
| 107 | +| `mixed_precision` | The computation is exposed at a different width from the declared io dtype — a widen → compute → narrow chain, or an accumulator wider than the operands. | Judgement only in the degenerate case where the widening is a single store-side cast; if any *arithmetic* happens at the other width, it holds. | | ||
| 108 | +| `index_dtypes` | An input is an index tensor whose element width and signedness participate in correctness (addressing, out-of-range and duplicate semantics). | None; either an index tensor exists or it does not. | | ||
| 109 | +| `dynamic_dims` | At least one extent is unknown at compile time and must be read from a runtime scalar or a tiling field. | None. | | ||
| 110 | +| `is_list` | An input is declared as a list whose **length is a runtime value**, which the DSL's fixed parameter list cannot express directly. | None; it is a declaration, not an inference. | | ||
| 111 | + | ||
| 112 | +Adding or removing a key here is a contract change — see "Changing the contract" below. | ||
| 113 | + | ||
| 88 | ## `KB_SELECTION.json` | 114 | ## `KB_SELECTION.json` |
| 89 | 115 | ||
| 90 | The planner derives this file from computation topology and dtype/shape properties, never | 116 | The planner derives this file from computation topology and dtype/shape properties, never |
| @@ -130,3 +156,6 @@ accepts a justified `not_applicable` claim. | |||
| 130 | Adding or removing a topology, property key, status, role scope, reference class or quantity | 156 | Adding or removing a topology, property key, status, role scope, reference class or quantity |
| 131 | rule is a contract change. Update `topology-map.json`, raise both schema versions, update this | 157 | rule is a contract change. Update `topology-map.json`, raise both schema versions, update this |
| 132 | document and the consuming skills, and keep the integrity checks green. | 158 | document and the consuming skills, and keep the integrity checks green. |
| 159 | + | ||
| 160 | +A **Correction/retraction** must chase and fix every other home of the corrected claim in the same | ||
| 161 | +change; a copy left standing elsewhere re-litigates the correction. | ||
| @@ -9,24 +9,54 @@ for signatures and platform-specific behavior. | |||
| 9 | | Select a retained study implementation | `pypto-pro-material-explore` | [examples/kernel-index.md](examples/kernel-index.md) | | 9 | | Select a retained study implementation | `pypto-pro-material-explore` | [examples/kernel-index.md](examples/kernel-index.md) | |
| 10 | | Choose a reusable dataflow | `pypto-pro-op-design` | [patterns/pattern-index.md](patterns/pattern-index.md) | | 10 | | Choose a reusable dataflow | `pypto-pro-op-design` | [patterns/pattern-index.md](patterns/pattern-index.md) | |
| 11 | | Check dtype and cast behavior | `pypto-pro-op-design` / `pypto-pro-op-develop` | [constraints/precision.md](constraints/precision.md) | | 11 | | Check dtype and cast behavior | `pypto-pro-op-design` / `pypto-pro-op-develop` | [constraints/precision.md](constraints/precision.md) | |
| 12 | -| Check tile shapes and memory placement | `pypto-pro-op-design` / `pypto-pro-op-develop` | [constraints/tiling.md](constraints/tiling.md), then [constraints/memory-layout.md](constraints/memory-layout.md) | | 12 | +| Check tile shapes and memory placement | `pypto-pro-op-design` / `pypto-pro-op-develop` | [constraints/tiling.md](constraints/tiling.md), then [constraints/memory-layout.md](constraints/memory-layout.md) — if a **wide fp32 accumulator stays resident in UB across an inner loop**, sum UB *first*: it can bind before L0A, and a sample's tile size then does not transfer | |
| 13 | | Check vector authoring choices | `pypto-pro-op-develop` | [constraints/vec.md](constraints/vec.md) | | 13 | | Check vector authoring choices | `pypto-pro-op-develop` | [constraints/vec.md](constraints/vec.md) | |
| 14 | -| Check synchronization | `pypto-pro-op-design` / `pypto-pro-op-develop` | [constraints/sync-stitch.md](constraints/sync-stitch.md) | | 14 | +| Pass a float constant to a `vf.*` op — an epsilon, a floor, a guard — or a result is right on ordinary data and wrong only on degenerate data | `pypto-pro-op-design` / `pypto-pro-op-develop` | [constraints/vec-scalar-immediate.md](constraints/vec-scalar-immediate.md) — a `vf` scalar immediate is emitted with `%f`; anything below ~`5e-7` is silently deleted | |
| 15 | +| Check synchronization | `pypto-pro-op-design` / `pypto-pro-op-develop` | [constraints/sync-stitch.md](constraints/sync-stitch.md) — a hand-written per-tile cube↔vector handoff inside **one** `@pl.jit` is measured to run, so a hang is not attributable to that construct | | ||
| 16 | +| A kernel returns a different answer on each run of the same input, or a fused cube+vector kernel emits NaN | any | [constraints/sync-stitch.md](constraints/sync-stitch.md) — the open cube↔vector race reduces to a **minimal reproducer** (one task, one reduction chunk), and the remaining handle is **bisecting the inner loop**; `auto_mutex` and the producer-local contraction loop are exonerated **by measurement**, so this is a design defect and not a framework bug, though the inner unroll is *not* positively identified; then [references/investigation-discipline.md](references/investigation-discipline.md) §18 — a low-rate race needs **thousands** of runs, and "clean" is only an upper bound on the rate | | ||
| 15 | | Check dynamic tails | `pypto-pro-op-design` / `pypto-pro-op-develop` | [constraints/tail-validshape.md](constraints/tail-validshape.md) | | 17 | | Check dynamic tails | `pypto-pro-op-design` / `pypto-pro-op-develop` | [constraints/tail-validshape.md](constraints/tail-validshape.md) | |
| 16 | -| Select A5 constraints; use limits only after target detection | design/develop/perf skills | [constraints/arch-a5.md](constraints/arch-a5.md) | | 18 | +| Select A5 constraints; use limits only after target detection | `pypto-pro-op-design` / `pypto-pro-op-develop` / `pypto-pro-op-perf-tune` | [constraints/arch-a5.md](constraints/arch-a5.md) | |
| 17 | | Decide what may run on the host | `pypto-pro-op-design` / `pypto-pro-op-develop` | [constraints/wrapper-boundary.md](constraints/wrapper-boundary.md) | | 19 | | Decide what may run on the host | `pypto-pro-op-design` / `pypto-pro-op-develop` | [constraints/wrapper-boundary.md](constraints/wrapper-boundary.md) | |
| 18 | | Measure and tune a correct kernel | `pypto-pro-op-perf-tune` | [`pypto-pro-op-perf-tune` evidence protocol](../pypto-pro-op-perf-tune/references/evidence-protocol.md) | | 20 | | Measure and tune a correct kernel | `pypto-pro-op-perf-tune` | [`pypto-pro-op-perf-tune` evidence protocol](../pypto-pro-op-perf-tune/references/evidence-protocol.md) | |
| 21 | +| Deciding whether a measured delta is real — how many arms do I need, can I compare two cards | `pypto-pro-op-perf-tune` / any | [`pypto-pro-op-perf-tune` evidence protocol](../pypto-pro-op-perf-tune/references/evidence-protocol.md) — paired arms on one card in one session; the measured floors are **0.291** within-session paired and **~0.908** across-session, and they are **per instrument** — ratios and absolute quantities do not convert; the *grader's* runners are an instrument too — one public runner reads **≈2.58 points low**, far above either floor, so never subtract two scores from different runners: [references/stage5-verdicts.md](references/stage5-verdicts.md) § External scores are instrument-qualified | | ||
| 19 | | Localise a numerical error | `pypto-pro-op-develop` | [playbooks/numerical-error-localisation.md](playbooks/numerical-error-localisation.md) | | 22 | | Localise a numerical error | `pypto-pro-op-develop` | [playbooks/numerical-error-localisation.md](playbooks/numerical-error-localisation.md) | |
| 23 | +| A clamp, epsilon or guard constant has no effect, or a small float constant reads back as `0.0` | `pypto-pro-op-develop` | [constraints/vec-scalar-immediate.md](constraints/vec-scalar-immediate.md) — same defect as the `vf` float-immediate row above, reached from the symptom rather than the cause: immediates are emitted at 6 decimal places, and the knee is ~5e-7, **above** the fp16 subnormal floor, so it is truncation and not underflow | | ||
| 24 | +| A kernel is exact on a small shape and corrupts on a larger one, with the same key and the same generated code | `pypto-pro-op-develop` | [patterns/buffer-reuse-lifetime.md](patterns/buffer-reuse-lifetime.md) — a declared buffer count that overruns the next group's base; then [references/investigation-discipline.md](references/investigation-discipline.md) §13.2 — nothing is observable at ≤ 1 work item per core | | ||
| 20 | | Quantize per row with a scale that is also an output | `pypto-pro-op-design` | [patterns/vec-per-token-dynamic-quant.md](patterns/vec-per-token-dynamic-quant.md) | | 25 | | Quantize per row with a scale that is also an output | `pypto-pro-op-design` | [patterns/vec-per-token-dynamic-quant.md](patterns/vec-per-token-dynamic-quant.md) | |
| 21 | | Feed an integer Cube contraction directly into a floating-point epilogue | `pypto-pro-op-design` / `pypto-pro-op-develop` | [patterns/cv-quant-matmul-direct-epilogue.md](patterns/cv-quant-matmul-direct-epilogue.md) | | 26 | | Feed an integer Cube contraction directly into a floating-point epilogue | `pypto-pro-op-design` / `pypto-pro-op-develop` | [patterns/cv-quant-matmul-direct-epilogue.md](patterns/cv-quant-matmul-direct-epilogue.md) | |
| 22 | | Changing a staged multi-phase Cube matmul and need target-version precision/performance gates before trusting an alternative | `pypto-pro-op-develop` / `pypto-pro-op-perf-tune` | [references/staged-cube-matmul-gates.md](references/staged-cube-matmul-gates.md) | | 27 | | Changing a staged multi-phase Cube matmul and need target-version precision/performance gates before trusting an alternative | `pypto-pro-op-develop` / `pypto-pro-op-perf-tune` | [references/staged-cube-matmul-gates.md](references/staged-cube-matmul-gates.md) | |
| 23 | | Choosing or varying a kernel's per-launch `block_dim` / core count | `pypto-pro-op-develop` / `pypto-pro-op-perf-tune` | [references/pypto-pro-launch-block-dim.md](references/pypto-pro-launch-block-dim.md) | | 28 | | Choosing or varying a kernel's per-launch `block_dim` / core count | `pypto-pro-op-develop` / `pypto-pro-op-perf-tune` | [references/pypto-pro-launch-block-dim.md](references/pypto-pro-launch-block-dim.md) | |
| 29 | +| Choosing a launch geometry or tile geometry — block count, `Rows`, "should I use every core" — or a launch is slower than its bytes while every core is busy | `pypto-pro-op-design` / `pypto-pro-op-perf-tune` | [constraints/tiling.md](constraints/tiling.md) — per-core load balance is the quantity the sweep ranks; sweep per shape; `section_vector` sizes from `vector_core_num`, **not** `core_num` | | ||
| 30 | +| An elementwise / activation kernel is slower than its byte count predicts, and the dataflow is already trivial | `pypto-pro-op-perf-tune` | [patterns/vec-elementwise-tile-and-balance.md](patterns/vec-elementwise-tile-and-balance.md) — work division and tile width are all that is left; three triggers checkable in the host wrapper | | ||
| 31 | +| Every core walks its own contiguous GM span, work is balanced, and the time still swings tens of percent between neighbouring tile widths / shard granules / buffer depths with no ordering | `pypto-pro-op-perf-tune` | [constraints/gm-concurrent-stride-aliasing.md](constraints/gm-concurrent-stride-aliasing.md) — one granule change tests it; the remedy is counter-indicated once the stream is at its move-only floor | | ||
| 32 | +| Need a vector GM bandwidth ceiling to price a lever against, or two circulating ceilings disagree | `pypto-pro-op-perf-tune` | [constraints/arch-a5.md](constraints/arch-a5.md) § the measured vector GM floor — the floor-probe method, and a **per-SKU** number that must not be quoted as universal | | ||
| 33 | +| Reduce rows whose last dimension is a handful of bytes | `pypto-pro-op-design` | [patterns/vec-narrow-row-packing.md](patterns/vec-narrow-row-packing.md) | | ||
| 34 | +| One row of the reduced axis does not fit one tile, so the kernel splits it across column blocks | `pypto-pro-op-design` / `pypto-pro-op-perf-tune` | [patterns/vec-colblk-resident-row.md](patterns/vec-colblk-resident-row.md) — keep the blocks resident and hoist gamma; `3D read + D write` → `D read + D write` | | ||
| 35 | +| One kernel must serve a last dimension spanning orders of magnitude, or several dtypes, or several ranks | `pypto-pro-op-design` / `pypto-pro-op-develop` | [patterns/tiling-key-width-ladder.md](patterns/tiling-key-width-ladder.md) | | ||
| 36 | +| A vector kernel recomputes the same loop-invariant quantity, or its register pressure is set at compile time | `pypto-pro-op-perf-tune` | [patterns/vec-compile-time-register-count.md](patterns/vec-compile-time-register-count.md) | | ||
| 37 | +| A compute chain is exposed at a narrower dtype than the operator's declared precision | `pypto-pro-op-design` / `pypto-pro-op-develop` | [patterns/vec-compute-exposed-narrow-dtype.md](patterns/vec-compute-exposed-narrow-dtype.md) | | ||
| 38 | +| A correct `vf.*` kernel is far slower than its memory roof | `pypto-pro-op-perf-tune` | **first check the row width, in BOTH directions**: below the strided-load knee go to [patterns/vec-narrow-row-packing.md](patterns/vec-narrow-row-packing.md); wider than one tile holds go to [patterns/vec-colblk-resident-row.md](patterns/vec-colblk-resident-row.md); otherwise [constraints/vec.md](constraints/vec.md), then [constraints/vec-alignment-and-rotation.md](constraints/vec-alignment-and-rotation.md) | | ||
| 24 | | Hit a framework limit, or debug something that makes no sense | any | [references/pypto-pro-framework-findings.md](references/pypto-pro-framework-findings.md) | | 39 | | Hit a framework limit, or debug something that makes no sense | any | [references/pypto-pro-framework-findings.md](references/pypto-pro-framework-findings.md) | |
| 25 | | An API looks unsupported, or a correct-looking call returns stale/wrong data | any | [references/pypto-pro-dsl-limitations-a5.md](references/pypto-pro-dsl-limitations-a5.md) — severity-ordered, silent failures first | | 40 | | An API looks unsupported, or a correct-looking call returns stale/wrong data | any | [references/pypto-pro-dsl-limitations-a5.md](references/pypto-pro-dsl-limitations-a5.md) — severity-ordered, silent failures first | |
| 26 | | An investigation keeps failing to converge, or you are about to trust a measurement | any | [references/investigation-discipline.md](references/investigation-discipline.md) | | 41 | | An investigation keeps failing to converge, or you are about to trust a measurement | any | [references/investigation-discipline.md](references/investigation-discipline.md) | |
| 42 | +| A sweep over one axis shows behaviour tracking one candidate's parameter and not another's, and you are about to name a cause | any | [references/investigation-discipline.md](references/investigation-discipline.md) §19 — a sweep falsifies only a parameter that is **not** 1-to-1 with the swept axis, so it buys **exonerations**; an attribution is conditional on the candidate list being complete, and must be stated with it | | ||
| 43 | +| Sizing a determinism or ablation matrix — how many configurations, how many runs per configuration | any | [references/investigation-discipline.md](references/investigation-discipline.md) §18.1 — breadth separates mechanisms, depth establishes presence/absence; a broad matrix of shallow cells is a table of unreliable booleans, and each cell must carry its run count and rate bound | | ||
| 27 | | A whole run failed at once (0/N), or a change "did nothing", or a number reproduces suspiciously well | any | [references/investigation-discipline.md](references/investigation-discipline.md) §2, §13, then `pypto-pro-environment-check` | | 44 | | A whole run failed at once (0/N), or a change "did nothing", or a number reproduces suspiciously well | any | [references/investigation-discipline.md](references/investigation-discipline.md) §2, §13, then `pypto-pro-environment-check` | |
| 28 | | Several agents are working in parallel on one shared record, or you are merging their branches | any | [references/investigation-discipline.md](references/investigation-discipline.md) §10 | | 45 | | Several agents are working in parallel on one shared record, or you are merging their branches | any | [references/investigation-discipline.md](references/investigation-discipline.md) §10 | |
| 29 | | Map a Chinese/English hardware, pipe, tiling, or layout term to its meaning | any | [references/terminology.md](references/terminology.md) | | 46 | | Map a Chinese/English hardware, pipe, tiling, or layout term to its meaning | any | [references/terminology.md](references/terminology.md) | |
| 47 | +| Recognise a failure someone has already hit | `pypto-pro-op-develop` / `pypto-pro-op-perf-tune` | [references/pitfall-records.md](references/pitfall-records.md) | | ||
| 48 | +| Judge whether an ops-per-register lever can pay on a narrow reduction, or are about to generalise a perf measurement across dtypes | `pypto-pro-op-perf-tune` | [references/narrow-trailing-dim-roofline-a5.md](references/narrow-trailing-dim-roofline-a5.md) | | ||
| 49 | +| Judging a Stage-5 tuning result, or a whole-set failure that might be the card rather than the kernel | `pypto-pro-op-perf-tune` | [references/stage5-verdicts.md](references/stage5-verdicts.md) — 结果词汇与环境故障分流；**PASS 判据以 `stage5-check` P1–P6/P8 为准** | | ||
| 50 | +| Recording which tuning candidates were considered, and why one was rejected — especially a rejection that cites a proximity guard | `pypto-pro-op-perf-tune` | [references/candidate-ledger-schema.md](references/candidate-ledger-schema.md) — the serialized 候选覆盖账本 and its checks; a guard margin must take its value and its limit from the **same** instrument | | ||
| 51 | +| A measured decomposition names a dominant cost term and you are choosing what to build next — or a run is about to call that term irreducible / write its completion section — or, with no decomposition yet, the case anchors show total work against core count at maximum legal parallelism so small that `t_hw` may be comparable to the card's per-launch fixed cost | `pypto-pro-op-perf-tune` | [playbooks/dominant-cost-attack-matrix.md](playbooks/dominant-cost-attack-matrix.md) — one row per transformation class over the largest measured term, each closed by a terminal verdict; the algebraic-reformulation row is the one no profiler suggests; the fixed-overhead / launch cost row is the one a-priori row, reachable before any decomposition exists | | ||
| 52 | +| A lever frees a shared capacity (UB bytes, registers, buffer slots, workspace) and its first spend did not pay — or you are about to generalise a perf conclusion across shapes, dtypes or work division | `pypto-pro-op-perf-tune` | [references/capacity-and-regime-method.md](references/capacity-and-regime-method.md) — a release closes per enumerated spend, never on one; every conclusion carries the regime cell it was measured in | | ||
| 53 | + | ||
| 54 | +[references/pitfall-records.md](references/pitfall-records.md) is a symptom | ||
| 55 | +index, not a rule page: it maps an observed error code, hang or wrong result to | ||
| 56 | +the cause that was actually found. Read it when something fails, not when | ||
| 57 | +designing. Each entry records the runtime it was observed on; confirm the | ||
| 58 | +behaviour still reproduces on the detected target before treating an entry as | ||
| 59 | +current. | ||
| 30 | 60 | ||
| 31 | Load [constraints/arch-a5.md](constraints/arch-a5.md) when runtime/build selects A5 or when | 61 | Load [constraints/arch-a5.md](constraints/arch-a5.md) when runtime/build selects A5 or when |
| 32 | the workflow default A5 applies; do not use its numerical limits until the exact device and source are confirmed. | 62 | the workflow default A5 applies; do not use its numerical limits until the exact device and source are confirmed. |
| @@ -8,10 +8,13 @@ Use these pages as focused supplements to the installed API documentation: | |||
| 8 | | [tiling.md](tiling.md) | tile shape, layout, and memory-space legality | | 8 | | [tiling.md](tiling.md) | tile shape, layout, and memory-space legality | |
| 9 | | [memory-layout.md](memory-layout.md) | address ownership, layout conversion, and overlap | | 9 | | [memory-layout.md](memory-layout.md) | address ownership, layout conversion, and overlap | |
| 10 | | [sync-stitch.md](sync-stitch.md) | tile-group mutexes and section synchronization | | 10 | | [sync-stitch.md](sync-stitch.md) | tile-group mutexes and section synchronization | |
| 11 | +| [vec-scalar-immediate.md](vec-scalar-immediate.md) | float immediates passed to `vf.*` ops, and the precision floor they are emitted at | | ||
| 12 | +| [wrapper-boundary.md](wrapper-boundary.md) | what may and may not happen on the host side of the kernel wrapper | | ||
| 11 | | [tail-validshape.md](tail-validshape.md) | dynamic dimensions and tail windows | | 13 | | [tail-validshape.md](tail-validshape.md) | dynamic dimensions and tail windows | |
| 12 | | [vec.md](vec.md) | conditional tile-op / vector-function authoring | | 14 | | [vec.md](vec.md) | conditional tile-op / vector-function authoring | |
| 13 | | [vec-alignment-and-rotation.md](vec-alignment-and-rotation.md) | vf lane and reduction-row alignment, buffer rotation, bare-tile sync | | 15 | | [vec-alignment-and-rotation.md](vec-alignment-and-rotation.md) | vf lane and reduction-row alignment, buffer rotation, bare-tile sync | |
| 14 | | [vec-mask-width.md](vec-mask-width.md) | converting a mask between b8/b16/b32 element widths | | 16 | | [vec-mask-width.md](vec-mask-width.md) | converting a mask between b8/b16/b32 element widths | |
| 17 | +| [gm-concurrent-stride-aliasing.md](gm-concurrent-stride-aliasing.md) | per-core contiguous GM spans walked in lockstep: when the shared stride aliases, and when rotating it costs | | ||
| 15 | | [arch-a5.md](arch-a5.md) | A5-only platform discovery and evidence gate | | 18 | | [arch-a5.md](arch-a5.md) | A5-only platform discovery and evidence gate | |
| 16 | 19 | ||
| 17 | Do not load a platform-specific page until the target architecture is | 20 | Do not load a platform-specific page until the target architecture is |
| @@ -45,18 +45,24 @@ version. | |||
| 45 | a SKU's core count. | 45 | a SKU's core count. |
| 46 | - Re-run correctness after any dtype, layout, buffering, or tile-size change. | 46 | - Re-run correctness after any dtype, layout, buffering, or tile-size change. |
| 47 | 47 | ||
| 48 | -## The UB capacity dispute is settled, and the settling generalises | 48 | +## A5 UB is 253952 B, and how that is established generalises |
| 49 | 49 | ||
| 50 | -Two values were in circulation for A5 UB: **248 KB**, from the installed tutorial | 50 | +**A5 UB is `ub_size = 253952` — 248 KB exactly.** `950PR_957x.ini` carries it, and so do |
| 51 | -(`.../tile_based_python_programming/multi_core_partitioning_and_Tiling.md`, stated | 51 | +`950DT_957x`, `950PR_958x` and `950DT_958x`. The installed tutorial |
| 52 | -once as the limit and once inside a worked FP16/FP32 budget example), and | 52 | +(`.../tile_based_python_programming/multi_core_partitioning_and_Tiling.md`) agrees, stating |
| 53 | -**256 KB / 216 KB-with-SIMT** from a sibling DSL's device profile for the same | 53 | +248 KB once as the limit and once inside a worked FP16/FP32 budget example. A sibling DSL's |
| 54 | -silicon. | 54 | +device profile for the same silicon says **256 KB / 216 KB-with-SIMT**; that is not the |
| 55 | +budget the toolchain compiles against on this SKU, and the platform file is what decides. | ||
| 55 | 56 | ||
| 56 | -**The platform file decides it.** `950PR_957x.ini` carries `ub_size=253952`, which | 57 | +🔴 **The physical buffer is larger than the budget, and that is not a licence to spend |
| 57 | -is 248 KB exactly, and so do `950DT_957x`, `950PR_958x` and `950DT_958x`. The | 58 | +it.** A direct probe (`references/pitfall-records.md`, the softmax UB-growth row) |
| 58 | -tutorial and the platform config agree; the 256 KB figure does not describe the | 59 | +allocates and computes bit-exact at **259072 B** and faults at **265216 B** — so the |
| 59 | -budget the toolchain compiles against on this SKU. | 60 | +usable silicon exceeds `ub_size`, and 262144 sits in an untested bracket between the |
| 61 | +two. Both facts hold at once. **Budget every design against the `.ini` `ub_size = | ||
| 62 | +253952`; treat anything above it as unsupported even where a probe happens to pass.** | ||
| 63 | +The single-tile gate that follows from it is `4 * TR * align64(D) * 4 ≤ 253952`, i.e. | ||
| 64 | +`align64(D) ≤ 1984` at `TR = 8` fp32 — and `1984 = 31 × 64`, so the bound is itself | ||
| 65 | +64-aligned and reachable. | ||
| 60 | 66 | ||
| 61 | Two things about *how* that was settled matter more than the number: | 67 | Two things about *how* that was settled matter more than the number: |
| 62 | 68 | ||
| @@ -79,6 +85,94 @@ makes the whole capacity question cheap: `l0_a_size`, `l0_b_size`, `l0_c_size`, | |||
| 79 | independently — are all keys in the same `.ini`. Read them there rather than from | 85 | independently — are all keys in the same `.ini`. Read them there rather than from |
| 80 | any page, including this one. | 86 | any page, including this one. |
| 81 | 87 | ||
| 88 | +### The core counts are the exception: the `.ini` and the live device disagree | ||
| 89 | + | ||
| 90 | +**Capacities come from the platform file; core counts come from the runtime | ||
| 91 | +platform query.** The two sources are not interchangeable, and on at least one | ||
| 92 | +A5 installation they differ: | ||
| 93 | + | ||
| 94 | +| source | cube / vector cores | | ||
| 95 | +|---|---| | ||
| 96 | +| `950PR_957x.ini` (and the other `957x` variants) | 28 / 56 | | ||
| 97 | +| the runtime platform query on the live device | **32 / 64** | | ||
| 98 | + | ||
| 99 | +Measured on Ascend950PR / CANN 9.2.0, 2026-08-24. A launch geometry sized from | ||
| 100 | +the `.ini` would leave a quarter of the array idle on that box while every | ||
| 101 | +device-free gate passed, because nothing in the toolchain compares the two. | ||
| 102 | + | ||
| 103 | +The rule this page already states — *read launch width from runtime platform | ||
| 104 | +information rather than hard-coding a SKU's core count* — is therefore not | ||
| 105 | +merely a portability preference: **the static file is a different number, not a | ||
| 106 | +stale copy of the same one.** Capacities have no such live query (the platform | ||
| 107 | +object exposes `soc_version`, `core_num`, `cube_core_num`, `vector_core_num` and | ||
| 108 | +a derived `arch`, and no memory sizes at all), so the split is forced: byte | ||
| 109 | +budgets from the `.ini` key, core counts from the query, and never the reverse. | ||
| 110 | + | ||
| 111 | +The permitted form, concretely: `from pypto_pro.runtime.platform import | ||
| 112 | +get_platform_info`, then `_VEC = int(getattr(get_platform_info(), | ||
| 113 | +"vector_core_num", <fallback>))`, read once at import and used to size the | ||
| 114 | +vector launch. That query is the **only** legal source of a core count in | ||
| 115 | +shipped code, and a literal is permitted **only** as the `getattr` fallback. | ||
| 116 | + | ||
| 117 | +🔴 **Neither number in this page's tables may be transcribed into a kernel.** | ||
| 118 | +Both tables are records of specific parts; the SKU a grading run lands on is not | ||
| 119 | +known at authoring time; and the two parts below report the same `soc_version`, | ||
| 120 | +so nothing a device-free gate can read tells them apart. A literal that is | ||
| 121 | +correct on the box in front of you therefore passes every local measurement and | ||
| 122 | +is still wrong where it is scored. `VECTOR_CORES = 56 # from the table` is the | ||
| 123 | +prohibited form, whichever row it was copied from — the tables here exist to | ||
| 124 | +show the sources disagree, not to be read from. **Citing this page beside a | ||
| 125 | +literal core count is a mis-citation.** | ||
| 126 | + | ||
| 127 | +**Three A5 parts in evidence here report DIFFERENT core counts**, and all are real: | ||
| 128 | + | ||
| 129 | +| part | cube / vector | | ||
| 130 | +|---|---| | ||
| 131 | +| `Ascend950PR_9579` | 28 / 56 | | ||
| 132 | +| `Ascend950PR_9589` | **32 / 64** | | ||
| 133 | +| `Ascend950PR_957b` | 28 / 56 (runtime query) | | ||
| 134 | + | ||
| 135 | +The `957b` row was added 2026-08-27 from a dynamic_quant Stage-5 run on box a5-238 card 3: | ||
| 136 | +`torch.npu.get_device_name(3)` -> `Ascend950PR_957b`, and `get_platform_info()` on the live | ||
| 137 | +device reported `core_num=28, cube_core_num=28, vector_core_num=56`, `soc_version=DAV_3510`. | ||
| 138 | +It is a fourth distinct part name in this KB once `Ascend950PR_957c` (the grader, recorded in | ||
| 139 | +`references/pitfall-records.md`) is counted, and it reports the same `DAV_3510` as the others | ||
| 140 | +— which is the discriminate-by-device-name rule below, observed again rather than argued. | ||
| 141 | + | ||
| 142 | +🔴 **A recorded instance of the prohibited form, from the same run.** A previous Stage-5 | ||
| 143 | +attempt on that operator shipped `VECTOR_CORES = 56 # A5 vector_core_num (constraints/arch-a5.md)` | ||
| 144 | +— a literal transcribed from the table above, *citing this page as its authority*. That is | ||
| 145 | +exactly the form and exactly the mis-citation this section names. It was numerically right on | ||
| 146 | +the box it was written on, which is precisely why it survived every local gate. The legal | ||
| 147 | +form is the `getattr` query below. Measured against the literal on the same card in the same | ||
| 148 | +session, the query moved exactly the cases whose tile count exceeds the core count and left the | ||
| 149 | +cases where the cap cannot bind unchanged; the magnitudes live with that run's own evidence, | ||
| 150 | +not on this page. | ||
| 151 | + | ||
| 152 | +So "56 blocks" on one page and "32 cores" on another are **different hardware, not a | ||
| 153 | +contradiction** — and a per-core figure measured on one does not port to the other. | ||
| 154 | + | ||
| 155 | +🔴 **Discriminate by DEVICE NAME, not by `soc_version`.** Both parts above | ||
| 156 | +report `DAV_3510`; `pypto-pro-op-perf-tune/references/evidence-protocol.md` states | ||
| 157 | +that `DAV_3510` plus a core count does **not** uniquely determine a SKU. Record | ||
| 158 | +`torch.npu.get_device_name(...)` beside every number, and read "56 blocks" on one | ||
| 159 | +page and "32 cores" on another as *different hardware*, not a contradiction. | ||
| 160 | + | ||
| 161 | +**`section_vector` must size from `vector_core_num`, not `core_num`.** Reading the | ||
| 162 | +wrong one launches on half the array. Magnitude and its caveat live in | ||
| 163 | +`constraints/tiling.md` § launch geometry — quote it from there, with the caveat. | ||
| 164 | +A block-count sweep result likewise does **not** port across SKUs: sweep on the part | ||
| 165 | +you will be measured on. | ||
| 166 | + | ||
| 167 | +Two consequences for a design document: | ||
| 168 | + | ||
| 169 | +- A UB budget must be a **named module constant citing its `.ini` key**, since | ||
| 170 | + there is no API to read it from. That is not a workaround for laziness; it is | ||
| 171 | + the only available form. | ||
| 172 | +- A core count written as a literal anywhere is a defect even when it is | ||
| 173 | + currently right, because the number that is right is the one the device | ||
| 174 | + reports at run time. | ||
| 175 | + | ||
| 82 | **What this does to the sibling DSL's device table.** Checked row by row against | 176 | **What this does to the sibling DSL's device table.** Checked row by row against |
| 83 | `950PR_957x.ini`, it is right about L0A/L0B (64 KB each), L0C (256 KB), L1 | 177 | `950PR_957x.ini`, it is right about L0A/L0B (64 KB each), L0C (256 KB), L1 |
| 84 | (512 KB), BT (4 KB), the 28/56 core split, and A2/A3's 192 KB UB — and wrong only | 178 | (512 KB), BT (4 KB), the 28/56 core split, and A2/A3's 192 KB UB — and wrong only |
| @@ -92,3 +186,115 @@ Platform configuration values are roofline inputs, not measured kernel | |||
| 92 | performance. Use the target profiler to determine the current bottleneck. See | 186 | performance. Use the target profiler to determine the current bottleneck. See |
| 93 | the platform-gated | 187 | the platform-gated |
| 94 | [A5 roofline workflow](../../pypto-pro-op-perf-tune/references/a5-roofline-and-levers.md). | 188 | [A5 roofline workflow](../../pypto-pro-op-perf-tune/references/a5-roofline-and-levers.md). |
| 189 | + | ||
| 190 | +### The measured vector GM floor — and why it is a per-SKU number | ||
| 191 | + | ||
| 192 | +A vector GM bandwidth ceiling is **not** in the platform `.ini`, so unlike the capacities | ||
| 193 | +above it cannot be resolved by reading a key. It has to be measured, and the answer is a | ||
| 194 | +per-SKU number that varies by roughly a factor of two across A5 parts: **~2.2 TB/s load / | ||
| 195 | +~2.35 TB/s store** (≈3.3–4.6 aggregate) on **Ascend950, 32 cores**, against **~1.48 TB/s** | ||
| 196 | +aggregate on the 56-vector-core part measured below. Those two decide a tuning verdict in | ||
| 197 | +opposite directions, which is why the floor probe below is run on the detected target rather | ||
| 198 | +than quoted from a page. | ||
| 199 | + | ||
| 200 | +**Method — the floor probe.** Take the kernel you actually ship and delete 100 % of the | ||
| 201 | +arithmetic, keeping the tiling, the launch geometry, the buffering and the DMA schedule | ||
| 202 | +identical: the vector body becomes `load_align` → `store_align`. The result is numerically | ||
| 203 | +wrong by construction and is never shipped; it prices the data movement of *that* | ||
| 204 | +implementation, which is the bound a compute-side lever is competing against. Run it at two | ||
| 205 | +sizes far past L2 so the measurement is DRAM-limited, one shape per process. | ||
| 206 | + | ||
| 207 | +**Result on `Ascend950PR_9579`** (28 cube / 56 vector, `soc_version DAV_3510`, CANN 9.2.0, | ||
| 208 | +2026-08-25), a `section_vector` load+store-only kernel on 56 blocks: | ||
| 209 | + | ||
| 210 | +| working set (read + write) | aggregate | | ||
| 211 | +|---|---| | ||
| 212 | +| 268 MB | **1.49 TB/s** | | ||
| 213 | +| 537 MB | **1.44 TB/s** | | ||
| 214 | + | ||
| 215 | +and the *full* kernels at the same geometry measure 1.53 and 1.44 TB/s — i.e. on the two | ||
| 216 | +largest shapes, deleting all the arithmetic changes nothing beyond the round-to-round spread. | ||
| 217 | +**So the ~1.48 TB/s figure is the one that describes this part**, and the 2.2/2.35 pair | ||
| 218 | +describes a different SKU with a different core count. | ||
| 219 | + | ||
| 220 | +### A third part, probed the same way: `Ascend950PR_957b` | ||
| 221 | + | ||
| 222 | +Same method, 2026-08-27, on a per-token dynamic-quant vector kernel (`section_vector`, 56 blocks | ||
| 223 | +from the runtime query, rung 16384 x 1 row, io and int8 tiles double-buffered). The shipped kernel | ||
| 224 | +had 100 % of its arithmetic deleted with the tiling, launch geometry, buffering and DMA schedule | ||
| 225 | +held identical; 10 timed launches, msprof `op_summary_PipeUtilization` Task Duration. | ||
| 226 | + | ||
| 227 | +| shape (bf16) | contractual traffic | move-only floor | achieved | | ||
| 228 | +|---|---|---|---| | ||
| 229 | +| 8192 x 16384 | 402.69 MB (268.44 read + 134.22 write int8 + 0.03 scale) | 324.2 us (median) | **1.24 TB/s** | | ||
| 230 | + | ||
| 231 | +So the aggregate move-only floor on this part reads below the ~1.44-1.53 TB/s recorded above for | ||
| 232 | +`Ascend950PR_9579`, on a comparable working set. Both are 28/56 parts by the runtime query, which | ||
| 233 | +is a reminder that the floor is a per-part measurement rather than a per-core-count one -- re-probe | ||
| 234 | +it, do not carry either number across. | ||
| 235 | + | ||
| 236 | +**Two method notes that changed the reading here.** The floor probe's `min` came in at 297.4 us | ||
| 237 | +while its 2nd-smallest and median were 320.6 and 324.2 -- the robust pair agreeing with each other | ||
| 238 | +and the min standing apart. Taking the min would have overstated the floor's bandwidth by ~9 % and | ||
| 239 | +understated how far the full kernel sits above it. The full kernel's own three statistics agreed | ||
| 240 | +within 0.2 %, so only the probe needed the judgement. And the same kernel measured through msprof | ||
| 241 | +and through the torch_npu profiler agreed to 0.01 % on this shape, so on this part the two | ||
| 242 | +front-ends are interchangeable for a target-kernel duration -- which is worth knowing before | ||
| 243 | +paying for a second collection to cross-check one. | ||
| 244 | + | ||
| 245 | +**🔴 The probe silently answers a different question on an L2-resident shape, and it does not | ||
| 246 | +announce that it has.** The method above says to run "at two sizes far past L2 so the measurement | ||
| 247 | +is DRAM-limited". Run on a shape that is *not* past L2 it still returns a number, and that number | ||
| 248 | +is fixed launch/teardown cost plus L2-resident traffic -- not a movement floor. Dividing the | ||
| 249 | +shape's bytes by it yields a plausible-looking TB/s that is an artifact of the division. | ||
| 250 | + | ||
| 251 | +**The discriminator is free, and it is the only thing that tells the two regimes apart: sweep at | ||
| 252 | +least two shapes whose byte counts differ, and check whether the FLOOR tracks the bytes.** Measured | ||
| 253 | +on one operator's five graded cases, 2026-08-27: across the four L2-resident cases the byte count | ||
| 254 | +spanned **16.5 %** while the measured floor spanned **2.2 %** -- one case moved 14 % fewer bytes | ||
| 255 | +than another and landed within 0.05 % of the same floor. The one case far past L2, on the same | ||
| 256 | +kernel and the same probe, took **156x** the floor for **127x** the bytes, i.e. roughly | ||
| 257 | +proportional. Same instrument, same day, opposite regimes. | ||
| 258 | + | ||
| 259 | +Consequence for a Stage-5 terminal-state claim: a floor probe only licenses a | ||
| 260 | +`data_movement_bound` / `balanced_compute_movement` reading on the shapes where its floor scales | ||
| 261 | +with size. On the L2-resident ones it licenses a statement about how much time the *arithmetic* | ||
| 262 | +accounts for (there, 70-74 %) and nothing about which side the bound sits on -- and "arithmetic is | ||
| 263 | +most of the time" is not `compute_bound`, which additionally needs the compute shown to be at a | ||
| 264 | +hardware limit. Reading it as one is reading a ratio as a bound. | ||
| 265 | + | ||
| 266 | +### The constant-geometry byte sweep | ||
| 267 | + | ||
| 268 | +**And the fix, which is cheap: sweep the byte count at CONSTANT geometry.** Hold the rung, | ||
| 269 | +rows-per-tile, launch width and DMA shape fixed and vary only how many bytes move, then fit the | ||
| 270 | +floor. On one A5 part, one operator, four points spanning 3.1 MB to 201.6 MB at a single | ||
| 271 | +geometry, the floor came out as **two regimes rather than one line**, with marginal slopes | ||
| 272 | +**0.135 us/MB** on the L2-resident points and **0.986 us/MB** on the DRAM-limited ones -- a | ||
| 273 | +**7.3x** difference on the same kernel and the same probe. The L2-regime fit | ||
| 274 | +`floor_us = 1.643 + 0.1352 * MB` puts the **size-independent term at 1.643 us**, which on a ~3 MB | ||
| 275 | +shape is four fifths of everything the probe returns. | ||
| 276 | + | ||
| 277 | +That is what makes a single-shape floor probe misleading below L2: the number is mostly the | ||
| 278 | +intercept. The sweep separates them, and it costs one extra run per point. Two uses follow: | ||
| 279 | + | ||
| 280 | +- a terminal-state claim on an L2-resident shape becomes possible, because `a` and `b` decompose | ||
| 281 | + the floor into fixed cost and real movement. On the same operator this put four graded cases at | ||
| 282 | + **5-6 %** of runtime in movement against **70-74 %** in arithmetic (the arithmetic measured by | ||
| 283 | + ablation, not inferred), which rules `data_movement_bound` out on the numbers rather than on a | ||
| 284 | + ratio reading; | ||
| 285 | +- the DRAM-regime slope from the sweep is directly comparable to a large-shape floor measured at a | ||
| 286 | + different geometry, which is a cross-check that both are measuring the same physical thing -- | ||
| 287 | + here 1.01 TB/s from the sweep against 1.24 TB/s from a rung-16384 single-row geometry. | ||
| 288 | + | ||
| 289 | +🔴 Do not read the L2-regime slope as a bandwidth to design against. It is an effective figure for | ||
| 290 | +a working set that fits, and it disappears the moment the shape does not. | ||
| 291 | + | ||
| 292 | +**Do not quote 1.44–1.53 TB/s as a universal A5 number.** It is one SKU, one core count, | ||
| 293 | +one CANN version, measured by the method above; it is recorded here so the next run can start | ||
| 294 | +from a measurement instead of a dispute, and so that the two circulating figures are visibly | ||
| 295 | +attached to the parts they were taken on. Re-run the probe on the detected target before using | ||
| 296 | +either. The consequence worth carrying is structural rather than numeric: **a modelled | ||
| 297 | +memory-time that implies more than the measured aggregate is unreachable by any pure-vector | ||
| 298 | +implementation on that part**, and a kernel measured at that floor has no compute-side lever | ||
| 299 | +left — which is also the counter-indication gate on | ||
| 300 | +[gm-concurrent-stride-aliasing.md](gm-concurrent-stride-aliasing.md). | ||
| @@ -0,0 +1,154 @@ | |||
| 1 | +# GM-side concurrent-stride aliasing across cores | ||
| 2 | + | ||
| 3 | +## Applies to | ||
| 4 | + | ||
| 5 | +Any kernel that gives each vector core its **own contiguous span** of one GM buffer and walks | ||
| 6 | +that span in lockstep with every other core. The elementwise partitioning | ||
| 7 | +`span = ceil(n / (cores * granule)) * granule`, with core `ci` starting at `ci * span` and | ||
| 8 | +stepping one tile at a time, is the canonical shape; a long-axis traversal with one span per | ||
| 9 | +core has it too. It is a property of the **partitioning**, not of the computation, so no | ||
| 10 | +topology label decides whether it applies — read the wrapper. | ||
| 11 | + | ||
| 12 | +This page is about the **GM** side. The UB-side pitch rules | ||
| 13 | +([vec-alignment-and-rotation.md](vec-alignment-and-rotation.md) § power-of-two store strides, | ||
| 14 | +and the gather pitch in | ||
| 15 | +[../patterns/vec-scan-prefix-dependent.md](../patterns/vec-scan-prefix-dependent.md)) are a | ||
| 16 | +different mechanism at a different memory, and neither predicts this one. | ||
| 17 | + | ||
| 18 | +**Instrument for every number below.** Ascend950PR_9579 (28 cube / 56 vector, | ||
| 19 | +`soc_version DAV_3510`), CANN 9.2.0, 2026-08-25, one flat elementwise kernel — 56 blocks, | ||
| 20 | +16 KB tiles, depth 2, one contiguous span per core, zero cross-lane ops. Figures marked | ||
| 21 | +`[iso]` are one shape per process, three rounds; the rest are a screening pass that runs a | ||
| 22 | +whole shape suite in one process. **Read the measurement-hygiene note below before trusting a | ||
| 23 | +screening figure** — it reversed the sign of a verdict here. | ||
| 24 | + | ||
| 25 | +## Symptom | ||
| 26 | + | ||
| 27 | +A kernel whose per-core work is provably balanced, whose UB budget is comfortable and whose | ||
| 28 | +vector body is already minimal runs **tens of percent** above its own move-only floor on | ||
| 29 | +*some* shapes and exactly at that floor on others, with no ordering in dtype, element count | ||
| 30 | +or tile count. Every knob you turn is **non-monotone**: buffer depths 3 and 6 measure worse | ||
| 31 | +than *both* depth 2 and depth 4 on one shape, and a tile-width sweep picks a different winner | ||
| 32 | +per shape with no pattern. | ||
| 33 | + | ||
| 34 | +Reproducibility is what separates this from noise: round-to-round spread stays under 1 %, so | ||
| 35 | +it is a deterministic address effect and does not average out. | ||
| 36 | + | ||
| 37 | +## Cause | ||
| 38 | + | ||
| 39 | +All `cores` cores issue their tile loads at the same rate from bases `ci * span`. At any | ||
| 40 | +instant the concurrently-outstanding DMA addresses are therefore separated by **exactly | ||
| 41 | +`span`** — and `span` is a value the wrapper computed from the element count, not a value | ||
| 42 | +anyone chose. When `span` lands on an unlucky stride the concurrent accesses fall in the same | ||
| 43 | +memory sets and serialise; a neighbouring value of `span` does not. | ||
| 44 | + | ||
| 45 | +## Diagnostic — move the stride without moving the work | ||
| 46 | + | ||
| 47 | +The decisive experiment is a change to `span` too small to change anything else. Raise the | ||
| 48 | +shard granule that `span` is rounded up to by one step, and re-measure: | ||
| 49 | + | ||
| 50 | +```python | ||
| 51 | +PART = 256 # elements; 32 B aligned for every dtype | ||
| 52 | +span = ((n + cores * PART - 1) // (cores * PART)) * PART | ||
| 53 | +``` | ||
| 54 | + | ||
| 55 | +Raising `PART` from 256 to 512 on a 16.8 M-element bf16 shape moves the span from 299 776 to | ||
| 56 | +300 032 elements — **0.09 %** — and moves the time **17.76 → 10.18 µs, −42.7 %**. (Screening | ||
| 57 | +pass; this shape reproduces isolated, 17.405–17.717 µs `[iso]` for the same in-order arm.) | ||
| 58 | + | ||
| 59 | +A 0.09 % change in a stride cannot change how many bytes move or how many instructions issue. | ||
| 60 | +It can only change which addresses are in flight together. **If the two granules land within | ||
| 61 | +your noise floor, you do not have this defect**, and the remedy below will cost you. | ||
| 62 | + | ||
| 63 | +### 🔴 Measurement hygiene: one shape per process | ||
| 64 | + | ||
| 65 | +Batching several shapes into one process changes L2 residency enough to **reverse the sign of | ||
| 66 | +a verdict**. On this kernel a 10.9 M fp32 shape read 19.07 µs batched and 21.10 µs `[iso]` on | ||
| 67 | +the same binary; a rotation variant read a 24 % *regression* batched and a 7.6 % | ||
| 68 | +*improvement* `[iso]`. A second shape's apparent +4.8 % regression under the shipped rotation | ||
| 69 | +turned out to be **−0.09 %** once isolated — i.e. one of the two numbers that would have | ||
| 70 | +rejected the lever did not exist. Measure one shape per process before any accept/reject on | ||
| 71 | +this page. | ||
| 72 | + | ||
| 73 | +## Remedy — rotate the per-core visiting order | ||
| 74 | + | ||
| 75 | +The rotation moves **no bytes**, changes no tiling and changes no output: core `ci` starts at | ||
| 76 | +tile `rot` of its own span and wraps, so the concurrent addresses stop being separated by a | ||
| 77 | +single constant. | ||
| 78 | + | ||
| 79 | +```python | ||
| 80 | +ntile = (span + TILE_ELEMS - 1) // TILE_ELEMS | ||
| 81 | +rot = pl.min(ci, ntile - 1) # no division; see the spelling note | ||
| 82 | +for t in pl.range(rot, ntile): # pl.range accepts a runtime scalar start | ||
| 83 | + body(t) | ||
| 84 | +for t in pl.range(0, rot): | ||
| 85 | + body(t) | ||
| 86 | +``` | ||
| 87 | + | ||
| 88 | +| shape, dtype | in order `[iso]` | rotated `[iso]` | Δ | | ||
| 89 | +|---|---|---|---| | ||
| 90 | +| 16.8 M bf16 | 17.405 / 17.507 / 17.717 µs | **10.388 / 10.487 / 10.537** | **−39.7 %** | | ||
| 91 | +| 10.9 M fp32 | 21.097 / 21.256 / 21.305 µs | **11.107 / 11.113 / 11.163** | **−47.4 %** | | ||
| 92 | + | ||
| 93 | +**The spelling is load-bearing, and the obvious one hides the result.** Written as | ||
| 94 | +`(t + rot) % ntile` inside the loop it pays two integer divisions per tile and costs the | ||
| 95 | +small, launch-dominated shapes **+3.1 % and +7.1 %** `[iso]` — enough to read as "rotation is | ||
| 96 | +bad for small inputs, full stop", which is false. The two division-free passes above remove | ||
| 97 | +one division; `pl.min(ci, ntile - 1)` in place of `ci % ntile` removes the last, and the | ||
| 98 | +small-shape cost then disappears (−2.7 % to +0.8 % over four ~1 M-element shapes). `pl.min` | ||
| 99 | +is an approximation of the modulus — it gives the first `ntile` cores distinct starting tiles | ||
| 100 | +and lets the rest share the last one — and it was sufficient wherever the rotation paid at all. | ||
| 101 | + | ||
| 102 | +## Counter-indication — it taxes a stream that is already at the floor | ||
| 103 | + | ||
| 104 | +Rotation scatters a sequential stream, and a sequential stream is the best case for DRAM row | ||
| 105 | +locality. Where the kernel already sits **at its move-only floor** — deleting 100 % of the | ||
| 106 | +arithmetic changes nothing — there is no aliasing left to fix and the rotation is pure cost: | ||
| 107 | + | ||
| 108 | +| shape, dtype | in order | rotated | Δ | | ||
| 109 | +|---|---|---|---| | ||
| 110 | +| 67 M fp16 (268 MB) | 174.99 µs | 185.50 (screening) | **+4.33 % `[iso]`**, +6.0 % screening | | ||
| 111 | + | ||
| 112 | +That is one confirmed counter-example, not a class: the *other* shape that appeared to | ||
| 113 | +regress under the same variant measured **−0.09 %** once isolated. Both facts belong here — | ||
| 114 | +the cost at the floor is real, and it is smaller and rarer than a batched screen suggests. | ||
| 115 | + | ||
| 116 | +So this is **not** a blanket recommendation. The gate is two predicates, both answerable | ||
| 117 | +before the rotation is written: | ||
| 118 | + | ||
| 119 | +1. the granule diagnostic moves the time by more than the noise floor, **and** | ||
| 120 | +2. the shape is not already at its move-only floor — measure that floor directly by running | ||
| 121 | + the same kernel with the vector body reduced to `load_align; store_align` | ||
| 122 | + (method and the per-SKU number: [arch-a5.md](arch-a5.md) § the measured vector GM floor). | ||
| 123 | + | ||
| 124 | +Both held on the two shapes that gained 40–47 % and neither held on the one that lost 4 %. | ||
| 125 | +Where the gate has to be decided per shape it wants a **compile-time rung** — a `tilingkey` | ||
| 126 | +field selecting a rotated body and an in-order body — rather than a runtime branch; see | ||
| 127 | +[../patterns/tiling-key-width-ladder.md](../patterns/tiling-key-width-ladder.md). | ||
| 128 | + | ||
| 129 | +## Probe | ||
| 130 | + | ||
| 131 | +One build, and no golden is needed, because the rotation is output-identical by construction: | ||
| 132 | + | ||
| 133 | +1. Write the kernel with per-core contiguous spans and a shard granule `PART` you can vary. | ||
| 134 | +2. Pick a shape whose working set is L2-resident or a small multiple of it. The effect is | ||
| 135 | + largest there and disappears into DRAM latency at HBM scale. | ||
| 136 | +3. Measure `PART` and `2 * PART` **one shape per process**, three rounds, reporting `min` | ||
| 137 | + next to the spread. A time difference far larger than the ~0.1 % span difference is the | ||
| 138 | + finding. | ||
| 139 | +4. Add the two-pass rotation and re-measure. Its output must be **bit-identical** to the | ||
| 140 | + in-order run; if any output word moved, the rotation is wrong and the memory system is not | ||
| 141 | + what you measured. | ||
| 142 | +5. Repeat step 3 on a shape you have separately shown to be at the move-only floor, and | ||
| 143 | + expect a few percent of cost there. That negative arm is what stops the finding from being | ||
| 144 | + generalised into a rule. | ||
| 145 | + | ||
| 146 | +## Related | ||
| 147 | + | ||
| 148 | +- [tiling.md](tiling.md) — launch geometry and per-core load balance. Balance and stride are | ||
| 149 | + independent quantities: the shapes above were already perfectly balanced. | ||
| 150 | +- [arch-a5.md](arch-a5.md) — the measured vector GM floor this page's gate refers to. | ||
| 151 | +- [../patterns/vec-elementwise-tile-and-balance.md](../patterns/vec-elementwise-tile-and-balance.md) | ||
| 152 | + — the partitioning that creates the lockstep span in the first place. | ||
| 153 | +- [vec-alignment-and-rotation.md](vec-alignment-and-rotation.md) — the UB-side pitch rules; | ||
| 154 | + different memory, different mechanism, not a substitute for the probe above. | ||
| @@ -18,6 +18,12 @@ updating the golden and acceptance criteria. | |||
| 18 | "Widening a narrow-dtype reduction" below. | 18 | "Widening a narrow-dtype reduction" below. |
| 19 | 4. For quantization, make scale, rounding, clamp range, and dequantization | 19 | 4. For quantization, make scale, rounding, clamp range, and dequantization |
| 20 | placement explicit in both kernel and golden. | 20 | placement explicit in both kernel and golden. |
| 21 | + Then **verify that every small constant survived code generation.** A float | ||
| 22 | + immediate below ~5e-7 — the magnitude a clamp floor, an epsilon or a denominator | ||
| 23 | + guard normally has — is emitted as a literal `0.0`, silently, so the guard is | ||
| 24 | + present in the source and absent from the kernel. See | ||
| 25 | + [vec.md](vec.md) § a float immediate below ~5e-7 for the mechanism and the | ||
| 26 | + multiplication workaround. | ||
| 21 | 5. Derive tolerances from the project standard for the actual output dtype and | 27 | 5. Derive tolerances from the project standard for the actual output dtype and |
| 22 | algorithm; never copy a tolerance from an unrelated sample. | 28 | algorithm; never copy a tolerance from an unrelated sample. |
| 23 | 6. Validate boundary values, tails, and representative random inputs. | 29 | 6. Validate boundary values, tails, and representative random inputs. |
| @@ -48,10 +54,24 @@ the current one: | |||
| 48 | softmax or normalization therefore has no native narrow path at all: widen to | 54 | softmax or normalization therefore has no native narrow path at all: widen to |
| 49 | FP32 before the exponential, not merely before the reduce. (Measured against | 55 | FP32 before the exponential, not merely before the reduce. (Measured against |
| 50 | the installed API docs on Ascend950PR / CANN 9.2.0.) | 56 | the installed API docs on Ascend950PR / CANN 9.2.0.) |
| 57 | + **`vf.div` has no BF16 row either** (installed `div` page, dtype table), | ||
| 58 | + which extends the rule from the exponential to the whole | ||
| 59 | + transcendental-or-divide class: widen to FP32 before **any** of them. A | ||
| 60 | + sigmoid-shaped activation is the case that catches people out, because it | ||
| 61 | + needs both — `exp` and then a reciprocal — so a bf16 chain that survived the | ||
| 62 | + reduce still has two more places to fail. | ||
| 51 | - The register-level cast is | 63 | - The register-level cast is |
| 52 | - `vf.astype(src, preg, *, dtype=..., layout=..., round_mode=..., saturate=...)`. | 64 | + `vf.astype(src, preg, *, dtype=..., layout=..., round_mode=..., saturate=..., mode=...)`. |
| 53 | The tile-level cast is `pl.cast(out, src, *, mode=...)`, where the target dtype | 65 | The tile-level cast is `pl.cast(out, src, *, mode=...)`, where the target dtype |
| 54 | - comes from `out`'s dtype. **There is no `vf.cast`.** | 66 | + comes from `out`'s dtype. **There is no `vf.cast`.** Note the two `mode=` kwargs |
| 67 | + are unrelated: at tile level it is the rounding mode, at register level it is the | ||
| 68 | + `MergeMode`, whose `MERGING` value the installed page marks | ||
| 69 | + 「当前不支持」. | ||
| 70 | +- 🔴 **`preg` is a mask of the SOURCE width**, for every `vf.astype` including the | ||
| 71 | + widening ones: *「mask_tensor根据输入的源操作数进行有效元素筛选」*. Passing a | ||
| 72 | + destination-width mask to a widening cast **returns zeros under `layout=ONE`, with | ||
| 73 | + no error** — see the negative control in | ||
| 74 | + [vec.md](vec.md#bf16-at-register-level-what-vfastype-can-and-cannot-do). | ||
| 55 | - **Both conversion APIs are platform-gated.** Their docs carry a | 75 | - **Both conversion APIs are platform-gated.** Their docs carry a |
| 56 | 「产品支持情况」 section, and support is not universal across Ascend | 76 | 「产品支持情况」 section, and support is not universal across Ascend |
| 57 | generations. Confirm availability for the *detected* target before a design | 77 | generations. Confirm availability for the *detected* target before a design |
| @@ -101,6 +121,71 @@ narrow register means **interleaving** the two results, not concatenating them. | |||
| 101 | `dtype=pl.DT_BF16` — which is also the evidence that the `vf.astype` doc | 121 | `dtype=pl.DT_BF16` — which is also the evidence that the `vf.astype` doc |
| 102 | table's silence on BF16 is a documentation gap, not a capability bound.) | 122 | table's silence on BF16 is a documentation gap, not a capability bound.) |
| 103 | 123 | ||
| 124 | +### Narrowing to INT8 has exactly one route, and the wrong one compiles | ||
| 125 | + | ||
| 126 | +The section above lists `vf.astype` and `pl.cast` as the two conversion APIs. | ||
| 127 | +For **FP32 → INT8** on Ascend950PR / CANN 9.2.0 neither of them is available, | ||
| 128 | +and the failure modes differ enough to matter: | ||
| 129 | + | ||
| 130 | +| candidate | result | | ||
| 131 | +|---|---| | ||
| 132 | +| `vf.astype` FP32 → INT8 | **absent** — no matching `vcvt` for the pair | | ||
| 133 | +| `pl.cast` FP32 → INT8 | **absent** — TCVT has no int8 destination | | ||
| 134 | +| `pl.cast` FP16 → INT8 | **compiles, and does not saturate**: `200` comes out as `−56` | | ||
| 135 | +| `pl.quant(out, src, scale, mode=pl.QuantMode.SYM)` | the route | | ||
| 136 | + | ||
| 137 | +🔴 **The only candidate that compiles is the one that wraps.** A design that | ||
| 138 | +reaches int8 by widening to FP16 first gets a clean build and silently modular | ||
| 139 | +arithmetic at the range boundary — and the boundary is where a quantizer spends | ||
| 140 | +its time. `pl.quant` saturates to `[−128, 127]` in hardware and rounds | ||
| 141 | +half-to-even (matching `torch.round`), so use it and check that the generated | ||
| 142 | +kernel contains no other narrowing. | ||
| 143 | + | ||
| 144 | +Note the rounding asymmetry while you are here: `pl.cast`'s default | ||
| 145 | +`CAST_ROUND → ROUND_A` is tie-away-from-zero and does **not** match | ||
| 146 | +`torch.round`; `mode=pl.RoundMode.CAST_RINT` is what agrees. | ||
| 147 | + | ||
| 148 | +### The max family propagates NaN — `maxNum` semantics are not used | ||
| 149 | + | ||
| 150 | +`vf.reduce_max`, `vf.max`, `vf.maxs` and `pl.maximum(dim=0)` **all** return NaN | ||
| 151 | +when either operand is NaN, measured on Ascend950PR / CANN 9.2.0. Only | ||
| 152 | +`vf.reduce_max` documents it. This is worth checking rather than assuming | ||
| 153 | +because the IEEE-754 `maxNum` alternative — *return the non-NaN operand* — is | ||
| 154 | +equally defensible and would silently convert an all-NaN row into a finite | ||
| 155 | +statistic, which the non-finite comparison rules below fail on position rather | ||
| 156 | +than on magnitude. `vf.abs` / `pl.abs` preserve NaN and map `−Inf → +Inf`. | ||
| 157 | + | ||
| 158 | +### The device's unsaturated `float → int` of a non-finite is not the host's | ||
| 159 | + | ||
| 160 | +The trap below says `float -> int` of a non-finite is undefined in C++ and | ||
| 161 | +differs between hosts. The device value is now measured: an **unsaturated** | ||
| 162 | +`float(+Inf) → int8` on Ascend950PR is **`−1`**, where an x86 host gives `0`. | ||
| 163 | +Both are legal; neither is a bug. The consequence is a design rule — | ||
| 164 | +**narrowing must saturate before the convert, not after** — and a validation | ||
| 165 | +rule: a hand-rolled narrowing path for non-finite inputs **cannot be validated | ||
| 166 | +on the host at all**, because the host answer is not evidence about the device. | ||
| 167 | +A primitive that saturates (see the INT8 table above) removes both problems. | ||
| 168 | + | ||
| 169 | +### A multiplier that must be exactly zero is not covered by a ULP budget | ||
| 170 | + | ||
| 171 | +When a reciprocal-and-multiply replaces a divide, the usual licence is a ULP | ||
| 172 | +sweep of the multiplier. That licence is valid on **finite** operands only, and | ||
| 173 | +the qualifier is routinely dropped when the result is restated. | ||
| 174 | + | ||
| 175 | +If any row's statistic can be `±Inf`, the exact multiplier is `k/Inf = +0.0`, | ||
| 176 | +and `±Inf × 0 = NaN` is the value the whole downstream mapping depends on. A | ||
| 177 | +fast reciprocal returning `+1` ULP — the smallest denormal — restores `±Inf`, | ||
| 178 | +which then saturates to the dtype extreme instead of mapping to zero: an error | ||
| 179 | +of the full output range, confined to exactly the rows a random sweep is least | ||
| 180 | +likely to weight. `−1` ULP is benign, so the failure is **one-sided** and a | ||
| 181 | +symmetric sweep reports a reassuring average over one case that cannot fail and | ||
| 182 | +one that always does. | ||
| 183 | + | ||
| 184 | +**Rule.** Where the reduced statistic can be non-finite, produce the multiplier | ||
| 185 | +with a true divide. Enforce it by grepping the *generated* kernels for the fast | ||
| 186 | +mnemonics (`vrec`, `vrsqrt`, a Newton iteration), not by reading the Python — | ||
| 187 | +a reciprocal can arrive through a helper or a backend rewrite. | ||
| 188 | + | ||
| 104 | ### A scale factor is not a substitute | 189 | ### A scale factor is not a substitute |
| 105 | 190 | ||
| 106 | Pre-scaling shifts the representable window; it cannot widen it. Compare the | 191 | Pre-scaling shifts the representable window; it cannot widen it. Compare the |
| @@ -255,7 +340,100 @@ and the last 5% to `+inf`, and `[nan, nan]` yields random values under a 50% NaN | |||
| 255 | mask. They are therefore reproducible offline — but `torch.randn` never reaches | 340 | mask. They are therefore reproducible offline — but `torch.randn` never reaches |
| 256 | these paths, so they need their own sweep. Note also that such a range applies to | 341 | these paths, so they need their own sweep. Note also that such a range applies to |
| 257 | *every* input it is declared for, so a non-finite can enter through a scale or | 342 | *every* input it is declared for, so a non-finite can enter through a scale or |
| 258 | -gamma vector rather than through the obvious data input. | 343 | +gamma vector rather than through the obvious data input. Establish **which** |
| 344 | +inputs it is declared for before relying on that: a **flat two-element** range — | ||
| 345 | +the form most contracts are written in — is declared for the first input only, | ||
| 346 | +which the next subsection is about. | ||
| 347 | + | ||
| 348 | +### A declared value range may not reach every input — check the generator | ||
| 349 | + | ||
| 350 | +**Trigger:** any argument of the form *"the inputs are symmetric about zero"*, | ||
| 351 | +*"a large fraction of outputs sit near zero"*, *"the data is bounded by ±1"* — | ||
| 352 | +i.e. any precision plan whose conclusion depends on the **distribution** of the | ||
| 353 | +inputs rather than on the arithmetic alone. It applies to multi-input operators; | ||
| 354 | +a single-input operator cannot hit it. | ||
| 355 | + | ||
| 356 | +**Rule: read what the generator does before reasoning about error regions.** | ||
| 357 | +A declared range and the data actually generated are two different things, and | ||
| 358 | +the gap is silent — the data is well-formed, in a plausible range, and nothing | ||
| 359 | +reports that four of five tensors were drawn from a distribution nobody chose. | ||
| 360 | + | ||
| 361 | +**Measured instance.** One generator, on a multi-input case set: a **flat | ||
| 362 | +two-element** `value_range` such as `[-1, 1]` is treated as the range of the | ||
| 363 | +**first input only**; the remaining inputs are padded with `None`, and `None` | ||
| 364 | +maps to the float default `(0.0, 1.0)`. So a contract that reads as "all inputs | ||
| 365 | +are U(-1,1)" produced one tensor at U(-1,1) and four at **U[0,1)**, mean ≈ 0.5. | ||
| 366 | +Confirmed twice: from the generator source, and by running the real generator and | ||
| 367 | +measuring the tensors. | ||
| 368 | + | ||
| 369 | +**Why it changes a decision rather than being trivia.** With the value tensor | ||
| 370 | +one-sided and non-negative, an attention output is a convex combination of | ||
| 371 | +non-negative numbers, so it concentrates near 0.5 and **essentially no output | ||
| 372 | +sits near zero**: the measured three-region census over 20 cases was **normal | ||
| 373 | +99.885–100.000 %**, small-value ≤ 0.006 %, cancellation ≤ 0.109 %. Under the | ||
| 374 | +symmetric distribution the contract *appeared* to declare, the same operator's | ||
| 375 | +census inverts — normal collapses to 0.0–70.5 %, cancellation rises to | ||
| 376 | +29.0–95.2 %, and two cases reach **0.000 % normal**. | ||
| 377 | + | ||
| 378 | +Two orders of magnitude of population moves between error regions on a reading of | ||
| 379 | +one line, and the region that dominates decides which bound is even reachable: | ||
| 380 | + | ||
| 381 | +- with the near-zero population empty, the small-value and cancellation regions | ||
| 382 | + are **never exercised**, and the only gate that can fire is the plain | ||
| 383 | + mean/max relative bound — the opposite of the usual advice, arrived at for the | ||
| 384 | + opposite reason; | ||
| 385 | +- a precision term justified as protection for cancellation is then | ||
| 386 | + **unmeasurable on the visible set**. Keep it if it is cheap and the hidden set | ||
| 387 | + may differ — but record it as *margin*, not as *demonstrated necessity*, or | ||
| 388 | + the next reader will delete it as an optimisation. | ||
| 389 | + | ||
| 390 | +**The check, in order:** (1) find the declared range in the case set; (2) find | ||
| 391 | +the generator function that consumes it and read how it normalises the field; | ||
| 392 | +(3) run the generator and measure per-input mean and min/max; (4) only then write | ||
| 393 | +down which error region dominates. Steps 1 and 4 alone are how the wrong premise | ||
| 394 | +survives review. | ||
| 395 | + | ||
| 396 | +Evidence class: **harness source read plus a live generator run**, one case set, | ||
| 397 | +2026-08-25. **Validation scope:** the flat-two-element form. A range declared | ||
| 398 | +per-input as a list of pairs is a different path and was not measured — do not | ||
| 399 | +assume it has the same defect, and do not assume it does not. | ||
| 400 | + | ||
| 401 | +### A lone ±Inf and a NaN do not have the same blast radius in a mean-of-squares normalisation | ||
| 402 | + | ||
| 403 | +Verified on NPU, the measuring card / Ascend950PR / CANN 9.2.0, 2026-08-14, in **fp32, fp16 and | ||
| 404 | +bf16 for both signs**, against `torch.nn.functional.rms_norm`, for | ||
| 405 | +`y = x / sqrt(mean(x²) + ε) · γ`: | ||
| 406 | + | ||
| 407 | +* a **NaN** anywhere in a row makes `mean(x²)` NaN, so **the whole row is NaN**; | ||
| 408 | +* a lone **±Inf** makes `mean(x²) = inf`, and `rsqrt(inf)` is exactly `+0.0`, so the **finite | ||
| 409 | + lanes of that row become exactly 0.0** and **only the Inf lane becomes NaN** (`inf * 0 = NaN`). | ||
| 410 | + | ||
| 411 | +A kernel that takes the shortcut "any non-finite in the row ⇒ NaN the whole row" is therefore | ||
| 412 | +wrong, and wrong in the way that costs the case outright: NaN positions are compared **before** | ||
| 413 | +MERE/MARE (previous subsection), so it fails on position mismatch rather than on error size. | ||
| 414 | + | ||
| 415 | +**The scored set cannot catch this, and that is the durable part.** In `rms_norm`'s public 20, | ||
| 416 | +the only case carrying `±inf` is `[1000003,2]` bf16 with `value_range [-inf, inf]`. The generator | ||
| 417 | +sets the first 5 % of the flat buffer to `-inf` and the last 5 % to `+inf`; with | ||
| 418 | +`n = 2000006 // 20 = 100000` **even** and `D = 2`, those blocks land exactly on row boundaries. | ||
| 419 | +Measured row census on the real case data: **900003 rows with 0 inf lanes, 100000 rows with 2, | ||
| 420 | +and none with 1.** No scored case contains a mixed Inf/finite row at all, so the blanket-NaN | ||
| 421 | +kernel scores **20/20 on the visible set and fails the hidden set**. Do not treat "20/20 on the | ||
| 422 | +public cases" as coverage for a non-finite rule; check the row structure the generator actually | ||
| 423 | +produces. | ||
| 424 | + | ||
| 425 | +*Getting it right is a matter of construction, not of a guard.* No clamp, no saturate, and in | ||
| 426 | +particular no `x == x` filter — `x == x` is **true** for ±Inf (see the trap above) and would | ||
| 427 | +manufacture a NaN where the reference has a finite `0.0`. Nothing in the chain may branch on | ||
| 428 | +non-finiteness: `x²` is never negative so the sum can never be `inf - inf`, `1/sqrt(+inf)` is | ||
| 429 | +exactly `+0.0`, and pad lanes must be neutralised by mask **zeroing**, never by multiply-by-zero | ||
| 430 | +— multiplying turns a stale NaN pad word into a row-poisoning NaN. | ||
| 431 | + | ||
| 432 | +*Detecting it takes a case the benchmark does not supply:* a row with exactly one `±Inf` among | ||
| 433 | +finite values, asserted with `==` and not `allclose` — exactly one NaN, at that column, and | ||
| 434 | +`(y[row][~nan] == 0).all()`. Repeat it with the Inf inside the ragged tail register and inside | ||
| 435 | +any packed segmented-reduction path, and run an all-NaN shape followed by a finite shape **in the | ||
| 436 | +same process** to prove pad lanes are zeroed rather than multiplied. | ||
| 259 | 437 | ||
| 260 | ## Evidence | 438 | ## Evidence |
| 261 | 439 | ||
| @@ -269,3 +447,71 @@ gamma vector rather than through the obvious data input. | |||
| 269 | 447 | ||
| 270 | Use `$PYPTO_DEVKIT_DIR/docs/pypto_pro/api/` as the primary source for supported | 448 | Use `$PYPTO_DEVKIT_DIR/docs/pypto_pro/api/` as the primary source for supported |
| 271 | dtype pairs and API semantics in the current SDK. | 449 | dtype pairs and API semantics in the current SDK. |
| 450 | + | ||
| 451 | +--- | ||
| 452 | + | ||
| 453 | +## BF16 narrowing at register level is bit-exact with the tile cast | ||
| 454 | + | ||
| 455 | +The note above says the register-level cast is `vf.astype(...)` and the tile-level | ||
| 456 | +one is `pl.cast(out, src, mode=...)`, and warns that both are platform-gated. For | ||
| 457 | +the **FP32 → BF16** direction on A5 / DAV_3510 the gate has been measured, and the | ||
| 458 | +answer is favourable enough to change how a BF16 output should be published: | ||
| 459 | + | ||
| 460 | +* the two produce **identical bytes**, verified on inputs that are exact BF16 ties | ||
| 461 | + (low 16 mantissa bits `== 0x8000`) as well as random values; | ||
| 462 | +* the `round_mode` kwarg does not reach the instruction — the hardware always | ||
| 463 | + rounds nearest-even, which is what `CAST_RINT` asks for anyway; | ||
| 464 | +* `CAST_RINT` and `CAST_HYBRID` do not compile for this pair at all. | ||
| 465 | + | ||
| 466 | +So a chain that computes in FP32 and ends in a single tile-level narrowing cast can | ||
| 467 | +instead publish BF16 straight from the register file, deleting a whole UB→UB pass | ||
| 468 | +(and its `auto_mutex` barrier) **without changing a single output byte**. Measured | ||
| 469 | +on `rms_norm`: publish traffic per element falls from 10 B (4 B FP32 store + 4 B | ||
| 470 | +cast read + 2 B cast write) to 2 B. | ||
| 471 | + | ||
| 472 | +### The reverse direction is symmetric: BF16 → FP32 reaches both lane parities | ||
| 473 | + | ||
| 474 | +A design that needs register-level widening does not have to be routed into a tile-level | ||
| 475 | +`pl.cast`. | ||
| 476 | + | ||
| 477 | +**Measured on Ascend950PR (`DAV_3510`), CANN 9.2.0, 2026-08-25, by two independent probes** — | ||
| 478 | +the second written from scratch rather than re-run — with a **source-width (b16)** mask: | ||
| 479 | + | ||
| 480 | +| `vf.astype(b16_reg, preg_b16, dtype=pl.DT_FP32, layout=…)` | result | | ||
| 481 | +|---|---| | ||
| 482 | +| `CastLayout.ZERO` | the **even** source elements, 64/64 lanes non-zero | | ||
| 483 | +| `CastLayout.ONE` | the **odd** source elements, 64/64 lanes non-zero | | ||
| 484 | + | ||
| 485 | +for **bf16 and fp16 alike**, and the widen → narrow round trip is **bit-identical over all | ||
| 486 | +128 lanes**. A `ZERO` + `ONE` pair therefore covers a whole narrow register with two | ||
| 487 | +instructions and needs no `unpack`, no store-and-reload, and no tile-level pass. | ||
| 488 | + | ||
| 489 | +**The mask width is the whole trap, and it is the transferable part.** A | ||
| 490 | +**destination-width (b32)** mask on a widening cast selects nothing under `layout=ONE`, and | ||
| 491 | +the instruction returns zeros — **0/64 lanes non-zero, silently, with no error and no | ||
| 492 | +warning.** One such reading looks exactly like an ISA that can only widen the even elements. | ||
| 493 | +The installed `astype` page states the rule: the mask filters by the **source** operand. | ||
| 494 | + | ||
| 495 | +**Keep the negative control as the diagnostic.** Running the b32/`ONE` arm next to the | ||
| 496 | +b16/`ONE` arm is what distinguishes "the hardware cannot do this" from "we asked wrongly", and | ||
| 497 | +it costs one extra arm: | ||
| 498 | + | ||
| 499 | +| mask width | `ZERO` | `ONE` | | ||
| 500 | +|---|---|---| | ||
| 501 | +| b16 (source width) | match | **match** | | ||
| 502 | +| b32 (destination width) | match | **all zeros** | | ||
| 503 | + | ||
| 504 | +The 2x2 is the point: sweeping the `layout` enum while holding the mask width fixed samples | ||
| 505 | +one column and reads a broken cell as a property of the enum. **When an operation has two | ||
| 506 | +type-carrying operands and one enum, sweep the cross product.** | ||
| 507 | + | ||
| 508 | +**What the open route does not change.** The only register-level bit reinterpretation, | ||
| 509 | +`vf.bit_cast`, is **same-element-width only**, so the "BF16 is the top 16 bits of FP32" trick | ||
| 510 | +is not expressible; and folding a tile-level *width-changing* cast into the vector function | ||
| 511 | +measured negative on a compute-exposed kernel — the *route* is open, the *payoff* is | ||
| 512 | +shape-dependent. On one shape class it does pay: a *same-width* fold that deletes two | ||
| 513 | +whole-tile staging passes measured up to 16 % faster | ||
| 514 | +([../patterns/vec-compute-exposed-narrow-dtype.md](../patterns/vec-compute-exposed-narrow-dtype.md)). | ||
| 515 | +The lane mechanics of | ||
| 516 | +both directions, and the narrowing side's op-count floor and the precondition it needs, are in | ||
| 517 | +[vec.md](vec.md#bf16-at-register-level-what-vfastype-can-and-cannot-do). | ||
| @@ -2,42 +2,419 @@ | |||
| 2 | 2 | ||
| 3 | ## Rule | 3 | ## Rule |
| 4 | 4 | ||
| 5 | -Use `make_tile_group(..., auto_mutex=True)` for rotation and intra-kernel | 5 | +Automatic ordering for rotation and intra-kernel buffer ownership is armed by |
| 6 | -buffer ownership that the target API documents as auto-managed. Do not add | 6 | +two separate things: `auto_mutex` on the **`@pl.jit` decorator**, and explicit |
| 7 | -manual synchronization to the same managed dependency without evidence. | 7 | +`mutex_ids=` on the `make_tile_group` it is to order. Use it for the buffer |
| 8 | +ownership the target API documents as auto-managed, and do not add manual | ||
| 9 | +synchronization to the same managed dependency without evidence. | ||
| 8 | 10 | ||
| 9 | Cross-section or AIC/AIV handoff is not implied by `auto_mutex`. When the data | 11 | Cross-section or AIC/AIV handoff is not implied by `auto_mutex`. When the data |
| 10 | flow crosses engines or sub-blocks, copy the producer/consumer event sequence | 12 | flow crosses engines or sub-blocks, copy the producer/consumer event sequence |
| 11 | from a matching official example for the installed SDK and validate it on the | 13 | from a matching official example for the installed SDK and validate it on the |
| 12 | target. | 14 | target. |
| 13 | 15 | ||
| 16 | +### How automatic ordering is armed, and how it is silently lost | ||
| 17 | + | ||
| 18 | +Getting the spelling wrong fails silently rather than loudly. Read against the | ||
| 19 | +installed parser on CANN 9.2.0: | ||
| 20 | + | ||
| 21 | +- `auto_mutex` is a **`@pl.jit` decorator kwarg** (`auto_mutex: bool = True`), | ||
| 22 | + not a `make_tile_group` kwarg. | ||
| 23 | +- `make_tile_group` accepts exactly four keywords — `type=`, `addrs=`, | ||
| 24 | + `mutex_ids=`, `depth=`. The parser collects the call's keywords into a dict | ||
| 25 | + and then reads those four by name. **It never checks for unknown ones**, so | ||
| 26 | + any other keyword — `auto_mutex=`, or the singular misspelling `mutex_id=`, or | ||
| 27 | + a typo — is **dropped without a diagnostic**. | ||
| 28 | +- What `mutex_ids` buys is the ordering metadata itself. A group declared with | ||
| 29 | + `depth=` and no usable `mutex_ids` is legal, rotates correctly, and carries | ||
| 30 | + **no mutex ids for `auto_mutex` to order against**. Rotation and | ||
| 31 | + synchronization are two separate things, and only one of them is what the | ||
| 32 | + ids provide. | ||
| 33 | + | ||
| 34 | +Put together, the failure is: a misspelled or misplaced keyword produces a group | ||
| 35 | +that looks synchronized in the source, rotates in the generated code, and has | ||
| 36 | +exactly the read-after-write exposure of a **bare `make_tile`** — the failure | ||
| 37 | +signature already recorded on | ||
| 38 | +[../references/pitfall-records.md](../references/pitfall-records.md) as | ||
| 39 | +"computed bare tile reads stale data, no error". | ||
| 40 | + | ||
| 41 | +Two checks worth putting in a gate, because neither costs a device: | ||
| 42 | + | ||
| 43 | +1. **Every rotating group in a cross-op dependency carries explicit | ||
| 44 | + `mutex_ids`.** Assert the count in a source check rather than eyeballing it — | ||
| 45 | + and scope the check to code, since a `mutex_ids=` written in a comment or a | ||
| 46 | + docstring inflates a naive `grep -c`. | ||
| 47 | +2. **`auto_mutex=` appears on the `@pl.jit` decorator and nowhere else.** Its | ||
| 48 | + default is `True` on this build, so an author who "set it on the group" gets | ||
| 49 | + the behaviour they wanted by accident and never learns the keyword did | ||
| 50 | + nothing — until the day the same idiom is copied onto a kernel that sets it | ||
| 51 | + `False`. | ||
| 52 | + | ||
| 53 | +#### The API page documents six keywords; the Python signature takes four | ||
| 54 | + | ||
| 55 | +`make_tile_group`'s **Python signature** takes only | ||
| 56 | +`type` / `addrs` / `mutex_ids` / `depth` and carries **no `**kwargs`**, so | ||
| 57 | +`auto_mutex=` there is silently dropped. The installed API *page* for | ||
| 58 | +`make_tile_group` nevertheless documents **six** keywords, adding `fwd_ids` and | ||
| 59 | +`bwd_ids`, and a shipped sample passes them. | ||
| 60 | + | ||
| 61 | +Both are right, and the reconciliation is worth stating because "the docs and | ||
| 62 | +the signature disagree" is otherwise read as one of them being stale: | ||
| 63 | + | ||
| 64 | +- `fwd_ids` / `bwd_ids` are read by the **AST parser**, not by the Python | ||
| 65 | + signature, which is why they work despite not appearing in `inspect.signature`. | ||
| 66 | +- They take effect **only when the generated pipeline is enabled**. On a kernel | ||
| 67 | + with a hand-written handoff they are accepted and inert. | ||
| 68 | + | ||
| 69 | +So the same call has **two different silent-ignore mechanisms** on it — one for | ||
| 70 | +keywords the parser does not know (`auto_mutex=`), one for keywords it knows but | ||
| 71 | +that the enabled transform never consumes (`fwd_ids`/`bwd_ids`). Neither warns. | ||
| 72 | +Evidence class: **installed source + installed API page**, CANN 9.2.0, read | ||
| 73 | +2026-08-25. | ||
| 74 | + | ||
| 75 | +--- | ||
| 76 | + | ||
| 77 | +## The hand-written per-tile cross-core handoff runs | ||
| 78 | + | ||
| 79 | +**What was measured.** On a5 / `DAV_3510`, CANN 9.2.0 (cann master `f0b3a2609`): | ||
| 80 | +one `@pl.jit` holding `section_cube` + `section_vector`, a **hand-written | ||
| 81 | +per-tile** `pl.system.set_cross_core` / `wait_cross_core` handoff, **6 event | ||
| 82 | +groups / 12 ids in `[0,16)`**, and two probability tiles in L1 written by the | ||
| 83 | +Vector side and read by the Cube side — **compiled, launched, and computed | ||
| 84 | +correct attention. It never hung, not once**, over a bring-up ladder, repeated | ||
| 85 | +runs, and a 20-case suite. On runs where the race below did not fire, the | ||
| 86 | +bring-up rungs matched the golden to `max_abs_error = 2.44e-4` — half the fp16 | ||
| 87 | +1-ULP budget at that output range — and an analytic causal-prefix probe returned | ||
| 88 | +the answer **exactly**, `max_abs_error = 0.0`. The immediately preceding campaign | ||
| 89 | +built the same construct for the same computation and went **0-for-13**: | ||
| 90 | +thirteen hypotheses, nine mutation-ladder rungs, `aicore timeout` every time, | ||
| 91 | +even though every ingredient passed in isolation — the cube half (including the | ||
| 92 | +transposed NT load and a dual-accumulator contraction) and the register-level | ||
| 93 | +softmax were each proven separately. A reference implementation written against | ||
| 94 | +a different kernel language hit the identical wall on the identical construct, | ||
| 95 | +so that failure is not an artifact of one language's lowering. | ||
| 96 | + | ||
| 97 | +> **Evidence class: measured on device.** One operator, one build, one session. | ||
| 98 | +> **Validation scope:** this establishes that the *construct* compiles, launches, | ||
| 99 | +> terminates, and can compute the right answer. It does **not** establish | ||
| 100 | +> shipping correctness: the same kernel carries an unresolved cube↔vector race | ||
| 101 | +> (next subsection), so every value above is a best-run figure and the same rung | ||
| 102 | +> can be an order of magnitude worse on the next run; 4 of 20 cases landed | ||
| 103 | +> outside tolerance, and the ladder aborted at the first failing rung, so the | ||
| 104 | +> upper rungs were never exercised. **"The construct runs" is the claim; "the | ||
| 105 | +> construct is solved" is not.** | ||
| 106 | +> | ||
| 107 | +> **What the analytic probe's `0.0` does and does not support.** That figure comes | ||
| 108 | +> from a causal-prefix probe whose operands are **degenerate by construction** — | ||
| 109 | +> `q = k = 0` with `v[…, j, :] = j`, giving the analytically known expectation | ||
| 110 | +> `y[i] = (prefix(i) − 1)/2`. It was built to pin the *length* of the causal | ||
| 111 | +> prefix, because the plain zero-input case turned out not to exercise the mask, | ||
| 112 | +> and for that purpose it is sound. | ||
| 113 | +> | ||
| 114 | +> - ✅ **Valid for the claim it is cited under.** Returning the analytically known | ||
| 115 | +> answer *exactly*, across both layouts and both causal settings, is not | ||
| 116 | +> something a kernel that hung, faulted, or computed the wrong reduction could | ||
| 117 | +> do. Liveness and the value chain are established by it. | ||
| 118 | +> - ❌ **Not valid as evidence of determinism**, of correct probability-tile | ||
| 119 | +> *ordering*, or of the absence of a race. With `q = k = 0` every score is | ||
| 120 | +> equal, so any permutation or duplication of probability tiles is numerically | ||
| 121 | +> invisible. This probe read clean on the control *and* on a kernel later | ||
| 122 | +> measured non-deterministic in five of five configurations. | ||
| 123 | +> | ||
| 124 | +> The generalisable form, and this is a clean instance of it: **a probe designed | ||
| 125 | +> to verify one property can be silently useless for another, and degenerate | ||
| 126 | +> operands are the usual reason.** The design was not the error; reading it as | ||
| 127 | +> reassurance about synchronisation was. See | ||
| 128 | +> [Locate a numerical error in a multi-stage kernel](../playbooks/numerical-error-localisation.md) | ||
| 129 | +> §2b. | ||
| 130 | + | ||
| 131 | +**Corroboration from the installed samples.** Of the 18 files under the | ||
| 132 | +installed `frontend/fa/` tree that call `set_cross_core`, **17 carry a single | ||
| 133 | +`@pl.jit` holding both sections**, and the handoff is genuinely *per tile* — | ||
| 134 | +set/wait pairs inside nested `pl.range` loops, not once per kernel. One of them, | ||
| 135 | +`frontend/fa/test_fa_perf_tkv_preload_dn_vf_bufid_bsnd.py`, closes with a | ||
| 136 | +**20-iteration determinism loop plus `assert_close(rtol=5e-3, atol=5e-3)`**. The | ||
| 137 | +two files that are *not* in the 17 are correctly excluded: one takes the | ||
| 138 | +`PipelineConfig` route, one stages through GM. | ||
| 139 | + | ||
| 140 | +### Attribution — one of the three differences is established, two are inference | ||
| 141 | + | ||
| 142 | +Three things differed from the attempts that produced `aicore timeout`. The | ||
| 143 | +page states which is which because a reader who picks the wrong one will | ||
| 144 | +"reproduce" the fix and get nothing. | ||
| 145 | + | ||
| 146 | +1. **Established.** The `slots >= blocks` claim inside framework finding | ||
| 147 | + *"Close the accumulator: the last matmul of a K-loop needs `AccPhase.Final`"* | ||
| 148 | + was **retracted** — *"two slots suffice at any depth"*, verified to K=7168. | ||
| 149 | + The failed design's structural-impossibility argument rested on it | ||
| 150 | + explicitly, so with it void the L0 budget moves off the ceiling to | ||
| 151 | + **L0A 50 % / L0B 50 % / L0C 25 %** and a 2-slot rotation becomes legal at any | ||
| 152 | + contraction depth. This is a fact about the budget, not about the sync. | ||
| 153 | +2. **Established that it changed; its causal role is inference.** The | ||
| 154 | + pipeline/sync engine was **rewritten** on this build, with **byte-identical | ||
| 155 | + `PipelineConfig` defaults** — so nothing in the API surface signals that the | ||
| 156 | + machinery underneath is different. An agent comparing two builds by their | ||
| 157 | + configuration will conclude, wrongly, that nothing moved. | ||
| 158 | +3. **Established as a removed defect; its causal role is inference.** `phase=` | ||
| 159 | + was never armed anywhere in the new design. See the section | ||
| 160 | + "`phase=` is a hardware handshake, and only `pl.store` can answer it" below — | ||
| 161 | + a mis-set `phase` is a hang cause with no diagnostic, so its absence is a real | ||
| 162 | + change, but nothing in this run isolates it as *the* change. | ||
| 163 | + | ||
| 164 | +Nothing separates the three. **Do not report any one of them as "the fix."** | ||
| 165 | + | ||
| 166 | +### The bring-up ladder is mandatory, not advisory | ||
| 167 | + | ||
| 168 | +When this construct does fail, it fails as an **opaque | ||
| 169 | +watchdog**. `aicore timeout` names the watchdog, not the mechanism. So add | ||
| 170 | +exactly one mechanism per rung and make **rung 1 validate the cube↔vector | ||
| 171 | +handshake alone** — one query tile, one KV step, no online recurrence, no mask, | ||
| 172 | +no tail. A ladder built that way localises a hang to the rung that introduced it; | ||
| 173 | +without one, you are back to enumerating hypotheses, which is how thirteen were | ||
| 174 | +spent. | ||
| 175 | + | ||
| 176 | +### Still open: a cube↔vector race that survives inside a single reduction chunk of a single task | ||
| 177 | + | ||
| 178 | +Recorded as an **open, characterised defect**, because the characterisation is | ||
| 179 | +the most reusable thing this campaign produced. It is not solved. | ||
| 180 | + | ||
| 181 | +**Symptom.** The kernel is **non-deterministic** — repeated runs on identical | ||
| 182 | +input differ. Severity is **heavy-tailed**, not a bounded-outlier problem: | ||
| 183 | + | ||
| 184 | +| observed | figure | | ||
| 185 | +|---|---| | ||
| 186 | +| exact | `max_abs_error` 2.44e-4 (half the fp16 1-ULP budget at that output range) | | ||
| 187 | +| mild | ~1.5 % of elements mismatched | | ||
| 188 | +| whole-tensor garbage | `matched_ratio = 0.000015`, `max_abs_error = 0.9956` — fifteen matching elements per million, at the full output dynamic range | | ||
| 189 | +| NaN | an analytic probe reported **22400 NaN position mismatches** | | ||
| 190 | + | ||
| 191 | +**The NaN route is the diagnostic one.** It means the race can corrupt the | ||
| 192 | +online-softmax statistics into a **zero or negative denominator** reaching | ||
| 193 | +`pl.expand_div`. A design argument that "the row sum is structurally ≥ 1" is a | ||
| 194 | +statement about the *intended* dataflow and **does not survive a race** — under | ||
| 195 | +corruption the invariant is simply not true. Treat any structural non-negativity | ||
| 196 | +or non-zero argument as void while a race is open. | ||
| 197 | + | ||
| 198 | +**Localisation already established — the method is the reusable part.** | ||
| 199 | + | ||
| 200 | +- A **determinism matrix** over 6 configurations, varying task count, KV-tile | ||
| 201 | + count and head dim independently, run repeatedly on identical input and scored | ||
| 202 | + by mismatch count rather than by pass/fail. | ||
| 203 | +- One **provable** defect was found and fixed by a full buffer live-range audit: | ||
| 204 | + a tile group's `.next()` cursor **free-runs across tasks** while a `kt % depth` | ||
| 205 | + event index **restarts each task**. The two agree only when the KV-tile count | ||
| 206 | + is even, so a release credit issued for slot 0 can authorise overwriting | ||
| 207 | + slot 1. **Fix:** index every event group by a **free-running counter that | ||
| 208 | + advances in lockstep with the cursor**, declared identically in both sections, | ||
| 209 | + never by `iteration % depth`. | ||
| 210 | +- **That fix did not remove the race**, and the decisive observation is that | ||
| 211 | + after it a **single-task** configuration — no task boundary at all — began | ||
| 212 | + racing. That excluded the task boundary, buffer lifetime/`depth`, and event | ||
| 213 | + pairing (re-derived balanced on both sides). It was then read as one remaining | ||
| 214 | + defect *inside one task's cube↔vector handshake*; the next subsection is the | ||
| 215 | + probe that split that reading in two. | ||
| 216 | + | ||
| 217 | +**Ruled out by measurement — do not re-spend these.** Buffer live range vs | ||
| 218 | +`depth` across every buffer; event set/wait balance; the task-boundary | ||
| 219 | +cursor/event misalignment (a real defect, found, fixed, race survived); | ||
| 220 | +`enable_slice` (two subsections down); and now **the task axis itself** — the | ||
| 221 | +residual race is present at 1, 2 and 128 tasks and absent at 1 and 8 tasks, so | ||
| 222 | +task count neither causes it nor prevents it. The dose-response sweep two | ||
| 223 | +subsections down adds three more: **`auto_mutex`**, the **producer-local | ||
| 224 | +contraction loop**, and every **cross-chunk** mechanism — cross-chunk recurrence, | ||
| 225 | +slot wraparound, event-counter parity — none of which exist in the minimal | ||
| 226 | +reproducer that still races. | ||
| 227 | + | ||
| 228 | +#### Two defects, separated by the head dimension | ||
| 229 | + | ||
| 230 | +**The intervention.** The lag-1 skew was collapsed: the consumer section waits on | ||
| 231 | +its producer in the **same** iteration rather than one behind. Six line edits — a | ||
| 232 | +loop trip count, two guards, two index expressions — and **no compensating change | ||
| 233 | +anywhere else**, so the inference is unconfounded. Recompilation was established | ||
| 234 | +before any result was read, three ways: a unique kernel name, a wiped build | ||
| 235 | +directory, and **328 differing lines** of emitted C++. | ||
| 236 | + | ||
| 237 | +**The matrix.** a5 / `DAV_3510`, CANN 9.2.0, fp16, identical inputs, outputs | ||
| 238 | +compared bitwise between runs. | ||
| 239 | + | ||
| 240 | +| task count | inner unroll trip count over the head dim | before collapse | after collapse | | ||
| 241 | +|---|---|---|---| | ||
| 242 | +| 1 | 2 | races | **clean** — 0 of 15 pairs differ, 8 launches, 2 processes | | ||
| 243 | +| 8 | 2 | races — **5 distinct outputs in 6 runs** | **clean** — 0 of 15 pairs differ | | ||
| 244 | +| 1 | 8 | races | **races** | | ||
| 245 | +| 2 | 8 | races | **races** | | ||
| 246 | +| 128 | 8 | races | **races** | | ||
| 247 | + | ||
| 248 | +Task count spans 1→8 across both clean cells and 1→2→128 across all three racing | ||
| 249 | +cells, and partitions neither. **Every remaining failure has the inner trip count | ||
| 250 | +at 8; every success has it at 2.** The correlation is exact, and it is the whole | ||
| 251 | +of the result. | ||
| 252 | + | ||
| 253 | +🔴 **Read "clean" in that table at the sample size that produced it.** Those cells | ||
| 254 | +are 6 runs / 15 pairs, and the sweep below establishes that at **16** runs five of | ||
| 255 | +its eight points read as false negatives. So the two clean cells bound their race | ||
| 256 | +rate loosely and **do not establish determinism**; what survives unweakened is the | ||
| 257 | +*contrast* between cells measured the same way, and the order-of-magnitude | ||
| 258 | +accuracy improvement, which is a separate observable. The sweep's `N <= 3` points | ||
| 259 | +are clean at 3516 runs each, which is *consistent* with those cells but is not a | ||
| 260 | +re-measurement of them — it was run on a different, minimal configuration. See | ||
| 261 | +[Investigation discipline](../references/investigation-discipline.md) §18. | ||
| 262 | + | ||
| 263 | +Two conclusions follow, and they differ in kind: | ||
| 264 | + | ||
| 265 | +1. **The lag-1 skew is a genuine, demonstrated defect.** Collapsing it made two | ||
| 266 | + configurations bit-stable and, in those configurations, improved accuracy by | ||
| 267 | + an order of magnitude (`max_abs_error` 2.6e-3 → 2.0e-4). **It is not adopted | ||
| 268 | + as a fix**: collapsing the lag forfeits the cube/vector overlap the construct | ||
| 269 | + exists for. It is a *diagnostic*, and a reusable one — recorded as such on | ||
| 270 | + [Buffer lifetime and rotating tile groups](../patterns/buffer-reuse-lifetime.md). | ||
| 271 | +2. **A second, independent defect remains, and it scales with the head | ||
| 272 | + dimension.** The task axis is exonerated **by measurement, not by argument**. | ||
| 273 | + | ||
| 274 | +The probe was designed as a binary discriminator: determinism returning would put | ||
| 275 | +the defect in the skew construction, not returning would put it in the handshake | ||
| 276 | +primitives. Determinism returned **at one trip count and not at another**, which | ||
| 277 | +neither branch anticipated. A cheap discriminator is worth running, but its | ||
| 278 | +premise is a hypothesis too. | ||
| 279 | + | ||
| 280 | +A 2-run pair is not a determinism measurement. The `8 tasks / trip count 2` cell | ||
| 281 | +reads deterministic on one pair and produces **five distinct outputs in six | ||
| 282 | +runs**, because it holds exactly one identical pair out of fifteen — and six runs | ||
| 283 | +is not a determinism measurement either. The run-count and equivalence-class | ||
| 284 | +protocol that follows is general method and lives on | ||
| 285 | +[Investigation discipline](../references/investigation-discipline.md) §18, not | ||
| 286 | +here. | ||
| 287 | + | ||
| 288 | +#### `auto_mutex` is exonerated, and the whole defect fits in a minimal reproducer | ||
| 289 | + | ||
| 290 | +A dose-response sweep over the head dimension was run **on the minimal | ||
| 291 | +reproducer** — single task, single reduction chunk, lag already collapsed. | ||
| 292 | + | ||
| 293 | +**What was varied, and why it discriminates.** The head dimension `D`, over eight | ||
| 294 | +points. Two structures scale with `D` and were confounded: | ||
| 295 | + | ||
| 296 | +- an **inner unroll**, trip count `N = D/64`, issuing one forward/backward | ||
| 297 | + handshake pair per iteration **across the section boundary** against **2-slot** | ||
| 298 | + groups, with four other cursors also advancing once per iteration through their | ||
| 299 | + own 2-slot groups; | ||
| 300 | +- a **producer-local contraction loop**, trip count `M = D/128`, lying entirely | ||
| 301 | + inside the producer section and ordered by **`auto_mutex`**. | ||
| 302 | + | ||
| 303 | +`N` is monotone `1..8` across the sweep; `M` is a staircase `1,1,2,2,3,3,4,4`. | ||
| 304 | +That difference is the whole discriminator, and it needs **no source edit**. Four | ||
| 305 | +passes at rising run counts (16 / 100 / 400 / 3000), separate processes, and a | ||
| 306 | +**separately compiled binary per point** — the build cache is keyed per tiling | ||
| 307 | +key, so every point ran its own freshly compiled code. | ||
| 308 | + | ||
| 309 | +| `D` | `N` | `M` | runs | runs deviating from the modal output | verdict | | ||
| 310 | +|---|---|---|---|---|---| | ||
| 311 | +| 64 / 128 / 192 | 1 / 2 / 3 | 1 / 1 / 2 | 3516 each | **0** | clean | | ||
| 312 | +| 256 | 4 | 2 | 516 | 159 (~31 %) | races | | ||
| 313 | +| 320 | 5 | 3 | 3516 | 2 | races | | ||
| 314 | +| 384 | 6 | 3 | 3516 | 9 | races | | ||
| 315 | +| 448 | 7 | 4 | 3516 | 5 | races | | ||
| 316 | +| 512 | 8 | 4 | 523 | 71 | races | | ||
| 317 | + | ||
| 318 | +**Onset at `N = 4`, perfectly monotone in `N`.** The decisive pair is | ||
| 319 | +`D = 192` against `D = 256`: **`M` is held constant at 2 across it** and the | ||
| 320 | +behaviour splits — 0 deviants in 3516 runs against 159 in 516. Every step the `M` | ||
| 321 | +staircase predicts is crossed with **no** change in behaviour. The three clean | ||
| 322 | +points pool to **0 in 10548 runs**; against the lowest racing rate seen anywhere | ||
| 323 | +in the sweep that is P ≈ 3e-7. | ||
| 324 | + | ||
| 325 | +**Verdict.** | ||
| 326 | + | ||
| 327 | +- **`auto_mutex` is exonerated, and the producer-local contraction loop with it.** | ||
| 328 | + `auto_mutex` delivers the ordering it promises. The residual is therefore a | ||
| 329 | + **synchronisation-design defect** — a credit authorising more cursor advances | ||
| 330 | + than its depth — and **not a framework bug**. That distinction is exactly why | ||
| 331 | + the alternative could not be assumed away, and it is now settled by measurement | ||
| 332 | + rather than by argument. | ||
| 333 | +- **The minimal reproducer still races.** One task, one reduction chunk, no | ||
| 334 | + cross-chunk recurrence, no slot wraparound, no event-counter parity — and still | ||
| 335 | + up to **4 distinct outputs**, the worst pair differing in ~46 % of the output | ||
| 336 | + elements. The whole defect fits inside one chunk of one task, so it is | ||
| 337 | + **bisectable by shortening the inner loop**: the ordinary method is available | ||
| 338 | + again. | ||
| 339 | +- The race can produce **NaN** (1536 of them in one pass). It can destroy an | ||
| 340 | + online-softmax running max/sum outright, not merely perturb summation order — | ||
| 341 | + which confirms, rather than supersedes, the NaN reading recorded above. | ||
| 342 | + | ||
| 343 | +**Mechanistic corroboration, offered as observation and not as proof.** The onset | ||
| 344 | +`N = 4` is exactly **twice the 2-slot depth**, and `N <= 3` is the regime in which | ||
| 345 | +those 2-slot groups never complete a second wrap inside a chunk. The story fits; | ||
| 346 | +nothing in this sweep tests it. | ||
| 347 | + | ||
| 348 | +🔴 **The limit on the inference — do not drop this half when quoting the result.** | ||
| 349 | +`N` is in **bijection with `D`** in this design, so **the sweep cannot falsify | ||
| 350 | +`N`**. What it measures is the **exoneration of `M`**, the only candidate carrying | ||
| 351 | +a falsifiable staircase, and of `auto_mutex`. `N` is merely left standing. The | ||
| 352 | +positive attribution to the inner unroll therefore rests on a **two-candidate | ||
| 353 | +frame**: any other structure 1-to-1 with `D` would fit this data equally well. | ||
| 354 | +**The directly measured results are the two exonerations.** The inner unroll was | ||
| 355 | +not positively identified by measurement. The **general** rule this instance | ||
| 356 | +produced — what a dose-response sweep can and cannot falsify, and how to state an | ||
| 357 | +attribution that is conditional on a candidate list — is on | ||
| 358 | +[Investigation discipline](../references/investigation-discipline.md) §19, not here. | ||
| 359 | + | ||
| 360 | +**What remains — one step, not a ladder. Bisect the inner loop inside the minimal | ||
| 361 | +reproducer.** Shorten it, or split its handshake pairs, and find the trip count at | ||
| 362 | +which determinism returns. That is now an ordinary bisection on the cheapest | ||
| 363 | +configuration available, and it is also the only move that can break the | ||
| 364 | +two-candidate frame, because it varies `N` independently of `D`. | ||
| 365 | + | ||
| 366 | +🔴 **Run counts, before any of this page's determinism wording is reused.** At | ||
| 367 | +**16 runs, five of these eight points read as false negatives**; two of them still | ||
| 368 | +looked deterministic at **400** runs and only revealed themselves at **3000**. | ||
| 369 | +"Clean" here is an **upper bound on the rate** (~0.03 % at `N <= 3`), never zero. | ||
| 370 | +Rates are also strongly **ambient-dependent** — a co-tenant workload ran | ||
| 371 | +throughout — so they do not compare across passes, and every conclusion above | ||
| 372 | +rests on presence/absence and on **within-process adjacent contrasts**, never on a | ||
| 373 | +ratio of rates. The general rule this forced is on | ||
| 374 | +[Investigation discipline](../references/investigation-discipline.md) §18. | ||
| 375 | + | ||
| 376 | +### `enable_slice` looks like the missing workaround. It is not. | ||
| 377 | + | ||
| 378 | +Many working fused cube+vector samples carry | ||
| 379 | +`@pypto.options(pass_options={"enable_slice": False})`, immediately above the | ||
| 380 | +kind of kernel that is racing, which reads like a required workaround for | ||
| 381 | +exactly this class. Measured on the installed build, it is a dead end: | ||
| 382 | + | ||
| 383 | +- the installed configuration documentation states *"`enable_slice : bool` — | ||
| 384 | + Whether to enable slice-related processing. **Defaults to False.**"*; | ||
| 385 | +- the shipped framework config file sets `"enable_slice": false`. | ||
| 386 | + | ||
| 387 | +**The samples are pinning a value it already has** — plausibly because the | ||
| 388 | +options setter is process-global, so another test in the same session could flip | ||
| 389 | +it. And it decorates a **pytest test function**, not a kernel, so it would not | ||
| 390 | +travel with a delivered operator even if it did something. Adopting it could only | ||
| 391 | +have made a local test pass while the delivered artifact still raced, i.e. | ||
| 392 | +**hidden** the bug. Recorded so the next campaign does not spend a round on it. | ||
| 393 | +Evidence class: **installed source + shipped config**, CANN 9.2.0. | ||
| 394 | + | ||
| 395 | +--- | ||
| 396 | + | ||
| 14 | ## On A5: which cube+vector fusion construct to reach for | 397 | ## On A5: which cube+vector fusion construct to reach for |
| 15 | 398 | ||
| 16 | -Two different constructs are covered here, and only one of them hangs. Decide in | 399 | +Two constructs are covered here. Decide in this order: |
| 17 | -this order — the same order the adjudication further down this page arrives at: | ||
| 18 | 400 | ||
| 19 | -1. **Try the generated pipeline path first.** `fwd_ids`/`bwd_ids` plus | 401 | +1. **A hand-written per-tile handoff inside one `@pl.jit` is a legitimate first |
| 20 | - `@pl.pipeline.stage` plus `PipelineConfig` **does work** here, measured on the | 402 | + choice** when the dataflow needs cube and vector to exchange a tile every |
| 21 | - reference kernel and on a full staged kernel | 403 | + iteration. It is what 17 of 18 installed FA samples do. **Copy the pipe/event |
| 22 | - ([framework-findings §17](../references/pypto-pro-framework-findings.md)). | 404 | + assignment from a sample rather than deriving it**, declare the shared groups |
| 23 | -2. **Fall back to single-sided launches with GM intermediates** when the fused | 405 | + outside both sections, and give each backward (release) group exactly `depth` |
| 24 | - form fails. This is the proven escape, not evidence that fusion is impossible. | 406 | + pre-published credits in the section prologue. Bring it up on the mandatory |
| 25 | -3. **Do not hand-write a per-tile cross-core event sequence inside one `@pl.jit`.** | 407 | + ladder above — when this construct fails, it fails as an opaque watchdog. |
| 26 | - That is the construct that hangs; see below. | 408 | +2. **The generated pipeline path** works where it applies — `fwd_ids`/`bwd_ids` |
| 27 | - | 409 | + plus `@pl.pipeline.stage` plus `PipelineConfig`, measured on the reference |
| 28 | -### The construct that does not run: hand-written per-tile handoff | 410 | + kernel and on a full staged kernel |
| 29 | - | 411 | + ([framework-findings §17](../references/pypto-pro-framework-findings.md)). It |
| 30 | -Measured, repeatedly. A single `@pl.jit` holding a `section_cube()` and a | 412 | + is not a safe default either: it was measured to fault on this same |
| 31 | -`section_vector()` with a **hand-written** cross-core handoff on every tile | 413 | + computation ("compiles with correct pipes, still faults"). |
| 32 | -compiles and then dies with `aicore timeout`. One attention kernel was taken | 414 | +3. **Single-sided launches with GM intermediates** remain the proven escape. |
| 33 | -through **thirteen hypotheses and nine mutation-ladder rungs** on that construct | 415 | + Reach for them when the fused form is blocked, not before — and check first |
| 34 | -and never ran once, even though every ingredient passed in isolation — the cube | 416 | + whether your delivery contract permits more than one kernel launch, because |
| 35 | -half (including the transposed NT load and a dual-accumulator contraction) and | 417 | + some do not. |
| 36 | -the register-level softmax were each proven separately. A reference | ||
| 37 | -implementation written against a different kernel language hit the identical | ||
| 38 | -wall on the identical construct, so the limit is in the construct rather than in | ||
| 39 | -one language's lowering. This says nothing about the generated pipeline path in | ||
| 40 | -step 1. | ||
| 41 | 418 | ||
| 42 | ### The proven fallback: decompose into single-sided launches | 419 | ### The proven fallback: decompose into single-sided launches |
| 43 | 420 | ||
| @@ -189,6 +566,43 @@ synchronization and replaces it with a hardware `unit_flag` that the paired | |||
| 189 | `matmul(phase=Final)` drained by `pl.move(..., acc_to_vec_mode=...)` arms a | 566 | `matmul(phase=Final)` drained by `pl.move(..., acc_to_vec_mode=...)` arms a |
| 190 | protocol whose other half cannot reply. Nothing rejects it. | 567 | protocol whose other half cannot reply. Nothing rejects it. |
| 191 | 568 | ||
| 569 | +### The trap chain, and why the resolution is "never arm it" | ||
| 570 | + | ||
| 571 | +Stated as a chain rather than as a rule, because the rule alone reads as | ||
| 572 | +arbitrary and the chain is what makes it checkable at **design** time: | ||
| 573 | + | ||
| 574 | +1. `pl.move`'s space table has **no `Acc → Mat` row** (previous section), so an | ||
| 575 | + L0C accumulator reaches L1 only via `Acc → Vec → Mat`, or through GM. | ||
| 576 | +2. Therefore, in any design whose accumulators drain on chip, **the drain is a | ||
| 577 | + `pl.move`**. | ||
| 578 | +3. **`pl.move` has no `phase` parameter.** `phase=` is a hardware handshake and | ||
| 579 | + only `pl.store` can answer it. | ||
| 580 | +4. Therefore `phase=` **must never be armed** in such a design. There is no | ||
| 581 | + spelling of the reply. | ||
| 582 | + | ||
| 583 | +The resolution is not "pass the right phase" — it is *not passing one*. | ||
| 584 | +`phase: Optional[AccPhase] = None` defaults to `None`, and `None` keeps the | ||
| 585 | +framework's **automatic M↔FixPipe synchronization on**. Arming `phase` turns | ||
| 586 | +that automatic sync **off** and substitutes a protocol whose other half cannot | ||
| 587 | +be written. | ||
| 588 | + | ||
| 589 | +This is worth writing down because every signal points the other way: the matmul | ||
| 590 | +samples all carry `phase=AccPhase.Partial/Final`, copying them is the default | ||
| 591 | +move, and **nothing rejects the result** — a mis-set `phase` is named on this | ||
| 592 | +page as a hang cause *with no diagnostic*. One design review caught it by | ||
| 593 | +walking the chain above and backing the `phase=` out of an already-reviewed | ||
| 594 | +dataflow; the shipped kernel has **zero** `phase=` occurrences and does not hang. | ||
| 595 | + | ||
| 596 | +**Design-time check, costs nothing:** for every accumulator, name its drain | ||
| 597 | +first. If the drain is `pl.move`, `phase=` is forbidden on every `matmul` and | ||
| 598 | +`matmul_acc` feeding that accumulator. Only if the drain is | ||
| 599 | +`pl.store`/`store_tile` is arming it even expressible. | ||
| 600 | + | ||
| 601 | +Evidence class: **installed API/source reading**, CANN 9.2.0, plus a device run | ||
| 602 | +of the resulting kernel (which did not hang). It is *not* a controlled A/B — | ||
| 603 | +`phase=` was removed at design time, so no armed variant of that kernel was ever | ||
| 604 | +built. | ||
| 605 | + | ||
| 192 | Checklist item, before diagnosing any Cube-side hang or L0C fault: | 606 | Checklist item, before diagnosing any Cube-side hang or L0C fault: |
| 193 | 607 | ||
| 194 | 1. For every `matmul` carrying `phase=`, name the drain. If it is not | 608 | 1. For every `matmul` carrying `phase=`, name the drain. If it is not |
| @@ -23,6 +23,58 @@ both maximum and sum. See | |||
| 23 | [online-softmax-tail.md](../patterns/online-softmax-tail.md), which is | 23 | [online-softmax-tail.md](../patterns/online-softmax-tail.md), which is |
| 24 | conceptual only. | 24 | conceptual only. |
| 25 | 25 | ||
| 26 | +## A static assert on a tile's shape binds the **declared** extent, not the working one | ||
| 27 | + | ||
| 28 | +This is the rule that turns checklist item 3 from a limitation into a free | ||
| 29 | +choice, and missing it makes whole shape bands look unimplementable. | ||
| 30 | + | ||
| 31 | +**The observation.** A `[R, 1]` FP32 tile in vector memory fails to store unless | ||
| 32 | +`R * sizeof(dtype) % 32 == 0` — for FP32, `R` a multiple of 8. Declaring | ||
| 33 | +`R = 12` fails the build: | ||
| 34 | + | ||
| 35 | +``` | ||
| 36 | +.../pto/npu/a5/TStore.hpp:178: static assertion failed ... | ||
| 37 | +((layout == Layout::DN) && (Tile<...,float,12,1,...>::Rows * sizeof(float) % 32 == 0)) || ... | ||
| 38 | +``` | ||
| 39 | + | ||
| 40 | +Read literally that says "rows per tile must be a multiple of 8", and a | ||
| 41 | +row-reduction or per-row-scale design then inherits `Rows % 8 == 0` as a | ||
| 42 | +constraint on its tiling. | ||
| 43 | + | ||
| 44 | +**The escape.** The assert is instantiated from the tile's **declared** template | ||
| 45 | +parameters. So declare a fixed, legal extent and narrow it at run time: | ||
| 46 | + | ||
| 47 | +```python | ||
| 48 | +scale_g = pl.make_tile_group( | ||
| 49 | + type=pl.TileType(shape=[64, 1], dtype=pl.DT_FP32, | ||
| 50 | + target_memory=pl.MemorySpace.Vec), ...) | ||
| 51 | +cur = scale_g.next() | ||
| 52 | +pl.set_validshape(cur, [vrows, 1]) # vrows is a runtime value, any 1..64 | ||
| 53 | +``` | ||
| 54 | + | ||
| 55 | +The declared `[64, 1]` is 256 B — eight whole 32-byte blocks — and satisfies the | ||
| 56 | +assert once, at compile time, for every `vrows`. | ||
| 57 | + | ||
| 58 | +**Verified on hardware with a negative control**, which is what separates this | ||
| 59 | +from a plausible reading of the assert. From a **single compile**, | ||
| 60 | +`vrows ∈ {1, 5, 7, 8, 12, 17, 33, 53, 64}` all produced bit-exact results and | ||
| 61 | +left rows beyond the valid window untouched; and declaring `[12, 1]` *without* | ||
| 62 | +`set_validshape` reproduced the assert above. Both halves are needed: the | ||
| 63 | +positive run alone would be consistent with the assert never having applied. | ||
| 64 | + | ||
| 65 | +**This escape is mandatory, not cosmetic, once the row is wide.** Taking | ||
| 66 | +`Rows % 8 == 0` at face value forces at least 8 rows into every tile. Eight rows | ||
| 67 | +of a 16384-wide FP32 row block is 512 KiB against a unified-buffer budget of | ||
| 68 | +roughly 248 KiB, so the widest band becomes unimplementable outright — not | ||
| 69 | +slower, impossible. A design that concludes "this shape cannot be done" from a | ||
| 70 | +shape assert should first ask whether the assert is on the declaration. | ||
| 71 | + | ||
| 72 | +**The general form**, worth carrying to any assert of this kind: a compile-time | ||
| 73 | +assertion can only see compile-time values. When it names a shape, check whether | ||
| 74 | +the DSL offers a runtime window over that shape before treating the assertion as | ||
| 75 | +a constraint on your tiling. The cost is one `set_validshape` per consuming | ||
| 76 | +operation — which checklist item 4 already requires. | ||
| 77 | + | ||
| 26 | ## Exception: `compact=1` tiles — the window is part of the layout | 78 | ## Exception: `compact=1` tiles — the window is part of the layout |
| 27 | 79 | ||
| 28 | Checklist item 4 is wrong for any tile declared `compact=1`, and following it | 80 | Checklist item 4 is wrong for any tile declared `compact=1`, and following it |
| @@ -40,6 +40,78 @@ identity is what `auto_mutex` orders against, so a slot counter that steps by | |||
| 40 | the wrong stride aliases two logical buffers onto one physical one while the | 40 | the wrong stride aliases two logical buffers onto one physical one while the |
| 41 | event machinery still issues two credits. | 41 | event machinery still issues two credits. |
| 42 | 42 | ||
| 43 | +## A resident fp32 accumulator makes UB — not L0A — the binding space | ||
| 44 | + | ||
| 45 | +**Trigger:** a kernel that carries an **fp32 output accumulator resident in UB | ||
| 46 | +across an inner loop**, whose width is the operator's largest dimension. Online | ||
| 47 | +softmax, any streaming normalisation, any running reduction over a long axis. | ||
| 48 | +The check is `accumulator_rows × wide_dim × 4` against the UB capacity, done | ||
| 49 | +*before* the tile size is chosen. | ||
| 50 | + | ||
| 51 | +**Rule:** for this shape, **sum UB first**. The instinct on a cube-heavy kernel | ||
| 52 | +is to budget L0A/L0B and treat UB as slack, and that instinct picks the wrong | ||
| 53 | +tile size. It is also why **copying an official sample's tile constants does not | ||
| 54 | +transfer** once the wide dimension grows: the sample's tiling was legal at *its* | ||
| 55 | +wide dimension, and nothing in the sample says which space was binding. | ||
| 56 | + | ||
| 57 | +**Worked arithmetic**, so it can be re-derived at another width. Attention with a | ||
| 58 | +query-tile height `TSQ` split across two vector sub-blocks and a head dim `D`, so | ||
| 59 | +the resident output accumulator each sub-block owns is `[TSQ/2, D]` in fp32: | ||
| 60 | + | ||
| 61 | +| quantity | at `TSQ = 128`, `D = 512` | at `TSQ = 128`, `D = 448` | | ||
| 62 | +|---|---|---| | ||
| 63 | +| output accumulator `[TSQ/2, D]` fp32 | `64 × 512 × 4` = **131072 B** | `64 × 448 × 4` = **114688 B** | | ||
| 64 | +| plus the minimum other resident tiles (the QK product staging, the two probability tiles, the PV staging, the narrowed output) | total **≥ 254976 B** | total **238592 B** | | ||
| 65 | +| UB capacity (`ub_size`, `950PR_957x.ini`) | **253952 B** | 253952 B | | ||
| 66 | +| verdict | **does not fit, even at one slot per group** | fits | | ||
| 67 | + | ||
| 68 | +So **`TSQ = TKV = 128` — the tile size every installed FA sample uses — is not | ||
| 69 | +implementable at `D = 512`.** 64 is forced. It *is* implementable at `D = 448`, | ||
| 70 | +which is why a sample or a design validated at a narrower head dim carries no | ||
| 71 | +warning. The same design's L0 budget at `TSQ = TKV = 64` sits at **L0A 50 % / | ||
| 72 | +L0B 50 % / L0C 25 %** — nowhere near binding, which is the whole point. | ||
| 73 | + | ||
| 74 | +Two consequences worth carrying: | ||
| 75 | + | ||
| 76 | +- **"Even at one slot per group" is the form the check must take.** A budget that | ||
| 77 | + only fails at two slots invites a slot reduction; this one fails at the floor, | ||
| 78 | + so it is a statement about the tile size and nothing else can absorb it. | ||
| 79 | +- **Carry the tile height as a tiling-key rung**, not as a module constant, if | ||
| 80 | + the wide dimension is dynamic. The legal height is a function of it. | ||
| 81 | + | ||
| 82 | +Evidence class: **derived from installed platform capacities and the design's own | ||
| 83 | +address map**, then confirmed by the address arithmetic of a kernel that ran on | ||
| 84 | +a5 / `DAV_3510`, CANN 9.2.0. **Validation scope:** the fp32-accumulator shape | ||
| 85 | +above. It is *not* a claim that UB binds for every cube+vector kernel — a design | ||
| 86 | +without a wide resident fp32 accumulator will land somewhere else entirely. The | ||
| 87 | +capacity figure comes from the platform `.ini` key, per | ||
| 88 | +[arch-a5.md](arch-a5.md); carry the key, not the number. | ||
| 89 | + | ||
| 90 | +### `TSQ == TKV` is load-bearing, and the ambiguity behind it is now closed | ||
| 91 | + | ||
| 92 | +The installed `insert` API page states its bound check as | ||
| 93 | +*"`row + src rows ≤ dst rows`, `col + src cols ≤ dst cols`"*. For an **NZ source | ||
| 94 | +inserted into a ZN destination** that admits **two self-consistent readings** — | ||
| 95 | +the offset pair interpreted in the destination's logical frame, or in the | ||
| 96 | +source's NZ frame — and **every installed sample has `TS == TKV == TD`**, so none | ||
| 97 | +of them discriminates. Choosing `TSQ == TKV` is valid under both readings and is | ||
| 98 | +the safe design-time move while the ambiguity stands. | ||
| 99 | + | ||
| 100 | +**This campaign closed it, both ways:** | ||
| 101 | + | ||
| 102 | +- **Empirically** — a one-hot probe returned the exact identity for every query | ||
| 103 | + row across **both** sub-blocks. Under the competing reading, the second | ||
| 104 | + sub-block would have written a different rectangle and the identity would have | ||
| 105 | + broken. | ||
| 106 | +- **In the generated code** — the emitted `TINSERT` takes the offset pair as | ||
| 107 | + **`(kv_row, query_col)` in the NZ source frame**, with a destination | ||
| 108 | + `Tile<Mat, half, 64, 64, ColMajor>` and a source | ||
| 109 | + `Tile<Vec, half, 64, 32, ColMajor>`. | ||
| 110 | + | ||
| 111 | +So the source-frame reading is the real one. `TSQ == TKV` remains the | ||
| 112 | +recommendation for anyone who has not re-verified this on their own build, since | ||
| 113 | +it costs nothing and is correct under either. | ||
| 114 | + | ||
| 43 | ## Cube-specific check | 115 | ## Cube-specific check |
| 44 | 116 | ||
| 45 | Obtain dtype-dependent contraction geometry and Left/Right/Acc layout | 117 | Obtain dtype-dependent contraction geometry and Left/Right/Acc layout |
| @@ -56,3 +128,325 @@ one K alignment or one fractal shape for all dtypes. | |||
| 56 | 128 | ||
| 57 | The installed `$PYPTO_DEVKIT_DIR/docs/pypto_pro/api/` pages and official | 129 | The installed `$PYPTO_DEVKIT_DIR/docs/pypto_pro/api/` pages and official |
| 58 | examples are the primary source for the current version. | 130 | examples are the primary source for the current version. |
| 131 | + | ||
| 132 | +--- | ||
| 133 | + | ||
| 134 | +## Launch geometry on a5 / DAV_3510 — measured, run H (rms_norm, 2026-08-12) | ||
| 135 | + | ||
| 136 | +Instrument: `cold_probe.py`-derived paired sweep (`probe/sweep.py`), **cold** L2 (192 MB flush | ||
| 137 | +inside the measured window), Level1 NPU profiler, all arms of a shape in ONE process so the arms | ||
| 138 | +share a compiled binary and differ only in launch geometry. The measuring card, on a shared box. | ||
| 139 | +🔴 Estimator note: on a shared box, contention is **one-sided** (it can only add time), so `min` | ||
| 140 | +over reps is the unbiased estimator of uncontended kernel time — but `min` is *not* robust to | ||
| 141 | +spuriously-fast profiler samples. Report `min`, 2nd-smallest and median together; where they | ||
| 142 | +disagree, the robust pair wins. This changed two verdicts in run H (a "−11.0 %" became +0.8 %, | ||
| 143 | +a "−66.8 %" became −36 %). | ||
| 144 | + | ||
| 145 | +### Shrinking blockDim for small inputs — DOES NOT APPLY to a `min(coreNum, n_tiles)` grid | ||
| 146 | + | ||
| 147 | +A widely-quoted rule (`blockDim = elems < SMALL_GRID_ELEMS ? min(coreNum,8) : coreNum`, with | ||
| 148 | +15000/30000 per dtype) is a repair for kernels that launch `coreNum` blocks **unconditionally**. | ||
| 149 | +A planner that already emits `blocks = max(1, min(coreNum, n_tiles))` **has the fix in a stronger | ||
| 150 | +form**, and applying the threshold on top is a no-op below `n_tiles = 8` and a large regression | ||
| 151 | +above it. Measured on `[11,13,17,67]` fp16 (162 877 elements, 304 tiles): grid 56 → 8 cost | ||
| 152 | +**4.7364 → 19.5766 µs = +4.13×** *(measured on a 56-vector-core part; the ratio is the point, the counts are that part's)*, because `ceil(304/56)=6` waves become `ceil(304/8)=38`. | ||
| 153 | +Break-even, derived from that measurement rather than assumed: capping an `N`-tile launch to 8 | ||
| 154 | +costs `(ceil(N/8)−1)·T_tile` and saves `(N−8)·d`; both scale as `N/8`, so the break-even sits at | ||
| 155 | +`T_tile ≈ 0.32 µs` **almost independently of N**. Check that number before porting the rule. | ||
| 156 | + | ||
| 157 | +### Wave alignment is NOT the objective; tile size is | ||
| 158 | + | ||
| 159 | +Tiles are handed to blocks strided (`for t in pl.range(ci, nt, nc)`), so wall-clock is | ||
| 160 | +`ceil(n_tiles/blocks) · T_tile`. It is tempting to minimise wave count. Measured, that is wrong: | ||
| 161 | +`[31,129,2049]` fp32's best rung has **7 → 18 waves** and is still 9 % faster, and | ||
| 162 | +`[3,7,11,4096]` fp32's one-wave rungs are 17–22 % **slower** than a three-wave rung. Twice more, | ||
| 163 | +a launch that leaves cores idle beat one that filled them: writing `C` for the count the runtime | ||
| 164 | +query returns, **`0.68·C` beat `C` by 30 % and `0.5·C` beat `C` by 41 %** — measured on a part | ||
| 165 | +where `C = 56`, and stated as fractions because the fraction is the only part that could ever | ||
| 166 | +travel. **Filling every core is not the objective.** | ||
| 167 | + | ||
| 168 | +🔴 **Those two magnitudes are per-shape observations, and as guidance they are | ||
| 169 | +falsified, on 20 shapes at once.** They are the only numbers on this row, so they | ||
| 170 | +read as an invitation to reduce the block count; the row's own instruction ("sweep it; do not | ||
| 171 | +assume it") is the part that transfers. Swept on Ascend950PR_9579 / CANN 9.2.0, 2026-08-25, on | ||
| 172 | +a flat elementwise kernel with per-core contiguous spans, substituting a half-array | ||
| 173 | +`block_dim = min(0.5·C, ceil(n / PART))` for `min(vector_core_num, …)`: **every one of the 20 | ||
| 174 | +shapes regressed**, by **+10.6 % to +85.3 %** (mean speedup 0.735), with the worst hits on the | ||
| 175 | +mid-size shapes where the tail wave is supposed to dominate. Full `vector_core_num` was correct | ||
| 176 | +on all of them. | ||
| 177 | + | ||
| 178 | +Read the two results together rather than picking one: the 30 %/41 % wins were measured on a | ||
| 179 | +`Rows`-tiled row-reduction where a *tail wave* dominated a small `n_tiles`; the 20 regressions | ||
| 180 | +were measured where each core owns one contiguous element span and halving the core count | ||
| 181 | +simply halves the achieved bandwidth. **Neither magnitude is a default.** Sweep the block | ||
| 182 | +count for your shape, and note that neither of the shipped artifacts on either side reduces | ||
| 183 | +it — see the paragraph below. | ||
| 184 | + | ||
| 185 | +🔴 **The shipped artifacts do NOT reduce the core count.** Verified in the two best-scoring | ||
| 186 | +row-normalisation submissions: `_BLOCKS = get_platform_info().vector_core_num`, | ||
| 187 | +read once at import, with a literal only as the `getattr` fallback — and nothing lowers it. | ||
| 188 | +That measured lever tuned **`Rows`**, not the block count. The sweep result below is a | ||
| 189 | +per-shape observation that never shipped. | ||
| 190 | + | ||
| 191 | +🔴 **Do not turn that into "use fewer cores".** It is a per-shape sweep result, valid only where | ||
| 192 | +`n_tiles` is small enough that the tail wave dominates — and the 20-shape sweep above measured the | ||
| 193 | +full queried count correct on every one of them. The opposite mistake has also been measured and | ||
| 194 | +*shipped*: sizing the grid from `get_platform_info().core_num` — the cube count, half the vector | ||
| 195 | +count on that part — instead of `vector_core_num` launched a `section_vector` kernel on half the | ||
| 196 | +array and cost a | ||
| 197 | +**1.900× median / 1.997× max** slowdown. That pair is recorded **here and nowhere else**, so | ||
| 198 | +treat it as one observation rather than as independently corroborated, and do not cite a second | ||
| 199 | +page for it. The only cross-check is order-of-magnitude and is arithmetic on a model rather than | ||
| 200 | +a second measurement: halving the per-core count in a load model gives a geometric mean of | ||
| 201 | +**1.985** over 20 shapes, consistent with the range above. The API half is directly confirmed — | ||
| 202 | +a `section_vector` launch must be sized from `vector_core_num`, and a pure-vector kernel that | ||
| 203 | +reads `core_num` gets exactly half the array. See | ||
| 204 | +[../references/pypto-pro-launch-block-dim.md](../references/pypto-pro-launch-block-dim.md). | ||
| 205 | + | ||
| 206 | +The rule is *sweep the block count for your shape*, never *prefer fewer cores*, and | ||
| 207 | +never read the count from the wrong constant. | ||
| 208 | + | ||
| 209 | +**This is not the other 28-vs-56 finding.** | ||
| 210 | +[`pypto-pro-op-perf-tune`](../../pypto-pro-op-perf-tune/references/a5-roofline-and-levers.md) | ||
| 211 | +reports that *merely declaring* `pl.section_cube()` costs **1.88x** on the vector path by halving the | ||
| 212 | +launch from 56 blocks to 28. That is a **declaration** shrinking the grid of an otherwise unchanged | ||
| 213 | +kernel; the paragraph above is an **API** mistake — reading `core_num` where `vector_core_num` was | ||
| 214 | +meant. Same arithmetic, two unrelated causes, and neither magnitude transfers to the other case. | ||
| 215 | + | ||
| 216 | +### There is no closed-form optimal `Rows` — but `Rows` derived from `S` alone is a real defect | ||
| 217 | + | ||
| 218 | +Swept every legal `Rows` rung on all 18 Mode-0 public shapes. A rule "pick the rung whose | ||
| 219 | +`Rows·MaxN` is nearest T" was evaluated offline against the measured table for | ||
| 220 | +T ∈ {7k…12k}: **every T regresses at least one case by ≥ 6.5 %**, up to +32 %. Same rung | ||
| 221 | +(`MaxN=1024`), same S band, opposite optima by dtype: fp32 wants `Rows=5`, fp16 wants `Rows=16` | ||
| 222 | +and is +32 % worse at 5. **Do not ship a global `Rows` rule.** | ||
| 223 | + | ||
| 224 | +What *is* real: a `_pick_rows` that computes `bal = S // (coreNum · MIN_TILES_PER_CORE)` is a | ||
| 225 | +function of **S alone and ignores MaxN**, so at the narrowest rung it starves the tile. At | ||
| 226 | +`MaxN = 128` it picked `Rows = 8` (a 1 KB tile) where the UB ceiling was 64. Growing `Rows` to | ||
| 227 | +the largest rung that still leaves `≥ coreNum/2` tiles measured **−12 % to −42 %, mean −27 % | ||
| 228 | +(fp16) / −21 % (bf16), zero regressions**, over `S` from 56 to 2431 — a 43× range, both dtypes. | ||
| 229 | +The same rule applied to **other** rungs regresses `MaxN=384` by **+40.8 %** and | ||
| 230 | +`MaxN=4096` by **+22.1 %**. Gate it to the rung it was measured on. | ||
| 231 | + | ||
| 232 | +### Regrouping rows into tiles is bit-neutral | ||
| 233 | + | ||
| 234 | +Changing `Rows` changes only how rows are grouped; each row's reduction is independent of the | ||
| 235 | +grouping. Verified byte-for-byte: **30 configurations, 5 shapes, 96 386 842 bytes compared, | ||
| 236 | +0 differing bytes**. A `Rows` change that alters even one output word means something else moved. | ||
| 237 | + | ||
| 238 | +### The narrow-rung `Rows` floor — where growing helps and where it overshoots (run H, extended) | ||
| 239 | + | ||
| 240 | +Same instrument as above. The `bal = S // (coreNum * MIN_TILES_PER_CORE)` defect (Rows chosen from | ||
| 241 | +S alone, MaxN ignored) starves the tile at EVERY narrow rung, not just the narrowest. Measured at | ||
| 242 | +`MaxN` 192 and 256, both dtype classes, `S in {56,127,254,896}`: the planner's pick is beaten by | ||
| 243 | +**-4.4 % to -48.8 %**, and `Rows = 8` is the best arm in 9 of those 12 points. | ||
| 244 | + | ||
| 245 | +But the FIX does not generalise the way the defect does. Growing `Rows` to the largest rung that | ||
| 246 | +still leaves >= coreNum/2 tiles is right at `MaxN = 128` (measured best-or-within-5 % over S from 56 | ||
| 247 | +to 14336) and **wrong at `MaxN >= 192` once S is large**: | ||
| 248 | + | ||
| 249 | +| MaxN | dt | S | grow-to-cap vs planner | | ||
| 250 | +|---|---|---|---| | ||
| 251 | +| 192 | fp32 | 7168 | **+14.6 %** | | ||
| 252 | +| 384 | fp16 | 4037 | **+17.6 %** | | ||
| 253 | +| 384 | fp32 | 4037 | **+38.5 %** | | ||
| 254 | + | ||
| 255 | +That last row is public case `[11,367,373]`, and +38.5 % here reproduces a +40.8 % measured from an | ||
| 256 | +independent sweep — the boundary is real. | ||
| 257 | + | ||
| 258 | +**The portable lesson: at rungs above the narrowest, use a FLOOR (`Rows >= 8`), not a target.** | ||
| 259 | +A floor is provably inert for `S >= 8 * MIN_TILES_PER_CORE * coreNum` — here `8 * 4 * vector_core_num`, which is 1792 on a 56-core part — so it cannot | ||
| 260 | +fire in the regime where growth overshoots, while still capturing the whole starved corner. Measured | ||
| 261 | ++0.0 % at all 12 large-S points and -4.4 %…-48.8 % at small S. Growth guards must be gated to the | ||
| 262 | +single rung they were measured on. | ||
| 263 | + | ||
| 264 | + | ||
| 265 | +--- | ||
| 266 | + | ||
| 267 | +## Per-core load balance is the quantity the `Rows` sweep is ranking (measured, 2026-08-14) | ||
| 268 | + | ||
| 269 | +Everything above stands: there is no closed-form optimal `Rows`, and you sweep. What was missing | ||
| 270 | +is the quantity the sweep is *ranking*, which turns a 27-point sweep into a shortlist of two or | ||
| 271 | +three. | ||
| 272 | + | ||
| 273 | +Tiles are handed to blocks strided and there is no barrier, so wall-clock tracks the **longest | ||
| 274 | +running core**, not the average one. For `S` rows at `Rows` rows per tile over `cores` cores, the | ||
| 275 | +elements resident on the busiest core are | ||
| 276 | + | ||
| 277 | + busiest(Rows) = ceil( ceil(S / Rows) / cores ) * Rows | ||
| 278 | + | ||
| 279 | +Measured on the measuring card / Ascend950PR / CANN 9.2.0, 2026-08-14, on `rms_norm`: 8 shapes × | ||
| 280 | +27 `Rows` points, each point a full pinned-harness single-case eval under both box locks. The | ||
| 281 | +ranking of `busiest(Rows)` reproduced the measured ranking **exactly on 5 of the 8 shapes and | ||
| 282 | +picked the winner on a 6th**. It **inverted on 2 of 8** — `[11,367,373]` fp32 and | ||
| 283 | +`[3,7,11,4096]` fp32, where a wider tile won despite worse balance. So it is a **shortlist | ||
| 284 | +generator, not a decision procedure**: take the two or three geometries it favours and time them. | ||
| 285 | + | ||
| 286 | +Shipped as a `_pick_rows` post-pass minimising `busiest(Rows)` over `[pick//2, pick)`, accepted | ||
| 287 | +only for a better-than-1/16 improvement, tie-break to the wider tile — **and gated to | ||
| 288 | +`tiles(pick) >= cores`, i.e. only a launch that already saturates every core.** It was the single | ||
| 289 | +largest lever of that tuning stage: | ||
| 290 | + | ||
| 291 | +| shape, dtype | rung | `Rows` | µs | Δ | | ||
| 292 | +|---|---|---|---|---| | ||
| 293 | +| `[11,367,373]` fp32 | 384 | 32 → 20 | 100.76 → **88.02** | **−12.6 %** | | ||
| 294 | +| `[33,127,769]` fp16 | 832 | 24 → 20 | 176.69 → 155.40 | −12.0 % | | ||
| 295 | +| `[32,128,1024]` fp32 | 1024 | 24 → 20 | 227.29 → 204.67 | −10.0 % | | ||
| 296 | +| `[32,128,768]` fp16 | 768 | 24 → 20 | 152.40 → 137.44 | −9.8 % | | ||
| 297 | +| `[7,1009,1021]` fp16 | 1024 | 20 → 16 | 291.61 → 269.98 | −7.4 % | | ||
| 298 | + | ||
| 299 | +**Why the saturation gate is not cosmetic.** Below `tiles >= cores` the launch is core-*starved*: | ||
| 300 | +every candidate sits in one wave, `busiest()` collapses onto the `pick//2` floor and stops | ||
| 301 | +carrying information — and that regime already has its own measured rules on this page and in | ||
| 302 | +the launch-geometry section of this page. The | ||
| 303 | +**ungated** form measured slightly *faster* (median 13.153 vs 13.157) and was **rejected**, | ||
| 304 | +because it moved a planner geometry that a frozen test pins. Both numbers are recorded so the | ||
| 305 | +next run knows the gate costs ~0.03 % on that suite and buys a regime where the model is | ||
| 306 | +meaningless. | ||
| 307 | + | ||
| 308 | +The same model also predicted a rejected lever's sign and rough magnitude on a *packed* narrow-D | ||
| 309 | +path: widening the packed rung from 512 to 2048 raises the busiest core from 36864 to 49152 | ||
| 310 | +elements (+33 %) and measured **+26 %** wall time. Enumerating every legal geometry that packing | ||
| 311 | +mode can express put the best reachable value at 35840 against the shipped 36864 — **2.9 %** — | ||
| 312 | +which is how that case was declared finished rather than searched further. | ||
| 313 | + | ||
| 314 | +### The `>= coreNum/2` endpoint slightly overshoots — a refinement, not an opposite sign | ||
| 315 | + | ||
| 316 | +The narrow-rung rule above grows `Rows` to the largest rung still leaving `>= coreNum/2` tiles, | ||
| 317 | +measured at −12 % to −42 % against a starved planner pick. A sweep taken **past** that endpoint, | ||
| 318 | +same operator and same shape `[11,13,17,67]` fp16 on the measuring card, 2026-08-14, measured `Rows` 64 → 48 | ||
| 319 | +at a further **−5.6 %**. Read the two together: growing out of the starved corner is the large | ||
| 320 | +and reliable move, and the endpoint it lands on is a few percent wide of the optimum. | ||
| 321 | + | ||
| 322 | +🔴 **The two comparisons are not the same comparison** and must not be collapsed into one number: | ||
| 323 | +the −12…−42 % figures (and the 30 % launch-geometry result recorded above) compare **38 vs 56** | ||
| 324 | +blocks, while this refinement compares **38 vs 51**. Nothing here contradicts *filling every core | ||
| 325 | +is not the objective* — the optimum is simply not exactly at the `coreNum/2` endpoint either. | ||
| 326 | +**Regime matters — this is the opposite corner from the starved one above.** In the *starved* corner (narrow rung, small `S`) `Rows` is too **small** and growing it wins −4.4…−48.8 %. In the regime swept here (wide rungs, ample `S`) the reverse holds: | ||
| 327 | +across all 8 shapes swept in that run the planner's `Rows` was too **large**, never too small, | ||
| 328 | +which is the opposite corner from the starvation defect the section above repairs; both are real | ||
| 329 | +and they are reached by different planners. The page's own instruction is what settles it: sweep. | ||
| 330 | + | ||
| 331 | +**Not established — recorded so it is not inherited as a constant.** The same run fitted a | ||
| 332 | +per-launch fixed cost of `T ≈ 31 µs + 7.37 µs/MB` on that card, which would be 73 % of the | ||
| 333 | +smallest case's entire 42.7 µs and would explain why the suite's worst ratio-to-floor case is | ||
| 334 | +also its smallest. It is a **two-point fit with zero degrees of freedom**: it cannot be | ||
| 335 | +falsified by its own data and it is not a measurement. **Do not use the 31 µs.** If a small-shape | ||
| 336 | +floor matters to a run, sweep transfer size properly and derive one, and note that no routed page | ||
| 337 | +models a launch constant at all. | ||
| 338 | + | ||
| 339 | +> **Where this statement lives, for anyone citing it.** The "do not use the 31 µs" | ||
| 340 | +> ruling is **this subsection** — 「Per-core load balance … > The `>= coreNum/2` | ||
| 341 | +> endpoint slightly overshoots …」 — not the wave-alignment section further up. | ||
| 342 | +> A design review cited it against the wrong section, and because both sections | ||
| 343 | +> discuss launch geometry the mis-citation read as plausible. When quoting a | ||
| 344 | +> retraction, quote the heading path it sits under, not the nearest heading on | ||
| 345 | +> the same subject: a retraction that is attached to the wrong section can be | ||
| 346 | +> re-litigated by anyone who reads that section and does not find it. | ||
| 347 | + | ||
| 348 | +## `set_validshape` on the ROW axis: cost tracks the CRITICAL-PATH VALID rows (measured 2026-08-27, corrected same day) | ||
| 349 | + | ||
| 350 | +🔴 **RETRACTION, in place.** This section previously read, under the heading | ||
| 351 | +`constraints/tiling.md › set_validshape narrows semantics, not cost — but only on the ROW axis`: | ||
| 352 | +"Tile-op cost tracks the **DECLARED ROW COUNT**, not the valid row count." **That is wrong on the | ||
| 353 | +row axis**, and the table below is why. The width finding in the next section is unaffected and | ||
| 354 | +still stands — do not read this retraction as touching it. | ||
| 355 | + | ||
| 356 | +The corrected rule is: | ||
| 357 | + | ||
| 358 | +> Tile-op cost on the row axis tracks the **CRITICAL-PATH VALID ROW COUNT** — the | ||
| 359 | +> `ceil(nrow / block_count)` rows the busiest core actually processes. The **declared** row count of | ||
| 360 | +> the tile type has **no demonstrated independent effect**. | ||
| 361 | + | ||
| 362 | +Why the original reading failed. The evidence was a rows-per-tile experiment on a per-token | ||
| 363 | +quantizer whose work division is a contiguous ROW SPAN. The entry claimed the rows actually | ||
| 364 | +processed per core were unchanged between the two variants, and that column was called | ||
| 365 | +load-bearing. It was load-bearing, and it was **incorrect**. The kernel launches | ||
| 366 | +`min(vector_core_num, ntiles)` blocks with `ntiles = ceil(T / rows_per_tile)`, so **raising | ||
| 367 | +rows-per-tile lowers the block count**, which *raises* `ceil(nrow / blocks)`. Declared rows and | ||
| 368 | +critical-path rows moved together by construction, and the experiment could not separate them. | ||
| 369 | +Block counts recovered from `Block Num` in the profiler's `op_summary.csv` for the same two | ||
| 370 | +variants: | ||
| 371 | + | ||
| 372 | +| declared rows | declared ratio | blocks launched | crit-path rows `ceil(nrow/blocks)` | measured slowdown | | ||
| 373 | +|---|---|---|---|---| | ||
| 374 | +| 64 → 64 (pinned by the `LANES` cap) | x1.00 | 56 → 56 | 64 → 64 | **none (0.99)** | | ||
| 375 | +| 39 → 62 | x1.59 | 53 → 34 | **39 → 61** | x1.41 | | ||
| 376 | +| 10 → 17 | x1.70 | 52 → 31 | **10 → 17** | x1.47 | | ||
| 377 | +| 5 → 8 | x1.60 | 52 → 32 | **5 → 8** | x1.38 | | ||
| 378 | + | ||
| 379 | +The rung that did not regress is still the control, but it controls for something different from | ||
| 380 | +what was claimed: its declared rows were pinned by `LANES`, so its **block count and critical path | ||
| 381 | +were also unchanged**. Every rung that regressed had its critical path lengthened. | ||
| 382 | + | ||
| 383 | +The separating experiment. A later variant changed **only** the launch clamp — | ||
| 384 | +`min(cores, ntiles)` → `min(cores, nrow)` — leaving every tile type, every rung and every | ||
| 385 | +declared row count untouched, and altering only how many rows the busiest core carries: | ||
| 386 | + | ||
| 387 | +| shape | declared rows | crit-path rows | measured | | ||
| 388 | +|---|---|---|---| | ||
| 389 | +| square, side = 1 rung width | 22, **unchanged** | 22 → 19 | **−9.3 %** | | ||
| 390 | +| tall, width just over a rung | 39, **unchanged** | 39 → 37 | **−3.4 %** | | ||
| 391 | +| control: crit-path pinned by the ceiling | 10, unchanged | 10 → 10 | **−0.6 % / +0.6 %** (null) | | ||
| 392 | + | ||
| 393 | +Declared rows are constant down that whole table. A strict declared-row model predicts zero | ||
| 394 | +everywhere; the measured effects are large and track the critical-path ratio instead. The third row | ||
| 395 | +is the control that rules out "more blocks is just better": it gains blocks with its critical path | ||
| 396 | +pinned, and does not move. | ||
| 397 | + | ||
| 398 | +**The law, and its range of validity.** Time scales as `(critical-path rows)^0.67` on this | ||
| 399 | +operator. The exponent was fitted on the *degradations* above (ratios 1.56–1.70) and then predicted | ||
| 400 | +the two *improvements* (ratios 0.86–0.95) to within 0.2 pp — out of sample, in the opposite | ||
| 401 | +direction. Treat 0.67 as an operator-specific fit, but treat the *shape* of the law — cost follows | ||
| 402 | +critical-path rows, sublinearly — as the transferable part. | ||
| 403 | + | ||
| 404 | +🔴 **This cuts against a neighbouring idiom, so read the scope carefully.** | ||
| 405 | +[../patterns/vec-row-reduce-broadcast.md](../patterns/vec-row-reduce-broadcast.md) prescribes | ||
| 406 | +over-declaring the `[R,1]` reduction *output* tile and narrowing it — and that is correct, because | ||
| 407 | +that tile is a couple of hundred bytes and its cost is irrelevant. The rule here is about **large | ||
| 408 | +data tiles**: the same idiom applied to one is a real cost. | ||
| 409 | + | ||
| 410 | +**Consequence for any "releaser" argument.** The standard chain — free bytes per element, get more | ||
| 411 | +rows per tile, go faster — has a negative second link, and the corrected mechanism makes it sharper | ||
| 412 | +than the original wording did. Spending freed capacity on rows-per-tile **shrinks `ntiles`, which | ||
| 413 | +shrinks a block count clamped to `ntiles`, which lengthens the critical path**. The cost is starved | ||
| 414 | +parallelism, not expensive declared rows. Freeing capacity is not a gain until you have somewhere | ||
| 415 | +profitable to spend it. Measure the spend, not just the release. | ||
| 416 | + | ||
| 417 | +**Corollary worth checking on any row-split kernel.** If the launch clamps blocks to `ntiles` while | ||
| 418 | +the kernel divides work by row span, the clamp is a leftover from a tile-index split and is | ||
| 419 | +leaving cores idle: `ceil(nrow/ncore)` is non-increasing in `ncore` and `nrow >= ntiles` always, so | ||
| 420 | +clamping to `nrow` instead is **weakly dominant on every shape** and is bit-neutral. On the operator | ||
| 421 | +above it was worth up to −9.3 % on the shapes where `ntiles < vector_core_num`, and exactly nothing | ||
| 422 | +on the shapes where it was already saturated — so it costs nothing to try and the shapes that | ||
| 423 | +benefit are identifiable in advance, without measuring, from `ntiles` alone. | ||
| 424 | + | ||
| 425 | +### The width axis: over-declaring is nearly free, and going below a 64-multiple backfires | ||
| 426 | + | ||
| 427 | +Same kernel chain, rows fixed, VALID width fixed, only the DECLARED width varied: | ||
| 428 | + | ||
| 429 | +| declared | valid | us | declared ratio | time ratio | | ||
| 430 | +|---|---|---|---|---| | ||
| 431 | +| 576 | 513 | 1.289 | x1.00 | x1.000 | | ||
| 432 | +| 1024 | 513 | 1.313 | x1.78 | x1.019 | | ||
| 433 | +| 2112 | 513 | 1.421 | x3.67 | x1.102 | | ||
| 434 | + | ||
| 435 | +Elasticity of time to declared extent is **0.04 on width**: a x3.67 width over-declare bought | ||
| 436 | +x1.10. Do not spend effort tightening declared width. There is no declared-extent counterpart on | ||
| 437 | +the row axis — cost there tracks the critical-path valid row count, not the declared count (see | ||
| 438 | +「The law, and its range of validity」 above). | ||
| 439 | + | ||
| 440 | +🔴 **And tightening width below a multiple of 64 costs more than it saves.** On a D=67 row, the | ||
| 441 | +tightest width still legal for an int8 output tile (`Cols % 32 == 0`) is 96, against the 128 a | ||
| 442 | +64-multiple ladder would pick: | ||
| 443 | + | ||
| 444 | +| declared | valid | us | | ||
| 445 | +|---|---|---| | ||
| 446 | +| 128 | 67 | **1.038** | | ||
| 447 | +| 96 | 67 | **1.195** | | ||
| 448 | + | ||
| 449 | +The narrower tile is **1.15x slower**. `LANES = 64` is stated elsewhere in this KB as a | ||
| 450 | +correctness constraint for `vf` register access; it is also a throughput effect on the **tile** | ||
| 451 | +path, and it dominates the width saving. A width ladder built from multiples of 64 is therefore | ||
| 452 | +right even where a tighter legal width exists. | ||
| @@ -25,7 +25,34 @@ starts at `(n_regs-1)*64` and runs off the end of a tile whose width is not a 64 | |||
| 25 | 25 | ||
| 26 | **Rule.** Size the tile to `align64(D)`. Then count UB slots honestly: double-buffered in and | 26 | **Rule.** Size the tile to `align64(D)`. Then count UB slots honestly: double-buffered in and |
| 27 | out is **four** slots, so at 256 KB of UB a single vf tile needs | 27 | out is **four** slots, so at 256 KB of UB a single vf tile needs |
| 28 | -`4 * TR * align64(D) * 4 ≤ 262144`, i.e. `align64(D) ≤ 2048` at `TR = 8`. | 28 | +`4 * TR * align64(D) * 4 ≤ 253952`, i.e. `align64(D) ≤ 1984` at `TR = 8`. Budget against the |
| 29 | +`.ini` `ub_size = 253952`, **not** 256 KB — see `constraints/arch-a5.md` for the derivation. | ||
| 30 | + | ||
| 31 | +## The read and write sides do not share an alignment contract | ||
| 32 | + | ||
| 33 | +**Symptom.** `vf.load_align` faults **`507035`** at a 4-byte element offset into a `[1, N]` | ||
| 34 | +FP32 Vec tile, while `vf.store_align(…, dist=pl.StoreDist.FIRST_ELEMENT)` at the **same** | ||
| 35 | +offsets in the **same** kernel completes and produces correct data. Witnessed at element | ||
| 36 | +offsets 0 / 4 / 8 bytes on a per-row-scalar staging path. | ||
| 37 | + | ||
| 38 | +**Cause.** They are different contracts, not one contract applied twice. `load_align` moves a | ||
| 39 | +**whole 64-lane register** and therefore inherits the register-granule rule at the top of this | ||
| 40 | +page. `FIRST_ELEMENT` is documented to align to the **dtype width**, which is what makes it | ||
| 41 | +the in-contract way to write a single 4-byte scalar; the general `NORM` store carries a | ||
| 42 | +32-byte constraint instead. | ||
| 43 | + | ||
| 44 | +**Rule.** Do not infer a legal load offset from a legal store offset, or the reverse. For a | ||
| 45 | +per-row scalar: write it with `dist=pl.StoreDist.FIRST_ELEMENT`, and **read it back through a | ||
| 46 | +32-byte-aligned row pitch** — give each row its own 32-byte staging row and load at | ||
| 47 | +`row * (32 / itemsize)` — rather than at the compact element offset the store accepted. This | ||
| 48 | +is the same 32-byte quantum as the per-token staging rule further down this page, arrived at | ||
| 49 | +from the load side. | ||
| 50 | + | ||
| 51 | +**Why it is worth a rule.** Nothing validates the offset on the way through: the backend's | ||
| 52 | +offset resolution is string arithmetic with no range or alignment check, and it emits the | ||
| 53 | +store instruction directly. A design that pays for 32-byte alignment on the read side and | ||
| 54 | +abandons it on the write side (or the reverse) is internally inconsistent and will look | ||
| 55 | +correct until the faulting side runs. | ||
| 29 | 56 | ||
| 30 | ## Reduction tiles: rows must be a multiple of 8 | 57 | ## Reduction tiles: rows must be a multiple of 8 |
| 31 | 58 | ||
| @@ -68,8 +95,44 @@ narrowed, which took every last-axis case from a two-pass kernel to single-pass. | |||
| 68 | > against the width budget, which shrank `TB` and took working bf16 rungs from 16 columns to 8, | 95 | > against the width budget, which shrank `TB` and took working bf16 rungs from 16 columns to 8, |
| 69 | > regressing 14/20 to 8/20 before anything improved. Budget the narrow tile first. | 96 | > regressing 14/20 to 8/20 before anything improved. Budget the narrow tile first. |
| 70 | 97 | ||
| 98 | +### An UNLABELLED Vec tile is RowMajor, so `Cols * sizeof(T) % 32 != 0` fails to build | ||
| 99 | + | ||
| 100 | +**Which disjunct of that assertion you get is decided by a layout you may never have written.** | ||
| 101 | +The full predicate is | ||
| 102 | +`(RowMajor && NoneBox && Cols*sizeof(DType) % 32 == 0) || (ColMajor && NoneBox && Rows*sizeof(DType) % 32 == 0) || …` | ||
| 103 | +with `alignedSize = 32` (the tile header's `static_assert`). The frontend's default-layout table | ||
| 104 | +`_DEFAULT_LAYOUTS_A5` has **no `MemorySpace.Vec` entry** (`pypto_pro/ir/op/block_ops.py`), and | ||
| 105 | +the lookup returns early when the default is `None`, so a Vec tile declared **without** an | ||
| 106 | +explicit layout keeps `layout=None`, the kwarg is dropped from the IR, and the **C++ template | ||
| 107 | +defaults** apply: `RowMajor`, `NoneBox`. Disjunct 1 is therefore the one being tested, whatever | ||
| 108 | +the tile is for. | ||
| 109 | + | ||
| 110 | +**The consequence bites exactly the per-row scalar operand.** A `[R, 1]` FP32 scale or | ||
| 111 | +reduction tile has `Cols * 4 = 4`, so disjunct 1 is false and bisheng fires the `static_assert`. | ||
| 112 | +**Padding `Rows` does nothing** — the RowMajor disjunct tests `Cols`, so the familiar | ||
| 113 | +over-declaration remedy protects the wrong axis. `Rows` padding only starts working once the | ||
| 114 | +tile is on the ColMajor disjunct. | ||
| 115 | + | ||
| 116 | +**Rule.** Declare a `[R, 1]` Vec tile as `layout=pl.DN` and treat the kwarg as **mandatory, | ||
| 117 | +not decorative**: it selects the ColMajor disjunct, where `Rows * 4 % 32 == 0` is satisfiable by | ||
| 118 | +rounding `Rows` up to a multiple of 8 (which is what the over-declaration idiom above is | ||
| 119 | +actually for). | ||
| 120 | + | ||
| 121 | +⚠ **Coupled caveat, already recorded in this KB:** a `DN` `[R, 1]` tile **rejects elementwise | ||
| 122 | +ops**, and the historical workaround — declaring a row-major alias at the same address — re-arms | ||
| 123 | +the tile-group aliasing corruption. Do not add the alias. Do the scalar arithmetic somewhere | ||
| 124 | +that is not the DN tile. | ||
| 125 | + | ||
| 126 | +**Diagnostic.** The assertion is a compile-time event, so it is observable only at build; and a | ||
| 127 | +shipped sample that omits the layout is **not** evidence the rule does not bind, because a | ||
| 128 | +sample that was never compiled for that key proves nothing. Compile the key, or read the | ||
| 129 | +predicate. | ||
| 130 | + | ||
| 71 | ## `.current()` does not advance the rotation — `.next()` does | 131 | ## `.current()` does not advance the rotation — `.next()` does |
| 72 | 132 | ||
| 133 | +> **Canonical home for the inert-rotation diagnostic.** Other pages cite this one; they must | ||
| 134 | +> not restate the threshold or the magnitudes. | ||
| 135 | + | ||
| 73 | **Symptom.** Adding a buffer changes performance by less than 2%, and multi-buffering looks | 136 | **Symptom.** Adding a buffer changes performance by less than 2%, and multi-buffering looks |
| 74 | worthless. | 137 | worthless. |
| 75 | 138 | ||
| @@ -107,6 +170,35 @@ reduction scratch or its row-major alias. | |||
| 107 | 170 | ||
| 108 | ## `vf.update_mask` per register costs 2.4x–6.5x — hoist it | 171 | ## `vf.update_mask` per register costs 2.4x–6.5x — hoist it |
| 109 | 172 | ||
| 173 | +🔴 **Gate before applying: read [vec.md](vec.md) § "Performance: the mask is the first thing to | ||
| 174 | +look at" and its degenerate-extent rules first.** The hoist replaces a per-register mask with a | ||
| 175 | +hoisted constant plus one peeled tail, and the register count that peel is written from decides | ||
| 176 | +whether it is safe: the `ceil(n/LANES) − 1` spelling reaches `update_mask(LANES)` and | ||
| 177 | +`update_mask(0)`, and reaching a cross-lane reduce through either has **hung a device (507034)** — | ||
| 178 | +a hang, not a fault code, so the arm returns nothing. The `#18` pointer at the end of this section | ||
| 179 | +is a different reference and does not cover it. | ||
| 180 | + | ||
| 181 | +> **Canonical home for this lever's magnitude.** Other pages point here; they must not restate a | ||
| 182 | +> number. Which figure to quote depends on what you are claiming: | ||
| 183 | +> - **A graded result exists and is NOT priced here.** The official per-case table (hidden set, | ||
| 184 | +> named cohort, attribution decided in advance) lives in | ||
| 185 | +> [vec.md](vec.md) § the mask lever — go there for the number you may quote as a *result*. The | ||
| 186 | +> trigger it came from is a **column-block / multi-tile-per-row** inner loop; that site, not the | ||
| 187 | +> operator, is what generalises. | ||
| 188 | +> - **Dev-box range: 2.4x–6.5x** (this page's table). Use it to *rank* candidates, never to claim a | ||
| 189 | +> score. Box and grader have disagreed in **sign** on the same change (+7.46 vs −1.92). | ||
| 190 | +> - A single box figure (e.g. 5.16× on one rms_norm host) is one row of the table below, not a | ||
| 191 | +> separate finding, and not a headline. | ||
| 192 | +> | ||
| 193 | +> If you are about to write a mask-hoist magnitude into another page, link this heading instead. | ||
| 194 | +> | ||
| 195 | +> **The same lever is measured independently upstream**, in | ||
| 196 | +> [`pypto-pro-op-perf-tune`](../../pypto-pro-op-perf-tune/references/a5-roofline-and-levers.md) | ||
| 197 | +> § "The per-register mask is a first-class cost, and hoisting it can be the whole win" — in ns per | ||
| 198 | +> register group plus a shape-class ratio table. Those are that page's numbers and that page's units; | ||
| 199 | +> the per-case table below is this page's. **Cite one or the other, never a figure assembled from | ||
| 200 | +> both** — they were derived on different instruments and are not interchangeable. | ||
| 201 | + | ||
| 110 | **Symptom.** A `vf` kernel whose body is a handful of arithmetic ops runs several times | 202 | **Symptom.** A `vf` kernel whose body is a handful of arithmetic ops runs several times |
| 111 | slower than its byte count implies, and profiles as vector-issue-bound rather than | 203 | slower than its byte count implies, and profiles as vector-issue-bound rather than |
| 112 | bandwidth-bound. | 204 | bandwidth-bound. |
| @@ -126,6 +218,37 @@ pure overhead — and on a short body they dominate. | |||
| 126 | **Rule.** Build one `vf.create_mask(pattern=pl.MaskPattern.ALL, ...)` outside the loop and | 218 | **Rule.** Build one `vf.create_mask(pattern=pl.MaskPattern.ALL, ...)` outside the loop and |
| 127 | use it for every register. | 219 | use it for every register. |
| 128 | 220 | ||
| 221 | +> 🔴 **The hoist is a CORRECTNESS requirement, not only the performance lever the tables | ||
| 222 | +> below measure.** `vf.update_mask` lowers to a special-purpose-register | ||
| 223 | +> write whose **POST_UPDATE is base behaviour, not an opt-in kwarg**, and the installed | ||
| 224 | +> `update_mask` page documents that it *mutates its scalar operand*: | ||
| 225 | +> `scalarValue = (scalarValue < VL_T) ? 0 : (scalarValue - VL_T)`. | ||
| 226 | +> | ||
| 227 | +> So a `vf.update_mask(tail, …)` written **inside** a per-row loop does not merely cost the | ||
| 228 | +> ratios in the tables below — it decrements `tail` on the first row, and every row from 1 | ||
| 229 | +> on then observes `tail == 0`, skips its tail peel and **silently drops up to 63 columns**. | ||
| 230 | +> Ragged widths lose data; nothing faults, and a suite of power-of-two widths cannot see it. | ||
| 231 | +> Two rules follow, and they hold even where the performance argument does not: | ||
| 232 | +> | ||
| 233 | +> * Hoist the mask out of the row loop **because the kernel is otherwise wrong**, then take | ||
| 234 | +> the speed as a consequence. | ||
| 235 | +> * **Do not reuse the scalar operand.** Feed `update_mask` a dedicated expression whose | ||
| 236 | +> only consumer is that call; a scalar that also bounds a loop, sizes a store or addresses | ||
| 237 | +> anything must not be handed to it. | ||
| 238 | +> | ||
| 239 | +> A cheap static guard, if the emitted code is inspectable: assert exactly one `plt_b32` per | ||
| 240 | +> generated kernel, emitted **before** the first `for` of the vector scope. That is a | ||
| 241 | +> mechanical check on an otherwise silent property. | ||
| 242 | + | ||
| 243 | +**Independently confirmed, and not only here.** The same lever is recorded in | ||
| 244 | +[`docs/pypto-pro-dsl-limitations.md`](../../../../docs/pypto-pro-dsl-limitations.md) #18 from a | ||
| 245 | +different campaign and a different operator family: hoisting the mask out of the per-register | ||
| 246 | +loop recovered **550.75 → 398.06 µs** (the DMA floor) on a 67M index fill, and on the reference | ||
| 247 | +generator's exp kernels moved mean SOL **0.300 → 0.961 across 20/20 cases**. Treat the two as | ||
| 248 | +corroborating measurements of one effect rather than as separate findings — and read the two | ||
| 249 | +caveats recorded with #18 (the tail loop must be its own pass; one hoisted variant faults the | ||
| 250 | +device) before applying it, since neither is visible from the numbers on this page. | ||
| 251 | + | ||
| 129 | **Whether the tail path can then be dropped depends on what the inactive lanes' output is | 252 | **Whether the tail path can then be dropped depends on what the inactive lanes' output is |
| 130 | used for, and getting this wrong corrupts results silently.** | 253 | used for, and getting this wrong corrupts results silently.** |
| 131 | 254 | ||
| @@ -159,6 +282,21 @@ one module so neither could be served the other's binary: | |||
| 159 | | 1538x1537 fp32, dim 0 | 8.36 us | 3.50 us | 2.4x | | 282 | | 1538x1537 fp32, dim 0 | 8.36 us | 3.50 us | 2.4x | |
| 160 | | 2039x65520 fp32 | 655.86 us | 649.88 us | 1.01x | | 283 | | 2039x65520 fp32 | 655.86 us | 649.88 us | 1.01x | |
| 161 | 284 | ||
| 285 | +The same lever on `rms_norm`, a two-pass reduce-and-broadcast body, on a second Ascend950PR | ||
| 286 | +host — interleaved A/B/A, run-to-run spread 0.02–0.11 %: | ||
| 287 | + | ||
| 288 | +| case | shape, dtype | per-register mask | hoisted | ratio | | ||
| 289 | +|---|---|---|---|---| | ||
| 290 | +| 4 | 16x256x4096 f16 | 158.83 us | 30.77 us | **5.16x** | | ||
| 291 | +| 18 | 2x511x8192 bf16 | 80.83 us | 17.02 us | 4.75x | | ||
| 292 | +| 8 | 16x511x2049 fp32 | 162.91 us | 96.63 us | 1.68x | | ||
| 293 | +| 16 | 11x13x17x67 f16 | 5.34 us | 3.50 us | 1.53x | | ||
| 294 | + | ||
| 295 | +Both tables are the same curve. The fp32 row lands at 1.68x for the reason case 5 lands at | ||
| 296 | +1.01x — it is the memory-bound shape in its suite, and the vector-side win has nowhere to go. | ||
| 297 | +On the two cache-resident cases the dynamic mask accounted for **81 %** and **79 %** of total | ||
| 298 | +kernel time. | ||
| 299 | + | ||
| 162 | **The gain scales with how far the kernel is from the DRAM roof, which is why it is easy to | 300 | **The gain scales with how far the kernel is from the DRAM roof, which is why it is easy to |
| 163 | dismiss.** The largest row moves 801 MB and is bandwidth-bound at 1.23 TB/s, so the mask costs | 301 | dismiss.** The largest row moves 801 MB and is bandwidth-bound at 1.23 TB/s, so the mask costs |
| 164 | it 1%; every shape that fits cache pays the full 2.4–6.5x. A kernel tuned only on its largest | 302 | it 1%; every shape that fits cache pays the full 2.4–6.5x. A kernel tuned only on its largest |
| @@ -205,23 +343,50 @@ class is invisible to flat-UB simulation. | |||
| 205 | scratch 再显式合并」。按活跃 lane 数给 scratch 定尺，store 会越过末尾写入相邻 UB tile， | 343 | scratch 再显式合并」。按活跃 lane 数给 scratch 定尺，store 会越过末尾写入相邻 UB tile， |
| 206 | 而症状出现在**另一个** tile 上，定位方向天然是错的。 | 344 | 而症状出现在**另一个** tile 上，定位方向天然是错的。 |
| 207 | 345 | ||
| 208 | -## A masked *continuous* store may round its active lanes up to a whole 32-byte block | 346 | +## A masked *continuous* store is LANE-EXACT |
| 209 | 347 | ||
| 210 | -**Unverified for `vf.store_align` — treat as a hypothesis and run the probe below | 348 | +A masked continuous store writes exactly its active lanes: it does **not** round them up to |
| 211 | -before relying on either behavior.** | 349 | +a whole 32-byte block. So it cannot spill past the active lanes onto the already-written |
| 350 | +first lane group of the next row when lane groups are the outer loop, and no defensive write | ||
| 351 | +ordering is needed to protect against that. | ||
| 212 | 352 | ||
| 213 | -A masked continuous store rounded its active lane count up to a whole 32-byte block | 353 | +**Measured**, with a discriminating sentinel tile and a **live control that fired** in the |
| 214 | -(`align8(n)` lanes at b32), so the spill lands past the active lanes. That is harmless | 354 | +same run — the control is what makes the negative result readable at all |
| 215 | -while rows are written in increasing order — it lands on data the loop has yet to write — | 355 | +([investigation-discipline §13](../references/investigation-discipline.md)): |
| 216 | -and wrong the moment a row spans several lane groups with the groups as the outer loop, | ||
| 217 | -because the last group of row `r` then lands on the already-written first group of row | ||
| 218 | -`r+1`. The *scatter* store on the same hardware was lane-exact: its mask is lane-wise and | ||
| 219 | -it writes exactly the active lanes. | ||
| 220 | 356 | ||
| 221 | -**Probe** (one build): store `n < 8` fp32 lanes under mask at a row boundary with the next | 357 | +| store form | active lanes | elements actually moved | |
| 222 | -row pre-filled with a sentinel, then read the sentinel back. If it was overwritten, masked | 358 | +|---|---|---| |
| 223 | -`vf.store_align` rounds up too — order the writes so the spill lands on not-yet-written | 359 | +| `vf.store_align(dist=pl.StoreDist.FIRST_ELEMENT)` | 1 | **exactly 1** | |
| 224 | -data, or store the boundary through `vf.scatter`. | 360 | +| `vf.store_align(dist=pl.StoreDist.NORM)`, masked | 3 | **exactly 3** | |
| 361 | + | ||
| 362 | +No `align8(n)` round-up, and no whole-256-byte register write. The three candidate | ||
| 363 | +behaviours — 1 element, `align8` = 8 elements, 256 bytes — are discriminated, and it is the | ||
| 364 | +first. | ||
| 365 | + | ||
| 366 | +**Scope, which is the part worth carrying.** The discrimination ran on a **RowMajor | ||
| 367 | +`[2, 256]`** tile. A `DN`-layout tile was **not** exercised, and neither was any distribution | ||
| 368 | +mode other than the two above. So this settles the continuous masked store on a row-major | ||
| 369 | +tile, and says nothing about the others. | ||
| 370 | + | ||
| 371 | +**This does NOT generalise to a distribution-mode store, and the two must not be | ||
| 372 | +conflated.** A masked `vf.store_align(dist=INTLV_B32)` **ignores its predicate entirely** and | ||
| 373 | +a distribution store writes the whole register — both are separately measured and are recorded | ||
| 374 | +below under "A distribution-mode store writes the whole register, mask or no mask". Lane-exact | ||
| 375 | +is a property of the *continuous* store. | ||
| 376 | + | ||
| 377 | +🔴 **Report the raw counts, never a probe's summary label.** The probe that produced the | ||
| 378 | +table above printed the string `align8 round-up` next to a lane-exact 3-element result — a | ||
| 379 | +label written before the data and never revisited. Anyone reading the log rather than the | ||
| 380 | +counts would have recorded a round-up as *confirmed*. When a probe's whole job is | ||
| 381 | +to discriminate between named branches, the branch names belong in the analysis, not in the | ||
| 382 | +instrument. | ||
| 383 | + | ||
| 384 | +## Below one register's width the mask hoist stops paying | ||
| 385 | + | ||
| 386 | +**That boundary has been hit twice.** | ||
| 387 | +At 8 or 16 columns the hoist measured +5 % with six cases 3–4 % *slower*; applying the identical | ||
| 388 | +hoist to a D=2 packed path measured **-1.0 %** and was rolled back as inside noise. The | ||
| 389 | +full-register loop does not execute there, so the hoist buys only bookkeeping. | ||
| 225 | 390 | ||
| 226 | --- | 391 | --- |
| 227 | 392 | ||
| @@ -246,3 +411,149 @@ One myth was busted in the same investigation and should not be reintroduced: `i | |||
| 246 | out-of-bounds access, misattributed. | 411 | out-of-bounds access, misattributed. |
| 247 | 412 | ||
| 248 | --- | 413 | --- |
| 414 | + | ||
| 415 | +## Per-token scalar staging wants one whole 32-byte row per token | ||
| 416 | + | ||
| 417 | +**Cross-DSL: EasyASC board measurement on Ascend 950. 未在 PyPTO-Pro 上验证——由 | ||
| 418 | +EasyASC 移植的假设 (unverified on PyPTO-Pro).** It does **not** override the | ||
| 419 | +PyPTO-Pro measurement it sits next to; read that one first. | ||
| 420 | + | ||
| 421 | +**What this KB already measured, and which stays authoritative.** An FP16 | ||
| 422 | +scalar-Tensor store is not safe at a 32-byte ownership boundary — accuracy 0.90 | ||
| 423 | +overall and 0.90625 at `block_dim=32`, while **64 B and 128 B boundaries were | ||
| 424 | +exact** and a whole-tile `pl.store` was exact at all three | ||
| 425 | +([framework-findings, A5 probe session](../references/pypto-pro-framework-findings.md)). | ||
| 426 | +The conclusion there is: prefer the beat-complete tile store. | ||
| 427 | + | ||
| 428 | +**What EasyASC adds, from the staging side rather than the store side.** Its | ||
| 429 | +`constraints/a5.md` §12 reports that repeatedly moving *one scalar per token* | ||
| 430 | +between GM and UB — `m > 1`, `n == 1` — into a compact `[1, L]` or `[L, 1]` UB | ||
| 431 | +layout **can misaddress later rows on hardware while its Python simulator | ||
| 432 | +passes**. Its prescription is to give each token one physical 32-byte UB row and | ||
| 433 | +use only the first element of it: | ||
| 434 | + | ||
| 435 | +``` | ||
| 436 | +scalar_ub : [L, C0] # C0 = 32 B / sizeof(dtype); 16 for bf16, 8 for fp32 | ||
| 437 | + # logical slice stays [L, 1]; the backing stays [L, C0] | ||
| 438 | +``` | ||
| 439 | + | ||
| 440 | +EasyASC is explicit that this is a **burst-pitch** requirement — a property of the | ||
| 441 | +repeated one-element transfer — and *not* a general ban on unaligned scalar | ||
| 442 | +addressing, which it says is fine for an isolated access. | ||
| 443 | + | ||
| 444 | +**Why it is worth carrying.** It is the same 32-byte quantum as the measured | ||
| 445 | +PyPTO-Pro finding above, reached from a different direction (staging pitch rather | ||
| 446 | +than concurrent-store ownership), by a different DSL, on the same silicon. Two | ||
| 447 | +independent arrivals at "give the scalar a whole beat" is the kind of agreement | ||
| 448 | +that should raise your prior before you spend a board session. It also predicts | ||
| 449 | +something the PyPTO-Pro measurement did not test: that the hazard survives even | ||
| 450 | +with a single writer, because it is about burst addressing rather than about two | ||
| 451 | +cores sharing a beat. | ||
| 452 | + | ||
| 453 | +**Named probe:** stage `L` per-token fp32 scalars into a `[L, 1]` UB tile and into | ||
| 454 | +an `[L, 8]` tile whose column 0 carries the payload, read both back, and compare | ||
| 455 | +against the host values at `L` large enough to need several bursts. A clean `[L, | ||
| 456 | +1]` result retires this section; a mismatch that appears only past the first burst | ||
| 457 | +confirms it. Note the control requirement — the probe is void unless the same run | ||
| 458 | +shows the checker can report a mismatch | ||
| 459 | +([investigation-discipline §13](../references/investigation-discipline.md)). | ||
| 460 | + | ||
| 461 | +## A distribution-mode store writes the whole register, mask or no mask | ||
| 462 | + | ||
| 463 | +Independent corroboration, from a second DSL, of a silent failure this KB already | ||
| 464 | +records — and a constructive consequence the original entry does not state. | ||
| 465 | + | ||
| 466 | +**Already measured here.** A masked `vf.store_align(dist=INTLV_B32)` **ignores its | ||
| 467 | +predicate entirely**, writing every lane; the store succeeds and the data is wrong | ||
| 468 | +only in the lanes the mask was supposed to protect | ||
| 469 | +([framework-findings, A5 probe session](../references/pypto-pro-framework-findings.md)). | ||
| 470 | + | ||
| 471 | +**The corroboration.** EasyASC's `constraints/a5.md` §6.2 reports, on the same | ||
| 472 | +silicon, that its `DIST_NORM_B8` store "writes the full 256-byte register even | ||
| 473 | +when only a prefix mask is active". Same shape of defect, a different DSL, a | ||
| 474 | +different distribution mode, and a *different* mask (a prefix rather than a | ||
| 475 | +scatter pattern). Two arrivals make this look like a property of the | ||
| 476 | +distribution-mode store path rather than an artifact of one lowering. | ||
| 477 | + | ||
| 478 | +**The consequence worth acting on, which is about allocation rather than | ||
| 479 | +correctness.** EasyASC states it as a sizing rule: the scratch a masked | ||
| 480 | +distribution store targets must be allocated for the **whole register**, 256 | ||
| 481 | +bytes, not for the active lane count. That is the constructive form — the | ||
| 482 | +PyPTO-Pro entry tells you the data will be wrong; this tells you the buffer will | ||
| 483 | +also be too small, which is the failure that shows up as corruption of whatever | ||
| 484 | +was allocated next to it. | ||
| 485 | + | ||
| 486 | +**未在 PyPTO-Pro 上验证 for the sizing rule specifically** (the mask-is-ignored | ||
| 487 | +half *is* measured here). To verify: place a sentinel immediately after a scratch | ||
| 488 | +tile sized to the active lane count, issue a masked distribution store into it, | ||
| 489 | +and read the sentinel back. | ||
| 490 | + | ||
| 491 | +## `D` not being a multiple of 64 costs nothing — register COUNT is what costs | ||
| 492 | + | ||
| 493 | +**Claim under test.** On `rms_norm`/A5, every public fp16 case with `D` a multiple of 64 beat | ||
| 494 | +the CANN reference (ratio 0.91–0.92) and every fp16 case with `D` not a multiple of 64 lost to | ||
| 495 | +it (1.08–1.48). A 7/7 split, and the near-controlled pair `[32,128,768]` vs `[33,127,769]` — | ||
| 496 | +one extra element of `D`, row count controlled to 2.3 % — showed **+21 %**. The natural reading | ||
| 497 | +is a ragged-column-tail or 32-byte-burst-alignment defect. | ||
| 498 | + | ||
| 499 | +**It is not.** Measured on the pinned `kernel_eval` harness (`--perf-metric-strategy | ||
| 500 | +kernel_details`), Ascend950PR, the measuring card, fp16, `S = 4096` held fixed while `D` walks. Same-shape | ||
| 501 | +repeats inside the run drifted **0.00 %, 0.00 %, −1.88 %**, so the instrument resolves ~2 %. | ||
| 502 | + | ||
| 503 | +The decisive triple shares the `MaxN = 1024` rung, 16 registers per row and one UB footprint — | ||
| 504 | +only alignment differs: | ||
| 505 | + | ||
| 506 | +| D | 32B-aligned | `D mod 64` | `D == MaxN` | µs | | ||
| 507 | +|---:|:--:|---:|:--:|---:| | ||
| 508 | +| 1008 | ✓ | **48** | ✗ | 10.50 | | ||
| 509 | +| 1023 | ✗ | 63 | ✗ | 10.47 | | ||
| 510 | +| 1024 | ✓ | 0 | ✓ | 10.44 | | ||
| 511 | + | ||
| 512 | +**0.6 % apart.** `D = 1023` — the width of the public case that loses 11 % — costs what | ||
| 513 | +`D = 1024` costs. The same holds one rung down: `D = 80` (`mod 64 = 16`, 32B-aligned) costs | ||
| 514 | +3.07 µs, exactly what `D = 128` (`mod 64 = 0`) costs, while `65/67/127` cost 3.22/3.19/3.14 — | ||
| 515 | +a residual of ≤ 3.7 % that **shrinks with `D` and is gone by `D ≈ 1000`**, the opposite of the | ||
| 516 | +direction the deficit needs. | ||
| 517 | + | ||
| 518 | +**So the modulus question is ill-posed: there is no modulus.** 64 is dead, and 128 and 256 die | ||
| 519 | +with it. What predicts cost is `ceil(D/64)` — the register count — a *ceiling*, not a residue | ||
| 520 | +class. Every `D` in `(64k, 64k+64]` costs the same, which is exactly why aligned and misaligned | ||
| 521 | +`D` are indistinguishable once register count is controlled. Fitting the three points that | ||
| 522 | +share `MaxN = 1024` and `Rows = 16` (13, 15, 16 registers → 9.04, 10.13, 10.47 µs) gives | ||
| 523 | +**2.75 µs fixed + 0.486 µs per register**, reproducing all three to ±1 %. | ||
| 524 | + | ||
| 525 | +**Why the public split looked so convincing — the confound.** In that suite the aligned fp16 | ||
| 526 | +cases sit at `D = 768, 4096, 4096` and the misaligned ones at `D = 67, 769, 1021, 1023`. There | ||
| 527 | +is **no aligned fp16 case anywhere in the `D ∈ [769, 2047]` band**, so "aligned vs misaligned" | ||
| 528 | +is perfectly confounded with "`D` at 768/4096 vs `D` near 1024". Probing `D = 960, 1008, 1024` | ||
| 529 | +— aligned and *in band* — breaks the confound: they are just as slow as 1023. Measured against | ||
| 530 | +the fused reference (`torch_npu.npu_rms_norm`) on one clock, our ratio is 1.19 at `D = 768` and | ||
| 531 | +1.40–1.42 at `D = 960/1008/1023/1024` alike. The ratio degrades with `D`-position on the rung | ||
| 532 | +ladder, not with alignment. | ||
| 533 | + | ||
| 534 | +**Two hypotheses this also kills**, since the same points control them: | ||
| 535 | + | ||
| 536 | +* *Row tiling / ragged `S`.* With `D = 768` fixed, the ragged row counts of the slow cases cost | ||
| 537 | + **less** per row than the round one: 1.8611 (S=4191) and 1.8645 (S=4221) against 1.8945 | ||
| 538 | + µs/1000 rows (S=4096). The remaining variation with `S` is `_pick_rows` landing on Rows 21 or | ||
| 539 | + 8 instead of 16 — a documented smooth function of `S`, not of its factorisation. | ||
| 540 | +* *Burst contiguity collapse* (`D == MaxN` ⟹ `dst_gap == 0` ⟹ the whole `[Rows, D]` block moves | ||
| 541 | + as one burst instead of `Rows` short ones). `D = 1024` has it, `D = 1008` and `1023` do not, | ||
| 542 | + and all three cost the same. Worth 0 here. | ||
| 543 | + | ||
| 544 | +**What the `768 → 769` +21 % actually is.** It reproduces (+16.9 % at controlled `S = 4096`) and | ||
| 545 | +then decomposes entirely into two already-known quantisations: 12 → 13 registers (+8.3 % of | ||
| 546 | +vector work) and a `MaxN` rung jump 768 → 1024 (a 33 % wider tile for one extra element, worth | ||
| 547 | +~10 % by the fit above). `D = 1040` clinches it — one register *more* than 1024, on the 2048 | ||
| 548 | +rung — at **10.20 µs, below** `D = 1024`'s 10.44. Only rung/`Rows` selection can do that; | ||
| 549 | +alignment cannot. | ||
| 550 | + | ||
| 551 | +**Rule.** Do not spend a run making `D` "aligned" for a `vf` row kernel of this shape. Size the | ||
| 552 | +tile to `align64(D)` because `load_align` moves a whole register and will otherwise fault | ||
| 553 | +`507035` (first rule on this page) — that is a **correctness** constraint. It is not a | ||
| 554 | +performance one. If a benchmark's slow cases all have ragged `D`, check first whether ragged | ||
| 555 | +`D` is confounded with `D` magnitude or with register count in that suite; in this one it was. | ||
| 556 | + | ||
| 557 | +**Diagnostic that settles it in one eval run:** probe three `D` values sharing a rung and a | ||
| 558 | +register count, one aligned-and-equal-to-rung, one aligned-but-below-rung, one misaligned. If | ||
| 559 | +they land together, alignment is not your mechanism. | ||
| @@ -0,0 +1,134 @@ | |||
| 1 | +# Float constants that reach a `vf.*` op | ||
| 2 | + | ||
| 3 | +## Trigger | ||
| 4 | + | ||
| 5 | +Your kernel passes a **Python float literal** as the scalar argument of a | ||
| 6 | +register-level op — `vf.maxs`, `vf.adds`, `vf.muls`, `vf.full`, any `vf.*s` | ||
| 7 | +family member — or folds a float constant into a `@pl.vector_function` body. | ||
| 8 | +That is the whole trigger; it does not depend on topology, dtype or shape. | ||
| 9 | + | ||
| 10 | +## Rule | ||
| 11 | + | ||
| 12 | +🔴 **A `vf.*` scalar immediate is rendered into the generated C++ with `%f`, so | ||
| 13 | +it keeps six fractional digits and nothing else. Any constant whose magnitude is | ||
| 14 | +below roughly `5e-7` becomes `0.000000f` — the constant is silently deleted.** | ||
| 15 | +Build such a constant in registers instead, and assert the bit-identity of the | ||
| 16 | +construction at import. | ||
| 17 | + | ||
| 18 | +There is no error at any layer. The Python is legal, the parser accepts it, the | ||
| 19 | +kernel compiles, the launch returns, and every ordinary input produces the right | ||
| 20 | +answer. The value only differs on the inputs the constant existed to protect, | ||
| 21 | +which is why this survives a correctness suite that does not deliberately | ||
| 22 | +contain degenerate data. | ||
| 23 | + | ||
| 24 | +## What was measured | ||
| 25 | + | ||
| 26 | +Ascend950PR / CANN 9.2.0, by emitting a throwaway probe kernel and reading its | ||
| 27 | +generated `kernel.cpp`: | ||
| 28 | + | ||
| 29 | +| written | emitted | | ||
| 30 | +|---|---| | ||
| 31 | +| `vf.maxs(reg, 1e-12, preg)` | `vmaxs(dst, src, 0.000000f, preg, MODE_ZEROING)` | | ||
| 32 | +| `vf.full(1e-12, preg, dtype=pl.DT_FP32)` | `vdup(dst, 0.000000f, preg, MODE_ZEROING)` | | ||
| 33 | +| `vf.adds(reg, 1e-12, preg)` | same truncation | | ||
| 34 | +| `vf.full(6.103515625e-5, …)` | `0.000061f` — representable, but not the value asked for | | ||
| 35 | +| `vf.full(-3.0e38, …)` | `-3e+38` — large magnitudes are unaffected | | ||
| 36 | + | ||
| 37 | +So the defect is **one-sided**: it destroys small magnitudes and leaves large | ||
| 38 | +ones alone. A sentinel is safe; a floor, an epsilon, a reciprocal-of-a-large- | ||
| 39 | +number and a denormal guard are not. `6.103515625e-5` is the smallest normal | ||
| 40 | +FP16 — a value that reaches this code path in any narrow-dtype design — and it | ||
| 41 | +already loses four significant digits. | ||
| 42 | + | ||
| 43 | +## The workaround, and the assertion that makes it safe | ||
| 44 | + | ||
| 45 | +Compose the constant from a representable one: | ||
| 46 | + | ||
| 47 | +```python | ||
| 48 | +_ROOT = 1e-6 # renders exactly as 0.000001f | ||
| 49 | +assert struct.pack("<f", _ROOT) == ... # see below | ||
| 50 | +... | ||
| 51 | +root = vf.full(_ROOT, preg, dtype=pl.DT_FP32) | ||
| 52 | +floor = vf.mul(root, root, preg) # exactly fp32(1e-12) | ||
| 53 | +safe = vf.max(rmax, floor, preg) | ||
| 54 | +``` | ||
| 55 | + | ||
| 56 | +This is exact rather than approximate for this particular pair: `fp32(1e-6)` | ||
| 57 | +squared is **bit-identical** to `fp32(1e-12)` (both `0x2b8cbccc`). That is a | ||
| 58 | +property of these two values, **not a general law about squaring a square | ||
| 59 | +root**, so the construction has to be checked, not assumed: | ||
| 60 | + | ||
| 61 | +```python | ||
| 62 | +import struct | ||
| 63 | +_F = lambda x: struct.unpack("<I", struct.pack("<f", x))[0] | ||
| 64 | +assert _F(np.float32(_ROOT) * np.float32(_ROOT)) == _F(TARGET) | ||
| 65 | +``` | ||
| 66 | + | ||
| 67 | +Put the assertion at **import** time, not in a test. It costs nothing, it runs | ||
| 68 | +in every process that loads the kernel, and it is the only thing standing | ||
| 69 | +between a future edit of `_ROOT` and a silently wrong floor. | ||
| 70 | + | ||
| 71 | +Two further points on the workaround: | ||
| 72 | + | ||
| 73 | +- **Two ops, not one.** The extra `vf.full` + `vf.mul` are loop-invariant, so | ||
| 74 | + hoist them out of the row loop; the cost is then per call rather than per row. | ||
| 75 | +- **Check the scalar op it replaces is gone.** `vf.maxs` with an immediate | ||
| 76 | + becomes `vf.max` against a register. Grep the generated kernels for the | ||
| 77 | + scalar mnemonic (`vmaxs`, `vadds`, …) and require a count of **zero** — a | ||
| 78 | + compile-time gate that cannot be satisfied by a passing correctness run. | ||
| 79 | + | ||
| 80 | +## The neighbouring failure: a constant that is not a number at all | ||
| 81 | + | ||
| 82 | +The same constant-folding path cannot express non-finite values. `float("nan")` | ||
| 83 | +and `float("inf")` are lowered as the **bare, undeclared C++ identifiers** | ||
| 84 | +`nanf` / `inff`: | ||
| 85 | + | ||
| 86 | +```cpp | ||
| 87 | +vdup(nanreg, nanf, preg, MODE_ZEROING); | ||
| 88 | +``` | ||
| 89 | +``` | ||
| 90 | +kernel.cpp:65:9: error: no matching function for call to 'vdup' | ||
| 91 | +kernel.cpp:38:135: error: use of undeclared identifier 'inff'; did you mean 'inc'? | ||
| 92 | +``` | ||
| 93 | + | ||
| 94 | +This one is **loud** — it fails the build — so it is cheap in comparison. What | ||
| 95 | +is worth recording is the **divergence**: the identical NaN and ±Inf values | ||
| 96 | +delivered through a real GM tensor work correctly through the whole vector | ||
| 97 | +path. Only the constant-fold route is broken. A kernel that needs a non-finite | ||
| 98 | +sentinel must either take it from memory or use a large finite stand-in | ||
| 99 | +(`-3.0e38`), and a design that assumed "the DSL has no NaN semantics" because | ||
| 100 | +this failed to compile has drawn the wrong conclusion. | ||
| 101 | + | ||
| 102 | +**This escapes the kernel body.** A module-level Python sequence is hoisted | ||
| 103 | +into the generated kernel even when the kernel never mentions it, so a list of | ||
| 104 | +floats containing a NaN or an Inf anywhere in the kernel module's import graph | ||
| 105 | +breaks the compile of an **unrelated** kernel. Keep such values as strings and | ||
| 106 | +convert them inside a function. | ||
| 107 | + | ||
| 108 | +## What this does not cover | ||
| 109 | + | ||
| 110 | +- Integer immediates. The `%f` truncation is a float-formatting defect; an | ||
| 111 | + `int` scalar reaches the instruction intact. | ||
| 112 | +- Tile-level ops. The measurement above is the register-level `vf.*` surface. | ||
| 113 | + A tile-level scalar-mode call (`pl.maximum(out, lhs, scalar)`) was not | ||
| 114 | + measured here; treat it as unknown and check the generated code before | ||
| 115 | + relying on a small tile-level immediate. | ||
| 116 | + | ||
| 117 | +## Related | ||
| 118 | + | ||
| 119 | +- [vec.md](vec.md) — the rest of the `vf` authoring rules. | ||
| 120 | +- [precision.md](precision.md) — where a deleted floor actually shows up: the | ||
| 121 | + divide-by-zero guard of a per-row scale. | ||
| 122 | +- [../patterns/vec-per-token-dynamic-quant.md](../patterns/vec-per-token-dynamic-quant.md) | ||
| 123 | + — the topology in which this was found, and the exact constant it deleted. | ||
| 124 | +- [../references/pypto-pro-dsl-limitations-a5.md](../references/pypto-pro-dsl-limitations-a5.md) | ||
| 125 | + — the upstream report entry, with the codegen evidence. | ||
| 126 | + | ||
| 127 | +## Validation | ||
| 128 | + | ||
| 129 | +**Hardware-measured** on Ascend950PR / CANN 9.2.0 by reading generated C++ for | ||
| 130 | +both the defect and the fix, and by an end-to-end all-zero-row case that fails | ||
| 131 | +with the immediate and passes with the register-built constant. The `%f` format | ||
| 132 | +string itself was not located in the emitter; the truncation is inferred from | ||
| 133 | +its signature (six fractional digits, magnitude-only dependence), which is | ||
| 134 | +consistent with every value measured but is not a source-level confirmation. | ||
Mcannbot-skills/ops/pypto-pro-op-perf-tune/scripts/tests/test_msprof_evidence.py+64-0文件内容审核中，请稍后刷新重试