Merged
feat(pro): Improve frontend diagnostics and contract checks #6868
ddddweAdd on 29 days ago
feat(pro): Improve frontend diagnostics and contract checks #6868
Merged
ddddweAdd on 29 days ago
ddddwe
29 days ago

Pro frontend errors and warnings carry DSL call and argument locations. Tile builder validation uses an explicit span when supplied and inherits current_span when omitted, including nested reinterpretation and capacity warnings. Capacity lookup failures produce located CapacityUnknown warnings for bounded on-chip buffers, including MX ScaleLeft/ScaleRight; DDR does not trigger this warning.

Tile capacity checks raise OutOfRange immediately on the first over-capacity declaration, in both normal compilation and pipeline probe parsing. Error rendering can omit source previews while preserving error codes, locations, hints and notes. Tests explicitly inspect the error message or rendered diagnostic, keeping assertions independent of source previews and pytest internals.

TileType equality checks effective MemRef presence and memory space. A known memory reference differs from an unspecified one; two unspecified references can match. Allocation addresses and sizes do not participate in type equality. Packed host shape binding normalizes the last axis of 4-bit tensors directly; explicit static_shapes already count logical elements. Compile-time binding and launch validation share accepted Torch storage dtypes: sub-byte data uses uint8, with native Torch FP4 also accepted for FP4E2M1. FP8 supports its matching Torch dtype or a uint8 carrier. INT16 and UINT16 each accept torch.int16 or torch.uint16 as bit-preserving 16-bit storage carriers. UINT32 accepts torch.uint32 or torch.int32 as 32-bit carriers. Integer carriers of the wrong width and unrelated floating-point dtypes remain invalid; no numerical dtype conversion is performed.

The frontend also checks dynamic slot reinterpretation, MX data/scale slot layouts and manual mutex aliases. IR observation exposes versioned attribute snapshots and invocation-local parse observers. Existing scalar arithmetic aliases have public declarations and documentation. Capacity validation uses the upstream Bias/Scaling and MX buffer mappings.

The existing gather comparison and gatherb STs use their original int32 storage directly. Their UINT32 arguments retain the same 32-bit bit patterns without a dtype conversion. This PR does not modify any ST case.

Validation:

  • PR branch baseline: c8367c8ece9f031c2a22fa64ee2244d63c10bcb0. The earlier diagnostics validation below is recorded against cb13c5555; current carrier compatibility and A5 integration checks are listed separately.
  • Recorded Pro Python UT on cb13c5555: 2742 passed, 21 existing architecture skips, one expected MutexMismatch warning
  • Diagnostics and contract regressions with all conftest files disabled (--noconftest): 67 passed
  • ST bundle collection: 3 tests collected successfully; this covers the duration_estimate import in test_kernel_bundle.py
  • First-error regressions cover normal and pipeline probe parsing; the first capacity error and its source location are preserved
  • Current carrier and packed-shape regression: 157 passed, including 74 carrier cases across compile binding and launch. Both signed/unsigned 16-bit carrier combinations and UINT32/int32 are accepted in compile binding and launch. Wider 32-bit buffers remain rejected for 16-bit parameters; narrower storage and floating-point mismatches are rejected for UINT32.
  • The recorded full Pro UT used pytest 8.4.2; current targeted checks and A5 ST use pytest 9.1.1.
  • Existing validation of the unchanged production code: C++ structural comparison suite 63 passed, upstream MX/Bias/Scaling capacity regression 1 passed, generated CCE compilation probes 5 passed. These interface-specific native checks retain their recorded results; current A5 integration validation is listed separately below
  • Applicable pre-commit checks passed for the current patch. The previously recorded CodeCheck comparison found no new findings across seven affected Python test files. The external agent log-spec hook was not run
  • Current A5 integration with upstream c859e50090de549a226c4e1805b516cffd673993: validation tree 7dd5abf7d8fbd9b3171a715892e24a4d67b34ed3 matches PR head c718e2e12. Native build inputs are unchanged from the full A5 build, whose native payload is reused in an isolated wheel with the updated Python JIT and regenerated RECORD. All 5182 source files, 1359 native-source/binding files and 158 Python files were verified; no Python overlay is used.
  • Current 16-bit compatibility passed 8 on-device bit-pattern copy combinations across INT16/UINT16 declarations and signed/unsigned input/output carriers. The original quant-lightning-indexer ST passed on the current integration with its unchanged int16 score buffer. The unchanged UINT32 path had both original gather comparison and gatherb STs pass on PR head 3244fbab0. The Tensor API softmax example passed on the earlier integration.
  • The single-card A5 ST sweep covered all 1833 selected cases (524 excluded by existing SoC/world-size markers): 1675 passed, 137 failed, 19 skipped, 2 expected failures. Completed-prefix evidence is bound to its earlier snapshots, and the remaining sweep used head 3244fbab0; these totals are not a complete rerun on c718e2e12. The UINT16 carrier rejection found in that sweep is addressed by the current patch. No ST cases are modified.
  • Unmodified upstream c859e5009 was independently built with the same CANN/PTO setup and all 137 failed nodes were rerun on the same device. 132 also failed upstream: 87 merged-axis cases, 18 core-limit cases, 4 HF8 scatter cases, 3 L0C-to-L1 cases, 1 per-channel-scale case, 4 VF control-flow cases, 10 blocking-launch/affected following cases, 1 missing-msprof case and 4 device-ELF dump assertions. Representative merged-load and L0C TEXTRACT/TINSERT compiler failures have matching generated code. These upstream/environment failures are outside this PR and their tests are unchanged.
  • Of the other five nodes, three GM Gather cases passed on both upstream and the current PR integration. The UINT16 quant-lightning-indexer regression passed after the carrier fix. The online-softmax shape test passed alone on both installations; running it after the Gather cases reproduces the native GetGraphInfo crash on pristine upstream as well, establishing an existing order-dependent failure. There is no full-suite pass claim.
likedislike
Pull Request has been successfully merged by CANN-robot.
(Thank you for ddddwe's contribution.)
Dddddwe
29 days ago Created pull request, commit 5cfc0f49
atomgit-bot
atomgit-bot
29 days ago Comments:

变更摘要

该 PR 主要围绕 Pro 前端诊断信息的可用性与若干契约校验的完备性展开。诊断侧,_errors.py 引入 PyptoProWarning 基类及 CapacityUnknown、CheckSkipped、MutexMismatch 等警告类别,新增 _warn_located、warn_check_skipped 与 render(with_source=...),使 DSL 调用位置可被结构化保留、警告可按类别过滤,并支持不渲染源码预览地输出诊断;配套 python/tests/conftest.py 让 pytest 的 match= 只匹配诊断文本而非源码预览。契约校验侧,block_ops.py、_call_parser.py、_buffer_parser.py 增加了动态槽位重解释、MX 缩放平面地址、手动 mutex 别名重叠、packed host 形状等检查,并通过 collect_capacity_overflows 聚合多处容量溢出。同时新增版本化 IR 属性快照 API、parse 观察者与标量算术别名,并修正相关文档。

主要改动

  • 诊断定位与警告分类: _errors.py 新增 PyptoProWarning、CapacityUnknown、CheckSkipped、MutexMismatch 及 _warn_located、warn_check_skipped,并让 PyptoProError.render() 支持 with_source=False;block_ops.py 中大量 raise 补传 span=span,使错误与警告携带 DSL 调用位置且可按类别过滤。

  • 容量校验聚合与告警: block_ops.py 新增 collect_capacity_overflows() 上下文管理器聚合单次 parse 中的多处容量溢出并统一抛出 OutOfRange,_get_memory_capacity 在可用空间容量未知时发出 CapacityUnknown,动态偏移检查则通过 warn_check_skipped 以可选方式提示。

  • 新增契约检查: _call_parser.py 增加 _check_mx_plane_slots/_mx_group_slot_addrs 校验每个 MX scale 槽位地址与数据槽位地址 >> 4 的对应关系,并拒绝 Vector 段使用 bar_m/bar_mte1/bar_fix;block_ops.py 与 _buffer_parser.py 借助 _tile_slot_source、_dynamic_slot_tiles 拒绝运行时槽位索引的 pl.reinterpret(),并新增 warn_manual_mutex_aliases() 对重叠但无共享 mutex ID 的手动别名缓冲发出 MutexMismatch。

  • 缓冲类型与形状策略: structural_equal.cpp 将 memorySpace_ 纳入 TileType 相等性比较(地址与大小仍属分配元信息);platform.h/platform.cpp 新增 kMemorySpaceBias、kMemorySpaceScaling 并映射到 btSize、fixQuantPreSize;shape_policy.py 将 _pack_factor 迁入并用于 bind_runtime_args 对 packed host 形状按逻辑元素归一。

  • 新增 API 与文档订正: ir.cpp 与 ir/__init__.py 暴露 get_attributes、attributes_api_version 以读取 Call/Function/ForStmt 的属性快照;kernel.py 的 parse_target_program 新增 observer 参数;language/__init__.py 与 _api.py 声明标量别名 adds、subs、muls,并订正 pl.struct、init_output、matmul bias、pto_assert、@pl.inline 等文档说明。

likedislike
Incorrect?
atomgit-bot
atomgit-bot
29 days ago Comments:

代码审查

✅ 未发现问题

likedislike
Incorrect?
CANN-robot
CANN-robotCollaborators
29 days ago Comments:

Hi @ddddwe, welcome to submitting your first PR to pypto!

PR Merge Steps

1. CLA Signing

If the current PR label includes cann-cla/yes, it means you have signed the CLA and can proceed to the next step. If the label includes cann-cla/no, please sign the CLA first. If you have any questions, please refer to the FAQ.

2. CI Check

Please comment /compile to trigger the CI pipeline check. If the CI run is successful, the PR will be tagged with ci-pipeline-passed and you can proceed to the next step. If the CI run fails, the PR will be tagged with ci-pipeline-failed, please check the CI logs to fix the issues in the PR. If you have any questions, please refer to the FAQ.

3. Code Review

After CI passes, please refer to the PR Approval Progress and proactively @ the committers in the table to review the code. After approval, committers will comment /lgtm and /approve. Once the lgtm and approved labels are successfully added, the PR will be merged automatically.

likedislike
CANN-robotCANN-robotCollaborators
29 days ago Added label: cann-cla/yes
218 messages are collapsed here。 See more
gaoxiang618Collaborators
17 days ago Comments:

/approve
/lgtm

likedislike
CANN-robotCANN-robotCollaborators
17 days ago Added label: lgtm
lyz0328
lyz0328Collaborators
17 days ago Comments:

/approve

likedislike
CANN-robotCANN-robotCollaborators
17 days ago Added label: approved
CANN-robotCANN-robotCollaborators
17 days ago Merged pull request