已开启
fix(op-verify): Validate adversarial suite against kernel contract, fix suite default path #586
fix(op-verify): Validate adversarial suite against kernel contract, fix suite default path #586
已开启
lizixuan创建于 6 天前
lizixuan
lizixuan
6 天前

Closes #117

问题

对抗测试套件与 kernel 静态契约对不上(12 个用例必然报 F00002)+ runner 按"当前目录"找套件文件:

  1. 用例生成不看契约:module_interfaces.yaml 写死 kv_cache 静态形状 [2048,…],但用例凑 block_num=b×ceil(s2/128)=16≠2048,进 kernel 必 F00002,却计入 PASS/FAIL 门禁统计。
  2. runner 路径缺陷--suite 默认 ./adversarial_suite.json(相对当前目录),从仓库根启动必 FileNotFoundError。

修复(仅模板 adversarial_runner_template.py.tmpl)

  1. --suite 默认值改为相对脚本自身目录(os.path.dirname(os.path.abspath(__file__)))。
  2. 新增 excluded 探针语义:对抗性/故意越界用例打 "excluded": true,只进 report[probes],不进 cases_total/cases_passed("对抗性"与"门禁统计"各归各)。
  3. 新增 --contract 契约校验:hook _contract_violation_reason(case)(verifier 按算子填充静态形状几何);_run 遇未标 excluded 却违反契约的用例直接 raise,--self-test 则返回 1 并逐条点名——越界从运行时噪音变成生成期硬错误。

验证

  • 模板语法 OK;_is_excluded_probe/_partition_cases/_check_contract 单元验证通过。
  • 改动仅在模板一个文件,不霰弹式扩散。
likedislike
合并受阻
lizixuanlizixuan
6 天前 创建了 pull request,commit d97aaef2
lizixuanlizixuan
6 天前 关联了issue:[Bug-Report|缺陷反馈]: [断裂点]对抗测试套件与 kernel 静态契约对不上(12 个用例必然报 F00002)+ runner 按"当前目录"找套件文件导致仓库根运行必崩
CANN-robotCANN-robot成员
6 天前 添加了label:cann-cla/yes
CANN-robot
CANN-robot成员
6 天前 评论:

Thanks for your pull-request.
The full list of commands accepted by me can be found at here.
You can get sig-info at here.
You can self-configure the PR merge rules for this repository. For more details, please refer to Here.
For more, you also can visit HICANN.


PR Approval Progress

⚠️ This PR does not yet meet the following requirements:lgtm (requires ≥ 2 person(s) per module)、approve (requires ≥ 1 person(s) per module)

Module Approval Details

module lgtm status approve status
repo-cann/pypto-gym ❌ (0/2)(You can also ask: DCGDDD, ym-HW, jason_yuan_ye, 冯思远, lim-jiunnbin) ❌ (0/1)(You can also ask: 孙文博, gaoxingwang, DCGDDD, pan-boyi, wangbing110)

💡 Tip:

  • Committer can comment /approve or /lgtm
  • Commenting /approve implies both code review (lgtm) and intent to merge (approve)

CLA Signature Pass

lizixuan1020, thanks for your pull request. All authors of the commits have signed the CLA. 👍

likedislike
CANN-robot
CANN-robot成员
6 天前 评论:

CLA Signature Pass

lizixuan1020, thanks for your pull request. All authors of the commits have signed the CLA. 👍

likedislike
lizixuanlizixuan
3 天前 update merge request[project id: 9743536, iid: 586, commit_id: f385d40c307ac16d8576a95bb3eecd27d79e0270] virtual merging success
lizixuanlizixuan
3 天前 推送  1 个提交:403530d0-fix(op-verify): Make static-shape case check explicit
lizixuanlizixuan
3 天前 update merge request[project id: 9743536, iid: 586, commit_id: acdf3b2de7284878bfc4271d6a6ede710b38dd7d] virtual merging success
CANN-robotCANN-robot成员
3 天前 删除了label:cann-cla/yes
CANN-robotCANN-robot成员
3 天前 添加了label:stat/needs-squash
CANN-robotCANN-robot成员
3 天前 添加了label:cann-cla/yes
lizixuanlizixuan
3 天前 update merge request[project id: 9743536, iid: 586, commit_id: 53bea6fd4a73f6893e9d8286944e3702bc0c76e2] virtual merging success
lizixuanlizixuan
3 天前 推送  1 个提交:f8f62e0b-fix(op-verify): Remove excluded probes from static-shape check
lizixuanlizixuan
3 天前 update merge request[project id: 9743536, iid: 586, commit_id: 3cd8afc7191773fac66702c617454708e46ef0e0] virtual merging success
CANN-robotCANN-robot成员
3 天前 删除了label:cann-cla/yes
CANN-robotCANN-robot成员
3 天前 添加了label:cann-cla/yes