草稿
[WIP] Stage-5 knowledge base, pl-native engine skills, and cann-bench targeting for pypto-pro-op-orchestrator #585
[WIP] Stage-5 knowledge base, pl-native engine skills, and cann-bench targeting for pypto-pro-op-orchestrator #585
草稿
hid22900163创建于 3 天前
hid22900163成员
3 天前

Was This PR authored or co-authored using generative AI tooling?

[ ] No
[x] Yes
__1. AI Agent : Claude Code (agentic CLI), driving this repo's own pypto-pro-op-orchestrator plugin
__2. AI Model : Claude Opus 5
__3. Prompt : No single prompt. Per-run prompts were generated by this repo's write-pypto-pro-prompt tooling from the operator SPEC plus live box facts (gym SHA, harness evaluator.py md5, card smoke, case anchors), then executed stage by stage by the orchestrator's own subagents. The knowledge-base pages in this PR are the write-back from those runs: each page was drafted from a measurement made on hardware, then reviewed and edited by hand before commit. Every load-bearing number in these pages is traceable to a run artifact, not to model recall.

What this PR does / why we need it:

Adds a Stage-5 (performance-tuning) knowledge base and the routing needed to reach it, plus the pl-native engine skills the tuning loop dispatches to. 26 commits, 42 files, +7128/−50 against 064be1d5. The bulk (25 files) is cannbot-skills/ops/pypto-pro-op-kb; the rest wires it into the orchestrator, pypto-pro-op-perf-tune, and the three pypto-pl-* engine skills.

The organising idea is an explicit objective precedence, because these three quantities had been conflated and several commits exist only to separate them:

  1. Acceptance — the SPEC's perf_target, falling back to per-P0-case golden_reference_ratio >= 1.0.
  2. In-loop selectionaverage_speedup = mean(baseline_i / candidate_i).
  3. The cann-bench score — target selection and interpretation only, never the acceptance caliber.

Why we need it: without a Stage-5 caliber the tuning loop has no stopping rule, and a measured improvement cannot be distinguished from noise. This branch supplies both, and the runs below are the evidence that it works and that it correctly refuses to ship when it should.

Which issue this PR fixes

(optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #

Special notes for your reviewer:

Measured Stage-5 contribution. Graded on the public cann-bench 950PR task set, jobs finished 2026-08-19 → 2026-08-24. Both signs shown; the negatives and the no-ships are the honest part.

run operator gym commit Stage 4 Stage 5 Δ
s13rn rms_norm not recorded (note 1) 74.1672 76.4570 +2.2898
s5o5 rms_norm f196bd49 74.1672 † 76.1690 +2.0018
s5o3 rms_norm f196bd49 74.1672 † 74.1520 −0.0152
s13sig sigmoid not recorded (note 1) 80.5713 80.5360 −0.0353
s13sig, hidden 80-case set sigmoid not recorded 92.2102 91.7891 −0.4211
sig5o1 sigmoid 0a0f8984 (note 2) shipped the Stage-4 snapshot 0.0000
conv2d conv_2d f196bd49 50.8488 shipped the Stage-4 snapshot 0.0000
s5o1 rms_norm ran on a different box seed predates the window 75.2177 not pairable in-window

† s5o5 and s5o3 were both seeded from s13rn's Stage-4 artifact, so they share its graded 74.1672 as baseline. They are not independent Stage-4 measurements and their deltas must not be summed with s13rn's.

The strongest claim available, stated precisely: s13rn and s5o5 are two independent Stage-5 runs from the same Stage-4 seed, landing +2.2898 and +2.0018 — agreeing to within 0.29, the campaign's measured across-session noise floor. A reproduced gain, not one lucky result.

The two zeros are the design working. In sig5o1 and conv2d, Stage 5 measured its candidate levers, could not clear the acceptance bar, and shipped the frozen Stage-4 snapshot under the "no gain → ship the snapshot" gate, preserving the tuned variants and recording the falsification. conv2d rejected a lever whose graded movement was +0.2111 — a real, band-confined, accuracy-neutral effect that sits below the measured within-session paired floor of 0.291 and is therefore not shippable evidence.

The negatives are small. s5o3 (−0.0152) and s13sig (−0.0353) are inside the noise floor and read as "no effect". The hidden-set sigmoid figure (−0.4211) is the one number that exceeds a floor, and it is not explained.

Provenance: Ascend 950PR, box a5-238, physical card 3 or 4 per run; CANN 9.2.0; pypto env snapshot pypto-master-20260730; scoring with --perf-metric-strategy kernel_details.

Note 1 — two runs do not record their gym commit. s13rn and s13sig (2026-08-19) carry no gym SHA in their workspace env.sh or setup script. The deployed tree of that vintage was gym-single-3cf7e0fb, and 3cf7e0fb is not an ancestor of this branch — it belongs to the pre-rebase lineage, before the rebase onto post-#444 master 064be1d5. That rebase was verified content-preserving on 39 of 41 files at the time, but those two runs were not executed against the commits in this PR, and the table says so rather than implying otherwise.

Note 2 — sig5o1 ran mid-branch. 0a0f8984 is commit 24 of the 26 here, two before the tip. This branch's content, just not its final state.

Note 3 — the pypto version string cannot identify the build. Our environment reports pypto 0.2.1; so does a materially different evaluation-fleet build whose pl.jit signature differs (compile_timeout rather than timeout, auto_mutex defaulting to True). The version string is not a usable handle, so we cite the env snapshot name and CANN version. For the record, our build has since been identified as master 86f508520 (2026-07-30).

What we would most like review on:

  1. The objective-precedence model — is golden_reference_ratio the right default acceptance bar when SPEC.perf_target is null, and is "cann-bench score is targeting-only" the rule you want enforced?
  2. The five accept gates, added after a diagnosed sigmoid regression slipped through: pinned instrument, cache-flush symmetry, inadmissible-case exclusion, measuring on the trigger band with its share stated, and the ship-the-snapshot rule.
  3. KB_USAGE.json — this branch requires a refresh of it in Stage 5, mapping selected reference → derived invariant → implementation location → implementation claim. We think it is the highest-leverage gate in the flow; we would like it endorsed or replaced.
  4. Whether 25 new kb pages is acceptable given selection is uncapped, or whether you would prefer a non-routed tier.

Known weak points, so you don't have to find them:

  • Two of eight runs in the table did not execute against commits in this PR (note 1).
  • The hidden-set sigmoid delta (−0.4211) exceeds the noise floor and is unexplained.
  • conv2d's Stage 5 shipped nothing, and its Stage-4 gate still carries one blocker (precision_failure, 18/20). That operator's ceiling is bounded by a framework capability, not by this branch.
  • No CI runs the kb's own check_kb_integrity.py; it must be invoked by hand.

Change Type / 改动类型

Testing / 测试信息

This branch adds knowledge-base pages and skill routing; it ships no library code, so the repo's UT and ST suites do not cover it. Verification was end-to-end on hardware, by running the pipeline these pages drive and grading the resulting kernels on the public cann-bench task set — the table in "Special notes" is that result, reported with both signs.

  • check_kb_integrity.py (the kb's own 16 checks, incl. every-page-routable, reachability, filename rules, link and magnitude-home checks): run by hand, passing. Note there is no CI invoking it.
  • 8 pipeline runs graded on Ascend 950PR, 2026-08-19 → 08-24; strongest result is a Stage-5 gain reproduced across two independent runs from one seed, +2.2898 and +2.0018, agreeing to within the 0.29 noise floor.

[ ] UT passed / 单元测试通过 — not applicable; no library code changed
[ ] ST passed / 集成测试通过 — not applicable
[x] Manual verified / 人工验证通过

Checklist / 检查清单

likedislike
合并受阻
Hhid22900163成员
3 天前 创建了 pull request,commit 9c9b247f
Hhid22900163成员
3 天前 修改了pull request 的描述
Hhid22900163成员
3 天前 修改了pull request 的描述
Hhid22900163成员
3 天前 修改了pull request 的描述
Hhid22900163成员
3 天前 修改了pull request 的描述
此处折叠了50条消息 查看更多
Hhid22900163成员
4 小时前 update merge request[project id: 9743536, iid: 585, commit_id: 12bed1c92f5c17427d800e108125b3adbee4abc4] virtual merging success
Hhid22900163成员
4 小时前 强制推送  12 个提交:68f667f2-9 commits from branch master6d5741d7-kb: stage-5 campaign knowledge, shape routing and integrity checks,e90e3d8e-skills: measurement discipline, lever ordering and candidate-ledger rules for stage 5,0654ebd0-orchestrator: stage gates hardened against false implementation records
Hhid22900163成员
4 小时前 update merge request[project id: 9743536, iid: 585, commit_id: 1b8be1ddec3e8b42bbbb3ff45e8493669fe5a375] virtual merging success
CANN-robotCANN-robot成员
4 小时前 删除了label:ai-co-authoredai-includes-codeai-compliance-failed
CANN-robot
CANN-robot成员
4 小时前 评论:

Notification

This pull request has been changed(code update) or closed, so removes the following label(s): ai-co-authored, ai-compliance-failed, ai-includes-code.

likedislike