| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
feat: add profiling interpolation phase1 Co-authored-by: zhenyu_zhang<a993040925@163.com> # message auto-generated for no-merge-commit merge: !262 merge phase1-profiling-interpolation-impl into master feat: add profiling interpolation phase1 Created-by: zhenyu_zhang Commit-by: zhenyu_zhang Merged-by: ascend-robot Description: Related Issue: #212 ## Summary This PR adds Phase1 profiling interpolation for TensorCast profiling data source. Scope of Phase1: - Compute interpolation for MatMul/GEMM-style kernels. MatMulV2 / MatMulV3 / MatMulCommon are conservative 1D-only in Phase1; other compute kernels keep multidim only where policy allows it. - Attention-special interpolation for FusedInferAttentionScore, including the existing sqrt_seq axis transform. - A narrow existing elementwise 1D fallback remains available for Add, but full elementwise support is not claimed in Phase1. - Runtime opt-out remains available through disable_profiling_interpolation. - GatherV2 / Index use generic_compute.axis: output_numel; output_numel means output tensor element count, while output tail shape stays in the regime key to avoid mixing different output structures. ## Runtime B2B On/Off Evidence GLM5 TensorCast profiling B2B was run with the valid v0.18 profiling DB: - DB: ATLAS_800_A3_752T_128G_DIE/vllm_ascend/vllm0.18.0_torch2.9.0_cann8.5 - On/off switch: default wrapper enabled vs disable_profiling_interpolation=True - Event counts are unchanged between enabled/disabled runs. - Non-finite event duration count: 0. | scenario | enabled latency | disabled latency | delta | INTERPOLATED delta | FALLBACK_ANALYTIC delta | |---|---:|---:|---:|---:|---:| | A3-32-prefill | 14.736785s | 14.030826s | +705.958ms (+5.0315%) | +4127 | -4127 | | A3-32-decode | 75.971ms | 77.881ms | -1.910ms (-2.4524%) | +931 | -931 | This B2B is used to validate runtime wiring, source distribution, and latency health under enabled/disabled modes. The accuracy evidence is the LOOCV section below. ## Phase1 LOOCV Kernel Highlights The table below reports leave-one-out recovery on the v0.18 profiling DB for Phase1-supported paths. Structural out-of-scope operators such as AddRmsNormBias are not mixed into the accuracy table. | kernel | path | success | median | p90 | max | >100% | methods | |---|---|---:|---:|---:|---:|---:|---| | FusedInferAttentionScore | attention_special | 102 | 0.00% | 0.00% | 98.76% | 0 | 1D sqrt=102 | | MatMulV2 | compute_matmul | 691 | 3.18% | 16.74% | 82.99% | 0 | 1D=691 | | MatMulV3 | compute_matmul | 137 | 2.66% | 14.98% | 28.33% | 0 | 1D=137 | | MatMulCommon | compute_matmul | 2 | 32.93% | 39.53% | 41.17% | 0 | 1D=2 | | QuantBatchMatmulV3 | compute_matmul | 950 | 2.29% | 14.06% | 98.15% | 0 | 1D=908, 2D=35, 3D=7 | | TransposeBatchMatMul | compute_matmul | 961 | 7.07% | 24.00% | 190.77% | 4 | 1D=842, 2D=115, 3D=4 | | RmsNorm | generic_compute_1d | 395 | 5.59% | 26.19% | 89.76% | 0 | 1D=395 | | SwiGlu | generic_compute_1d | 426 | 5.49% | 33.82% | 297.54% | 1 | 1D=426 | | SoftmaxV2 | generic_compute_1d | 415 | 4.50% | 14.86% | 88.57% | 0 | 1D=415 | | DynamicQuant | generic_compute_1d | 433 | 4.18% | 30.15% | 363.90% | 1 | 1D=433 | | GatherV2 | generic_compute_1d | 2009 | 8.25% | 28.42% | 79.59% | 0 | 1D=2009 | | Index | generic_compute_1d | 2012 | 4.28% | 26.33% | 77.05% | 0 | 1D=2012 | | Add | elementwise_1d | 447 | 4.96% | 24.63% | 306.19% | 2 | 1D=447 | FusedInferAttentionScore in this table is direct kernel-level LOOCV. Composite MLA/MLAPO is covered in this PR by routing/regime/details regressions; full composite end-to-end profiling ground-truth validation is follow-up. The latency-source grouping materially reduces severe long-tail cases: - QuantBatchMatmulV3: >100% count 7 -> 0, max error 3819.67% -> 98.15%. - SwiGlu: >100% count 6 -> 1. - DynamicQuant: >100% count 6 -> 1. ## Remaining Long-Tail Explanation The remaining rel_error > 100% points are understood and do not change the merge decision for Phase1. ### TransposeBatchMatMul Remaining count: 4. All four remaining points are 1D. They are not caused by 2D/3D interpolation. The issue is sparse local profiling coverage on the M axis. Example boundaries include: - target M=4096, boundary 512 -> 32768 - target M=4096, boundary 384 -> 8192 - target M=2048, boundary 192 -> 32768 The target is interpolated across a wide M-axis interval, while the right boundary latency is much higher than local measured behavior. This causes over-estimation. Follow-up data issue: issue 174. This is a data densification follow-up and is not required for this PR to merge. ### SwiGlu / DynamicQuant Remaining count: 2 total. - SwiGlu: one local non-monotonic boundary remains. Boundary latency is 14.40us -> 3.76us, while target actual is 3.16us. - DynamicQuant: one local low-latency target remains. Fallback-column boundary points are both much higher than the held-out point. These look like local dirty-data / hidden-regime / sparse-neighborhood cases rather than interpolation implementation bugs. Follow-up data issue: issue 175. This is a data cleanup / densification follow-up and is not required for this PR to merge. ### Add Remaining count: 2. Add is an elementwise operator. The PR keeps a narrow elementwise 1D fallback because the overall result is useful, but full elementwise interpolation is not part of the Phase1 core scope. The two long-tail points are same-axis-coordinate cases, which suggests hidden regime differences such as broadcast/layout/stride/copy pattern. This is a Phase2/3 elementwise regime problem, not a blocker for Phase1 compute / attention interpolation. ## Additional Review Fixes This update keeps the Phase1 scope unchanged and closes the latest review gaps: - ProfilingDataSource exact lookup is intentionally unchanged in this PR; wrapper interpolation keeps its own finite-positive latency scanning and fallback handling. - Attention interpolation keeps wrapper-side layout guards for Runtime input_layout; base exact sparse/layout inference is intentionally unchanged in this PR. - Attention interpolation now refuses layout-less targets when the CSV carries Runtime input_layout, instead of silently allowing cross-layout matching. - Attention batch-axis handling is group-local: batchless candidates are not dropped just because the same regime also contains batched points; all-batched groups require a single batch value. - sqrt_seq diagnostics now distinguish transformed target coordinates from original pre-transform sequence length. - generic_compute.axis: output_numel now keeps (1, H) and (N, H) in the same tail-shape regime, and alternate kernels can inherit the primary kernel's generic-compute policy when they do not define their own. - Elementwise 1D unknown signatures no longer include the token axis when it is already the interpolation axis. - lookup() no longer mutates a shared QueryResult when it needs to add default interpolation shape-match info. - Attention target construction now records attention_sparse_mode_unknown as miss details and returns analytic fallback instead of raising when sparse mode cannot be inferred. - generic_compute.axis: output_numel now supports only single-output kernels in Phase1; multi-output cases return the specific miss reason generic_compute_output_numel_multi_output_unsupported instead of using only the first output, and scalar output () is treated as one element. - The Phase1 RFC wording now matches current behavior: repo-tracked M1/M3/M5 coverage is synthetic aggregation coverage, real DB on/off evidence remains PR-local evidence, and failed wrapper retry from PARTIAL returns None for analytic fallback. ## Verification Coverage | area | command | result | |---|---|---:| | Focused interpolation regression | python -m pytest tests/regression/tensor_cast/test_profiling_interpolation_phase1.py tests/regression/tensor_cast/test_interpolating_data_source.py -q | 124 passed | | Phase1 regression pack | python -m pytest tests/regression/tensor_cast/test_profiling_interpolation_phase1.py tests/regression/tensor_cast/test_interpolating_data_source.py tests/regression/tensor_cast/test_interpolation_math.py -q | 135 passed | | Benchmark perf DB regression | python -m pytest tests/benchmark/ops/perf_database -q | 442 passed, 4 subtests passed | | Final sparse-mode regression | python -m pytest tests/regression/tensor_cast/test_profiling_interpolation_phase1.py tests/regression/tensor_cast/test_interpolating_data_source.py tests/regression/tensor_cast/test_interpolation_math.py tests/benchmark/ops/perf_database/test_fia_enriched_lookup.py -q | 185 passed | | Changed-file checks | git diff --check and pre-commit run --files <changed files> | passed | Local ci_gate was also attempted with the repo-local test map. The bash path is missing pydantic; the Windows Python path progressed further but timed out after falling back to a stale-map full-suite run. No project code or script failure was observed, and this PR does not modify CI scripts. Additional reviewer-requested runtime validation: All four requested profiling-mode text_generate scenarios completed successfully before this update and are kept as B2B evidence for default-on behavior. Each scenario was run before/after with --disable-profiling-interpolation vs default profiling interpolation enabled. The exported empirical metrics cover M1-M5, and compute_m6.py completed on the exported traces for M6. | workload | mode | M1 raw op hit | M2 fused op hit | M3 fused no-zero-cost hit | M4 shape hit | M5 latency coverage | M6 Empirical/Real | |---|---|---:|---:|---:|---:|---:|---:| | Qwen3-32B | prefill, 4112 tokens | 96.43% -> 98.21% (+1.79pp) | 92.00% -> 96.00% (+4.00pp) | 81.82% -> 90.91% (+9.09pp) | 89.47% -> 94.74% (+5.26pp) | 99.38% -> 99.76% (+0.38pp) | 0.905 -> 0.908 (+0.003) | | Qwen3-32B | decode, 16 queries, ctx 4096 | 96.15% -> 96.15% (+0.00pp) | 92.31% -> 92.31% (+0.00pp) | 83.33% -> 83.33% (+0.00pp) | 88.24% -> 88.24% (+0.00pp) | 98.77% -> 98.77% (+0.00pp) | 0.840 -> 0.840 (+0.000) | | DeepSeek-V3 | prefill, 4099 tokens | 96.97% -> 96.97% (+0.00pp) | 94.29% -> 94.29% (+0.00pp) | 88.89% -> 88.89% (+0.00pp) | 93.33% -> 93.33% (+0.00pp) | 98.69% -> See merge request: Ascend/msmodeling!262 | 5 天前 | |
【同步】【非开发代码】代码从 develop 同步到 master Co-authored-by: yydyzr<liuyuncong1@huawei.com> Co-authored-by: gcw_61YBRfIt<chuzhenxing@huawei.com> Co-authored-by: 孔炳翔<1120200577@qq.com> Co-authored-by: zhengxinqian<qianzhengxin@huawei.com> Co-authored-by: hw_whx<wanghexiang7@huawei.com> Co-authored-by: jgong5<steven.gong@gmail.com> Co-authored-by: hw_whx<2952154980@qq.com> # message auto-generated for no-merge-commit merge: !330 merge master into master 【同步】【非开发代码】代码从 develop 同步到 master Created-by: AvadaKedavrua Commit-by: liujiawang;ascend-robot;AvadaKedavrua;lutean;Horacehxw;eveyin1;minghang_c;zwt__;tt0cool;elrond-g;jia_ya_nan;zhenyu_zhang;ChenHuiwen;wangshen001;Hudingyi;wendellX;Secluded_Ocean;jhon-117;yaohan404;jiangruitao;zhenghaojie;stormchasingg;panyj1993;cmh1056291129;yuyinkai1;sunguozhong;genius52;liu_jiaxu;HongMaoShuiGuai;zhengxinqian;weixin_43368449;jsez-li-bin;jgong5;wqh17101;w00609794;yydyzr;JieZhang679;sppedforcy;gcw_61YBRfIt;Jiong Gong;hw_whx;gongjiong;孔炳翔 Merged-by: ascend-robot Description: 代码从 develop 同步到 master，后续基于 master 演进，并支持打包 See merge request: Ascend/msmodeling!330 | 30 天前 |
| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
| 5 天前 | ||
| 30 天前 |