| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
feat: add profiling interpolation phase1 Co-authored-by: zhenyu_zhang<a993040925@163.com> # message auto-generated for no-merge-commit merge: !262 merge phase1-profiling-interpolation-impl into master feat: add profiling interpolation phase1 Created-by: zhenyu_zhang Commit-by: zhenyu_zhang Merged-by: ascend-robot Description: Related Issue: #212 ## Summary This PR adds Phase1 profiling interpolation for TensorCast profiling data source. Scope of Phase1: - Compute interpolation for MatMul/GEMM-style kernels. MatMulV2 / MatMulV3 / MatMulCommon are conservative 1D-only in Phase1; other compute kernels keep multidim only where policy allows it. - Attention-special interpolation for FusedInferAttentionScore, including the existing sqrt_seq axis transform. - A narrow existing elementwise 1D fallback remains available for Add, but full elementwise support is not claimed in Phase1. - Runtime opt-out remains available through disable_profiling_interpolation. - GatherV2 / Index use generic_compute.axis: output_numel; output_numel means output tensor element count, while output tail shape stays in the regime key to avoid mixing different output structures. ## Runtime B2B On/Off Evidence GLM5 TensorCast profiling B2B was run with the valid v0.18 profiling DB: - DB: ATLAS_800_A3_752T_128G_DIE/vllm_ascend/vllm0.18.0_torch2.9.0_cann8.5 - On/off switch: default wrapper enabled vs disable_profiling_interpolation=True - Event counts are unchanged between enabled/disabled runs. - Non-finite event duration count: 0. | scenario | enabled latency | disabled latency | delta | INTERPOLATED delta | FALLBACK_ANALYTIC delta | |---|---:|---:|---:|---:|---:| | A3-32-prefill | 14.736785s | 14.030826s | +705.958ms (+5.0315%) | +4127 | -4127 | | A3-32-decode | 75.971ms | 77.881ms | -1.910ms (-2.4524%) | +931 | -931 | This B2B is used to validate runtime wiring, source distribution, and latency health under enabled/disabled modes. The accuracy evidence is the LOOCV section below. ## Phase1 LOOCV Kernel Highlights The table below reports leave-one-out recovery on the v0.18 profiling DB for Phase1-supported paths. Structural out-of-scope operators such as AddRmsNormBias are not mixed into the accuracy table. | kernel | path | success | median | p90 | max | >100% | methods | |---|---|---:|---:|---:|---:|---:|---| | FusedInferAttentionScore | attention_special | 102 | 0.00% | 0.00% | 98.76% | 0 | 1D sqrt=102 | | MatMulV2 | compute_matmul | 691 | 3.18% | 16.74% | 82.99% | 0 | 1D=691 | | MatMulV3 | compute_matmul | 137 | 2.66% | 14.98% | 28.33% | 0 | 1D=137 | | MatMulCommon | compute_matmul | 2 | 32.93% | 39.53% | 41.17% | 0 | 1D=2 | | QuantBatchMatmulV3 | compute_matmul | 950 | 2.29% | 14.06% | 98.15% | 0 | 1D=908, 2D=35, 3D=7 | | TransposeBatchMatMul | compute_matmul | 961 | 7.07% | 24.00% | 190.77% | 4 | 1D=842, 2D=115, 3D=4 | | RmsNorm | generic_compute_1d | 395 | 5.59% | 26.19% | 89.76% | 0 | 1D=395 | | SwiGlu | generic_compute_1d | 426 | 5.49% | 33.82% | 297.54% | 1 | 1D=426 | | SoftmaxV2 | generic_compute_1d | 415 | 4.50% | 14.86% | 88.57% | 0 | 1D=415 | | DynamicQuant | generic_compute_1d | 433 | 4.18% | 30.15% | 363.90% | 1 | 1D=433 | | GatherV2 | generic_compute_1d | 2009 | 8.25% | 28.42% | 79.59% | 0 | 1D=2009 | | Index | generic_compute_1d | 2012 | 4.28% | 26.33% | 77.05% | 0 | 1D=2012 | | Add | elementwise_1d | 447 | 4.96% | 24.63% | 306.19% | 2 | 1D=447 | FusedInferAttentionScore in this table is direct kernel-level LOOCV. Composite MLA/MLAPO is covered in this PR by routing/regime/details regressions; full composite end-to-end profiling ground-truth validation is follow-up. The latency-source grouping materially reduces severe long-tail cases: - QuantBatchMatmulV3: >100% count 7 -> 0, max error 3819.67% -> 98.15%. - SwiGlu: >100% count 6 -> 1. - DynamicQuant: >100% count 6 -> 1. ## Remaining Long-Tail Explanation The remaining rel_error > 100% points are understood and do not change the merge decision for Phase1. ### TransposeBatchMatMul Remaining count: 4. All four remaining points are 1D. They are not caused by 2D/3D interpolation. The issue is sparse local profiling coverage on the M axis. Example boundaries include: - target M=4096, boundary 512 -> 32768 - target M=4096, boundary 384 -> 8192 - target M=2048, boundary 192 -> 32768 The target is interpolated across a wide M-axis interval, while the right boundary latency is much higher than local measured behavior. This causes over-estimation. Follow-up data issue: issue 174. This is a data densification follow-up and is not required for this PR to merge. ### SwiGlu / DynamicQuant Remaining count: 2 total. - SwiGlu: one local non-monotonic boundary remains. Boundary latency is 14.40us -> 3.76us, while target actual is 3.16us. - DynamicQuant: one local low-latency target remains. Fallback-column boundary points are both much higher than the held-out point. These look like local dirty-data / hidden-regime / sparse-neighborhood cases rather than interpolation implementation bugs. Follow-up data issue: issue 175. This is a data cleanup / densification follow-up and is not required for this PR to merge. ### Add Remaining count: 2. Add is an elementwise operator. The PR keeps a narrow elementwise 1D fallback because the overall result is useful, but full elementwise interpolation is not part of the Phase1 core scope. The two long-tail points are same-axis-coordinate cases, which suggests hidden regime differences such as broadcast/layout/stride/copy pattern. This is a Phase2/3 elementwise regime problem, not a blocker for Phase1 compute / attention interpolation. ## Additional Review Fixes This update keeps the Phase1 scope unchanged and closes the latest review gaps: - ProfilingDataSource exact lookup is intentionally unchanged in this PR; wrapper interpolation keeps its own finite-positive latency scanning and fallback handling. - Attention interpolation keeps wrapper-side layout guards for Runtime input_layout; base exact sparse/layout inference is intentionally unchanged in this PR. - Attention interpolation now refuses layout-less targets when the CSV carries Runtime input_layout, instead of silently allowing cross-layout matching. - Attention batch-axis handling is group-local: batchless candidates are not dropped just because the same regime also contains batched points; all-batched groups require a single batch value. - sqrt_seq diagnostics now distinguish transformed target coordinates from original pre-transform sequence length. - generic_compute.axis: output_numel now keeps (1, H) and (N, H) in the same tail-shape regime, and alternate kernels can inherit the primary kernel's generic-compute policy when they do not define their own. - Elementwise 1D unknown signatures no longer include the token axis when it is already the interpolation axis. - lookup() no longer mutates a shared QueryResult when it needs to add default interpolation shape-match info. - Attention target construction now records attention_sparse_mode_unknown as miss details and returns analytic fallback instead of raising when sparse mode cannot be inferred. - generic_compute.axis: output_numel now supports only single-output kernels in Phase1; multi-output cases return the specific miss reason generic_compute_output_numel_multi_output_unsupported instead of using only the first output, and scalar output () is treated as one element. - The Phase1 RFC wording now matches current behavior: repo-tracked M1/M3/M5 coverage is synthetic aggregation coverage, real DB on/off evidence remains PR-local evidence, and failed wrapper retry from PARTIAL returns None for analytic fallback. ## Verification Coverage | area | command | result | |---|---|---:| | Focused interpolation regression | python -m pytest tests/regression/tensor_cast/test_profiling_interpolation_phase1.py tests/regression/tensor_cast/test_interpolating_data_source.py -q | 124 passed | | Phase1 regression pack | python -m pytest tests/regression/tensor_cast/test_profiling_interpolation_phase1.py tests/regression/tensor_cast/test_interpolating_data_source.py tests/regression/tensor_cast/test_interpolation_math.py -q | 135 passed | | Benchmark perf DB regression | python -m pytest tests/benchmark/ops/perf_database -q | 442 passed, 4 subtests passed | | Final sparse-mode regression | python -m pytest tests/regression/tensor_cast/test_profiling_interpolation_phase1.py tests/regression/tensor_cast/test_interpolating_data_source.py tests/regression/tensor_cast/test_interpolation_math.py tests/benchmark/ops/perf_database/test_fia_enriched_lookup.py -q | 185 passed | | Changed-file checks | git diff --check and pre-commit run --files <changed files> | passed | Local ci_gate was also attempted with the repo-local test map. The bash path is missing pydantic; the Windows Python path progressed further but timed out after falling back to a stale-map full-suite run. No project code or script failure was observed, and this PR does not modify CI scripts. Additional reviewer-requested runtime validation: All four requested profiling-mode text_generate scenarios completed successfully before this update and are kept as B2B evidence for default-on behavior. Each scenario was run before/after with --disable-profiling-interpolation vs default profiling interpolation enabled. The exported empirical metrics cover M1-M5, and compute_m6.py completed on the exported traces for M6. | workload | mode | M1 raw op hit | M2 fused op hit | M3 fused no-zero-cost hit | M4 shape hit | M5 latency coverage | M6 Empirical/Real | |---|---|---:|---:|---:|---:|---:|---:| | Qwen3-32B | prefill, 4112 tokens | 96.43% -> 98.21% (+1.79pp) | 92.00% -> 96.00% (+4.00pp) | 81.82% -> 90.91% (+9.09pp) | 89.47% -> 94.74% (+5.26pp) | 99.38% -> 99.76% (+0.38pp) | 0.905 -> 0.908 (+0.003) | | Qwen3-32B | decode, 16 queries, ctx 4096 | 96.15% -> 96.15% (+0.00pp) | 92.31% -> 92.31% (+0.00pp) | 83.33% -> 83.33% (+0.00pp) | 88.24% -> 88.24% (+0.00pp) | 98.77% -> 98.77% (+0.00pp) | 0.840 -> 0.840 (+0.000) | | DeepSeek-V3 | prefill, 4099 tokens | 96.97% -> 96.97% (+0.00pp) | 94.29% -> 94.29% (+0.00pp) | 88.89% -> 88.89% (+0.00pp) | 93.33% -> 93.33% (+0.00pp) | 98.69% -> See merge request: Ascend/msmodeling!262 | 4 天前 | |
【同步】【非开发代码】代码从 develop 同步到 master Co-authored-by: yydyzr<liuyuncong1@huawei.com> Co-authored-by: gcw_61YBRfIt<chuzhenxing@huawei.com> Co-authored-by: 孔炳翔<1120200577@qq.com> Co-authored-by: zhengxinqian<qianzhengxin@huawei.com> Co-authored-by: hw_whx<wanghexiang7@huawei.com> Co-authored-by: jgong5<steven.gong@gmail.com> Co-authored-by: hw_whx<2952154980@qq.com> # message auto-generated for no-merge-commit merge: !330 merge master into master 【同步】【非开发代码】代码从 develop 同步到 master Created-by: AvadaKedavrua Commit-by: liujiawang;ascend-robot;AvadaKedavrua;lutean;Horacehxw;eveyin1;minghang_c;zwt__;tt0cool;elrond-g;jia_ya_nan;zhenyu_zhang;ChenHuiwen;wangshen001;Hudingyi;wendellX;Secluded_Ocean;jhon-117;yaohan404;jiangruitao;zhenghaojie;stormchasingg;panyj1993;cmh1056291129;yuyinkai1;sunguozhong;genius52;liu_jiaxu;HongMaoShuiGuai;zhengxinqian;weixin_43368449;jsez-li-bin;jgong5;wqh17101;w00609794;yydyzr;JieZhang679;sppedforcy;gcw_61YBRfIt;Jiong Gong;hw_whx;gongjiong;孔炳翔 Merged-by: ascend-robot Description: 代码从 develop 同步到 master，后续基于 master 演进，并支持打包 See merge request: Ascend/msmodeling!330 | 30 天前 | |
【同步】【非开发代码】代码从 develop 同步到 master Co-authored-by: yydyzr<liuyuncong1@huawei.com> Co-authored-by: gcw_61YBRfIt<chuzhenxing@huawei.com> Co-authored-by: 孔炳翔<1120200577@qq.com> Co-authored-by: zhengxinqian<qianzhengxin@huawei.com> Co-authored-by: hw_whx<wanghexiang7@huawei.com> Co-authored-by: jgong5<steven.gong@gmail.com> Co-authored-by: hw_whx<2952154980@qq.com> # message auto-generated for no-merge-commit merge: !330 merge master into master 【同步】【非开发代码】代码从 develop 同步到 master Created-by: AvadaKedavrua Commit-by: liujiawang;ascend-robot;AvadaKedavrua;lutean;Horacehxw;eveyin1;minghang_c;zwt__;tt0cool;elrond-g;jia_ya_nan;zhenyu_zhang;ChenHuiwen;wangshen001;Hudingyi;wendellX;Secluded_Ocean;jhon-117;yaohan404;jiangruitao;zhenghaojie;stormchasingg;panyj1993;cmh1056291129;yuyinkai1;sunguozhong;genius52;liu_jiaxu;HongMaoShuiGuai;zhengxinqian;weixin_43368449;jsez-li-bin;jgong5;wqh17101;w00609794;yydyzr;JieZhang679;sppedforcy;gcw_61YBRfIt;Jiong Gong;hw_whx;gongjiong;孔炳翔 Merged-by: ascend-robot Description: 代码从 develop 同步到 master，后续基于 master 演进，并支持打包 See merge request: Ascend/msmodeling!330 | 30 天前 | |
【同步】【非开发代码】代码从 develop 同步到 master Co-authored-by: yydyzr<liuyuncong1@huawei.com> Co-authored-by: gcw_61YBRfIt<chuzhenxing@huawei.com> Co-authored-by: 孔炳翔<1120200577@qq.com> Co-authored-by: zhengxinqian<qianzhengxin@huawei.com> Co-authored-by: hw_whx<wanghexiang7@huawei.com> Co-authored-by: jgong5<steven.gong@gmail.com> Co-authored-by: hw_whx<2952154980@qq.com> # message auto-generated for no-merge-commit merge: !330 merge master into master 【同步】【非开发代码】代码从 develop 同步到 master Created-by: AvadaKedavrua Commit-by: liujiawang;ascend-robot;AvadaKedavrua;lutean;Horacehxw;eveyin1;minghang_c;zwt__;tt0cool;elrond-g;jia_ya_nan;zhenyu_zhang;ChenHuiwen;wangshen001;Hudingyi;wendellX;Secluded_Ocean;jhon-117;yaohan404;jiangruitao;zhenghaojie;stormchasingg;panyj1993;cmh1056291129;yuyinkai1;sunguozhong;genius52;liu_jiaxu;HongMaoShuiGuai;zhengxinqian;weixin_43368449;jsez-li-bin;jgong5;wqh17101;w00609794;yydyzr;JieZhang679;sppedforcy;gcw_61YBRfIt;Jiong Gong;hw_whx;gongjiong;孔炳翔 Merged-by: ascend-robot Description: 代码从 develop 同步到 master，后续基于 master 演进，并支持打包 See merge request: Ascend/msmodeling!330 | 30 天前 | |
【同步】【非开发代码】代码从 develop 同步到 master Co-authored-by: yydyzr<liuyuncong1@huawei.com> Co-authored-by: gcw_61YBRfIt<chuzhenxing@huawei.com> Co-authored-by: 孔炳翔<1120200577@qq.com> Co-authored-by: zhengxinqian<qianzhengxin@huawei.com> Co-authored-by: hw_whx<wanghexiang7@huawei.com> Co-authored-by: jgong5<steven.gong@gmail.com> Co-authored-by: hw_whx<2952154980@qq.com> # message auto-generated for no-merge-commit merge: !330 merge master into master 【同步】【非开发代码】代码从 develop 同步到 master Created-by: AvadaKedavrua Commit-by: liujiawang;ascend-robot;AvadaKedavrua;lutean;Horacehxw;eveyin1;minghang_c;zwt__;tt0cool;elrond-g;jia_ya_nan;zhenyu_zhang;ChenHuiwen;wangshen001;Hudingyi;wendellX;Secluded_Ocean;jhon-117;yaohan404;jiangruitao;zhenghaojie;stormchasingg;panyj1993;cmh1056291129;yuyinkai1;sunguozhong;genius52;liu_jiaxu;HongMaoShuiGuai;zhengxinqian;weixin_43368449;jsez-li-bin;jgong5;wqh17101;w00609794;yydyzr;JieZhang679;sppedforcy;gcw_61YBRfIt;Jiong Gong;hw_whx;gongjiong;孔炳翔 Merged-by: ascend-robot Description: 代码从 develop 同步到 master，后续基于 master 演进，并支持打包 See merge request: Ascend/msmodeling!330 | 30 天前 | |
【同步】【非开发代码】代码从 develop 同步到 master Co-authored-by: yydyzr<liuyuncong1@huawei.com> Co-authored-by: gcw_61YBRfIt<chuzhenxing@huawei.com> Co-authored-by: 孔炳翔<1120200577@qq.com> Co-authored-by: zhengxinqian<qianzhengxin@huawei.com> Co-authored-by: hw_whx<wanghexiang7@huawei.com> Co-authored-by: jgong5<steven.gong@gmail.com> Co-authored-by: hw_whx<2952154980@qq.com> # message auto-generated for no-merge-commit merge: !330 merge master into master 【同步】【非开发代码】代码从 develop 同步到 master Created-by: AvadaKedavrua Commit-by: liujiawang;ascend-robot;AvadaKedavrua;lutean;Horacehxw;eveyin1;minghang_c;zwt__;tt0cool;elrond-g;jia_ya_nan;zhenyu_zhang;ChenHuiwen;wangshen001;Hudingyi;wendellX;Secluded_Ocean;jhon-117;yaohan404;jiangruitao;zhenghaojie;stormchasingg;panyj1993;cmh1056291129;yuyinkai1;sunguozhong;genius52;liu_jiaxu;HongMaoShuiGuai;zhengxinqian;weixin_43368449;jsez-li-bin;jgong5;wqh17101;w00609794;yydyzr;JieZhang679;sppedforcy;gcw_61YBRfIt;Jiong Gong;hw_whx;gongjiong;孔炳翔 Merged-by: ascend-robot Description: 代码从 develop 同步到 master，后续基于 master 演进，并支持打包 See merge request: Ascend/msmodeling!330 | 30 天前 | |
【同步】【非开发代码】代码从 develop 同步到 master Co-authored-by: yydyzr<liuyuncong1@huawei.com> Co-authored-by: gcw_61YBRfIt<chuzhenxing@huawei.com> Co-authored-by: 孔炳翔<1120200577@qq.com> Co-authored-by: zhengxinqian<qianzhengxin@huawei.com> Co-authored-by: hw_whx<wanghexiang7@huawei.com> Co-authored-by: jgong5<steven.gong@gmail.com> Co-authored-by: hw_whx<2952154980@qq.com> # message auto-generated for no-merge-commit merge: !330 merge master into master 【同步】【非开发代码】代码从 develop 同步到 master Created-by: AvadaKedavrua Commit-by: liujiawang;ascend-robot;AvadaKedavrua;lutean;Horacehxw;eveyin1;minghang_c;zwt__;tt0cool;elrond-g;jia_ya_nan;zhenyu_zhang;ChenHuiwen;wangshen001;Hudingyi;wendellX;Secluded_Ocean;jhon-117;yaohan404;jiangruitao;zhenghaojie;stormchasingg;panyj1993;cmh1056291129;yuyinkai1;sunguozhong;genius52;liu_jiaxu;HongMaoShuiGuai;zhengxinqian;weixin_43368449;jsez-li-bin;jgong5;wqh17101;w00609794;yydyzr;JieZhang679;sppedforcy;gcw_61YBRfIt;Jiong Gong;hw_whx;gongjiong;孔炳翔 Merged-by: ascend-robot Description: 代码从 develop 同步到 master，后续基于 master 演进，并支持打包 See merge request: Ascend/msmodeling!330 | 30 天前 | |
【同步】【非开发代码】代码从 develop 同步到 master Co-authored-by: yydyzr<liuyuncong1@huawei.com> Co-authored-by: gcw_61YBRfIt<chuzhenxing@huawei.com> Co-authored-by: 孔炳翔<1120200577@qq.com> Co-authored-by: zhengxinqian<qianzhengxin@huawei.com> Co-authored-by: hw_whx<wanghexiang7@huawei.com> Co-authored-by: jgong5<steven.gong@gmail.com> Co-authored-by: hw_whx<2952154980@qq.com> # message auto-generated for no-merge-commit merge: !330 merge master into master 【同步】【非开发代码】代码从 develop 同步到 master Created-by: AvadaKedavrua Commit-by: liujiawang;ascend-robot;AvadaKedavrua;lutean;Horacehxw;eveyin1;minghang_c;zwt__;tt0cool;elrond-g;jia_ya_nan;zhenyu_zhang;ChenHuiwen;wangshen001;Hudingyi;wendellX;Secluded_Ocean;jhon-117;yaohan404;jiangruitao;zhenghaojie;stormchasingg;panyj1993;cmh1056291129;yuyinkai1;sunguozhong;genius52;liu_jiaxu;HongMaoShuiGuai;zhengxinqian;weixin_43368449;jsez-li-bin;jgong5;wqh17101;w00609794;yydyzr;JieZhang679;sppedforcy;gcw_61YBRfIt;Jiong Gong;hw_whx;gongjiong;孔炳翔 Merged-by: ascend-robot Description: 代码从 develop 同步到 master，后续基于 master 演进，并支持打包 See merge request: Ascend/msmodeling!330 | 30 天前 | |
【同步】【非开发代码】代码从 develop 同步到 master Co-authored-by: yydyzr<liuyuncong1@huawei.com> Co-authored-by: gcw_61YBRfIt<chuzhenxing@huawei.com> Co-authored-by: 孔炳翔<1120200577@qq.com> Co-authored-by: zhengxinqian<qianzhengxin@huawei.com> Co-authored-by: hw_whx<wanghexiang7@huawei.com> Co-authored-by: jgong5<steven.gong@gmail.com> Co-authored-by: hw_whx<2952154980@qq.com> # message auto-generated for no-merge-commit merge: !330 merge master into master 【同步】【非开发代码】代码从 develop 同步到 master Created-by: AvadaKedavrua Commit-by: liujiawang;ascend-robot;AvadaKedavrua;lutean;Horacehxw;eveyin1;minghang_c;zwt__;tt0cool;elrond-g;jia_ya_nan;zhenyu_zhang;ChenHuiwen;wangshen001;Hudingyi;wendellX;Secluded_Ocean;jhon-117;yaohan404;jiangruitao;zhenghaojie;stormchasingg;panyj1993;cmh1056291129;yuyinkai1;sunguozhong;genius52;liu_jiaxu;HongMaoShuiGuai;zhengxinqian;weixin_43368449;jsez-li-bin;jgong5;wqh17101;w00609794;yydyzr;JieZhang679;sppedforcy;gcw_61YBRfIt;Jiong Gong;hw_whx;gongjiong;孔炳翔 Merged-by: ascend-robot Description: 代码从 develop 同步到 master，后续基于 master 演进，并支持打包 See merge request: Ascend/msmodeling!330 | 30 天前 | |
feat(tensor_cast): add mla_sparse_attention op for DSA path (#128) Co-authored-by: Hudingyi<305949481@qq.com> # message auto-generated for no-merge-commit merge: !363 merge feat/mla-sparse-attention-op into master feat(tensor_cast): add mla_sparse_attention op for DSA path (#128) Created-by: Hudingyi Commit-by: Hudingyi Merged-by: ascend-robot Description: 原始问题 TensorCast 对 DeepSeek-V3.2 和 GLM-5.1 的性能仿真结果不准确。这两个模型在 NPU 上走 AscendSFABackend，使用 npu_sparse_flash_attention（SFA）kernel，但profiling database 查询时因找不到对应条目而全量 miss，退化到 analytic fallback，导致仿真数据失真。 --- 根因 TensorCast 中 DeepseekSparseAttention（服务 V3.2 / GLM-5.1）继承自 MultiheadLatentAttentionTensorCast，其 forward() 硬编码调用torch.ops.tensor_cast.multihead_latent_attention——与 dense MLA（V3 / V3.1 / Qwen3）共用同一个 TC op。 profiling database 的 COMPOSITE_DECOMPOSERS 中，multihead_latent_attention 固定 decompose 到 FusedInferAttentionScore（FIA），没有SparseFlashAttention（SFA）路径。 vllm-ascend 实际的分叉逻辑是：hf_config 含 index_topk 属性 → AscendSFABackend → SFA；不含 → AscendMLABackend → FIA。TensorCast 侧没有对应的 op区分，所以 SFA 模型的查询路径根本不存在。 --- 修改方案 最小改动原则：不改 dense 路径的任何行为，新增 MLA的sparse 算子。 1. 新增两个 TC op（tensor_cast/ops/mla.py） 注册 mla_sparse_attention 和 mla_sparse_attention_quant，签名与 dense 版完全相同（分别对应 BF16 和量化场景），语义等价，仅 op 名不同。 2. 引入 _get_attention_op 钩子（tensor_cast/layers/mla.py） 父类 MultiheadLatentAttentionTensorCast 新增钩子，默认返回 dense op；DeepseekSparseAttention override 返回 sparse op。forward()改用钩子替代硬编码，dense 路径（V3 / V3.1 / Qwen3）完全不受影响。 3. decomposer 支持 SFA（profiling_data_source.py） _decompose_mla_common 加 attention_kernel_type 参数（默认 "FusedInferAttentionScore"），现有调用零修改。新增 _decompose_mla_sparse /_decompose_mla_sparse_quant，传入 "SparseFlashAttention"，注册到 COMPOSITE_DECOMPOSERS。 4. 注册 analytic properties（performance_model/__init__.py） 两个 sparse op 复用 _multihead_latent_attention_properties_helper，analytic 数值与 dense 版一致。 5. op_mapping.yaml × 4 四个版本目录（vllm 0.13 / 0.15 / 0.18 / 0.18-shape-generated）的 section 7 MLA 末尾追加两个条目，composite: true, decomposer: true。 --- 验证结果 ## 测试结果 **环境**:ATLAS_800_A3_752T_128G_DIE / vllm0.18.0_torch2.9.0_cann8.5,DeepSeek-V3.2, decode seq=3500,INT8 attention。Profiling DB 为 master 现有数据(SFA.csv 经 Git-LFS, SHA256 add617f8…,23 行,已校验与 master 一致)。 > GLM5 完整 forward 受本地 transformers modeling_glm_moe_dsa.py 库 bug 阻断(与本 PR 无关), > 其映射用「模型构建 + 模块 dispatch」动态验证。 ### 项1:DSA 模型正确映射到 mla_sparse_attention ✓ | 模型 | 判定 | 结果 | |---|---|---| | DeepSeek-V3.2 (deepseek_v32, index_topk=2048) | forward 后 runtime 表 | mla_sparse_attention_quant.default ✓ + dsa_indexer.default ✓ | | GLM5 (glm_moe_dsa, index_topk=2048, 78层, W8A8) | 构建出的 attention 模块 | 3× Glm5SparseAttention;_get_attention_op() → mla_sparse_attention[_quant] ✓ | - 继承链:Glm5SparseAttention → DeepseekSparseAttention → MultiheadLatentAttentionTensorCast。 - 单测 **26** 项(SFA/映射)+ DSA 回归 **91** 项全绿。 ### 项2:Analytical 模型性能/Memory 保持不变 ✓(离线子集) | 项 | 数据 | |---|---| | DSV3.2 analytic 总时延 | **2.354121 s**(forward 干净,无报错) | | 离线回归 (v32/v4/mla) | **91 passed, 0 failed** | | 本 PR 对 analytic 路径的改动 | **无**(profiling_data_source.py 未被 merge 改动;dense 模型不进 SFA 路径) | ### 项3:DSA 模型 Empirical 匹配率 / 准确性(量化) DSV3.2 经真实 profiling DB + InterpolatingDataSource,共 **151** 个算子: **按命中来源(算子数 / 占比 / 加权解析时延)** | 来源 | 算子数 | 占比 | 解析时延占比 | |---|---:|---:|---:| | MEASURED(精确命中) | 77 | 51.0% | 7.5% (1841.6 µs) | | INTERPOLATED(插值) | 24 | 15.9% | 35.1% (8652.8 µs) | | ANALYTIC-FALLBACK(miss) | 50 | 33.1% | 57.4% (14774.5 µs) | | **命中率(MEAS+INTERP)** | **101** | **66.9%(按数量)** | **41.5%(按时延)** | M1-M6指标： | 配置 | M1 | M2 | M3 | M4 | M5 | M6 | | Qwen3 PF | 55/56 (98.2%) | 24/25 (96.0%) | 10/11 (90.9%) | 18/19 (94.7%) | 99.8% | 90.8% (153.3/168.9ms) | | Qwen3 PF | 55/56 (98.2%) | 24/25 (96.0%) | 10/11 (90.9%) | 18/19 (94.7%) | 99.8% | 90.8% (153.3/168.9ms) | | Qwen3 DC | 50/52 (96.2%) | 24/26 (92.3%) | 10/12 (83.3%) | 15/17 (88.2%) | 98.8% | 84.0% (13.0/15.4ms) | | DSv3 PF | 64/66 (97.0%) | 33/35 (94.3%) | 16/18 (88.9%) | 28/30 (93.3%) | 98.7% | 69.8% (321.5/460.8ms) | | DSv3 DC | 64/66 (97.0%) | 33/35 (94.3%) | 16/18 (88.9%) | 28/30 (93.3%) | 99.6% | 83.7% (33.0/39.4ms) | 与master当前对比是一致，本PR不会下降： | 配置 | 指标 | 纯 master | master+本PR | 一致 | |---|---|---|---|---| | DSv3 PF | M1/M2/M3/M4/M5/M6 | 64/66·33/35·16/18·28/30·98.7%·0.698 | 64/66·33/35·16/18·28/30·98.7%·0.698 | ✅ | | Qwen3 DC | M1/M2/M3/M4/M5/M6 | 50/52·24/26·10/12·15/17·98.8%·0.840 | 50/52·24/26·10/12·15/17·98.8%·0.840 | ✅ | See merge request: Ascend/msmodeling!363 | 2 天前 | |
【同步】【非开发代码】代码从 develop 同步到 master Co-authored-by: yydyzr<liuyuncong1@huawei.com> Co-authored-by: gcw_61YBRfIt<chuzhenxing@huawei.com> Co-authored-by: 孔炳翔<1120200577@qq.com> Co-authored-by: zhengxinqian<qianzhengxin@huawei.com> Co-authored-by: hw_whx<wanghexiang7@huawei.com> Co-authored-by: jgong5<steven.gong@gmail.com> Co-authored-by: hw_whx<2952154980@qq.com> # message auto-generated for no-merge-commit merge: !330 merge master into master 【同步】【非开发代码】代码从 develop 同步到 master Created-by: AvadaKedavrua Commit-by: liujiawang;ascend-robot;AvadaKedavrua;lutean;Horacehxw;eveyin1;minghang_c;zwt__;tt0cool;elrond-g;jia_ya_nan;zhenyu_zhang;ChenHuiwen;wangshen001;Hudingyi;wendellX;Secluded_Ocean;jhon-117;yaohan404;jiangruitao;zhenghaojie;stormchasingg;panyj1993;cmh1056291129;yuyinkai1;sunguozhong;genius52;liu_jiaxu;HongMaoShuiGuai;zhengxinqian;weixin_43368449;jsez-li-bin;jgong5;wqh17101;w00609794;yydyzr;JieZhang679;sppedforcy;gcw_61YBRfIt;Jiong Gong;hw_whx;gongjiong;孔炳翔 Merged-by: ascend-robot Description: 代码从 develop 同步到 master，后续基于 master 演进，并支持打包 See merge request: Ascend/msmodeling!330 | 30 天前 | |
【同步】【非开发代码】代码从 develop 同步到 master Co-authored-by: yydyzr<liuyuncong1@huawei.com> Co-authored-by: gcw_61YBRfIt<chuzhenxing@huawei.com> Co-authored-by: 孔炳翔<1120200577@qq.com> Co-authored-by: zhengxinqian<qianzhengxin@huawei.com> Co-authored-by: hw_whx<wanghexiang7@huawei.com> Co-authored-by: jgong5<steven.gong@gmail.com> Co-authored-by: hw_whx<2952154980@qq.com> # message auto-generated for no-merge-commit merge: !330 merge master into master 【同步】【非开发代码】代码从 develop 同步到 master Created-by: AvadaKedavrua Commit-by: liujiawang;ascend-robot;AvadaKedavrua;lutean;Horacehxw;eveyin1;minghang_c;zwt__;tt0cool;elrond-g;jia_ya_nan;zhenyu_zhang;ChenHuiwen;wangshen001;Hudingyi;wendellX;Secluded_Ocean;jhon-117;yaohan404;jiangruitao;zhenghaojie;stormchasingg;panyj1993;cmh1056291129;yuyinkai1;sunguozhong;genius52;liu_jiaxu;HongMaoShuiGuai;zhengxinqian;weixin_43368449;jsez-li-bin;jgong5;wqh17101;w00609794;yydyzr;JieZhang679;sppedforcy;gcw_61YBRfIt;Jiong Gong;hw_whx;gongjiong;孔炳翔 Merged-by: ascend-robot Description: 代码从 develop 同步到 master，后续基于 master 演进，并支持打包 See merge request: Ascend/msmodeling!330 | 30 天前 | |
【同步】【非开发代码】代码从 develop 同步到 master Co-authored-by: yydyzr<liuyuncong1@huawei.com> Co-authored-by: gcw_61YBRfIt<chuzhenxing@huawei.com> Co-authored-by: 孔炳翔<1120200577@qq.com> Co-authored-by: zhengxinqian<qianzhengxin@huawei.com> Co-authored-by: hw_whx<wanghexiang7@huawei.com> Co-authored-by: jgong5<steven.gong@gmail.com> Co-authored-by: hw_whx<2952154980@qq.com> # message auto-generated for no-merge-commit merge: !330 merge master into master 【同步】【非开发代码】代码从 develop 同步到 master Created-by: AvadaKedavrua Commit-by: liujiawang;ascend-robot;AvadaKedavrua;lutean;Horacehxw;eveyin1;minghang_c;zwt__;tt0cool;elrond-g;jia_ya_nan;zhenyu_zhang;ChenHuiwen;wangshen001;Hudingyi;wendellX;Secluded_Ocean;jhon-117;yaohan404;jiangruitao;zhenghaojie;stormchasingg;panyj1993;cmh1056291129;yuyinkai1;sunguozhong;genius52;liu_jiaxu;HongMaoShuiGuai;zhengxinqian;weixin_43368449;jsez-li-bin;jgong5;wqh17101;w00609794;yydyzr;JieZhang679;sppedforcy;gcw_61YBRfIt;Jiong Gong;hw_whx;gongjiong;孔炳翔 Merged-by: ascend-robot Description: 代码从 develop 同步到 master，后续基于 master 演进，并支持打包 See merge request: Ascend/msmodeling!330 | 30 天前 | |
fix: align shape grid model ids and database Co-authored-by: Secluded_Ocean<tangchuxiao0709@qq.com> # message auto-generated for no-merge-commit merge: !348 merge codex/fix-shape-grid-profile-db into master fix: align shape grid model ids and database Created-by: Secluded_Ocean Commit-by: Secluded_Ocean Merged-by: ascend-robot Description: ## Summary - align generate_shape_grid.py --target-models with text_generate model_id naming and reject legacy short names such as dsv3 - keep --rows effective when sampling is capped but rng/seed is None - replace vllm0.18.0_torch2.9.0_cann8.5 with the shape_generated database after validating it has more effective data ## Validation - Database comparison before replacement: old=68 CSV / 823 valid shape rows / 823 positive metric rows; shape_generated=104 CSV / 36198 valid shape rows / 15008 positive metric rows - Final database path: 104 CSV / 36198 shape rows / 15008 positive metric rows; shape_generated path removed - python -m py_compile tools/perf_data_collection/generate_shape_grid.py tools/perf_data_collection/grid_generator/model_configs.py tools/perf_data_collection/grid_generator/theory_router.py tools/perf_data_collection/grid_generator/generators/fused_attention.py - pytest tests/regression/cli/test_shape_grid_model_configs.py tests/regression/cli/test_model_configs.py tests/regression/cli/test_runner.py tests/regression/cli/test_theory_router_pure.py tests/regression/cli/test_generate_shape_grid.py -q See merge request: Ascend/msmodeling!348 | 18 天前 | |
feat: add profiling interpolation phase1 Co-authored-by: zhenyu_zhang<a993040925@163.com> # message auto-generated for no-merge-commit merge: !262 merge phase1-profiling-interpolation-impl into master feat: add profiling interpolation phase1 Created-by: zhenyu_zhang Commit-by: zhenyu_zhang Merged-by: ascend-robot Description: Related Issue: #212 ## Summary This PR adds Phase1 profiling interpolation for TensorCast profiling data source. Scope of Phase1: - Compute interpolation for MatMul/GEMM-style kernels. MatMulV2 / MatMulV3 / MatMulCommon are conservative 1D-only in Phase1; other compute kernels keep multidim only where policy allows it. - Attention-special interpolation for FusedInferAttentionScore, including the existing sqrt_seq axis transform. - A narrow existing elementwise 1D fallback remains available for Add, but full elementwise support is not claimed in Phase1. - Runtime opt-out remains available through disable_profiling_interpolation. - GatherV2 / Index use generic_compute.axis: output_numel; output_numel means output tensor element count, while output tail shape stays in the regime key to avoid mixing different output structures. ## Runtime B2B On/Off Evidence GLM5 TensorCast profiling B2B was run with the valid v0.18 profiling DB: - DB: ATLAS_800_A3_752T_128G_DIE/vllm_ascend/vllm0.18.0_torch2.9.0_cann8.5 - On/off switch: default wrapper enabled vs disable_profiling_interpolation=True - Event counts are unchanged between enabled/disabled runs. - Non-finite event duration count: 0. | scenario | enabled latency | disabled latency | delta | INTERPOLATED delta | FALLBACK_ANALYTIC delta | |---|---:|---:|---:|---:|---:| | A3-32-prefill | 14.736785s | 14.030826s | +705.958ms (+5.0315%) | +4127 | -4127 | | A3-32-decode | 75.971ms | 77.881ms | -1.910ms (-2.4524%) | +931 | -931 | This B2B is used to validate runtime wiring, source distribution, and latency health under enabled/disabled modes. The accuracy evidence is the LOOCV section below. ## Phase1 LOOCV Kernel Highlights The table below reports leave-one-out recovery on the v0.18 profiling DB for Phase1-supported paths. Structural out-of-scope operators such as AddRmsNormBias are not mixed into the accuracy table. | kernel | path | success | median | p90 | max | >100% | methods | |---|---|---:|---:|---:|---:|---:|---| | FusedInferAttentionScore | attention_special | 102 | 0.00% | 0.00% | 98.76% | 0 | 1D sqrt=102 | | MatMulV2 | compute_matmul | 691 | 3.18% | 16.74% | 82.99% | 0 | 1D=691 | | MatMulV3 | compute_matmul | 137 | 2.66% | 14.98% | 28.33% | 0 | 1D=137 | | MatMulCommon | compute_matmul | 2 | 32.93% | 39.53% | 41.17% | 0 | 1D=2 | | QuantBatchMatmulV3 | compute_matmul | 950 | 2.29% | 14.06% | 98.15% | 0 | 1D=908, 2D=35, 3D=7 | | TransposeBatchMatMul | compute_matmul | 961 | 7.07% | 24.00% | 190.77% | 4 | 1D=842, 2D=115, 3D=4 | | RmsNorm | generic_compute_1d | 395 | 5.59% | 26.19% | 89.76% | 0 | 1D=395 | | SwiGlu | generic_compute_1d | 426 | 5.49% | 33.82% | 297.54% | 1 | 1D=426 | | SoftmaxV2 | generic_compute_1d | 415 | 4.50% | 14.86% | 88.57% | 0 | 1D=415 | | DynamicQuant | generic_compute_1d | 433 | 4.18% | 30.15% | 363.90% | 1 | 1D=433 | | GatherV2 | generic_compute_1d | 2009 | 8.25% | 28.42% | 79.59% | 0 | 1D=2009 | | Index | generic_compute_1d | 2012 | 4.28% | 26.33% | 77.05% | 0 | 1D=2012 | | Add | elementwise_1d | 447 | 4.96% | 24.63% | 306.19% | 2 | 1D=447 | FusedInferAttentionScore in this table is direct kernel-level LOOCV. Composite MLA/MLAPO is covered in this PR by routing/regime/details regressions; full composite end-to-end profiling ground-truth validation is follow-up. The latency-source grouping materially reduces severe long-tail cases: - QuantBatchMatmulV3: >100% count 7 -> 0, max error 3819.67% -> 98.15%. - SwiGlu: >100% count 6 -> 1. - DynamicQuant: >100% count 6 -> 1. ## Remaining Long-Tail Explanation The remaining rel_error > 100% points are understood and do not change the merge decision for Phase1. ### TransposeBatchMatMul Remaining count: 4. All four remaining points are 1D. They are not caused by 2D/3D interpolation. The issue is sparse local profiling coverage on the M axis. Example boundaries include: - target M=4096, boundary 512 -> 32768 - target M=4096, boundary 384 -> 8192 - target M=2048, boundary 192 -> 32768 The target is interpolated across a wide M-axis interval, while the right boundary latency is much higher than local measured behavior. This causes over-estimation. Follow-up data issue: issue 174. This is a data densification follow-up and is not required for this PR to merge. ### SwiGlu / DynamicQuant Remaining count: 2 total. - SwiGlu: one local non-monotonic boundary remains. Boundary latency is 14.40us -> 3.76us, while target actual is 3.16us. - DynamicQuant: one local low-latency target remains. Fallback-column boundary points are both much higher than the held-out point. These look like local dirty-data / hidden-regime / sparse-neighborhood cases rather than interpolation implementation bugs. Follow-up data issue: issue 175. This is a data cleanup / densification follow-up and is not required for this PR to merge. ### Add Remaining count: 2. Add is an elementwise operator. The PR keeps a narrow elementwise 1D fallback because the overall result is useful, but full elementwise interpolation is not part of the Phase1 core scope. The two long-tail points are same-axis-coordinate cases, which suggests hidden regime differences such as broadcast/layout/stride/copy pattern. This is a Phase2/3 elementwise regime problem, not a blocker for Phase1 compute / attention interpolation. ## Additional Review Fixes This update keeps the Phase1 scope unchanged and closes the latest review gaps: - ProfilingDataSource exact lookup is intentionally unchanged in this PR; wrapper interpolation keeps its own finite-positive latency scanning and fallback handling. - Attention interpolation keeps wrapper-side layout guards for Runtime input_layout; base exact sparse/layout inference is intentionally unchanged in this PR. - Attention interpolation now refuses layout-less targets when the CSV carries Runtime input_layout, instead of silently allowing cross-layout matching. - Attention batch-axis handling is group-local: batchless candidates are not dropped just because the same regime also contains batched points; all-batched groups require a single batch value. - sqrt_seq diagnostics now distinguish transformed target coordinates from original pre-transform sequence length. - generic_compute.axis: output_numel now keeps (1, H) and (N, H) in the same tail-shape regime, and alternate kernels can inherit the primary kernel's generic-compute policy when they do not define their own. - Elementwise 1D unknown signatures no longer include the token axis when it is already the interpolation axis. - lookup() no longer mutates a shared QueryResult when it needs to add default interpolation shape-match info. - Attention target construction now records attention_sparse_mode_unknown as miss details and returns analytic fallback instead of raising when sparse mode cannot be inferred. - generic_compute.axis: output_numel now supports only single-output kernels in Phase1; multi-output cases return the specific miss reason generic_compute_output_numel_multi_output_unsupported instead of using only the first output, and scalar output () is treated as one element. - The Phase1 RFC wording now matches current behavior: repo-tracked M1/M3/M5 coverage is synthetic aggregation coverage, real DB on/off evidence remains PR-local evidence, and failed wrapper retry from PARTIAL returns None for analytic fallback. ## Verification Coverage | area | command | result | |---|---|---:| | Focused interpolation regression | python -m pytest tests/regression/tensor_cast/test_profiling_interpolation_phase1.py tests/regression/tensor_cast/test_interpolating_data_source.py -q | 124 passed | | Phase1 regression pack | python -m pytest tests/regression/tensor_cast/test_profiling_interpolation_phase1.py tests/regression/tensor_cast/test_interpolating_data_source.py tests/regression/tensor_cast/test_interpolation_math.py -q | 135 passed | | Benchmark perf DB regression | python -m pytest tests/benchmark/ops/perf_database -q | 442 passed, 4 subtests passed | | Final sparse-mode regression | python -m pytest tests/regression/tensor_cast/test_profiling_interpolation_phase1.py tests/regression/tensor_cast/test_interpolating_data_source.py tests/regression/tensor_cast/test_interpolation_math.py tests/benchmark/ops/perf_database/test_fia_enriched_lookup.py -q | 185 passed | | Changed-file checks | git diff --check and pre-commit run --files <changed files> | passed | Local ci_gate was also attempted with the repo-local test map. The bash path is missing pydantic; the Windows Python path progressed further but timed out after falling back to a stale-map full-suite run. No project code or script failure was observed, and this PR does not modify CI scripts. Additional reviewer-requested runtime validation: All four requested profiling-mode text_generate scenarios completed successfully before this update and are kept as B2B evidence for default-on behavior. Each scenario was run before/after with --disable-profiling-interpolation vs default profiling interpolation enabled. The exported empirical metrics cover M1-M5, and compute_m6.py completed on the exported traces for M6. | workload | mode | M1 raw op hit | M2 fused op hit | M3 fused no-zero-cost hit | M4 shape hit | M5 latency coverage | M6 Empirical/Real | |---|---|---:|---:|---:|---:|---:|---:| | Qwen3-32B | prefill, 4112 tokens | 96.43% -> 98.21% (+1.79pp) | 92.00% -> 96.00% (+4.00pp) | 81.82% -> 90.91% (+9.09pp) | 89.47% -> 94.74% (+5.26pp) | 99.38% -> 99.76% (+0.38pp) | 0.905 -> 0.908 (+0.003) | | Qwen3-32B | decode, 16 queries, ctx 4096 | 96.15% -> 96.15% (+0.00pp) | 92.31% -> 92.31% (+0.00pp) | 83.33% -> 83.33% (+0.00pp) | 88.24% -> 88.24% (+0.00pp) | 98.77% -> 98.77% (+0.00pp) | 0.840 -> 0.840 (+0.000) | | DeepSeek-V3 | prefill, 4099 tokens | 96.97% -> 96.97% (+0.00pp) | 94.29% -> 94.29% (+0.00pp) | 88.89% -> 88.89% (+0.00pp) | 93.33% -> 93.33% (+0.00pp) | 98.69% -> See merge request: Ascend/msmodeling!262 | 4 天前 | |
【同步】【非开发代码】代码从 develop 同步到 master Co-authored-by: yydyzr<liuyuncong1@huawei.com> Co-authored-by: gcw_61YBRfIt<chuzhenxing@huawei.com> Co-authored-by: 孔炳翔<1120200577@qq.com> Co-authored-by: zhengxinqian<qianzhengxin@huawei.com> Co-authored-by: hw_whx<wanghexiang7@huawei.com> Co-authored-by: jgong5<steven.gong@gmail.com> Co-authored-by: hw_whx<2952154980@qq.com> # message auto-generated for no-merge-commit merge: !330 merge master into master 【同步】【非开发代码】代码从 develop 同步到 master Created-by: AvadaKedavrua Commit-by: liujiawang;ascend-robot;AvadaKedavrua;lutean;Horacehxw;eveyin1;minghang_c;zwt__;tt0cool;elrond-g;jia_ya_nan;zhenyu_zhang;ChenHuiwen;wangshen001;Hudingyi;wendellX;Secluded_Ocean;jhon-117;yaohan404;jiangruitao;zhenghaojie;stormchasingg;panyj1993;cmh1056291129;yuyinkai1;sunguozhong;genius52;liu_jiaxu;HongMaoShuiGuai;zhengxinqian;weixin_43368449;jsez-li-bin;jgong5;wqh17101;w00609794;yydyzr;JieZhang679;sppedforcy;gcw_61YBRfIt;Jiong Gong;hw_whx;gongjiong;孔炳翔 Merged-by: ascend-robot Description: 代码从 develop 同步到 master，后续基于 master 演进，并支持打包 See merge request: Ascend/msmodeling!330 | 30 天前 |
| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
| 4 天前 | ||
| 30 天前 | ||
| 30 天前 | ||
| 30 天前 | ||
| 30 天前 | ||
| 30 天前 | ||
| 30 天前 | ||
| 30 天前 | ||
| 30 天前 | ||
| 2 天前 | ||
| 30 天前 | ||
| 30 天前 | ||
| 30 天前 | ||
| 18 天前 | ||
| 4 天前 | ||
| 30 天前 |