Closed
添加moe_init_routing_v4和moe_re_routing_v2算子的st&标杆 #4543
cpy_123456Add on  Aug 24Closed at  Aug 25
cpy_123456Collaborators
Aug 24 Create

MoE 系列算子 ST 用例描述与执行报告

生成时间:2026-08-24
测试环境:Ascend950 (arch35), CANN 9.1.0, TTK 3.0.0
测试模式:ttk kernel -b release(Binary Release 模式)


一、moe_init_routing_v4(25 个用例,全部 PASS)

1.1 场景覆盖矩阵

维度 覆盖值 用例数
输入数据类型 (x) int8 2
float16 7
bfloat16 8
float32 8
quant_mode -1(非量化,DropLess) 13
1(动态量化) 12
expert_tokens_num_type (etnt) 1(直方图统计) 13
2(key-value 对) 12
drop_pad_mode 0(DropLess 模式) 25
expert_tokens_num_flag True 25
row_idx_type 0(gather) 7
1(scatter) 18
scale 输入 有 10
无 15
topk_weight 输入 有 25
expert_num 范围 256 ~ 2930 —
active_expert_range 跨度 6 ~ 580 —

1.2 用例分组描述

第 1 组:标准场景(small shape, k 较小)

用例编号:[0][2][4][6][8][10][12][16][18][20][22][24]

  • token 数 2~35,k=3~85,expert_num=256~1024
  • 覆盖 4 种 dtype × 2 种 quant_mode × 2 种 etnt
  • 验证基础路由、topk_weight 展开、expert_tokens_count 计算的正确性

第 2 组:大 k 场景(fuzz, k=270~2930)

用例编号:[1][3][5][7][9][11][13][15][17][19][21][23]

  • token 数 1~109,k 高达 2930,expert_num 高达 2930
  • 覆盖大 expert 展开因子下的内存布局与输出 shape 正确性
  • 包含最大输出用例 [15](59296×6553)和 [3][17](69513×3719)

第 3 组:row_idx_type=0(gather 模式)

用例编号:[4][5][7][8][11][14][20][21][24]

  • expanded_row_idx 使用 gather 方式生成
  • 与 scatter 模式(row_idx_type=1)对比验证索引正确性

1.3 用例明细

# 用例名 dtype quant_mode etnt row_idx_type x shape expert_idx (k) expert_num active_range scale topk_weight 输出 shape perf (us)
0 moe_init_routing_v4_0029 bfloat16 -1 1 1 (34, 4) (34, 21) k=21 1024 [19, 356] ✗ ✓ (714,4),(714,),(337,),(714,),(714,1) 14.86
1 MoeInitRoutingV4_bfloat16_ND_fuzz_000046 bfloat16 -1 2 1 (40, 24) (40, 2753) k=2753 2753 [1731, 1734] ✓ ✓ (110120,24),...,(2753,2),... 155.70
2 moe_init_routing_v4_0002 bfloat16 1 1 1 (27, 14) (27, 4) k=4 256 [33, 39] ✓ ✓ (108,14),(108,),(6,),... 7.60
3 MoeInitRoutingV4_bfloat16_ND_fuzz_000011 bfloat16 1 2 1 (47, 3719) (47, 1479) k=1479 1479 [969, 1042] ✗ ✓ (69513,3719),... 141.56
4 moe_init_routing_v4_0039 float16 -1 1 0 (5, 3) (5, 38) k=38 1024 [3, 118] ✗ ✓ (190,3),... 9.93
5 MoeInitRoutingV4_float16_ND_fuzz_000014 float16 -1 2 0 (1, 192) (1, 1333) k=1333 1333 [597, 787] ✗ ✓ (1333,192),... 26.11
6 moe_init_routing_v4_0007 float16 1 1 1 (26, 35) (26, 30) k=30 256 [111, 208] ✓ ✓ (780,35),... 17.54
7 MoeInitRoutingV4_float16_ND_fuzz_000052 float16 1 2 0 (1, 10) (1, 1367) k=1367 1367 [98, 331] ✓ ✓ (1367,10),... 26.50
8 moe_init_routing_v4_0090 float32 -1 1 0 (35, 2) (35, 85) k=85 256 [102, 194] ✗ ✓ (2975,2),... 56.70
9 MoeInitRoutingV4_float32_ND_fuzz_000036 float32 -1 2 1 (1, 661) (1, 2695) k=2695 2695 [1878, 2114] ✓ ✓ (2695,661),... 56.41
10 moe_init_routing_v4_0036 float32 1 1 1 (10, 13) (10, 25) k=25 1024 [454, 865] ✓ ✓ (250,13),... 12.46
11 MoeInitRoutingV4_float32_ND_fuzz_000034 float32 1 2 0 (1, 192) (1, 270) k=270 270 [6, 167] ✗ ✓ (270,192),... 12.54
12 moe_init_routing_v4_0001 int8 -1 1 1 (21, 34) (21, 34) k=34 256 [82, 212] ✓ ✓ (714,34),... 19.62
13 MoeInitRoutingV4_int8_ND_fuzz_000030 int8 -1 2 1 (3, 1) (3, 1244) k=1244 1244 [967, 1038] ✓ ✓ (3732,1),... 64.33
14 moe_init_routing_v4_0136 bfloat16 -1 1 0 (36, 20) (36, 160) k=160 1024 [272, 852] ✗ ✓ (5760,20),... 45.37
15 MoeInitRoutingV4_bfloat16_ND_fuzz_000092 bfloat16 -1 2 1 (109, 6553) (109, 544) k=544 544 [284, 391] ✗ ✓ (59296,6553),... 157.09
16 moe_init_routing_v4_0011 bfloat16 1 1 1 (6, 26) (6, 34) k=34 256 [93, 201] ✗ ✓ (204,26),... 11.29
17 MoeInitRoutingV4_bfloat16_ND_fuzz_000011 bfloat16 1 2 1 (47, 3719) (47, 1479) k=1479 1479 [969, 1042] ✗ ✓ (69513,3719),... 142.74
18 moe_init_routing_v4_0126 float16 -1 1 1 (4, 25) (4, 364) k=364 1024 [99, 304] ✗ ✓ (1456,25),... 23.05
19 MoeInitRoutingV4_float16_ND_fuzz_000042 float16 -1 2 1 (1, 1) (1, 2930) k=2930 2930 [2596, 2791] ✓ ✓ (2930,1),... 54.21
20 moe_init_routing_v4_0017 float16 1 1 0 (27, 18) (27, 10) k=10 256 [38, 159] ✓ ✓ (270,18),... 12.59
21 MoeInitRoutingV4_float16_ND_fuzz_000086 float16 1 2 0 (1, 4001) (1, 2727) k=2727 2727 [1728, 1953] ✗ ✓ (2727,4001),... 28.82
22 moe_init_routing_v4_0364 float32 -1 1 1 (7, 458) (7, 3) k=3 1024 [36, 323] ✗ ✓ (21,458),... 11.24
23 MoeInitRoutingV4_float32_ND_fuzz_000050 float32 -1 2 1 (53, 118) (53, 962) k=962 962 [43, 49] ✗ ✓ (50986,118),... 69.02
24 moe_init_routing_v4_0332 float32 1 1 0 (2, 291) (2, 36) k=36 256 [33, 43] ✗ ✓ (72,291),... 7.47

1.4 执行时间统计

指标 值
25 用例总执行时间(wall-clock) 63.0 s
单用例执行时间(含框架开销) ~15 s
最快用例 AiCore 耗时 moe_init_routing_v4_0002 — 7.60 us
最慢用例 AiCore 耗时 MoeInitRoutingV4_bfloat16_ND_fuzz_000092 — 157.09 us

说明:wall-clock 时间包含 TTK 框架初始化、知识库加载、tiling 编译、NPU 执行、golden 计算与精度比对等全流程开销,AiCore 耗时仅为 NPU 上算子执行的 profiling 采集值。


二、moe_re_routing_v2(25 个用例,全部 PASS)

2.1 场景覆盖矩阵

维度 覆盖值 用例数
tokens 数据类型 int8 8
float16 8
bfloat16 9
idx_type 0(正向索引,permute_token_idx=src_offset) 12
1(反向索引,permute_token_idx=dst_offset) 13
per_token_scales 输入 有 12
无 13
expert_topk_weight 输入 有 25
rank_num(专家并行数) 1 17
2 3
3 1
4~5 2
7~52 2
expert_num(每 rank 专家数) 1 17
2 5
3 1
5~6 2
tokens 第二维 (hidden_size) 9 ~ 15267 —

2.2 用例分组描述

第 1 组:单 rank 单 expert(基础路由)

用例编号:[0][2][4][6][8][9][10][14][16][19][22][24]

  • rank_num=1, expert_num=1,最简单的 token 重排
  • 覆盖 3 种 dtype × 2 种 idx_type × 有/无 scales
  • 验证 permute_tokens、permute_topk_weight、permute_token_idx 的正确性

第 2 组:多 rank 多 expert(EP 场景)

用例编号:[5][11][12][15][17][20][21][23]

  • rank_num=2~19, expert_num=2~6
  • 验证跨 rank token 汇聚、expert_token_num 聚合、per_token_scales 重排

第 3 组:大 hidden_size(带宽压力)

用例编号:[0][3][12][13][14][16]

  • hidden_size 高达 4884~15267
  • 验证大 token 维度下的内存拷贝正确性与性能

第 4 组:idx_type=1(反向索引)

用例编号:[2][3][6][7][10][11][14][15][18][19][22][23][24]

  • permute_token_idx 写入 src 位置而非 dst 位置
  • 与 idx_type=0 对比验证索引语义正确性

2.3 用例明细

# 用例名 dtype idx_type tokens shape expert_token_num (rank×expert) scales topk_weight 输出 shape perf (us)
0 MoeReRoutingV2_bfloat16_ND_fuzz_000376_topkWeight bfloat16 0 (35, 15267) (35, 1) ✗ ✓ (35,15267),(35,2),(35,),(1,),(35,1) 6.29
1 MoeReRoutingV2_bfloat16_ND_fuzz_000053_topkWeight bfloat16 0 (23, 467) (23, 1) ✓ ✓ (23,467),(23,4),(23,),(1,),(23,1) 5.06
2 MoeReRoutingV2_bfloat16_ND_fuzz_000240_topkWeight bfloat16 1 (14, 46) (14, 1) ✗ ✓ (14,46),(14,2),(14,),(1,),(14,1) 4.05
3 MoeReRoutingV2_bfloat16_ND_fuzz_000085_topkWeight bfloat16 1 (20, 8109) (20, 1) ✓ ✓ (20,8109),(20,3),(20,),(1,),(20,1) 4.92
4 MoeReRoutingV2_float16_ND_fuzz_000284_topkWeight float16 0 (34, 93) (34, 1) ✗ ✓ (34,93),(34,3),(34,),(1,),(34,1) 6.27
5 MoeReRoutingV2_float16_ND_fuzz_000333_topkWeight float16 0 (8, 72) (4, 2) ✓ ✓ (8,72),(8,1),(8,),(2,),(8,1) 3.59
6 MoeReRoutingV2_float16_ND_fuzz_000332_topkWeight float16 1 (25, 17) (25, 1) ✗ ✓ (25,17),(25,2),(25,),(1,),(25,1) 5.29
7 MoeReRoutingV2_float16_ND_fuzz_000022_topkWeight float16 1 (11, 14) (11, 1) ✓ ✓ (11,14),(11,4),(11,),(1,),(11,1) 3.74
8 MoeReRoutingV2_int8_ND_fuzz_000092_topkWeight int8 0 (11, 80) (11, 1) ✗ ✓ (11,80),(11,4),(11,),(1,),(11,1) 3.79
9 MoeReRoutingV2_int8_ND_fuzz_000300_topkWeight int8 0 (11, 79) (11, 1) ✓ ✓ (11,79),(11,1),(11,),(1,),(11,1) 3.77
10 MoeReRoutingV2_int8_ND_fuzz_000179_topkWeight int8 1 (10, 70) (10, 1) ✗ ✓ (10,70),(10,3),(10,),(1,),(10,1) 3.79
11 MoeReRoutingV2_int8_ND_fuzz_000319_topkWeight int8 1 (9, 7168) (3, 3) ✓ ✓ (9,7168),(9,3),(9,),(3,),(9,1) 3.82
12 MoeReRoutingV2_bfloat16_ND_fuzz_000237_topkWeight bfloat16 0 (46, 12654) (23, 2) ✗ ✓ (46,12654),(46,2),(46,),(2,),(46,1) 6.65
13 MoeReRoutingV2_bfloat16_ND_fuzz_000322_topkWeight bfloat16 0 (28, 6057) (28, 1) ✓ ✓ (28,6057),(28,4),(28,),(1,),(28,1) 5.66
14 MoeReRoutingV2_bfloat16_ND_fuzz_000358_topkWeight bfloat16 1 (29, 6326) (29, 1) ✗ ✓ (29,6326),(29,752),(29,),(1,),(29,1) 5.66
15 MoeReRoutingV2_bfloat16_ND_fuzz_000359_topkWeight bfloat16 1 (30, 41) (5, 6) ✓ ✓ (30,41),(30,3),(30,),(6,),(30,1) 5.89
16 MoeReRoutingV2_float16_ND_fuzz_000119_topkWeight float16 0 (35, 4884) (35, 1) ✗ ✓ (35,4884),(35,3),(35,),(1,),(35,1) 6.21
17 MoeReRoutingV2_float16_ND_fuzz_000082_topkWeight float16 0 (21, 25) (7, 3) ✓ ✓ (21,25),(21,4),(21,),(3,),(21,1) 4.88
18 MoeReRoutingV2_float16_ND_fuzz_000145_topkWeight float16 1 (28, 769) (14, 2) ✗ ✓ (28,769),(28,757),(28,),(2,),(28,1) 5.58
19 MoeReRoutingV2_float16_ND_fuzz_000270_topkWeight float16 1 (15, 551) (15, 1) ✓ ✓ (15,551),(15,4),(15,),(1,),(15,1) 4.30
20 MoeReRoutingV2_int8_ND_fuzz_000192_topkWeight int8 0 (20, 70) (4, 5) ✗ ✓ (20,70),(20,2),(20,),(5,),(20,1) 4.91
21 MoeReRoutingV2_int8_ND_fuzz_000185_topkWeight int8 0 (22, 9) (11, 2) ✓ ✓ (22,9),(22,4),(22,),(2,),(22,1) 4.83
22 MoeReRoutingV2_int8_ND_fuzz_000265_topkWeight int8 1 (12, 69) (12, 1) ✗ ✓ (12,69),(12,4),(12,),(1,),(12,1) 3.90
23 MoeReRoutingV2_int8_ND_fuzz_000289_topkWeight int8 1 (38, 630) (19, 2) ✓ ✓ (38,630),(38,562),(38,),(2,),(38,1) 5.92
24 MoeReRoutingV2_bfloat16_ND_fuzz_000104_topkWeight bfloat16 0 (52, 21) (52, 1) ✗ ✓ (52,21),(52,3),(52,),(1,),(52,1) 6.63

2.4 执行时间统计

指标 值
25 用例总执行时间(wall-clock) 57.3 s
单用例执行时间(含框架开销) ~14.3 s
最快用例 AiCore 耗时 MoeReRoutingV2_int8_ND_fuzz_000300 — 3.77 us
最慢用例 AiCore 耗时 MoeReRoutingV2_bfloat16_ND_fuzz_000237 — 6.65 us

说明:wall-clock 时间包含 TTK 框架初始化、知识库加载、tiling 编译、NPU 执行、golden 计算与精度比对等全流程开销,AiCore 耗时仅为 NPU 上算子执行的 profiling 采集值。


三、总体汇总

算子 用例数 PASS FAIL 总执行时间 (wall-clock) 单用例含框架开销
moe_init_routing_v4 25 25 0 63.0 s ~15 s
moe_re_routing_v2 25 25 0 57.3 s ~14.3 s
合计 50 50 0 120.3 s —

所有用例均从源 CSV(/workspace/log/ttk/st/moe_init_routing_v4.csv、/workspace/log/ttk/st/moe_re_routing_v2.csv)中提取带 topk_weight 的用例,覆盖了 4 种数据类型(int8/float16/bfloat16/float32)、2 种量化模式、2 种索引模式、有/无 scale/scales 输入、不同 rank/expert 组合等关键场景。


四、测试命令参考

# 环境准备
source /home/developer/Ascend/cann/set_env.sh
export LD_LIBRARY_PATH=/home/developer/Ascend/cann-9.1.0/opp/vendors/custom_transformer/op_api/lib:${LD_LIBRARY_PATH}
cd /workspace/ops-test-kit

# moe_init_routing_v4 kernel 测试
python3 -m ttk kernel \
  -i /workspace/ops-transformer_3876/moe/moe_init_routing_v4/tests/st/test_moe_init_routing_v4.csv \
  -o /tmp/result_init_routing_v4.csv \
  -d false -b release --no-memory-check \
  --plugin /workspace/ops-transformer_3876/moe/moe_init_routing_v4/tests/assets/golden.py \
  --pc 1 --ti=0-24

# moe_re_routing_v2 kernel 测试
python3 -m ttk kernel \
  -i /workspace/ops-transformer_3876/moe/moe_re_routing_v2/tests/st/test_moe_re_routing_v2.csv \
  -o /tmp/result_re_routing_v2.csv \
  -d false -b release --no-memory-check \
  --plugin /workspace/ops-transformer_3876/moe/moe_re_routing_v2/tests/assets/golden.py \
  --pc 1 --ti=0-24

注意:不要设置 ASCEND_CUSTOM_OPP_PATH,否则会覆盖 vendor 路径导致 dynamic impl 模块加载失败。仅设置 LD_LIBRARY_PATH 即可。


五、测试文件清单

moe_init_routing_v4

文件 说明
tests/assets/golden.py TestSpec(含 MoeInitRoutingV4TestSpec kernel golden + AclnnMoeInitRoutingV4TestSpec aclnn golden)
tests/st/test_moe_init_routing_v4.csv 25 个 kernel ST 用例
tests/st/test_aclnn_moe_init_routing_v4.csv 25 个 ACLNN ST 用例

moe_re_routing_v2

文件 说明
tests/assets/golden.py TestSpec(含 MoeReRoutingV2TestSpec kernel golden)
tests/st/test_moe_re_routing_v2.csv 25 个 kernel ST 用例

moe_re_routing_v2 无 ACLNN C 接口实现(op_host/op_api/ 目录不存在),仅支持 kernel 模式测试。

likedislike
weihao18Collaborators
Aug 24 Comments:

/assign @cpy_123456

likedislike
CANN-robotCANN-robotCollaborators
Aug 24 Assign cpy_123456 as responsible person
CANN-robotCANN-robotCollaborators
Aug 25 Close issue
CANN-robotCANN-robotCollaborators
Aug 25 Added label: resolved
CPZhouCPZhouCollaborators
Sep 3 Added label: requirement
CPZhouCPZhouCollaborators
Sep 3 Removed label: requirement
CPZhouCPZhouCollaborators
Sep 3 Added label: torch-npu
CPZhouCPZhouCollaborators
Sep 3 Removed label: torch-npu