Star20
31
代码
代码
Issues5
Pull Requests2
项目讨论
Wiki
分析
Star20
31
已合并
perf: conv1d_and_silu consume/produce [B,S,C] directly, drop transposes #476
Igor Prusov创建于 5 天前
perf: conv1d_and_silu consume/produce [B,S,C] directly, drop transposes #476
已合并
Igor Prusov创建于 5 天前
从igor-prusov/GVirteliminate_transpose合入到openeuler/GVirtmaster
I
Igor Prusov
5 天前

Was This PR authored or co-authored using generative AI tooling?

[ ] No

[x] Yes
__1. AI Agent : OpenCode
__2. AI Model : DeepSeek V4 Pro
__3. Prompt :

Reference: openEuler Community Generative AI Use and Open Source Contribution Policy

What this PR does / why we need it:

Optimize conv1d_and_silu kernel by switching to [B, S, C] layout. This allows to drop 2 transpose calls and optimizes the data layout. With this change conv1d_and_silu.py test completes ~13% faster.

Which issue this PR fixes

(optional, in fixes #<issue number>(, fixes #<issue_number>, ...) format, will close that issue when PR gets merged): fixes #

Special notes for your reviewer:

Even after optimization the conv1d_and_silu is slower compared to conv1d_and_silu_token, so we still need that option for prefill.

likedislike
Pull Request已成功合入, 合并人@openeuler-ci-bot
（感谢 Igor Prusov 的贡献）
IIgor Prusov
5 天前 创建了 pull request，commit 2065f9f9
atomgit-bot
atomgit-bot
5 天前 评论：

变更摘要

该 PR 将 conv1d_and_silu 融合卷积核从 [B, C, S] 布局切换为直接消费/产出 [B, S, C] 布局（channel 为内层连续维度），从而在 prefill 路径中去掉了两次 XliteOpTranspose_1_2 转置调用，并使测试运行提速约 13%。核心改动集中在 conv1d_and_silu.h 内核重写：64 通道向量化维度由 sequence 改为 channel，每个 tap 由原先的 vgather 改为连续且 32B 对齐的向量加载（vmul + vadd），并按 kTile 分块处理；同时删除了旧的 packed 通道加载/存储路径与标量跨窗口计算逻辑。

主要改动

  • 内核布局重构（conv1d_and_silu.h）：将统一布局（seqLen != 0）从 input/output [B, C, S] 改为 [B, S, C]（channel 为内层连续维），权重 [C, 1, K] 保持不变；新增 LoadTileInto / StoreTile / ConvertTile / ProcessSequence 等按 kTile=128 分块加载与写回的逻辑，SiLU 直接写 out_tile，删除了 LoadPackedChannel / StorePackedChannel 及旧的全宽 vgather 循环。
  • 权重与状态重排：新增 LoadWeights / LoadState / WriteBackState，将 [w, K] 的权重和状态重排为 [K, kBlock]（tap j 位于连续 64 通道向量），使每次卷积 tap 为连续向量乘加而非逐元素 gather；WriteBackState 基于 LoadStateWin 加载的尾部输入窗口推导新状态。
  • 管道同步改为事件标志：用 FlagMTE2V / FlagVMTE2 / FlagVMTE3 / FlagMTE3V / FlagMTE2S / FlagVS / FlagSV / FlagSMTE3 等显式事件标志替代依赖 pipe_barrier 的隐式顺序，并统一通过 EVENT_ID0/1/2 编号管理窗口与输出 tile 流水线的 MTE2/V/MTE3/S 依赖。
  • 上层调用去除转置（model.cpp 与 op.cpp）：prefill 分支删除两次 XliteOpTranspose_1_2，将 [B, S, C] 的 mix3d 直接传给 XliteOpConv1dAndSiLU 并直接产出 [B, S, C] 的 convSeq3d；op.cpp 的形状校验改为 [B,*,C] 约定（channels = input.shape[2]、seqLen = input.shape[1]），错误信息同步更新。
  • 测试与类型桩更新：conv1d_and_silu.py 测试移除 transpose_1_2 调用；conv1d_and_silu_token.py 的 perf 对比改为 channel 并行路径直接消费/产出 [B, S, C] 并与 token 路径做数值交叉校验；_C.pyi 中 linear_att_conv_and_silu 的 mix_qkv/output 形状注释更新为 [B, S, C]。
likedislike
atomgit-bot
atomgit-bot
5 天前 评论：

代码审查

✅ 未发现问题

likedislike
openeuler-ci-botopeneuler-ci-bot成员
5 天前 将juntianlinux,sinever,luanjianhai设为审查人
openeuler-ci-botopeneuler-ci-bot成员
5 天前 添加了label：sig/sig-Long
openeuler-ci-bot
openeuler-ci-bot成员
5 天前 评论：

Welcome To openEuler Community

Hey @igor-prusov , thanks for your contribution to the community.

Bot Usage Manual

I'm the Bot here serving you. You can find the instructions on how to interact with me at Here . That means you can comment below every pull request or issue to trigger Bot Commands. You can self-configure the PR merge rules for this repository. For more details, please refer to Here.

Contact Guide

If you have any questions, please contact the SIG: sig-Long ,
and any of the maintainers: @juntianlinux, @luanjianhai, @sinever ,
and any of the committers: @JungleXX, @hao-mingzhe, @lulina1, @rubickrt .

likedislike
openeuler-ci-botopeneuler-ci-bot成员
5 天前 添加了label：ai-co-authored
openeuler-ci-botopeneuler-ci-bot成员
5 天前 添加了label：openeuler-cla/yes
openeuler-ci-botopeneuler-ci-bot成员
5 天前 添加了label：ai-includes-code
openeuler-ci-bot
openeuler-ci-bot成员
5 天前 评论：

CLA Signature Pass

igor-prusov, thanks for your pull request. All authors of the commits have signed the CLA. 👍

likedislike
openeuler-ci-botopeneuler-ci-bot成员
5 天前 添加了label：ai-compliance-successful
C
ci bot
5 天前 评论：

CI Test Results

1. GVirt/xlite Static Check

Status: Passed

Check Results:

  • C++: Passed
  • Python: Passed
Click to see detailed output 
Checking 174 C++ files...
Building project to generate operator header files...
Generating compile_commands.json...
  Checked 1/174 files
  Checked 2/174 files
  Checked 3/174 files
  Checked 4/174 files
  Checked 5/174 files
  Checked 6/174 files
  Checked 7/174 files
  Checked 8/174 files
  Checked 9/174 files
  Checked 10/174 files
  Checked 11/174 files
  Checked 12/174 files
  Checked 13/174 files
  Checked 14/174 files
  Checked 15/174 files
  Checked 16/174 files
  Checked 17/174 files
  Checked 18/174 files
  Checked 19/174 files
  Checked 20/174 files
  Checked 21/174 files
  Checked 22/174 files
  Checked 23/174 files
  Checked 24/174 files
  Checked 25/174 files
  Checked 26/174 files
  Checked 27/174 files
  Checked 28/174 files
  Checked 29/174 files
  Checked 30/174 files
  Checked 31/174 files
  Checked 32/174 files
  Checked 33/174 files
  Checked 34/174 files
  Checked 35/174 files
  Checked 36/174 files
  Checked 37/174 files
  Checked 38/174 files
  Checked 39/174 files
  Checked 40/174 files
  Checked 41/174 files
  Checked 42/174 files
  Checked 43/174 files
  Checked 44/174 files
  Checked 45/174 files
  Checked 46/174 files
  Checked 47/174 files
  Checked 48/174 files
  Checked 49/174 files
  Checked 50/174 files
  Checked 51/174 files
  Checked 52/174 files
  Checked 53/174 files
  Checked 54/174 files
  Checked 55/174 files
  Checked 56/174 files
  Checked 57/174 files
  Checked 58/174 files
  Checked 59/174 files
  Checked 60/174 files
  Checked 61/174 files
  Checked 62/174 files
  Checked 63/174 files
  Checked 64/174 files
  Checked 65/174 files
  Checked 66/174 files
  Checked 67/174 files
  Checked 68/174 files
  Checked 69/174 files
  Checked 70/174 files
  Checked 71/174 files
  Checked 72/174 files
  Checked 73/174 files
  Checked 74/174 files
  Checked 75/174 files
  Checked 76/174 files
  Checked 77/174 files
  Checked 78/174 files
  Checked 79/174 files
  Checked 80/174 files
  Checked 81/174 files
  Checked 82/174 files
  Checked 83/174 files
  Checked 84/174 files
  Checked 85/174 files
  Checked 86/174 files
  Checked 87/174 files
  Checked 88/174 files
  Checked 89/174 files
  Checked 90/174 files
  Checked 91/174 files
  Checked 92/174 files
  Checked 93/174 files
  Checked 94/174 files
  Checked 95/174 files
  Checked 96/174 files
  Checked 97/174 files
  Checked 98/174 files
  Checked 99/174 files
  Checked 100/174 files
  Checked 101/174 files
  Checked 102/174 files
  Checked 103/174 files
  Checked 104/174 files
  Checked 105/174 files
  Checked 106/174 files
  Checked 107/174 files
  Checked 108/174 files
  Checked 109/174 files
  Checked 110/174 files
  Checked 111/174 files
  Checked 112/174 files
  Checked 113/174 files
  Checked 114/174 files
  Checked 115/174 files
  Checked 116/174 files
  Checked 117/174 files
  Checked 118/174 files
  Checked 119/174 files
  Checked 120/174 files
  Checked 121/174 files
  Checked 122/174 files
  Checked 123/174 files
  Checked 124/174 files
  Checked 125/174 files
  Checked 126/174 files
  Checked 127/174 files
  Checked 128/174 files
  Checked 129/174 files
  Checked 130/174 files
  Checked 131/174 files
  Checked 132/174 files
  Checked 133/174 files
  Checked 134/174 files
  Checked 135/174 files
  Checked 136/174 files
  Checked 137/174 files
  Checked 138/174 files
  Checked 139/174 files
  Checked 140/174 files
  Checked 141/174 files
  Checked 142/174 files
  Checked 143/174 files
  Checked 144/174 files
  Checked 145/174 files
  Checked 146/174 files
  Checked 147/174 files
  Checked 148/174 files
  Checked 149/174 files
  Checked 150/174 files
  Checked 151/174 files
  Checked 152/174 files
  Checked 153/174 files
  Checked 154/174 files
  Checked 155/174 files
  Checked 156/174 files
  Checked 157/174 files
  Checked 158/174 files
  Checked 159/174 files
  Checked 160/174 files
  Checked 161/174 files
  Checked 162/174 files
  Checked 163/174 files
  Checked 164/174 files
  Checked 165/174 files
  Checked 166/174 files
  Checked 167/174 files
  Checked 168/174 files
  Checked 169/174 files
  Checked 170/174 files
  Checked 171/174 files
  Checked 172/174 files
  Checked 173/174 files
  Checked 174/174 files

✅ All C++ checks passed!
Checking 3 Python files...
  Checking xlite/__init__.py
  Checking xlite/tools/__init__.py
  Checking xlite/tools/fp8_cast_bf16.py
Running flake8...
Running mypy...

✅ All Python checks passed!

============================================================
xlite Static Checker
============================================================

============================================================
Running C++ Static Checks
============================================================


============================================================
Running Python Static Checks
============================================================


============================================================
Summary
============================================================
  CPP       : ✅ PASSED
  PYTHON    : ✅ PASSED

✅ All checks passed!

2. vllm-ascend Static Check

Status: Passed

Click to see detailed output 
Success: no issues found in 1 source file

3. UT Test

Status: Passed

Click to see detailed output 
Running accuracy test for qwen3_0.6B...
(R0) Euler Xlite Model Inited! [tensor paralled(1), data parallel(1), expert parallel(1)]
Qwen3ModelArgs(max_batch_size=4, max_seq_len=1024, dim=1024, head_dim=128, inter_dim=3072, vocab_size=151936, n_layers=28, n_heads=16, n_kv_heads=8, norm_eps=1e-06, rope_theta=1000000.0, dtype='bfloat16', tie_word_embeddings=True, qkv_bias=False, qk_norm=True, model_type='qwen3', config_path=PosixPath('/mnt/nvme0n1/models/Qwen3-0.6B/config.json'))
Memory usage: Model: 1433 MB KV Cache: 448 MB Tensor pool: 104 MB
start to run
Query: Hello, my name is
Completion:  Lina. I'm a 22-year-old student from China. I'm interested in studying in the US. I'm looking for a job in the
Query: The president of the United States is
Completion:  the same as the president of the United Nations. This is because the president of the United States is the same as the president of the United Nations. The president
Query: The capital of France is
Completion:  Paris. The capital of Italy is Rome. The capital of Spain is Madrid. The capital of China is Beijing. The capital of Japan is Tokyo. The capital
Query: The future of AI is
Completion:  not just a technological challenge but a profound transformation of how we live, work, and interact with the world. As we stand at the intersection of artificial intelligence and
generate 146 tokens take 411.38 ms. avg: 354.90 tokens/s @ 12.10 ms bs: 4
The results have been written into the input_file.

========================================
Accuracy Test Results
========================================
✓ Test 0 PASSED
✓ Test 1 PASSED
✓ Test 2 PASSED
✓ Test 3 PASSED

========================================
Summary: 4 passed, 0 failed out of 4 tests
========================================
All accuracy tests PASSED!

This comment was automatically generated by CI bot

likedislike
C
ci bot
5 天前 评论：

CI Test Results

1. GVirt/xlite Static Check

Status: Passed

Check Results:

  • C++: Passed
  • Python: Passed
Click to see detailed output 
Checking 174 C++ files...
Building project to generate operator header files...
Generating compile_commands.json...
  Checked 1/174 files
  Checked 2/174 files
  Checked 3/174 files
  Checked 4/174 files
  Checked 5/174 files
  Checked 6/174 files
  Checked 7/174 files
  Checked 8/174 files
  Checked 9/174 files
  Checked 10/174 files
  Checked 11/174 files
  Checked 12/174 files
  Checked 13/174 files
  Checked 14/174 files
  Checked 15/174 files
  Checked 16/174 files
  Checked 17/174 files
  Checked 18/174 files
  Checked 19/174 files
  Checked 20/174 files
  Checked 21/174 files
  Checked 22/174 files
  Checked 23/174 files
  Checked 24/174 files
  Checked 25/174 files
  Checked 26/174 files
  Checked 27/174 files
  Checked 28/174 files
  Checked 29/174 files
  Checked 30/174 files
  Checked 31/174 files
  Checked 32/174 files
  Checked 33/174 files
  Checked 34/174 files
  Checked 35/174 files
  Checked 36/174 files
  Checked 37/174 files
  Checked 38/174 files
  Checked 39/174 files
  Checked 40/174 files
  Checked 41/174 files
  Checked 42/174 files
  Checked 43/174 files
  Checked 44/174 files
  Checked 45/174 files
  Checked 46/174 files
  Checked 47/174 files
  Checked 48/174 files
  Checked 49/174 files
  Checked 50/174 files
  Checked 51/174 files
  Checked 52/174 files
  Checked 53/174 files
  Checked 54/174 files
  Checked 55/174 files
  Checked 56/174 files
  Checked 57/174 files
  Checked 58/174 files
  Checked 59/174 files
  Checked 60/174 files
  Checked 61/174 files
  Checked 62/174 files
  Checked 63/174 files
  Checked 64/174 files
  Checked 65/174 files
  Checked 66/174 files
  Checked 67/174 files
  Checked 68/174 files
  Checked 69/174 files
  Checked 70/174 files
  Checked 71/174 files
  Checked 72/174 files
  Checked 73/174 files
  Checked 74/174 files
  Checked 75/174 files
  Checked 76/174 files
  Checked 77/174 files
  Checked 78/174 files
  Checked 79/174 files
  Checked 80/174 files
  Checked 81/174 files
  Checked 82/174 files
  Checked 83/174 files
  Checked 84/174 files
  Checked 85/174 files
  Checked 86/174 files
  Checked 87/174 files
  Checked 88/174 files
  Checked 89/174 files
  Checked 90/174 files
  Checked 91/174 files
  Checked 92/174 files
  Checked 93/174 files
  Checked 94/174 files
  Checked 95/174 files
  Checked 96/174 files
  Checked 97/174 files
  Checked 98/174 files
  Checked 99/174 files
  Checked 100/174 files
  Checked 101/174 files
  Checked 102/174 files
  Checked 103/174 files
  Checked 104/174 files
  Checked 105/174 files
  Checked 106/174 files
  Checked 107/174 files
  Checked 108/174 files
  Checked 109/174 files
  Checked 110/174 files
  Checked 111/174 files
  Checked 112/174 files
  Checked 113/174 files
  Checked 114/174 files
  Checked 115/174 files
  Checked 116/174 files
  Checked 117/174 files
  Checked 118/174 files
  Checked 119/174 files
  Checked 120/174 files
  Checked 121/174 files
  Checked 122/174 files
  Checked 123/174 files
  Checked 124/174 files
  Checked 125/174 files
  Checked 126/174 files
  Checked 127/174 files
  Checked 128/174 files
  Checked 129/174 files
  Checked 130/174 files
  Checked 131/174 files
  Checked 132/174 files
  Checked 133/174 files
  Checked 134/174 files
  Checked 135/174 files
  Checked 136/174 files
  Checked 137/174 files
  Checked 138/174 files
  Checked 139/174 files
  Checked 140/174 files
  Checked 141/174 files
  Checked 142/174 files
  Checked 143/174 files
  Checked 144/174 files
  Checked 145/174 files
  Checked 146/174 files
  Checked 147/174 files
  Checked 148/174 files
  Checked 149/174 files
  Checked 150/174 files
  Checked 151/174 files
  Checked 152/174 files
  Checked 153/174 files
  Checked 154/174 files
  Checked 155/174 files
  Checked 156/174 files
  Checked 157/174 files
  Checked 158/174 files
  Checked 159/174 files
  Checked 160/174 files
  Checked 161/174 files
  Checked 162/174 files
  Checked 163/174 files
  Checked 164/174 files
  Checked 165/174 files
  Checked 166/174 files
  Checked 167/174 files
  Checked 168/174 files
  Checked 169/174 files
  Checked 170/174 files
  Checked 171/174 files
  Checked 172/174 files
  Checked 173/174 files
  Checked 174/174 files

✅ All C++ checks passed!
Checking 3 Python files...
  Checking xlite/__init__.py
  Checking xlite/tools/__init__.py
  Checking xlite/tools/fp8_cast_bf16.py
Running flake8...
Running mypy...

✅ All Python checks passed!

============================================================
xlite Static Checker
============================================================

============================================================
Running C++ Static Checks
============================================================


============================================================
Running Python Static Checks
============================================================


============================================================
Summary
============================================================
  CPP       : ✅ PASSED
  PYTHON    : ✅ PASSED

✅ All checks passed!

2. vllm-ascend Static Check

Status: Passed

Click to see detailed output 
Success: no issues found in 1 source file

3. UT Test

Status: Passed

Click to see detailed output 
Running accuracy test for qwen3_0.6B...
(R0) Euler Xlite Model Inited! [tensor paralled(1), data parallel(1), expert parallel(1)]
Qwen3ModelArgs(max_batch_size=4, max_seq_len=1024, dim=1024, head_dim=128, inter_dim=3072, vocab_size=151936, n_layers=28, n_heads=16, n_kv_heads=8, norm_eps=1e-06, rope_theta=1000000.0, dtype='bfloat16', tie_word_embeddings=True, qkv_bias=False, qk_norm=True, model_type='qwen3', config_path=PosixPath('/mnt/nvme0n1/models/Qwen3-0.6B/config.json'))
Memory usage: Model: 1433 MB KV Cache: 448 MB Tensor pool: 104 MB
start to run
Query: Hello, my name is
Completion:  Lina. I'm a 22-year-old student from China. I'm interested in studying in the US. I'm looking for a job in the
Query: The president of the United States is
Completion:  the same as the president of the United Nations. This is because the president of the United States is the same as the president of the United Nations. The president
Query: The capital of France is
Completion:  Paris. The capital of Italy is Rome. The capital of Spain is Madrid. The capital of China is Beijing. The capital of Japan is Tokyo. The capital
Query: The future of AI is
Completion:  not just a technological challenge but a profound transformation of how we live, work, and interact with the world. As we stand at the intersection of artificial intelligence and
generate 146 tokens take 419.95 ms. avg: 347.66 tokens/s @ 12.35 ms bs: 4
The results have been written into the input_file.

========================================
Accuracy Test Results
========================================
✓ Test 0 PASSED
✓ Test 1 PASSED
✓ Test 2 PASSED
✓ Test 3 PASSED

========================================
Summary: 4 passed, 0 failed out of 4 tests
========================================
All accuracy tests PASSED!

This comment was automatically generated by CI bot

likedislike
lulina1
lulina1成员
4 天前 评论：

/lgtm

likedislike
openeuler-ci-botopeneuler-ci-bot成员
4 天前 添加了label：lgtm
openeuler-ci-bot
openeuler-ci-bot成员
4 天前 评论：

Review Code Feedback

  • The label lgtm was added to this pull request. It means that lulina1 reviewed the code changes. 👋
Tips
  • If this pull request is not merged while all conditions are met, comment /check-pr to try again. 😄
likedislike
lulina1
lulina1成员
4 天前 评论：

/approve

likedislike
openeuler-ci-botopeneuler-ci-bot成员
4 天前 添加了label：approved
openeuler-ci-bot
openeuler-ci-bot成员
4 天前 评论：

Review Code Feedback

  • The label approved was added to this pull request. It means that lulina1 reviewed the code changes. 👋
Tips
  • If this pull request is not merged while all conditions are met, comment /check-pr to try again. 😄
likedislike
openeuler-ci-botopeneuler-ci-bot成员
4 天前 合入了pull request，合并节点 SHA：40b1629943b4d8c32ac2f988eb0f2730a7858ec3