matmul_dequant_pipeline-GVirt-AtomGit

已开启

matmul_dequant_pipeline #436

liuweiqi创建于 20 天前从lwq030909/GVirt： pr_branch 合入到 openeuler/GVirt： master

matmul_dequant_pipeline #436

已开启

liuweiqi创建于 20 天前

从lwq030909/GVirt pr_branch合入到openeuler/GVirt master

liuweiqi

20 天前

当前PR是否有AI参与:

[x] 否

__1. AI Agent 平台:
__2. AI 模型:
__3. Prompt上下文 :

参考：openEuler社区《生成式AI工具使用与开源贡献政策》

PR功能描述 / 为什么需要这个合入**:

 进行反量化算子和前序matmul算子的融合，通过流水线的形式进行优化

该PR关联的issue

(格式为fixes #<issue号>, 或者resolves #<issue号>): fixes #
无

希望检视人员了解:

无

合并受阻

liuweiqi

20 天前创建了 pull request，commit 98638704

atomgit-bot

20 天前评论：

变更摘要

本 PR 新增了一个将 INT8 Matmul 与反量化（dequant）融合到单个 MIX 内核的流水线算子 fusion_operator_matmul_dequant_pipeline，用于在 AIC 完成矩阵乘 tile 写回 GM 后立即通过 AIC↔AIV 跨核同步信号触发 AIV 执行 FP16→FP32→BF16 的 tile 级反量化，替代原先「先 matmul_dequant 写 FP16、再 dequant 转 BF16」的串行路径。改动涵盖新增内核实现与实例化文件、Python 绑定与 .pyi 声明、XliteOpFusionOperatorMatmulDequantPipeline 启动逻辑、ForwardLinear 中的融合调用分支，以及通过环境变量 XLITE_FUSED_DENSE_W8A8 控制的开关，并补充了对应正确性测试。

主要改动

新增融合内核实现：新增 fusion_operator_matmul_dequant_pipeline.h，定义 FusionMatmulDequantPipeline 类及 FusionOperatorMatmulDequantPipelineRun 入口，通过 FUSION_MATMUL_DQ_* 同步标志实现 AIC 与 AIV 的 ping-pong 跨核同步，AIC 侧沿用 matmul 的 tiling/copy/mmad/fixpipe 结构，AIV 侧执行 FusionDequantTileFp16ToBf16 完成 FP16→FP32→BF16 转换。
新增内核实例化与导出：新增 fusion_operator_matmul_dequant_pipeline_bfloat16_t.cpp，通过宏 FUSION_OPERATOR_MATMUL_DEQUANT_PIPELINE_FUNC_DEFINE 生成 fusion_operator_matmul_dequant_pipeline_bfloat16_t 内核入口。
新增 Python 绑定与类型声明：在 _C.cpp 中新增 FusionOperatorMatmulDequantPipeline 并注册为 fusion_operator_matmul_dequant_pipeline，在 xlite/_C.pyi 中补充对应函数签名与说明。
新增算子启动逻辑：在 op.cpp/op.h 中新增 XliteOpFusionOperatorMatmulDequantPipeline，负责输入形状/数据类型校验、tiling 参数计算与 swizzle 选择，并通过 aclrtlaunch_fusion_operator_matmul_dequant_pipeline_bfloat16_t 启动内核，输出视图在 FP16 与 BF16 间切换。
接入 ForwardLinear 与运行开关：在 model.cpp 的 XModel::ForwardLinear 中新增 DYNAMIC_QUANT/STATIC_QUANT 场景下的融合分支；runtime.h/runtime.cpp 新增 enableFusedDenseW8A8 字段并支持通过环境变量 XLITE_FUSED_DENSE_W8A8 开启。

openeuler-ci-bot成员

20 天前将juntianlinux,sinever,luanjianhai设为审查人

atomgit-bot

20 天前评论：

代码审查

✅ 未发现问题

openeuler-ci-bot成员

20 天前添加了label：sig/sig-Long

此处折叠了57条消息查看更多

lulina1成员

3 天前设置为草稿状态

lulina1成员

3 天前修改标题为 “[WIP] matmul_dequant_pipeline”，原标题为“matmul_dequant_pipeline”

lulina1成员

2 天前取消了草稿状态

lulina1成员

2 天前修改标题为 “matmul_dequant_pipeline”，原标题为“[WIP] matmul_dequant_pipeline”

ci bot

2 天前评论：

CI Test Results

1. GVirt/xlite Static Check

Status: Passed

Check Results:

C++: Passed
Python: Passed

Click to see detailed output

Checking 154 C++ files...
Building project to generate operator header files...
Generating compile_commands.json...
[K  Checking csrc/_C.cpp
[K  Checking csrc/ascend.h
[K  Checking csrc/auto_tuner.cpp
[K  Checking csrc/auto_tuner.h
[K  Checking csrc/base.cpp
[K  Checking csrc/base.h
[K  Checking csrc/ccl.cpp
[K  Checking csrc/ccl.h
[K  Checking csrc/core_assigner.cpp
[K  Checking csrc/core_assigner.h
[K  Checking csrc/debug.cpp
[K  Checking csrc/debug.h
[K  Checking csrc/kernels/add.h
[K  Checking csrc/kernels/add_bfloat16_t.cpp
[K  Checking csrc/kernels/add_bias.h
[K  Checking csrc/kernels/add_bias_bfloat16_t.cpp
[K  Checking csrc/kernels/add_bias_float.cpp
[K  Checking csrc/kernels/add_bias_float16_t.cpp
[K  Checking csrc/kernels/add_float16_t.cpp
[K  Checking csrc/kernels/all_gather.cpp
[K  Checking csrc/kernels/all_reduce.cpp
[K  Checking csrc/kernels/attention.h
[K  Checking csrc/kernels/attention_bfloat16_t.cpp
[K  Checking csrc/kernels/attention_float16_t.cpp
[K  Checking csrc/kernels/beta_decay.h
[K  Checking csrc/kernels/beta_decay_bfloat16_t.cpp
[K  Checking csrc/kernels/beta_decay_float.cpp
[K  Checking csrc/kernels/beta_decay_float16_t.cpp
[K  Checking csrc/kernels/cast.h
[K  Checking csrc/kernels/cast_bfloat16_t_float.cpp
[K  Checking csrc/kernels/cast_float_bfloat16_t.cpp
[K  Checking csrc/kernels/cce_stub.h
[K  Checking csrc/kernels/concat.cpp
[K  Checking csrc/kernels/config_macro.h
[K  Checking csrc/kernels/conv1d_and_silu.h
[K  Checking csrc/kernels/conv1d_and_silu_bfloat16_t.cpp
[K  Checking csrc/kernels/conv1d_and_silu_float.cpp
[K  Checking csrc/kernels/conv1d_and_silu_float16_t.cpp
[K  Checking csrc/kernels/debug.h
[K  Checking csrc/kernels/dequant.h
[K  Checking csrc/kernels/dequant_float16_t.cpp
[K  Checking csrc/kernels/einsum_mht_hdt_mhd.h
[K  Checking csrc/kernels/einsum_mht_hdt_mhd_bfloat16_t.cpp
[K  Checking csrc/kernels/einsum_mht_hdt_mhd_float16_t.cpp
[K  Checking csrc/kernels/einsum_mht_htd_mhd.h
[K  Checking csrc/kernels/einsum_mht_htd_mhd_bfloat16_t.cpp
[K  Checking csrc/kernels/einsum_mht_htd_mhd_float16_t.cpp
[K  Checking csrc/kernels/embed_kernel.h
[K  Checking csrc/kernels/embed_kernel_bfloat16_t.cpp
[K  Checking csrc/kernels/embed_kernel_float16_t.cpp
[K  Checking csrc/kernels/experts_counts_sum.cpp
[K  Checking csrc/kernels/flash_attention.h
[K  Checking csrc/kernels/flash_attention_bfloat16_t.cpp
[K  Checking csrc/kernels/flash_attention_float16_t.cpp
[K  Checking csrc/kernels/flash_mla_v2.h
[K  Checking csrc/kernels/flash_mla_v2_bfloat16_t.cpp
[K  Checking csrc/kernels/fusion_operator_matmul_dequant_pipeline.h
[K  Checking csrc/kernels/fusion_operator_matmul_dequant_pipeline_bfloat16_t.cpp
[K  Checking csrc/kernels/gather_sparse_kv_cache.h
[K  Checking csrc/kernels/gather_sparse_kv_cache_bfloat16_t.cpp
[K  Checking csrc/kernels/group_matmul.h
[K  Checking csrc/kernels/group_matmul_bfloat16_t.cpp
[K  Checking csrc/kernels/group_matmul_float.cpp
[K  Checking csrc/kernels/group_matmul_float16_t.cpp
[K  Checking csrc/kernels/group_matmul_int8_t.cpp
[K  Checking csrc/kernels/indexer_prepare.h
[K  Checking csrc/kernels/indexer_prepare_bfloat16_t.cpp
[K  Checking csrc/kernels/indexer_prepare_float16_t.cpp
[K  Checking csrc/kernels/indexer_scores.h
[K  Checking csrc/kernels/indexer_scores_bfloat16_t.cpp
[K  Checking csrc/kernels/indexer_scores_float16_t.cpp
[K  Checking csrc/kernels/indexer_topk.h
[K  Checking csrc/kernels/indexer_topk_bfloat16_t.cpp
[K  Checking csrc/kernels/indexer_topk_float16_t.cpp
[K  Checking csrc/kernels/kernel_macro.h
[K  Checking csrc/kernels/kernel_param.h
[K  Checking csrc/kernels/matmul.h
[K  Checking csrc/kernels/matmul_bfloat16_t.cpp
[K  Checking csrc/kernels/matmul_float.cpp
[K  Checking csrc/kernels/matmul_float16_t.cpp
[K  Checking csrc/kernels/matmul_int4b_t.cpp
[K  Checking csrc/kernels/matmul_int8_t.cpp
[K  Checking csrc/kernels/mla_prepare.h
[K  Checking csrc/kernels/mla_prepare_bfloat16_t.cpp
[K  Checking csrc/kernels/mla_prepare_float16_t.cpp
[K  Checking csrc/kernels/mla_v2.h
[K  Checking csrc/kernels/mla_v2_bfloat16_t.cpp
[K  Checking csrc/kernels/mla_v3.h
[K  Checking csrc/kernels/mla_v3_bfloat16_t.cpp
[K  Checking csrc/kernels/msd_merge_dequant.h
[K  Checking csrc/kernels/msd_merge_dequant_int8_t.cpp
[K  Checking csrc/kernels/muls.h
[K  Checking csrc/kernels/muls_bfloat16_t.cpp
[K  Checking csrc/kernels/muls_float16_t.cpp
[K  Checking csrc/kernels/norm.h
[K  Checking csrc/kernels/norm_bfloat16_t.cpp
[K  Checking csrc/kernels/norm_float16_t.cpp
[K  Checking csrc/kernels/permutation.cpp
[K  Checking csrc/kernels/qk_rms_norm.h
[K  Checking csrc/kernels/qk_rms_norm_bfloat16_t.cpp
[K  Checking csrc/kernels/qk_rms_norm_float16_t.cpp
[K  Checking csrc/kernels/quant.h
[K  Checking csrc/kernels/quant_bfloat16_t.cpp
[K  Checking csrc/kernels/quant_dyn.h
[K  Checking csrc/kernels/quant_dyn_bfloat16_t.cpp
[K  Checking csrc/kernels/recurrent_gated_delta_rule.h
[K  Checking csrc/kernels/recurrent_gated_delta_rule_bfloat16_t.cpp
[K  Checking csrc/kernels/recurrent_gated_delta_rule_float.cpp
[K  Checking csrc/kernels/recurrent_gated_delta_rule_float16_t.cpp
[K  Checking csrc/kernels/reduce_scatter.cpp
[K  Checking csrc/kernels/reorder_moe.cpp
[K  Checking csrc/kernels/rope_and_cache.h
[K  Checking csrc/kernels/rope_and_cache_bfloat16_t.cpp
[K  Checking csrc/kernels/rope_and_cache_float16_t.cpp
[K  Checking csrc/kernels/rope_complex_and_cache.h
[K  Checking csrc/kernels/rope_complex_and_cache_bfloat16_t.cpp
[K  Checking csrc/kernels/rope_complex_and_cache_float16_t.cpp
[K  Checking csrc/kernels/sigmoid_gate_mul.h
[K  Checking csrc/kernels/sigmoid_gate_mul_bfloat16_t.cpp
[K  Checking csrc/kernels/sigmoid_gate_mul_float16_t.cpp
[K  Checking csrc/kernels/sigmoid_topk.h
[K  Checking csrc/kernels/sigmoid_topk_bfloat16_t.cpp
[K  Checking csrc/kernels/sigmoid_topk_float.cpp
[K  Checking csrc/kernels/silu_and_mul.h
[K  Checking csrc/kernels/silu_and_mul_bfloat16_t.cpp
[K  Checking csrc/kernels/silu_and_mul_float.cpp
[K  Checking csrc/kernels/silu_and_mul_float16_t.cpp
[K  Checking csrc/kernels/softmax.cpp
[K  Checking csrc/kernels/softmax_attn_aiv.h
[K  Checking csrc/kernels/softmax_topk.h
[K  Checking csrc/kernels/softmax_topk_bfloat16_t.cpp
[K  Checking csrc/kernels/softmax_topk_float.cpp
[K  Checking csrc/kernels/split.cpp
[K  Checking csrc/kernels/topk.h
[K  Checking csrc/kernels/topk_bfloat16_t.cpp
[K  Checking csrc/kernels/topk_float.cpp
[K  Checking csrc/kernels/transpose_1_2.h
[K  Checking csrc/kernels/transpose_1_2_bfloat16_t.cpp
[K  Checking csrc/kernels/transpose_1_2_float16_t.cpp
[K  Checking csrc/kernels/unpack_activation.h
[K  Checking csrc/kernels/unpack_activation_int8_t.cpp
[K  Checking csrc/kernels/unpermutation.h
[K  Checking csrc/kernels/unpermutation_bfloat16_t.cpp
[K  Checking csrc/kernels/unpermutation_float.cpp
[K  Checking csrc/model.cpp
[K  Checking csrc/model.h
[K  Checking csrc/op.cpp
[K  Checking csrc/op.h
[K  Checking csrc/runtime.cpp
[K  Checking csrc/runtime.h
[K  Checking csrc/sock.cpp
[K  Checking csrc/sock.h
[K  Checking csrc/swizzle.cpp
[K  Checking csrc/swizzle.h

✅ All C++ checks passed!
Checking 3 Python files...
[K  Checking xlite/__init__.py
[K  Checking xlite/tools/__init__.py
[K  Checking xlite/tools/fp8_cast_bf16.py
Running flake8...
Running mypy...

✅ All Python checks passed!

============================================================
xlite Static Checker
============================================================

============================================================
Running C++ Static Checks
============================================================


============================================================
Running Python Static Checks
============================================================


============================================================
Summary
============================================================
  CPP       : ✅ PASSED
  PYTHON    : ✅ PASSED

✅ All checks passed!

2. vllm-ascend Static Check

Status: Passed

Click to see detailed output

Success: no issues found in 1 source file

3. UT Test

Status: Passed

Click to see detailed output

Running accuracy test for qwen3_0.6B...
[38;5;210m(R0) [0mEuler Xlite Model Inited! [tensor paralled(1), data parallel(1), expert parallel(1)]
Qwen3ModelArgs(max_batch_size=4, max_seq_len=1024, dim=1024, head_dim=128, inter_dim=3072, vocab_size=151936, n_layers=28, n_heads=16, n_kv_heads=8, norm_eps=1e-06, rope_theta=1000000.0, dtype='bfloat16', tie_word_embeddings=True, qkv_bias=False, qk_norm=True, model_type='qwen3')
Memory usage: Model: 1433 MB KV Cache: 448 MB Tensor pool: 104 MB
start to run
Query: Hello, my name is
Completion:  Lina. I'm a 22-year-old student from China. I'm interested in studying in the US. I'm looking for a job in the
Query: The president of the United States is
Completion:  the same as the president of the United Nations. This is because the president of the United States is the same as the president of the United Nations. The president
Query: The capital of France is
Completion:  Paris. The capital of Italy is Rome. The capital of Spain is Madrid. The capital of China is Beijing. The capital of Japan is Tokyo. The capital
Query: The future of AI is
Completion:  not just a technological challenge but a profound transformation of how we live, work, and interact with the world. As we stand at the intersection of artificial intelligence and
generate 146 tokens take 392.36 ms. avg: 372.10 tokens/s @ 11.54 ms bs: 4
The results have been written into the input_file.

========================================
Accuracy Test Results
========================================
✓ Test 0 PASSED
✓ Test 1 PASSED
✓ Test 2 PASSED
✓ Test 3 PASSED

========================================
Summary: 4 passed, 0 failed out of 4 tests
========================================
All accuracy tests PASSED!

This comment was automatically generated by CI bot