FilesLast commitLast update
1 month ago
3 months ago
9 days ago
2 months ago
1 month ago
1 month ago
19 days ago
1 month ago
1 month ago
2 months ago
1 month ago
2 months ago
2 months ago
2 months ago
19 days ago
1 month ago
2 months ago
1 month ago
README

mixed_quant_sparse_flash_mla Operator Test Framework

English | 简体中文

Function Description

Based on the pytest test framework, this framework implements function verification of the mixed_quant_sparse_flash_mla operator:

  • CPU side: Reproduces the operator function to generate golden data
  • NPU side: Calls the operator directly or in graph mode through TorchNPU to obtain actual data
  • Precision comparison: Performs precision comparison between CPU and NPU results to verify the operator function

Current Implementation Scope

Parameter Restrictions

  • Data formats:
    • layout_q: BSND, TND
    • layout_kv: PA_BBND

Environment Configuration

Prerequisites

  1. Confirm that TorchNPU is the latest version

Custom Package Invocation

Custom package invocation is supported.

File Structure

pytest File Structure Description

  • test_run.sh # Test case execution script
  • mixed_quant_sparse_flash_mla_golden.py # Operator input parameter processing and CPU-side golden implementation
  • result_compare_method.py # Precision comparison script between CPU golden and NPU results
  • pytest.ini # Creates test markers

Single test case:

  • test_mixed_quant_sparse_flash_mla_single.py # Main program for running a single pytest test case
  • mixed_quant_sparse_flash_mla_paramset.py # Parameter configuration for a single test case

Batch testing:

  • test_mixed_quant_sparse_flash_mla_batch.py # Main program for batch test case testing. It reads pt test cases, obtains NPU outputs, and saves the results to an Excel file
  • ./batch/test_mixed_quant_sparse_flash_mla_pt_save.py # Reads an Excel table and generates test case pt files in batches
  • ./batch/mixed_quant_sparse_flash_mla_process.py # Calls the operator API to obtain NPU outputs

Usage

Run in the pytest folder path:

Running Test Cases

Single Test Case

  1. Manually configure the parameters in mixed_quant_sparse_flash_mla_paramset.py

  2. Run the command:

bash test_run.sh single

Batch Testing

Three batch testing modes are supported. For each mode, you can specify the Excel file, sheet name, and pt save directory through options:

Command Function
batch_save Reads test cases from Excel → golden computation → saves pt files
batch_exec Reads pt files → NPU testing (batch_save is required first)
batch Full process: batch_save + batch_exec. pt files are cleaned up by default after completion
Options
Option Description Default Value Applicable Commands
--excel <path> Specifies the Excel file path ./excel/example.xlsx batch_save / batch_exec / batch
--sheet <name> Specifies the Excel sheet name decode batch_save / batch
--pt-dir <directory> Specifies the directory for saving/reading pt files qsmla_testcase batch_save / batch_exec / batch
--keep-pt Keeps pt files after execution Cleaned up by default batch only
Usage Examples
# batch_save: generate pt files only
bash test_run.sh batch_save
bash test_run.sh batch_save --excel my.xlsx --sheet prefill --pt-dir my_pt

# batch_exec: run NPU testing only (pt files generated by batch_save are required)
bash test_run.sh batch_exec
bash test_run.sh batch_exec --pt-dir my_pt

# batch: full process (pt files are cleaned up by default)
bash test_run.sh batch

# batch: full process, keep pt files
bash test_run.sh batch --keep-pt

# batch: full process, customize all parameters
bash test_run.sh batch --excel my.xlsx --sheet prefill --pt-dir my_pt --keep-pt