MindSpeed/mindspeed/core/qat · guofanfeng23/MindSpeed

ascend-robotfeat(qat): add W4A4 MXFP4 quantization-aware training support

文件	最后提交记录	最后更新时间
layers.py	feat(qat): add W4A4 MXFP4 quantization-aware training support Co-authored-by: n_nddddd<lanshangwei1@huawei.com> # message auto-generated for no-merge-commit merge: !3614 merge feat_qat_w4a4_1 into master feat(qat): add W4A4 MXFP4 quantization-aware training support Created-by: n_nddddd Commit-by: n_nddddd Merged-by: ascend-robot Description: title: "feat: add w4a4 qat feature" labels: ["feat"] assignees: lanshangwei What this PR does / why we need it? add feature of qat w4a4 Does this PR introduce any user-facing change? no, just add a new qat type How was this patch tested? 实验报告 https://wiki.huawei.com/domains/159368/wiki/325164/WIKI2026070211708534 UT运行报告： ============================= test session starts ============================== platform linux -- Python 3.10.19, pytest-9.0.2, pluggy-1.6.0 -- /home/anaconda3/envs/llm_lsw/bin/python cachedir: .pytest_cache rootdir: /home/l00611484/workspace/MindSpeed plugins: mock-3.15.1, jaxtyping-0.3.4, hydra-core-1.3.2, anyio-4.12.0 collecting ... collected 17 items tests_extend/unit_tests/features/qat/test_w4a4_core_functions.py::TestW4A4FakeQuantization::test_output_shape PASSED [ 5%] tests_extend/unit_tests/features/qat/test_w4a4_core_functions.py::TestW4A4FakeQuantization::test_output_not_nan PASSED [ 11%] tests_extend/unit_tests/features/qat/test_w4a4_core_functions.py::TestW4A4FakeQuantization::test_deterministic PASSED [ 17%] tests_extend/unit_tests/features/qat/test_w4a4_core_functions.py::TestW4A4FakeQuantization::test_dtype_preserved PASSED [ 23%] tests_extend/unit_tests/features/qat/test_w4a4_core_functions.py::TestW4A4FakeQuantization::test_zero_tensor PASSED [ 29%] tests_extend/unit_tests/features/qat/test_w4a4_core_functions.py::TestW4A4FakeQuantization::test_small_values PASSED [ 35%] tests_extend/unit_tests/features/qat/test_w4a4_core_functions.py::TestW4A4FakeQuantization::test_ste_backward_exists PASSED [ 41%] tests_extend/unit_tests/features/qat/test_w4a4_core_functions.py::TestW4A4FakeQuantization::test_ste_grad_values PASSED [ 47%] tests_extend/unit_tests/features/qat/test_w4a4_core_functions.py::TestW4A4LinearForward::test_forward_shape PASSED [ 52%] tests_extend/unit_tests/features/qat/test_w4a4_core_functions.py::TestW4A4LinearForward::test_forward_with_bias PASSED [ 58%] tests_extend/unit_tests/features/qat/test_w4a4_core_functions.py::TestW4A4LinearForward::test_forward_not_nan PASSED [ 64%] tests_extend/unit_tests/features/qat/test_w4a4_core_functions.py::TestW4A4LinearForward::test_forward_differentiable PASSED [ 70%] tests_extend/unit_tests/features/qat/test_w4a4_core_functions.py::TestW4A4LinearBackward::test_backward_shapes PASSED [ 76%] tests_extend/unit_tests/features/qat/test_w4a4_core_functions.py::TestW4A4LinearBackward::test_backward_not_nan PASSED [ 82%] tests_extend/unit_tests/features/qat/test_w4a4_core_functions.py::TestW4A4LinearBackward::test_dw_uses_quantized_input PASSED [ 88%] tests_extend/unit_tests/features/qat/test_w4a4_core_functions.py::TestIntegration::test_end_to_end PASSED [ 94%] tests_extend/unit_tests/features/qat/test_w4a4_core_functions.py::TestIntegration::test_gradient_flow PASSED [100%] See merge request: Ascend/MindSpeed!3614	2 个月前
w4a16_fake_quantization.py	feat: add w4a16 quant Co-authored-by: xusiyang<xusiyang2@huawei.com> # message auto-generated for no-merge-commit merge: !3334 merge master into master feat: add w4a16 quant Created-by: weixin_44492126 Commit-by: xusiyang;weixin_44492126 Merged-by: ascend-robot Description: What this PR does / why we need it? QAT支持W4A16伪量化 Does this PR introduce any user-facing change? 详细说明见：docs/zh/features/qat_quant.md How was this patch tested? 参数添加"--qat-scheme w4a16-mxf4"时启用伪量化 See merge request: Ascend/MindSpeed!3334	6 个月前
w4a4_fake_quantization.py	feat(qat): add W4A4 MXFP4 quantization-aware training support Co-authored-by: n_nddddd<lanshangwei1@huawei.com> # message auto-generated for no-merge-commit merge: !3614 merge feat_qat_w4a4_1 into master feat(qat): add W4A4 MXFP4 quantization-aware training support Created-by: n_nddddd Commit-by: n_nddddd Merged-by: ascend-robot Description: title: "feat: add w4a4 qat feature" labels: ["feat"] assignees: lanshangwei What this PR does / why we need it? add feature of qat w4a4 Does this PR introduce any user-facing change? no, just add a new qat type How was this patch tested? 实验报告 https://wiki.huawei.com/domains/159368/wiki/325164/WIKI2026070211708534 UT运行报告： ============================= test session starts ============================== platform linux -- Python 3.10.19, pytest-9.0.2, pluggy-1.6.0 -- /home/anaconda3/envs/llm_lsw/bin/python cachedir: .pytest_cache rootdir: /home/l00611484/workspace/MindSpeed plugins: mock-3.15.1, jaxtyping-0.3.4, hydra-core-1.3.2, anyio-4.12.0 collecting ... collected 17 items tests_extend/unit_tests/features/qat/test_w4a4_core_functions.py::TestW4A4FakeQuantization::test_output_shape PASSED [ 5%] tests_extend/unit_tests/features/qat/test_w4a4_core_functions.py::TestW4A4FakeQuantization::test_output_not_nan PASSED [ 11%] tests_extend/unit_tests/features/qat/test_w4a4_core_functions.py::TestW4A4FakeQuantization::test_deterministic PASSED [ 17%] tests_extend/unit_tests/features/qat/test_w4a4_core_functions.py::TestW4A4FakeQuantization::test_dtype_preserved PASSED [ 23%] tests_extend/unit_tests/features/qat/test_w4a4_core_functions.py::TestW4A4FakeQuantization::test_zero_tensor PASSED [ 29%] tests_extend/unit_tests/features/qat/test_w4a4_core_functions.py::TestW4A4FakeQuantization::test_small_values PASSED [ 35%] tests_extend/unit_tests/features/qat/test_w4a4_core_functions.py::TestW4A4FakeQuantization::test_ste_backward_exists PASSED [ 41%] tests_extend/unit_tests/features/qat/test_w4a4_core_functions.py::TestW4A4FakeQuantization::test_ste_grad_values PASSED [ 47%] tests_extend/unit_tests/features/qat/test_w4a4_core_functions.py::TestW4A4LinearForward::test_forward_shape PASSED [ 52%] tests_extend/unit_tests/features/qat/test_w4a4_core_functions.py::TestW4A4LinearForward::test_forward_with_bias PASSED [ 58%] tests_extend/unit_tests/features/qat/test_w4a4_core_functions.py::TestW4A4LinearForward::test_forward_not_nan PASSED [ 64%] tests_extend/unit_tests/features/qat/test_w4a4_core_functions.py::TestW4A4LinearForward::test_forward_differentiable PASSED [ 70%] tests_extend/unit_tests/features/qat/test_w4a4_core_functions.py::TestW4A4LinearBackward::test_backward_shapes PASSED [ 76%] tests_extend/unit_tests/features/qat/test_w4a4_core_functions.py::TestW4A4LinearBackward::test_backward_not_nan PASSED [ 82%] tests_extend/unit_tests/features/qat/test_w4a4_core_functions.py::TestW4A4LinearBackward::test_dw_uses_quantized_input PASSED [ 88%] tests_extend/unit_tests/features/qat/test_w4a4_core_functions.py::TestIntegration::test_end_to_end PASSED [ 94%] tests_extend/unit_tests/features/qat/test_w4a4_core_functions.py::TestIntegration::test_gradient_flow PASSED [100%] See merge request: Ascend/MindSpeed!3614	2 个月前
w8a16_fake_quantization.py	add w8a16 quant Co-authored-by: wangjunhang<wangjunhang7@huawei.com> # message auto-generated for no-merge-commit merge: !3460 merge Dev_W8A16 into master add w8a16 quant Created-by: goodflower9 Commit-by: wangjunhang Merged-by: ascend-robot Description: What this PR does / why we need it? This PR adds W8A16 MXFP8 QAT support based on the existing QAT flow. Does this PR introduce any user-facing change? Yes. Users can enable W8A16 MXFP8 QAT with: `--qat-scheme w8a16-mxfp8` Related doc: `docs/zh/features/qat_quant.md` How was this patch tested? Enable fake quantization by adding the parameter `--qat-scheme w8a16-mxfp8` https://wiki.huawei.com/domains/76578/wiki/233229/WIKI2026051211063362 See merge request: Ascend/MindSpeed!3460	4 个月前