提交提案之前，请先检索仓库内是否已有相同的提案，如已有请在同一提案中进行讨论。

💻 需求背景、当前现状、期望实现的功能内容、具体的设计方案、以及测试方案

After merging https://gitcode.com/cann/ops-nn/pull/1187 aclnnQuantMatmulWeightNz kernel now support pertoken w4a4 quantization with up to 1.6x acceleration.
But when trying to pass nz weights in torch_npu.npu_quant_matmul(), aclnnQuantMatmulV4 kernel is selected and an error occurs.
(It's because there is no processing of the w4a4 nz scenario).

Code to reproduce:

import torch
import torch_npu

x1 = torch.rand(size=[2048, 5120], dtype=torch.bfloat16).npu()

x2 = torch.randint(low=-8, high=7, size=[5120, 5120], dtype=torch.int32).npu()
new_x2 = torch_npu.npu_convert_weight_to_int4pack(x2)
new_x2 = torch_npu.npu_format_cast(new_x2, 29)

new_scale = torch.rand([5120], dtype=torch.bfloat16).npu()

start_time = time.time()
new_x1, dynamic_scale = torch_npu.npu_dynamic_quant(x1, dst_type=torch.quint4x2)
mid_time = time.time()
new_res = torch_npu.npu_quant_matmul(
                new_x1,
                new_x2,
                new_scale,
                pertoken_scale=dynamic_scale,
                output_dtype=torch.bfloat16,
            )
end_time = time.time()

替代方案

补充说明

欢迎加入社区，感谢您对社区的贡献 🎉!