main

GGitHub[Quantization][Feature] Support compressed tensors moe w4a8 dynamic weight (#5889)

45a573cf创建于 2月2日历史提交

文件	最后提交记录	最后更新时间
w4a8_dynamic_moe.py	[Quantization][Feature] Support compressed tensors moe w4a8 dynamic weight (#5889) ### What this PR does / why we need it? While using the LLM Compressor quantization tool from the VLLM community to generate quantized weights, the VLLM Ascend engine needs to be adapted to support the compressed tensors quantization format. 1. Support Moe model W4A8 dynamic weight. - vLLM version: v0.13.0 - vLLM main: https://github.com/vllm-project/vllm/commit/bde38c11df0ea066a740efe9b77fff5418be45df --------- Signed-off-by: LHXuuu <scut_xlh@163.com> Signed-off-by: menogrey <1299267905@qq.com> Co-authored-by: menogrey <1299267905@qq.com>	3 个月前
w8a8_int8.py	[Lint]Style: Convert example to ruff format (#5863) ### What this PR does / why we need it? This PR fixes linting issues in the example/ to align with the project's Ruff configuration. - vLLM version: v0.13.0 - vLLM main: https://github.com/vllm-project/vllm/commit/bde38c11df0ea066a740efe9b77fff5418be45df Signed-off-by: root <root@LAPTOP-VQKDDVMG.localdomain> Co-authored-by: root <root@LAPTOP-VQKDDVMG.localdomain>	4 个月前
w8a8_int8_dynamic.py	[Lint]Style: Convert example to ruff format (#5863) ### What this PR does / why we need it? This PR fixes linting issues in the example/ to align with the project's Ruff configuration. - vLLM version: v0.13.0 - vLLM main: https://github.com/vllm-project/vllm/commit/bde38c11df0ea066a740efe9b77fff5418be45df Signed-off-by: root <root@LAPTOP-VQKDDVMG.localdomain> Co-authored-by: root <root@LAPTOP-VQKDDVMG.localdomain>	4 个月前
w8a8_int8_dynamic_moe.py	[CI] Fix lint CI (#5880) Quick fix for lint CI - vLLM version: v0.13.0 - vLLM main: https://github.com/vllm-project/vllm/commit/bde38c11df0ea066a740efe9b77fff5418be45df Signed-off-by: wangxiyuan <wangxiyuan1007@gmail.com>	4 个月前