Foundation Model Support Matrix

Notes:

• You can click the link in the Model Type column to go to the best practices page recommended by msModelSlim. This page describes the quantization commands and configurations in detail.
• √ indicates that the quantization strategy is officially verified by msModelSlim. It features complete functions and stable performance and is recommended for use.
• - indicates that the quantization strategy has not been officially verified by msModelSlim. You can configure and try the strategy as needed, but the quantization effect and function stability cannot be guaranteed.
• The combination of "model name-quantization mode (such as w8a8s)" marked with "quick quantization" can be used to execute the following quick quantization command to perform model quantization after installation.
• Because newer versions of the Qwen series have been released with more powerful capabilities, the Qwen1.5-14B/32B/72B models have exceeded the maintenance period. Old models in this series will be sunset, and maintenance support for the quantization modes of their live-network versions will be discontinued.

msmodelslim quant --model_path ${MODEL_PATH} --save_path ${SAVE_PATH} --device npu --model_type ${MODEL_TYPE} --quant_type ${QUANT_TYPE} --trust_remote_code True

• For best practices not marked with "quick quantization", read the best practices page of the corresponding model type. Execute the command in the subdirectory of the model type within the example directory, such as DeepSeek or Qwen3.

Quantization Mode Naming Convention

The format of the quantization mode name is W{weight_bit}A{activation_bit}[C{cache_bit}][S], where

• {weight_bit} indicates weight quantization bits (such as 8, 4, or 16).
• {activation_bit} indicates activation quantization bits (such as 8 or 16).
• {cache_bit} (optional) indicates KV cache quantization bits (such as 8).
• S (optional) indicates sparse quantization.

Supported LLMs

!!! info "Note" This table contains a large amount of data. If it is not fully visible, use the scroll bar at the bottom or hold the mouse wheel to scroll horizontally.

Notes:

• ¹ The w8a16 quantization mode is supported only by MindIE.
• ² Both KVCache and FA3 quantization are categorized under c8, as both methods quantize the K and V caches within the LLM. Only MindIE supports c8 quantization modes, such as w8a8c8 and w4a8c8.
• ³ For optimal performance, use the decompression features of the Atlas 300I Duo products after compression. Only MindIE supports sparse quantization modes, including w8a8s and w16a16s.
• ⁴ Only MindIE supports best practices that employ the PDMIX quantization scheme.

Supported MLLMs

Notes:

• ¹ For optimal performance, use the decompression features of the Atlas 300I Duo series after compression. Only MindIE supports sparse quantization modes.
• ² FLUX.1-dev, HunyuanVideo, Wan2.2, and Qwen-Image-Edit-2509 support MXFP quantization.