已关闭
docs: fix broken relative-link anchors (batch 1) #272
不喜欢雨天ༀ创建于 8月25日关闭于 9月22日
docs: fix broken relative-link anchors (batch 1) #272
已关闭
共 3 个文件变更+9-16
| @@ -1,6 +1,6 @@ | |||
| 1 | # 训练后量化配置参数说明 | 1 | # 训练后量化配置参数说明 |
| 2 | 2 | ||
| 3 | -如果通过[create\_quant\_config](../api/create_quant_config.md)接口生成的config.json量化配置文件,推理精度不满足要求,则需要参见该章节不断调整config.json文件中的内容(用户修改JSON文件时,请确保层名唯一),直至精度满足要求,json量化配置文件样例请参见接口中的[调用示例](../api/create_quant_config.md#zh-cn_topic_0240187365_section64231658994)部分。 | 3 | +如果通过[create\_quant\_config](../api/create_quant_config.md)接口生成的config.json量化配置文件,推理精度不满足要求,则需要参见该章节不断调整config.json文件中的内容(用户修改JSON文件时,请确保层名唯一),直至精度满足要求,json量化配置文件样例请参见接口中的[调用示例](../api/create_quant_config.md)部分。 |
| 4 | 4 | ||
| 5 | 配置文件中参数说明如下: | 5 | 配置文件中参数说明如下: |
| 6 | 6 | ||
| @@ -738,4 +738,3 @@ | |||
| 738 | </tr> | 738 | </tr> |
| 739 | </tbody> | 739 | </tbody> |
| 740 | </table> | 740 | </table> |
| 741 | - | ||
| @@ -73,7 +73,7 @@ python -m amct_pytorch.eval \ | |||
| 73 | 基准测试精度结果: | 73 | 基准测试精度结果: |
| 74 | `Wikitext2-ppl=6.2825` | 74 | `Wikitext2-ppl=6.2825` |
| 75 | 75 | ||
| 76 | -更多参数说明请参见[参数说明](../../../docs/zh/AMCT_Pytorch_LLM.md#31-通用参数) | 76 | +更多参数说明请参见[参数说明](../../../docs/zh/AMCT_Pytorch_LLM.md#31-%E9%80%9A%E7%94%A8%E5%8F%82%E6%95%B0) |
| 77 | 77 | ||
| 78 | ### 直转量化精度评估 | 78 | ### 直转量化精度评估 |
| 79 | 根据YAML的bit配置,进行直转量化精度测试,评估与基准精度的差距,当前方案默认为对`quant-target`做全A8W4的int量化: | 79 | 根据YAML的bit配置,进行直转量化精度测试,评估与基准精度的差距,当前方案默认为对`quant-target`做全A8W4的int量化: |
| @@ -97,7 +97,7 @@ python -m amct_pytorch.eval \ | |||
| 97 | 直转量化精度结果: | 97 | 直转量化精度结果: |
| 98 | `Wikitext2-ppl=7.0407` | 98 | `Wikitext2-ppl=7.0407` |
| 99 | 99 | ||
| 100 | -更多参数说明请参见[参数说明](../../../docs/zh/AMCT_Pytorch_LLM.md#32-ppl-评估参数) | 100 | +更多参数说明请参见[参数说明](../../../docs/zh/AMCT_Pytorch_LLM.md#32-ppl-%E8%AF%84%E4%BC%B0%E5%8F%82%E6%95%B0) |
| 101 | 101 | ||
| 102 | ### PTQ数据提取 | 102 | ### PTQ数据提取 |
| 103 | 根据不同的量化对象`quant_target`提取相应的 PTQ 校准数据集: | 103 | 根据不同的量化对象`quant_target`提取相应的 PTQ 校准数据集: |
| @@ -115,7 +115,7 @@ python -m amct_pytorch.extract_ptq_data \ | |||
| 115 | 必要传参解释: | 115 | 必要传参解释: |
| 116 | - data_dir:提取数据目录 | 116 | - data_dir:提取数据目录 |
| 117 | 117 | ||
| 118 | -更多参数说明请参见[参数说明](../../../docs/zh/AMCT_Pytorch_LLM.md#33-数据提取参数) | 118 | +更多参数说明请参见[参数说明](../../../docs/zh/AMCT_Pytorch_LLM.md#33-%E6%95%B0%E6%8D%AE%E6%8F%90%E5%8F%96%E5%8F%82%E6%95%B0) |
| 119 | 119 | ||
| 120 | ### Post-Training Quantization | 120 | ### Post-Training Quantization |
| 121 | 引入量化算法,对量化过程做优化,以减少量化带来的损失,以autoround为例: | 121 | 引入量化算法,对量化过程做优化,以减少量化带来的损失,以autoround为例: |
| @@ -147,7 +147,7 @@ python -m amct_pytorch.ptq \ | |||
| 147 | 为提升训练效率,我们提供多卡下的训练脚本 | 147 | 为提升训练效率,我们提供多卡下的训练脚本 |
| 148 | 多卡环境请参照脚本[ptq_multi_npu](../../ptq_multi_npu.sh) | 148 | 多卡环境请参照脚本[ptq_multi_npu](../../ptq_multi_npu.sh) |
| 149 | 149 | ||
| 150 | -更多参数说明请参见[参数说明](../../../docs/zh/AMCT_Pytorch_LLM.md#35-ptq-参数) | 150 | +更多参数说明请参见[参数说明](../../../docs/zh/AMCT_Pytorch_LLM.md#35-ptq-%E5%8F%82%E6%95%B0) |
| 151 | 151 | ||
| 152 | ### 基于Post-Training Quantization的直转量化精度评估 | 152 | ### 基于Post-Training Quantization的直转量化精度评估 |
| 153 | 完成PTQ后,在直转量化精度评估中加入量化算法,与基准测试、无量化算法的直转量化精度比对,验证量化算法的效果: | 153 | 完成PTQ后,在直转量化精度评估中加入量化算法,与基准测试、无量化算法的直转量化精度比对,验证量化算法的效果: |
| @@ -55,7 +55,6 @@ The one-stop platform has pre-configured deployment runtime environment. When us | |||
| 55 | 55 | ||
| 56 | ```shell | 56 | ```shell |
| 57 | python -m amct_pytorch.eval \ | 57 | python -m amct_pytorch.eval \ |
| 58 | - --trust_remote_code \ | ||
| 59 | --model /data/models/Qwen3.6-35B-A3B \ | 58 | --model /data/models/Qwen3.6-35B-A3B \ |
| 60 | --model_name qwen3_6_moe \ | 59 | --model_name qwen3_6_moe \ |
| 61 | --seq_len 4096 \ | 60 | --seq_len 4096 \ |
| @@ -73,13 +72,12 @@ Required parameter explanation: | |||
| 73 | Baseline test accuracy result: | 72 | Baseline test accuracy result: |
| 74 | `Wikitext2-ppl=6.2825` | 73 | `Wikitext2-ppl=6.2825` |
| 75 | 74 | ||
| 76 | -For more detailed parameter explanation, please refer to [Parameter Description](../../../docs/zh/AMCT_Pytorch_LLM.md#31-通用参数) | 75 | +For more detailed parameter explanation, please refer to [Parameter Description](../../../docs/zh/AMCT_Pytorch_LLM.md#31-%E9%80%9A%E7%94%A8%E5%8F%82%E6%95%B0) |
| 77 | 76 | ||
| 78 | ### Direct Conversion Quantization Accuracy Evaluation | 77 | ### Direct Conversion Quantization Accuracy Evaluation |
| 79 | According to YAML bit configuration, perform direct conversion quantization accuracy test, evaluate the gap with baseline accuracy. Current solution defaults to full A8W4 int quantization on `quant-target`: | 78 | According to YAML bit configuration, perform direct conversion quantization accuracy test, evaluate the gap with baseline accuracy. Current solution defaults to full A8W4 int quantization on `quant-target`: |
| 80 | ```shell | 79 | ```shell |
| 81 | python -m amct_pytorch.eval \ | 80 | python -m amct_pytorch.eval \ |
| 82 | - --trust_remote_code \ | ||
| 83 | --model /data/models/Qwen3.6-35B-A3B \ | 81 | --model /data/models/Qwen3.6-35B-A3B \ |
| 84 | --model_name qwen3_6_moe \ | 82 | --model_name qwen3_6_moe \ |
| 85 | --seq_len 4096 \ | 83 | --seq_len 4096 \ |
| @@ -97,13 +95,12 @@ Required parameter explanation: | |||
| 97 | Direct conversion quantization accuracy result: | 95 | Direct conversion quantization accuracy result: |
| 98 | `Wikitext2-ppl=7.0407` | 96 | `Wikitext2-ppl=7.0407` |
| 99 | 97 | ||
| 100 | -For more detailed parameter explanation, please refer to [Parameter Description](../../../docs/zh/AMCT_Pytorch_LLM.md#32-ppl-评估参数) | 98 | +For more detailed parameter explanation, please refer to [Parameter Description](../../../docs/zh/AMCT_Pytorch_LLM.md#32-ppl-%E8%AF%84%E4%BC%B0%E5%8F%82%E6%95%B0) |
| 101 | 99 | ||
| 102 | ### PTQ Data Extraction | 100 | ### PTQ Data Extraction |
| 103 | Extract corresponding PTQ calibration dataset according to different quantization objects `quant_target`: | 101 | Extract corresponding PTQ calibration dataset according to different quantization objects `quant_target`: |
| 104 | ```shell | 102 | ```shell |
| 105 | python -m amct_pytorch.extract_ptq_data \ | 103 | python -m amct_pytorch.extract_ptq_data \ |
| 106 | - --trust_remote_code \ | ||
| 107 | --model /data/models/Qwen3.6-35B-A3B \ | 104 | --model /data/models/Qwen3.6-35B-A3B \ |
| 108 | --model_name qwen3_6_moe \ | 105 | --model_name qwen3_6_moe \ |
| 109 | --seq_len 4096 \ | 106 | --seq_len 4096 \ |
| @@ -115,14 +112,13 @@ python -m amct_pytorch.extract_ptq_data \ | |||
| 115 | Required parameter explanation: | 112 | Required parameter explanation: |
| 116 | - data_dir: Extracted data directory | 113 | - data_dir: Extracted data directory |
| 117 | 114 | ||
| 118 | -For more detailed parameter explanation, please refer to [Parameter Description](../../../docs/zh/AMCT_Pytorch_LLM.md#33-数据提取参数) | 115 | +For more detailed parameter explanation, please refer to [Parameter Description](../../../docs/zh/AMCT_Pytorch_LLM.md#33-%E6%95%B0%E6%8D%AE%E6%8F%90%E5%8F%96%E5%8F%82%E6%95%B0) |
| 119 | 116 | ||
| 120 | ### Post-Training Quantization | 117 | ### Post-Training Quantization |
| 121 | Introduce quantization algorithm to optimize the quantization process to reduce quantization loss, using autoround as an example: | 118 | Introduce quantization algorithm to optimize the quantization process to reduce quantization loss, using autoround as an example: |
| 122 | #### Single-Card Environment | 119 | #### Single-Card Environment |
| 123 | ```shell | 120 | ```shell |
| 124 | python -m amct_pytorch.ptq \ | 121 | python -m amct_pytorch.ptq \ |
| 125 | - --trust_remote_code \ | ||
| 126 | --model /data/models/Qwen3.6-35B-A3B \ | 122 | --model /data/models/Qwen3.6-35B-A3B \ |
| 127 | --model_name qwen3_6_moe \ | 123 | --model_name qwen3_6_moe \ |
| 128 | --seq_len 4096 \ | 124 | --seq_len 4096 \ |
| @@ -147,13 +143,12 @@ Required parameter explanation: | |||
| 147 | To improve training efficiency, we provide training scripts under multi-card | 143 | To improve training efficiency, we provide training scripts under multi-card |
| 148 | For multi-card environment, please refer to script [ptq_multi_npu](../../ptq_multi_npu.sh) | 144 | For multi-card environment, please refer to script [ptq_multi_npu](../../ptq_multi_npu.sh) |
| 149 | 145 | ||
| 150 | -For more detailed parameter explanation, please refer to [Parameter Description](../../../docs/zh/AMCT_Pytorch_LLM.md#35-ptq-参数) | 146 | +For more detailed parameter explanation, please refer to [Parameter Description](../../../docs/zh/AMCT_Pytorch_LLM.md#35-ptq-%E5%8F%82%E6%95%B0) |
| 151 | 147 | ||
| 152 | ### Direct Conversion Quantization Accuracy Evaluation Based on Post-Training Quantization | 148 | ### Direct Conversion Quantization Accuracy Evaluation Based on Post-Training Quantization |
| 153 | After completing PTQ, add quantization algorithm in direct conversion quantization accuracy evaluation, compare with baseline test and direct conversion quantization accuracy without quantization algorithm, verify quantization algorithm effectiveness: | 149 | After completing PTQ, add quantization algorithm in direct conversion quantization accuracy evaluation, compare with baseline test and direct conversion quantization accuracy without quantization algorithm, verify quantization algorithm effectiveness: |
| 154 | ```shell | 150 | ```shell |
| 155 | python -m amct_pytorch.eval \ | 151 | python -m amct_pytorch.eval \ |
| 156 | - --trust_remote_code \ | ||
| 157 | --model /data/models/Qwen3.6-35B-A3B \ | 152 | --model /data/models/Qwen3.6-35B-A3B \ |
| 158 | --model_name qwen3_6_moe \ | 153 | --model_name qwen3_6_moe \ |
| 159 | --seq_len 4096 \ | 154 | --seq_len 4096 \ |
| @@ -173,7 +168,6 @@ Required parameter explanation: | |||
| 173 | After all the preceding steps are complete, export the quantized weights for loading. The weight_map corresponding to the weights is consistent with that on the huggingface official website. | 168 | After all the preceding steps are complete, export the quantized weights for loading. The weight_map corresponding to the weights is consistent with that on the huggingface official website. |
| 174 | ```shell | 169 | ```shell |
| 175 | python -m amct_pytorch.deploy \ | 170 | python -m amct_pytorch.deploy \ |
| 176 | - --trust_remote_code \ | ||
| 177 | --model /data/models/Qwen3.6-35B-A3B \ | 171 | --model /data/models/Qwen3.6-35B-A3B \ |
| 178 | --model_name qwen3_6_moe \ | 172 | --model_name qwen3_6_moe \ |
| 179 | --granularity block \ | 173 | --granularity block \ |