已关闭
docs: fix broken relative-link anchors (batch 1) #272
不喜欢雨天ༀ创建于 8月25日关闭于 9月22日
docs: fix broken relative-link anchors (batch 1) #272
已关闭
不喜欢雨天ༀ创建于 8月25日关闭于 9月22日
共 3 个文件变更+9-16
@@ -1,6 +1,6 @@
1# 训练后量化配置参数说明1# 训练后量化配置参数说明
2 2 
3-如果通过[create\_quant\_config](../api/create_quant_config.md)接口生成的config.json量化配置文件,推理精度不满足要求,则需要参见该章节不断调整config.json文件中的内容(用户修改JSON文件时,请确保层名唯一),直至精度满足要求,json量化配置文件样例请参见接口中的[调用示例](../api/create_quant_config.md#zh-cn_topic_0240187365_section64231658994)部分。3+如果通过[create\_quant\_config](../api/create_quant_config.md)接口生成的config.json量化配置文件,推理精度不满足要求,则需要参见该章节不断调整config.json文件中的内容(用户修改JSON文件时,请确保层名唯一),直至精度满足要求,json量化配置文件样例请参见接口中的[调用示例](../api/create_quant_config.md)部分。
4 4 
5配置文件中参数说明如下:5配置文件中参数说明如下:
6 6 
@@ -738,4 +738,3 @@
738</tr>738</tr>
739</tbody>739</tbody>
740</table>740</table>
741- 
@@ -73,7 +73,7 @@ python -m amct_pytorch.eval \
73基准测试精度结果:73基准测试精度结果:
74`Wikitext2-ppl=6.2825`74`Wikitext2-ppl=6.2825`
75 75 
76-更多参数说明请参见[参数说明](../../../docs/zh/AMCT_Pytorch_LLM.md#31-通用参数)76+更多参数说明请参见[参数说明](../../../docs/zh/AMCT_Pytorch_LLM.md#31-%E9%80%9A%E7%94%A8%E5%8F%82%E6%95%B0)
77 77 
78### 直转量化精度评估78### 直转量化精度评估
79根据YAML的bit配置,进行直转量化精度测试,评估与基准精度的差距,当前方案默认为对`quant-target`做全A8W4的int量化:79根据YAML的bit配置,进行直转量化精度测试,评估与基准精度的差距,当前方案默认为对`quant-target`做全A8W4的int量化:
@@ -97,7 +97,7 @@ python -m amct_pytorch.eval \
97直转量化精度结果:97直转量化精度结果:
98`Wikitext2-ppl=7.0407`98`Wikitext2-ppl=7.0407`
99 99 
100-更多参数说明请参见[参数说明](../../../docs/zh/AMCT_Pytorch_LLM.md#32-ppl-评估参数)100+更多参数说明请参见[参数说明](../../../docs/zh/AMCT_Pytorch_LLM.md#32-ppl-%E8%AF%84%E4%BC%B0%E5%8F%82%E6%95%B0)
101 101 
102### PTQ数据提取102### PTQ数据提取
103根据不同的量化对象`quant_target`提取相应的 PTQ 校准数据集:103根据不同的量化对象`quant_target`提取相应的 PTQ 校准数据集:
@@ -115,7 +115,7 @@ python -m amct_pytorch.extract_ptq_data \
115必要传参解释:115必要传参解释:
116- data_dir:提取数据目录116- data_dir:提取数据目录
117 117 
118-更多参数说明请参见[参数说明](../../../docs/zh/AMCT_Pytorch_LLM.md#33-数据提取参数)118+更多参数说明请参见[参数说明](../../../docs/zh/AMCT_Pytorch_LLM.md#33-%E6%95%B0%E6%8D%AE%E6%8F%90%E5%8F%96%E5%8F%82%E6%95%B0)
119 119 
120### Post-Training Quantization120### Post-Training Quantization
121引入量化算法,对量化过程做优化,以减少量化带来的损失,以autoround为例:121引入量化算法,对量化过程做优化,以减少量化带来的损失,以autoround为例:
@@ -147,7 +147,7 @@ python -m amct_pytorch.ptq \
147为提升训练效率,我们提供多卡下的训练脚本147为提升训练效率,我们提供多卡下的训练脚本
148多卡环境请参照脚本[ptq_multi_npu](../../ptq_multi_npu.sh)148多卡环境请参照脚本[ptq_multi_npu](../../ptq_multi_npu.sh)
149 149 
150-更多参数说明请参见[参数说明](../../../docs/zh/AMCT_Pytorch_LLM.md#35-ptq-参数)150+更多参数说明请参见[参数说明](../../../docs/zh/AMCT_Pytorch_LLM.md#35-ptq-%E5%8F%82%E6%95%B0)
151 151 
152### 基于Post-Training Quantization的直转量化精度评估152### 基于Post-Training Quantization的直转量化精度评估
153完成PTQ后,在直转量化精度评估中加入量化算法,与基准测试、无量化算法的直转量化精度比对,验证量化算法的效果:153完成PTQ后,在直转量化精度评估中加入量化算法,与基准测试、无量化算法的直转量化精度比对,验证量化算法的效果:
@@ -55,7 +55,6 @@ The one-stop platform has pre-configured deployment runtime environment. When us
55 55 
56 ```shell56 ```shell
57python -m amct_pytorch.eval \57python -m amct_pytorch.eval \
58- --trust_remote_code \
59 --model /data/models/Qwen3.6-35B-A3B \58 --model /data/models/Qwen3.6-35B-A3B \
60 --model_name qwen3_6_moe \59 --model_name qwen3_6_moe \
61 --seq_len 4096 \60 --seq_len 4096 \
@@ -73,13 +72,12 @@ Required parameter explanation:
73Baseline test accuracy result:72Baseline test accuracy result:
74`Wikitext2-ppl=6.2825`73`Wikitext2-ppl=6.2825`
75 74 
76-For more detailed parameter explanation, please refer to [Parameter Description](../../../docs/zh/AMCT_Pytorch_LLM.md#31-通用参数)75+For more detailed parameter explanation, please refer to [Parameter Description](../../../docs/zh/AMCT_Pytorch_LLM.md#31-%E9%80%9A%E7%94%A8%E5%8F%82%E6%95%B0)
77 76 
78### Direct Conversion Quantization Accuracy Evaluation77### Direct Conversion Quantization Accuracy Evaluation
79According to YAML bit configuration, perform direct conversion quantization accuracy test, evaluate the gap with baseline accuracy. Current solution defaults to full A8W4 int quantization on `quant-target`:78According to YAML bit configuration, perform direct conversion quantization accuracy test, evaluate the gap with baseline accuracy. Current solution defaults to full A8W4 int quantization on `quant-target`:
80 ```shell79 ```shell
81python -m amct_pytorch.eval \80python -m amct_pytorch.eval \
82- --trust_remote_code \
83 --model /data/models/Qwen3.6-35B-A3B \81 --model /data/models/Qwen3.6-35B-A3B \
84 --model_name qwen3_6_moe \82 --model_name qwen3_6_moe \
85 --seq_len 4096 \83 --seq_len 4096 \
@@ -97,13 +95,12 @@ Required parameter explanation:
97Direct conversion quantization accuracy result:95Direct conversion quantization accuracy result:
98`Wikitext2-ppl=7.0407`96`Wikitext2-ppl=7.0407`
99 97 
100-For more detailed parameter explanation, please refer to [Parameter Description](../../../docs/zh/AMCT_Pytorch_LLM.md#32-ppl-评估参数)98+For more detailed parameter explanation, please refer to [Parameter Description](../../../docs/zh/AMCT_Pytorch_LLM.md#32-ppl-%E8%AF%84%E4%BC%B0%E5%8F%82%E6%95%B0)
101 99 
102### PTQ Data Extraction100### PTQ Data Extraction
103Extract corresponding PTQ calibration dataset according to different quantization objects `quant_target`:101Extract corresponding PTQ calibration dataset according to different quantization objects `quant_target`:
104 ```shell102 ```shell
105python -m amct_pytorch.extract_ptq_data \103python -m amct_pytorch.extract_ptq_data \
106- --trust_remote_code \
107 --model /data/models/Qwen3.6-35B-A3B \104 --model /data/models/Qwen3.6-35B-A3B \
108 --model_name qwen3_6_moe \105 --model_name qwen3_6_moe \
109 --seq_len 4096 \106 --seq_len 4096 \
@@ -115,14 +112,13 @@ python -m amct_pytorch.extract_ptq_data \
115Required parameter explanation:112Required parameter explanation:
116- data_dir: Extracted data directory113- data_dir: Extracted data directory
117 114 
118-For more detailed parameter explanation, please refer to [Parameter Description](../../../docs/zh/AMCT_Pytorch_LLM.md#33-数据提取参数)115+For more detailed parameter explanation, please refer to [Parameter Description](../../../docs/zh/AMCT_Pytorch_LLM.md#33-%E6%95%B0%E6%8D%AE%E6%8F%90%E5%8F%96%E5%8F%82%E6%95%B0)
119 116 
120### Post-Training Quantization117### Post-Training Quantization
121Introduce quantization algorithm to optimize the quantization process to reduce quantization loss, using autoround as an example:118Introduce quantization algorithm to optimize the quantization process to reduce quantization loss, using autoround as an example:
122#### Single-Card Environment119#### Single-Card Environment
123 ```shell120 ```shell
124python -m amct_pytorch.ptq \121python -m amct_pytorch.ptq \
125- --trust_remote_code \
126 --model /data/models/Qwen3.6-35B-A3B \122 --model /data/models/Qwen3.6-35B-A3B \
127 --model_name qwen3_6_moe \123 --model_name qwen3_6_moe \
128 --seq_len 4096 \124 --seq_len 4096 \
@@ -147,13 +143,12 @@ Required parameter explanation:
147To improve training efficiency, we provide training scripts under multi-card143To improve training efficiency, we provide training scripts under multi-card
148For multi-card environment, please refer to script [ptq_multi_npu](../../ptq_multi_npu.sh)144For multi-card environment, please refer to script [ptq_multi_npu](../../ptq_multi_npu.sh)
149 145 
150-For more detailed parameter explanation, please refer to [Parameter Description](../../../docs/zh/AMCT_Pytorch_LLM.md#35-ptq-参数)146+For more detailed parameter explanation, please refer to [Parameter Description](../../../docs/zh/AMCT_Pytorch_LLM.md#35-ptq-%E5%8F%82%E6%95%B0)
151 147 
152### Direct Conversion Quantization Accuracy Evaluation Based on Post-Training Quantization148### Direct Conversion Quantization Accuracy Evaluation Based on Post-Training Quantization
153After completing PTQ, add quantization algorithm in direct conversion quantization accuracy evaluation, compare with baseline test and direct conversion quantization accuracy without quantization algorithm, verify quantization algorithm effectiveness:149After completing PTQ, add quantization algorithm in direct conversion quantization accuracy evaluation, compare with baseline test and direct conversion quantization accuracy without quantization algorithm, verify quantization algorithm effectiveness:
154 ```shell150 ```shell
155python -m amct_pytorch.eval \151python -m amct_pytorch.eval \
156- --trust_remote_code \
157 --model /data/models/Qwen3.6-35B-A3B \152 --model /data/models/Qwen3.6-35B-A3B \
158 --model_name qwen3_6_moe \153 --model_name qwen3_6_moe \
159 --seq_len 4096 \154 --seq_len 4096 \
@@ -173,7 +168,6 @@ Required parameter explanation:
173After all the preceding steps are complete, export the quantized weights for loading. The weight_map corresponding to the weights is consistent with that on the huggingface official website.168After all the preceding steps are complete, export the quantized weights for loading. The weight_map corresponding to the weights is consistent with that on the huggingface official website.
174 ```shell169 ```shell
175python -m amct_pytorch.deploy \170python -m amct_pytorch.deploy \
176- --trust_remote_code \
177 --model /data/models/Qwen3.6-35B-A3B \171 --model /data/models/Qwen3.6-35B-A3B \
178 --model_name qwen3_6_moe \172 --model_name qwen3_6_moe \
179 --granularity block \173 --granularity block \