[ST][MS][全量]llama2-70b awq w4a16量化报错ValueError: not enough values to unpack (expected 3, got 2)-mindformers-AtomGit

已关闭

[ST][MS][全量]llama2-70b awq w4a16量化报错ValueError: not enough values to unpack (expected 3, got 2) #507

wang_xingyan创建于 2024年12月26日关闭于 2024年12月31日

wang_xingyan

2024年12月26日创建

1.Describe the current behavior / 问题描述 (Mandatory / 必填)

[ST][MS][全量]llama2-70b awq w4a16量化报错ValueError: not enough values to unpack (expected 3, got 2)

2.Environment / 环境信息 (Mandatory / 必填)

Hardware Environment / 硬件环境(Mandatory / 必填):
Hardware (e.g.Ascend910B1/Mac CPU)

样例：

后端类型	硬件具体类别
Ascend	910B3
CPU	Mac CPU/Win CPU

Software Environment / 软件环境 (Mandatory / 必填):
迭代版本新增问题样例：（根据实际修改和增删）

Software	Version(根据实际修改，必填)
MindSpore	MindSpore 2.5.0B031配套版本

3.Related testcase / 关联用例 (Mandatory / 必填)

Testcase Name/ 用例名 (Mandatory / 必填):
-- Testcase Name(e.g.test_mf_llama_7b_wiki4096_train_cell_dp1mp4pp4mb4_910_16p_0005): *
test_ms_llama2_70b_quant_ptq_awq_w4a16_infer_4p_0001.py

Excute Mode / 执行模式 (Mandatory / 必填):
-- Excute Mode(e.g., Graph\PyNative): *
-- Excute Mode(e.g., O0\O1\O2)：O*

4.Steps to reproduce the issue / 重现步骤 (Mandatory / 必填)

（1）安装mindspore mindformers mindspore_gs
（2）执行awq w4a16量化
msrun --worker_num=4 --local_worker_num=4 --join=True --cluster_time_out=300 --log_dir=quant_ckpt.py -q ptq -c ceval/dev/ -w int4 -a none -k none -o awq -b lm_head -wg per_group -g 128 > log 2>&1 &

（3）基于量化后权重进行推理
（4）验证网络推理精度是否正常

5.Describe the expected behavior / 预期结果 (Mandatory / 必填)

【预期结果】：量化成功，推理精度正常

6.Related log / screenshot / 日志 / 截图 (Mandatory / 必填)

  File "/home/miniconda3/envs/ci_310/lib/python3.10/site-packages/mindspore_gs/ptq/ptq/wrappers/mindformers/linear_smooth_wrappers.py", line 332, in smooth
    smooth_scale = self._search_best_scale(smooth_alpha)
  File "/home/miniconda3/envs/ci_310/lib/python3.10/site-packages/mindspore_gs/ptq/ptq/wrappers/mindformers/linear_smooth_wrappers.py", line 254, in _search_best_scale
    best_scale = self._compute_best_scale(alpha)
  File "/home/miniconda3/envs/ci_310/lib/python3.10/site-packages/mindspore_gs/ptq/ptq/wrappers/mindformers/linear_smooth_wrappers.py", line 269, in _compute_best_scale
    fp16_output = self._module_forward()
  File "/home/miniconda3/envs/ci_310/lib/python3.10/site-packages/mindspore_gs/ptq/ptq/wrappers/mindformers/linear_smooth_wrappers.py", line 325, in _module_forward
    return self.forward_module(self.cat_samples)
  File "/home/miniconda3/envs/ci_310/lib/python3.10/site-packages/mindspore/nn/cell.py", line 743, in __call__
    return self.construct(*args, **kwargs)
  File "/data/jenkins_workspace/TDT_deployment/solution_test/cases/01frame_func/20quantization/infer/test_ms_llama2_70b_quant_ptq_awq_w4a16_infer_4p_0001/mindformers/experimental/infer/core/transformer.py", line 181, in construct
    bs, seq_len, _ = x.shape
ValueError: not enough values to unpack (expected 3, got 2)

7.Special notes for this issue/备注 (Optional / 选填)

**【定位人】**尤日帆

wang_xingyan

2024年12月26日创建了Bug-Report

此处折叠了15条事件消息查看更多

尤尤日帆

2024年12月27日将任务状态从 ACCEPTED 修改为WIP

尤

尤日帆

2024年12月27日评论：

mf新增qkv、ffn重排特性，ffn部分新增xshape为3维。
gs需要适配x的shape大小。pr: https://gitee.com/mindspore/golden-stick/pulls/748

H hangangqiang

2024年12月27日将任务状态从 WIP 修改为VALIDATION

此处折叠了6条事件消息查看更多

尤尤日帆

2024年12月28日将任务状态从 WIP 修改为VALIDATION

wang_xingyan

2024年12月31日评论：

回归版本：
ms: master 52135efe
mf: dev efbdb9e2
gs: master 7b6aeee3
回归步骤：
参考回归步骤
回归结果：

2024-12-31 02:44:46,557 - mindformers[mindformers/generation/text_generator.py:727] - INFO - Generation Config is: {'max_length': 4096, 'max_new_tokens': 1, 'min_length': 0, 'min_new_tokens': None, 'num_beams': 1, 'do_sample': False, 'use_past': True, 'temperature': 1.0, 'top_k': 3, 'top_p': 1, 'repetition_penalty': 1, 'encoder_repetition_penalty': 1.0, 'renormalize_logits': False, 'return_dict_in_generate': False, 'output_scores': False, 'output_logits': False, 'pad_token_id': 0, 'bos_token_id': 1, 'eos_token_id': [2], 'parallel_decoding': False, 'window_size': 5, 'level': 5, 'guess_set_size': 3, '_from_model_config': True}
2024-12-31 02:44:46,558 - mindformers[mindformers/generation/text_generator.py:792] - INFO - The generation mode will be **GREEDY_SEARCH**.
2024-12-31 02:44:46,558 - mindformers[mindformers/modules/block_tables.py:63] - INFO - init cache engine success.
2024-12-31 02:44:46,559 - mindformers[mindformers/experimental/infer/models/llama/llama.py:127] - INFO - Set dynamic input for llama.
2024-12-31 02:44:46,564 - mindformers[mindformers/modules/block_tables.py:126] - INFO - Clear block table cache engines.
Running PTQ...:   0%|                                                                                                                  | 0/80 [00:00<?, ?it/s].Running PTQ...: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████| 80/80 [2:09:27<00:00, 97.09s/it]
[WARNING] GoldenStick(304):2024-12-31 04:54:14,220 [/home/jenkins/solution_test/cases/01frame_func/20quantization/infer/test_ms_llama2_70b_quant_ptq_awq_w4a16_infer_4p_0001/../quant_ckpt.py:197 quant_net] - Convert to real quantize cost time is 0.0002739429473876953 s.
[WARNING] GoldenStick(304):2024-12-31 04:54:14,221 [/home/jenkins/solution_test/cases/01frame_func/20quantization/infer/test_ms_llama2_70b_quant_ptq_awq_w4a16_infer_4p_0001/../quant_ckpt.py:198 quant_net] - Quant Network cost total time is 7831.4105219841 s.
[WARNING] GoldenStick(304):2024-12-31 04:54:14,222 [/home/jenkins/solution_test/cases/01frame_func/20quantization/infer/test_ms_llama2_70b_quant_ptq_awq_w4a16_infer_4p_0001/../quant_ckpt.py:225 ] - Saving checkpoint...
[WARNING] GoldenStick(304):2024-12-31 04:54:28,565 [/home/jenkins/solution_test/cases/01frame_func/20quantization/infer/test_ms_llama2_70b_quant_ptq_awq_w4a16_infer_4p_0001/../quant_ckpt.py:244 ] - Save checkpoint cost time is 14.342804908752441 s.
[WARNING] GoldenStick(304):2024-12-31 04:54:28,565 [/home/jenkins/solution_test/cases/01frame_func/20quantization/infer/test_ms_llama2_70b_quant_ptq_awq_w4a16_infer_4p_0001/../quant_ckpt.py:245 ] - Checkpoint saved to ./output/ptq_ckpt/rank_0...
[INFO] PS(304,fffd397af120,python):2024-12-31-04:54:32.477.565 [mindspore/ccsrc/ps/core/communicator/tcp_client.cc:318] Start] Event base dispatch success!
[INFO] PS(304,fffd39fbf120,python):2024-12-31-04:54:32.477.565 [mindspore/ccsrc/ps/core/communicator/tcp_server.cc:220] Start] Event base dispatch success!

回归结论：
回归通过

wang_xingyan

2024年12月31日将任务状态从 VALIDATION 修改为DONE

I ivss成员

2025年9月10日移除了sig/quant（已删除）标签

I ivss成员

2025年9月11日添加了   bug 标签

I ivss成员

2025年9月11日移除了kind/bug（已删除）标签

A AtomGit-Bot

2025年12月14日关联了看板：MindSpore Bug Tracking System