您好，Qwen2.5-VL冻结参数需要在model_xb.json里面配置。参数为：对应模块下面的freeze参数。

您好，Qwen2.5-VL冻结参数需要在model_xb.json里面配置。参数为：对应模块下面的freeze参数。

冻结llm，现在只能在pretrain_vlm.py 109行添加一行freeze_text_decoder=True

您好，Qwen2.5-VL冻结参数需要在model_xb.json里面配置。参数为：对应模块下面的freeze参数。

冻结llm，现在只能在pretrain_vlm.py 109行添加一行freeze_text_decoder=True

不行奥，会报错，提示优化器中的参数为空，虽然只冻结了llm，没有冻结vit，但还是报错了。

learning rate decay style: cosine
[rank5]: Traceback (most recent call last):
[rank5]: File "/home/code/MindSpeed-MM/pretrain_vlm.py", line 159, in
[rank5]: pretrain(
[rank5]: File "/home/code/MindSpeed-MM/mindspeed_mm/training.py", line 186, in pretrain
[rank5]: model, optimizer, opt_param_scheduler = setup_model_and_optimizer(
[rank5]: File "/home/code/MindSpeed-MM/MindSpeed/mindspeed/core/training.py", line 543, in wrapper
[rank5]: model, optimizer, opt_param_scheduler = setup_model_and_optimizer(*args, **kwargs)
[rank5]: File "/home/code/MindSpeed-MM/megatron/training/training.py", line 544, in setup_model_and_optimizer
[rank5]: optimizer = get_megatron_optimizer(config, model, no_wd_decay_cond,
[rank5]: File "/home/code/MindSpeed-MM/megatron/core/optimizer/init.py", line 338, in get_megatron_optimizer
[rank5]: _get_megatron_optimizer_based_on_param_groups(
[rank5]: File "/home/code/MindSpeed-MM/megatron/core/optimizer/init.py", line 194, in _get_megatron_optimizer_based_on_param_groups
[rank5]: optimizer = Adam(
[rank5]: File "/home/code/MindSpeed-MM/MindSpeed/mindspeed/optimizer/adamw.py", line 95, in init
[rank5]: super(AdamW, self).init(params, defaults)
[rank5]: File "/work/model/miniconda3/envs/mindspeed2.1.0/lib/python3.10/site-packages/torch/optim/optimizer.py", line 372, in init
[rank5]: raise ValueError("optimizer got an empty parameter list")
[rank5]: ValueError: optimizer got an empty parameter list

您好，Qwen2.5-VL冻结参数需要在model_xb.json里面配置。参数为：对应模块下面的freeze参数。

冻结llm，现在只能在pretrain_vlm.py 109行添加一行freeze_text_decoder=True

不行奥，会报错，提示优化器中的参数为空，虽然只冻结了llm，没有冻结vit，但还是报错了。

learning rate decay style: cosine
[rank5]: Traceback (most recent call last):
[rank5]: File "/home/code/MindSpeed-MM/pretrain_vlm.py", line 159, in
[rank5]: pretrain(
[rank5]: File "/home/code/MindSpeed-MM/mindspeed_mm/training.py", line 186, in pretrain
[rank5]: model, optimizer, opt_param_scheduler = setup_model_and_optimizer(
[rank5]: File "/home/code/MindSpeed-MM/MindSpeed/mindspeed/core/training.py", line 543, in wrapper
[rank5]: model, optimizer, opt_param_scheduler = setup_model_and_optimizer(*args, **kwargs)
[rank5]: File "/home/code/MindSpeed-MM/megatron/training/training.py", line 544, in setup_model_and_optimizer
[rank5]: optimizer = get_megatron_optimizer(config, model, no_wd_decay_cond,
[rank5]: File "/home/code/MindSpeed-MM/megatron/core/optimizer/init.py", line 338, in get_megatron_optimizer
[rank5]: _get_megatron_optimizer_based_on_param_groups(
[rank5]: File "/home/code/MindSpeed-MM/megatron/core/optimizer/init.py", line 194, in _get_megatron_optimizer_based_on_param_groups
[rank5]: optimizer = Adam(
[rank5]: File "/home/code/MindSpeed-MM/MindSpeed/mindspeed/optimizer/adamw.py", line 95, in init
[rank5]: super(AdamW, self).init(params, defaults)
[rank5]: File "/work/model/miniconda3/envs/mindspeed2.1.0/lib/python3.10/site-packages/torch/optim/optimizer.py", line 372, in init
[rank5]: raise ValueError("optimizer got an empty parameter list")
[rank5]: ValueError: optimizer got an empty parameter list

不知道是不是PP的问题，可以查看文档添加参数https://gitcode.com/Ascend/MindSpeed-MM/blob/master/docs/features/dummy_optimizer.md

model.json中默认vit是冻结的，如果想冻结llm，放开vit训练，可以在上述修改前提下将vit模块的freeze参数置为false

您好，我在finetune Qwen2.5VL时也想冻结LLM放开ViT。我按照上述在freeze里加了freeze_text_decoder=True，在model配置文件里将image projector的freeze设为了false。我用的PP=2去跑, vision:[32,0], llm:[12,16]，跑finetune时报错说learning_rate is None。是否是因为第2个PP上没有可以train的部分而报错，有什么解决方案吗？

[rank2]: Traceback (most recent call last):
[rank2]: File "/home/ma-user/work/MindSpeed-MM/pretrain_qwen2vl.py", line 163, in
[rank2]: pretrain(
[rank2]: File "/home/ma-user/work/MindSpeed-MM/mindspeed_mm/training.py", line 244, in pretrain
[rank2]: iteration, num_floating_point_operations_so_far = train(
[rank2]: File "/home/ma-user/work/MindSpeed-MM/mindspeed_mm/training.py", line 487, in train
[rank2]: report_memory_flag = training_log(
[rank2]: File "/home/ma-user/work/MindSpeed-MM/mindspeed_mm/training.py", line 923, in training_log
[rank2]: raise AssertionError
[rank2]: AssertionError
[rank3]: Traceback (most recent call last):
[rank3]: File "/home/ma-user/work/MindSpeed-MM/pretrain_qwen2vl.py", line 163, in
[rank3]: pretrain(
[rank3]: File "/home/ma-user/work/MindSpeed-MM/mindspeed_mm/training.py", line 244, in pretrain
[rank3]: iteration, num_floating_point_operations_so_far = train(
[rank3]: File "/home/ma-user/work/MindSpeed-MM/mindspeed_mm/training.py", line 487, in train
[rank3]: report_memory_flag = training_log(
[rank3]: File "/home/ma-user/work/MindSpeed-MM/mindspeed_mm/training.py", line 923, in training_log
[rank3]: raise AssertionError
[rank3]: AssertionError

您好，我在finetune Qwen2.5VL时也想冻结LLM放开ViT。我按照上述在freeze里加了freeze_text_decoder=True，在model配置文件里将image projector的freeze设为了false。我用的PP=2去跑, vision:[32,0], llm:[12,16]，跑finetune时报错说learning_rate is None。是否是因为第2个PP上没有可以train的部分而报错，有什么解决方案吗？

[rank2]: Traceback (most recent call last):
[rank2]: File "/home/ma-user/work/MindSpeed-MM/pretrain_qwen2vl.py", line 163, in
[rank2]: pretrain(
[rank2]: File "/home/ma-user/work/MindSpeed-MM/mindspeed_mm/training.py", line 244, in pretrain
[rank2]: iteration, num_floating_point_operations_so_far = train(
[rank2]: File "/home/ma-user/work/MindSpeed-MM/mindspeed_mm/training.py", line 487, in train
[rank2]: report_memory_flag = training_log(
[rank2]: File "/home/ma-user/work/MindSpeed-MM/mindspeed_mm/training.py", line 923, in training_log
[rank2]: raise AssertionError
[rank2]: AssertionError
[rank3]: Traceback (most recent call last):
[rank3]: File "/home/ma-user/work/MindSpeed-MM/pretrain_qwen2vl.py", line 163, in
[rank3]: pretrain(
[rank3]: File "/home/ma-user/work/MindSpeed-MM/mindspeed_mm/training.py", line 244, in pretrain
[rank3]: iteration, num_floating_point_operations_so_far = train(
[rank3]: File "/home/ma-user/work/MindSpeed-MM/mindspeed_mm/training.py", line 487, in train
[rank3]: report_memory_flag = training_log(
[rank3]: File "/home/ma-user/work/MindSpeed-MM/mindspeed_mm/training.py", line 923, in training_log
[rank3]: raise AssertionError
[rank3]: AssertionError

@hx37790928

你好，这个文档里说的参数是否添加 https://gitcode.com/Ascend/MindSpeed-MM/blob/master/docs/features/dummy_optimizer.md

您好，Qwen2.5-VL冻结参数需要在model_xb.json里面配置。参数为：对应模块下面的freeze参数。

冻结llm，现在只能在pretrain_vlm.py 109行添加一行freeze_text_decoder=True

不行奥，会报错，提示优化器中的参数为空，虽然只冻结了llm，没有冻结vit，但还是报错了。

learning rate decay style: cosine
[rank5]: Traceback (most recent call last):
[rank5]: File "/home/code/MindSpeed-MM/pretrain_vlm.py", line 159, in
[rank5]: pretrain(
[rank5]: File "/home/code/MindSpeed-MM/mindspeed_mm/training.py", line 186, in pretrain
[rank5]: model, optimizer, opt_param_scheduler = setup_model_and_optimizer(
[rank5]: File "/home/code/MindSpeed-MM/MindSpeed/mindspeed/core/training.py", line 543, in wrapper
[rank5]: model, optimizer, opt_param_scheduler = setup_model_and_optimizer(*args, **kwargs)
[rank5]: File "/home/code/MindSpeed-MM/megatron/training/training.py", line 544, in setup_model_and_optimizer
[rank5]: optimizer = get_megatron_optimizer(config, model, no_wd_decay_cond,
[rank5]: File "/home/code/MindSpeed-MM/megatron/core/optimizer/init.py", line 338, in get_megatron_optimizer
[rank5]: _get_megatron_optimizer_based_on_param_groups(
[rank5]: File "/home/code/MindSpeed-MM/megatron/core/optimizer/init.py", line 194, in _get_megatron_optimizer_based_on_param_groups
[rank5]: optimizer = Adam(
[rank5]: File "/home/code/MindSpeed-MM/MindSpeed/mindspeed/optimizer/adamw.py", line 95, in init
[rank5]: super(AdamW, self).init(params, defaults)
[rank5]: File "/work/model/miniconda3/envs/mindspeed2.1.0/lib/python3.10/site-packages/torch/optim/optimizer.py", line 372, in init
[rank5]: raise ValueError("optimizer got an empty parameter list")
[rank5]: ValueError: optimizer got an empty parameter list

@ZhihuaH

您好，因为本issue超过1周没有更新，现先关闭此issue，后续有问题，欢迎继续交流。

您好，我在finetune Qwen2.5VL时也想冻结LLM放开ViT。我按照上述在freeze里加了freeze_text_decoder=True，在model配置文件里将image projector的freeze设为了false。我用的PP=2去跑, vision:[32,0], llm:[12,16]，跑finetune时报错说learning_rate is None。是否是因为第2个PP上没有可以train的部分而报错，有什么解决方案吗？

[rank2]: Traceback (most recent call last):
[rank2]: File "/home/ma-user/work/MindSpeed-MM/pretrain_qwen2vl.py", line 163, in
[rank2]: pretrain(
[rank2]: File "/home/ma-user/work/MindSpeed-MM/mindspeed_mm/training.py", line 244, in pretrain
[rank2]: iteration, num_floating_point_operations_so_far = train(
[rank2]: File "/home/ma-user/work/MindSpeed-MM/mindspeed_mm/training.py", line 487, in train
[rank2]: report_memory_flag = training_log(
[rank2]: File "/home/ma-user/work/MindSpeed-MM/mindspeed_mm/training.py", line 923, in training_log
[rank2]: raise AssertionError
[rank2]: AssertionError
[rank3]: Traceback (most recent call last):
[rank3]: File "/home/ma-user/work/MindSpeed-MM/pretrain_qwen2vl.py", line 163, in
[rank3]: pretrain(
[rank3]: File "/home/ma-user/work/MindSpeed-MM/mindspeed_mm/training.py", line 244, in pretrain
[rank3]: iteration, num_floating_point_operations_so_far = train(
[rank3]: File "/home/ma-user/work/MindSpeed-MM/mindspeed_mm/training.py", line 487, in train
[rank3]: report_memory_flag = training_log(
[rank3]: File "/home/ma-user/work/MindSpeed-MM/mindspeed_mm/training.py", line 923, in training_log
[rank3]: raise AssertionError
[rank3]: AssertionError

@hx37790928

你好，该issue我这边先关闭了哈。如果添加上面提到的参数并不能解决问题，欢迎重新提一个新的issue，这样我们这边比较容易看见，其他用户搜索的时候也容易搜索到。