[Bug]: Qwen3.5 SFT packing模式下micro_batch_size>1时训练错误：“expected sequence of length 32 at dim 1 (got 57)”-MindSpeed-MM-AtomGit

已关闭

[Bug]: Qwen3.5 SFT packing模式下micro_batch_size>1时训练错误：“expected sequence of length 32 at dim 1 (got 57)” #437

JSCHEN创建于 5月30日关闭于 7月1日

JSCHEN

5月30日创建

环境信息

软硬件信息

操作系统：ubuntu22.04.5
昇腾硬件信息：910B2
CANN软件版本：8.5.2
torch： torch 2.7.1, torch_npu 2.7.1
安装的对应软件版本：MindSpeed-MM master： commit: 1afa7360f00576fbd0001368cb5e2d76e0ac3f92

模型：

Qwen3.5-9B

数据集：

modelscope download --dataset AI-ModelScope/alpaca-data-gpt4-chinese --local_dir /data/datasets/alpaca-data-gpt4-chinese
sed  '/<image>/d' /data/datasets/alpaca-data-gpt4-chinese/Alpaca_data_gpt4_zh.jsonl > /data/datasets/alpaca-data-gpt4-chinese/Alpaca_data_gpt4_zh_filter.jsonl

配置：

# 并行策略
parallel:
  fully_shard_parallel_size: auto
  fsdp_plan:
    apply_modules: # 如果要开prefetch的话，请不要随意修改apply_modules的顺序
      - model.visual
      - model.visual.blocks.{*}
      - model.language_model
      - model.language_model.embed_tokens
      - model.language_model.layers.{*}
      - lm_head
      - mtp
    param_dtype: bf16
    reduce_dtype: fp32
  ulysses_parallel_size: 1 # 开启 ulysses-cp 时, 请将 model 的 attn_implementation 设置为 flash_attention_2

### 数据相关配置
data:
  dataset_param:
    dataset_type: huggingface
    #数据集属性
    attr:
      formatting: alpaca
      # alpaca 格式化
      prompt: instruction
      query: input
      response: output
      # sharegpt 格式化
      #images: images
      #messages: conversations
      #role_tag: from
      #content_tag: value
      #user_tag: human
      #assistant_tag: gpt
      #tools: tools
      #observation_tag: observation
      #function_tag: function_call
      #system_tag: system

    # 数据预处理
    preprocess_parameters:
      model_name_or_path: /weight/Qwen3.5-9B # 替换为原始hf权重
      use_fast_tokenizer: true
      split_special_tokens: false
      image_max_pixels: 262144
      image_min_pixels: 1024
      video_max_pixels: 16384
      video_min_pixels: 0
      video_fps: 2.0
      video_maxlen: 64

    basic_parameters:
      cutoff_len: 16384
      template: qwen3_vl_nothink
      enable_thinking: false
      train_on_prompt: false
      mask_history: false
      dataset_dir: ./data
      formatting: alpaca
      dataset: /deeplink/datasets/alpaca-data-gpt4-chinese/Alpaca_data_gpt4_zh_filter.jsonl
      #dataset: /deeplink/datasets/NousResearch/hermes-function-calling-v1/func-calling.json
      cache_dir: /dev/shm
      overwrite_cache: false
      preprocessing_batch_size: 2000
      preprocessing_num_workers: 8
      max_samples: null
      packing: true
      neat_packing: false

  # 数据加载
  dataloader_param:
    pin_memory: true
    shuffle: false
    dataloader_mode: sampler
    drop_last: true
    sampler_type: BaseRandomBatchSampler
    num_workers: 8
    collate_param:
      model_name: qwen3vl
      ignore_pad_token_for_loss: true
    enable_preload: false

# 模型配置
model:
  model_id: qwen3_5
  model_name_or_path: /weight/Qwen3.5-9B
  trust_remote_code: true
  attn_implementation: flash_attention_2
  freeze:
    - model.visual
  # 融合算子配置
  gdn_implementation: eager #triton # AscendC
  causal_conv1d_implementation: triton

# 优化特性配置
features:
  # loss 配置
  loss_cfg:
    loss_type: default   # If you want raw loss in model, loss_type can be set to "raw".
    router_aux_loss_coef: 0.0
  # 重计算配置
  recompute: true
  recompute_plan:
      apply_modules:
        - model.visual.blocks.{*}
        - model.language_model.layers.{*}
  # chunkloss配置
  enable_chunk_loss: true
  chunkloss_plan:
    apply_module: lm_head
    chunk_size: 1024
  # activation offload 配置
  enable_activation_offload: false
  activation_offload_plan:
    apply_modules:
     - model.visual.blocks.{*}
     - model.language_model.layers.{*}
  # chunkmbs配置
  enable_chunk_mbs: false
  chunkmbs_plan:
    apply_modules:
     - model.language_model.layers.{*}
    chunk_mbs: 2 # 这个表示的是chunk之后的micro batchsize
    batch_dim: 0
    chunk_arg_indexs: [0]
    chunk_kwarg_names: ["position_embeddings", "position_ids", "rope_deltas", "attention_mask"]

# 训练配置
training:
  micro_batch_size: 2
  gradient_accumulation_steps: 1
  seed: 42
  lr: 1.0e-5
  adam_epsilon: 1e-8
  adam_beta1: 0.9
  adam_beta2: 0.999
  lr_decay_style: cosine
  lr_warmup_ratio: 0.0
  weight_decay: 0
  train_iters: 30
  #train_epochs: 3
  clip_grad: 1.0
  init_model_with_meta_device: true
  optimizer: adamw
  adam_fused: true
  save_interval: 10000
  no_load_optim: true  # Do not load optimizer state; remove if loading is needed.
  no_load_rng: true  # Do not load RNG state; remove if loading is needed.
  no_save_optim: true  # Do not save optimizer state; remove if saving is needed.
  no_save_rng: true  # Do not save RNG state; remove if saving is needed.
  load: /weight/Qwen3.5-9B-dcp  # 替换为转换后的dcp权重
  #save: ./save_path
  use_deter_comp: false
  plugin:
    - mindspeed_mm/fsdp/models/qwen3_5
    - mindspeed_mm/fsdp/data/datasets/huggingface

# 工具配置
tools:
  profile:
    enable: false
    profile_type: static
    ranks: [0]
    static_param:
      level: level1
      with_stack: false
      with_memory: false
      record_shapes: false
      with_cpu: true
      save_path: ./profiling
      start_step: 10
      end_step: 11
      data_simplification: false
      aic_metrics_type: PipeUtilization
  memory_profile:
      enable: false
      start_step: 1
      end_step: 2
      save_path: ./memory_snapshot
      dump_ranks: [0]
      stacks: all
      max_entries: null
      mem_info: false

🐛 问题描述

packing模式下mbs只能配置为1，配置为2时data colloctor执行错误：

[rank4]: Traceback (most recent call last):
[rank4]:   File "/deeplink/jschen/MindSpeed-MM-A2-Master/mindspeed_mm/fsdp/train/trainer.py", line 367, in <module>
[rank4]:     trainer.train()
[rank4]:   File "/deeplink/jschen/MindSpeed-MM-A2-Master/mindspeed_mm/fsdp/train/trainer.py", line 349, in train
[rank4]:     self.trainer.train()
[rank4]:   File "/deeplink/jschen/MindSpeed-MM-A2-Master/mindspeed_mm/fsdp/train/train_engine.py", line 209, in train
[rank4]:     loss_dict = self.train_step(train_dataloader_iter)
[rank4]:                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank4]:   File "/deeplink/jschen/MindSpeed-MM-A2-Master/mindspeed_mm/fsdp/train/train_engine.py", line 133, in train_step
[rank4]:     batch_data = self.get_batch(train_dataloader_iter)
[rank4]:                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank4]:   File "/deeplink/jschen/MindSpeed-MM-A2-Master/mindspeed_mm/fsdp/train/train_engine.py", line 96, in get_batch
[rank4]:     batch = next(data_iterator)
[rank4]:             ^^^^^^^^^^^^^^^^^^^
[rank4]:   File "/deeplink/jschen/MindSpeed-MM-A2-Master/mindspeed_mm/fsdp/data/data_utils/utils.py", line 11, in _cyclic_iter
[rank4]:     for x in dl:
[rank4]:   File "/usr/local/python3.11.15/lib/python3.11/site-packages/torchdata/stateful_dataloader/stateful_dataloader.py", line 450, in __next__
[rank4]:     return super().__next__()
[rank4]:            ^^^^^^^^^^^^^^^^^^
[rank4]:   File "/usr/local/python3.11.15/lib/python3.11/site-packages/torch/utils/data/dataloader.py", line 733, in __next__
[rank4]:     data = self._next_data()
[rank4]:            ^^^^^^^^^^^^^^^^^
[rank4]:   File "/usr/local/python3.11.15/lib/python3.11/site-packages/torchdata/stateful_dataloader/stateful_dataloader.py", line 1456, in _next_data
[rank4]:     return self._process_data(data, worker_id, state_dict)
[rank4]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank4]:   File "/usr/local/python3.11.15/lib/python3.11/site-packages/torchdata/stateful_dataloader/stateful_dataloader.py", line 1543, in _process_data
[rank4]:     data.reraise()
[rank4]:   File "/usr/local/python3.11.15/lib/python3.11/site-packages/torch/_utils.py", line 750, in reraise
[rank4]:     raise exception
[rank4]: ValueError: Caught ValueError in DataLoader worker process 0.
[rank4]: Original Traceback (most recent call last):
[rank4]:   File "/usr/local/python3.11.15/lib/python3.11/site-packages/torchdata/stateful_dataloader/worker.py", line 242, in _worker_loop
[rank4]:     data = fetcher.fetch(index)  # type: ignore[union-attr]
[rank4]:            ^^^^^^^^^^^^^^^^^^^^
[rank4]:   File "/usr/local/python3.11.15/lib/python3.11/site-packages/torch/utils/data/_utils/fetch.py", line 55, in fetch
[rank4]:     return self.collate_fn(data)
[rank4]:            ^^^^^^^^^^^^^^^^^^^^^
[rank4]:   File "/deeplink/jschen/MindSpeed-MM-A2-Master/mindspeed_mm/fsdp/data/dataloader/data_collator.py", line 44, in __call__
[rank4]:     return self.data_collator(*args, **kwargs)
[rank4]:            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank4]:   File "/deeplink/jschen/MindSpeed-MM-A2-Master/mindspeed_mm/fsdp/data/data_utils/func_utils/collator.py", line 182, in __call__
[rank4]:     packed_postion_ids = torch.tensor(packed_position_ids_list, dtype=torch.long)
[rank4]:                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank4]: ValueError: expected sequence of length 16383 at dim 1 (got 16376)

欢迎加入社区，感谢您对社区的贡献 🎉!