您好，感谢您的反馈，根据您反馈的信息您提供的数据格式是alpaca格式的，您可以直接使用数据转换脚本 https://gitcode.com/Ascend/MindSpeed-LLM/blob/master/examples/mcore/qwen3/data_convert_qwen3_pretrain.sh 进行转后使用。

您好，感谢您的反馈，根据您反馈的信息您提供的数据格式是alpaca格式的，您可以直接使用数据转换脚本 https://gitcode.com/Ascend/MindSpeed-LLM/blob/master/examples/mcore/qwen3/data_convert_qwen3_pretrain.sh 进行转后使用。

@LinShua

转换脚本如下：

source /usr/local/Ascend/ascend-toolkit/set_env.sh
mkdir ./dataset

python ./preprocess_data.py
--input /home/ai_train2/sft_1230_5class_dataset.json
--tokenizer-name-or-path /home/qwen3_8b
--tokenizer-type PretrainedFromHF
--handler-name GeneralPretrainHandler
--output-prefix ./dataset/enwiki
--json-keys text
--workers 4
--log-interval 1000

报错如下：

"""
Traceback (most recent call last):
File "/usr/local/lib/python3.10/site-packages/multiprocess/pool.py", line 125, in worker
result = (True, func(*args, **kwds))
File "/usr/local/lib/python3.10/site-packages/datasets/utils/py_utils.py", line 586, in _write_generator_to_queue
for i, result in enumerate(func(**kwargs)):
File "/usr/local/lib/python3.10/site-packages/datasets/arrow_dataset.py", line 3650, in _map_single
for i, example in iter_outputs(shard_iterable):
File "/usr/local/lib/python3.10/site-packages/datasets/arrow_dataset.py", line 3624, in iter_outputs
yield i, apply_function(example, i, offset=offset)
File "/usr/local/lib/python3.10/site-packages/datasets/arrow_dataset.py", line 3547, in apply_function
processed_inputs = function(*fn_args, *additional_args, **fn_kwargs)
File "/home/MindSpeed-Core-MS/MindSpeed-LLM/mindspeed_llm/tasks/preprocess/data_handler.py", line 273, in _filter
text = sample[key]
File "/usr/local/lib/python3.10/site-packages/datasets/formatting/formatting.py", line 282, in getitem
value = self.data[key]
KeyError: 'text'
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/home/MindSpeed-Core-MS/MindSpeed-LLM/./preprocess_data.py", line 370, in
main()
File "/home/MindSpeed-Core-MS/MindSpeed-LLM/./preprocess_data.py", line 334, in main
handler.serialize_to_disk()
File "/home/MindSpeed-Core-MS/MindSpeed-LLM/mindspeed_llm/tasks/preprocess/data_handler.py", line 240, in serialize_to_disk
self._serialize_to_disk(iteration_batch_size=iteration_batch_size)
File "/home/MindSpeed-Core-MS/MindSpeed-LLM/mindspeed_llm/tasks/preprocess/data_handler.py", line 168, in _serialize_to_disk
self.tokenized_dataset = self.get_tokenized_data()
File "/home/MindSpeed-Core-MS/MindSpeed-LLM/mindspeed_llm/tasks/preprocess/data_handler.py", line 72, in get_tokenized_data
return self.raw_datasets.map(self._filter, remove_columns=remove_columns, **proc_kwargs)
File "/usr/local/lib/python3.10/site-packages/datasets/arrow_dataset.py", line 560, in wrapper
out: Union["Dataset", "DatasetDict"] = func(self, *args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/datasets/arrow_dataset.py", line 3309, in map
for rank, done, content in iflatmap_unordered(
File "/usr/local/lib/python3.10/site-packages/datasets/utils/py_utils.py", line 626, in iflatmap_unordered
[async_result.get(timeout=0.05) for async_result in async_results]
File "/usr/local/lib/python3.10/site-packages/datasets/utils/py_utils.py", line 626, in
[async_result.get(timeout=0.05) for async_result in async_results]
File "/usr/local/lib/python3.10/site-packages/multiprocess/pool.py", line 774, in get
raise self._value
KeyError: 'text'
[ERROR] 2026-01-31-14:06:46 (PID:1230, Device:-1, RankID:-1) ERR99999 UNKNOWN applicaiton exception

您好，根据您跑的情况来看，建议换成 https://gitcode.com/Ascend/MindSpeed-LLM/blob/master/examples/mcore/qwen3/data_convert_qwen3_instruction.sh 该脚本来尝试使用

您好，根据您跑的情况来看，建议换成 https://gitcode.com/Ascend/MindSpeed-LLM/blob/master/examples/mcore/qwen3/data_convert_qwen3_instruction.sh 该脚本来尝试使用

@LinShua

模型转换如下：

export CUDA_DEVICE_MAX_CONNECTIONS=1
source /usr/local/Ascend/ascend-toolkit/set_env.sh

python convert_ckpt_v2.py
--load-model-type hf
--save-model-type mg
--target-tensor-parallel-size 1
--target-pipeline-parallel-size 4
--load-dir /home/qwen3_8b/
--save-dir /home/qwen3_mcore/
--model-type-hf qwen3

数据集转换如下：

source /usr/local/Ascend/ascend-toolkit/set_env.sh
mkdir ./finetune_dataset

python ./preprocess_data.py
--input /home/ai_train2/sft_1230_5class_dataset.json
--tokenizer-name-or-path /home/qwen3_8b
--output-prefix ./finetune_dataset/alpaca
--handler-name AlpacaStyleInstructionHandler
--tokenizer-type PretrainedFromHF
--workers 4
--log-interval 1000
--enable-thinking false
--prompt-type qwen3

数据集文件如下：

-rw-r--r-- 1 root root 54358572 Feb 2 10:33 alpaca_packed_attention_mask_document.bin
-rw-r--r-- 1 root root 586682 Feb 2 10:33 alpaca_packed_attention_mask_document.idx
-rw-r--r-- 1 root root 54358572 Feb 2 10:33 alpaca_packed_input_ids_document.bin
-rw-r--r-- 1 root root 586682 Feb 2 10:33 alpaca_packed_input_ids_document.idx
-rw-r--r-- 1 root root 54358572 Feb 2 10:33 alpaca_packed_labels_document.bin
-rw-r--r-- 1 root root 586682 Feb 2 10:33 alpaca_packed_labels_document.idx

训练参数如下：

export HCCL_CONNECT_TIMEOUT=1800
export CUDA_DEVICE_MAX_CONNECTIONS=1

NPUS_PER_NODE=8
MASTER_ADDR=localhost
MASTER_PORT=6000
NNODES=1
NODE_RANK=0
WORLD_SIZE=8

please fill these path configurations

CKPT_LOAD_DIR="/home/qwen3_mcore"
CKPT_SAVE_DIR="/llm_model/ckpt/qwen3_8b"
DATA_PATH="/home/MindSpeed-Core-MS/MindSpeed-LLM/fintune_dataset" # "/home/MindSpeed-Core-MS/MindSpeed-LLM/dataset" /home/ai_train2/alpaca/alpaca_text_document
TOKENIZER_PATH="/home/qwen3_8b/"

TP=1
PP=4
SEQ_LENGTH=2048
TRAIN_ITERS=8000

DISTRIBUTED_ARGS="
--nproc_per_node $NPUS_PER_NODE
--nnodes $NNODES
--node_rank $NODE_RANK
--master_addr $MASTER_ADDR
--master_port $MASTER_PORT
"

OPTIMIZE_ARGS="
--use-flash-attn
--use-fused-rotary-pos-emb
--use-rotary-position-embeddings
--use-fused-swiglu
--use-fused-rmsnorm
--no-masked-softmax-fusion
--use-distributed-optimizer
"

TRAIN_ARGS="
--micro-batch-size 4
--global-batch-size 64
--lr 5e-5
--lr-decay-style cosine
--min-lr 5e-7
--weight-decay 1e-1
--lr-warmup-fraction 0.01
--attention-dropout 0.0
--init-method-std 0.01
--hidden-dropout 0.0
--clip-grad 1.0
--adam-beta1 0.9
--adam-beta2 0.95
--initial-loss-scale 4096
--seed 42
--bf16
--train-iters ${TRAIN_ITERS}
--seq-length ${SEQ_LENGTH}
--no-shared-storage
"

MODEL_PARALLEL_ARGS="
--tensor-model-parallel-size ${TP}
--pipeline-model-parallel-size ${PP}
"

GPT_ARGS="
--use-mcore-models
--spec mindspeed_llm.tasks.models.spec.qwen3_spec layer_spec
--kv-channels 128
--qk-layernorm
--tokenizer-name-or-path ${TOKENIZER_PATH}
--max-position-embeddings ${SEQ_LENGTH}
--num-layers 36
--hidden-size 4096
--ffn-hidden-size 12288
--num-attention-heads 32
--tokenizer-type PretrainedFromHF
--make-vocab-size-divisible-by 1
--padded-vocab-size 151936
--rotary-base 1000000
--untie-embeddings-and-output-weights
--disable-bias-linear
--position-embedding-type rope
--normalization RMSNorm
--swiglu
--attention-softmax-in-fp32
--no-gradient-accumulation-fusion
--group-query-attention
--num-query-groups 8
"

DATA_ARGS="
--data-path $DATA_PATH
--split 80,20,0
"

OUTPUT_ARGS="
--load ${CKPT_LOAD_DIR}
--save ${CKPT_SAVE_DIR}
--log-interval 10
--save-interval 50
--eval-interval 50
--eval-iters 10
--no-load-optim
--no-load-rng
"

TUNE_ARGS="
--finetune
--stage sft
--is-instruction-dataset
--tokenizer-not-use-fast
--prompt-type qwen3
--no-pad-to-seq-lengths
--lora-r 16
--lora-alpha 32
--lora-fusion
--lora-target-modules linear_qkv linear_proj linear_fc1 linear_fc2
"

torchrun $DISTRIBUTED_ARGS posttrain_gpt.py
$GPT_ARGS
$DATA_ARGS
$OUTPUT_ARGS
$OPTIMIZE_ARGS
$TRAIN_ARGS
$TUNE_ARGS
$MODEL_PARALLEL_ARGS
--distributed-backend nccl
| tee logs/tune_qwen3_8b_lora_ptd.log

报错如下：

[rank4]: IndexError: list index out of range
[rank2]: Traceback (most recent call last):
[rank2]: File "/home/MindSpeed-Core-MS/MindSpeed-LLM/posttrain_gpt.py", line 12, in
[rank2]: launch()
[rank2]: File "/home/MindSpeed-Core-MS/MindSpeed-LLM/posttrain_gpt.py", line 7, in launch
[rank2]: trainer = AutoTrainer()
[rank2]: File "/home/MindSpeed-Core-MS/MindSpeed-LLM/mindspeed_llm/tasks/posttrain/launcher.py", line 45, in init
[rank2]: self.trainer = get_trainer(self.args.stage)
[rank2]: File "/home/MindSpeed-Core-MS/MindSpeed-LLM/mindspeed_llm/tasks/posttrain/launcher.py", line 20, in get_trainer
[rank2]: return SFTTrainer()
[rank2]: File "/home/MindSpeed-Core-MS/MindSpeed-LLM/mindspeed_llm/tasks/posttrain/sft/sft_trainer.py", line 31, in init
[rank2]: super().init()
[rank2]: File "/home/MindSpeed-Core-MS/MindSpeed-LLM/mindspeed_llm/tasks/posttrain/base/base_trainer.py", line 55, in init
[rank2]: self.initialize()
[rank2]: File "/home/MindSpeed-Core-MS/MindSpeed-LLM/mindspeed_llm/tasks/posttrain/base/base_trainer.py", line 71, in initialize
[rank2]: self.train_args, self.test_data_iterator_list = build_train_args(
[rank2]: File "/home/MindSpeed-Core-MS/MindSpeed-LLM/mindspeed_llm/training/training.py", line 359, in build_train_args
[rank2]: = build_train_valid_test_data_iterators(
[rank2]: File "/home/MindSpeed-Core-MS/Megatron-LM/megatron/training/training.py", line 2413, in build_train_valid_test_data_iterators
[rank2]: build_train_valid_test_data_loaders(
[rank2]: File "/home/MindSpeed-Core-MS/MindSpeed-LLM/mindspeed_llm/training/training.py", line 208, in wrapper
[rank2]: train_dataloader, valid_dataloader, test_dataloader = fn(build_train_valid_test_datasets_provider)
[rank2]: File "/home/MindSpeed-Core-MS/Megatron-LM/megatron/training/training.py", line 2374, in build_train_valid_test_data_loaders
[rank2]: train_ds, valid_ds, test_ds = build_train_valid_test_datasets(
[rank2]: File "/home/MindSpeed-Core-MS/Megatron-LM/megatron/training/training.py", line 2344, in build_train_valid_test_datasets
[rank2]: return build_train_valid_test_datasets_provider(train_valid_test_num_samples)
[rank2]: File "/home/MindSpeed-Core-MS/MindSpeed-LLM/mindspeed_llm/tasks/posttrain/utils.py", line 62, in train_valid_test_datasets_provider
[rank2]: train_ds, valid_ds, test_ds = build_instruction_dataset(
[rank2]: File "/home/MindSpeed-Core-MS/MindSpeed-LLM/mindspeed_llm/tasks/preprocess/decoder_packed_mtf_dataset.py", line 54, in build_train_valid_test_datasets
[rank2]: all_train_datasets, all_valid_datasets, all_test_datasets = _build_train_valid_test_datasets(
[rank2]: File "/home/MindSpeed-Core-MS/MindSpeed-LLM/mindspeed_llm/tasks/preprocess/decoder_packed_mtf_dataset.py", line 81, in _build_train_valid_test_datasets
[rank2]: total_num_of_documents = len(list(packed_indexed_dataset.values())[0])
[rank2]: IndexError: list index out of range
[ERROR] 2026-02-02-10:34:49 (PID:4322, Device:5, RankID:-1) ERR99999 UNKNOWN applicaiton exception
[ERROR] 2026-02-02-10:34:49 (PID:4321, Device:4, RankID:-1) ERR99999 UNKNOWN applicaiton exception
[ERROR] 2026-02-02-10:34:49 (PID:4324, Device:7, RankID:-1) ERR99999 UNKNOWN applicaiton exception
[ERROR] 2026-02-02-10:34:49 (PID:4318, Device:1, RankID:-1) ERR99999 UNKNOWN applicaiton exception
[ERROR] 2026-02-02-10:34:50 (PID:4320, Device:3, RankID:-1) ERR99999 UNKNOWN applicaiton exception
[ERROR] 2026-02-02-10:34:50 (PID:4319, Device:2, RankID:-1) ERR99999 UNKNOWN applicaiton exception
[ERROR] 2026-02-02-10:34:50 (PID:4317, Device:0, RankID:-1) ERR99999 UNKNOWN applicaiton exception
[ERROR] 2026-02-02-10:34:50 (PID:4323, Device:6, RankID:-1) ERR99999 UNKNOWN applicaiton exception
W0202 10:34:54.962000 4248 site-packages/torch/distributed/elastic/multiprocessing/api.py:900] Sending process 4317 closing signal SIGTERM
W0202 10:34:54.963000 4248 site-packages/torch/distributed/elastic/multiprocessing/api.py:900] Sending process 4319 closing signal SIGTERM
W0202 10:34:54.963000 4248 site-packages/torch/distributed/elastic/multiprocessing/api.py:900] Sending process 4320 closing signal SIGTERM
W0202 10:34:54.964000 4248 site-packages/torch/distributed/elastic/multiprocessing/api.py:900] Sending process 4321 closing signal SIGTERM
W0202 10:34:54.965000 4248 site-packages/torch/distributed/elastic/multiprocessing/api.py:900] Sending process 4322 closing signal SIGTERM
W0202 10:34:54.965000 4248 site-packages/torch/distributed/elastic/multiprocessing/api.py:900] Sending process 4323 closing signal SIGTERM
E0202 10:34:55.259000 4248 site-packages/torch/distributed/elastic/multiprocessing/api.py:874] failed (exitcode: 1) local_rank: 1 (pid: 4318) of binary: /usr/local/bin/python3.10
Traceback (most recent call last):
File "/usr/local/bin/torchrun", line 8, in
sys.exit(main())
File "/usr/local/lib/python3.10/site-packages/torch/distributed/elastic/multiprocessing/errors/init.py", line 355, in wrapper
return f(*args, **kwargs)
File "/usr/local/lib/python3.10/site-packages/torch/distributed/run.py", line 892, in main
run(args)
File "/usr/local/lib/python3.10/site-packages/torch/distributed/run.py", line 883, in run
elastic_launch(
File "/usr/local/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 139, in call
return launch_agent(self._config, self._entrypoint, list(args))
File "/usr/local/lib/python3.10/site-packages/torch/distributed/launcher/api.py", line 270, in launch_agent
raise ChildFailedError(
torch.distributed.elastic.multiprocessing.errors.ChildFailedError:

posttrain_gpt.py FAILED

Failures:
[1]:
time : 2026-02-02_10:34:54
host : icp-icmx01-s-6u8ap
rank : 7 (local_rank: 7)
exitcode : 1 (pid: 4324)
error_file: <N/A>
traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html

Root Cause (first observed failure):
[0]:
time : 2026-02-02_10:34:54
host : icp-icmx01-s-6u8ap
rank : 1 (local_rank: 1)
exitcode : 1 (pid: 4318)
error_file: <N/A>
traceback : To enable traceback see: https://pytorch.org/docs/stable/elastic/errors.html

[ERROR] 2026-02-02-10:34:55 (PID:4248, Device:-1, RankID:-1) ERR99999 UNKNOWN applicaiton exception

您好，感谢您的反馈，请您按照 https://gitcode.com/Ascend/MindSpeed-LLM#版本说明 提供对应的配套软件版本信息，方便我们复现定位问题。

/label add pending

你的问题是微调脚本中，数据集路径少了一个前缀alpaca
DATA_PATH="/home/MindSpeed-Core-MS/MindSpeed-LLM/fintune_dataset" 改成DATA_PATH="/home/MindSpeed-Core-MS/MindSpeed-LLM/fintune_dataset/alpaca"应该就不会报list index out of range了

您好，list index out of range报错是由于DATA_PATH数据集路径填错导致，请依据
https://gitcode.com/Ascend/MindSpeed-LLM/blob/master/docs/pytorch/solutions/finetune/datasets/alpaca_dataset.md
进行数据集的转化

并根据
https://gitcode.com/Ascend/MindSpeed-LLM/blob/master/docs/pytorch/solutions/finetune/instruction_finetune.md
正确配置DATA_PATH路径

DATA_PATH：数据集路径。请注意实际数据预处理生成文件末尾会增加_input_ids_document等后缀，该参数填写到数据集的前缀即可。例如实际的数据集相对路径是./finetune_dataset/alpaca/alpaca_packed_input_ids_document.bin等，那么只需要填./finetune_dataset/alpaca/alpaca即可。

添加对应前缀。

希望我的回答有帮到您！

如果您目前没有其他问题，我们将根据流程关闭此 issue。
如后续有新的疑问或补充内容，欢迎随时重新打开本 issue，或另起一个新的 issue 进行提问。
感谢您的理解与支持！