| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
docs:fix error Co-authored-by: z60119525<zhaotao68@h-partners.com> # message auto-generated for no-merge-commit merge: !2311 merge master into master docs:fix error Created-by: kali00 Commit-by: z60119525 Merged-by: ascend-robot Description: ## What this PR does / why we need it? Please describe the background and detailed changes of the PR. If it is a bugfix, please attach the related issue. ## Does this PR introduce any user-facing change? Please describe whether the PR will result in any user-facing usage changes. If there is related documentation, please specify its path. ## How was this patch tested? Please explain how to verify the correctness and effectiveness of this feature, as well as its usage constraints and limitations. See merge request: Ascend/MindSpeed-MM!2311 | 6 个月前 | |
docs: fix text pretraining configuration guide Co-authored-by: yeqm<yeqiangmao@huawei.com> # message auto-generated for no-merge-commit merge: !2974 merge docs/fix-text-pretrain-guide into master docs: fix text pretraining configuration guide Created-by: ye_qm Commit-by: yeqm Merged-by: ascend-robot Description: ## What this PR does / why we need it? The FSDP2 text pretraining example in docs/zh/features/pretrain.md does not match the current implementation. The existing example only sets attr.pretrain: true. However, the FSDP2 backend selects PretrainDatasetProcessor through basic_parameters.stage. Without stage: pretrain, the stage defaults to sft, causing text-only samples without a response field to be dropped and eventually resulting in StopIteration. This PR: - Adds the required basic_parameters.stage: pretrain setting for the FSDP2 backend. - Replaces the deprecated collate_param.model_name with collate_param.collator_id. - Clarifies the configuration differences between the FSDP2 and Megatron backends. - Corrects the packing description: text is packed within each preprocessing batch, so each batch must contain at least cutoff_len valid tokens to generate a training sample. - Improves the formatting and readability of the guide. Related issue: N/A. The problem was found while verifying the documented workflow against the current master implementation. ## Does this PR introduce any user-facing change? Yes. This PR corrects the user-facing configuration required to run text-only pretraining with the FSDP2 backend. Users should configure: ```yaml data: dataset_param: attr: formatting: alpaca prompt: text basic_parameters: stage: pretrain template: default dataloader_param: collate_param: collator_id: llm_pretrain See merge request: Ascend/MindSpeed-MM!2974 | 2 个月前 | |
style: pre-commit autofix cleancode (base check) Co-authored-by: liyingxuan<liyingxuan3@huawei.com> # message auto-generated for no-merge-commit merge: !2616 merge master into master style: pre-commit autofix cleancode (base check) Created-by: liyx616 Commit-by: liyingxuan Merged-by: ascend-robot Description: ## What this PR does / why we need it? Please describe the background and detailed changes of the PR. If it is a bugfix, please attach the related issue. ## Does this PR introduce any user-facing change? Please describe whether the PR will result in any user-facing usage changes. If there is related documentation, please specify its path. ## How was this patch tested? Please explain how to verify the correctness and effectiveness of this feature, as well as its usage constraints and limitations. See merge request: Ascend/MindSpeed-MM!2616 | 4 个月前 |