| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
feat(pytorch): add DeepSeek4 fine-tuning template Co-authored-by: HanhuiChen<chenhanhui1@h-partners.com> # message auto-generated for no-merge-commit merge: !4436 merge dsv4 into master feat(pytorch): add DeepSeek4 fine-tuning template Created-by: HANHU1CHEN Commit-by: HanhuiChen Merged-by: ascend-robot Description: ## What this PR does / why we need it? Adds a fine-tuning template for the DeepSeek4 model series to support its specific prompt format, including thinking mode, tool calling (DSML format), and reasoning effort control. ## Does this PR introduce any user-facing change? Yes — users can now select --prompt-type deepseek4 to fine-tune DeepSeek4 models. Two new behaviors are also exposed: - --enable-thinking controls thinking vs chat mode - --reasoning-effort {max,high} inserts a max-effort instruction prefix; only valid when thinking is enabled - --drop-thinking controls whether reasoning content is kept in each turn ## How was this patch tested? Tested with byte-level alignment against the official encoding_dsv4 script. See merge request: Ascend/MindSpeed-LLM!4436 | 5 个月前 | |
| 1 年前 | ||
feat: support GLM-5.2 ShareIndex with custom VPP layouts Co-authored-by: LinShua<707894133@qq.com> # message auto-generated for no-merge-commit merge: !4961 merge master_share_index_vpp into master feat: support GLM-5.2 ShareIndex with custom VPP layouts Created-by: LinShua Commit-by: LinShua Merged-by: ascend-robot Description: Fixes #1782 ## What this PR does / why we need it? 1. Adds PP/VPP-aware global decoder-layer mapping for GLM-5.2 ShareIndex so each Share layer resolves the intended Compute source inside the same process-local PP/VPP chunk, and rejects groups split across chunk boundaries. 2. Reuses MindSpeed PipelineModelParallelLayoutFeature and PipelineParallelLayerLayout as the runtime layout parser, validator, layer-count provider, and layer-offset provider. --num-layer-list remains the legacy non-VPP uneven-PP mode, while --pipeline-model-parallel-layout independently describes custom PP/VPP placement; the two modes are now rejected when configured together. 3. Uses the MindSpeed layout offset when MindSpeed-LLM selects global-layer-dependent TransformerBlock behavior, including Dense/MoE, attention-pattern, and noop-layer decisions. 4. Persists pipeline_model_parallel_layout in checkpoint arguments, detects distribution-mode conflicts after checkpoint restoration, and extends HF-to-MCore and MCore-to-HF conversion with the same VPP-major/PP-major global-layer mapping, including uneven layouts and layout-defined uneven PP without VPP. 5. Keeps legacy num_layer_list, uniform VPP, and DualPipeV paths unchanged when no custom layout is configured, and updates the GLM-5.2 POC command to use the layout as the sole custom PP/VPP distribution argument. ## Does this PR introduce any user-facing change? Yes. --num-layer-list and --pipeline-model-parallel-layout are mutually exclusive alternatives. For PP=4/VPP=5 custom placement, use --pipeline-model-parallel-layout 'Ett|(tttt|)*18ttttL' without --num-layer-list. The legacy --num-layer-list "18,20,20,20" mode remains available for uneven PP without custom VPP layout. Quote the layout in conversion shell commands because it contains the shell pipe character |. ## How was this patch tested? 1. Ran the repository-configured pre-commit hooks on all 10 PR files. Trailing-whitespace, end-of-file, added-large-file, merge-conflict, private-key, Ruff check/format, Codespell, Pylint, Bandit, and Typos checks passed. The repository-local Gitleaks hook could not run because the required ./gitleaks executable is absent; the same changed-file scope was rerun with only that hook skipped and all other hooks passed. 2. Ran Python byte-code compilation for all 9 changed Python files successfully. 3. Ran lightweight Python assertions for conversion argument mutual exclusion, PP=4/VPP=5 layout mapping, layout-defined uneven PP without VPP, independent num_layer_list behavior, MLA layout offsets, and rejection of num_layer_list with VPP. 4. A functional pytest suite was not executed locally because the available bundled Python runtime does not provide pytest, torch, or transformers. 5. Force-updated the squashed branch; GitCode remote Git Hooks Checking passed. See merge request: Ascend/MindSpeed-LLM!4961 | 2 个月前 | |
feat: support GLM-5.2 ShareIndex with custom VPP layouts Co-authored-by: LinShua<707894133@qq.com> # message auto-generated for no-merge-commit merge: !4961 merge master_share_index_vpp into master feat: support GLM-5.2 ShareIndex with custom VPP layouts Created-by: LinShua Commit-by: LinShua Merged-by: ascend-robot Description: Fixes #1782 ## What this PR does / why we need it? 1. Adds PP/VPP-aware global decoder-layer mapping for GLM-5.2 ShareIndex so each Share layer resolves the intended Compute source inside the same process-local PP/VPP chunk, and rejects groups split across chunk boundaries. 2. Reuses MindSpeed PipelineModelParallelLayoutFeature and PipelineParallelLayerLayout as the runtime layout parser, validator, layer-count provider, and layer-offset provider. --num-layer-list remains the legacy non-VPP uneven-PP mode, while --pipeline-model-parallel-layout independently describes custom PP/VPP placement; the two modes are now rejected when configured together. 3. Uses the MindSpeed layout offset when MindSpeed-LLM selects global-layer-dependent TransformerBlock behavior, including Dense/MoE, attention-pattern, and noop-layer decisions. 4. Persists pipeline_model_parallel_layout in checkpoint arguments, detects distribution-mode conflicts after checkpoint restoration, and extends HF-to-MCore and MCore-to-HF conversion with the same VPP-major/PP-major global-layer mapping, including uneven layouts and layout-defined uneven PP without VPP. 5. Keeps legacy num_layer_list, uniform VPP, and DualPipeV paths unchanged when no custom layout is configured, and updates the GLM-5.2 POC command to use the layout as the sole custom PP/VPP distribution argument. ## Does this PR introduce any user-facing change? Yes. --num-layer-list and --pipeline-model-parallel-layout are mutually exclusive alternatives. For PP=4/VPP=5 custom placement, use --pipeline-model-parallel-layout 'Ett|(tttt|)*18ttttL' without --num-layer-list. The legacy --num-layer-list "18,20,20,20" mode remains available for uneven PP without custom VPP layout. Quote the layout in conversion shell commands because it contains the shell pipe character |. ## How was this patch tested? 1. Ran the repository-configured pre-commit hooks on all 10 PR files. Trailing-whitespace, end-of-file, added-large-file, merge-conflict, private-key, Ruff check/format, Codespell, Pylint, Bandit, and Typos checks passed. The repository-local Gitleaks hook could not run because the required ./gitleaks executable is absent; the same changed-file scope was rerun with only that hook skipped and all other hooks passed. 2. Ran Python byte-code compilation for all 9 changed Python files successfully. 3. Ran lightweight Python assertions for conversion argument mutual exclusion, PP=4/VPP=5 layout mapping, layout-defined uneven PP without VPP, independent num_layer_list behavior, MLA layout offsets, and rejection of num_layer_list with VPP. 4. A functional pytest suite was not executed locally because the available bundled Python runtime does not provide pytest, torch, or transformers. 5. Force-updated the squashed branch; GitCode remote Git Hooks Checking passed. See merge request: Ascend/MindSpeed-LLM!4961 | 2 个月前 | |
feat: extend train_from_hf support for legacy scripts Co-authored-by: daixzh<daixuezheng@huawei.com> # message auto-generated for no-merge-commit merge: !4880 merge Train_from_HF into master feat: extend train_from_hf support for legacy scripts Created-by: daixzh Commit-by: daixzh Merged-by: ascend-robot Description: ## What this PR does / why we need it? Enabled automatic data preprocessing for all training scripts under examples and tests/poc. Enabled automatic weight conversion for training scripts of all models below 300B parameters under examples and tests/poc (excluding DeepSeek3, DeepSeek4, and LongCat). ## Does this PR introduce any user-facing change? The updated scripts integrate data preprocessing, weight conversion, and training into a single workflow, allowing users to launch training jobs with a single script. The default --tokenizer-type in the Llama2 training script has been changed from Llama2Tokenizer to PretrainedFormHF, and --tokenizer-model ${TOKENIZER_MODEL} has been changed to --tokenizer-name-or-path ${TOKENIZER_MODEL}. TOKENIZER_MODEL only needs to be configured to the directory containing the target model's tokenizer. ## How was this patch tested? This PR was validated through smoke testing. See merge request: Ascend/MindSpeed-LLM!4880 | 2 个月前 | |
[pytorch][feature][mindcluster] Integration of elastic-training-related callback code Co-authored-by: 李鸣沼<limingzhao3@h-partners.com> # message auto-generated for no-merge-commit merge: !3661 merge elastic_training into master [pytorch][feature][mindcluster] Integration of elastic-training-related callback code Created-by: lmztju Commit-by: lmztju;李鸣沼 Merged-by: ascend-robot Description: move takd elastic-training callback to MindSpeed-LLM See merge request: Ascend/MindSpeed-LLM!3661 | 11 个月前 | |
fix(pytorch): keep TFT rollback training logs consistent Co-authored-by: guoywang<wangguoyan6@h-partners.com> # message auto-generated for no-merge-commit merge: !5120 merge fix/master-tft-training-log-repair into master fix(pytorch): keep TFT rollback training logs consistent Created-by: guoywang Commit-by: guoywang Merged-by: ascend-robot Description: ## What this PR does / why we need it? Fixes [#1909](https://gitcode.com/Ascend/MindSpeed-LLM/issues/1909) - The training log output rank makes one rollback replay decision and broadcasts it so all ranks take the same collective path. - Mark an iteration complete only after training_log() returns, both in normal training and rollback repair. A fault before completion leaves the step eligible for replay; a fault after output but before the marker may duplicate the loss. - Preserve the existing no-cache behavior: if the output rank has no cached loss, all ranks skip repair without an error. ## Does this PR introduce any user-facing change? NA ## How was this patch tested? Injected a TFT-recoverable exception immediately before args.iteration = iteration. Fast recovery completed successfully, and all ranks followed the same training-log repair branch, with no branch divergence observed in the logs. See merge request: Ascend/MindSpeed-LLM!5120 | 1 个月前 | |
feat(pytorch): integrate DeepSeek V4 compressed-attention context parallel Co-authored-by: Liz<lizhi166@huawei.com> # message auto-generated for no-merge-commit merge: !5017 merge stream-master-0902 into master feat(pytorch): integrate DeepSeek V4 compressed-attention context parallel Created-by: Liz_ Commit-by: Liz Merged-by: ascend-robot Description: ## What this PR does / why we need it? DS V4 CP adaptor ## Does this PR introduce any user-facing change? NAN ## How was this patch tested? https://wiki.huawei.com/domains/64729/wiki/343185/WIKI2026090512720782 See merge request: Ascend/MindSpeed-LLM!5017 | 1 个月前 |
| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
| 5 个月前 | ||
| 1 年前 | ||
| 2 个月前 | ||
| 2 个月前 | ||
| 2 个月前 | ||
| 11 个月前 | ||
| 1 个月前 | ||
| 1 个月前 |