| feat: Add ds v4 fine-tuning template for fsdp2 and fixed related issues Co-authored-by: Rostellaria<xiaoshaoan@huawei.com> # message auto-generated for no-merge-commit merge: !4964 merge master into master feat: Add ds v4 fine-tuning template for fsdp2 and fixed related issues Created-by: Rostellaria Commit-by: Rostellaria Merged-by: ascend-robot Description: [#1786](https://gitcode.com/Ascend/MindSpeed-LLM/issues/1786) ## What this PR does / why we need it? This PR aligns FSDP2 DeepSeek-V4 training precision with Megatron: it removes the two legacy ÷ep_size expert grad-division hooks that stacked on top of FSDPTurbo's native gradient_divide_factor (introduced by 4095b313/cfbeb48) and broke grad-norm parity by ~-2.8% whenever EP>1, and ports the mcore DeepSeek-V4 data template, indexer-loss definition/logging/backward scaling, and hc_head final-norm placement. ## Does this PR introduce any user-facing change? Yes in behavior but not in usage: no arguments or scripts change, yet expert-parallel (EP>1) runs now produce correct (larger) expert gradients and matching grad norms — previously over-divided results were silently wrong. ## How was this patch tested? Verified against mcore on a 4-layer/64-expert DSv4 model with 8 NPUs: step-by-step lm-loss/grad-norm parity at 16/50/2000 steps (0/2000 steps exceed the 2% tolerance on loss), plus EP=8/EP=1 A/B runs isolating the hooks (-2.8% vs -0.3% grad norm). Limitation: validated on NPU bf16 with eager attention (non-fused kernels); fused-path parity is not covered here. See merge request: Ascend/MindSpeed-LLM!4964 | 7 天前 |
| test: sink deepseek4-flash fsdp2 and deepseek-v3 gate cases to pipeline to shorten CI time, fix mamba3 tee misreport Co-authored-by: Rostellaria<xiaoshaoan@huawei.com> # message auto-generated for no-merge-commit merge: !4963 merge master into master test: sink deepseek4-flash fsdp2 and deepseek-v3 gate cases to pipeline to shorten CI time, fix mamba3 tee misreport Created-by: Rostellaria Commit-by: Rostellaria Merged-by: ascend-robot Description: [#1784](https://gitcode.com/Ascend/MindSpeed-LLM/issues/1784) ## What this PR does / why we need it? This PR sinks two redundant or long-running gate cases (pretrain_deepseek4_flash_1k_fsdp2, deepseek_v3_mcore_tp1_pp2_ep4) into the nightly pipeline to shorten gate CI time, as their guarded features are fully covered by remaining gate cases. It also fixes the mamba3 pipeline script, whose tee into a non-existent logs/ directory falsely failed the case with "Script has failed" in clean CI checkouts. ## Does this PR introduce any user-facing change? No. This PR only reassigns test cases between the gate and pipeline suites and updates the registrations in tests/README.md and tests/resource_record.md; no framework code or user-facing usage changes are involved. ## How was this patch tested? All changed scripts passed bash -n syntax checks, the gate table now matches the 11 remaining .sh files one-to-one, and mamba3 was re-run 3 times on 8 NPUs with loss/grad-norm matching baseline values. The moved cases are still auto-discovered and executed nightly by the pipeline runner. See merge request: Ascend/MindSpeed-LLM!4963 | 14 天前 |
| ci(torch): run st with pytest Co-authored-by: iansheng<shengjiayi@huawei.com> # message auto-generated for no-merge-commit merge: !4403 merge ci412 into master ci(torch): run st with pytest Created-by: iansheng Commit-by: iansheng Merged-by: ascend-robot Description: ## What this PR does / why we need it? Run st test suit with Pytest instead of Shell. ## Does this PR introduce any user-facing change? no ## How was this patch tested? Yes See merge request: Ascend/MindSpeed-LLM!4403 | 4 个月前 |
| test(megatron): CI run added pipeline testcase Co-authored-by: guihaowen666<guihaowen@huawei.com> # message auto-generated for no-merge-commit merge: !4670 merge feature/pipeline-case-gate into master test(megatron): CI run added pipeline testcase Created-by: guihaowen666 Commit-by: guihaowen666 Merged-by: ascend-robot Description: ## What this PR does / why we need it? When adding or modifying pipeline test cases, not only are UT and ST test cases executed, but the newly added or modified pipeline test cases are also tested. This prevents pipeline test cases that would cause merge failures from being merged. ## Does this PR introduce any user-facing change? The access control architecture is modified, but the service code is not modified. Therefore, the user plane is not affected. ## How was this patch tested? Updated the baseline of the pretrain_minimax_m2p7_4K_fsdp2 test case. This test case was tested in the CI gate. See merge request: Ascend/MindSpeed-LLM!4670 | 1 个月前 |
