| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
[Feature] Add the muon optimizer and adapt it for FSDP2. Co-authored-by: hanyyy<hanyue42@huawei.com> # message auto-generated for no-merge-commit merge: !1964 merge master into master [Feature] Add the muon optimizer and adapt it for FSDP2. Created-by: vasileone Commit-by: hanyyy Merged-by: ascend-robot Description: ## Motivation Added the Muon optimizer and enabled its compatibility with FSDP2. ## Modification - Added Muon optimizer implementation with FSDP2 compatibility. - Added patches for Megatron-core to enable zero-code-change injection. - Fixed the issue of Muon optimizer failing under FSDP2 context: sharded parameter handling, gradient sync, and step behavior. ## Self-test (Optional) If modifications to this PR may cause/fix function/accuracy/performance DTSs/issues, a self-inspection record needs to be attached. ## BC-breaking (Optional) If there are compatibility issues, such as dependencies on cann/torch_npu versions, they need to be explained in the PR. ## Checklist **Before PR**: - The new code needs to comply with the Clean Code specification. - The PR content is self-checked, and the expression can be clear and the writing standardized **After PR**: - CLA has been signed and all committers have signed the CLA in this PR. - The ci-pipeline is passed, Code Check is passed. See merge request: Ascend/MindSpeed-MM!1964 | 6 个月前 | |
[Feature] Add the muon optimizer and adapt it for FSDP2. Co-authored-by: hanyyy<hanyue42@huawei.com> # message auto-generated for no-merge-commit merge: !1964 merge master into master [Feature] Add the muon optimizer and adapt it for FSDP2. Created-by: vasileone Commit-by: hanyyy Merged-by: ascend-robot Description: ## Motivation Added the Muon optimizer and enabled its compatibility with FSDP2. ## Modification - Added Muon optimizer implementation with FSDP2 compatibility. - Added patches for Megatron-core to enable zero-code-change injection. - Fixed the issue of Muon optimizer failing under FSDP2 context: sharded parameter handling, gradient sync, and step behavior. ## Self-test (Optional) If modifications to this PR may cause/fix function/accuracy/performance DTSs/issues, a self-inspection record needs to be attached. ## BC-breaking (Optional) If there are compatibility issues, such as dependencies on cann/torch_npu versions, they need to be explained in the PR. ## Checklist **Before PR**: - The new code needs to comply with the Clean Code specification. - The PR content is self-checked, and the expression can be clear and the writing standardized **After PR**: - CLA has been signed and all committers have signed the CLA in this PR. - The ci-pipeline is passed, Code Check is passed. See merge request: Ascend/MindSpeed-MM!1964 | 6 个月前 |
| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
| 6 个月前 | ||
| 6 个月前 |