master

ascend-robot[Bugfix] Fix Megatron checkpoint saving&loading compatibility for torch_dcp format

96bc0a3b创建于 2025年12月6日历史提交

文件	最后提交记录	最后更新时间
__init__.py	!2237 refactor: 通信512pad重构 Merge pull request !2237 from wangruiqi/master	1 年前
buffer_pad.py	!2237 refactor: 通信512pad重构 Merge pull request !2237 from wangruiqi/master	1 年前
layerzero.py	!2331 feat: layerzero Merge pull request !2331 from 邓佳/master_layerzero_v2	1 年前
reset_bucket_group_order_feature.py	重排bucket group；在模型定义顺序与执行顺序混乱的情况下，可以明显提升开启overlap param gather后的吞吐 Co-authored-by: zjuCC<caizeyong1@huawei.com> # message auto-generated for no-merge-commit merge: merge feature/reset_bucket_group into master 重排bucket group；在模型定义顺序与执行顺序混乱的情况下，可以明显提升开启overlap param gather后的吞吐 Created-by: zjuCC Commit-by: zjuCC Merged-by: ascend-robot Description: 在大模型的训练过程中，模型的定义顺序和执行不一致是非常常见的问题，尤其是重定义常见的transformer组件或者使用多模态大模型时。这直接导致overlap-param-gather参数时会出现精度问题和计算通信串行的问题。目前megatron 0.12.1的方案解决了精度问题，然而不可避免地会出现计算和通信串行的问题。 具体实验设计参考https://clouddocs.huawei.com/wapp/doc/a5f778cd-fa07-49fc-9b23-3d14c4c67c85 See merge request: Ascend/MindSpeed!2840	7 个月前
torch_fully_sharded_data_parallel.py	[Bugfix] Fix Megatron checkpoint saving&loading compatibility for torch_dcp format Co-authored-by: 林明哲<linmingzhe3@huawei.com> # message auto-generated for no-merge-commit merge: !3077 merge fix1202 into master [Bugfix] Fix Megatron checkpoint saving&loading compatibility for torch_dcp format Created-by: LinMingZhe Commit-by: 林明哲 Merged-by: ascend-robot Description: Fix Megatron checkpoint saving&loading compatibility for torch_dcp format See merge request: Ascend/MindSpeed!3077	5 个月前