0
代码
代码
Issues
Pull Requests
讨论
Wiki
分析
0
ascend-robotascend-robotfeat(pytorch): FSDP2 support GDN cp
84083190创建于 6 天前历史提交
文件最后提交记录最后更新时间
ring_context_parallel
feat(pytorch):ring cp support MLA/GQA Co-authored-by: mhh111<mahonghao1@huawei.com> # message auto-generated for no-merge-commit merge: !4412 merge ring0418 into master feat(pytorch):ring cp support MLA/GQA Created-by: mhh111 Commit-by: mhh111 Merged-by: ascend-robot Description: ## What this PR does / why we need it? ring cp support MLA/GQA ## Does this PR introduce any user-facing change? Please describe whether the PR will result in any user-facing usage changes. If there is related documentation, please specify its path. ## How was this patch tested? Please explain how to verify the correctness and effectiveness of this feature, as well as its usage constraints and limitations. See merge request: Ascend/MindSpeed-LLM!44121 个月前
ulysses_context_parallel
feat(pytorch): FSDP2 support GDN cp Co-authored-by: mhh111<mahonghao1@huawei.com> # message auto-generated for no-merge-commit merge: !4510 merge 0522_1 into master feat(pytorch): FSDP2 support GDN cp Created-by: mhh111 Commit-by: mhh111 Merged-by: ascend-robot Description: ## What this PR does / why we need it? Please describe the background and detailed changes of the PR. If it is a bugfix, please attach the related issue. ## Does this PR introduce any user-facing change? Please describe whether the PR will result in any user-facing usage changes. If there is related documentation, please specify its path. ## How was this patch tested? Please explain how to verify the correctness and effectiveness of this feature, as well as its usage constraints and limitations. See merge request: Ascend/MindSpeed-LLM!45106 天前
__init__.py
[pytorch][feature] fsdp2 context parallel feature Co-authored-by: mhh111<mahonghao1@huawei.com> # message auto-generated for no-merge-commit merge: !4090 merge master_fsdpcp into master [pytorch][feature] fsdp2 context parallel feature Created-by: mhh111 Commit-by: mhh111 Merged-by: ascend-robot Description: FSDP2 框架 cp并行的支持 See merge request: Ascend/MindSpeed-LLM!40904 个月前
context_parallel_functions.py
feat(pytorch):ring cp support MLA/GQA Co-authored-by: mhh111<mahonghao1@huawei.com> # message auto-generated for no-merge-commit merge: !4412 merge ring0418 into master feat(pytorch):ring cp support MLA/GQA Created-by: mhh111 Commit-by: mhh111 Merged-by: ascend-robot Description: ## What this PR does / why we need it? ring cp support MLA/GQA ## Does this PR introduce any user-facing change? Please describe whether the PR will result in any user-facing usage changes. If there is related documentation, please specify its path. ## How was this patch tested? Please explain how to verify the correctness and effectiveness of this feature, as well as its usage constraints and limitations. See merge request: Ascend/MindSpeed-LLM!44121 个月前
context_parallel_manager.py
feat(pytorch):ring cp support MLA/GQA Co-authored-by: mhh111<mahonghao1@huawei.com> # message auto-generated for no-merge-commit merge: !4412 merge ring0418 into master feat(pytorch):ring cp support MLA/GQA Created-by: mhh111 Commit-by: mhh111 Merged-by: ascend-robot Description: ## What this PR does / why we need it? ring cp support MLA/GQA ## Does this PR introduce any user-facing change? Please describe whether the PR will result in any user-facing usage changes. If there is related documentation, please specify its path. ## How was this patch tested? Please explain how to verify the correctness and effectiveness of this feature, as well as its usage constraints and limitations. See merge request: Ascend/MindSpeed-LLM!44121 个月前
context_parallel_mappings.py
feat(pytorch): FSDP2 support GDN cp Co-authored-by: mhh111<mahonghao1@huawei.com> # message auto-generated for no-merge-commit merge: !4510 merge 0522_1 into master feat(pytorch): FSDP2 support GDN cp Created-by: mhh111 Commit-by: mhh111 Merged-by: ascend-robot Description: ## What this PR does / why we need it? Please describe the background and detailed changes of the PR. If it is a bugfix, please attach the related issue. ## Does this PR introduce any user-facing change? Please describe whether the PR will result in any user-facing usage changes. If there is related documentation, please specify its path. ## How was this patch tested? Please explain how to verify the correctness and effectiveness of this feature, as well as its usage constraints and limitations. See merge request: Ascend/MindSpeed-LLM!45106 天前
gdn_context_parallel.py
feat(pytorch): FSDP2 support GDN cp Co-authored-by: mhh111<mahonghao1@huawei.com> # message auto-generated for no-merge-commit merge: !4510 merge 0522_1 into master feat(pytorch): FSDP2 support GDN cp Created-by: mhh111 Commit-by: mhh111 Merged-by: ascend-robot Description: ## What this PR does / why we need it? Please describe the background and detailed changes of the PR. If it is a bugfix, please attach the related issue. ## Does this PR introduce any user-facing change? Please describe whether the PR will result in any user-facing usage changes. If there is related documentation, please specify its path. ## How was this patch tested? Please explain how to verify the correctness and effectiveness of this feature, as well as its usage constraints and limitations. See merge request: Ascend/MindSpeed-LLM!45106 天前