fsdp2_dev

ascend-robot[Feature]Added performance optimization for the Self-Forcing

e60306de创建于 2025年12月1日历史提交

文件	最后提交记录	最后更新时间
attention.py	[Feature] Add Self-Forcing Co-authored-by: qq_31502577<liuqiyuan6@huawei.com> # message auto-generated for no-merge-commit merge: !1716 merge master into master [Feature] Add Self-Forcing Created-by: qq_31502577 Commit-by: qq_31502577 Merged-by: ascend-robot Description: ## Motivation 新增 Add Self-Forcing模型 ## Modification 1、增加Self-Forcing相关的原仓适配patch 2、增加Self-Forcing README ## Self-test (Optional) 1、根据README测试环境安装、训练、推理流程正常 2、12小时长跑对比精度，critic loss平均绝对误差0.0099，generator loss平均绝对误差0.014 - critic loss对比：  - generator loss：  ## BC-breaking (Optional) If there are compatibility issues, such as dependencies on cann/torch_npu versions, they need to be explained in the PR. ## Checklist **Before PR**: - [x] The new code needs to comply with the Clean Code specification. - [x] The PR content is self-checked, and the expression can be clear and the writing standardized **After PR**: - [x] CLA has been signed and all committers have signed the CLA in this PR. - [x] The ci-pipeline is passed, Code Check is passed. See merge request: Ascend/MindSpeed-MM!1716	7 个月前
inference.py	[Feature] Add Self-Forcing Co-authored-by: qq_31502577<liuqiyuan6@huawei.com> # message auto-generated for no-merge-commit merge: !1716 merge master into master [Feature] Add Self-Forcing Created-by: qq_31502577 Commit-by: qq_31502577 Merged-by: ascend-robot Description: ## Motivation 新增 Add Self-Forcing模型 ## Modification 1、增加Self-Forcing相关的原仓适配patch 2、增加Self-Forcing README ## Self-test (Optional) 1、根据README测试环境安装、训练、推理流程正常 2、12小时长跑对比精度，critic loss平均绝对误差0.0099，generator loss平均绝对误差0.014 - critic loss对比：  - generator loss：  ## BC-breaking (Optional) If there are compatibility issues, such as dependencies on cann/torch_npu versions, they need to be explained in the PR. ## Checklist **Before PR**: - [x] The new code needs to comply with the Clean Code specification. - [x] The PR content is self-checked, and the expression can be clear and the writing standardized **After PR**: - [x] CLA has been signed and all committers have signed the CLA in this PR. - [x] The ci-pipeline is passed, Code Check is passed. See merge request: Ascend/MindSpeed-MM!1716	7 个月前
patch.py	[Feature]Added performance optimization for the Self-Forcing Co-authored-by: qq_31502577<liuqiyuan6@huawei.com> # message auto-generated for no-merge-commit merge: !1832 merge master into master [Feature]Added performance optimization for the Self-Forcing Created-by: qq_31502577 Commit-by: qq_31502577 Merged-by: ascend-robot Description: ## Motivation 新增self forcing模型的性能优化点 ## Modification 1、使能rms_norm融合算子 2、将rope_param默认精度从float64修改为float32 ## Self-test (Optional) 1、经验证，修改后性能提升25% 2、性能优化后，精度符合精度标准  ## BC-breaking (Optional) If there are compatibility issues, such as dependencies on cann/torch_npu versions, they need to be explained in the PR. ## Checklist **Before PR**: - [ ] The new code needs to comply with the Clean Code specification. - [ ] The PR content is self-checked, and the expression can be clear and the writing standardized **After PR**: - [ ] CLA has been signed and all committers have signed the CLA in this PR. - [ ] The ci-pipeline is passed, Code Check is passed. See merge request: Ascend/MindSpeed-MM!1832	7 个月前
requirements.txt	[Feature] Add Self-Forcing Co-authored-by: qq_31502577<liuqiyuan6@huawei.com> # message auto-generated for no-merge-commit merge: !1716 merge master into master [Feature] Add Self-Forcing Created-by: qq_31502577 Commit-by: qq_31502577 Merged-by: ascend-robot Description: ## Motivation 新增 Add Self-Forcing模型 ## Modification 1、增加Self-Forcing相关的原仓适配patch 2、增加Self-Forcing README ## Self-test (Optional) 1、根据README测试环境安装、训练、推理流程正常 2、12小时长跑对比精度，critic loss平均绝对误差0.0099，generator loss平均绝对误差0.014 - critic loss对比：  - generator loss：  ## BC-breaking (Optional) If there are compatibility issues, such as dependencies on cann/torch_npu versions, they need to be explained in the PR. ## Checklist **Before PR**: - [x] The new code needs to comply with the Clean Code specification. - [x] The PR content is self-checked, and the expression can be clear and the writing standardized **After PR**: - [x] CLA has been signed and all committers have signed the CLA in this PR. - [x] The ci-pipeline is passed, Code Check is passed. See merge request: Ascend/MindSpeed-MM!1716	7 个月前
self_forcing_dmd.yaml	[Feature] Add Self-Forcing Co-authored-by: qq_31502577<liuqiyuan6@huawei.com> # message auto-generated for no-merge-commit merge: !1716 merge master into master [Feature] Add Self-Forcing Created-by: qq_31502577 Commit-by: qq_31502577 Merged-by: ascend-robot Description: ## Motivation 新增 Add Self-Forcing模型 ## Modification 1、增加Self-Forcing相关的原仓适配patch 2、增加Self-Forcing README ## Self-test (Optional) 1、根据README测试环境安装、训练、推理流程正常 2、12小时长跑对比精度，critic loss平均绝对误差0.0099，generator loss平均绝对误差0.014 - critic loss对比：  - generator loss：  ## BC-breaking (Optional) If there are compatibility issues, such as dependencies on cann/torch_npu versions, they need to be explained in the PR. ## Checklist **Before PR**: - [x] The new code needs to comply with the Clean Code specification. - [x] The PR content is self-checked, and the expression can be clear and the writing standardized **After PR**: - [x] CLA has been signed and all committers have signed the CLA in this PR. - [x] The ci-pipeline is passed, Code Check is passed. See merge request: Ascend/MindSpeed-MM!1716	7 个月前
train.py	[Feature] Add Self-Forcing Co-authored-by: qq_31502577<liuqiyuan6@huawei.com> # message auto-generated for no-merge-commit merge: !1716 merge master into master [Feature] Add Self-Forcing Created-by: qq_31502577 Commit-by: qq_31502577 Merged-by: ascend-robot Description: ## Motivation 新增 Add Self-Forcing模型 ## Modification 1、增加Self-Forcing相关的原仓适配patch 2、增加Self-Forcing README ## Self-test (Optional) 1、根据README测试环境安装、训练、推理流程正常 2、12小时长跑对比精度，critic loss平均绝对误差0.0099，generator loss平均绝对误差0.014 - critic loss对比：  - generator loss：  ## BC-breaking (Optional) If there are compatibility issues, such as dependencies on cann/torch_npu versions, they need to be explained in the PR. ## Checklist **Before PR**: - [x] The new code needs to comply with the Clean Code specification. - [x] The PR content is self-checked, and the expression can be clear and the writing standardized **After PR**: - [x] CLA has been signed and all committers have signed the CLA in this PR. - [x] The ci-pipeline is passed, Code Check is passed. See merge request: Ascend/MindSpeed-MM!1716	7 个月前