0
代码
代码
Issues
Pull Requests
讨论
Wiki
分析
0
ascend-robotascend-robot[Feature] qwen3.5 support new dcp convert and fused rmsnorm
2a44e12f创建于 3月7日历史提交
文件最后提交记录最后更新时间
converters
[Feature] qwen3.5 support new dcp convert and fused rmsnorm Co-authored-by: young256<liumingyang16@huawei.com> # message auto-generated for no-merge-commit merge: !2246 merge fsdp2_dev_qwen35_publish into master [Feature] qwen3.5 support new dcp convert and fused rmsnorm Created-by: young256 Commit-by: young256 Merged-by: ascend-robot Description: ## What this PR does / why we need it? Qwen3.5 needs to support new DCP weight conversion to reduce the vector operator overhead caused by weight transpose in the program. The mul vector operator caused by RMSNorm and RMSNormGate needs to be eliminated. ## Does this PR introduce any user-facing change? DCP weight conversion instructions have changed. ## How was this patch tested? The weight verification after weight conversion is valid. The fused operator replacement has completed precision alignment. See merge request: Ascend/MindSpeed-MM!22462 个月前
__init__.py
!1121 [Refactor] understanding model weight convert code restructure Merge pull request !1121 from pjgao/master 11 个月前
config.py
[feature] hetero pp ckpt converter for vl and omni model Co-authored-by: xiaoyue994<xiaoyuanhang@huawei.com> # message auto-generated for no-merge-commit merge: !2214 merge mm_ckpt_convert into master [feature] hetero pp ckpt converter for vl and omni model Created-by: xiaoyue994 Commit-by: xiaoyue994 Merged-by: ascend-robot Description: ## Motivation Add hetero-pp-ckpt-converter for hetero parallel when powering pipeline parallel. ## Modification Modify args check and pp converter func. ## Self-test (Optional) If modifications to this PR may cause/fix function/accuracy/performance DTSs/issues, a self-inspection record needs to be attached. ## BC-breaking (Optional) If there are compatibility issues, such as dependencies on cann/torch_npu versions, they need to be explained in the PR. ## Checklist **Before PR**: - [ ] The new code needs to comply with the Clean Code specification. - [ ] The PR content is self-checked, and the expression can be clear and the writing standardized **After PR**: - [ ] CLA has been signed and all committers have signed the CLA in this PR. - [ ] The ci-pipeline is passed, Code Check is passed. See merge request: Ascend/MindSpeed-MM!22143 个月前
hf_to_mm.py
[feature] hetero pp ckpt converter for vl and omni model Co-authored-by: xiaoyue994<xiaoyuanhang@huawei.com> # message auto-generated for no-merge-commit merge: !2214 merge mm_ckpt_convert into master [feature] hetero pp ckpt converter for vl and omni model Created-by: xiaoyue994 Commit-by: xiaoyue994 Merged-by: ascend-robot Description: ## Motivation Add hetero-pp-ckpt-converter for hetero parallel when powering pipeline parallel. ## Modification Modify args check and pp converter func. ## Self-test (Optional) If modifications to this PR may cause/fix function/accuracy/performance DTSs/issues, a self-inspection record needs to be attached. ## BC-breaking (Optional) If there are compatibility issues, such as dependencies on cann/torch_npu versions, they need to be explained in the PR. ## Checklist **Before PR**: - [ ] The new code needs to comply with the Clean Code specification. - [ ] The PR content is self-checked, and the expression can be clear and the writing standardized **After PR**: - [ ] CLA has been signed and all committers have signed the CLA in this PR. - [ ] The ci-pipeline is passed, Code Check is passed. See merge request: Ascend/MindSpeed-MM!22143 个月前
mm_to_hf.py
[feature] hetero pp ckpt converter for vl and omni model Co-authored-by: xiaoyue994<xiaoyuanhang@huawei.com> # message auto-generated for no-merge-commit merge: !2214 merge mm_ckpt_convert into master [feature] hetero pp ckpt converter for vl and omni model Created-by: xiaoyue994 Commit-by: xiaoyue994 Merged-by: ascend-robot Description: ## Motivation Add hetero-pp-ckpt-converter for hetero parallel when powering pipeline parallel. ## Modification Modify args check and pp converter func. ## Self-test (Optional) If modifications to this PR may cause/fix function/accuracy/performance DTSs/issues, a self-inspection record needs to be attached. ## BC-breaking (Optional) If there are compatibility issues, such as dependencies on cann/torch_npu versions, they need to be explained in the PR. ## Checklist **Before PR**: - [ ] The new code needs to comply with the Clean Code specification. - [ ] The PR content is self-checked, and the expression can be clear and the writing standardized **After PR**: - [ ] CLA has been signed and all committers have signed the CLA in this PR. - [ ] The ci-pipeline is passed, Code Check is passed. See merge request: Ascend/MindSpeed-MM!22143 个月前
operator.py
[mindspore] ckpt convert for qwen3vl between hf and megatron format Co-authored-by: chengpeng25<chengpeng9@huawei.com> # message auto-generated for no-merge-commit merge: !1765 merge dev_convert_ckpt into master [mindspore] ckpt convert for qwen3vl between hf and megatron format Created-by: chengpeng25 Commit-by: chengpeng25 Merged-by: ascend-robot Description: ## Motivation Ckpt converter between huggingface and megatron format for qwen3vl. ## Modification qwen3vl ckpt converter between hf and megatron, including Dense and moe models. ## Self-test (Optional) If modifications to this PR may cause/fix function/accuracy/performance DTSs/issues, a self-inspection record needs to be attached. ## BC-breaking (Optional) If there are compatibility issues, such as dependencies on cann/torch_npu versions, they need to be explained in the PR. ## Checklist **Before PR**: - [x] The new code needs to comply with the Clean Code specification. - [x] The PR content is self-checked, and the expression can be clear and the writing standardized **After PR**: - [x] CLA has been signed and all committers have signed the CLA in this PR. - [x] The ci-pipeline is passed, Code Check is passed. See merge request: Ascend/MindSpeed-MM!17656 个月前