文件最后提交记录最后更新时间
[Feature] fsdp2 qwen3vl 30B support lora finetune Co-authored-by: LKONE<wanglikai4@huawei.com> # message auto-generated for no-merge-commit merge: !2189 merge master into master [Feature] fsdp2 qwen3vl 30B support lora finetune Created-by: wanglikai1019 Commit-by: LKONE Merged-by: ascend-robot Description: ## Motivation During gradient recomputation in the backward pass, it is necessary to obtain the gradient function outside this block as the entry point for the backward pass. Therefore, a patch is added to enable the gradient function of the embed_tokens layer, allowing LoRA fine-tuning to enter the gradient recomputation backward pass. Related history PR: https://gitee.com/ascend/MindSpeed-LLM/pulls/1498 ## Modification 1.Add a new LoRA fine-tuning script for 30B models 2.Add parsing code for LoRA parameters 3.Add a LoRA patch for the gradient recomputation scenario ## Self-test (Optional) If modifications to this PR may cause/fix function/accuracy/performance DTSs/issues, a self-inspection record needs to be attached. ## BC-breaking (Optional) If there are compatibility issues, such as dependencies on cann/torch_npu versions, they need to be explained in the PR. ## Checklist **Before PR**: - [ ] The new code needs to comply with the Clean Code specification. - [ ] The PR content is self-checked, and the expression can be clear and the writing standardized **After PR**: - [ ] CLA has been signed and all committers have signed the CLA in this PR. - [ ] The ci-pipeline is passed, Code Check is passed. See merge request: Ascend/MindSpeed-MM!21893 个月前
feat(torch):Support text-only pretraining Co-authored-by: yaoyaoxu<xuyaoyao.824404@huawei.com> # message auto-generated for no-merge-commit merge: !2261 merge pretrain_data_preprocess into master feat(torch):Support text-only pretraining Created-by: yaoyaoxu Commit-by: yaoyaoxu Merged-by: ascend-robot Description: ## What this PR does / why we need it? 1.支持纯fsdp的纯文本预训练 2.支持megatron+fsdp双后端的纯文本预训练 3.提供预训练特性文档 ## Does this PR introduce any user-facing change? Please describe whether the PR will result in any user-facing usage changes. If there is related documentation, please specify its path. ## How was this patch tested? Please explain how to verify the correctness and effectiveness of this feature, as well as its usage constraints and limitations. See merge request: Ascend/MindSpeed-MM!22612 个月前
[Bugfix] bugfix for clip grad & empty ep Co-authored-by: htwang<wanghaitao60@huawei.com> # message auto-generated for no-merge-commit merge: !2382 merge 26.0.0 into 26.0.0 [Bugfix] bugfix for clip grad & empty ep Created-by: htwang Commit-by: htwang Merged-by: ascend-robot Description: ## What this PR does / why we need it? 1、EP使能时,当部分ep rank没有收到tokens时,保持空运算,防止专家参数失去梯度 2、修复不开EP切clip grad norm大于0时,clip grad 计算错误的问题 ## Does this PR introduce any user-facing change? Please describe whether the PR will result in any user-facing usage changes. If there is related documentation, please specify its path. ## How was this patch tested? Please explain how to verify the correctness and effectiveness of this feature, as well as its usage constraints and limitations. See merge request: Ascend/MindSpeed-MM!23821 个月前
[Modify][mindspore] remove temporal patchs and update preprocess_dataset patch Co-authored-by: weixin_47897441<wuyouqi1@h-partners.com> # message auto-generated for no-merge-commit merge: !2100 merge master-0129 into master [Modify][mindspore] remove temporal patchs and update preprocess_dataset patch Created-by: weixin_47897441 Commit-by: weixin_47897441 Merged-by: ascend-robot Description: ## Motivation To optimize the patch management and ensure the normal launch of model, unnecessary patches need to be removed and existing patches need to be updated. ## Modification 1. Removed the unnecessary patches for qwen2vl_vit_model.py to clean up redundant code. 2. Updated the patch for the preprocess_dataset function in convert.py to ensure the normal launch of models. ## Self-test (Optional) - Verified that the model can be successfully launched after the patch adjustments, with no functional abnormalities in dataset preprocessing and model initialization. ![image.png](https://raw.gitcode.com/user-images/assets/7404510/9de18e82-35e3-46fb-ab30-a43f3eac6516/image.png 'image.png') ## BC-breaking (Optional) This PR only involves patch adjustments for specific files and does not introduce compatibility issues with CANN/torch_npu versions or other dependencies. ## Checklist **Before PR**: - [x] The new code needs to comply with the Clean Code specification. - [x] The PR content is self-checked, and the expression can be clear and the writing standardized **After PR**: - [x] CLA has been signed and all committers have signed the CLA in this PR. - [x] The ci-pipeline is passed, Code Check is passed. See merge request: Ascend/MindSpeed-MM!21003 个月前
cleancode Co-authored-by: liyingxuan<liyingxuan3@huawei.com> # message auto-generated for no-merge-commit merge: !2323 merge master into 26.0.0 cleancode Created-by: liyx616 Commit-by: liyingxuan Merged-by: ascend-robot Description: ## What this PR does / why we need it? cleancode整改 ## Does this PR introduce any user-facing change? cleancode整改 ## How was this patch tested? Please explain how to verify the correctness and effectiveness of this feature, as well as its usage constraints and limitations. See merge request: Ascend/MindSpeed-MM!23232 个月前
[Docs] Annotation Standardization Rectification Co-authored-by: LKONE<wanglikai4@huawei.com> # message auto-generated for no-merge-commit merge: !2010 merge master into master [Docs] Annotation Standardization Rectification Created-by: wanglikai1019 Commit-by: LKONE Merged-by: ascend-robot Description: ## Motivation Rectify the annotations for the code repository in accordance with the annotation specification requirements. ## Modification 1. Replace Chinese annotations with English ones; 2. Organize the environment variable documentation and add explanations in the startup scripts (currently implemented on Qwen3VL and Wan2.2); 3. Add annotations to the public dataset functions and public model classes of Wan2.2 and Qwen3VL; 4. Adjust the annotation format (e.g., number of indentations). ## Self-test (Optional) If modifications to this PR may cause/fix function/accuracy/performance DTSs/issues, a self-inspection record needs to be attached. ## BC-breaking (Optional) If there are compatibility issues, such as dependencies on cann/torch_npu versions, they need to be explained in the PR. ## Checklist **Before PR**: - [ ] The new code needs to comply with the Clean Code specification. - [ ] The PR content is self-checked, and the expression can be clear and the writing standardized **After PR**: - [ ] CLA has been signed and all committers have signed the CLA in this PR. - [ ] The ci-pipeline is passed, Code Check is passed. See merge request: Ascend/MindSpeed-MM!20104 个月前
[docs] update wan2.2 14b t2v lora finetune readme Co-authored-by: LKONE<wanglikai4@huawei.com> # message auto-generated for no-merge-commit merge: !2230 merge master into master [docs] update wan2.2 14b t2v lora finetune readme Created-by: wanglikai1019 Commit-by: LKONE Merged-by: ascend-robot Description: ## Motivation Please describe the motivation of this PR and the goal you want to achieve through this PR. ## Modification Please briefly describe what modification is made in this PR. ## Self-test (Optional) If modifications to this PR may cause/fix function/accuracy/performance DTSs/issues, a self-inspection record needs to be attached. ## BC-breaking (Optional) If there are compatibility issues, such as dependencies on cann/torch_npu versions, they need to be explained in the PR. ## Checklist **Before PR**: - [ ] The new code needs to comply with the Clean Code specification. - [ ] The PR content is self-checked, and the expression can be clear and the writing standardized **After PR**: - [ ] CLA has been signed and all committers have signed the CLA in this PR. - [ ] The ci-pipeline is passed, Code Check is passed. See merge request: Ascend/MindSpeed-MM!22302 个月前
[Feature] Hunyuanvideo_1.5 add pipeline of infer Co-authored-by: z_yaoxue<zhouyaoxue@huawei.com> # message auto-generated for no-merge-commit merge: !2243 merge hy15_infer_pipeline into master [Feature] Hunyuanvideo_1.5 add pipeline of infer Created-by: z_yaoxue Commit-by: z_yaoxue Merged-by: ascend-robot Description: ## What this PR does / why we need it? 混元视频1.5:添加推理pipeline ## Does this PR introduce any user-facing change? Please describe whether the PR will result in any user-facing usage changes. If there is related documentation, please specify its path. ## How was this patch tested? Please explain how to verify the correctness and effectiveness of this feature, as well as its usage constraints and limitations. See merge request: Ascend/MindSpeed-MM!22432 个月前
[featre] Qwen3vl-32b add PMCC feature Co-authored-by: cxiaolong<2845907121@qq.com> # message auto-generated for no-merge-commit merge: !2264 merge master into master [featre] Qwen3vl-32b add PMCC feature Created-by: cxiaolong Commit-by: cxiaolong Merged-by: ascend-robot Description: ## What this PR does / why we need it? PMCC(Privacy and Model Confidential Computing)是昇腾提供的一种隐私计算解决方案,用于保护模型训练过程中的模型权重和数据隐私。 当前 PMCC 特性已适配Qwen3VL数据集并在Qwen3VL-32B模型上进行了验证。 ## Does this PR introduce any user-facing change? Please describe whether the PR will result in any user-facing usage changes. If there is related documentation, please specify its path. ## How was this patch tested? Please explain how to verify the correctness and effectiveness of this feature, as well as its usage constraints and limitations. See merge request: Ascend/MindSpeed-MM!22642 个月前
feat(torch): hunyuanvideo 1.5 i2v adapt Co-authored-by: Miss_min<qiaoxiaomin@huawei.com> # message auto-generated for no-merge-commit merge: !2221 merge master into master feat(torch): hunyuanvideo 1.5 i2v adapt Created-by: Miss_min Commit-by: Miss_min Merged-by: ascend-robot Description: ## Motivation hunyuanvideo 1.5 i2v 适配 ## Modification Please briefly describe what modification is made in this PR. ## Self-test (Optional) If modifications to this PR may cause/fix function/accuracy/performance DTSs/issues, a self-inspection record needs to be attached. ## BC-breaking (Optional) If there are compatibility issues, such as dependencies on cann/torch_npu versions, they need to be explained in the PR. ## Checklist **Before PR**: - [ ] The new code needs to comply with the Clean Code specification. - [ ] The PR content is self-checked, and the expression can be clear and the writing standardized **After PR**: - [ ] CLA has been signed and all committers have signed the CLA in this PR. - [ ] The ci-pipeline is passed, Code Check is passed. See merge request: Ascend/MindSpeed-MM!22212 个月前
feat(torch): Squash merge fsdp2_dev into master. Co-authored-by: zs-derrick1<1434012475@qq.com> # message auto-generated for no-merge-commit merge: !2223 merge master into master feat(torch): Squash merge fsdp2_dev into master. Created-by: zs-derrick1 Commit-by: zs-derrick1 Merged-by: ascend-robot Description: ## Motivation Squash merge fsdp2_dev into master. ## Modification fsdp2_dev分支合并至master ## Self-test (Optional) If modifications to this PR may cause/fix function/accuracy/performance DTSs/issues, a self-inspection record needs to be attached. ## BC-breaking (Optional) If there are compatibility issues, such as dependencies on cann/torch_npu versions, they need to be explained in the PR. ## Checklist **Before PR**: - [x] The new code needs to comply with the Clean Code specification. - [x] The PR content is self-checked, and the expression can be clear and the writing standardized **After PR**: - [x] CLA has been signed and all committers have signed the CLA in this PR. - [x] The ci-pipeline is passed, Code Check is passed. See merge request: Ascend/MindSpeed-MM!22233 个月前
[feature] Support qwen3vl lora training based on fsdp2 Co-authored-by: suis-moi<zhushixiang@huawei.com> Co-authored-by: zhushixiang<1318709014@qq.com> # message auto-generated for no-merge-commit merge: !2061 merge master into master [feature] Support qwen3vl lora training based on fsdp2 Created-by: suis-moi Commit-by: suis-moi;zhushixiang Merged-by: ascend-robot Description: Support qwen3vl lora training based on fsdp2 and torch dcp format a. supoport qwen3vl lora training and lora weights convert, must enable fsdp2 and using torch_dcp fomat. b. support MoE module lora training and reuires Pert>=18.0.1 See merge request: Ascend/MindSpeed-MM!20614 个月前
[Bugfix] fix ep grad caluation and clip grad Co-authored-by: htwang<wanghaitao60@huawei.com> # message auto-generated for no-merge-commit merge: !2070 merge master into master [Bugfix] fix ep grad caluation and clip grad Created-by: htwang Commit-by: htwang Merged-by: ascend-robot Description: 1、修改fsdp2场景下ep场景下grad norm计算逻辑bug(moe参数在ep group上未累加) 2、修复clip grad时因dtype导致的报错 See merge request: Ascend/MindSpeed-MM!20704 个月前