| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
[bugfix] fix qwen3vl cp Co-authored-by: cxiaolong<2845907121@qq.com> # message auto-generated for no-merge-commit merge: !1960 merge master into master [bugfix] fix qwen3vl cp Created-by: cxiaolong Commit-by: cxiaolong Merged-by: ascend-robot Description: ## Motivation 1. fix qwen3vl cp; 2. add readme for attention usage. ## Modification Please briefly describe what modification is made in this PR. ## Self-test (Optional) If modifications to this PR may cause/fix function/accuracy/performance DTSs/issues, a self-inspection record needs to be attached. ## BC-breaking (Optional) If there are compatibility issues, such as dependencies on cann/torch_npu versions, they need to be explained in the PR. ## Checklist **Before PR**: - [x] The new code needs to comply with the Clean Code specification. - [x] The PR content is self-checked, and the expression can be clear and the writing standardized **After PR**: - [x] CLA has been signed and all committers have signed the CLA in this PR. - [x] The ci-pipeline is passed, Code Check is passed. See merge request: Ascend/MindSpeed-MM!1960 | 6 个月前 | |
[Feature]Add ring attention for qwen3vl Co-authored-by: zs-derrick1<1434012475@qq.com> # message auto-generated for no-merge-commit merge: !1895 merge ring into master [Feature]Add ring attention for qwen3vl Created-by: zs-derrick1 Commit-by: zs-derrick1 Merged-by: ascend-robot Description: ## Motivation Add ring attention for qwen3vl https://wiki.huawei.com/domains/149956/wiki/300791/WIKI202511048896138?title=_c3cd4b74 ## Modification 1、modeling_qwen3_vl.py && modules.py add ring and hybrid function. 2、transformers_model.py adds the functionality to split labels in ring and hybrid scenarios. 3、pretrain_transformers.py modifies the get_batch and forward_step functions to further optimize memory usage. ## Self-test (Optional) If modifications to this PR may cause/fix function/accuracy/performance DTSs/issues, a self-inspection record needs to be attached. ## BC-breaking (Optional) If there are compatibility issues, such as dependencies on cann/torch_npu versions, they need to be explained in the PR. ## Checklist **Before PR**: - [x] The new code needs to comply with the Clean Code specification. - [x] The PR content is self-checked, and the expression can be clear and the writing standardized **After PR**: - [x] CLA has been signed and all committers have signed the CLA in this PR. - [x] The ci-pipeline is passed, Code Check is passed. See merge request: Ascend/MindSpeed-MM!1895 | 6 个月前 | |
[Feature] transformers model support aux_loss Co-authored-by: liyingxuan<liyingxuan3@huawei.com> # message auto-generated for no-merge-commit merge: !1965 merge aux_loss into master [Feature] transformers model support aux_loss Created-by: liyx616 Commit-by: liyingxuan Merged-by: ascend-robot Description: ## Motivation transformers model support aux_loss. ## Modification transformers_model支持aux_loss，配置方式： 在model.json中，加入下面的字段 json "loss_cfg": { "router_aux_loss_coef": 0.1 } 其中该系数对应的loss计算为：lm_loss + router_aux_loss_coef * aux_loss 如果不配置该字段，默认不开启aux loss计算，通过该字段开启aux loss计算需要transformers模型本身支持aux loss计算，否则不生效。 遗留问题： 当前per-token-loss通过缩放模型梯度实现，会导致auxloss也被错误的缩放了一个系数，所以有兼容性问题。 ## Self-test (Optional) If modifications to this PR may cause/fix function/accuracy/performance DTSs/issues, a self-inspection record needs to be attached. ## BC-breaking (Optional) If there are compatibility issues, such as dependencies on cann/torch_npu versions, they need to be explained in the PR. ## Checklist **Before PR**: - [x] The new code needs to comply with the Clean Code specification. - [x] The PR content is self-checked, and the expression can be clear and the writing standardized **After PR**: - [x] CLA has been signed and all committers have signed the CLA in this PR. - [x] The ci-pipeline is passed, Code Check is passed. See merge request: Ascend/MindSpeed-MM!1965 | 6 个月前 | |
[bugfix] fix qwen3vl cp Co-authored-by: cxiaolong<2845907121@qq.com> # message auto-generated for no-merge-commit merge: !1960 merge master into master [bugfix] fix qwen3vl cp Created-by: cxiaolong Commit-by: cxiaolong Merged-by: ascend-robot Description: ## Motivation 1. fix qwen3vl cp; 2. add readme for attention usage. ## Modification Please briefly describe what modification is made in this PR. ## Self-test (Optional) If modifications to this PR may cause/fix function/accuracy/performance DTSs/issues, a self-inspection record needs to be attached. ## BC-breaking (Optional) If there are compatibility issues, such as dependencies on cann/torch_npu versions, they need to be explained in the PR. ## Checklist **Before PR**: - [x] The new code needs to comply with the Clean Code specification. - [x] The PR content is self-checked, and the expression can be clear and the writing standardized **After PR**: - [x] CLA has been signed and all committers have signed the CLA in this PR. - [x] The ci-pipeline is passed, Code Check is passed. See merge request: Ascend/MindSpeed-MM!1960 | 6 个月前 | |
[Modify] Refactor the fsdp2 model Co-authored-by: htwang<wanghaitao60@huawei.com> # message auto-generated for no-merge-commit merge: !1793 merge master into master [Modify] Refactor the fsdp2 model Created-by: htwang Commit-by: htwang Merged-by: ascend-robot Description: ## Motivation 1、修改modelzoo至modelhub 2、修改hf_src至transformers ## Modification 1、修改modelzoo至modelhub 2、修改hf_src至transformers ## Self-test (Optional) If modifications to this PR may cause/fix function/accuracy/performance DTSs/issues, a self-inspection record needs to be attached. ## BC-breaking (Optional) If there are compatibility issues, such as dependencies on cann/torch_npu versions, they need to be explained in the PR. ## Checklist **Before PR**: - [x] The new code needs to comply with the Clean Code specification. - [x] The PR content is self-checked, and the expression can be clear and the writing standardized **After PR**: - [x] CLA has been signed and all committers have signed the CLA in this PR. - [x] The ci-pipeline is passed, Code Check is passed. See merge request: Ascend/MindSpeed-MM!1793 | 7 个月前 | |
[Feature] transformers model support aux_loss Co-authored-by: liyingxuan<liyingxuan3@huawei.com> # message auto-generated for no-merge-commit merge: !1965 merge aux_loss into master [Feature] transformers model support aux_loss Created-by: liyx616 Commit-by: liyingxuan Merged-by: ascend-robot Description: ## Motivation transformers model support aux_loss. ## Modification transformers_model支持aux_loss，配置方式： 在model.json中，加入下面的字段 json "loss_cfg": { "router_aux_loss_coef": 0.1 } 其中该系数对应的loss计算为：lm_loss + router_aux_loss_coef * aux_loss 如果不配置该字段，默认不开启aux loss计算，通过该字段开启aux loss计算需要transformers模型本身支持aux loss计算，否则不生效。 遗留问题： 当前per-token-loss通过缩放模型梯度实现，会导致auxloss也被错误的缩放了一个系数，所以有兼容性问题。 ## Self-test (Optional) If modifications to this PR may cause/fix function/accuracy/performance DTSs/issues, a self-inspection record needs to be attached. ## BC-breaking (Optional) If there are compatibility issues, such as dependencies on cann/torch_npu versions, they need to be explained in the PR. ## Checklist **Before PR**: - [x] The new code needs to comply with the Clean Code specification. - [x] The PR content is self-checked, and the expression can be clear and the writing standardized **After PR**: - [x] CLA has been signed and all committers have signed the CLA in this PR. - [x] The ci-pipeline is passed, Code Check is passed. See merge request: Ascend/MindSpeed-MM!1965 | 6 个月前 | |
[bugfix]qwen3vl ulysses bugfix Co-authored-by: zs-derrick1<1434012475@qq.com> # message auto-generated for no-merge-commit merge: !1973 merge ring into master [bugfix]qwen3vl ulysses bugfix Created-by: zs-derrick1 Commit-by: zs-derrick1 Merged-by: ascend-robot Description: ## Motivation qwen3vl ulysses bugfix ## Modification qwen3vl ulysses bugfix ## Self-test (Optional) If modifications to this PR may cause/fix function/accuracy/performance DTSs/issues, a self-inspection record needs to be attached. ## BC-breaking (Optional) If there are compatibility issues, such as dependencies on cann/torch_npu versions, they need to be explained in the PR. ## Checklist **Before PR**: - [x] The new code needs to comply with the Clean Code specification. - [x] The PR content is self-checked, and the expression can be clear and the writing standardized **After PR**: - [x] CLA has been signed and all committers have signed the CLA in this PR. - [x] The ci-pipeline is passed, Code Check is passed. See merge request: Ascend/MindSpeed-MM!1973 | 6 个月前 |
| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
| 6 个月前 | ||
| 6 个月前 | ||
| 6 个月前 | ||
| 6 个月前 | ||
| 7 个月前 | ||
| 6 个月前 | ||
| 6 个月前 |