华为昇腾面向大规模分布式训练的多模态大模型套件，支撑多模态生成、多模态理解。

文件	最后提交记录	最后更新时间
.agents	feat: add skill - mindspeed-mm-fsdp2-model-only-migration Co-authored-by: hanyyy<hanyue42@huawei.com> # message auto-generated for no-merge-commit merge: !2595 merge model-migration into master feat: add skill - mindspeed-mm-fsdp2-model-only-migration Created-by: vasileone Commit-by: hanyyy Merged-by: ascend-robot Description: ## What this PR does / why we need it? 新增 mindspeed-mm-fsdp2-model-only-migration skill，用于指导仅模型开源、缺少训练 pipeline 的模型迁移到 MindSpeed-MM 插件式 FSDP2 后端。同时移除旧的 VLM 专用 skill，补充 FSDP2 框架地图 reference 和迁移流程示意图，提升模型迁移流程的泛化性和可执行性。 ## Does this PR introduce any user-facing change? 无 ## How was this patch tested? 文档/skill 更新，不涉及运行测试。 See merge request: Ascend/MindSpeed-MM!2595	4 个月前
.devcontainer	chore: add VS Code devcontainer configuration. Co-authored-by: zhangxubin<zhangxubin3@h-partners.com> # message auto-generated for no-merge-commit merge: !2911 merge master into master chore: add VS Code devcontainer configuration. Created-by: MoCuishle-M Commit-by: zhangxubin Merged-by: ascend-robot Description: ## What this PR does / why we need it? 当前仓库缺乏 Dev Container 配置（.devcontainer/devcontainer.json 及配套 Dockerfile），导致开发者在不同环境中需要手动配置依赖，降低了协作效率。为 MindSpeed-MM 新增 VS Code Dev Container 配置（.devcontainer/），一键启动标准化的容器开发环境。同时对dockerfile做一定适配简化。 ## Does this PR introduce any user-facing change? 可选使用，不影响已有代码。VS Code 打开仓库 → F1 → Reopen in Container 即可。 ## How was this patch tested? 在 arm64 NPU 服务器上 Reopen in Container 验证通过：镜像构建、依赖安装、pre-commit hooks、torch_npu.npu.is_available() 均正常。 See merge request: Ascend/MindSpeed-MM!2911	2 个月前
.gitcode	docs: modify pull request template Co-authored-by: liyingxuan<liyingxuan3@huawei.com> # message auto-generated for no-merge-commit merge: !2225 merge master into master docs: modify pull request template Created-by: liyx616 Commit-by: liyingxuan Merged-by: ascend-robot Description: ## What this PR does / why we need it? 按照最新的标准修改pr模板 ## Does this PR introduce any user-facing change? 不涉及代码功能 ## How was this patch tested? 不涉及 See merge request: Ascend/MindSpeed-MM!2225	7 个月前
UserGuide	docs:rename Ascend Extension for PyTorch Co-authored-by: z60112595<zhaotao68@h-partners.com> # message auto-generated for no-merge-commit merge: !2836 merge master into master docs:rename Ascend Extension for PyTorch Created-by: kali00 Commit-by: z60112595 Merged-by: ascend-robot Description: ## What this PR does / why we need it? https://gitcode.com/Ascend/MindSpeed-MM/issues/506 1、更新Ascend Extension for PyTorch名称为TorchNPU 2、补充快速入门环境最低配置 ## Does this PR introduce any user-facing change? Please describe whether the PR will result in any user-facing usage changes. If there is related documentation, please specify its path. ## How was this patch tested? Please explain how to verify the correctness and effectiveness of this feature, as well as its usage constraints and limitations. See merge request: Ascend/MindSpeed-MM!2836	3 个月前
bridge	修复 qwen25vl_7B mbridge 特性报错的问题 Co-authored-by: bonjour_rzq1998<ruanzeqi1@huawei.com> # message auto-generated for no-merge-commit merge: !2906 merge feature/fix-mbridge-qwen25vl into master 修复 qwen25vl_7B mbridge 特性报错的问题 Created-by: bonjour_rzq1998 Commit-by: bonjour_rzq1998 Merged-by: ascend-robot Description: What this PR does / why we need it? 1、修改bridge/models/qwen_vl/__init__.py中对于异常情况的容忍范围。 Does this PR introduce any user-facing change? 不涉及 How was this patch tested? 不涉及 See merge request: Ascend/MindSpeed-MM!2906	3 个月前
checkpoint	feat: Adapt for save_async feature Co-authored-by: LKONE<wanglikai4@huawei.com> # message auto-generated for no-merge-commit merge: !2825 merge MM-0712 into master feat: Adapt for save_async feature Created-by: wanglikai1019 Commit-by: LKONE Merged-by: ascend-robot Description: ## What this PR does / why we need it? 1. 校正dcp_to_hf函数的缩进语法问题，从八个空格改为四个空格； 2. 修改save_async配置说明，当前仅支持dcp权重异步保存，非dcp权重保存且同时开启了异步保存功能时，打印提示； 3. save_checkpointer.save方法调用时传入save_async参数。 ## Does this PR introduce any user-facing change? 无 ## How was this patch tested? 测试dcp权重异步保存功能是否正常。 See merge request: Ascend/MindSpeed-MM!2825	3 个月前
ci	feat: Adapt to the new PR pipeline Co-authored-by: zhangxubin<zhangxubin3@h-partners.com> # message auto-generated for no-merge-commit merge: !2881 merge master into master feat: Adapt to the new PR pipeline Created-by: MoCuishle-M Commit-by: zhangxubin Merged-by: ascend-robot Description: ## What this PR does / why we need it? ci_trigger.sh只需要维护本分支的相关信息，所以简化里的逻辑。 ## Does this PR introduce any user-facing change? 对最终用户无使用接口变化。本 PR 主要影响 CI 内部流程与 ST 测试脚本，不改变 MindSpeed-MM 对外暴露的训练/推理用法。 ## How was this patch tested? 无。通过UT、ST代表功能正常。 See merge request: Ascend/MindSpeed-MM!2881	3 个月前
docker	Docs: Add gawk installation steps Co-authored-by: zhangxubin<zhangxubin3@h-partners.com> # message auto-generated for no-merge-commit merge: !2919 merge master into master Docs: Add gawk installation steps Created-by: MoCuishle-M Commit-by: zhangxubin Merged-by: ascend-robot Description: ## What this PR does / why we need it? fla 的编译安装依赖OS上的gawk。在qwen3.5 3.6的readme里添加上相关说明。 dockerfile里添加gawk的安装。 ## Does this PR introduce any user-facing change? 无。 ## How was this patch tested? 无。不影响。 See merge request: Ascend/MindSpeed-MM!2919	2 个月前
docs	docs:update supported_models Co-authored-by: z60112595<zhaotao68@h-partners.com> # message auto-generated for no-merge-commit merge: !2872 merge master into master docs:update supported_models Created-by: kali00 Commit-by: z60112595 Merged-by: ascend-robot Description: ## What this PR does / why we need it? https://gitcode.com/Ascend/MindSpeed-MM/issues/517 更新支持的模型列表，删除下架模型 ## Does this PR introduce any user-facing change? Please describe whether the PR will result in any user-facing usage changes. If there is related documentation, please specify its path. ## How was this patch tested? Please explain how to verify the correctness and effectiveness of this feature, as well as its usage constraints and limitations. See merge request: Ascend/MindSpeed-MM!2872	3 个月前
examples	Docs: Add gawk installation steps Co-authored-by: zhangxubin<zhangxubin3@h-partners.com> # message auto-generated for no-merge-commit merge: !2919 merge master into master Docs: Add gawk installation steps Created-by: MoCuishle-M Commit-by: zhangxubin Merged-by: ascend-robot Description: ## What this PR does / why we need it? fla 的编译安装依赖OS上的gawk。在qwen3.5 3.6的readme里添加上相关说明。 dockerfile里添加gawk的安装。 ## Does this PR introduce any user-facing change? 无。 ## How was this patch tested? 无。不影响。 See merge request: Ascend/MindSpeed-MM!2919	2 个月前
mindspeed_mm	fix precommit issue	2 个月前
pre-commit	feat: add gitleaks secret detection Co-authored-by: zhangxubin<zhangxubin3@h-partners.com> # message auto-generated for no-merge-commit merge: !2817 merge master into master feat: add gitleaks secret detection Created-by: MoCuishle-M Commit-by: zhangxubin Merged-by: ascend-robot Description: ## What this PR does / why we need it? 1. 引入gitleaks二进制离线扫描工具 2. 新增pre-commit/.gitleaks.toml配置，继承官方全部检测规则 3. 配置pre-commit钩子，提交前自动扫描密钥硬编码风险。 ## Does this PR introduce any user-facing change? 无 ## How was this patch tested? PR流水线pre-commit检测新增敏感信息检测. See merge request: Ascend/MindSpeed-MM!2817	3 个月前
scripts	feat: for the FSDP2 backend README, always pull from the MindSpeed master branch. Co-authored-by: zhangxubin<zhangxubin3@h-partners.com> # message auto-generated for no-merge-commit merge: !2864 merge master into master feat: for the FSDP2 backend README, always pull from the MindSpeed master branch. Created-by: MoCuishle-M Commit-by: zhangxubin Merged-by: ascend-robot Description: ## What this PR does / why we need it? 使用 fsdp2 后端的模型，修改成每次都拉取最新的MindSpeed master 分支代码详细变更： 1. scripts/install.sh 新增分支参数支持 - 新增 -b, --msbranch BRANCH 参数，用于指定 MindSpeed 分支名 - 与 --msid 互斥，同时指定时报错退出 - 克隆时使用 git clone -b <branch> 一步完成分支切换；已存在目录时使用 git fetch origin <branch> && git checkout <branch> 保持最新 - 配置打印与安装日志根据参数显示 branch/commit/最新版 2. fsdp2 后端 README 统一改为拉取 master 分支 - 判断逻辑：脚本中存在 torchrun $DISTRIBUTED_ARGS mindspeed_mm/fsdp/train/trainer.py 即代表使用 fsdp2 后端 - 使用 bash scripts/install.sh 的 README 增加 --msbranch master 参数 - 使用手动 git clone 的 README 改为 git clone -b master - 存在 README_v1.md 的目录（qwen3vl、qwen3omni）只修改 v1，不动 README.md 涉及文件： - scripts/install.sh 、 examples/qwen3_5/README.md （首次提交） - examples/kimik2_5/README.md 、 examples/ltx2/README.md 、 examples/mimov2_5/README.md 、 examples/moss_tts/README.md 、 examples/qwen3_6/README.md 、 examples/qwen3omni/README_v1.md 、 examples/qwen3tts/README.md 、 examples/qwen3vl/README_v1.md 、 examples/step3_vl/README.md （第二次提交） ## Does this PR introduce any user-facing change? 是。使用 fsdp2 后端的用户安装流程变化： - 原方式： bash scripts/install.sh --msid eb10b92 或手动 git clone https://gitcode.com/Ascend/MindSpeed.git - 新方式： bash scripts/install.sh --msbranch master 或 git clone -b master https://gitcode.com/Ascend/MindSpeed.git 用户不再需要查询和维护 MindSpeed commit ID，每次安装自动拉取最新 master 分支代码。相关文档： - scripts/install.sh - 新增 -b/--msbranch 参数（ --msid 保留向后兼容） - examples/qwen3_5/README.md - examples/qwen3_6/README.md - examples/mimov2_5/README.md - examples/kimik2_5/README.md - examples/step3_vl/README.md - examples/qwen3tts/README.md - examples/moss_tts/README.md - examples/ltx2/README.md - examples/qwen3omni/README_v1.md - examples/qwen3vl/README_v1.md ## How was this patch tested? 1. 语法校验：通过 bash -n scripts/install.sh 验证脚语法正确 2. 互斥校验：同时指定 --msid 和 --msbranch 时正确报错退出 3. 参数解析：验证 -b master 和 --msbranch master 两种写法均可正确解析 4. 克隆逻辑：新目录场景使用 git clone -b ，已存在目录场景使用 fetch + checkout 使用约束与限制： - --msid 参数保留向后兼容，未修改的 README（bagel、hunyuanvideo_1.5、wan2.2、qwen3vl/README.md、mcore/qwen3_5 等）仍使用 commit ID 方式 - 分支名必须为远端存在的有效分支，否则 git fetch / git clone -b 会失败 - 离线环境（无法访问 gitcode.com）不适用此方式，仍需使用 --msid 配合本地缓存 See merge request: Ascend/MindSpeed-MM!2864	3 个月前
sources	更新微信公众号二维码图片 Co-authored-by: bonjour_rzq1998<ruanzeqi1@huawei.com> # message auto-generated for no-merge-commit merge: !2915 merge fix/MM_wechat_qrcodes-20260803 into master 更新微信公众号二维码图片 Created-by: bonjour_rzq1998 Commit-by: bonjour_rzq1998 Merged-by: ascend-robot Description: What this PR does / why we need it? Update sources/images/MM_wechat_qrcode.jpg with the latest QR code for the MindSpeed MM community group. Does this PR introduce any user-facing change? Yes. Users can scan the updated QR code in the README to join the latest community group. How was this patch tested? The image was resized to 757 × 757 pixels to match the existing format. The processed QR code was successfully decoded and verified to be scannable. See merge request: Ascend/MindSpeed-MM!2915	2 个月前
tests	feat:add A5 script for flux DanceGRPO Co-authored-by: zhangxubin<zhangxubin3@h-partners.com> # message auto-generated for no-merge-commit merge: !2904 merge master into master feat:add A5 script for flux DanceGRPO Created-by: MoCuishle-M Commit-by: zhangxubin Merged-by: ascend-robot Description: ## What this PR does / why we need it? 为flux DanceGRPO添加A5 性能测试脚本。 1. 添加一个用于A5性能测试的脚本 2. 添加异步保存图片的功能 3. 添加更多的训练信息打印，例如增加rollout time、sft time、E2E time打印。 4. tests/ut/tasks/dancegrpo/test_flux_grpo_trainer.py 增加缺少的参数 ## Does this PR introduce any user-facing change? 接口无变化。 ## How was this patch tested? 不改任何核心训练代码，无精度、性能影响。 See merge request: Ascend/MindSpeed-MM!2904	2 个月前
verl_plugin	docs: fix master doc ci Co-authored-by: liuxi_<liuxi75@huawei.com> # message auto-generated for no-merge-commit merge: !2651 merge doc-scan-fix into master docs: fix master doc ci Created-by: liuxi_ Commit-by: liuxi_ Merged-by: ascend-robot Description: ## What this PR does / why we need it? fix doc ci ## Does this PR introduce any user-facing change? fix doc ci ## How was this patch tested? fix doc ci See merge request: Ascend/MindSpeed-MM!2651	4 个月前
.clang-format	修改完善pre-commit工具配置 Co-authored-by: wujinyuan1<wujinyuan1@huawei.com> # message auto-generated for no-merge-commit merge: !2470 merge master into master 修改完善pre-commit工具配置 Created-by: wujinyuan1 Commit-by: wujinyuan1 Merged-by: ascend-robot Description: ## What this PR does / why we need it? pre-commit 工具规则配置修改：试运行阶段发现python bandit工具检测过严，超出昇腾编程规范。此次修改重点修改bandit规则。 ## Does this PR introduce any user-facing change? 门禁codecheck检测规则精简，提升代码合入效率 ## How was this patch tested? 代码扫描工具配置，不涉及代码仓功能 See merge request: Ascend/MindSpeed-MM!2470	5 个月前
.gitignore	chore: add VS Code devcontainer configuration. Co-authored-by: zhangxubin<zhangxubin3@h-partners.com> # message auto-generated for no-merge-commit merge: !2911 merge master into master chore: add VS Code devcontainer configuration. Created-by: MoCuishle-M Commit-by: zhangxubin Merged-by: ascend-robot Description: ## What this PR does / why we need it? 当前仓库缺乏 Dev Container 配置（.devcontainer/devcontainer.json 及配套 Dockerfile），导致开发者在不同环境中需要手动配置依赖，降低了协作效率。为 MindSpeed-MM 新增 VS Code Dev Container 配置（.devcontainer/），一键启动标准化的容器开发环境。同时对dockerfile做一定适配简化。 ## Does this PR introduce any user-facing change? 可选使用，不影响已有代码。VS Code 打开仓库 → F1 → Reopen in Container 即可。 ## How was this patch tested? 在 arm64 NPU 服务器上 Reopen in Container 验证通过：镜像构建、依赖安装、pre-commit hooks、torch_npu.npu.is_available() 均正常。 See merge request: Ascend/MindSpeed-MM!2911	2 个月前
.pre-commit-config.yaml	chore: add VS Code devcontainer configuration. Co-authored-by: zhangxubin<zhangxubin3@h-partners.com> # message auto-generated for no-merge-commit merge: !2911 merge master into master chore: add VS Code devcontainer configuration. Created-by: MoCuishle-M Commit-by: zhangxubin Merged-by: ascend-robot Description: ## What this PR does / why we need it? 当前仓库缺乏 Dev Container 配置（.devcontainer/devcontainer.json 及配套 Dockerfile），导致开发者在不同环境中需要手动配置依赖，降低了协作效率。为 MindSpeed-MM 新增 VS Code Dev Container 配置（.devcontainer/），一键启动标准化的容器开发环境。同时对dockerfile做一定适配简化。 ## Does this PR introduce any user-facing change? 可选使用，不影响已有代码。VS Code 打开仓库 → F1 → Reopen in Container 即可。 ## How was this patch tested? 在 arm64 NPU 服务器上 Reopen in Container 验证通过：镜像构建、依赖安装、pre-commit hooks、torch_npu.npu.is_available() 均正常。 See merge request: Ascend/MindSpeed-MM!2911	2 个月前
CONTRIBUTING.md	docs: 移除文档中 MindSpore 相关展示 Co-authored-by: liuxi_<liuxi75@huawei.com> # message auto-generated for no-merge-commit merge: !2670 merge docs-fix into master docs: 移除文档中 MindSpore 相关展示 Created-by: liuxi_ Commit-by: liuxi_ Merged-by: ascend-robot Description: ## What this PR does / why we need it? 移除文档中 MindSpore 相关展示(目录树/Profiler项),仅保留 PyTorch 移除出海描述 ## Does this PR introduce any user-facing change? 移除文档中 MindSpore 相关展示(目录树/Profiler项),仅保留 PyTorch 移除出海描述 ## How was this patch tested? read See merge request: Ascend/MindSpeed-MM!2670	4 个月前
LICENSE	style: pre-commit autofix cleancode (base check) Co-authored-by: liyingxuan<liyingxuan3@huawei.com> # message auto-generated for no-merge-commit merge: !2616 merge master into master style: pre-commit autofix cleancode (base check) Created-by: liyx616 Commit-by: liyingxuan Merged-by: ascend-robot Description: ## What this PR does / why we need it? Please describe the background and detailed changes of the PR. If it is a bugfix, please attach the related issue. ## Does this PR introduce any user-facing change? Please describe whether the PR will result in any user-facing usage changes. If there is related documentation, please specify its path. ## How was this patch tested? Please explain how to verify the correctness and effectiveness of this feature, as well as its usage constraints and limitations. See merge request: Ascend/MindSpeed-MM!2616	4 个月前
MANIFEST.in	!325 【测试】添加InternVL2-8B ST & build打包内容完善 Merge pull request !325 from 陆劲夫/master	1 年前
OWNERS	style: pre-commit autofix cleancode (base check) Co-authored-by: liyingxuan<liyingxuan3@huawei.com> # message auto-generated for no-merge-commit merge: !2616 merge master into master style: pre-commit autofix cleancode (base check) Created-by: liyx616 Commit-by: liyingxuan Merged-by: ascend-robot Description: ## What this PR does / why we need it? Please describe the background and detailed changes of the PR. If it is a bugfix, please attach the related issue. ## Does this PR introduce any user-facing change? Please describe whether the PR will result in any user-facing usage changes. If there is related documentation, please specify its path. ## How was this patch tested? Please explain how to verify the correctness and effectiveness of this feature, as well as its usage constraints and limitations. See merge request: Ascend/MindSpeed-MM!2616	4 个月前
README.md	docs: add Kimi-K3 release news	3 个月前
SECURITYNOTE.md	docs:rename Ascend Extension for PyTorch Co-authored-by: z60112595<zhaotao68@h-partners.com> # message auto-generated for no-merge-commit merge: !2836 merge master into master docs:rename Ascend Extension for PyTorch Created-by: kali00 Commit-by: z60112595 Merged-by: ascend-robot Description: ## What this PR does / why we need it? https://gitcode.com/Ascend/MindSpeed-MM/issues/506 1、更新Ascend Extension for PyTorch名称为TorchNPU 2、补充快速入门环境最低配置 ## Does this PR introduce any user-facing change? Please describe whether the PR will result in any user-facing usage changes. If there is related documentation, please specify its path. ## How was this patch tested? Please explain how to verify the correctness and effectiveness of this feature, as well as its usage constraints and limitations. See merge request: Ascend/MindSpeed-MM!2836	3 个月前
Third-Party Open Source Software Notice.txt	fix: KimiK3 use fused npu_rms_norm in KimiRMSNorm, fix chunk_kda. Co-authored-by: suis-moi<zhushixiang@huawei.com> # message auto-generated for no-merge-commit merge: !2899 merge master into master fix: KimiK3 use fused npu_rms_norm in KimiRMSNorm, fix chunk_kda. Created-by: suis-moi Commit-by: suis-moi Merged-by: ascend-robot Description: ## What this PR does / why we need it? 1、重构MOE模块 2、更新开源最新版代码 3、修复chunk kda.py 4、替换kimi RMSNORM为融合算子 ## Does this PR introduce any user-facing change? Please describe whether the PR will result in any user-facing usage changes. If there is related documentation, please specify its path. ## How was this patch tested? 本地已经验证通过 See merge request: Ascend/MindSpeed-MM!2899	2 个月前
evaluate_gen.py	style: pre-commit autofix cleancode (base check) Co-authored-by: liyingxuan<liyingxuan3@huawei.com> # message auto-generated for no-merge-commit merge: !2616 merge master into master style: pre-commit autofix cleancode (base check) Created-by: liyx616 Commit-by: liyingxuan Merged-by: ascend-robot Description: ## What this PR does / why we need it? Please describe the background and detailed changes of the PR. If it is a bugfix, please attach the related issue. ## Does this PR introduce any user-facing change? Please describe whether the PR will result in any user-facing usage changes. If there is related documentation, please specify its path. ## How was this patch tested? Please explain how to verify the correctness and effectiveness of this feature, as well as its usage constraints and limitations. See merge request: Ascend/MindSpeed-MM!2616	4 个月前
evaluate_vlm.py	style: pre-commit autofix cleancode (base check) Co-authored-by: liyingxuan<liyingxuan3@huawei.com> # message auto-generated for no-merge-commit merge: !2616 merge master into master style: pre-commit autofix cleancode (base check) Created-by: liyx616 Commit-by: liyingxuan Merged-by: ascend-robot Description: ## What this PR does / why we need it? Please describe the background and detailed changes of the PR. If it is a bugfix, please attach the related issue. ## Does this PR introduce any user-facing change? Please describe whether the PR will result in any user-facing usage changes. If there is related documentation, please specify its path. ## How was this patch tested? Please explain how to verify the correctness and effectiveness of this feature, as well as its usage constraints and limitations. See merge request: Ascend/MindSpeed-MM!2616	4 个月前
inference_sora.py	bugfix: 合法合规风险数据修复 Co-authored-by: yaoyaoxu<xuyaoyao.824404@huawei.com> # message auto-generated for no-merge-commit merge: !2763 merge codecheck_260702 into master bugfix: 合法合规风险数据修复 Created-by: yaoyaoxu Commit-by: yaoyaoxu Merged-by: ascend-robot Description: ## What this PR does / why we need it? 合法合规风险数据修复：补充缺少的Copyright ## Does this PR introduce any user-facing change? 不涉及用户界面修改 ## How was this patch tested? 通过已有ut和st See merge request: Ascend/MindSpeed-MM!2763	3 个月前
inference_vlm.py	style: pre-commit autofix cleancode (base check) Co-authored-by: liyingxuan<liyingxuan3@huawei.com> # message auto-generated for no-merge-commit merge: !2616 merge master into master style: pre-commit autofix cleancode (base check) Created-by: liyx616 Commit-by: liyingxuan Merged-by: ascend-robot Description: ## What this PR does / why we need it? Please describe the background and detailed changes of the PR. If it is a bugfix, please attach the related issue. ## Does this PR introduce any user-facing change? Please describe whether the PR will result in any user-facing usage changes. If there is related documentation, please specify its path. ## How was this patch tested? Please explain how to verify the correctness and effectiveness of this feature, as well as its usage constraints and limitations. See merge request: Ascend/MindSpeed-MM!2616	4 个月前
posttrain_flux_dancegrpo.py	[Bugfix] fix protobuf resource conflict bug Co-authored-by: LKONE<wanglikai4@huawei.com> # message auto-generated for no-merge-commit merge: !2343 merge master into master [Bugfix] fix protobuf resource conflict bug Created-by: wanglikai1019 Commit-by: LKONE Merged-by: ascend-robot Description: ## What this PR does / why we need it? 当环境中存在tensorflow时，直接拉起训练脚本，在import transformers时会因框架不兼容导致段错误，程序异常退出 ## Does this PR introduce any user-facing change? 无 ## How was this patch tested? 在装有tensorflow库的环境中拉起训练脚本，看是否能够正常运行 See merge request: Ascend/MindSpeed-MM!2343	6 个月前
posttrain_sora_dpo.py	style: pre-commit autofix cleancode (base check) Co-authored-by: liyingxuan<liyingxuan3@huawei.com> # message auto-generated for no-merge-commit merge: !2616 merge master into master style: pre-commit autofix cleancode (base check) Created-by: liyx616 Commit-by: liyingxuan Merged-by: ascend-robot Description: ## What this PR does / why we need it? Please describe the background and detailed changes of the PR. If it is a bugfix, please attach the related issue. ## Does this PR introduce any user-facing change? Please describe whether the PR will result in any user-facing usage changes. If there is related documentation, please specify its path. ## How was this patch tested? Please explain how to verify the correctness and effectiveness of this feature, as well as its usage constraints and limitations. See merge request: Ascend/MindSpeed-MM!2616	4 个月前
pretrain_omni.py	style: pre-commit autofix cleancode (base check) Co-authored-by: liyingxuan<liyingxuan3@huawei.com> # message auto-generated for no-merge-commit merge: !2616 merge master into master style: pre-commit autofix cleancode (base check) Created-by: liyx616 Commit-by: liyingxuan Merged-by: ascend-robot Description: ## What this PR does / why we need it? Please describe the background and detailed changes of the PR. If it is a bugfix, please attach the related issue. ## Does this PR introduce any user-facing change? Please describe whether the PR will result in any user-facing usage changes. If there is related documentation, please specify its path. ## How was this patch tested? Please explain how to verify the correctness and effectiveness of this feature, as well as its usage constraints and limitations. See merge request: Ascend/MindSpeed-MM!2616	4 个月前
pretrain_sora.py	style: pre-commit autofix cleancode (base check) Co-authored-by: liyingxuan<liyingxuan3@huawei.com> # message auto-generated for no-merge-commit merge: !2616 merge master into master style: pre-commit autofix cleancode (base check) Created-by: liyx616 Commit-by: liyingxuan Merged-by: ascend-robot Description: ## What this PR does / why we need it? Please describe the background and detailed changes of the PR. If it is a bugfix, please attach the related issue. ## Does this PR introduce any user-facing change? Please describe whether the PR will result in any user-facing usage changes. If there is related documentation, please specify its path. ## How was this patch tested? Please explain how to verify the correctness and effectiveness of this feature, as well as its usage constraints and limitations. See merge request: Ascend/MindSpeed-MM!2616	4 个月前
pretrain_transformers.py	[modify] modify the threshold for gc in qwen3_vl 30B Co-authored-by: ye_qm<yeqiangmao@huawei.com> # message auto-generated for no-merge-commit merge: !2593 merge fix-qwen3vl-gc into master [modify] modify the threshold for gc in qwen3_vl 30B Created-by: ye_qm Commit-by: ye_qm Merged-by: ascend-robot Description: ## What this PR does / why we need it? This PR adds an optional GC threshold setting for Qwen3-VL 30B LoRA fine-tuning. Background: During Qwen3-VL 30B training, Python GC behavior may introduce performance fluctuation in some runs. This PR allows the Qwen3-VL 30B LoRA training script to explicitly enable a tuned GC threshold before training starts. Changes: - Adds ENABLE_GC_THRESHOLD=1 to examples/qwen3vl/finetune_lora_qwen3vl_30B.sh. - Updates pretrain_transformers.py to read ENABLE_GC_THRESHOLD. - When ENABLE_GC_THRESHOLD is set to "1", pretrain_transformers.py applies gc.set_threshold(700, 10, 1000). - If the environment variable is not set, the default behavior is unchanged. ## Does this PR introduce any user-facing change? Yes, but it is limited to an optional environment variable. Users can enable the GC threshold behavior by setting: export ENABLE_GC_THRESHOLD=1 The Qwen3-VL 30B LoRA example script enables this variable by default. Other training scripts are not affected unless they also set ENABLE_GC_THRESHOLD=1. No public API is changed. Related script: examples/qwen3vl/finetune_lora_qwen3vl_30B.sh ## How was this patch tested? Please explain how to verify the correctness and effectiveness of this feature, as well as its usage constraints and limitations. See merge request: Ascend/MindSpeed-MM!2593	5 个月前
pretrain_vlm.py	style: pre-commit autofix cleancode (base check) Co-authored-by: liyingxuan<liyingxuan3@huawei.com> # message auto-generated for no-merge-commit merge: !2616 merge master into master style: pre-commit autofix cleancode (base check) Created-by: liyx616 Commit-by: liyingxuan Merged-by: ascend-robot Description: ## What this PR does / why we need it? Please describe the background and detailed changes of the PR. If it is a bugfix, please attach the related issue. ## Does this PR introduce any user-facing change? Please describe whether the PR will result in any user-facing usage changes. If there is related documentation, please specify its path. ## How was this patch tested? Please explain how to verify the correctness and effectiveness of this feature, as well as its usage constraints and limitations. See merge request: Ascend/MindSpeed-MM!2616	4 个月前
pyproject.toml	feat: upgrade diffusers Co-authored-by: htwang<wanghaitao60@huawei.com> # message auto-generated for no-merge-commit merge: !2766 merge master into master feat: upgrade diffusers Created-by: htwang Commit-by: htwang Merged-by: ascend-robot Description: ## What this PR does / why we need it? 升级diffusers版本到0.38.0 ## Does this PR introduce any user-facing change? 1、升级diffusers依赖版本 2、删掉日落模型部分代码 ## How was this patch tested? ST、UT看护验证 See merge request: Ascend/MindSpeed-MM!2766	3 个月前

简介

MindSpeed MM：面向大规模分布式训练的昇腾多模态大模型套件，支持业界主流多模态大模型训练，旨在为华为昇腾芯片提供端到端的多模态训练解决方案, 包含预置业界主流模型，数据工程，分布式训练及加速，预训练、微调、后训练、在线推理任务等特性。

未来规划

📅未来规划会动态刷新在MindSpeed MM RoadMap中，欢迎大家通过此链接进行互动并提出诉求

社区会议

MindSpeed系列TC及SIG会议安排请查看Ascend会议中心

加入我们

为了交流开发经验、分享使用心得、及时获取项目更新，我们创建了MindSpeed MM官方微信群。

无论你是正在使用这个项目，还是有奇思妙想，都欢迎加入👋

加入方式：

直接扫码加入微信交流群（二维码7天有效，定期更新，当前1群已达到扫码加入人数上限，可加入2群）
添加昇腾开源小助手，获取群链接，进入MindSpeed MM社区交流群

MindSpeed MM社区交流群

昇腾开源小助手

目录结构

├─bridge          # mbridge在线权重转换
├─checkpoint      # 离线权重转换工具
├─ci              # Continuous Integration
├─docs            # 项目文档目录
│  └─zh           # 中文文档目录
├─examples        # 预置模型，包括模型配置、数据集配置、训练脚本、推理脚本等文件
├─mindspeed_mm    # 核心代码目录
├─scripts         # 脚本目录
├─sources         # 图片视频目录
├─tests           # 测试代码目录
│  ├─st           # 系统测试用例
│  └─ut           # 单元测试用例
├─UserGuide       # 用户指南目录
└─verl_plugin     # verl插件模块

效果展示

文生视频： Wan 2.2 T2V

Prompt: Ultra HD, 4K, cinematic composition, low contrast ratio, low saturation, cool tone; The queen wears an iron crown and rides on the dragon over the city. She holds a big flag that shows:" MindSpeed MM".

文生视频： OpensoraPlan 1.5 T2V

Prompt: A fluffy white rabbit with soft, velvety fur and twitching pink nose sits curiously near a rustic wooden fence, surrounded by a lush garden of vibrant wildflowers and tall grasses swaying gently in the breeze. The rabbit's large, expressive eyes scan the environment, reflecting the golden hues of the setting sun. As it nibbles on a patch of clover, its ears perk up at the distant sound of chirping birds. The fence, weathered and covered in patches of moss, adds a charming, pastoral backdrop to this serene scene, capturing the essence of a peaceful countryside moment.

Prompt: A majestic Berlin tower stands tall against the night sky, its structure bathed in a mesmerizing array of vibrant lights, casting a kaleidoscope of colors across the cityscape. The tower's intricate architectural details are highlighted by the illumination, creating a stunning contrast against the deep indigo sky. As the camera pans upward, the lights shift, revealing a dynamic play of shadows and hues that dance across the tower's surface. The surrounding city lights twinkle in harmony, enhancing the tower's grandeur and creating a breathtaking visual symphony that captures the essence of Berlin's vibrant nightlife.

文生图：Qwen-Image -> 图片编辑 Flux.1-Kontext

Prompt for generation: A coffee shop entrance features a chalkboard sign reading "MindSpeed Coffee 😊 $2 per cup," with a neon light displaying "MindSpeed MM". Next to it hangs a poster showing a beautiful Chinese woman, and beneath the poster is written "Welcome to use MindSpeed MM". Ultra HD, 4K, cinematic composition. (Qwen-Image)

Prompt for edition: Change the decoration of the coffee shop to a modern style with white painting. (Flux.1-Kontext)

理解模型：Qwen2VL

Input image for both models:

Input text for both models: Please describe the image shortly

Qwen2VL推理结果: The image depicts a serene lakeside scene with a wooden dock extending into the calm waters. The dock is made of weathered wooden planks and leads to a small platform with a ladder, suggesting it is used for swimming or diving. The lake is surrounded by lush green forests and mountains in the background, creating a picturesque and tranquil setting. The sky is overcast, adding to the calm and peaceful atmosphere of the scene.

Input text for Qwen2VL: 请用中文简短描述这张照片

Qwen2VL推理结果: 这张图片展示了一座木制码头延伸到平静的湖面上，背景是连绵的山脉和茂密的森林。天空多云，整体色调偏冷，给人一种宁静和自然的感觉。

版本说明

MindSpeed MM支持Atlas 800T A2等昇腾训练硬件形态，软件版本配套表如下：

MindSpeed MM版本	MindSpeed版本	Megatron版本	PyTorch版本	TorchNPU版本	CANN版本	Python版本
master（在研版本）	master（在研版本）	Core 0.12.1	2.7.1	在研版本	在研版本	Python3.10
26.0.0（商用）	26.0.0_core_r0.12.1	Core 0.12.1	2.7.1	26.0.0	9.0.0	Python3.10
2.3.0（商用）	2.3.0_core_r0.12.1	Core 0.12.1	2.6.0, 2.7.1	7.3.0	8.5.0	Python3.10
2.2.0（商用）	2.2.0_core_r0.12.1	Core 0.12.1	2.6.0, 2.7.1	7.2.0	8.3.RC1	Python3.10
2.1.0（商用）	2.1.0_core_r0.8.0	Core 0.8.0	2.1.0, 2.6.0	7.1.0	8.2.RC1	Python3.8, Python3.10
2.0.0（商用）	2.0.0_core_r0.8.0	Core 0.8.0	2.1.0	7.0.0	8.1.RC1	Python3.8, Python3.10
1.0.0（商用）	1.0.0_core_r0.6.0	Core 0.6.0	2.1.0	6.0.0	8.0.0	Python3.8, Python3.10

Note

“在研版本”指当前正处于开发迭代中的版本，由于该版本的功能仍处于持续迭代与优化阶段，其配套依赖项即使采用已发布的商用版本，仍可能存在兼容性风险或运行不稳定性，如需稳定使用，建议优先使用已正式发布的商用版本。

更多详情请参考版本配套表。

安装

MindSpeed MM具体的安装请参考安装指南。当前qwen3vl、wan2.2模型已支持一键安装，一键安装使用说明详见一键安装使用说明。

快速上手

MindSpeed MM将以Qwen2.5-VL-3B和Wan2.1-T2V-1.3B模型为例，引导开发者快速上手预置模型在昇腾NPU上的高效运行。具体的操作请参考快速上手。

特性/模型介绍

已支持特性概览

模型 \ 特性	TP	TP-SP	VPP	PP	CP	Distributed Optimizer	Recomputation	LoRA	RL	FSDP2
Magistral-Small-2509							✔	✔		✔
InternVL3.5-30B							✔			✔
Qwen3-VL-8B							✔			✔
Qwen3-VL-30B							✔			✔
Wan2.2					CP (Ulysses)		✔			✔
OpenSoraPlan1.5-T2V	✔	✔					✔
Wan2.1					CP (Ulysses)	✔	✔	✔		✔
HunyuanVideo	✔	✔			CP (Ulysses)	✔	✔	✔
HunyuanVideo1.5						✔	✔			✔
CogVideoX系列-T2V	✔	✔			CP (Ulysses)	✔	✔	✔
CogVideoX系列-I2V	✔	✔			CP (Ulysses)	✔	✔	✔
OpensoraPlan1.3-T2V	✔	✔	✔	✔	CP (Ulysses)	✔	✔
OpensoraPlan1.3-I2V	✔	✔	✔	✔	CP (Ulysses)	✔	✔
GLM-4.1V				✔		✔	✔
Qwen2VL-2B	✔	✔		✔	CP (Ulysses)	✔	✔	✔
Qwen2VL-7B	✔	✔		✔	CP (Ulysses)	✔	✔	✔
Qwen2VL-72B	✔	✔		✔	CP (Ulysses)	✔	✔	✔	DPO
Qwen2.5VL-3B	✔	✔		✔		✔	✔		GRPO
Qwen2.5VL-7B	✔	✔		✔		✔	✔		GRPO
Qwen2.5VL-32B	✔	✔		✔		✔	✔		GRPO
Qwen2.5VL-72B	✔	✔		✔		✔	✔	✔
Qwen2.5Omni-7B	✔			✔		✔		✔
Qwen3-Omni							✔			✔
InternVL3-8B	✔	✔	✔	✔	CP (Ring)	✔	✔
InternVL3-78B	✔	✔	✔	✔	CP (Ring)	✔	✔

备注：

TP: Tensor Parallel
TP-SP: Tensor Parallel with Sequence Parallel
VPP: Virtual Pipeline Parallel
PP: Pipeline Parallel
DSP: Dynamic Sequence Parallel
CP (Ulysses): Context Parallel by leveraging Deepspeed Ulysses with Sequence Parallel
CP (Ring Attention): Context Parallel with Ring Attention
Distributed Optimizer: Zero Redundancy Optimizer (ZeRO)
Recomputation: Reducing Activation Recomputation
LoRA: Low-Rank Adaptation
RL: Reinforcement Learning
FSDP2: Fully Sharded Data Parallelism

配套版本与支持模型

MindSpeed MM 预置了覆盖多模态生成、多模态理解等任务的丰富模型；各模型的参数规模、训练任务、推荐集群、实测性能与认证状态，详见 MindSpeed MM 支持模型列表。

大语言模型（稠密模型、稀疏模型和状态空间模型）由 MindSpeed-LLM 专项维护，如需进行大语言模型训练，请访问 MindSpeed-LLM 获取详细使用说明。

常用参数解释说明

针对MindSpeed MM套件中运行所使用的参数做解释说明，具体见README

特性规划

【模型特性】 CogVideoX: PP
【模型特性】 OpensoraPlan1.3: CP (Ring Attention)
【模型特性】 Qwen2VL: VPP, CP (Ulysses & Ring Attention)
【模型特性】 InternVL2: TP, CP (Ulysses & Ring Attention)
【基础特性】 Hetero-parallel

工具使用

昇腾Profiling采集工具

MindSpeed MM集成了昇腾profiling采集工具，以提供对模型运行情况的分析。该工具能够依照配置采集模型的算子、显存等关键信息，同时支持动静态两种采集方式，协助开发者分析模型瓶颈，并可根据实际场景需求选择使用。

具体方法见 README 的profiling章节

MindStudio Insight性能分析工具

针对大模型集群场景的性能调优，这里推荐一款优秀的可视化调优工具MindStudio Insight。 MindStudio Insight提供了包括Timeline视图、通信分析、计算耗时等的可视化呈现，以便用户分析潜在的性能瓶颈，并指导如何采取措施消除或减少这些瓶颈。

具体安装及使用方法见《MindStudio Insight操作指南》

Sora类模型特征提取

MindSpeed MM支持提取视频和文本特征并保存

具体方法见 README 的Sora类模型特征提取章节

内存快照提取

MindSpeed MM集成了昇腾内存快照采集工具，以提供对模型运行情况的分析。

具体方法见 README 的内存快照提取章节

Tensorboard使用

MindSpeed MM支持Tensorboard的使用

具体方法见 README 的Tensorboard使用章节

版本维护

MindSpeed MM版本有以下五个维护阶段：

状态	时间	说明
计划	1—3 个月	计划特性
开发	3 个月	开发特性
维护	6-12 个月	合入所有已解决的问题并发布版本，针对不同的MindSpeed MM版本采取不同的维护策略，常规版本和长期支持版本维护周期分别为6个月和12个月
无维护	0—3 个月	合入所有已解决的问题，无专职维护人员，无版本发布
生命周期终止（EOL）	N/A	分支不再接受任何修改

MindSpeed MM已发布版本维护策略：

MindSpeed MM版本	维护策略	当前状态	发布时间	后续状态
26.0.0	常规版本	维护	2026/03/30	预计2026/09/30起无维护
2.3.0	常规版本	无维护	2025/12/30	预计2026/06/30起无维护
2.2.0	常规版本	无维护	2025/09/30	预计2026/03/30起无维护
2.1.0	常规版本	无维护	2025/06/30	预计2025/12/30起无维护
2.0.0	常规版本	无维护	2025/03/30	预计2025/09/30起无维护
1.0.0	常规版本	无维护	2024/12/30	预计2025/06/30起无维护
1.0.RC3	常规版本	无维护	2024/09/30	预计2025/03/30起无维护

常见问题

安全声明

MindSpeed MM 安全声明

免责声明

致MindSpeed MM使用者

MindSpeed MM提供的模型仅供您用于非商业目的。
对于各模型，MindSpeed MM平台仅提示性地向您建议可用于训练的数据集，华为不提供任何数据集，如您使用这些数据集进行训练，请您特别注意应遵守对应数据集的License，如您因使用数据集而产生侵权纠纷，华为不承担任何责任。
如您在使用MindSpeed MM模型过程中，发现任何问题（包括但不限于功能问题、合规问题），请在Gitcode提交issue，我们将及时审视并解决。
MindSpeed MM功能依赖的Megatron等第三方开源软件，均由第三方社区提供和维护，因第三方开源软件导致的问题的修复依赖相关社区的贡献和反馈。您应理解，MindSpeed MM仓库不保证第三方开源软件本身的问题进行修复，也不保证会测试，纠正所有第三方开源软件的漏洞和错误。

致数据集所有者

如果您不希望您的数据集在MindSpeed MM中的模型被提及，或希望更新MindSpeed MM中的模型关于您的数据集的描述，请在Gitcode提交issue，我们将根据您的issue要求删除或更新您的数据集描述。衷心感谢您对MindSpeed MM的理解和贡献。

License声明

Ascend MindSpeed MM提供的模型，如模型目录下存在License的，以该License为准。如模型目录下不存在License的，以Apache 2.0许可证许可，对应许可证文本可查阅Ascend MindSpeed MM根目录LICENSE文件，docs目录下的文档适用CC-BY 4.0许可证，具体参见文档LICENSE。

贡献声明

1. 报告问题

如果您发现任何问题，请先查看仓库的issues列表，尝试寻找类似问题或解决方案。
如果现有issues列表中没有您遇到的问题，可以提交一个新的issue，并尽量提供清晰的问题描述、复现步骤与环境信息。

2. 贡献代码流程

若您希望提交代码改动，请遵循以下简要步骤：

在您的个人分支上开发并提交，然后向本项目仓库发起Pull Request（PR）；
在我们的SIG例会PR评审申请登记中，参照既定格式申请PR评审，并按时参加对应的评审会议；
根据评审意见进行修改，并更新PR；
PR通过评审后，在评论区输入compile以触发门禁流水线（CI）；
当PR的CI通过且获得足够的标签后，仓库Committer将进行最终审核，并合入在研分支。

感谢您的参与与贡献！我们期待与您共同推动项目发展。

致谢

MindSpeed MM 由华为公司的下列部门及昇腾生态合作伙伴联合贡献：

华为公司：

计算产品线
公共开发部
2012实验室
华为云

生态合作伙伴：

360 AI Research
北大OpenSoraPlan团队
微信技术架构部基础架构中心
京东零售九数研发技术部

感谢来自社区的每一个PR，欢迎贡献 MindSpeed MM。

项目介绍