Star0
1
代码
代码
Issues
Pull Requests
项目讨论
Wiki
分析
Star0
1
文件最后提交记录最后更新时间
auto_settings
feat: add A2G3 device type support across codebase Co-authored-by: fanlu5<fanlu5@huawei.com> # message auto-generated for no-merge-commit merge: !3587 merge master into master feat: add A2G3 device type support across codebase Created-by: fanlu5 Commit-by: fanlu5 Merged-by: ascend-robot Description: ## What this PR does / why we need it? feat: add A2G3 device type support across codebase. 添加A2G3设备类型的适配支持，将其与现有的910B设备同等对待，覆盖版本检测、搜索空间裁剪、通信硬件信息、MoE相关逻辑等多个模块 ## Does this PR introduce any user-facing change? NA ## How was this patch tested? NA See merge request: Ascend/MindSpeed!3587
2 个月前
core
feat: support W4A8-QAT Co-authored-by: xusiyang<xusiyang2@huawei.com> # message auto-generated for no-merge-commit merge: !3664 merge master into master feat: support W4A8-QAT Created-by: weixin_44492126 Commit-by: xusiyang Merged-by: ascend-robot Description: ## What this PR does / why we need it? 本PR 新增了对 W4A8-QAT（权重MXFP4/激活 MXFP8 量化感知训练，仅限 MoE 场景），并量化粒度分别支持：32(标准MXFP4量化粒度)/128(deepseek-V4采用量化粒度） ## Does this PR introduce any user-facing change? Yes. 用户可以通过在训练命令中添加以下参数来启用W4A8功能，默认量化粒度32： --qat-scheme w4a8-moe-only --qat-quant-block-size 128 ## How was this patch tested? 添加 --qat-scheme w4a8-moe-only 参数启动量化粒度为32的W4A8-QAT训练，添加--qat-quant-block-size 128启动量化粒度为128的W4A8-QAT训练 实现方案和实验结果如下：https://wiki.huawei.com/domains/171785/wiki/358154/WIKI2026071511870516 昇腾 950DT上测试通过：基于减层DeepSeek-V4，分别训练100step对比BF16 loss误差：量化粒度为32时loss误差在0.4%，量化粒度为128时loss误差在0.79% See merge request: Ascend/MindSpeed!3664
1 个月前
features_manager
feat: support W4A8-QAT Co-authored-by: xusiyang<xusiyang2@huawei.com> # message auto-generated for no-merge-commit merge: !3664 merge master into master feat: support W4A8-QAT Created-by: weixin_44492126 Commit-by: xusiyang Merged-by: ascend-robot Description: ## What this PR does / why we need it? 本PR 新增了对 W4A8-QAT（权重MXFP4/激活 MXFP8 量化感知训练，仅限 MoE 场景），并量化粒度分别支持：32(标准MXFP4量化粒度)/128(deepseek-V4采用量化粒度） ## Does this PR introduce any user-facing change? Yes. 用户可以通过在训练命令中添加以下参数来启用W4A8功能，默认量化粒度32： --qat-scheme w4a8-moe-only --qat-quant-block-size 128 ## How was this patch tested? 添加 --qat-scheme w4a8-moe-only 参数启动量化粒度为32的W4A8-QAT训练，添加--qat-quant-block-size 128启动量化粒度为128的W4A8-QAT训练 实现方案和实验结果如下：https://wiki.huawei.com/domains/171785/wiki/358154/WIKI2026071511870516 昇腾 950DT上测试通过：基于减层DeepSeek-V4，分别训练100step对比BF16 loss误差：量化粒度为32时loss误差在0.4%，量化粒度为128时loss误差在0.79% See merge request: Ascend/MindSpeed!3664
1 个月前
fsdp
[Features] mxfp8: fix hard code Co-authored-by: yangx_sy<sunyang49@huawei.com> # message auto-generated for no-merge-commit merge: !3493 merge fsdp2_fp8 into master [Features] mxfp8: fix hard code Created-by: yangx_sy Commit-by: yangx_sy Merged-by: ascend-robot Description: What this PR does / why we need it? mxfp8: fix hard code Does this PR introduce any user-facing change? Please describe whether the PR will result in any user-facing usage changes. If there is related documentation, please specify its path. How was this patch tested? Please explain how to verify the correctness and effectiveness of this feature, as well as its usage constraints and limitations. See merge request: Ascend/MindSpeed!3493
2 个月前
functional
fix: NPU datadump level: L0 & mix Co-authored-by: yulelanmei<huangyijie8@huawei.com> # message auto-generated for no-merge-commit merge: !3351 merge master into master fix: NPU datadump level: L0 & mix Created-by: yulelanmei Commit-by: yulelanmei Merged-by: ascend-robot Description: What this PR does / why we need it? 当前--npu-datadump未适配 L0及mix 的dump等级，需要增强功能 Does this PR introduce any user-facing change? N/A How was this patch tested? 开启--npu-datadump，config.json配置level为L0或mix 测试：https://wiki.huawei.com/domains/148330/wiki/296621/WIKI2026032510543405 See merge request: Ascend/MindSpeed!3351
5 个月前
lite
fix(triton): harden sort_chunks_by_idx memory access
26 天前
mindspore
[modify][mindspore] register patchs for coalescing_manager Co-authored-by: weixin_47897441<wuyouqi1@h-partners.com> # message auto-generated for no-merge-commit merge: !3233 merge master-0129 into master [modify][mindspore] register patchs for coalescing_manager Created-by: weixin_47897441 Commit-by: weixin_47897441 Merged-by: ascend-robot Description: [modify][mindspore] register patchs for coalescing_manager： The coalescing_manager-related patches have recently been removed from megtron_basic.py. Given that MSA currently lacks support for communication operators associated with coalescing_manager, these removed patches are migrated to MindSpore to ensure normal launch of models such as qwen3vl. See merge request: Ascend/MindSpeed!3233
7 个月前
model
remove deprecated code 2 Co-authored-by: 赵一帆<zhaoyifan15@huawei.com> # message auto-generated for no-merge-commit merge: !2972 merge master into master remove deprecated code 2 Created-by: zhao-yifan27 Commit-by: 赵一帆 Merged-by: ascend-robot Description: 删除废弃代码 mindspeed/model下废弃代码 mindspeed/moe下ampipe废弃代码 See merge request: Ascend/MindSpeed!2972
9 个月前
moe
remove deprecated code 2 Co-authored-by: 赵一帆<zhaoyifan15@huawei.com> # message auto-generated for no-merge-commit merge: !2972 merge master into master remove deprecated code 2 Created-by: zhao-yifan27 Commit-by: 赵一帆 Merged-by: ascend-robot Description: 删除废弃代码 mindspeed/model下废弃代码 mindspeed/moe下ampipe废弃代码 See merge request: Ascend/MindSpeed!2972
9 个月前
multi_modal
!872 [feature: conv3d支持depth维度并行] Merge pull request !872 from Shitong Li/master_new_conv3d
1 年前
op_builder
refactor: Remove SFA/SFAG/SLI Co-authored-by: JialiZheng<jializheng@huawei.com> # message auto-generated for no-merge-commit merge: !3591 merge master into master refactor: Remove SFA/SFAG/SLI Created-by: JialiZheng1 Commit-by: JialiZheng Merged-by: ascend-robot Description: 删除临时版本算子适配，相关功能通过正式算子承载 See merge request: Ascend/MindSpeed!3591
2 个月前
ops
fix(triton): harden sort_chunks_by_idx memory access
26 天前
optimizer
fix: TE LayerNormLinear init weight order align NVTE Co-authored-by: clc2025<chenlucong@huawei.com> # message auto-generated for no-merge-commit merge: !3569 merge fix_lnliner_initweight into master fix: TE LayerNormLinear init weight order align NVTE Created-by: clc2025 Commit-by: clc2025 Merged-by: ascend-robot Description: ## What this PR does / why we need it? 具体查看关联issue ## Does this PR introduce any user-facing change? NA ## How was this patch tested? 基于脚本用例，从GPU上保存权重，NPU加载权重后断点续训，精度能对齐且无功能报错 See merge request: Ascend/MindSpeed!3569
2 个月前
run
!716 perf: gpt_dataset and initialize in megatron Merge pull request !716 from 邓佳/dev_patch
1 年前
te
fix: MindSpeedTEDotProductAttention reads context_parallel_algo from global args instead of stale config Co-authored-by: clc2025<chenlucong@huawei.com> # message auto-generated for no-merge-commit merge: !3640 merge test_repatch into master fix: MindSpeedTEDotProductAttention reads context_parallel_algo from global args instead of stale config Created-by: clc2025 Commit-by: clc2025 Merged-by: ascend-robot Description: ## What this PR does / why we need it? Fix: MindSpeedTEDotProductAttention 在 repatch() 切换 context_parallel_algo 后仍然读取到旧的 CP 配置，导致 context parallel 失败。 问题背景：当调用 repatch() 将 context_parallel_algo 从 megatron_cp_algo 切换为 kvallgather_cp_algo 时，已存在的 TransformerConfig 对象仍保留旧值。MindSpeedTEDotProductAttention 从 self.config.context_parallel_algo 读取 cp_comm_type，得到的是过时的 megatron_cp_algo，该值不在 CPStrategyFactory._STRATEGY_MAP 中，导致 CP 初始化报错。 修复方案：改为从 get_full_args().context_parallel_algo 读取，该方法始终返回最新的全局 args（repatch 通过 setattr 就地更新全局 args）。 ## Does this PR introduce any user-facing change? NA ## How was this patch tested? 可通过以下步骤验证： 1. 使用 megatron_cp_algo 启动训练 2. 调用 repatch() 切换为 kvallgather_cp_algo 3. 确认 context parallel 正常初始化和运行，不再因 megatron_cp_algo 不在 _STRATEGY_MAP 中而报错 See merge request: Ascend/MindSpeed!3640
1 个月前
tokenizer
!2238 refactor: tokenizer重构 Merge pull request !2238 from wangruiqi/master
1 年前
__init__.py
!2114 实现pipeline parallel的noop layer的重构 Merge pull request !2114 from liurong1995/feature_noop
1 年前
args_utils.py
!2827 fix: fix get_full_args Merge pull request !2827 from YE ZHENYUAN/master0910
11 个月前
arguments.py
Support TransformerEngine Co-authored-by: MingzhenWang<wangmingzhen4@huawei.com> Co-authored-by: Muu<koimuu@163.com> Co-authored-by: x30061065<xuyuanhui3@h-partners.com> Co-authored-by: 耿瑞良<gengruiliang@huawei.com> # message auto-generated for no-merge-commit merge: !2947 merge lingqu_master into master Support TransformerEngine Created-by: mingzhenwang Commit-by: mingzhenwang;Muu;MingzhenWang;x30061065;耿瑞良 Merged-by: ascend-robot Description: 1. 支持TELinear层 2. 支持FP8计算，quantmatmul/gmm 3. 支持多种数据类型FP8/HiF8 4. 支持多种量化策略delayed/tensorwise/blockwise/mxfp8 5. TELinear层支持通算融合 See merge request: Ascend/MindSpeed!2947
9 个月前
checkpointing.py
[Bugfix] Fix Megatron checkpoint saving&loading compatibility for torch_dcp format Co-authored-by: 林明哲<linmingzhe3@huawei.com> # message auto-generated for no-merge-commit merge: !3077 merge fix1202 into master [Bugfix] Fix Megatron checkpoint saving&loading compatibility for torch_dcp format Created-by: LinMingZhe Commit-by: 林明哲 Merged-by: ascend-robot Description: Fix Megatron checkpoint saving&loading compatibility for torch_dcp format See merge request: Ascend/MindSpeed!3077
8 个月前
deprecate.py
!2228 add MindSpeedFeaturesManager Merge pull request !2228 from Jializheng/master
1 年前
initialize.py
!2308 Adaptation core_r0.12.0 Merge pull request !2308 from 邓佳/core_r0.12.0_dev
1 年前
log_config.py
!2675 Add once warning and args check Merge pull request !2675 from Jializheng/master
1 年前
megatron_adaptor.py
【bugfix!!!】fbov COC&share_expert_sync fix Co-authored-by: EX_mitsu<yangjie409@h-partners.com> # message auto-generated for no-merge-commit merge: !3005 merge master into master 【bugfix!!!】fbov COC&share_expert_sync fix Created-by: EX_mitsuX Commit-by: EX_mitsuX;EX_mitsu Merged-by: ascend-robot Description: 修复TP1状态下开启COC未进行检查的BUG（非预期场景。TP1开COC应当无收益。）。 重新调整计算流，追加等待及同步，修复计算速度过快时可能产生的同步问题（问题场景：同时开启COC，permute融合算子及共享专家出现，使用launch_blocking该问题消失）。 修复不再兼容的TE检测。 修复TE部分module的属性缺失。 See merge request: Ascend/MindSpeed!3005
9 个月前
patch_utils.py
fix：Add mindspeed config to subclass of transformer config Co-authored-by: JialiZheng<jializheng@huawei.com> # message auto-generated for no-merge-commit merge: !3284 merge master into master fix：Add mindspeed config to subclass of transformer config Created-by: JialiZheng1 Commit-by: JialiZheng Merged-by: ascend-robot Description: Add mindspeed config to subclass of transformer config See merge request: Ascend/MindSpeed!3284
5 个月前
training.py
增加MOE专家负载均衡功能 Co-authored-by: zhanggaolu2<252028123@qq.com> # message auto-generated for no-merge-commit merge: !2845 merge expert_loadbalance2master into master 增加MOE专家负载均衡功能 Created-by: zhanggaolu2 Commit-by: zhanggaolu2 Merged-by: ascend-robot Description: 增加MOE专家负载均衡功能 See merge request: Ascend/MindSpeed!2845
10 个月前
utils.py
fix: fix the alltoall_seq token dispatcher Nan bug Co-authored-by: guofanfeng<guofanfeng1@huawei.com> # message auto-generated for no-merge-commit merge: !3249 merge bug_fix into master fix: fix the alltoall_seq token dispatcher Nan bug Created-by: guofanfeng23 Commit-by: guofanfeng Merged-by: ascend-robot Description: fix the alltoall_seq token dispatcher Nan bug https://wiki.huawei.com/domains/152732/wiki/307991/WIKI2026020210028614 See merge request: Ascend/MindSpeed!3249
6 个月前
yaml_arguments.py
!1825 长序列支持BNSD和arguments整改 Merge pull request !1825 from YE ZHENYUAN/master
1 年前