| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
docs: fix Chinese documentation issues from AIDD review Co-authored-by: fanlu5<fanlu5@huawei.com> # message auto-generated for no-merge-commit merge: !3704 merge master into master docs: fix Chinese documentation issues from AIDD review Created-by: fanlu5 Commit-by: fanlu5 Merged-by: ascend-robot Description: ## What this PR does / why we need it? docs: fix Chinese documentation issues from AIDD review 本次修改根据AIDD文档评审反馈，修复了中文文档中的多处问题，包括： - 术语表述不准确（如"高保序性"、"低保真数据"等） - 公式错误（如分布式归一化中的方差计算公式） - 参数依赖关系描述不清（如 ulysses-degree-in-cp 的整除关系） - 特性描述与代码实现不一致（如 swap-optimizer 的 D2H/H2D 时序） - 性能提升概念混淆（如训练吞吐与收敛速度的区别） ## Does this PR introduce any user-facing change? NA ## How was this patch tested? NA  See merge request: Ascend/MindSpeed!3704 | 1 个月前 | |
feat: kvallgather cp thd load balance Co-authored-by: m0_50947149<chenkaihu201@outlook.com> # message auto-generated for no-merge-commit merge: !3820 merge feature/kvallgather-thd-load-balance into master feat: kvallgather cp thd load balance Created-by: m0_50947149 Commit-by: m0_50947149 Merged-by: ascend-robot Description: ## What this PR does / why we need it? This PR adds load-balanced KVAllGather context parallelism for the THD format. It uses EOD-aware symmetric sequence partitioning and implements a THD attention path with direct rank-major K/V indexing and a single TND fused-attention call. Metadata caching is added to reduce repeated calculations. MHA, GQA, and expanded MLA are supported. https://gitcode.com/Ascend/MindSpeed/issues/154 https://gitcode.com/Ascend/MindSpeed/issues/232 ## Does this PR introduce any user-facing change? Yes. Users can now use THD packed sequences with kvallgather_cp_algo, causal attention, and EOD attention-mask reset without introducing new command-line options. Each packed subsequence length must be divisible by 2 * context_parallel_size. ## How was this patch tested? Unit tests cover: - Load-balanced batch partitioning and metadata generation - Metadata cache reuse and invalidation - MHA, GQA, and MLA - FP16 and BF16 - Distributed forward and backward consistency against a non-CP reference The modified files also pass Ruff, Pylint, and whitespace checks. Test_results:  See merge request: Ascend/MindSpeed!3820 | 2 天前 | |
perf: defer mxfp8 backward quantization Co-authored-by: guihaowen666<guihaowen@huawei.com> # message auto-generated for no-merge-commit merge: !3852 merge br_master_fp8_quant_fb_split into master perf: defer mxfp8 backward quantization Created-by: guihaowen666 Commit-by: guihaowen666 Merged-by: ascend-robot Description: ## What this PR does / why we need it? When doing low-precision matmul and low-precision GMM calculations, using the double-axis quantization method saves the quantized weights and scales for the model's backward pass, which takes up a lot of GPU memory. This PR changes the double-axis quantization to single-axis quantization, so the model doesn’t save the quantized weights and scales during the forward pass but recalculates them during the backward pass, saving GPU memory. ## Does this PR introduce any user-facing change? This is a newly developed feature. Use the --mxfp8-defer-backward-quant parameter to control whether to enable this feature. It doesn't affect other features and involves no user-facing changes. ## How was this patch tested? Local checks and CI checks passed See merge request: Ascend/MindSpeed!3852 | 2 天前 | |
[feat] add mxfp4_to_bf16_dequantization kernel Co-authored-by: yhr6385<yanghongru2@huawei.com> # message auto-generated for no-merge-commit merge: !3624 merge master into master [feat] add mxfp4_to_bf16_dequantization kernel Created-by: yhr6385 Commit-by: yhr6385 Merged-by: ascend-robot Description: ## What this PR does / why we need it? 添加 MXFP4 反量化 BF16 的 triton 算子实现，用于 W4A16 的低精度 RL 后训练。 ## Does this PR introduce any user-facing change? N/A ## How was this patch tested? 在 950DT 单卡使用大小为（1024,4096）的矩阵测试耗时 0.1011ms。 正确性测试结果：https://wiki.huawei.com/domains/150199/wiki/301399/WIKI2026071911918935 See merge request: Ascend/MindSpeed!3624 | 27 天前 | |
fix: NPU datadump level: L0 & mix Co-authored-by: yulelanmei<huangyijie8@huawei.com> # message auto-generated for no-merge-commit merge: !3351 merge master into master fix: NPU datadump level: L0 & mix Created-by: yulelanmei Commit-by: yulelanmei Merged-by: ascend-robot Description: What this PR does / why we need it? 当前--npu-datadump未适配 L0及mix 的dump等级，需要增强功能 Does this PR introduce any user-facing change? N/A How was this patch tested? 开启--npu-datadump，config.json配置level为L0或mix 测试：https://wiki.huawei.com/domains/148330/wiki/296621/WIKI2026032510543405 See merge request: Ascend/MindSpeed!3351 | 5 个月前 | |
harden sort_chunks_by_idx memory access Co-authored-by: guofanfeng<guofanfeng1@huawei.com> # message auto-generated for no-merge-commit merge: !3316 merge sort_kernel into master harden sort_chunks_by_idx memory access Created-by: guofanfeng23 Commit-by: guofanfeng Merged-by: ascend-robot Description: What this PR does / why we need it? 1、修复内存越界访问 分析：https://wiki.huawei.com/domains/152732/wiki/325378/WIKI2026071611884648 2、ut和pytorch小算子比较 补充大shape用例 Does this PR introduce any user-facing change? No How was this patch tested? ut pass See merge request: Ascend/MindSpeed!3316 | 12 天前 | |
[modify][mindspore] register patchs for coalescing_manager Co-authored-by: weixin_47897441<wuyouqi1@h-partners.com> # message auto-generated for no-merge-commit merge: !3233 merge master-0129 into master [modify][mindspore] register patchs for coalescing_manager Created-by: weixin_47897441 Commit-by: weixin_47897441 Merged-by: ascend-robot Description: [modify][mindspore] register patchs for coalescing_manager： The coalescing_manager-related patches have recently been removed from megtron_basic.py. Given that MSA currently lacks support for communication operators associated with coalescing_manager, these removed patches are migrated to MindSpore to ensure normal launch of models such as qwen3vl. See merge request: Ascend/MindSpeed!3233 | 6 个月前 | |
feat: add mhc recompute Co-authored-by: wuweiqiang24<wuweiqiang11@huawei.com> # message auto-generated for no-merge-commit merge: !3619 merge mhc-recompute-fboverlap into master feat: add mhc recompute Created-by: wuweiqiang24 Commit-by: wuweiqiang24 Merged-by: ascend-robot Description: ## What this PR does / why we need it? This PR adds fine-grained recomputation for DeepSeek V4 to reduce peak NPU memory usage. Main changes: - Add CSA recomputation controlled by --recompute-csa-attention. - Add fused MHC recomputation controlled by --mhc-recompute. - Support regular Transformer and MoE FB overlap paths. - Adapt PP layout, MTP, norm recomputation and swap layer input. - Replace legacy G2 naming with CSA terminology. ## Does this PR introduce any user-facing change? Two new options are added: --recompute-csa-attention --mhc-recompute ## Does this PR introduce any user-facing change? - Compared loss, gradient norm and peak NPU memory with recomputation-disabled and full-recomputation configurations. See merge request: Ascend/MindSpeed!3619 | 6 天前 | |
remove deprecated code 2 Co-authored-by: 赵一帆<zhaoyifan15@huawei.com> # message auto-generated for no-merge-commit merge: !2972 merge master into master remove deprecated code 2 Created-by: zhao-yifan27 Commit-by: 赵一帆 Merged-by: ascend-robot Description: 删除废弃代码 mindspeed/model下废弃代码 mindspeed/moe下ampipe废弃代码 See merge request: Ascend/MindSpeed!2972 | 9 个月前 | |
| 1 年前 | ||
refactor: Remove SFA/SFAG/SLI Co-authored-by: JialiZheng<jializheng@huawei.com> # message auto-generated for no-merge-commit merge: !3591 merge master into master refactor: Remove SFA/SFAG/SLI Created-by: JialiZheng1 Commit-by: JialiZheng Merged-by: ascend-robot Description: 删除临时版本算子适配，相关功能通过正式算子承载 See merge request: Ascend/MindSpeed!3591 | 1 个月前 | |
harden sort_chunks_by_idx memory access Co-authored-by: guofanfeng<guofanfeng1@huawei.com> # message auto-generated for no-merge-commit merge: !3316 merge sort_kernel into master harden sort_chunks_by_idx memory access Created-by: guofanfeng23 Commit-by: guofanfeng Merged-by: ascend-robot Description: What this PR does / why we need it? 1、修复内存越界访问 分析：https://wiki.huawei.com/domains/152732/wiki/325378/WIKI2026071611884648 2、ut和pytorch小算子比较 补充大shape用例 Does this PR introduce any user-facing change? No How was this patch tested? ut pass See merge request: Ascend/MindSpeed!3316 | 12 天前 | |
fix: TE LayerNormLinear init weight order align NVTE Co-authored-by: clc2025<chenlucong@huawei.com> # message auto-generated for no-merge-commit merge: !3569 merge fix_lnliner_initweight into master fix: TE LayerNormLinear init weight order align NVTE Created-by: clc2025 Commit-by: clc2025 Merged-by: ascend-robot Description: ## What this PR does / why we need it? 具体查看关联issue ## Does this PR introduce any user-facing change? NA ## How was this patch tested? 基于脚本用例，从GPU上保存权重，NPU加载权重后断点续训，精度能对齐且无功能报错 See merge request: Ascend/MindSpeed!3569 | 2 个月前 | |
| 1 年前 | ||
feat: kvallgather cp thd load balance Co-authored-by: m0_50947149<chenkaihu201@outlook.com> # message auto-generated for no-merge-commit merge: !3820 merge feature/kvallgather-thd-load-balance into master feat: kvallgather cp thd load balance Created-by: m0_50947149 Commit-by: m0_50947149 Merged-by: ascend-robot Description: ## What this PR does / why we need it? This PR adds load-balanced KVAllGather context parallelism for the THD format. It uses EOD-aware symmetric sequence partitioning and implements a THD attention path with direct rank-major K/V indexing and a single TND fused-attention call. Metadata caching is added to reduce repeated calculations. MHA, GQA, and expanded MLA are supported. https://gitcode.com/Ascend/MindSpeed/issues/154 https://gitcode.com/Ascend/MindSpeed/issues/232 ## Does this PR introduce any user-facing change? Yes. Users can now use THD packed sequences with kvallgather_cp_algo, causal attention, and EOD attention-mask reset without introducing new command-line options. Each packed subsequence length must be divisible by 2 * context_parallel_size. ## How was this patch tested? Unit tests cover: - Load-balanced batch partitioning and metadata generation - Metadata cache reuse and invalidation - MHA, GQA, and MLA - FP16 and BF16 - Distributed forward and backward consistency against a non-CP reference The modified files also pass Ruff, Pylint, and whitespace checks. Test_results:  See merge request: Ascend/MindSpeed!3820 | 2 天前 | |
| 1 年前 | ||
| 1 年前 | ||
| 11 个月前 | ||
Support TransformerEngine Co-authored-by: MingzhenWang<wangmingzhen4@huawei.com> Co-authored-by: Muu<koimuu@163.com> Co-authored-by: x30061065<xuyuanhui3@h-partners.com> Co-authored-by: 耿瑞良<gengruiliang@huawei.com> # message auto-generated for no-merge-commit merge: !2947 merge lingqu_master into master Support TransformerEngine Created-by: mingzhenwang Commit-by: mingzhenwang;Muu;MingzhenWang;x30061065;耿瑞良 Merged-by: ascend-robot Description: 1. 支持TELinear层 2. 支持FP8计算，quantmatmul/gmm 3. 支持多种数据类型FP8/HiF8 4. 支持多种量化策略delayed/tensorwise/blockwise/mxfp8 5. TELinear层支持通算融合 See merge request: Ascend/MindSpeed!2947 | 9 个月前 | |
[Bugfix] Fix Megatron checkpoint saving&loading compatibility for torch_dcp format Co-authored-by: 林明哲<linmingzhe3@huawei.com> # message auto-generated for no-merge-commit merge: !3077 merge fix1202 into master [Bugfix] Fix Megatron checkpoint saving&loading compatibility for torch_dcp format Created-by: LinMingZhe Commit-by: 林明哲 Merged-by: ascend-robot Description: Fix Megatron checkpoint saving&loading compatibility for torch_dcp format See merge request: Ascend/MindSpeed!3077 | 8 个月前 | |
| 1 年前 | ||
| 1 年前 | ||
| 1 年前 | ||
【bugfix!!!】fbov COC&share_expert_sync fix Co-authored-by: EX_mitsu<yangjie409@h-partners.com> # message auto-generated for no-merge-commit merge: !3005 merge master into master 【bugfix!!!】fbov COC&share_expert_sync fix Created-by: EX_mitsuX Commit-by: EX_mitsuX;EX_mitsu Merged-by: ascend-robot Description: 修复TP1状态下开启COC未进行检查的BUG（非预期场景。TP1开COC应当无收益。）。 重新调整计算流，追加等待及同步，修复计算速度过快时可能产生的同步问题（问题场景：同时开启COC，permute融合算子及共享专家出现，使用launch_blocking该问题消失）。 修复不再兼容的TE检测。 修复TE部分module的属性缺失。 See merge request: Ascend/MindSpeed!3005 | 8 个月前 | |
fix：Add mindspeed config to subclass of transformer config Co-authored-by: JialiZheng<jializheng@huawei.com> # message auto-generated for no-merge-commit merge: !3284 merge master into master fix：Add mindspeed config to subclass of transformer config Created-by: JialiZheng1 Commit-by: JialiZheng Merged-by: ascend-robot Description: Add mindspeed config to subclass of transformer config See merge request: Ascend/MindSpeed!3284 | 5 个月前 | |
增加MOE专家负载均衡功能 Co-authored-by: zhanggaolu2<252028123@qq.com> # message auto-generated for no-merge-commit merge: !2845 merge expert_loadbalance2master into master 增加MOE专家负载均衡功能 Created-by: zhanggaolu2 Commit-by: zhanggaolu2 Merged-by: ascend-robot Description: 增加MOE专家负载均衡功能 See merge request: Ascend/MindSpeed!2845 | 10 个月前 | |
fix: fix the alltoall_seq token dispatcher Nan bug Co-authored-by: guofanfeng<guofanfeng1@huawei.com> # message auto-generated for no-merge-commit merge: !3249 merge bug_fix into master fix: fix the alltoall_seq token dispatcher Nan bug Created-by: guofanfeng23 Commit-by: guofanfeng Merged-by: ascend-robot Description: fix the alltoall_seq token dispatcher Nan bug https://wiki.huawei.com/domains/152732/wiki/307991/WIKI2026020210028614 See merge request: Ascend/MindSpeed!3249 | 6 个月前 | |
| 1 年前 |
| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
| 1 个月前 | ||
| 2 天前 | ||
| 2 天前 | ||
| 27 天前 | ||
| 5 个月前 | ||
| 12 天前 | ||
| 6 个月前 | ||
| 6 天前 | ||
| 9 个月前 | ||
| 1 年前 | ||
| 1 个月前 | ||
| 12 天前 | ||
| 2 个月前 | ||
| 1 年前 | ||
| 2 天前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 11 个月前 | ||
| 9 个月前 | ||
| 8 个月前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 1 年前 | ||
| 8 个月前 | ||
| 5 个月前 | ||
| 10 个月前 | ||
| 6 个月前 | ||
| 1 年前 |