0
代码
代码
Issues
Pull Requests
讨论
Wiki
分析
0
ascend-robotascend-robotfeat: add custom pp layout
94e8ac1c创建于 10 小时前历史提交
文件最后提交记录最后更新时间
custom_layers
!2114 实现pipeline parallel的noop layer的重构 Merge pull request !2114 from liurong1995/feature_noop 1 年前
flash_attention
fix：fix atten_mask_shape error when using transformer_engine Co-authored-by: Keilo_W<wangkaiyu11@h-partners.com> # message auto-generated for no-merge-commit merge: !3293 merge master into master fix：fix atten_mask_shape error when using transformer_engine Created-by: Keilo_W Commit-by: Keilo_W Merged-by: ascend-robot Description: An atten_mask_shape error will occur if --attention-mask-type causal is used together with --transformer-impl transformer_engine. To avoid this, you must also enable the --use-flash-attn option. See merge request: Ascend/MindSpeed!32932 个月前
moe
feat: add custom pp layout Co-authored-by: wuweiqiang24<wuweiqiang11@huawei.com> # message auto-generated for no-merge-commit merge: !3496 merge add_pp_layout into master feat: add custom pp layout Created-by: wuweiqiang24 Commit-by: wuweiqiang24 Merged-by: ascend-robot Description: 新增pipeline-model-parallel-layout功能，支持自定义PP每个stage的层排布 验证链接：https://wiki.huawei.com/domains/137239/wiki/268925/WIKI2026052611233549 issue: https://gitcode.com/Ascend/MindSpeed/issues/166 See merge request: Ascend/MindSpeed!349610 小时前
multi_head_latent_attention
fix: bugfix for mla in ulysses Co-authored-by: wuweiqiang24<wuweiqiang11@huawei.com> # message auto-generated for no-merge-commit merge: !3170 merge bugfix_ulysses_tnd into master fix: bugfix for mla in ulysses Created-by: wuweiqiang24 Commit-by: wuweiqiang24 Merged-by: ascend-robot Description: 1、适配Ulysses+TND Causal场景 2、修复了Ulysses+TND场景下MLA-CP的bug See merge request: Ascend/MindSpeed!31704 个月前
__init__.py
!359 change ascendspeed to mindspeed Merge pull request !359 from 邓佳/master 1 年前
attention.py
!2477 fix v1 old code Merge pull request !2477 from yanzhixiao/fix-v1-code 11 个月前
dot_product_attention.py
fix: different scale for mla in master Co-authored-by: wuweiqiang24<wuweiqiang11@huawei.com> # message auto-generated for no-merge-commit merge: !2977 merge fixbug_mla_master into master fix: different scale for mla in master Created-by: wuweiqiang24 Commit-by: wuweiqiang24 Merged-by: ascend-robot Description: 与magatron中mla内的scale参数计算不一致，导致精度无法对齐 See merge request: Ascend/MindSpeed!29776 个月前
mlp.py
!2040 refactor：tp2d重构 Merge pull request !2040 from glhyy/tp2d 1 年前
module.py
!607 MOE适配swiglu及激活函数重计算 Merge pull request !607 from 王智伟/master2 1 年前
transformer.py
fix: TE + recompute_norm refix Co-authored-by: yulelanmei<huangyijie8@huawei.com> # message auto-generated for no-merge-commit merge: !3325 merge master into master fix: TE + recompute_norm refix Created-by: yulelanmei Commit-by: yulelanmei Merged-by: ascend-robot Description: What this PR does / why we need it? refix TE + recompute_norm Does this PR introduce any user-facing change? No How was this patch tested? Test using MindSpeed-Core ST cases and LLM+core case See merge request: Ascend/MindSpeed!33252 个月前
transformer_block.py
Support TransformerEngine Co-authored-by: MingzhenWang<wangmingzhen4@huawei.com> Co-authored-by: Muu<koimuu@163.com> Co-authored-by: x30061065<xuyuanhui3@h-partners.com> Co-authored-by: 耿瑞良<gengruiliang@huawei.com> # message auto-generated for no-merge-commit merge: !2947 merge lingqu_master into master Support TransformerEngine Created-by: mingzhenwang Commit-by: mingzhenwang;Muu;MingzhenWang;x30061065;耿瑞良 Merged-by: ascend-robot Description: 1. 支持TELinear层 2. 支持FP8计算，quantmatmul/gmm 3. 支持多种数据类型FP8/HiF8 4. 支持多种量化策略delayed/tensorwise/blockwise/mxfp8 5. TELinear层支持通算融合 See merge request: Ascend/MindSpeed!29476 个月前
transformer_config.py
!2285 v2 l0 UT Merge pull request !2285 from glhyy/masterdev 1 年前