MindSpeed/mindspeed/core/transformer · Ascend/MindSpeed - AtomGit

文件	最后提交记录	最后更新时间
custom_layers	!2114 实现pipeline parallel的noop layer的重构 Merge pull request !2114 from liurong1995/feature_noop	1 年前
flash_attention	fix：fix atten_mask_shape error when using transformer_engine Co-authored-by: Keilo_W<wangkaiyu11@h-partners.com> # message auto-generated for no-merge-commit merge: !3293 merge master into master fix：fix atten_mask_shape error when using transformer_engine Created-by: Keilo_W Commit-by: Keilo_W Merged-by: ascend-robot Description: An atten_mask_shape error will occur if `--attention-mask-type causal` is used together with `--transformer-impl transformer_engine`. To avoid this, you must also enable the `--use-flash-attn` option. See merge request: Ascend/MindSpeed!3293	2 个月前
moe	feat: fp8_reuse_quant_w Co-authored-by: Jia_Austin<dengjia6@huawei.com> # message auto-generated for no-merge-commit merge: !3358 merge feat_fp8_reuse_quant_w into master feat: fp8_reuse_quant_w Created-by: Jia_Austin Commit-by: Jia_Austin Merged-by: ascend-robot Description: What this PR does / why we need it? Please describe the background and detailed changes of the PR. If it is a bugfix, please attach the related issue. Does this PR introduce any user-facing change? Please describe whether the PR will result in any user-facing usage changes. If there is related documentation, please specify its path. How was this patch tested? Please explain how to verify the correctness and effectiveness of this feature, as well as its usage constraints and limitations. See merge request: Ascend/MindSpeed!3358	2 个月前
multi_head_latent_attention	fix: bugfix for mla in ulysses Co-authored-by: wuweiqiang24<wuweiqiang11@huawei.com> # message auto-generated for no-merge-commit merge: !3170 merge bugfix_ulysses_tnd into master fix: bugfix for mla in ulysses Created-by: wuweiqiang24 Commit-by: wuweiqiang24 Merged-by: ascend-robot Description: 1、适配Ulysses+TND Causal场景 2、修复了Ulysses+TND场景下MLA-CP的bug See merge request: Ascend/MindSpeed!3170	4 个月前
__init__.py	!359 change ascendspeed to mindspeed Merge pull request !359 from 邓佳/master	1 年前
attention.py	!2477 fix v1 old code Merge pull request !2477 from yanzhixiao/fix-v1-code	11 个月前
dot_product_attention.py	fix: different scale for mla in master Co-authored-by: wuweiqiang24<wuweiqiang11@huawei.com> # message auto-generated for no-merge-commit merge: !2977 merge fixbug_mla_master into master fix: different scale for mla in master Created-by: wuweiqiang24 Commit-by: wuweiqiang24 Merged-by: ascend-robot Description: 与magatron中mla内的scale参数计算不一致，导致精度无法对齐 See merge request: Ascend/MindSpeed!2977	6 个月前
mlp.py	!2040 refactor：tp2d重构 Merge pull request !2040 from glhyy/tp2d	1 年前
module.py	!607 MOE适配swiglu及激活函数重计算 Merge pull request !607 from 王智伟/master2	1 年前
transformer.py	fix: TE + recompute_norm refix Co-authored-by: yulelanmei<huangyijie8@huawei.com> # message auto-generated for no-merge-commit merge: !3325 merge master into master fix: TE + recompute_norm refix Created-by: yulelanmei Commit-by: yulelanmei Merged-by: ascend-robot Description: What this PR does / why we need it? refix TE + recompute_norm Does this PR introduce any user-facing change? No How was this patch tested? Test using MindSpeed-Core ST cases and LLM+core case See merge request: Ascend/MindSpeed!3325	2 个月前
transformer_block.py	Support TransformerEngine Co-authored-by: MingzhenWang<wangmingzhen4@huawei.com> Co-authored-by: Muu<koimuu@163.com> Co-authored-by: x30061065<xuyuanhui3@h-partners.com> Co-authored-by: 耿瑞良<gengruiliang@huawei.com> # message auto-generated for no-merge-commit merge: !2947 merge lingqu_master into master Support TransformerEngine Created-by: mingzhenwang Commit-by: mingzhenwang;Muu;MingzhenWang;x30061065;耿瑞良 Merged-by: ascend-robot Description: 1. 支持TELinear层 2. 支持FP8计算，quantmatmul/gmm 3. 支持多种数据类型FP8/HiF8 4. 支持多种量化策略delayed/tensorwise/blockwise/mxfp8 5. TELinear层支持通算融合 See merge request: Ascend/MindSpeed!2947	6 个月前
transformer_config.py	!2285 v2 l0 UT Merge pull request !2285 from glhyy/masterdev	1 年前