master

ascend-robotfix：fix atten_mask_shape error when using transformer_engine

3712cfe8创建于 3月4日历史提交

文件	最后提交记录	最后更新时间
data_handler.py	fix：fix atten_mask_shape error when using transformer_engine Co-authored-by: Keilo_W<wangkaiyu11@h-partners.com> # message auto-generated for no-merge-commit merge: !3293 merge master into master fix：fix atten_mask_shape error when using transformer_engine Created-by: Keilo_W Commit-by: Keilo_W Merged-by: ascend-robot Description: An atten_mask_shape error will occur if --attention-mask-type causal is used together with --transformer-impl transformer_engine. To avoid this, you must also enable the --use-flash-attn option. See merge request: Ascend/MindSpeed!3293	2 个月前
preprocess_data.py	Add offline pad_data Co-authored-by: wuweiqiang24<wuweiqiang11@huawei.com> # message auto-generated for no-merge-commit merge: !2938 merge revise_preprocess_data into master Add offline pad_data Created-by: wuweiqiang24 Commit-by: wuweiqiang24 Merged-by: ascend-robot Description: 增加离线预处理pack数据集功能，可提前将数据padding到2\*CP倍，在线使用CP功能时可节约padding部分耗时 * 精度与非离线padding版本存在一定差异  * Llama2-7b，单机16k，GBS=8场景下，性能提升4.8%  See merge request: Ascend/MindSpeed!2938	6 个月前