master

ascend-robotstyle: pre-commit autofix cleancode (base check)

37ccda36创建于 12 小时前历史提交

文件	最后提交记录	最后更新时间
data_utils	style: pre-commit autofix cleancode (base check) Co-authored-by: liyingxuan<liyingxuan3@huawei.com> # message auto-generated for no-merge-commit merge: !2616 merge master into master style: pre-commit autofix cleancode (base check) Created-by: liyx616 Commit-by: liyingxuan Merged-by: ascend-robot Description: ## What this PR does / why we need it? Please describe the background and detailed changes of the PR. If it is a bugfix, please attach the related issue. ## Does this PR introduce any user-facing change? Please describe whether the PR will result in any user-facing usage changes. If there is related documentation, please specify its path. ## How was this patch tested? Please explain how to verify the correctness and effectiveness of this feature, as well as its usage constraints and limitations. See merge request: Ascend/MindSpeed-MM!2616	12 小时前
dataloader	style: pre-commit autofix cleancode (base check) Co-authored-by: liyingxuan<liyingxuan3@huawei.com> # message auto-generated for no-merge-commit merge: !2616 merge master into master style: pre-commit autofix cleancode (base check) Created-by: liyx616 Commit-by: liyingxuan Merged-by: ascend-robot Description: ## What this PR does / why we need it? Please describe the background and detailed changes of the PR. If it is a bugfix, please attach the related issue. ## Does this PR introduce any user-facing change? Please describe whether the PR will result in any user-facing usage changes. If there is related documentation, please specify its path. ## How was this patch tested? Please explain how to verify the correctness and effectiveness of this feature, as well as its usage constraints and limitations. See merge request: Ascend/MindSpeed-MM!2616	12 小时前
datasets	style: pre-commit autofix cleancode (base check) Co-authored-by: liyingxuan<liyingxuan3@huawei.com> # message auto-generated for no-merge-commit merge: !2616 merge master into master style: pre-commit autofix cleancode (base check) Created-by: liyx616 Commit-by: liyingxuan Merged-by: ascend-robot Description: ## What this PR does / why we need it? Please describe the background and detailed changes of the PR. If it is a bugfix, please attach the related issue. ## Does this PR introduce any user-facing change? Please describe whether the PR will result in any user-facing usage changes. If there is related documentation, please specify its path. ## How was this patch tested? Please explain how to verify the correctness and effectiveness of this feature, as well as its usage constraints and limitations. See merge request: Ascend/MindSpeed-MM!2616	12 小时前
__init__.py	feat: fsdp2 supports streaming dataload and async preprocessing Co-authored-by: xiaoyue994<xiaoyuanhang@huawei.com> # message auto-generated for no-merge-commit merge: !2554 merge fsdp2_streaming into master feat: fsdp2 supports streaming dataload and async preprocessing Created-by: xiaoyue994 Commit-by: xiaoyue994 Merged-by: ascend-robot Description: ## What this PR does / why we need it? fsdp2的流式数据加载（base mode）、异步数据预处理特性。 特性RFC：https://gitcode.com/xiaoyue994/MindSpeed-MM/blob/7144b3a40195848b60e5e8170dab17c039dea334/docs/zh/features/async_preprocess_iterable_dataset.md ## Does this PR introduce any user-facing change? 用户可以在配置文件中，开启streaming开关，开启流式数据加载、数据异步预处理 basic_parameters: streaming: true async_preprocess: true async_preprocess_buffer_size: 32（可自行设置） ## How was this patch tested? 通过长跑5000step，进行精度验证。 See merge request: Ascend/MindSpeed-MM!2554	6 天前