| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
feat(pytorch): add glm52 in fsdp2 Co-authored-by: guozhihua2<guozhihua2@huawei.com> # message auto-generated for no-merge-commit merge: !4614 merge add_glm52_in_fsdp2 into master feat(pytorch): add glm52 in fsdp2 Created-by: guozhihua2 Commit-by: guozhihua2 Merged-by: ascend-robot Description: ## What this PR does / why we need it? This PR adds GLM52 model support in the FSDP2 training framework. Main changes include: 1. Add the GLM52 model implementation for FSDP2, including model definition, configuration adaptation, and registration logic. 2. Support GLM52 pretraining under the FSDP2 framework, enabling users to launch GLM52 training with the FSDP2 training entry and related scripts. 3. Support GLM52 chat/inference flow, so the adapted GLM52 model can be used for basic generation and chat validation after loading. 4. Adapt GLM52-specific model logic in FSDP2, including attention/indexer-related behavior and model forward compatibility required by GLM52. 5. Provide related ST coverage to ensure the GLM52 FSDP2 model path can be built, loaded, and executed correctly. ## Does this PR introduce any user-facing change? Yes. Users can now use the GLM52 model in the FSDP2 framework, including GLM52 pretraining and chat/inference workflows. Existing model usage is not expected to be affected. ## How was this patch tested？ Verified by running the GLM52 ST test. pipeline/st/glm52/pretrain_glm52_38b_4k_fsdp2_A3.sh The test covers the GLM52 FSDP2 model build and execution path, including pretraining-related runtime validation and basic chat/inference functionality. See merge request: Ascend/MindSpeed-LLM!4614 | 1 个月前 | |
feat: FSDP2 Parameter Extraction from YAML to Shell Scripts Co-authored-by: HANHU1CHEN<chenhanhui1@h-partners.com> # message auto-generated for no-merge-commit merge: !4722 merge master into master feat: FSDP2 Parameter Extraction from YAML to Shell Scripts Created-by: Rostellaria Commit-by: HANHU1CHEN Merged-by: ascend-robot Description: [#1586](https://gitcode.com/Ascend/MindSpeed-LLM/issues/1586) ## What this PR does / why we need it? Refactors FSDP2 training/inference scripts: extracts parallel , training , optimization , and quantization parameters from YAML configs into shell scripts as inline torchrun CLI arguments, enabling easier per-run overrides without editing YAML files. Keeps data.dataset.file_name in YAML due to argparse Dict-type field constraints. ## Does this PR introduce any user-facing change? Yes. Users now configure parallelism, batch size, output dir, and optimization params directly in .sh scripts instead of .yaml files. YAML files only retain model, data, and recompute configs. Affected files: all examples/fsdp2/**/*.sh|.yaml and tests/**/*fsdp2*.sh|.yaml . ## How was this patch tested? Ran pretrain_qwen3_8b_4k_fsdp2.sh in docker with conda env, completed 15 training steps successfully (exit code 0). See merge request: Ascend/MindSpeed-LLM!4722 | 1 个月前 |
| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
| 1 个月前 | ||
| 1 个月前 |