| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
refactor: remove deepseek4 redundant Triton paths and align CSA/MHC CLI Co-authored-by: dingzicha1997<dingzilin@huawei.com> # message auto-generated for no-merge-commit merge: !4871 merge remove-triton-mhc into master refactor: remove deepseek4 redundant Triton paths and align CSA/MHC CLI Created-by: dingzicha1997 Commit-by: dingzicha1997 Merged-by: ascend-robot Description: ## What this PR does / why we need it? This PR simplifies the DeepSeek-V4 CSA and MHC implementations after the Ascend fused paths became stable. Main changes: - Remove redundant Triton MHC, Sinkhorn, RMSNorm, and SFA implementations. - Retain the Triton pre_only kernel required by the MHC head stage. - Keep eager implementations as fallback paths. - Rename G2-related components to DeepSeek-V4 CSA/HCA terminology. - Relocate MHC operators into the common ops directories. - Consolidate CSA eager attention implementations. - Align DeepSeek-V4 CLI arguments with Megatron and MindSpeed conventions. - Remove incompatible legacy RoPE options from all DeepSeek-V4 scripts. ## Does this PR introduce any user-facing change? Yes. DeepSeek-V4 CLI configurations need to be updated as follows: | Type | Previous option | New option / behavior | |---|---|---| | Renamed | --use-ascend-mhc | --use-fused-mhc | | Renamed | --g2-window-size 128 | --sliding-window-size 128 | | Removed | --rope-head-dim 64 | --qk-pos-emb-head-dim 64 | | Removed | --norm-eps 1e-6 | --norm-epsilon 1e-6 | | Renamed | --position-embedding-type g2 | --position-embedding-type deepseek4 | | Replaced | --use-g2-indexer-loss | --indexer-loss-coeff 1.0; use 0 for inference | | Removed | --use-triton-sfa | --use-sparse-flash-attn | | Removed | --use-g2-attention | No replacement required | | Removed | --use-triton-sinkhorn | Use fused MHC or the eager fallback | | Removed | --use-triton-rmsnorm-without-weight | Use fused MHC or the eager fallback | | Removed | --mhc-recompute | No replacement required | | Removed from V4 scripts | --use-fused-rotary-pos-emb | Incompatible with the DeepSeek-V4 RoPE path | | Removed from V4 scripts | --use-rotary-position-embeddings | Overrides the DeepSeek-V4 position embedding type | | Retained | --use-triton-mhc | Only controls the MHC-head pre_only kernel | | Behavior changed | --use-fused-lightning-indexer and --use-fused-lightning-indexer-loss | Must be enabled or disabled together; the framework automatically enables both if only one is provided | Recommended DeepSeek-V4 position embedding configuration: shell --position-embedding-type deepseek4 --qk-pos-emb-head-dim 64 Do not combine it with: shell --use-fused-rotary-pos-emb --use-rotary-position-embeddings ## How was this patch tested? - Checked all DeepSeek-V4 training, fine-tuning, inference, POC, and ST scripts. - Verified that removed CLI options have no remaining DeepSeek-V4 references. - Ran bash -n against all affected DeepSeek-V4 shell scripts. - Ran Python compilation and applicable code-quality checks for the modified Python modules. - Ran git diff --check. - Applicable local pre-commit hooks passed. The local Gitleaks hook could not run because the ./gitleaks binary was unavailable. - GitCode remote Git Hooks passed successfully. - Verified that importing the repository without Triton remains protected by the existing try/except logic. See merge request: Ascend/MindSpeed-LLM!4871 | 1 个月前 | |
fix(megatron): dsv4flash scripts fix Co-authored-by: guihaowen666<guihaowen@huawei.com> # message auto-generated for no-merge-commit merge: !5043 merge br_master_dsv4_scripts_fix into master fix(megatron): dsv4flash scripts fix Created-by: guihaowen666 Commit-by: guihaowen666 Merged-by: ascend-robot Description: ## What this PR does / why we need it? DeepSeekV4-Flash Model Script Parameter Update ## Does this PR introduce any user-facing change? Script format updated, no user-facing changes ## How was this patch tested? Local verification and CI gate verification passed. See merge request: Ascend/MindSpeed-LLM!5043 | 1 天前 | |
fix(megatron): dsv4flash scripts fix Co-authored-by: guihaowen666<guihaowen@huawei.com> # message auto-generated for no-merge-commit merge: !5043 merge br_master_dsv4_scripts_fix into master fix(megatron): dsv4flash scripts fix Created-by: guihaowen666 Commit-by: guihaowen666 Merged-by: ascend-robot Description: ## What this PR does / why we need it? DeepSeekV4-Flash Model Script Parameter Update ## Does this PR introduce any user-facing change? Script format updated, no user-facing changes ## How was this patch tested? Local verification and CI gate verification passed. See merge request: Ascend/MindSpeed-LLM!5043 | 1 天前 | |
fix(megatron): dsv4flash scripts fix Co-authored-by: guihaowen666<guihaowen@huawei.com> # message auto-generated for no-merge-commit merge: !5043 merge br_master_dsv4_scripts_fix into master fix(megatron): dsv4flash scripts fix Created-by: guihaowen666 Commit-by: guihaowen666 Merged-by: ascend-robot Description: ## What this PR does / why we need it? DeepSeekV4-Flash Model Script Parameter Update ## Does this PR introduce any user-facing change? Script format updated, no user-facing changes ## How was this patch tested? Local verification and CI gate verification passed. See merge request: Ascend/MindSpeed-LLM!5043 | 1 天前 | |
chore(megatron) DeepSeek V4 A3 long scripts Co-authored-by: guihaowen666<guihaowen@huawei.com> # message auto-generated for no-merge-commit merge: !5015 merge br_master_v4flash_long_A3_scripts into master chore(megatron) DeepSeek V4 A3 long scripts Created-by: guihaowen666 Commit-by: guihaowen666 Merged-by: ascend-robot Description: ## What this PR does / why we need it? This PR is used to add DeepSeekV4 A3 long sequence (64K, 156K) scripts ## Does this PR introduce any user-facing change? Only scripts are added, and service code is not modified. There is no user-facing change ## How was this patch tested? Local self-validation and CI gate access verification passed. See merge request: Ascend/MindSpeed-LLM!5015 | 11 天前 | |
fix(megatron): dsv4flash scripts fix Co-authored-by: guihaowen666<guihaowen@huawei.com> # message auto-generated for no-merge-commit merge: !5043 merge br_master_dsv4_scripts_fix into master fix(megatron): dsv4flash scripts fix Created-by: guihaowen666 Commit-by: guihaowen666 Merged-by: ascend-robot Description: ## What this PR does / why we need it? DeepSeekV4-Flash Model Script Parameter Update ## Does this PR introduce any user-facing change? Script format updated, no user-facing changes ## How was this patch tested? Local verification and CI gate verification passed. See merge request: Ascend/MindSpeed-LLM!5043 | 1 天前 | |
feat(poc): add DeepSeek4-Flash 256k mxfp8 A5 pretrain scripts Co-authored-by: EVA1<jingsiyu1@huawei.com> # message auto-generated for no-merge-commit merge: !5041 merge master into master feat(poc): add DeepSeek4-Flash 256k mxfp8 A5 pretrain scripts Created-by: EVA1 Commit-by: EVA1 Merged-by: ascend-robot Description: Close #1853 ### 变更内容 在 tests/poc/deepseek4_flash/ 新增两个 256k 长序列 mxfp8 精度（A5）预训练脚本，补齐 256k × mxfp8 组合缺口： 1. **pretrain_deepseek4_flash_256k_fp8_A5_ptd.sh**（全量模型） - 16 机 128 卡，PP=2 / VPP=11，EP=64，CP=32（kvallgather_cp_algo），44 层 + MTP=1，SEQ_LEN=262144 - 在现有 BF16 版本基础上仅新增 --mxfp8-defer-backward-quant --fp8-format e4m3 --fp8-recipe mxfp8 2. **pretrain_deepseek4_flash_3.23b_256k_fp8_A5_ptd.sh**（3.23B 小模型） - 单机 8 卡，CP=8，4 层，SEQ_LEN=262144，GBS=16 - 相对 BF16 版本调整：--indexer-loss-coeff 0.1，开启 --use-triton-mhc --use-fused-mhc，同样启用 mxfp8 三参数 其余参数与对应 BF16 版本保持一致，不修改任何训练运行时代码与公共 API。 ### 自测 - [x] bash -n 语法检查通过（两个脚本） - [x] pre-commit 检查通过（trailing-whitespace / end-of-file-fixer / codespell / typos / detect-private-key / check-added-large-files 等；gitleaks-offline-scan 因本地无 ./gitleaks 二进制按仓库惯例跳过） - [x] 与对应 BF16 脚本逐行 diff 复核，差异仅为上述 mxfp8 参数及明确列出的调优项 See merge request: Ascend/MindSpeed-LLM!5041 | 1 天前 | |
chore: commit the single‑machine pruning script for DeepSeek V4 with 256k long context Co-authored-by: heyuzhen<heyuzhen5@huawei.com> # message auto-generated for no-merge-commit merge: !5020 merge master into master chore: commit the single‑machine pruning script for DeepSeek V4 with 256k long context Created-by: heyuzhen Commit-by: heyuzhen Merged-by: ascend-robot Description: ## What this PR does / why we need it? This PR adds a single‑machine pruning script for DeepSeek V4 with 256k long context, supporting A3/A5 configurations. It enables efficient model pruning to reduce resource consumption for single‑node deployment, while maintaining critical capability for long‑sequence processing. ## Does this PR introduce any user-facing change? No. Only new scripts are added; existing service code is not modified. Therefore, there is no user‑facing change. ## How was this patch tested? The script was validated locally on a single machine with A3/A5 settings, and also passed the CI gate access verification. See merge request: Ascend/MindSpeed-LLM!5020 | 5 天前 | |
chore: commit the single‑machine pruning script for DeepSeek V4 with 256k long context Co-authored-by: heyuzhen<heyuzhen5@huawei.com> # message auto-generated for no-merge-commit merge: !5020 merge master into master chore: commit the single‑machine pruning script for DeepSeek V4 with 256k long context Created-by: heyuzhen Commit-by: heyuzhen Merged-by: ascend-robot Description: ## What this PR does / why we need it? This PR adds a single‑machine pruning script for DeepSeek V4 with 256k long context, supporting A3/A5 configurations. It enables efficient model pruning to reduce resource consumption for single‑node deployment, while maintaining critical capability for long‑sequence processing. ## Does this PR introduce any user-facing change? No. Only new scripts are added; existing service code is not modified. Therefore, there is no user‑facing change. ## How was this patch tested? The script was validated locally on a single machine with A3/A5 settings, and also passed the CI gate access verification. See merge request: Ascend/MindSpeed-LLM!5020 | 5 天前 | |
feat(poc): add DeepSeek4-Flash 256k mxfp8 A5 pretrain scripts Co-authored-by: EVA1<jingsiyu1@huawei.com> # message auto-generated for no-merge-commit merge: !5041 merge master into master feat(poc): add DeepSeek4-Flash 256k mxfp8 A5 pretrain scripts Created-by: EVA1 Commit-by: EVA1 Merged-by: ascend-robot Description: Close #1853 ### 变更内容 在 tests/poc/deepseek4_flash/ 新增两个 256k 长序列 mxfp8 精度（A5）预训练脚本，补齐 256k × mxfp8 组合缺口： 1. **pretrain_deepseek4_flash_256k_fp8_A5_ptd.sh**（全量模型） - 16 机 128 卡，PP=2 / VPP=11，EP=64，CP=32（kvallgather_cp_algo），44 层 + MTP=1，SEQ_LEN=262144 - 在现有 BF16 版本基础上仅新增 --mxfp8-defer-backward-quant --fp8-format e4m3 --fp8-recipe mxfp8 2. **pretrain_deepseek4_flash_3.23b_256k_fp8_A5_ptd.sh**（3.23B 小模型） - 单机 8 卡，CP=8，4 层，SEQ_LEN=262144，GBS=16 - 相对 BF16 版本调整：--indexer-loss-coeff 0.1，开启 --use-triton-mhc --use-fused-mhc，同样启用 mxfp8 三参数 其余参数与对应 BF16 版本保持一致，不修改任何训练运行时代码与公共 API。 ### 自测 - [x] bash -n 语法检查通过（两个脚本） - [x] pre-commit 检查通过（trailing-whitespace / end-of-file-fixer / codespell / typos / detect-private-key / check-added-large-files 等；gitleaks-offline-scan 因本地无 ./gitleaks 二进制按仓库惯例跳过） - [x] 与对应 BF16 脚本逐行 diff 复核，差异仅为上述 mxfp8 参数及明确列出的调优项 See merge request: Ascend/MindSpeed-LLM!5041 | 1 天前 | |
test: update DeepSeek4 Flash A3 4K POC settings Co-authored-by: dingzicha1997<dingzilin@huawei.com> # message auto-generated for no-merge-commit merge: !4970 merge codex/deepseekv4-flash-a3-4k-master into master test: update DeepSeek4 Flash A3 4K POC settings Created-by: dingzicha1997 Commit-by: dingzicha1997 Merged-by: ascend-robot Description: ## What this PR does / why we need it? Update the DeepSeek4 Flash A3 4K PTD POC script: tests/poc/deepseek4_flash/pretrain_deepseek4_flash_4k_A3_ptd.sh Changes: - Set --train-iters from 2000 to 30. - Set --lr-warmup-iters from 500 to 0. - Enable --use-triton-swiglu-limit next to the existing --swiglu-limit 10.0. These changes shorten the POC validation run and enable the Triton SwiGLU limit path. ## Does this PR introduce any user-facing change? No production API or model interface changes are introduced. The default runtime configuration of this POC script is updated as described above. ## How was this patch tested? - Passed bash -n syntax validation. - Passed git diff --check. - Verified the script contains: - --train-iters 30 - --lr-warmup-iters 0 - --use-triton-swiglu-limit A full training run was not performed because it requires the target Ascend NPU environment and model/data paths. See merge request: Ascend/MindSpeed-LLM!4970 | 25 天前 | |
fix(megatron): dsv4flash scripts fix Co-authored-by: guihaowen666<guihaowen@huawei.com> # message auto-generated for no-merge-commit merge: !5043 merge br_master_dsv4_scripts_fix into master fix(megatron): dsv4flash scripts fix Created-by: guihaowen666 Commit-by: guihaowen666 Merged-by: ascend-robot Description: ## What this PR does / why we need it? DeepSeekV4-Flash Model Script Parameter Update ## Does this PR introduce any user-facing change? Script format updated, no user-facing changes ## How was this patch tested? Local verification and CI gate verification passed. See merge request: Ascend/MindSpeed-LLM!5043 | 1 天前 | |
fix(megatron): dsv4flash scripts fix Co-authored-by: guihaowen666<guihaowen@huawei.com> # message auto-generated for no-merge-commit merge: !5043 merge br_master_dsv4_scripts_fix into master fix(megatron): dsv4flash scripts fix Created-by: guihaowen666 Commit-by: guihaowen666 Merged-by: ascend-robot Description: ## What this PR does / why we need it? DeepSeekV4-Flash Model Script Parameter Update ## Does this PR introduce any user-facing change? Script format updated, no user-facing changes ## How was this patch tested? Local verification and CI gate verification passed. See merge request: Ascend/MindSpeed-LLM!5043 | 1 天前 | |
chore(megatron) DeepSeek V4 A3 long scripts Co-authored-by: guihaowen666<guihaowen@huawei.com> # message auto-generated for no-merge-commit merge: !5015 merge br_master_v4flash_long_A3_scripts into master chore(megatron) DeepSeek V4 A3 long scripts Created-by: guihaowen666 Commit-by: guihaowen666 Merged-by: ascend-robot Description: ## What this PR does / why we need it? This PR is used to add DeepSeekV4 A3 long sequence (64K, 156K) scripts ## Does this PR introduce any user-facing change? Only scripts are added, and service code is not modified. There is no user-facing change ## How was this patch tested? Local self-validation and CI gate access verification passed. See merge request: Ascend/MindSpeed-LLM!5015 | 11 天前 | |
fix(megatron): dsv4flash scripts fix Co-authored-by: guihaowen666<guihaowen@huawei.com> # message auto-generated for no-merge-commit merge: !5043 merge br_master_dsv4_scripts_fix into master fix(megatron): dsv4flash scripts fix Created-by: guihaowen666 Commit-by: guihaowen666 Merged-by: ascend-robot Description: ## What this PR does / why we need it? DeepSeekV4-Flash Model Script Parameter Update ## Does this PR introduce any user-facing change? Script format updated, no user-facing changes ## How was this patch tested? Local verification and CI gate verification passed. See merge request: Ascend/MindSpeed-LLM!5043 | 1 天前 | |
refactor: remove deepseek4 redundant Triton paths and align CSA/MHC CLI Co-authored-by: dingzicha1997<dingzilin@huawei.com> # message auto-generated for no-merge-commit merge: !4871 merge remove-triton-mhc into master refactor: remove deepseek4 redundant Triton paths and align CSA/MHC CLI Created-by: dingzicha1997 Commit-by: dingzicha1997 Merged-by: ascend-robot Description: ## What this PR does / why we need it? This PR simplifies the DeepSeek-V4 CSA and MHC implementations after the Ascend fused paths became stable. Main changes: - Remove redundant Triton MHC, Sinkhorn, RMSNorm, and SFA implementations. - Retain the Triton pre_only kernel required by the MHC head stage. - Keep eager implementations as fallback paths. - Rename G2-related components to DeepSeek-V4 CSA/HCA terminology. - Relocate MHC operators into the common ops directories. - Consolidate CSA eager attention implementations. - Align DeepSeek-V4 CLI arguments with Megatron and MindSpeed conventions. - Remove incompatible legacy RoPE options from all DeepSeek-V4 scripts. ## Does this PR introduce any user-facing change? Yes. DeepSeek-V4 CLI configurations need to be updated as follows: | Type | Previous option | New option / behavior | |---|---|---| | Renamed | --use-ascend-mhc | --use-fused-mhc | | Renamed | --g2-window-size 128 | --sliding-window-size 128 | | Removed | --rope-head-dim 64 | --qk-pos-emb-head-dim 64 | | Removed | --norm-eps 1e-6 | --norm-epsilon 1e-6 | | Renamed | --position-embedding-type g2 | --position-embedding-type deepseek4 | | Replaced | --use-g2-indexer-loss | --indexer-loss-coeff 1.0; use 0 for inference | | Removed | --use-triton-sfa | --use-sparse-flash-attn | | Removed | --use-g2-attention | No replacement required | | Removed | --use-triton-sinkhorn | Use fused MHC or the eager fallback | | Removed | --use-triton-rmsnorm-without-weight | Use fused MHC or the eager fallback | | Removed | --mhc-recompute | No replacement required | | Removed from V4 scripts | --use-fused-rotary-pos-emb | Incompatible with the DeepSeek-V4 RoPE path | | Removed from V4 scripts | --use-rotary-position-embeddings | Overrides the DeepSeek-V4 position embedding type | | Retained | --use-triton-mhc | Only controls the MHC-head pre_only kernel | | Behavior changed | --use-fused-lightning-indexer and --use-fused-lightning-indexer-loss | Must be enabled or disabled together; the framework automatically enables both if only one is provided | Recommended DeepSeek-V4 position embedding configuration: shell --position-embedding-type deepseek4 --qk-pos-emb-head-dim 64 Do not combine it with: shell --use-fused-rotary-pos-emb --use-rotary-position-embeddings ## How was this patch tested? - Checked all DeepSeek-V4 training, fine-tuning, inference, POC, and ST scripts. - Verified that removed CLI options have no remaining DeepSeek-V4 references. - Ran bash -n against all affected DeepSeek-V4 shell scripts. - Ran Python compilation and applicable code-quality checks for the modified Python modules. - Ran git diff --check. - Applicable local pre-commit hooks passed. The local Gitleaks hook could not run because the ./gitleaks binary was unavailable. - GitCode remote Git Hooks passed successfully. - Verified that importing the repository without Triton remains protected by the existing try/except logic. See merge request: Ascend/MindSpeed-LLM!4871 | 1 个月前 |
| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
| 1 个月前 | ||
| 1 天前 | ||
| 1 天前 | ||
| 1 天前 | ||
| 11 天前 | ||
| 1 天前 | ||
| 1 天前 | ||
| 5 天前 | ||
| 5 天前 | ||
| 1 天前 | ||
| 25 天前 | ||
| 1 天前 | ||
| 1 天前 | ||
| 11 天前 | ||
| 1 天前 | ||
| 1 个月前 |