| 文件 | 最后提交记录 | 最后更新时间 |
|---|---|---|
fix: use deterministic placeholder initialization for DeepSeek4 hash layers Co-authored-by: dingzicha1997<dingzilin@huawei.com> # message auto-generated for no-merge-commit merge: !4898 merge codex/hash-layer-init into master fix: use deterministic placeholder initialization for DeepSeek4 hash layers Created-by: dingzicha1997 Commit-by: dingzicha1997 Merged-by: ascend-robot Description: ## What this PR does / why we need it? This PR updates the initialization logic of the DeepSeek4 hash-layer tid2eid table. The previous implementation generated one torch.randperm(moe_router_topk) for each token. This only permuted expert IDs within [0, moe_router_topk) and consumed a large amount of RNG state during model construction. Following NVIDIA/Megatron-LM#4481, this PR: - replaces the random initialization with deterministic round-robin initialization across all experts; - keeps tid2eid as a non-trainable parameter; - preserves the int64 dtype required by the existing torch.gather and torch.scatter routing path; - documents that this initialization is only a placeholder for constructing a runnable model from scratch. DeepSeek4 checkpoints provide a pre-trained tid2eid table. No public recipe for initializing this table for real training is currently available, so the placeholder initialization in this PR must not be treated as a real training initialization strategy. Reference: https://github.com/NVIDIA/Megatron-LM/pull/4481 The remaining changes in router.py are formatting and static-check fixes produced while making the file pass the repository pre-commit checks. They do not change the existing routing behavior. ## Does this PR introduce any user-facing change? No CLI or configuration interface is changed. For models loaded from a valid DeepSeek4 checkpoint, the checkpoint-provided tid2eid table overwrites the placeholder value, so checkpoint loading behavior is unchanged. For hash-layer models constructed without a checkpoint, the initial routing table changes from random permutations limited to the first moe_router_topk experts to deterministic round-robin assignments across all experts. The placeholder initialization is not suitable for real-world training. ## How was this patch tested? - Ran the complete pre-commit checks for: - mindspeed_llm/core/transformer/moe/router.py - Verified: - tid2eid shape and deterministic round-robin values; - expert indices remain within the configured expert range; - tid2eid remains an int64 non-trainable parameter; - compatibility with the existing torch.gather and torch.scatter routing path; - non-hash layers do not create tid2eid. - Ran the DeepSeek4 checkpoint-loading ST case for 15 iterations. Training completed successfully and the checkpoint loaded without state-dict errors. - The existing DeepSeek4 ST baseline JSON is intentionally not modified by this PR. See merge request: Ascend/MindSpeed-LLM!4898 | 1 小时前 | |
feat(moe): add triton swiglu with limit Co-authored-by: wlf-darkmatter<wlf_x_contion@sina.com> # message auto-generated for no-merge-commit merge: !4864 merge triton-swiglu-limit into master feat(moe): add triton swiglu with limit Created-by: wlf-darkmatter Commit-by: wlf-darkmatter Merged-by: ascend-robot Description: ## What this PR does / why we need it? 在已有 --swiglu-limit 能力（torch + npu_swiglu）基础上，新增 Triton Ascend 融合实现，将 clamp + SiLU + mul 的前反向合并为单个 kernel，降低激活函数路径上的算子调度与中间显存开销。 主要变更： 1. 新增 mindspeed_llm/ops/triton/triton_swiglu_with_limit.py：实现 SwiGLU-with-limit 的 Triton fwd/bwd（含 autotune）。 2. 在 fused_bias_swiglu.py 增加 triton_swiglu_limit 入口，复用全局 swiglu_limit，并兼容 2D/3D 输入。 3. 在 SwigluLimitFeature 中新增 --use-triton-swiglu-limit；与 --swiglu-limit > 0 同时开启时 patch 到 Triton 实现，否则保持原有 fused_swiglu_with_limit 行为。 实测表明，在 950 芯片上，该算子性能相比于 fused_bias_swiglu.py 中的实现，性能提升 200% 到 300%左右。 ## Does this PR introduce any user-facing change? Yes. 新增可选参数 --use-triton-swiglu-limit。 - 默认关闭，不影响现有 --swiglu-limit / fused swiglu 路径。 - 开启后需同时设置 --swiglu-limit > 0，才会走 Triton Ascend kernel。 示例： bash --swiglu-limit 7.0 \ --use-triton-swiglu-limit refer to issue https://gitcode.com/Ascend/MindSpeed-LLM/issues/1710 See merge request: Ascend/MindSpeed-LLM!4864 | 4 天前 | |
docs: add loss_ctx parameter docstring for DeepseekV4 and MiniMaxM2 CausalLM Co-authored-by: Rostellaria<xiaoshaoan@huawei.com> # message auto-generated for no-merge-commit merge: !4930 merge master into master docs: add loss_ctx parameter docstring for DeepseekV4 and MiniMaxM2 CausalLM Created-by: Rostellaria Commit-by: Rostellaria Merged-by: ascend-robot Description: [#1745](https://gitcode.com/Ascend/MindSpeed-LLM/issues/1745) ## What this PR does / why we need it? This PR documents the previously undocumented loss_ctx parameter in the forward methods of DeepseekV4ForCausalLM and MiniMaxM2ForCausalLM , clarifying its callable signature, behavior, and the side-effect that labels remain active for DeepseekV4's MTP loss path. Without this, callers cannot discover the parameter from auto-generated docs and may be surprised by MTP still consuming labels. ## Does this PR introduce any user-facing change? No runtime behavior changes; this is a docstring-only patch. The added descriptions appear in the auto-generated API docstring for DeepseekV4ForCausalLM.forward at modeling_deepseek_v4.py#L1704-L1707 and MiniMaxM2ForCausalLM.forward at modeling_minimax_m2.py#L823-L826 . ### How was this patch tested? Verification is static: the rendered docstring was cross-checked against the loss_ctx branching logic in both forward methods and the LMHead.forward implementation in modules.py#L8-L32 to confirm signature, logits-behavior, and the DeepseekV4 MTP label-usage note match actual code semantics. No runtime tests are required for a doc-only change. See merge request: Ascend/MindSpeed-LLM!4930 | 3 天前 | |
feat: Add DeepSeek-V4 inference module Co-authored-by: sunjunjie1587<sunjunjie8@huawei.com> # message auto-generated for no-merge-commit merge: !4660 merge master into master feat: Add DeepSeek-V4 inference module Created-by: sunjunjie1587 Commit-by: sunjunjie1587 Merged-by: ascend-robot Description: ## What this PR does / why we need it? This PR adds an independent DeepSeek-V4 inference path for MindSpeed-LLM. DeepSeek-V4 uses a model implementation different from the existing GPT inference model, so it cannot cleanly share the original inference.py GPT model provider. This PR keeps the original inference path unchanged and adds a dedicated inference_deepseek4.py entrypoint plus DeepSeek4-specific inference wrappers. Main changes: - Add inference_deepseek4.py as the DeepSeek-V4 inference entrypoint. - Add DeepSeek4-specific inference modules under mindspeed_llm/tasks/inference/: - deepseek4_module.py: inherits the common inference module and isolates DeepSeek4-specific generation behavior. - infer_deepseek4.py: inherits/reuses common inference task logic and isolates DeepSeek4 chat streaming/stdout handling. - Keep common inference modules compatible with the original GPT inference flow, avoiding behavior changes to existing inference.py usage. - Add DeepSeek4 example script: - examples/mcore/deepseek4_flash/generate_deepseek4_flash_4k_A3_ptd.sh - Support DeepSeek4-specific inference requirements, including MTP inference state handling, multi-EOS truncation, streaming delta output, MHC receive-buffer shape adaptation, and DeepSeek4 DSA/G2 attention index alignment. ## Does this PR introduce any user-facing change? Yes. This PR introduces a new DeepSeek-V4 inference entrypoint and example script. Users running DeepSeek-V4 inference should use: bash bash examples/mcore/deepseek4_flash/generate_deepseek4_flash_4k_A3_ptd.sh or directly run: bash torchrun ... inference_deepseek4.py ... Existing GPT/model inference through inference.py remains unchanged. Related files: - inference_deepseek4.py - examples/mcore/deepseek4_flash/generate_deepseek4_flash_4k_A3_ptd.sh - mindspeed_llm/tasks/inference/deepseek4_module.py - mindspeed_llm/tasks/inference/infer_deepseek4.py ## How was this patch tested? This patch was verified with static and syntax checks: bash PYTHONPYCACHEPREFIX=/private/tmp/mindspeed_pycache python3 -m py_compile \ inference_deepseek4.py \ mindspeed_llm/tasks/inference/infer_deepseek4.py \ mindspeed_llm/tasks/inference/deepseek4_module.py \ mindspeed_llm/tasks/inference/infer_base.py \ mindspeed_llm/tasks/inference/module.py bash bash -n examples/mcore/deepseek4_flash/generate_deepseek4_flash_4k_A3_ptd.sh bash git diff --check origin/master..HEAD Expected runtime verification: 1. Prepare DeepSeek-V4 tokenizer and Megatron-format checkpoint. 2. Update TOKENIZER_PATH and CHECKPOINT in examples/mcore/deepseek4_flash/generate_deepseek4_flash_4k_A3_ptd.sh. 3. Launch the script on the target NPU environment. 4. Enter chat prompts and verify that output is streamed normally without repeated full-text printing or delayed large blank output. Constraints and limitations: - DeepSeek-V4 inference should use inference_deepseek4.py, not the generic inference.py. - The provided example is configured for the DeepSeek4 Flash 4K setup and may need TP/PP/EP/CP/path adjustments for other clusters. - Full end-to-end NPU inference needs to be validated in the target hardware environment with the actual checkpoint. 验证记录： INFO:root:=========================================================== INFO:root:1. If you want to quit, please entry one of [q, quit, exit] INFO:root:2. To create new title, please entry one of [clear, new] INFO:root:=========================================================== You >> nihao MindSpeed-LLM: 你好！有什么我可以帮你的吗？😊 You >> nishishei /usr/local/python3.11.15/lib/python3.11/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via init_process_group or barrier . Using the current device set by the user. warnings.warn( # warn only once MindSpeed-LLM: 我是 **DeepSeek，由深度求索公司创造的AI助手！我是 **DeepSeek**，由深度求索公司创造的AI助手！🤖 简单介绍一下自己： -身份纯文本模型知识截止于2025年月能力可以回答问题、写作翻译编程分析文件等特点免费使用支持上传图片PDFWord还能联网搜索（需要手动开启） 版本最新你可以把我当成一个乐于人的智能有什么尽管问我无论是学习工作还是日常闲聊都很乐意帮忙。我是 **DeepSeek**，由深度求索公司创造的AI助手！🤖 简单介绍一下自己： - **身份**：纯文本AI模型，知识截止于2025年5月 - **能力**：可以回答问题、写作、翻译、编程、分析文件等 - **特点**：免费使用，支持上传图片、PDF、Word等文件，还能联网搜索（需要手动开启） - **版本**：DeepSeek最新版模型 你可以把我当成一个乐于助人的智能助手，有什么问题尽管问我！无论是学习、工作还是日常闲聊，我都很乐意帮忙。😊那么今天我可以为你做的吗？ See merge request: Ascend/MindSpeed-LLM!4660 | 1 个月前 | |
[pytorch][feature]Decouple the 'variable-seq-lengths' and 'no-pad-to-seq-lengths' parameters Co-authored-by: mhh001<mahonghao1@huawei.com> # message auto-generated for no-merge-commit merge: !3579 merge master_pad into master [pytorch][feature]Decouple the 'variable-seq-lengths' and 'no-pad-to-seq-lengths' parameters Created-by: mhh111 Commit-by: mhh001 Merged-by: ascend-robot Description: [pytorch][feature]Decouple the 'variable-seq-lengths' and 'no-pad-to-seq-lengths' parameters See merge request: Ascend/MindSpeed-LLM!3579 | 9 个月前 | |
feat: deepseek-v4 support cp+tnd Co-authored-by: wanggangguo<wanggangguo@huawei.com> # message auto-generated for no-merge-commit merge: !4842 merge cp-master into master feat: deepseek-v4 support cp+tnd Created-by: isfrapples Commit-by: wanggangguo Merged-by: ascend-robot Description: ## What this PR does / why we need it? 1. Implements TND layout support for kvallgather CP in DeepSeek-v4 attention 2. Adds prefix KV mode with rank-order gather and per-batch rearrangement 3. Refactors kvallgather kernels into TND/BSND layout dispatchers 4. Fixes indexer loss scale capture to avoid multi-layer backward race ## Does this PR introduce any user-facing change? 1. Adds --ring-degree argument for ring context parallel configuration 2. Requires cmp_residual_k explicitly for TND layout in npu_lightning_indexer 3. DeepSeek4 now supports kvallgather_cp_algo with packed TND sequences 4. No breaking changes to existing BSND/CP paths ## How was this patch tested? 1. Verified step loss alignment between CP=1 and CP=2 with TND packed data 2. Confirmed prefix KV correctness via per-layer indexer loss comparison 3. Checked gradient propagation through indexer loss across CP ranks 4. Validated rotary position embedding alignment under continuous CP sharding fixes#1663 See merge request: Ascend/MindSpeed-LLM!4842 | 20 天前 | |
feat(moe): add triton swiglu with limit Co-authored-by: wlf-darkmatter<wlf_x_contion@sina.com> # message auto-generated for no-merge-commit merge: !4864 merge triton-swiglu-limit into master feat(moe): add triton swiglu with limit Created-by: wlf-darkmatter Commit-by: wlf-darkmatter Merged-by: ascend-robot Description: ## What this PR does / why we need it? 在已有 --swiglu-limit 能力（torch + npu_swiglu）基础上，新增 Triton Ascend 融合实现，将 clamp + SiLU + mul 的前反向合并为单个 kernel，降低激活函数路径上的算子调度与中间显存开销。 主要变更： 1. 新增 mindspeed_llm/ops/triton/triton_swiglu_with_limit.py：实现 SwiGLU-with-limit 的 Triton fwd/bwd（含 autotune）。 2. 在 fused_bias_swiglu.py 增加 triton_swiglu_limit 入口，复用全局 swiglu_limit，并兼容 2D/3D 输入。 3. 在 SwigluLimitFeature 中新增 --use-triton-swiglu-limit；与 --swiglu-limit > 0 同时开启时 patch 到 Triton 实现，否则保持原有 fused_swiglu_with_limit 行为。 实测表明，在 950 芯片上，该算子性能相比于 fused_bias_swiglu.py 中的实现，性能提升 200% 到 300%左右。 ## Does this PR introduce any user-facing change? Yes. 新增可选参数 --use-triton-swiglu-limit。 - 默认关闭，不影响现有 --swiglu-limit / fused swiglu 路径。 - 开启后需同时设置 --swiglu-limit > 0，才会走 Triton Ascend kernel。 示例： bash --swiglu-limit 7.0 \ --use-triton-swiglu-limit refer to issue https://gitcode.com/Ascend/MindSpeed-LLM/issues/1710 See merge request: Ascend/MindSpeed-LLM!4864 | 4 天前 | |
fix: Fix the issue where the LoRA A/B matrices of the DeepSeek V4 linear_o_down_proj layer are not updated. Co-authored-by: xiejiahao2333<xiejiahao11@huawei.com> # message auto-generated for no-merge-commit merge: !4922 merge fix-lora-down-proj into master fix: Fix the issue where the LoRA A/B matrices of the DeepSeek V4 linear_o_down_proj layer are not updated. Created-by: xiejiahao2333 Commit-by: xiejiahao2333 Merged-by: ascend-robot Description: https://gitcode.com/Ascend/MindSpeed-LLM/issues/1772 ## What this PR does / why we need it? Please describe the background and detailed changes of the PR. If it is a bugfix, please attach the related issue. Fix the issue where the A/B matrices of the DeepSeek V4 linear_o_down_proj layer are not updated. 一、**Issue:**：The B matrix of linear_o_down_proj remains all zeros and is not updated. The A matrix also shows cases where it is not updated. 二、**Root cause:**： The original implementation at /opt/MindSpeed-LLM/mindspeed_llm/tasks/models/transformer/deepseek4/g2_attention.py:572 directly reads self.linear_o_down_proj_weight for the einsum computation. This is equivalent to using only the original weight of this layer for computation, without adding the LoRA A/B matrices. As a result, the A/B matrices of linear_o_down_proj do not receive gradients and are not updated. ## Does this PR introduce any user-facing change? Please describe whether the PR will result in any user-facing usage changes. If there is related documentation, please specify its path. **Solution:**： The newly added logic, self._linear_o_down_proj_lora(), works as follows: a. First, check whether linear_o_down_proj has LoRA A/B matrices. b. If LoRA is not enabled, directly return the original output, so normal training is not affected. c. If LoRA is enabled, retrieve lora_A, lora_B, and scaling. d. Use the input grouped_o to compute an additional LoRA branch: python lora_delta = grouped_o * lora_A * lora_B * scaling e. Add the LoRA delta back to the original result: python output = base_output + lora_delta   ## How was this patch tested? Please explain how to verify the correctness and effectiveness of this feature, as well as its usage constraints and limitations. Commit ID: 1ed31e0e36aefdd2e868ec8e865164fd0b030480 Training configuration: { "task_type": "sft", "model_type": "deepseek4", "peft_type": "full", "custom": { "use_mp_training": true, "train_dir": "/dsV4/train_chatml.json" }, "train": { "tokenizer-name-or-path": "/models/deepseek-ai/DeepSeek-V4-Flash-BF16", "tokenizer-type": "PretrainedFromHF", "load": "/dsV4/weight/DeepSeek-V4-Flash-0731-Megatron-pp4tp1ep4", "save": "./save_lora", "prompt_type": "deepseek4", "finetune": true, "no-load-optim": true, "no-load-rng": true, "seed": 1234, "split": "100,0,0", "lr": 1e-5, "min-lr": "1.0e-7", "lr-decay-style": "cosine", "lr-warmup-iters": 5, "bf16": true, "weight-decay": "1e-2", "clip-grad": "1.0", "adam-beta1": "0.9", "adam-beta2": "0.999", "train-iters": 2000, "micro-batch-size": 1, "global-batch-size": 128, "tensor-model-parallel-size": 1, "pipeline-model-parallel-size": 4, "expert-model-parallel-size": 4, "expert-tensor-parallel-size": 1, "compress-rope-theta": 160000.0, "context-parallel-size": 1, "rope_factor": 16.0, "sequence-parallel": true, "seq-length": 4096, "mtp-num-layers": 0, "initial-loss-scale": 65536.0, "log-interval": 1, "eval-interval": 1000, "eval-iters": 1000, "save-interval": 50, "use-flash-attn": true, "use-distributed-optimizer": true, "tensorboard-queue-size": 1000, "tensorboard-dir": "./log_dir", "recompute-granularity": "full", "swap-optimizer": true, "use-mcore-models": true, "is-instruction-dataset": true, "use-triton-mhc": false, "ckpt-format": "torch", "moe-permute-fusion": true, "moe-grouped-gemm": true, "moe-shared-expert-overlap": false, "recompute-method": "uniform", "recompute-num-layers": 1, "no-save-optim": true, "no-save-rng": true, "log-throughput": true, "fix-router": false, "lora-r": 16, "lora-alpha": 32, "lora-fusion": true, "lora-target-modules": ["linear_q", "linear_kv", "linear_fc1", "linear_fc2", "linear_q_up_proj", "linear_o_down_proj", "linear_o_up_proj"], "moe-token-dispatcher-type": "alltoall_seq" } } Loss curve:  Update status of the A/B matrices：  vLLM test result after training:  See merge request: Ascend/MindSpeed-LLM!4922 | 22 小时前 | |
fix(pytorch)：fix kvallgather for pipeline. Co-authored-by: yanzhixiao<yanzhixiao@h-partners.com> # message auto-generated for no-merge-commit merge: !4409 merge bugfix-pipeline-0414 into master fix(pytorch)：fix kvallgather for pipeline. Created-by: yanzhixiao23 Commit-by: yanzhixiao Merged-by: ascend-robot Description: ## What this PR does / why we need it? Fix kvallgather for pipeline. ## Does this PR introduce any user-facing change? NA ## How was this patch tested? NA See merge request: Ascend/MindSpeed-LLM!4409 | 3 个月前 | |
feat: extend train_from_hf support for legacy scripts Co-authored-by: daixzh<daixuezheng@huawei.com> # message auto-generated for no-merge-commit merge: !4880 merge Train_from_HF into master feat: extend train_from_hf support for legacy scripts Created-by: daixzh Commit-by: daixzh Merged-by: ascend-robot Description: ## What this PR does / why we need it? Enabled automatic data preprocessing for all training scripts under examples and tests/poc. Enabled automatic weight conversion for training scripts of all models below 300B parameters under examples and tests/poc (excluding DeepSeek3, DeepSeek4, and LongCat). ## Does this PR introduce any user-facing change? The updated scripts integrate data preprocessing, weight conversion, and training into a single workflow, allowing users to launch training jobs with a single script. The default --tokenizer-type in the Llama2 training script has been changed from Llama2Tokenizer to PretrainedFormHF, and --tokenizer-model ${TOKENIZER_MODEL} has been changed to --tokenizer-name-or-path ${TOKENIZER_MODEL}. TOKENIZER_MODEL only needs to be configured to the directory containing the target model's tokenizer. ## How was this patch tested? This PR was validated through smoke testing. See merge request: Ascend/MindSpeed-LLM!4880 | 1 小时前 | |
[pytorch][feature]FSDP2 backend supports the Transformers training process. Co-authored-by: sunjunjie1587<sunjunjie8@huawei.com> # message auto-generated for no-merge-commit merge: !4043 merge master into master [pytorch][feature]FSDP2 backend supports the Transformers training process. Created-by: sunjunjie1587 Commit-by: sunjunjie1587 Merged-by: ascend-robot Description: FSDP2 backend supports the Transformers training process. See merge request: Ascend/MindSpeed-LLM!4043 | 6 个月前 |