| feat: Add DeepSeek-V4 inference module Co-authored-by: sunjunjie1587<sunjunjie8@huawei.com> # message auto-generated for no-merge-commit merge: !4660 merge master into master feat: Add DeepSeek-V4 inference module Created-by: sunjunjie1587 Commit-by: sunjunjie1587 Merged-by: ascend-robot Description: ## What this PR does / why we need it? This PR adds an independent DeepSeek-V4 inference path for MindSpeed-LLM. DeepSeek-V4 uses a model implementation different from the existing GPT inference model, so it cannot cleanly share the original inference.py GPT model provider. This PR keeps the original inference path unchanged and adds a dedicated inference_deepseek4.py entrypoint plus DeepSeek4-specific inference wrappers. Main changes: - Add inference_deepseek4.py as the DeepSeek-V4 inference entrypoint. - Add DeepSeek4-specific inference modules under mindspeed_llm/tasks/inference/: - deepseek4_module.py: inherits the common inference module and isolates DeepSeek4-specific generation behavior. - infer_deepseek4.py: inherits/reuses common inference task logic and isolates DeepSeek4 chat streaming/stdout handling. - Keep common inference modules compatible with the original GPT inference flow, avoiding behavior changes to existing inference.py usage. - Add DeepSeek4 example script: - examples/mcore/deepseek4_flash/generate_deepseek4_flash_4k_A3_ptd.sh - Support DeepSeek4-specific inference requirements, including MTP inference state handling, multi-EOS truncation, streaming delta output, MHC receive-buffer shape adaptation, and DeepSeek4 DSA/G2 attention index alignment. ## Does this PR introduce any user-facing change? Yes. This PR introduces a new DeepSeek-V4 inference entrypoint and example script. Users running DeepSeek-V4 inference should use: bash bash examples/mcore/deepseek4_flash/generate_deepseek4_flash_4k_A3_ptd.sh or directly run: bash torchrun ... inference_deepseek4.py ... Existing GPT/model inference through inference.py remains unchanged. Related files: - inference_deepseek4.py - examples/mcore/deepseek4_flash/generate_deepseek4_flash_4k_A3_ptd.sh - mindspeed_llm/tasks/inference/deepseek4_module.py - mindspeed_llm/tasks/inference/infer_deepseek4.py ## How was this patch tested? This patch was verified with static and syntax checks: bash PYTHONPYCACHEPREFIX=/private/tmp/mindspeed_pycache python3 -m py_compile \ inference_deepseek4.py \ mindspeed_llm/tasks/inference/infer_deepseek4.py \ mindspeed_llm/tasks/inference/deepseek4_module.py \ mindspeed_llm/tasks/inference/infer_base.py \ mindspeed_llm/tasks/inference/module.py bash bash -n examples/mcore/deepseek4_flash/generate_deepseek4_flash_4k_A3_ptd.sh bash git diff --check origin/master..HEAD Expected runtime verification: 1. Prepare DeepSeek-V4 tokenizer and Megatron-format checkpoint. 2. Update TOKENIZER_PATH and CHECKPOINT in examples/mcore/deepseek4_flash/generate_deepseek4_flash_4k_A3_ptd.sh. 3. Launch the script on the target NPU environment. 4. Enter chat prompts and verify that output is streamed normally without repeated full-text printing or delayed large blank output. Constraints and limitations: - DeepSeek-V4 inference should use inference_deepseek4.py, not the generic inference.py. - The provided example is configured for the DeepSeek4 Flash 4K setup and may need TP/PP/EP/CP/path adjustments for other clusters. - Full end-to-end NPU inference needs to be validated in the target hardware environment with the actual checkpoint. 验证记录： INFO:root:=========================================================== INFO:root:1. If you want to quit, please entry one of [q, quit, exit] INFO:root:2. To create new title, please entry one of [clear, new] INFO:root:=========================================================== You >> nihao MindSpeed-LLM: 你好！有什么我可以帮你的吗？😊 You >> nishishei /usr/local/python3.11.15/lib/python3.11/site-packages/torch/distributed/distributed_c10d.py:4631: UserWarning: No device id is provided via init_process_group or barrier . Using the current device set by the user. warnings.warn( # warn only once MindSpeed-LLM: 我是 **DeepSeek，由深度求索公司创造的AI助手！我是 **DeepSeek**，由深度求索公司创造的AI助手！🤖 简单介绍一下自己： -身份纯文本模型知识截止于2025年月能力可以回答问题、写作翻译编程分析文件等特点免费使用支持上传图片PDFWord还能联网搜索（需要手动开启） 版本最新你可以把我当成一个乐于人的智能有什么尽管问我无论是学习工作还是日常闲聊都很乐意帮忙。我是 **DeepSeek**，由深度求索公司创造的AI助手！🤖 简单介绍一下自己： - **身份**：纯文本AI模型，知识截止于2025年5月 - **能力**：可以回答问题、写作、翻译、编程、分析文件等 - **特点**：免费使用，支持上传图片、PDF、Word等文件，还能联网搜索（需要手动开启） - **版本**：DeepSeek最新版模型 你可以把我当成一个乐于助人的智能助手，有什么问题尽管问我！无论是学习、工作还是日常闲聊，我都很乐意帮忙。😊那么今天我可以为你做的吗？ See merge request: Ascend/MindSpeed-LLM!4660 | 4 个月前 |