Star197
282
代码
代码
Issues55
Pull Requests78
流水线
项目讨论
Wiki
分析
Star197
282
ascend-robotascend-robotdocs(pytorch): Add document for en
78008ac4创建于 7月2日历史提交

GitHub Documentation

Introduction

MindSpeed LLM is a distributed large language model training toolkit built on the Ascend ecosystem. It aims to provide ecosystem partners in the Huawei Ascend chips ecosystem with an end-to-end large language model training solution, including distributed pretraining, distributed instruction fine-tuning, and the corresponding development toolchain, such as data preprocessing, weight conversion, online inference, and baseline evaluation.

Note: The original repository name, ModelLink, has been changed to MindSpeed LLM, and the original package name, modellink, has been changed to mindspeed_llm.

Roadmap

The latest roadmap updates are published in MindSpeed LLM RoadMap. Visit there for the latest LLM plans and updates.

Community Meetings

Please see the Ascend Meeting Center for the schedule of MindSpeed LLM TC and SIG meetings.

Join Us

To share development experience, exchange usage tips, and stay up to date on project releases, we created the MindSpeed LLM community group. Whether you already use this project or have new ideas, you are welcome to join.

Ways to join:

  1. Scan the QR code to join the WeChat group directly. The QR code is valid for 7 days and is updated regularly.
  2. Add the Ascend open-source assistant to get the group link and join the MindSpeed LLM community group.
MindSpeed LLM community group
MindSpeed LLM WeChat group
Ascend open-source assistant
Ascend assistant WeChat

Latest News

Note: 【Prototype】 indicates that the feature has not been fully validated. If you encounter any issues while using it, please report them in the issue tracker.

Directory Structure

The MindSpeed LLM project code is organized according to modular design principles. For details, see the Project Guide.

MindSpeed-LLM/
 ├── ci                        # CI watchdog.
 ├── configs                   # Configuration files.
 ├── docs                      # Project documentation.
 ├── examples                  # Model example scripts.
 ├── mindspeed_llm             # Core code.
 ├── tests                     # Test cases.
 ├── convert_ckpt.py           # Weight conversion tool.
 ├── convert_ckpt_v2.py        # Weight conversion tool v2.
 ├── preprocess_data.py        # Data preprocessing tool.
 ├── pretrain_gpt.py           # Pretraining workflow.
 ├── pretrain_mamba.py         # Pretraining workflow for Mamba models.
 ├── posttrain_gpt.py          # Post-training workflow.
 ├── preprocess_prompt.py      # Prompt preprocessing tool.
 ├── rlhf_gpt.py               # RLHF training workflow.
 ├── train_fsdp2.py            # FSDP2 training workflow.
 ├── inference.py              # Model inference tool.
 ├── evaluation.py             # Model evaluation tool.
 ├── setup.py                  # Installation configuration file.
 ├── README.md                 # Project overview document.

Documentation Navigation

The Documentation Guide provides the complete usage guide for MindSpeed LLM and covers the following core topics:

  • Environment setup guide: installation and configuration instructions for MindSpeed LLM.
  • Quick start: beginner guidance from environment setup to launching training.
  • Model list: models supported by the PyTorch and MindSpore frameworks.
  • Feature list: performance optimization and memory optimization features.
  • Training solutions: complete solutions for pretraining, fine-tuning, inference, and evaluation.
  • Toolchain: usage instructions for weight conversion, dataset processing, performance collection and analysis, deterministic computation, and other tools.

Release Notes

See the release notes.

Installation

Quick Start

This section helps you quickly launch large language model pretraining and fine-tuning tasks. See the following guides for details:

Supported Models

MindSpeed LLM now includes built-in support for pretraining and fine-tuning more than 100 widely used large language models. See the supported model lists:

Training Solutions and Features

MindSpeed LLM includes training solutions such as distributed pretraining and distributed fine-tuning. For details, see Training Solutions and Features.

Online Inference

Feature MCore Released Contributor
Streaming inference ✅ ✅ 【NAIE】
Chat conversation ✅ ✅ 【NAIE】
YARN context extension ✅ ❌ 【Ascend】

Open Dataset Evaluation

See the open dataset evaluation baselines for model baselines in the repository.

Scenario Dataset MCore Released Contributor
Evaluation MMLU ✅ ❌ 【NAIE】
CEval ✅ ❌ 【NAIE】
BoolQ ✅ ❌ 【NAIE】
BBH ✅ ❌ 【NAIE】
AGIEval ✅ ❌ 【NAIE】
HumanEval ✅ ❌ 【NAIE】

Developer Toolchain

Weight Conversion

MindSpeed LLM supports two-way weight conversion between Hugging Face and Megatron-core formats, and it supports merging LoRA weights. For parameters and usage instructions for the weight conversion feature, see Weight Conversion.

Source format Target format Sharding features LoRA Contributor Released
Hugging Face Megatron-core tp, pp, dpp, vpp, cp, ep, loop layer ❌ 【Ascend】 ❌
Megatron-core Hugging Face ✅
Megatron-core tp, pp, dpp, vpp, cp, ep, loop layer ✅

Data Preprocessing

MindSpeed LLM supports data preprocessing for pretraining, instruction fine-tuning, and other tasks.

Task scenario Dataset MCore Released Contributor
Pretraining Pretraining data processing ✅ ✅ 【Ascend】
Fine-tuning Alpaca style ✅ ✅
ShareGPT style ✅ ✅

Performance Collection

Scenario Feature MCore Released Contributor
Performance collection Collect profiling data on Ascend chips ✅ ❌ 【Ascend】

High Availability

Scenario Feature MCore Released Contributor
High availability Enable deterministic computation on Ascend chips ✅ ❌ 【Ascend】

Version Maintenance Policy

MindSpeed LLM versions go through the following five maintenance stages:

Status Time Description
Planning 1 to 3 months Planned features.
Development 3 months Features under development.
Maintenance 6 to 12 months Merge all resolved issues and release versions. Different MindSpeed LLM versions follow different maintenance strategies. The maintenance period is six months for regular releases and twelve months for long-term support releases.
No maintenance 0 to 3 months Merge all resolved issues. No dedicated maintenance staff. No version release.
End of life (EOL) N/A The branch no longer accepts any changes.

MindSpeed LLM released version maintenance policy:

MindSpeed LLM version Corresponding tag Maintenance policy Current status Release date Next status EOL date
2.3.0 v2.3.0 Regular release Maintenance 2025/12/30 Expected to enter no-maintenance status starting 2026/6/30
2.2.0 v2.2.0 Regular release Maintenance 2025/9/30 Expected to enter no-maintenance status starting 2026/3/30
2.1.0 v2.1.0 Regular release EOL 2025/6/30 End of life 2025/12/30
2.0.0 v2.0.0 Regular release EOL 2025/3/30 End of life 2025/9/30
1.0.0 v1.0.0 Regular release EOL 2024/12/30 End of life 2025/6/30
1.0.RC3 v1.0.RC3.0 Regular release EOL 2024/09/30 End of life 2025/3/30
1.0.RC2 v1.0.RC2.0 Regular release EOL 2024/06/30 End of life 2024/12/30
1.0.RC1 v1.0.RC1.0 Regular release EOL 2024/03/30 End of life 2024/9/30
bk_origin_23 \ Demo EOL 2023 End of life 2024/6/30

Security Statement

MindSpeed LLM Security Statement

Disclaimer

To MindSpeed LLM Users

  1. The models provided by MindSpeed LLM are for non-commercial use only.
  2. Third-party open source software that MindSpeed LLM depends on, such as Megatron, is provided and maintained by third-party communities. Fixes for problems caused by third-party open source software depend on contributions from and feedback to the relevant communities. You should understand that the MindSpeed LLM repository does not guarantee fixes for problems in the third-party open source software itself, and it also does not guarantee that it tests or corrects all vulnerabilities and errors in third-party open source software.
  3. For each model, the MindSpeed LLM platform only suggests datasets that you can use for training. Huawei does not provide any datasets. If you use these datasets for training, you must especially ensure that you comply with the license of the corresponding dataset. If you become involved in an infringement dispute because of your use of a dataset, Huawei bears no responsibility.
  4. If you find any issues while using MindSpeed LLM models, including but not limited to functional issues and compliance issues, please submit an issue on GitCode. We will review and address it in a timely manner.

To Dataset Owners

If you do not want your dataset to be mentioned in MindSpeed LLM model descriptions, or if you want to update the description of your dataset in MindSpeed LLM, please submit an issue on GitCode. We will delete or update the dataset description according to your request. We sincerely thank you for your understanding and contribution to MindSpeed LLM.

License Statement

  • The license for the MindSpeed LLM product is described in LICENSE.
  • The documents in the docs directory of the MindSpeed LLM tool are subject to the CC-BY 4.0 license. See LICENSE for details.

Contribution Statement

If you want to report issues or contribute code to MindSpeed LLM, see the Contribution Guide.

FAQ

For common questions about basic MindSpeed LLM usage, see the MindSpeed LLM FAQ. If the FAQ does not cover your question, you can search for similar issues in the repository issues list, or submit a new issue.

Acknowledgments

MindSpeed LLM is jointly contributed by the following Huawei departments and Ascend ecosystem partners:

Huawei:

  • Computing Product Line: Ascend
  • Public Development Department: NAIE
  • Global Technical Service Department: GTS
  • Huawei Cloud Computing: Cloud

Ecosystem partners:

  • Mobile Cloud (China Mobile Cloud): Dayun Zhenze Intelligent Computing Platform
  • Big Data and Artificial Intelligence Lab of the Software Development Center of Industrial and Commercial Bank of China

Thank you for every PR from the community. Contributions to MindSpeed LLM are welcome.