Welcome to the meMemScope Memory Analysis Tool Tutorial ✨
🌟 What's New
-
[2026.2.02] 🎉 MindStudio MemScope 26.0.0-alpha.1 launched! It supports data collection via Python APIs, memory snapshot collection in the PyTorch framework, identification of memory page table attributes and flushing, and new memory allocation API of the driver.
-
[2025.12.30]: The MindStudio MemScope project was debuted.
🌏 Overview
MindStudio MemScope (msMemScope) is a memory analysis tool developed based on the Ascend hardware to locate memory problems during model training and inference. It provides functions such as memory leak detection, memory comparison, memory block monitoring, memory decomposition, and identification of inefficient memory, helping you locate and handle problems.
🚀 Core functions
msMemScope provides memory collection and analysis functions.
| Function | Description | Sub-function | Application Scenario |
|---|---|---|---|
| Memory collection | msMemScope can collection memory events and allows custom memory collection scopes and items to provide raw data for subsequent analysis. | Collection via Python APIs | Information is collected through Python APIs, supporting custom collection scopes and items, memory events, and Python Trace events, for precise collection and efficient analysis. |
| Collection via CLIs | Information is collected through CLIs and memory event collection and memory analysis capabilities in non-Python scenarios are supported. | ||
| Memory analysis | msMemScope provides analysis capabilities such as leak detection, comparison, monitoring, decomposition, and identification of inefficient memory based on the collected memory data, helping you quickly diagnose and optimize memory problems. | Memory leak analysis | If the memory is not deallocated for a long time or a memory leak occurs, msMemScope provides memory leak analysis and change analysis at the kernel launch level to locate and analyze alarms. |
| Memory comparison | If the memory usage differs between two steps, it may lead to excessive memory usage or even out of memory (OOM) errors. In this case, use the memory comparison analysis function of msMemScope to locate and analyze the problem. | ||
| Memory block monitoring | In foundation model scenarios, if it is difficult to locate memory corruption, msMemScope can monitor the specified memory blocks before and after operator execution through Python APIs and CLIs. Based on changes in the memory block data, it can quickly determine the scope or exact location of memory corruption between operators. | ||
| Memory decomposition | msMemScope supports memory decomposition to analyze the memory usage of the CANN layer and Ascend Extension for PyTorch framework and outputs model weights, activations, gradients, and optimizer and other component memory usage. | ||
| Identification of inefficient memory | During model training and inference, some memory blocks may not be used immediately after being allocated or may not be deallocated in a timely manner after being used. msMemScope identifies the inefficient memory usage to optimize model training and inference. |
👉 Recommended Learning Path
To get started with msMemScope, follow this learning path:
- Read Installation Guide to get started with msMemScope and correctly configure an environment.
- Read Quick Start to learn how to configure and run msMemScope to locate memory problems.
- Read Memory Collection, Memory Analysis, and Output File Specifications to better understand memory problem locating.
- Read API Reference to quickly analyze memory status.
- Read Developer Guide to learn how to develop msMemScope.
📬 Suggestions and Feedback
The Huawei MindStudio full-pipeline development toolchain team is dedicated to providing an end-to-end solution for building Ascend AI applications, accelerating the processes of training, inference, and operator development. You can learn more about the Huawei MindStudio team through the following channels:
Ascend Assistant:
You are welcome to contribute to the community. If you have any questions or suggestions, see Communication Guide to contact us.
:maxdepth: 2
:caption: 🚀 Getting Started
:hidden:
install_guide
quick_start
:maxdepth: 1
:caption: 🧭 Function Description
:hidden:
memory_profile
memory_analysis
output_file_spec
:maxdepth: 2
:caption: 🔬 APIs
:hidden:
api
:maxdepth: 2
:caption: 💪 Development Guide
:hidden:
development_guide/development_guide.md
:maxdepth: 2
:caption: 🔍 Communication Guide
:hidden:
communication_guide/communication.md