apex:基于昇腾AI处理器的Apex训练优化项目

Ascend apex adapter

Branch24Tags15
FilesLast commitLast update
5 years ago
10 months ago
1 year ago
11 months ago
2 years ago
2 years ago
1 year ago
5 years ago
2 years ago
1 year ago
10 months ago
1 year ago
5 years ago

Ascend apex

Full Code Generation and Compilation

Obtain the apex source code.

Run the following command in the root directory:

git clone -b master https://gitcode.com/Ascend/apex.git
cd apex/

Compile the binary package of apex.

  1. Ensure that torch is installed and the version of setuptools is less than or equal to 65.7.0 (otherwise run: pip install setuptools==41.2.0).

  2. Run the following command(python3.7-3.10 is supported):

bash scripts/build.sh --python=3.7

The generated binary package is stored in the current dist directory.

Installation

*using docker is recommended:Link

Run the following command:

cd apex/dist/
pip3 uninstall apex
pip3 install --upgrade apex-0.1+ascend-{version}.whl *version* indicates the python version and cpu architecture.

Features

Supported features:

Note:

In the current version, apex is implemented using Python and does not support AscendCL or CUDA optimization.

Method of Use

Mixed precision:

For details, see https://nvidia.github.io/apex/amp.html.

**combine grad for unscale: **

In amp.initialize(), set combine_grad to True.

**npu fused optimizer: **

Replace the original optimizer with apex.optimizers.xxx, where xxx indicates the name of the fusion optimizer.

Introduction

Ascend apex adapter

Customize your domain