Installation (Ascend NPU)

Note: Ascend NPU support is maintained on the ascend-v1.0.5 branch rather than main. All code, configurations, and examples referenced in this document refer to that branch — make sure to check it out as described below.

Prerequisites

Hardware Requirements

The following hardware configuration has been extensively tested:

• NPU: 16x NPU per node
• CPU: 64 cores per node
• Memory: 1TB per node
• Network: RoCE 3.2 Tbps
• Storage:
  • 1TB local storage for single-node experiments
  • 10TB shared storage (NAS) for distributed experiments

Software Requirements

Component	Version
Operating System	Ubuntu, EulerOS or any system meeting the requirements below
Ascend HDK	25.5.1
CANN	9.0.1
Git LFS	Required for downloading models, datasets, and AReaL code. See installation guide
Docker	27.2.0
AReaL Image (A2)	ghcr.io/hwvanici/areal_npu:v1.0.5-a2 (see details below)
AReaL Image (A3)	ghcr.io/hwvanici/areal_npu:v1.0.5-a3 (see details below)

Note: This tutorial does not cover the installation of CANN, or shared storage mounting, as these depend on your specific node configuration and system version. Please complete these installations independently. You can check out more details from the vLLM-Ascend community at this page.

Runtime Environment

We recommend using Docker with our provided image for NPU. The A2 image targets Atlas A2 and the A3 image targets Atlas A3; both are built from the same recipe (see Dockerfile.a2 / Dockerfile.a3 on the ascend-v1.0.5 branch) and ship the following pre-built stack. The versions below match the published v1.0.5 image:

Component	Version
Base image (A2)	quay.io/ascend/cann:9.0.1-910b-ubuntu22.04-py3.11
Base image (A3)	quay.io/ascend/cann:9.0.1-a3-ubuntu22.04-py3.11
Runtime OS	Ubuntu 22.04.5 LTS
Python	3.11.15
CANN	9.0.1
PyTorch	2.10.0
torch_npu	2.10.0.post2
vLLM	0.23.0 (patched v0.23.0 source)
vLLM-Ascend	releases/v0.23.0 at eaefc536
Transformers	5.5.4
Megatron-Core	core_v0.16.1
MindSpeed	core_r0.16.0 at 79626c13
Megatron-Bridge	de93536e (compatible with Megatron-Core 0.16.x)

All other AReaL Python dependencies are pre-installed from pyproject.npu.toml. Megatron-LM, MindSpeed, and Megatron-Bridge sources live under /areal-workspace and are made importable via PYTHONPATH.

Create Container

WORK_DIR=<your_workspace>
CONTAINER_WORK_DIR=<your_container_workspace>

# Use A2/A3 image depending on your hardware type
# IMAGE=ghcr.io/hwvanici/areal_npu:v1.0.5-a2
IMAGE=ghcr.io/hwvanici/areal_npu:v1.0.5-a3
CONTAINER_NAME=areal_npu

cd ${WORK_DIR}

docker pull ${IMAGE}

docker run -itd --cap-add=SYS_PTRACE --net=host \
--device=/dev/davinci0 \
--device=/dev/davinci1 \
--device=/dev/davinci2 \
--device=/dev/davinci3 \
--device=/dev/davinci4 \
--device=/dev/davinci5 \
--device=/dev/davinci6 \
--device=/dev/davinci7 \
--device=/dev/davinci8 \
--device=/dev/davinci9 \
--device=/dev/davinci10 \
--device=/dev/davinci11 \
--device=/dev/davinci12 \
--device=/dev/davinci13 \
--device=/dev/davinci14 \
--device=/dev/davinci15 \
--device=/dev/davinci_manager \
--device=/dev/devmm_svm \
--device=/dev/hisi_hdc \
--shm-size=1200g \
-v /usr/local/sbin/npu-smi:/usr/local/sbin/npu-smi \
-v /usr/local/dcmi:/usr/local/dcmi \
-v /etc/ascend_install.info:/etc/ascend_install.info \
-v /sys/fs/cgroup:/sys/fs/cgroup:ro \
-v /usr/local/Ascend/driver:/usr/local/Ascend/driver \
-v /var/log/npu/:/usr/slog \
-v ${WORK_DIR}:${CONTAINER_WORK_DIR} \
--privileged=true \
--name ${CONTAINER_NAME} \
${IMAGE}  \
/bin/bash

For multi-node training: Ensure a shared storage path is mounted on every node (and mounted to the container if you are using Docker). This path will be used to save checkpoints and logs.

Custom Environment Installation

The image includes a built-in copy of the AReaL source code under /AReaL, but it may be out of date. We recommend removing it and installing AReaL from the latest source. This only replaces the AReaL package itself — all dependencies (including MindSpeed and Megatron-Bridge under /areal-workspace) stay untouched.

rm -rf /AReaL

git clone https://github.com/areal-project/AReaL
cd AReaL

# Checkout to ascend branch
git checkout ascend-v1.0.5

# Install AReaL. All Python dependencies (from pyproject.npu.toml) are already
# installed in the image, so only the AReaL package itself needs to be installed.
uv pip install --no-deps -e . --system

(Optional) Launch Ray Cluster for Distributed Training

On the first node, start the Ray Head:

ray start --head

On all other nodes, start the Ray Worker:

# Replace with the actual IP address of the first node
RAY_HEAD_IP=xxx.xxx.xxx.xxx
ray start --address $RAY_HEAD_IP

You should see the Ray resource status displayed when running ray status.

Properly set the n_nodes argument in AReaL's training command, then AReaL's training script will automatically detect the resources and allocate workers to the cluster.

Next Steps

Check the quickstart section to get familiar with launching AReaL jobs. On NPU, we recommend starting from the vision-language model (VLM) GRPO examples in examples/vlm_npu/ on the ascend-v1.0.5 branch, which train on the Geometry3K dataset. For example, to train Qwen2.5-VL-3B on a single node (16 NPUs, vLLM rollout + Megatron training):

# Some models are optimized by vllm-ascend, but the optimized variants
# may be unsuitable for RLHF training. Disable them before launching.
export USE_OPTIMIZED_MODEL=0

python examples/vlm/geometry3k_grpo.py \
    --config examples/vlm_npu/qwen2_5_vl_3b_geometry3k_grpo.yaml

See the examples/vlm_npu/README.md on the ascend-v1.0.5 branch for the full list of configurations (Qwen2.5-VL, Qwen3-VL, Qwen3.5, Qwen3.6 dense and MoE), dataset preparation, and multi-node training with Ray. If you want to run multi-node training, make sure your Ray cluster is started as described above before launching the job.

Note: On Ascend NPU, rollout is supported through the vllm engine (via the vLLM-Ascend plugin); SGLang is not available. Both the fsdp and megatron (through MindSpeed) training engines are supported.