zbbbcfix: Optimize the logic unification of local compilation scripts and pipeline scripts.

vNPU

Overview

The vNPU project implements Ascend NPU user-space virtualization (soft partitioning) based on CANN ACL Runtime API hijacking technology and the volcano scheduler, supporting multi-container device sharing while providing isolation capabilities for memory and AICore resources, improving resource utilization.

The main purpose of NPU soft partitioning is to address some limitations of Ascend's native NPU virtualization instance feature (hard partitioning):

Finer partitioning granularity, soft partitioning supports up to 1-to-20 partitioning, thus supporting more containers sharing a single card.
More flexible partitioning, AICore and memory resources support independent partitioning ratios, while hard partitioning partitions resources based on fixed templates, with AICore and memory only partitionable in the same ratio.
No hardware dependency, wide applicability.

Key Features

Supports NPU partitioning, a single NPU card can be partitioned into up to 20 vNPUs. AICore partitioning granularity is minimum 5%, memory partitioning granularity is minimum 1Gi.
Supports AICore and memory isolation. ACore is time-division multiplexed, not strictly isolated, computing power isolation fluctuation is approximately less than 10% of the whole card. Memory is strictly isolated.
NPU resource scheduling strategy supports binpack, trying to allocate tasks to the same NPU.
Supports limiting only memory without limiting AICore, each container can be set independently.
Supports collecting vNPU-related monitoring metrics for reporting to Prometheus.
Supports whole card scheduling, some containers on the same node use whole cards, some containers use vNPU.
Supports hard partitioning, some nodes in the same cluster use hard partitioning, some nodes use soft partitioning.
(Planned) Supports resource oversubscription and priority scheduling.

Usage

POD Declaration of NPU Resources

Business Pods only need to declare the required vNPU resources, and the vNPU scheduler and device plugin are responsible for automatically scheduling Pods to suitable nodes and suitable NPU devices and mounting devices.

Supports Pods applying for resources by vNPU quantity, AICore, and memory.

spec:
    schedulerName: volcano  # Specify scheduler name as volcano
    containers:
        resources:
          limits:
            huawei.com/vnpu-number: 1  # Required, vnpu quantity. If limiting AICore, can only be 1. Using whole card supports greater than 1. If not configured, no NPU resources will be allocated to this container.
            huawei.com/vnpu-cores: 5  # Optional, aicore usage rate available to container, unit is 5%, must be a multiple of 5. If not configured, defaults to 100, indicating exclusive use of whole card. Configured as 0 means no limit, and other containers can also share the same card.
            huawei.com/vnpu-memory.1Gi: 8 # Optional, memory available to container, unit is 1Gi. If not configured, defaults to using total single card memory. Configuring to 0 is not allowed.
          requests:
            huawei.com/vnpu-number: 1
            huawei.com/vnpu-cores: 5
            huawei.com/vnpu-memory.1Gi: 8

For example, Pod declares 1 vNPU, 50% AICore, 8G memory:

spec:
    schedulerName: volcano
    containers:
        resources:
          limits:
            huawei.com/vnpu-number: 1  
            huawei.com/vnpu-cores: 50  
            huawei.com/vnpu-memory.1Gi: 8 
          requests:
            huawei.com/vnpu-number: 1
            huawei.com/vnpu-cores: 50
            huawei.com/vnpu-memory.1Gi: 8

Only limit memory to 8G, do not limit AICore:

spec:
    schedulerName: volcano
    containers:
        resources:
          limits:
            huawei.com/vnpu-number: 1  
            huawei.com/vnpu-cores: 0 
            huawei.com/vnpu-memory.1Gi: 8 
          requests:
            huawei.com/vnpu-number: 1
            huawei.com/vnpu-cores: 0
            huawei.com/vnpu-memory.1Gi: 8

Pod declares using two whole cards:

spec:
    schedulerName: volcano
    containers:
        resources:
          limits:
            huawei.com/vnpu-number: 2 
          requests:
            huawei.com/vnpu-number: 2

Constraints and Limitations

Currently limited to a maximum of one vNPU per container.
Currently limited to only one process per container being able to use vNPU.
Only supports partitioning and limiting AICore and memory capacity, does not support partitioning control of AI CPU, VPC, VDEC, JEPGD and other cores.
Temporarily does not support GE graph engine interface.

Code Structure

direct: Hijacks CANN runtime API, implements AICore and memory isolation.
volcano-xpu-plugin: Volcano plugin, implements NPU virtualization resource scheduling.
xpu-device-plugin: Device plugin, implements NPU resource reporting and allocation.
client_update: Responsible for deployment of direct hijacking library.
xpu-exporter: Responsible for vNPU metric reporting.
ci: Compilation and build scripts.
charts: Files related to vNPU component deployment.

Compilation and Build

Tool Preparation

Install necessary tools:

git
gcc
go
cmake: version > 3.22.0
make
docker
helm

Download Code

git clone https://gitcode.com/openFuyao/vNPU.git
cd vNPU
git submodule update --init --recursive

Install Ascend HDK Driver and CANN

NPU hijacking library depends on HDK driver and CANN header files and dynamic libraries.

Version requirements: HDK recommends using Ascend-hdk-910b-npu-driver_25.x.x or Ascend-hdk-310p-npu-driver_25.x.x, CANN recommends using Ascend-cann-toolkit_8.x.x.

Install HDK driver to specified directory, for example /usr/local/Ascend, then set environment variable:

export ENPU_ASCEND_DRIVER_PATH=/usr/local/Ascend

Install CANN Ascend-cann-toolkit package to specified directory, for example /opt/tmp/CANN/ascend-toolkit, then execute source /opt/tmp/CANN/ascend-toolkit/set_env.sh to set environment variable ASCEND_TOOLKIT_HOME.

Start Build

cd vNPU/ci
# Build with default parameters, image version defaults to 1.0.0
sh build_xpu_package.sh
# Specify image version as 1.1.1
sh build_xpu_package.sh 1.1.1

# Build artifacts are in vNPU/output

Installation

Deploy Ascend Docker Runtime, recommend using 7.2.RC1+ version. Taking 7.2.RC1 version as an example, for detailed deployment methods please refer to Ascend official documentation.
Add label to nodes that need to deploy vnpu components: kubectl label node {nodename} huawei.com/vnpu=ready
Create xpu Namespace: kubectl apply -f charts/yaml/namespace.yaml

Import image packages. Taking containerd local image as an example:

# Image package names should be based on actual situation
ctr -n k8s.io images import acl_client_update-1.0.0-aarch64.tar
ctr -n k8s.io images import npu_device_plugin-1.0.0-aarch64.tar
ctr -n k8s.io images import xpu_exporter-1.0.0-aarch64.tar
ctr -n k8s.io images import vc_controller_manager-1.9.0-aarch64.tar
ctr -n k8s.io images import vc_scheduler-1.9.0-aarch64.tar
ctr -n k8s.io images import vc_webhook_manager-1.9.0-aarch64.tar

Deploy npu-client-update, npu-device-plugin, xpu-exporter through helm chart: helm upgrade --install vxpu vxpu-1.0.0.tgz -n xpu --wait.
First use root user to create log directories /var/log/volcano-admission, /var/log/volcano-controller, /var/log/volcano-scheduler on the host OS, then deploy volcano through charts/yaml/volcano-deployment.yaml: kubectl apply -f charts/yaml/volcano-deployment.yaml.
Refer to Ascend official documentation to enable device sharing mode. For 910b chip, HDK driver 25.5.0 and above versions support sharing mode. For scenarios that do not support sharing mode, business containers need to be set as privileged containers. For example, set container sharing mode for all chips of device 0 to enabled: npu-smi set -t device-share -i 0 -d 1.

FAQ

Supported NPU Models

vNPU theoretically does not depend on specific NPU models, currently only fully verified on 310P and 910B, other NPU models not yet verified.

Supported HDK and CANN Versions

Currently adapted to HDK 25.x.x and CANN versions below 8.5.0.

Supported Volcano Versions

Currently supports volcano 1.9.0 version, latest version is planned.