Physical machine (PM)^a

PM (bare metal)

NPUs run directly on PMs (bare metal).

PM^a

PM + unprivileged container

Passthrough the physical server's NPU device into an unprivileged container for use.

PM^a

PM + privileged container

Passthrough the physical server's NPU device into a privileged container for use.

Virtual machine (VM)^b

VM passthrough

Dedicate an entire physical NPU for exclusive use by a single VM. Supported passthrough modes include: full-system (all NPUs), partial (a subset of NPUs), and single-device (one NPU)

VM^b

VM passthrough + unprivileged container

Allocate an entire physical NPU to a VM and then pass its resources through to an unprivileged container running inside that VM.

VM^b

VM passthrough + privileged container

Allocate an entire physical NPU to a VM and then pass its resources through to a privileged container running inside that VM.

Ascend virtual instance (AVI)^c

PM + container-based AVI

On a PM, the npu-smi tool or DCMI is used to partition the NPU hardware resources into vNPUs based on user-defined requirements. The created vNPUs are then mounted to containers for use.

AVI^c

PM + VM-based AVI

On a PM, the npu-smi tool or DCMI is used to partition the NPU hardware resources into vNPUs based on user-defined requirements. The created vNPUs are then allocated to the VM for use. A single vNPU can be allocated to only one VM, but a single VM can be mounted with multiple vNPUs.

AVI^c

VM + container-based AVI

Partition the NPU hardware resources into vNPUs using the npu-smi tool or DCMI, based on user-defined requirements. Then allocate specified vNPUs to a VM, and mount the vNPUs to containers in the VM for use.

AVI^c

VM passthrough + container-based AVI

Allocate a single physical NPU entirely to a VM. Then, within that VM, the NPU is partitioned into vNPUs using the npu-smi tool or DCMI based on user-defined requirements. Finally, these vNPUs are passed through to containers for use.

a: Operating NPU resources in a PM environment, including two scenarios: PM and PM + containers

b: Operating NPU resources in a VM environment, including two scenarios: VM passthrough and VM passthrough + containers

c: Use resource virtualization technology to partition one NPU into several vNPUs for mounting to VMs or containers, allowing one NPU to handle multiple computing tasks simultaneously. AVI implements this through computing power allocation, enabling secure, multi-tenant resource pooling and on-demand management, which greatly reduces costs and complexity.