Image Overview: Atlas torch-onnx-inference Environment

Quick Reference

The torch-onnx-inference runtime environment is maintained by the ModelZoo Community
Where to Get Help

Supported Hardware: Atlas 300I DUO / Atlas 800I A2
Base OS Options: Ubuntu 22.04 LTS (AArch64) / openEuler 24.03 LTS (AArch64)
CANN Version: 8.3.RC1
Pre-installed Software Stack: Python 3.11, Miniconda, PyTorch 2.1.0, torch_npu, MindIE-SD, ais_bench, MSIT

torch-onnx-inference

torch-onnx-inference is a high-performance deep learning inference deployment and full-stack development runtime environment tailored specifically for edge and device-side scenarios.

The core mission of this environment is to host and execute the end-to-end inference workflows for classic and mainstream AI models within the ModelZoo repository. It seamlessly integrates the PyTorch framework, the torch_npu native acceleration library, MindIE-SD, and a comprehensive suite of ONNX graph optimization and performance benchmarking tools (including onnxslim, ais_bench, and MSIT).

Image Tags and Dockerfile Paths

Full Image Path (`<Registry>/<ImageName>:<Tag>`)	Base OS	Target Hardware	Dockerfile Archive Path
`torch-onnx-inference:cann8.3.rc1_torch2.1.0-300I-DUO-ubuntu22.04-py3.11-aarch64`	Ubuntu 22.04	Atlas 300I DUO	`ModelZoo-PyTorch/ACL_PyTorch/docker/cann8.3.rc1_torch2.1.0/Dockerfile.300IDUO.ubuntu`
`torch-onnx-inference:cann8.3.rc1_torch2.1.0-300I-DUO-openeuler24.03-py3.11-aarch64`	openEuler 24.03	Atlas 300I DUO	`ModelZoo-PyTorch/ACL_PyTorch/docker/cann8.3.rc1_torch2.1.0/Dockerfile.300IDUO.openeuler`
`torch-onnx-inference:cann8.3.rc1_torch2.1.0-800I-A2-ubuntu22.04-py3.11-aarch64`	Ubuntu 22.04	Atlas 800I A2	`ModelZoo-PyTorch/ACL_PyTorch/docker/cann8.3.rc1_torch2.1.0/Dockerfile.800I_A2.ubuntu`
`torch-onnx-inference:cann8.3.rc1_torch2.1.0-800I-A2-openeuler24.03-py3.11-aarch64`	openEuler 24.03	Atlas 800I A2	`ModelZoo-PyTorch/ACL_PyTorch/docker/cann8.3.rc1_torch2.1.0/Dockerfile.800I_A2.openeuler`

Quick Start

Prerequisites

Ensure that the Atlas NPU Driver and Firmware have been correctly installed on the host machine, and the NPU devices are healthy.

Check NPU status on host: npu-smi info

Running the Container

Select the corresponding launch command based on your hardware type.

docker run -it -d --net=host --shm-size=1g \  
   --name <container-name> \
   --device=/dev/davinci_manager:rwm \
   --device=/dev/hisi_hdc:rwm \
   --device=/dev/devmm_svm:rwm \
   --device=/dev/davinci0:rwm \
   -v /usr/local/Ascend/driver:/usr/local/Ascend/driver:ro \
   -v /usr/local/Ascend/firmware/:/usr/local/Ascend/firmware:ro \
   -v /usr/local/sbin:/usr/local/sbin:ro \
   -v /path-to-weights:/path-to-weights:ro \
   torch-onnx-inference:cann8.3.rc1_torch2.1.0-800I-A2-openeuler24.03-py3.11-aarch64 bash

Local Build

To rebuild the image locally, navigate to the docker directory and run:

Example for Atlas 800I A2 openEuler

docker build \
  -t <YOUR_IMAGE_REGISTRY>/torch-onnx-inference:cann8.3.rc1_torch2.1.0-800I-A2-openeuler24.03-py3.11-aarch64 \
  -f ModelZoo-PyTorch/ACL_PyTorch/docker/cann8.3.rc1_torch2.1.0/Dockerfile.800I_A2.openeuler .

Secondary Development

Enter the container:

docker exec -it torch-onnx-inference-800I-A2 bash

Environment Activation: The Conda base environment (Python 3.11) is activated automatically upon login.

Verify NPU Stack:

python3 -c "import torch; import torch_npu; print(torch.npu.is_available())"

Hardware Support & Compatibility

Compute Architecture: Compatible with NPU architectures (Atlas 300I DUO, Atlas 800I A2).
Driver Requirements: Requires Host Driver version 24.1.RC3 corresponding driver or later to ensure full compatibility with backward operator execution.

License & Disclaimer

torch-onnx-inference 1.0

Container image Copyright (c) 2026, Huawei Technologies Co., Ltd. All rights reserved. This container image and its contents are governed by the Huawei Container License Agreement ("Li cense"). By pulling and using the container, you accept the terms and conditions of this License. A copy of this License is made available in this container at: https://www.hiascend.com/en/legal/ascend hub-download Note: You agree and undertake that when using Huawei or third-party software in this image, you will comply with the license agreement of the corresponding Huawei or third-party software.