llm

Quick reference

• The official llm docker image.

• Maintained by: openEuler CloudNative SIG

• Where to get help: openEuler CloudNative SIG, openEuler

Build reference

1. Build images and push:

docker buildx build -t "openeuler/llm:$TAG" --platform linux/arm64 ./$TAG --push

We are using buildx in here to generate ARM64 images on different host, see more in Docker Buildx. Note that you may need to configure the proxy for cloning LLMs from huggingface while building.

How to use this image

Please run container with this image on Ascend platform of ARM64.

docker run --device $DEVICE --device /dev/davinci_manager --device /dev/devmm_svm --device /dev/hisi_hdc -v /usr/local/dcmi:/usr/local/dcmi -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi -v /usr/local/Ascend/driver/lib64/:/usr/local/Ascend/driver/lib64/ -v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info -it openeuler/llm:$TAG

Supported tags and respective Dockerfile links

• chatglm2_6b-pytorch2.1.0.a1-cann7.0.RC1.alpha002-oe2203sp2
• fastchat-pytorch2.1.0.a1-cann7.0.RC1.alpha002-oe2203sp2

Operating System

Linux/Unix, ARM64 architecture.