Star18
24
代码
代码
Issues1
Pull Requests
流水线
讨论
Wiki
分析
Star18
24
m0_49192959m0_49192959update: 更新文件 installation_guide.md
27bc4169创建于 13 天前历史提交

Installation Guide

Learn how to install and deploy OmniStateStore.

Environment Requirements

Hardware Requirements

Before installing OmniStateStore, ensure that the hardware environment meets the requirements described in the following table.

Table 1 Hardware requirements

Processor Kunpeng 920 or Kunpeng 950
Memory size 256 GB or above
Memory frequency 4,800 MT/s
NIC NA
Drive At least one 3.6 TB or 7.68 TB NVMe SSD

Software Requirements

The following table describes the OS and dependency software installation requirements.

Table 2 Software requirements

Software Name Software Version How to Obtain
OS openEuler 22.03 LTS SP3 Link
JDK JDK 1.8.0_432 Link
Maven Apache Maven 3.6.3 Link
GCC 10.3.1 Link
Flink 1.16.3 Link
Docker 18.09.0 -

Obtaining the Software Package

Table 3 OmniStateStore software list

Software Package File Name Release Type Description How to Obtain
OmniStateStore package BoostKit-omniruntime-omnistatestore-1.3.0.zip Open source OmniStateStore software installation package Link

Installing Dependencies

Installing the JDK

Install and configure the JDK to provide a runtime environment for running the Flink cluster.

  1. Navigate to the /usr/local directory of the physical machine and download bisheng-jdk-8u342-linux-aarch64.tar.gz.
cd /usr/local
wget --no-check-certificate https://mirror.iscas.ac.cn/kunpeng/archive/compiler/bisheng_jdk/bisheng-jdk-8u342-linux-aarch64.tar.gz
  1. Enter the /usr/local directory and extract the bisheng-jdk-8u342-linux-aarch64.tar.gz file. Then, change the ownership of the extracted JDK directory to root user and root group.
tar -zxvf bisheng-jdk-8u342-linux-aarch64.tar.gz
chown -R root /usr/local/bisheng-jdk1.8.0_342
chgrp -R root /usr/local/bisheng-jdk1.8.0_342
  1. Configure the environment variables by adding the following information to the /etc/profile file.
export JAVA_HOME=/usr/local/bisheng-jdk1.8.0_342
export JRE_HOME=$JAVA_HOME/jre
export CLASSPATH=.:$JAVA_HOME/lib:$JRE_HOME/lib
export PATH=$JAVA_HOME/bin:$PATH
  1. Update the environment variables.
source /etc/profile
java -version    
javac -version   # View the JDK version.

If the correct version is displayed, the installation is successful.

Installing Maven

  1. Download the Maven software package.

  2. Place the Maven software package in the installation directory (for example, /opt) and deploy the software package.

cd /opt
unzip apache-maven-3.6.3-bin.zip
rm -rf apache-maven-3.6.3-bin.zip
  1. Configure the environment variables by adding the following information to the /etc/profile file.
export MAVEN_HOME=/opt/apache-maven-3.6.3
export PATH=$MAVEN_HOME/bin:$PATH
  1. Update and verify the environment variables.
source /etc/profile
mvn -version # View the Maven version.

If the correct version is displayed, the installation is successful.

Installing the GCC

  1. Download the GCC binary installation package.

  2. Place the software package in the installation directory (for example, /opt) and deploy the software package.

cd /opt
tar -zxvf gcc-10.3.1-2021.09-aarch64-linux.tar.gz
mv gcc-10.3.1-2021.09-aarch64-linux gcc-10.3.1
rm -rf gcc-10.3.1-2021.09-aarch64-linux.tar.gz
  1. Configure the environment variables by adding the following information to the /etc/profile file.
export GCC_HOME=/opt/gcc-10.3.1
export PATH=$GCC_HOME/bin:$PATH
export LD_LIBRARY_PATH=$GCC_HOME/lib64:$GCC_HOME/lib:$LD_LIBRARY_PATH
export CPLUS_INCLUDE_PATH=$GCC_HOME/include/c++/10.3.1:$GCC_HOME/include:$CPLUS_INCLUDE_PATH
  1. Update the environment variables.
source /etc/profile
gcc --version
g++ --version  # View the GCC and G++ version.

If the correct version is displayed, the installation is successful.

  1. Download Flink, for example, flink-1.16.3-bin-scala_2.12.tgz for Scala 2.12.

  2. Place the software package in the installation directory (for example, /opt) and deploy the software package.

cd /opt
tar -zxvf flink-1.16.3-bin-scala_2.12.tgz
mv flink-1.16.3-bin-scala_2.12 flink-1.16.3
rm -rf flink-1.16.3-bin-scala_2.12.tgz
  1. Configure the environment variables by adding the following information to the /etc/profile file.
export FLINK_HOME=/opt/flink-1.16.3
export PATH=$FLINK_HOME/bin:$PATH
  1. Update the environment variables.
source /etc/profile

Installing Docker

Install Docker and deploy multiple containers to set up the Flink environment. If the server cannot connect to the Internet, configure a local yum repository according to your environment to ensure a smooth installation.

  1. Install Docker and import the base image. For details, see the Docker Installation Guide (CentOS & openEuler).
cd /opt
wget --no-check-certificate https://mirrors.huaweicloud.com/openeuler/openEuler-22.03-LTS-SP4/docker_img/aarch64/openEuler-docker.aarch64.tar.xz
docker load < openEuler-docker.aarch64.tar.xz
  1. Create a network in bridge mode and check whether the network is successfully created.
docker network create -d bridge flink-network
docker network ls
  1. Create and start three Docker containers. The container flavor is 8C32G, and the containers are named flink_jm_8c32g, flink_tm1_8c32g, and flink_tm2_8c32g. After all containers are started, the command execution process automatically exits.
docker run -it -d --name flink_jm_8c32g --cpus=8 --memory=32g --network flink-network openeuler-22.03-lts-sp4 /bin/bash 
docker run -it -d --name flink_tm1_8c32g --cpus=8 --memory=32g --network flink-network openeuler-22.03-lts-sp4 /bin/bash 
docker run -it -d --name flink_tm2_8c32g --cpus=8 --memory=32g --network flink-network openeuler-22.03-lts-sp4 /bin/bash
docker ps
  1. Log in to all containers, enable the SSH service in the containers, and configure password-free login.
docker exec -it flink_jm_8c32g /bin/bash
docker exec -it flink_tm1_8c32g /bin/bash
docker exec -it flink_tm2_8c32g /bin/bash
yum -y install openssh-clients openssh-server passwd vim findutils net-tools libXext libXrender gcc cmake make gcc-c++ unzip wget libXtst # Install the dependencies for the SSH service.
ssh-keygen -A # Generate the RSA key.
/usr/sbin/sshd -D & # Start the SSH service in the container.
passwd [user password] # Set a password for the container.
ssh-keygen -t rsa # Generate the RSA key again. Press "Enter" when prompted.
exit # Exit the container.
docker exec -it flink_jm_8c32g /bin/bash
ssh-copy-id -i ~/.ssh/id_rsa.pub root@flink_tm1_8c32g
ssh-copy-id -i ~/.ssh/id_rsa.pub root@flink_tm2_8c32g # Configure SSH password-free login from the "flink\_jm\_8c32g" container to the other containers.

Installing OmniStateStore

  1. Obtain the software package BoostKit-omniruntime-omnistatestore-1.3.0.zip based on [OmniStateStore software list](#Obtaining the software package).

  2. Configure the environment variable by specifying FLINK_HOME, JAVA_HOME, and LD_LIBRARY_PATH.

LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$JAVA_HOME/lib:$JAVA_HOME/jre/lib/aarch64:$JAVA_HOME/jre/lib/aarch64/server:/usr/local/lib
  1. Log in to the installation node, extract BoostKit-omniruntime-omnistatestore-1.3.0.zip to $FLINK_HOME/lib, copy librocksdb.so.6 to /usr/local/lib, and save flink-alg-falcon.jar to the current directory.
unzip BoostKit-omniruntime-omnistatestore-1.3.0.zip
mv librocksdb.so.6 /usr/local/lib
rm -rf BoostKit-omniruntime-omnistatestore-1.3.0.zip

Uninstalling OmniStateStore

If you need to uninstall OmniStateStore, go to the $FLINK_HOME/lib directory to uninstall related software packages.

rm -rf /usr/local/lib/librocksdb.so.6
rm -rf flink-alg-falcon.jar