ArkVM Development guide

ArkVM and ArkTS and related projects are hosted at Gitee and GitCode.

Table of content

Fast introduce:

1. Quick setup
2. Launch "Hello world!" on ArkVM

Detailed info:

1. How to use es2panda, ark, ark_aot, verifier
2. Build modes
3. Toolchains
4. Enable CPU features
5. Plugins system
6. Building with GN (still not all targets are supported)
7. Building with LLVM Backend
8. Launch cross-compiled ARM64/ARM32 tests with QEMU
9. Install third party
10. Testing your changes

Quick setup

Download project

mkdir arkcompiler
cd arkcompiler

BASELINE_BRANCH="OpenHarmony_feature_20241108"

git clone --branch ${BASELINE_BRANCH} https://gitee.com/openharmony/arkcompiler_runtime_core.git runtime_core

git clone --branch ${BASELINE_BRANCH} https://gitee.com/openharmony/arkcompiler_ets_frontend ets_frontend

# Set specific link for ets_frontend
ln -s ../../../ets_frontend/ets2panda runtime_core/static_core/tools/es2panda

# (optional) Install ecmascript plugin
git clone --branch ${BASELINE_BRANCH} https://gitcode.com/openharmony-sig/arkcompiler_ets_runtime.git runtime_core/static_core/plugins/ecmascript

Install dependent programs

Ubuntu 22.04 is actual base system for development.

cd runtime_core/static_core/
sudo ./scripts/install-deps-ubuntu -i=dev -i=test -i=arm-all -i=llvm-prebuilts
# It is minimal pack of tools for start develop on Linux x86-64, you can see more options by pass "--help" to script

Build project

Let's configure a common build in Debug mode using Clang-14 (more information about the build configuration in Build modes and Toolchains):

mkdir build
cd build

cmake <PATH_TO_ARKCOMPILER>/runtime_core/static_core -GNinja -DCMAKE_BUILD_TYPE=Debug -DCMAKE_TOOLCHAIN_FILE=cmake/toolchain/host_clang_14.cmake
ninja ark ark_aot es2panda

Congratulation, project and primary programs are built!

Launch "Hello world!" on ArkVM

echo 'console.log("Hello world!");' > hello.ets

BUILD=<PATH_TO_BUILD>
# Generate .abc files from source code
${BUILD}/bin/es2panda --extension=ets --output hello.abc hello.ets

# Launch Virtual Machine
${BUILD}/bin/ark --load-runtimes=ets --boot-panda-files=${BUILD}/plugins/ets/etsstdlib.abc hello.abc hello.ETSGLOBAL::main

Congratulation, you launch the ArkTS code on ArkVM!

Detailed description

How to use es2panda, ark, ark_aot, verifier

The most frequent used options for each program are described here.

These tables are quick-start subsets, not exhaustive option references. Current source-of-truth definitions live in:

• runtime/options.yaml for ark
• compiler/tools/paoc/paoc.yaml for ark_aot
• compiler/docs/paoc.md for compiler-owned ark_aot workflow notes

es2panda

es2panda is used to compile source code to .abc file.

Option name	Type of option	Default value	Description
--extension	string	"ets"	Parse the input as the given extension
--opt-level	number	1	Compiler optimization level
--output	string	""*	Path to output file

Compile ets file:

./bin/es2panda --extension=ets <file>.ets

You can see more options by pass --help.

* If --output is omitted, es2panda derives <input-basename>.abc.

---

ark

ark is virtual machine, is used to run .abc files.

Option name	Type of option	Default value	Description
--load-runtimes	string	"core"	Load specified class and intrinsic spaces and define runtime type
--boot-panda-files	string[]	$ORIGIN/../pandastdlib/arkstdlib.abc	Boot panda files (.abc) separated by colon
--interpreter-type	string	llvm*	Interpreter implementation type
--compiler-enable-jit	bool	true	Enables/disables JIT compiler
--no-async-jit	bool	false	Perform compilation in the main thread or in parallel worker
--compiler-hotness-threshold	uint32_t	3000	Threshold for "hotness" counter of the method after that it will be compiled
--boot-an-location	string	/data/service/el1/public/for-all-app/framework_ark_cache	Preferred location for boot .an files when --enable-an is used
--aot-files	string[]	[]	List of aot files (.an) to be loaded separated by colon
--enable-an	bool	false	Try to load ARK .an files from --boot-an-location first, then from the .abc file location
--enable-an:force	bool	false	Crash if there is no .an file for location based on .abc file

* Runtime may downgrade the default choice to irtoc or cpp when LLVM/IRTOC support is unavailable or restricted by build or runtime configuration. For reproducible setups, pass --interpreter-type=... explicitly.

Launch ets abc file:

./bin/ark --load-runtimes=ets --boot-panda-files=plugins/ets/etsstdlib.abc <file>.abc <file>.ETSGLOBAL::main

You can see more options by pass --help.

---

ark_aot

ark_aot tool aims to compile input panda files into the single AOT file, that can be consumed by ark.

Option name	Type of option	Default value	Description
--load-runtimes	string	"core"	Load specified class and intrinsic spaces and define runtime type
--paoc-panda-files	string[]	[]	List of input panda files to compile separated by comma
--boot-panda-files	string[]	$ORIGIN/../pandastdlib/arkstdlib.abc	Boot panda files (.abc) separated by colon
--paoc-location	string	""	Location path of the input panda file written into the AOT file
--paoc-mode	string	aot	Compiler mode: aot, llvm, jit, or osr
--paoc-output	string	out.an	Path to output file

Compile an ets abc file to AOT file:

./bin/ark_aot --boot-panda-files=plugins/ets/etsstdlib.abc --load-runtimes=ets --paoc-panda-files=<file>.abc --paoc-output=<file>.an

NOTE: boot-panda-files should be the same as it will be during launch ark, otherwise will be throw error during execution about mismach class hierarchy.

For the maintained compiler-side workflow and current paoc option map, see compiler/docs/paoc.md.

---

verifier

verifier tool aims to verify specified Panda files.

Option name	Type of option	Default value	Description
--load-runtimes	string	"core"	Load specified class and intrinsic spaces and define runtime type
--boot-panda-files	string[]	$ORIGIN/../pandastdlib/arkstdlib.abc	Boot panda files (.abc) separated by colon

./bin/verifier --load-runtimes=ets --boot-panda-files=plugins/ets/etsstdlib.abc <file>.abc

NOTE: boot-panda-files should be the same as it will be during launch ark or ark_aot.

You can see more options by pass --help.

Build modes

Recommended way to build Panda is to set CMAKE_BUILD_TYPE variable explicitly during configurations. Supported values are:

Mode	Assertions	Optimizations	Debug info
Debug	Yes	None (CMake default)	-g (CMake default)
Release	No	-O3 (CMake default)	None (CMake default)
FastVerify	Yes	-O2	-ggdb3

Notes:

• Other common modes (RelWithDebInfo, MinSizeRel, DebugDetailed) should work but they are not tested in CI.
• Unlike RelWithDebInfo, FastVerify preserves assertions (and provides more verbose debug information). Use this build type for running heavy test suites when you want both fast-running code and debuggability.
• DebugDetailed gives more debug information than Debug, it can be useful for debugging unit tests for example.

Toolchains

All types of toolchains are located by path: runtime_core/static_core/cmake/toolchain. Main toolchains which are used in development:

• cmake/toolchain/host_clang_14.cmake - build by clang-14 on Linux
• cmake/toolchain/host_gcc_11.cmake - build by gcc-11 on Linux
• cmake/toolchain/cross-clang-14-qemu-aarch64.cmake - cross build by clang-14 for arm64 with launch on Linux x86-64
• cmake/toolchain/cross-clang-14-qemu-arm-linux-gnueabi.cmake - cross build for arm32 (gnueabi) with launch on Linux x86-64 by clang-14
• cmake/toolchain/cross-clang-14-x86_64-w64-mingw32-static.cmake - build by mingw clang-14 on Windows x86-64

You can already see how the toolchain name is built and what it is used for. By analogy, so for all the others.

For more details, see the build system.

Enable CPU features

To specify what CPU features will be supported by the target CPU, the PANDA_TARGET_CPU_FEATURES variable may be set. This list of the features will be passed into numerous custom tools during the building of the VM itself, e.g., in irtoc and influence what instructions will be selected by these tools:

$ cmake -DPANDA_TARGET_CPU_FEATURES=jscvt,atomics

Plugins system

ArkVM has a multilanguage virtual machine, each language attach as a plugin. All plugins located as a separate folder by path runtime_core/static_core/plugins/. By default, extension is plugged in configure if folder is exist.

Example: there is folder runtime_core/static_core/plugins/ets/, so language ets will implicitly be added to configure for building.

You can explicitly enable/disable language by cmake option -DPANDA_WITH_<LANG>.

Example: option for ets will be --PANDA_WITH_ETS=ON/OFF.

Building with GN

Using bootstrap

Build arkts_bin, ark_aot, es2panda, verifier_bin and ets_interop_js_napi targets.Launch cross-compiled ARM64/ARM32 tests with QEMU

cd runtime_core/static_core
./scripts/build-panda-with-gn

Manually

1. Getting GN binary

$ git clone https://gn.googlesource.com/gn
$ cd gn
$ python build/gen.py
$ ninja -C out

2. Build panda using gn (arkts_asm, arkts_disasm, ark_aot, ark_aotdump, arkts_bin, es2panda, and verifier_bin targets are supported)

$ cd /path/to/panda/repository
$ /path/to/gn/repository/out/gn gen out
$ ninja -C out arkts_bin

When standard system, use

$ cd /path/to/panda/repository
$ /path/to/gn/repository/out/gn --args=is_standard_system=true gen out
$ ninja -C out <target name>

By default, gn build uses LLVM Backend, so one must provide llvm_target_dir="/path/to/llvm-15-{type}-{arch}" if it is not in default location. To build without llvm add the following arguments:

$ /path/to/gn/repository/out/gn out is_llvmbackend=false ...

Option is_llvmbackend=true in gn is the same scenarios as -DPANDA_LLVM_BACKEND=true option in cmake builds

Building with LLVM Backend

If you want to build Ark with LLVM Backend you need to special modified LLVM 15 binaries, there are 2 way to get them:

1. Download prebuilds (preferred way)

   The prebuilds will be installed in the folder /opt/:

   cd runtime_core/static_core/
   sudo ./scripts/install-deps-ubuntu -i=llvm-prebuilts
   

2. Build binaries locally

   • Clone repository from gitee.com
   • Check README file about build process
   • Use build script

If modified LLVM available in /opt, the following two options are necessary to build Ark with LLVM Backend functions.

cmake -DPANDA_LLVM_BACKEND=true -DLLVM_TARGET_PATH=/opt/llvm-15-{type}-{arch} ...

The PANDA_LLVM_BACKEND enables:

1. LLVM Irtoc Interpreter. Use -DPANDA_LLVM_INTERPRETER=OFF to disable.
2. LLVM Fastpaths compilation. Use -DPANDA_LLVM_FASTPATH=OFF to disable.
3. LLVM Interpreter inlining. Use -DPANDA_LLVM_INTERPRETER_INLINING=OFF to disable.
4. LLVM AOT compiler. Use -DPANDA_LLVM_AOT=OFF to disable.

PANDA_LLVM_INTERPRETER, PANDA_LLVM_FASTPATH, and PANDA_LLVM_AOT are ON if PANDA_LLVM_BACKEND is turned on.

It is recommended to choose clang compiler using toolchain files: -DCMAKE_TOOLCHAIN_FILE=cmake/toolchain/host_clang_14.cmake. By default GNU compiler c++ is used, but some features are not available in such gcc builds.

Building with LLVM Backend for cross ARM64 build

For cross-build, when e.g. -DCMAKE_TOOLCHAIN_FILE=cmake/toolchain/cross-clang-14-qemu-aarch64.cmake is used, two LLVM-paths options are required when all LLVM Backend functions are enabled.

• First one is target LLVM, like -DLLVM_TARGET_PATH=/opt/llvm-15-debug-aarch64
  • It is required for AOT, so, alternatively, you can use -DPANDA_LLVM_AOT=OFF.
• Second one is host LLVM, like -DLLVM_HOST_PATH=/opt/llvm-15-debug-x86_64
  • It is required for Irtoc compilation, so, alternatively, you can disable appropriate Interpreter and FastPath options (see above).

Launch cross-compiled ARM64/ARM32 tests with QEMU

Recommended QEMU version for running tests is 6.2.0 (but 5.1+ should be ok, too). By default, it is downloaded and installed during environment bootstrap. Any system-installed package is left intact. If recommended QEMU version is not accessible via $PATH it can be specified during configuration time:

# If QEMU is available as /opt/qemu-6.2.0/bin/qemu-aarch64 or download manually
$ cmake -DQEMU_PREFIX=/opt/qemu-6.2.0 ...

Install third party

Panda uses third party libraries. During firs configure of cmake third-party will be installed automatically. To manually download libraries run:

cd runtime_core/static_core
./scripts/install-third-party

Useful options:

• --force-clone - force download libraries, usually used when one of part third-party have updated
• --repo-name=repo - install specific repo
• --node - install node

You can see more options by pass --help to script

Testing your changes

For testing, the following umbrella targets that guarantee building prior to running may be used:

• ninja tests, for running all testing suites.
• ninja tests_full, for running all testing suites and various code linters.

clang-format and clang-tidy checks are integrated into build system and can be called by target of build system:

• ninja code-style-check - clang-format
• ninja clang-tidy-check - clang-tidy

NOTE ninja is launched from the build directory