Sample Usage Guide

1. Function Description

This sample demonstrates the offline graph compilation and execution workflow. For more information about compiling graphs into offline models, refer to Generating Offline Models.

2. Directory Structure

python/
├── src/
|   ├── single_model/               // Single model sample
|   |   ├── build_add_model.py      // Offline compile Add graph, generate add_sample.om
|   |   └── run_add_model.py        // Load add_sample.om and run inference
|   ├── bundle_model/               // Bundle model sample
|   |   ├── build_bundle_model.py   // Bundle Add/Mul multiple graphs, generate bundle_sample.om
|   |   └── run_bundle_model.py     // Load bundle_sample.om, execute sub-models sequentially
|   └── common.py                   // Common logic
├── README.md                       // README file
├── run_sample.sh                   // Execution script

3. Usage

3.1 Prepare CANN Package

• Refer to Environment Preparation section "Method 3: Manual Package Installation > Scenario 1: Experience master version capabilities or develop based on master version", install the latest toolkit and ops packages.
• Set environment variables (assuming the package is installed in /usr/local/Ascend/)

source /usr/local/Ascend/cann/set_env.sh

3.2 Graph Compilation and Execution

Execute single model sample:

bash run_sample.sh -t sample_and_run_python

This command will:

1. Build Add graph, compile offline and generate add_sample.om
2. Load and execute the offline model

Execute bundle sample:

bash run_sample.sh -t sample_and_run_bundle_python

This command will:

1. Bundle Add graph and Mul graph, compile offline and generate bundle_sample.om
2. Load Bundle and execute two sub-models separately

For offline compilation in cardless scenarios where you need to specify target chip version, add --soc-version:

bash run_sample.sh --soc-version Ascend910B1 -t sample_and_run_python
bash run_sample.sh --soc-version Ascend910B1 -t sample_and_run_bundle_python

You can also split into "graph compilation only" and "graph execution only" phases:

bash run_sample.sh -t build_model
bash run_sample.sh -t run_infer

bash run_sample.sh -t build_bundle_model
bash run_sample.sh -t run_bundle_infer

After successful execution you will see:

[Success] sample execution successful

Output Files Description

After successful execution, the following files are generated in the current directory:

• add_sample.om - Single model offline file
• bundle_sample.om - Bundle offline model file

3.3 Log Printing

If you need log printing to help troubleshoot during executable program execution, you can set the following environment variables before bash run_sample.sh to print logs to screen

export ASCEND_SLOG_PRINT_TO_STDOUT=1 # Print logs to screen
export ASCEND_GLOBAL_LOG_LEVEL=0 # Log level is debug

4. Core Workflow Introduction

4.1 Single Model Offline Compilation and Execution

• Use build_initialize to initialize compilation environment
• Build Graph and generate offline model via build_model
• Save om file using save_model
• Execute offline model via acl.mdl.load_from_file, acl.mdl.execute

4.2 Bundle Offline Compilation and Execution

• Organize multiple Graph using GraphWithOptions
• Build Bundle model at once via bundle_build_model
• Save bundle_sample.om using bundle_save_model
• Load Bundle via acl.mdl.bundle_load_from_file and execute sub-models sequentially