Sample Usage Guide

1. Functional Description

This sample uses dynamic output operators for graph construction, aiming to help graph construction developers quickly understand the definition and usage of dynamic output

2. Directory Structure

cpp/
├── src/
|   └── CMakeLists.txt           // CMake build file
|   └── es_showcase.h            // Header file
|   └── make_split_graph.cpp     // sample file
├── CMakeLists.txt               // CMake build file
├── main.cpp                     // Program main entry
├── README.md                    // README file
├── run_sample.sh                // Execution script
├── utils.h                      // Utility file

3. Usage

3.1. Prepare CANN Package

Correctly install toolkit and ops packages through installation guide Environment Preparation
Set environment variables (assuming packages are installed in /usr/local/Ascend/)


source /usr/local/Ascend/cann/set_env.sh

3.2. Compilation and Execution

Simply run the following command to complete cleanup, interface generation, graph construction and DUMP graph:


bash run_sample.sh

Current run_sample.sh behavior: first automatically clean up old build, build sample and execute sample dump by default. When you see the following information, it means successful execution:

[Success] sample executed successfully, pbtxt dump generated in current directory. This file starts with ge_onnx_ and can be opened in netron for display

You can view computation results through data files

Output File Description

After successful execution, the following files will be generated in the current directory:

ge_onnx_*.pbtxt - Graph structure protobuf text format, can be viewed with netron

Build Graph and Execute

Besides basic graph construction and dump functionality, esb_sample also supports building graphs and executing actual computation.


bash run_sample.sh -t sample_and_run

This command will:

Automatically generate ES interfaces
Compile sample program
Generate dump graph, run graph and output computation results

After successful execution you will see:

[Success] sample_and_run executed successfully, pbtxt and data output dump generated in current directory

3.3. Log Printing

If you need log printing to assist debugging during executable program execution, you can set the following environment variables before bash run_sample.sh to print logs to screen


export ASCEND_SLOG_PRINT_TO_STDOUT=1 #Print logs to screen
export ASCEND_GLOBAL_LOG_LEVEL=0 #Log level is debug level

3.4. DUMP Graph During Graph Compilation Process

During executable program execution, if you need to DUMP graph to assist debugging graph compilation process, you can set the following environment variable before bash run_sample.sh -t sample_and_run to DUMP graph to execution path

export DUMP_GE_GRAPH=2

4. Core Concept Introduction

4.1. Graph Construction Steps

Create graph builder (used to provide context, workspace and construction-related methods needed for graph construction)
Add start nodes (start nodes refer to nodes without input dependencies, usually including graph inputs (like Data nodes) and weight constants (like Const nodes))
Add intermediate nodes (intermediate nodes are computation nodes with input dependencies, usually generated by user graph construction logic, and connected through existing nodes as inputs)
Set graph output (explicitly specify graph output nodes as endpoints of computation results)

4.2. Dynamic Output

Concept Description: Dynamic output refers to operators whose output count is not fixed; for example, Split operator, which is a dynamic multi-output operator

Graph Construction API Features:

Requires a parameter to pass the count of dynamic outputs

For example, the Split operator prototype is shown below, the ES graph construction generated API is Split (C++) or EsSplit (C)

  REG_OP(Split)
    .INPUT(split_dim, TensorType({DT_INT32}))
    .INPUT(x, TensorType({DT_COMPLEX128, DT_COMPLEX64, DT_DOUBLE, DT_FLOAT, DT_FLOAT16, DT_INT16,
     DT_INT32, DT_INT64, DT_INT8, DT_QINT16, DT_QINT32, DT_QINT8,

     DT_QUINT16, DT_QUINT8, DT_UINT16, DT_UINT32, DT_UINT64, DT_UINT8,

     DT_BF16, DT_BOOL}))
    .DYNAMIC_OUTPUT(y, TensorType({DT_COMPLEX128, DT_COMPLEX64, DT_DOUBLE, DT_FLOAT, DT_FLOAT16, DT_INT16,
     DT_INT32, DT_INT64, DT_INT8, DT_QINT16, DT_QINT32, DT_QINT8,

     DT_QUINT16, DT_QUINT8, DT_UINT16, DT_UINT32, DT_UINT64, DT_UINT8,

     DT_BF16, DT_BOOL}))
    .REQUIRED_ATTR(num_split, Int)
    .OP_END_FACTORY_REG(Split)

Its corresponding function prototype is:

Function name: Split (C++) or EsSplit (C)
Parameters: 4 in total, namely split_dim, x, y_num, num_split
Return value: output y

Note: Because the current IR definition does not explicitly describe the mapping relationship of "input/attribute → dynamic output count", the graph construction API cannot automatically deduce the output count, so it needs to be specified manually; If you need to customize API, please refer to custom_es_api.md

In C API:

typedef struct {
  EsCTensorHolder **y;
  int64_t y_num;
} EsSplitOutput;
EsSplitOutput* EsSplit(EsCTensorHolder *split_dim, EsCTensorHolder *x, int64_t y_num, int64_t num_split);

C++ API:

std::vector<EsTensorHolder> Split(const EsTensorLike &split_dim, const EsTensorLike &x, int64_t y_num, int64_t num_split);

Note: Use TensorLike type to express input, to support the case where actual parameters can directly pass values