Sample Usage Guide

1. Function Description

This sample uses BatchNorm operator's optional inputs for graph construction, aiming to help graph construction developers quickly understand the definition of optional inputs and how to use this type of operator for graph construction.

2. Directory Structure

python/
├── src/
|   └── make_batchnorm_graph.py   // Sample file
├── run_sample.sh                 // Execution script
├── CMakeLists.txt                // Build script
├──README.md                     // README file

3. Usage

3.1 Prepare CANN Package

Correctly install toolkit and ops packages following the installation guide Environment Preparation
Set environment variables (assuming the package is installed in /usr/local/Ascend/)

source /usr/local/Ascend/cann/set_env.sh

3.2 Build and Execute

Note: Compared with C/C++ graph construction, Python graph construction requires additional LD_LIBRARY_PATH and PYTHONPATH settings (refer to the configuration in sample)

bash run_sample.sh -t sample_and_run_python

This command will:

Automatically generate ES interface
Compile sample program
Generate dump graph and run the graph

After successful execution, you will see:

[Success] sample executed successfully, pbtxt dump has been generated in current directory. This file starts with ge_onnx_ and can be opened in netron for display

Output File Description

After successful execution, the following file will be generated in current directory:

ge_onnx_*.pbtxt - Protobuf text format of graph structure, can be viewed with netron

3.3 Log Printing

If log printing is needed during executable program execution to assist debugging, set the following environment variables before bash run_sample.sh -t sample_and_run_python to print logs to screen:

export ASCEND_SLOG_PRINT_TO_STDOUT=1 #Print logs to screen
export ASCEND_GLOBAL_LOG_LEVEL=0 #Log level set to debug level

3.4 DUMP Graph During Graph Compilation

If DUMP graph is needed during executable program execution to assist graph compilation debugging, set the following environment variables before bash run_sample.sh -t sample_and_run_python to DUMP graph to execution path:

export DUMP_GE_GRAPH=2

4. Core Concept Introduction

4.1 Graph Construction Steps

Create graph builder (to provide context, workspace and build-related methods needed for graph construction)
Add starting nodes (starting nodes refer to nodes without input dependencies, usually including graph inputs (like Data nodes) and weight constants (like Const nodes))
Add intermediate nodes (intermediate nodes are computation nodes with input dependencies, usually generated by user graph construction logic, and connected using existing nodes as inputs)
Set graph output (explicitly specify graph output nodes as endpoints of computation results)

4.2 Concept Description

Optional input refers to certain inputs of an operator that are non-mandatory inputs.

Graph Construction API Features:

Input is non-mandatory parameter during graph construction

For example, BatchNorm operator prototype is shown below, ES graph construction generated API is BatchNorm(), supporting use at Python layer

  REG_OP(BatchNorm)
    .INPUT(x, TensorType({DT_FLOAT16,DT_FLOAT}))
    .INPUT(scale, TensorType({DT_FLOAT}))
    .INPUT(offset, TensorType({DT_FLOAT}))
    .OPTIONAL_INPUT(mean, TensorType({DT_FLOAT}))
    .OPTIONAL_INPUT(variance, TensorType({DT_FLOAT}))
    .OUTPUT(y, TensorType({DT_FLOAT16,DT_FLOAT}))
    .OUTPUT(batch_mean, TensorType({DT_FLOAT}))
    .OUTPUT(batch_variance, TensorType({DT_FLOAT}))
    .OUTPUT(reserve_space_1, TensorType({DT_FLOAT}))
    .OUTPUT(reserve_space_2, TensorType({DT_FLOAT}))
    .OUTPUT(reserve_space_3, TensorType({DT_FLOAT}))
    .ATTR(epsilon, Float, 0.0001f)
    .ATTR(data_format, String, "NHWC")
    .ATTR(is_training, Bool, true)
    .ATTR(exponential_avg_factor, Float, 1.0)
    .OP_END_FACTORY_REG(BatchNorm)

Its corresponding function prototype is:

Function name: BatchNorm
Parameters: Total 9, sequentially x, scale, offset, mean (optional input), variance (optional input), epsilon, data_format, is_training, exponential_avg_factor
Return values: Outputs y, batch_mean, batch_variance, reserve_space_1, reserve_space_2, reserve_space_3

In Python API:

BatchNorm(x: Union[TensorHolder, TensorLike], scale: Union[TensorHolder, TensorLike], offset: Union[TensorHolder, TensorLike], mean: Optional[Union[TensorHolder, TensorLike]] = None, variance: Optional[Union[TensorHolder, TensorLike]] = None,
epsilon: float = 0.00100, data_format: str = "NHWC", is_training: bool = True, exponential_avg_factor: float = 0.00100) -> BatchNormOutput:

class BatchNormOutput:
    def __init__(self, y: TensorHolder, batch_mean: TensorHolder, batch_variance: TensorHolder, reserve_space_1: TensorHolder, reserve_space_2: TensorHolder, reserve_space_3: TensorHolder)
        self.y = y
        self.batch_mean = batch_mean
        self.batch_variance = batch_variance
        self.reserve_space_1 = reserve_space_1
        self.reserve_space_2 = reserve_space_2
        self.reserve_space_3 = reserve_space_3

Note:

Use TensorLike type to express input, to support cases where actual parameters can directly pass numeric values

Python Layer API Example

Method: Directly Call BatchNorm()

from ge.es.graph_builder import GraphBuilder, TensorHolder
from ge.graph import Tensor
from ge.graph.types import DataType, Format
from ge.graph import Graph
from ge.es.all import BatchNorm

# 1. Create graph builder
builder = GraphBuilder("control_dep_example")
# 2. Create nodes
input_tensor_holder = builder.create_input(
    index=0,
    name="input",
    data_type=DataType.DT_FLOAT,
    shape=[2, 3]
)
variance = builder.create_input(
    index=1,
    name="variance",
    data_type=DataType.DT_FLOAT,
    shape=[2, 3]
)
scale = builder.create_vector_int64([3, 1])
offset = builder.create_vector_int64([3, 0])
# 3. Optional input mean is None, variance has input
batchNorm_tensor_holder = BatchNorm(input_tensor_holder, scale, offset, None, variance)
# 4. Set output and build
builder.set_graph_output(batchNorm_tensor_holder.y, 0)
graph = builder.build_and_reset()