Model Conversion and Code Integration

[ English | 简体中文 ]

In openvela development, due to the limited RAM resources of microcontrollers (MCUs) and the fact that a complete file system may not be mounted, directly reading .tflite files is often not feasible. The standard practice is to convert the trained TensorFlow Lite model into a C language array and compile it into the application firmware as read-only data (RODATA), allowing for direct execution from Flash.

This section guides developers on how to convert a model into a C array and integrate it into an openvela C++ application (such as helloxx).

I. Model Conversion (TFLite to C Array)

To embed the model into firmware, we need to use tools to convert the binary .tflite file into a C source code file.

1. Prepare Model File

This tutorial uses the TensorFlow Lite Micro official Hello World model (sine wave prediction). To match the code logic below, we need to download the Float32 (floating point) version of the model.

• Download Link: hello_world_float.tflite (Google Official Example)

Please rename the downloaded file to converted_model.tflite and place it in the current directory.

2. Convert Using xxd Tool

In a Linux/Unix environment, use the xxd command to generate the source file containing the model data:

# Convert converted_model.tflite to model_data.cc
xxd -i converted_model.tflite > model_data.cc

3. Optimize Model Array Declaration

The default output generated by xxd is similar to the following:

unsigned char converted_model_tflite[] = {  0x18, 0x00, ...};
unsigned int converted_model_tflite_len = 18200;

Key Optimization Steps:

To save valuable RAM resources and ensure stable program operation, you must modify the generated array:

1. Add const: Place model data in Flash (RODATA segment) to avoid occupying RAM.
2. Add Memory Alignment: TFLite Micro requires the model data start address to be 16-byte aligned.

Please open model_data.cc, copy the array content, and paste it directly into the main program file helloxx_main.cxx (recommended):

// Add alignas(16) to meet TFLite memory alignment requirements
// Add const to place data in Flash, saving RAM
alignas(16) const unsigned char converted_model_tflite[] = { 
    0x18, 0x00, ... 
};
const unsigned int converted_model_tflite_len = 18200;

II. Integration into Application

This section uses the modification of the standard C++ example program apps/examples/helloxx in openvela to demonstrate how to integrate TFLite Micro.

1. Modify Build System

When compiling the application, the TFLite Micro header file paths and build rules need to be included. Edit apps/examples/helloxx/CMakeLists.txt; you can refer to the following content:

if(CONFIG_EXAMPLES_HELLOXX)
  nuttx_add_application(
    NAME
    helloxx
    STACKSIZE
    10240
    MODULE
    ${CONFIG_EXAMPLES_HELLOXX}
    SRCS
    helloxx_main.cxx
    DEPENDS
    tflite_micro
    DEFINITIONS
    TFLITE_WITH_STABLE_ABI=0
    TFLITE_USE_OPAQUE_DELEGATE=0
    TFLITE_SINGLE_ROUNDING=0
    TF_LITE_STRIP_ERROR_STRINGS
    TF_LITE_STATIC_MEMORY
    COMPILE_FLAGS
    -Wno-error)
endif()

2. Modify Configuration

• Refer to Configure TFLite Micro Development Environment to configure the compilation environment and dependent libraries.
• Enable the example application: In the configuration menu (menuconfig), navigate to Application Configuration -> Examples, and check "Hello, World!" C++ example (i.e., helloxx).

3. Implement Inference Logic

Integrating TFLite Micro in the code mainly involves five standard steps:

1. Load Model: Load the model structure from the C array.
2. Register Operators: Instantiate OpResolver and register the operators required by the model.
3. Prepare Environment: Instantiate Interpreter and allocate the Tensor Arena (tensor memory pool).
4. Write Input: Fill the input tensor with sensor data or test data.
5. Execute and Read: Call Invoke() and read the output tensor.

Open apps/examples/helloxx/helloxx_main.cxx, which needs to include the following core logic:

#include <cstdio>
#include <syslog.h>
#include "tensorflow/lite/micro/micro_mutable_op_resolver.h"
#include "tensorflow/lite/micro/micro_interpreter.h"
#include "tensorflow/lite/schema/schema_generated.h"

// ==========================================================
// Model Data Definition (Recommend pasting content generated by xxd directly and modifying modifiers)
// ==========================================================
alignas(16) const unsigned char converted_model_tflite[] = {
    // ... Paste specific hex data generated by xxd -i here ...
    0x1c, 0x00, 0x00, 0x00, 0x54, 0x46, 0x4c, 0x33, // Example header
    // ... Middle data omitted ...
};
const unsigned int converted_model_tflite_len = 18200; // Please fill in the actual length

static void test_inference(const void* file_data, size_t arenaSize) {
  // 1. Load model
  const tflite::Model* model = tflite::GetModel(file_data); 

  // 2. Register operators
  // Note: Only the FullyConnected operator is registered here; add others based on actual model requirements
  tflite::MicroMutableOpResolver<1> resolver;
  resolver.AddFullyConnected(tflite::Register_FULLY_CONNECTED());

  // 3. Allocate memory and instantiate interpreter
  std::unique_ptr<uint8_t[]> pArena(new uint8_t[arenaSize]);
  // Create an interpreter instance. The interpreter requires the model, operator resolver, and memory buffer as inputs
  tflite::MicroInterpreter interpreter(model,
    resolver, pArena.get(), arenaSize);

  // Allocate tensor memory
  interpreter.AllocateTensors();
  
  // 4. Write input data
  TfLiteTensor* input_tensor = interpreter.input(0);
  float* input_tensor_data = tflite::GetTensorData<float>(input_tensor);
  
  // Test case: Input x = pi/2 (1.5708), expected model output y approx 1.0
  float x_value = 1.5708f;  
  input_tensor_data[0] = x_value;

   // 5. Execute inference
  interpreter.Invoke();

  // Read output result
  TfLiteTensor* output_tensor = interpreter.output(0);
  float* output_tensor_data = tflite::GetTensorData<float>(output_tensor);
  printf("Output value after inference: %f\n", output_tensor_data[0]);
}

4. Verify Results

After compiling and flashing the firmware, run the helloxx command. The terminal should output the following inference result:

Output value after inference:0.99999

If the output value is close to 1.0, it indicates that the model has been successfully loaded on the openvela platform and has completed a sine wave inference calculation.