CANN/ge离线图编译执行示例

Sample Usage Guide

【免费下载链接】geGE（Graph Engine）是面向昇腾的图编译器和执行器，提供了计算图优化、多流并行、内存复用和模型下沉等技术手段，加速模型执行效率，减少模型内存占用。 GE 提供对 PyTorch、TensorFlow 前端的友好接入能力，并同时支持 onnx、pb 等主流模型格式的解析与编译。项目地址: https://gitcode.com/cann/ge

1. Function Description

This sample demonstrates the offline graph compilation and execution workflow. For more information about compiling graphs into offline models, refer to Generating Offline Models.

2. Directory Structure

cpp/ ├── CMakeLists.txt // CMake build file ├── main.cpp // Program main entry ├── run_sample.sh // Execution script ├── README.md // README file └── src/ ├── CMakeLists.txt // CMake build file ├── common.h / common.cpp // Common logic files ├── single_model/ // Single model compilation and inference └── bundle_model/ // Bundle compilation and inference

3. Usage

3.1 Prepare CANN Package

• Correctly installtoolkitandopspackages through Environment Preparation section "Method 3: Manual Package Installation > Scenario 1: Experience master version capabilities or develop based on master version"
• Set environment variables (assuming package is installed in /usr/local/Ascend/)

source /usr/local/Ascend/cann/set_env.sh

3.2 Graph Compilation and Execution

Execute single model sample:

bash run_sample.sh -t sample_and_run

This command will:

1. Compile C++ executable program
2. BuildAddgraph, compile offline and generateadd_sample.om
3. Load and execute the offline model

Execute bundle sample:

bash run_sample.sh -t sample_and_run_bundle

This command will:

1. Compile C++ executable program
2. BundleAddgraph andMulgraph, compile offline and generatebundle_sample.om
3. Load Bundle and execute two sub-models separately

For offline compilation in cardless scenarios where you need to specify target chip version, add--soc-version:

bash run_sample.sh -s Ascend910B1 -t sample_and_run bash run_sample.sh -s Ascend910B1 -t sample_and_run_bundle

You can also split into "graph compilation only" and "graph execution only" phases:

bash run_sample.sh -t build_model bash run_sample.sh -t run_infer bash run_sample.sh -t build_bundle_model bash run_sample.sh -t run_bundle_infer

run_infer/run_bundle_inferrequires om model to already exist

After successful execution you will see:

[Success] sample execution successful

Output Files Description

After successful execution, the following files are generated in the current directory:

• add_sample.om- Single model offline file
• bundle_sample.om- Bundle offline model file

3.3 Log Printing

If you need log printing to help troubleshoot during executable program execution, you can set the following environment variables beforebash run_sample.shto print logs to screen

export ASCEND_SLOG_PRINT_TO_STDOUT=1 # Print logs to screen export ASCEND_GLOBAL_LOG_LEVEL=0 # Log level is debug

4. Core Workflow Introduction

4.1 Single Model Offline Compilation and Execution

• UseaclgrphBuildInitializeto initialize compilation environment
• BuildGraphand generate offline model viaaclgrphBuildModel
• Saveomfile usingaclgrphSaveModel
• Execute offline model viaaclmdlLoadFromFile,aclmdlExecute

4.2 Bundle Offline Compilation and Execution

• Organize Bundle using multipleGraph
• Build Bundle model at once viaaclgrphBundleBuildModel
• Savebundle_sample.omusingaclgrphBundleSaveModel
• Load Bundle viaaclmdlBundleLoadFromFileand execute sub-models sequentially

【免费下载链接】geGE（Graph Engine）是面向昇腾的图编译器和执行器，提供了计算图优化、多流并行、内存复用和模型下沉等技术手段，加速模型执行效率，减少模型内存占用。 GE 提供对 PyTorch、TensorFlow 前端的友好接入能力，并同时支持 onnx、pb 等主流模型格式的解析与编译。项目地址: https://gitcode.com/cann/ge