6.2. Getting Started Guide
6.2.1. Overview
This guide introduces the usage workflow of the Post-Training Quantization (PTQ) method in the algorithm toolchain. If you are a first-time user of the algorithm toolchain, it is recommended to follow the steps outlined in this guide. After completing the content in this beginner guide, you may proceed to the Quick Start section to perform model conversion and on-board execution for your private models.
For more information about the algorithm toolchain, please refer to the Advanced Guide section.
6.2.2. Environment Setup
This section mainly describes the necessary environment preparations before using the algorithm toolchain.
6.2.2.1. Hardware Requirements
To use the algorithm toolchain smoothly, we recommend your development machine meets the following requirements:
| Hardware/OS | Requirements |
|---|---|
| CPU | Intel I3 or equivalent E3/E5 processors |
| Memory | 16GB or higher |
| GPU (Optional) | CUDA 11.6, driver version Linux: >= 510.39.01* Supported GPUs include but are not limited to: 1) GeForce RTX 3090 2) GeForce RTX 2080 Ti 3) NVIDIA TITAN V 4) Tesla V100S-PCIE-32GB 5) A100 |
| OS | Ubuntu 20.04 |
6.2.2.2. Development Machine Deployment
Note: Before performing model conversion, ensure that Anaconda3 supporting Python 3.10.x has been installed on your development machine running Ubuntu or CentOS.
Run the following commands on the development machine to download the model conversion package:
wget -c ftp://oeftp@sdk.d-robotics.cc/oe_v1.2.8/install_env.sh --ftp-password=Oeftp~123$%
wget -c ftp://oeftp@sdk.d-robotics.cc/oe_v1.2.8/manual_deployment_package.tar.gz --ftp-password=Oeftp~123$%
wget -c ftp://oeftp@sdk.d-robotics.cc/model_convert_sample/yolov5s_v2.0.tar.gz --ftp-password=Oeftp~123$%
wget -c ftp://oeftp@sdk.d-robotics.cc/torch_env/torch_package.tar.xz --ftp-password=Oeftp~123$%
Tips:
To obtain more public model conversion examples, run:
wget -c ftp://oeftp@sdk.d-robotics.cc/model_convert_sample/horizon_model_convert_sample.tar.xz --ftp-password=Oeftp~123$%.A Docker image supporting model conversion is also provided. For Docker setup, please refer to the Advanced Guide - Using Docker Environment section.
Create the model conversion environment:
// oe_env is the environment name, which can be customized
conda create -n oe_env python=3.10 -y
Activate the model conversion environment:
// oe_env is the Python environment name created above. The conda command may vary across operating systems. Choose one of the following commands that successfully activates the conda environment.
source activate oe_env or conda activate oe_env
Extract the model conversion environment and example model packages, then install required dependencies:
sh install_env.sh
After successful installation, verify the setup by running hb_mapper --help. If the following output appears, the environment has been successfully installed:
hb_mapper --help
Usage: hb_mapper [OPTIONS] COMMAND [ARGS]...
hb_mapper is an offline model transform tool provided by horizon.
Options:
--version Show the version and exit.
-h, --help Show this message and exit.
Commands:
checker check whether the model meet the requirements.
infer inference and dump output feature as float vector.
makertbin transform caffe model to quantization model, generate runtime...
Tips:
Before performing model conversion, always enter the environment using
source activate oe_envorconda activate oe_env. The total size of the algorithm toolchain installation package is approximately200MB. The download and installation time, which depends on network speed, usually takes around 30 minutes. Please wait patiently for completion.
6.2.3. Quick Start
This section introduces the basic usage flow of the PTQ solution in the algorithm toolchain, enabling you to get started quickly. We use the X5 development board running the yolov5s model as an example for demonstration. For more detailed information about the PTQ solution, please refer to the Advanced Guide - PTQ Principles and Step-by-Step Guide section.
6.2.3.1. Development Environment Preparation
If the development environment is not yet set up, please refer to the Environment Setup section for instructions.
6.2.3.2. Model Preparation
Once the development environment is ready, enter the model conversion environment using the command: source activate oe_env or conda activate oe_env.
Run the following command to check if the yolov5s floating-point model exists:
ls -l yolov5s_v2.0/04_detection/03_yolov5s/mapper
After execution, if the following log appears, the model preparation is complete:
-rwxrwxr-x 1 10488 10501 645 Mar 9 2023 01_check.sh
-rwxrwxr-x 1 10488 10501 661 Mar 9 2023 02_preprocess.sh
-rwxrwxr-x 1 10488 10501 603 Mar 9 2023 03_build.sh
-rwxrwxr-x 1 10488 10501 13039 Mar 9 2023 postprocess.py
-rwxrwxr-x 1 10488 10501 3133 Mar 9 2023 preprocess.py
-rwxrwxr-x 1 10488 10501 2752 Mar 9 2023 README.cn.md
-rwxrwxr-x 1 10488 10501 10773 Mar 28 20:13 yolov5s_config.yaml
-rwxr-xr-x 1 10488 10501 29999538 Mar 9 2023 YOLOv5s.onnx
If the above log does not appear, please refer to the Environment Setup section to download the model example package.
6.2.3.3. Model Verification
After the floating-point model is ready, follow the steps below to verify the model and ensure it meets the constraints supported by the X3 processor.
Enter the directory of the yolov5s model conversion example
cd yolov5s_v2.0/04_detection/03_yolov5s/mapper
Model checking
# Check model structure and operator support, and display hardware allocation for each operator (BPU/CPU)
bash 01_check.sh
After execution, if the following log appears, the model verification is successful:
2024-03-28 20:24:43,193 INFO End to Horizon NN Model Convert.
2024-03-28 20:24:43,205 INFO ONNX model output num : 3
2024-03-28 20:24:43,220 INFO End model checking....
6.2.3.4. Model Conversion
After successful model checking, proceed with model conversion as follows.
Perform calibration data preprocessing
bash 02_preprocess.sh
After execution, if the following log appears without any errors, the data preprocessing is successful:
write:./calibration_data_rgb_f32/COCO_val2014_000000181677.rgb
write:./calibration_data_rgb_f32/COCO_val2014_000000181714.rgb
write:./calibration_data_rgb_f32/COCO_val2014_000000181739.rgb
Model conversion
# The configuration file required for conversion, yolov5s_config.yaml, is already located in the same directory as the 03_build.sh script.
bash 03_build.sh
After execution, if the following log appears without any errors, the model conversion is successful:
2024-03-28 20:31:13,228 INFO Convert to runtime bin file successfully!
2024-03-28 20:31:13,228 INFO End Model Convert
After model conversion, the model files and static performance evaluation files will be saved in the model_output folder:
torch-jit-export_subgraph_0.html # Static performance evaluation file (better readability)
torch-jit-export_subgraph_0.json # Static performance evaluation file
yolov5s_672x672_nv12.bin # Model for loading and running on the processor
yolov5s_672x672_nv12_calibrated_model.onnx # Intermediate model file, usable for subsequent accuracy verification
yolov5s_672x672_nv12_optimized_float_model.onnx # Intermediate model file, usable for subsequent accuracy verification
yolov5s_672x672_nv12_original_float_model.onnx # Intermediate model file, usable for subsequent accuracy verification
yolov5s_672x672_nv12_quantized_model.onnx # Intermediate model file, usable for subsequent accuracy verification
6.2.3.5. On-Board Model Execution
Note: Before running the model on the board, ensure that the on-board environment has been deployed according to the System Installation section.
Please refer to the sunrise_camera User Guide document to copy the sunrise_camera_vxxx.tar.gz package to the development board and extract it. Replace the fixed-point model yolov5s_672x672_nv12.bin into the model_zoom directory of sunrise_camera, then run the following command:
sh ./start_app.sh
Upon successful execution, detection results will be displayed. Refer to the sunrise_camera User Guide - Web Client Instructions document: in the web interface, set Algorithm Model (model): to yolov5s, click Submit to save, and the actual performance of the yolov5s model will be displayed.
For more information about model inference API usage, please refer to the Model Inference API Usage Guide section.