1. OE Document Introduction

This section provides a comprehensive guide to the development process for all developers using the D-Robotics X5 processor. To give you a full understanding of the overall process, we recommend that you first go through this section, which briefly describes all the sub-sections.

1. OE Document Introduction (current section)

This section provides you with an overview of the contents of relevant sections and content jump links, as well as the recommended reading order of the document.

2. Product Introduction

Section

Section Introduction

OpenExplorer Introduction

This section introduces OpenExplorer, as well as a brief overview of the contents in the release package and some of the key concepts that may be referenced repeatedly as you read.

Toolchain Overview

This section briefly describes the toolchain and the overall usage process.

3. Key Concepts

This section provides you with some common key concepts and commonly used background knowledge.

4. Environmental Installation

Section

Section Introduction

Environment Deployment

This section describes the environment deployment that needs to be performed in advance in the development and runtime environments.

Version Management of OE Package

This section describes how to use the version management tool (ddk_vc) of the OE package.

5. Quick Start

Section

Section Introduction

PTQ and QAT Introduction

This section briefly introduces the two quantification methods: PTQ and QAT.

Algorithm Model PTQ + On-board Deployment Quick Start

This section provides a quick start sample of algorithm model quantization + on board using the PTQ scheme to help you understand the basic process of the post-training quantization (PTQ) and on board deployment of the floating-point conversion toolchain.

Algorithm Model QAT + Deployment Quick Start

This section provides a quick start of algorithm model quantization + deployment using the QAT scheme to help you understand the basic process of the quantized awareness training (QAT) and deployment.

6. Post-training Quantization (PTQ)

Section

Section Introduction

PTQ Conversion Principle And Process

This section introduces you to the overall process of PTQ model conversion and details of the whole process.

PTQ Conversion Tools Guide

This section provides you with a detailed introduction to the PTQ toolkit provided by algorithm toolchain.

PTQ Conversion Steps

This section provides instructions on how to use it from model preparation, model checking, prepare calibration data, model quantization and compilation, performance analysis, accuracy analysis, and so on.

PTQ Conversion Samples Guide

This section introduces conversion sample package of the horizon_model_convert_sample model and its usage instructions. Provide a quick sample of converting a floating-point model to a fixed-point model using the floating-point model conversion toolchain, including a single inference and accuracy verification sample.

Custom OP Development

If you are using a model with operators that are not supported by the D-Robotics toolchain, refer to this section for custom operator development.

FAQ And Common Failure Resolutions

This section provides you with answers to some common questions about the PTQ conversion process as well as generalized suggestions for solving common trouble-shooting phenomena.

Appendix

This section introduces the descriptions and analysis of norm_type related parameters and related calculation formulas, as well as the concept of each transformer used in image scaling and cropping, parameter descriptions and examples, and general suggestions for solving common abnormalities and failures.

7. Quantized Awareness Training (QAT)

Section

Section Introduction

Environmental Dependence

This section introduces you to the environment-dependent requirements for quantized awareness training.

QAT Quick Start

This section provides a quick start of algorithm model quantization + deployment using the QAT scheme to help you understand the basic process of the quantized awareness training (QAT) and deployment.

TUTORIAL

This section provides you with instructions to guide you through the Quantized Awareness Training development process.

API REFERRENCE

This section provides an introduction to the API interface for QAT.

NOTE

This section describes the similarities and differences in using the QAT interface compared to the community.

Calibration

This section provides an introduction to the usage of Calibration.

8. Model Performance/Accuracy Tuning Guide

Section

Section Introduction

Model Performance Optimization

This section provides you with D-Robotics’s recommendations and measures for improving the performance of a model when a performance analysis is performed and if the performance does not meet your expectations.

PTQ Model Accuracy Optimization

This section provides you with D-Robotics’s recommendations and measures to improve the accuracy of the model when a PTQ conversion is performed and if the performance does not meet your expected.

QAT Model Accuracy Optimization

This section provides you with D-Robotics’s recommendations and measures to improve the performance of the model when a QAT conversion is performed and if the performance does not meet your expected.

9. Embedded Application Development (Runtime) Manual

Section

Section Introduction

Embedded Application Development

This section introduces the steps to deploy the quantitative training models to the X5 hardware platform, related considerations, the common multi-model control concepts, and the application tuning recommendations.

BPU SDK API DOC

This section introduces the API, data, structures, layout and alignment rules of D-Robotics OpenExplorer Toolchain Runtime, so that you can easily perform operations such as loading or releasing models, retrieving information, and performing inference by using APIs on the D-Robotics hardware platform.

Basic Sample User Guide

This section provides you with the following three reference samples: BPU SDK API teaching sample, Custom OP (custom operator) and other special function samples, Miscellaneous samples of non-NV12 input models. You can experience these examples and use them to develop your own applications.

AI Benchmark User Guide

This section provides you with performance and accuracy evaluation samples of common classification, detection, segmentation, and optical flow estimation models for embedded application development. The sample packages come with pre-built source code, executables, and evaluation scripts that you can experience on the D-Robotics development board and develop applications based on these samples directly.

Tool Introduction

This section introduces the parameter descriptions, usage and usage examples of the hrt_bin_dump tool, the layer dump tool for the model, the parameter descriptions, usage and usage examples of the model execution tool, hrt_model_exec tool, and auxiliary tools and common operations.

10. Benchmark of Model Performance

This section introduces the model Benchmark related data under certain test conditions, so that you can clearly know the model performance.

11. Appendix

Section

Section Introduction

Toolchain Operator Support Constraint List

This section provides a list of operators supported by D-Robotics, as well as their types, constraints, and general usage restrictions.

Dataset Download

In this section, we provide download links to the datasets that will be used when using the sample models for your reference.

Common Abbreviations

This section introduces some common abbreviations and their full names and meanings in this document.