2.1. OpenExplorer Introduction

2.1.1. What is OE

OE is the abbreviation of OpenExplorer, which is an full lifecycle development platform based on Horizon’s self-developed computing platform.

It mainly includes three functional modules: model compilation optimization toolset, algorithm warehouse and application development SDK. The application reference solutions developed based on these three functional modules provide case supports for smart driving, smart IoT and other industry solutions.

1. Model compilation optimization toolset: As shown in the yellow box in the figure above, it focuses on business scenarios, including a series of software tool sets involved in the entire process of algorithm model conversion and deployment, with functions such as model quantization, optimization, compilation, simulation, deployment, and debugging.

2. Algorithm warehouse: It contains three types of algorithm resources, product-level algorithms, basic algorithms, and product reference algorithms, enabling Horizon partners to develop their own products faster and more economically.

3. Application Development SDK: It provides rich and highly reusable algorithm modules, business strategy modules, application components, and scenario application reference solutions, aiming at accelerating customers from business model integration to application development process.

OE provides Horizon partners with rich and diverse algorithm resources, flexible and efficient development tools, and easy-to-use development frameworks. The features and advantages of OE are as follows:

To help you deploy various solutions to a series of Horizon dev boards, we provide a full-volume development package, called OE package, which integrates the dev board system image, development environment deployment, application reference solution sample code, user manuals, etc. After you get the OE package, you can first follow the steps below to understand the OE:

1. First refer to Release Content for the directory structure of the release package.

2. Then refer to System Image Upgrade for board-side system image installation/upgrade.

3. Next, refer to Post-training Quantization (PTQ) , Quantized Awareness Training (QAT) and Embedded Application Development to complete the entire process of model conversion and deployment.

For more tutorials on using the OE package, please refer to the instruction manual below. We believe Horizon’s OE package can make your development more efficient and easier!

For more information about the hardware applicable to the OE, refer to Development Board Kits in the appendix.

2.1.2. Release Content

2.1.2.1. bsp

The bsp directory contains the resolve.sh script and the tools folder.

Among them:

• The resolve.sh script is used to download the image package, but the downloaded image is only for J5 DVB board, if you need other board images, please contact Horizon to get them.

• The resolve.sh script in the tools folder is used to download the hbupdate board-side update tools and the USB_Driver that you will use under Windows. The hbupdate tool is available for multiple platforms (win/linux). Contents include:

  • hbupdate_linux_cli_{version}.tgz.

  • hbupdate_linux_gui_{version}.tgz.

  • hbupdate_win64_{version}.zip.

2.1.2.2. ddk

The ddk directory is the main development component directory, which contains package and samples folders.

2.1.2.2.1. ddk.package

The ddk.package directory contains some base libraries and components for the distribution to run.

• ddk.package.board

  • ddk.package.board contains the board-side executables.

  • hrt_model_exec is a model execution tool that can be used directly on the dev board to evaluate the inference performance of the model and obtain model information, which provides funcitons such as infer, perf for model performance analysis, and model_info for viewing model information.

  • hrt_bin_dump is the layer dump tool for the ptq debug model, which outputs binary files.

  • install.sh is an installation script that installs the hrt tool to the specified dev board with one click.

• ddk.package.host

The ddk.package.host folder contains the environment dependencies and tool dependencies for the distribution in the x86 development environment. By executing the script install.sh in this directory, you can install all environment and tool dependencies on the development machine. By executing the script resolve.sh in this directory, you can download the cross-compilation tools and torch etc. dependencies.

2.1.2.2.2. ddk.samples

The ddk.samples contains ai_benchmark, ai_toolchain, ai_forward_view_sample, vdsp_rpc_sample, and model_zoo .

ai_benchmark provides evaluation samples of common classification, detection, and segmentation models, including performance evaluation and accuracy evaluation.

ai_toolchain provides a series of samples of some model algorithms (horizon_model_train_samples: samples of floating-point model training frameworks; horizon_model_convert_sample: a conversion sample for floating-point to fixed-point models; horizon_runtime_sample: board deployment sample for fixed-point models).

ai_forward_view_sample shows how to run an inspection model on J5 and display the object frame.

vdsp_rpc_sample shows how to use DSP to process tasks on J5.

model_zoo is a model library that is used to place the original model and runtime model for the toolchain sample model compilation.

2.1.2.3. resolve_all.sh

Script for automatically downloading all downloadable dependencies within the OE package.

Running this script will sequentially download the following:

1.Execute resolve.sh under the ddk/package/host path to download the cross-compilation tools and torch etc. dependencies.

2.Execute resolve_ai_benchmark_ptq.sh under the ddk/samples/ai_toolchain/model_zoo/runtime/ai_benchmark path to download the bin model used on the board.

3.Execute resolve_ai_benchmark_qat.sh under the ddk/samples/ai_toolchain/model_zoo/runtime/ai_benchmark path to download the hbm model used on the board.

4.Execute resolve_runtime_sample.sh under the ddk/samples/ai_toolchain/model_zoo/runtime/horizon_runtime_sample path to download the bin model used on the board for the corresponding sample.

5.Execute resolve.sh under bsp/tools path to download the hbupdate board-side update tools and the USB_Driver that you will use under Windows.

6.Execute resolve.sh under bsp path to download the image package(the downloaded image is only for J5 DVB board, if you need other board images, please contact Horizon to get them).

7.Execute all the 00_init.sh scripts in the ddk/samples/ai_toolchain/horizon_model_convert_sample folder to download the calibration dataset and the original model for the sample.

2.1.2.4. run_docker.sh

In case the evaluation dataset and the required docker download for the OE package are completed, you can use the command sh run_docker.sh {dataset path} to automatically mount the OE package and start the docker.