1.2. OpenExplorer Introduction

1.2.1. What is OE

OE is the abbreviation of OpenExplorer, which is an full lifecycle development platform based on D-Robotics’ self-developed computing platform.

It mainly includes three functional modules: model compilation optimization toolset, algorithm warehouse and application development SDK. The application reference solutions developed based on these three functional modules provide case supports for smart driving, smart IoT and other industry solutions.

../../../_images/learn_oe_1.png

  1. Model compilation optimization toolset: It focuses on intelligent business scenarios, including a series of software toolsets involved in completing the algorithmic model transformation and compilation process, providing model quantization, optimization, compilation, debugging and other functions.

  2. Algorithm warehouse: It fits the basic algorithmic structure of the current robotics field, and the key business algorithmic technology points, D-Robotics provides a wealth of open source best practices, empowering D-Robotics partners to develop their own smart products faster and more economically.

  3. Application Development SDK: It provides rich basic interfaces and tools to support the deployment of business algorithms on the Sunrise platform, which can completely support the whole process of customer deployment in simulation and physical environment.

OE provides D-Robotics partners with rich and diverse algorithm resources, flexible and efficient development tools, and easy-to-use development frameworks. The features and advantages of OE are as follows:

../../../_images/learn_oe_2.png

To help users deploy various solutions to a series of D-Robotics dev boards, we provide a full-volume development package, called OE package, which integrates the dev board system image, development environment deployment, application reference solution sample code, user manuals, etc. After you get the OE package, you can first follow the steps below to understand the OE:

  1. First refer to Release Content for the directory structure of the release package.

  2. Then refer to Development Board Kit and System Image Description for board-side system image installation/upgrade.

  3. Next, refer to Floating-point Model Convert To Fixed-point Model Guide and Embedded Application Development to complete the entire process of model conversion and deployment.

For more tutorials on using the OE package, please refer to the instruction manual below. We believe D-Robotics’ OE package can make your development more efficient and easier!

For more information about the hardware applicable to the OE, refer to Development Board Kit and System Image Description in the appendix.

1.2.2. Release Content

1.2.2.1. bsp

The bsp directory contains the resolve.sh script and the tools folder.

Among them:

  • The resolve.sh script is used to download the image package.

    Attention

    The image package downloaded by this script in the X3 OE package includes X3 SDB and X3 DVB board-side system images. If you need other development board images, please contact D-Robotics to get them.

  • The resolve.sh script in the tools folder is used to download the hbupdate board-side update tools and the USB_Driver that you will use under Windows. The hbupdate tool is available for multiple platforms (win/linux). Contents include:

    • hbupdate_linux_cli_{version}.tgz.

    • hbupdate_linux_gui_{version}.tgz.

    • hbupdate_win64_{version}.zip.

1.2.2.2. ddk

The ddk directory is the main development component directory, which contains package and samples folders.

1.2.2.2.1. ddk.package

The ddk.package directory contains some base libraries and components for the distribution to run.

  • ddk.package.board

    • ddk.package.board contains the board-side executables.

    • hrt_model_exec is a model execution tool that can be used directly on the dev board to evaluate the inference performance of the model and obtain model information, which provides funcitons such as infer, perf for model performance analysis, and model_info for viewing model information.

    • hrt_bin_dump is the layer dump tool for model, which outputs binary files.

    • install.sh is an installation script that installs the hrt tool to the specified dev board with one click.

  • ddk.package.host

The ddk.package.host folder contains the environment dependencies and tool dependencies for the distribution in the x86 development environment. By executing the script install.sh in this directory, you can install all environment and tool dependencies on the development machine. By executing the script resolve.sh in this directory, you can download the cross-compilation tools and torch etc. dependencies.

1.2.2.2.2. ddk.samples

The ddk.samples contains ai_benchmark, ai_toolchain and model_zoo .

ai_benchmark provides evaluation samples of common classification, detection, and segmentation models, including performance evaluation and accuracy evaluation.

ai_toolchain provides a series of samples of some model algorithms (horizon_model_train_samples: samples of floating-point model training frameworks; horizon_model_convert_sample: a conversion sample for floating-point to fixed-point models; horizon_runtime_sample: board deployment sample for fixed-point models).

model_zoo is a model library that is used to place the original model and runtime model for the toolchain sample model compilation.

1.2.2.3. resolve_all.sh

Script for automatically downloading all downloadable dependencies within the OE package.

Running this script will sequentially download the following:

1.Execute resolve.sh under the ddk/package/host path to download the cross-compilation tools and torch etc. dependencies.

2.Execute resolve_ai_benchmark_ptq.sh under the ddk/samples/ai_toolchain/model_zoo/runtime/ai_benchmark path to download the bin model used on the board.

3.Execute resolve_runtime_sample.sh under the ddk/samples/ai_toolchain/model_zoo/runtime/horizon_runtime_sample path to download the bin model used on the board for the corresponding sample.

4.Execute resolve.sh under bsp/tools path to download the hbupdate board-side update tools and the USB_Driver that you will use under Windows.

5.Execute resolve.sh under bsp path to download the image package.

6.Execute all the 00_init.sh scripts in the ddk/samples/ai_toolchain/horizon_model_convert_sample folder to download the calibration dataset and the original model for the sample.

1.2.2.4. run_docker.sh

In case the evaluation dataset and the required docker download for the OE package are completed, you can use the following command to automatically mount the OE package and start the docker.

sh run_docker.sh {dataset path}

sh run_docker.sh {dataset path} cpu

1.2.3. Key Concepts

  • Original floating-point model: Available models obtained from the DL framework training like TensorFlow, PyTorch, etc. This model is computed with a precision of float32.

  • Hybrid Heterogeneous Model: A model format suitable for running on the D-Robotics computing platform.

  • Model conversion: Process of converting the original floating-point model or the ONNX model converted by QAT into a D-Robotics hybrid heterogeneous model.

  • Model quantization: Currently one of the most effective model optimization methods in industry. Quantization is to establish data mapping relationships between fixed-point data and floating-point data to achieve inference performance gains with little precision loss, which can be simply understood as using “low-bit” numbers to represent FP32 or other types of values, e.g., FP32 –> INT8 can achieve 4 times parameter compression, and faster calculations can be achived while memory usage is reduced.

  • hbm file: D-Robotics BPU Model (HBM), a model file generated by HBDK compiler. The HBM contains BPU instructions, model parameters, input and output descriptions, etc.

  • PTQ: PTQ conversion scheme, a quantization method that first trains a floating-point model and then uses a calibration image to calculate quantization parameters to convert the floating-point model into a quantized model.

  • QAT: QAT (quantization-aware training) scheme, which intervenes in the floating-point model structure during the floating-point training to enable the model to perceive the loss from quantization and reduce the quantization loss accuracy.

For more information about abbreviations in the documents, please refer to the section Common Abbreviations.