9.3. Basic Sample User Guide

The Basic Sample Package is composed by samples in 3 categories:

1. dnn API know-how samples.

2. Demonstrations of special features, e.g. custom OP, etc.

3. Miscellaneous samples of Non-NV12 input model.

Developers can experience and develop applications based on these samples, lowering the development threshold.

9.3.1. Deliverables Description

The deliverables mainly include the following:

RELEASE PACKAGE	DESCRIPTIONS
horizon_runtime_sample	including sample codes and scripts.

Note

The on-board model needs to be obtained first by executing the resolve_runtime_sample.sh script in the samples/ai_toolchain/model_zoo/runtime/horizon_runtime_sample directory of OE package.

Directory of the Basic Sample Package is shown as below:

+---horizon_runtime_sample
  ├── code                        # source code of samples
  │   ├── 00_quick_start          # a quick-start single image inference sample using the MobileNetv1 model
  │   │   ├── CMakeLists.txt
  │   │   ├── CMakeLists_x86.txt
  │   │   └── src
  │   ├── 01_api_tutorial         # BPU SDK API sample code
  │   │   ├── CMakeLists.txt
  │   │   ├── mem
  │   │   ├── model
  │   │   ├── roi_infer
  │   │   └── tensor
  │   ├── 02_advanced_samples     # special feature sample
  │   │   ├── CMakeLists.txt
  │   │   ├── custom_identity
  │   │   ├── multi_input
  │   │   ├── multi_model_batch
  │   │   └── nv12_batch
  │   ├── 03_misc                 # miscellaneous sample
  │   │   ├── CMakeLists.txt
  │   │   ├── lenet_gray
  │   │   └── resnet_feature
  │   ├── build_x5.sh             # compilation script
  │   ├── build_x86.sh            # compilation script for x86
  │   ├── CMakeLists.txt
  │   ├── CMakeLists_x86.txt
  │   └── deps_gcc11.3              # compilation dependencies
  │       ├── aarch64
  │       └── x86
  ├── x5
  │   ├── data                    # preset data
  │   │   ├── cls_images
  │   │   ├── det_images
  │   │   ├── misc_data
  │   │   ├── custom_identity_data
  │   ├── model
  │   │   ├── README.md
  │   │   └── runtime -> ../../../model_zoo/runtime/horizon_runtime_sample   # soft link, pointed to the model folder in OE package. In the board-side runtime environment, you need to specify the model path
  │   └── script                  # aarch64 scripts to run samples
  │   │   ├── 00_quick_start
  │   │   ├── 01_api_tutorial
  │   │   ├── 02_advanced_samples
  │   │   ├── 03_misc
  │   │   ├── aarch64             # aarch64 executable programs generated by the compilation and dependencies
  │   │   ├── README.md
  │   └── script_x86              # x86 scripts to run samples
  │       ├── 00_quick_start
  │       ├── x86                 # x86 executable programs generated by the compilation and dependencies
  │       └── README.md
  └── README.md

• The code directory contains source code.

• The code/00_quick_start folder contains a quick-start sample based on the dnn API. This script inference a single image and parse the results using MobileNetv1.

• The code/01_api_tutorial folder contains API teaching code, including 4 parts: mem, model, roi_infer and tensor.

• The code/02_advanced_samples folder contains custom_identity, multi_input, multi_model_batch and nv12_batch samples.

• The code/03_misc folder contains those miscellaneous samples of non-NV12 input models.

• The code/build_x5.sh is a quick application compilation script.

• The code/build_x86.sh is a quick application compilation script for x86 Platform.

• The code/deps_gcc11.3 folder contains those sample code required third party dependencies. Users can replace or delete depending on their own needs.

• The x5 folder contains scripts to run samples, preset data and related models.

9.3.2. Prerequisites

9.3.2.1. Prepare the Dev Board

1.After getting the development board, upgrade the system image file to the version recommended by the sample package.

2.Make sure the local development machine and development board can be connected remotely.

9.3.2.2. Compilation

Take the following steps to perform the compilation:

1.Install the arm-gnu-toolchain-11.3.rel1-x86_64-aarch64-none-linux-gnu cross-compilation tool.

2.In real machine environment, execute the build_x5.sh script in the horizon_runtime_sample/code folder to compile executable programs. The executable programs and corresponding dependencies will be copied into the aarch64 sub-folder of the x5/script folder.

In x86 simulation environment, execute the build_x86.sh script in the horizon_runtime_sample/code folder to compile executable programs. The executable programs and corresponding dependencies will be copied into the x86 sub-folder of the x5/script_x86 folder.

Note

By default, the cross-compilation tool specified in the build_x5.sh script is located in the /opt directory. You can manually modify the build_x5.sh script if you want to change the location.

export CC=/opt/arm-gnu-toolchain-11.3.rel1-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu-gcc
export CXX=/opt/arm-gnu-toolchain-11.3.rel1-x86_64-aarch64-none-linux-gnu/bin/aarch64-none-linux-gnu-g++

9.3.3. How to Use (Basic Samples)

Sample scripts are in the x5/script folder, and the directory structure after compilation is shown as follows:

script:
├── 00_quick_start
│   ├── README.md
│   └── run_mobilenetV1.sh
├── 01_api_tutorial
│   ├── model.sh
│   ├── README.md
│   ├── roi_infer.sh
│   ├── sys_mem.sh
│   └── tensor.sh
├── 02_advanced_samples
│   ├── plugin
│   │   └── custom_arm_op_custom_identity.sh
│   ├── README.md
│   ├── run_multi_input.sh
│   ├── run_multi_model_batch.sh
│   └── run_nv12_batch.sh
├── 03_misc
│   ├── README.md
│   ├── run_lenet.sh
│   └── run_resnet50_feature.sh
├── aarch64                        # executable programs generated by the compilation and dependencies
│   ├── bin
│   │   ├── model_example
│   │   ├── roi_infer
│   │   ├── run_custom_op
│   │   ├── run_lenet_gray
│   │   ├── run_mobileNetV1_224x224
│   │   ├── run_multi_input
│   │   ├── run_multi_model_batch
│   │   ├── run_nv12_batch
│   │   ├── run_resnet_feature
│   │   ├── sys_mem_example
│   │   └── tensor_example
│   └── lib
│       ├── libdnn.so
│       ├── libhbrt_bayes_aarch64.so
│       └── libopencv_world.so.3.4
└── README.md

script_x86:
├── 00_quick_start
│   ├── README.md
│   └── run_mobilenetV1.sh
├── x86                        # executable programs generated by the compilation and dependencies
│   ├── bin
│   │   └── run_mobileNetV1_224x224
│   └── lib
│       ├── libdnn.so
│       ├── libhbdk_sim_x86.so
│       └── libopencv_world.so.3.4
└── README.md

Note

• To obtain the model publisher for the horizon_runtime_sample sample package, you can execute the resolve_runtime_sample.sh in the samples/ai_toolchain/model_zoo/runtime/horizon_runtime_sample dictionary of the OE package.

• The model folder contains the path of the model, For x86 env, where the runtime folder is a soft link and the link path is ../../../model_zoo/runtime/horizon_runtime_sample, run x86 script in OE directly, it will automatically find the model folder path in OE. For the board env environment, the model publisher needs to be placed in the model folder.

9.3.3.1. quick_start

The quick start sample in the 00_quick_start directory is shown as below:

00_quick_start/
├── README.md
└── run_mobilenetV1.sh

• run_mobilenetV1.sh: This script implements a sample function that uses mobilenetv1 to read a single image for inference. To run it, go to the 00_quick_start, and then execute sh run_mobilenetV1.sh directly, as shown in the following code block:

  #!/bin/sh
  
  root@x5dvb:/userdata/app/horizon/basic_samples/x5/script/00_quick_start# sh run_mobilenetV1.sh
  ../aarch64/bin/run_mobileNetV1_224x224 --model_file=../../model/runtime/mobilenetv1/mobilenetv1_224x224_nv12.bin --image_file=../../data/cls_images/zebra_cls.jpg --top_k=5
  I0000 00:00:00.000000 10765 vlog_is_on.cc:197] RAW: Set VLOG level for "*" to 3
  [BPU_PLAT]BPU Platform Version(1.3.3)!
  [HBRT] set log level as 0. version = 3.15.18.0
  [DNN] Runtime version = 1.17.2_(3.15.18 HBRT)
  [A][DNN][packed_model.cpp:225][Model](2023-04-11,17:51:17.206.804) [HorizonRT] The model builder version = 1.15.0
  I0411 17:51:17.244180 10765 run_mobileNetV1_224x224.cc:135] DNN runtime version: 1.17.2_(3.15.18 HBRT)
  I0411 17:51:17.244376 10765 run_mobileNetV1_224x224.cc:252] input[0] name is data
  I0411 17:51:17.244508 10765 run_mobileNetV1_224x224.cc:268] output[0] name is prob
  I0411 17:51:17.260176 10765 run_mobileNetV1_224x224.cc:159] read image to tensor as nv12 success
  I0411 17:51:17.262075 10765 run_mobileNetV1_224x224.cc:194] TOP 0 result id: 340
  I0411 17:51:17.262118 10765 run_mobileNetV1_224x224.cc:194] TOP 1 result id: 292
  I0411 17:51:17.262148 10765 run_mobileNetV1_224x224.cc:194] TOP 2 result id: 282
  I0411 17:51:17.262177 10765 run_mobileNetV1_224x224.cc:194] TOP 3 result id: 83
  I0411 17:51:17.262205 10765 run_mobileNetV1_224x224.cc:194] TOP 4 result id: 290
  

9.3.3.2. api_tutorial

The API tutorial refers to the samples in the 01_api_tutorial folder. These samples help developers understand how to use the embedded APIs. This folder contains the following scripts:

├── model.sh
├── roi_infer.sh
├── sys_mem.sh
└── tensor.sh

• The model.sh script reads model information. To run it, go to the 01_api_tutorial, and then execute sh model.sh, as shown in the following code block:

  #!/bin/sh
  
  root@x5dvb-hynix8G:/userdata/horizon/x5/script/01_api_tutorial# sh model.sh
  ../aarch64/bin/model_example --model_file_list=../../model/runtime/mobilenetv1/mobilenetv1_224x224_nv12.bin
  I0000 00:00:00.000000 10810 vlog_is_on.cc:197] RAW: Set VLOG level for "*" to 3
  [BPU_PLAT]BPU Platform Version(1.3.3)!
  [HBRT] set log level as 0. version = 3.15.18.0
  [DNN] Runtime version = 1.17.2_(3.15.18 HBRT)
  [A][DNN][packed_model.cpp:225][Model](2023-04-11,17:53:28.970.396) [HorizonRT] The model builder version = 1.15.0
  I0411 17:53:29.007853 10810 model_example.cc:104] model count:1, model[0]: mobilenetv1_224x224_nv12
  I0411 17:53:29.007939 10810 model_example.cc:112] hbDNNGetModelHandle [mobilenetv1_224x224_nv12] success!
  I0411 17:53:29.008011 10810 model_example.cc:186] [mobilenetv1_224x224_nv12] Model Info:  input num: 1, input[0] validShape: ( 1, 3, 224, 224 ), alignedShape: ( 1, 3, 224, 224 ), tensorType: 1, output num: 1, output[0] validShape: ( 1, 1000, 1, 1 ), alignedShape: ( 1, 1000, 1, 1 ), tensorType: 13
  

• The roi_infer.sh script helps understand how to use the hbDNNRoiInfer API. The function of the sample code is to convert an image into nv12 data, and then use the given ROI box for inference.

• The sys_mem.sh script helps understand how to use the hbSysAllocMem, hbSysFlushMem and hbSysFreeMem APIs. To run it, go to the 01_api_tutorial folder, and then execute sh sys_mem.sh.

• The tensor.sh script helps understand how to prepare model input and output tensors, print the properties and data layout of the tensor and de-quantize it using the quantizeAxis parameter. To run it, go to the 01_api_tutorial folder, and then execute sh tensor.sh, as shown in the following code block:

  root@x5dvb-hynix8G:/userdata/horizon/x5/script/01_api_tutorial# sh tensor.sh
  *****************************test_prepare_free_fn*************************************************
  Tensor data type:0, Tensor layout: 2, shape:1x1x721x1836, aligned shape:1x1x721x1840
  Tensor data type:1, Tensor layout: 2, shape:1x3x773x329, aligned shape:1x3x773x336
  Tensor data type:2, Tensor layout: 2, shape:1x3x108x1297, aligned shape:1x3x108x1312
  Tensor data type:5, Tensor layout: 2, shape:1x3x858x477, aligned shape:1x3x858x477
  Tensor data type:5, Tensor layout: 0, shape:1x920x102x3, aligned shape:1x920x102x3
  Tensor data type:4, Tensor layout: 2, shape:1x3x723x1486, aligned shape:1x3x723x1486
  Tensor data type:4, Tensor layout: 0, shape:1x372x366x3, aligned shape:1x372x366x3
  Tensor data type:3, Tensor layout: 2, shape:1x3x886x291, aligned shape:1x3x886x291
  Tensor data type:3, Tensor layout: 0, shape:1x613x507x3, aligned shape:1x613x507x3
  *****************************test_prepare_free_fn************************************************
  
  *****************************test_info_fn********************************************************
  Tensor data type:14, shape:1x1x1x3x2, stride:24x24x24x8x4, ndim: 5, data:
    [[[[[0, 1], [2, 3], [4, 5]]]]]
  Tensor data type:9, shape:3x3x1x2x1, stride:6x2x2x1x1, ndim: 5, data:
    [[[[[0], [1]]], [[[2], [3]]], [[[4], [5]]]], [[[[6], [7]]], [[[8], [9]]],
      [[[10], [11]]]], [[[[12], [13]]], [[[14], [15]]], [[[16], [17]]]]]
  *****************************test_info_fn********************************************************
  
  ********************test_dequantize_fn***********************************************************
  Tensor data type:8, shape:1x1x2x4, ndim: 4, quantiType: 2, quantizeAxis: 1,
    quantizeValue: (0.1,), data: [[[[0, 1, 2, 3], [4, 5, 6, 7]]]],
    dequantize data: [[[[0, 0.1, 0.2, 0.3], [0.4, 0.5, 0.6, 0.7]]]]
  Tensor data type:8, shape:2x4x1x1, ndim: 4, quantiType: 2, quantizeAxis: 3,
    quantizeValue: (0.1,),
    data: [[[[0]], [[1]], [[2]], [[3]]], [[[4]], [[5]], [[6]], [[7]]]],
    dequantize data: [[[[0]], [[0.1]], [[0.2]], [[0.3]]], [[[0.4]], [[0.5]], [[0.6]], [[0.7]]]]
  ********************test_dequantize_fn***********************************************************
  

9.3.3.3. advanced_samples

The advanced samples are located in the 02_advanced_samples folder, which are used for demonstrating how to use the custom OP special feature. The folder contains the following scripts:

├── plugin
│    └── custom_arm_op_custom_identity.sh
├── README.md
├── run_multi_input.sh
├── run_multi_model_batch.sh
└── run_nv12_batch.sh

• The custom_arm_op_custom_identity.sh script implements model inference using user-customized operators. To run it, go to the 02_advanced_samples folder, and then execute sh custom_arm_op_custom_identity.sh, as shown in the following code block:

  root@x5dvb-hynix8G:/userdata/horizon/x5/script/02_advanced_samples# sh custom_arm_op_custom_identity.sh
  ../../aarch64/bin/run_custom_op --model_file=../../../model/runtime/custom_op/custom_op_featuremap.bin --input_file=../../../data/custom_identity_data/input0.bin,../../../data/custom_identity_data/input1.bin
  I0000 00:00:00.000000 10841 vlog_is_on.cc:197] RAW: Set VLOG level for "*" to 3
  I0411 17:55:59.928918 10841 main.cpp:212] hbDNNRegisterLayerCreator success
  I0411 17:55:59.929064 10841 main.cpp:217] hbDNNRegisterLayerCreator success
  [BPU_PLAT]BPU Platform Version(1.3.3)!
  [HBRT] set log level as 0. version = 3.15.18.0
  [DNN] Runtime version = 1.17.2_(3.15.18 HBRT)
  [A][DNN][packed_model.cpp:225][Model](2023-04-11,17:56:00.667.991) [HorizonRT] The model builder version = 1.15.0
  I0411 17:56:00.676071 10841 main.cpp:232] hbDNNGetModelNameList success
  I0411 17:56:00.676204 10841 main.cpp:239] hbDNNGetModelHandle success
  I0411 17:56:00.676276 10841 main.cpp:245] hbDNNGetInputCount success
  file length: 602112
  file length: 602112
  I0411 17:56:00.687402 10841 main.cpp:268] hbDNNGetOutputCount success
  I0411 17:56:00.687788 10841 main.cpp:297] hbDNNInfer success
  I0411 17:56:00.695663 10841 main.cpp:302] task done
  I0411 17:56:03.145243 10841 main.cpp:306] write output tensor
  

  The first output data of the model is saved to the output0.txt file.

• The run_multi_input.sh script implements the multi-input model inference function. To run it, go to the 02_advanced_samples folder, and then execute sh run_multi_input.sh, as shown in the following code block:

  root@x5dvb:/userdata/horizon/x5/script/02_advanced_samples# sh run_multi_input.sh
  ../aarch64/bin/run_multi_input --model_file=../../model/runtime/mobilenetv2/mobilenetv2_multi_224x224_gray.bin --image_file=../../data/cls_images/zebra_cls.jpg --top_k=5
  I0000 00:00:00.000000 10893 vlog_is_on.cc:197] RAW: Set VLOG level for "*" to 3
  [BPU_PLAT]BPU Platform Version(1.3.3)!
  [HBRT] set log level as 0. version = 3.15.18.0
  [DNN] Runtime version = 1.17.2_(3.15.18 HBRT)
  [A][DNN][packed_model.cpp:225][Model](2023-04-11,17:57:03.277.375) [HorizonRT] The model builder version = 1.15.0
  I0411 17:57:03.327527 10893 multi_input.cc:148] read image to tensor as bgr success
  I0411 17:57:03.329546 10893 multi_input.cc:183] TOP 0 result id: 340
  I0411 17:57:03.329598 10893 multi_input.cc:183] TOP 1 result id: 292
  I0411 17:57:03.329628 10893 multi_input.cc:183] TOP 2 result id: 352
  I0411 17:57:03.329656 10893 multi_input.cc:183] TOP 3 result id: 351
  I0411 17:57:03.329684 10893 multi_input.cc:183] TOP 4 result id: 282
  

• The run_multi_model_batch.sh script mainly implements the batch inference function for multiple small models. To run it, go to the 02_advanced_samples directory, and then execute sh run_multi_model_batch.sh directly, as shown in the following code block:

  root@x5dvb-hynix8G:/userdata/horizon/x5/script/02_advanced_samples# sh run_multi_model_batch.sh
  ../aarch64/bin/run_multi_model_batch --model_file=../../model/runtime/googlenet/googlenet_224x224_nv12.bin,../../model/runtime/mobilenetv2/mobilenetv2_224x224_nv12.bin --input_file=../../data/cls_images/zebra_cls.jpg,../../data/cls_images/zebra_cls.jpg
  I0000 00:00:00.000000 10916 vlog_is_on.cc:197] RAW: Set VLOG level for "*" to 3
  [BPU_PLAT]BPU Platform Version(1.3.3)!
  [HBRT] set log level as 0. version = 3.15.18.0
  [DNN] Runtime version = 1.17.2_(3.15.18 HBRT)
  [A][DNN][packed_model.cpp:225][Model](2023-04-11,17:57:43.547.52) [HorizonRT] The model builder version = 1.15.0
  [A][DNN][packed_model.cpp:225][Model](2023-04-11,17:57:51.811.477) [HorizonRT] The model builder version = 1.15.0
  I0411 17:57:51.844280 10916 main.cpp:117] hbDNNInitializeFromFiles success
  I0411 17:57:51.844388 10916 main.cpp:125] hbDNNGetModelNameList success
  I0411 17:57:51.844424 10916 main.cpp:139] hbDNNGetModelHandle success
  I0411 17:57:51.875140 10916 main.cpp:153] read image to nv12 success
  I0411 17:57:51.875686 10916 main.cpp:170] prepare input tensor success
  I0411 17:57:51.875875 10916 main.cpp:182] prepare output tensor success
  I0411 17:57:51.876082 10916 main.cpp:216] infer success
  I0411 17:57:51.878844 10916 main.cpp:221] task done
  I0411 17:57:51.878948 10916 main.cpp:226] googlenet class result id: 340
  I0411 17:57:51.879084 10916 main.cpp:230] mobilenetv2 class result id: 340
  I0411 17:57:51.879177 10916 main.cpp:234] release task success
  

• The run_nv12_batch.sh script mainly implements the batch model inference function. Infer1 is used to set the address of each batch of the input tensor separately, while Infer2 is used to set only one address that contains all the batches. To run it, go to the 02_advanced_samples directory, and then execute sh run_multi_model_batch.sh , as shown in the following code block:

  root@x5dvb:/userdata/horizon/x5/script/02_advanced_samples# sh run_nv12_batch.sh
  ../aarch64/bin/run_nv12_batch --model_file=../../model/runtime/googlenet/googlenet_4x224x224_nv12.bin --image_file=../../data/cls_images/zebra_cls.jpg,../../data/cls_images/cat_cls.jpg,../../data/cls_images/zebra_cls.jpg,../../data/cls_images/cat_cls.jpg --top_k=5
  I0000 00:00:00.000000 21511 vlog_is_on.cc:197] RAW: Set VLOG level for "*" to 3
  [BPU_PLAT]BPU Platform Version(1.3.3)!
  [HBRT] set log level as 0. version = 3.15.18.0
  [DNN] Runtime version = 1.17.2_(3.15.18 HBRT)
  I0705 11:39:43.429180 21511 nv12_batch.cc:151] Infer1 start
  I0705 11:39:43.488143 21511 nv12_batch.cc:166] read image to tensor as nv12 success
  I0705 11:39:43.491156 21511 nv12_batch.cc:201] Batch[0]:
  I0705 11:39:43.491211 21511 nv12_batch.cc:203] TOP 0 result id: 340
  I0705 11:39:43.491240 21511 nv12_batch.cc:203] TOP 1 result id: 83
  I0705 11:39:43.491266 21511 nv12_batch.cc:203] TOP 2 result id: 41
  I0705 11:39:43.491298 21511 nv12_batch.cc:203] TOP 3 result id: 912
  I0705 11:39:43.491324 21511 nv12_batch.cc:203] TOP 4 result id: 292
  I0705 11:39:43.491348 21511 nv12_batch.cc:201] Batch[1]:
  I0705 11:39:43.491374 21511 nv12_batch.cc:203] TOP 0 result id: 282
  I0705 11:39:43.491398 21511 nv12_batch.cc:203] TOP 1 result id: 281
  I0705 11:39:43.491422 21511 nv12_batch.cc:203] TOP 2 result id: 285
  I0705 11:39:43.491447 21511 nv12_batch.cc:203] TOP 3 result id: 287
  I0705 11:39:43.491472 21511 nv12_batch.cc:203] TOP 4 result id: 283
  I0705 11:39:43.491497 21511 nv12_batch.cc:201] Batch[2]:
  I0705 11:39:43.491514 21511 nv12_batch.cc:203] TOP 0 result id: 340
  I0705 11:39:43.491539 21511 nv12_batch.cc:203] TOP 1 result id: 83
  I0705 11:39:43.491564 21511 nv12_batch.cc:203] TOP 2 result id: 41
  I0705 11:39:43.491587 21511 nv12_batch.cc:203] TOP 3 result id: 912
  I0705 11:39:43.491612 21511 nv12_batch.cc:203] TOP 4 result id: 292
  I0705 11:39:43.491637 21511 nv12_batch.cc:201] Batch[3]:
  I0705 11:39:43.491662 21511 nv12_batch.cc:203] TOP 0 result id: 282
  I0705 11:39:43.491685 21511 nv12_batch.cc:203] TOP 1 result id: 281
  I0705 11:39:43.491710 21511 nv12_batch.cc:203] TOP 2 result id: 285
  I0705 11:39:43.491734 21511 nv12_batch.cc:203] TOP 3 result id: 287
  I0705 11:39:43.491760 21511 nv12_batch.cc:203] TOP 4 result id: 283
  I0705 11:39:43.492235 21511 nv12_batch.cc:223] Infer1 end
  I0705 11:39:43.492276 21511 nv12_batch.cc:228] Infer2 start
  I0705 11:39:43.549713 21511 nv12_batch.cc:243] read image to tensor as nv12 success
  I0705 11:39:43.552248 21511 nv12_batch.cc:278] Batch[0]:
  I0705 11:39:43.552292 21511 nv12_batch.cc:280] TOP 0 result id: 340
  I0705 11:39:43.552320 21511 nv12_batch.cc:280] TOP 1 result id: 83
  I0705 11:39:43.552345 21511 nv12_batch.cc:280] TOP 2 result id: 41
  I0705 11:39:43.552371 21511 nv12_batch.cc:280] TOP 3 result id: 912
  I0705 11:39:43.552397 21511 nv12_batch.cc:280] TOP 4 result id: 292
  I0705 11:39:43.552421 21511 nv12_batch.cc:278] Batch[1]:
  I0705 11:39:43.552445 21511 nv12_batch.cc:280] TOP 0 result id: 282
  I0705 11:39:43.552469 21511 nv12_batch.cc:280] TOP 1 result id: 281
  I0705 11:39:43.552495 21511 nv12_batch.cc:280] TOP 2 result id: 285
  I0705 11:39:43.552520 21511 nv12_batch.cc:280] TOP 3 result id: 287
  I0705 11:39:43.552567 21511 nv12_batch.cc:280] TOP 4 result id: 283
  I0705 11:39:43.552592 21511 nv12_batch.cc:278] Batch[2]:
  I0705 11:39:43.552616 21511 nv12_batch.cc:280] TOP 0 result id: 340
  I0705 11:39:43.552641 21511 nv12_batch.cc:280] TOP 1 result id: 83
  I0705 11:39:43.552665 21511 nv12_batch.cc:280] TOP 2 result id: 41
  I0705 11:39:43.552690 21511 nv12_batch.cc:280] TOP 3 result id: 912
  I0705 11:39:43.552716 21511 nv12_batch.cc:280] TOP 4 result id: 292
  I0705 11:39:43.552739 21511 nv12_batch.cc:278] Batch[3]:
  I0705 11:39:43.552763 21511 nv12_batch.cc:280] TOP 0 result id: 282
  I0705 11:39:43.552788 21511 nv12_batch.cc:280] TOP 1 result id: 281
  I0705 11:39:43.552812 21511 nv12_batch.cc:280] TOP 2 result id: 285
  I0705 11:39:43.552837 21511 nv12_batch.cc:280] TOP 3 result id: 287
  I0705 11:39:43.552861 21511 nv12_batch.cc:280] TOP 4 result id: 283
  I0705 11:39:43.553154 21511 nv12_batch.cc:300] Infer2 end
  

9.3.3.4. misc

The miscellaneous samples are located in the 03_misc folder. They are used for demonstrating how to use the non-NV12 input models. It contains the following scripts:

├── run_lenet.sh
└── run_resnet50_feature.sh

• The run_lenet.sh script is used to implement the Lenet model inference using Y data input. To run it, go to the 03_misc folder, and then execute sh run_lenet.sh , as shown in the following code block:

  root@x5dvb-hynix8G:/userdata/horizon/x5/script/03_misc# sh run_lenet.sh
  ../aarch64/bin/run_lenet_gray --model_file=../../model/runtime/lenet_gray/lenet_28x28_gray.bin --data_file=../../data/misc_data/7.bin --image_height=28 --image_width=28 --top_k=5
  I0000 00:00:00.000000 10979 vlog_is_on.cc:197] RAW: Set VLOG level for "*" to 3
  [BPU_PLAT]BPU Platform Version(1.3.3)!
  [HBRT] set log level as 0. version = 3.15.18.0
  [DNN] Runtime version = 1.17.2_(3.15.18 HBRT)
  [A][DNN][packed_model.cpp:225][Model](2023-04-11,18:02:12.605.436) [HorizonRT] The model builder version = 1.15.0
  I0411 18:02:12.613317 10979 run_lenet_gray.cc:128] hbDNNInitializeFromFiles success
  I0411 18:02:12.613404 10979 run_lenet_gray.cc:136] hbDNNGetModelNameList success
  I0411 18:02:12.613440 10979 run_lenet_gray.cc:143] hbDNNGetModelHandle success
  I0411 18:02:12.614181 10979 run_lenet_gray.cc:159] prepare y tensor success
  I0411 18:02:12.614310 10979 run_lenet_gray.cc:172] prepare tensor success
  I0411 18:02:12.614503 10979 run_lenet_gray.cc:182] infer success
  I0411 18:02:12.615538 10979 run_lenet_gray.cc:187] task done
  [W][DNN][hb_sys.cpp:108][Mem](2023-04-11,18:02:12.615.583) memory is noncachable, ignore flush operation
  I0411 18:02:12.615624 10979 run_lenet_gray.cc:192] task post process finished
  I0411 18:02:12.615667 10979 run_lenet_gray.cc:198] TOP 0 result id: 7
  I0411 18:02:12.615698 10979 run_lenet_gray.cc:198] TOP 1 result id: 9
  I0411 18:02:12.615727 10979 run_lenet_gray.cc:198] TOP 2 result id: 3
  I0411 18:02:12.615754 10979 run_lenet_gray.cc:198] TOP 3 result id: 4
  I0411 18:02:12.615782 10979 run_lenet_gray.cc:198] TOP 4 result id: 2
  

• The run_resnet50_feature.sh script implements resnet50 model inference using the feature input. The sample code quantizes and pads the feature data to meet the input conditions of the model, and then inputs it to the model for infer. To run it, go to the 03_misc directory, and then execute sh run_resnet50_feature.sh, as shown in the following code block:

  root@x5dvb-hynix8G:/userdata/horizon/x5/script/03_misc# sh run_resnet50_feature.sh
  ../aarch64/bin/run_resnet_feature --model_file=./resnet50_64x56x56_featuremap_modified.bin --data_file=../../data/misc_data/np_0 --top_k=5
  I0000 00:00:00.000000 11024 vlog_is_on.cc:197] RAW: Set VLOG level for "*" to 3
  [BPU_PLAT]BPU Platform Version(1.3.3)!
  [HBRT] set log level as 0. version = 3.15.18.0
  [DNN] Runtime version = 1.17.2_(3.15.18 HBRT)
  [A][DNN][packed_model.cpp:225][Model](2023-04-11,18:03:30.317.594) [HorizonRT] The model builder version = 1.15.1
  I0411 18:03:30.523054 11024 run_resnet_feature.cc:160] hbDNNInitializeFromFiles success
  I0411 18:03:30.523152 11024 run_resnet_feature.cc:168] hbDNNGetModelNameList success
  I0411 18:03:30.523188 11024 run_resnet_feature.cc:175] hbDNNGetModelHandle success
  I0411 18:03:30.529860 11024 run_resnet_feature.cc:346] input data size: 802816; input valid size: 200704; input aligned size: 229376
  I0411 18:03:30.536860 11024 run_resnet_feature.cc:357] tensor padding featuremap success
  I0411 18:03:30.536912 11024 run_resnet_feature.cc:190] prepare feature tensor success
  I0411 18:03:30.537052 11024 run_resnet_feature.cc:200] prepare tensor success
  I0411 18:03:30.537197 11024 run_resnet_feature.cc:210] infer success
  I0411 18:03:30.541096 11024 run_resnet_feature.cc:215] task done
  [W][DNN][hb_sys.cpp:108][Mem](2023-04-11,18:03:30.541.149) memory is noncachable, ignore flush operation
  I0411 18:03:30.541409 11024 run_resnet_feature.cc:220] task post process finished
  I0411 18:03:30.541453 11024 run_resnet_feature.cc:226] TOP 0 result id: 74
  I0411 18:03:30.541483 11024 run_resnet_feature.cc:226] TOP 1 result id: 815
  I0411 18:03:30.541512 11024 run_resnet_feature.cc:226] TOP 2 result id: 73
  I0411 18:03:30.541538 11024 run_resnet_feature.cc:226] TOP 3 result id: 78
  I0411 18:03:30.541565 11024 run_resnet_feature.cc:226] TOP 4 result id: 72
  

9.3.4. Helper Tools (Log)

There are 2 types of logs: sample log and dnn log. Wherein, sample log refers to the log in the Basic Sample Package release package; while dnn log refers to the log in the embedded dnn library. Developers can specify logs according to their own needs.

9.3.4.1. Sample Log

The sample logs are mainly vlog in glog, and all log contents will be output in the examples covered by basic_samples.

9.3.4.2. dnn Log

For the configuration of dnn logs, please read the Configuration Info section in the BPU SDK API DOC.