中文 EN

6.3. Advanced Guide

This guide is suitable for users who have already understood the content of the beginner guide. It mainly introduces the detailed explanation of the principles and steps of the Post-Training Quantization (PTQ) method in the D-Robotics algorithm toolchain, the support instructions for model operators, and the embedded application development instructions for model board operation, etc. If you are a first-time user of the D-Robotics algorithm toolchain, it is recommended that you jump to Getting Started Guide to study the chapters.

• 6.3.1. Environment Installation
  • 6.3.1.1. Instructions for Using Delivered Packages
    • Source Code Directory Structure of the Sample Package
  • 6.3.1.2. Development Machine Deployment
    • Development Machine Preparation
    • Using Docker Environment
  • 6.3.1.3. Board Deployment
  • 6.3.1.4. Version Management Tool Usage Instructions
    • ddk_vcs list
    • ddk_vcs install
    • ddk_vcs uninstall
    • ddk_vcs patch
    • ddk_vcs show
• 6.3.2. PTQ Principle and Step-by-Step Guide
  • 6.3.2.1. Introduction
  • 6.3.2.2. Model Preparation
  • 6.3.2.3. Model Verification
    • Using the hb_mapper checker Tool to Verify Models
    • Handling Verification Exceptions
    • Interpreting Verification Results
    • Optimization Guidance from Verification Results
  • 6.3.2.4. Model Conversion
    • Preparing Calibration Data
    • Using the hb_mapper makertbin Tool for Model Conversion
    • Model Conversion YAML Configuration Parameters
      • Model parameter group
      • Enter information parameter group
      • Calibration parameter group
      • Compile parameter group
      • Custom operator parameter group
      • X5 int16 configuration instructions
      • Instructions for preprocessing HzPreprocess operator
      • [Reference] Supported calibration methods
    • Interpretation internal process
    • Interpretation of conversion results
    • Interpretation of Conversion Outputs
  • 6.3.2.5. Model Performance Analysis
    • Performance Evaluation on Development Machine
    • Performance Measurement on Development Board
    • Model Performance Optimization
      • Check YAML Parameters Affecting Model Performance
      • Handling CPU Operators
      • High-Performance Model Design Recommendations
  • 6.3.2.6. Model Accuracy Analysis
    • Accuracy Analysis
    • Accuracy Optimization
      • Significant Accuracy Loss (Over 4%)
      • Minor Accuracy Loss Optimization (1.5%~3%)
      • Accuracy Debug Tool
        • Usage Instructions
        • Function Description
        • get_sensitivity_of_nodes
        • plot_acc_error
        • plot_distribution
        • get_channelwise_data_distribution
        • sensitivity_analysis
  • 6.3.2.7. Improving Model Accuracy Using QAT (Quantization-Aware Training)
  • 6.3.2.8. Other Tool Usage Instructions
    • hb_mapper infer Tool
    • hb_perf Tool
    • hb_pack Tool
    • hb_model_info Tool
    • hb_model_modifier Tool
    • hb_verifier Tool
    • hb_eval_preprocess Tool
    • HB_ONNXRuntime Inference Library
• 6.3.3. Model Operator Support List
  • 6.3.3.1. Usage Limitations
  • 6.3.3.2. X5 Supported Caffe Operators
  • 6.3.3.3. X5 Supported ONNX Operators
• 6.3.4. Model board running application development instructions
  • 6.3.4.1. Model Inference DNN API usage example description
    • Overview
      • Introduction to the sample code package structure
    • Environment Construction
      • Development Board Preparation
      • Compilation
    • Example usage
      • basic_samples example
      • quick_start
      • api_tutorial
      • advanced_samples
      • misc
    • Auxiliary Tools and Common Operations
      • Logs
      • Example Logs
      • Model Inference DNN API Logs
  • 6.3.4.2. Public Model Performance and Accuracy Evaluation Guide
    • Introduction
    • Release Package Description
      • Example Code Package Structure
      • Example Models
      • Public Datasets
    • Environment Setup
      • Development Board Preparation
      • Compilation Environment Preparation
    • Evaluation Example Usage Guide
      • JSON Configuration File Parameter Information
        • Input Configuration (input_config)
        • Output Configuration (output_config)
      • Performance Evaluation
        • Evaluation Script Usage
          • Command-line Parameter Explanation
          • Result Visualization
      • Accuracy Evaluation
        • Data Preprocessing
        • Generating lst Files
        • Data Mounting
        • Model Inference
        • Accuracy Evaluation
    • Model Integration
      • Preprocessing
        • Preprocessing File and Header File Addition
        • Model Preprocessing Configuration File Addition
        • Single-Frame Latency Evaluation for Preprocessing
      • Postprocessing
        • Postprocessing File and Header File Addition
        • Add Model Execution Scripts and Configuration Files
    • Auxiliary Tools and Common Operations
      • Logging System Usage Instructions
        • Example Logs
        • Model Inference API DNN Logs
      • Operator Latency Explanation
      • Dump Tool
  • 6.3.4.3. On-Board Model Analysis Tool Description
    • Overview
    • hrt_model_exec Tool Usage Instructions
      • Input Parameter Description
      • Usage Instructions
      • model_info
      • Additional Input Parameter Explanation
      • infer
      • Multi-Input Model Explanation
      • Additional Input Parameter Explanation
      • perf
      • Multi-threaded Latency Data Explanation
      • Input Parameter Additional Notes
      • Common Questions
        • How are Latency and FPS data calculated?
        • Why does FPS derived from Latency differ from the tool-measured FPS?
        • How does the tool evaluate models with custom operators?
    • hrt_bin_dump Tool Usage Guide
      • Input Parameter Description
      • Usage Instructions
      • Example Explanation
• 6.3.5. Model Inference Interface Description
  • 6.3.5.1. Model Inference Library Version Information Retrieval API
    • hbDNNGetVersion()
  • 6.3.5.2. Model Loading/Release API
    • hbDNNInitializeFromFiles()
    • hbDNNInitializeFromDDR()
    • hbDNNRelease()
  • 6.3.5.3. Model Information Retrieval API
    • hbDNNGetModelNameList()
    • hbDNNGetModelHandle()
    • hbDNNGetInputCount()
    • hbDNNGetInputName()
    • hbDNNGetInputTensorProperties()
    • hbDNNGetOutputCount()
    • hbDNNGetOutputName()
    • hbDNNGetOutputTensorProperties()
  • 6.3.5.4. Model Inference API
    • hbDNNInfer()
    • hbDNNRoiInfer()
    • hbDNNWaitTaskDone()
    • hbDNNReleaseTask()
  • 6.3.5.5. Model Memory Operation API
    • hbSysAllocMem()
    • hbSysAllocCachedMem()
    • hbSysFlushMem()
    • hbSysFreeMem()
    • hbSysWriteMem()
    • hbSysReadMem()
    • hbSysRegisterMem()
    • hbSysUnregisterMem()
  • 6.3.5.6. Data Types and Data Structures
    • Version Information
    • Model-related Types
    • System-related Types
    • Preprocessing Classes
  • 6.3.5.7. Data Layout and Alignment Rules
    • Data Layout
    • BPU Alignment Restrictions
      • Model Input Requirements
      • Alignment and Valid Data
    • Introduction to NV12
      • YUV Format
      • NV12 Layout
  • 6.3.5.8. Model Inference DEBUG Methods
    • Error Codes
    • Configuration Information
    • Notes on Using Simulator on Development Machine