4.1.1.1. PTQ Conversion Process and Steps

Model conversion is the process of converting the original floating-point model to a D-Robotics hybrid heterogeneous model.

The original floating-point model (also referred to as a floating-point model in sections of the document) is an available model trained by a DL framework such as TensorFlow/PyTorch, with computation precision of float32; the hybrid heterogeneous model is a model format suitable for running on the D-Robotics computing platform.

This section will repeatedly use the two model terms. To avoid ambiguity, please understand the concept before reading the following section.

The complete the model development process with the D-Robotics toolchain involves five important stages: Floating-point Model Preparation, Model Checking, Model Conversion, Performance Evaluation, and Accuracy Evaluation, as shown in the figure below.

The Floating-point model, as the output of the Floating-point Model Preparation stage, will serve as the input of the model conversion tool. The floating-point model is usually trained on basis of some open source deep learning frameworks. Note that the model must be exported to a format supported by D-Robotics. For more information, please refer to the Floating-point Model Preparation.

The Model Checking stage is used to ensure that the model is computing platform compliant. D-Robotics provides specified tools to complete model validation, and for non-compliance, such tools will explicitly give you the specific operator information for the non-compliance, so that you can easily adjust the model with the description of the operator constraints. For more information, please refer to the Verify model.

The Model Conversion stage converts the floating-point model to the hybrid heterogeneous model supported by D-Robotics. To run models efficiently on the D-Robotics computing platform, critical steps such as model optimization, quantization, and compilation are completed by D-Robotics’ model conversion tools. D-Robotics’ model quantization method has undergone long-term technological and production validation, and can guarantee an accuracy loss of less than 1% on most typical deep learning models. For more details about model conversion please refer to the Prepare Calibration Data and Model Quantization and Compilation.

The Performance Evaluation stage contains a series of tools to evaluate the model performance. Before deploying your application, you can use these tools to verify that the model performance meets application requirements. For some cases where the performance is not as good as expected, you can optimize the models based on D-Robotics’ model optimization advices. For more information, please refer to the Model Performance Analysis and Optimization.

The Accuracy Evaluation stage contains a series of tools to evaluate the accuracy of the model. In most cases, D-Robotics’ converted-models can maintain almost the same accuracy as the original floating-point model. Before application deployment, you can use these tools to verify that the accuracy of the model meets the expectations. For some cases where the accuracy is not as good as expected, you can optimize the models based on D-Robotics’ model optimization advices. For more information about evaluation, please refer to the Model Accuracy Analysis and Optimization.

Attention

• In general, you can get a qualified runtime model after model conversion. However, make sure that the performance and accuracy of the model are in line with the application requirements. D-Robotics strongly suggest you that each conversion shall be followed by the evaluation steps of performance and accuracy.

• The model conversion process will generate onnx models, which are intermediate products and only facilitate users to verify the accuracy of the model. Therefore, the compatibility between versions is not guaranteed. When using the evaluation script in the example to evaluate the onnx model in a single image or on a test set, please use the onnx model generated by the current version of the tool.