4.2.6.1. Calibration (Experimental Support)

In quantization-aware-training (QAT) of horizon_pytorch_plugin, an important step is to determine the quantization parameter scale. A reasonable scale can significantly improve the model training results and speed up the model convergence speed. A common method of calculating scale works as follows:

def compute_scale(data, quant_min, quant_max):
    fmax = data.abs().max()
    scale = fmax * 2 / (quant_max - quant_min)
    return scale

When calculating the scale of the feature map , since each forward can only calculate the fmax of the current batch, the feature map calculated by each forward may be inaccurate for the entire dataset. Therefore, the calibration method is introduced.

4.2.6.1.1. Calibration Method

Calibration is a method that uses floating-point model statistics to calculate scale Prior to QAT. Take the following steps:

1. Floating-point model forward, collect statistics of floating-point model.

2. Use the statistics from step 1, get the quantization parameters of the feature map by calibration.

3. Use the quantization parameters obtained in step 2 to initialize the quantization parameters of the QAT model.

4. Quantization aware training (QAT) based on step 3.

4.2.6.1.2. How to Use Calibration in Plugin

D-Robotics_plugin_pytorch provides the default calibration configuration. You can use calibration by setting float_model.qconfig = get_default_calib_qconfig() .

horizon.quantization.get_default_calib_qconfig()

4.2.6.1.3. Limitations of Calibration in Plugin

1. It only supports the calibration for feature map.

2. Modules with inconsistent schemas of train() and eval() are not supported.