7.3.9. Ideas of Quantization Precision Debug

7.3.9.1. BaseLine

Generally, the precision problem of QAT or quantized models can be measured by a unified standard. When it is lower than this standard, it can be considered that there is a problem with the accuracy of quantization. If this standard does not meet the needs of special tasks, it needs to be handled separately.

7.3.9.1.1. Quantization Precision Standard

General metric: quantitative model metric >= floating point model metric * 0.99 .

  • Single task: the general metric >= floating point * 0.99 meets the conditions. Some sub-metrics (such as the detection metrics of a single object) may fluctuate.

  • Multitask: the metric of each task >= floating point * 0.99 meets the conditions. In some cases, there will be fluctuations affected by multi-task training. Some tasks have high metrics (greater than 100%), and some tasks have slightly lower metrics (below 99%).

In real situations, it may be found that the QAT accuracy is less than the quantization accuracy standard, such as a 0.x% index gap. This can be solved by adjusting parameters first, and the experience of parameter adjustment can be referred to QAT Experience Summary. If the gap is large, users can solve any low accuracy situation according to the following insights.

7.3.9.2. Flow Chart

7.3.9.3. NAN

In the QAT stage, the metrics may be completely unchanged, or there may be an obvious NAN error. In this case, it can generally be considered as NAN in the network, which is actually similar to the floating point stage. The following methods can be used to check and avoid the NAN in the QAT stage.

  1. Check if the precision of the floating point model is normal. If the model in floating point stage has problems or the accuracy is very low, QAT training will have convergence problems.

  2. Check whether there are any dubious points in the model that affect the convergence, such as common bugs. The check method is consistent with the training divergence in floating point model. This location may be troublesome, and it is more conventional to quantify part of the model and analyze the impact of this part of quantification.

  3. If no problems are found, the lr of the QAT stage can be adjusted appropriately. Sometimes adding warmup can also solve the problem, but you need to consider the impact of warmup on the final metric.

7.3.9.4. Low Calibration Precision

In Calibration (Experimental Support), we suggest that when there is a problem with QAT, calibration can be used by default. Calibration can provide a better initialized scale value for QAT, but Calibration may not solve all accuracy problems. If there is an obvious problem with the accuracy of calibration, consider the following situations:

  1. Use the feature map similarity comparison tool provided in Model Precision Debug Tool to find the layer with a large quantization loss. For layers with large quantization loss, it can be properly converted to int16.

  2. Use the step quantization tool provided in Model Precision Debug Tool to find the module with large quantization loss, and conduct a fine-grained analysis of the module.

On one hand, the analysis of calibration accuracy can be solved by setting int16 quantization of some ops. On the other hand, users can also get the quantization-friendly analysis related to the floating point model referring to Build Quantization-Friendly Floating Point Model.

7.3.9.5. Low QAT Precision

QAT is a dynamic training process, so there are many factors that affect the accuracy. According to previous practical experience, the improvement of the entire QAT after the first epoch is about 1-3 points. In generally, if the metric of the first epoch is low, it is necessary to consider that QAT has obvious accuracy problems. After adding calibration by default, if there is still an issue with accuracy, users can analyze it by the following tools:

  1. According to the experience provided by QAT Experience Summary, make some appropriate adjustments to the QAT process, including quantization methods, hyperparameters, training strategies, etc.

  2. Analyze floating point models and find operations that may be not friendly to quantization by referring to Build Quantization-Friendly Floating Point Model. Make appropriate adjustments to the floating point models based on the analysis.

  3. Use the debug tool provided by Model Precision Debug Tool to analyze the obvious GAP between the floating point and QAT models. Solve the GAP by adjusting some ops quantized by int16, or even keeping some in floating point.

7.3.9.6. Low Quantized Precision

The current QAT->Quantized process is fixed, and it has been verified that the conversion to quantized models do not cause obvious accuracy problems in most scenarios. Therefore, when the quantized accuracy is low, users may need to cooperate with the development team to deal with it. However, users can also use the similarity comparison tool provided in Model Precision Debug Tool to help analyze which branches and layers lead to the low final quantized accuracy.