7.6.2. Calibration v2 (Experimental Support)¶

D-Robotics Plugin Pytorch supports a new calibration method as of version 1.2.1. Compared with the original calibration, the new calibration supports more calibration methods, and its usage is more flexible. We recommend users to try the new calibration method first. The original calibration is still compatible, but will be gradually deprecated in later versions.

7.6.2.1. Usage¶

The following figure shows the overall process of calibration and QAT:

../../../_images/calibration_v2_workflow.svg

Take the following steps:

Build and train a float model. Refer to subsections Building a floating point model and Pre-train A Floating Point Model in the “Quick Start” chapter.
Convert float model to QAT model. Refer to subsections Set BPU Architectures , Operator Fusion and Convert a Floating Point Model to a quantitative Model in the “Quick Start” chapter. Before using the prepare_qat method to convert a float model, you need to set qconfig for the model.
```
model.qconfig = horizon.quantization.get_default_qconfig()
```
get_default_qconfig can set different fake_quant and observer for weight and activation. Currently, the supported fake quant methods are “fake_quant”, “lsq” and “pact”, and the supported observers are “min_max”, “fixed_scale”, “clip”, “percentile” and “clip_std”. If there is no special requirement, it is recommended to use the default “fake_quant” method for activation_fake_quant and weight_fake_quant , and the default “min_max” for weight_observer . activation_observer is recommended to use the default “min_max” if the qconfig is set for the QAT phase, and it is recommended to use “percentile” if the qconfig is set for the calibration phase. Optional observers for calibration are “min_max”, “percentile”, and “clip_std”. See the calibration experience summary for special usage and debugging tips.
```
def get_default_qconfig(
    activation_fake_quant: Optional[str] = "fake_quant",
    weight_fake_quant: Optional[str] = "fake_quant",
    activation_observer: Optional[str] = "min_max",
    weight_observer: Optional[str] = "min_max",
    activation_qkwargs: Optional[Dict] = None,
    weight_qkwargs: Optional[Dict] = None,
):
```
Set the state of fake quantize as CALIBRATION .
```
horizon.quantization.set_fake_quantize(model, horizon.quantization.FakeQuantState.CALIBRATION)
```
There are 3 states of fake quantize in total, which need to be set before QAT, calibration and validation respectively. In the calibration state, only the statistics of the input and output of each operator are observed. In the QAT state, fake quantization operations are performed in addition to the observed statistics. In the validation state, no statistics are observed, and only fake quantization operations are performed.
```
class FakeQuantState(Enum):
    QAT = "qat"
    CALIBRATION = "calibration"
    VALIDATION = "validation"
```
Calibration. The calibration data is fed to the model, and the relevant statistics are observed by the observer during the forward process of the model.

Set the state of fake quantize as VALIDATION .

horizon.quantization.set_fake_quantize(model, horizon.quantization.FakeQuantState.VALIDATION)

Verify the accuracy of calibration model. If the accuracy is satisfied, go to step 7. If not, adjust the parameters in calibration qconfig to get a better accuracy.
Starting from the floating point model, follow the process of step 2 to build the QAT model again. Please note the difference of the qconfig settings between the QAT phase and the calibration phase.

Load the parameters observed in calibration.

horizon.quantization.load_observer_params(calibration_model, qat_model)

Set the state of fake quantize as QAT .

horizon.quantization.set_fake_quantize(model, horizon.quantization.FakeQuantState.QAT)

Perform the QAT.

Set the state of fake quantize as VALIDATION and verify the accuracy of QAT model.

horizon.quantization.set_fake_quantize(model, horizon.quantization.FakeQuantState.VALIDATION)

7.6.2.2. Limitations of Calibration in Plugin¶

Attention

Modules with different train and eval schemas are not supported.