4.2.5.1. Similarities and Differences in Using the QAT Interface Compared to the Community

4.2.5.1.1. Operator Fusion Function

	torch	Plugin
horizon_plugin_pytorch.quantization.fuse_modules	Call this interface	Call this interface
The parameter settings of fuser_func related to the above interfaces	Use the community internal interface	horizon_plugin_pytorch.quantization.fuse_known_modules

As shown in the table above, both the community and Plugin call the horizon_plugin_pytorch.quantization.fuse_modules interface when fusing operators:

fuse_modules(model, modules_to_fuse, inplace=False, fuser_func=fuse_known_modules, fuse_custom_config_dict=None)

Difference: When using Plugin for QAT, you should use the horizon_plugin_pytorch.quantization.fuse_known_modules interface as a parameter to assign values to fuser_func in the above interface, which utilizes the operator fusion rules defined by Plugin.

4.2.5.1.2. QConfig Parameter Settings

	torch	Plugin
torch.quantization.QConfig	Call this interface	Call this interface
QConfig member variable: activation	Community-provided parameters	horizon_plugin_pytorch.quantization.default_8bit_fake_quant
QConfig member variable: weight	Community-provided parameters	horizon_plugin_pytorch.quantization.default_weight_8bit_fake_quant

As shown in the above table, when setting up the model’s qconfig , both the community and Plugin use torch.qconfig.QConfig . However, the Plugin customizes the parameters used by the QConfig on initialization and that quantify either the output or the weights.

Plugin also provides two interfaces for obtaining commonly used QConfig :

horizon_plugin_pytorch.quantization.get_default_qat_qconfig(bits=8, backend="")
horizon_plugin_pytorch.quantization.get_default_qat_out_qconfig(bits=8, backend="")

You can directly obtain the corresponding QConfig by using the above interface.

4.2.5.1.3. Model Conversion between Different Stages

Plugin provides horizon_plugin_pytorch.quantization.prepare_qat to convert a floating-point model to a QAT model, and uses horizon_plugin_pytorch.quantization.convert to convert a QAT model to a fixed-point prediction stage model.