6.3.3. Prepare Calibration Data

Note

If you need to do this process in the sample folder, you need to execute the 00_init.sh script in the folder first to get the corresponding original model and dataset.

When converting models, 20~100 samples are required at the calibration stage, each is an independent data file. To ensure the accuracy of the converted models, these calibration samples better come from the training or validation dataset when training the models. In addition, please try NOT to use rare samples, e.g. single colored images or those images who don’t contain any detection or classification targets in them.

The ON and OFF states of the mentioned preprocess_on parameter in conversion configuration file respectively correspond to 2 different requirements for pre-processing samples. Detailed parameter configuration can be found in the Calibration Parameters section.

When the preprocess_on is OFF, you need to pre-process the samples taken from the training/validation dataset in the same way as you did before inference. The pre-processed calibration samples will hold the same data type (input_type_train ), size (input_shape ), and layout (input_layout_train ) as the original models. For the models input by the featuremap, you can save them one-by-one into binary as independent files. For example, there is an ImageNet trained original classification floating-point model with only one input node, it should be described as below:

  • Input type: BGR.

  • Input layout: NCHW.

  • Input size: 1x3x224x224.

When using the validation dataset for inference, take the following steps to pre-process the dataset:

  1. Uniformly scale the image and resize the shorter side to 256.

  2. Get 224x224 image using the center_crop method.

  3. Convert the input layout to the NCHW required by the model.

  4. Convert the color space to the BGR required by the model.

  5. Normalization.

According to the preprocess_on principle of sample file creation in the closed state, the sample processing code for the above example model is as follows (to avoid excessive code length, some simple transformer implementation codes are ignored, the usage of transformer can be found in Image Processing .

# this sample uses skimage, mind the differences when using opencv
# note that there are not mean subtraction or scale multiplication implementations in the following transformers
# note that the mean and scale operations are fused into the model
# as previously described in the norm_type/mean_values/scale_values
import skimage
import skimage.io
import numpy as np
from horizon_tc_ui.data.transformer import (CenterCropTransformer,
                                            HWC2CHWTransformer,
                                            RGB2BGRTransformer,
                                            ScaleTransformer,
                                            ShortSideResizeTransformer)
def data_transformer():
  transformers = [
  # uniformly scale the image and resize the shorter side to 256
  ShortSideResizeTransformer(short_size=256),
  # get 224x224 image using the CenterCrop
  CenterCropTransformer(crop_size=224),
  # read the NHWC layout results using the skimage and convert into the model required NCHW layout
  HWC2CHWTransformer(),
  # read the RGB channel sequence results using the skimage and convert into the model required BGR
  RGB2BGRTransformer(),
  # read the value range between [0.0,1.0] using the skimage and adjust into the model required value range
  ScaleTransformer(scale_value=255)
  ]

  return transformers

# the src_image refers to the source images in sample dataset
# the dst_file refers to the filename to save the final sample datasets
def convert_image(src_image, dst_file, transformers):
    image = [skimage.img_as_float(
        skimage.io.imread(src_image)).astype(np.float32)]
    for trans in transformers:
        image = trans(image)
    # type of input_type_train BGR value specified by the model is UINT8
    image = image[0].astype(np.uint8)
    # save samples into data files as binary
    image.tofile(dst_file)

if __name__ == '__main__':
  # refer to the original sample images, fake-code
  src_images = ['ILSVRC2012_val_00000001.JPEG', ...]
  # denote the filename (no restrictions on suffix) of the final samples, fake-code
  # calibration_data_bgr_f32 refers to your specified cal_data_dir in the configuration file
  dst_files = ['./calibration_data_bgr_f32/ILSVRC2012_val_00000001.bgr', ...]

  transformers = data_transformer()
  for src_image, dst_file in zip(src_images, dst_files):
      convert_image(src_image, dst_file, transformers)

Tip

When the preprocess_on is ON, use image format files supported by skimage to read samples. After reading the images, the conversion tool will scales them to the size required by the model input node and uses the result as input for calibration. Such operations are easier, but cannot guarantee the accuracy of the quantized models. Therefore, we strongly recommend you to use the pre-process_on OFF method.

Attention

Note that the input_shape parameter in the yaml file serves to specify the input data size of the original floating-point model. If it is a dynamic input model, you can use this parameter to set the converted input size, and the shape size of the calibration data should be consistent with input_shape.

For example, if the original floating-point model input node shape is ?x3x224x224 (“?” sign represents the placeholder, i.e., the first dimension of the model is dynamic input), and the input_shape: 8x3x224x224 is set in the conversion profile, then the size of each calibration data that you need to prepare is 8x3x224x224 (Please be aware that the input_batch parameter does not support modifying the model batch information for models with the first dimension of the input shape not equal to 1).