4.1.2.4. The vec_diff Tool¶

The vec_diff tool is used to help you locate model accuracy problems, which may be caused by any of the following:

There is error on a certain layer caused by either calibration or quantization, the error was somehow enlarged and eventually leads to larger accuracy problem.
Some unknown problems exist in certain steps during the model conversion. During the conversion process from floating-point to fixed-point model, the data are optimized and transformed for a few times, where something might go wrong and lead to accuracy problems.
Software bugs: Due to the software bugs, error occurs in the places where numbers are supposed to be consistent.

The vec_diff tool is developed to find out the root causes. You can use this vector comparison tool to find the differences of conv layers outputs at different stages.

4.1.2.4.1. How to Use¶

The vec_diff tool can diff the output features at different model conversion stages.

The first step is to obtain the results of the hb_mapper infer tool.

The hb_mapper infer tool can be used to obtain the vector files generated by the model. For details of the tool, refer to Inference Tool (hb_mapper infer).

You can obtain the intermediate vector output by running *.bin model on the dev board.

Please refer to document on the runtime tool hrt_bin_dump tool introduction.

vec_diff [OPTIONS] left_file/folder right_file/folder

4.1.2.4.2. Parameters¶

Parameters:

--version: Displays the version information.

left_file/folder: Specifies the file name or folder name.
right_file/folder: Specifies the file name or folder name.

-o, --output-file FILENAME: Specifies the filename of output results.
--help: Displays the help information and exits.

4.1.2.4.3. Output Contents¶

CSV files and lists specified by the vec_diff -o command, including Left Files, Right Files, Cosine Similarity, Relative Euclidean Distance, Max Absolute Error, and Mean Square Error, as follows:

Left Files	Right Files	Cosine Similarity	Relative Euclidean Distance	Max Absolute Error	Mean Square Error
Layerxxx-quanti-input.txt	Layerxxx-float-input.txt	xxx	xxx	xxx	xxx
Layerxxx-quanti-param.txt	Layerxxx-float-param.txt	xxx	xxx	xxx	xxx