How to convert pytorch (YoloV5) model to INT8

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Description

I am trying to convert YoloV5 (Pytorch) model to tensorrt INT8. I have taken 90 images which I stored in calibration folder and I have created the image directory text file (valid_calibartion.txt)

Error

undefined reference to Int8EntropyCalibrator2::Int8EntropyCalibrator2(int, int, int, char const*, char const*, char const*, bool)' CMakeFiles/yolov5.dir/yolov5.cpp.o: In function createEngine_m(unsigned int, nvinfer1::IBuilder*, nvinfer1::IBuilderConfig*, nvinfer1::DataType)‘:
/Downloads/tensorrtx-master_2/yolov5/tensorrtx-master/yolov5/INT8_yolov5/yolov5.cpp:214: undefined reference to Int8EntropyCalibrator2::Int8EntropyCalibrator2(int, int, int, char const*, char const*, char const*, bool)' CMakeFiles/yolov5.dir/yolov5.cpp.o: In function createEngine_l(unsigned int, nvinfer1::IBuilder*, nvinfer1::IBuilderConfig*, nvinfer1::DataType)’:
/Downloads/tensorrtx-master_2/yolov5/tensorrtx-master/yolov5/INT8_yolov5/yolov5.cpp:313: undefined reference to Int8EntropyCalibrator2::Int8EntropyCalibrator2(int, int, int, char const*, char const*, char const*, bool)' CMakeFiles/yolov5.dir/yolov5.cpp.o: In function createEngine_x(unsigned int, nvinfer1::IBuilder*, nvinfer1::IBuilderConfig*, nvinfer1::DataType)‘:
/tensorrtx-master_2/yolov5/tensorrtx-master/yolov5/INT8_yolov5/yolov5.cpp:413: undefined reference to `Int8EntropyCalibrator2::Int8EntropyCalibrator2(int, int, int, char const*, char const*, char const*, bool)’
collect2: error: ld returned 1 exit status
CMakeFiles/yolov5.dir/build.make:114: recipe for target ‘yolov5’ failed
make[2]: *** [yolov5] Error 1
CMakeFiles/Makefile2:67: recipe for target ‘CMakeFiles/yolov5.dir/all’ failed
make[1]: *** [CMakeFiles/yolov5.dir/all] Error 2
Makefile:83: recipe for target ‘all’ failed
make: *** [all] Error 2

Environment

TensorRT Version: 6.0.1.5
GPU Type: GeForce RTX 2060/PCIe/SSE2
Nvidia Driver Version: 418.67
CUDA Version: 10.1
CUDNN Version: 10
Operating System + Version: Ubuntu +16.04
Python Version (if applicable): 3.6
PyTorch Version (if applicable): 1.6.0

Relevant Files

Github Link : tensorrtx/yolov5 at master · wang-xinyu/tensorrtx · GitHub

2

Hi, Request you to share the ONNX model and the script so that we can assist you better.

Alongside you can try validating your model with the below snippet

check_model.py

import sys
import onnx
filename = yourONNXmodel
model = onnx.load(filename)
onnx.checker.check_model(model).

Alternatively, you can try running your model with trtexec command.
https://github.com/NVIDIA/TensorRT/tree/master/samples/opensource/trtexec

Thanks!

3

Thank you so much for the quick response.
I am not using onnx model.
I am converting yolov5.pt file to yolov5.wts
using gen_wts1.py please find attached file.

gen_wts (copy).py (877 Bytes)

i am creating yolov5.engine file using below link.
link: tensorrtx/yolov5 at master · wang-xinyu/tensorrtx · GitHub

Thank you

4

Hi @yugendra,
Can you please check the below link for your reference.

Also, we recommend you to try the latest TRT release for improved performance.

Thanks!

5

Hi @AakankshaS,

  1. What is the format of the calibration dataset so that i can create my own dataset ?
  2. How many images have to be in this calibration dataset ?
  3. How can i create calibration table?

Thank you.

6

Hi @yugendra,
Please refer to the below link
https://github.com/NVIDIA/TensorRT/tree/master/samples/opensource/sampleINT8#setup-the-calibrator

Thanks!