TensorRT INT8 inference accuracy

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Description

When I quantify my segmentation model to fp16, the reasoning accuracy can reach 0.92, but when I quantify it to int8, the reasoning accuracy is only 0.59.

Environment

TensorRT Version: 7.2.0.14
GPU Type:
Nvidia Driver Version: NVIDIA Xavier NX
CUDA Version: 10.2
CUDNN Version: 8.0.0
Operating System + Version: ubuntu 18.04
Python Version (if applicable): 3.6
TensorFlow Version (if applicable):
PyTorch Version (if applicable): 1.6.0
Baremetal or Container (if container which image + tag):

Relevant Files

Please attach or include links to any models, data, files, or scripts necessary to reproduce your issue. (Github repo, Google Drive, Dropbox, etc.)

Steps To Reproduce

Please include:

  • Exact steps/commands to build your repro
  • Exact steps/commands to run your repro
  • Full traceback of errors encountered
2

Hi, Please refer to the below links to perform inference in INT8

Thanks!

5

Hi,

Also looks like you’re using an old version of the TensorRT. We recommend you please try on the latest version 8.4.

Thank you.