Accuracy decrease with TensorRT 7.2

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Description

Hi, I use pytorch 1.4 with cuda 10.1, cudnn 7.6.5 to train my model, and use onnx 1.7.0 to produce a *.onnx model. And then I separately use TensorRT 7.0.0 and TensorRT 7.2.1 to parse model and do inference, and the environment as followings:
TensorRT 7.0.0 with cuda 10.0, cudnn 7.6.5
TensorRT 7.2.1 with cuda 10.2, cudnn 8.0.4
I find that in these two situation, the accuracy varies greatly. When I use TensorRT 7.2.1, the recall of the model decrease by 3%. Could you tell me why this happens and what I can do to improve the accuracy?

Thanks very much!

3

Hi @jiangnan,
Can you please share your onnx model with us, so that we can try reproducing the issue.

Thanks!

4

I am so sorry that I cannot supply you the onnx model because of the project is not allowed. Don’t you do the test with the same onnx model for different TensorRT version? Or could you give me some advice what may lead this result?

Thanks very much!

5

Hi @jiangnan,
Can you try running your onnx model with trtxec command, and share the verbose results?

Thanks!