Accuracy drop in TensorRT

I have a tensorflow trained model and tested at tensorflow with accuracy achieved 95%.

Tensorflow model is converted to ONNX and converted to TensorRT.
TensorRT engine runs with 16-bit precision.
In TensorRT, accuracy drops to 75%.
Tested same images for both tests and same input size.

Where should I look at for this accuracy drop? The only difference is I do batch inference in TensorRT, but not in Tensorflow.

This link also has the same issue. But he said because of image reading.
For me, images are read using OpenCV for both Tensorflow and TensorRT.

I am using TensorRT7.1 and running on Jetson AGX Xavier.

Even I tested with kTF32, accuracy is still 75%. Is there sth worng?

@AakankshaS any idea on this issue? Why I have accuracy drop?

Hi @edit_or,
Request you to provide the code so that i can try it from my end.


My TRT program and tf model?

I have serious problem for accuracy drop from tensorflow to tensorrt.
My tensorflow modle is

My script is NumPlateRecognition.cpp (35.2 KB)

Can please help me on this?

Now I found the issue. Tensorflow did normalization to input image by multiplying 1/255.0. But in tensorrt normalization, it is 1- x/255.0. That is the issue. Now I have same accuracy.