Hi there,
I got a saved model converted to onnx in order to run inference using TensorRT c++ api;
but ouput results are different compared to python inference and I don’t why. seems like values are slightly shifted.
infos:
tensorrt 8.2
cuDNN 8.2
onnx 1.8
cuda 11,
and did not set the BuilderFlag::kFP16 because my machine does not have that.
any help will be more than appreciated!
Environment
TensorRT Version: GPU Type: Nvidia Driver Version: CUDA Version: CUDNN Version: Operating System + Version: Python Version (if applicable): TensorFlow Version (if applicable): PyTorch Version (if applicable): Baremetal or Container (if container which image + tag):
Relevant Files
Please attach or include links to any models, data, files, or scripts necessary to reproduce your issue. (Github repo, Google Drive, Dropbox, etc.)
Hi,
Request you to share the ONNX model and the script if not shared already so that we can assist you better.
Alongside you can try few things:
validating your model with the below snippet
check_model.py
import sys
import onnx
filename = yourONNXmodel
model = onnx.load(filename)
onnx.checker.check_model(model).
2) Try running your model with trtexec command. https://github.com/NVIDIA/TensorRT/tree/master/samples/opensource/trtexec
In case you are still facing issue, request you to share the trtexec “”–verbose"" log for further debugging
Thanks!
Could you please let us know the which version of TenosrRT are you using. We recommend you to please try latest TRT version 8.2 EA. Also we recommend you to please verify result using onnx-runtime to confirm if problem is with onnx model.
If you still face an issue, please share us onnx model and sample data to try from our end for better debugging.
Hi,
Thank you for your response; and yes, i was already using TRT 8.2.0.6 EA for windows.
I have also tested onnx checker which successfully passed.
As I explained earlier, my algorithm is for image ring corrections but results from c++ TRT inférence are a bit different compared to python tensorflow inference which has better results (seems like values are a bit shifted and i dont really know why); the saved model is trained by python tensorflow and converted with tf2onnx.
I am not very sure about sharing the file onnx model; need to ask to my superior.
