I am trying to convert tf-keras segmentation model to tensorrt engine and perform inference on it. The tensorrt engine gives incorrect/noisy result output. I have tried porting classification model and it works. The problem is with tensorflow models only.
Following are the steps I followed:
- Convert the model to ONNX model using tf2onnx using following command:
python -m tf2onnx.convert --saved-model “Path_To_TF_Model” --output “Path_To_Output_Model\Model.onnx” --verbose
I performed inference on this onnx model using onnxruntime in python. It gives correct output.
This ONNX model I converted to TensorRT engine using following command:
trtexec.exe --onnx=“Patrh_To_ONNX_Model\model.onnx” --saveEngine=“PAth_To_TRT_Engine\model.engine” --verbose --explicitBatch
I performed inference on this tensorrt engine in c++. Inference outputs are all wrong and noisy.
There are no errors or warnings in the verbose output while generating the engine. In fact the onnx model inference gives the correct output. So I am confused now.
I can share my code, verbsose outputs, tf and onnx models as well.
Platform: Windows 10
CUDA : 11.1
TensorRT version : 7.2.2
CuDNN version : 8.0.4
Please let me know if you need any other input from my side.