I have a Faster R-CNN ResNet50 V1 pretrained model I’ve trained on some image data I have using the object detection API. It works fine and on a Xavier NX images take about 150ms to run. We need much better performance and want to explore using TensorRT.
From what I can see online the TensorFlow to TRT workflow seems to be based around TF1 frozen graphs and UFF, which aren’t supported in Tensorflow2, which I’m using. There’s another workflow that uses TF2 → Onnx → TRT, but I’m running into an issue moving the model from Onnx to TRT which is some sort of UINT8 error. TRT doesn’t support UINT8 so I’m stuck until I can swap data types. From what I can tell, this error is unique to the TF2 Object Detection API.
What is the current recommended workflow to move my model (using the saved model directory format) from TF2, to TRT given my use of the Object Detection API?
Environment
GPU Type: GTX1050
Nvidia Driver Version: 470.63.01
CUDA Version: 11.4
Operating System + Version: Ubuntu 20.04
TensorFlow Version (if applicable): TF 2.6