I am trying to convert a custom trained frozen object detection model (.pb) to a TensorRT model for faster inference on the NVIDIA Jetson Xavier-NX (or the Nano). Thus far, I have trained a 2-class object detector using Tensorflow’s Object Detection API with Mobilenet_v1 assuming this would generate the fastest inference possible on the Xavier-NX/Nano. the fastest inference possible. Right now, when I run the output frozen file (.pb) on the Jetson Xavier NX, I get about 1-FPS. I followed this tutorial:
To increase the Jetson Xavier-NX FPS, I read that if the frozen model (.pb) is converted to TensorRT, you can get much faster inference time. From what I understand, this requires converting the frozen model (.pb) to a UFF model (.uff). This UFF model is then used to create an ‘engine’ for which you can do inference. I was successful in converting my frozen (.pb) to UFF (.uff) using the following command when running on the Xavier-NX:
python3 convert-to-uff.py frozen_inference_graph.pb -O NMS -p config.py
That said, I am stuck as to what the next step is. I know I have to somehow convert my UFF file to an ‘engine file’–and after which I can somehow perform inference from this engine file. This is where I am looking for guidance–how do you convert the generated UFF file to an ENGINE file for NVIDIA Xavier-NX inference?
I am a beginner with Machine-Learning, and as such, if you are offering suggestions, please do not skip steps to reduce the difficulty in following.