Convert custom Tensorflow model to TensorRT

Hello. I have a tensorflow model (model.pb) and a labels file (labels.txt). I have not been able to find a readable tutorial online for how to convert this model into a tensorrt file. Please explain to me, in simple terms that a beginner can understand, how to make this conversion.

Hi,

You can follow our sample which is located at /usr/src/tensorrt/samples/sampleUffSSD/.

Thanks.

I’m sorry, but I am returning my Nano. It has been a nightmare to even set up. Poor documentation, unreliable, hardware, and difficult to customize. The entire reason I bought one was because I wanted to use my own pre-trained model on it, and that is proving to require 10+ years of software development training. I looked at the sample you recommended, it is equally opaque to other instructions online.

You market this as a maker board, I assumed it would be on the same level of difficulty as a raspberry pi. Given the issues and errors and difficulty I have encountered, I am surprised anyone has bought the Nano. It is a horrible product and I can’t recommend it to anybody.

5 Likes

Hi tylersuard, thanks for your feedback - we are always striving to improve resources available for Jetson and make it easier to use.

Regarding your model, the beginner’s approach would be to start by running the model through TensorFlow first. You can find the documentation for installing TensorFlow on Jetson here. Then you would just run your TensorFlow script and model like normal (which still uses GPU acceleration).

Next, would be to utilize the TF-TRT interface. This allows you to accelerate your TensorFlow graph with TensorRT while keeping the TensorFlow environment to run any layers that may not be supported by TensorRT. See here for resources on using TF-TRT:

The final way, which does not rely on TensorFlow at runtime after the conversion has taken place, is to convert the graph to UFF and load it with the TensorRT C++ or Python runtime API. It can give the best performance but requires the conversion. Here are some resources on the UFF method:

There are also UFF samples included with the TensorRT SDK that can be found under /usr/src/tensorrt/samples

3 Likes

Since this is the top google result for “convert tensorflow model for TensorRT”, I would like inform people reading that UFF has been deprecated for TensorRT and so that workflow has become very difficult. The new worklflow is TensorFlow-ONYX-TensorRT. Please see this nVidia Developer blog:

I am stuck with this problem for the last two weeks… There are many links and links inside them, but I cant find the exact thing to do.

I am trying to convert a two-class ssd-mobilent-v2 object detection model for a custom dataset. Now I am trying your approach. If I am successful, I will do a write-up…

1 Like