Difference between TF-TFT and uff->tensorrt

Robotics & Edge Computing Jetson & Embedded Systems Jetson Nano
1

I found that we can optimize the Tensorflow model in several ways. Someone asked the same question in github, but no one answered. I think maybe i can get the answer here.

1- Using TF-TRT This API developer by tensorflow and integreted TensoRT to Tensorflow and this API called as :

from tensorflow.python.compiler.tensorrt import trt_convert as trt

This API can be applied to any tensorflow models (new and old version models) without any converting error, because If this API don’t support any new layers, don’t consider these layers for TensorRT engines and these layers remain for Tensorflow engine and run on Tensorflow. right?

2- Using TensorRT, This API by developed by NVIDA and is independent of Tenorflow library (Not integrated to Tensorflow), and this API called as:

import tensorrt as trt

If we want to use this api, first, we must converting the tensorflow graph to UFF using uff-convertor and then parse the UFF graph to this API.
In this case, If the Tensorflow graph have unsupported layers we must use plugin or custom code for these layers, right?

2

Hi,

1. TF-TRT is a TensorRT plugin embedded in the TensorFlow frameworks.
As you know, the advantage is that it will use TensorFlow implementation for those non-supported layer.
However, TensorFlow is not an optimal solution on Jetson, especially on performance and memory usage.

2. YES. You will need to make sure all the layers can be executed directly with TensorRT.
If yes, it will give you a much better performance.

My suggestion is to give pure TensorRT a try.
For most common architecture, pure TensorRT should work for them now.

Thanks.

3

Thank you for your answer!

I have another question. For object detection, I found several repos and threads showing real good demo/tutorial about how to use TF-TRT to do the job.

Since you mentioned the pure TensorRt provides better performance, do you have any good tutorial for me to follow? especially for the jetson Nano.

Many thanks.

4

Hi,

You can check following sample first:
https://github.com/dusty-nv/jetson-inference/blob/master/docs/detectnet-console-2.md
https://github.com/AastaNV/TRT_object_detection

Thanks.