How does TF to TRT work?


Sorry I am still learning tensorRT and its basics.
I was ran and tested the tf_to_trt samples you guys provided, but I don’t get how it actually works. As far as I know, tensorRT python api is not compatible with the tx2, so how are the python scripts working?
I ask because I want to see if I can adapt the scripts for other usage.

Thanks in advanced.

Hi jpeiwang99, you may want to view this webinar on the topic if you haven’t already:

The Python scripts are intended to be run on an x86 machine, to convert the frozen PB graph into a UFF file. Then the UFF is copied to the Jetson where it can be imported into TensorRT using the TensorRT C++ API. See this thread for more info:

Also, TensorFlow now is integrating TFTRT integration natively, so you may like to try that too:

Hi Dusty_nv,

Thanks for the reply.
I seem to have successfully run all the scripts on the TX2, meaning I converted the frozen Protobuf graphs into UFF files on the TX2. Benchmarking the tensorflow and tensorRT models also showed improvement, which leads me to assume that they were successfully optimized.

I guess to clarify what I’m asking, looking at the thread you linked me, I performed all the steps on the TX2 and got it to work, but is that supposed to happen since the python api is not supported on the TX2?

Thanks again.

The pre-process conversion scripts may be run on the Jetson if you install them there, however the TensorRT Python runtime API isn’t supported on Jetson at this time, so the runtime component should be run from C++.