Differenct between converting to TensorRT from ONNX, Tensortflow or PyTorch (The best way to get TensorRT)

Autonomous Machines Jetson & Embedded Systems Jetson Nano
1

Hi
Is there a difference between converting to TensorRT from ONNX, TensorFlow, or PyTorch?

I’m trying to choose a framework in which I’ll train a NN and convert it to TensorRT. And the most important thing is to choose the framework that gives the best performance (in FPS) after converting to TensorRT.

And if there is no difference, then what is the best way to convert the PyTorch model to TensorRT?

I’m asking this question because my converted MobileNetV3 (small version) to TensorRT model only gives me about 20 FPS. As i know, MobileNetV3 faster than MobileNetV2 but MobileNetV2 can hit 64 FPS on Jetson by this article Jetson Benchmarks | NVIDIA Developer. Moreover I used 224x224 input size when 300x300 was used in article. So my suggession is that my TensorRT model is not optimal.
My converting Pipeline is: Pytorch → ONNX → TensorRT (with trtexec). Converted to fp16 of course

3

Hi,

Your converting pipeline looks good to us.
Just in case you don’t know, have you maximized the device performance?

$ sudo nvpmodel -m 0
$ sudo jetson_clocks

More, could you follow the same way to convert a MobileNetV2 model to see if you can reach the 64fps on Nano?

Thanks.

5

Sorry for the late answer, I didn’t have access to the jetson
The performance of the device was maximized.
Problem solved with using trtexec FILE (/usr/src/tensorrt/bin/trtexec) but NOT tensorrt library in python3. I still didn’t get 60 fps performance but close enough (about 55)

P.S. yes i thought i was using trtexec but i used tensorrt library in python3 :(
and converting using tensorrt library in python3 is not optimal