Is there a difference between converting to TensorRT from ONNX, TensorFlow, or PyTorch?
I’m trying to choose a framework in which I’ll train a NN and convert it to TensorRT. And the most important thing is to choose the framework that gives the best performance (in FPS) after converting to TensorRT.
And if there is no difference, then what is the best way to convert the PyTorch model to TensorRT?
I’m asking this question because my converted MobileNetV3 (small version) to TensorRT model only gives me about 20 FPS. As i know, MobileNetV3 faster than MobileNetV2 but MobileNetV2 can hit 64 FPS on Jetson by this article Jetson Nano: Deep Learning Inference Benchmarks | NVIDIA Developer. Moreover I used 224x224 input size when 300x300 was used in article. So my suggession is that my TensorRT model is not optimal.
My converting Pipeline is: Pytorch → ONNX → TensorRT (with trtexec). Converted to fp16 of course