I recently got my hands on Jetson Nano and deployed simple image classification which I created in keras with only 3 classes. I followed this blog to convert to tensorrt with FP32 precision.
I did inference using webcam, model loaded in approx. 14 secs and got avg. 7.5 FPS utilizing 1.5GB ram.
I also did inference using Tensorflow model, model loaded in approx. 4 mins and also for this I had to increase swapfile size to 6GB in order to meet it’s memory demand after utilizing it’s 4GB ram memory or else process would get killed. This tf model was giving avg. 17 FPS
The question is why TensorRT is not giving better FPS as it optimized, am I missing something?