I am experimenting on my jetson nano with TensorRT Python API. I am using this repo https://github.com/penolove/yolov3-tensorrt for all operations necessary to run Yolov3 on TRT, i added
builder.fp16_mode=True
into engine creation block.
I have Jetpack 4.2.2 on my Nano. I get 0.5sec per frame inference time on TRT inference while getting 0.38sec per frame when using pure pytorch(v1.3) and torchvision(v0.5) implemantation of YOLO. I wasnt expecting 30FPS inference of course but this so dissappointing.
P.S I timed only the forward passes in both inference scripts.