TensorRT for Jetson Nano

[enesergul.11]

When I run darknet over jetson nano, I get 99% average gpu usage of 15 fps. But when I use TensorRT, 22 fps gpu usage is around 70%. Why is TensorRT not using the whole gpu.

Jetson Nano Jetpack = 4.6

and using rtx 2060 laptop

Darknet AVG FPS = 315
TensorRT AVG FPS =165

Considering the GPU usages, the system draws an average of 14 watts of power at no load, while it draws 100 watts with darknet and 40 watts with TensorRT.

When the fps rate per watt is calculated,
Darknet 315/86 = 3.66 fps
TensorRT 165/26 =6.34 fps

Is it possible to use the gpu more?

[AastaLLL]

Hi,

Did you test it with trtexec? If not, please give it a try.

If GPU needs to wait for the input data, the resources cannot be fully utilized.
So it’s recommended to try trtexec first.
It can measure the pure inference time without waiting for image preprocessing.

Thanks.