Deal the output cost more time when use the tensorRT, and why?

output = model_trt(input)
# it costs about 1000 times than using the pytorch model to get the output
# when use the pytorch -- 0.00024
# when use the tensorrt -- 0.1923


Do you use PyTorch API?

When calling .cpu(), PyTorch will move the buffer from GPU back to CPU.
So this might takes some time according to the IO bandwidth.

To accelerate, you can try to boost the device with the following commands.

$ sudo nvpmodel -m 0
$ sudo jetson_clocks


This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.