output = model_trt(input) output.cpu() # it costs about 1000 times than using the pytorch model to get the output # when use the pytorch -- 0.00024 # when use the tensorrt -- 0.1923
Do you use PyTorch API?
.cpu(), PyTorch will move the buffer from GPU back to CPU.
So this might takes some time according to the IO bandwidth.
To accelerate, you can try to boost the device with the following commands.
$ sudo nvpmodel -m 0 $ sudo jetson_clocks
This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.