I am working on deploy models on TX2
The model is ok on desktop.
On TX2, If the engine is builded without EXPLICIT_BATCH, everything is still fine. But if I build engine with EXPLICIT_BATCH, Inference is much slower than pytorch.
I have add some IPluginV2DynamicExt in my model, If I can’t use explicit batch, all these plugins have to be rewrited.
Is it possible to speed up the engine?
TensorRT Version: 6.0.1
GPU Type: Jetson TX2
Nvidia Driver Version:
CUDA Version: 10.0
Operating System + Version: Ubuntu 18.04
Python Version (if applicable): 3.6.9
TensorFlow Version (if applicable):
PyTorch Version (if applicable): 1.1
Baremetal or Container (if container which image + tag):