Loading TensorRT model is very slow on Jetson Nano

[AastaLLL]

Hi,

First, you can try to convert it into TensorRT engine with trtexec.

$ /usr/src/tensorrt/bin/trtexec --onnx=[your/model]

If all goes good, you can deploy it with the same below:

Custom ResNet Jetson Xavier

Hi, The error comes from dynamic shape usage. If you don’t need to reshape the model at the runtime, please set EXPLICIT_BATCH to use static shape instead. Here is an example of onnx model for your reference: import cv2 import time import numpy as np import tensorrt as trt import pycuda.autoinit import pycuda.driver as cuda EXPLICIT_BATCH = 1 << (int)(trt.NetworkDefinitionCreationFlag.EXPLICIT_BATCH) TRT_LOGGER = trt.Logger(trt.Logger.INFO) runtime = trt.Runtime(TRT_LOGGER) host_inputs = …

Thanks.