Tensorrt Inference in Real time

Autonomous Machines Jetson & Embedded Systems Jetson Nano
, , ,
10

Hi,

1. Please check Q3.

2. Have you tried to infer the model on a dGPU?
Could you share the inference time?

3. It’s expected that TensorRT output the same result as ONNXRuntime.
If you didn’t get the correct results, it indicates there are some issues when converting the model into ONNX.
For this case, please check it with the tf2onnx team directly.

4. Usually, the TensorRT engine can be generated with trtexec.

$ /usr/src/tensorrt/bin/trtexec --onnx=[file]

5. The script shared above is for TensorRT8.4.
Please check the below change to make it works with TensorRT8.2:

Thanks.