Jetson AGX Xavier engine unexpected exceptions

Recently I am experiencing an [TRT] [E] 1: Unexpected exception (see attached trtinference_fp16_failure.txt) when running a script (attached below) doing tensorrt inferences. I created the engine using trtexec (see below). When creating the engine without the “–fp16” flag I get no exception and everything works.

I am running on Jetpack 4.6.2 with TensorRT
Any ideas or recommendations?

Steps to reproduce the issue:
Pull docker image:
docker pull allu1234/pysot-xavier-torch19:1.0

Create engine with trtexec:

trtexec --onnx=OT_target_net4refit_fp16.onnx --fp16 --saveEngine=OT_target_net4refit_fp16.engine



You may have to modify the path to the engine that should be loaded.

trtinference_fp16_failure.txt (2.3 KB)

Additional resources [1/2]: (1.3 KB)

Additional resources [2/2]:
onnx-model: OT_target_net4refit_fp16.onnx - Google Drive


Could you try if you can run the model with trtexec directly?

$ /usr/src/tensorrt/bin/trtexec --loadEngine=OT_target_net4refit_fp16.engin

If yes, there might be some issue in your python implementation.
Below is an example to load a TensorRT engine with python for your reference:


I could run the model with trtexec directly, so I have reworked my python code for allocating the buffers and executing inference based on your given example and now it works. So the issue was in my python implementation.

Thanks for the quick response.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.