Jetson AGX Xavier engine unexpected exceptions

[allu12345]

Recently I am experiencing an [TRT] [E] 1: Unexpected exception (see attached trtinference_fp16_failure.txt) when running a script (attached below) doing tensorrt inferences. I created the engine using trtexec (see below). When creating the engine without the “–fp16” flag I get no exception and everything works.

I am running on Jetpack 4.6.2 with TensorRT 8.2.1.8.
Any ideas or recommendations?
Thanks!

Steps to reproduce the issue:
Pull docker image:
docker pull allu1234/pysot-xavier-torch19:1.0

Create engine with trtexec:

trtexec --onnx=OT_target_net4refit_fp16.onnx --fp16 --saveEngine=OT_target_net4refit_fp16.engine

Run trtenginetest.py:

python3 trtenginetest.py

You may have to modify the path to the engine that should be loaded.

trtinference_fp16_failure.txt (2.3 KB)

[allu12345]

Additional resources [1/2]:
trtenginetest.py (1.3 KB)

[allu12345]

Additional resources [2/2]:
onnx-model: OT_target_net4refit_fp16.onnx - Google Drive

[AastaLLL]

Hi,

Could you try if you can run the model with trtexec directly?

$ /usr/src/tensorrt/bin/trtexec --loadEngine=OT_target_net4refit_fp16.engin

If yes, there might be some issue in your python implementation.
Below is an example to load a TensorRT engine with python for your reference:
https://elinux.org/Jetson/L4T/TRT_Customized_Example#OpenCV_with_PLAN_model

Thanks.

[allu12345]

I could run the model with trtexec directly, so I have reworked my python code for allocating the buffers and executing inference based on your given example and now it works. So the issue was in my python implementation.

Thanks for the quick response.