Troubleshooting TensorRT Engine Deserialization Issue: Null Engine

Robotics & Edge Computing Jetson & Embedded Systems Jetson Orin Nano
,
1

import tensorrt as trt
import os

trt_logger = trt.Logger(trt.Logger.VERBOSE)

trt_file_path = “./yolox_tiny.trt”
if not os.path.exists(trt_file_path):
raise FileNotFoundError(f"The TensorRT file does not exist: {trt_file_path}")

try:
with open(trt_file_path, “rb”) as f:
engine_data = f.read()
if len(engine_data) == 0:
raise RuntimeError(“The TensorRT file is empty”)

    runtime = trt.Runtime(trt_logger)
    engine = runtime.deserialize_cuda_engine(engine_data)

if engine is None:
    raise RuntimeError("The engine failed to load")

context = engine.create_execution_context()
print("The engine and context have been loaded correctly")

except Exception as e:
print(f"An error has occurred: {e}")

In this program, I am loading the trt file that I created, but in ‘engine = runtime.deserialize_cuda_engine(engine_data)’, the engine becomes Null and I get the error ‘The engine failed to load’. Why is this happening? I appreciate your response.

Jetpack : 5.1.2-b104,
cuda : 11.4
cudnn : 8.6
TensorRT : 8.5.2.2

3

Hi,

Could you share the TensorRT verbose log with us as well?
It should output on the console since you already enabled it in your app.

Thanks.

4

Thank you for your response. The log is as follows. I appreciate your cooperation.

[07/09/2024-16:41:05] [TRT] [I] Loaded engine size: 11 MiB
[07/09/2024-16:41:05] [TRT] [W] Using an engine plan file across different models of devices is not recommended and is likely to affect performance or even cause errors.
[07/09/2024-16:41:05] [TRT] [E] 1: [pluginV2Runner.cpp::load::299] Error Code 1: Serialization (Serialization assertion creator failed.Cannot deserialize plugin since corresponding IPluginCreator not found in Plugin Registry)
[07/09/2024-16:41:05] [TRT] [E] 4: [runtime.cpp::deserializeCudaEngine::65] Error Code 4: Internal Error (Engine deserialization failed.)

5

Hi,

Please try below comment:

Thanks.

6

Hello. Thank you for your response. I did not register the plugin.

Thanks.

7

There is no update from you for a period, assuming this is not an issue any more.
Hence we are closing this topic. If need further support, please open a new one.
Thanks

Hi,

Does your app work after registering the plugin?

Thanks.