Troubleshooting TensorRT Engine Deserialization Issue: Null Engine

import tensorrt as trt
import os

trt_logger = trt.Logger(trt.Logger.VERBOSE)

trt_file_path = “./yolox_tiny.trt”
if not os.path.exists(trt_file_path):
raise FileNotFoundError(f"The TensorRT file does not exist: {trt_file_path}")

try:
with open(trt_file_path, “rb”) as f:
engine_data = f.read()
if len(engine_data) == 0:
raise RuntimeError(“The TensorRT file is empty”)

    runtime = trt.Runtime(trt_logger)
    engine = runtime.deserialize_cuda_engine(engine_data)

if engine is None:
    raise RuntimeError("The engine failed to load")

context = engine.create_execution_context()
print("The engine and context have been loaded correctly")

except Exception as e:
print(f"An error has occurred: {e}")

In this program, I am loading the trt file that I created, but in ‘engine = runtime.deserialize_cuda_engine(engine_data)’, the engine becomes Null and I get the error ‘The engine failed to load’. Why is this happening? I appreciate your response.

Jetpack : 5.1.2-b104,
cuda : 11.4
cudnn : 8.6
TensorRT : 8.5.2.2

Hi,

Could you share the TensorRT verbose log with us as well?
It should output on the console since you already enabled it in your app.

Thanks.

Thank you for your response. The log is as follows. I appreciate your cooperation.

[07/09/2024-16:41:05] [TRT] [I] Loaded engine size: 11 MiB
[07/09/2024-16:41:05] [TRT] [W] Using an engine plan file across different models of devices is not recommended and is likely to affect performance or even cause errors.
[07/09/2024-16:41:05] [TRT] [E] 1: [pluginV2Runner.cpp::load::299] Error Code 1: Serialization (Serialization assertion creator failed.Cannot deserialize plugin since corresponding IPluginCreator not found in Plugin Registry)
[07/09/2024-16:41:05] [TRT] [E] 4: [runtime.cpp::deserializeCudaEngine::65] Error Code 4: Internal Error (Engine deserialization failed.)

Hi,

Please try below comment:

Thanks.

Hello. Thank you for your response. I did not register the plugin.

Thanks.

Hi,

Does your app work after registering the plugin?

Thanks.