Python TensorRT loaded engine failed

AaronCoC · April 23, 2023, 8:02am

I try the tutorial, and I export the “model_gn.engine” successfully.
I want to load the engine file with code:

import tensorrt as trt

engine_path="model_gn.engine"

TRT_LOGGER = trt.Logger(trt.Logger.WARNING)
with open(engine_path, "rb") as f, trt.Runtime(TRT_LOGGER) as runtime:
    engine = runtime.deserialize_cuda_engine(f.read())

Then I got the error

[04/23/2023-15:53:59] [TRT] [E] 1: [pluginV2Runner.cpp::load::292] Error Code 1: Serialization (Serialization assertion creator failed.Cannot deserialize plugin since corresponding IPluginCreator not found in Plugin Registry)
[04/23/2023-15:53:59] [TRT] [E] 4: [runtime.cpp::deserializeCudaEngine::49] Error Code 4: Internal Error (Engine deserialization failed.)

Is there any problem?

Here is my environment info:

pip list | grep tensorrt
tensorrt                      8.4.1.5

AastaLLL · April 24, 2023, 2:43am

Hi,

The model uses the plugin library.
Could you init the plugin before deserializing the model?

TRT_LOGGER = trt.Logger(trt.Logger.WARNING)
trt.init_libnvinfer_plugins(self.logger, namespace="")

Below is a Python sample for your reference:

github.com

NVIDIA/TensorRT/blob/release/8.4/samples/python/detectron2/infer.py

#
# SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
# SPDX-License-Identifier: Apache-2.0
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#

import os
import sys
import argparse

This file has been truncated. show original

Thanks.

AaronCoC · April 24, 2023, 3:10am

That works! Thanks a lot!

AaronCoC · April 24, 2023, 3:42am

Can you give a example with C++?

AastaLLL · April 25, 2023, 3:38am

Hi,

Please find the sample below:

github.com

NVIDIA/TensorRT/blob/release/8.5/samples/sampleOnnxMNIST/sampleOnnxMNIST.cpp

/*
 * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
 * SPDX-License-Identifier: Apache-2.0
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 * http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */

//!
//! sampleOnnxMNIST.cpp
//! This file contains the implementation of the ONNX MNIST sample. It creates the network using

This file has been truncated. show original

Thanks.

AaronCoC · April 25, 2023, 3:58am

Okay! Thanks!

Topic		Replies	Views
Troubleshooting TensorRT Engine Deserialization Issue: Null Engine Jetson Orin Nano tensorrt , cudnn	5	510	July 12, 2024
How to load and deserialize the .engine file? TensorRT	3	4928	January 30, 2020
How to load and deserialize the .engine file? Jetson TX2	1	1472	August 1, 2019
To deserialize an engine without a factory, please use IPluginV2 instead TensorRT tensorrt	5	1160	March 29, 2021
Python engine inference API bug TensorRT	1	566	February 13, 2023
TensorRT C++ engine deserealization failed. Windows 10 TensorRT	3	693	June 28, 2022
Error when Deserializing TensorRT Engine with Custom Plug-In TensorRT tensorrt	2	1135	September 12, 2020
Error: Cannot deserialize plugin since corresponding IPluginCreator not found in Plugin Registry with Yolov4 TensorRT engine TensorRT	5	3550	April 14, 2023
Deserialize TensorRT Engine Failure with Python API TensorRT	0	901	June 19, 2019
Python: Unable to load .trt model, but loads fine using trtexec TensorRT	3	2317	July 6, 2021

Python TensorRT loaded engine failed

Related topics