TensorRT C++ engine deserealization failed. Windows 10

valyok94 · June 27, 2022, 8:54pm

Description

Hello, I’m trying to deserialize .engine file in C++, using the following code:

bool didInitPlugins = initLibNvInferPlugins(nullptr, “”);
nvinfer1::ICudaEngine* engine = infer->deserializeCudaEngine(model_data, model_size, nullptr);

But then I get the following error in Windows console output:

Assertion failed: d == a + length
C:_src\plugin\batchedNMSPlugin\batchedNMSPlugin.cpp:115

Engine file was created, using the same CUDA, TensorRT and GPU as on machine where I’m trying to deserialize it.

Environment

TensorRT Version: 8.2.5.1
GPU Type: Nvidia GeForce RTX 2070 Super
Nvidia Driver Version: 516.01
CUDA Version: 11.7
CUDNN Version: 8.4.0
Operating System + Version: Windows 10 version 21H2
Python Version (if applicable):
TensorFlow Version (if applicable):
PyTorch Version (if applicable):
Baremetal or Container (if container which image + tag):

Relevant Files

7 zip archive with C++ Visual Studio 2019 project
(cuda11.7_ETmodel.zip - Google Drive)

Steps To Reproduce

Launch “cuda11.7_ETmodel.exe” in “cuda11.7_ETmodel\x64\Release” folder

NVES · June 27, 2022, 9:07pm

Hi,
Please refer to below links related custom plugin implementation and sample:

While IPluginV2 and IPluginV2Ext interfaces are still supported for backward compatibility with TensorRT 5.1 and 6.0.x respectively, however, we recommend that you write new plugins or refactor existing ones to target the IPluginV2DynamicExt or IPluginV2IOExt interfaces instead.

Thanks!

valyok94 · June 28, 2022, 12:44pm

Can you please provide some example how to use “IPluginV2DynamicExt or IPluginV2IOExt interfaces” to desearilize .engine in C++ code?

spolisetty · June 28, 2022, 5:28pm

Hi,

Hope the following samples are helpful to you.

github.com

NVIDIA/TensorRT/blob/b55c4710ce01f076c26710a48879fcb2661be4a9/plugin/efficientNMSPlugin/efficientNMSPlugin.h

/*
 * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
 * SPDX-License-Identifier: Apache-2.0
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 * http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
#ifndef TRT_EFFICIENT_NMS_PLUGIN_H
#define TRT_EFFICIENT_NMS_PLUGIN_H

#include <vector>

This file has been truncated. show original

github.com

NVIDIA/TensorRT/blob/b55c4710ce01f076c26710a48879fcb2661be4a9/plugin/nmsPlugin/nmsPlugin.h

/*
 * SPDX-FileCopyrightText: Copyright (c) 1993-2022 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
 * SPDX-License-Identifier: Apache-2.0
 *
 * Licensed under the Apache License, Version 2.0 (the "License");
 * you may not use this file except in compliance with the License.
 * You may obtain a copy of the License at
 *
 * http://www.apache.org/licenses/LICENSE-2.0
 *
 * Unless required by applicable law or agreed to in writing, software
 * distributed under the License is distributed on an "AS IS" BASIS,
 * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
 * See the License for the specific language governing permissions and
 * limitations under the License.
 */
#ifndef TRT_NMS_PLUGIN_H
#define TRT_NMS_PLUGIN_H
#include "common/kernel.h"
#include "common/nmsUtils.h"

This file has been truncated. show original

Thank you.

Topic		Replies	Views
Error in tensorrt test TRT file TensorRT tensorrt , onnx	3	1352	July 5, 2022
Load TensorRT engine and deserialize in C++ TensorRT	11	4977	July 14, 2021
Runtime.deserialize_cuda_engine return a NoneType, how to fix ti? TensorRT tensorrt	10	2341	July 15, 2022
Assertion failed: *tensor = importer_ctx->network()->addInput( input.name().c_str(), trt_dtype, trt_dims) TensorRT tensorrt	15	924	June 24, 2022
getPluginCreator could not find plugin BatchedNMS_TRT version 1 TensorRT	5	3985	December 23, 2020
To deserialize an engine without a factory, please use IPluginV2 instead TensorRT tensorrt	5	1051	March 29, 2021
Python TensorRT loaded engine failed Jetson AGX Orin tensorrt	6	500	May 17, 2023
Efficient NMS plugin to TensorRT engine at runtime TensorRT	4	6457	May 17, 2022
Could not find any supported formats consistent with input/output data types TensorRT	1	816	April 11, 2023
Gain insight into TensorRT Engine TensorRT tensorrt	3	504	May 3, 2022

TensorRT C++ engine deserealization failed. Windows 10

Description

Environment

Relevant Files

Steps To Reproduce

Related topics