Abou pytorch2onnx2TRT,serialize and deserialize

I try to use my model from pytorch 2 onnx,then 2 trt and it works well.
but ,when I try to save onnx as ‘.engin’ in c++ trt and load , I got some err like this:
getPluginCreator could not find plugin ResizeNearest version 001 namespace
Cannot deserialize plugin ResizeNearest
Some one says this because Somehow the ResizeNearest plugin is implemented in tensorrt onnx parser.
link:
https://devtalk.nvidia.com/default/topic/1050845/tensorrt/tensorrt-5-1-c-api-cannot-deserialize-retinanet-trt-engine/post/5333756/#5333756

I get the .onnx as follow:
from onnx import helper
helper.make_node(
‘Upsample’,
mode=‘nearest’,
scales=[1.0, 1.0, 2, 2],
inputs=inputs,
outputs=[layer_name],
name=layer_name,
)…
and I got :
onnx::Upsample[mode = ‘nearest’, scales = [1, 1, 2, 2]]…

How to slove this problem?

Hello, can you provide details on the platforms you are using?

Linux distro and version
GPU type
nvidia driver version
CUDA version
CUDNN version
Python version [if using python]
Tensorflow version
TensorRT version

Also a small repro of the source/model that demonstrate the error you are seeing will help us debug with you too.

Under WIN10 enviroment:
Cuda :cuda_9.0.176_win10
cudnn : cudnn-9.0-windows10-x64-v7.5.0.56

I got the .onnx model from python3.5 ,pytorch1.0(make some modify about export becuase upsample operation will cause fault, link :GitHub - NVIDIA/retinanet-examples: Fast and accurate object detection with end-to-end GPU optimization).
I run the this onnx molde on VS2015 tensorRT 5.0.4.3, and get the right result, it works well.

ONNX IR version: 0.0.3
Opset version: 9
Producer name: pytorch
Producer version: 0.4
Domain:
Model version: 0
Doc string:

But ,when i try save the model as follow:

void onnxToTRTModel(const std::string& modelFile,
unsigned int maxBatchSize,
IHostMemory*& trtModelStream)
{
int verbosity = (int) nvinfer1::ILogger::Severity::kWARNING;
IBuilder* builder = createInferBuilder(gLogger);
nvinfer1::INetworkDefinition* network = builder->createNetwork();
auto parser = nvonnxparser::createParser(*network, gLogger);
if (!parser->parseFromFile(locateFile(modelFile, directories).c_str(), verbosity))
{
string msg(“failed to parse onnx file”);
gLogger.log(nvinfer1::ILogger::Severity::kERROR, msg.c_str());
exit(EXIT_FAILURE);
}

builder->setMaxBatchSize(maxBatchSize);
builder->setMaxWorkspaceSize(1 << 20);
samplesCommon::enableDLA(builder, gUseDLACore);
ICudaEngine* engine = builder->buildCudaEngine(*network);
assert(engine);
parser->destroy();
trtModelStream = engine->serialize();
engine->destroy();
network->destroy();
builder->destroy();


/*               save trtModelStream as      .engine */
std::fstream file;
file.open("./serialize_engine_output.engine", ios::binary | ios::out);
file.write((const char*)trtModelStream->data(), trtModelStream->size());
file.close();

}

and try load this model as follow:


IRuntime* runtime = createInferRuntime(gLogger);
std::fstream file;
file.open(“./serialize_engine_output.engine”, ios::binary | ios::in);
file.seekg(0, ios::end);
int length_ = file.tellg();
file.seekg(0, ios::beg);
std::unique_ptr<char>data_(new char[length_]);
file.read(data_.get(), length_);
file.close();
ICudaEngine* engine = runtime->deserializeCudaEngine(data_.get(), length_, nullptr);


Then ,the fualt like the follow:

ERROR: 00007FFBC2DCF2D0ResizeNearest version 001 namespace
ERROR: Cannot deserialize plugin ResizeNearest
ERROR: 00007FFBC2DCF2D0ResizeNearest version 001 namespace
ERROR: Cannot deserialize plugin ResizeNearest
ERROR: 00007FFBC2DCF2D0ResizeNearest version 001 namespace
ERROR: Cannot deserialize plugin ResizeNearest
ERROR: 00007FFBC2DCF2D0ResizeNearest version 001 namespace
ERROR: Cannot deserialize plugin ResizeNearest