TensorRT custom layer help

AI & Data Science Deep Learning (Training & Inference) TensorRT
#1

I am trying to write a custom layer in TensorRT using addPluginV2. I see this error:
[06/10/2020-22:47:08] [E] [TRT] Parameter check failed at: …/builder/Network.cpp::addPluginV2::997, condition: containsTensor(inputs[i])

i.e. the layer after my custom layer cannot find the output of the custom layer. Any pointers?

Thanks.

#2

Error log implies that the input tensors are not owned by the network object.
Can you post a piece of sample code, so we can help better?

Thanks

#3

last_conv->setPadding(DimsHW{0, 0});

IActivationLayer* relu1 = network->addActivation(*last_conv->getOutput(0), ActivationType::kRELU);

std::cout << relu1->getOutput(0)->getDimensions() << std::endl;
IPoolingLayer* avg_pool = network->addPooling(*relu1->getOutput(0),PoolingType::kAVERAGE,DimsHW{7,7});
assert(avg_pool);
std::cout << avg_pool->getOutput(0)->getDimensions() << std::endl;



//add the plugin to the TensorRT network using the network API
nvinfer1::ITensor* const* point = (nvinfer1::ITensor* const*) &(*avg_pool->getOutput(0));

nvinfer1::PluginFieldCollection plugfc;
plugfc.nbFields = 0;
plugfc.fields = nullptr;
nvinfer1::IPluginV2* pluginV2 = getPluginRegistry()->getPluginCreator("Identity", "2")->createPlugin("testing", &plugfc);
assert(pluginV2);

auto layer = network->addPluginV2(point, 1280 * 4, *pluginV2);
layer->setName("testing");


nvinfer1::Weights fc_weight, fc_bias;
auto arr4 = cnpy::npy_load("mbnetv2/final_dense_kernel.npy");
fc_weight.values = arr4.data<short>();
fc_weight.count = arr4.shape[0] * arr4.shape[1];
auto arr5 = cnpy::npy_load("mbnetv2/final_dense_bias.npy");
fc_bias.values = arr5.data<short>();
fc_bias.count = arr5.shape[0];
fc_weight.type = DataType::kHALF;
fc_bias.type = DataType::kHALF;
IFullyConnectedLayer * fc = network->addFullyConnected(*layer->getOutput(0), 1000, fc_weight, fc_bias);
#4

It seems that you are using the dim instead of input count. So TRT will check an out of bound memory, it is invalid (not owned by network)
Please refer to below API link:
https://docs.nvidia.com/deeplearning/tensorrt/archives/tensorrt-710-ea/api/c_api/classnvinfer1_1_1_i_network_definition.html#a0c6e2a0b4e1c8a4df1722a24cc7c0473

Thanks

#5

Thank you.