IPluginV2 deserialization, how?

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

first of all, i use caffe, :)

about sample fasterRCNN

in sample fasterRCNN, we call createRPNROIPlugin in IPluginFactoryV2.createPlugin, so the plugin be registed in runtime

so that, when we deserialize, we could get the right thing even though we pass nullptr into deserializeCudaEngine

FRCNNPluginFactory pluginFactory;
ICudaEngine* engine = runtime->deserializeCudaEngine(trtModelStream->data(), trtModelStream->size(), nullptr);

hope that i understand right about sample fasterRCNN registery process

about deserializeCudaEngine

but, what if i have a FooPlugin, and clearly there is not such CreateFooPlugin function in NvInferPlugin.h

so, i write a FooPluginV2, and write a FooPluginFactoryV2, and serialize engine and write IHostMemory->data() into a file

class FooPluginV2 : public nvinfer1::IPluginV2{
}

class FooPluginFactoryV2 : public nvcaffeparser1::IPluginFactoryV2{
}

FooPluginFactoryV2 foo_factoryv2;

parser->setPluginFactoryV2(foo_factoryv2);

// we build done

trtModelStream = engine->serialize();
ifile.write((char*)trtModelStream->data(), trtModelStream->size());

// serialize done

still, i can not deserialize from file

runtime->deserializeCudaEngine accept IPluginFactory* but not IPluginV2Factory*

// not IPluginFactoryV2!!!!!!!
nvinfer1::ICudaEngine* deserializeCudaEngine(const void* blob, std::size_t size, IPluginFactory* pluginFactory)

so i can not pass my FooIPluginFactoryV2 into runtime->deserializeCudaEngine

now, i am stuck

IPluginCreator

well, a lot of questions

from sdk docs, what i should do:

class FooPluginCreator : public IPluginCreator
{
	...implement all creator methods here
};
REGISTER_TENSORRT_PLUGIN(FooPluginCreator);

what if i have some layers, different names, but same layer type

like 5 layers, name foo1, foo2, foo3, foo4, foo5, and they are same layer type FoolType

so i must create one IPluginCreator for each one layer?

class IPluginCreator
{

    // return name, not pass a name!
    virtual const char* getPluginName() const = 0;

// but why pass a name?
    virtual IPluginV2* deserializePlugin(const char* name, const void* serialData, size_t serialLength) = 0;

}
1 Like
2

for your last question, you may do as below. Here every layer that has foo is mapped to the same plugin. You may add more control on the names if needed

bool PluginFactory::isPluginV2(const char *name)
{
    std::string strName{name};
    std::transform(strName.begin(), strName.end(), strName.begin(), ::tolower);
    return (strName.find("foo") != std::string::npos);
}

IPluginV2 *PluginFactory::createPlugin(const char *layerName, const Weights* weights, int nbWeights, const char* libNamespace)
{
    assert(isPluginV2(layerName));

    std::string strName{layerName};
    std::transform(strName.begin(), strName.end(), strName.begin(), ::tolower);

    if (strName.find("foo") != std::string::npos)
    {
        _nvPlugins[layerName] = (IPluginV2 *)(new FOOPlugin(weights, nbWeights));
        return _nvPlugins.at(layerName);
    }
    else
    {
        std::cout << "warning : " << layerName << std::endl;
        assert(0);
        return nullptr;
    }
}
3

Even I am having the same exact doubt as the first qstn.
I am upgrading from TrT 5 → TrT 7. Due to the deprecated API’s I am forced to change the interfacing via IPluginV2. These are some places were I’m left clueless.

  1. How is de-serialization handled for IPluginV2 layers ?
    As discussed above,
    nvinfer1::ICudaEngine* deserializeCudaEngine(const void* blob, std::size_t size, IPluginFactory* pluginFactory)
    I cannot pass an IPluginFactoryV2, into the deserialize API ? Is there a work around ?

  2. I was using

    createYOLORegionPlugin , createPReLUPlugin
    etc for YOLO support, With the TensorRT 7, these layers are deprecated(?) and

: createRegionPlugin,createLReLUPlugin
are shown as alternatives. But I couldn’t find the
createLReLUPlugin (serialData, serialLength)
constructor, as it used to be in TrT 5.

  1. createYOLORegionPlugin, (YOLO)
    createPReLUPlugin (YOLO),
    createConcatPlugin (SSD),
    createSSDPermutePlugin (SSD)
    are the deprecated API’s in TensorRT 7.
    Are there any alternatives for these, or is the implementations of these layers accessible ?

Thanks
@nvidiadev2