How to use `PluginFieldCollection` during writing custom TenosrRT plugins?

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

I have met the PluginFieldCollection in the TensorRT Developer Guide, samplePlugin and the sampleUffSSD samples. I noticed that in the FlattenConcat plugins included in sampleUffSSD sample, this class defined a static member variable whose type is PluginFieldCollection and this member is used as the construct function parameters.

But I still cannot figure out the usage of this member, especially, when and how this static member field is used. Or, is it used during serialize Engine or deserialization when calling deserialize() or serialize(), or when calling initialize(), getOutputDimensions() … ?

This object has confused me very long time, thanks for any explains.

2

Hi,

You don’t need to use PluginFieldCollection if you know all parameters required to execute a plugin.
It is actually required by ONNX/Caffe/UFF parsers. For example:
For Caffe/UffParser parser, after calling CaffeParser/UffParser::parse(), all fields that are parsed will be inserted into PluginFieldCollection.
For ONNX parser, it is loaded by OP TRT_PluginV2, or any other OPs that are not supported will attempt to import as plugins.
PluginFieldCollection is the only data structure to pass parameters from Models → TensorRT engine → TensorRT runtime.

If you are going to use PluginFieldCollection or not, the corresponding serialization/deserialialization must be implemented as shown in TRT samples.
TensorRT will call APIs accordingly, e.g., Engine::deserialize() calls IPluginCreator::createPlugin() to create plugin from PluginFieldCollection buffers,
ONNX OP TRT_PluginV2 calls IPluginCreator::deserializePlugin to create plugin from ONNX attributes “data”,
PluginFieldCollection can be serialized by Engine::serialize() (implemented by IPluginV2xxx::serialize()).

Thanks

3

Thanks for such detailed explanation. I looked into the samples and tensorrt document again, and concluded following several points:

  • PluginFieldCollection will be used in IPluginCreator::createPlugin(). Defining the INetworkDefinition object from TensorRT C++ API or parsing the ONNX/Uff/Caffe files will call IPluginCreator::createPlugin() function
  • During serialize a Engine to a PLAN file, the PluginFieldCollections elements should be written to the output file
  • During deserialize a PLAN file to load Engine object, the IPlugin constructor function that accepts a void * and int length as parameters will be used to load the plugin, so PluginFieldCollection can be not used
  • Finally, PluginFieldCollection member of IPluginCreator class is served as the configure settings of corresponding layer, but not the I/O datas (from bottom layers), just like the stride, kenrel_size attributes of a Convolution layer

Are these conclusions right ?

4

Item 3:
Users can implement their own constructor to accepts either void* and int length or PluginFieldCollection – it depends on their applications

Item 4:
The usage is not limited by TensorRT, users can actually put any parameters required into PluginFieldCollection, because the serialization and deserialization functions are implemented by users, while TensorRT only calls them.

Thanks

5

Ok, thanks