some plugin needs max batch size to allocate cpu buffers in initialize, but it is missing in configurePlugin of IPluginV2IOExt.
Max Batch Size is a property of implicit networks, IPluginV2IOExt and IPluginV2DynamicExt are for full dims or dynamic shape networks.
If you want the same functionality as implicit networks, create a network with -1 in the first dimension and then create a profile with kMAX as the max batch size in that dimension.
I just need fixed batch size, but I can’t get batch dim in configurePlugin of IPluginV2IOExt, the dims of the PluginTensorDesc don’t contain batch dim.
Is there a way to get explicit batch size of a tensor before enqueue when using IPluginV2IOExt?
You can save explicit batch size (or even the dims of all I/O tensors) when you implement configurePlugin to members, and serialize it via void serialize(void* buffer) const.
When you do inference at runtime stage, that serialized batch will be deserialized via IPluginV2* IPluginCreator::deserializePlugin(const char* name, …).
The problem is that arguments of the configurePlugin don’t contain batch size even if I use explicit batch axis network:
virtual void nvinfer1::IPluginV2IOExt::configurePlugin ( const PluginTensorDesc * in,
const PluginTensorDesc * out,
My workaround is use second axis as batch axis.