Inconsistent behaviour of plugin enqueue method when inputs has empty shapes(i.e. 0 on batch dimension)


i got a onnx model(maskrcnn) and converted it into trt engine by using trtexec, there are 2 custom ops in onnx model, so i implemented 2 corresponding plugins in tensorrt, one is NMSRotatedPlugin and another is ROIAlignRotatedPlugin. when testing inference on the converted trt engine, i found that when inputs has empty shapes, the inputs/outputs parameters of enqueue method of these 2 plugins are given differently by tensorrt.

As for ROIAlignRotatedPlugin, it has 2 inputs and 1 output, when the 1st dimension of the 2nd input is 0(which means no rois), the output should be empty(0 in the 1st dim). the inputs/outputs parameter are non-null pointer and the “checked the 0 condition and return” logic works fine, as show in the following picture

As for NMSRotatedPlugin, it has 3 inputs and 2 outputs, when the 1st dim of the first 2 inputs are 0(which means no bbox were found), the 1st output should also be empty(0 in the 1st dim). but this time the inputs/outputs parameter are given null pointers, and after “checked the 0 condition and return” logic, tensorrt throws out “illegal memory access” error, as show in followig pictures:

I think the illegal memory access is related to the null-pointers and i don’t know why the parameter passing behaviour are different(i.e. one gets non-null pointers and the other get null) on these 2 plugins.


TensorRT Version: 8.6.0
GPU Type: RTX 3090
Nvidia Driver Version: 470.129.06
CUDA Version: 11.4
CUDNN Version: 8.8.0
Operating System + Version: Ubuntu 20.04.4 LTS
Python Version (if applicable): 3.8
TensorFlow Version (if applicable): none
PyTorch Version (if applicable): 1.9
Baremetal or Container (if container which image + tag): container

Relevant Files

Please attach or include links to any models, data, files, or scripts necessary to reproduce your issue. (Github repo, Google Drive, Dropbox, etc.)

Steps To Reproduce

Please include:

  • Exact steps/commands to build your repro
  • Exact steps/commands to run your repro
  • Full traceback of errors encountered

Please refer to below links related custom plugin implementation and sample:

While IPluginV2 and IPluginV2Ext interfaces are still supported for backward compatibility with TensorRT 5.1 and 6.0.x respectively, however, we recommend that you write new plugins or refactor existing ones to target the IPluginV2DynamicExt or IPluginV2IOExt interfaces instead.


thanks for response, but i go through these 2 articles and don’t see anything related to my question, i derived my plugins from IPluginV2DynamicExt already.

Hi @kfiring ,
Are you still facing the issue?