Plugins with dynamic shapes

I’m creating two plugins CTCGreedyDecoder and SparseToDense (d_predictions) as shown in Figure.

CTCGreedyDecoder has fixed input shape and dynamic output shapes (?,2) and (?). Output shape is same as character length. We don’t know in advance, only know in runtime.

CTCGreedyDecoder output is fed to SparseToDense (d_predictions).
So SparseToDense (d_predictions) has dynamic input shape (?,2) and (?) and fixed output shape (1, 20).

Which plugin type (IPluginV2IOExt, or IPluginV2DynamicExt) should I derive from for each?

In this getOutputDimensions, for the dynamic size (?), what should I return for CTCGreedyDecoder as it has two outputs?

    Dims getOutputDimensions(int index, const Dims* inputs, int nbInputDims) override
    {
        assert(index == 0 && nbInputDims == 2 && inputs[0].nbDims == 3 && inputs[0].nbDims == 1);
        int height = (inputs[0].d[1] + mPoolingParams.pH * 2 - mPoolingParams.mR) / mPoolingParams.mU + 1;
        int width = (inputs[0].d[2] + mPoolingParams.pW * 2 - mPoolingParams.mS) / mPoolingParams.mV + 1;
        DimsHW outDims(height, width);
        return Dims3(inputs[0].d[0], outDims.h(), outDims.w());
    }

Hi @edit_or,
To support dynamic shapes, IPluginV2DynamicExt is preferred
Dynamic part will automatically adjust based on the input size.It can be dynamic input or output.

Refer to the below post addressing the query.


Thanks!