Plugins with dynamic shapes

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

I’m creating two plugins CTCGreedyDecoder and SparseToDense (d_predictions) as shown in Figure.

CTCGreedyDecoder has fixed input shape and dynamic output shapes (?,2) and (?). Output shape is same as character length. We don’t know in advance, only know in runtime.

CTCGreedyDecoder output is fed to SparseToDense (d_predictions).
So SparseToDense (d_predictions) has dynamic input shape (?,2) and (?) and fixed output shape (1, 20).

Which plugin type (IPluginV2IOExt, or IPluginV2DynamicExt) should I derive from for each?

Screenshot from 2020-08-18 16-30-30
Screenshot from 2020-08-18 16-30-301600×900 161 KB

In this getOutputDimensions, for the dynamic size (?), what should I return for CTCGreedyDecoder as it has two outputs?

    Dims getOutputDimensions(int index, const Dims* inputs, int nbInputDims) override
    {
        assert(index == 0 && nbInputDims == 2 && inputs[0].nbDims == 3 && inputs[0].nbDims == 1);
        int height = (inputs[0].d[1] + mPoolingParams.pH * 2 - mPoolingParams.mR) / mPoolingParams.mU + 1;
        int width = (inputs[0].d[2] + mPoolingParams.pW * 2 - mPoolingParams.mS) / mPoolingParams.mV + 1;
        DimsHW outDims(height, width);
        return Dims3(inputs[0].d[0], outDims.h(), outDims.w());
    }
2

Hi @edit_or,
To support dynamic shapes, IPluginV2DynamicExt is preferred
Dynamic part will automatically adjust based on the input size.It can be dynamic input or output.

Refer to the below post addressing the query.

Thanks!