Input dimensions are different for plugins

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

I have test.onnx model.

Tested with two plugin types. IPluginV2IOExt and IPluginV2DynamicExt.

The same approach was used to parse onnx model for both.

    auto parser = samplesCommon::infer_object(nvonnxparser::createParser(*network, sample::gLogger.getTRTLogger()));
    bool parsingSuccess = parser->parseFromFile(locateFile(mParams.onnxFileName, mParams.dataDirs).c_str(), static_cast<int>(sample::gLogger.getReportableSeverity()));
    if (!parsingSuccess)
    {
        sample::gLogError << "Failed to parse model." << std::endl;
        return false;
    }

In IPluginV2IOExt plugin, inside
Dims getOutputDimensions(int index, const Dims* inputs, int nbInputDims) override
checked the dimension of input[0]. It gave

(gdb) p inputs[0]
$1 = {static MAX_DIMS = 8, nbDims = 2, d = {10, 48, 0, 0, 0, 0, 0, 0}, type = {
    nvinfer1::DimensionType::kSPATIAL, nvinfer1::DimensionType::kSPATIAL, 
    nvinfer1::DimensionType::kSPATIAL, nvinfer1::DimensionType::kSPATIAL, 
    nvinfer1::DimensionType::kSPATIAL, nvinfer1::DimensionType::kSPATIAL, 
    nvinfer1::DimensionType::kSPATIAL, nvinfer1::DimensionType::kSPATIAL}}

It is only 2D. Actual dimension is 3D (88,10,48).

   Then for IPluginV2DynamicExt, inside `DimsExprs getOutputDimensions(int outputIndex, const DimsExprs* inputs, int nbInputs, IExprBuilder& exprBuilder) override`, printed out inputs[0]

(gdb) p inputs[0]
$1 = {nbDims = 3, d = {88, 10, 48, 0, 0, 0, 0, 0}}

It gave the correct dimensions.

Why I don’t have correct input shape for plugin when used IPluginV2IOExt even though same onnx model was parsed?

2

Hi @edit_or,
IPluginV2IOExt is an implicit batch plugin, so the batch size is specified at runtime(enqueue), need to use IPluginV2DynamicExtplugin and getOutputDimensions to get batch size .

Thanks!

3

Thank you.Let me check.