Order size is not matching the number dimensions of TensorRT

I have a convolution layer which was preceded by a padding layer(custom plugin).

input placeholder : 1×3x361×641
pad (custom layer) : 1×3x367×647
conv out expected : 1×32x181×321

Note : pad custom plugin done as there is issue in tensorrt pad as shared in previous topic

I have verified the custom layer output and observed works as expected.

i have tried using

Dims getOutputDimensions(int index, const Dims* inputs, int nbInputDims) override
	{

option a) tried
		return DimsNCHW(1,inputs[0].d[0], inputs[0].d[1] + 2 * hPad, inputs[0].d[2] + 2 * wPad);

// wPad and hPad is 3
and using of 
option b) tried
		return DimsCHW(inputs[0].d[0], inputs[0].d[1] + 2 * hPad, inputs[0].d[2] + 2 * wPad);

	}

error obtained option a)

Parser error: SharedFeatureExtractor/resnet_v1_50/conv1/Conv2D: Order size is not matching the number dimensions of TensorRT

error obtianed for option b)
ERROR: SharedFeatureExtractor/resnet_v1_50/conv1/Conv2D: image size is smaller than filter size
ERROR: UFFParser: Parser error: MarkOutput_0: Order size is not matching the number dimensions of TensorRT

however the conv filter is 7x7x3x32 in tensorflow

any idea why this error is obtained.

@ Moderator…

uploading the entire code standalone with following folder

  1. tensorflowcode – the refference tensorflow code
  2. uff conversion – conversion file used
  3. TensoRT - TensorRT code (built from sample uff ssd). uff file also provided.

Expected logs:
Begin parsing model…
Inside get getOutputDimensions
3 361 641
ERROR: UFFParser: Parser error: Conv2D: Order size is not matching the number dimensions of TensorRT
ERROR: MYEXP: Fail to parse
BUG_CONV_PAD.zip (64.8 KB)

Hello,

I have solved this. I want to send my code to you, but I don’t know how to attach it. So sorry.

The solutions are,

1. You should specify the output name as a valid name which is in the graph, not MarkOutput0.

2. If you want to use batch dimension, you should add a 'transpose' layer after relu, which transpose the data format from 'NHWC' to 'NCHW'. And register the transpose layer as the output.

BUG_CONV_PAD_Modified.7z (799 KB)

@lingchao.zhu, in the above comment that u have commented , check on the top right, u should see attach file as second option.

You should specify the output name as a valid name which is in the graph, not MarkOutput0.
since i used the below command for conversion , Conv2D would be registered as MarkOutput0( as from sample uff ssd example)
convert-to-uff -o myexp.uff --input-file myexp.pb -p config_icnet_pad.py -O Conv2D -t
I however tried using parser->registerOutput(“Conv2D”); there is no change in logs however.

If you want to use batch dimension, you should add a ‘transpose’ layer after relu, which transpose the data format from ‘NHWC’ to ‘NCHW’. And register the transpose layer as the output.

network works in NCHW so i think should be fine for now

its intresting to hear that you have solved the problem, thanks a lot :-) waiting for ur reply

Hello,

I have attached it.

Check it please.

hi lingchao.zhu , thanks a lot for uploading the code.

i took the code you shared and came up with following observations:

transpose as output


tensorflow - NHWC
input placeholder mentioned = data_format=‘NCHW’
Code 1 - parser->registerInput(“Input”, DimsNCHW(1,3, 361, 641), UffInputOrder::kNCHW); - RUNS
Code 2 -parser->registerInput(“Input”, DimsCHW(3, 361, 641), UffInputOrder::kNCHW); -RUNS

Relu as output


tensorflow - NHWC
input placeholder mentioned = data_format=‘NCHW’

Code 1 -
parser->registerInput(“Input”, DimsNCHW(1,3, 361, 641), UffInputOrder::kNCHW); - FAILS ( MarkOutput_0: Order size is not matching the number dimensions of TensorRT)

Code 2 -
parser->registerInput(“Input”, DimsCHW(3, 361, 641), UffInputOrder::kNCHW); -RUNS
parser->registerOutput(“Relu”);


ACTUAL ISSUE

Note:
/////////////////////////////////////////////
To be frank, i am working one level deeper, input -> resize (custom layer) -> pad layer -> conv2d -> …
However i am trying to reproduce the error with least number of layers !!!
//////////////////////////////////////////////


How to produce error in your code

now in the same code you shared please try making pad layer as a custom layer (custom code interface already present in my code…)

WHEN PAD LAYER IS FORCED AS A CUSTOM LAYER , YOU WILL SEE the bug

Conv2D: Order size is not matching the number dimensions of TensorRT
or
Conv2D: image size is smaller than filter size

i still assume this is some corner case , and would push the knowledge boundary. if possible please try.

Hi,

I rebuild the model and make the pad as a customer plugin.

The error occurred,

Conv2D: Order size is not matching the number dimensions of TensorRT
or
Conv2D: image size is smaller than filter size

  1. Error1,
    I don’t know how to solve the ‘Order size is not matching the number dimensions of TensorRT’ error.
    Maybe it should make the conv layer to be ‘NCHW’ format for tensorflow model. But I don’t have enough time to try it.

  2. Error2,
    I have also some projects that I should make a Resizebilinear plugin. My experience is that the convolution which format is ‘NHWC’ in tensorflow model, only supports ‘NHWC’ in TensorRT. So you should return a ‘NHWC’ size in the previous layer which linked with convolution layer.
    In the function, Dims getOutputDimensions(int index, const Dims* inputs, int nbInputDims) override, if you return DimsCHW(inputs[0].d[1] + 6, inputs[0].d[2] + 6, inputs[0].d[0]). it should be OK.
    BUG_CONV_PAD.7z (804 KB)

@lingchao.zhu Thanks for the wholehearted help.

Solution


yes it should be in HWC format

Option 1:
parser->registerInput(“Input”, DimsNCHW(1,3, 361, 641), UffInputOrder::kNCHW);
getOutputDimensions for Custom Pad -
return DimsCHW( inputs[0].d[2] + 6, inputs[0].d[3] + 6, inputs[0].d[1]);
Option 2:
parser->registerInput(“Input”, DimsCHW(3, 361, 641), UffInputOrder::kNCHW);
getOutputDimensions for Custom Pad -
return DimsCHW( inputs[0].d[1] + 6, inputs[0].d[2] + 6, inputs[0].d[0]);

personally i am not pretty convinced with all of the reasons , mainly because for ssd-resnet ( tensorflow standard object detction model ) had NHWC in tensorflow, but all the custom layers like Nearest neighbour was coded as if it was NCHW in cuda and also mentioned as NCHW in getoutput dimensions. still figuring out a damm good reason for this behaviour here!!

Hello,

I have tested,

placeholder->pad->transpose(to NCHW)->conv2d(data_format='NCHW')

and also used ‘Custom_Pad’ to replace pad. The error, Conv2D: Order size is not matching the number dimensions of TensorRT, still occurred when the input is with batch. And It can be solved by return a ‘HWC’ dimension. It’s strange.

I still want to solve this, because large batches processing will give an obvious improvement of speed, especially for fp16 and int8.

ya, The following flow works:
placeholder(CHW)->pad_custom(CHW)->transpose(to HWC)->conv2d
** Transpose used has following permutation = [0,2,3,1] values
this is with parser->registerInput(“Input”, DimsCHW(3, 361, 641), UffInputOrder::kNCHW);

input with batch size
I haven’t worked on multiple batch size till now. however trying to return a combination of
return DimsNCHW(inputs[0].d[0], inputs[0].d[1], inputs[0].d[2]+6 , inputs[0].d[3]+6);
from pad layer ( as transpose takes care of conversion later) and
parser->registerInput(“Input”, DimsNCHW(8,3, 361, 641), UffInputOrder::kNCHW);
went in vain.

Question on batch size:
as per sampleuffSSD , isnt multiple batch size handled without registering,

parser->registerInput("Input", DimsNCHW(8,3, 361, 641), UffInputOrder::kNCHW);

but just using

parser->registerInput("Input", DimsCHW(3, 361, 641), UffInputOrder::kNCHW);