Error after adding pre-processing layers (Order size is not matching the number dimensions of TensorRT)

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

I have a Tensorflow model (.pb file) that I want to convert into a TensorRT model (.plan file). I am able to successfully convert a base version of the network from Tensorflow to TensorRT. However, after adding some pre-processing layers into the network, conversion to a .plan file now fails.

I simply added 3 layers (ExpandDims, Transpose, and Div) to the beginning of my frozen Tensorflow model in a manner similar to this: python - Add low layers in a Tensorflow model - Stack Overflow

Here is what the input layers to the base model look like (also attached): Imgur: The magic of the Internet

Here is what the input layers to my modified model look like (also attached): Imgur: The magic of the Internet

Here is the error that it produces when I try to convert it to a .plan file (after successfully re-freezing the Tensorflow model and converting it to UFF) (also attached): Imgur: The magic of the Internet

Since the error says “Order size is not matching the number dimensions of TensorRT” for the Conv2D layer immediately after my pre-processing layers, it seems that my dimensions are not matching up. But I was able to successfully add the pre-processing layers in Tensorflow and re-freeze the model without any errors, and then my conversion from .pb to .uff was done successfully. Also, it seems like the dimensions should match up:

Base model has an input of shape: 1x3x608x960

Modified model has an input of shape: 608x960x3 → ExpandDims (1x608x960x3) → Transpose (1x3x608x960) → Div (1x3x608x960)

So both models should be feeding a 1x3x608x960 tensor into the Conv2D layer.

Does anyone have any insight into why this error is happening or what other approaches I could use to successfully convert my Tensorflow model into TensorRT? I am new to TensorRT conversion so any help would be greatly appreciated!

Thank you!

base_model.png
base_model.png540×555 26.1 KB

modified_model.png
modified_model.png539×906 31.7 KB

error.png
error.png1311×360 73.2 KB

2

Bumping this thread since it wasn’t visible until now.

3

hi were you able to solve this problem??

4

I was not able to successfully generate a .PLAN file with my modified model of shape 608x960x3 (HWC). I had to keep the input shape as 1x3x608x960 (NCHW) in order for it to work. So yes, I was able to resolve this issue by using NCHW format.

5

yes, As per my understanding NCHW is the standard expected input for TensorRT. I am facing a wierd issue where i do use NCHW but still problem exists.

it would be of great help , if u could just have a glance on this thread.

https://devtalk.nvidia.com/default/topic/1061861/tensorrt/order-size-is-not-matching-the-number-dimensions-of-tensorrt/post/5377437/#5377437

I have created a simple standalone with 2 layers representing this issue.

Thanks in advance!! Thanks for replying!!

6

I am not sure why you are getting that error. I can tell you that I never had any luck using 3 dimensions (CHW, your option B). I always had to represent everything in 4 dimensions (NCHW, your option A).