Dimension error in Uff parser

Robotics & Edge Computing Jetson & Embedded Systems Jetson TX2
1

I have successfully created an uff-file from a tensorflow model which I want to parse using the Tensor RT C++ API on a Jetson TX2. I am using the code from sampleUffMNIST.cpp where I changed the following:

Name of uff-file
name of input layer
dimensions of input layer
name of output layer

When I run the file I get the error:

TEST_sample_uff: Layers.h:460: virtual nvinfer1::Dims nvinfer1::ConcatenationLayer::getOutputDimensions(int, const std::vectornvinfer1::Dims&) const: Assertion `j==inputDims[0].nbDims-3 || inputDims[i].d[j] == inputDims[0].d[j]’ failed.
Aborted (core dumped)

What is the best way to debug this error?

One possible explanation could be that the tensorflow model is using the NHWC format. However, that does not completely makes sense because the layer that generates the error is not the first layer, so it seems like the layers before were parsed correctly.

Thanks in advance!

2

Hi,

Have your model included a reshape layer?

For TensorRT 3 RC, reshape only apply on constant weights.
Tensor reshapes will automatically drop when importing UFF model to a TensorRT engine, which may cause dimension out of expectation.

Thanks.

3

Thank you for your answer!

I use the reshape function, but I created the UFF file from a frozen model so the weights should be constant. I changed the format to NCHW in my tensorflow model, and now I get the following error message when i try to parse it in tensorflow:

ERROR: UFFParser: Parser error: model_1/norm_1/batchnorm_1/mul_1: Invalid scale mode, nbWeights: 416

Do you have any solutions for this?

4

Hello again.

I fixed the error with batch normalization by removing all batch normalization layers.

However, I still cannot create the engine. I use reshape to change tensors with dimensions (None, 64, 26, 26) to (None, 256, 13, 13), where the dimensions are in NCHW format. After that, there is a concatenate layer that merges the tensor (None, 256, 13, 13) with a tensor with dimension (None, 1024, 13, 13). The output of the concatenate layer has the dimensions (None, 1280, 13, 13).

To freeze the network and create the UFF file I use the following code:

checkpoint_prefix = os.path.join(‘.’, “saved_checkpoint”)

saver = saver_lib.Saver(write_version=saver_pb2.SaverDef.V2)
checkpoint_path = saver.save(sess, checkpoint_prefix, global_step=0,
latest_filename=‘checkpoint_state’)
graph_io.write_graph(sess.graph, ‘.’, ‘tmp.pb’)

freeze_graph.freeze_graph(‘./tmp.pb’, ‘’,
False, checkpoint_path, “output_node0”,
“save/restore_all”, “save/Const:0”,
‘frozen_graph.pb’, False, “”)

output_names=[‘reshape_1/Reshape’]
uff_model = uff.from_tensorflow_frozen_model(“frozen_graph.pb”, output_names, output_filename = “model.uff”)

Then I try to create the tensorRT engine with the following code:

parser = uffparser.create_uff_parser()
parser.register_input(“input_1”, (3, 416, 416), 0)
parser.register_output(“reshape_1/Reshape”)

engine = trt.utils.uff_file_to_trt_engine(G_LOGGER, uff_file, parser, 1, 1 << 20, trt.infer.DataType.FLOAT)

I get the error message:

[TensorRT] ERROR: Parameter check failed at: Network.cpp::addConcatenation::152, condition: first->getDimensions().d[j] == dims.d[j] && “All non-channel dimensions must match across tensors.”
Segmentation fault (core dumped)

I tried to comment out the concatenate layer and then the engine was created as it should. Could you please take a look at the code and see if I freeze the model the wrong way somehow?

Thanks!

5

The function I use to reshape the tensor looks like this:

def space_to_depth_x2(x):
#return tf.space_to_depth(x, block_size=2, data_format=‘NCHW’)
block_size = 2
dims = x.get_shape().as_list()
print(“dims”, dims)
height = int(dims[2]/block_size)
width = int(dims[3]/block_size)
depth = dims[1]
y = tf.reshape(x, [tf.shape(x)[0], depth, height, block_size, width, block_size])
y = tf.transpose(y, perm=[0, 1, 2, 4, 3, 5])
y = tf.reshape(y, [tf.shape(x)[0], block_sizeblock_sizedepth, height, width]) #-1])
return(y)

As you can see, the tensorflow functions I use are tf.reshape and tf.transpose. Are both of them supported by TensorRT?

6

Hi,

This is a Tensor reshape operation.
It will be automatically dropped when creating a TensorRT engine from UFF.

Thanks.

7

Hi AastaLLL,

I am having a similar problem… just to clarify this is currently a bug in TensorRT, right? There is nothing we can do at the moment, is it correct?

Thanks

8

Hi,

This is a not support feature of TensorRT.
Currently, workaround is to use Caffe framework to have Plugin API support.

Thanks.

9

I see.
I guess that another workaround would be to manually build the network with the TensorRT network definition API, right?
Thanks

10

Hi,

Do you want to use it on Jetson?
Python API doesn’t support Jetson platform currently.

Thanks.

11

Yes, on the jeston. I was thinking to use the C++ API.
Thanks

12

Hi,

Here is a sample to create TensorRT engine from pyTorch model.
Its extract the weight with python and directly use it to create TensorRT engine.

If you can extract TensorFlow weight with C++ correctly, it should be a possible workaround to go.
We don’t have too much experience on TensorFlow C++ interface and not very sure if any limitation exists.

Thanks and please let us know the result.

13

Hi Aasta,

I tried to build a network in python but I got an error as soon as I instantiated a Conv layer. You can see here: TensorRT Python API giving segmentation fault - TensorRT - NVIDIA Developer Forums

I haven’t tried yet with C++. Maybe you can point me to the documentation that describes the weights format that TensorRT accepts, so I can convert them from Tensorflow. I have found only this API Reference :: NVIDIA Deep Learning TensorRT Documentation

Thanks!

14

Hi,

Please find this sample to create an TensorRT engine from API:
/usr/src/tensorrt/samples/sampleMNISTAPI

Guess that it will be challenge to convert a TensorFlow model into TensorRT since TF is a operation-based framework.
Thanks.