Cannot get custom .uff file to work

Autonomous Machines Jetson & Embedded Systems Jetson TX2
1

I am trying to get my custom .uff file to work on TX2. However I get the following error:

[loadModelAndCreateEngine] Parse UFF File: uff_ready/output_model.uff
ERROR: UFFParser: Validator error: net_vlad_layer_1/l2_normalize/Square: Invalid enum field 'func': bad type or not present
ERROR: sample_uff_mnist: Fail to parse
ERROR: sample_uff_mnist: Model load failed

I am using TensorRT C++ API. I had a keras model (which inturn had a custom layer defined).

I converted the keras model (from h5) to tensorflow’s .pb. The used the convert-to-uff utility from nvidia to get the uff file.

All the data files are: tx2_whole_image_desc_server/standalone/uff_ready at master · mpkuse/tx2_whole_image_desc_server · GitHub

On looking at the pbtxt file (which is the same as the .pb file but in readable text),
it seems like I had several nodes with same prefix name as in the output.

Please advice how can this is resolved.

2

Hi,

Which TensorRT version do you use?
There is an uff-parser issue will hit this error but it is already fixed in the v5.0.

If you are not using JetPack4.2, could you give it a try?

Thanks.

3

I am using tensorrt5.1.5.0 on my x86 that this uff file was produced. I am not 100% sure which TensorRT version on my TX2. Although I can see the file
/usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.4.0.4 so I assume i have tensorrt v4 on tx2.

I am actually using DJI Manifold2-G device.

Anyways, let me try Jetpack4.2 and getback.

4

Hi,
I tried setting up jetpack4.2 on my TX2 using the nvidia-sdkmanager. Installation went well.
On the new ubuntu (on tx2) I could compile the samples and run them fine.

I modified my standalone code to

    parser->registerInput("Input_0", DimsCHW(1, 28, 28), UffInputOrder::kNCHW);

from older version

    parser->registerInput("Input_0", DimsCHW(1, 28, 28));

in the standalone folder, I compiled as:

$ g++ -std=c++11 image_desc_try.cpp  common.cpp -I /usr/local/cuda/include -L /usr/local/cuda/lib64 -L /usr/lib/aarch64-linux-gnu/libnvparsers.so  -lcudart -lnvinfer -lnvparsers

compilation worked fine. However, the UFFParser still failed. How can this be fixed?

uff_ready/output_model.uff
[loadModelAndCreateEngine] Parse UFF File: uff_ready/output_model.uff
ERROR: UFFParser: Validator error: conv_pw_7_bn/cond/Switch_1: Unsupported operation _Switch
ERROR: sample_uff_mnist: Fail to parse
ERROR: sample_uff_mnist: Model load failed
5

I created a new uff, output_nvinfer.uff (corresponding pbtxt is output_nvinfer.pbtxt, generated with -t option to convert-to-uff utility). tx2_whole_image_desc_server/standalone/uff_ready at master · mpkuse/tx2_whole_image_desc_server · GitHub

I used the input and output tags as generated from the convert-to-uff utility output

  parser->registerInput("input_1", DimsCHW(3, 240, 320),  UffInputOrder::kNCHW);
  parser->registerOutput("net_vlad_layer_1/l2_normalize_1");

With this uff I get the following error:

[loadModelAndCreateEngine] Parse UFF File: uff_ready/output_nvinfer.uff
ERROR: UFFParser: Validator error: net_vlad_layer_1/ExpandDims_1: Unsupported operation ExpandDims
ERROR: sample_uff_mnist: Fail to parse
ERROR: sample_uff_mnist: Model load failed

So, my guess is operation ExpandDims is not supported? What is a possible way around this?
In my keras code, in custom layer, there are things like a = K.expand_dims( a, -2 )
is this the source of the problem?

6

Hi,

The error is caused by the nonsupported layer.
You can find our supported list here:
[url]https://docs.nvidia.com/deeplearning/sdk/tensorrt-support-matrix/index.html#layers-matrix[/url]

What is your model?
Here is a sample for ssd_mobilenet for your reference:
[url]https://github.com/AastaNV/TRT_object_detection[/url]

Thanks.

7

The base network is mobilenet, however, I do have a custom layer which has expand_dims. I looked at the list of supported and under “These are the operations that are supported in a TensorFlow framework:” on the link, it mentions expand_dims as a supported operation.

Is my understanding correct?

I looked at your mobilenet code, and it seems like you load it from an existing PB.
Can you tell me how you produced the .pb from keras model. I am trying to do this, but

convert-to-uff throws 2 issues:
a) I have no idea how to fix this :(

Warning: No conversion function registered for layer: Merge yet.
Converting conv_pw_5_bn/cond/Merge as custom op: Merge

b) Based on Wrong order of dependencies after running freeze_graph and/or optimize_for_inference · Issue #8404 · tensorflow/tensorflow · GitHub,
I could fix this by converting Switch to Identity. Identity is on the supported list of TensorRT.

Warning: No conversion function registered for layer: Switch yet.
Converting conv_pw_5_bn/cond/FusedBatchNorm/Switch_2 as custom op: Switch

I created my mobilenet using keras and I need an intermediate layer as output.

input_img = keras.layers.Input( shape=(240, 320, 3 ) )
base_model = keras.applications.mobilenet.MobileNet( weights=None, include_top=False, input_tensor=input_img )
base_model_out = base_model.get_layer( layer_name ).output # can also try conv_pw_7_relu etc.
model = keras.models.Model( inputs=input_img, outputs=base_model_out )

I firstly try to get remove the custom layer and just get my mobilenet to work.
No success yet.

FYI, if I use the VGG16 (from keras.applications) it seem to work with UFFParser on the TX2. Thats a good sign :)

Once I could run my mobilenet on TX2 I will deal with my custom layer. Sounds fair?

8

Here my update,

I figured one needs to do K.set_learning_phase(0) even before keras graph construction and not when you save the protobuffer. With this got everything to work.
The Merge warnings went away, Switch warnings went away.

I was able to load the uff file with tensorrt on my x86. Also with C++ API UFFParse was successfully able to load the uff for mobilenet and vgg.

^^^^ Here is the entire script that does the trick.
Read every line of code before using the script though.
Thanks AaastaLLL for pointing out the correct resources.

Now the only problem remaining is that, my custom layer has tensorflow’s expand_dims. I tried with reshape but that doesn’t work yet. The problem is shape has None (ie. shape determined at run time) which is causing UFFParser to throw dimension mismatch error. Is expand_dims support coming soon? How can I fix this?

9

Hi,

expand_dims is in our future plan but not ready yet.
Suppose you can get it work by replacing with a reshape layer.

Does your tensor have dynamic dimension?
Please noticed that dynamic tensor allocation is not supported by TensorRT yet.
TensorRT requires the exact tensor size to allocate GPU memory beforehand.

If you are not using a dynamic dimension, would you mind to update the dimension directly as a workaround?

Thanks.