Uff-Parser switches dimensions unintended

AI & Data Science Deep Learning (Training & Inference) TensorRT
,
1

Description

I think the Uff parser might have a bug and sometimes switche dimensions back to [H,W,C].

Here’s my problem:
I use build_enginy.py to convert my ssd_mobilenet_v3 to uff. Hereby all unsupported operations are replaced by other operations or plugins.

When parsing the uff file I encounter the following error:
[TensorRT] ERROR: FeatureExtractor/MobilenetV3/expanded_conv/squeeze_excite/mul: elementwise inputs must have same dimensions or follow broadcast rules (input dimensions were [38,38,8] and [8,1,1]).

But this is not simply a dimension mismatch in my model:

The output of the layer FeatureExtractor/MobilenetV3/expanded_conv/depthwise/Relu is of dimension [8,38,38] (check logfile line 114).

But the next layer (the mul-layer from above) that has the Relu layer as Input gets the dimension [38,38,8] passed (Check logfile line 163+164). Therefore it fails with a dimension mismatch.

I think it is an error of the uff parser to switch the dimensions in that way, since it ususally works.
Additionally, I added a reshape node to bring the dimensions back in shape. This fixes the error but the network is not usable anymore (bad predictions) because reshaping is of course not the inverse of the transposition that has taken place.

Would be great to solve that!

BTW: I know that Uff will be deprecated after the next major release and we should switch to onnx.
But uff is great and easy to handle, for example to replace NMS with the TRT-PLugin (a lot harder in ONNX). So since the Uff parser is not deprecated yet, it would be great to find a solution

Environment

TensorRT Version: 7.1
GPU Type: Tegra
Plattform: Jetson AGX Xavier
CUDA Version: 10.2.89
CUDNN Version: 8.0.0.145
Operating System + Version: Ubuntu 18.04 for arm
Python Version (if applicable): 3.6.9
TensorFlow Version (if applicable): 1.15.2
Baremetal or Container (if container which image + tag): Baremetal

Relevant Files

log_reverse_order.txt (25.0 KB)
build_engine.py.txt (15.6 KB)

Steps To Reproduce

Bug/Error can also be reproduced with the original ssd_mobilenet_v3_coco models:

  1. Download model pb (e.g. from GitHub)
  2. In build_engine.py specify the path to the frozen_graph.pb in the MODEL_SPECS for ssd_mobilenet_v3_small_coco
  3. run python3 build_enginy.py -v ssd_mobilenet_v3_small_coco
1 Like
2

Hi @joel.oswald,

The error you are getting is addressed in the below link

However, as you mentioned, UFF conversion has been deprecated from TRT>=7.
The recommended way is to use
TF << ONNX << TRT
or
use TF-TRT

Please check the below links for reference.
https://github.com/NVIDIA/TensorRT/tree/master/samples/opensource/sampleOnnxMNIST

Thanks!

3

Hi @AakankshaS,
thanks for your reply!

I am aware that the caused error is due to a dimension missmatch!

But the problem I see is the cause of the dimension missmatch:
I pointed out the following: The output of the layer “…/…/ReLu” is of shape [8,38,38].
But when this output is passed to “…/…/mul” the dimensions are [38,38,8]. They have been reverted for some reason. This reverting does not make sense, because the output is only used for the mul-operation.
The reverting is causing the dimension missmatch. If the dimension would stay at [8,38,38] that would perfectly match with [8,1,1] for the mul-operation.

And as I read it, Uff-Parser will still be tested for release 8.0 , so I hope we can find a solution here?

4

Hi @AakankshaS, any update on the topic?

5

Hi @joel.oswald,
Because UFF has been deprecated from TRT 7 onwards, as a result, we plan to remove the support in the subsequent major release.
I could see your main concern is with plugins, which has been made a lot easier in TRT7.1
Hence would recommend you try the same using onnx-packnet.

Thanks!