Unable to convert maskrcnn h5 to uff file

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Hi,
I have retrained the maskrcnn default coco model on my custom dataset now to run it with deepstream i need to convert it to uff first but i am unable to do so
I followed this
https://github.com/NVIDIA/TensorRT/tree/master/samples/opensource/sampleUffMaskRCNN but once i run the final command i get this error

python3 mrcnn_to_trt_single.py -w /root/Mask_RCNN/mask_rcnn_cafe_0006.h5 -o /root/Mask_RCNN/mrcnn_nchw.uff -p ./config.py

Traceback (most recent call last):
File "mrcnn_to_trt_single.py", line 166, in 
main()
File "mrcnn_to_trt_single.py", line 115, in main
model = modellib.MaskRCNN(mode="inference", model_dir=LOG_DIR, config=config).keras_model
File "/usr/local/lib/python3.6/dist-packages/mrcnn/model.py", line 1832, in init
self.keras_model = self.build(mode=mode, config=config)
File "/usr/local/lib/python3.6/dist-packages/mrcnn/model.py", line 1896, in build
stage5=True, train_bn=config.TRAIN_BN)
File "/usr/local/lib/python3.6/dist-packages/mrcnn/model.py", line 180, in resnet_graph
x = KL.Conv2D(64, (7, 7), strides=(2, 2), name='conv1', use_bias=True)(x)
File "/usr/local/lib/python3.6/dist-packages/keras/engine/base_layer.py", line 463, in call
self.build(unpack_singleton(input_shapes))
File "/usr/local/lib/python3.6/dist-packages/keras/layers/convolutional.py", line 132, in build
raise ValueError('The channel dimension of the inputs '
ValueError: The channel dimension of the inputs should be defined. Found None.`
2

Hi,

You will need to replace the TensorRT OSS plugin and use a custom Deepstream pipeline here:
https://github.com/NVIDIA-AI-IOT/deepstream_4.x_apps

Here is detail steps to run MaskRCNN with deepstream.
You may already do part of the steps.

1. Setup

  • JetPack4.3
  • Deepstream4.0.2
  • TensorRT 6.0.1

2. cmake

$ wget https://github.com/Kitware/CMake/releases/download/v3.13.5/cmake-3.13.5.tar.gz
$ tar xvf cmake-3.13.5.tar.gz
$ cd cmake-3.13.5/ && ./configure && make && sudo make install

3. scipy

$ wget https://github.com/scipy/scipy/releases/download/v1.3.3/scipy-1.3.3.tar.gz
$ tar -xzvf scipy-1.3.3.tar.gz scipy-1.3.3
$ cd scipy-1.3.3/ && python3 setup.py install --user

4. TensorRT OSS plugin

$ git clone -b release/6.0 https://github.com/nvidia/TensorRT  && cd  TensorRT
$ git submodule update --init --recursive && export TRT_SOURCE=`pwd`
$ cd $TRT_SOURCE
$ mkdir -p build && cd build
$ cd ..
$ usr/local/bin/cmake .. -DTRT_BIN_DIR=`pwd`/out
$ make nvinfer_plugin -j3
$ sudo cp /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.6.0.1 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.6.0.1.bak
$ sudo cp out/libnvinfer_plugin.so.6.0.1 /usr/lib/aarch64-linux-gnu/libnvinfer_plugin.so.6.0.1

5. Generate mrcnn_nchw.uff

Install dependency

$ cd $TRT_SOURCE
$ sudo apt-get install python3-matplotlib
$ pip install -r $TRT_SOURCE/samples/opensource/sampleUffMaskRCNN/converted/requirements.txt

Apply this change to /usr/lib/python3.6/dist-packages/uff/converters/tensorflow/converter_functions.py

uff_graph.conv_transpose(
    inputs[0], inputs[2], inputs[1],
    strides, padding,
    dilation=None, number_groups=number_groups,
    left_format=lhs_fmt, right_format=rhs_fmt,
    name=name, fields=fields
    )

Get MaskRCNN source

$ git clone https://github.com/matterport/Mask_RCNN.git
$ export PYTHONPATH=$PYTHONPATH:$PWD/Mask_RCNN
$ cd Mask_RCNN
$ git checkout 3deaec5
$ git am $TRT_SOURCE/samples/opensource/sampleUffMaskRCNN/converted/0001-Update-the-Mask_RCNN-model-from-NHWC-to-NCHW.patch

Convert uff model

$ cd $TRT_SOURCE/samples/opensource/sampleUffMaskRCNN/converted/
$ wget https://github.com/matterport/Mask_RCNN/releases/download/v2.0/mask_rcnn_coco.h5
$ python mrcnn_to_trt_single.py -w mask_rcnn_coco.h5 -o mrcnn_nchw.uff -p ./config.py

6. RUN

$ git clone https://github.com/NVIDIA-AI-IOT/deepstream_4.x_apps.git
$ export DS_SRC_PATH=/opt/nvidia/deepstream/deepstream-4.0/
$ cd nvdsinfer_customparser_mrcnn_uff
$ make
$ cd ..
$ make
$ cp $TRT_SOURCE/samples/opensource/sampleUffMaskRCNN/converted/mrcnn_nchw.uff .
$ ./deepstream-custom pgie_mrcnn_uff_config.txt /opt/nvidia/deepstream/deepstream-4.0/samples/streams/sample_720p.h264

Thanks.

3

Hi,
Thanks for the detailed reply just to clerify i am using Tesla V100’s.

4

Hi,

For desktop environment, please replace step 1 with the instruction shared in this document:
https://docs.nvidia.com/metropolis/deepstream/dev-guide/index.html

In short, please get TensorRT 6.0.1 and Deepstream 4.0.2 installed.
Thanks.