IndexError: list index out of range(Object Detection And Instance Segmentations With A TensorFlow Ma...

Hello,everyone.I am following the tutorial:
https://github.com/NVIDIA/TensorRT/tree/release/6.0/samples/opensource/sampleUffMaskRCNN
to perform inference on the Mask R-CNN network in TensorRT.
I followed the steps,when I implemented the step 7,namely “Convert the h5 model to the UFF model” I encountered a problem.Here are the information:

UFF Version 0.6.3
=== Automatically deduced input nodes ===
[name: "input_image"
op: "Placeholder"
attr {
  key: "dtype"
  value {
    type: DT_FLOAT
  }
}
attr {
  key: "shape"
  value {
    shape {
      dim {
        size: -1
      }
      dim {
        size: 3
      }
      dim {
        size: 1024
      }
      dim {
        size: 1024
      }
    }
  }
}
]
=========================================

Using output node mrcnn_detection
Using output node mrcnn_mask/Sigmoid
Converting to UFF graph
Traceback (most recent call last):
  File "mrcnn_to_trt_single.py", line 171, in <module>
    main()
  File "mrcnn_to_trt_single.py", line 130, in main
    text=True, list_nodes=list_nodes)
  File "mrcnn_to_trt_single.py", line 164, in convert_model
    debug_mode = False
  File "/home/zhaoyu/anaconda3/envs/test/lib/python3.7/site-packages/uff/converters/tensorflow/conversion_helpers.py", line 233, in from_tensorflow_frozen_model
    return from_tensorflow(graphdef, output_nodes, preprocessor, **kwargs)
  File "/home/zhaoyu/anaconda3/envs/test/lib/python3.7/site-packages/uff/converters/tensorflow/conversion_helpers.py", line 181, in from_tensorflow
    debug_mode=debug_mode)
  File "/home/zhaoyu/anaconda3/envs/test/lib/python3.7/site-packages/uff/converters/tensorflow/converter.py", line 94, in convert_tf2uff_graph
    uff_graph, input_replacements, debug_mode=debug_mode)
  File "/home/zhaoyu/anaconda3/envs/test/lib/python3.7/site-packages/uff/converters/tensorflow/converter.py", line 79, in convert_tf2uff_node
    op, name, tf_node, inputs, uff_graph, tf_nodes=tf_nodes, debug_mode=debug_mode)
  File "/home/zhaoyu/anaconda3/envs/test/lib/python3.7/site-packages/uff/converters/tensorflow/converter.py", line 47, in convert_layer
    return cls.registry_[op](name, tf_node, inputs, uff_graph, **kwargs)
  File "/home/zhaoyu/anaconda3/envs/test/lib/python3.7/site-packages/uff/converters/tensorflow/converter_functions.py", line 40, in convert_add
    uff_graph.binary(inputs[0], inputs[1], 'add', name)
IndexError: list index out of range

I found it has created the temp.pb file, but it failed to convert .pb to .uff.
I don’t know how to deal with the problem,if you have the same problem or know how to solve it,I will be appreciated to your resopnse.

Hi,

Do you use the the model downloaded from the step.6?

wget https://github.com/matterport/Mask_RCNN/releases/download/v2.0/mask_rcnn_coco.h5

Thanks.

No,I have downloaded the mask_rcnn_coco.h5 file before,so I just used the previous one.
You meant the mask_rcnn_coco.h5 resulted the errors?

I have tried to redownload the mask_rcnn_coco.h5 file, unfortunately it still has the same error.Any other suggestions?

Hi,

Thanks for your feedback.

We are trying to reproduce this issue on our side.
Will update more information with you.

Stay tuned.

Thanks, AastaLLL.

Hi,

Thanks for your patience.
Here is steps to generate MaskRCNN uff on the Jetson.

1. Install prerequisite

sudo apt-get update
sudo apt-get install -y build-essential libatlas-base-dev gfortran libfreetype6-dev python3-setuptools
sudo apt-get install -y protobuf-compiler libprotobuf-dev openssl libssl-dev libcurl4-openssl-dev

2. Upgrade cmake

wget http://www.cmake.org/files/v3.13/cmake-3.13.0.tar.gz
tar xpvf cmake-3.13.0.tar.gz cmake-3.13.0/
cd cmake-3.13.0/
./bootstrap --system-curl 
make -j8
echo 'export PATH=/home/nvidia/cmake-3.13.0/bin/:$PATH' >> ~/.bashrc
source ~/.bashrc

3. Install scipy

wget https://github.com/scipy/scipy/releases/download/v1.3.3/scipy-1.3.3.tar.gz
tar -xzvf scipy-1.3.3.tar.gz scipy-1.3.3
cd scipy-1.3.3/
python3 setup.py install --user

4. Compile TensorRT OSS

git clone https://github.com/nvidia/TensorRT TensorRT
cd TensorRT
git submodule update --init --recursive
mkdir -p build && cd build 
cmake -DCUBLASLT_LIB=/usr/local/cuda-10.0/targets/aarch64-linux/lib/libcublas.so ..
make -j8

5. Run SampleMaskRCNN

Install requirement

pip install -r $TRT_SOURCE/samples/opensource/sampleUffMaskRCNN/converted/requirements.txt

Modify /usr/lib/python3.6/dist-packages/uff/converters/tensorflow/converter_functions.py

uff_graph.conv_transpose(
    inputs[0], inputs[2], inputs[1],
    strides, padding,
    dilation=None, number_groups=number_groups,
    left_format=lhs_fmt, right_format=rhs_fmt,
    name=name, fields=fields
    )

Setting Mast_RCNN

git clone https://github.com/matterport/Mask_RCNN.git
export PYTHONPATH=$PYTHONPATH:$PWD/Mask_RCNN
cd Mask_RCNN
git checkout 3deaec5
git am $TRT_SOURCE/samples/opensource/sampleUffMaskRCNN/converted/0001-Update-the-Mask_RCNN-model-from-NHWC-to-NCHW.patch

Download data

cd /path/to/data
wget https://github.com/matterport/Mask_RCNN/releases/download/v2.0/mask_rcnn_coco.h5
cp /usr/src/tensorrt/data/faster-rcnn/001763.ppm .
cp /usr/src/tensorrt/data/faster-rcnn/004545.ppm .

Run

cd $TRT_SOURCE/samples/opensource/sampleUffMaskRCNN/converted/
python3 mrcnn_to_trt_single.py -w /path/to/data/mask_rcnn_coco.h5 -o /path/to/data/mrcnn_nchw.uff -p ./config.py
cd $TRT_SOURCE/build
./sample_uff_maskRCNN -d /path/to/data

Please noticed that we found there is issue when running the sample with PyramidROIAlign plugin.
This issue is passed to our internal team and will be updated here if we got any response.

Thanks.

According to your suggestion, I try to build on Xavier but still meet some errors during make procedure:

-- Configuring incomplete, errors occurred!
See also "/home/nvidia/TensorRT/build/third_party.protobuf_aarch64/src/third_party.protobuf_aarch64/CMakeFiles/CMakeOutput.log".
See also "/home/nvidia/TensorRT/build/third_party.protobuf_aarch64/src/third_party.protobuf_aarch64/CMakeFiles/CMakeError.log".
CMakeFiles/third_party.protobuf_aarch64.dir/build.make:108: recipe for target 'third_party.protobuf_aarch64/src/third_party.protobuf_aarch64-stamp/third_party.protobuf_aarch64-configure' failed
make[2]: *** [third_party.protobuf_aarch64/src/third_party.protobuf_aarch64-stamp/third_party.protobuf_aarch64-configure] Error 1
CMakeFiles/Makefile2:173: recipe for target 'CMakeFiles/third_party.protobuf_aarch64.dir/all' failed
make[1]: *** [CMakeFiles/third_party.protobuf_aarch64.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....

Hi,

Sorry for the late.

Would you mind to turn off the parser and try it again?

https://github.com/NVIDIA/TensorRT/blob/master/CMakeLists.txt#L57

option(BUILD_PARSERS "Build TensorRT parsers" OFF)

Thanks.

Thanks a lot for your help. After I turned off this option, the build finally finished.

Hi,

Good to know this.

Another information is that we confirm that this sample can run correctly with TensorRT 6.0. (JetPack4.3).
The error of PyramidROIAlign won’t occur on the TRT6.0.

Thanks.

Hi, AastaLLL

I followed the steps to generate MaskRCNN uff on the Jetson XAVIER.
And I also encountered a problem.
Here are some informations:

/home/nvidia/TensorRT/plugin/common/kernels/maskRCNNKernels.cu, 1149, 48, no kernel image is available for execution on the device
/home/nvidia/TensorRT/plugin/common/kernels/maskRCNNKernels.cu, 1225, 48, no kernel image is available for execution on the device
/home/nvidia/TensorRT/plugin/common/kernels/maskRCNNKernels.cu, 1231, 48, no kernel image is available for execution on the device
/home/nvidia/TensorRT/plugin/common/kernels/maskRCNNKernels.cu, 1236, 48, no kernel image is available for execution on the device
/home/nvidia/TensorRT/plugin/common/kernels/maskRCNNKernels.cu, 1000, 48, no kernel image is available for execution on the device
/home/nvidia/TensorRT/plugin/common/kernels/maskRCNNKernels.cu, 1049, 48, no kernel image is available for execution on the device
/home/nvidia/TensorRT/plugin/common/kernels/maskRCNNKernels.cu, 1055, 48, no kernel image is available for execution on the device
/home/nvidia/TensorRT/plugin/common/kernels/maskRCNNKernels.cu, 1060, 48, no kernel image is available for execution on the device
[11/26/2019-16:03:20] [I] Run for 10 times with Batch Size 1
[11/26/2019-16:03:20] [I] Average inference time is 410.036 ms/frame
output31
[11/26/2019-16:03:20] [I] The results are stored in current directory: 0.ppm
&&&& PASSED TensorRT.sample_maskrcnn # ./sample_uff_maskRCNN -d …/samples/opensource/sampleUffMaskRCNN/data

Could you give me some advice?

Hi, AastaLLL

I followed the steps to generate MaskRCNN uff on the Jetson XAVIER.
And I also encountered a problem.
Here are some informations:

/home/nvidia/TensorRT/plugin/common/kernels/maskRCNNKernels.cu, 1149, 48, no kernel image is available for execution on the device
/home/nvidia/TensorRT/plugin/common/kernels/maskRCNNKernels.cu, 1225, 48, no kernel image is available for execution on the device
/home/nvidia/TensorRT/plugin/common/kernels/maskRCNNKernels.cu, 1231, 48, no kernel image is available for execution on the device
/home/nvidia/TensorRT/plugin/common/kernels/maskRCNNKernels.cu, 1236, 48, no kernel image is available for execution on the device
/home/nvidia/TensorRT/plugin/common/kernels/maskRCNNKernels.cu, 1000, 48, no kernel image is available for execution on the device
/home/nvidia/TensorRT/plugin/common/kernels/maskRCNNKernels.cu, 1049, 48, no kernel image is available for execution on the device
/home/nvidia/TensorRT/plugin/common/kernels/maskRCNNKernels.cu, 1055, 48, no kernel image is available for execution on the device
/home/nvidia/TensorRT/plugin/common/kernels/maskRCNNKernels.cu, 1060, 48, no kernel image is available for execution on the device
[11/26/2019-16:03:20] [I] Run for 10 times with Batch Size 1
[11/26/2019-16:03:20] [I] Average inference time is 410.036 ms/frame
output31
[11/26/2019-16:03:20] [I] The results are stored in current directory: 0.ppm
&&&& PASSED TensorRT.sample_maskrcnn # ./sample_uff_maskRCNN -d …/samples/opensource/sampleUffMaskRCNN/data

Could you give me some advice?

Hi,

/home/nvidia/TensorRT/plugin/common/kernels/maskRCNNKernels.cu, 1060, 48, no kernel image is available for execution on the device
This indicates that the source doesn’t compile with correct GPU architecture.
For Xavier, the GPU architecture is 72.

Please help to update the CMakeLists.txt here:
https://github.com/NVIDIA/TensorRT/blob/master/CMakeLists.txt#L84

list(APPEND GPU_ARCHS
      35
      53
      61
      70
      <b>72</b>
      75
    )

And rebuild the repository.
Thanks.