Ssd_Mobilenet_V2 on Deepstream-app Aborted (core dumped)

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

I trained ssd_mobilenet_v2 and converted to uff successfully. I tested using the inference instructions in https://devtalk.nvidia.com/default/topic/1050377/jetson-nano/deep-learning-inference-benchmarking-instructions/ and it ran properly. I also tested on https://github.com/AastaNV/TRT_object_detection with no problems. However while running it using the deepstream-app this was the error.

sharan@jetson-nano:~/deepstream_sdk_v4.0.1_jetson/sources/objectDetector_SSD$ deepstream-app -c deepstream_app_config_ssd.txt

Using winsys: x11
0:00:01.397554664 26563     0x1657e4d0 WARN                 nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:useEngineFile(): Failed to read from model engine file
0:00:01.397651333 26563     0x1657e4d0 INFO                 nvinfer gstnvinfer.cpp:519:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:initialize(): Trying to create engine from model files
deepstream-app: nvdsiplugin_ssd.cpp:53: FlattenConcat::FlattenConcat(int, bool): Assertion `mConcatAxisID == 1 || mConcatAxisID == 2 || mConcatAxisID == 3' failed.
Aborted (core dumped)
2

Hi,

The default configure in objectDetector_SSD is for model ssd_inception_v2_coco.
Please check this comment for the change of ssd_mobilenet_v2:
https://devtalk.nvidia.com/default/topic/1066088/deepstream-sdk/how-to-use-ssd_mobilenet_v2/post/5399649/#5399649

Thanks.

3

Yes. I changed it for mobilenet. The default coco trained SSD-Mobilenetv2 seems to run fine. This error occurs only when I use a custom trained SSD-Mobilenetv2 detector.

4

The problem seems to be with the nvdsiplugin_ssd.cpp FlattenConcat part. But since I am able to run it without deepstream I assume there is no problem with the weights conversion part. So is the deepstream version(4.0.1) a problem? What else could be the reason?

5

Hi,

Do you mean the model can be inferenced with TensorRT but meet the #1 error with Deepstream?
If yes, could you share the model with us so we can check it further.

Thanks.

6

Yes. The model can be inferenced using tensorrt. The error arises only when using deepstream.

Sent the model files in private message. Please do the necessary.

7

Hi,

It looks like your class number is 2, is it correct?
Except from the class number, do you have any customized item for your model?

Thanks.

8

Hi, the class number is 1 only. Other than that I didnt modify any parameters during training.

9

Hi,

Thanks.
We are checking this issue. Will update more information with you asap.

10

Hi,

Thanks for your patience.
We can run your model with the following change. Please give it a try.

1. Convert .pb file into .uff with following config.py

import graphsurgeon as gs
import tensorflow as tf

Input = gs.create_node("Input",
    op="Placeholder",
    dtype=tf.float32,
    shape=[1, 3, 300, 300])
PriorBox = gs.create_plugin_node(name="GridAnchor", op="GridAnchor_TRT",
    numLayers=6,
    minSize=0.2,
    maxSize=0.95,
    aspectRatios=[1.0, 2.0, 0.5, 3.0, 0.33],
    variance=[0.1,0.1,0.2,0.2],
    featureMapShapes=[19, 10, 5, 3, 2, 1])
NMS = gs.create_plugin_node(name="NMS", op="NMS_TRT",
    shareLocation=1,
    varianceEncodedInTarget=0,
    backgroundLabelId=0,
    confidenceThreshold=1e-8,
    nmsThreshold=0.6,
    topK=100,
    keepTopK=100,
    numClasses=2,
    inputOrder=[0, 2, 1],
    confSigmoid=1,
    isNormalized=1,
    scoreConverter="SIGMOID")
concat_priorbox = gs.create_node(name="concat_priorbox", op="ConcatV2", dtype=tf.float32, axis=2)
concat_box_loc = gs.create_plugin_node("concat_box_loc", op="FlattenConcat_TRT", dtype=tf.float32, axis=1, ignoreBatch=0)
concat_box_conf = gs.create_plugin_node("concat_box_conf", op="FlattenConcat_TRT", dtype=tf.float32, axis=1, ignoreBatch=0)

namespace_plugin_map = {
    "MultipleGridAnchorGenerator": PriorBox,
    "Postprocessor": NMS,
    "Preprocessor": Input,
    "ToFloat": Input,
    "image_tensor": Input,
    "Concatenate": concat_priorbox,
    "concat": concat_box_loc,
    "concat_1": concat_box_conf
}

def preprocess(dynamic_graph):
    # Now create a new graph by collapsing namespaces
    dynamic_graph.collapse_namespaces(namespace_plugin_map)
    # Remove the outputs, so we just have a single output node (NMS).
    dynamic_graph.remove(dynamic_graph.graph_outputs, remove_exclusive_dependencies=False)

    dynamic_graph.find_nodes_by_op("NMS_TRT")[0].input.remove("Input")

2. Apply following change in /opt/nvidia/deepstream/deepstream-4.0/sources/objectDetector_SSD

diff --git a/sources/objectDetector_SSD/config_infer_primary_ssd.txt b/sources/objectDetector_SSD/config_infer_primary_ssd.txt
index bafdff7..089ac89 100755
--- a/sources/objectDetector_SSD/config_infer_primary_ssd.txt
+++ b/sources/objectDetector_SSD/config_infer_primary_ssd.txt
@@ -64,13 +64,13 @@ offsets=127.5;127.5;127.5
 model-color-format=0
 model-engine-file=sample_ssd_relu6.uff_b1_fp32.engine
 labelfile-path=ssd_coco_labels.txt
-uff-file=sample_ssd_relu6.uff
+uff-file=///home/nvidia/topic_1067003/topic_1067003.uff
 uff-input-dims=3;300;300;0
 uff-input-blob-name=Input
 batch-size=1
 ## 0=FP32, 1=INT8, 2=FP16 mode
 network-mode=0
-num-detected-classes=91
+num-detected-classes=2
 interval=0
 gie-unique-id=1
 is-classifier=0

3. Run

$ cd /opt/nvidia/deepstream/deepstream-4.0/sources/objectDetector_SSD
$ cp /usr/src/tensorrt/data/ssd/ssd_coco_labels.txt .
$ export CUDA_VER=10.0
$ make -C nvdsinfer_custom_impl_ssd
$ deepstream-app -c deepstream_app_config_ssd.txt

Thanks and please let us know your result.