Node Creation for Concat Layer while converting SSD MobilenetV1 frozen graph to .uff

AI & Data Science Deep Learning (Training & Inference) TensorRT
1


Ubuntu 16.04
TensorRT container 19.02-py3 (TensorRT v5.0.2)
NVIDIA P4

Hi,
I converted the frozen inference graph for SSD Mobilenet V1 using convert-to-uff script. I have used the following config for the conversion:

import graphsurgeon as gs
import tensorflow as tf

Input = gs.create_node("Input",
    op="Placeholder",
    dtype=tf.float32,
    shape=[1, 3, 300, 300])

Fill = gs.create_node("Fill",
    op="Fill",
    dtype=tf.float32)

PriorBox = gs.create_plugin_node(name="GridAnchor", op="GridAnchor_TRT",
    numLayers=6,
    minSize=0.2,
    maxSize=0.95,
    aspectRatios=[1.0, 2.0, 0.5, 3.0, 0.33],
    variance=[0.1,0.1,0.2,0.2],
    featureMapShapes=[19, 10, 5, 3, 2, 1])

NMS = gs.create_plugin_node(name="NMS", op="NMS_TRT",
    shareLocation=1,
    varianceEncodedInTarget=0,
    backgroundLabelId=0,
    confidenceThreshold=1e-8,
    nmsThreshold=0.6,
    topK=100,
    keepTopK=100,
    numClasses=2,
    inputOrder=[0, 2, 1],
    confSigmoid=1,
    isNormalized=1,
    scoreConverter="SIGMOID")

concat_priorbox = gs.create_node(name="concat_priorbox", op="ConcatV2", dtype=tf.float32, axis=2)
concat_box_loc = gs.create_plugin_node("concat_box_loc", op="FlattenConcat_TRT", dtype=tf.float32, axis=1, ignoreBatch=0)
concat_box_conf = gs.create_plugin_node("concat_box_conf", op="FlattenConcat_TRT", dtype=tf.float32, axis=1, ignoreBatch=0)

namespace_plugin_map = {
    "MultipleGridAnchorGenerator":PriorBox,
    "Postprocessor":NMS,
    "Preprocessor":Input,
    "image_tensor":Input,
    "ToFloat":Input,
    "MultipleGridAnchorGenerator/Concatenate": concat_priorbox,
    "concat": concat_box_loc,
    "concat_1": concat_box_conf,
    "FeatureExtractor/MobilenetV1/zeros":Fill,
    "FeatureExtractor/MobilenetV1/zeros_1":Fill,
    "FeatureExtractor/MobilenetV1/zeros_2":Fill,
    "FeatureExtractor/MobilenetV1/zeros_3":Fill,
    "FeatureExtractor/MobilenetV1/zeros_4":Fill,
    "FeatureExtractor/MobilenetV1/zeros_5":Fill,
    "FeatureExtractor/MobilenetV1/zeros_6":Fill,
    "FeatureExtractor/MobilenetV1/zeros_7":Fill
}

def preprocess(dynamic_graph):
    dynamic_graph.collapse_namespaces(namespace_plugin_map)

    dynamic_graph.remove(dynamic_graph.graph_outputs, remove_exclusive_dependencies = False)

I referred to this link for implementation:
https://devtalk.nvidia.com/default/topic/1047429/tensorrt/sampleuffssd-with-custom-ssd_mobilenet_v1-model/

I have few assumptions based on my understanding. Correct me if I am wrong.

Assumptions:

  1. For the layers which UFF converter is not able to convert, we create a node/plugin node in the config.
  2. If the layer is one of the DLA supported layers then we create a node
  3. If it is either in the TRT plugin registry or if we need to create a custom implementation for the layer we create a plugin node.

Based on these assumptions I have written the above config and have the following questions:

Questions

  1. According to the DLA supported layers, “DLA supports concatenation only along the channel axis”.
    The node definition in .pbtxt for the graph is,
nodes {
    id: "MultipleGridAnchorGenerator/Concatenate/concat"
    inputs: "MultipleGridAnchorGenerator/concat"
    inputs: "MultipleGridAnchorGenerator/concat_1"
    inputs: "MultipleGridAnchorGenerator/concat_2"
    inputs: "MultipleGridAnchorGenerator/concat_3"
    inputs: "MultipleGridAnchorGenerator/concat_4"
    inputs: "MultipleGridAnchorGenerator/concat_5"
    operation: "Concat"
    fields {
      key: "axis"
      value {
        i: 0
      }
    }
  }

concat_priorbox = gs.create_node(name="concat_priorbox", op="ConcatV2", dtype=tf.float32, axis=2)

Here axis=0 but while node creation for concat_priorbox we give axis=2 . Same node is also used in sampleUFFSSD for SSDInceptionV2. Is there a reason for using different axis values?
Can someone throw more light on the limitations of the supported concatenation operation?

  1. While conversion I also get the following operations in the list unsupported layers:
Warning: No conversion function registered for layer: Fill yet.
Converting FeatureExtractor/MobilenetV1/zeros_7 as custom op: Fill
Warning: No conversion function registered for layer: Fill yet.
Converting FeatureExtractor/MobilenetV1/zeros_5 as custom op: Fill
Warning: No conversion function registered for layer: Fill yet.
Converting FeatureExtractor/MobilenetV1/zeros_3 as custom op: Fill
Warning: No conversion function registered for layer: Fill yet.
Converting FeatureExtractor/MobilenetV1/zeros_1 as custom op: Fill
Warning: No conversion function registered for layer: Fill yet.
Converting FeatureExtractor/MobilenetV1/zeros as custom op: Fill
Warning: No conversion function registered for layer: Fill yet.
Converting FeatureExtractor/MobilenetV1/zeros_2 as custom op: Fill
Warning: No conversion function registered for layer: Fill yet.
Converting FeatureExtractor/MobilenetV1/zeros_4 as custom op: Fill
Warning: No conversion function registered for layer: Fill yet.
Converting FeatureExtractor/MobilenetV1/zeros_6 as custom op: Fill

But for the same layers there is no node creation in the sample configs to which I referred.
Here we fill the input with 0s. Can’t this can be considered as a scaling layer or an elementwise layer?
Is it due the difference in TensorRT versions that this layer is not supported for 5.0.2?

2

Anyone who can respond to any of the queries here?