convert_to_uff.py failed to parse nodes from config.py when trying to convert resnet50_v1_fpn frozen graph to UFF

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

I am trying to convert frozen graph of ssd_resnet50_fpn_coco from Tensorflow detection model zoo to UFF.

In process of conversion, I changed my config.py as follows:

import graphsurgeon as gs
import tensorflow as tf

Input = gs.create_node("Input",
    op="Placeholder",
    dtype=tf.float32,
    shape=[1, 3, 640, 640])
PriorBox = gs.create_node("PriorBox",
    #numLayers=6,
    #minScale=0.2,
    #maxScale=0.95,
    min_level= 3,
    max_level = 7,
    anchor_scale = 4.0,
    aspectRatios=[1.0, 2.0, 0.5],
    layerVariances=[0.1,0.1,0.2,0.2],
    featureMapShapes=[80, 40, 20, 10, 5])
NMS = gs.create_node("NMS",
    scoreThreshold=0.3,
    iouThreshold=0.6,
    maxDetectionsPerClass=100,
    maxTotalDetections=100,
    numClasses=91,
    scoreConverter="SIGMOID")
concat_priorbox = gs.create_node("concat_priorbox", dtype=tf.float32, axis=2)
concat_box_loc = gs.create_node("concat_box_loc")
concat_box_conf = gs.create_node("concat_box_conf")

namespace_plugin_map = {
    "Postprocessor": NMS,
    "MultiscaleGridAnchorGenerator": PriorBox,
    "Preprocessor": Input,
    # "ToFloat": Input,
    # "image_tensor": Input,
    #"Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/Concatenate": concat_priorbox,
    "concat": concat_box_loc,
    "concat_1": concat_box_conf,
    "Postprocessor/BatchMultiClassNonMaxSuppression/map/while/MultiClassNonMaxSuppression/Concatenate": concat_priorbox,
    "MultiscaleGridAnchorGenerator": PriorBox,
}

namespace_remove = {
    "ToFloat",
    "image_tensor",
    "Preprocessor/map/TensorArrayStack_1/TensorArrayGatherV3",
}



def preprocess(dynamic_graph):
    # remove the unrelated or error layers

    dynamic_graph.remove(dynamic_graph.find_nodes_by_path(namespace_remove), remove_exclusive_dependencies=False)

    # Now create a new graph by collapsing namespaces

    dynamic_graph.collapse_namespaces(namespace_plugin_map)

    # Remove the outputs, so we just have a single output node (NMS).

    dynamic_graph.remove(dynamic_graph.graph_outputs, remove_exclusive_dependencies=False)

    # Remove the Squeeze to avoid "Assertion `isPlugin(layerName)' failed"

    # Squeeze = dynamic_graph.find_node_inputs_by_name(dynamic_graph.graph_outputs[0], 'Squeeze')

    # dynamic_graph.forward_inputs(Squeeze)

After running the command:

python convert_to_uff.py tensorflow --input-file ssd_resnet50_v1_fpn_shared_box_predictor_640x640_coco14_sync_2018_07_03/frozen_inference_graph.pb -O NMS -p config_ssd2018_resnet.py

The output log is as follows:

Loading ssd_resnet50_v1_fpn_shared_box_predictor_640x640_coco14_sync_2018_07_03/frozen_inference_graph.pb
Using output node NMS
Converting to UFF graph
Warning: No conversion function registered for layer: NMS yet.
Converting as custom op NMS NMS
name: "NMS"
op: "NMS"
input: "concat_box_loc"
input: "concat_box_conf"
input: "concat_priorbox"
attr {
  key: "iouThreshold_u_float"
  value {
    f: 0.600000023842
  }
}
attr {
  key: "maxDetectionsPerClass_u_int"
  value {
    i: 100
  }
}
attr {
  key: "maxTotalDetections_u_int"
  value {
    i: 100
  }
}
attr {
  key: "numClasses_u_int"
  value {
    i: 91
  }
}
attr {
  key: "scoreConverter_u_str"
  value {
    s: "SIGMOID"
  }
}
attr {
  key: "scoreThreshold_u_float"
  value {
    f: 0.300000011921
  }
}

Warning: No conversion function registered for layer: concat_priorbox yet.
Converting as custom op concat_priorbox concat_priorbox
name: "concat_priorbox"
op: "concat_priorbox"
input: "NMS"
attr {
  key: "axis_u_int"
  value {
    i: 2
  }
}
attr {
  key: "dtype"
  value {
    type: DT_FLOAT
  }
}

Warning: No conversion function registered for layer: concat_box_conf yet.
Converting as custom op concat_box_conf concat_box_conf
name: "concat_box_conf"
op: "concat_box_conf"
input: "WeightSharedConvolutionalBoxPredictor/Reshape_1"
input: "WeightSharedConvolutionalBoxPredictor_1/Reshape_1"
input: "WeightSharedConvolutionalBoxPredictor_2/Reshape_1"
input: "WeightSharedConvolutionalBoxPredictor_3/Reshape_1"
input: "WeightSharedConvolutionalBoxPredictor_4/Reshape_1"

Warning: No conversion function registered for layer: concat_box_loc yet.
Converting as custom op concat_box_loc concat_box_loc
name: "concat_box_loc"
op: "concat_box_loc"
input: "WeightSharedConvolutionalBoxPredictor/Reshape"
input: "WeightSharedConvolutionalBoxPredictor_1/Reshape"
input: "WeightSharedConvolutionalBoxPredictor_2/Reshape"
input: "WeightSharedConvolutionalBoxPredictor_3/Reshape"
input: "WeightSharedConvolutionalBoxPredictor_4/Reshape"

No. nodes: 877
UFF Output written to ssd_resnet50_v1_fpn_shared_box_predictor_640x640_coco14_sync_2018_07_03/frozen_inference_graph.pb.uff

From the above log, It is evident that “MultiscaleGridAnchorGenerator”: PriorBox isn’t parsed. Does TensorRT support FPN(Feature Pyramid Network) based object detection models? If yes, How do I modify config.py such that it generates UFF for the frozen graph of ssd_resnet50_v1_fpn_shared_box_predictor_640x640_coco14_sync_2018_07_03 model?

2

Have you solved the problem?