Discussion about the uff_ssd of python sample

Hi everyone,

I once asked a question which was relevant with how to add custom nodes in gs.create_plugin_node.

Link: https://devtalk.nvidia.com/default/topic/1069216/tensorrt/the-uff_custom_plugin-of-python-sample/

However, I still have a question about my previous mention. (If I need to ask in that post, please let me know. Thanks!)

In the uff_ssd of python sample, we knew the part of adding plugin nodes, ssd_unsupported_nodes_to_plugin_nodes, which is located at
~/TensorRT-7.0.0.11/samples/python/uff_ssd/utils/model.py.

Today, if I get a trained model such as the uff_ssd sample would download an existing model of ssd_inception_v2_coco_2017_11_17 from online, how do we know that which exact layer or which exact node should we add in gs.create_plugin_node?

I knew that there is a list of supported operations on Nvidia TensorRT website, but I wanna know which specific layers/ nodes should I add it in gs.create_plugin_node. Is there any method or standard steps to deal with this part?

As the notes from the ssd_unsupported_nodes_to_plugin_nodes, it mentioned “Note: This specific implementation works only for ssd_inception_v2_coco_2017_11_17 network.”, so if I changed a new model such as ssd_mobilenet_v1_coco_2018_01_28, how to identify its nodes or layers for custom plugin node?

On the other hand, I can get the output by using ssd_graph.graph_outputs.
It showed

[name: "detection_boxes"
op: "Identity"
input: "Postprocessor/BatchMultiClassNonMaxSuppression/map/TensorArrayStack/TensorArrayGatherV3"
attr {
  key: "T"
  value {
    type: DT_FLOAT
  }
}
, name: "detection_scores"
op: "Identity"
input: "Postprocessor/BatchMultiClassNonMaxSuppression/map/TensorArrayStack_1/TensorArrayGatherV3"
attr {
  key: "T"
  value {
    type: DT_FLOAT
  }
}
, name: "num_detections"
op: "Identity"
input: "Postprocessor/ToFloat"
attr {
  key: "T"
  value {
    type: DT_FLOAT
  }
}
, name: "detection_classes"
op: "Identity"
input: "add_6"
attr {
  key: "T"
  value {
    type: DT_FLOAT
  }
}
]

And I saw the description of comment that it removed all of outputs.

# Remove the outputs, so we just have a single output node (NMS).
    # If remove_exclusive_dependencies is True, the whole graph will be removed!
    ssd_graph.remove(ssd_graph.graph_outputs, remove_exclusive_dependencies=False)

What reason does this step let NMS being a new output node instead of removing original output nodes?
Is it due to TensorRT not supporting these TensorFlow operations then removing these nodes?

Thank you for giving me any thoughts.

Best regards,
Chieh

Hi @Chieh,

I would expect that you’d have to use gs.create_plugin_node on any nodes that fail to parse; however, this also implies that you’d have to have built plugin implementations of the unsupported nodes as well.

With the deprecation of UFF parser in TRT 7, I encourage using tf2onnx and the ONNX parser instead, as more ops should be supported by default, and plugin support should be coming soon.

Hi @NVES_R,

Got it. Thanks for your suggestion!