Hello everyone,
I’m using TensorRT in order to do inference of a Tensorflow FasterRCNN model. However I have some issues trying to have the graph_surgeon script working properly.
My config.py is the following:
[i]import graphsurgeon as gs
import tensorflow as tf
Input = gs.create_node(
“Input”,
op=“Placeholder”,
dtype=tf.float32,
shape=[1, 3, 300, 300]
)
PriorBox = gs.create_plugin_node(
name=“GridAnchor”,
op=“GridAnchor_TRT”,
numLayers=6,
minSize=0.2,
maxSize=0.95,
aspectRatios=[1.0, 2.0, 0.5, 3.0, 0.33],
variance=[0.1, 0.1, 0.2, 0.2],
featureMapShapes=[19, 10, 5, 3, 2, 1]
)
NMS = gs.create_plugin_node(
name=“NMS”,
op=“NMS_TRT”,
shareLocation=1,
varianceEncodedInTarget=0,
backgroundLabelId=0,
confidenceThreshold=1e-8,
nmsThreshold=0.6,
topK=100,
keepTopK=100,
numClasses=91,
inputOrder=[0, 2, 1],
confSigmoid=1,
isNormalized=1
)
concat_priorbox = gs.create_node(
name=“concat_priorbox”,
op=“ConcatV2”,
dtype=tf.float32,
axis=2
)
concat_box_loc = gs.create_plugin_node(
“concat_box_loc”,
op=“FlattenConcat_TRT”,
dtype=tf.float32,
axis=1,
ignoreBatch=0
)
concat_box_conf = gs.create_plugin_node(
“concat_box_conf”,
op=“FlattenConcat_TRT”,
dtype=tf.float32,
axis=1,
ignoreBatch=0
)
namespace_plugin_map = {
“GridAnchorGenerator”: PriorBox,
"Postprocessor": NMS,
"Preprocessor": Input,
"ToFloat": Input,
"image_tensor": Input,
“GridAnchorGenerator/concat”: concat_priorbox,
“GridAnchorGenerator/Identity”: concat_priorbox,
“concat”: concat_box_loc,
“concat_1”: concat_box_conf
}
#namespace_remove = {
“ToFloat”,
“image_tensor:0”,
“Preprocessor/map/TensorArrayStack_1/TensorArrayGatherV3”
}
def preprocess(dynamic_graph):
# remove the unrelated or error layers
dynamic_graph.remove(
dynamic_graph.find_nodes_by_path(
namespace_remove
),
remove_exclusive_dependencies=False
)
# Now create a new graph by collapsing namespaces
dynamic_graph.collapse_namespaces(namespace_plugin_map)
# Remove the outputs, so we just have a single output node (NMS).
dynamic_graph.remove(
dynamic_graph.graph_outputs,
remove_exclusive_dependencies=False
)[/i]
I’m invoking graph_surgeon using the following command:
convert-to-uff frozen_inference_graph.pb -O NMS -p graph_surgeon_example.py
Obtaining the following output:
[i]WARNING: Logging before flag parsing goes to stderr.
W0627 19:34:58.118373 140132127528704 deprecation_wrapper.py:119] From /usr/lib/python3.5/dist-packages/uff/converters/tensorflow/conversion_helpers.py:18: The name tf.GraphDef is deprecated. Please use tf.compat.v1.GraphDef instead.
Loading frozen_inference_graph.pb
W0627 19:34:58.120106 140132127528704 deprecation_wrapper.py:119] From /usr/lib/python3.5/dist-packages/uff/converters/tensorflow/conversion_helpers.py:231: The name tf.gfile.GFile is deprecated. Please use tf.io.gfile.GFile instead.
NOTE: UFF has been tested with TensorFlow 1.12.0. Other versions are not guaranteed to work
WARNING: The version of TensorFlow installed on this system is not guaranteed to work with UFF.
W0627 19:34:59.743098 140132127528704 deprecation_wrapper.py:119] From /usr/lib/python3.5/dist-packages/graphsurgeon/_utils.py:2: The name tf.NodeDef is deprecated. Please use tf.compat.v1.NodeDef instead.
WARNING: To create TensorRT plugin nodes, please use the create_plugin_node function instead.
WARNING: To create TensorRT plugin nodes, please use the create_plugin_node function instead.
UFF Version 0.6.3
=== Automatically deduced input nodes ===
[name: “Input”
op: “Placeholder”
attr {
key: “dtype”
value {
type: DT_FLOAT
}
}
attr {
key: “shape”
value {
shape {
dim {
size: 1
}
dim {
size: 3
}
dim {
size: 300
}
dim {
size: 300
}
}
}
}
]
Using output node NMS
Converting to UFF graph
Warning: No conversion function registered for layer: NMS_TRT yet.
Converting NMS as custom op: NMS_TRT
W0627 19:35:00.486479 140132127528704 deprecation_wrapper.py:119] From /usr/lib/python3.5/dist-packages/uff/converters/tensorflow/converter.py:179: The name tf.AttrValue is deprecated. Please use tf.compat.v1.AttrValue instead.
No. nodes: 2
UFF Output written to frozen_inference_graph.uff[/i]
The thing is that I have just 2 nodes in the output, the pbtxt generated is this:
version: 1
descriptor_core_version: 1
descriptors {
id: “tensorflow_extension”
version: 1
}
descriptors {
id: “custom”
version: 1
}
graphs {
id: “main”
nodes {
id: “NMS”
operation: “_NMS_TRT”
fields {
key: “backgroundLabelId_u_int”
value {
i: 0
}
}
fields {
key: “confSigmoid_u_int”
value {
i: 1
}
}
fields {
key: “confidenceThreshold_u_float”
value {
d: 9.99999993922529e-09
}
}
fields {
key: “inputOrder_u_ilist”
value {
i_list {
val: 0
val: 2
val: 1
}
}
}
fields {
key: “isNormalized_u_int”
value {
i: 1
}
}
fields {
key: “keepTopK_u_int”
value {
i: 100
}
}
fields {
key: “nmsThreshold_u_float”
value {
d: 0.6000000238418579
}
}
fields {
key: “numClasses_u_int”
value {
i: 91
}
}
fields {
key: “shareLocation_u_int”
value {
i: 1
}
}
fields {
key: “topK_u_int”
value {
i: 100
}
}
fields {
key: “varianceEncodedInTarget_u_int”
value {
i: 0
}
}
}
nodes {
id: “MarkOutput_0”
inputs: “NMS”
operation: “MarkOutput”
}
}
Do you have some idea about why I am just obtaining these 2 nodes. Some additional information about graph_surgeon use ??
Thanks for your help.