@NVES,
I have made some progress in the issue.
First, I tried another *.pb graph from here and had the exact same issue (TF=137ms, TRT=147ms).
I also found that in both cases, if I tried to run the TRT session before the TF session, there would be a bug during inference saying: ‘No attr named ‘identical_element_shapes’ in NodeDef’ (see below for full error stack). This is the same bug recently reported here, where you said there was a bug w/ TF-TRT which was recently fixed in new release. Do you know how I can access this fix?
FULL ERROR:
---------------------------------------------------------------------------
NotFoundError Traceback (most recent call last)
~/Projects/SANATA/trt_test_SSDlite.py in <module>
73 outputs=output_names,
74 max_batch_size=1,
---> 75 precision_mode='FP16', # 'INT8'/'FP16'
76 )
77
~/Projects/SANATA/.venv/lib/python3.5/site-packages/tensorflow/contrib/tensorrt/python/trt_convert.py in create_inference_graph(input_graph_def, outputs, max_batch_size, max_workspace_size_bytes, precision_mode, minimum_segment_size)
113 # pylint: disable=protected-access
114 raise _impl._make_specific_exception(None, None, ";".join(msg[1:]),
--> 115 int(msg[0]))
116 # pylint: enable=protected-access
117 output_graph_def = graph_pb2.GraphDef()
NotFoundError: No attr named 'identical_element_shapes' in NodeDef:
[[Node: Preprocessor/map/TensorArray_2 = TensorArrayV3[clear_after_read=true, dtype=DT_INT32, dynamic_size=false, element_shape=<unknown>, tensor_array_name=""](Preprocessor/map/TensorArrayUnstack/strided_slice)]] for 'Preprocessor/map/TensorArray_2' (op: 'TensorArrayV3') with input shapes: [].