Provide details on the platforms you are using:
Linux distro and version - Ubuntu 18.04.4 LTS
GPU type - Tesla T4 (AWS g4dn.xlarge) and Volta V100 (AWS p3.2xlarge)
Nvidia driver version - 450.36.06
CUDA version - 10.0
CUDNN version - 7.6
Python version [if using python] - 3.6
Tensorflow version - 1.14
TensorRT version - 7
Describe the problem
Using tensorrt 5 (tf 1.15) for conversion, results:
numb. of all_nodes in frozen graph: 3408
numb. of trt_engine_nodes in TensorRT graph: 7
numb. of all_nodes in TensorRT graph: 892
using Tensort 7.0.0 (tf 1.14) for conversion. results:
numb. of all_nodes in frozen graph: 3408
numb. of trt_engine_nodes in TensorRT graph: 0
numb. of all_nodes in TensorRT graph: 1789
Files
Code used for conversion:
import tensorflow as tf
from tensorflow.python.platform import gfile
from tensorflow.python.compiler.tensorrt import trt_convert as trt
frozen_graph = ‘/home/ubuntu/Tensorrt/frozen_model.pb’
with open(frozen_graph, ‘rb’) as f:
frozen_graph_gd = tf.GraphDef()
frozen_graph_gd.ParseFromString(f.read())
if tf.test.gpu_device_name():
print(‘Default GPU Device:{}’.format(tf.test.gpu_device_name()))
else:
print(“Please install GPU version of TF”)
trt_graph = trt.create_inference_graph(
input_graph_def = frozen_graph_gd,
is_dynamic_op=True,
outputs=[‘num_detections:0’, ‘detection_boxes:0’, ‘detection_scores:0’,‘detection_classes:0’],
max_batch_size=32,
max_workspace_size_bytes=2*(10**9),
precision_mode=“FP16”)
with gfile.FastGFile(“/home/ubuntu/Tensorrt/TensorRT_model.pb”, ‘wb’) as f:
f.write(trt_graph.SerializeToString())
print(“TensorRT model is successfully stored!”)
all_nodes = len([1 for n in frozen_graph_gd.node])
print(“numb. of all_nodes in frozen graph:”, all_nodes)
#check how many ops that is converted to TensorRT engine
trt_engine_nodes = len([1 for n in trt_graph.node if str(n.op) == ‘TRTEngineOp’])
print(“numb. of trt_engine_nodes in TensorRT graph:”, trt_engine_nodes)
all_nodes = len([1 for n in trt_graph.node])
print(“numb. of all_nodes in TensorRT graph:”, all_nodes)
Include any logs, source, models (.uff, .pb, etc.) that would be helpful to diagnose the problem.
Cannot share .pb due to infosec.
No error reported in logs. Identical for both cases.
Reproducibility
Using the above code and any detection model.