Description
Hello,
I have been trying to convert a custom tensorflow model to tensor RT, using TF-TRT workflow.
The conversion prouduces a usable graph but it looks like no optimization was performed. When trying to serialize to the .plan file, no operation is found as “TRTEngineOp”, all the nodes are excluded and no .plan file is generated.
The architecture is mostly composed of basic convolutional blocks (convolution, pooling, non-linearity), plus some bilinear resize.
I attach an extract from the output to give an idea of the type of node that are excluded:
Exclude Node: BiasAdd, architecture_layer1_nin_block_nin_0_BiasAdd
Exclude Node: Mul, architecture_layer1_nin_block_nin_0_mul
Exclude Node: Maximum, architecture_layer1_nin_block_nin_0_leaky_relu
Exclude Node: Conv2D, architecture_layer1_nin_block_nin_1_Conv2D
Exclude Node: BiasAdd, architecture_layer1_nin_block_nin_1_BiasAdd
Exclude Node: Mul, architecture_layer1_nin_block_nin_1_mul
Exclude Node: Maximum, architecture_layer1_nin_block_nin_1_leaky_relu
Exclude Node: MaxPool, architecture_layer1_pooling
Exclude Node: ResizeBilinear, architecture_layer1_resize_regL
Environment
TensorRT Version: 7.0.0.11
GPU Type: RTX2080ti
Nvidia Driver Version: 450.102.04
CUDA Version: 10.2
CUDNN Version: 7.6.5
Operating System + Version: Ubuntu 18.04
TensorFlow Version: 1.14.0
Relevant Files
The script I used to convert the graph
import tensorflow as tf
from tensorflow.python.compiler.tensorrt import trt_convert as trt
with tf.Session() as sess:
# First deserialize your frozen graph:
with tf.gfile.GFile(pb_model_path, 'rb') as f:
frozen_graph = tf.GraphDef()
frozen_graph.ParseFromString(f.read())
# Now you can create a TensorRT inference graph from your
# frozen graph:
converter = trt.TrtGraphConverter(
input_graph_def=frozen_graph,
is_dynamic_op=False,
nodes_blacklist=output_nodes)
trt_graph = converter.convert()
# Import the TensorRT graph into a new graph and run:
output_node = tf.import_graph_def(
trt_graph,
return_elements=output_nodes)
for nn in trt_graph.node:
if nn.op == "TRTEngineOp":
print("Node: %s, %s" % (nn.op, nn.name.replace("/", "_")))
with tf.gfile.GFile("%s.plan" % (nn.name.replace("/", "_")), 'wb') as f:
f.write(nn.attr["serialized_segment"].s)
else:
print("Exclude Node: %s, %s" % (nn.op, nn.name.replace("/", "_")))