I have been working with Tensor RT 4 with Tensor flow 1.7 on Ubuntu 16.04. Such an excellent design and powerful, fast inference.
In my tests with Inception V3, I see that the TensorRT output my_trt_op0 has dimension [?, 2048, 1,1] while the intended is [?,2048]. How can I avoid this issue?
Just wanted to add my workaround, in case it might help any one.
Basically, somehow the average pool layer output before the dense layer (1000 output) of Inception V3 is being understood as (?, 2048, 1,1). This was actually (?, 2048) in TensorBoard when I look at the graph.
I have explicitly stacked a Reshape() function in Keras to process this layer into (2048,) shape
One more point is TensorRT needs to know the explicit input shape. This needs to be input when making the InceptionV3() model
The below model compressed the most of the model into a tensor RT node :)
=====Code snippet for model ======
model = tf.keras.applications.inception_v3.InceptionV3([b]input_shape[/b]=(299,299,3),include_top=False, pooling='avg')
[b]flatten[/b] = tf.keras.layers.Reshape((2048,))
dense= tf.keras.layers.Dense(NUM_CLASSES,input_shape=(None,2048), kernel_initializer=tf.keras.initializers.he_normal(), activation='softmax')
model = tf.keras.models.Model(model.input, dense(flatten(model(model.input))))
print('Model layers=' + str(len(model.layers)))