Description
I trained a model using tensorflow / keras and tried to convert to TensorRT.
While the tensorflow saved model size was around 19MB, the converted TensorRT file size was around 900MB.
The output file size is too huge; Do you have any suggestions for improvement?
Thanks
My environment,
python 3.7.6
tf 2.3.1
keras 2.4.0
cuda 11.0
This is how I created the model.
cnn_base = MobileNet(input_shape=shape[1:],weights="imagenet",include_top=False)
cnn_reshape = Reshape((28,1024))(cnn_base.output)
cnn = tf.keras.models.Model(cnn_base.input, cnn_reshape)
cnn.trainable = False
timestep_inputs = [Input(shape[1:],name='sinput_'+str(idx)) for idx in range(shape[0])]
conv_outputs = []
for x in timestep_inputs:
y = cnn(x)
conv_outputs.append(y)
x = concatenate(conv_outputs, axis = 1)
print('x',x.shape)
y = LSTM(128, return_sequences=False, return_state=False, stateful=False, dropout=0.5,activation='tanh')(x)
hidden_layer = Dense(128, activation="relu")(y)
outputs = Dense(nbout, activation="softmax")(hidden_layer)
model = tf.keras.models.Model(timestep_inputs, outputs)
This is model summary
Model: “functional_3”
Layer (type) Output Shape Param # Connected to
sinput_0 (InputLayer) [(None, 132, 251, 3) 0
sinput_1 (InputLayer) [(None, 132, 251, 3) 0
.
.
.
sinput_69 (InputLayer) [(None, 132, 251, 3) 0
functional_1 (Functional) (None, 28, 1024) 3228864 sinput_0[0][0]
sinput_1[0][0]
.
.
.
sinput_69[0][0]
concatenate (Concatenate) (None, 1960, 1024) 0 functional_1[0][0]
functional_1[1][0]
.
.
.
functional_1[69][0]
lstm (LSTM) (None, 128) 590336 concatenate[0][0]
dense (Dense) (None, 128) 16512 lstm[0][0]
dense_1 (Dense) (None, 2) 258 dense[0][0]
Total params: 3,835,970
Trainable params: 607,106
Non-trainable params: 3,228,864
None
This is how I converted the model.
import tensorflow as tf
from tensorflow.python.compiler.tensorrt import trt_convert as trt
converter = trt.TrtGraphConverterV2(input_saved_model_dir='drown_model2_m2')
converter.convert()
#converter.build(input_fn=my_input_fn)
converter.save('t')
With our without build the size is around 900MB.