How to convert keras .h5 model and use it in tensorrt

HI All,

I am pretty new to the DL so please bear with me if I asked anything stupid.

I had defined a custom model using module tensorflow.python.keras. I had serval TimeDistributed Layer and LSTM layer in the custom model and I successfully save it as follow:


When I tried to free the graph following instruction from various the internet (for example this:

I received the following error message.

File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/util/", line 324, in new_func
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.6/dist-packages/tensorflow/python/framework/", line 302, in convert_variables_to_constants
    raise ValueError("Cannot find the variable that is an input "
ValueError: Cannot find the variable that is an input to the ReadVariableOp.

But if I use a model with no LSTM and TimeDistributed(with only ConvNet and Dense), tf is able to freeze the graph and generate the .pb file

Did I do something wrong or I need to wait for the TensorFlow update in order to use tensorrt? Is there any beginner guide to tensorrt using jetson Nano?


A possible reason is that some of your layers may not be supported from the TensorRT compiler.
Would you mind to follow our official sample to freeze the model first?



Most of the code is the same. The problem is in this line of code:

frozen_graph = tf.graph_util.convert_variables_to_constants(sess, sess.graph.as_graph_def(), output_names)

for your information, inside tensorflow library ( I am using tf1.14 precompile for jetson Nano)

map_name_to_node[source_op_name].op # 'Enter'
source_op_name # 'lstm/while/split/ReadVariableOp/Enter'

Is it because LSTM is not supported? Does it support GRU / basic RNN?


This depends on the layers used in your model.
For example, TensorRT support basic LSTM and you can find an example here:



The Layer is pretty straight forward and please find the details as follow:
I am able to convert the model from .h5 to pb than uff without LSTM layer.

input_dim = (seq_length,) + img_in
    img_in = Input(shape=input_dim, name='img_in')
    imu_in = Input(shape=(seq_length, imu_in), name="imu_in")

    x = TimeDistributed(Convolution2D(24, (5, 5), strides=(2, 2), activation='relu'))(img_in)
    x = Dropout(drop)(x)
    x = TimeDistributed(Convolution2D(32, (5, 5), strides=(2, 2), activation='relu'))(x)
    x = Dropout(drop)(x)
    x = TimeDistributed(Convolution2D(32, (3, 3), strides=(2, 2), activation='relu'))(x)
    x = Dropout(drop)(x)
    x = TimeDistributed(Convolution2D(32, (3, 3), strides=(2, 2), activation='relu'))(x)
    x = Dropout(drop)(x)
    x = TimeDistributed(Flatten())(x)
    x = Dropout(drop)(x)
    x = TimeDistributed(Dense(64, activation='relu'))(x)
    x = Dropout(drop)(x)

    y = TimeDistributed(Dense(14, activation='relu'))(imu_in)
    y = Dropout(drop)(y)
    y = TimeDistributed(Dense(14, activation='relu'))(y)
    y = Dropout(drop)(y)

    z = concatenate([x, y])

    z = LSTM(128, return_sequences=True, activation='tanh')(z)
    z = LSTM(128, return_sequences=False, activation='tanh')(z)

    z = Dense(64, activation='relu')(z)
    z = Dropout(drop)(z)
    z = Dense(32, activation='relu')(z)
    z = Dropout(drop)(z)

    out_1 = Dense(31, activation='softmax', name='out_1')(z)

    out_2 = Dense(31, activation='softmax', name='out_2')(z)

    model = Model(inputs=[img_in,imu_in], outputs=[out_1, out_2])


It doesn’t depends on the model complexity but the layer you used.

The TimeDistributed layer is not in our supported matrix.
That’s why you cannot convert it into TensorRT.