This version of TensorRT does not support dynamic ReverseSequence length


I build a model with Bidirectional LSTM, and I save the model to .pb format, then convert the pb file to onnx. When I use trtexec to convert the onnx to trt engine, it failed.
model code:

max_features = 20000  # Only consider the top 20k words
maxlen = 200  # Only consider the first 200 words of each movie review

# Input for variable-length sequences of integers
inputs = keras.Input(shape=(200,), dtype="int32")
# Embed each integer in a 128-dimensional vector
x = layers.Embedding(max_features, 128)(inputs)
# Add 2 bidirectional LSTMs
x = layers.Bidirectional(layers.LSTM(64, return_sequences=True))(x)
x = layers.Bidirectional(layers.LSTM(64))(x)
# Add a classifier
outputs = layers.Dense(1, activation="sigmoid")(x)
model = keras.Model(inputs, outputs)

(x_train, y_train), (x_val, y_val) =
print(len(x_train), "Training sequences")
print(len(x_val), "Validation sequences")
x_train = keras.preprocessing.sequence.pad_sequences(x_train, maxlen=maxlen)
x_val = keras.preprocessing.sequence.pad_sequences(x_val, maxlen=maxlen)

model.compile("adam", "binary_crossentropy", metrics=["accuracy"]), y_train, batch_size=32, epochs=2, validation_data=(x_val, y_val))


TensorRT Version: 8.0.16
GPU Type: V100:32GB
Nvidia Driver Version: 450.119.04
CUDA Version: 10.2
CUDNN Version: 8.0.2
Operating System + Version: ubuntu 16.04
Python Version (if applicable): 3.8.0
TensorFlow Version (if applicable): 2.3

Relevant Files

the log file
the onnx file

I make the input’shape fixed, but failed again.

inputs = keras.Input(shape=(200,), batch_size=32, dtype="int32")

onnx file
log file


Regarding primary error you’re facing, currently dynamic dimensions not supported for the ReverseSequence operator.

We tried to build engine using onnx model you’ve shared, but facing following error.
TRT support tensor at most 2G elements.

[10/18/2021-16:55:07] [E] [TRT] ModelImporter.cpp:772: --- End node ---
[10/18/2021-16:55:07] [E] [TRT] ModelImporter.cpp:775: ERROR: ModelImporter.cpp:179 In function parseGraph:
[6] Invalid Node - StatefulPartitionedCall/model/bidirectional/forward_lstm/PartitionedCall/while_loop
[graphShapeAnalyzer.cpp::processCheck::582] Error Code 4: Internal Error ((Unnamed Layer* 60) [LoopOutput]_output: tensor volume exceeds (2^31)-1, dimensions are [2147483647,32,64])
[10/18/2021-16:55:07] [E] Failed to parse onnx file
[10/18/2021-16:55:07] [I] Finish parsing network model
[10/18/2021-16:55:07] [E] Parsing model failed
[10/18/2021-16:55:07] [E] Failed to create engine from model.
[10/18/2021-16:55:07] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8200] # /usr/src/tensorrt/bin/trtexec --onnx=bi_list_fixed.onnx --verbose --workspace=8000

Following similar issue may help you.

Thank you.