Keras CRNN model conversion to tensorrt engine error

Description

tensorflow to onnx to trt engine conversion failure for crnn model with bidirectional layer with ReverseSequence Op on Jetson Xavier with trt version 8.0.1.6

Environment

TensorRT Version: ‘8.0.1.6’
GPU Type: Jetson Xavier
Nvidia Driver Version:
CUDA Version: 4.6-b197
CUDNN Version: 4.6-b197
Operating System + Version: Ubuntu 18.04
Python Version (if applicable): Python 3.6.9
TensorFlow Version (if applicable): 1.15.2
PyTorch Version (if applicable): -
Baremetal or Container (if container which image + tag): -

Relevant Files

Package: nvidia-jetpack
Version: 4.6-b197
Architecture: arm64
Maintainer: NVIDIA Corporation
Installed-Size: 194
Depends: nvidia-cuda (= 4.6-b197), nvidia-opencv (= 4.6-b197), nvidia-cudnn8 (= 4.6-b197), nvidia-tensorrt (= 4.6-b197), nvidia-visionworks (= 4.6-b197), nvidia-container (= 4.6-b197), nvidia-vpi (= 4.6-b197), nvidia-l4t-jetson-multimedia-api (>> 32.6-0), nvidia-l4t-jetson-multimedia-api (<< 32.7-0)
Homepage: Autonomous Machines | NVIDIA Developer
Priority: standard
Section: metapackages
Filename: pool/main/n/nvidia-jetpack/nvidia-jetpack_4.6-b197_arm64.deb

keras/tensorflow crnn model with bidirectional lstm layer

Steps To Reproduce

tensorrt engine conversion from onnx fails at ReverseSequence Op
[TensorRT] VERBOSE: lstm_10_back_reverse_seq_rev_seq [ReverseSequence] inputs: [lstm_10_back_transpose → (-1, 50, 128)[FLOAT]], [lstm_10_back_reverse_seq_expand0 → (-1)[INT32]],
[TensorRT] ERROR: 4: [network.cpp::validate::2411] Error Code 4: Internal Error (Network must have at least one output)
engine conversion failed
Even though Trt ‘8.0.1.6’ version supports ReverseSequence Op

Onnx model node names:
permute
permute/transpose:0_cropping
Transpose28
conv_1
Relu7
conv_2
Relu6
conv_3
Relu5
bn_3
bn_3/FusedBatchNormV3:0_pooling
conv_4
Relu4
conv_5
Relu3
bn_5
bn_5/FusedBatchNormV3:0_pooling
conv_6
Relu2
conv_7
Relu1
bn_7
Transpose9
reshape
fc_9
Add2
Relu
Transpose4
Transpose6
lstm_10
lstm_10_back
Squeeze2
Squeeze3
Transpose5
Transpose7
Shape1
Gather2
Gather3
lstm_10_back_reverse_seq_expand
lstm_10_back_reverse_seq_rev_seq
Add1
Transpose
Transpose2
lstm_11
lstm_11_back
Squeeze
Squeeze1
Transpose1
Transpose3
Shape
Gather
Gather1
lstm_11_back_reverse_seq_expand
lstm_11_back_reverse_seq_rev_seq
concatenate
fc_12
Add
Softmax
fc_12/truediv:0_cropping

Hi,
Request you to share the ONNX model and the script if not shared already so that we can assist you better.
Alongside you can try few things:

  1. validating your model with the below snippet

check_model.py

import sys
import onnx
filename = yourONNXmodel
model = onnx.load(filename)
onnx.checker.check_model(model).
2) Try running your model with trtexec command.

In case you are still facing issue, request you to share the trtexec “”–verbose"" log for further debugging
Thanks!

Thanks Nves for quick reply, I did run the 2 options you told me to perform.

  1. onnx.checker.check_model(model). - did not return any o/p
  2. ./trtexec failed with this error
    [04/06/2022-17:41:23] [E] [TRT] ModelImporter.cpp:720: While parsing node number 39 [ReverseSequence → “lstm_10_back_reverse_seq_rev_seq0”]:
    [04/06/2022-17:41:23] [E] [TRT] ModelImporter.cpp:721: — Begin node —
    [04/06/2022-17:41:23] [E] [TRT] ModelImporter.cpp:722: input: “lstm_10_back_transpose”
    input: “lstm_10_back_reverse_seq_expand0”
    output: “lstm_10_back_reverse_seq_rev_seq0”
    name: “lstm_10_back_reverse_seq_rev_seq”
    op_type: “ReverseSequence”
    attribute {
    name: “batch_axis”
    i: 0
    type: INT
    }
    attribute {
    name: “time_axis”
    i: 1
    type: INT
    }
    domain: “”

[04/06/2022-17:41:23] [E] [TRT] ModelImporter.cpp:723: — End node —
[04/06/2022-17:41:23] [E] [TRT] ModelImporter.cpp:726: ERROR: builtin_op_importers.cpp:3496 In function importReverseSequence:
[8] Assertion failed: size != -1 && “This version of TensorRT does not support dynamic ReverseSequence lengths!”
[04/06/2022-17:41:23] [E] Failed to parse onnx file
[04/06/2022-17:41:23] [I] Finish parsing network model
[04/06/2022-17:41:23] [E] Parsing model failed
[04/06/2022-17:41:23] [E] Engine creation failed
[04/06/2022-17:41:23] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec [TensorRT v8001] # ./trtexec --onnx=/home/nxdev8/vision-research-pranav/crnn_pretrained/crnn_pretrained_nx.onnx --verbose=True

Attaching onnx file
crnn_pretrained_nx.onnx (30.0 MB)

Hi,

Currently ReverseSequence Op, which as we state in the support document, does not yet support dynamic length inputs (i.e., lengths of the sequences in a batch, sequence_lens, cannot be dynamic). Maybe we can try for a static batch.

Thank you.