IRecurrenceLayer cannot be used to compute a shape tensor)

AI & Data Science Deep Learning (Training & Inference) TensorRT
1

Description

I used trtexec to load onnx model produced by tf2onnx, but the execution is failed:

[08/18/2021-21:47:20] [V] [TRT] Searching for input: rnn_2_1/gru2/while/strided_slice:0
[08/18/2021-21:47:20] [E] Error[9]: [graph.cpp::computeInputExecutionUses::510] Error Code 9: Internal Error ((Unnamed Layer* 355) [Recurrence]: IRecurrenceLayer cannot be used to compute a shape tensor)
[08/18/2021-21:47:20] [V] [TRT] rnn_2_1/gru2/while/strided_slice__230 [Squeeze] inputs: [rnn_2_1/gru2/while/strided_slice:0 → ()[FLOAT]],
[08/18/2021-21:47:20] [E] Error[9]: [graph.cpp::computeInputExecutionUses::510] Error Code 9: Internal Error ((Unnamed Layer* 355) [Recurrence]: IRecurrenceLayer cannot be used to compute a shape tensor)
[08/18/2021-21:47:20] [E] [TRT] /root/code/code/TensorRT/parsers/onnx/ModelImporter.cpp:725: While parsing node number 119 [Scan → “custom_rnn_scan_Scan__61:0”]:
[08/18/2021-21:47:20] [E] [TRT] /root/code/code/TensorRT/parsers/onnx/ModelImporter.cpp:726: — Begin node —
[08/18/2021-21:47:20] [E] [TRT] /root/code/code/TensorRT/parsers/onnx/ModelImporter.cpp:727: input: “rnn_2_1/gru2/VecAttGRUCellZeroState/zeros:0”

Environment

TensorRT Version: 8.0
GPU Type: T4-8C
Nvidia Driver Version: 450.102.04
CUDA Version: 11.0
CUDNN Version: 7.0
Operating System + Version: Linux VM-116-42-centos 3.10.107-1-tlinux2_kvm_guest-004

Relevant Files

截屏2021-08-18 下午10.01.44
截屏2021-08-18 下午10.01.441532×927 68.6 KB

full logs:
a.txt (194.4 KB)

Steps To Reproduce

python -m tf2onnx.convert --saved-model ./ --output model_11.onnx --opset 11
./trtexec --onnx=model_11.onnx --verbose

about my model:

According to the logs, the error occured when parsing custom_rnn_scan_Scan__61:0. I guess the meaning is that the output of this node cannot be used as shape(such as reshape OP), but the upcoming nodes are almost Matmul or Bias (without any shape OP).

Besides, I have tried almost all TensorRT versions( 7.1 7.2 8.0), and tried the ONNX IR versions with 8, 9, 11 and 13. However, the errors still exists.

Can anybody provide some tips about it?

2

Hi,
Request you to share the ONNX model and the script if not shared already so that we can assist you better.
Alongside you can try few things:

  1. validating your model with the below snippet

check_model.py

import sys
import onnx
filename = yourONNXmodel
model = onnx.load(filename)
onnx.checker.check_model(model).
2) Try running your model with trtexec command.
https://github.com/NVIDIA/TensorRT/tree/master/samples/opensource/trtexec
In case you are still facing issue, request you to share the trtexec “”–verbose"" log for further debugging
Thanks!

4

Thank you for suggestion! I have tried the check_model.py，and every thing is OK( without any errors or warnings). I also run the onnx model in onnx-runtime, and everything is OK too.

The full log of trtexec command is attached in previous post naming “a.log”.

I would share my onnx model as below:model_11.onnx (2.7 MB)

5

Hi @vivienfanghua,

We could reproduce the issue. Please allow us sometime to get back on this.

Thank you.

6

Hi,

TRT does not support shapes that are computed using loops/recurrences.

Thank you.