Assertion failed: stride overflow

I am receiving an error similar to ERROR: StatefulPartitionedCall/functional_5/lstm_1/PartitionedCall/while_loop:7: region stride in units of elements overflows

This occurs while generating an engine from an ONNX model file. The ONNX model was generated with tf2onnx from a TF2 SavedModel.

I have attached the ONNX model for your inspection. I note that TRT’s ONNX parsing succeeds and the ONNX checker reports success, so I don’t think there’s a problem with the conversion to ONNX.

Description

Below is the error:

[04/20/2021-10:56:37] [V] [TRT] Constructing optimization profile number 0 [1/1].
[04/20/2021-10:56:37] [E] [TRT] dense: region stride in units of elements overflows int32_t. Region dimensions are [1,2147483647,4].
[04/20/2021-10:56:37] [F] [TRT] Assertion failed: stride overflow
../builder/tacticOptimizer.cpp:2734
Aborting...
[04/20/2021-10:56:37] [V] [TRT] Builder timing cache: created 0 entries, 0 hit(s)
[04/20/2021-10:56:37] [E] [TRT] ../builder/tacticOptimizer.cpp (2734) - Assertion Error in accumulatePitch: 0 (stride overflow)
[04/20/2021-10:56:37] [E] Engine creation failed
[04/20/2021-10:56:37] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec # trtexec --explicitBatch --onnx=./model.onnx --saveEngine=engine.trt --workspace=100 --verbose --minShapes='input_1:0':1x20x12 --optShapes='input_1:0':1x20x12 --maxShapes='input_1:0':1x20x12

Environment

TensorRT Version: 7.2.3.4
GPU Type: Quadro P520
Nvidia Driver Version: 460.56
CUDA Version: 11.2
CUDNN Version: 8.1.1
Operating System + Version: Ubuntu 20.04 x86-64
Python Version (if applicable): 3.8.5
TensorFlow Version (if applicable): 2.3.0
PyTorch Version (if applicable):
Baremetal or Container (if container which image + tag):
TF2ONNX version: 1.8.4
ONNX opset: 11

Relevant Files

ONNX attached.model.onnx (144.7 KB)

Steps To Reproduce

I attempted to build using trtexec:

trtexec --explicitBatch --onnx=./model.onnx --saveEngine=engine.trt --workspace=100 --verbose --minShapes=\'input_1:0\':1x20x12 --optShapes=\'input_1:0\':1x20x12 --maxShapes=\'input_1:0\':1x20x12

Please let me know if I can provide more details. Thanks for your time.

Hi,
We recommend you to check the below samples links, as they might answer your concern
https://docs.nvidia.com/deeplearning/frameworks/tf-trt-user-guide/index.html#samples
https://docs.nvidia.com/deeplearning/tensorrt/archives/tensorrt-722/quick-start-guide/index.html#framework-integration
https://docs.nvidia.com/deeplearning/frameworks/tf-trt-user-guide/index.html#integrate-ovr
https://docs.nvidia.com/deeplearning/frameworks/tf-trt-user-guide/index.html#usingtftrt

If issue persist, request you to share the model and script so that we can try reproducing the issue at our end.
Thanks!

Thank you for your quick response.

Unfortunately all of these samples use TF-TRT.

I will be ultimately trying to build engines on the Jetson AGX Xavier with this ONNX model, although I am currently building on a laptop. As far as I am aware, there are many disadvantages to using TF-TRT, such as the high runtime memory usage and disk space required for installing TensorFlow.

I also do not believe it is also possible to directly generate .plan files for models generated with TensorFlow 2. This is based on my tests with the Python TF-TRT API, and fact that the parameter is_dynamic_op=True for TensorFlow 2. Please correct me if I am mistaken, as that would be a suitable workaround.

Is it possible to generate a pure TensorRT engine for this ONNX model? I have shared the model and script (single command) so you may try to reproduce the issue.

Hi @alok.deshpande,

We could reproduce the same error. Please allow us some time to work on this.
Meanwhile could you please confirm are you able to use onnx model for inference using onnx-runtime without any errors ?

Thank you.

Hi @spolisetty,

No problem. Thank you for keeping me updated.

Yes, I have no problem with ONNX inference. Here is a sample script:

import onnxruntime as nxrun
import numpy as np

sess = nxrun.InferenceSession("./model.onnx")

input_name = sess.get_inputs()[0].name

print("The model expects input shape: ", sess.get_inputs()[0].shape)

x=np.zeros((1,20,12),dtype=‘f’)

result = sess.run(None, {input_name: x})

print("Result of inference is ", result)

I’m also experiencing the same issue. Everything works fine with ONNX inference.

Hi @spolisetty ,

Would you happen to have any update on this? Please let us know whenever you do.

Hi @NVES @spolisetty ,

I am facing the same issue. I simplified my algorithm to maximum possible extent. No matter which approach I use for generating the TRT engine i.e., using TensorRT python API or using trtexec, I see the same error eventually after resolving all other conversion issues.

Here is the error output:

I simplified the ONNX model using ONNX-simplifier but the same error persists.

Please suggest a workaround, I did not find any solution in other discussion threads.