Assertion failed: stride overflow

alok.deshpande · April 20, 2021, 3:13pm

I am receiving an error similar to ERROR: StatefulPartitionedCall/functional_5/lstm_1/PartitionedCall/while_loop:7: region stride in units of elements overflows

This occurs while generating an engine from an ONNX model file. The ONNX model was generated with tf2onnx from a TF2 SavedModel.

I have attached the ONNX model for your inspection. I note that TRT’s ONNX parsing succeeds and the ONNX checker reports success, so I don’t think there’s a problem with the conversion to ONNX.

Description

Below is the error:

[04/20/2021-10:56:37] [V] [TRT] Constructing optimization profile number 0 [1/1].
[04/20/2021-10:56:37] [E] [TRT] dense: region stride in units of elements overflows int32_t. Region dimensions are [1,2147483647,4].
[04/20/2021-10:56:37] [F] [TRT] Assertion failed: stride overflow
../builder/tacticOptimizer.cpp:2734
Aborting...
[04/20/2021-10:56:37] [V] [TRT] Builder timing cache: created 0 entries, 0 hit(s)
[04/20/2021-10:56:37] [E] [TRT] ../builder/tacticOptimizer.cpp (2734) - Assertion Error in accumulatePitch: 0 (stride overflow)
[04/20/2021-10:56:37] [E] Engine creation failed
[04/20/2021-10:56:37] [E] Engine set up failed
&&&& FAILED TensorRT.trtexec # trtexec --explicitBatch --onnx=./model.onnx --saveEngine=engine.trt --workspace=100 --verbose --minShapes='input_1:0':1x20x12 --optShapes='input_1:0':1x20x12 --maxShapes='input_1:0':1x20x12

Environment

TensorRT Version: 7.2.3.4
GPU Type: Quadro P520
Nvidia Driver Version: 460.56
CUDA Version: 11.2
CUDNN Version: 8.1.1
Operating System + Version: Ubuntu 20.04 x86-64
Python Version (if applicable): 3.8.5
TensorFlow Version (if applicable): 2.3.0
PyTorch Version (if applicable):
Baremetal or Container (if container which image + tag):
TF2ONNX version: 1.8.4
ONNX opset: 11

Relevant Files

ONNX attached.model.onnx (144.7 KB)

Steps To Reproduce

I attempted to build using trtexec:

trtexec --explicitBatch --onnx=./model.onnx --saveEngine=engine.trt --workspace=100 --verbose --minShapes=\'input_1:0\':1x20x12 --optShapes=\'input_1:0\':1x20x12 --maxShapes=\'input_1:0\':1x20x12

Please let me know if I can provide more details. Thanks for your time.

NVES · April 20, 2021, 6:50pm

Hi,
We recommend you to check the below samples links, as they might answer your concern

If issue persist, request you to share the model and script so that we can try reproducing the issue at our end.
Thanks!

alok.deshpande · April 20, 2021, 7:46pm

Thank you for your quick response.

Unfortunately all of these samples use TF-TRT.

I will be ultimately trying to build engines on the Jetson AGX Xavier with this ONNX model, although I am currently building on a laptop. As far as I am aware, there are many disadvantages to using TF-TRT, such as the high runtime memory usage and disk space required for installing TensorFlow.

I also do not believe it is also possible to directly generate .plan files for models generated with TensorFlow 2. This is based on my tests with the Python TF-TRT API, and fact that the parameter is_dynamic_op=True for TensorFlow 2. Please correct me if I am mistaken, as that would be a suitable workaround.

Is it possible to generate a pure TensorRT engine for this ONNX model? I have shared the model and script (single command) so you may try to reproduce the issue.

spolisetty · April 22, 2021, 5:46am

Hi @alok.deshpande,

We could reproduce the same error. Please allow us some time to work on this.
Meanwhile could you please confirm are you able to use onnx model for inference using onnx-runtime without any errors ?

Thank you.

alok.deshpande · April 22, 2021, 12:47pm

Hi @spolisetty,

No problem. Thank you for keeping me updated.

Yes, I have no problem with ONNX inference. Here is a sample script:

import onnxruntime as nxrun
import numpy as np

sess = nxrun.InferenceSession(“./model.onnx”)

input_name = sess.get_inputs()[0].name

print("The model expects input shape: ", sess.get_inputs()[0].shape)

x=np.zeros((1,20,12),dtype=‘f’)

result = sess.run(None, {input_name: x})

print("Result of inference is ", result)

jeichel · April 30, 2021, 6:01pm

I’m also experiencing the same issue. Everything works fine with ONNX inference.

alok.deshpande · May 4, 2021, 1:04pm

Hi @spolisetty ,

Would you happen to have any update on this? Please let us know whenever you do.

msripooja · June 16, 2021, 10:17am

Hi @NVES @spolisetty ,

I am facing the same issue. I simplified my algorithm to maximum possible extent. No matter which approach I use for generating the TRT engine i.e., using TensorRT python API or using trtexec, I see the same error eventually after resolving all other conversion issues.

Here is the error output:

I simplified the ONNX model using ONNX-simplifier but the same error persists.

Please suggest a workaround, I did not find any solution in other discussion threads.

alok.deshpande · December 2, 2021, 6:09am

Just so you are aware, this is still a problem as of TensorRT V8.0.1.

Any update @NVES @spolisetty? Thanks for your time.

spolisetty · December 23, 2021, 6:28am

Hi @alok.deshpande, @msripooja, @jeichel,

This issue will be fixed in future releases. Currently, we are not aware of ETA.

Thank you.