`std::out_of_range` converting `.onnx` to engine

Autonomous Machines Jetson & Embedded Systems Jetson Xavier NX
1

I have a .onnx model in NCHW format. But when I use it with DeepStream or try to convert it manually withtrtexec, I’m getting an out of range error. Input shape is 1x3x299x299.

...
----------------------------------------------------------------
Input filename:   qa_edge_latest.onnx
ONNX IR version:  0.0.7
Opset version:    13
Producer name:    tf2onnx
Producer version: 1.9.1
Domain:
Model version:    0
Doc string:
----------------------------------------------------------------
[08/19/2021-16:31:33] [W] [TRT] onnx2trt_utils.cpp:220: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
terminate called after throwing an instance of 'std::out_of_range'
  what():  Attribute not found: axes
Aborted

Any ideas what might be causing this?

3

Hi,

The error indicates an expected attribute (axis) doesn’t present.
Could you run it with --verbose so we know which layer causes this issue?

Thanks.

4

Thanks! That’s very helpful!

...
[09/10/2021-12:45:23] [V] [TRT] ModelImporter.cpp:103: Parsing node: StatefulPartitionedCall/model_1/mixed10/concat [Concat]
[09/10/2021-12:45:23] [V] [TRT] ModelImporter.cpp:119: Searching for input: StatefulPartitionedCall/model_1/activation_86/Relu:0
[09/10/2021-12:45:23] [V] [TRT] ModelImporter.cpp:119: Searching for input: StatefulPartitionedCall/model_1/activation_88/Relu:0
[09/10/2021-12:45:23] [V] [TRT] ModelImporter.cpp:119: Searching for input: StatefulPartitionedCall/model_1/activation_89/Relu:0
[09/10/2021-12:45:23] [V] [TRT] ModelImporter.cpp:119: Searching for input: StatefulPartitionedCall/model_1/activation_92/Relu:0
[09/10/2021-12:45:23] [V] [TRT] ModelImporter.cpp:119: Searching for input: StatefulPartitionedCall/model_1/activation_93/Relu:0
[09/10/2021-12:45:23] [V] [TRT] ModelImporter.cpp:119: Searching for input: StatefulPartitionedCall/model_1/activation_94/Relu:0
[09/10/2021-12:45:23] [V] [TRT] ModelImporter.cpp:125: StatefulPartitionedCall/model_1/mixed10/concat [Concat] inputs: [StatefulPartitionedCall/model_1/activation_86/Relu:0 -> (1, 320, 8, 8)], [StatefulPartitionedCall/model_1/activation_88/Relu:0 -> (1, 384, 8, 8)], [StatefulPartitionedCall/model_1/activation_89/Relu:0 -> (1, 384, 8, 8)], [StatefulPartitionedCall/model_1/activation_92/Relu:0 -> (1, 384, 8, 8)], [StatefulPartitionedCall/model_1/activation_93/Relu:0 -> (1, 384, 8, 8)], [StatefulPartitionedCall/model_1/activation_94/Relu:0 -> (1, 192, 8, 8)], 
[09/10/2021-12:45:23] [V] [TRT] ImporterContext.hpp:141: Registering layer: StatefulPartitionedCall/model_1/mixed10/concat for ONNX node: StatefulPartitionedCall/model_1/mixed10/concat
[09/10/2021-12:45:23] [V] [TRT] ImporterContext.hpp:116: Registering tensor: StatefulPartitionedCall/model_1/mixed10/concat:0 for ONNX tensor: StatefulPartitionedCall/model_1/mixed10/concat:0
[09/10/2021-12:45:23] [V] [TRT] ModelImporter.cpp:179: StatefulPartitionedCall/model_1/mixed10/concat [Concat] outputs: [StatefulPartitionedCall/model_1/mixed10/concat:0 -> (1, 2048, 8, 8)], 
[09/10/2021-12:45:23] [V] [TRT] ModelImporter.cpp:103: Parsing node: StatefulPartitionedCall/model_1/sequential_4/global_average_pooling2d_1/Mean [GlobalAveragePool]
[09/10/2021-12:45:23] [V] [TRT] ModelImporter.cpp:119: Searching for input: StatefulPartitionedCall/model_1/mixed10/concat:0
[09/10/2021-12:45:23] [V] [TRT] ModelImporter.cpp:125: StatefulPartitionedCall/model_1/sequential_4/global_average_pooling2d_1/Mean [GlobalAveragePool] inputs: [StatefulPartitionedCall/model_1/mixed10/concat:0 -> (1, 2048, 8, 8)], 
[09/10/2021-12:45:23] [V] [TRT] ImporterContext.hpp:141: Registering layer: StatefulPartitionedCall/model_1/sequential_4/global_average_pooling2d_1/Mean for ONNX node: StatefulPartitionedCall/model_1/sequential_4/global_average_pooling2d_1/Mean
[09/10/2021-12:45:23] [V] [TRT] ImporterContext.hpp:116: Registering tensor: StatefulPartitionedCall/model_1/sequential_4/global_average_pooling2d_1/Mean:0 for ONNX tensor: StatefulPartitionedCall/model_1/sequential_4/global_average_pooling2d_1/Mean:0
[09/10/2021-12:45:23] [V] [TRT] ModelImporter.cpp:179: StatefulPartitionedCall/model_1/sequential_4/global_average_pooling2d_1/Mean [GlobalAveragePool] outputs: [StatefulPartitionedCall/model_1/sequential_4/global_average_pooling2d_1/Mean:0 -> (1, 2048, 1, 1)], 
[09/10/2021-12:45:23] [V] [TRT] ModelImporter.cpp:103: Parsing node: StatefulPartitionedCall/model_1/sequential_4/global_average_pooling2d_1/Mean_Squeeze__496 [Squeeze]
[09/10/2021-12:45:23] [V] [TRT] ModelImporter.cpp:119: Searching for input: StatefulPartitionedCall/model_1/sequential_4/global_average_pooling2d_1/Mean:0
[09/10/2021-12:45:23] [V] [TRT] ModelImporter.cpp:119: Searching for input: const_axes__495
[09/10/2021-12:45:23] [V] [TRT] ModelImporter.cpp:125: StatefulPartitionedCall/model_1/sequential_4/global_average_pooling2d_1/Mean_Squeeze__496 [Squeeze] inputs: [StatefulPartitionedCall/model_1/sequential_4/global_average_pooling2d_1/Mean:0 -> (1, 2048, 1, 1)], [const_axes__495 -> (2)], 
terminate called after throwing an instance of 'std::out_of_range'
  what():  Attribute not found: axes
Searching for input: const_axes__495
[09/10/2021-12:45:23] [V] [TRT] ModelImporter.cpp:125: StatefulPartitionedCall/model_1/sequential_4/global_average_pooling2d_1/Mean_Squeeze__496 [Squeeze] inputs: [StatefulPartitionedCall/model_1/sequential_4/global_average_pooling2d_1/Mean:0 -> (1, 2048, 1, 1)], [const_axes__495 -> (2)], 
terminate called after throwing an instance of 'std::out_of_range'
  what():  Attribute not found: axes

Searching for input: const_axes__495
[09/10/2021-12:45:23] [V] [TRT] ModelImporter.cpp:125: StatefulPartitionedCall/model_1/sequential_4/global_average_pooling2d_1/Mean_Squeeze__496 [Squeeze] inputs: [StatefulPartitionedCall/model_1/sequential_4/global_average_pooling2d_1/Mean:0 -> (1, 2048, 1, 1)], [const_axes__495 -> (2)], 
terminate called after throwing an instance of 'std::out_of_range'
  what():  Attribute not found: axes

Searching for input: const_axes__495
[09/10/2021-12:45:23] [V] [TRT] ModelImporter.cpp:125: StatefulPartitionedCall/model_1/sequential_4/global_average_pooling2d_1/Mean_Squeeze__496 [Squeeze] inputs: [StatefulPartitionedCall/model_1/sequential_4/global_average_pooling2d_1/Mean:0 -> (1, 2048, 1, 1)], [const_axes__495 -> (2)], 
terminate called after throwing an instance of 'std::out_of_range'
  what():  Attribute not found: axes

Aborted (core dumped)

So, seems it’s GlobalAveragePool? Any advice I can relay back to the data scientist who created the model, or is this fixable with some conversion flags?

5

Interesting. Same .onnx works fine on x86 with TensorRT. This seems to be a Tegra specific issue. @AastaLLL have you had a chance to look at the file I sent via pm?

6

Hi,

Sorry for the delay.

Just check the NCHW-based model from the PM.
It seems that the issue comes from the different TensorRT versions.

We test this model with JetPack4.5.1 (TensorRT v7.1.3) and meet the same axes error.
But if we try with the latest JetPack4.6 (TensorRT 8.0), the model can be converted successfully.

It seems there are some related issues fix or features enhancement in the latest release.
Would you mind also giving it a try?

Thanks.

7

Thanks for looking into it! I suspected it might have something to do with the TRT version. Unfortunately there’s currently no DeepStream for JP 4.6. I’ve let my client know. We’re going to do inference in the cloud for now since capture and the rest of it already works for our product. We’ll revisit if and when DeepStream is updated.