`std::out_of_range` converting `.onnx` to engine

I have a .onnx model in NCHW format. But when I use it with DeepStream or try to convert it manually withtrtexec, I’m getting an out of range error. Input shape is 1x3x299x299.

...
----------------------------------------------------------------
Input filename:   qa_edge_latest.onnx
ONNX IR version:  0.0.7
Opset version:    13
Producer name:    tf2onnx
Producer version: 1.9.1
Domain:
Model version:    0
Doc string:
----------------------------------------------------------------
[08/19/2021-16:31:33] [W] [TRT] onnx2trt_utils.cpp:220: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32.
terminate called after throwing an instance of 'std::out_of_range'
  what():  Attribute not found: axes
Aborted

Any ideas what might be causing this?

Hi,

The error indicates an expected attribute (axis) doesn’t present.
Could you run it with --verbose so we know which layer causes this issue?

Thanks.

Thanks! That’s very helpful!

...
[09/10/2021-12:45:23] [V] [TRT] ModelImporter.cpp:103: Parsing node: StatefulPartitionedCall/model_1/mixed10/concat [Concat]
[09/10/2021-12:45:23] [V] [TRT] ModelImporter.cpp:119: Searching for input: StatefulPartitionedCall/model_1/activation_86/Relu:0
[09/10/2021-12:45:23] [V] [TRT] ModelImporter.cpp:119: Searching for input: StatefulPartitionedCall/model_1/activation_88/Relu:0
[09/10/2021-12:45:23] [V] [TRT] ModelImporter.cpp:119: Searching for input: StatefulPartitionedCall/model_1/activation_89/Relu:0
[09/10/2021-12:45:23] [V] [TRT] ModelImporter.cpp:119: Searching for input: StatefulPartitionedCall/model_1/activation_92/Relu:0
[09/10/2021-12:45:23] [V] [TRT] ModelImporter.cpp:119: Searching for input: StatefulPartitionedCall/model_1/activation_93/Relu:0
[09/10/2021-12:45:23] [V] [TRT] ModelImporter.cpp:119: Searching for input: StatefulPartitionedCall/model_1/activation_94/Relu:0
[09/10/2021-12:45:23] [V] [TRT] ModelImporter.cpp:125: StatefulPartitionedCall/model_1/mixed10/concat [Concat] inputs: [StatefulPartitionedCall/model_1/activation_86/Relu:0 -> (1, 320, 8, 8)], [StatefulPartitionedCall/model_1/activation_88/Relu:0 -> (1, 384, 8, 8)], [StatefulPartitionedCall/model_1/activation_89/Relu:0 -> (1, 384, 8, 8)], [StatefulPartitionedCall/model_1/activation_92/Relu:0 -> (1, 384, 8, 8)], [StatefulPartitionedCall/model_1/activation_93/Relu:0 -> (1, 384, 8, 8)], [StatefulPartitionedCall/model_1/activation_94/Relu:0 -> (1, 192, 8, 8)], 
[09/10/2021-12:45:23] [V] [TRT] ImporterContext.hpp:141: Registering layer: StatefulPartitionedCall/model_1/mixed10/concat for ONNX node: StatefulPartitionedCall/model_1/mixed10/concat
[09/10/2021-12:45:23] [V] [TRT] ImporterContext.hpp:116: Registering tensor: StatefulPartitionedCall/model_1/mixed10/concat:0 for ONNX tensor: StatefulPartitionedCall/model_1/mixed10/concat:0
[09/10/2021-12:45:23] [V] [TRT] ModelImporter.cpp:179: StatefulPartitionedCall/model_1/mixed10/concat [Concat] outputs: [StatefulPartitionedCall/model_1/mixed10/concat:0 -> (1, 2048, 8, 8)], 
[09/10/2021-12:45:23] [V] [TRT] ModelImporter.cpp:103: Parsing node: StatefulPartitionedCall/model_1/sequential_4/global_average_pooling2d_1/Mean [GlobalAveragePool]
[09/10/2021-12:45:23] [V] [TRT] ModelImporter.cpp:119: Searching for input: StatefulPartitionedCall/model_1/mixed10/concat:0
[09/10/2021-12:45:23] [V] [TRT] ModelImporter.cpp:125: StatefulPartitionedCall/model_1/sequential_4/global_average_pooling2d_1/Mean [GlobalAveragePool] inputs: [StatefulPartitionedCall/model_1/mixed10/concat:0 -> (1, 2048, 8, 8)], 
[09/10/2021-12:45:23] [V] [TRT] ImporterContext.hpp:141: Registering layer: StatefulPartitionedCall/model_1/sequential_4/global_average_pooling2d_1/Mean for ONNX node: StatefulPartitionedCall/model_1/sequential_4/global_average_pooling2d_1/Mean
[09/10/2021-12:45:23] [V] [TRT] ImporterContext.hpp:116: Registering tensor: StatefulPartitionedCall/model_1/sequential_4/global_average_pooling2d_1/Mean:0 for ONNX tensor: StatefulPartitionedCall/model_1/sequential_4/global_average_pooling2d_1/Mean:0
[09/10/2021-12:45:23] [V] [TRT] ModelImporter.cpp:179: StatefulPartitionedCall/model_1/sequential_4/global_average_pooling2d_1/Mean [GlobalAveragePool] outputs: [StatefulPartitionedCall/model_1/sequential_4/global_average_pooling2d_1/Mean:0 -> (1, 2048, 1, 1)], 
[09/10/2021-12:45:23] [V] [TRT] ModelImporter.cpp:103: Parsing node: StatefulPartitionedCall/model_1/sequential_4/global_average_pooling2d_1/Mean_Squeeze__496 [Squeeze]
[09/10/2021-12:45:23] [V] [TRT] ModelImporter.cpp:119: Searching for input: StatefulPartitionedCall/model_1/sequential_4/global_average_pooling2d_1/Mean:0
[09/10/2021-12:45:23] [V] [TRT] ModelImporter.cpp:119: Searching for input: const_axes__495
[09/10/2021-12:45:23] [V] [TRT] ModelImporter.cpp:125: StatefulPartitionedCall/model_1/sequential_4/global_average_pooling2d_1/Mean_Squeeze__496 [Squeeze] inputs: [StatefulPartitionedCall/model_1/sequential_4/global_average_pooling2d_1/Mean:0 -> (1, 2048, 1, 1)], [const_axes__495 -> (2)], 
terminate called after throwing an instance of 'std::out_of_range'
  what():  Attribute not found: axes
Searching for input: const_axes__495
[09/10/2021-12:45:23] [V] [TRT] ModelImporter.cpp:125: StatefulPartitionedCall/model_1/sequential_4/global_average_pooling2d_1/Mean_Squeeze__496 [Squeeze] inputs: [StatefulPartitionedCall/model_1/sequential_4/global_average_pooling2d_1/Mean:0 -> (1, 2048, 1, 1)], [const_axes__495 -> (2)], 
terminate called after throwing an instance of 'std::out_of_range'
  what():  Attribute not found: axes

Searching for input: const_axes__495
[09/10/2021-12:45:23] [V] [TRT] ModelImporter.cpp:125: StatefulPartitionedCall/model_1/sequential_4/global_average_pooling2d_1/Mean_Squeeze__496 [Squeeze] inputs: [StatefulPartitionedCall/model_1/sequential_4/global_average_pooling2d_1/Mean:0 -> (1, 2048, 1, 1)], [const_axes__495 -> (2)], 
terminate called after throwing an instance of 'std::out_of_range'
  what():  Attribute not found: axes

Searching for input: const_axes__495
[09/10/2021-12:45:23] [V] [TRT] ModelImporter.cpp:125: StatefulPartitionedCall/model_1/sequential_4/global_average_pooling2d_1/Mean_Squeeze__496 [Squeeze] inputs: [StatefulPartitionedCall/model_1/sequential_4/global_average_pooling2d_1/Mean:0 -> (1, 2048, 1, 1)], [const_axes__495 -> (2)], 
terminate called after throwing an instance of 'std::out_of_range'
  what():  Attribute not found: axes

Aborted (core dumped)

So, seems it’s GlobalAveragePool? Any advice I can relay back to the data scientist who created the model, or is this fixable with some conversion flags?

Interesting. Same .onnx works fine on x86 with TensorRT. This seems to be a Tegra specific issue. @AastaLLL have you had a chance to look at the file I sent via pm?