What could cause negative sized output bindings?

[brad.steiner1]

Description

I am trying to run inference using a trt engine, but when allocating buffers it is showing my output bindings as negative sized, and preventing me from running the engine. What could cause this?

Here is the snippet to get sizes:

        for binding in self.engine:
            print(binding)
            size = trt.volume(self.engine.get_tensor_shape(binding))
            print(size)

Output:

scores
-1
labels
-1
boxes
-4

Environment

TensorRT Version : 8.5
GPU Type : Jetson Xavier
Nvidia Driver Version :
CUDA Version : 11.4
CUDNN Version :
Operating System + Version : Ubuntu 20
Python Version (if applicable) : 3.8.10
TensorFlow Version (if applicable) :
PyTorch Version (if applicable) : 1.13
Baremetal or Container (if container which image + tag) :

[spolisetty]

Hi,

Could you please confirm that you using the dynamic shape?

Thank you.

[brad.steiner1]

I am not setting a dynamic shape explicitly, the input is a fixed size and batch size of 1. I can understand how the output size could be dynamic since there could be any number of detections in the image.

[spolisetty]

Hi,

This may happen due to dynamic dims, which are represented with -1.
To get the size of a tensor with dynamic dims, you will need to query the min/opt/max sizes by checking the optimization profiles.

Thank you.