Nvvideoconvert -> Nvinfer Aspect Ratio Change

[Y-T-G]

I want to know if I resize the video using nvvideoconvert before nvinfer, does nvinfer use the original aspect ratio when resizing it again to the network input.

Let’s say the original aspect ratio was 4:3, nvvideoconvert resized it to a resolution with aspect ratio 16:9. When nvinfer is used with maintain-aspect-ratio: 1, will it resize to the aspect ratio 4:3 or 16:9?

The pipeline also has nvstreammux. Does that affect the aspect ratio too based on the width and height property:

nvurisrcbin (4:3) → nvvideoconvert (16:9) → nvstreammux (?) → nvinfer (?)

[fanzh]

Please provide complete information as applicable to your setup.
• Hardware Platform (Jetson / GPU)
• DeepStream Version
• JetPack Version (valid for Jetson only)
• TensorRT Version
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type( questions, new requirements, bugs)
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

[Y-T-G]

@fanzh

• Hardware Platform (Jetson / GPU)
• DeepStream Version
• JetPack Version (valid for Jetson only)
• TensorRT Version
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type( questions, new requirements, bugs)
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

• Hardware Platform GPU
• DeepStream Version 6.2
• TensorRT Version 8.5.2
• NVIDIA GPU Driver Version 525.60.15
• Issue Type question

[fanzh]

@Y-T-G

will it resize to the aspect ratio 4:3 or 16:9?

it resize to the aspect ratio 16:9 because nvvideoconvert outputs aspect ratio 16:9.

@Y-T-G

reammux (?) → nvinfer (?)

it is configurable. please refer to the doc… The muxer scales all input frames to this resolution. The enable-padding property can be set to true to preserve the input aspect ratio while scaling by padding with black bands.

[Y-T-G]

@fanzh

The enable-padding property can be set to true to preserve the input aspect ratio while scaling by padding with black bands.

Does that mean if I don’t have enable-padding set to true the maintain-aspect-ratio: 1 in the nvinfer config is useless?

[fanzh]

enable-padding is only the property of nvstreammux. nvinfer will scale the frame to the size of model. it has own maintain-aspect-ratio property. you can find in the explanation doc. BTW, nvinfer is opensource. you can find the maintain-aspect-ratio logics in get_converted_buffer.

[Y-T-G]

But the width and height scaling from nvstreammux will affect downstream elements wouldn’t it?

If my pipeline has:
source: Resolution R1 (4:3)
nvstreammux: Resolution R2 as per width and height (16:9)
nvinfer: Resolution R3 (1:1)

Since nvstreammux didn’t preserve the aspect ratio going from (4:3) to (16:9), it will affect nvinfer too correct? nvinfer will preserve the aspect ratio of nvstreammux and not the source?

[fanzh]

@Y-T-G

source: Resolution R1 (4:3)
nvstreammux: Resolution R2 as per width and height (16:9)
nvinfer: Resolution R3 (1:1)

In this case, nvsterammux can preserve the aspect ratio 4:3 by setting enable-padding to 1. and the nvstreammux 's output is 16:9 because width :height is 16:9.
if maintain-aspect-ratio is 1, nvinfer will pad image to output 1:1.
you can set nvstreammux’s width:hieght to 1:1 and enable-padding to 1.