Support of nvinfer for dynamic resolutions

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

• Hardware Platform: GPU
**• DeepStream Version: 6.3 **
**• TensorRT Version: 8.5.3 **
• NVIDIA GPU Driver Version: 555.42.06
• Issue Type( questions, new requirements, bugs)
Hello,

I’m manually optimizing my models using the following trtexec command:

trtexec \
    --workspace=3000 \
    --fp16 \
    --onnx=model.onnx \
    --minShapes=input:1x3x32x32 \
    --optShapes=input:1x3x1280x1280 \
    --maxShapes=input:1x3x1920x1920 \
    --buildOnly \
    --threads \
    --saveEngine=model.engine

With these settings, the engine supports dynamic input shapes. I’m using the same configuration with nvinfer in a custom setup, and it works correctly with multiple input resolutions.

Now, I want to achieve the same behavior within a DeepStream pipeline. I’ve optimized the model with dynamic shapes and switched to the new nvstreammux element to preserve the original resolution of each input stream without automatic resizing.

However, during post-processing, I noticed that even when the actual input resolution is, for example, 1280x720, nvinfer still resizes all inputs to the optimal shape (1280x1280 in this case). I also tried using the nvdspreprocess element, but it still performs resizing to the opt shape.

How can I configure DeepStream so that nvinfer accepts and processes inputs at their original resolutions without resizing them to the optimal shape?

3
  1. From the trtexec cmd, why will the shape change from 32x32 to 1920x1920? please refer this sample.
  2. how did you nvinfer resize the inputs from 1280x720 to 1280x1280? you can dump the media pipeline to check if the resolution changes. please refer to this faq for how to dump pipeline graph.
  3. the driver version is incompatible with DS6.3. please refer to the compatibility table.
4

Thanks for the sharing! are you using DeepStream nvinfer plugin? could you share the configuration file of nvinfer?

Currently, nvinfer will resize the resolution of input to the resolution of optShapes for inference. If you don’t want to resize in nvinfer. you can set the resolution of input to optShapes when converting engine with trtexec.
Or you can set infer-dims to specify the model input resolution. please refer to this sample.

5

Thank you for the fast answer!

Yes, I am using the nvinfer plugin. My configuration currently looks like this:

[property]
gpu-id=0
gie-unique-id=2
batch-size=1

model-engine-file=model.engine
network-mode=2
network-type=0
num-detected-classes=1
model-color-format=0

offsets=123.675;116.28;103.53
maintain-aspect-ratio=1
net-scale-factor=0.01735207
network-input-order=0

output-tensor-meta=1

That’s exactly what I’m trying to avoid. Since I already converted the engine with dynamic shapes, I want nvinfer to use the input as-is for inference, without resizing it to the optimal shape. Naturally, any input larger than the defined maxShape should be resized down. But for any input size within the range between minShape and maxShape, inference should work directly without additional resizing.

6

why do you try to avoid resizing? for accuracy?
DeepStream nvinfer calls TensorRT to do inference and nvinfer plugin and low-level lib are opensurce. If TensorRT supports this feature, you may modify nvinfer code to customize. please refer to initInferenceInfo in /opt/nvidia/deepstream/deepstream/sources/libs/nvdsinfer/nvdsinfer_context_impl.cpp, where m_NetworkInfo is set.

7

Thanks again for your detailed reply!

You’re right — the main reason I want to avoid resizing is to preserve accuracy , especially for aspect ratio–sensitive tasks. When the original resolution already falls within the [minShapes, maxShapes] range, I’d like to leverage TensorRT’s dynamic shape capabilities directly without any intermediate preprocessing that might distort input data or degrade model performance.

I’m aware that DeepStream wraps TensorRT, and I appreciate the pointer to the initInferenceInfo() function in nvdsinfer_context_impl.cpp . I’ll definitely take a closer look at how m_NetworkInfo is populated and see if there’s a way to bypass the resizing logic or conditionally apply it only for out-of-range inputs.

Ideally, it would be great if this behavior could be toggled via config (something like preserve-input-shape=1 ) — especially now that nvstreammux can handle per-stream resolution. For now, I’ll explore modifying the nvinfer plugin accordingly.

Thanks again for the support — and if you have any internal roadmap insights about planned improvements for this use case, I’d be curious to hear them!

Best regards,
kmtrn

8

please note that initInferenceInfo is called before pipeline negotiation, initInferenceInfo does not know the resolution of input source when calling.

9

There is no update from you for a period, assuming this is not an issue anymore. Hence we are closing this topic. If need further support, please open a new one. Thanks

10

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.