ROI Positioning Issue with nvdspreprocess and Mixed Resolutions Using New Streammux

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

I’m encountering an issue with nvdspreprocess in a DeepStream pipeline where I’m using the new streammux (with USE_NEW_NVSTREAMMUX=yes).

Current Setup:

  • DeepStream 7.1
  • Pipeline with 2 video sources:
    • Source 0: 1280×720
    • Source 1: 640×320
  • Using nvdspreprocess for ROI (Region of Interest) processing
  • New streammux enabled (USE_NEW_NVSTREAMMUX=yes and NVSTREAMMUX_ADAPTIVE_BATCHING=yes)

Observed Behavior:

  • When displaying each stream individually, ROIs appear correctly positioned
  • When displaying both streams together (via tiler/multistream), the ROI for the 640×320 stream shifts position

Key Details:

  1. Using absolute coordinates in config_preprocess.txt
  2. Streammux batch size = number of sources (batch-size=2)
  3. Tried both with and without maintain-aspect-ratio=1

Configuration Snippet:

[property]
processing-width=640
processing-height=640
maintain-aspect-ratio=1

[group-0]
src-ids=0;1
process-on-roi=1
roi-params-src-0=150;110;850;500 # For 1280×720 source
roi-params-src-1=75;55;425;250 # Scaled coordinates for 640×320

[property] processing-width=640 processing-height=640 maintain-aspect-ratio=1 [group-0] src-ids=0;1 process-on-roi=1 roi-params-src-0=150;110;850;500 # For 1280×720 source roi-params-src-1=75;55;425;250 # Scaled coordinates for 640×320

Questions:

  1. What’s the recommended way to handle ROIs with mixed resolutions?
  2. Should ROI coordinates be normalized or maintained as absolute value
3
  1. which sample are you testing or referring to? what is the complete media pipeline? new-streammux can’t handle different resolutions directly. Please refer to this heterogeneous-batching for how to handle different resolutions with new-streammux.
  2. what do you mean by "normalized " and “maintained as absolute value”?
4

I mean normalized so that each ROI configured for the same video, even with multiple resolutions, becomes consistent — similar to the property used in your plugin, nvdsanalytics.

5

Please refer to link in my last comment, if inputting multiple resolutions to new-streammux, please convert the resolutions to the same resolution with nvideocovnert. Is this still an DeepStream issue to support? Thanks!

6

If I apply a resolution resizing using the new nvstreammux, what is the added value compared to the old streammux? The main advantage of the new nvstreammux is its ability to preserve aspect ratio and dynamically handle multiple input resolutions.??

7

new-streammux does not support resizing. you need to use nvvideocovnert to do resizing before new-streammux if inputting different resolutions. About the other two questions, please refer to new-streammux for the added value and new features.