ROI Positioning Issue with nvdspreprocess and Mixed Resolutions Using New Streammux

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Dear NVIDIA Team and Community,

I’m encountering an issue with nvdspreprocess in a DeepStream pipeline where I’m using the new streammux (with USE_NEW_NVSTREAMMUX=yes).

Current Setup:

  • DeepStream 7.1
  • Pipeline with 2 video sources:
    • Source 0: 1280×720
    • Source 1: 640×320
  • Using nvdspreprocess for ROI (Region of Interest) processing
  • New streammux enabled (USE_NEW_NVSTREAMMUX=yes and NVSTREAMMUX_ADAPTIVE_BATCHING=yes)

Observed Behavior:

  • When displaying each stream individually, ROIs appear correctly positioned
  • When displaying both streams together (via tiler/multistream), the ROI for the 640×320 stream shifts position

Key Details:

  1. Using absolute coordinates in config_preprocess.txt
  2. Streammux batch size = number of sources (batch-size=2)
  3. Tried both with and without maintain-aspect-ratio=1

Configuration Snippet:

[property]
processing-width=640
processing-height=640
maintain-aspect-ratio=1

[group-0]
src-ids=0;1
process-on-roi=1
roi-params-src-0=150;110;850;500 # For 1280×720 source
roi-params-src-1=75;55;425;250 # Scaled coordinates for 640×320

[property] processing-width=640 processing-height=640 maintain-aspect-ratio=1 [group-0] src-ids=0;1 process-on-roi=1 roi-params-src-0=150;110;850;500 # For 1280×720 source roi-params-src-1=75;55;425;250 # Scaled coordinates for 640×320

Questions:

  1. Does nvdspreprocess fully support the new streammux architecture?
  2. What’s the recommended way to handle ROIs with mixed resolutions?
  3. Should ROI coordinates be normalized or maintained as absolute value
2

Any response?

3

Should ROI coordinates be normalized or maintained as absolute value??

4

Hi @amine.ghamgui.anavid ,
Thank you for reaching out, moving your query to Deepstream Forum.

Thanks

6

Yes

Generally speaking, you can directly form a batch. However, if you encounter problems, you can consider adding nvvideoconvert before new streammux to scale different resolutions to a unified one.

Refer to this link

maintained as absolute value

I did a test using /opt/nvidia/deepstream/deepstream/sources/apps/sample_apps/deepstream-preprocess-test and it works fine

modify config_preprocess.txt

[group-0]
src-ids=0;1
custom-input-transformation-function=CustomAsyncTransformation
process-on-roi=1
roi-params-src-0=100;100;200;200
roi-params-src-1=100;100;200;200

USE_NEW_NVSTREAMMUX=yes ./deepstream-preprocess-test config_preprocess.txt config_infer.txt file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_1080p_h264.mp4 file:///opt/nvidia/deepstream/deepstream/samples/streams/sample_720p.h264

Since the two sources have different resolutions and different ROIs are set, nvmutilstreamtiler will display the ROI in slightly different areas