Disconnected RTSP streams reduce the frame rate of other streams

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU)
2070 Super
• DeepStream Version
6.1

• JetPack Version (valid for Jetson only)
N/A

• TensorRT Version
8.2.5.1

• NVIDIA GPU Driver Version (valid for GPU only)
515.48.07

• Issue Type( questions, new requirements, bugs)
Bug

• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
Stream from multiple RTSP cameras and disconnect one or more cameras.

• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)

When streaming from RTSP cameras and one (or more) cameras disconnect all the other streams are capped at 4.79 FPS. Packets are still coming in at normal frame rate, but nvstreammux appear to only push out frames at a very limited rate.

Here’s sample output from deepstream-app when power cycling one camera 1:

**PERF:  FPS 0 (Avg)	FPS 1 (Avg)	FPS 2 (Avg)	
**PERF:  26.32 (26.26)	29.68 (29.59)	26.32 (26.26)	
**PERF:  29.40 (27.92)	29.40 (29.48)	29.40 (27.92)	
**PERF:  13.07 (22.76)	29.98 (22.38)	13.07 (22.76)	
**PERF:  4.79 (18.14)	0.00 (16.44)	4.79 (18.14)	
** WARN: <watch_source_status:739>: No data from source 1 since last 10 sec. Trying reconnection
** INFO: <reset_source_pipeline:1572>: Resetting source 1
**PERF:  4.79 (15.41)	0.00 (12.99)	4.79 (15.41)	
**PERF:  4.79 (13.61)	0.00 (10.74)	4.79 (13.61)	
**PERF:  4.79 (12.33)	0.00 (9.16)	4.79 (12.33)	
** ERROR: <cb_newpad3:500>: Failed to link depay loader to rtsp src
** INFO: <bus_callback:180>: Pipeline running

**PERF:  16.75 (12.90)	2.54 (9.83)	16.75 (12.90)	
**PERF:  30.00 (14.82)	30.00 (12.13)	30.00 (14.82)	
**PERF:  29.80 (16.32)	29.80 (13.92)	29.80 (16.32)	
**PERF:  29.60 (17.55)	29.60 (15.39)	29.60 (17.55)

Here camera 0 and 2 (same RTSP camera but two different streams) are disconnected and then reconnected:

**PERF:  FPS 0 (Avg)	FPS 1 (Avg)	FPS 2 (Avg)	
**PERF:  20.72 (20.60)	29.23 (28.95)	20.72 (20.60)	
**PERF:  29.20 (25.16)	29.20 (29.11)	29.20 (25.16)	
**PERF:  29.60 (26.70)	29.60 (29.30)	29.60 (26.70)	
**PERF:  29.59 (27.45)	29.59 (29.38)	29.59 (27.45)	
**PERF:  17.55 (22.19)	6.13 (24.27)	26.28 (22.15)	
** WARN: <watch_source_status:739>: No data from source 0 since last 10 sec. Trying reconnection
** INFO: <reset_source_pipeline:1572>: Resetting source 0
**PERF:  0.00 (18.42)	4.79 (20.78)	0.00 (18.39)	
** WARN: <watch_source_status:739>: No data from source 2 since last 13 sec. Trying reconnection
** INFO: <reset_source_pipeline:1572>: Resetting source 2
**PERF:  0.00 (15.75)	4.79 (18.35)	0.00 (15.72)	
**PERF:  0.00 (13.75)	4.79 (16.56)	0.00 (13.72)	
**PERF:  0.00 (12.20)	4.79 (15.19)	0.00 (12.18)	
ERROR from src_elem0: Could not open resource for reading and writing.
Debug info: gstrtspsrc.c(7871): gst_rtspsrc_retrieve_sdp (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin0/GstRTSPSrc:src_elem0:
Failed to connect. (Generic error)
**PERF:  0.00 (10.97)	4.79 (14.10)	0.00 (10.95)	
ERROR from src_elem2: Could not open resource for reading and writing.
Debug info: gstrtspsrc.c(7871): gst_rtspsrc_retrieve_sdp (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin2/GstRTSPSrc:src_elem2:
Failed to connect. (Generic error)
**PERF:  0.00 (9.96)	4.79 (13.22)	0.00 (9.94)	
**PERF:  0.00 (9.12)	4.79 (12.49)	0.00 (9.10)	
**PERF:  0.00 (8.41)	4.79 (11.88)	0.00 (8.40)	
**PERF:  0.00 (7.81)	4.79 (11.36)	0.00 (7.79)	
**PERF:  0.00 (7.28)	4.79 (10.91)	0.00 (7.27)	
**PERF:  0.00 (6.82)	4.79 (10.52)	0.00 (6.81)	
**PERF:  0.00 (6.42)	4.79 (10.17)	0.00 (6.41)	
** INFO: <reset_source_pipeline:1572>: Resetting source 0
ERROR from src_elem0: Could not open resource for reading and writing.
Debug info: gstrtspsrc.c(7871): gst_rtspsrc_retrieve_sdp (): /GstPipeline:pipeline/GstBin:multi_src_bin/GstBin:src_sub_bin0/GstRTSPSrc:src_elem0:
Failed to connect. (Generic error)
**PERF:  0.00 (6.06)	4.79 (9.87)	0.00 (6.05)	
** INFO: <reset_source_pipeline:1572>: Resetting source 2
Warning: Color primaries 5 not present and will be treated BT.601
**PERF:  0.00 (5.74)	4.79 (9.59)	0.05 (5.77)	
**PERF:  0.00 (5.45)	4.79 (9.35)	4.79 (5.72)	

**PERF:  FPS 0 (Avg)	FPS 1 (Avg)	FPS 2 (Avg)	
**PERF:  0.00 (5.19)	4.79 (9.13)	4.79 (5.68)	
**PERF:  0.00 (4.95)	4.79 (8.93)	4.79 (5.64)	
**PERF:  0.00 (4.74)	4.79 (8.74)	4.79 (5.60)	
**PERF:  0.00 (4.54)	4.79 (8.58)	4.79 (5.57)	
**PERF:  0.00 (4.36)	4.79 (8.42)	4.79 (5.54)	
**PERF:  0.00 (4.19)	4.79 (8.28)	4.79 (5.51)	
**PERF:  0.00 (4.03)	4.79 (8.15)	4.79 (5.48)	
**PERF:  0.00 (3.89)	4.79 (8.02)	4.79 (5.45)	
**PERF:  0.00 (3.75)	4.79 (7.91)	4.79 (5.43)	
** INFO: <reset_source_pipeline:1572>: Resetting source 0
Warning: Color primaries 5 not present and will be treated BT.601
**PERF:  0.18 (3.78)	8.52 (7.94)	8.52 (5.54)	
**PERF:  30.02 (4.64)	30.02 (8.67)	30.02 (6.34)	
**PERF:  29.93 (5.43)	29.93 (9.34)	29.93 (7.07)	
**PERF:  30.07 (6.18)	30.07 (9.98)	30.07 (7.78)	
**PERF:  29.67 (6.87)	29.67 (10.56)	29.67 (8.42)	

As you can see in the above output, as soon as a camera is disconnected it takes down the rest of the system to 4.79 FPS. Configuration:

[application]
enable-perf-measurement=1
perf-measurement-interval-sec=5

[tiled-display]
enable=1
rows=2
columns=2
width=1920
height=1080
gpu-id=0
nvbuf-memory-type=0

[source0]
enable=1
type=4
uri=rtsp://192.168.4.11:8554/fhd
rtsp-reconnect-attempts=1000
rtsp-reconnect-interval-sec=10
num-sources=1
gpu-id=0
cudadec-memtype=0

[source1]
enable=1
type=4
uri=rtsp://192.168.4.10:554/s0
rtsp-reconnect-attempts=1000
rtsp-reconnect-interval-sec=10
num-sources=1
gpu-id=0
cudadec-memtype=0

[source2]
enable=1
type=4
uri=rtsp://192.168.4.11:8554/fhd
rtsp-reconnect-attempts=1000
rtsp-reconnect-interval-sec=10
num-sources=1
gpu-id=0
cudadec-memtype=0

[sink0]
enable=1
type=2
sync=0
source-id=0
gpu-id=0
nvbuf-memory-type=0

[osd]
enable=0

[streammux]
gpu-id=0
live-source=1
batch-size=30
batched-push-timeout=40000
width=1920
height=1080
enable-padding=0
nvbuf-memory-type=0

[primary-gie]
enable=0

[tests]
file-loop=0

deepstream-app works with batched data, when there is one stream late, nvstreammux need to wait for a timeout period, the frame rate of each stream is impacted by the latency of the whole batch.

https://docs.nvidia.com/metropolis/deepstream/dev-guide/text/DS_plugin_gst-nvstreammux.html#gst-nvstreammux

Hi Fiona, how do I fix it? Changing batched-push-timeout doesn’t seem to have any impact.

You don’t need to fix it. The frame rate is just for a period. When the steams become stable, the frame rate will increase.

That doesn’t make sense. A stream can be offline for an indefinite amount of time. deepstream-app seems to have the same issue if the stream is offline when you start the application.

The log in this topic just show low fps during the rtsp connection period. In this period, there is no stable stream data received, so the fps is correct.

Can you set “batch-size=3” in [streammux] group in the configuration file.

There is no update from you for a period, assuming this is not an issue anymore.
Hence we are closing this topic. If need further support, please open a new one.
Thanks

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.