Performance drop in multiple source RTSP streaming

RTSP streaming using deepstream is very inconsistent

**• Hardware Platform GPU
**• DeepStream Version : 5

**• NVIDIA GPU Driver Version (valid for GPU only) : | NVIDIA-SMI 440.64.00 Driver Version: 440.64.00 CUDA Version: 10.2

**PERF: 5.07 (6.41)     5.20 (8.18)
**PERF: 1.50 (6.30)     9.60 (8.21)
**PERF: 4.53 (6.27)     5.60 (8.15)
**PERF: 4.92 (6.24)     7.00 (8.13)
**PERF: 1.92 (6.15)     8.20 (8.13)
**PERF: 5.31 (6.13)     4.80 (8.06)
**PERF: 1.40 (6.03)     9.07 (8.08)
**PERF: 2.10 (5.95)     7.14 (8.06)
**PERF: 3.15 (5.90)     7.40 (8.05)
**PERF: 2.50 (5.83)     8.80 (8.06)
**PERF: 4.56 (5.81)     5.00 (8.01)
**PERF: 1.77 (5.73)     8.60 (8.02)
**PERF: 1.20 (5.66)     9.47 (8.04)
**PERF: 8.68 (5.68)     9.72 (8.07)
**PERF: 1.21 (5.61)     8.09 (8.07)
**PERF: 1.90 (5.54)     7.71 (8.07)
**PERF: 2.19 (5.48)     8.67 (8.08)

**PERF: FPS 0 (Avg)     FPS 1 (Avg)
**PERF: 1.88 (5.43)     8.73 (8.09)
**PERF: 5.65 (5.42)     7.80 (8.08)
**PERF: 1.20 (5.35)     8.60 (8.09)
**PERF: 2.75 (5.32)     3.66 (8.02)
**PERF: 4.85 (5.31)     6.89 (8.00)
**PERF: 1.03 (5.24)     8.00 (8.00)
**PERF: 3.78 (5.23)     7.83 (8.00)
**PERF: 1.93 (5.18)     8.51 (8.00)
**PERF: 0.94 (5.12)     9.47 (8.03)
**PERF: 3.92 (5.09)     6.21 (8.00)
**PERF: 1.23 (5.04)     9.13 (8.02)
**PERF: 2.88 (5.01)     4.80 (7.98)
**PERF: 3.44 (4.99)     9.20 (7.99)
**PERF: 1.67 (4.95)     7.93 (7.99)
**PERF: 2.88 (4.92)     7.54 (7.98)
**PERF: 1.47 (4.87)     9.08 (8.00)
**PERF: 1.35 (4.83)     9.48 (8.02)
**PERF: 1.89 (4.79)     9.99 (8.04)
**PERF: 4.96 (4.78)     9.07 (8.06)
**PERF: 1.00 (4.74)     9.45 (8.08)

**PERF: FPS 0 (Avg)     FPS 1 (Avg)
**PERF: 1.42 (4.70)     9.62 (8.10)
**PERF: 3.30 (4.67)     9.59 (8.11)

As you can see one of the stream is inconsistent , and eventually it will drop to 0 . However upon restart it still picks from higher number and randomly one of the stream will go to 0

My config :
# Copyright (c) 2018-2020 NVIDIA Corporation. All rights reserved.
#
# NVIDIA Corporation and its licensors retain all intellectual property
# and proprietary rights in and to this software, related documentation
# and any modifications thereto. Any use, reproduction, disclosure or
# distribution of this software and related documentation without an express
# license agreement from NVIDIA Corporation is strictly prohibited.

[application]
enable-perf-measurement=1
perf-measurement-interval-sec=5
#gie-kitti-output-dir=streamscl
kitti-track-output-dir=../../../kitti_track/

[tiled-display]
enable=0
rows=2
columns=2
width=1280
height=720
gpu-id=0
#(0): nvbuf-mem-default - Default memory allocated, specific to particular platform
#(1): nvbuf-mem-cuda-pinned - Allocate Pinned/Host cuda memory, applicable for Tesla
#(2): nvbuf-mem-cuda-device - Allocate Device cuda memory, applicable for Tesla
#(3): nvbuf-mem-cuda-unified - Allocate Unified cuda memory, applicable for Tesla
#(4): nvbuf-mem-surface-array - Allocate Surface Array memory, applicable for Jetson
nvbuf-memory-type=0

[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI 4=RTSP
type=4
#uri=file://../../streams/sample_1080p_h264.mp4
uri=rtsp://admin:@192.168.1.10:554/mode=real&idc=1&ids=1
num-sources=1
#drop-frame-interval=2
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0

[source1]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI 4=RTSP
type=4
#uri=file://../../streams/sample_1080p_h264.mp4
uri=rtsp://admin:@192.168.1.12:554/mode=real&idc=1&ids=1
num-sources=1
#drop-frame-interval=2
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0

[sink0]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=1
sync=0
source-id=0
gpu-id=0
nvbuf-memory-type=0

[sink1]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=1
sync=0
source-id=0
gpu-id=0
nvbuf-memory-type=0

[sink2]
enable=0
#Type - 1=FakeSink 2=EglSink 3=File 4=RTSPStreaming
type=4
#1=h264 2=h265
codec=1
#encoder type 0=Hardware 1=Software
enc-type=0
sync=0
#iframeinterval=10
bitrate=400000
#H264 Profile - 0=Baseline 2=Main 4=High
#H265 Profile - 0=Main 1=Main10
profile=0
# set below properties in case of RTSPStreaming
rtsp-port=8554
udp-port=5400

[osd]
enable=0
gpu-id=0
border-width=1
text-size=15
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Serif
show-clock=0
clock-x-offset=800
clock-y-offset=820
clock-text-size=12
clock-color=1;0;0;0
nvbuf-memory-type=0

[streammux]
gpu-id=0
##Boolean property to inform muxer that sources are live
live-source=1
batch-size=2
##time out in usec, to wait after the first buffer is available
##to push the batch even if the complete batch is not formed
batched-push-timeout=40000
## Set muxer output width and height
width=2560
height=1440
##Enable to maintain aspect ratio wrt source, and allow black borders, works
##along with width, height properties
enable-padding=0
nvbuf-memory-type=0
## If set to TRUE, system timestamp will be attached as ntp timestamp
## If set to FALSE, ntp timestamp from rtspsrc, if available, will be attached
# attach-sys-ts-as-ntp=1

# config-file property is mandatory for any gie section.
# Other properties are optional and if set will override the properties set in
# the infer config file.
[primary-gie]
enable=1
gpu-id=0
model-engine-file=../../models/Primary_Detector/resnet10.caffemodel_b4_gpu0_int8.engine
batch-size=2
#Required by the app for OSD, not a plugin property
bbox-border-color0=1;0;0;1
bbox-border-color1=0;1;1;1
bbox-border-color2=0;0;1;1
bbox-border-color3=0;1;0;1
interval=0
gie-unique-id=1
nvbuf-memory-type=0
config-file=config_infer_primary.txt

[tracker]
enable=1
# For the case of NvDCF tracker, tracker-width and tracker-height must be a multiple of 32, respectively
tracker-width=640
tracker-height=384
#ll-lib-file=/opt/nvidia/deepstream/deepstream-5.0/lib/libnvds_mot_iou.so
ll-lib-file=/opt/nvidia/deepstream/deepstream-5.0/lib/libnvds_nvdcf.so
#ll-lib-file=/opt/nvidia/deepstream/deepstream-5.0/lib/libnvds_mot_klt.so
#ll-config-file required for DCF/IOU only
#ll-config-file=tracker_config.yml
#ll-config-file=iou_config.txt
gpu-id=0
#enable-batch-process applicable to DCF only
enable-batch-process=1

[secondary-gie0]
enable=0
model-engine-file=../../models/Secondary_VehicleTypes/resnet18.caffemodel_b16_gpu0_int8.engine
gpu-id=0
batch-size=16
gie-unique-id=4
operate-on-gie-id=1
operate-on-class-ids=0;
config-file=config_infer_secondary_vehicletypes.txt

[secondary-gie1]
enable=0
model-engine-file=../../models/Secondary_CarColor/resnet18.caffemodel_b16_gpu0_int8.engine
batch-size=16
gpu-id=0
gie-unique-id=5
operate-on-gie-id=1
operate-on-class-ids=0;
config-file=config_infer_secondary_carcolor.txt

[secondary-gie2]
enable=0
model-engine-file=../../models/Secondary_CarMake/resnet18.caffemodel_b16_gpu0_int8.engine
batch-size=16
gpu-id=0
gie-unique-id=6
operate-on-gie-id=1
operate-on-class-ids=0;
config-file=config_infer_secondary_carmake.txt

[ds-example]
enable=1
processing-width=2560
processing-height=1440
full-frame=0
unique-id=15
gpu-id=0

[tests]
file-loop=0

When I change source-id in sink1 to 1 , application crashes

bump

I figured out , as I was using twi sinks for same source may be thats why, but when I intend to use , separate sink for separate source my app crashes

If you want to watch two streams at the same time, use tiled display instead of two sinks. Also you can increase interval in your primary-gie since you are using a tracker. That would improve the FPS

I dont want to watch stream I am just processing them thats all and using ds-example to save images

@hrsht.sarma Can you try patch in BUG REPORT: the deepstream5.0 reference application process sub_bins index incorrectly in demux setting to fix the crash with two different sinks problem?
You need to apply the patch to opt/nvidia/deepstream/deepstream-5.0/sources/apps/sample_apps/deepstream-app/deepstream_app.c file and rebuild all sample apps. Don’t forget replacing the app binary files in opt/nvidia/deepstream/deepstream-5.0/bin with the new apps.