RTSP multi-camera issues with Deepstream 4.0.1

neophyte1 · September 20, 2019, 7:14am

Hi Guys,

I just flashed the latest JetPack 4.2.2 on Jetson Nano. I installed Deepstream 4.0.1 according to the instructions mentioned in the manual. The demo application ‘deepstream-app’ works fine with multiple video files. I get an fps of over 25 fps. However, the application does not perform well at all with RTSP streams especially when the number of streams is above 2. Following are some of the observations with 4 RTSP streams from IP cameras ( HIKVision Model - DS-2CD202WF-I ):

When I turn on tiled display, the fps drops to 8-9 fps.
When I turn off tiled display, the fps stays around 14-15 fps. However, the renderer gets stuck.

Queries:

When I face issues with RTSP streams with Deepstream 4.0, I was asked to replace few libraries. Please find the link below :

Should I replace the libraries for better performance.

Which camera models have been tested? Could this be an issue with HIKVision cameras?

Kindly help me out.

Thanks.

DaneLLL · September 20, 2019, 8:18am

Hi,
You do not need the libraries since it is part of DS4.0.1.

The reference config file is source8_1080p_dec_infer-resnet_tracker_tiled_display_fp16_nano.txt.
For 4 sources, please set batch-size=4 and run in sync=0

[sink0]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=5
<b>sync=0</b>
source-id=0
gpu-id=0
qos=0
nvbuf-memory-type=0
overlay-id=1

We use HIKVision cameras for verification. Will update the model ID for reference.

neophyte1 · September 23, 2019, 10:43am

Hi DaneLLL,

Thanks for your response. THe performance improved significantly. However, I am still not getting the expeected 25 fps. Please find the output below :

**PERF: 15.23 (15.23)	16.03 (16.03)	15.62 (15.62)	15.80 (15.80)	
**PERF: 14.86 (14.99)	14.86 (15.29)	14.86 (15.12)	14.86 (15.20)	
**PERF: 14.57 (14.82)	12.78 (14.32)	14.57 (14.91)	14.57 (14.96)	
**PERF: 15.17 (14.92)	14.57 (14.39)	15.17 (14.98)	14.77 (14.90)

Config:

[application]
enable-perf-measurement=1
perf-measurement-interval-sec=5
#gie-kitti-output-dir=streamscl

[tiled-display]
enable=0
rows=4
columns=2
width=1280
height=720
gpu-id=0
#(0): nvbuf-mem-default - Default memory allocated, specific to particular platform
#(1): nvbuf-mem-cuda-pinned - Allocate Pinned/Host cuda memory, applicable for Tesla
#(2): nvbuf-mem-cuda-device - Allocate Device cuda memory, applicable for Tesla
#(3): nvbuf-mem-cuda-unified - Allocate Unified cuda memory, applicable for Tesla
#(4): nvbuf-mem-surface-array - Allocate Surface Array memory, applicable for Jetson
nvbuf-memory-type=0

[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI 4=RTSP
type=4
uri=rtsp://admin:edge1234@192.168.0.201:554/Streaming/Channels/1
#uri=file://../../streams/sample_1080p_h264.mp4
num-sources=1
#drop-frame-interval=2
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0
#source-id=0

[source1]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI 4=RTSP
type=4
uri=rtsp://admin:edge1234@192.168.0.202:554/Streaming/Channels/1
#uri=file://../../streams/sample_1080p_h264.mp4
num-sources=1
#drop-frame-interval=2
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0
#source-id=0

[source2]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI 4=RTSP
type=4
uri=rtsp://admin:edge1234@192.168.0.203:554/Streaming/Channels/1
#uri=file://../../streams/sample_1080p_h264.mp4
num-sources=1
#drop-frame-interval=2
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0
#source-id=0

[source3]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI 4=RTSP
type=4
uri=rtsp://admin:edge1234@192.168.0.204:554/Streaming/Channels/1
#uri=file://../../streams/sample_1080p_h264.mp4
num-sources=1
#drop-frame-interval=2
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0
#source-id=0


[sink0]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=2
sync=0
source-id=0
gpu-id=0
qos=0
nvbuf-memory-type=0
overlay-id=1

[sink1]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=2
sync=0
source-id=1
gpu-id=0
qos=0
nvbuf-memory-type=0
overlay-id=1

[sink2]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=2
sync=0
source-id=2
gpu-id=0
qos=0
nvbuf-memory-type=0
overlay-id=1

[sink3]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=2
sync=0
source-id=3
gpu-id=0
qos=0
nvbuf-memory-type=0
overlay-id=1



[osd]
enable=1
gpu-id=0
border-width=1
text-size=15
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Serif
show-clock=0
clock-x-offset=800
clock-y-offset=820
clock-text-size=12
clock-color=1;0;0;0
nvbuf-memory-type=0

[streammux]
gpu-id=0
##Boolean property to inform muxer that sources are live
live-source=0
batch-size=4
##time out in usec, to wait after the first buffer is available
##to push the batch even if the complete batch is not formed
batched-push-timeout=40000
## Set muxer output width and height
width=1280
height=720
##Enable to maintain aspect ratio wrt source, and allow black borders, works
##along with width, height properties
enable-padding=0
nvbuf-memory-type=0

# config-file property is mandatory for any gie section.
# Other properties are optional and if set will override the properties set in
# the infer config file.
[primary-gie]
enable=1
gpu-id=0
model-engine-file=/home/edgetensor/deepstream_sdk_v4.0.1_jetson/samples/models/Primary_Detector_Nano/resnet10.caffemodel_b4_fp16.engine
batch-size=4
#Required by the app for OSD, not a plugin property
bbox-border-color0=1;0;0;1
bbox-border-color1=0;1;1;1
bbox-border-color2=0;0;1;1
bbox-border-color3=0;1;0;1
interval=4
gie-unique-id=1
nvbuf-memory-type=0
config-file=config_infer_primary_nano.txt

[tracker]
enable=1
tracker-width=480
tracker-height=270
#ll-lib-file=/opt/nvidia/deepstream/deepstream-4.0/lib/libnvds_mot_iou.so
ll-lib-file=/opt/nvidia/deepstream/deepstream-4.0/lib/libnvds_mot_klt.so
#ll-config-file required for IOU only
#ll-config-file=iou_config.txt
gpu-id=0


[ds-example]
enable=0
processing-width=1280
processing-height=720
full-frame=1
unique-id=15
gpu-id=0


[tests]
file-loop=0

When I try type=5 for sink the RTSP frames occupy the entire screen and the application freezes. Kindly let me know if this is the expected behavior.

Thanks.

DaneLLL · September 24, 2019, 3:08am

Hi,
For four sources, you need to enable tiled-display and enable only sink0 in type=5

[tiled-display]
[b]enable=1
rows=2
columns=2[/b]

neophyte1 · September 24, 2019, 5:15am

Hi DaneLLL,

The display issue seems to have solved. Could you please explain why only sink0 needed to be of type 5 ? What is type 5 ? Also, the fps stays around the same.

**PERF: 16.13 (17.77)	13.85 (16.51)	13.66 (16.44)	14.58 (16.03)	
**PERF: 13.44 (16.08)	14.88 (15.93)	15.59 (16.13)	15.47 (15.82)	
**PERF: 15.66 (15.97)	14.89 (15.65)	14.45 (15.68)	14.59 (15.47)

Thanks.

DaneLLL · September 24, 2019, 8:06am

Hi,
Please disable sink1, sink2, and sink3. With tiled-display, you should need sink0 only.
type 5 is nvoverlaysink which does not take any GPU loading.
[url]NVIDIA DeepStream SDK Developer Guide — DeepStream 6.1.1 Release documentation

Also please execute ‘sudo jetson_clocks’ to run max performance. If the performance is still not as expected, please run ‘sudo tegrastats’ to profile system loading.

DaneLLL · September 26, 2019, 6:52am

Hi,
rtsp is a live source, so please also configure ‘live-source=1’ in [streammux].