Deepstream 4.0.2 multiple sources and multiple RTSP sinks

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

hi!
• Hardware Platform ( GPU)
• DeepStream Version 4.0.2
• TensorRT Version6.0.1
• NVIDIA GPU Driver Version 440+
my config is

[application]
enable-perf-measurement=1
perf-measurement-interval-sec=5
#gie-kitti-output-dir=streamscl

[tiled-display]
enable=0
rows=1
columns=1
width=1280
height=720
gpu-id=0
#(0): nvbuf-mem-default - Default memory allocated, specific to particular platform
#(1): nvbuf-mem-cuda-pinned - Allocate Pinned/Host cuda memory, applicable for Tesla
#(2): nvbuf-mem-cuda-device - Allocate Device cuda memory, applicable for Tesla
#(3): nvbuf-mem-cuda-unified - Allocate Unified cuda memory, applicable for Tesla
#(4): nvbuf-mem-surface-array - Allocate Surface Array memory, applicable for Jetson
nvbuf-memory-type=0

[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
#uri=file:///home/ubuntu/program/deepstream_sdk_v4.0.2_x86_64/samples/streams/sample_720p.mp4
uri=rtsp://admin:12345678a@172.16.16.10/Streaming/Channels/1
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0

[sink0]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=4
sync=0

rtsp-port=21000
udp-port=31000
bitrate=1200000
codec=1
source-id=0
gpu-id=0
nvbuf-memory-type=0

[source1]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
#uri=file:///home/ubuntu/program/deepstream_sdk_v4.0.2_x86_64/samples/streams/sample_720p.mp4
uri=rtsp://admin:12345678a@172.16.16.10/Streaming/Channels/1
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0

[sink1]
enable=4
#Type - 1=FakeSink 2=EglSink 3=File
type=4
sync=0

rtsp-port=21001
udp-port=31001
bitrate=4000000
codec=1
source-id=1
gpu-id=0
nvbuf-memory-type=0

[source2]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=2
#uri=file:///home/ubuntu/program/deepstream_sdk_v4.0.2_x86_64/samples/streams/sample_720p.mp4
uri=rtsp://admin:12345678a@192.168.90.136/Streaming/Channels/1
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0

[sink2]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=4
sync=0

rtsp-port=21002
udp-port=31002
bitrate=1200000
codec=1
source-id=2
gpu-id=0
nvbuf-memory-type=0


[osd]
enable=1
gpu-id=0
border-width=1
text-size=15
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Serif
show-clock=0
clock-x-offset=800
clock-y-offset=820
clock-text-size=12
clock-color=1;0;0;0
nvbuf-memory-type=0

[streammux]
gpu-id=0
##Boolean property to inform muxer that sources are live
live-source=1
batch-size=3
##time out in usec, to wait after the first buffer is available
##to push the batch even if the complete batch is not formed
batched-push-timeout=40000
## Set muxer output width and height
width=1920
height=1080
##Enable to maintain aspect ratio wrt source, and allow black borders, works
##along with width, height properties
enable-padding=0
nvbuf-memory-type=0

# config-file property is mandatory for any gie section.
# Other properties are optional and if set will override the properties set in
# the infer config file.
[primary-gie]
enable=1
gpu-id=0
model-engine-file=model_b3_int8.engine
labelfile-path=labels.txt
batch-size=3
#Required by the app for OSD, not a plugin property
bbox-border-color0=1;0;0;1
bbox-border-color1=0;1;1;1
bbox-border-color2=0;0;1;1
bbox-border-color3=0;1;0;1
#interval=0
gie-unique-id=1
nvbuf-memory-type=0
config-file=config_infer_primary_yoloV3.txt

[tests]
file-loop=0

if i use one source and one sink ,everything is ok, but 3 sources 3 rtsp, the error shows below

 *** DeepStream: Launched RTSP Streaming at rtsp://localhost:21000/ds-test ***


 *** DeepStream: Launched RTSP Streaming at rtsp://localhost:21001/ds-test ***


 *** DeepStream: Launched RTSP Streaming at rtsp://localhost:21002/ds-test ***

Creating LL OSD context new
Creating LL OSD context new
Creating LL OSD context new
Deserialize yoloLayerV3 plugin: yolo_83
Deserialize yoloLayerV3 plugin: yolo_95
Deserialize yoloLayerV3 plugin: yolo_107
0:00:04.220658011 44628 0x7f1708002240 WARN                 nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:log(): TensorRT was linked against cuDNN 7.6.3 but loaded cuDNN 7.4.1
0:00:04.233676724 44628 0x7f1708002240 WARN                 nvinfer gstnvinfer.cpp:515:gst_nvinfer_logger:<primary_gie_classifier> NvDsInferContext[UID 1]:log(): TensorRT was linked against cuDNN 7.6.3 but loaded cuDNN 7.4.1
cb_sourcesetup set 100 latency
cb_sourcesetup set 100 latency
cb_sourcesetup set 100 latency

Runtime commands:
	h: Print this help
	q: Quit

	p: Pause
	r: Resume

** INFO: <bus_callback:189>: Pipeline ready

** INFO: <bus_callback:175>: Pipeline running

Creating LL OSD context new
Creating LL OSD context new
Creating LL OSD context new
ERROR from sink_sub_bin_encoder1: Device '/dev/nvhost-msenc' failed during initialization
Debug info: gstv4l2object.c(4050): gst_v4l2_object_set_format_full (): /GstPipeline:pipeline/GstBin:processing_bin_0/GstBin:sink_bin/GstBin:sink_sub_bin1/nvv4l2h264enc:sink_sub_bin_encoder1:
Call to S_FMT failed for YM12 @ 1920x1080: Unknown error -1
ERROR from sink_sub_bin_encoder1: Device '/dev/nvhost-msenc' failed during initialization
Debug info: gstv4l2object.c(4050): gst_v4l2_object_set_format_full (): /GstPipeline:pipeline/GstBin:processing_bin_0/GstBin:sink_bin/GstBin:sink_sub_bin1/nvv4l2h264enc:sink_sub_bin_encoder1:
Call to S_FMT failed for YM12 @ 1920x1080: Unknown error -1
ERROR from sink_sub_bin_encoder1: Device '/dev/nvhost-msenc' failed during initialization
Debug info: gstv4l2object.c(4050): gst_v4l2_object_set_format_full (): /GstPipeline:pipeline/GstBin:processing_bin_0/GstBin:sink_bin/GstBin:sink_sub_bin1/nvv4l2h264enc:sink_sub_bin_encoder1:
Call to S_FMT failed for YM12 @ 1920x1080: Unknown error -1
ERROR from sink_sub_bin_encoder1: Device '/dev/nvhost-msenc' failed during initialization
Debug info: gstv4l2object.c(4050): gst_v4l2_object_set_format_full (): /GstPipeline:pipeline/GstBin:processing_bin_0/GstBin:sink_bin/GstBin:sink_sub_bin1/nvv4l2h264enc:sink_sub_bin_encoder1:
Call to S_FMT failed for YM12 @ 1920x1080: Unknown error -1
ERROR from sink_sub_bin_encoder1: Device '/dev/nvhost-msenc' failed during initialization
Debug info: gstv4l2object.c(4050): gst_v4l2_object_set_format_full (): /GstPipeline:pipeline/GstBin:processing_bin_0/GstBin:sink_bin/GstBin:sink_sub_bin1/nvv4l2h264enc:sink_sub_bin_encoder1:
Call to S_FMT failed for YM12 @ 1920x1080: Unknown error -1
ERROR from sink_sub_bin_encoder1: Device '/dev/nvhost-msenc' failed during initialization
Debug info: gstv4l2object.c(4050): gst_v4l2_object_set_format_full (): /GstPipeline:pipeline/GstBin:processing_bin_0/GstBin:sink_bin/GstBin:sink_sub_bin1/nvv4l2h264enc:sink_sub_bin_encoder1:
Call to S_FMT failed for YM12 @ 1920x1080: Unknown error -1
Quitting
ERROR from sink_sub_bin_encoder1: Device '/dev/nvhost-msenc' failed during initialization
Debug info: gstv4l2object.c(4050): gst_v4l2_object_set_format_full (): /GstPipeline:pipeline/GstBin:processing_bin_0/GstBin:sink_bin/GstBin:sink_sub_bin1/nvv4l2h264enc:sink_sub_bin_encoder1:
Call to S_FMT failed for YM12 @ 1920x1080: Unknown error -1
ERROR from sink_sub_bin_encoder1: Device '/dev/nvhost-msenc' failed during initialization
Debug info: gstv4l2object.c(4050): gst_v4l2_object_set_format_full (): /GstPipeline:pipeline/GstBin:processing_bin_0/GstBin:sink_bin/GstBin:sink_sub_bin1/nvv4l2h264enc:sink_sub_bin_encoder1:
Call to S_FMT failed for YM12 @ 1920x1080: Unknown error -1
ERROR from sink_sub_bin_encoder1: Device '/dev/nvhost-msenc' failed during initialization
Debug info: gstv4l2object.c(4050): gst_v4l2_object_set_format_full (): /GstPipeline:pipeline/GstBin:processing_bin_0/GstBin:sink_bin/GstBin:sink_sub_bin1/nvv4l2h264enc:sink_sub_bin_encoder1:
Call to S_FMT failed for YM12 @ 1920x1080: Unknown error -1
ERROR from sink_sub_bin_encoder1: Device '/dev/nvhost-msenc' failed during initialization
Debug info: gstv4l2object.c(4050): gst_v4l2_object_set_format_full (): /GstPipeline:pipeline/GstBin:processing_bin_0/GstBin:sink_bin/GstBin:sink_sub_bin1/nvv4l2h264enc:sink_sub_bin_encoder1:
Call to S_FMT failed for YM12 @ 1920x1080: Unknown error -1
3

Hi,
The config file looks fine. Does it work in 3 sources + 3 file save? Or 2 sources + 2 RTSP?

4

i try 2 sources + 2 rtsp sinks, if [source 0] and [source 1] uri is same, it can work,
likes this
[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=3
uri=rtsp://admin:12345678a@172.16.16.10/Streaming/Channels/1
num-sources=1
gpu-id=0
# (0): memtype_device - Memory type Device
# (1): memtype_pinned - Memory type Host Pinned
# (2): memtype_unified - Memory type Unified
cudadec-memtype=0

[sink0]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=4
sync=0

rtsp-port=21000
udp-port=31000
bitrate=1200000
codec=1
source-id=0
gpu-id=0
nvbuf-memory-type=0

[source1]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=3
uri=rtsp://admin:12345678a@172.16.16.10/Streaming/Channels/1
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0

[sink1]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=4
sync=0

rtsp-port=21001
udp-port=31001
bitrate=1200000
codec=1
source-id=1
gpu-id=0
nvbuf-memory-type=0

but if i change the second source uri to file , the same error happen.
[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=3
uri=rtsp://admin:12345678a@172.16.16.10/Streaming/Channels/1
num-sources=1
gpu-id=0
# (0): memtype_device - Memory type Device
# (1): memtype_pinned - Memory type Host Pinned
# (2): memtype_unified - Memory type Unified
cudadec-memtype=0

[sink0]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=4
sync=0

rtsp-port=21000
udp-port=31000
bitrate=1200000
codec=1
source-id=0
gpu-id=0
nvbuf-memory-type=0

[source1]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=3
uri=file:///home/ubuntu/program/deepstream_sdk_v4.0.2_x86_64/samples/streams/sample_720p.mp4
num-sources=1
gpu-id=0
# (0): memtype_device   - Memory type Device
# (1): memtype_pinned   - Memory type Host Pinned
# (2): memtype_unified  - Memory type Unified
cudadec-memtype=0

[sink1]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=4
sync=0

rtsp-port=21001
udp-port=31001
bitrate=1200000
codec=1
source-id=1
gpu-id=0
nvbuf-memory-type=0
5

hi!
ok, i try to update DS 5.0
here is my config

[application]
enable-perf-measurement=1
perf-measurement-interval-sec=5
#gie-kitti-output-dir=streamscl

[tiled-display]
enable=0
rows=1
columns=1
width=1280
height=720
gpu-id=0
#(0): nvbuf-mem-default - Default memory allocated, specific to particular platform
#(1): nvbuf-mem-cuda-pinned - Allocate Pinned/Host cuda memory, applicable for Tesla
#(2): nvbuf-mem-cuda-device - Allocate Device cuda memory, applicable for Tesla
#(3): nvbuf-mem-cuda-unified - Allocate Unified cuda memory, applicable for Tesla
#(4): nvbuf-mem-surface-array - Allocate Surface Array memory, applicable for Jetson
nvbuf-memory-type=0

[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=3
#uri=file://../../samples/streams/sample_720p.h264
uri=rtsp://admin:12345678a@172.16.16.10/Streaming/Channels/1
num-sources=1
gpu-id=0
cudadec-memtype=0

[sink0]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=4
sync=0
rtsp-port=10000
udp-port=10000
bitrate = 1000000
codec=1
enc-type=0
source-id=0
gpu-id=0
nvbuf-memory-type=0

[osd]
enable=1
gpu-id=0
border-width=1
text-size=15
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Serif
show-clock=0
clock-x-offset=800
clock-y-offset=820
clock-text-size=12
clock-color=1;0;0;0
nvbuf-memory-type=0

[streammux]
gpu-id=0
##Boolean property to inform muxer that sources are live
live-source=1
batch-size=2
##time out in usec, to wait after the first buffer is available
##to push the batch even if the complete batch is not formed
batched-push-timeout=40000
## Set muxer output width and height
width=1280
height=720
##Enable to maintain aspect ratio wrt source, and allow black borders, works
##along with width, height properties
enable-padding=0
nvbuf-memory-type=0

# config-file property is mandatory for any gie section.
# Other properties are optional and if set will override the properties set in
# the infer config file.
[primary-gie]
enable=1
gpu-id=0
model-engine-file=model_b2_gpu0_int8.engine
labelfile-path=labels.txt
batch-size=2
#Required by the app for OSD, not a plugin property
bbox-border-color0=1;0;0;1
bbox-border-color1=0;1;1;1
bbox-border-color2=0;0;1;1
bbox-border-color3=0;1;0;1
interval=2
gie-unique-id=1
nvbuf-memory-type=0
config-file=config_infer_primary_yoloV3.txt

[tracker]
enable=0
tracker-width=640
tracker-height=384
ll-lib-file=/opt/nvidia/deepstream/deepstream-5.0/lib/libnvds_mot_klt.so

[tests]
file-loop=0

yes, 1 source 1 RTSP sink , everything is ok, but i change to 2 source + 2 rtsp sink, the config

[application]
enable-perf-measurement=1
perf-measurement-interval-sec=5
#gie-kitti-output-dir=streamscl

[tiled-display]
enable=0
rows=1
columns=1
width=1280
height=720
gpu-id=0
#(0): nvbuf-mem-default - Default memory allocated, specific to particular platform
#(1): nvbuf-mem-cuda-pinned - Allocate Pinned/Host cuda memory, applicable for Tesla
#(2): nvbuf-mem-cuda-device - Allocate Device cuda memory, applicable for Tesla
#(3): nvbuf-mem-cuda-unified - Allocate Unified cuda memory, applicable for Tesla
#(4): nvbuf-mem-surface-array - Allocate Surface Array memory, applicable for Jetson
nvbuf-memory-type=0

[source0]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=3
#uri=file://../../samples/streams/sample_720p.h264
uri=rtsp://admin:12345678a@172.16.16.10/Streaming/Channels/1
num-sources=1
gpu-id=0
cudadec-memtype=0

[sink0]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=4
sync=0
rtsp-port=10000
udp-port=10000
bitrate = 1000000
codec=1
enc-type=0
source-id=0
gpu-id=0
nvbuf-memory-type=0

[source1]
enable=1
#Type - 1=CameraV4L2 2=URI 3=MultiURI
type=3
#uri=file://../../samples/streams/sample_720p.h264
uri=rtsp://admin:12345678a@172.16.16.10/Streaming/Channels/1
num-sources=1
gpu-id=0
cudadec-memtype=0

[sink1]
enable=1
#Type - 1=FakeSink 2=EglSink 3=File
type=4
sync=0
rtsp-port=10001
udp-port=10001
bitrate = 1000000
codec=1
enc-type=0
source-id=1
gpu-id=0
nvbuf-memory-type=0


[osd]
enable=1
gpu-id=0
border-width=1
text-size=15
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Serif
show-clock=0
clock-x-offset=800
clock-y-offset=820
clock-text-size=12
clock-color=1;0;0;0
nvbuf-memory-type=0

[streammux]
gpu-id=0
##Boolean property to inform muxer that sources are live
live-source=1
batch-size=2
##time out in usec, to wait after the first buffer is available
##to push the batch even if the complete batch is not formed
batched-push-timeout=40000
## Set muxer output width and height
width=1280
height=720
##Enable to maintain aspect ratio wrt source, and allow black borders, works
##along with width, height properties
enable-padding=0
nvbuf-memory-type=0

# config-file property is mandatory for any gie section.
# Other properties are optional and if set will override the properties set in
# the infer config file.
[primary-gie]
enable=1
gpu-id=0
model-engine-file=model_b2_gpu0_int8.engine
labelfile-path=labels.txt
batch-size=2
#Required by the app for OSD, not a plugin property
bbox-border-color0=1;0;0;1
bbox-border-color1=0;1;1;1
bbox-border-color2=0;0;1;1
bbox-border-color3=0;1;0;1
interval=2
gie-unique-id=1
nvbuf-memory-type=0
config-file=config_infer_primary_yoloV3.txt

[tracker]
enable=0
tracker-width=640
tracker-height=384
ll-lib-file=/opt/nvidia/deepstream/deepstream-5.0/lib/libnvds_mot_klt.so

[tests]
file-loop=0

the error is

 *** DeepStream: Launched RTSP Streaming at rtsp://localhost:10000/ds-test ***


 *** DeepStream: Launched RTSP Streaming at rtsp://localhost:10001/ds-test ***


(deepstream-app:42406): GStreamer-CRITICAL **: 10:26:42.179: gst_element_get_static_pad: assertion 'GST_IS_ELEMENT (element)' failed
Segfault (core dumped)

so, i need your help, thanks for you replay.

6

Hi,
Does it work in 3 sources + 3 file save?

7

I have the same problem

8

Hi,
We would suggest use default mode: use nvmultistreamtiler to composite all sources into single frame and do single RTSP streaming.

9
10

@1319501722 Can you try the patch in BUG REPORT: the deepstream5.0 reference application process sub_bins index incorrectly in demux setting to check if it can fix your problem or not?
It is for Deepstream5.0, so you need to apply the patch manually, rebuild and replace the deepstream apps.

12

hi ,what I need is two rtsp-out streaming for two input streaming(even more),and I use udpsink+rtsp-server for single streaming,
How can I get two rtsp-out streaming simultaneously?reate two udpsink+rtsp-server using different ports?
I hope you can give me some advice,thanks

13

Hi,

Yes. We have the demonstration in deepstream-app. Please check create_udpsink_bin() in

deepstream-5.0\sources\apps\apps-common\src\deepstream_sink_bin.c