Some problems about RTSP streams on DS5.0

Hi,

Env.
GPU:NVIDIA T4, Ubuntu 18.04, GStreamer 1.14.1, NVIDIA driver 440+, CUDA 10.2, TensorRT 7.0, Deepstream 5.0

We used original deepstream-app to run 5~8 rtsp streams. Tiler is disabled (stand alone output for each video).
Some video frames are delivered to wrong channel after nvdemuxer.
Attachments are screen captured file and app config file.

Thanks.

20200525182937

A part of config file:
[source0]
enable=1
type=4
uri=rtsp://192.168.170.65:554/xxx
num-sources=1
gpu-id=0

[sink0]
enable=1
type=4
sync=0
source-id=0
gpu-id=0
nvbuf-memory-type=0
rtsp-port=21000
udp-port=31000
bitrate=1200000
codec=1

[osd]
enable=1
gpu-id=0
border-width=1
text-size=15
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Serif
show-clock=0
clock-x-offset=800
clock-y-offset=820
clock-text-size=12
clock-color=1;0;0;0
nvbuf-memory-type=0
process-mode=1

[streammux]
gpu-id=0
live-source=1
batch-size=1
batched-push-timeout=40000
width=1920
height=1080
enable-padding=0

[primary-gie]
enable=1
gpu-id=0
model-engine-file=model_b1_gpu0_int8.engine
labelfile-path=labels.txt
batch-size=1
bbox-border-color0=1;0;0;1
bbox-border-color1=0;1;1;1
bbox-border-color2=0;0;1;1
bbox-border-color3=0;1;0;1
gie-unique-id=1
nvbuf-memory-type=0
config-file=config_infer_primary_yoloV3.txt

I have seen this before, but only when I was testing and set sync=1 on my sinks. It gave me the exact results your seeing.
Sync is meant to be 0 for live sources I believe and with that setting I don’t get the issue.

However there are various timing issues with streammux. Recorded files have strange timing issues and rewind (kind of) with streammux live-sources set to 1 which is what you are meant to do for rtsp sources.
So strangely I have to set live sources to 0 to get decent file recordings. Even though they are still a big odd an occasionally seek to run in fast forward or slow motion. I have other threads open on these issues but no response yet.

Hi,
In [streammux], batch-size shoud match with your source number. Setting to 1 means single source.

Hi,
We have doen some tests, base on source numer = batch-size in [streammux].
It works well, when source number <=5.
And same problem occured, when source muber >=6. Especially, 8 rtsp cameras.
That would never happen in DS4.0, even batch-size =1.

Hi,
According to frequently Q/A. Frequently Asked Questions
image
So we think batch-size just affects performance, and not caused any wrong results.

Hi,
Can the issue be reproduced with local video playback? Or specific to multiple RTSP sources? We would like to reproduce it but don’t have enough IP cameras. If it can be reproduced with local video playback, we can try to run
source12_1080p_dec_infer-resnet_tracker_tiled_display_fp16_tx2.txt

Hi,
It’s OK for 8 source mp4 files and batch-size = 8.
And not OK for 8 source mp4 files and batch-size = 1.

Hi,
In [streammux], batch-size has to be identical to source number. You can check reference config files such as
source6_csi_dec_infer_resnet_int8.txt
source8_1080p_dec_infer-resnet_tracker_tiled_display_fp16_nano.txt
source30_1080p_dec_infer-resnet_tiled_display_int8.txt

You have reported an issue in the comment. Is it specific to RTPS sources? If you have eight RTSP sources and set batch-size=8, the misoder is reproduced?

Hi,
The misorder will absolutely hapen for eight RTSP sources and set batch-size=8(nvstreammux) batch-size=8(primary-gie) .

Hi,
Please attach the config file that can be run with deepstream-app to reproduce the issue.

[application]
enable-perf-measurement=1
perf-measurement-interval-sec=5

[source0]
enable=1
type=4
uri=rtsp://xxx
num-sources=1
gpu-id=0

[sink0]
enable=1
type=4
codec=1
bitrate=2000000
rtsp-port=8554
udp-port=5400
sync=0
source-id=0
gpu-id=0

[source1]
enable=1
type=4
uri=rtsp://xxx
num-sources=1
gpu-id=0

[sink1]
enable=1
type=4
codec=1
bitrate=2000000
rtsp-port=8555
udp-port=5401
sync=0
source-id=1
gpu-id=0

[source2]
enable=1
type=4
uri=rtsp://xxx
num-sources=1
gpu-id=0

[sink2]
enable=1
type=4
codec=1
bitrate=2000000
rtsp-port=8556
udp-port=5402
sync=0
source-id=2
gpu-id=0

[source3]
enable=1
type=4
uri=rtsp://xxx
num-sources=1
gpu-id=0

[sink3]
enable=1
type=4
codec=1
bitrate=2000000
rtsp-port=8557
udp-port=5403
sync=0
source-id=3
gpu-id=0

[source4]
enable=1
type=4
uri=rtsp://xxx
num-sources=1
gpu-id=0

[sink4]
enable=1
type=4
codec=1
bitrate=2000000
rtsp-port=8558
udp-port=5404
sync=0
source-id=4
gpu-id=0

[source5]
enable=6
type=4
uri=rtsp://xxx
num-sources=1
gpu-id=0

[sink5]
enable=1
type=4
codec=1
bitrate=2000000
rtsp-port=8559
udp-port=5405
sync=0
source-id=5
gpu-id=0


[source6]
enable=1
type=4
uri=rtsp://xxx
num-sources=1
gpu-id=0

[sink6]
enable=1
type=4
codec=1
bitrate=2000000
rtsp-port=8560
udp-port=5406
sync=0
source-id=6
gpu-id=0

[source7]
enable=1
type=4
uri=rtsp://xxx
num-sources=1
gpu-id=0

[sink7]
enable=1
type=4
codec=1
bitrate=2000000
rtsp-port=8561
udp-port=5407
sync=0
source-id=7
gpu-id=0


[osd]
enable=1
gpu-id=0
border-width=1
text-size=15
text-color=1;1;1;1;
text-bg-color=0.3;0.3;0.3;1
font=Serif
show-clock=0
clock-x-offset=800
clock-y-offset=820
clock-text-size=12
clock-color=1;0;0;0
process-mode=1

[streammux]
enable=1
gpu-id=0
live-source=1
batch-size=8
batched-push-timeout=40000
width=1920
height=1080

[primary-gie]
enable=1
process-mode=1
gpu-id=0
model-engine-file=model_b10_gpu0_int8.engine
labelfile-path=labels.txt
batch-size=8
bbox-border-color0=1;0;0;1
bbox-border-color1=0;1;1;1
bbox-border-color2=0;0;1;1
bbox-border-color3=0;1;0;1
gie-unique-id=1
config-file=config_infer_primary_yoloV3.txt

Hi,
Is the issue observed with nveglglessink? Using windowed sink is easier to set up in the test.

Hi,
We run this app on a GPU server, and has no vision output.
We used filesink to save output videos as mp4 files, the misorder is same as rstp sink.

Hi,
Please check if batched-push-timeout need to be adjusted:

If your sources are not 30fps, you have to adjust the value accordingly.

We have different cameras with different fps. what we we do?
Anyway, for users the difference of batch-size , cameras, rtsp streams… should not cause wrong result.

Hi,
We support the case that all sources are in identical framerate. If it is not identical, nvstreammux may not work properly. Please list frame rate of your sources. We would need the information and discuss with teams about the usecase.

Hi,
In the actual project, cameras may have different fps.
They may be any fps between 10~30.

Hi,
Please upgrade to 5.0 GA and try again.