Multi stream video artifacts

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU)
Jetson TX2
• DeepStream Version
5.0.1-20.09
• JetPack Version (valid for Jetson only)
4.4 [L4T 32.4.3]
• TensorRT Version
7.1.3.0

I am using the following pipeline to decode 3 video streams from my local network:

rtspsrc latency=150 drop-on-latency=true location=rtsp://10.42.0.1:8554/doors_0 name=stream_0 ! tee ! queue ! rtph264depay ! h264parse ! nvv4l2decoder bufapi-version=1 enable-max-performance=1 ! nvvideoconvert compute-hw=0 src-crop=410:510:1280:720 ! 'video/x-raw(memory:NVMM), format=RGBA' ! mux_0.sink_0 rtspsrc latency=150 drop-on-latency=true location=rtsp://10.42.0.1:8554/corridor_0 name=stream_1 ! tee ! queue ! rtph264depay ! h264parse ! nvv4l2decoder bufapi-version=1 enable-max-performance=1 ! nvvideoconvert compute-hw=0 src-crop=0:700:1100:1200 ! 'video/x-raw(memory:NVMM), format=RGBA' ! mux_0.sink_1 rtspsrc latency=150 drop-on-latency=true location=rtsp://10.42.0.1:8554/corridor_1 name=stream_2 ! tee ! queue ! rtph264depay ! h264parse ! nvv4l2decoder bufapi-version=1 enable-max-performance=1 ! nvvideoconvert compute-hw=0 src-crop=800:200:700:600 ! 'video/x-raw(memory:NVMM), format=RGBA' ! mux_0.sink_2 nvstreammux live-source=1 name=mux_0 batch-size=3 width=960 height=544 batched-push-timeout=4000000 attach-sys-ts=false ! nvvideoconvert ! nvmultistreamtiler width=1440 height=544 ! nvegltransform ! nveglglessink

The pipeline works but I get a lot of weird video temporal artifacts, it looks like ghosting:

decode
decode479×539 83 KB

On the other hand, everything works as expected when I decode a single stream (any of the three).

Moreover, I have also tried uridecodebin3 instead of rtspsrc ! ... ! nvv4l2decoder, it works a little bit better but too much slower.

3

The artifact is caused by packet lost with RTSP. Can you use larger “latency”(the default value is 2000 but you use 150 and disable “drop-on-latency”? And there are many quality related properties of rtspsrc, you can try to tune them. rtspsrc

uridecodebin works better and slower is because it will setup a RTP session and use rtpjitterbuffer to maintain the packets.

playing
playing16970×1093 707 KB

https://gstreamer.freedesktop.org/documentation/additional/rtp.html
There are a lot of resources for gstreamer RTSP, you can google it.

4

Thank you @Fiona.Chen. You are right that’s the problem.

I have a new issue related to these multi rtsp stream setup. I start the pipeline, then I get some gstreamer logs complaining about the NTP sync and finally I see the streams on the jetson display. The problem is that most of the times, the video completely freezes without raising any error, it just keep the same frame and it displays a new frame after several seconds. On the other, sometimes I start the same pipeline and it just works as expected. Therefore, I need to restart the pipeline many times (about 6) in order to get it work properly.

NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 

(python3:1): GStreamer-CRITICAL **: 17:29:21.038: gst_mini_object_unref: assertion 'mini_object != NULL' failed
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 

(python3:1): GStreamer-CRITICAL **: 17:29:21.135: gst_mini_object_unref: assertion 'mini_object != NULL' failed
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 

(python3:1): GStreamer-CRITICAL **: 17:29:21.354: gst_mini_object_unref: assertion 'mini_object != NULL' failed
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 
2020-12-03 17:29:21 pervasive-desktop app[1] WARNING gst-library-error-quark: Either host or Source 1 seems to be out of NTP sync          SYS TIME = 2020-12-03T17:29:21.511Z CALCULATED NTP TIME = 1970-01-01T00:00:00.000Z (5)/dvs/git/dirty/git-master_linux/deepstream/sdk/src/gst-plugins/gst-nvmultistream2/gstnvstreammux_ntp.cpp(97): check_if_sys_rtcp_time_is_ntp_sync (): /GstPipeline:pipeline0/GstNvStreamMux:mux_0
2020-12-03 17:29:21 pervasive-desktop app[1] WARNING gst-library-error-quark: Either host or Source 2 seems to be out of NTP sync          SYS TIME = 2020-12-03T17:29:21.517Z CALCULATED NTP TIME = 1970-01-01T00:00:00.000Z (5)/dvs/git/dirty/git-master_linux/deepstream/sdk/src/gst-plugins/gst-nvmultistream2/gstnvstreammux_ntp.cpp(97): check_if_sys_rtcp_time_is_ntp_sync (): /GstPipeline:pipeline0/GstNvStreamMux:mux_0
2020-12-03 17:29:21 pervasive-desktop app[1] WARNING gst-library-error-quark: Either host or Source 0 seems to be out of NTP sync          SYS TIME = 2020-12-03T17:29:21.675Z CALCULATED NTP TIME = 1970-01-01T00:00:00.000Z (5)/dvs/git/dirty/git-master_linux/deepstream/sdk/src/gst-plugins/gst-nvmultistream2/gstnvstreammux_ntp.cpp(97): check_if_sys_rtcp_time_is_ntp_sync (): /GstPipeline:pipeline0/GstNvStreamMux:mux_0

Do you have any suggestion?

5

I can not tell you anything with only these log. You can enable rtspsrc debug message to check the RTSP problems. rtspsrc
And RTSP spec may help you too. RFC 2326: Real Time Streaming Protocol (RTSP)

6

Thank you. These are the logs I get when it is not working (no matter whether I use uridecodebin or rtspsrc ! ... ! nvv4l2decoder) :

0:00:08.027100926     1     0x23ac9630 WARN                    v4l2 gstv4l2object.c:4430:gst_v4l2_object_probe_caps:<nvv4l2decoder0:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
0:00:08.027167327     1     0x23ac9630 WARN                    v4l2 gstv4l2object.c:2372:gst_v4l2_object_add_interlace_mode:0x7ec813b540 Failed to determine interlace mode
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 
0:00:08.046334900     1     0x23aca630 WARN                    v4l2 gstv4l2object.c:4430:gst_v4l2_object_probe_caps:<nvv4l2decoder1:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
0:00:08.046485781     1     0x23aca630 WARN                    v4l2 gstv4l2object.c:2372:gst_v4l2_object_add_interlace_mode:0x7ed40ddf40 Failed to determine interlace mode
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
0:00:08.053773070     1     0x23acb370 WARN                    v4l2 gstv4l2object.c:4430:gst_v4l2_object_probe_caps:<nvv4l2decoder2:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
0:00:08.054449267     1     0x23acb370 WARN                    v4l2 gstv4l2object.c:2372:gst_v4l2_object_add_interlace_mode:0x7edc09f570 Failed to determine interlace mode
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 
NvMMLiteBlockCreate : Block : BlockType = 261 
0:00:08.137650588     1     0x23ac9630 WARN                    v4l2 gstv4l2object.c:4430:gst_v4l2_object_probe_caps:<nvv4l2decoder0:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
0:00:08.137717213     1     0x23ac9630 WARN                    v4l2 gstv4l2object.c:2372:gst_v4l2_object_add_interlace_mode:0x7ec813b540 Failed to determine interlace mode
0:00:08.140341201     1     0x23ac9630 FIXME               basesink gstbasesink.c:3145:gst_base_sink_default_event:<mask_sink> stream-start event without group-id. Consider implementing group-id handling in the upstream elements
0:00:08.168530157     1     0x23acb370 WARN                    v4l2 gstv4l2object.c:4430:gst_v4l2_object_probe_caps:<nvv4l2decoder2:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
0:00:08.170108825     1     0x23acb370 WARN                    v4l2 gstv4l2object.c:2372:gst_v4l2_object_add_interlace_mode:0x7edc09f570 Failed to determine interlace mode

0:00:08.186803707     1     0x23aca630 WARN                    v4l2 gstv4l2object.c:4430:gst_v4l2_object_probe_caps:<nvv4l2decoder1:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
0:00:08.187718595     1     0x23aca630 WARN                    v4l2 gstv4l2object.c:2372:gst_v4l2_object_add_interlace_mode:0x7ed40ddf40 Failed to determine interlace mode

0:00:08.193446319     1     0x23aa3770 FIXME               basesink gstbasesink.c:3145:gst_base_sink_default_event:<door_sink> stream-start event without group-id. Consider implementing group-id handling in the upstream elements
0:00:08.353377486     1     0x23ac9630 FIXME               basesink gstbasesink.c:3145:gst_base_sink_default_event:<mask_sink> stream-start event without group-id. Consider implementing group-id handling in the upstream elements
0:00:08.353690513     1     0x23acb370 FIXME               basesink gstbasesink.c:3145:gst_base_sink_default_event:<mask_sink> stream-start event without group-id. Consider implementing group-id handling in the upstream elements
0:00:08.355205244     1     0x23ac9630 WARN            v4l2videodec gstv4l2videodec.c:1632:gst_v4l2_video_dec_decide_allocation:<nvv4l2decoder0> Duration invalid, not setting latency
0:00:08.357154476     1     0x23acb370 WARN            v4l2videodec gstv4l2videodec.c:1632:gst_v4l2_video_dec_decide_allocation:<nvv4l2decoder2> Duration invalid, not setting latency
0:00:08.363934145     1     0x23aa3230 WARN           basetransform gstbasetransform.c:1355:gst_base_transform_setcaps:<door_1_crop> transform could not transform video/x-raw(memory:NVMM), width=(int)2560, height=(int)1440, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)20/1, format=(string)RGBA, batch-size=(int)3, num-surfaces-per-frame=(int)1 in anything we support
0:00:08.366391668     1     0x23aa3850 WARN           basetransform gstbasetransform.c:1355:gst_base_transform_setcaps:<door_2_crop> transform could not transform video/x-raw(memory:NVMM), width=(int)2560, height=(int)1440, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)20/1, format=(string)RGBA, batch-size=(int)3, num-surfaces-per-frame=(int)1 in anything we support
0:00:08.366655926     1     0x23aa38f0 WARN           basetransform gstbasetransform.c:1355:gst_base_transform_setcaps:<door_3_crop> transform could not transform video/x-raw(memory:NVMM), width=(int)2560, height=(int)1440, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)20/1, format=(string)RGBA, batch-size=(int)3, num-surfaces-per-frame=(int)1 in anything we support
0:00:08.367627709     1     0x23aca630 WARN            v4l2videodec gstv4l2videodec.c:1632:gst_v4l2_video_dec_decide_allocation:<nvv4l2decoder1> Duration invalid, not setting latency
0:00:08.407117617     1     0x23acb370 WARN          v4l2bufferpool gstv4l2bufferpool.c:1057:gst_v4l2_buffer_pool_start:<nvv4l2decoder2:pool:src> Uncertain or not enough buffers, enabling copy threshold
0:00:08.413920166     1   0x7edc007e80 WARN          v4l2bufferpool gstv4l2bufferpool.c:1503:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder2:pool:src> Driver should never set v4l2_buffer.field to ANY
0:00:08.416973598     1     0x23ac9630 WARN          v4l2bufferpool gstv4l2bufferpool.c:1057:gst_v4l2_buffer_pool_start:<nvv4l2decoder0:pool:src> Uncertain or not enough buffers, enabling copy threshold
0:00:08.423810867     1     0x23aca630 WARN          v4l2bufferpool gstv4l2bufferpool.c:1057:gst_v4l2_buffer_pool_start:<nvv4l2decoder1:pool:src> Uncertain or not enough buffers, enabling copy threshold
0:00:08.435874641     1   0x7ed406d4a0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1503:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder0:pool:src> Driver should never set v4l2_buffer.field to ANY
0:00:08.444422708     1   0x7ec87c4b70 WARN          v4l2bufferpool gstv4l2bufferpool.c:1503:gst_v4l2_buffer_pool_dqbuf:<nvv4l2decoder1:pool:src> Driver should never set v4l2_buffer.field to ANY
0:00:08.446323715     1     0x23aa3850 WARN           basetransform gstbasetransform.c:1355:gst_base_transform_setcaps:<door_2_crop> transform could not transform video/x-raw(memory:NVMM), width=(int)2560, height=(int)1440, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)20/1, format=(string)RGBA, batch-size=(int)3, num-surfaces-per-frame=(int)1 in anything we support
0:00:08.447651405     1     0x23aa3230 WARN           basetransform gstbasetransform.c:1355:gst_base_transform_setcaps:<door_1_crop> transform could not transform video/x-raw(memory:NVMM), width=(int)2560, height=(int)1440, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)20/1, format=(string)RGBA, batch-size=(int)3, num-surfaces-per-frame=(int)1 in anything we support
0:00:08.447865775     1     0x23aa3850 WARN           basetransform gstbasetransform.c:1355:gst_base_transform_setcaps:<door_2_crop> transform could not transform video/x-raw(memory:NVMM), width=(int)2560, height=(int)1440, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)20/1, format=(string)RGBA, batch-size=(int)3, num-surfaces-per-frame=(int)1 in anything we support
0:00:08.447600941     1     0x23aa38f0 WARN           basetransform gstbasetransform.c:1355:gst_base_transform_setcaps:<door_3_crop> transform could not transform video/x-raw(memory:NVMM), width=(int)2560, height=(int)1440, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)20/1, format=(string)RGBA, batch-size=(int)3, num-surfaces-per-frame=(int)1 in anything we support
0:00:08.448349075     1     0x23aa38f0 WARN           basetransform gstbasetransform.c:1355:gst_base_transform_setcaps:<door_3_crop> transform could not transform video/x-raw(memory:NVMM), width=(int)2560, height=(int)1440, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)20/1, format=(string)RGBA, batch-size=(int)3, num-surfaces-per-frame=(int)1 in anything we support
0:00:08.447991440     1     0x23aa3230 WARN           basetransform gstbasetransform.c:1355:gst_base_transform_setcaps:<door_1_crop> transform could not transform video/x-raw(memory:NVMM), width=(int)2560, height=(int)1440, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)20/1, format=(string)RGBA, batch-size=(int)3, num-surfaces-per-frame=(int)1 in anything we support
0:00:08.549132197     1     0x23aa4720 WARN         nvstreammux_ntp gstnvstreammux_ntp.cpp:97:check_if_sys_rtcp_time_is_ntp_sync:<mux_0> warning: Either host or Source 2 seems to be out of NTP sync          SYS TIME = 2020-12-04T14:54:39.003Z CALCULATED NTP TIME = 1970-01-01T00:00:00.000Z
2020-12-04 14:54:39 pervasive-desktop app[1] WARNING gst-library-error-quark: Either host or Source 2 seems to be out of NTP sync          SYS TIME = 2020-12-04T14:54:39.003Z CALCULATED NTP TIME = 1970-01-01T00:00:00.000Z (5)/dvs/git/dirty/git-master_linux/deepstream/sdk/src/gst-plugins/gst-nvmultistream2/gstnvstreammux_ntp.cpp(97): check_if_sys_rtcp_time_is_ntp_sync (): /GstPipeline:pipeline0/GstNvStreamMux:mux_0
0:00:08.558628079     1     0x23aa3850 WARN           basetransform gstbasetransform.c:1355:gst_base_transform_setcaps:<door_2_crop> transform could not transform video/x-raw(memory:NVMM), width=(int)2560, height=(int)1440, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)20/1, format=(string)RGBA, batch-size=(int)3, num-surfaces-per-frame=(int)1 in anything we support
0:00:08.559802904     1     0x23aa3230 WARN           basetransform gstbasetransform.c:1355:gst_base_transform_setcaps:<door_1_crop> transform could not transform video/x-raw(memory:NVMM), width=(int)2560, height=(int)1440, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)20/1, format=(string)RGBA, batch-size=(int)3, num-surfaces-per-frame=(int)1 in anything we support
0:00:08.559715927     1     0x23aa38f0 WARN           basetransform gstbasetransform.c:1355:gst_base_transform_setcaps:<door_3_crop> transform could not transform video/x-raw(memory:NVMM), width=(int)2560, height=(int)1440, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)1/1, framerate=(fraction)20/1, format=(string)RGBA, batch-size=(int)3, num-surfaces-per-frame=(int)1 in anything we support
0:00:08.584179574     1     0x23aa4720 WARN         nvstreammux_ntp gstnvstreammux_ntp.cpp:97:check_if_sys_rtcp_time_is_ntp_sync:<mux_0> warning: Either host or Source 1 seems to be out of NTP sync          SYS TIME = 2020-12-04T14:54:39.038Z CALCULATED NTP TIME = 1970-01-01T00:00:00.000Z
2020-12-04 14:54:39 pervasive-desktop app[1] WARNING gst-library-error-quark: Either host or Source 1 seems to be out of NTP sync          SYS TIME = 2020-12-04T14:54:39.038Z CALCULATED NTP TIME = 1970-01-01T00:00:00.000Z (5)/dvs/git/dirty/git-master_linux/deepstream/sdk/src/gst-plugins/gst-nvmultistream2/gstnvstreammux_ntp.cpp(97): check_if_sys_rtcp_time_is_ntp_sync (): /GstPipeline:pipeline0/GstNvStreamMux:mux_0
0:00:08.597247068     1     0x23aa4720 WARN         nvstreammux_ntp gstnvstreammux_ntp.cpp:97:check_if_sys_rtcp_time_is_ntp_sync:<mux_0> warning: Either host or Source 0 seems to be out of NTP sync          SYS TIME = 2020-12-04T14:54:39.051Z CALCULATED NTP TIME = 1970-01-01T00:00:00.000Z
2020-12-04 14:54:39 pervasive-desktop app[1] WARNING gst-library-error-quark: Either host or Source 0 seems to be out of NTP sync          SYS TIME = 2020-12-04T14:54:39.051Z CALCULATED NTP TIME = 1970-01-01T00:00:00.000Z (5)/dvs/git/dirty/git-master_linux/deepstream/sdk/src/gst-plugins/gst-nvmultistream2/gstnvstreammux_ntp.cpp(97): check_if_sys_rtcp_time_is_ntp_sync (): /GstPipeline:pipeline0/GstNvStreamMux:mux_0
0:00:09.095631434     1   0x7f280034f0 WARN         nvstreammux_ntp gstnvstreammux_ntp.cpp:97:check_if_sys_rtcp_time_is_ntp_sync:<mux_1> warning: Either host or Source 0 seems to be out of NTP sync          SYS TIME = 2020-12-04T14:54:39.550Z CALCULATED NTP TIME = 1970-01-01T00:00:00.000Z
2020-12-04 14:54:39 pervasive-desktop app[1] WARNING gst-library-error-quark: Either host or Source 0 seems to be out of NTP sync          SYS TIME = 2020-12-04T14:54:39.550Z CALCULATED NTP TIME = 1970-01-01T00:00:00.000Z (5)/dvs/git/dirty/git-master_linux/deepstream/sdk/src/gst-plugins/gst-nvmultistream2/gstnvstreammux_ntp.cpp(97): check_if_sys_rtcp_time_is_ntp_sync (): /GstPipeline:pipeline0/GstNvStreamMux:mux_1

I think the importan part is in transform could not transform video/x-raw(memory:NVMM).... Everything goes wrong when this message appears. Sometimes it displays a completely green frame and other times it just freezes in the first frame.

7

No. This is the result but not the reason. The log shows there is some negotiation for the data received. Seems no correct data received. You need to look for the reason why there is no correct data received.

Can you get any rtsp log?