Deepstream & gstreamer rtsp input issue

**• Hardware Platform : Jetson AGX Xavier **
**• DeepStream Version : 5.0.1 / gstreamer 1.14.5 **
**• JetPack Version : 4.3 **
**• TensorRT Version : 7.1.3 **
**• NVIDIA GPU Driver Version : 10.2 **
Hello
I am testing deepstream(python) + tlt.

I have a problem with deepstream. IP camera rtsp input used for deepstream-uridecodebin pipeline is not working properly.

We are currently using IP camera and streaming to rtsp(tcp) - h264.

The interesting thing is that when video capture is being done using opencv, uridecodebin works perfectly in deepstream (+ gstreamer). However, when deepstream (or gstreamer) is soley used without any external connection, rtsp input is not received properly and the app gets frozen.
This is happening for deepstream python example code as well - deepstream-app(sample).

I have no idea why this is happening. Could you please help me?

[log]
** freezing **

xavier@xavier-desktop:~/Desktop$ gst-launch-1.0 uridecodebin uri=rtspt:///main ! nvoverlaysink
0:00:00.058084388 17081 0x5573578630 WARN omx gstomx.c:2826:plugin_init: Failed to load configuration file: Valid key file could not be found in search dirs (searched in: /home/xavier/.config:/etc/xdg/xdg-unity:/etc/xdg as per GST_OMX_CONFIG_DIR environment variable, the xdg user config directory (or XDG_CONFIG_HOME) and the system config directory (or XDG_CONFIG_DIRS)
Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Progress: (open) Opening Stream
Progress: (connect) Connecting to rtspt:///main
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (request) SETUP stream 1
Progress: (open) Opened Stream
Setting pipeline to PLAYING …
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request
Opening in BLOCKING MODE
0:00:00.314199828 17081 0x7fac080c00 WARN v4l2 gstv4l2object.c:4435:gst_v4l2_object_probe_caps:nvv4l2decoder0:src Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
0:00:00.314292312 17081 0x7fac080c00 WARN v4l2 gstv4l2object.c:2372:gst_v4l2_object_add_interlace_mode:0x7f8405eb00 Failed to determine interlace mode
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261

** freezing & ctrl + c **

^Chandling interrupt.
Interrupt: Stopping pipeline …
Execution ended after 0:00:23.461234930
Setting pipeline to PAUSED …
Setting pipeline to READY …
^C
xavier@xavier-desktop:~/Desktop$ ^C
xavier@xavier-desktop:~/Desktop$

** GST_DEBUG=5 **
0:00:02.773111506 23074 0x556b86a000 DEBUG rtspsrc gstrtspsrc.c:5139:gst_rtspsrc_loop_interleaved: doing receive with timeout 52 seconds, 981992 usec
0:00:02.773250648 23074 0x556b86a000 DEBUG rtspsrc gstrtspsrc.c:5149:gst_rtspsrc_loop_interleaved: we received a server message
0:00:02.773266105 23074 0x556b86a000 DEBUG rtspsrc gstrtspsrc.c:5182:gst_rtspsrc_loop_interleaved: got data message
0:00:02.773280890 23074 0x556b86a000 DEBUG GST_MEMORY gstmemory.c:139:gst_memory_init: new memory 0x7f8c313ad0, maxsize:1448 offset:0 size:1448
0:00:02.773295098 23074 0x556b86a000 DEBUG rtspsrc gstrtspsrc.c:4948:gst_rtspsrc_handle_data: pushing data of size 1448 on channel 0
0:00:02.773311451 23074 0x556b86a000 DEBUG GST_SCHEDULING gstpad.c:4320:gst_pad_chain_data_unchecked:manager:recv_rtp_sink_0 calling chainfunction &gst_proxy_pad_chain_default with buffer buffer: 0x7f8c310bf0, pts 99:99:99.999999999, dts 99:99:99.999999999, dur 99:99:99.999999999, size 1448, offset none, offset_end none, flags 0x4000
0:00:02.773329084 23074 0x556b86a000 DEBUG GST_SCHEDULING gstpad.c:4320:gst_pad_chain_data_unchecked:funnel0:funnelpad0 calling chainfunction &gst_funnel_sink_chain with buffer buffer: 0x7f8c310bf0, pts 99:99:99.999999999, dts 99:99:99.999999999, dur 99:99:99.999999999, size 1448, offset none, offset_end none, flags 0x4000
0:00:02.773344028 23074 0x556b86a000 DEBUG funnel gstfunnel.c:323:gst_funnel_sink_chain_object:funnel0:funnelpad0 received buffer: 0x7f8c310bf0, pts 99:99:99.999999999, dts 99:99:99.999999999, dur 99:99:99.999999999, size 1448, offset none, offset_end none, flags 0x4000
0:00:02.773359773 23074 0x556b86a000 DEBUG GST_SCHEDULING gstpad.c:4320:gst_pad_chain_data_unchecked:rtpsession0:recv_rtp_sink calling chainfunction &0x7f90151980 with buffer buffer: 0x7f8c310bf0, pts 99:99:99.999999999, dts 99:99:99.999999999, dur 99:99:99.999999999, size 1448, offset none, offset_end none, flags 0x4000
0:00:02.773388383 23074 0x556b86a000 DEBUG GST_CLOCK gstclock.c:1046:gst_clock_get_internal_time: internal time 3:55:31.602736599
0:00:02.773402719 23074 0x556b86a000 DEBUG GST_CLOCK gstclock.c:1091:gst_clock_get_time: adjusted time 3:55:31.602736599
0:00:02.773417056 23074 0x556b86a000 DEBUG GST_CLOCK gstclock.c:1046:gst_clock_get_internal_time: internal time 3:55:31.602779609
0:00:02.773429536 23074 0x556b86a000 DEBUG GST_CLOCK gstclock.c:1091:gst_clock_get_time: adjusted time 3:55:31.602779609
0:00:02.773452546 23074 0x556b86a000 DEBUG GST_SCHEDULING gstpad.c:4320:gst_pad_chain_data_unchecked:rtpstorage0:sink calling chainfunction &0x7f900e2098 with buffer buffer: 0x7f8c310bf0, pts 99:99:99.999999999, dts 99:99:99.999999999, dur 99:99:99.999999999, size 1448, offset none, offset_end none, flags 0x4000
0:00:02.773472450 23074 0x556b86a000 DEBUG GST_SCHEDULING gstpad.c:4320:gst_pad_chain_data_unchecked:rtpssrcdemux0:sink calling chainfunction &0x7f9013c558 with buffer buffer: 0x7f8c310bf0, pts 99:99:99.999999999, dts 99:99:99.999999999, dur 99:99:99.999999999, size 1448, offset none, offset_end none, flags 0x4000
0:00:02.773492483 23074 0x556b86a000 DEBUG rtpssrcdemux gstrtpssrcdemux.c:625:gst_rtp_ssrc_demux_chain: received buffer of SSRC ddde56a9
0:00:02.773508964 23074 0x556b86a000 DEBUG GST_SCHEDULING gstpad.c:4320:gst_pad_chain_data_unchecked:rtpjitterbuffer0:sink calling chainfunction &gst_rtp_jitter_buffer_chain with buffer buffer: 0x7f8c310bf0, pts 99:99:99.999999999, dts 99:99:99.999999999, dur 99:99:99.999999999, size 1448, offset none, offset_end none, flags 0x4000
0:00:02.773527205 23074 0x556b86a000 DEBUG GST_CLOCK gstclock.c:1046:gst_clock_get_internal_time: internal time 3:55:31.602887966
0:00:02.773545318 23074 0x556b86a000 DEBUG GST_CLOCK gstclock.c:1091:gst_clock_get_time: adjusted time 3:55:31.602887966
0:00:02.773562119 23074 0x556b86a000 DEBUG rtpjitterbuffer gstrtpjitterbuffer.c:2957:gst_rtp_jitter_buffer_chain: Received packet #1309 at time 0:00:02.022027312, discont 0, rtx 0
0:00:02.773576647 23074 0x556b86a000 DEBUG rtpjitterbuffer gstrtpjitterbuffer.c:3055:gst_rtp_jitter_buffer_chain: expected #1309, got #1309, gap of 0
0:00:02.773587112 23074 0x556b86a000 DEBUG rtpjitterbuffer gstrtpjitterbuffer.c:3084:gst_rtp_jitter_buffer_chain: Clearing gap packets
0:00:02.773601544 23074 0x556b86a000 DEBUG rtpjitterbuffer gstrtpjitterbuffer.c:3230:gst_rtp_jitter_buffer_chain: Pushed packet #1309, now 1305 packets, head: 0, percent -1
0:00:02.773614825 23074 0x556b86a000 DEBUG GST_SCHEDULING gstpad.c:4326:gst_pad_chain_data_unchecked:rtpjitterbuffer0:sink called chainfunction &gst_rtp_jitter_buffer_chain with buffer 0x7f8c310bf0, returned ok
0:00:02.773628234 23074 0x556b86a000 DEBUG GST_SCHEDULING gstpad.c:4326:gst_pad_chain_data_unchecked:rtpssrcdemux0:sink called chainfunction &0x7f9013c558 with buffer 0x7f8c310bf0, returned ok
0:00:02.773641610 23074 0x556b86a000 DEBUG GST_SCHEDULING gstpad.c:4326:gst_pad_chain_data_unchecked:rtpstorage0:sink called chainfunction &0x7f900e2098 with buffer 0x7f8c310bf0, returned ok
0:00:02.773654411 23074 0x556b86a000 DEBUG GST_SCHEDULING gstpad.c:4326:gst_pad_chain_data_unchecked:rtpsession0:recv_rtp_sink called chainfunction &0x7f90151980 with buffer 0x7f8c310bf0, returned ok
0:00:02.773668363 23074 0x556b86a000 DEBUG GST_SCHEDULING gstpad.c:4326:gst_pad_chain_data_unchecked:funnel0:funnelpad0 called chainfunction &gst_funnel_sink_chain with buffer 0x7f8c310bf0, returned ok
0:00:02.773681388 23074 0x556b86a000 DEBUG GST_SCHEDULING gstpad.c:4326:gst_pad_chain_data_unchecked:manager:recv_rtp_sink_0 called chainfunction &gst_proxy_pad_chain_default with buffer 0x7f8c310bf0, returned ok
0:00:02.773695533 23074 0x556b86a000 DEBUG rtspsrc gstrtspsrc.c:5139:gst_rtspsrc_loop_interleaved: doing receive with timeout 52 seconds, 981407 usec
0:00:02.773724654 23074 0x556b86a000 DEBUG rtspsrc gstrtspsrc.c:5149:gst_rtspsrc_loop_interleaved: we received a server message


** when video capture is being done using opencv **

xavier@xavier-desktop:~$ gst-launch-1.0 uridecodebin uri=rtspt:///main ! nvoverlaysink
Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Progress: (open) Opening Stream
Progress: (connect) Connecting to rtspt:///main
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (request) SETUP stream 1
Progress: (open) Opened Stream
Setting pipeline to PLAYING …
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request
Opening in BLOCKING MODE
NvMMLiteOpen : Block : BlockType = 261
NVMEDIA: Reading vendor.tegra.display-size : status: 6
NvMMLiteBlockCreate : Block : BlockType = 261
reference in DPB was never decoded

** doing well & ctrl + c **

^Chandling interrupt.
Interrupt: Stopping pipeline …
Execution ended after 0:00:03.073757789
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to NULL …
Freeing pipeline …

Can you explain what does “video capture is being done using opencv” mean? What does “deepstream (or gstreamer) is soley used without any external connection” mean? The followed command are both “gst-launch” commands.

“video capture is being done using openv” means capturing camera inputs in the background using a code snippet below.


import cv2 as cv

cap2 = cv.VideoCapture(‘rtsp:///main’) # it can be rtsp or http $

while True:
ret, frame = cap2.read()


I ran “gst-launch” in the same way for both conditions (using and not using openCV)
(gst-launch-1.0 uridecodebin uri=rtspt:///ip:port/main ! nvoverlaysink)

I have absolutely no idea why it is happening.

We don’t know the mechanism with opencv. But when I run “gst-launch-1.0 uridecodebin uri=rtspt://xxxxx ! nvoverlaysink” command in my Xavier board, it works well.

As far as I know, opencv VideoCapture only supports appsink pipeline, so the pipeline is different to “gst-launch-1.0 uridecodebin uri=rtspt://xxxxx ! nvoverlaysink”.

It is not a deepstream issue.

Thank you for answer.

Find out more reasons for the problem and share if I can confirm it.