Rtsp decoding deadlocks

Robotics & Edge Computing Jetson & Embedded Systems Jetson Xavier NX
,
1

Hello everyone,

I’m using Gstreamer in order to receive live RTSP stream, and decode it using nvv4l2decoder.
The pipeline is the following:
rtspsrc location=%s name=rtsp timeout=100000 tcp-timeout=100000 ! rtph264depay name=depay ! h264parse ! nvv4l2decoder disable-dpb=true ! nvvidconv ! video/x-raw,format=RGBA,width=%d,height=%d ! appsink name=app_sink sync=false

Since the source is not stable, it may disconnect from time to time, or may not connect in the first attemp.
To handle this, whenever I receive an error in the bus, I’m waiting 1 second, and then just throw away the entire pipeline, and construct it again (by setting state to NULL and unrefing the pipeline).

Everything works fine 99% of the time. However, in some cases, when I try to throw away the pipeline, gst_element_set_state(pipeline, GST_STATE_NULL) just deadlocks. Also, I’m not calling this function from any streaming threads but rather the application thread.

I haven’t noticed that behaviour when I’m using avdec_h264, so I’m assuming it’s related to the nvv4l2 decoder.
Also, it always occurs when the decoder moves to PLAYING state but no data has flown into it yet (from what I’ve seen at least). I know this because the decoder prints some messages:
`
NvMMLiteOpen : Block : BlockType = 261

NVMEDIA: Reading vendor.tegra.display-size : status: 6

NvMMLiteBlockCreate : Block : BlockType = 261
`
Whenever I see this, and I don’t get any video on my appsink - I know I’m in for a treat.

As I said, the RTSP source is not stable and many times a connection is made, but no data is flowing from it. Maybe during this time I’m not allowed to change the deocder state to NULL?

I’d appreciate any help with this issue.
Thank you very much.

Omer.

3

Hi,
Since gst-v4l2 is open source and in the package:
https://developer.nvidia.com/embedded/l4t/r32_release_v5.1/r32_release_v5.1/sources/t186/public_sources.tbz2
We suggest download it to add some debug prints to check where the deadlock happens.

If there is a consistent way to replicate the deadlock, please share the steps so that we can give it a try.

4

Hello again,

The issue is now reproducing every time. Nothing is changed with my implementation, but I think it’s related to my RTSP server overheating.

I have discovered that it only occurs when using h264parse before nvv4l2decoder. Without it - it works fine.
Also, I’m getting a CRITICAL error when using it. That error was always there when used the h264parse before the decoder and usually it worked fine.
Please see the output:

$ GST_DEBUG=2 gst-launch-1.0 -e rtspsrc location=rtsp://192.168.0.31:551/mux1.sdp ! rtph264depay ! queue ! h264parse ! identity dump=true ! nvv4l2decoder ! fakesink
Setting pipeline to PAUSED ...
Opening in BLOCKING MODE
Opening in BLOCKING MODE 
0:00:00.188231451 14093   0x55c8bb1120 WARN                    v4l2 gstv4l2object.c:4434:gst_v4l2_object_probe_caps:<nvv4l2decoder0:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
0:00:00.188355005 14093   0x55c8bb1120 WARN                    v4l2 gstv4l2object.c:2375:gst_v4l2_object_add_interlace_mode:0x55c8ba1b90 Failed to determine interlace mode
0:00:00.188434782 14093   0x55c8bb1120 WARN                    v4l2 gstv4l2object.c:2375:gst_v4l2_object_add_interlace_mode:0x55c8ba1b90 Failed to determine interlace mode
Pipeline is live and does not need PREROLL ...
Progress: (open) Opening Stream
Progress: (connect) Connecting to rtsp://192.168.0.31:551/mux1.sdp
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (open) Opened Stream
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 
0:00:03.453110565 14093   0x55c8b3a8a0 WARN               h264parse gsth264parse.c:1197:gst_h264_parse_handle_frame:<h264parse0> input stream is corrupt; it contains a NAL unit of length 0
0:00:03.453290311 14093   0x55c8b3a8a0 WARN               h264parse gsth264parse.c:1197:gst_h264_parse_handle_frame:<h264parse0> input stream is corrupt; it contains a NAL unit of length 0

(gst-launch-1.0:14093): GStreamer-CRITICAL **: 13:52:49.833: gst_mini_object_unref: assertion 'mini_object != NULL' failed
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 
00000000 (0x7f4c013b80): 00 00 00 01 09 f0 00 00 00 01 67 42 40 1e a6 80  ..........gB@...
00000010 (0x7f4c013b90): b4 12 64                                         ..d

Then the pipeline is just stuck and EOS is never received.

When using avdec_h264:

$ GST_DEBUG=2 gst-launch-1.0 -e rtspsrc location=rtsp://192.168.0.31:551/mux1.sdp ! rtph264depay ! queue ! h264parse ! avdec_h264 ! fakesink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Progress: (open) Opening Stream
Progress: (connect) Connecting to rtsp://192.168.0.31:551/mux1.sdp
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (open) Opened Stream
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request

No warnings or errors.

Why is the h264parse cause this issue? Can I just remove it from the pipeline and not worry that things will go horribly wrong?

Thanks ahead,
Omer

5

Hi,
Do you use r32.5.1 or r32.4.4? If you use previous release, may hit this known issue:
Issues decoding RTSP stream using nvv4l2decoder with Jetpack 4.4 - #27 by DaneLLL

7

This topic was automatically closed 2 days after the last reply. New replies are no longer allowed.