Gst-play-1.0, gst-launch-1.0 unable to display RTSP stream on Xavier AGX

Autonomous Machines Jetson & Embedded Systems Jetson AGX Xavier
, ,
1

I need to display an RTSP stream using the gst-play-1.0 and/ or gst-launch-1.0 commands on an NVIDIA Jetson Xavier AGX device.

These are the commands I have tried so far:

1. gst-play-1.0

$ gst-play-1.0 rtsp://192.168.1.xxx:8554/main.264

in which case the terminal remains stuck at:

Press 'k' to see a list of keyboard shortcuts.
Now playing rtsp://192.168.1.xxx:8554/main.264
Pipeline is live.
Opening in BLOCKING MODE 
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 
Prerolled.

2. gst-launch-1.0

$ gst-launch-1.0 rtspsrc location=rtsp://192.168.1.xxx:8554/main.264 latency=0 buffer-mode=auto ! queue ! rtph264depay ! h264parse ! avdec_h264 ! videoconvert ! videoscale ! video/x-raw,width=1920,height=1080 ! autovideosink

in which case the terminal remains stuck at:

Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Progress: (open) Opening Stream
Progress: (connect) Connecting to rtsp://192.168.1.xxx:8554/main.264
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (open) Opened Stream
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request

After pressing Ctrl+C:

^Chandling interrupt.
Interrupt: Stopping pipeline ...
Execution ended after 0:00:02.188911578
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

3. gst-launch-1.0

gst-launch-1.0 rtspsrc location=rtsp://username:password@192.168.1.xxx:554 latency=500 ! queue ! rtph264depay ! h264parse ! nvv4l2decoder ! nvvidconv ! 'video/x-raw(memory:NVMM),width=1920,height=1080' ! nvvidconv ! autovideosink

and the terminal displayed the following:

Setting pipeline to PAUSED ...
Opening in BLOCKING MODE 
Pipeline is live and does not need PREROLL ...
Progress: (open) Opening Stream
Progress: (connect) Connecting to rtsp://username:password@192.168.1.xxx:554
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (open) Opened Stream
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 

(gst-launch-1.0:14920): GStreamer-CRITICAL **: 21:59:35.771: gst_mini_object_unref: assertion 'mini_object != NULL' failed
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261

The URLs are typically of the following formats:

rtsp://192.168.1.xxx:8554/main.264
rtsp://username:password@192.168.1.xxx:554

I am able to use the commands on a x86 PC with Ubuntu 20.04 and Gstreamer 1.16.3 as well as a Jetson Nano. So, the camera feeds themselves are fine.

But, the commands don’t work on the Xavier device.

Output from gst-discoverer-1.0 rtsp://username:password@192.168.1.xxx:554 is:

Analyzing rtsp://username:password@192.168.1.xxx:554
Opening in BLOCKING MODE 
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 
Done discovering rtsp://username:password@192.168.1.xxx:554

Topology:
  unknown: application/x-rtp
    video: H.264 (Main Profile)

Properties:
  Duration: 99:99:99.999999999
  Seekable: no
  Live: yes
  Tags: 
      video codec: H.264 (Main Profile)

NVIDIA Jetson- Xavier AGX device info:

L4T 32.6.1 [ JetPack 4.6 ]

Ubuntu 18.04.6 LTS

Kernel Version: 4.9.253-tegra

GStreamer 1.14.5

CUDA 10.2.300

CUDA Architecture: NONE

OpenCV version: 4.1.1

OpenCV Cuda: NO

CUDNN: 8.2.1.32

TensorRT: 8.0.1.6

Vision Works: 1.6.0.501

VPI: 1.1.12

Vulcan: 1.2.70

Thanks for going through the post. Any hint(s) or guidance would be most appreciated.

Thank you. :)

3

Hi,
Please try the command on AGX Xavier and see if video preview is shown:

$ gst-launch-1.0 rtspsrc location=rtsp://username:password@192.168.1.xxx:554 latency=500 ! queue ! rtph264depay ! h264parse ! nvv4l2decoder ! nvvidconv ! 'video/x-raw(memory:NVMM)' ! nv3dsink sync=0
4

Hey @DaneLLL , thanks for the peompt response. The above command did not work the first time, although after restarting the device it did. Also, the following was displayed on the terminal:

Setting pipeline to PAUSED ...
Opening in BLOCKING MODE 
Pipeline is live and does not need PREROLL ...
Progress: (open) Opening Stream
Progress: (connect) Connecting to rtsp://username:password@192.168.1.xxx:554
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (open) Opened Stream
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 

(gst-launch-1.0:21045): GStreamer-CRITICAL **: 03:58:49.713: gst_mini_object_unref: assertion 'mini_object != NULL' failed
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261

Is the above a cause for concern?

The following command also worked:

gst-launch-1.0 -v uridecodebin uri=rtsp://username:password@192.168.1.xxx:554 ! nvvidconv ! nvegltransform ! nveglglessink

I also have C++ code which uses uridecodebin, nvstreammux at the beginning of the pipeline. I tried measuring the latency using the NVDS_ENABLE_COMPONENT_LATENCY_MEASUREMENT method, it turns out that nvv4l2decoder and nvstreamux timings are fluctuating wildly, sometimes even crossing 500ms individually. It is causing a delay to constantly build up. I have checked jtop and can confirm that NVDEC is being used. What could be the possible reason for this? Any help/ guidance at all is welcome.

Regards.

5

Hi,
Please apply the patch to gst-v4l2 and rebuild the prebuilt lib:
Jetson/L4T/r32.6.x patches - eLinux.org

[gstreamer] Memory leak in UDP streaming

This fixes a memory leak in UDP/RTSP streaming. The source code of gst-v4l2 is in
https://developer.nvidia.com/embedded/linux-tegra-r3261

L4T Driver Package (BSP) Sources

1 Like
6

Hey @DaneLLL ! Thanks again for the solution.
I just wanted to confirm, if in your experience, this issue shows up randomly while running a pipeline or is it a consistent issue?

Also, currently the device is being used for development work, so burning the OS again and installing all other dependencies may not be feasible time wise. Would any other workaround be possible?

Regards.

7

Hi,
From our experience, it looks to be an issue in the RTSP source. Looks like decoding cannot begin in certain condition. For further check, you can try to launch RTSP server through test-launch. There are steps in
Jetson Nano FAQ

Q: Is there any example of running RTSP streaming?

We would expect the stream through test-launch* can be well decoded each time.

1 Like
8

Hey @DaneLLL ,
Thanks a lot, for your prompt guidance.
I was able to run and use test-launch like an RTSP server and it did make a difference:

$ ./test-launch videotestsrc is-live=1 ! x264enc ! rtph264pay name=pay0 pt=96

But then, in the real world, i won’t have control over the RTSP source. So, what steps can i take then? Are there any particular settings, i should target on the RTSP server end?

I also did some further testing, and it seems that the time taken by nvv4l2decoder(uridecodebin), nvstreammux changes based on other plugins which i use. For example, the compute timings for the above 2 plugins are very different in these 2 cases:

The timings are significantly higher for the second option. Any reason why nvv4l2decoder(uridecodebin), nvstreammux should have different compute timings **
** because of different plugins for object detection?
The difference in the compute timings for the 2 different object detection plugins is still understandable. The difference is present even when using test-launch as the RTSP server.

Thanks once again, for your time.

Regards.

9

Hi,

For this, you may check if the source has AU delimiter, SPS/PPS in each IDR frames, and if there is VUI in SPS. And may try to change IDR interval.

I think this is misunderstanding. It should be the decoded buffers are all used and no available buffers for putting next decoded frame data. If each buffer is occupied in nvinfer for a longer-than-frame-date time, the whole pipeline gets slowed down. This is due to heavier model that nvinfer is running.

10

Hey @DaneLLL ,

Thanks once again for taking an interest in this query. I want to update and convey a few points, regarding this.

1(a). I tested with 2 RTSP cameras and the decoder time is indeed different for both of them for the same frame size. Just to confirm what you asked me to check with test-launch. I have yet to get around to checking the parameters you mentioned, though in the real world, i may not always have that luxury.
By the way, does test-launch have an option to set frame size? The default is 320x240, and i would like to try with a 1920x1080 frame.

1(b). In nvstreammux, when i set live-source=1, for any possible improvement in performance, the pipeline gets stuck after 25-30 frames. Could you please point out where am I doing wrong here?

  1. I want to better explain the other issue better.

I have a pipeline which has the following flow:

urisourcebin → nvstreammux → object_detector_plugin → object_tracker_plugin → rest_of_the_pipeline(ending with udpsink)

(a) object_detector_plugin: nvinfer(yolov4-tiny)
object_tracker_plugin: nvtracker

(b) object_detector_plugin: custom_plugin_yolov7(yolo_deepstream/tensorrt_yolov7/samples/detect.cpp at main · NVIDIA-AI-IOT/yolo_deepstream · GitHub with yolov7_qat_640.engine)
object_tracker_plugin: custom_plugin_tracker

What i want to convey is that compute timings of specifically nvv4l2decoder(uridecodebin), nvstreammux are different for option (a) and (b), and not that the whole pipeline’s timing is affected because of a heavier model, which of course it logically will.

Why do plugins which come after nvv4l2decoder(uridecodebin), nvstreammux, affect their timings? These two plugins’ compute timings have approximately 90% contribution to the pipeline’s total compute timing.

Regards.