Gstreamer with Opencv for rtsp stream


I am having trouble using the gstreamer rstp with opencv on Jetson.
I can use gst launcher command to open and display the rstp source.
I searched online that some people has successfully using this code to read rtsp in opencv.

import cv2

def open_cam_rtsp(uri, width, height, latency):
    gst_str = ("rtspsrc location={} latency={} ! rtph264depay ! h264parse ! omxh264dec ! "
               "nvvidconv ! video/x-raw, width=(int){}, height=(int){}, format=(string)BGRx ! "
               "videoconvert ! appsink").format(uri, latency, width, height)

    return cv2.VideoCapture(gst_str, cv2.CAP_GSTREAMER)

uri = 'rtsp://'
latency = 100
width = 640
height = 480

cap = open_cam_rtsp(uri, width, height, latency)


But somehow i get this error:

[ WARN:0] global /home/correct-ai/opencv-4.4.0/modules/videoio/src/cap_gstreamer.cpp (1761) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module udpsrc3 reported: Internal data stream error.
[ WARN:0] global /home/correct-ai/opencv-4.4.0/modules/videoio/src/cap_gstreamer.cpp (888) open OpenCV | GStreamer warning: unable to start pipeline
[ WARN:0] global /home/correct-ai/opencv-4.4.0/modules/videoio/src/cap_gstreamer.cpp (480) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created

And cap.isOpened() gives me False.
Did I make a mistake somewhere?


We have deprecated omx plugins. Please replace omxh264dec with nvv4l2decoder for a try.

Please also try this sample:
Doesn't work nvv4l2decoder for decoding RTSP in gstreamer + opencv - #3 by DaneLLL


Thanks for the reply,
After replace omxh264dec with nvv4l2decoder , it still cannot open the cap. (cap.isOpened returns False), and the output is the same.

I also tried your example with public link, it works well (with some warnings showing on the terminal).
But when I replaced the rtsp location to my camera, it does not work anymore.

I use gst-launch-1.0 command and it works fine with the camera, which suggests no connection issue.

So this command works:

$ gst-launch-1.0 rtspsrc location='rtsp://' ! rtph264depay ! h264parse ! nvv4l2decoder ! nvvidconv ! video/x-raw,format=BGRx ! videoconvert ! video/x-raw,format=BGR ! fakesink

but it fails if you apply it to cv2.VideoCapture() with appsink. A bit strange for this situation. Generally if it works in gst-launch-1.0 command, it should also work in cv2.VideoCapture().


Sorry for the late reply, actually the command you put there is not what i used,
I was using this command:

$ gst-launch-1.0 -v playbin uri=rtsp:// uridecodebin0::source::latency=10

it shows the video feed correctly.

When I try the command you provided, it cannot open the video feed:
The output:

Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Progress: (open) Opening Stream
Progress: (connect) Connecting to rtsp://
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (request) SETUP stream 1
Progress: (open) Opened Stream
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request
WARNING: from element /GstPipeline:pipeline0/GstRTSPSrc:rtspsrc0: Delayed linking failed.
Additional debug info:
./grammar.y(510): gst_parse_no_more_pads (): /GstPipeline:pipeline0/GstRTSPSrc:rtspsrc0:
failed delayed linking some pad of GstRTSPSrc named rtspsrc0 to some pad of GstRtpH264Depay named rtph264depay0
ERROR: from element /GstPipeline:pipeline0/GstRTSPSrc:rtspsrc0/GstUDPSrc:udpsrc3: Internal data stream error.
Additional debug info:
gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstRTSPSrc:rtspsrc0/GstUDPSrc:udpsrc3:
streaming stopped, reason not-linked (-1)
Execution ended after 0:00:02.109489167
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

Please set

$ export GST_DEBUG=*FACTORY*:4

Adn run the working command. The elements picked by playbin will be printed. It shows how the gstreamer pipeline is constructed.

Thank you very much, It seems the camera has been using h265.

If the source is h265 stream, please try rtph265depay and h265parse:

$ gst-launch-1.0 rtspsrc location='rtsp://' ! rtph265depay ! h265parse ! nvv4l2decoder ! nvvidconv ! video/x-raw,format=BGRx ! videoconvert ! video/x-raw,format=BGR ! fakesink

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.