Reading RTSP stream using Gsteamer in Opencv in C++

Robotics & Edge Computing Jetson & Embedded Systems Jetson Nano
1

Hi,

I am trying to read RTSP stream from a Hikvision IP camera in the network using Gsteamer in Opencv in C++. I had a working code with Opencv version 3.3.1 and Gstreamer version 1.14.1.

const char *gst = "rtspsrc location=rtsp://username:passowrd@ip:port/streamPath  ! application/x-rtp, media=(string)video ! decodebin ! video/x-raw, format=(string)NV12 ! videoconvert ! video/x-raw, format=(string)BGR  ! appsink";

The same is not working on Opencv Version 4.0.0 and Gstreamer version 1.14.5.

The latest jetpack comes up bundled with Gstreamer version 1.14.5 and downgrading it is not a feasible solution for me.

Has anyone recently tried to read the RTSP stream in a similar way?

Thanks in advance for your help.

2

Hi,

I don’t think that the GStreamer version is your problem because our pipelines with rtspsrc are very similar and they work just fine on GStreamer 1.14.5. You can test if your pipeline has errors outside OpenCV with gst-launch. On a terminal run:

export GST_DEBUG=2; gst-launch-1.0 rtspsrc location=rtsp://username:passowrd@ip:port/streamPath  ! decodebin ! videoconvert ! video/x-raw,format=BGR ! fakesink

I removed some caps that aren’t really needed and switched the appsink for a fakesink. ‘export GST_DEBUG=2’ will let you see all GStreamer errors and warnings.

3

Hi,

Got following response after adding your changes

cctv-ola3@cctvola3:~$ export GST_DEBUG=2
cctv-ola3@cctvola3:~$ gst-launch-1.0 rtspsrc location=rtsp://admin:welcome123@10.218.10.22:554/Streaming/Channels/102 ! decodebin ! videoconvert ! video/x-raw,format=BGR ! fakesink

Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Progress: (open) Opening Stream
Progress: (connect) Connecting to rtsp://admin:welcome123@10.218.10.22:554/Streaming/Channels/102
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (open) Opened Stream
Setting pipeline to PLAYING ...
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request
Opening in BLOCKING MODE 
0:00:00.555802930 16463   0x7f680028f0 WARN                    v4l2 gstv4l2object.c:4408:gst_v4l2_object_probe_caps:<nvv4l2decoder0:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
0:00:00.555874181 16463   0x7f680028f0 WARN                    v4l2 gstv4l2object.c:2370:gst_v4l2_object_add_interlace_mode:0x7f580b2350 Failed to determine interlace mode
NvMMLiteOpen : Block : BlockType = 261 
NVMEDIA: Reading vendor.tegra.display-size : status: 6 
NvMMLiteBlockCreate : Block : BlockType = 261 
0:00:00.660062487 16463   0x7f680028f0 WARN                    v4l2 gstv4l2object.c:4408:gst_v4l2_object_probe_caps:<nvv4l2decoder0:src> Failed to probe pixel aspect ratio with VIDIOC_CROPCAP: Unknown error -1
0:00:00.660154885 16463   0x7f680028f0 WARN                    v4l2 gstv4l2object.c:2370:gst_v4l2_object_add_interlace_mode:0x7f580b2350 Failed to determine interlace mode
0:00:00.661328189 16463   0x7f680028f0 WARN           basetransform gstbasetransform.c:1355:gst_base_transform_setcaps:<videoconvert0> transform could not transform video/x-raw(memory:NVMM), format=(string)NV12, width=(int)640, height=(int)480, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)126/95, chroma-site=(string)jpeg, colorimetry=(string)bt601, framerate=(fraction)25/1 in anything we support
0:00:00.661555120 16463   0x7f680028f0 WARN           basetransform gstbasetransform.c:1355:gst_base_transform_setcaps:<videoconvert0> transform could not transform video/x-raw(memory:NVMM), format=(string)NV12, width=(int)640, height=(int)480, interlace-mode=(string)progressive, multiview-mode=(string)mono, multiview-flags=(GstVideoMultiviewFlagsSet)0:ffffffff:/right-view-first/left-flipped/left-flopped/right-flipped/right-flopped/half-aspect/mixed-mono, pixel-aspect-ratio=(fraction)126/95, chroma-site=(string)jpeg, colorimetry=(string)bt601, framerate=(fraction)25/1 in anything we support
0:00:00.661607517 16463   0x7f680028f0 WARN                GST_PADS gstpad.c:4226:gst_pad_peer_query:<decodebin0:src_0> could not send sticky events
0:00:00.661937992 16463   0x7f680028f0 WARN            v4l2videodec gstv4l2videodec.c:1433:gst_v4l2_video_dec_decide_allocation:<nvv4l2decoder0> Duration invalid, not setting latency
0:00:00.662312895 16463   0x7f680028f0 WARN          v4l2bufferpool gstv4l2bufferpool.c:1054:gst_v4l2_buffer_pool_start:<nvv4l2decoder0:pool:src> Uncertain or not enough buffers, enabling copy threshold
0:00:00.664698045 16463   0x7f680028f0 WARN           basetransform gstbasetransform.c:1355:gst_base_transform_setcaps:<capsfilter1> transform could not transform video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, level=(string)3, profile=(string)main, pixel-aspect-ratio=(fraction)126/95, width=(int)640, height=(int)480, framerate=(fraction)25/1, interlace-mode=(string)progressive, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true in anything we support
0:00:00.664769713 16463   0x7f680028f0 WARN           basetransform gstbasetransform.c:1415:gst_base_transform_reconfigure:<capsfilter1> warning: not negotiated
0:00:00.664836642 16463   0x7f680028f0 WARN           basetransform gstbasetransform.c:1415:gst_base_transform_reconfigure:<capsfilter1> warning: not negotiated
WARNING: from element /GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstCapsFilter:capsfilter1: not negotiated
Additional debug info:
gstbasetransform.c(1415): gst_base_transform_reconfigure (): /GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstCapsFilter:capsfilter1:
not negotiated
0:00:00.666117760 16463   0x7f680028f0 WARN           basetransform gstbasetransform.c:1355:gst_base_transform_setcaps:<capsfilter1> transform could not transform video/x-h264, stream-format=(string)byte-stream, alignment=(string)au, level=(string)3, profile=(string)main, pixel-aspect-ratio=(fraction)126/95, width=(int)640, height=(int)480, framerate=(fraction)25/1, interlace-mode=(string)progressive, chroma-format=(string)4:2:0, bit-depth-luma=(uint)8, bit-depth-chroma=(uint)8, parsed=(boolean)true in anything we support
0:00:00.666157292 16463   0x7f680028f0 WARN           basetransform gstbasetransform.c:1415:gst_base_transform_reconfigure:<capsfilter1> warning: not negotiated
0:00:00.666179272 16463   0x7f680028f0 WARN           basetransform gstbasetransform.c:1415:gst_base_transform_reconfigure:<capsfilter1> warning: not negotiated
WARNING: from element /GstPipeline:pipeline0/GstDecodeBin:decodebin0/GstCapsFilter:capsfilter1: not negotiated
4

It seems your stream is encoded in h264 and you don’t have the depayloader in the pipeline. Please try this one:

export GST_DEBUG=2; gst-launch-1.0 rtspsrc location=rtsp://username:passowrd@ip:port/streamPath  ! rtph264depay ! decodebin ! videoconvert ! video/x-raw,format=BGR ! fakesink
5

Tried with omxh264dec and it worked.

export GST_DEBUG=2; gst-launch-1.0 rtspsrc location=rtsp://username:password@ip:port/streamPath  ! rtph264depay ! h264parse ! omxh264dec ! videoconvert ! fakesink

The same is working with Opencv in C++.

const char *gst = "rtspsrc location=rtsp://username:password@ip:port/streamPath ! rtph264depay ! h264parse ! omxh264dec ! videoconvert ! appsink";