Python opencv gstreamer RTSP stream error

wang_jin_1999 · March 3, 2022, 1:10pm

I have 3 ip camera which can be get by rtsp,but when receiving videos from 3 cameras at the same time, the video will have a 5s delay(reciving 1 camera is real-time).
The receiving code is just like cv2.VideoCapture(url),url is the rtsp of ip camera.
Maybe receiving by Gstreamer will be real-time…,so I run follows code:

image_width = 1920  
image_height = 1080  
rtsp_latency = 200
url1 = "rtsp://169.254.224.11/user=admin&password=&channel=1&stream=0.sdp?"
gurl1 = (
    "rtspsrc location={} latency={} ! rtph264depay ! h264parse ! omxh264dec ! nvvidconv ! video/x-raw, width=(int){}, height=(int){}, format=(string)BGRx ! videoconvert ! appsink sync=false") \
    .format(url1, rtsp_latency, image_width, image_height)  
a = cv2.VideoCapture(gurl1,cv2.CAP_GSTREAMER)
if not a.isOpened():
    sys.exit("plz dont print")
while 1:
    r1,f1=a.read()
    cv2.imshow('1',f1)
    cv2.waitKey(1)

Terminal:
ace@ace-desktop:~/acecombat$ python3 ipc.py
[ WARN:0] global /home/ace/opencv/modules/videoio/src/cap_gstreamer.cpp (2076) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module rtspsrc0 reported: Internal data stream error.
[ WARN:0] global /home/ace/opencv/modules/videoio/src/cap_gstreamer.cpp (1053) open OpenCV | GStreamer warning: unable to start pipeline
[ WARN:0] global /home/ace/opencv/modules/videoio/src/cap_gstreamer.cpp (616) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
plz dont print

plus:
cv2.getBuildInformation()show Gstreamer is YES.
i have run sudo apt install nvidia-l4t-* but it can not work.

Is the pipeline wrong or the Gstreamer config have error?
Thanks.

DaneLLL · March 7, 2022, 2:26am

Hi,
This is possible in OpenCV since there is additional memory copy. Please refer to discussion in
[Gstreamer] nvvidconv, BGR as INPUT

The performance would better if you can run pure gstreamer pipeline, or use cv::gpu::gpuMat as demonstrated in the sample:
Nano not using GPU with gstreamer/python. Slow FPS, dropped frames - #8 by DaneLLL

If you must run gstreamer pipeline in cv2.VideoCapture(), please excute sudo nvpmodel -m 0 and sudo jetson_clocks. This fixes CPU cores at max clock and is the optimal solution for the use-case.

wang_jin_1999 · March 8, 2022, 1:48am

I wonder if Gstreamer can solve the video stream delay problem of jetson connecting three ip cameras?
I’m not sure if the pipeline I wrote is correct，can you show me the correct pipeline code that py-opencv read rtsp video by gstreamer?
Very very thanks.

DaneLLL · March 8, 2022, 2:23am

Hi,
Please try gst-launch-1.0 command like:

$ gst-launch-1.0 uridecodebin uri='rtsp://192.168.68.3:8554/test' ! nvvidconv ! nvegltransform ! nveglglessink

You can run three gst-launch-1.0 commands.

wang_jin_1999 · March 8, 2022, 1:37pm

Thank you for your help ,my problem has been solved.
I have another question to ask you.
I want to use yolo to detect the acquired images, and then convert the opencv image into rtsp and push it to another website. How do I use Gstreamer to do it?
Get the rtsp video I use cv2.VideoCapture("Gstreamer string command"), but I dont know how to push, what opencv function I need?
Thank you for your generous answer.

DaneLLL · March 9, 2022, 1:45am

Hi,
For this use-case we suggest use DeepStream SDK. Please install it through SDKManager or apt install command, and then take a look at

/opt/nvidia/deepstream/deepstream-6.0/sources/objectDetector_Yolo/

You can try the default config file and then modify it to RTSP sink.

And suggest run Yolo tiny models on Jetson Nano.

wang_jin_1999 · March 9, 2022, 3:54am

Thanks.
Can deepstreamer use the yolov5 model that I trained myself? I also need to get the coordinates of the target to control the aircraft.

DaneLLL · March 9, 2022, 4:55am

Hi,
Please check discussion in this topic:
Low fps when doing object detection on jetson nano - #7 by AastaLLL

You would need to convert it to be TensorRT model.

wang_jin_1999 · March 9, 2022, 7:18am

Thank u.

DaneLLL · March 9, 2022, 8:41am

Hi,
Please also check this:
GitHub - marcoslucianops/DeepStream-Yolo: NVIDIA DeepStream SDK 6.0 configuration for YOLO models

It is from community contribution.

wang_jin_1999 · March 9, 2022, 9:02am

OK,I 'll try.
Thku.
best wishes

wang_jin_1999 · March 11, 2022, 7:10am

Hi,I use the command gst-launch-1.0 uridecodebin uri=‘rtsp://169.254.224.11/user=admin&password=&channel=1&stream=0.sdp?’ ! nvvidconv ! nvegltransform ! nveglglessink
Setting pipeline to PAUSED …

terminal showes:

Using winsys: x11
Pipeline is live and does not need PREROLL …
Got context from element ‘eglglessink0’: gst.egl.EGLDisplay=context, display=(GstEGLDisplay)NULL;
Progress: (open) Opening Stream
Progress: (connect) Connecting to rtsp://169.254.224.11/user=admin&password=&channel=1&stream=0.sdp?
ERROR: from element /GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstRTSPSrc:source: Could not open resource for reading and writing.
Additional debug info:
gstrtspsrc.c(7469): gst_rtspsrc_retrieve_sdp (): /GstPipeline:pipeline0/GstURIDecodeBin:uridecodebin0/GstRTSPSrc:source:
Failed to connect. (Generic error)
ERROR: pipeline doesn’t want to preroll.
Setting pipeline to PAUSED …
Setting pipeline to READY …
Setting pipeline to NULL …
Freeing pipeline …

DaneLLL · March 11, 2022, 7:38am

Hi,
The URI looks invalid. Please try with fakesink and check if it can be successfully run:

$ gst-launch-1.0 rtspsrc location="rtsp://169.254.224.11/user=admin&password=&channel=1&stream=0.sdp?" ! fakesink

wang_jin_1999 · March 11, 2022, 7:43am

thks.
gst-launch-1.0 rtspsrc location=“rtsp://169.254.224.11/user=admin&password=&channel=1&stream=0.sdp?” ! fakesink
Setting pipeline to PAUSED …
Pipeline is live and does not need PREROLL …
Progress: (open) Opening Stream
Progress: (connect) Connecting to rtsp://169.254.224.11/user=admin&password=&channel=1&stream=0.sdp?
Progress: (open) Retrieving server options
Progress: (open) Retrieving media info
Progress: (request) SETUP stream 0
Progress: (request) SETUP stream 1
Progress: (open) Opened Stream
Setting pipeline to PLAYING …
New clock: GstSystemClock
Progress: (request) Sending PLAY request
Progress: (request) Sending PLAY request
Progress: (request) Sent PLAY request
WARNING: from element /GstPipeline:pipeline0/GstRTSPSrc:rtspsrc0: Could not read from resource.
Additional debug info:
gstrtspsrc.c(5427): gst_rtspsrc_reconnect (): /GstPipeline:pipeline0/GstRTSPSrc:rtspsrc0:
Could not receive any UDP packets for 5.0000 seconds, maybe your firewall is blocking it. Retrying using a tcp connection.

however I use python cv2.VideoCapture(“rtsp://169.254.224.11/user=admin&password=&channel=1&stream=0.sdp?”) and cv2.imshow(), I can get the video

wang_jin_1999 · March 11, 2022, 7:45am

wang_jin_1999 · March 11, 2022, 7:50am

I change the VideoCapture to the format of Gstreamer , it cant work.

terminal:
fwav@fwav-desktop:~/qwe$ python3 ipc2.py
[ WARN:0] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (1757) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module rtspsrc0 reported: Internal data stream error.
[ WARN:0] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (886) open OpenCV | GStreamer warning: unable to start pipeline
[ WARN:0] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (480) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
Gtk-Message: 15:48:35.961: Failed to load module “canberra-gtk-module”
False
Traceback (most recent call last):
File “ipc2.py”, line 31, in
cv2.imshow(“1”, f1)
cv2.error: OpenCV(4.1.1) /home/nvidia/host/build_opencv/nv_opencv/modules/highgui/src/window.cpp:352: error: (-215:Assertion failed) size.width>0 && size.height>0 in function ‘imshow’

Here is code:
import cv2

#print(cv2.getBuildInformation())

image_width = 1920 # 摄像头视频分辨率中宽度参数
image_height = 1080 # 摄像头视频分辨率中高度参数
rtsp_latency = 50
url1 = “rtsp://169.254.224.11/user=admin&password=&channel=1&stream=0.sdp?”
gurl1 = (
“rtspsrc location={} latency={} ! rtph264depay ! h264parse ! omxh264dec ! nvvidconv ! video/x-raw, width=(int){}, height=(int){}, format=(string)BGRx ! videoconvert ! appsink”)
.format(url1, rtsp_latency, image_width, image_height) # 开启硬件加速
a = cv2.VideoCapture(gurl1)
cv2.namedWindow(‘1’, cv2.WINDOW_NORMAL)
while 1:
r1, f1 = a.read()
print(r1)
cv2.imshow(“1”, f1)
cv2.waitKey(1)

DaneLLL · March 11, 2022, 8:49am

Hi,
You would need to figure out a valid URI which can be read correctly in rtspsrc. So that can use nvv4l2decoder for hardware decoding. Please check if you can contact the camera vendor for help.

Not sure if it works but may add real_stream at tail like

location="rtsp://169.254.224.11/user=admin&password=&channel=1&stream=0.sdp?real_stream"

Certain IP camera has it in URI. May apply it for a try.

wang_jin_1999 · March 11, 2022, 9:41am

Hi,thank you suggestion, I add real_stream,and it still cant work…
Rtsp link i written is according to the supplier format given.I add the port in link and it cant receive source,but the port is got in computer software…

terminal:
fwav@fwav-desktop:~/qwe$ python3 ipc.py
[ WARN:0] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (1757) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module rtspsrc0 reported: Internal data stream error.
[ WARN:0] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (886) open OpenCV | GStreamer warning: unable to start pipeline
[ WARN:0] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (480) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
Gtk-Message: 16:04:30.149: Failed to load module “canberra-gtk-module”
False
Traceback (most recent call last):
File “ipc.py”, line 18, in
cv2.imshow(“1”, f1)
cv2.error: OpenCV(4.1.1) /home/nvidia/host/build_opencv/nv_opencv/modules/highgui/src/window.cpp:352: error: (-215:Assertion failed) size.width>0 && size.height>0 in function ‘imshow’

code:
import cv2
#print(cv2.getBuildInformation())
image_width = 1920 # 摄像头视频分辨率中宽度参数
image_height = 1080 # 摄像头视频分辨率中高度参数
rtsp_latency = 50
url1 = “rtsp://169.254.224.11:34567/user=admin&password=&channel=1&stream=0.sdp?real_stream”

gurl1 = (
“rtspsrc location={} latency={} ! rtph264depay ! h264parse ! omxh264dec ! nvvidconv ! video/x-raw, width=(int){}, height=(int){}, format=(string)BGRx ! videoconvert ! appsink”)
.format(url1, rtsp_latency, image_width, image_height) # 开启硬件加速
a = cv2.VideoCapture(gurl1,cv2.CAP_GSTREAMER)
cv2.namedWindow(‘1’, cv2.WINDOW_NORMAL)
while 1:
r1, f1 = a.read()
print(r1)
cv2.imshow(“1”, f1)
cv2.waitKey(1)

system · April 6, 2022, 3:11am

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.