I am able to use the simple, non-accelerated gstreamer pipeline in Python 3.6 OpenCV cv.VideoCapture().
cv2.VideoCapture( “v4l2src device=/dev/video0 ! video/x-raw, width=640, height=360 ! videoconvert ! appsink”)
Unfortunately, this simplistic pipeline does not use any NVIDIA hardware decoders or accelerations, so it uses too much CPU.
Furthermore, my cameras support faster MJPEG framerates than YUY2 raw, and it would be nice to use NVJPG hardware accelerated decoding.
The following pipelines work when launched from script, but using them inside cv2.VideoCapture() either does not work at all, or shows only greyscale image.
- NVJPG decoding (I can see NVJPG turning on in jtop)
gst-launch-1.0 v4l2src device=/dev/video2 io-mode=2 ! image/jpeg, width=1280, height=720, framerate=30/1, format=MJPG ! nvjpegdec ! video/x-raw ! nvvidconv ! ‘video/x-raw(memory:NVMM), width=1280, height=720’ ! nvoverlaysink
- Similar to above, but visualizing with xvimagesink
gst-launch-1.0 v4l2src device=/dev/video0 io-mode=2 ! image/jpeg,width=640,height=360,framerate=60/1 ! nvv4l2decoder mjpeg=1 ! nvvidconv ! video/x-raw ! videoconvert ! video/x-raw ! xvimagesink
- Also works with nvv4l2decoder
gst-launch-1.0 v4l2src device=/dev/video2 io-mode=2 ! image/jpeg, width=1280, height=720, framerate=30/1 ! nvv4l2decoder mjpeg=1 ! nvvidconv ! video/x-raw ! videoconvert ! video/x-raw ! xvimagesink
All of the above do not work inside cv2.VideoCapture().
- Number one in OpenCV:
cap = cv2.VideoCapture(“v4l2src device=/dev/video0 io-mode=2 ! image/jpeg,width=640,height=360,framerate=60/1 ! nvv4l2decoder mjpeg=1 ! nvvidconv ! video/x-raw ! videoconvert ! video/x-raw ! appsink”, cv2.CAP_GSTREAMER)
gives error:
(python3:23088): GStreamer-CRITICAL **: 22:32:32.992: gst_element_make_from_uri: assertion ‘gst_uri_is_valid (uri)’ failed
(python3:23088): GStreamer-CRITICAL **: 22:32:32.995: gst_element_link_pads_filtered: assertion ‘GST_IS_BIN (parent)’ failed
(python3:23088): GStreamer-CRITICAL **: 22:32:32.996: gst_element_link_pads_filtered: assertion ‘GST_IS_BIN (parent)’ failed
[ WARN:0] global /home/datemike/opencv/modules/videoio/src/cap_gstreamer.cpp (713) open OpenCV | GStreamer warning: Error opening bin: syntax error
[ WARN:0] global /home/datemike/opencv/modules/videoio/src/cap_gstreamer.cpp (480) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
- Number two and three in OpenCV:
cap = cv2.VideoCapture(“v4l2src device=/dev/video0 io-mode=2 ! image/jpeg, width=640, height=360, framerate=60/1 ! nvv4l2decoder mjpeg=1 ! nvvidconv ! video/x-raw ! videoconvert ! video/x-raw ! appsink”)
cap = cv2.VideoCapture(“v4l2src device=/dev/video0 io-mode=2 ! image/jpeg, width=640, height=360, framerate=60/1 ! nvv4l2decoder mjpeg=1 ! nvvidconv ! video/x-raw ! videoconvert ! video/x-raw ! appsink”)
Displays black and white only!
Looking at the output array, the ‘color’ dimension is only size 1, instead of 3.
I tried adding and deleting the ‘cv2.CAP_GSTREAMER’ argument at the end, and it makes no difference.