Unable to retrieve 16-bit greyscale images

Robotics & Edge Computing Jetson & Embedded Systems Jetson Nano
,
1

I’m attempting to create a gstreamer pipeline using OpenCV to simply open and retrieve raw data from a Y16 uvc webcam. Looking at the output from a gst-launch-1.0 command, it appears as if the GRAY16_LE pixel format is supported, and the one I want. Output from the command:

~/Documents/dev$ gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,format='GRAY16_LE' ! fakesink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock

However, when I go to implement the pipeline and get data out in OpenCV, either C++ or python, I’m unable to get 16-bit data.

cap = cv2.VideoCapture("v4l2src ! videoconvert ! video/x-raw,format=GRAY16_LE! appsink", 
cv2.CAP_GSTREAMER)
ret, gray = cap.read()
print(gray[0,0])
cv2.imshow('frame', gray)

Command line errors:

~/Documents/dev$ python3 cam.py 
[ WARN:0] global /home/opencv-4.1.1/modules/videoio/src/cap_gstreamer.cpp (1757) handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module v4l2src0 reported: Internal data stream error.
[ WARN:0] global /home/opencv-4.1.1/modules/videoio/src/cap_gstreamer.cpp (886) open OpenCV | GStreamer warning: unable to start pipeline
[ WARN:0] global /home/opencv-4.1.1/modules/videoio/src/cap_gstreamer.cpp (480) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
Traceback (most recent call last):
  File "cam.py", line 13, in <module>
cv2.imshow('frame', gray)
cv2.error: OpenCV(4.1.1) /home/opencv-4.1.1/modules/highgui/src/window.cpp:352: error: (-215:Assertion failed) size.width>0 && size.height>0 in function 'imshow'

When I change the format to GRAY8, it seems to grab 8 bit data fine with the following warning:

~/Documents/dev$ python3 cam.py 
[ WARN:0] global /home/opencv-4.1.1/modules/videoio/src/cap_gstreamer.cpp (933) open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1
Gtk-Message: 12:52:51.939: Failed to load module "canberra-gtk-module"

What’s the proper method to grab Y16 data?

2

I think opencv expects GRAY8 only for grayscale. You may try:

cap = cv2.VideoCapture("v4l2src device=/dev/video0 ! video/x-raw,format=GRAY16_LE ! videoconvert ! video/x-raw, format=GRAY8 ! appsink", cv2.CAP_GSTREAMER)
1 Like
3

Thanks, I think you’re right on both counts. It looks like openCV is only getting the MSB into the image, however, I think I can use the V4L2 pipe to get what I actually want after looking at the Arducam github. This code seems to get full resolution 16 bit data in:
cap=cv2.VideoCapture(0, cv2.CAP_V4L2)
cap.set(cv2.CAP_PROP_FOURCC, cv2.VideoWriter.fourcc(‘Y’,‘1’,‘6’,’ '))
cap.set(cv2.CAP_PROP_CONVERT_RGB, False)

2 Likes