OpenCV capture with GStreamer results in grayscale images, expected color

Robotics & Edge Computing Jetson & Embedded Systems Jetson Orin Nano
, ,
1

Issue: When I run a gstreamer pipeline in bash with an xvimagesink I get a color video as expected. When I run a similar pipeline in OpenCV however the images are 8-bit mono and display as grayscale.

bash script pipeline: gst-launch-1.0 nvarguscamerasrc ! "video/x-raw(memory:NVMM),width=1920,height=1080,framerate=30/1,format=NV12" ! nvvidconv ! xvimagesink
– Correctly displays color video

pipeline passed to OpenCV: cap = cv2.VideoCapture('nvarguscamerasrc ! video/x-raw(memory:NVMM),width=1920,height=1080,framerate=30/1,format=NV12 ! nvvidconv ! appsink', cv2.CAP_GSTREAMER)
– Issue: mono grayscale video appears.

Variations on OpenCV pipeline that resulted in issues:

  1. Try using videoconvert to convert to BGR before passing to appsink (as suggested by this post): cap = cv2.VideoCapture('nvarguscamerasrc ! video/x-raw(memory:NVMM),width=1920,height=1080,framerate=30/1,format=NV12 ! nvvidconv ! video/x-raw(memory:NVMM),format=BGRx ! videoconvert ! video/x-raw(memory:NVMM),format=BGR ! appsink', cv2.CAP_GSTREAMER)
    – This results in a failure to initialize the pipeline:
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 4032 x 3040 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 22.250000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 3840 x 2160 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 22.250000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 22.250000; Exposure Range min 13000, max 683709000;

GST_ARGUS: Running with following settings:
   Camera index = 0 
   Camera mode  = 2 
   Output Stream W = 1920 H = 1080 
   seconds to Run    = 0 
   Frame Rate = 59.999999 
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
[ WARN:0@0.690] global cap_gstreamer.cpp:2784 handleMessage OpenCV | GStreamer warning: Embedded video playback halted; module nvarguscamerasrc0 reported: Internal data stream error.
GST_ARGUS: Cleaning up
CONSUMER: Done Success
GST_ARGUS: Done Success
[ WARN:0@5.715] global cap_gstreamer.cpp:1679 open OpenCV | GStreamer warning: unable to start pipeline
[ WARN:0@5.715] global cap_gstreamer.cpp:1164 isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
[ WARN:0@5.715] global cap.cpp:204 open VIDEOIO(GSTREAMER): backend is generally available but can't be used to capture by name

Additional Info:
v4l2-ctl --list-formats yields:

ioctl: VIDIOC_ENUM_FMT
	Type: Video Capture

	[0]: 'RG10' (10-bit Bayer RGRG/GBGB)

Any suggestions for what to try next would be greatly appreciated!

2

Found a hacky fix, perhaps a better solution exists:

  1. convert to I420 format before app sink: cap = cv2.VideoCapture('nvarguscamerasrc ! nvvidconv ! video/x-raw,width=1920,height=1080,framerate=30/1,format=I420 ! appsink', cv2.CAP_GSTREAMER)
  2. use cvtColor() to convert YUV to BGR:
import cv2

cap = cv2.VideoCapture('nvarguscamerasrc ! nvvidconv ! video/x-raw,width=1920,height=1080,framerate=30/1,format=I420 ! appsink', cv2.CAP_GSTREAMER)
while True:
    ret, img = cap.read()
    if not ret:
        break

    img = cv2.cvtColor(img, cv2.COLOR_YUV2BGR_I420)
    cv2.imshow('cam', img)
    k = cv2.waitKey(1)
    if k & 0xFF == ord('q'):
        break

cap.release()
cv2.destroyAllWindows()
4

Hi,
Please try

'nvarguscamerasrc ! video/x-raw(memory:NVMM),width=1920,height=1080,framerate=30/1,format=NV12 ! nvvidconv ! video/x-raw,format=BGRx ! videoconvert ! video/x-raw,format=BGR ! appsink'

The videoconvert plugin is a software convert and supports only CPU buffer. Please use nvvidconv plugin to convert NVMM buffer to CPU buffer, and send the buffer to videoconvert plugin.

1 Like
5

Solution worked, thanks!

1 Like
7

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.