Green screen when using Raspberry pi camera v2 attached to Jetson nano and cv2 (openCV)

Autonomous Machines Jetson & Embedded Systems Jetson Nano
, ,
#1

I using the Nvidia Jetson Nano and a Paspberry Pi NoIR Camera V2. https://www.raspberrypi.org/products/pi-noir-camera-v2/

When ich using cv2, my output is a green srceen.

But when I using a normal USB-Cam than run fine.

I don’t know why. I hope any of you already solved this problem.

This is my code:

import numpy as np
import cv2

cap = cv2.VideoCapture(0)

while(True):
    # Capture frame-by-frame
    ret, im= cap.read()

    # Display the resulting frame
    cv2.imshow('frame', im)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# When everything done, release the capture
cap.release()
cv2.destroyAllWindows()
#2

The IMX219 driver in L4T only provides bayer format in V4L. So you cannot use cv2.VideoCapture(0) because opencv videio doesn’t support bayer formats but rather expects BGR for color frames.
So you would use a gstreamer pipeline instead, using nvarguscamerasrc for debayering with ISP, nvvidconv to convert from NV12 format in NVMM memory into BGRx format in CPU memory, then videoconvert to provide BGR format to opencv:

cv2.VideoCapture cap("nvarguscamerasrc ! nvvidconv ! video/x-raw, format=BGRx ! videoconvert ! video/x-raw, format=BGR ! appsink", cv2.CAP_GSTREAMER)

Also note that imshow is CPU only and not very efficient on Jetson (better first try with low resolution). It also depends on GUI backend of your opencv version (GTK, Qt…).

#3

ok.

Do you think I should try to convert
cap = cv.VideoCapture(0)
to
cap = cv2.VideoCapture("nvarguscamerasrc ! nvvidconv ! video/x-raw, format=BGRx' ! videoconvert ! video/x-raw, format=BGR ! appsink", cv2.CAP_GSTREAMER)

But when I do that I have a error like this:

Traceback (most recent call last):
File “yolov5_cam_check.py”, line 12, in
cv2.imshow(‘frame’, im)
cv2.error: OpenCV(4.5.1) /tmp/pip-req-build-zuuo394f/opencv/modules/highgui/src/window.cpp:376: error: (-215:Assertion failed) size.width>0 && size.height>0 in function ‘imshow’

My new code is:

import numpy as np
import cv2

cap = cv2.VideoCapture("nvarguscamerasrc ! nvvidconv ! video/x-raw, format=BGRx' ! videoconvert ! video/x-raw, format=BGR ! appsink", cv2.CAP_GSTREAMER)


while(True):
    # Capture frame-by-frame
    ret, im= cap.read()

    # Display the resulting frame
    cv2.imshow('frame', im)
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# When everything done, release the capture
cap.release()
cv2.destroyAllWindows()
#4

Sorry, there was a typo in my command (extra ’ after BGRx). I’ve edited it for fixing that.
As a result of the wrong pipeline, the videoCapture failed to open and you were reading void frames.
You may check that videoCapture was correctly opened (if cap.isOpened()), and you may also check that frame read succeeded.

#5

It’s really fun.
It works with Python 2, but with Python 3 it jumps to else and prints in the output “Open camera failed”.

Do you also have an idea why that is?

import numpy as np
import cv2

cap = cv2.VideoCapture("nvarguscamerasrc ! nvvidconv ! video/x-raw, width=1024, height=576, format=BGRx ! videoconvert ! video/x-raw, format=BGR ! appsink", cv2.CAP_GSTREAMER)

if cap.isOpened():
    while(True):
        # Capture frame-by-frame
        ret, im= cap.read()

        # Display the resulting frame
        cv2.imshow('frame', im)
        if cv2.waitKey(1) & 0xFF == ord('q'):
            break
else:
    print("camera open failed")

# When everything done, release the capture
cap.release()
cv2.destroyAllWindows()
#6

Probably you have different versions installed for python2 and python3.
You may check with:

import cv2
print(cv2.__version__)
#7

My cv2. version for python2 is 4.1.1, for python3 the latest version is 4.5.1
Do you think that’s the reason

cv2_version-python2-und-3
cv2_version-python2-und-3745×493 36.9 KB

#8

Would you change “format=BGRx” to “format=(string)NV12” ?

#9

You would check if your Opencv-4.5.1 version has support for gstreamer. Use:

import cv2
print(cv2.getBuildInformation())

and check for GSTREAMER support in Video I/O section:

GStreamer:                   YES (1.14.5)

If not available, you would reconfigure a new opencv build enabling it with -D WITH_GSTREAMER=ON

#10

@Honey_Patouceul thank you for the answer.

I checked the GSTREAMER support in video I / O:

Gstreamer: NO

I tried to set GSTREAMER to YES with the description from Install OpenCV 4.5 on Jetson Nano - Q-engineering

But the GSTREAMER stays on NO

Do you have another idea how I get the GSTREAMER on YES and can you explain it to me?

grafik

#12

Hi,
Probably you manaully upgrade the version. By default you should see OpenCV 4.1.1:

nvidia@nvidia-desktop:~$ python3
Python 3.6.9 (default, Oct  8 2020, 12:12:24)
[GCC 8.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import cv2
>>> print(cv2.__version__)
4.1.1
>>>

Suggest you re-install system image and try again. The gstreamer support is enabled in default OpenCV 4.1.1 package.