Issue with Raspberry Pi V2 Camera: error: (-215:Assertion failed) !_src.empty() in function 'cvtColor'

Hy,

I’m using a jetson nano and a raspberry pi camera module V2 and want to run a code from a course about YOLO + object detection.

When i try to run the code with Spyder (python 3.6) this error occures:

File “/home/dlinano/Documents/Python/UdemyTrainingYOLO/activity-1/detecting-object.py”, line 44, in
frame_HSV = cv2.cvtColor(frame_BGR, cv2.COLOR_BGR2HSV)

error: OpenCV(4.1.1) /home/nvidia/host/build_opencv/nv_opencv/modules/imgproc/src/color.cpp:182: error: (-215:Assertion failed) !_src.empty() in function ‘cvtColor’

This is the program:

# Detecting Object with chosen Colour Mask
#
# Algorithm:
# Reading RGB image --> Converting to HSV --> Implementing Mask -->
# --> Finding Contour Points --> Extracting Rectangle Coordinates -->
# --> Drawing Bounding Box --> Putting Label
#
# Result:
# Window with Detected Object, Bounding Box and Label in Real Time
# Importing needed library
import cv2

# Defining lower bounds and upper bounds of founded Mask
min_blue, min_green, min_red = 21, 222, 70
max_blue, max_green, max_red = 176, 255, 255

# Getting version of OpenCV that is currently used
# Converting string into the list by dot as separator
# and getting first number
v = cv2.__version__.split('.')[0]

# Defining object for reading video from camera
camera = cv2.VideoCapture(0)


# Defining loop for catching frames
while True:
    # Capture frame-by-frame from camera
    _, frame_BGR = camera.read()

    # Converting current frame to HSV
    frame_HSV = cv2.cvtColor(frame_BGR, cv2.COLOR_BGR2HSV)

    # Implementing Mask with founded colours from Track Bars to HSV Image
    mask = cv2.inRange(frame_HSV,
                       (min_blue, min_green, min_red),
                       (max_blue, max_green, max_red))

    # Showing current frame with implemented Mask
    # Giving name to the window with Mask
    # And specifying that window is resizable
    cv2.namedWindow('Binary frame with Mask', cv2.WINDOW_NORMAL)
    cv2.imshow('Binary frame with Mask', mask)

    # Finding Contours
    # Pay attention!
    # Different versions of OpenCV returns different number of parameters
    # when using function cv2.findContours()

    # In OpenCV version 3 function cv2.findContours() returns three parameters:
    # modified image, found Contours and hierarchy
    # All found Contours from current frame are stored in the list
    # Each individual Contour is a Numpy array of(x, y) coordinates
    # of the boundary points of the Object
    # We are interested only in Contours

    # Checking if OpenCV version 3 is used
    if v == '3':
        _, contours, _ = cv2.findContours(mask, cv2.RETR_TREE, cv2.CHAIN_APPROX_NONE)

    # In OpenCV version 4 function cv2.findContours() returns two parameters:
    # found Contours and hierarchy
    # All found Contours from current frame are stored in the list
    # Each individual Contour is a Numpy array of(x, y) coordinates
    # of the boundary points of the Object
    # We are interested only in Contours

    # Checking if OpenCV version 4 is used
    else:
        contours, _ = cv2.findContours(mask, cv2.RETR_TREE, cv2.CHAIN_APPROX_NONE)

    # Finding the biggest Contour by sorting from biggest to smallest
    contours = sorted(contours, key=cv2.contourArea, reverse=True)

    # Extracting Coordinates of the biggest Contour if any was found
    if contours:
        # Getting rectangle coordinates and spatial size from biggest Contour
        # Function cv2.boundingRect() is used to get an approximate rectangle
        # around the region of interest in the binary image after Contour was found
        (x_min, y_min, box_width, box_height) = cv2.boundingRect(contours[0])

        # Drawing Bounding Box on the current BGR frame
        cv2.rectangle(frame_BGR, (x_min - 15, y_min - 15),
                      (x_min + box_width + 15, y_min + box_height + 15),
                      (0, 0, 255), 3)

        # Preparing text for the Label
        label = 'Detected Object'

        # Putting text with Label on the current BGR frame
        cv2.putText(frame_BGR, label, (x_min - 5, y_min - 25),
                    cv2.FONT_HERSHEY_SIMPLEX, 1.0, (0, 0, 255), 2)

    # Showing current BGR frame with Detected Object
    # Giving name to the window with Detected Object
    # And specifying that window is resizable
    cv2.namedWindow('Detected Object', cv2.WINDOW_NORMAL)
    cv2.imshow('Detected Object', frame_BGR)

    # Breaking the loop if 'q' is pressed
    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

# Destroying all opened windows
cv2.destroyAllWindows()

I already checked the following things:

1.)
dlinano@jetson-nano:~$ ls -l /dev/video0
crw-rw----+ 1 root video 81, 0 Mar 21 10:56 /dev/video0
(this is fine i think)

2.)
gst-launch-1.0 nvarguscamerasrc sensor_mode=0 ! ‘video/x-raw(memory:NVMM),width=3820, height=2464, framerate=21/1, format=NV12’ ! nvvidconv flip-method=0 ! ‘video/x-raw,width=960, height=616’ ! nvvidconv ! nvegltransform ! nveglglessink -e
(everything worked fine with a good image quality)

3.)
Running the same code on my Windows 10 notebook with Spyder (python 3.6) and the notebooks onboard camera: result: everything worked fine

Someone can help me?

BR chris

The PiV2 Camera is a Bayer sensor, and the driver only exposes Bayer formats to V4L.
Debayering can be done through ISP using nvarguscamerasrc plugin in gstreamer as you’ve noticed.

So you would just change the arguments of cv2.VideoCapture so that it uses a gstreamer pipeline rather than direct V4L API:

gst_cap = 'nvarguscamerasrc sensor_mode=0 ! video/x-raw(memory:NVMM),width=3820, height=2464, framerate=21/1, format=NV12 ! nvvidconv  ! video/x-raw, format=BGRx, width=960, height=616 ! videoconvert ! video/x-raw, format=BGR ! appsink'
camera = cv2.VideoCapture(gst_cap, cv2.CAP_GSTREAMER)
1 Like

It works!
Thanks a lot :-) !