Python OpenCV video capture problem

imranhassan84 · April 19, 2021, 10:43pm

I am trying to read the camera image on Jetson Xavier (ubuntu 18). I am facing a problem. When I run the following code it gives a warning and gives a black (full) image.

[ WARN:0] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (933) open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1

cam=cv2.VideoCapture(0)
if cam.isOpened():
    grab,img = cam.read()
    if grab is True:
        cv2.imshow('sample image',img)
    else:
        print(f"Image not found")
else:
    print("Camera not openedd")
       

cv2.waitKey(0) # waits until a key is pressed
cv2.destroyAllWindows() # destroys the window showing image

If I use ‘dev/video0’ to read the image i.e.

cam=cv2.VideoCapture('dev/video0')

I get the warning and custom error message of camera not opened

[ WARN:0] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (711) open OpenCV | GStreamer warning: Error opening bin: no element “dev”
[ WARN:0] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (480) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
Camera not opened

Then I created gstream string and passed that to video capture as shown below.
the string is as follow

gstr = 'varguscamerasrc ! video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)60/1 ! nvvidconv flip-method=0 ! video/x-raw, width=(int)1280, height=(int)720, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink'

cap = cv2.VideoCapture(gstr, cv2.CAP_GSTREAMER)

I get the following error

rror generated. /dvs/git/dirty/git-master_linux/multimedia/nvgstreamer/gst-nvarguscamera/gstnvarguscamerasrc.cpp, execute:645 No cameras available

(python3:15402): GStreamer-CRITICAL **: 19:08:54.835: gst_mini_object_set_qdata: assertion 'object != NULL' failed
[ WARN:0] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (933) open OpenCV | GStreamer warning: Cannot query video position: status=0, value=-1, duration=-1
Traceback (most recent call last):

I am new to jetson please guide me. Thanks

Honey_Patouceul · April 20, 2021, 12:01am

There is a typo here, it lacks the leading ‘n’.
Also, you may tell what camera model you are using. If using a RPi v2 cam, you may use 30 fps with 1080p:

gstr = 'nvarguscamerasrc ! video/x-raw(memory:NVMM), width=1920, height=1080, format=NV12, framerate=30/1 ! nvvidconv ! video/x-raw, width=1280, height=720, format=BGRx ! videoconvert ! video/x-raw, format=BGR ! appsink drop=1 '

DaneLLL · April 20, 2021, 3:28am

Hi,
Please try Honey_Patouceul’s suggestion. If you use USB cameras, may refer to
Jetson Nano FAQ
Q: I have a USB camera. How can I launch it on Jetson Nano?

imranhassan84 · April 20, 2021, 4:28am

Oh sorry . It was copy paste mistake. In actual code it is correct. Problem is still same

imranhassan84 · April 20, 2021, 11:12am

I used the method and find this

ioctl: VIDIOC_ENUM_FMT
	Index       : 0
	Type        : Video Capture
	Pixel Format: 'MJPG' (compressed)
	Name        : Motion-JPEG
		Size: Discrete 1920x1080
			Interval: Discrete 0.017s (60.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 640x480
			Interval: Discrete 0.017s (60.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 800x600
			Interval: Discrete 0.017s (60.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 1024x768
			Interval: Discrete 0.017s (60.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 1280x720
			Interval: Discrete 0.017s (60.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 1280x960
			Interval: Discrete 0.017s (60.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 1280x1024
			Interval: Discrete 0.017s (60.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 1360x768
			Interval: Discrete 0.017s (60.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 1400x900
			Interval: Discrete 0.017s (60.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 1440x900
			Interval: Discrete 0.017s (60.000 fps)
			Interval: Discrete 0.033s (30.000 fps)

	Index       : 1
	Type        : Video Capture
	Pixel Format: 'YUYV'
	Name        : YUYV 4:2:2
		Size: Discrete 1920x1080
			Interval: Discrete 0.017s (60.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 640x480
			Interval: Discrete 0.017s (60.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 800x600
			Interval: Discrete 0.017s (60.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 1024x768
			Interval: Discrete 0.017s (60.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 1280x720
			Interval: Discrete 0.017s (60.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 1280x960
			Interval: Discrete 0.017s (60.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 1280x1024
			Interval: Discrete 0.017s (60.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 1360x768
			Interval: Discrete 0.017s (60.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 1400x900
			Interval: Discrete 0.017s (60.000 fps)
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 1440x900
			Interval: Discrete 0.017s (60.000 fps)
			Interval: Discrete 0.033s (30.000 fps)

When I use this command as told in the FAQ

gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,format=YUY2,width=1920,height=1080,framerate=60/1 ! nvvidconv ! 'video/x-raw(memory:NVMM),format=NV12' ! nvoverlaysink

I get the following response

Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock

and nothing happened. Then use gstreamer

gst_str = 'nvarguscamerasrc ! video/x-raw,format=YUY2,width=1920,height=1080,framerate=60/1 ! nvvidconv ! video/x-raw(memory:NVMM),format=NV12 ! nvoverlaysink'

I got this error and warning

[ WARN:0] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (711) open OpenCV | GStreamer warning: Error opening bin: could not link nvarguscamerasrc0 to nvvconv0, nvarguscamerasrc0 can't handle caps video/x-raw, format=(string)YUY2, width=(int)1920, height=(int)1080, framerate=(fraction)60/1
[ WARN:0] global /home/nvidia/host/build_opencv/nv_opencv/modules/videoio/src/cap_gstreamer.cpp (480) isPipelinePlaying OpenCV | GStreamer warning: GStreamer: pipeline have not been created
Camera not opened

I am still confused, what is exact problem. I am using python3 and OpenCV 4.1.1 with jetpack 4.5.1

Honey_Patouceul · April 20, 2021, 11:44am

nvarguscamerasrc is for bayer sensors connected with CSI.

If using a USB cam, you would use v4l2src instead.

First try with a low resolution * fps. The following should display your camera feed in a X window:

gst-launch-1.0 -v v4l2src device=/dev/video0 ! video/x-raw,format=YUY2,width=640,height=480,framerate=30/1 ! xvimagesink

If it works, in opencv you would use :

cam = cv2.VideoCapture("v4l2src device=/dev/video0 ! video/x-raw,format=YUY2,width=640,height=480,framerate=30/1 ! videoconvert ! video/x-raw, format=BGR ! appsink drop=1 ", cv2.CAP_GSTREAMER)
if not cam.isOpened() :
    print("Failed to open camera")
    exit()

Note that videoconvert is CPU only and may result in high CPU usage and might be a bottleneck for high resolution * fps. You may add a queue after it or use its n-threads property.
You may also use nvvidconv for YUV to BGRx conversion with hardware and have videoconvert just remove the extra 4th byte for getting BGR format expected by opencv:

cam = cv2.VideoCapture("v4l2src device=/dev/video0 ! video/x-raw,format=YUY2,width=640,height=480,framerate=30/1 ! nvvidconv ! video/x-raw(memory:NVMM) ! nvvidconv ! video/x-raw, format=BGRx ! videoconvert ! video/x-raw, format=BGR ! appsink drop=1 ", cv2.CAP_GSTREAMER)

imranhassan84 · April 20, 2021, 6:51pm

This working perfectly fine and getting image … but using openCV again getting black image. What is problem with opencv.

Honey_Patouceul · April 20, 2021, 6:56pm

Sorry I missed that, but you need to use waitKey after imshow for opencv drawing thread to be scheduled:

     if cap.isOpened():
         cv2.namedWindow("demo", cv2.WINDOW_AUTOSIZE)
         while True:
             ret_val, img = cap.read();
             if not ret_val:
                  break
             cv2.imshow('demo',img)
             if cv2.waitKey(1) == ord('q'):
                  break
     else:
         print ("camera open failed")

imranhassan84 · April 21, 2021, 8:11am

Honey_Patouceul:

  if cap.isOpened():
         cv2.namedWindow("demo", cv2.WINDOW_AUTOSIZE)
         while True:
             ret_val, img = cap.read();
             if not ret_val:
                  break
             cv2.imshow('demo',img)
             if cv2.waitKey(1) == ord('q'):
                  break
     else:
         print ("camera open failed")

Thanks a lot, dear. It worked fine. The first image is black and then others are fine.

I am curious why the first image is black?

imranhassan84 · June 18, 2021, 1:40pm

@Honey_Patouceul

My solution was working perfectly fine with all possible resolutions and fps but suddenly it stopped working. The thing I did is I installed SSD NVMe.

When I use this command I get error

gst-launch-1.0 -v v4l2src device=/dev/video0 ! video/x-raw,format=YUY2,width=640,height=480,framerate=30/1 ! xvimagesink

Error is

Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data stream error.
Additional debug info:
gstbasesrc.c(3055): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming stopped, reason not-negotiated (-4)
Execution ended after 0:00:00.000622382
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

I tried to change the resolution but it does not work. If I use

cap = cv2.VideoCapture(0)

It canputures frames but at low resolution 640x480 and some time gives error

premature end of stream

Can you help me?

Best Regards

DaneLLL · June 23, 2021, 8:24am

Hi,
Please check if you can run this gst-launch-1.0 command:

gst-launch-1.0 -v v4l2src device=/dev/video0 ! video/x-raw,format=YUY2,width=640,height=480,framerate=30/1 ! videoconvert ! video/x-raw,format=BGR ! fakesink

If yes, please apply this pipeline:

v4l2src device=/dev/video0 ! video/x-raw,format=YUY2,width=640,height=480,framerate=30/1 ! videoconvert ! video/x-raw,format=BGR ! appsink

to cv2.VideoCapture().

Reference python sample:
V4l2src using OpenCV Gstreamer is not working in Jetson Xavier NX - #3 by DaneLLL

Topic		Replies	Views
OpenCV2 VideoCapture issues Jetson Nano	5	16651	October 14, 2021
Problem with two USB cameras Jetson Nano camera , usb	6	2699	October 15, 2021
Recording video from camera Jetson AGX Xavier python	38	4668	October 18, 2021
Streaming an openCV capture with gstreamer in Python Jetson AGX Xavier opencv , gstreamer	17	4898	October 18, 2021
Python opencv gstreamer RTSP stream error Jetson Nano gstreamer	18	6191	April 6, 2022
Opencv streaming issue Jetson Xavier NX camera , opencv , gstreamer	19	2433	October 18, 2021
PiCam in OpenCV nano_built_opencv error Jetson Nano camera , opencv , gstreamer	9	1936	December 22, 2021
nvarguscamerasrc OpenCV (Solved) Jetson AGX Xavier opencv	52	30022	October 18, 2021
Opencv Camera Properties : CAP_PROP_FOURCC Not working Jetson Nano opencv	28	4024	October 15, 2021
Jetpack 4.5: USB Camera does not work in Cheese and opencv 4.1 Jetson Xavier NX camera , opencv , usb , gstreamer , nvbugs	18	6280	September 15, 2021

Python OpenCV video capture problem

Related topics