Python, OpenCV and GSTREAMER - USB Camera Problem

Hello again. I’m asking too many questions sorry for that but I like working on Jetson TK1. :D

My new problem is about Gstreamer using with usb camera.

My code is like this:

import cv2
import sys

video_capture = cv2.VideoCapture("nvcamerasrc ! video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)I420, framerate=(fraction)24/1 ! nvvidconv flip-method=6 ! video/x-raw, format=(string)I420 ! videoconvert ! video/x-raw, format=(string)BGR ! appsink")

while True:
   
    ret, frame = video_capture.read()

cv2.imshow('Video', frame)

    if cv2.waitKey(1) & 0xFF == ord('q'):
        break

video_capture.release()
cv2.destroyAllWindows()

Problem is:

python camera.py
NvCamHwHalDeviceRead: 18 - -1
NvCamHwHalDeviceRead: 19 - -1
NvCamHwHalDeviceRead: 20 - -1
NvCamHwHalDeviceRead: 21 - -1
NvCamHwHalDeviceRead: 22 - -1
NvCamHwHalDeviceRead: 23 - -1
NvCamHwHalDeviceRead: 24 - -1
NvCamHwHalDeviceRead: 25 - -1
NvCamHwHalDeviceRead: 26 - -1
IMX135 ****  Can not open camera device: No such file or directory
NvOdmImagerOpenExpanded 462: Sensor ERR
NvOdmImagerOpenExpanded FAILED!
camera_open failed

(python:4758): GStreamer-CRITICAL **: 
Trying to dispose element capsfilter2, but it is in PAUSED instead of the NULL state.
You need to explicitly set elements to the NULL state before
dropping the final reference, to allow them to clean up.
This problem may also be caused by a refcounting bug in the
application or some element.

(python:4758): GStreamer-CRITICAL **: 
Trying to dispose element capsfilter1, but it is in PAUSED instead of the NULL state.
You need to explicitly set elements to the NULL state before
dropping the final reference, to allow them to clean up.
This problem may also be caused by a refcounting bug in the
application or some element.

(python:4758): GStreamer-CRITICAL **: 
Trying to dispose element capsfilter0, but it is in PAUSED instead of the NULL state.
You need to explicitly set elements to the NULL state before
dropping the final reference, to allow them to clean up.
This problem may also be caused by a refcounting bug in the
application or some element.

(python:4758): GStreamer-CRITICAL **: 
Trying to dispose element nvvconv0, but it is in PAUSED instead of the NULL state.
You need to explicitly set elements to the NULL state before
dropping the final reference, to allow them to clean up.
This problem may also be caused by a refcounting bug in the
application or some element.

(python:4758): GStreamer-CRITICAL **: 
Trying to dispose element nvcamerasrc0, but it is in READY instead of the NULL state.
You need to explicitly set elements to the NULL state before
dropping the final reference, to allow them to clean up.
This problem may also be caused by a refcounting bug in the
application or some element.

(python:4758): GStreamer-CRITICAL **: 
Trying to dispose element pipeline0, but it is in READY instead of the NULL state.
You need to explicitly set elements to the NULL state before
dropping the final reference, to allow them to clean up.
This problem may also be caused by a refcounting bug in the
application or some element.

OpenCV Error: Unspecified error (GStreamer: unable to start pipeline
) in cvCaptureFromCAM_GStreamer, file /home/ubuntu/opencv/modules/highgui/src/cap_gstreamer.cpp, line 816
Traceback (most recent call last):
  File "camera.py", line 6, in <module>
    video_capture = cv2.VideoCapture("nvcamerasrc ! video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)I420, framerate=(fraction)24/1 ! nvvidconv flip-method=6 ! video/x-raw, format=(string)I420 ! videoconvert ! video/x-raw, format=(string)BGR ! appsink")
cv2.error: /home/ubuntu/opencv/modules/highgui/src/cap_gstreamer.cpp:816: error: (-2) GStreamer: unable to start pipeline
 in function cvCaptureFromCAM_GStreamer

And When I tried like this to use the Gstreamer on videos:

cap = cv2.VideoCapture('project_video.mp4src ! video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)I420, framerate=(fraction)24/1 ! \
			nvvidconv flip-method=2 ! video/x-raw, format=(string)I420 ! \
			videoconvert ! video/x-raw, format=(string)BGR ! \
			appsink')

It is waiting.


Edit: I tried the gst on terminal like this and it is working.:

VELEM="v4l2src device=/dev/video0"
VCAPS="image/jpeg, width=1920, height=1080, framerate=30/1"
VSOURCE="$VELEM ! $VCAPS"
VIDEO_SINK="videoconvert ! videoscale ! xvimagesink sync=false"
VIDEO_DECODE="jpegparse ! jpegdec"

gst-launch-1.0 -vvv \
	$VSOURCE \
   	! $VIDEO_DECODE \
   	! $VIDEO_SINK

But if I use that code with opencv VideoCapture like this:

video_capture = cv2.VideoCapture("-vvv v4l2src device=/dev/video0 ! image/jpeg, width=1280, height=720, framerate=30/1 ! jpegparse ! jpegdec ! videoconvert ! videoscale ! xvimagesink sync=false")

the Error is:

cv2.error: /home/ubuntu/opencv/modules/highgui/src/window.cpp:261: error: (-215) size.width>0 && size.height>0 in function imshow

EDIT 2:
I tried to play the video like this:

gst-launch-1.0 playbin uri=file:/home/ubuntu/Desktop/project_video.mp4

After that:

Setting pipeline to PAUSED ...
Pipeline is PREROLLING ...
Inside NvxLiteH264DecoderLowLatencyInitNvxLiteH264DecoderLowLatencyInit set DPB and MjstreamingNvMMLiteOpen : Block : BlockType = 261 
TVMR: NvMMLiteTVMRDecBlockOpen: 5107: NvMMLiteBlockOpen 
NvMMLiteBlockCreate : Block : BlockType = 261 
TVMR: cbBeginSequence: 584: BeginSequence  1280x720, bVPR = 0
TVMR: cbBeginSequence: 826: DecodeBuffers = 6 
TVMR: cbBeginSequence: 850: Display Resolution : (1280x720) 
TVMR: cbBeginSequence: 851: Display Aspect Ratio : (1280x720) 
TVMR: cbBeginSequence: 1015: SurfaceLayout = 3
TVMR: cbBeginSequence: 1045: NumOfSurfaces = 10, InteraceStream = 0, InterlaceEnabled = 0, bSecure = 0, MVC = 0 Semiplanar = 1, bReinit = 1 
Allocating new output: 1280x720 (x 12), ThumbnailMode = 0
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
New clock: GstPulseSinkClock
TVMR: NvMMLiteTVMRDecDoWork: 4138: NVMMLITE_TVMR: EOS detected
TVMR: TVMRBufferProcessing: 3518: Processing of EOS Done
Got EOS from element "playbin0".
Execution ended after 0:00:50.364402184
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
TVMR: TVMRFrameStatusReporting: 3883: Closing TVMR Frame Status Thread -------------
TVMR: TVMRFrameDelivery: 3739: Closing TVMR Frame Delivery Thread -------------
TVMR: NvMMDecTVMRDestroyParser: 4290: NvAvpClose
TVMR: NvMMLiteTVMRDecBlockClose: 5263: Done 
Setting pipeline to NULL ...
Freeing pipeline ...
ubuntu@tegra-ubuntu:~/Desktop$
1 Like

Hi mustafamertunali,

Your pipeline is not correct for opencv. Opencv only supports appsink to catch the buffer from previous element. xvimagesink is a rendering element for video preview. Also, openCV 3.2 only supports gray8 and BGRA. That is why we need videoconv before appsink.