builtin and USB cameras work in gstreamer but not python

Autonomous Machines Jetson & Embedded Systems Jetson TX1
1

We are trying to port some Python code from a Raspberry Pi to Jetson TX1 (24.1 64bit). We have the devkit camera and a USB camera working from command line using gstreamer.

gst-launch-1.0 v4l2src device=“/dev/video1” ! ‘video/x-raw, width=640, height=480, format=(string)I420’ ! videoconvert ! ximagesink

gst-launch -v nvcamerasrc fpsRange=“30.0 30.0” ! ‘video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1’ ! nvvidconv flip-method=2 tnr=6 ! ‘video/x-raw, width=(int)1920, height=(int)1080, format=(string)BGRx’ ! tee name=tp tp. ! queue max-size-buffers=2 ! ximagesink name=highres tp. ! queue max-size-buffers=2 ! videocrop left=240 right=240 ! videoscale ! ‘video/x-raw, width=(int)640, height=(int)480, format=(string)BGRx, pixel-aspect-ratio=(fraction)1/1’ ! ximagesink name=lowres

Frame rates are marginal and would like to use the opencv4tegra acceleration (not to excited about moving to OpenCV3.1). Seems acc not realistic for USB and we could not get nvtee to split the stream correctly. Ideas on nvtee would be welcome but not the main issue.

The old code used a USB camera, so we would like to get that working first. The following seems to connect to the camera, but we dont know how to link the gst-pipe into openCV.

# copied from https://gist.github.com/jampekka/ec4a3f3a3748fd2a281d/
import numpy as np
import cv2
import gi
gi.require_version('Gst', '1.0')
from gi.repository import GObject, Gst

# This doesn't seem to make any difference
#GObject.threads_init()

Gst.init(None)

# This works

pipe = Gst.parse_launch("""v4l2src device=/dev/video1 !
       appsink sync=false max-buffers=2 drop=true name=sink emit-signals=true""")

sink = pipe.get_by_name('sink')
pipe.set_state(Gst.State.PLAYING)

while True:
	print "Getting a sample"
	sample = sink.emit('pull-sample')
	print sample.get_buffer().get_size()

The following fails to open the camera:
** (cam.py:8145): WARNING **: Couldn’t connect to accessibility bus: Failed to connect to socket /tmp/dbus-HmKP8cTCj6: Connection refused
2016-08-02 13:41:31,340 - ERROR - Camera Open Failed

import numpy as np
import cv2
import sys
import gi
gi.require_version('Gst', '1.0')
from gi.repository import GObject, Gtk
from gi.repository import Gst as gst
#from gi.repository import GdkX11, GstVideo
GObject.threads_init()
gst.init(None)
#mygstpipe = gst.Pipeline()

import logging
logging.basicConfig(level=logging.DEBUG, format='%(asctime)s - %(levelname)s - %(message)s')

mycam = cv2.VideoCapture("v4l2src device='/dev/video1' ! 'video/x-raw, width=1280, height=720, format=I420' ! videoconvert ! appsink sync=false max-buffers=2 drop=true name=sink emit-signals=true")

#mycam = cv2.VideoCapture("nvcamerasrc ! 'video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1' ! nvvidconv flip-method=2 ! 'video/x-raw, format=(string)BGRx' ! videoconvert ! appsink")

if mycam.isOpened():
	rval, frame = mycam.read()
	logging.info('Hit ESC to close')
else:
	rval = False
	logging.error('Camera Open Failed')

while rval:
	cv2.imshow("feed", frame)
	rval, frame = mycam.read()
	key = cv2.waitKey(20)
	if key == 27: # exit on ESC
		break

cv2.destroyWindow("feed")

Thoughts on where to go next?

2

Hi gpetilli,

Regarding onboard camera, due to pixel format issue, it can’t work well with OpenCV4Tegra, but should be no problem with OpenCV3.x.

We did the test with OpenCV 3.1.0 with onboard camera, we can see it work successfully.
Here are some tips to open the default camera

  1. Please try openCV version be 3.0.0 or higher. we’re using 3.1.0
  2. GST 1.0 and related plugins must be installed.
  3. test program must be linked against built openCV 3.1.0 libs
  4. test code looks like:
VideoCapture cap("nvcamerasrc ! video/x-raw(memory:NVMM), width=(int)1280, height=(int)720,format=(string)I420, framerate=(fraction)24/1 ! nvvidconv flip-method=2 ! video/x-raw, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink"); //open the default camera

if(!cap.isOpened()) { // check if we succeeded
cerr << "Fail to open camera " << endl;
return -1;
}
for(;;)
{
Mat frame;
cap >> frame; // get a new frame from camera
imshow("original", frame);
waitKey(1);
}
// the camera will be deinitialized automatically in VideoCapture destructor
cap.release();

Thanks