Open CV camera error jetson tx2

Autonomous Machines Jetson & Embedded Systems Jetson TX2
1

I am unable to capture camera output in opencv.

I can successfully run:

gst-launch-1.0 nvcamerasrc fpsRange=“30.0 30.0” ! ‘video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1’ ! nvtee ! nvvidconv flip-method=2 ! ‘video/x-raw(memory:NVMM), format=(string)I420’ ! nvoverlaysink -e

also I can run

nvgstcapture-1.0 -m 2

But when I use opencv it gives error:

If I use:

cap = cv2.VideoCapture(“nvcamerasrc ! ‘video/x-raw(memory:NVMM), width=(int)640, height=(int)480, format=(string)I420, framerate=(fraction)30/1’ ! nvvidconv flip-method=2 ! ‘video/x-raw, format=(string)I420’ ! videoconvert ! ‘video/x-raw, format=(string)BGR’ ! appsink”)

I get None in frames.

I have also tried to use

cap = cv2.VideoCapture(“nvcamerasrc fpsRange=‘30 30’ ! ‘video/x-raw(memory:NVMM), width=(int)3840, height=(int)2160,format=(string)I420, framerate=(fraction)30/1’ ! nvtee ! nvivafilter cuda-process=true customer-lib-name=‘libnvsample_cudaprocess.so’ ! ‘video/x-raw(memory:NVMM), format=(string)NV12’ ! nvoverlaysink -e! appsink”)

which actually works in command line (I mean without open cv but with gst-launch)
It gives me the error:

appsink: Unable to query number of channels

If I just use :

cap = cv2.VideoCapture(0)

I get blank frames.

I have also tried to set

export LD_PRELOAD=“/usr/lib/aarch64-linux-gnu/libv4l/v4l1compat.so”
and
export LD_PRELOAD=“/usr/lib/libv4l/v4l2convert.so”

I am not sure where the problem is

with same results

2

Are you using opencv4tegra ? It’s not able to support gstreamer. You have to build your own opencv lib enabling gstreamer support.
It should work with a gstreamer-1.0 enabled opencv.
I’ve not upgraded to R28, but I’ve seen posts mentioning problems with the onboard camera with this release, but I cannot say much more…you may search the forum.

Since nvoverlaysink is a pure sink, it has no output, and appsink fails to get an input channel thus the error message.
If you want to have 2 outputs, you have to use plugin tee as such:

cap = cv2.VideoCapture("nvcamerasrc  ! 'video/x-raw(memory:NVMM), width=(int)3840, height=(int)2160,format=(string)I420, framerate=(fraction)30/1' ! nvtee ! nvivafilter cuda-process=true customer-lib-name='libnvsample_cudaprocess.so' ! 'video/x-raw(memory:NVMM), format=(string)NV12' ! <b>tee name=t ! queue </b>! nvoverlaysink -e     <b>t. ! queue !</b> appsink")

In this case you are using v4l2 interface on /dev/video0. But the onboard camera is a bayer sensor, frame format should not match with what opencv expects. I’m surprised it succeeded to open the camera.

3

@Honey_Patouceul

I used the command you sent but still the same result.
I only want one output and that needs to go to the opencv video capture.

I have built opencv 3.2 as well I am not using Opencv4Tegra

I also tried to use just appsink without nvoverlaysink

Same result

cap = cv2.VideoCapture("nvcamerasrc fpsRange='30 30' ! 'video/x-raw(memory:NVMM), width=(int)3840, height=(int)2160,format=(string)I420, framerate=(fraction)30/1' ! nvtee ! nvivafilter cuda-process=true customer-lib-name='libnvsample_cudaprocess.so' ! 'video/x-raw(memory:NVMM), format=(string)NV12' ! appsink")
4

Opencv needs to read frames in either gray8 or BGR format. NV12 is not supported. It also needs to read from cpu memory, NVMM memory is not supported.

For now, you should keep trying with

cap = cv2.VideoCapture("nvcamerasrc ! 'video/x-raw(memory:NVMM), width=(int)640, height=(int)480, format=(string)I420, framerate=(fraction)30/1' ! nvvidconv flip-method=2 ! 'video/x-raw, format=(string)I420' ! videoconvert ! 'video/x-raw, format=(string)BGR' ! appsink")

Have you enabled python2 or python3 support when configuring opencv-3.2 ?
What gives the following in python ?

import sys
import cv2
cv2.__version__
5

Hi,

Its giving me same result. I am not sure what is wrong and I know its difficult for you to diagnose.
But still can you give me some insight so as where to look?

Python 2.7.12 (default, Nov 19 2016, 06:48:10) 
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> import cv2
>>> cv2.__version__
'3.2.0'
6

Ok, so opencv-3.2.0 is correctly installed for python.

Could you try this code and send errors if any:

import sys
import cv2

def read_cam():
    cap = cv2.VideoCapture("nvcamerasrc ! video/x-raw(memory:NVMM), width=(int)640, height=(int)480,format=(string)I420, framerate=(fraction)30/1 ! nvvidconv flip-method=2 ! video/x-raw, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink")
    if cap.isOpened():
        cv2.namedWindow("demo", cv2.WINDOW_AUTOSIZE)
        while True:
            ret_val, img = cap.read();
            cv2.imshow('demo',img)
            cv2.waitKey(10)
    else:
       print "camera open failed"

    cv2.destroyAllWindows()

if __name__ == '__main__':
    read_cam()
7

Its the same error:

VIDEOIO ERROR: V4L: device nvcamerasrc ! video/x-raw(memory:NVMM), width=(int)640, height=(int)480,format=(string)I420, framerate=(fraction)30/1 ! nvvidconv flip-method=2 ! video/x-raw, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink: Unable to query number of channels
camera open failed

Do you think its a syntax error?

When I tried it from command line I had to add single inverts without which it was telling me syntax error and then I got the following error:

gst-launch-1.0 nvcamerasrc ! 'video/x-raw(memory:NVMM), width=(int)640, height=(int)480,format=(string)I420, framerate=(fraction)30/1' ! nvvidconv flip-method=2 ! 'video/x-raw, format=(string)BGR' ! videoconvert ! 'video/x-raw, format=(string)BGR'

(gst-launch-1.0:4154): GStreamer-CRITICAL **: gst_element_make_from_uri: assertion 'gst_uri_is_valid (uri)' failed
WARNING: erroneous pipeline: no element "video"

root@tegra-ubuntu:/mydrive/exp# gst-launch-1.0 nvcamerasrc ! 'video/x-raw(memory:NVMM), width=(int)640, height=(int)480,format=(string)I420, framerate=(fraction)30/1' ! nvvidconv flip-method=2 ! 'video/x-raw, format=(string)BGR' ! videoconvert ! 'video/x-raw, format=(string)BGR' ! appsink
WARNING: erroneous pipeline: could not link nvvconv0 to videoconvert0
root@tegra-ubuntu:/mydrive/exp#
8

For gst-launch, can you run

gst-launch-1.0 -ev nvcamerasrc ! 'video/x-raw(memory:NVMM), width=640, height=480, format=I420, framerate=30/1' ! nvvidconv flip-method=2 ! 'video/x-raw, format=BGRx' ! ximagesink

Using appsink as for opencv, adding videoconvert to get BGR:

gst-launch-1.0 -ev nvcamerasrc ! 'video/x-raw(memory:NVMM), width=640, height=480, format=I420, framerate=30/1' ! nvvidconv flip-method=2 ! 'video/x-raw, format=BGRx' ! videoconvert ! 'video/x-raw, format=BGR' ! appsink

For using gstreamer pipeline in opencv, you need an opencv library built with -DWITH_GSTREAMER=ON.
Opencv4tegra doesn’t support gstreamer.

9

I was using opencv build and not tegra

This time I built it again with these options:

-DWITH_GSTREAMER=ON
-DWITH_GSTREAMER_0_10=ON \

Still getting the same result.

I did not uninstall and reinstall… Just ran sudo make install after building it.

The commands works in the command line:

nvcamerasrc ! 'video/x-raw(memory:NVMM), width=(int)640, height=(int)480,format=(string)I420, framerate=(fraction)30/1' ! nvvidconv flip-method=2 ! 'video/x-raw, format=(string)BGRx' ! videoconvert ! 'video/x-raw, format=(string)BGR' ! appsink
10

Ok, so now we know remaining problems are with opencv.

I’m not sure it is a good idea to enable gstreamer-0.10 support.
Could you post the command you’ve launched for configuring opencv-3.2.0 ? Or attach the CMakeCache.txt file from your build directory.
Does it set -DMAKE_BUILD_TYPE=RELEASE ?

You may check [url]http://dev.t7.ai/jetson/opencv/[/url] or [url]http://docs.opencv.org/3.2.0/d6/d15/tutorial_building_tegra_cuda.html[/url]. Configuring for TX2 is similar to TX1, just change -DCUDA_ARCH_BIN=5.3 to -DCUDA_ARCH_BIN=6.2.

Did you set -DCMAKE_INSTALL_PREFIX to something else than /usr ? If you didn’t, it may have installed in /usr, but if you had not purged opencv4tegra, some files of opencv4tegra may remain. If so, I’d suggest to delete any file/folder matching /usr/include/opencv* or /usr/lib/libopencv*, and reinstall opencv-3.2.0.

[EDIT: one extra file to be removed for cleanup is the file for pkg-config: /usr/lib/pkgconfig/opencv.pc. make install should install the new one in its location according to CMAKE_INSTALL_PREFIX]

11

I configured and made using:

cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local -D WITH_CUDA=ON -D CUDA_ARCH_BIN=“6.2” -D CUDA_ARCH_PTX=“” -D WITH_OPENGL=ON -D WITH_LIBV4L=ON -D CMAKE_SHARED_LINKER_FLAGS=-Wl,-Bsymbolic …

If you enable gstreamer-0.10 support, it says it cannot support gstreamer-1.0 so I think you must disable it.

In addition, maybe not relevant but, if you cannot open video files (not camera), refer to [url]python - Can't open video using opencv - Stack Overflow.

12

I configured and made using:

cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local -D WITH_CUDA=ON -D CUDA_ARCH_BIN=“6.2” -D CUDA_ARCH_PTX=“” -D WITH_OPENGL=ON -D WITH_LIBV4L=ON -D CMAKE_SHARED_LINKER_FLAGS=-Wl,-Bsymbolic …

In addition, maybe not relevant but, if you cannot open video files (not camera), refer to [url]python - Can't open video using opencv - Stack Overflow.

13

@worthlessfella

How about this one?

cap = cv2.VideoCapture("nvcamerasrc ! video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1 ! nvtee ! nvvidconv flip-method=2 ! video/x-raw(memory:NVMM), format=(string)I420 ! nvoverlaysink -e ! appsink")

I am using OpenCV 3.2. and I can read from camera.javascript:void();

14

Here’s how I built OpenCV-3.3.0 on Jetson TX2. I can run Honey_Patouceul’s sample python code and see inverted camera images with the TX2 on-board camera. FYI~

[url]https://jkjung-avt.github.io/opencv3-on-tx2/[/url]

15

I would like to thank everyone for your help but special thanks to @Honey_Patouceul!!

Lessons learnt:

I had installed opencv4tegra and then i installed opencv by building.

Mistakes:

-DWITH_GSTREAMER was set to OFF while building.

The opencv4tegra files were not properly removed.

Addressing the above problems as suggested by @Honey_Patouceul helped.

16

I am a college student from Taiwan, and my English is not good.

I succeeded too.
it’s my method.

First of all, a original Jetpack environment (I use Jetpack 3.1 on TX2)

step1:purge opencv 2.x & opencv4tegra

sudo apt purge libopencv*
sudo apt-get purge libopencv4tegra-dev libopencv4tegra
sudo apt-get purge libopencv4tegra-repo
sudo apt-get update

step2: check remove

apt-cache show libopencv4tegra
pkg-config --modversion opencv
pkg-config --cflags --libs opencv
sudo apt search opencv | grep installed

step3:install necessary items

sudo apt-get install libv4l-dev 
sudo apt-get install v4l2ucp
sudo apt-get install v4l-utils
sudo apt-get install \
    libglew-dev \
    libtiff5-dev \
    zlib1g-dev \
    libjpeg-dev \
    libpng12-dev \
    libjasper-dev \
    libavcodec-dev \
    libavformat-dev \
    libavutil-dev \
    libpostproc-dev \
    libswscale-dev \
    libeigen3-dev \
    libtbb-dev \
    libgtk2.0-dev \
    pkg-config

step4:download opencv sourse code (opencv 3.2.0) (https://opencv.org/releases.html).

step5:cmake opencv

sudo apt-get install cmake
cd /your opencv dir/
cmake \
    -DCMAKE_BUILD_TYPE= RELEASE\
    -DCMAKE_INSTALL_PREFIX=/usr \
    -DBUILD_PNG=OFF \
    -DBUILD_TIFF=OFF \
    -DBUILD_TBB=OFF \
    -DBUILD_JPEG=OFF \
    -DBUILD_JASPER=OFF \
    -DBUILD_ZLIB=OFF \
    -DBUILD_EXAMPLES=ON \
    -DBUILD_opencv_java=OFF \
    -DBUILD_opencv_python2=ON \
    -DBUILD_opencv_python3=OFF \
    -DENABLE_PRECOMPILED_HEADERS=OFF \
    -DWITH_OPENCL=OFF \
    -DWITH_OPENMP=OFF \
    -DWITH_FFMPEG=ON \
    -DWITH_GSTREAMER=ON \
    -DWITH_GSTREAMER_0_10=OFF \
    -DWITH_CUDA=ON \
    -DWITH_GTK=ON \
    -DWITH_VTK=OFF \
    -DWITH_TBB=ON \
    -DWITH_1394=OFF \
    -DWITH_OPENEXR=OFF \
    -DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-8.0 \
    -DCUDA_ARCH_BIN=6.2 \
    -DCUDA_ARCH_PTX="" \
    -DINSTALL_C_EXAMPLES=ON \
    -DINSTALL_TESTS=OFF \
    -DOPENCV_TEST_DATA_PATH=../opencv_extra/testdata \
    ../opencv-3.2.0

note:
-DCMAKE_BUILD_TYPE= RELEASE
-DWITH_GSTREAMER=ON
-DCUDA_ARCH_BIN=6.2

step6:make opencv

make -j6

step7:install opencv

sudo make install

It’s my python code,you can try it.

import numpy as np
import cv2

cap = cv2.VideoCapture("nvcamerasrc ! video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)I420, framerate=(fraction)24/1 ! nvvidconv flip-method=6 ! video/x-raw, format=(string)I420 ! videoconvert ! video/x-raw, format=(string)BGR ! appsink")
#cap=cv2.VideoCapture(0)
#cap=cv2.VideoCapture("/dev/video0")

if not cap.isOpened() :
	print("not capture")
	exit()
ret,frame = cap.read()
while ret :
	cv2.imshow('frame',frame)
	ret,frame = cap.read()
	if(cv2.waitKey(1) & 0xFF == ord('q')):
		break;

cap.release()
cv2.destroyAllWindows()

Welcome advice.

17

Hi eeee2222345,

It’s great to seeing you share the experience, and looking forward to see the interesting project from you.

Cheers