Jetson TX1 OpenCV 3 camera failed

Hello there,

I just got a jetson TX1 and I want to use openCV with the module camera on the board. I tried many things and nothing works.

I installed -Jepack 3.0 (kernel 3.10.96 / ubuntu 16.04) and built openCV 3 using this tutorial: http://docs.opencv.org/3.2.0/d6/d15/tutorial_building_tegra_cuda.html

Is not possible to initialize the camera. when I the camera with gst-launch-1.0, it works!!!

gst-launch-1.0 nvcamerasrc fpsRange=“30.0 30.0” ! ‘video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1’ ! nvtee ! nvvidconv flip-method=2 ! ‘video/x-raw(memory:NVMM), format=(string)I420’ ! nvoverlaysink -e

I compiled my source code with this command : g++ face1.cpp -o test pkg-config --cflags --libs opencv

my program used all the openCV libs :
ubuntu@tegra-ubuntu:~/test$ ldd test
linux-vdso.so.1 => (0x0000007f86e1e000)
libopencv_highgui.so.3.1 => /usr/lib/libopencv_highgui.so.3.1 (0x0000007f86de0000)
libopencv_videoio.so.3.1 => /usr/lib/libopencv_videoio.so.3.1 (0x0000007f86da7000)
libopencv_core.so.3.1 => /usr/lib/libopencv_core.so.3.1 (0x0000007f86a1f000)

my source is:

#include
#include <stdio.h>
#include <opencv2/opencv.hpp>
#include “opencv2/videoio.hpp”

[i]using namespace cv;
using namespace std;

int main()
{
VideoCapture cap(“nvcamerasrc ! video/x-raw(memory:NVMM), width=(int)1280, height=(int)720,format=(string)I420, framerate=(fraction)24/1 ! nvvidconv flip-method=2 ! video/x-raw, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink”);

if (!cap.isOpened())
{ 
	cerr << "Fail to open camera " << endl;
	return -1;
}
for(;;)
{
	Mat frame;
	cap >> frame; 
	imshow("window", frame);
	waitKey(1);
}
cap.release();
return 0;

}[/i]

I would like to known how to use it. Please help me.

Best Regards
Denis

Hi Denis,
Please try

cap = cv2.VideoCapture("nvcamerasrc ! video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1 ! nvtee ! nvvidconv flip-method=2 ! video/x-raw(memory:NVMM), format=(string)I420 ! nvoverlaysink -e ! appsink")

Hello WayneWWW,

What is cv2 ? I can’t compile the source code. Or maybe I have to add an include from openCV.

Best Regards
Denis

Hi Denis,

Sorry for misleading. “cv2” is for python code.

What I meant is you could try the gstreamer pipeline.

I’m a new in the jetson field.

if you mean by using “gst-launch-1.0”. it works very well. how can I check if openCV calls gstreamer ?

Do you have a tutorial from scratch to make everything well. I would like to restart from the beginning.

Thanks a lot

Ldd shows opencv3 librairies. Did you install opencv3 through a package ? If not and you compiled and installed your own version of opencv3, then the flags returned by pkg-config --cflags --libs opencv may refer to opencv4tegra 2.4’s include files. Try to pass -I<path_to_opencv3_include>.

Furthermore, I think there was some opencv bug where using:

const char* gst = " ...your gst pipeline ";
VideoCapture cap(gst);

could be a fix.

Just last detail: if you are using >> operator for reading, then the only way to know when it fails is catching exception. Otherwise, you can check return code using cap.read().

You can also check this example as starting point:https://devtalk.nvidia.com/default/topic/1001696/jetson-tx1/failed-to-open-tx1-on-board-camera/post/5117370/#5117370

Hello Honey

I compiled and installed from this procedure: http://docs.opencv.org/3.2.0/d6/d15/tutorial_building_tegra_cuda.html. I tried to install openCV 3 without openCV 2.4.

the cap.isOpened() return false after creating the object VideoCapture.

I have no clue what to do now

Hi Denis,
Not sure what you meant with install opencv3 without opencv2-4.
In a few words, opencv4tegra-2.4.13 is installed in /usr by nvidia packaging of libopencv4tegra. So its include files are in /usr/include/opencv2 (and legacy /usr/include/opencv) and its libs are in /usr/lib. These are default paths for most compilers.
Furthermore, you are using flags from pkg-config, but when you look at what pkg-config returns from the package known by apt or dpkg as opencv, you should find only some references to this installation of opencv4tegra-2.4.13:

pkg-config --cflags opencv
-I/usr/include/opencv

pkg-config --libs opencv
-L/usr/local/cuda-8.0/lib64 -lopencv_calib3d -lopencv_contrib -lopencv_core -lopencv_features2d -lopencv_flann <b>-lopencv_gpu</b> -lopencv_highgui -lopencv_imgproc -lopencv_legacy -lopencv_ml -lopencv_objdetect -lopencv_photo -lopencv_stitching -lopencv_superres -lopencv_ts -lopencv_video -lopencv_videostab -lopencv_detection_based_tracker -lopencv_esm_panorama -lopencv_facedetect -lopencv_imuvstab -lopencv_tegra -lopencv_vstab -lcufft -lnpps -lnppi -lnppc -lcudart -latomic -ltbb -lrt -lpthread -lm -ldl

I bolded libopencv_gpu because it is typical 2.4, as opencv3 has a different naming scheme explicitly naming cuda for cuda backends.

So, assuming you’ve installed your opencv3.2.0 version in /usr/local/opencv-3.2.0, you should compile with your own flags for opencv-3.2.0 include files and libs:

export LD_LIBRARY_PATH=/usr/local/cuda/nvvm/lib:/usr/local/cuda/lib64:/usr/local/opencv-3.2.0/lib
g++ -I/usr/local/opencv-3.2.0/include your_main.cpp -L/usr/local/opencv-3.2.0/lib -lopencv_core -lopencv_videoio  -o your_app

If it still fails to open camera when you do so, then probably there is a problem in your pipeline. In such case, you may try the example I mentionned before and when you get it working start changing your pipeline one thing at a time.

Which video configuration did you enable in yourOpenCV cmake configuration? In the link provided, GStreamer is set to OFF for the TX1, it’s unlikely that the camera would work. Here’s a different thread that might be worth reading through:

https://devtalk.nvidia.com/default/topic/965134/jetson-tx1/opencv-3-1-compilation-on-tx1-lets-collect-the-quot-definitive-quot-cmake-settings-/post/5067204/#5067204

Hello,

I’m restarting from the beginning. I mean to re-flash the TX1 from JetPack 3.0.

After flashing evrything on the TX1, I’m going to compile openCV3 with this procedure giving by Honey: http://dev.t7.ai/jetson/opencv/

and then I’ll try to run my small program using the camera on the TX1.

int main()
{
VideoCapture cap("nvcamerasrc ! video/x-raw(memory:NVMM), width=(int)1280, height=(int)720,format=(string)I420, framerate=(fraction)24/1 ! nvvidconv flip-method=2 ! video/x-raw, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink");

if (!cap.isOpened())
{
cerr << "Fail to open camera " << endl;
return -1;
}

if I have to do something else, please tell me

Thanks a lot

The instructions that you noted do not appear to enable Gstreamer, which your code appears to reference. The two flags for GStreamer are WITH_GSTREAMER, WITH_GSTREAMER_0_10

In the Install document you referenced, the GStreamer support libraries are installed, but the actual build doesn’t include them. It looks like the support libraries are for GStreamer 1.0.

Also the ENABLE_NEON is superfluous on the TX1, the compiler assumes all aarch64 machines are NEON enabled. I don’t think anything bad happens, it’s just ignored.

hello,

I got the bellow message when I tried to compile openCV version 3.2.0 from this procedure: http://dev.t7.ai/jetson/opencv/

it seems that some headers are not synchronized.

Scanning dependencies of target opencv_core
[ 36%] Building CXX object modules/core/CMakeFiles/opencv_core.dir/src/array.cpp.o
[ 36%] Building CXX object modules/core/CMakeFiles/opencv_core.dir/src/out.cpp.o
[ 36%] Building CXX object modules/core/CMakeFiles/opencv_core.dir/src/datastructs.cpp.o
[ 36%] Building CXX object modules/core/CMakeFiles/opencv_core.dir/src/stl.cpp.o
[ 36%] Building CXX object modules/core/CMakeFiles/opencv_core.dir/src/dxt.cpp.o
[ 36%] Building CXX object modules/core/CMakeFiles/opencv_core.dir/src/lpsolver.cpp.o
[ 36%] Building CXX object modules/core/CMakeFiles/opencv_core.dir/src/opengl.cpp.o
In file included from /home/ubuntu/opencv/modules/core/src/opengl.cpp:48:0:
/usr/local/cuda-8.0/include/cuda_gl_interop.h:64:2: error: #error Please include the appropriate gl headers before including cuda_gl_interop.h
 #<b>error Please include the appropriate gl headers before including cuda_gl_interop.h</b><u></u>
  ^
modules/core/CMakeFiles/opencv_core.dir/build.make:241: recipe for target 'modules/core/CMakeFiles/opencv_core.dir/src/opengl.cpp.o' failed
make[2]: *** [modules/core/CMakeFiles/opencv_core.dir/src/opengl.cpp.o] Error 1
make[2]: *** Waiting for unfinished jobs....
CMakeFiles/Makefile2:1532: recipe for target 'modules/core/CMakeFiles/opencv_core.dir/all' failed
make[1]: *** [modules/core/CMakeFiles/opencv_core.dir/all] Error 2
Makefile:160: recipe for target 'all' failed
make: *** [all] Error 2

For compiling with opengl, you have to patch file opencv-3.1.0/modules/cudalegacy/src/graphcuts.cpp by replacing:

#if !defined (HAVE_CUDA) || defined (CUDA_DISABLER)

with

#if !defined (HAVE_CUDA) || defined (CUDA_DISABLER)  || (CUDART_VERSION >= 8000)

[EDIT: I copy/pasted that from a previous post related to opencv-3.1.0, but the same should apply to opencv-3.2.0]

[EDIT2: Reading this today, I suppose I was very tired when posting this.
What I wanted to advise was to modify file /usr/local/cuda/samples/common/inc/helper_gl.h :

line105: --- #if !defined(GLX_EXTENSION_NAME) || defined(__arm__) || defined(__aarch64__)
line105: +++ #if !defined(GLX_EXTENSION_NAME) || !defined(GL_VERSION_1_3)

]

[EDIT3: read this for a more efficient way: https://devtalk.nvidia.com/default/topic/1007290/jetson-tx2/building-opencv-with-opengl-support-/post/5141945/#5141945]

Ok I’ll make a test with those changes.

Using an another procedure, it works. I use openCV 3.1 including some changes for cuda and so on.

thanks a lot

Maybe you should build opencv with gstreamer.
I build your code with my lib(3.2).It’s ok.
thanks.