OpenCV VideoCapture with onboard CSI Camera?

Hi there,

I am trying to get a little camera preview window running with some OpenCV code.

I naively tried the example here first: http://docs.opencv.org/3.0-beta/modules/videoio/doc/reading_and_writing_video.html

and the result was a failure of opening the capture with another error: HIGHGUI ERROR: V4L2: Pixel format of incoming image is unsupported by OpenCV

After some reading, it turned out that you have to write

VideoCapture cap("nvcamerasrc ! video/x-raw(memory:NVMM), width=(int)640, height=(int)480, format=(string)I420, framerate=(fraction)30/1 ! nvvidconv flip-method=2 ! video/x-raw, format=(string)I420 ! videoconvert ! video/x-raw, format=(string)BGR ! appsink");

instead of

VideoCapture cap(0);

However, this still does not succeed! There is no more HIGHGUI error, but the capture is not working.

Am I missing something? I tried compiling with g++ and cmake, both have the same result.

Thanks

Hi,

Please check if you have enabled gstreamer option when building.
More information can be found here:
https://devtalk.nvidia.com/default/topic/1001696/

How can I check this?

I’m not sure if I’m doing this right, when building the application I write

cmake -DCUDA_USE_STATIC_CUDA_RUNTIME=OFF -DWITH_GSTREAMER=ON ..

but it later says

CMake Warning:
  Manually-specified variables were not used by the project:

    WITH_GSTREAMER

Also, I tried the code from the link you posted, but it seems that I do not have “opencv2/videoio.hpp”, is that the problem?

Thanks for your help.

edit: I will try upgrading my OpenCV from 2.4.13 to 3.1.0 and see what happens …

Have you installed gstreamer dev packages ?

sudo apt-get install libgstreamer1.0-dev libgstreamer-plugins-base1.0-dev

You may check http://dev.t7.ai/jetson/opencv/ for other required or useful packages to be installed for building opencv.

Furthermore, which version of opencv are you building ? I would suggest 3.2.0, but 3.1.0 also works.

If this doesn’t work, post your configure command and its output.

Yup I have all of those, and I can access the camera running gstreamer straight from command line.

I will do 3.2.0, but which brings up another question, why does it not come with the latest Jetpack (which I flashed)???

edit: is OpenCV4Tegra much faster than OpenCV ?

Opencv4tegra is a Nvidia closed source version, based on opencv-2.4.13 (.12 in older versions), built with many optimizations for tegra chips. AFAIK, it doesn’t support gstreamer.

So you just have to build your own opencv-3.2.0 library (this implies configure with cmake and build with make), I would advise to build it on a disk having at least 4GB free, and then install it on your TX1 with

sudo make install

that will install it into the location provided at configure time with CMAKE_INSTALL_PREFIX.

Be aware if you keep opencv4tegra, it is installed in /usr, and will therefore be found by gcc instead of opencv3 includes if you don’t provide opencv3 include path namely with -I. Same applies for opencv librairies path at link time with -L.
Check the previously mentioned example for the syntax if you’re not familiar with it.

[EDIT answer: for speed, you just have to benchmark. Building with linaro gcc5 gives also faster code (opencv4tegra is built with gcc4), opencv-3.2.0 uses nvidia carotene low level optimizations library. Depends on what you want to do. But if gstreamer pipeline is mandatory, IMHO you have no choice.]

Thank you! I believe this is true, the OpenCV that comes with Jetpack simply doesn’t support gstreamer.

That’s kind of weird.

Thanks again for the tips.

Never believe me ! ;-) I also give sometimes weird advices…
Just let us know if it works or post error logs if not.