Onboard camera /dev/video0

Hello.
I want to use /dev/video0 for onboard camera since “cvcapturefromcam(arg)” function is used in the downloaded program.
Some can use onboard camera using videocapture() function and pipelining gststreamer.
However, gststreamer can not work with cvcapturefromcam().
The webcam (/dev/video1) works fine.
It seems that the onboard camera is allocated to /dev/video0 as below.
However, it did not work.
Do you have any solution?

nvidia@tegra-ubuntu:~/opencv/samples/cpp/cpp$ v4l2-ctl --list-devices
VIDIOC_QUERYCAP: failed: Inappropriate ioctl for device
VIDIOC_QUERYCAP: failed: Inappropriate ioctl for device
vi-output, ov5693 2-0036 (platform:15700000.vi:2):
/dev/video0

SAMSUNG (SPC-A400MB) (usb-3530000.xhci-2.4):
/dev/video1
/dev/v4l-subdev1
/dev/v4l-subdev0

Hi junggun,

gstreamer pipeline is enabled after opencv3.x. Which version of opencv do you install?

Opencv version is 3.2.0.
When I run the program, it shows the below error message.

VIDEOIO ERROR: V4L2: Pixel format of incoming image is unsupported by OpenCV

nvidia@tegra-ubuntu:~/caffe-ssd/ssd-tx1$ v4l2-ctl -d /dev/video0 --list-formats
ioctl: VIDIOC_ENUM_FMT
Index : 0
Type : Video Capture
Pixel Format: ‘RGGB’
Name : 8-bit Bayer RGRG/GBGB

    Index       : 1
    Type        : Video Capture
    Pixel Format: 'RG10'
    Name        : 10-bit Bayer RGRG/GBGB

    Index       : 2
    Type        : Video Capture
    Pixel Format: 'RG12'
    Name        : 12-bit Bayer RGRG/GBGB

nvidia@tegra-ubuntu:~/caffe-ssd/ssd-tx1$ v4l2-ctl -d /dev/video1 --list-formats
ioctl: VIDIOC_ENUM_FMT
Index : 0
Type : Video Capture
Pixel Format: ‘YUYV’
Name : YUYV 4:2:2

The usb camera on /dev/video1 provides YUYV, but the onboard camera on /dev/video0 is a bayer sensor, as you can see above its 3 formats are all bayer type. You have to debayer it and convert into a suitable format for OpenCV.
Using gstreamer, you can use nvcamerasrc that will do that for you with HW.

If you have enabled gstreamer-1.0 in your opencv-3.2.0 build, it is correctly built and installed, with no interference of opencv4tegra (be sure to give opencv3 include and libs paths when building your application), then it might work, but not sure at all. You may just try changing:

cvcapturefromcam(arg)

to

const char *gst = "nvcamerasrc ! video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)I420, framerate=(fraction)30/1 ! \
	           nvvidconv flip-method=6 ! video/x-raw, format=(string)I420 ! \
		   videoconvert ! video/x-raw, format=(string)BGR ! \
		   appsink";
CvCapture* myCam = cvCaptureFromFile(gst);

[EDIT: For an example of using gstreamer with OpenCv C API, check https://devtalk.nvidia.com/default/topic/1026062/jetson-tx1/jetson-tx1-onboard-camera-cvcapturefromcam/post/5219738/#5219738]

You may also activate gstreamer debug traces by adding this before:

putenv("GST_DEBUG=*:3");

This activates verbosity level 3 for all elements of the pipeline. Understanding these traces and how gstreamer links elements also takes some time.

I have tried the below code in advance (C++)
However, I got the error message.
gst-lanch-1.0 is working well.

gst-launch-1.0 nvcamerasrc fpsRange=“30.0 30.0” ! ‘video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)I420, framerate=(fraction)30/1’ ! nvtee ! nvvidconv flip-method=2 ! ‘video/x-raw(memory:NVMM), format=(string)I420’ ! nvoverlaysink -e

It seems that opencv problem.

#include <iostream>
#include <opencv2/opencv.hpp>
#include <opencv2/videoio.hpp>
#include <opencv2/highgui.hpp>


int main()
{
   const char* gst =  "nvcamerasrc ! video/x-raw(memory:NVMM), width=(int)1280, height=(int)720, format=(string)I420, framerate=(fraction)120/1 ! \
			nvvidconv flip-method=6 ! video/x-raw, format=(string)I420 ! \
			videoconvert ! video/x-raw, format=(string)BGR ! \
			appsink";

  cv::VideoCapture cap(gst);


    if(!cap.isOpened())
    {
	std::cout<<"Failed to open camera."<<std::endl;
	return -1;
    }

    unsigned int width = cap.get(CV_CAP_PROP_FRAME_WIDTH); 
    unsigned int height = cap.get(CV_CAP_PROP_FRAME_HEIGHT); 
    unsigned int pixels = width*height;
    std::cout <<"Frame size : "<<width<<" x "<<height<<", "<<pixels<<" Pixels "<<std::endl;

    cv::namedWindow("MyCameraPreview", CV_WINDOW_AUTOSIZE);
    cv::Mat frame_in(width, height, CV_8UC3);

    while(1)
    {
    	if (!cap.read(frame_in)) {
		std::cout<<"Capture read error"<<std::endl;
		break;
    	}
	else {
   		cv::imshow("MyCameraPreview",frame_in);
    		cv::waitKey(1000/120); // let imshow draw and wait for next frame 8 ms for 120 fps
	}	
    }

    cap.release();

    return 0;
}

nvidia@tegra-ubuntu:~/opencv/samples/cpp$ g++ video.cpp -o video pkg-config --cflags --libs opencv
nvidia@tegra-ubuntu:~/opencv/samples/cpp$ ./video
Failed to open camera.
nvidia@tegra-ubuntu:~/opencv/samples/cpp$

The question is does the used opencv library supports gstreamer. What gives:

pkg-config --cflags --libs opencv

This is what I have.
It seems that the gstreamer is not included.
I have tried to build with gstreamer-1.0.
However, it shows the same results.

nvidia@tegra-ubuntu:~/opencv/samples/cpp$ pkg-config --cflags --libs opencv
-I/usr/include/opencv -lopencv_cudabgsegm -lopencv_cudaobjdetect -lopencv_cudastereo -lopencv_shape -lopencv_stitching -lopencv_cudafeatures2d -lopencv_superres -lopencv_cudacodec -lopencv_videostab -lopencv_cudaoptflow -lopencv_cudalegacy -lopencv_calib3d -lopencv_features2d -lopencv_objdetect -lopencv_highgui -lopencv_videoio -lopencv_photo -lopencv_imgcodecs -lopencv_cudawarping -lopencv_cudaimgproc -lopencv_cudafilters -lopencv_video -lopencv_ml -lopencv_imgproc -lopencv_flann -lopencv_cudaarithm -lopencv_core -lopencv_cudev

nvidia@tegra-ubuntu:~/opencv/samples/cpp$ g++ video.cpp -o video pkg-config --cflags --libs gstreamer-1.0 opencv
nvidia@tegra-ubuntu:~/opencv/samples/cpp$ ./video
Failed to open camera.

The flags returned by pkg-config make me wonder: Have you installed opencv-3.2.0 in /usr (I suppose you had uninstalled opencv4tegra before) ? Not sure about the include path…In this case it should be -I/usr/include [EDIT: looks like that’s normal way for opencv pkgconfig files, although not what I’d use (opencv1?)… but this may be out of topic.]

You may also activate g++ warnings (-Wall) and post if any.

You can see which dynamic libraries will be loaded for your app with

ldd ./video

You may try to compile without pkg-config and give explicitly the paths for include and libs to g++ as in post #5.

I think building Opencv 3.2.0 with GStreamer will solve the problem, but the question is how…
I try to build it according to JetsonHacks ‘http://www.jetsonhacks.com/2017/04/05/build-opencv-nvidia-jetson-tx2/’, and set ‘-DWITH_GSTREAMER=ON’ and ‘-D WITH_GSTREAMER_0_10=ON’ but it does not work.