I've plugged in the MIPI CSI camera, but nothing comes up when I open the cam.

Something went quit wrong when I tried to open my cam by index 0
. Of course I’ve plugged in my MIPI CSI cam. I’ve compiled the darknet with opencv, everything worked fine with USB cam, however it was not able to meet our needs with 30 fps on object detection.
I’ve installed gstreamer, opencv 3.4.5, cuda 10 and cudnn.

Is there something missing on my OS, or the MIPI CSI cam needs additional driver to function ?
I’d appreciate it if anyone can help me.

hello huxiaochuan2011,

  1. may I know is your CSI camera being detected by kernel driver? please check /dev/video* to confirm device node has generated.
  2. please also share your commands to access camera sensor
  3. please refer to Camera Architecture Stack and please access camera with v4l2-ctl.
  1. Only USB camera can be detected by opencv

Now here’s my code

#include<opencv2/opencv.hpp>
using namespace cv;
int main()
{
	Mat frame;
	VideoCapture cap;
	cap.open(0);
	  if(!cap.isOpened())
	  return 0;
	while(1)
	{
		cap >> frame;
		if(frame.empty())
			break;
		imshow("cam", frame);
		waitKey(20);
	}
	cap.release();

I do compiled it :
g++ camread.cpp -o camread pkg-config --cflags --libs opencv
And when I tried to run it :

(camread:9086): GStreamer-CRITICAL **: 18:49:28.446: gst_element_get_state: assertion ‘GST_IS_ELEMENT (element)’ failed
VIDEOIO ERROR: V4L: can’t open camera by index 0

I’ve also tried a raspberry pi CSI camera, it didn’t work as well.
Everything worked fine with USB cam.

  1. When I tried
v4l2-ctl --all

terminal showed:

bash: v4l2-ctl: command not found

I’m new to Jetson nano and opencv, so forgive me if there’s anything I ignored.

Hi, huxiaochuan2011

For a CSI camera, you need to access via nvcamerasrc. It’s not exactly the same with V4L2 framework.
Follow the topic here, hope could help:

https://devtalk.nvidia.com/default/topic/987537/jetson-tx1/videocapture-fails-to-open-onboard-camera-l4t-24-2-1-opencv-3-1/post/5064902/#5064902

Thanks alanz and JerryChang. I’ve decided not to use CSI camera and turned to a USB camera instead.
And I’ve installed V4L-utils successfully. Here’s my output by using the command below

v4l2-ctl -d /dev/video0 --list-formats-ext

The terminal prints:

ioctl: VIDIOC_ENUM_FMT
	Index       : 0
	Type        : Video Capture
	Pixel Format: 'MJPG' (compressed)
	Name        : Motion-JPEG
		Size: Discrete 1920x1080
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 1280x720
			Interval: Discrete 0.017s (60.000 fps)
		Size: Discrete 1024x768
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 640x480
			Interval: Discrete 0.008s (120.101 fps)
		Size: Discrete 800x600
			Interval: Discrete 0.017s (60.000 fps)
		Size: Discrete 1280x1024
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 320x240
			Interval: Discrete 0.008s (120.101 fps)

	Index       : 1
	Type        : Video Capture
	Pixel Format: 'YUYV'
	Name        : YUYV 4:2:2
		Size: Discrete 1920x1080
			Interval: Discrete 0.167s (6.000 fps)
		Size: Discrete 1280x720
			Interval: Discrete 0.111s (9.000 fps)
		Size: Discrete 1024x768
			Interval: Discrete 0.167s (6.000 fps)
		Size: Discrete 640x480
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 800x600
			Interval: Discrete 0.050s (20.000 fps)
		Size: Discrete 1280x1024
			Interval: Discrete 0.167s (6.000 fps)
		Size: Discrete 320x240
			Interval: Discrete 0.033s (30.000 fps)

I’ve been searching the nvcamerasrc for the past few days and I used the code below to open my
camera

#include<opencv2/opencv.hpp>
using namespace cv;
int main()
{
	Mat frame;
	VideoCapture cap;
	cap.open("nvarguscamerasrc ! video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080,format=(string)NV12, framerate=(fraction)30/1 ! nvvidconv ! video/x-raw, format=(string)BGRx ! videoconvert ! video/x-raw, format=(string)BGR ! appsink");
	  if(!cap.isOpened())
	  return 0;
	while(1)
	{
		cap >> frame;
		if(frame.empty())
			break;
		imshow("cam", frame);
		waitKey(20);
	}
	cap.release();
}

However it failed once again. Here’s the information:

GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
Segmentation fault (core dumped)

I felt that I’m almost succeed, could you please point out my mistake?

Hi, huxiaochuan2011

I tried with the same code on my Nano board with IMX219 MIPI CSI camera. It works well.
Maybe you can check with dmesg to if there are some driver error with your sensor.

jetbot@jetbot:~$ vim cvtest.cpp 
jetbot@jetbot:~$ g++ -o test cvtest.cpp `pkg-config --cflags --libs opencv`
jetbot@jetbot:~$ export DISPLAY=:0
jetbot@jetbot:~$ ./test 
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 3280 x 2464 FR = 21.000000 fps Duration = 47619048 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 3280 x 1848 FR = 28.000001 fps Duration = 35714284 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1280 x 720 FR = 120.000005 fps Duration = 8333333 ; Analog Gain range min 1.000000, max 10.625000; Exposure Range min 13000, max 683709000;

GST_ARGUS: Running with following settings:
   Camera index = 0 
   Camera mode  = 2 
   Output Stream W = 1920 H = 1080 
   seconds to Run    = 0 
   Frame Rate = 29.999999 
GST_ARGUS: PowerService: requested_clock_Hz=13608000
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.

Also you can check with command: to have a quick view of the mipi csi camera.

nvgstcapture-1.0

Thanks alanz very much ! It is actually the driver’s problem. I’ve successfully opened my CSI camera.