Open CSI camera on Jetson TX2 except the onboard camera


I could open onboard csi camera on TX2 with OpenCV gstreamer command:

VideoCapture cap(“nvarguscamerasrc ! video/x-raw(memory:NVMM), width=(int)2592, height=(int)1458, format=(string)NV12, framerate=(fraction)30/1 ! nvvidconv flip-method=0 ! video/x-raw, width=640, height=360, format=(string)BGRx ! videoconvert ! appsink”);

Now I want to use 2 csi cameras on the Jetson TX2 board, so I connected the Jetson TX2 board to the Xilinx FPGA board, then I imitate the onboard camera driver and write my own camera driver.

My camera is a grayscale camera, video out 8-bit raw data, 1024*768. After conversion through FPGA, the raw data becomes 10-bit Bayer data into the TX2.

I could open my csi camera with v4l2-ctl and v4l2 API. The v4l2-ctl command like these:

$ ./v4l2-ctl -d /dev/video0 --list-formats
Index : 0
Type : Video Capture
Pixel Format: ‘BG10’
Name : 10-bit Bayer BGBG/GRGR

$ ./v4l2-ctl -d /dev/video0 --set-fmt-video=width=1024,height=768,pixelformat=BG10 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=1 --stream-to=test0.raw

Then I got a raw format picture named test0.raw. I can open it as a 16-bit grayscale picture.

In the v4l2 API, I set pixelformat to V4L2_PIX_FMT_SBGGR10, width to 1024, height to 768, I use cv::Mat Img(768, 1024, CV_16UC1, bufdata) to receive raw10 data, now I can open my csi camera on TX2 using OpenCV.

But I find that the v4l2 API method consumes a lot of CPU resources, I want to increase frame rate, so is there a gstreamer way to open other csi cameras, like onboard camera’s command “nvarguscamerasrc ! video/x-raw(memory:NVMM)…”?

Any comments or suggestions would be greatly appreciated.


Have a reference to 12_camera_v4l2_cuda of tegra_multimedia_api to avoid memory copy cause performance issue.