How can I capture images from V4L2 camera with gstreamer?

Autonomous Machines Jetson & Embedded Systems Jetson AGX Orin
,
5

Format conversion with opencv may not be the best solution, you may get better performance with gstreamer.

For using v4l2src, you may try to run a gstreamer pipeline to display for checking such as:

gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,format=UYVY,width=3840,height=2160,framerate=30/1 ! nvvidconv ! autovideosink

Maybe for your case this might also work;

gst-launch-1.0 nvv4l2camerasrc device=/dev/video0 ! 'video/x-raw(memory:NVMM),format=UYVY,width=3840,height=2160,framerate=30/1' ! nvvidconv ! autovideosink

Then the conversion to BGR for opencv may be done such as:

# Using CPU-based videoconvert
gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,format=UYVY,width=3840,height=2160,framerate=30/1 ! videoconvert ! video/x-raw,format=BGR ! fakesink -v

# Using HW VIC-based nvvidconv for conversion into BGRx and CPU-based for removing 4th byte into BGR format:
gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,format=UYVY,width=3840,height=2160,framerate=30/1 ! nvvidconv ! 'video/x-raw(memory:NVMM)' ! nvvidconv ! video/x-raw,format=BGRx ! videoconvert ! video/x-raw, format=BGR ! fakesink -v

# Same with nvv4l2camerasrc would be better as saving first copy from system memory into NVMM
gst-launch-1.0 nvv4l2camerasrc device=/dev/video0 ! 'video/x-raw(memory:NVMM),format=UYVY,width=3840,height=2160,framerate=30/1' ! nvvidconv ! video/x-raw,format=BGRx ! videoconvert ! video/x-raw, format=BGR ! fakesink -v

Choose the one performing best and just replace fakesink with appsink drop=1 and remove single quotes into gstreamer pipeline for opencv appsink such as:

cv::VideoCapture cap("nvv4l2camerasrc device=/dev/video0 ! video/x-raw(memory:NVMM),format=UYVY, width=3840,height=2160,framerate=30/1 ! nvvidconv ! video/x-raw,format=BGRx ! videoconvert ! video/x-raw, format=BGR ! appsink drop=1", cv::CAP_GSTREAMER);
// Then check for cap.isOpened()

Searching this forum you would find many samples about how to read and display the frames with opencv.

1 Like