How to handle raw from USB cameras

What is the current state of how to handle raw unprocessed images from a USB camera? I’m using a Leopard Imaging MT9M021C USB 3 camera. I can get an image using v4l2, but it just shows the bayer data. I’ve been reading on these forums as of about 6 months ago the nvcamerasrc processes the bayer data and outputs RGB, but it only works with the onboard camera - not USB cameras. Is this still the case?

I’ve also tried the gstreamer bayer2rgb element, but I get some assertion errors.

ubuntu@tegra-ubuntu:~$ gst-launch-1.0 v4l2src device=/dev/video1 ! bayer2rgb ! xvimagesink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock

(gst-launch-1.0:15727): GStreamer-CRITICAL **: gst_caps_get_structure: assertion 'index < GST_CAPS_LEN (caps)' failed

(gst-launch-1.0:15727): GStreamer-CRITICAL **: gst_structure_get_value: assertion 'structure != NULL' failed

(gst-launch-1.0:15727): GStreamer-CRITICAL **: gst_structure_set_value: assertion 'G_IS_VALUE (value)' failed

(gst-launch-1.0:15727): GStreamer-CRITICAL **: gst_structure_get_value: assertion 'structure != NULL' failed

(gst-launch-1.0:15727): GStreamer-CRITICAL **: gst_structure_set_value: assertion 'G_IS_VALUE (value)' failed

(gst-launch-1.0:15727): GStreamer-CRITICAL **: gst_structure_get_value: assertion 'structure != NULL' failed

(gst-launch-1.0:15727): GStreamer-CRITICAL **: gst_structure_set_value: assertion 'G_IS_VALUE (value)' failed
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data flow error.
Additional debug info:
gstbasesrc.c(2948): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming task paused, reason not-negotiated (-4)
Execution ended after 0:00:00.037026844
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...
ubuntu@tegra-ubuntu:~$ v4l2-ctl --list-formats-ext -d /dev/video1
ioctl: VIDIOC_ENUM_FMT
        Index       : 0
        Type        : Video Capture
        Pixel Format: 'YUYV'
        Name        : YUV 4:2:2 (YUYV)
                Size: Discrete 1280x720
                        Interval: Discrete 0.017s (60.000 fps)
                Size: Discrete 800x460
                        Interval: Discrete 0.011s (90.000 fps)
                Size: Discrete 640x480
                        Interval: Discrete 0.017s (60.000 fps)
                Size: Discrete 1280x960
                        Interval: Discrete 0.017s (60.000 fps)

Here’s what it looks like when I just display the raw image without the bayer2rgb element.

The log from the command “v4l2-ctl --list-formats-ext -d /dev/video1” shows it output YUV not Bayer RAW data. Did you pass whole log here? For the YUV you can just show it with display sink don’t need the bayer2rgb element.

Yeah I think you’ve hit on the problem here. The Leopard Imaging camera incorrectly reports its pixel format as YUV but it is, in fact, Bayer Raw. I have a few other Leopard Imaging cameras that output bayer raw and also report it as YUV, but I don’t have another correct example of a camera correctly reporting its pixel format as bayer raw.

I’ll follow up with leopard imaging about this.