How to handle raw from USB cameras

Autonomous Machines Jetson & Embedded Systems Jetson TX1
1

What is the current state of how to handle raw unprocessed images from a USB camera? I’m using a Leopard Imaging MT9M021C USB 3 camera. I can get an image using v4l2, but it just shows the bayer data. I’ve been reading on these forums as of about 6 months ago the nvcamerasrc processes the bayer data and outputs RGB, but it only works with the onboard camera - not USB cameras. Is this still the case?

I’ve also tried the gstreamer bayer2rgb element, but I get some assertion errors.

ubuntu@tegra-ubuntu:~$ gst-launch-1.0 v4l2src device=/dev/video1 ! bayer2rgb ! xvimagesink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
Setting pipeline to PLAYING ...
New clock: GstSystemClock

(gst-launch-1.0:15727): GStreamer-CRITICAL **: gst_caps_get_structure: assertion 'index < GST_CAPS_LEN (caps)' failed

(gst-launch-1.0:15727): GStreamer-CRITICAL **: gst_structure_get_value: assertion 'structure != NULL' failed

(gst-launch-1.0:15727): GStreamer-CRITICAL **: gst_structure_set_value: assertion 'G_IS_VALUE (value)' failed

(gst-launch-1.0:15727): GStreamer-CRITICAL **: gst_structure_get_value: assertion 'structure != NULL' failed

(gst-launch-1.0:15727): GStreamer-CRITICAL **: gst_structure_set_value: assertion 'G_IS_VALUE (value)' failed

(gst-launch-1.0:15727): GStreamer-CRITICAL **: gst_structure_get_value: assertion 'structure != NULL' failed

(gst-launch-1.0:15727): GStreamer-CRITICAL **: gst_structure_set_value: assertion 'G_IS_VALUE (value)' failed
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data flow error.
Additional debug info:
gstbasesrc.c(2948): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
streaming task paused, reason not-negotiated (-4)
Execution ended after 0:00:00.037026844
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

ubuntu@tegra-ubuntu:~$ v4l2-ctl --list-formats-ext -d /dev/video1
ioctl: VIDIOC_ENUM_FMT
        Index       : 0
        Type        : Video Capture
        Pixel Format: 'YUYV'
        Name        : YUV 4:2:2 (YUYV)
                Size: Discrete 1280x720
                        Interval: Discrete 0.017s (60.000 fps)
                Size: Discrete 800x460
                        Interval: Discrete 0.011s (90.000 fps)
                Size: Discrete 640x480
                        Interval: Discrete 0.017s (60.000 fps)
                Size: Discrete 1280x960
                        Interval: Discrete 0.017s (60.000 fps)

Here’s what it looks like when I just display the raw image without the bayer2rgb element.

External Image

2

The log from the command “v4l2-ctl --list-formats-ext -d /dev/video1” shows it output YUV not Bayer RAW data. Did you pass whole log here? For the YUV you can just show it with display sink don’t need the bayer2rgb element.

3

Yeah I think you’ve hit on the problem here. The Leopard Imaging camera incorrectly reports its pixel format as YUV but it is, in fact, Bayer Raw. I have a few other Leopard Imaging cameras that output bayer raw and also report it as YUV, but I don’t have another correct example of a camera correctly reporting its pixel format as bayer raw.

I’ll follow up with leopard imaging about this.