Image capture from monochrome and color cameras

Autonomous Machines Jetson & Embedded Systems Jetson Xavier NX
1

Hi,

we want to connect one color and two monochrome cameras to a Jetson Xavier NX. My question now is regarding the access from the software side.

In other projects, we use libargus to capture images from some color cameras which is working fine. As far I understand, monochrome cameras are not supported by libargus. So what would be the best way to capture images from both types of cameras? Should I use libargus for the color and v4l for the monochrome cameras or is there any better solution?

Even if this works, there are some open questions:

  1. Synchronization
    The cameras have to be synchronized which will be done using a hardware trigger. Nevertheless I also must be able to synchronize the image streams in software. Is there any metadata field that can be used (e.g. frame count, timestamp) in both APIs?

  2. Auto controls
    Am I right that auto controls only work in libargus? Do I have to implement a separate auto exposure algorithm for the monochrome cameras or does v4l2 (or some other library) also has some auto exposure functionality?

I would appreciate if someone can give me some hints to find the best solution. Thanks!

3

Does it Bayer monochrome or YUV? If it’s Bayer you can using argus.

Dual camera frame synchronization
suggest to check Topic 1070823 for reference.

BTW, AWB shouldn’t working for the monochrome sensor. Also not verify the AE with monochrome sensor too.

4

Hi ShaneCCC,

one cameras is color (Bayer), two cameras monochrome. I can use the color camera with argus, but what is the best way to get images from the monochrome image sensors?

Here is the output from v4l2-ctl --all for the monochrome sensor:

Driver Info (not using libv4l2):
        Driver name   : tegra-video
        Card type     : vi-output, imx296 9-001a
        Bus info      : platform:15c10000.vi:0
        Driver version: 4.9.201
        Capabilities  : 0x84200001
                Video Capture
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps   : 0x04200001
                Video Capture
                Streaming
                Extended Pix Format
Priority: 2
Video input : 0 (Camera 0: ok)
Format Video Capture:
        Width/Height      : 1440/1080
        Pixel Format      : 'Y10 '
        Field             : None
        Bytes per Line    : 2880
        Size Image        : 3110400
        Colorspace        : Raw
        Transfer Function : Default (maps to None)
        YCbCr/HSV Encoding: Default (maps to ITU-R 601)
        Quantization      : Default (maps to Full Range)
        Flags             : 

Camera Controls

                     group_hold 0x009a2003 (bool)   : default=0 value=0 flags=execute-on-write
                    sensor_mode 0x009a2008 (int64)  : min=0 max=0 step=0 default=0 value=0 flags=slider
                           gain 0x009a2009 (int64)  : min=0 max=0 step=0 default=0 value=0 flags=slider
                       exposure 0x009a200a (int64)  : min=0 max=0 step=0 default=0 value=29 flags=slider
                     frame_rate 0x009a200b (int64)  : min=0 max=0 step=0 default=0 value=10 flags=slider
           sensor_configuration 0x009a2032 (u32)    : min=0 max=0 step=0 default=0 flags=read-only, volatile, has-payload
         sensor_mode_i2c_packet 0x009a2033 (u32)    : min=0 max=0 step=0 default=0 flags=read-only, volatile, has-payload
      sensor_control_i2c_packet 0x009a2034 (u32)    : min=0 max=0 step=0 default=0 flags=read-only, volatile, has-payload
                    bypass_mode 0x009a2064 (intmenu): min=0 max=1 default=0 value=0
                override_enable 0x009a2065 (intmenu): min=0 max=1 default=0 value=0
                   height_align 0x009a2066 (int)    : min=1 max=16 step=1 default=1 value=1
                     size_align 0x009a2067 (intmenu): min=0 max=2 default=0 value=0
               write_isp_format 0x009a2068 (int)    : min=1 max=1 step=1 default=1 value=1
       sensor_signal_properties 0x009a2069 (u32)    : min=0 max=0 step=0 default=0 flags=read-only, has-payload
        sensor_image_properties 0x009a206a (u32)    : min=0 max=0 step=0 default=0 flags=read-only, has-payload
      sensor_control_properties 0x009a206b (u32)    : min=0 max=0 step=0 default=0 flags=read-only, has-payload
              sensor_dv_timings 0x009a206c (u32)    : min=0 max=0 step=0 default=0 flags=read-only, has-payload
               low_latency_mode 0x009a206d (bool)   : default=0 value=0
               preferred_stride 0x009a206e (int)    : min=0 max=65535 step=1 default=0 value=0
                   sensor_modes 0x009a2082 (int)    : min=0 max=30 step=1 default=30 value=2 flags=read-only

Is there any chance to access this sensor also using argus?

5

Looks like the monochrome sensor is output YUV didn’t need ISP debayer? If so you can use v4l2src or nvv4l2camerasrc for it. And use argus for the color sensor.
If need debayer you need to modify the pixel type as any or RGB type to cheat ARGUS for it.