Problem with displaying a "broken" image on the UC-788 Rev.B (OV9281) camera on Jetson Orin NX

Good afternoon! I encountered a problem when working with the UC-788 Rev camera.B with OV9281 sensor (black and white sensor with MIPI CSI-2 interface) on Jetson Orin NX DevKit platform. I am using JetPack 6.2 (L4T 36.3.0) and the 5.136-tegra kernel.
To connect the camera, the driver provided by the link taken from the Arducam forum was used:

When connecting the camera and using the configured parameters, I get a “broken” picture (noise, stripes), despite the fact that all parameters, including the capture format, are set correctly, as I understand it. The attached image shows that instead of the expected image, only noise is output. The problem seems to be related to incorrect initialization of the sensor or incorrect loading of registers via I2C, but I did not find a clear solution.

System information:

Camera: UC-788 Rev.B
Sensor: OV9281 (black and white, MIPI CSI-2)
Platform: Jetson Orin NX DevKit
JetPack: 6.2 / L4T 36.3.0
Core: 5.136-tegra

Questions:
Can you tell me if such errors (error 22 when accessing the sensor’s I2C device) are a known problem with the OV9281 driver?
Perhaps it is necessary to make changes to the initialization settings (I2C initialization, sensor register configuration) for the OV9281 to work correctly on this platform (Jetson Orin NX, JetPack 6.2, core 5.15.136-tegra)?
Do you have any updates or patches to fix this issue?
What additional parameters or tests can be performed to diagnose and correct the situation?

I would be grateful for the prompt response and recommendations provided, as this problem significantly hinders the further development of the project.

Output of the v4l2-ctl -d /dev/video0 --all command:

Driver Info:
  Driver name      : tegra-video
  Card type        : vi-output, arducam-csi2 10-000c
  Bus info         : platform:tegra-capture-vi:1
  Driver version   : 5.15.136
  Capabilities     : 0x84200001
    Video Capture
    Streaming
    Extended Pix Format
    Device Capabilities
  Device Caps      : 0x04200001
    Video Capture
    Streaming
    Extended Pix Format
Media Driver Info:
  Driver name      : tegra-camrtc-ca
  Model            : NVIDIA Tegra Video Input Device
  Serial           : 
  Bus info         : 
  Media version    : 5.15.136
  Hardware revision: 0x00000003 (3)
  Driver version   : 5.15.136
Interface Info:
  ID               : 0x0300000b
  Type             : V4L Video
Entity Info:
  ID               : 0x00000009 (9)
  Name             : vi-output, arducam-csi2 10-000c
  Function         : V4L2 I/O
  Pad 0x0100000a   : 0: Sink
    Link 0x0200000f: from remote pad 0x1000003 of entity '13e00000.host1x:nvcsi@15a00000-' (Unknown sub-device (0002000a)): Data, Enabled
Priority: 2
Video input : 0 (Camera 1: no power)
Format Video Capture:
  Width/Height      : 640/400
  Pixel Format      : 'GREY' (8-bit Greyscale)
  Field             : None
  Bytes per Line    : 640
  Size Image        : 256000
  Colorspace        : sRGB
  Transfer Function : Default (maps to sRGB)
  YCbCr/HSV Encoding: Default (maps to ITU-R 601)
  Quantization      : Default (maps to Full Range)
  Flags             : 

User Controls:
  exposure 0x00980911 (int)    : min=1 max=65523 step=1 default=681 value=681
  horizontal_flip 0x00980914 (bool)   : default=0 value=0
  vertical_flip 0x00980915 (bool)   : default=0 value=0
  trigger_mode 0x00981901 (bool)   : default=0 value=0
  disable_frame_timeout 0x00981902 (bool)   : default=0 value=0
  frame_timeout 0x00981903 (int)    : min=100 max=12000 step=1 default=2000 value=2000
  frame_rate 0x00981906 (int)    : min=5 max=240 step=1 default=240 value=240

Camera Controls:
  error 22 getting ext_ctrl Sensor configuration
  error 22 getting ext_ctrl Sensor mode I2C packet
  error 22 getting ext_ctrl Sensor control I2C packet
  bypass_mode 0x009a2064 (intmenu): min=0 max=1 default=0 value=0
  override_enable 0x009a2065 (intmenu): min=0 max=1 default=0 value=0
  height_align 0x009a2066 (int)    : min=1 max=16 step=1 default=1 value=1
  size_align 0x009a2067 (intmenu): min=0 max=2 default=0 value=0
  write_isp_format 0x009a2068 (int)    : min=1 max=1 step=1 default=1 value=1
  sensor_signal_properties 0x009a2069 (u32)    : dims=[30][18] flags=read-only, has-payload
  sensor_image_properties 0x009a206a (u32)    : dims=[30][16] flags=read-only, has-payload
  sensor_control_properties 0x009a206b (u32)    : dims=[30][36] flags=read-only, has-payload
  sensor_dv_timings 0x009a206c (u32)    : dims=[30][16] flags=read-only, has-payload
  low_latency_mode 0x009a206d (bool)   : default=0 value=0
  preferred_stride 0x009a206e (int)    : default=0 value=0
  sensor_modes 0x009a2082 (int)    : min=0 max=30 step=1 default=30 value=1 flags=read-only

Image Source Controls:
  vertical_blanking 0x009e0901 (int)    : min=143 max=25686 step=1 default=143 value=143
  horizontal_blanking 0x009e0902 (int)    : min=96 max=96 step=1 default=96 value=96
  analogue_gain 0x009e0903 (int)    : min=100 max=1500 step=1 default=100 value=100

Image Processing Controls:
  pixel_rate 0x009f0902 (int64)  : min=96000000 max=96000000 step=1 default=96000000 value=96000000 flags=read-only
1 Like

Set the preferred_stride to 256 alignment by below command to try.

v4l2-ctl -c preferred_stride=768

Good afternoon, I’m still getting the same result:(

jetson@jetson-desktop:~$ v4l2-ctl -c preferred_stride=768
jetson@jetson-desktop:~$ v4l2-ctl --device=/dev/video0 \
     --set-fmt-video=width=640,height=400,pixelformat=GREY \
     --stream-mmap --stream-count=1 --stream-to=frame.raw
<
jetson@jetson-desktop:~$ ls -lh frame.raw
-rw-rw-r-- 1 jetson jetson 300K апр 15 17:40 frame.raw
jetson@jetson-desktop:~$ ffmpeg -f rawvideo -pixel_format gray -video_size 768x400 -i frame.raw -vf "crop=640:400:0:0" frame.png
ffmpeg version 4.4.2-1ubuntu0.1 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 11 (Ubuntu 11.4.0-1ubuntu1~22.04)
jetson@jetson-desktop:~$ v4l2-ctl -c preferred_stride=768
jetson@jetson-desktop:~$ v4l2-ctl --device=/dev/video0 --set-fmt-video=width=640,height=400,pixelformat=GREY --stream-mmap --stream-count=1 --stream-to=frame.raw
<
jetson@jetson-desktop:~$ ffmpeg -f rawvideo -pixel_format gray -video_size 640x400 -i frame.raw frame.png
ffmpeg version 4.4.2-1ubuntu0.1 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 11 (Ubuntu 11.4.0-1ubuntu1~22.04)
  configuration: --prefix=/usr --enable-nvv4l2dec --enable-libv4l2 --enable-shared --extra-libs='-L/usr/lib/aarch64-linux-gnu/tegra -lv4l2 -lnvbufsurface -lnvbufsurftransform' --extra-cflags=-I/usr/src/jetson_multimedia_api/include/ --extra-version=1ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/aarch64-linux-gnu --incdir=/usr/include/aarch64-linux-gnu --arch=arm64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
[rawvideo @ 0xaaab0bc63b50] Estimating duration from bitrate, this may be inaccurate
Input #0, rawvideo, from 'frame.raw':
  Duration: 00:00:00.04, start: 0.000000, bitrate: 61440 kb/s
  Stream #0:0: Video: rawvideo (Y800 / 0x30303859), gray, 640x400, 51200 kb/s, 25 tbr, 25 tbn, 25 tbc
File 'frame.png' already exists. Overwrite? [y/N] y
Stream mapping:
  Stream #0:0 -> #0:0 (rawvideo (native) -> png (native))
Press [q] to stop, [?] for help
Output #0, image2, to 'frame.png':
  Metadata:
    encoder         : Lavf58.76.100
  Stream #0:0: Video: png, gray(progressive), 640x400, q=2-31, 200 kb/s, 25 fps, 25 tbn
    Metadata:
      encoder         : Lavc58.134.100 png
[rawvideo @ 0xaaab0bc63b50] Packet corrupt (stream = 0, dts = 1).peed=   0x    
frame.raw: corrupt input packet in stream 0
[rawvideo @ 0xaaab0bc6ebe0] Invalid buffer size, packet size 51200 < expected frame_size 256000
Error while decoding stream #0:0: Invalid argument
frame=    1 fps=0.0 q=-0.0 Lsize=N/A time=00:00:00.04 bitrate=N/A speed=1.15x    
video:54kB audio:0kB subtitle:0kB other streams:0kB global headers:0kB muxing overhead: unknown
jetson@jetson-desktop:~$ v4l2-ctl --device=/dev/video0 \
     --set-fmt-video=width=640,height=400,pixelformat=GREY \
     --stream-mmap --stream-count=1 --stream-to=frame.raw
<
jetson@jetson-desktop:~$ ls -lh frame.raw
-rw-rw-r-- 1 jetson jetson 300K апр 15 17:40 frame.raw
jetson@jetson-desktop:~$ ffmpeg -f rawvideo -pixel_format gray -video_size 768x400 -i frame.raw -vf "crop=640:400:0:0" frame.png
ffmpeg version 4.4.2-1ubuntu0.1 Copyright (c) 2000-2021 the FFmpeg developers
  built with gcc 11 (Ubuntu 11.4.0-1ubuntu1~22.04)

Please try preferred_stride = 1280 again.

Thanks

good afternoon!

  1. Confirm the byte per line after set the preferred_stride.
v4l2-ctl -c preferred_stride=1280
v4l2-ctl --all
  1. Dump the raw by v4l2-ctl --stream-to like below command to confirm by yuv tools.
v4l2-ctl --set-fmt-video=width=640,height=400 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=1 --stream-to=t.raw -d /dev/video0 

Thanks for the feedback.

jetson@jetson-desktop:~$ v4l2-ctl -c preferred_stride=1280
jetson@jetson-desktop:~$ v4l2-ctl --all
Driver Info:
  Driver name      : tegra-video
  Card type        : vi-output, arducam-csi2 10-000c
  Bus info         : platform:tegra-capture-vi:1
  Driver version   : 5.15.136
  Capabilities     : 0x84200001
    Video Capture
    Streaming
    Extended Pix Format
    Device Capabilities
  Device Caps      : 0x04200001
    Video Capture
    Streaming
    Extended Pix Format
Media Driver Info:
  Driver name      : tegra-camrtc-ca
  Model            : NVIDIA Tegra Video Input Device
  Serial           : 
  Bus info         : 
  Media version    : 5.15.136
  Hardware revision: 0x00000003 (3)
  Driver version   : 5.15.136
Interface Info:
  ID               : 0x0300000b
  Type             : V4L Video
Entity Info:
  ID               : 0x00000009 (9)
  Name             : vi-output, arducam-csi2 10-000c
  Function         : V4L2 I/O
  Pad 0x0100000a   : 0: Sink
    Link 0x0200000f: from remote pad 0x1000003 of entity '13e00000.host1x:nvcsi@15a00000-' (Unknown sub-device (0002000a)): Data, Enabled
Priority: 2
Video input : 0 (Camera 1: no power)
Format Video Capture:
  Width/Height      : 640/400
  Pixel Format      : 'GREY' (8-bit Greyscale)
  Field             : None
  Bytes per Line    : 1280
  Size Image        : 512000
  Colorspace        : sRGB
  Transfer Function : Default (maps to sRGB)
  YCbCr/HSV Encoding: Default (maps to ITU-R 601)
  Quantization      : Default (maps to Full Range)
  Flags             : 

User Controls

                       exposure 0x00980911 (int)    : min=1 max=65523 step=1 default=681 value=681
                horizontal_flip 0x00980914 (bool)   : default=0 value=0
                  vertical_flip 0x00980915 (bool)   : default=0 value=0
                   trigger_mode 0x00981901 (bool)   : default=0 value=0
          disable_frame_timeout 0x00981902 (bool)   : default=0 value=0
                  frame_timeout 0x00981903 (int)    : min=100 max=12000 step=1 default=2000 value=2000
                     frame_rate 0x00981906 (int)    : min=5 max=240 step=1 default=240 value=240

Camera Controls

error 22 getting ext_ctrl Sensor configuration
error 22 getting ext_ctrl Sensor mode I2C packet
error 22 getting ext_ctrl Sensor control I2C packet
                    bypass_mode 0x009a2064 (intmenu): min=0 max=1 default=0 value=0 (0 0x0)
        0: 0 (0x0)
        1: 1 (0x1)
                override_enable 0x009a2065 (intmenu): min=0 max=1 default=0 value=0 (0 0x0)
        0: 0 (0x0)
        1: 1 (0x1)
                   height_align 0x009a2066 (int)    : min=1 max=16 step=1 default=1 value=1
                     size_align 0x009a2067 (intmenu): min=0 max=2 default=0 value=0 (1 0x1)
        0: 1 (0x1)
        1: 65536 (0x10000)
        2: 131072 (0x20000)
               write_isp_format 0x009a2068 (int)    : min=1 max=1 step=1 default=1 value=1
       sensor_signal_properties 0x009a2069 (u32)    : min=0 max=4294967295 step=1 default=0 dims=[30][18] flags=read-only, has-payload
        sensor_image_properties 0x009a206a (u32)    : min=0 max=4294967295 step=1 default=0 dims=[30][16] flags=read-only, has-payload
      sensor_control_properties 0x009a206b (u32)    : min=0 max=4294967295 step=1 default=0 dims=[30][36] flags=read-only, has-payload
              sensor_dv_timings 0x009a206c (u32)    : min=0 max=4294967295 step=1 default=0 dims=[30][16] flags=read-only, has-payload
               low_latency_mode 0x009a206d (bool)   : default=0 value=0
               preferred_stride 0x009a206e (int)    : min=0 max=65535 step=1 default=0 value=1280
                   sensor_modes 0x009a2082 (int)    : min=0 max=30 step=1 default=30 value=1 flags=read-only

Image Source Controls

              vertical_blanking 0x009e0901 (int)    : min=143 max=25686 step=1 default=143 value=143
            horizontal_blanking 0x009e0902 (int)    : min=96 max=96 step=1 default=96 value=96
                  analogue_gain 0x009e0903 (int)    : min=100 max=1500 step=1 default=100 value=100

Image Processing Controls
pixel_rate 0x009f0902 (int64)  : min=96000000 max=96000000 step=1 default=96000000 value=96000000 flags=read-only
jetson@jetson-desktop:~$ v4l2-ctl --set-fmt-video=width=640,height=400 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=1 --stream-to=t.raw -d /dev/video0
<
jetson@jetson-desktop:~$ ffplay -f rawvideo -pixel_format gray -video_size 640x400 frame.raw
ffplay version 4.4.2-1ubuntu0.1 Copyright (c) 2003-2021 the FFmpeg developers
  built with gcc 11 (Ubuntu 11.4.0-1ubuntu1~22.04)
  configuration: --prefix=/usr --enable-nvv4l2dec --enable-libv4l2 --enable-shared --extra-libs='-L/usr/lib/aarch64-linux-gnu/tegra -lv4l2 -lnvbufsurface -lnvbufsurftransform' --extra-cflags=-I/usr/src/jetson_multimedia_api/include/ --extra-version=1ubuntu0.1 --toolchain=hardened --libdir=/usr/lib/aarch64-linux-gnu --incdir=/usr/include/aarch64-linux-gnu --arch=arm64 --enable-gpl --disable-stripping --enable-gnutls --enable-ladspa --enable-libaom --enable-libass --enable-libbluray --enable-libbs2b --enable-libcaca --enable-libcdio --enable-libcodec2 --enable-libdav1d --enable-libflite --enable-libfontconfig --enable-libfreetype --enable-libfribidi --enable-libgme --enable-libgsm --enable-libjack --enable-libmp3lame --enable-libmysofa --enable-libopenjpeg --enable-libopenmpt --enable-libopus --enable-libpulse --enable-librabbitmq --enable-librubberband --enable-libshine --enable-libsnappy --enable-libsoxr --enable-libspeex --enable-libsrt --enable-libssh --enable-libtheora --enable-libtwolame --enable-libvidstab --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libx265 --enable-libxml2 --enable-libxvid --enable-libzimg --enable-libzmq --enable-libzvbi --enable-lv2 --enable-omx --enable-openal --enable-opencl --enable-opengl --enable-sdl2 --enable-pocketsphinx --enable-librsvg --enable-libdc1394 --enable-libdrm --enable-libiec61883 --enable-chromaprint --enable-frei0r --enable-libx264 --enable-shared
  libavutil      56. 70.100 / 56. 70.100
  libavcodec     58.134.100 / 58.134.100
  libavformat    58. 76.100 / 58. 76.100
  libavdevice    58. 13.100 / 58. 13.100
  libavfilter     7.110.100 /  7.110.100
  libswscale      5.  9.100 /  5.  9.100
  libswresample   3.  9.100 /  3.  9.100
  libpostproc    55.  9.100 / 55.  9.100
[rawvideo @ 0xffff78000c10] Estimating duration from bitrate, this may be inaccurate
Input #0, rawvideo, from 'frame.raw':
  Duration: 00:00:00.08, start: 0.000000, bitrate: 51200 kb/s
  Stream #0:0: Video: rawvideo (Y800 / 0x30303859), gray, 640x400, 51200 kb/s, 25 tbr, 25 tbn, 25 tbc
^Cjetson@jetson-desktop:~$  aq=    0KB vq=    0KB sq=    0B f=0/0

What’s the modification for GREY support in VI driver?

Hello! We are colleagues. This camera is monochrome. He compiled the driver without modifications. Do I need to recompile the driver for grayscale mode, and how to do it? We are very grateful for your help.

You must need to add “GREY” format support by VI kernel driver otherwise the default driver don’t support it.