V4L Reports AR24 instead of RGB888

Hello all. We have a YUYV camera, we try to bring up. We convert it to RGB888 at a FPGA and stream CSI to jetson directly.

{ // 29.04.2019 Burak
		MEDIA_BUS_FMT_RGB888_1X24,
		V4L2_COLORSPACE_SRGB,
		V4L2_PIX_FMT_RGB24,
	},

But the camera is getting reported as;

nvidia@tegra-ubuntu:~$ v4l2-ctl --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
	Index       : 0
	Type        : Video Capture
	Pixel Format: 'AR24'
	Name        : 32-bit BGRA 8-8-8-8

and

nvidia@tegra-ubuntu:~$ v4l2-ctl --all
Driver Info (not using libv4l2):
	Driver name   : tegra-video
	Card type     : vi-output, ov5693 6-0036
	Bus info      : platform:54080000.vi:0
	Driver version: 4.4.38
	Capabilities  : 0x84200001
		Video Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps   : 0x04200001
		Video Capture
		Streaming
		Extended Pix Format
Priority: 2
Video input : 0 (Camera 0: ok)
Format Video Capture:
	Width/Height      : 1280/720
	Pixel Format      : 'AR24'
	Field             : None
	Bytes per Line    : 5120
	Size Image        : 3686400
	Colorspace        : sRGB
	Transfer Function : Default
	YCbCr Encoding    : Default
	Quantization      : Default
	Flags             : 

Camera Controls

                   frame_length (int)    : min=0 max=32767 step=1 default=1984 value=1984 flags=slider
                    coarse_time (int)    : min=2 max=32761 step=1 default=1978 value=1978 flags=slider
              coarse_time_short (int)    : min=2 max=32761 step=1 default=1978 value=1978 flags=slider
                     group_hold (intmenu): min=0 max=1 default=0 value=0
                     hdr_enable (intmenu): min=0 max=1 default=0 value=0
                       otp_data (str)    : min=0 max=1024 step=2 value='000d210000000000c0fbe8ffc0ffffff001e00fbc0ffffff0000000000000000c080400200000000fc2c7a00c0ffffff200020000100000000402afbc0ffffffb07a2afbc0ffffffbcf71b00c0ffffff001e00fbc0ffffff4000000000000000c080400200000000fc2c7a00c0fffffffc2c7a00c0ffffff000000000000000002eb010000000000001e00fbc0ffffffe07a2afbc0ffffff00a05501c0ffffff607a2afbc0ffffff24a9b600c0ffffff58303f30c0ffffff2000200001000000070000000000000000863e30c0ffffffb07a2afbc0ffffffd4f71b00c0ffffff00903e30c0ffffff4000000000000000c080400200000000fc2c7a00c0fffffffc2c7a00c0ffffff000000000000000002eb010000000000a8950c00c0ffffffe07a2afbc0ffffffa82b1c00c0ffffff00402afbc0ffffff001e00fbc0ffffff207b2afbc0ffffff0cc8b600c0ffffff307b2afbc0ffffff6ccdb600c0ffffff007b2afbc0ffffffa8950c00c0ffffff507b2afbc0ffffffc87a7b00c0ffffff010000000000000000853e30c0ffffff000000000000000000853e30c0ffffff88303f30c0ffffffd0797b00c0ffffff507b2afbc0ffffff7c7a7b00c0ffffff907b2afbc0ffffff682d7a00c0ffffff18303f30c0ffffff80c14101c0ffffff2068c4f9c0ffffff58303f30c0ffffff000000000000000005209a0000000000' flags=read-only, has-payload
                        fuse_id (str)    : min=0 max=16 step=2 value='f82a6100c0ffffff' flags=read-only, has-payload
                           gain (int)    : min=256 max=4096 step=1 default=256 value=256 flags=slider
                    bypass_mode (intmenu): min=0 max=1 default=0 value=0
                override_enable (intmenu): min=0 max=1 default=0 value=0
                   height_align (int)    : min=1 max=16 step=1 default=1 value=1
                     size_align (intmenu): min=0 max=2 default=0 value=0
               write_isp_format (bool)   : default=0 value=0
       sensor_signal_properties (u32)    : min=0 max=0 step=0 default=0 flags=read-only, has-payload
        sensor_image_properties (u32)    : min=0 max=0 step=0 default=0 flags=read-only, has-payload
      sensor_control_properties (u32)    : min=0 max=0 step=0 default=0 flags=read-only, has-payload
              sensor_dv_timings (u32)    : min=0 max=0 step=0 default=0 flags=read-only, has-payload
                   sensor_modes (int)    : min=0 max=30 step=1 default=30 value=1 flags=read-only

When we stream the camera with pipe;

gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,format=BGRA,width=1280,height=720 ! videoconvert ! omxh265enc qp-range=30,50:30,50:30,50 control-rate=1 bitrate=3000000 ! "video/x-h265, stream-format=(string)byte-stream" ! rtph265pay ! udpsink host=192.168.1.100 port=5600 sync=false async=false

and watch it with;

gst-launch-1.0 udpsrc port=5600 ! application/x-rtp, encoding-name=H265, payload=96 ! rtph265depay ! h265parse ! queue ! avdec_h265 ! fpsdisplaysink sync = false

We get 50 fps, the full fps of the camera. And the video is good.

When we set format=RGB instead of format=BGRA, the fps drops to around 37~38 fps.

I suspected it was because V4L really reports our format as BGRA, and got a frame with the v4l2-ctl command;

v4l2-ctl --set-fmt-video=width=1280,height=720 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=100 -d /dev/video0 --stream-to=ov491.raw

RAW and opened it with a program named 7YUV that can open .raw images.

The image was fine if I selected BGRA8888. But when I tried to open it as RGB888, it got distorted.

When I checked the image as HEX’s, I saw it has 4 channels instead of 3, as I was suspected, it was BGRA8888 instead of RGB888. The 4th channel was just filled with FF values.

https://ibb.co/0sgpgvV
https://ibb.co/fC2W5XH
https://ibb.co/KDn3tNL

To summarize, the FPGA is streaming RGB888 into the Jetson. v4l2-ctl reports it as AR24 and gstreamer captures it as BGRA.

Tried different colorpsaces; for example

{ // 29.04.2019 Burak
		MEDIA_BUS_FMT_RGB888_1X24,
		V4L2_COLORSPACE_ADOBERGB,
		V4L2_PIX_FMT_RGB24,
	},
{ // 29.04.2019 Burak
		MEDIA_BUS_FMT_RGB888_1X24,
		V4L2_COLORSPACE_RAW,
		V4L2_PIX_FMT_RGB24,
	},

But neither solved the problem.

hello hburaksaruhan,

  1. may I know which JetPack release you’re working with.
  2. according to Topic 1038421, we had fix for V4L format conversion issue. could you please also apply the kernel patch for verification.
    thanks

The JetPack release is 3.3
And the L4T version is 28.2

Did the patch 20181012_Topic1038421_add-format-code-conversions.tar.gz, format report didn’t change. Only thing that changed was;

nvidia@tegra-ubuntu:~$ v4l2-ctl --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
	Index       : 0
	Type        : Video Capture
	Pixel Format: 'AR24'
	Name        : 32-bit BGRA 8-8-8-8
		Size: Discrete 1280x720
			Interval: Discrete 0.018s (55.00 fps)

It added the “Size” and “Interval” parameters, which didn’t exist before this patch. but ‘AR24’ is still intact.

nvidia@tegra-ubuntu:~$ v4l2-ctl --all
Driver Info (not using libv4l2):
	Driver name   : tegra-video
	Card type     : vi-output, ov5693 6-0036
	Bus info      : platform:54080000.vi:0
	Driver version: 4.4.38
	Capabilities  : 0x84200001
		Video Capture
		Streaming
		Extended Pix Format
		Device Capabilities
	Device Caps   : 0x04200001
		Video Capture
		Streaming
		Extended Pix Format
Priority: 2
Video input : 0 (Camera 0: ok)
Format Video Capture:
	Width/Height      : 1280/720
	Pixel Format      : 'AR24'
	Field             : None
	Bytes per Line    : 5120
	Size Image        : 3686400
	Colorspace        : sRGB
	Transfer Function : Default
	YCbCr Encoding    : Default
	Quantization      : Default
	Flags             : 

Camera Controls

                   frame_length (int)    : min=0 max=32767 step=1 default=1984 value=1984 flags=slider
                    coarse_time (int)    : min=2 max=32761 step=1 default=1978 value=1978 flags=slider
              coarse_time_short (int)    : min=2 max=32761 step=1 default=1978 value=1978 flags=slider
                     group_hold (intmenu): min=0 max=1 default=0 value=0
                     hdr_enable (intmenu): min=0 max=1 default=0 value=0
                       otp_data (str)    : min=0 max=1024 step=2 value='00402afbc0ffffff001e00fbc0ffffffc080400200000000c8907b00c0ffffff407a2afbc0ffffffc8907b00c0ffffff58483e30c0ffffff01009a000000000007209a000000000000bf007dc0fffffff0792afbc0ffffffa8950c00c0ffffff207a2afbc0ffffff54a9b600c0ffffff58483e30c0ffffff00d3007dc0ffffff07209a000000000000bf007dc0ffffff307a2afbc0ffffffa8950c00c0ffffff607a2afbc0ffffff54a9b600c0ffffff58483e30c0ffffff0100000000000000070000000000000000bf007dc0ffffff807a2afbc0ffffffc88c7b00c0ffffff01000000000000000000000000000000407b2afbc0ffffffe08e7b00c0ffffff00c14101c0ffffff80c14101c0ffffffb07a2afbc0ffffffa8950c00c0ffffffe07a2afbc0ffffffa82b1c00c0ffffff00402afbc0ffffff001e00fbc0ffffff207b2afbc0ffffff3cc8b600c0ffffff307b2afbc0ffffff9ccdb600c0ffffff007b2afbc0ffffffa8950c00c0ffffff507b2afbc0ffffffc87a7b00c0ffffff010000000000000000be007dc0ffffff000000000000000000be007dc0ffffff88483e30c0ffffffd0797b00c0ffffff507b2afbc0ffffff7c7a7b00c0ffffff907b2afbc0ffffff682d7a00c0ffffff18483e30c0ffffff80c14101c0ffffff20f0c4f9c0ffffff58483e30c0ffffff000000000000000005209a0000000000' flags=read-only, has-payload
                        fuse_id (str)    : min=0 max=16 step=2 value='f82a6100c0ffffff' flags=read-only, has-payload
                           gain (int)    : min=256 max=4096 step=1 default=256 value=256 flags=slider
                    bypass_mode (intmenu): min=0 max=1 default=0 value=0
                override_enable (intmenu): min=0 max=1 default=0 value=0
                   height_align (int)    : min=1 max=16 step=1 default=1 value=1
                     size_align (intmenu): min=0 max=2 default=0 value=0
               write_isp_format (bool)   : default=0 value=0
       sensor_signal_properties (u32)    : min=0 max=0 step=0 default=0 flags=read-only, has-payload
        sensor_image_properties (u32)    : min=0 max=0 step=0 default=0 flags=read-only, has-payload
      sensor_control_properties (u32)    : min=0 max=0 step=0 default=0 flags=read-only, has-payload
              sensor_dv_timings (u32)    : min=0 max=0 step=0 default=0 flags=read-only, has-payload
                   sensor_modes (int)    : min=0 max=30 step=1 default=30 value=1 flags=read-only

hello hburaksaruhan,

kernel driver reads the pixel phase through device tree settings, and there are functions to determine the pixel formats. could you please add some debug prints in below two kernel sources.
thanks

$TOP/kernel_src/kernel/kernel-4.4/drivers/media/platform/tegra/camera/sensor_common.c
$TOP/kernel_src/kernel/kernel-4.4/drivers/media/platform/tegra/camera/camera_common.c