Monochrome camera with pixel format Y10

Robotics & Edge Computing Jetson & Embedded Systems Jetson Orin NX
1

Hi all,

Given:
L4T - 35.4.1
JetPack 5.1.2
Ubuntu 18.04 x64 LTS
Jetson Orin NX 16 GB

We have a 10 bit monochrome camera sensor. Of course under NDA. I2C & GPIO’s is managed from user space linux. Linux driver without I2C, in *dtsi file settings added:

 dynamic_pixel_bit_depth = "10";
                        csi_pixel_bit_depth = "10";
                        mode_type = "gray";
                        pixel_phase = "y10";

In JetPack_5.1.2_Linux_JETSON_ORIN_NX_TARGETS/Linux_for_Tegra/sources/kernel/nvidia/drivers/media/platform/tegra/camera/camera_common.c I added

static const struct camera_common_colorfmt camera_common_color_fmts[] = {
...
{
MEDIA_BUS_FMT_Y10_1X10,
V4L2_COLORSPACE_RAW,
V4L2_PIX_FMT_Y10,
},
{
MEDIA_BUS_FMT_Y12_1X12,
V4L2_COLORSPACE_RAW,
V4L2_PIX_FMT_Y12,
},
};

in JetPack_5.1.2_Linux_JETSON_ORIN_NX_TARGETS/Linux_for_Tegra/sources/kernel/nvidia/drivers/media/platform/tegra/camera/sensor_common.c

static int extract_pixel_format(const char *pixel_t, u32 *format)
{
…
else if (strncmp(pixel_t, "y10", size) == 0)

*format = V4L2_PIX_FMT_Y10;

else if (strncmp(pixel_t, "y12", size) == 0)

*format = V4L2_PIX_FMT_Y12;
}

in /JetPack_5.1.2_Linux_JETSON_ORIN_NX_TARGETS/Linux_for_Tegra/sources/kernel/nvidia/drivers/media/platform/tegra/camera/vi/vi2_formats.h

static const struct tegra_video_format vi2_video_formats[] = {
…
TEGRA_VIDEO_FORMAT(RAW10, 10, Y10_1X10, 2, 1, T_R16_I,RAW10, Y10, "GRAY10"),

TEGRA_VIDEO_FORMAT(RAW12, 12, Y12_1X12, 2, 1, T_R16_I,RAW12, Y12, "GRAY12"),
}

in /JetPack_5.1.2_Linux_JETSON_ORIN_NX_TARGETS/Linux_for_Tegra/sources/kernel/nvidia/drivers/media/platform/tegra/camera/vi/vi4_formats.h

static const struct tegra_video_format vi4_video_formats[] = {
…
TEGRA_VIDEO_FORMAT(RAW10, 10, Y10_1X10, 2, 1, T_R16_I,RAW10, Y10, "GRAY10"),

TEGRA_VIDEO_FORMAT(RAW12, 12, Y12_1X12, 2, 1, T_R16_I,RAW12, Y12, "GRAY12"),
}

in /JetPack_5.1.2_Linux_JETSON_ORIN_NX_TARGETS/Linux_for_Tegra/sources/kernel/nvidia/drivers/media/platform/tegra/camera/vi/vi5_formats.h

static const struct tegra_video_format vi5_video_formats[] = {
…
TEGRA_VIDEO_FORMAT(RAW10, 10, Y10_1X10, 2, 1, T_R16,RAW10, Y10, "GRAY10"),

TEGRA_VIDEO_FORMAT(RAW12, 12, Y12_1X12, 2, 1, T_R16,RAW12, Y12, "GRAY12"),
}

When running the camera driver sudo insmod nv_*.ko, in dmesg the following errors appear:

Unsupported pixel format
Failed to read mode0 image props
Could not initialize sensor properties.

What needs to be added to make the errors disappear and the pixel format Y10 appear correctly?
Maybe the pixel format should be explicitly specified in the Linux driver?

Sorry, how to move the topic to Jetson Orin NX?

Thanks!

3

hello demonoff2024,

you’ve to process with v4l2 IOCTL or v4l2src plugin directly.
however, as you can see… $ gst-inspect-1.0 v4l2src, Y10 looks to be 10-but GRAY format.
it is not supported in v4l2src.

      video/x-raw
                 format: { ... (string)GRAY8, (string)GRAY16_LE, (string)GRAY16_BE, ...

you’ll need some workaround to process Y10, please see-also Topic 255247 for reference.

1 Like
4

Hello, JerryChang

Then the question remains: how to compile the tegra_camera.ko driver (now is L4T 35.3.1)?

There are default options in defconfig:

#
# NVIDIA overlay V4L platform devices
#
CONFIG_TEGRA_MIPI_CAL=y
CONFIG_VIDEO_CAMERA=y
CONFIG_VIDEO_TEGRA_VI=y
CONFIG_VIDEO_TEGRA_VI_TPG=m
CONFIG_VIDEO_CAMERA_SKT=m
CONFIG_VIDEO_ISC=m
CONFIG_VIDEO_CDI=m
# CONFIG_VIDEO_TEGRA_VIVID is not set
# end of NVIDIA overlay V4L platform devices
...
CONFIG_TEGRA_CAMERA_PLATFORM=y

What options should be set to compile the tegra_camera.ko driver? I replaced CONFIG_VIDEO_TEGRA_VI=y with CONFIG_VIDEO_TEGRA_VI=m and errors appeared during assembly:

In file cti_tegra_defconfig I change CONFIG_VIDEO_TEGRA_VI=y to CONFIG_VIDEO_TEGRA_VI=m and I get a build error:
nvidia/nvidia_sdk/JetPack_5.1.1_Linux_JETSON_ORIN_NX_TARGETS/Linux_for_Tegra/sources/kernel/nvidia/drivers/video/tegra/host/capture/capture-support.c:179:12: ошибка: «t19_vi_thi_info» не описан здесь (не в функции); имелось в виду «t19_isp_thi_info»?
179 | .data = &t19_vi_thi_info,
| ^~~~~~~~~~~~~~~
| t19_isp_thi_info
In file included from nvidia/nvidia_sdk/JetPack_5.1.1_Linux_JETSON_ORIN_NX_TARGETS/Linux_for_Tegra/sources/kernel/nvidia/drivers/video/tegra/host/capture/capture-support.c:183:
nvidia/nvidia_sdk/JetPack_5.1.1_Linux_JETSON_ORIN_NX_TARGETS/Linux_for_Tegra/sources/kernel/nvidia/drivers/video/tegra/host/capture/capture-support-t23x.h:24:12: ошибка: «t23x_vi0_thi_info» не описан здесь (не в функции); имелось в виду «t23x_isp_thi_info»?
24 | .data = &t23x_vi0_thi_info,
| ^~~~~~~~~~~~~~~~~
| t23x_isp_thi_info
nvidia/nvidia_sdk/JetPack_5.1.1_Linux_JETSON_ORIN_NX_TARGETS/Linux_for_Tegra/sources/kernel/nvidia/drivers/video/tegra/host/capture/capture-support-t23x.h:29:12: ошибка: «t23x_vi1_thi_info» не описан здесь (не в функции); имелось в виду «t23x_isp_thi_info»?
29 | .data = &t23x_vi1_thi_info,
| ^~~~~~~~~~~~~~~~~
| t23x_isp_thi_info

How to fix it?

Thanks!

5

hello demonoff2024,

please see-also developer guide Kernel Customization to build the kernel sources.
BTW, since Orin NX uses VI-5 drivers. you don’t need to revise VI-2 and VI-4, (i.e. vi2_formats.h and vi4_formats.h)

6

Hello, JerryChang

The question is:

When building according to the item Preparing to Build External Kernel Modules, will the module tegra_camera.ko appear?

7

hello demonoff2024,

you may updating kernel image, it’s /boot/Image.