Built-in imx219 sensor driver

chiaueng · December 29, 2024, 4:16am

[Version] r36.4
[Environment] Jetson Orin 64GB Developer Kit (IMX219 sensor is during purchasing process, not arrived yet)

Steps:

Following the instructions from this link: Kernel Customization — NVIDIA Jetson Linux Developer Guide 1 documentation

On Host PC:

cd <install-path>/Linux_for_Tegra/source
export CROSS_COMPILE=<toolchain-path>/bin/aarch64-buildroot-linux-gnu-
Edit kernel/kernel-jammy-src/arch/arm64/configs/defconfig using vim: CONFIG_MEDIA_SUPPORT=y CONFIG_VIDEO_IMX219=y CONFIG_VIDEO_V4L2=y
Build the kernel: make -C kernel

After successful build, I can see the object file: kernel/kernel-jammy-src/drivers/media/i2c/imx219.o

Set the installation path for modules: export INSTALL_MOD_PATH= <install-path>/Linux_for_Tegra/rootfs/
Install the kernel modules: sudo -E make install -C kernel
Copy the kernel image: cp kernel/kernel-jammy-src/arch/arm64/boot/Image <install-path>/Linux_for_Tegra/kernel/Image
Update the kernel using the SDK manager

On Jetson Platform:

After flashing and rebooting the system, I can see the sys path exists: /sys/bus/i2c/drivers/imx219
Open a terminal and run sudo dmesg. I see the kernel updated information: "SMP PREEMPT Sat Dec 28 23:09:28 CST 2024" (This timestamp matches the build time of the Image)
However, the keyword “imx219” is not present in the dmesg output.

Question:

Even though the IMX219 sensor is not physically connected, I believe it should be probed during system boot. Did I miss any steps?

carolyuu · December 29, 2024, 4:30am

Hi,
Here are some suggestions for the common issues:

1. Performance

Please run the below command before benchmarking deep learning use case:

$ sudo nvpmodel -m 0
$ sudo jetson_clocks

2. Installation

Installation guide of deep learning frameworks on Jetson:

TensorFlow: Installing TensorFlow for Jetson Platform - NVIDIA Docs
PyTorch: Installing PyTorch for Jetson Platform - NVIDIA Docs
We also have containers that have frameworks preinstalled:
Data Science, Machine Learning, AI, HPC Containers | NVIDIA NGC

3. Tutorial

Startup deep learning tutorial:

Jetson-inference: Hello AI World guide to deploying deep-learning inference networks and deep vision primitives with TensorRT and NVIDIA Jetson
TensorRT sample: Jetson/L4T/TRT Customized Example - eLinux.org

4. Report issue

If these suggestions don’t help and you want to report an issue to us, please attach the model, command/step, and the customized app (if any) with us to reproduce locally.

Thanks!

chiaueng · December 29, 2024, 2:08pm

Hi @carolyuu ,
I don’t quite understand what you mean here. My purpose is to bring up the imx219 sensor driver, after that , i will use V4L2 to stream the video. I won’t use the deep learning functions for my use case. Why do i need to install TensorFlow or PyTorch?

BTW, when typing cat /lib/modules/5.15.148-tegra/modules.builtin, I can see kernel/drivers/media/i2c/imx219.ko, but I still can’t see any probing message in dmesg. (Please note I did not plug in the imx219 sensor. I am not sure if it affects or not.)

Thanks in advance.

JerryChang · December 30, 2024, 6:03am

hello chiaueng,

you should also have device tree overlay applied.
please see-also r36.4 Release Notes for [4.2. Device Registration] section.

chiaueng · January 1, 2025, 6:40am

Hi Jerry,
Because I am very new to this topic, I tried to check the device-tree overlay steps :.

Add your .dtsi file to the camera conguration .dtsi file.
tegra234-p3737-camera-modules.dtsi is for Jetson AGX-orin.
→ #include "tegra234-p3737-0000-camera-imx274-dual.dtsi" is alreay in tegra234-p3737-camera-modules.dtsi
Set the status of your device tree nodes to disabled.
imx274 is alreay disabled
→

imx274_a@1a{ 
     status = "disabled";
};

Add the overlay information as fragments below to a new .dts file"
→ tegra234-p3737-camera-dual-imx274-lverlay.dts has the overlay information.
Update the .dts file with proper overlay information and a compatible string
–>In tegra234-p3737-camera-dual-imx274-overlay.dts, I saw the following description which set the overlays status=okay

fragment@0 {
    target-path = "/bus@0/i2c@3180000/tca9546@70/i2c@0/imx274_a@1a";
    board_config {
        ids = "LPRD-dual-imx274-002";
        sw-modules = "kernel";
    };
    __overlay__ {
        status = "okay";
    };
};

Compile the .dts le to generate a .dtbo
→ Follow the steps in the ofllowing link to generate tegra234-p3737-camera-dual-imx274-overlay.dtbo Kernel Customization — NVIDIA Jetson Linux Developer Guide 1 documentation
Before you flash, move the .dtbo to flash_folder/kernel/dtb/.
→ cp kernel-devicetree/generic-dts/dtbs/* <install-path>/Linux_for_Tegra/kernel/dtb/
Add the following lines to the .conf file.
→ Because my platform is Jetson AGX Orin 64GB module, I check jetson-agx-orin-devkit.conf. OVERLAY_DTB_FILE already includes tegra234-p3737-camera-dual-imx274-lverlay.dtbo
Flash the target board by SDK manager

However, after flashing & reboot the target board and type “dmesg”, I cannot see any imx274 probe information. Could you give me some guidance for this problem?

Thank you inadvance

JerryChang · January 2, 2025, 2:05am

hello chiaueng,

you don’t need to modify the code,
there’s already /boot/tegra234-p3737-camera-dual-imx274-overlay.dtbo if you flashed Jetpack-6.1 public release.
please refer to Configuring the CSI Connector to run Jetson-IO utility to apply device tree overlay.

FYI, Raspberry Pi Camera Module 2, IMX219, it’s supported on Orin NX/Nano by default.

chiaueng · January 3, 2025, 8:24am

Hi @JerryChang ,
After selecting the compatible hardware as imx274 in Jetson-IO. It can work now. I can also modify the overlay file as imx219 in /boot/extlinux/extlinux.conf , then in dmesg I can see imx219 probe message.

My additional question is : How can i add imx219 in the following menu :

i checked the link below, but still have no idea how to do it. Sensor Software Driver Programming — NVIDIA Jetson Linux Developer Guide 1 documentation .
If this should be a new thread, i can create a new topic.

Thank you in advance.

JerryChang · January 3, 2025, 8:35am

hello chiaueng,

as you can see… it’s a specific dtbo file, /boot/tegra234-p3737-camera-dual-imx274-overlay.dtbo.
please also dig into kernel release sources, you may develop an overlay file by yourself.
for instance, $public_sources/hardware/nvidia/t23x/nv-public/overlay/

chiaueng · January 4, 2025, 4:00pm

Hi @JerryChang ,
Thanks for your prompt reply. I checked this Link again Sensor Software Driver Programming — NVIDIA Jetson Linux Developer Guide 1 documentation

Locate and edit the .dtsi file:
<top>/hardware/nvidia/t23x/nv-public/overlay/tegra234-p3737-0000+p3701-0000-dynamic.dts
Replace the following line with an #include statement specifying the .dtsi file for your new device.
(1) remove #include "tegra234-p3737-camera-modules.dtsi"
(2) add #include tegra234-p3767-camera-p3768-imx219-dual.dtboand save the file
make dtbs
cp kernel-devicetree/generic-dts/dtbs/* /Linux_for_Tegra/kernel/dtb/
flash device and reboot
After the above steps, I still see the menu of Jetson-IO as below and no IMX219 item is added.

GetImage479×415 21.2 KB

It seems i did not take the correct actions. May I have your help to give me your suggestions?

Thank you.

JerryChang · January 6, 2025, 1:59am

hello chiaueng,

it seems you’ve misunderstood.
please dig into device tree overlay file,
for instance,
here is the sources of Jetson Camera Dual-IMX274.
$public_sources/kernel_src/hardware/nvidia/t23x/nv-public/overlay/tegra234-p3737-camera-dual-imx274-overlay.dts

        overlay-name = "Jetson Camera Dual-IMX274";
        jetson-header-name = "Jetson AGX CSI Connector";
        compatible = JETSON_COMPATIBLE;

since you’re working with AGX Orin, which doesn’t support IMX219 by default.
even though there’s IMX219 device tree overlay file, you still need some modification too.
for instance,
here’s sources of IMX219 device tree overlay
$public_sources/kernel_src/hardware/nvidia/t23x/nv-public/overlay/tegra234-p3767-camera-p3768-imx219-A.dts

        overlay-name = "Camera IMX219-A";
        jetson-header-name = "Jetson 24pin CSI Connector";
        compatible = JETSON_COMPATIBLE_P3768;

as you can see… they’re using different compatible property, that’s simply adding the dtbo file not works.

JerryChang · January 6, 2025, 2:02am

hello chiaueng,

may I also know how you connect a IMX219 to AGX Orin?
please see-also Camera Design Guide, AGX Orin and Orin Nano/NX they’re having different camera connectors.
if you’re developing your own camera board, please consider it’s another sensor bringup works.

chiaueng · January 6, 2025, 4:05pm

Hi @JerryChang ,
I checked it again, and think for tegra234-p3767-camera-p3768-imx219-A.dts, I need to

replace include <dt-bindings/tegra234-p3767-0000-common.h>
to
include <dt-bindings/tegra234-p3737-0000+p3701-0000.h>
Change jetson-header-name to “Jetson AGX CSI Connector”
Change compatible to JETSON_COMPATIBLE.
But besides this, i am not sure what else need to modfiy.
I am very confused now & have some questions :

After modifying it, if I follow the steps in the sub-section device-tree-overlay, will the imx219 appear in jetson-io menu ?
Where to configure the IO pins for SCL and SDA ? I can’t find it either in imx219.c or dts files.
We will connect imx219 to the CSI interface. Is it the OK to modify tegra234-p3767-camera-p3768-imx219-A.dts ? or do you suggest which dts file to refer ?

Thank you.

JerryChang · January 7, 2025, 2:37am

hello chiaueng,

Jetson-IO menu looking for below,
you’ll see a new option available once you’ve add device tree overlay to your target correctly,
for instance,

        overlay-name = "NEW CAMERA OVERLAY";
        jetson-header-name = "Jetson AGX CSI Connector";
        compatible = JETSON_COMPATIBLE;

>> I can’t find it either in imx219.c or dts files.

here’s path of IMX219 sensor driver and, device tree.
$public_sources/r36.4.0/Linux_for_Tegra/source/kernel_src/nvidia-oot/drivers/media/i2c/nv_imx219.c
$public_sources/r36.4.0/Linux_for_Tegra/source/kernel_src/hardware/nvidia/t23x/nv-public/overlay/tegra234-camera-rbpcv2-imx219.dtsi

>> Is it the OK to modify tegra234-p3767-camera-p3768-imx219-A.dts ? or do you suggest which dts file to refer ?

as you can see… it’s sensor device tree with such hardware, Raspberry Pi Camera, IMX219.
please develop another new device tree since you’re going to connect IMX219 to AGX Orin’s CSI interface,
you may dig into that’s cameras from AGX Orin menu of Jetson-IO for reference,
for instance,
$public_sources/r36.4.0/Linux_for_Tegra/source/kernel_src/hardware/nvidia/t23x/nv-public/overlay/tegra234-p3737-camera-dual-imx274-overlay.dts

chiaueng · January 7, 2025, 3:50am

Hi @JerryChang ,
I will dig into imx274 dts first. Thank you.

For the following information, may I know where it mentions the IO pins for SDA/SCL ? From Camera Design Guide, i see there are i2c3, i2c2, i2c4 i2c5, but i can’t see the correponding description both in driver and dts file

*$public_sources/r36.4.0/Linux_for_Tegra/source/kernel_src/nvidia-oot/drivers/media/i2c/nv_imx219.c*
*$public_sources/r36.4.0/Linux_for_Tegra/source/kernel_src/hardware/nvidia/t23x/nv-public/overlay/tegra234-camera-rbpcv2-imx219.dtsi*

I see there is nv_imx219.c and imx219.c, which driver can I use ? I asked this question because when I insmod or modprobe nv_imx219 or nv_imx274,there is nothing appears in dmesg.

JerryChang · January 7, 2025, 5:51am

hello chiaueng,

as you can see from [Figure 2-3. Single Camera Case Block Diagram],
it uses CAM_I2C, which is I2C3, you may also refer to below…
$public_sources/r36.4.0/Linux_for_Tegra/source/kernel_src/hardware/nvidia/t23x/nv-public/tegra234.dtsi
for instance,

                cam_i2c: i2c@3180000 {
                        compatible = "nvidia,tegra194-i2c";
                        clocks = <&bpmp TEGRA234_CLK_I2C3>,
                                 <&bpmp TEGRA234_CLK_PLLP_OUT0>;

for the multi-camera use-case, it’ll use the GPIO-based i2c bus multiplexer to route the i2c signals.
you may see-also Topic 192486, comment #17 for reference.

did you download the latest r36.4 public release sources package?
it should changes to nv_imx219.c

you’ll also need to apply VI/CSI port bindings, which is done by device tree overlay.

chiaueng · January 8, 2025, 2:42am

Hi @JerryChang ,
(1) I still can’t get the same result as you described here :

To narrow down the porblem, I test with the following steps.
In Host Ubuntu

copy all the imx185 tdsi to imx219 dtsi, rename the imx185 keywords to imx219
tegra234-camera-imx219-a00.dtsi.txt (14.6 KB)
tegra234-p3737-0000-camera-imx219-a00.dtsi.txt (1.4 KB)
tegra234-p3737-camera-imx219-overlay.dts.txt (5.2 KB)
In tegra234-p3737-camera-modules.dtsi, add :

#include "tegra234-p3737-0000-camera-imx219-a00.dtsi" 
.... 
imx219_a@1a {
    status = "disabled";
};

tegra234-p3737-camera-modules.dtsi.txt (8.5 KB)

In ~/nvidia/nvidia_sdk/JetPack_6.1_Linu_JETSON_AGX_ORIN_TARGETS/Linux_for_Tegra/source/hardware/nvidia/t23x/nv-public/overlay/Makefile
add:
dtbo-y += tegra234-p3737-camera-imx219-overlay.dtbo
Makefile.txt (3.6 KB)
Build dtbs

$ export CROSS_COMPILE=~/l4t-gcc/aarch64--glibc--stable--2022.08-1/bin/aarch64-buildroot-linux-gnu-
$ export KERNEL_HEADERS=~/nvidia/nvidia_sdk/JetPack_6.1_Linux_JETSON_AGX_ORIN_TARGETS/Linux_for_Tegra/source/kernel/kernel-jammy-src
$ make dtbs
$ cp kernel-devicetree/generic-dts/dtbs/* ~/nvidia/nvidia_sdk/etPack_6.1_Linux_JETSON_AGX_ORIN_TARGETS/Linux_for_Tegra/kernel/dtb/

In Linux_for_Tegra/Jetson-agx-orin-devkit.conf, add tegra234-p3737-camera-imx219-overlay.dtbo
jetson-agx-orin-devkit.conf.txt (3.1 KB)
flash image either by sdk manger or flash command:

  sudo ./flash.sh jetson-agx-orin-devlit.conf mmcblk0p1

In Jetson Orin:
boot system, check device tree, I can see imx219 device tree as expected. However, when type the command

sudo /opt/nvidia/jetson-io/jetson-io.py
--> Configure Jetson AGX CSI Connector
--> Configure for compatible hardware

There is still no IMX219 menu

(2) For loadable kernel module
I checked the device tree of imx185, the VI/CSI port bindgs are already described in the overlay dtsi.
tegra234-p3737-camera-imx185-overlay.dts.txt (5.2 KB)
So I select imx185 in jetson-io.py, and the extlinux.conf is updated after reboot:

after system reboot, input the modprobe command

sudo mpdprobe nv_imx185

nv_imx185 module can be seen when lsmod, but dmesg print nothing
dmesg_no_imx185.txt (65.1 KB)

Could you give me the debug suggestions ? I checked the official doc again but still don’t know what steps are missing.

JerryChang · January 8, 2025, 6:17am

hello chiaueng,

you may putting the dtbo to your target instead of re-flashing.

here’s demonstration.
you may download this attachment, tegra234-p3737-camera-test-overlay.7z (1.9 KB), which has included sample overlay sources and dtbo file.
and, please copy dtbo file via ssh to… /boot/tegra234-p3737-camera-test-overlay.dtbo
by launching Jetson-IO to configure Jetson AGX CSI connector, you’ll see a new option available.
for instance,

 =================== Jetson Expansion Header Tool ===================
 |                                                                    |
 |                                                                    |
 |                Select one of the following options:                |
 |                                                                    |
...
 |                      Jetson Camera TEST TEST                       |

chiaueng · January 8, 2025, 10:34am

Hi @JerryChang
With your example, I can see it on jetson-io menu now. Let me try to modify it with my own setting. Thank you.

JerryChang · January 9, 2025, 2:08am

hello chiaueng,

FYI, it’s overlay file duplicated from tegra234-p3737-camera-dual-imx274-overlay.dts
you may based-on that to develop your new sensor driver, let’s have another new forum thread if you need further supports.

Topic		Replies	Views
Adding a new camera sensor driver without flashing Jetson AGX Orin camera	19	447	August 13, 2024
Enable camera sensor connected to i2c mux Jetson AGX Orin camera	26	629	July 30, 2024
Custom camera driver for TX2 NX Jetson TX2 camera	25	2394	July 16, 2024
Add new sensor driver with Orin NX (t234) Jetson Orin NX camera	29	674	June 14, 2024
On Orin NX R36.3 I can't configure imx477 image capture from two virtual channels FPD Link ds90ub954 - ds90ub953 Jetson Orin NX camera	23	187	September 23, 2024
Jetson Orin Nano 在内核增加IMX586摄像头驱动后，无法检测到该摄像头 Jetson Orin Nano camera , chinese	19	1116	December 6, 2023
Orin can not access /dev/video0 Jetson AGX Orin camera	41	4035	August 16, 2022
I cannot see the video from the imx219 camera on the Jetson Orin Nano + custom carrier board Jetson Orin Nano camera , board-design	24	1226	October 17, 2023
imx219 in TX1,Jetpack 4.2.2 Jetson TX1	4	1107	October 18, 2021
Orin nx dp enable fail Jetson Orin NX dp-display	45	2393	April 9, 2023

Built-in imx219 sensor driver

1. Performance

2. Installation

3. Tutorial

4. Report issue

Related topics