Enable camera sensor connected to i2c mux

Good morning,
I’m trying to port my system based on 4 camera sensors connected to a Orin AGX through a camera interposer module based on I2C mux.

With Jetpack 5.1.3 everything worked perfectly. Now I’m trying to do the same with Jetpack 6.0

First I downloaded all the source package and i rebuilt kernel images, modules and device tree files. following the guide
https://docs.nvidia.com/jetson/archives/r36.3/DeveloperGuide/SD/Kernel/KernelCustomization.html

I verified that the system works with updated files.

Even if my camera is not based on IMX185 sensor, it has the same i2c default address. Also the i2c mux has the same i2c default address of the sample configuration.

The only thing I need to change is to replace the camera i2c channel from 0 to 4. So i modified the tegra234-p3737-camera-modules.dtsi and rebuild everything. After update all the system files, following the indication here
https://docs.nvidia.com/jetson/archives/r36.3/DeveloperGuide/SD/CameraDevelopment/SensorSoftwareDriverProgramming.html#loadable-kernel-module-lkm

i tried to insert IMX185 driver with modprobe

Nevertheless I cannot see any effect. From dmesg I cannot see the mux driver output during probe. The i2x mux driver seems not loaded and i cannot see the sensor i2c device

UPDATE:
I also tried the approach based on JetsonIO Tools. I added the imx185 overlay, and a new boot configuration that load tegra234-p3737-camera-imx185-overlay.dtbo has been created. After reboot i choose this new boot configuration, but I cannot see any differences

Is it posssible to have a clear documentation of the steps required to enable a sensor on Jetpack 6.0?

Best regards

Does the system load the sensor driver?
Dump the device tree to confirm.

sudo dtc -I fs -O dts -o extracted_proc.dts /proc/device-tree

Hi @ShaneCCC ,
I repeat the whole process to be sure about the steps:

  • compile kernel, modules, dtb for 36.3 and update the board with new generated files
  • enable IMX185 overlay with JetsonIO Tools
  • reboot the system using the new boot option with IMX185 overlay
  • I run the command you suggested. Attach you can find the relative files “extracted_proc.dts.before”
  • modify the dts files in order to change the i2c mux port connected to sensor
  • attach you can find the patch file with modification
  • rebuild only dtb
  • update the following files to the board
    tegra234-p3737-0000+p3701-0000-dynamic.dtbo
    tegra234-p3737-0000+p3701-0000-nv.dtb
    tegra234-p3737-camera-imx185-overlay.dtbo
  • reboot the system a run again your command. Attach you can find the relative files “extracted_proc.dts.after”

I tried to compare extracted_proc.dts.before and extracted_proc.dts.after but the files are equal.

UPDATE:
I laso tried to convert the generated tegra234-p3737-camera-imx185-overlay.dtbo to dts file and the converted source file is correct, so with i2c@4 insted of i2c@0. It seems that after reboot the new tegra234-p3737-camera-imx185-overlay.dtbo overlay file is not used

Another issue is that the i2c mux pca954x driver is not loaded. I tried to add some kernel debug message in driver source but from dmesg i cannot see anything

What I’m doing wrong?

Regards

change-i2c-mux-port.zip (106.4 KB)

Do you add the FDT in the /boot/extlinux/extlinux.conf to assign your dtb?

Hi @ShaneCCC ,
yes the extlinux.conf is automatically modified after using the JetsonIO tools utility and enabling the overlay from there

orin@ubuntu:~$ cat /boot/extlinux/extlinux.conf
TIMEOUT 30
DEFAULT JetsonIO

MENU TITLE L4T boot options

LABEL primary
MENU LABEL primary kernel
LINUX /boot/Image
INITRD /boot/initrd
APPEND ${cbootargs} root=PARTUUID=f6f7f0a8-5dbf-45ef-99e8-9f5ff9618f51 rw rootwait rootfstype=ext4 mminit_loglevel=4 console=ttyTCU0,115200 console=ttyAMA0,115200 firmware_class.path=/etc/firmware fbcon=map:0 net.ifnames=0 nospectre_bhb video=efifb:off console=tty0 nv-auto-config

LABEL backup
MENU LABEL backup kernel
LINUX /boot/Image.backup
INITRD /boot/initrd
APPEND ${cbootargs} root=PARTUUID=f6f7f0a8-5dbf-45ef-99e8-9f5ff9618f51 rw rootwait rootfstype=ext4 mminit_loglevel=4 console=ttyTCU0,115200 console=ttyAMAA0,115200 firmware_class.path=/etc/firmware fbcon=map:0 net.ifnames=0 nospectre_bbhb video=efifb:off console=tty0 nv-auto-config

LABEL JetsonIO
MENU LABEL Custom Header Config:
LINUX /boot/Image
FDT /boot/dtb/kernel_tegra234-p3737-0000+p3701-0000-nv.dtb
INITRD /boot/initrd
APPEND ${cbootargs} root=PARTUUID=f6f7f0a8-5dbf-45ef-99e8-9f5ff9618f51 rw rootwait rootfstype=ext4 mminit_loglevel=4 console=ttyTCU0,115200 console=ttyAMA0,115200 firmware_class.path=/etc/firmware fbcon=map:0 net.ifnames=0 nospectre_bhb video=efifb:off console=tty0 nv-auto-config
OVERLAYS /boot/tegra234-p3737-camera-imx185-overlay.dtbo

Looks like you didn’t override the kernel_tegra234-p3737-0000+p3701-0000-nv.dtb by tegra234-p3737-0000+p3701-0000-nv.dtb that build by your host.

I convert the 2 files to dts and then I compared them, are exactly the same.

kernel_tegra234-p3737-0000+p3701-0000-nv.dtb file does not contain any information regards the cameras sensor. This part is included in tegra234-p3737-0000+p3701-0000-dynamic.dtb file

Other question that is not clear, do I need to modprobe the sensor kernel module after boot or is automatically done after parsing the imx185 overlay dtb?

I have no idea of what is happening, the dtb files on board are correctly mdoified since i tried to decode them. For example, the tegra234-p3737-0000+p3701-0000-dynamic contains only tca9548@70 on i2c@3180000 bus.

But after boot I have
ls -l /proc/device-tree/bus@0/i2c@3180000/
total 0
-r–r–r-- 1 root root 4 May 29 08:41 ‘#address-cells
-r–r–r-- 1 root root 8 May 29 08:41 assigned-clock-parents
-r–r–r-- 1 root root 8 May 29 08:41 assigned-clocks
-r–r–r-- 1 root root 4 May 29 08:41 clock-frequency
-r–r–r-- 1 root root 15 May 29 08:41 clock-names
-r–r–r-- 1 root root 16 May 29 08:41 clocks
-r–r–r-- 1 root root 20 May 29 08:41 compatible
-r–r–r-- 1 root root 0 May 29 08:41 dma-coherent
-r–r–r-- 1 root root 6 May 29 08:41 dma-names
-r–r–r-- 1 root root 16 May 29 08:41 dmas
-r–r–r-- 1 root root 12 May 29 08:41 interrupts
-r–r–r-- 1 root root 8 May 29 08:41 iommus
-r–r–r-- 1 root root 4 May 29 08:41 name
-r–r–r-- 1 root root 4 May 29 08:41 phandle
drwxr-xr-x 7 root root 0 May 29 08:41 prod-settings
-r–r–r-- 1 root root 16 May 29 08:41 reg
-r–r–r-- 1 root root 4 May 29 08:41 reset-names
-r–r–r-- 1 root root 8 May 29 08:41 resets
-r–r–r-- 1 root root 4 May 29 08:41 ‘#size-cells
-r–r–r-- 1 root root 5 May 29 08:41 status
drwxr-xr-x 2 root root 0 May 29 08:41 tca6408@21
drwxr-xr-x 4 root root 0 May 29 08:41 tca9546@70
drwxr-xr-x 6 root root 0 May 29 08:41 tca9548@70
drwxr-xr-x 8 root root 0 May 29 08:41 tca9548@77

It seems that /proc/device-tree/ contains the original (Jetpack 6 default) device tree map and all my modifications are not applied

Regards

Hi,

Please first make sure which set of extlinux config the device is booting with.
If the L4T Boot Mode option in UEFI menu is ExtLinux, and there is no FDT entry specified, then the device tree from the bootloader partition will be loaded; if FDT is specified, then the dtb file from rootfs will be loaded.
If the L4T Boot Mode option is Kernel Partition, then the device tree from A/B_kernel-dtb partition will be loaded regardless of whether FDT is specified.

Hi @DaveYYY ,
I check the boot settings and I also check the graph here
https://docs.nvidia.com/jetson/archives/r36.3/DeveloperGuide/SD/Bootloader/UEFI.html#dtb-support

The L4T boot mode is ExtLinux, and my /boot/extlinux/extlinux.conf file has this configuration
LABEL primary
MENU LABEL primary kernel
LINUX /boot/Image
FDT /boot/dtb/kernel_tegra234-p3737-0000+p3701-0000-nv.dtb
INITRD /boot/initrd
APPEND ${cbootargs} root=/dev/nvme0n1p1 rw rootwait rootfstype=ext4 mminit_loglevel=4 console=ttyTCU0,115200 console=ttyAMA0,115200 firmware_class.path=/etc/firmware fbcon=map:0 net.ifnames=0 nospectre_bhb video=efifb:off console=tty0 nv-auto-config
OVERLAYS /boot/tegra234-p3737-camera-imx185-overlay.dtbo

So LINUX, FDT and OVERLAYS entry are present but nothing change in device tree loaded

How is possible to know which device tree is actually loaded?

Regars

You can add some random nodes in the top level of the device tree, and check their contents with procfs.
Like what we do with these two nodes in JetPack 5, which have been removed in JetPack 6:

nvidia,dtsfilename = FILE;
nvidia,dtbbuildtime = DATE, TIME;

Hi,
I added a top level node in file tegra234-p3737-0000+p3701-0000.dts. After rebuild dtbs I replace tegra234-p3737-0000+p3701-0000-nv.dtb in root targetfs at /boot/dtb/kernel_tegra234-p3737-0000+p3701-0000-nv.dtb.

After rebuild, actually I can find my new node in /proc/device-tree.

Since the modifications relative to camera modules are in tegra234-p3737-0000+p3701-0000-dynamic.dtbo and tegra234-p3737-camera-imx185-overlay.dtbo, first i replaced the new version of these files in /boot/ and next I tried to add the keyword

OVERLAYS /boot/tegra234-p3737-0000+p3701-0000-dynamic.dtbo

in extlinux.conf, and I can see the modification. How can I add mutiple overlay in order to apply tegra234-p3737-0000+p3701-0000-dynamic.dtbo and tegra234-p3737-camera-imx185-overlay.dtbo?

I found a solution directly editing tegra234-p3737-camera-modules.dtsi and tegra234-camera-imx* files. Now at boot the sensor is succesfully probed but i cannot see /dev/video0 even if I can see /dev/media0

hello liviolima80,

FYI,
here’s what tegra234-p3737-0000+p3701-0000-dynamic.dts has included.

  9 #include "tegra234-p3737-camera-modules.dtsi"
 10 
 11 / {
 12         overlay-name = "Tegra234 p3737-0000+p3701-xxxx Dynamic Overlay";

as you can see…
it’s tegra234-p3737-camera-modules.dtsi to include all those camera nodes.
you may use overlay to update those properties, or please see-also [To register a device using the main platform device tree file] to specify the .dtsi file for your new device.

did you meant there’s no /dev/video0 created?

during kernel initialization stage, it’s step for camera device registration to setup a video device node to linux kernel. sensor probing only run once during kernel initialization stage of system boot-up.
for a typical camera application running cycle, the driver will Power On the sensor, Start Sensor Streaming, sending relevant v4l2 controls, and finally power off the sensor.
so… if there’s a error returns, it’ll not register a video node, (i.e. /dev/video0).

Hi @JerryChang ,
there is no /dev/video0

What i can see from dmesg is that power_on driver function is called, but after probe no start_streaming, power_off and stop_streaming

With new Jetpack 6 for the driver I’m using the source code i wrote for Jetpack 5. The driver source file is correctly compiled. Do you think that this could be a problem? It seems that jetpack 5 sensor drivers are compatible to jetpack 6

please gather the logs for reference. you may also adding some debug messages to ensure the probing function has return correctly.

Thank you @JerryChang ,
I’ll try to debug through the steps. I think the problem is that the device is not registered even if the probe is succesfully done. Could you indicate me the kernl source file where the device registration takes place?

Regards

What’s your sensor? IMX185?

Hi @ShaneCCC ,
no I’m using a custom camera board based on IMX675

What’s the pixel format? Need apply below patch if format is BGGR 12bit.
Otherwise you may need trace in to tegracam_device_register() and tegracam_v4l2subdev_register().