Xavier NX R32.7.1 changes to DTSI Overlay for cameras has no effect

Hello,

I’m developing custom device tree source based on the source for the Jakku carrier board found in /hardware/nvidia/platform/t19x/jakku/kernel-dts. For now, I’m just trying to adjust the cam node and i2c mux layout to match our board, and enable the imx477 cam node by default. The former is simply removing imx477 cam1 instance, and enabling access to cam0 with i2cmux index 1, and not the focus of this post. The latter I’m trying to do by adjusting the dts overlays, but it’s not working.

I’m trying to modify the source so that the imx477 device node receives the fragment property status = okay and the imx219 nodes are disabled. I’ve deleted the three default overlays from the above mentioned folder, including:

  • tegra194-p3668-all-p3509-0000-camera-imx219-dual.dts
  • tegra194-p3668-all-p3509-0000-camera-imx477-dual.dts
  • tegra194-p3668-all-p3509-0000-camera-imx477-imx219.dts

I’ve also removed these from the Makefile. Next, I created a copy of the dts overlay for the imx477-dual configuration, shown below, and added it to the Makefile. I modified the copy to keep disabling both imx219 nodes and enable the imx477 components.

/dts-v1/;
/plugin/;

/ {
        overlay-name = "Camera IMX477 Dual";
        jetson-header-name = "Jetson Nano CSI Connector";
        compatible = "nvidia,p3449-0000+p3668-0000", "nvidia,p3449-0000+p3668-0001", "nvidia,p3509-0000+p3668-0000", "nvidia,p3509-0000+p3668-0001";

        /* IMX477 dual sensor module */
        fragment@0 {
                target = <&imx477_cam0>;
                __overlay__ {
                        status = "okay";
                };
        };
        fragment@1 {
                target = <&cam_module0>;
                __overlay__ {
                        status = "okay";
                        badge = "jakku_front_RBPCV3";
                        position = "front";
                        orientation = "1";
                };
        };
        fragment@2 {
                target = <&cam_module0_drivernode0>;
                __overlay__ {
                        status = "okay";
                        pcl_id = "v4l2_sensor";
                        devname = "imx477 9-001a";
                        proc-device-tree = "/proc/device-tree/cam_i2cmux/i2c@1/rbpcv3_imx477_a@1a";
                };
        };
        fragment@6 {
                target = <&imx219_cam0>;
                __overlay__ {
                        status = "disabled";
                };
        };
        fragment@7 {
                target = <&rbpcv3_imx477_vi_in0>;
                __overlay__ {
                        status = "okay";
                        port-index = <0>;
                        bus-width = <2>;
                        remote-endpoint = <&rbpcv3_imx477_csi_out0>;
                };
        };
        fragment@9 {
                target = <&rbpcv3_imx477_csi_in0>;
                __overlay__ {
                        status = "okay";
                        port-index = <0>;
                        bus-width = <2>;
                        remote-endpoint = <&rbpcv3_imx477_dual_out0>;
                };
        };
        fragment@10 {
                target = <&rbpcv3_imx477_csi_out0>;
                __overlay__ {
                        status = "okay";
                };
        };
        fragment@13 {
                target = <&imx219_cam1>;
                __overlay__ {
                        status = "disabled";
                };
        };
};

I’ve made other changes to other dts files to achieve what I described above, but I haven’t yet included them for brevity, can include on request. Upon building, I can decompile the resulting DTO and see my desired changes to the i2cmux/imx477 configuration take effect, but the status attributes are not present.

// this shows the desired configuration but status disabled
                i2c@1 {
                        reg = <0x1>;
                        #address-cells = <0x1>;
                        #size-cells = <0x0>;
                        linux,phandle = <0x174>;
                        phandle = <0x174>;

                        rbpcv3_imx477_a@1a {
                                compatible = "ridgerun,imx477";
                                reg = <0x1a>;
                                devnode = "video0";
                                physical_w = "3.680";
                                physical_h = "2.760";
                                sensor_model = "imx477";
                                use_sensor_mode_id = "true";
                                status = "disabled";
                                reset-gpios = <0x13 0x7c 0x0>;
                                linux,phandle = <0x175>;

later in that file …

// No status disabled attribute!
                        rbpcv2_imx219_c@10 {
                                compatible = "nvidia,imx219";
                                reg = <0x10>;
                                devnode = "video1";
                                physical_w = "3.680";
                                physical_h = "2.760";
                                sensor_model = "imx219";
                                use_sensor_mode_id = "true";
                                reset-gpios = <0x13 0x7d 0x0>;
                                linux,phandle = <0x177>;
                                phandle = <0x177>;

Again, my focus in this post is just on controlling the status attribute from the overlay dtsis provided for the jakku board.

Thanks!

If just want to enable IMX477 by default you can just modify the status to okay in imx477 fragment

Thanks for the response Shane. You’re right, I was able to achieve the desired statuses by just hardcoding the status in the actual imx477 fragment.

However, I would still like to understand how the fragment overlays work. As a starting point, can you explain which of the default overlays (imx477-dual, imx477-imx219, imx219 dual) takes precedence if all them are included in the default Makefile?

The default enable is IMX219 dual.

Yes I’ve witnessed that with fresh installs. My question is how IMX219 Dual is designated as the default in the build step.

The status of imx219 default set to okay

yes I understand that status is set by tegra194-p3668-all-p3509-0000-camera-imx477-imx219.dtbo. When is this file loaded? Is it baked into board dts or loaded at boot time?

Suppose you need to know detail information of the device tree overlay.
Have a check if below help on it.

https://docs.kernel.org/devicetree/overlay-notes.html

Thanks for sharing the link Shane, though I can’t say it helped very much.

This task can be marked resolved, as I figured out answers to my questions. I’ll try to capture my findings here, with the obvious disclaimer that I can only report what I’ve seen and gotten working, and my answers might be misguided or incomplete.

How was the imx219 set as the default?

The stock device tree source is included in the kernel sources. For finding where the sources can be downloaded and how to build them, I found the script made available in this post unbelievably helpful in getting off the ground with building and flashing the kernel source. Shout out to @sgroesz and @james.rhodes for fighting the good fight. For posterity, note that the script didn’t work as is–I updated to use newer sources, used the build step and then flashed the resulting dtb using flash.sh instead of the payload update script.

Anyway, the answer to my original question here was obvious in hindsight. The first thing I want to highlight for noob posterity is that the Jakku device tree defines all four sensor instances in question (2x imx477 and 2x imx219) simultaneously, and then just selectively defines which are enabled and disabled. You can see this by looking at the includes in hardware/nvidia/platform/t19x/jakku/common/tegra194-p3509-0000-a00.dtsi:

 19 #include "tegra194-camera-jakku-rbpcv3-imx477.dtsi"
 20 #include "tegra194-camera-jakku-rbpcv2-imx219.dtsi"

This means under the cam_i2cmux element has all four sensors defined. Then you can see the default dts files describing the driver configuration leave the imx477 status as disabled, and leave the status attribute unset for the imx219, which appears to then default to okay/enabled. So the actual dts for the imager configuration is set up for dual imx219 until the overlays say otherwise.

Excerpt from the stock hardware/nvidia/platform/t19x/jakku/common/tegra194-camera-jakku-rbpcv3-imx477.dtsi, showing the default state of disabled:

    cam_i2cmux {
        compatible = "i2c-mux-gpio";
        #address-cells = <1>;
        #size-cells = <0>;
        mux-gpios = <&tegra_aon_gpio CAM_I2C_MUX GPIO_ACTIVE_HIGH>;
        i2c-parent = <&i2c7>; // @shoelick: I'm pretty sure this is a bug, had to change to match imx219
        i2c@0 {
            reg = <0>;
            #address-cells = <1>;
            #size-cells = <0>;
            rbpcv3_imx477_a@1a {
                status = "disabled"; // emphasis 
                reset-gpios = <&tegra_main_gpio CAM0_PWDN GPIO_ACTIVE_HIGH>;
            };
        };
        i2c@1 {
            reg = <1>;
            #address-cells = <1>;
            #size-cells = <0>;
            rbpcv3_imx477_c@1a {
                status = "disabled"; // emphasis
                reset-gpios = <&tegra_main_gpio CAM1_PWDN GPIO_ACTIVE_HIGH>;
            };
        };


Excerpt from the stock hardware/nvidia/platform/t19x/jakku/common/tegra194-camera-jakku-rbpcv2-imx219.dtsi, showing no state attribute:

   cam_i2cmux{
        compatible = "i2c-mux-gpio";
        #address-cells = <1>;
        #size-cells = <0>;
        i2c-parent = <&cam_i2c>;
        mux-gpios = <&tegra_aon_gpio CAM_I2C_MUX GPIO_ACTIVE_HIGH>;
        i2c@0 {
            reg = <0>;
            #address-cells = <1>;
            #size-cells = <0>;
            rbpcv2_imx219_a@10 {
                reset-gpios = <&tegra_main_gpio CAM0_PWDN GPIO_ACTIVE_HIGH>;
            };
        };
        i2c@1 {
            reg = <1>;
            #address-cells = <1>;
            #size-cells = <0>;
            rbpcv2_imx219_c@10 {
                reset-gpios = <&tegra_main_gpio CAM1_PWDN GPIO_ACTIVE_HIGH>;
            };
        };

At this point gaining an understanding of the video pipeline defined in the device tree is also helpful. Reading this post got me another “aha” moment and explains why the overlays apply statuses the other elements defined in common/tegra194-p3668-all-p3509-0000-camera-imx219-dual.dts and common/tegra194-p3668-all-p3509-0000-camera-imx477-imx219.dts. You have to get these elements right to have an actual working video node when your system boots.

Bonus knowledge: these files are also where the imx drivers and associated pipeline are tied to specific CSI lanes.

jetson-io.py and config-by-hardware.py

The second place I’ve seen overlays applied is the infamous /opt/nvidia/jetson-io/jetson-io.py script. This script appears to find any compatible .dto files in /boot (based on the headers), makes nice ncurses menus out of it, and when you choose a setting in the ncurses interface, it grabs the base device tree (maybe from /boot/dts or something, not actually sure which), applies the overlay you’ve chosen, compiles this combo into one new dtb, writes it to /boot, and then makes sets a new default extlinux.conf boot entry that loads the new dtb with the overlay applied.

That said, with our custom carrier board this method with a dtb specified with a boot entry in the extlinux.conf doesn’t seem to work. The settings from jetson-io don’t take effect. In fact, I can put gibberish in the /boot/extlinux/extlinux.conf and the system boots with the last flashed dtb, ignoring whatever is in extlinux.conf.

Unanswered questions

Within the kernel sources, down in hardware/nvidia/platform/t19x/jakku/, you can see in the makefile that all three overlays are included in the build.

dtbo-$(CONFIG_ARCH_TEGRA_19x_SOC) += tegra194-p3668-all-p3509-0000-camera-imx477-imx219.dtbo
dtbo-$(CONFIG_ARCH_TEGRA_19x_SOC) += tegra194-p3668-all-p3509-0000-camera-imx477-dual.dtbo
dtbo-$(CONFIG_ARCH_TEGRA_19x_SOC) += tegra194-p3668-all-p3509-0000-camera-imx219-dual.dtbo

One small question that remains is whether these overlays are actually applied to the resulting base device tree build, or just included on the filesystem for future usage. I may test this when I have time.

The jetson documentation still remains incomplete based on some things I’ve seen. Some other questions remain for me, but they’re outside the scope of this post.

  • Why did extlinux stop taking effect on our custom board? How’s this thing even booting…?!
  • This post mentions a separate dtb partition, which I can only presume is where the dtb is being loaded from when my extlinux.conf is being ignored as described above.
  • I have only found so far that flash.sh as a reliable way of updating the dtb/applying new overlays. For past l4t versions there used to be a separate script to apply update payloads, but it doesn’t seem to be supported since Jetson Nano.

Other helpful references

2 Likes

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.