Create and include own .dtsi files for L4T 24.2 fails at endpoint definition

schwartz · November 7, 2016, 2:56pm

Hi!
we are porting our soc_camera driver to 24.2 to be able to use the ISP. Unfortunately, we cannot compile the dtb files. (make dtbs fails).
I’m using the NVIDIA Tegra Linux Driver Package Development Guide for reference.

Is my assumption correct that to use the media controller api I cannot use arch/arm64/boot/mach-tegra/board-t210ref-camera.c to declare my cameras? Because we already have a working file for our setup.

What I did until this point:
I copied
arch/boot/dts/tegra210-platforms/tegra210-camera-e3326-a00.dtsi
arch/boot/dts/tegra210-platforms/tegra210-jetson-cv-camera-e3326-a00.dtsi
which are the device tree files for the ov5693 used in the Driver Package Development Guide.

I renamed them to tegra210-jetson-cv-camera-g2-a00.dtsi and tegra210-camera-g2-a00.dtsi and renamed the modules inside the files to our sensor. I will attach these two files. I checked with diff for typos, but there seem to be none.

Then, following the guide (“Device Registration”), I removed the camera-plugin-manager from tegra210-plugin-manager/tegra210-jetson-cv-plugin-manager.dtsi
and included tegra210-jetson-cv-camera-g2-a00.dtsi in kernel/arch/arm64/boot/dts/tegra210-jetson-cv-base-p2597-2180-a00.dts

When I now try to compile with make dtbs, I get the following error:

DTC     arch/arm64/boot/dts/tegra210-p4573-2180.dtb
ERROR (phandle_references): Reference to non-existent node or label "g2_vi_in0"

ERROR: Input tree has errors, aborting (use -f to force output)
make[1]: *** [arch/arm64/boot/dts/tegra210-p4573-2180.dtb] Error 2
make: *** [dtbs] Error 2

If I include tegra210-jetson-cv-camera-e3326-a00.dtsi in the tegra210-jetson-cv-base-p2597-2180-a00.dts file, compilation works just fine. I tried full text search for “e3326_vi_in0” and it was only found in the two files named above and in tegra210-jetson-cv-camera-plugin-manager.dtsi, which include I just removed following the guide.

So how exactly do I add my own sensor to the device tree?
tegra210-camera-g2-a00.txt (3.72 KB)
tegra210-jetson-cv-camera-g2-a00.txt (1.55 KB)

wilkinsw · November 7, 2016, 5:37pm

Had the exact same issue! (https://devtalk.nvidia.com/default/topic/973760/issues-compiling-dtbs/#5009039)

From reading dmesg the kernel loads tegra210-jetson-tx1-p2597-2180-a01-devkit.dtb, so you don’t need to worry about any blobs except for that one. I just deleted all the other blobs from /boot/ and only use the one.

The reason you get that error is tegra210-p4573-2180 has a /delete-nodes/ instruction that removes your label.

I’m wrestling with the device tree too, let me know if you get everything working!

schwartz · November 8, 2016, 9:27am

That seems to have worked, many thanks!

schwartz · November 8, 2016, 10:37am

While you are still here,
could you shortly elaborate on how to actually use this device tree?

I have a sensor on CSI-E, using i2c-2

All .dtsi files for image sensors (e.g. tegra210-camera-e3326-a00.dtsi) use i2c@546c0000, which is defined in tegra210-soc-base.dtsi as i2c7
There are only 6 i2c busses on the tegra, so I don’t know what this is supposed to mean.
To be honest I have no idea how to register my device. Where do I define the CSI port? Is it the csi-port field in the vi-port? Why is it a number, when the CSI ports are labeled with letters?

Could you help me with this please and tell me where to set the CSI port and the i2c port for a sensor? We want to use 6 sensors at the end, but even looking at tegra210-camera-e3322-a00.dtsi didn’t help, since there a no comments or any description whatsoever.

Best regards!

wilkinsw · November 8, 2016, 4:02pm

From experimentation there are actually 7 i2c busses (0-6, including 0).

i2c@546c0000 refers to i2c bus 6, otherwise known as CAM_I2C.

A strategy I used to confirm which bus my device was on was by using i2cdetect. It’s a utility that prints all the devices on the specified bus. Just enter 0-6 for the bus id.

i2cdetect -y -r bus_id

As for the CSI port question I’m currently figuring that out myself. Based on my research I believe you are correct in that the csi-port field in the vi-port node is where it is specified. It’s probably a number just to make things easier. I’ve just been assuming it’s enumerated with 0=A, 1=B, etc.

From what I’ve gathered there are three things you need to add to the device tree for your device. I attached my dtsi and made some notes below.

host1vx -> vi -> ports -> ...
i2c2@... -> device@... -> ...
tegra-camera-platform -> modules -> ...

Here’s the vi section from the driver i’m working on. I wrote this under the assumption that csi-port 0=A. Note the daxc02_vi_in0 label and the remote endpoint location. The port has to be connected to your i2c driver’s port endpoint so that vi can find and control it.

host1x {
        vi {
            num-channels = <0x1>;
            ports {
                #address-cells = <0x1>;
                #size-cells = <0x0>;

                port@0 {
                    reg = <0x0>;
                    status = "okay";
                    daxc02_vi_in0: endpoint {
                        status = "okay";
                        csi-port = <0x0>;
                        bus-width = <0x4>;
                        remote-endpoint = <&daxc02_out0>;
                    };
                };      
            };   
        };

Now here’s the corresponding section for the i2c device. Fill in the mode and device information for your device. The important part here is the ports section. Note that I created a port on the same bus and it is connected both directions to the vi port above. I used ov5693_c@36 as an example.

i2c@546c0000 {
            #address-cells = <1>;
            #size-cells = <0>;

            daxc02@0e {
                // all the information for your device here...
                ports {
                    #address-cells = <0x1>;
                    #size-cells = <0x0>;

                    port@0 {
                        reg = <0x0>;
                        daxc02_out0: endpoint {
                            csi-port = <0x0>;
                            bus-width = <0x4>;
                            remote-endpoint = <&daxc02_vi_in0>;
                        };
                    };
                };
            };

And finally the tegra-camera-platform:

tegra-camera-platform {
        compatible = "nvidia, tegra-camera-platform";
        modules {
            module0 {
                badge = "daxc02_master_mt9m021";
                position = "front";
                orientation = "1";
                status = "okay";
                drivernode0 {
                    pcl_id = "v4l2_sensor";
                    devname = "daxc02 6-000e";
                    proc-device-tree = "/proc/device-tree/host1x/i2c@546c0000/daxc02@0e";
                	status = "okay";
                };
                drivernode1 {
                    status = "okay";
                    pcl_id = "v4l2_focuser_stub";
                    proc-device-tree = [00];
                };
            };
        };
    };

Another useful trick I came across is how to set the power regulators to always on:

regulators {
        /* VDD_SYS_EN */
        en_vdd_cam: regulator@5 {
            regulator-boot-on;
            regulator-always-on;
        };

        /* CAM_VDD_2V8_EN */
        en_vdd_cam_hv_2v8: regulator@206 {
            regulator-boot-on;
            regulator-always-on;
        };

        /* CAM_VDD_1V8_EN */
        en_vdd_cam_1v8: regulator@211 {
            regulator-boot-on;
            regulator-always-on;
        };

        /* CAM_VDD_1V2_EN */
        en_vdd_cam_1v2: regulator@209 {
            regulator-boot-on;
            regulator-always-on;
        };
    };
};

What really helped me in troubleshooting was de-compiling the blob to see if anything was getting overwritten.

sudo apt-get install device-tree-compiler
dtc -I dtb tegra210-jetson-tx1-p2597-2180-a01-devkit.dtb | less

tegra210-daxc02.txt (8.78 KB)

schwartz · November 10, 2016, 3:46pm

Hello Atrer,

thank you for your reply, it really helped us a lot so far.

I excluded all other .dtb files from the Makefile, a crazy error we still have is that we know that our sensor is on i2c-2, we checked with i2cdetect and looked at the schematics.
i2c-2 would be at i2c@7000c500, but our probe function only gets called when we leave the i2c@546c0000 as the i2c bus. Changing it to the right i2c bus leads to our driver not even being called, which is crazy to us.
Did this happen to you as well or do you have any idea where to look for the problem?

And just to make it really sure, you didn’t alter the board-t210ref-camera.c file? To our knowledge it only works with the now obsolete soc_camera_platform.

Best regards and thanks for all the help, even with the most recent documentation it’s really a mess to get these dtsi files to work!

ShaneCCC · November 11, 2016, 6:44am

Only the i2c@546c0000 is under the host1x, must of others are the same level the host1x.

proc-device-tree = “/proc/device-tree/host1x/i2c@546c0000/g2_e@10”;
proc-device-tree = “/proc/device-tree/i2c@7000c500/g2_e@10”;

Rejeesh_QueST · November 24, 2016, 4:57am

Hi,

I am also trying to port my SoC sensor driver to MCF driver (for 24.2). And as it was the case with schwartz, my sensor is also connected to I2C_2 (i2c@7000c500). I too had this issue of probe function not being called initially, bot following the suggstion from NVIDIA,now my probe function is called (proc-device-tree = “/proc/device-tree/i2c@7000c500/g2_e@10”) .

Now, my issue is that neither /dev/video0 nor /dev/media0 are created.

Note:

My driver is not built as static module. I am inserting it separately. When I insert my driver, probe function is successfully completed.
I am referring to “Sensor Driver Programming Guide” coming with “Tegra_Linux_Driver_Package_Documents_R24.2.tar” documenation package
I am using “tegra210-jetson-cv-camera-imx274.dtsi” as my reference dtsi file
I replaced “tegra210-jetson-cv-camera-modules.dtsi” inclusion in “tegra210-jetson-cv-base-p2597-2180-a00.dts” with “tegra210-jetson-cv-camera-imx274.dtsi”.

Any pointers to as what could be the reason why /dev/video0 or /dev/media0 are not created for me?

schwartz · November 24, 2016, 8:20am

Did you try to compile your driver static? Because if you compile your driver as a module, but the media controller/vi remains static, your driver_probe will run, but it will be too late for the vi to register your device.

Rejeesh_QueST · November 24, 2016, 8:25am

Thanks Schwartz for the quick response.

I shall try building my driver as static.

However, with my design I need to load my driver as a module. Is it possible to get it working by any means?

May be, by building media controller/vi too as a module?

ShaneCCC · November 24, 2016, 8:35am

What design need it as loadable driver?
You can make it as build in module to bring it up first. That try to figure the later things out.

Rejeesh_QueST · November 24, 2016, 8:51am

I tried it with my driver as static and still /dev/video0 is not created.

To make my driver static, I just added “obj-y += mysensdrv.o” to /drivers/media/i2c/Makefile and rebuild the kernel (make zImage).

Even after rebooting with new kernel, I did not see /dev/video0/ and no debug messages from my driver in boot message, indicating that my driver init is not called.

Should I do anything else for my driver to be probed?

schwartz · November 24, 2016, 9:04am

Hi,
by making it static I meant to not compile it as a module.
Also please look at the vi debug output in dmesg.
make zImage only builds the kernel file, make modules would build your driver if it’s configured to be built as a module.

ShaneCCC · November 24, 2016, 9:05am

Your driver will not load should be cause by the incorrect device tree. You need take time to check you dts file.

Rejeesh_QueST · November 29, 2016, 11:39am

As Shane suggested, I spent some time on my dtsi files and now /dev/video0 and /dev/media0 are created for me.

However, I am getting all 0’s (black screen) during streaming using standard V4L2 application(using /dev/video0).

Below is the kernel log during streaming. All prints starting with “drivers/media/i2c/mysens.c” are from my driver.

mysens_v4l2 2-0010: camera_common_mclk_enable: enable MCLK with 27000000 Hz
drivers/media/i2c/mysens.c : mysens_power_on() ENTER
drivers/media/i2c/mysens.c : mysens_power_on() EXIT
drivers/media/i2c/mysens.c : mysens_set_fmt() ENTER
mysens_v4l2 2-0010: camera_common_try_fmt: size 1280 x 720
drivers/media/i2c/mysens.c : mysens_set_fmt() EXIT
drivers/media/i2c/mysens.c : mysens_set_fmt() ENTER
mysens_v4l2 2-0010: camera_common_s_fmt(12303) size 1280 x 720
mysens_v4l2 2-0010: camera_common_try_fmt: size 1280 x 720
drivers/media/i2c/mysens.c : mysens_set_fmt() EXIT
misc tegra_camera_ctrl: vi_v4l2_update_isobw:Set iso bw 55296 kbyteps at 3456 KHz
misc tegra_camera_ctrl: tegra_camera_isomgr_request++ bw=55296, lt=4
misc tegra_camera_ctrl: tegra_camera_isomgr_request: tegra_camera isomgr latency is 4 usec
drivers/media/i2c/mysens.c : mysens_s_stream() ENTER
drivers/media/i2c/mysens.c : mysens_s_stream() EXIT
<i>vi vi: Selected no CSI lane, cannot do calibration</i>

Will this log (especially, the one marked in italic) help you guys suggest what is missing in my setup?

I shall also attach my dtsi files and kernel log during boot as well

tegra210-jetson-cv-camera-mysens.dtsi

/*
 * Copyright (c) 2015-2016, NVIDIA CORPORATION.  All rights reserved.
 *
 * This program is free software; you can redistribute it and/or modify
 * it under the terms of the GNU General Public License as published by
 * the Free Software Foundation; either version 2 of the License, or
 * (at your option) any later version.
 *
 * This program is distributed in the hope that it will be useful, but WITHOUT
 * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
 * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
 * more details.
 *
 * You should have received a copy of the GNU General Public License
 * along with this program.  If not, see <http://www.gnu.org/licenses/>.
 */

#include <tegra210-platforms/tegra210-camera-mysens.dtsi>

#define CAM1_RST_L	TEGRA_GPIO(S, 5)

/* camera control gpio definitions */

/ {
	i2c@7000c500 {
		mysens@10 {
			clocks = <&tegra_car TEGRA210_CLK_ID_CLK_OUT_3>;
			clock-names = "mclk";
			clock-frequency = <24000000>;

			mclk = "cam_mclk1";
			reset-gpios = <&gpio CAM1_RST_L GPIO_ACTIVE_HIGH>;
		};
	};

	gpio: gpio@6000d000 {
		camera-control {
			gpio-input = <
				TEGRA_GPIO(X, 2)
				>;
			gpio-output-low = <
				CAM1_RST_L
				TEGRA_GPIO(S, 4)
				>;
		};
	};
};

tegra210-camera-mysens.dtsi

/*
 * Copyright (c) 2015-2016, NVIDIA CORPORATION.  All rights reserved.
 *
 * This program is free software; you can redistribute it and/or modify
 * it under the terms of the GNU General Public License as published by
 * the Free Software Foundation; either version 2 of the License, or
 * (at your option) any later version.
 *
 * This program is distributed in the hope that it will be useful, but WITHOUT
 * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
 * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
 * more details.
 *
 * You should have received a copy of the GNU General Public License
 * along with this program.  If not, see <http://www.gnu.org/licenses/>.
 */

#include <dt-bindings/media/camera.h>
#include <dt-bindings/platform/t210/t210.h>

/ {
	host1x {		
		vi {
		    num-channels = <1>;
		    ports {
			#address-cells = <1>;
			#size-cells = <0>;
		        port@0 {
		            status = "okay";
					mysens_vi_in0: endpoint {
						status = "okay";
						csi-port = <4>; // CSI EF
						bus-width = <4>; // 4 lane
						remote-endpoint = <&mysens_out0>;
					};
		        };
		    };
		};
     };

	i2c@7000c500 { // I2C_2
		status = "okay";
		#address-cells = <1>;
		#size-cells = <0>;
		mysens@10 {
			compatible = "nvidia,mysens_v4l2";
			/* I2C device address */
			reg = <0x10>;

			/* Physical dimensions of sensor */
			physical_w = "3.674";
			physical_h = "2.738";

			/* Sensor Model */
			sensor_model ="mysens";

			post_crop_frame_drop = "0";

			mode0 { // MODE_3840X2160@30fps
				mclk_khz = "24000";
				num_lanes = "4";
				tegra_sinterface = "serial_e";
				discontinuous_clk = "yes";
				cil_settletime = "0";

				active_w = "3864";
				active_h = "2160";
				pixel_t = "bayer_rggb";
				readout_orientation = "90";
				line_length = "4200";
				inherent_gain = "1";
				mclk_multiplier = "25";
				pix_clk_hz = "576000000";

				min_gain_val = "1.0";
				max_gain_val = "22.2";
				min_hdr_ratio = "1";
				max_hdr_ratio = "64";
				min_framerate = "1.462526";
				max_framerate = "30";
				min_exp_time = "16.165";
				max_exp_time = "33191";
				embedded_metadata_height = "0";
			};

			mode1 { // MODE_3840X2160@60fps
				mclk_khz = "24000";
				num_lanes = "4";
				tegra_sinterface = "serial_e";
				discontinuous_clk = "yes";
				cil_settletime = "0";

				active_w = "3864";
				active_h = "2160";
				pixel_t = "bayer_rggb";
				readout_orientation = "90";
				line_length = "4200";
				inherent_gain = "1";
				mclk_multiplier = "25";
				pix_clk_hz = "576000000";

				min_gain_val = "1.0";
				max_gain_val = "22.2";
				min_hdr_ratio = "1";
				max_hdr_ratio = "64";
				min_framerate = "1.462526";
				max_framerate = "60";
				min_exp_time = "16.165";
				max_exp_time = "33191";
				embedded_metadata_height = "0";
			};

			mode2 { // MODE_1920X1080@60fps
				mclk_khz = "24000";
				num_lanes = "4";
				tegra_sinterface = "serial_e";
				discontinuous_clk = "yes";
				cil_settletime = "0";

				active_w = "1932";
				active_h = "1080";
				pixel_t = "bayer_rggb";
				readout_orientation = "90";
				line_length = "4200";
				inherent_gain = "1";
				mclk_multiplier = "25";
				pix_clk_hz = "576000000";

				min_gain_val = "1.0";
				max_gain_val = "22.2";
				min_hdr_ratio = "1";
				max_hdr_ratio = "64";
				min_framerate = "1.462526";
				max_framerate = "60";
				min_exp_time = "16.165";
				max_exp_time = "33191";
				embedded_metadata_height = "0";
			};

			ports {
				#address-cells = <1>;
				#size-cells = <0>;

				port@0 {
					reg = <0>;
					mysens_out0: endpoint {
						csi-port = <4>;
						bus-width = <4>;
						remote-endpoint = <&mysens_vi_in0>;
					};
				};
			};
		};
	};

	tegra-camera-platform {
		compatible = "nvidia, tegra-camera-platform";

		/**
		* The general guideline for naming badge_info contains 3 parts, and is as follows,
		* The first part is the camera_board_id for the module; if the module is in a FFD
		* platform, then use the platform name for this part.
		* The second part contains the position of the module, ex. “rear” or “front”.
		* The third part contains the last 6 characters of a part number which is found
		* in the module's specsheet from the vender.
		*/
		modules {
			module0 {
				badge = "mysens_front_A6V24";
				position = "rear";
				orientation = "1";
				status = "okay";
				drivernode0 {
					/* Declare PCL support driver (classically known as guid)  */
					pcl_id = "v4l2_sensor";
					/* Driver's v4l2 device name */
					devname = "my sensor";
					/* Declare the device-tree hierarchy to driver instance */
					proc-device-tree = "/proc/device-tree/i2c@7000c500/mysens@10";
					status = "okay";
				};
			};
		};
	};
};

Relevant kernel log during booting

i2c /dev entries driver
[1322] : drivers/media/i2c/mysens.c : mysens_init() ENTER
[756] : drivers/media/i2c/mysens.c : mysens_probe() ENTER
Node name: mysens
[637] : drivers/media/i2c/mysens.c : mysens_parse_dt() ENTER
[688] : drivers/media/i2c/mysens.c : mysens_parse_dt() EXIT
[389] : drivers/media/i2c/mysens.c : mysens_power_get() ENTER
[405] : drivers/media/i2c/mysens.c : mysens_power_get() EXIT
Inside camera_common_parse_ports()
mysens_v4l2 2-0010: camera_common_parse_ports: csi port 4 num of lanes 4
Exiting camera_common_parse_ports()
[mysens]CSI Port:4 Lane conf: 4
[578] : drivers/media/i2c/mysens.c : mysens_ctrls_init() ENTER
[541] : drivers/media/i2c/mysens.c : mysens_s_ctrl() ENTER
[541] : drivers/media/i2c/mysens.c : mysens_s_ctrl() ENTER
[541] : drivers/media/i2c/mysens.c : mysens_s_ctrl() ENTER
[541] : drivers/media/i2c/mysens.c : mysens_s_ctrl() ENTER
[541] : drivers/media/i2c/mysens.c : mysens_s_ctrl() ENTER
[541] : drivers/media/i2c/mysens.c : mysens_s_ctrl() ENTER
[616] : drivers/media/i2c/mysens.c : mysens_ctrls_init() EXIT
[928] : drivers/media/i2c/mysens.c : mysens_probe() EXIT
[1339] : drivers/media/i2c/mysens.c : mysens_init() EXIT
virtual_init
[IMX135] sensor driver loading
[imx132] sensor driver loading
[ar0261] sensor driver loading
.
.
.
platform d3.regulator: Driver reg-fixed-sync-voltage requests probe deferral
vi vi: vi_probe: ++
vi vi: vi_intr_init: ++
vi vi: initialized
vi vi: parsing node /host1x/vi
vi vi: handling endpoint /host1x/vi/ports/port@0/endpoint
vi vi: parsing node /i2c@7000c500/mysens@10
vi vi: handling endpoint /i2c@7000c500/mysens@10/ports/port@0/endpoint
vi vi: subdev mysens_v4l2 2-0010 bound
vi vi: notify complete, all subdevs registered
vi vi: creating links for entity mysens_v4l2 2-0010
vi vi: processing endpoint /i2c@7000c500/mysens@10/ports/port@0/endpoint
vi vi: skipping channel port /i2c@7000c500/mysens@10:0
vi vi: creating links for channels
vi vi: processing endpoint /host1x/vi/ports/port@0/endpoint
vi vi: creating link for channel vi-output-255
vi vi: creating mysens_v4l2 2-0010:0 -> vi-output-255:0 link
mysens_v4l2 2-0010: camera_common_g_fmt++
reg-fixed-sync-voltage 5.regulator: Consumer c1 does not have device name
.
.
.
[699] : drivers/media/i2c/mysens.c : mysens_open() ENTER
[701] : drivers/media/i2c/mysens.c : mysens_open() EXIT
mysens_v4l2 2-0010: camera_common_mclk_enable: enable MCLK with 27000000 Hz
[280] : drivers/media/i2c/mysens.c : mysens_power_on() ENTER
[322] : drivers/media/i2c/mysens.c : mysens_power_on() EXIT
[340] : drivers/media/i2c/mysens.c : mysens_power_off() ENTER
[370] : drivers/media/i2c/mysens.c : mysens_power_off() EXIT
mysens_v4l2 2-0010: camera_common_mclk_disable: disable MCLK

Please note that, in the dtsi file, I have put i2c info outside host1x node since I am using I2C_2 . Is there something wrong with my dtsi file or should I concentrate on my driver code? Any pointer/help is really appreciated as I am kind of stuck at the moment.

Rejeesh_QueST · November 30, 2016, 7:18am

I got it sorted out. So, now everything working perfectly.

Now one final query:

Now, my module is built as static module. As I mentioned in an earlier post, I need my module to be built as separate module and loaded as a module after the kernel is boot.

What are the changes I should make in the kernel (may be, build some built in modules like vi as separate module?) for streaming to work?

Right now, When I insert my module as a separate module, /dev/video0 & /dev/media0 are not created.

ShaneCCC · November 30, 2016, 7:55am

You may need build the …/kernel/drivers/media/platform/tegra/vi/. as module, and you should need to fix a lot of compiling error due to lot of code dependency. The csi, camera, mipical, tpg should be the relative code dependency.

Rejeesh_QueST · November 30, 2016, 8:11am

Thanks ShaneCCC for quick response. Seems hard time ahead for me :)

btw, has anyone did this before?

wilkinsw · November 30, 2016, 3:00pm

I tried, did not go well.

sensor_test · August 10, 2018, 9:40am

Hello ALL,
I can not fix the error:

ERROR (phandle_references): Reference to non-existent node or label

Who can help me fix this?