UB960/UB963/IMX477 on AGX Xavier, JP 5.1.3

Hello Nvidia,

We have working pipeline imx477/ub953/ub960 (D3 FPD Link iii) on Jetson AGX Xavier JP4.6.1.
I am trying to port that pipeline to JP5.1.3.

Whatever I do/change I stuck with trace log:

kworker/0:4-499     [000] ....    68.729674: rtcpu_dbg_set_loglevel: tstamp:2790389833 old:0 new:3
     kworker/0:4-499     [000] ....    68.729679: rtcpu_dbg_exit: tstamp:2790390004
  nvargus-daemon-2673    [003] ....    74.211166: tegra_channel_open: vi-output, imx477 2-0060
  nvargus-daemon-2673    [003] ....    74.211395: tegra_channel_close: vi-output, imx477 2-0060
  nvargus-daemon-2673    [003] ....    74.215298: tegra_channel_open: vi-output, imx477 2-0060
  nvargus-daemon-2673    [003] ....    74.215458: tegra_channel_close: vi-output, imx477 2-0060
  nvargus-daemon-2673    [003] ....    74.242849: tegra_channel_open: vi-output, imx477 2-0060
  nvargus-daemon-2673    [003] ....    74.242887: tegra_channel_close: vi-output, imx477 2-0060
  nvargus-daemon-2673    [003] ....    74.242918: tegra_channel_open: vi-output, imx477 2-0060
  nvargus-daemon-2673    [003] ....    74.243098: tegra_channel_close: vi-output, imx477 2-0060
  nvargus-daemon-2673    [003] ....    74.243142: tegra_channel_open: vi-output, imx477 2-0060
  nvargus-daemon-2673    [003] ....    74.243290: tegra_channel_close: vi-output, imx477 2-0060
 CaptureSchedule-2689    [000] ....    74.739725: tegra_channel_open: vi-output, imx477 2-0060
 CaptureSchedule-2689    [000] ....    74.740259: tegra_channel_set_power: imx477 2-0060 : 0x1
 CaptureSchedule-2689    [000] ....    74.740266: camera_common_s_power: status : 0x1
 CaptureSchedule-2689    [000] ....    75.040208: tegra_channel_set_power: 13e10000.host1x:nvcsi@15a00000- : 0x1
 CaptureSchedule-2689    [000] ....    75.040216: csi_s_power: enable : 0x1
 CaptureSchedule-2689    [000] ....    75.044320: tegra_channel_set_stream: enable : 0x1
 CaptureSchedule-2689    [000] ....    75.062726: tegra_channel_set_stream: 13e10000.host1x:nvcsi@15a00000- : 0x1
 CaptureSchedule-2689    [000] ....    75.062730: csi_s_stream: enable : 0x1
 CaptureSchedule-2689    [000] ....    75.062733: tegra_channel_set_stream: imx477 2-0060 : 0x1
     kworker/0:4-499     [000] ....    75.347445: rtcpu_vinotify_event: tstamp:2997110225 cch:1 vi:0 tag:VIFALC_TDSTATE channel:0x0b frame:0 vi_tstamp:95906338080 data:0xcd9c550010000000
     kworker/0:4-499     [000] ....    75.347448: rtcpu_vinotify_event: tstamp:2997110419 cch:1 vi:0 tag:VIFALC_TDSTATE channel:0x0b frame:0 vi_tstamp:95906380864 data:0x0000000031000001
     kworker/0:4-499     [000] ....    75.403391: rtcpu_vinotify_event: tstamp:2998195508 cch:1 vi:0 tag:VIFALC_TDSTATE channel:0x0b frame:0 vi_tstamp:95940114336 data:0xcd9c520010000000
     kworker/0:4-499     [000] ....    75.403412: rtcpu_vinotify_event: tstamp:2998195681 cch:1 vi:0 tag:VIFALC_TDSTATE channel:0x0b frame:0 vi_tstamp:95940157760 data:0x0000000031000002
     kworker/0:6-503     [000] ....    76.634559: rtcpu_dbg_enter: tstamp:3036817906, data:4
     kworker/0:6-503     [000] ....    76.634563: rtcpu_dbg_set_loglevel: tstamp:3036818039 old:3 new:3
     kworker/0:6-503     [000] ....    76.634565: rtcpu_dbg_exit: tstamp:3036818179
 CaptureSchedule-2689    [000] ....    77.955260: tegra_channel_set_stream: enable : 0x0
 CaptureSchedule-2689    [000] ....    77.955265: tegra_channel_set_stream: imx477 2-0060 : 0x0
 CaptureSchedule-2689    [000] ....    77.955958: tegra_channel_set_stream: 13e10000.host1x:nvcsi@15a00000- : 0x0
 CaptureSchedule-2689    [000] ....    77.955963: csi_s_stream: enable : 0x0
 CaptureSchedule-2689    [000] ....    77.967709: tegra_channel_set_power: imx477 2-0060 : 0x0
 CaptureSchedule-2689    [000] ....    77.967732: camera_common_s_power: status : 0x0
 CaptureSchedule-2689    [000] ....    77.967818: tegra_channel_set_power: 13e10000.host1x:nvcsi@15a00000- : 0x0
 CaptureSchedule-2689    [000] ....    77.967856: csi_s_power: enable : 0x0
  nvargus-daemon-2673    [001] ....    78.433682: tegra_channel_close: vi-output, imx477 2-0060
     kworker/0:6-503     [000] ....    78.480826: rtcpu_isp_falcon_task_start: tstamp:3093724298 ch:0 task:HANDLE_EVENT
     kworker/0:6-503     [000] ....    78.480831: rtcpu_isp_falcon_task_end: tstamp:3093724333 task:HANDLE_EVENT

Here is my device-tree:

/ {
    i2c@3180000 {

        /* Microchip 24AA1026 (U26) */
        eeprom@50 {
            status = "okay";
            compatible = "atmel,24c1024";
            reg = <0x50>;
            pagesize = <128>;
        };

        /* I2C GPIO expander PCAL9539A (U34) */
        gpio_i2c_3_74: gpio@74 {
            status = "okay";
            compatible = "nxp,pca9539";
            reg = <0x74>;
            #gpio-cells = <2>;
            gpio-controller;
            vcc-supply = <&p2822_vdd_1v8_cvb>;
        };

        /* I2C GPIO expander PCAL9539A (U35) */
        gpio_i2c_3_75: gpio@75 {
            status = "okay";
            compatible = "nxp,pca9539";
            reg = <0x75>;
            #gpio-cells = <2>;
            gpio-controller;
            vcc-supply = <&p2822_vdd_1v8_cvb>;
        };

        /* I2C GPIO expander PCAL9539A (U36) */
        gpio_i2c_3_76: gpio@76 {
            status = "okay";
            compatible = "nxp,pca9539";
            reg = <0x76>;
            #gpio-cells = <2>;
            gpio-controller;
            vcc-supply = <&p2822_vdd_1v8_cvb>;
        };

        /* ub960 deserializers will be included here (U1, U4, U7, U10) */
		ub690_30: ub960@30 {
			status = "okay";
			compatible = "d3,ub960";
			reg = <0x30>;
			iovdd-supply = <&p2822_vdd_1v8_cvb>;
			avdd-supply = <&fpd_vdd_0_3>;
			pdb-gpios = <&gpio_i2c_3_74 13 GPIO_ACTIVE_LOW>;
			
			csi-lane-count = <4>;
			csi-tx-speed-mbps = <1600>;
			csi-continuous-clock = <1>;

			/* Frame Synchronization Mode (FS_MODE field in FS_CTL register)
			 *
			 * 0x0 = internally generated framesync
			 * 0x8 = external framesync from gpio 0
			 * 0x9 = external framesync from gpio 1
			 * 0xa = external framesync from gpio 2
			 * 0xb = external framesync from gpio 3
			 *
			 * If this field is omitted, 0x0 is used.  It's value can be ignored
			 * if the imager is not operating in a triggered mode.
			 */
			frame-sync-mode = <8>;

			/* Internally generated FrameSync's high and low time, in microseconds.
			 *
			 * If these fields are omitted, they are configured for 15 ms high,
			 * 35 ms low (20 Hz). Their value can be ignored if an internal
			 * FrameSync is not being used
			 */
			frame-sync-high-time-us = <15000>;
			frame-sync-low-time-us = <35000>;

			#address-cells = <1>;
			#size-cells = <0>;
			ub690_30_link0: link@0 {
				status = "okay";
				reg = <0>;
				#address-cells = <1>;
				#size-cells = <0>;
				no-mux;

				ub953_0: ub953_0@40 {
					compatible = "d3,ub953";
					status = "okay";
					physical-addr = <0x18>;
					#address-cells = <1>;
					#size-cells = <0>;
					reg = <0x40>;
					csi-lane-count = <2>;
					csi-continuous-clock = <1>;
					i2c-voltage-sel = <0x0>;
					hs-clk-div = <0x4>;
					div-m-val = <0x1b>;
					div-n-val = <0xfa>;


/*
					ub953_gpio_0: ub953_gpio_0 {
						compatible = "d3,ub953-gpio";
						status = "okay";
						gpio-controller;
						#gpio-cells = <2>;
						ngpios = <4>;
						};
		*/
				};

				imx477_0: imx477@60 {
					status = "okay";
					compatible = "ridgerun,imx477";
					reg = <0x60>;
					physical-addr = <0x1a>;
					deserializer = <&ub690_30>;
					serializer = <&ub953_0>;
					devnode = "video0";
					physical_w = "3.680";
					physical_h = "2.760";
					sensor_model = "imx477";
					use_sensor_mode_id = "true";
					mode0 { /* IMX477_MODE_3840x2160 */
						mclk_khz = "24000";
						num_lanes = "2";
						tegra_sinterface = "serial_a";
						phy_mode = "DPHY";
						discontinuous_clk = "no";
						set_mode_delay_ms = "800";
						dpcm_enable = "false";
						cil_settletime = "0";
						lane_polarity = "0";
						vc_id = "0";
						active_w = "3840";
						active_h = "2160";
						mode_type = "bayer";
						pixel_phase = "rggb";
						csi_pixel_bit_depth = "10";
						readout_orientation = "90";
						line_length = "11200";
						inherent_gain = "1";
						mclk_multiplier = "80";
						pix_clk_hz = "300000000";
						serdes_pix_clk_hz = "535000000";
						gain_factor = "16";
						framerate_factor = "1000000";
						exposure_factor = "1000000";
						min_gain_val = "16"; /* 1.00x */
						max_gain_val = "356"; /* 22x */
						step_gain_val = "1";
						default_gain = "16"; /* 1.00x */
						min_hdr_ratio = "1";
						max_hdr_ratio = "1";
						min_framerate = "2000000"; /* 2.0 fps */
						max_framerate = "30000000"; /* 30.0 fps */
						step_framerate = "1";
						default_framerate = "30000000"; /* 30.0 fps */
						min_exp_time = "13"; /* us */
						max_exp_time = "683709"; /* us */
						step_exp_time = "1";
						default_exp_time = "2495"; /* us */
						
						embedded_metadata_height = "2";
					};
					
					mode1 { /* IMX477_MODE_1920X1080 */
						mclk_khz = "24000";
						num_lanes = "2";
						tegra_sinterface = "serial_a";
						phy_mode = "DPHY";
						discontinuous_clk = "no";
						set_mode_delay_ms = "800";
						dpcm_enable = "false";
						cil_settletime = "0";
						lane_polarity = "0";
						vc_id = "0";
						active_w = "1920";
						active_h = "1080";
						mode_type = "bayer";
						pixel_phase = "rggb";
						csi_pixel_bit_depth = "10";
						readout_orientation = "90";
						line_length = "7000";
						inherent_gain = "1";
						mclk_multiplier = "80";
						pix_clk_hz = "300000000";
						serdes_pix_clk_hz = "535000000";
						gain_factor = "16";
						framerate_factor = "1000000";
						exposure_factor = "1000000";
						min_gain_val = "16"; /* 1.00x */
						max_gain_val = "356"; /* 22x */
						step_gain_val = "1";
						default_gain = "16"; /* 1.00x */
						min_hdr_ratio = "1";
						max_hdr_ratio = "1";
						min_framerate = "2000000"; /* 2.0 fps */
						max_framerate = "60000000"; /* 60.0 fps */
						step_framerate = "1";
						default_framerate = "60000000"; /* 60.0 fps */
						min_exp_time = "13"; /* us */
						max_exp_time = "683709"; /* us */
						step_exp_time = "1";
						default_exp_time = "2495"; /* us */
	
						embedded_metadata_height = "2";
					};
					
					ports {
						#address-cells = <1>;
						#size-cells = <0>;

						sen_port_imx477_0: port@0 {
							reg = <0>;
							status = "okay";

							sen_out_imx477_0: endpoint {
								status = "okay";
								vc-id = <0>;
								port-index = <0>;
								bus-width = <2>;
								remote-endpoint = <&csi_in_0>;
							};
						};
					};
				};
			};
		};
    };



	

	/* all cameras are disabled by default */
	tegra-capture-vi {
		num-channels = <1>;
		#address-cells = <1>;
		#size-cells = <0>;
		ports {
			status = "okay";
			vi_port0: port@0 {
				#address-cells = <1>;
				#size-cells = <0>;
				status = "okay";
				reg = <0>;

				vi_in0: endpoint {
					status = "okay";
					vc-id = <0>;				 
					port-index = <0>;
					bus-width = <2>;
					remote-endpoint = <&csi_out_0>;
				};
			};
		};
	};

	host1x@13e00000 {
		csi_base: nvcsi@15a00000 {
			num-channels = <1>;
			#address-cells = <1>;
			#size-cells = <0>;
			csi_chan0: channel@0 {
				reg = <0>;
				status = "okay";
				ports {
					#address-cells = <1>;
					#size-cells = <0>;

					csi_chan0_port0: port@0 {
						status = "okay";
						reg = <0>;

						csi_in_0: endpoint@0 {
							status = "okay";
							port-index = <0>;
							bus-width = <2>;
							 remote-endpoint = <&sen_out_imx477_0>;
						};
					};

					 csi_chan0_port1: port@1 {
						status = "okay";
						reg = <1>;

						 csi_out_0: endpoint@1 {
							status = "okay";
							remote-endpoint = <&vi_in0>;
						};
					};
				};
			};

		};
	};


	tegra-camera-platform {
		compatible = "nvidia, tegra-camera-platform";
		num_csi_lanes = <16>;
		max_lane_speed = <1500000>;
		min_bits_per_pixel = <10>;
		/* max_pixel_rate = <222750>; */
		/* max_pixel_rate = <445500>; */
		max_pixel_rate = <750000>;
		
		vi_peak_byte_per_pixel = <2>;
		vi_bw_margin_pct = <25>;
		isp_peak_byte_per_pixel = <5>;
		isp_bw_margin_pct = <25>;
		
		
		modules {
			cam_module0: module0 {
				status = "okay";
				badge = "galen_front_IMX477";
				position = "front";
				orientation = "1";
				cam_module0_drivernode0: drivernode0 {
					status = "okay";
					pcl_id = "v4l2_sensor";
					devname = "imx477 2-0060";
					proc-device-tree = "/proc/device-tree/i2c@3180000/ub960@30/link@0/imx477@60";
				};
			};
		};
	};
		
		
	i2c@c240000 {

        /* I2C GPIO expander PCAL9539A (U37) */
        gpio_i2c_2_77: gpio@77 {
            status = "okay";
            compatible = "nxp,pca9539";
            reg = <0x77>;
            #gpio-cells = <2>;
            gpio-controller;
            vcc-supply = <&p2822_vdd_1v8_cvb>;
        };
    };

    fixed-regulators {
		
        /* 1.1V regulator (U21) */
        fpd_vdd_1v1: regulator@300 {
            status = "okay";
            reg = <0x12c>;
            regulator-name = "fpd-1v1";
            compatible = "regulator-fixed";
            regulator-always-on;
            regulator-min-microvolt = <1100000>;
            regulator-max-microvolt = <1100000>;
            vin-supply = <&p2822_vdd_1v8_cvb>;
        };
        /* J19: 1-2 = VDD_12V0, 2-3 = VDD_5V0 */
        vdd_in_fpd: regulator@301 {
            compatible = "regulator-fixed";
            reg = <301>;
            regulator-name = "vdd_in_fpd";
            regulator-min-microvolt = <12000000>;
            regulator-max-microvolt = <12000000>;
        };

        /* FPD-Link 0-3 regulator (U13) */
        fpd_vdd_0_3: regulator@302 {
            status = "okay";
            reg = <302>;
            regulator-name = "fpd_vdd_1_4";
            compatible = "regulator-fixed";
            enable-active-high;
            regulator-always-on;
            gpio = <&gpio_i2c_2_77 0 GPIO_ACTIVE_HIGH>;
        };
        /* FPD-Link 4-7 regulator (U16) */
        fpd_vdd_4_7: regulator@303 {
            status = "okay";
            reg = <303>;
            regulator-name = "fpd_vdd_5_8";
            compatible = "regulator-fixed";
            enable-active-high;
            regulator-always-on;
            gpio = <&gpio_i2c_2_77 3 GPIO_ACTIVE_HIGH>;
        };
        /* FPD-Link 8-11 regulator (U17) */
        fpd_vdd_8_11: regulator@304 {
            status = "okay";
            reg = <304>;
            regulator-name = "fpd_vdd_9_12";
            compatible = "regulator-fixed";
            enable-active-high;
            regulator-always-on;
            gpio = <&gpio_i2c_2_77 8 GPIO_ACTIVE_HIGH>;
        };
        /* FPD-Link 12-15 regulator (U20) */
        fpd_vdd_12_15: regulator@305 {
            status = "okay";
            reg = <305>;
            regulator-name = "fpd_vdd_13_16";
            compatible = "regulator-fixed";
            enable-active-high;
            regulator-always-on;
            gpio = <&gpio_i2c_2_77 11 GPIO_ACTIVE_HIGH>;
        };
    };
};

Output of media-ctl:

Media controller API version 5.10.192

Media device information
------------------------
driver          tegra-camrtc-ca
model           NVIDIA Tegra Video Input Device
serial          
bus info        
hw revision     0x3
driver version  5.10.192

Device topology
- entity 1: 13e10000.host1x:nvcsi@15a00000- (2 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev0
	pad0: Sink
		<- "imx477 2-0060":0 [ENABLED]
	pad1: Source
		-> "vi-output, imx477 2-0060":0 [ENABLED]

- entity 4: imx477 2-0060 (1 pad, 1 link)
            type V4L2 subdev subtype Sensor flags 0
            device node name /dev/v4l-subdev1
	pad0: Source
		[fmt:SRGGB10_1X10/3840x2160 field:none colorspace:srgb]
		-> "13e10000.host1x:nvcsi@15a00000-":0 [ENABLED]

- entity 6: vi-output, imx477 2-0060 (1 pad, 1 link)
            type Node subtype V4L flags 0
            device node name /dev/video0
	pad0: Sink
		<- "13e10000.host1x:nvcsi@15a00000-":1 [ENABLED]

I’ve already played with serdes_pix_clk_hz, num_lanes 2 or 4, tried to limit csi-tx-speed-mbps = <1600>; to 1200 to avoid deskew.

I tried to add :

if (priv->deserializer) {
    ub960_s_stream(priv->deserializer, priv->i2c_client, 1);
}

to imx477_start_streaming, I guess it sends ‘deskew’.

In this case the trace is slightly different, two new errors appear:
CaptureSchedule-3724 [000] … 118.681063: tegra_channel_set_stream: imx477 2-0060 : 0x1
kworker/0:5-551 [000] … 118.957505: rtcpu_nvcsi_intr: tstamp:4358710613 class:GLOBAL type:PHY_INTR0 phy:0 cil:0 st:0 vc:0 status:0x00000088
kworker/0:5-551 [000] … 118.957510: rtcpu_nvcsi_intr: tstamp:4358714669 class:GLOBAL type:PHY_INTR0 phy:0 cil:0 st:0 vc:0 status:0x01c00000
kworker/0:5-551 [000] … 118.957513: rtcpu_vinotify_event: tstamp:4359049772 cch:1 vi:0 tag:VIFALC_TDSTATE channel:0x0b frame:0 vi_tstamp:139488765024 data:0xcd9c550010000000
kworker/0:5-551 [000] … 118.957514: rtcpu_vinotify_event: tstamp:4359049943 cch:1 vi:0 tag:VIFALC_TDSTATE channel:0x0b frame:0 vi_tstamp:139488807808 data:0x0000000031000001
kworker/0:6-803 [000] … 119.013555: rtcpu_vinotify_event: tstamp:4360135132 cch:1 vi:0 tag:VIFALC_TDSTATE channel:0x0b frame:0 vi_tstamp:139522576608 data:0xcd9c520010000000
kworker/0:6-803 [000] … 119.013560: rtcpu_vinotify_event: tstamp:4360135299 cch:1 vi:0 tag:VIFALC_TDSTATE channel:0x0b frame:0 vi_tstamp:139522620096 data:0x0000000031000002

Kernel log messages related to ub953/960

Jul 22 16:14:12 kernel: [    6.088823] ub960 2-0030: probe enter
Jul 22 16:14:12 kernel: [    6.088836] ub960 2-0030: loading csi parameters
Jul 22 16:14:12 kernel: [    6.088848] ub960 2-0030: csi speed:1472-1664 lanes:4 continuous-clock:yes
Jul 22 16:14:12 kernel: [    6.094470] ub960 2-0030: loading fpdlink port parameters
Jul 22 16:14:12 kernel: [    6.094480] ub960 2-0030: Configure port 0
Jul 22 16:14:12 kernel: [    6.095171] ub960 2-0030: could not find regulator: avdd
Jul 22 16:14:12 kernel: [    6.372646] hub 1-0:1.0: USB hub found
Jul 22 16:14:12 kernel: [    6.376550] tegra_nvdisp_bandwidth_register_max_config: max config hubclk = 358400000 Hz
Jul 22 16:14:12 kernel: [    6.383051] hub 1-0:1.0: 4 ports detected
Jul 22 16:14:12 kernel: [    6.529335] hub 2-0:1.0: USB hub found
Jul 22 16:14:12 kernel: [    6.529367] hub 2-0:1.0: 4 ports detected
Jul 22 16:14:12 kernel: [    6.533388] ub960 2-0030: probe enter
Jul 22 16:14:12 kernel: [    6.539997] ub960 2-0030: loading csi parameters
Jul 22 16:14:12 kernel: [    6.550839] ub960 2-0030: csi speed:1472-1664 lanes:4 continuous-clock:yes
Jul 22 16:14:12 kernel: [    6.558795] ub960 2-0030: loading fpdlink port parameters
Jul 22 16:14:12 kernel: [    6.569489] ub960 2-0030: Configure port 0
Jul 22 16:14:12 kernel: [    6.588619] ub960 2-0030: using pdb_gpio for reset
Jul 22 16:14:12 kernel: [    6.597241] ub960 2-0030: err = 0 after regmap_bulk_read(UB960_REG_FPD3_RX_ID0)
Jul 22 16:14:12 kernel: [    6.597423] ub960 2-0030: revision mask id: 0x04
Jul 22 16:14:12 kernel: [    6.597552] ub960 2-0030: detected: DS90UB960-Q1 A0
Jul 22 16:14:12 kernel: [    6.597682] ub960 2-0030: using pdb_gpio for reset
Jul 22 16:14:12 kernel: [    6.606322] ub960 2-0030: err = 0 after regmap_bulk_read(UB960_REG_FPD3_RX_ID0)
Jul 22 16:14:12 kernel: [    7.811526] ub960 2-0030: STATUS: 81
Jul 22 16:14:12 kernel: [    7.811532] ub960 2-0030: HANDLE PORT 0
Jul 22 16:14:12 kernel: [    7.812220] ub960 2-0030: [0] RX_PORT_STS1: 13
Jul 22 16:14:12 kernel: [    7.812241] ub960 2-0030: [0] RX_PORT_STS2: 04
Jul 22 16:14:12 kernel: [    7.812245] ub960 2-0030: [0] CSI_RX_STS: 00
Jul 22 16:14:12 kernel: [    7.812251] ub960 2-0030: port->lock_work scheduled
Jul 22 16:14:12 kernel: [    8.063292] ub960 2-0030: [0] ub960_port_check_lock_sts enter
Jul 22 16:14:12 kernel: [    8.063973] ub960 2-0030: [0] RX_PORT_STS1: 03
Jul 22 16:14:12 kernel: [    8.063979] ub960 2-0030: [0] RX_PORT_STS2: 04
Jul 22 16:14:12 kernel: [    8.063983] ub960 2-0030: [0] CSI_RX_STS: 00
Jul 22 16:14:12 kernel: [    8.063988] ub960 2-0030: [0] CHECK LOCK: locked=1, port->locked=0, port->enabled=1, port->configured=1
Jul 22 16:14:12 kernel: [    8.063992] ub960 2-0030: [0] start port
Jul 22 16:14:12 kernel: [    8.064816] ub960 2-0030: [0] scan for child i2c nodes
Jul 22 16:14:12 kernel: [    8.064997] ub960 2-0030: [0] Found child node: ub953_0@40
Jul 22 16:14:12 kernel: [    8.066545] ub960 2-0030: [0] Create client for ub953_0@40
Jul 22 16:14:12 kernel: [    8.066551] ub960 2-0030: [0] Alias serializer from 18 to 40
Jul 22 16:14:12 kernel: [    8.066820] ub953 2-0040: probe
Jul 22 16:14:12 kernel: [    8.072303] ub953 2-0040: div-m-val=27 hs-clk-div=4 div-n-val=250 i2c-voltage-sel=0
Jul 22 16:14:12 kernel: [    8.073108] ub953 2-0040: No fsync gpio found
Jul 22 16:14:12 kernel: [    8.073127] ub953 2-0040: probe success
Jul 22 16:14:12 kernel: [    8.074546] ub960 2-0030: [0] Found child node: imx477@60
Jul 22 16:14:12 kernel: [    8.076072] ub960 2-0030: [0] Create client for imx477@60
Jul 22 16:14:12 kernel: [    8.076078] ub960 2-0030: [0] Alias #0 from 1a to 60
Jul 22 16:14:12 kernel: [    8.076393] ub960 2-0030: [0] probe success
Jul 22 16:14:12 kernel: [   10.627084] imx477 2-0060: imx477 found compatible ub960 1073741823
Jul 22 16:14:12 kernel: [   10.944263] tegra-camrtc-capture-vi tegra-capture-vi: subdev imx477 2-0060 bound
Jul 22 16:16:37 kernel: [  118.947567] ub960 2-0030: imx477 started a new stream; 1 active streams
Jul 22 16:16:39 kernel: [  121.474822] ub960 2-0030: imx477 stopped a stream; 0 active streams
Jul 22 16:16:39 kernel: [  121.474827] ub960 2-0030: imx477 notified end-of-stream, performing digital reset

related imx477

Jul 22 16:14:12 kernel: [    8.074546] ub960 2-0030: [0] Found child node: imx477@60
Jul 22 16:14:12 kernel: [    8.076072] ub960 2-0030: [0] Create client for imx477@60
Jul 22 16:14:12 kernel: [   10.626854] imx477 2-0060: probing v4l2 sensor at addr 0x60
Jul 22 16:14:12 kernel: [   10.626953] imx477 2-0060: mclk name not present, assume sensor driven externally
Jul 22 16:14:12 kernel: [   10.626960] imx477 2-0060: avdd, iovdd and/or dvdd reglrs. not present, assume sensor powered independently
Jul 22 16:14:12 kernel: [   10.627049] imx477 2-0060: tegracam sensor driver:imx477_v2.0.6
Jul 22 16:14:12 kernel: [   10.627084] imx477 2-0060: imx477 found compatible ub960 1073741823
Jul 22 16:14:12 kernel: [   10.627094] imx477 2-0060: deserializer present
Jul 22 16:14:12 kernel: [   10.627098] imx477 2-0060: imx477_power_on: power on
Jul 22 16:14:12 kernel: [   10.929060] imx477 2-0060: imx477_board_setup: invalid sensor model id: 77
Jul 22 16:14:12 kernel: [   10.944136] imx477 2-0060: imx477_power_off: power off
Jul 22 16:14:12 kernel: [   10.944263] tegra-camrtc-capture-vi tegra-capture-vi: subdev imx477 2-0060 bound
Jul 22 16:14:12 kernel: [   10.945414] imx477 2-0060: detected imx477 sensor
Jul 22 16:14:13 kernel: [   12.221015] imx477 2-0060: imx477_open:
Jul 22 16:14:58 kernel: [   20.554012] imx477 2-0060: imx477_open:
Jul 22 16:16:36 kernel: [  118.369208] imx477 2-0060: imx477_power_on: power on
Jul 22 16:16:36 kernel: [  118.681111] imx477 2-0060: imx477_set_mode:
Jul 22 16:16:37 kernel: [  118.939614] imx477 2-0060: imx477_set_gain: Setting gain control to: 16
Jul 22 16:16:37 kernel: [  118.939628] imx477 2-0060: imx477_set_gain: val: 16 (/16) [times], gain: 0
Jul 22 16:16:37 kernel: [  118.940815] imx477 2-0060: imx477_set_exposure: Setting exposure control to: 33330
Jul 22 16:16:37 kernel: [  118.940826] imx477 2-0060: imx477_set_exposure: val: 33330 [us], coarse_time: 2499 [lines]
Jul 22 16:16:37 kernel: [  118.942184] imx477 2-0060: imx477_set_frame_rate: Setting framerate control to: 30000000
Jul 22 16:16:37 kernel: [  118.942194] imx477 2-0060: imx477_set_frame_rate: val: 30000000e-6 [fps], frame_length: 2500 [lines]
Jul 22 16:16:37 kernel: [  118.943348] imx477 2-0060: imx477_start_streaming:
Jul 22 16:16:37 kernel: [  118.947567] ub960 2-0030: imx477 started a new stream; 1 active streams
Jul 22 16:16:39 kernel: [  121.474808] imx477 2-0060: imx477_stop_streaming:
Jul 22 16:16:39 kernel: [  121.474822] ub960 2-0030: imx477 stopped a stream; 0 active streams
Jul 22 16:16:39 kernel: [  121.474827] ub960 2-0030: imx477 notified end-of-stream, performing digital reset
Jul 22 16:16:39 kernel: [  121.491044] imx477 2-0060: imx477_power_off: power off

What is suspicious here is “imx477_board_setup: invalid sensor model id: 77” but then probe success.

Everything is overclocked.

Could you please give me a clue what I am doing wrong?

Thank you.

Hi Vladimir,

It looks like the camera driver has established I2C communications and UB953/960 probe correctly. Trace log with ub960_s_stream() call implemented shows LP sequence error detected on data lane but then successful DPHY deskew calibration done on data lane. After successful deskew calibration, the Jetson doesn’t receive any MIPI data and times out. FYI you can decode the trace logs by running nvcapture-status-decoder [trace log line] on the Jetson.

Looking at the Devicetree there is a discrepancy between ub960 csi-lane-count = <4>; and related vi port bus-width = <2>; as well as ub953/imx477 csi-lane-count and num_lanes. If this is based on NVIDIA’s IMX477 driver, it does read num_lanes in mode Devicetree and will automatically adjust. Can you try setting all ‘number of CSI lane’ properties to 4 and seeing if that resolves the issue?

Also, when this previously worked in JetPack 4.6.1, were you using D3 hardware then too?

Hi D3,

Thank you for quick feedback. After I changed all csi-lane-count/num_lanes/bus-width to 4 the trace logg is:

 kworker/3:13-211     [003] ....    75.027652: rtcpu_nvcsi_intr: tstamp:2986634589 class:GLOBAL type:PHY_INTR0 phy:0 cil:0 st:0 vc:0 status:0x00000088
    kworker/3:13-211     [003] ....    75.027658: rtcpu_nvcsi_intr: tstamp:2986634589 class:GLOBAL type:PHY_INTR0 phy:0 cil:1 st:0 vc:0 status:0x00000088
    kworker/3:13-211     [003] ....    75.027659: rtcpu_nvcsi_intr: tstamp:2986634589 class:CORRECTABLE_ERR type:PHY_INTR phy:0 cil:0 st:0 vc:0 status:0x00000088
    kworker/3:13-211     [003] ....    75.027660: rtcpu_nvcsi_intr: tstamp:2986634589 class:CORRECTABLE_ERR type:PHY_INTR phy:0 cil:1 st:0 vc:0 status:0x00000088
    kworker/3:13-211     [003] ....    75.027661: rtcpu_nvcsi_intr: tstamp:2986638948 class:GLOBAL type:PHY_INTR0 phy:0 cil:0 st:0 vc:0 status:0x0e000000
    kworker/3:13-211     [003] ....    75.027662: rtcpu_nvcsi_intr: tstamp:2986638948 class:GLOBAL type:PHY_INTR0 phy:0 cil:1 st:0 vc:0 status:0x06000000
    kworker/3:13-211     [003] ....    75.027663: rtcpu_nvcsi_intr: tstamp:2986638948 class:CORRECTABLE_ERR type:PHY_INTR phy:0 cil:0 st:0 vc:0 status:0x00000e00
    kworker/3:13-211     [003] ....    75.027665: rtcpu_nvcsi_intr: tstamp:2986638948 class:CORRECTABLE_ERR type:PHY_INTR phy:0 cil:1 st:0 vc:0 status:0x00000600
    kworker/3:13-211     [003] ....    75.027667: rtcpu_vinotify_event: tstamp:2986955673 cch:1 vi:0 tag:VIFALC_TDSTATE channel:0x0b frame:0 vi_tstamp:95581933312 data:0xcd9c550010000000
    kworker/3:13-211     [003] ....    75.027668: rtcpu_vinotify_event: tstamp:2986955843 cch:1 vi:0 tag:VIFALC_TDSTATE channel:0x0b frame:0 vi_tstamp:95581976096 data:0x0000000031000001
    kworker/3:13-211     [003] ....    75.083518: rtcpu_vinotify_event: tstamp:2988041064 cch:1 vi:0 tag:VIFALC_TDSTATE channel:0x0b frame:0 vi_tstamp:95615672992 data:0xcd9c520010000000
    kworker/3:13-211     [003] ....    75.083521: rtcpu_vinotify_event: tstamp:2988041234 cch:1 vi:0 tag:VIFALC_TDSTATE channel:0x0b frame:0 vi_tstamp:95615716448 data:0x0000000031000002

Maybe I need to adjust serdes_pix_clk_hz?

I also moved eeprom inside ub690_30_link0: link@0 section (according to schema in ub960.c)
another change: I uncommented ub953_gpio_0: ub953_gpio_0 section (in device tree above) and added
reset-gpios = <&ub953_gpio_0 0 GPIO_ACTIVE_HIGH>; to imx477_0: imx477@60 (according to D3 dirver for FPD Link / imx390)

Also, when this previously worked in JetPack 4.6.1, were you using D3 hardware then too?

I would say I use the same hardware units that worked well couple days ago under JP4.6.1

Yes, I think adjusting serdes_pix_clk_hz may help. We use 640000000 for our marketplace IMX390/ISX031/AR0234 cameras with the UB960.

Also, did you remove the ub960_s_stream() call in the driver? It should be present for sensors with >1.5Gbps data rate. Trace log is showing DPHY deskew calibration not complete on data lane

Could you please explain “for sensors with >1.5Gbps data rate”?

Does it mean if I use imx477 with 1920x1080x10bitx30fps = ~0.62Gbps mode I don’t need ‘deskew’, the same sensor with 4056x3040x10bitx30fps mode = ~3.7Gbps I must send ‘deskew’?
Does send/not send depends on number of CSI lines? or data rate in general is mandatory condition?

Today I discovered interesting fact that can be related to the issue. I am working with Arducam IMX477, I’ve already mentioned that camera reports weird “imx477_board_setup: invalid sensor model id: 77”, the revision of the camera unit is E (see photo). I have also Rev B and C. If I try B or C the camera probe fails with:

Jul 23 09:05:03 kernel: [   10.310025] imx477 2-0060: imx477_power_on: power on
Jul 23 09:05:03 kernel: [   10.610819] imx477 2-0060: imx477_board_setup: error during i2c read probe (-121)
Jul 23 09:05:03 kernel: [   10.611919] imx477 2-0060: imx477_power_off: power off
Jul 23 09:05:03 kernel: [   10.611954] imx477 2-0060: board setup failed
Jul 23 09:05:03 kernel: [   10.613164] imx477: probe of 2-0060 failed with error -121

So only Rev E probe is success but with message “imx477_board_setup: invalid sensor model id: 77”, other two revisions failed.

All these cameras work well on JP 4.6.1. Can the issue be related to some changes in camera gpio, i2c power supply?

Could you please explain “for sensors with >1.5Gbps data rate”?

Yes, in JetPack 5, NVIDIA implemented MIPI deskew requirement. The standard requires any sensor >1.5Gbps data rate must send initial skew calibration signal (periodic skew calibration optional). I’m not exactly sure how the Jetson determines if skew calibration is required, I think it is related to the serdes_pix_clk_hz/pix_clk_hz properties but not certain.

Can the issue be related to some changes in camera gpio, i2c power supply?

It is probably related to power or reset GPIO since -121 means device didn’t respond to I2C request. Also the IMX477 driver may have changed significantly b/w JetPack 4 and 5 so that is an avenue to look into.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.