FPD-Link III - IMX477 - Streaming 2 virtual channel simultaneously

[TobidieTopfpflanze]

Hello together,

Overview

I am currently trying to implement FPD-Link using UB960 Deserializer and the Rbpi IMX477. My end goal is to get 4 sensors on each 4-Lane CSI-Interface.
My current FPD-Link III & camera configuration only uses 2 of the 4 available lanes.

                           ┌─────────────────────┐
    ┌─────────────────┐    │       tdes960       │    ┌────────────────────┐  ┌───────────────┐
    │  CSI-Interface  ├────┤ CSI-0        Port-0 ├────┤ tser953        CSI ├──┤ Camera sensor │
    └─────────────────┘    │                     │    └────────────────────┘  └───────────────┘
    ┌─────────────────┐    │                     │    ┌────────────────────┐  ┌───────────────┐
    │  I2C            ├────┤ I2C-0        Port-1 ├────┤ tser953        CSI ├──┤ Camera sensor │
    └─────────────────┘    │                     │    └────────────────────┘  └───────────────┘
    ┌─────────────────┐    │                     │    ┌────────────────────┐  ┌───────────────┐
    │  CSI-Interface  ├────┤ CSI-1        Port-2 ├────┤ tser953        CSI ├──┤ Camera sensor │
    └─────────────────┘    │                     │    └────────────────────┘  └───────────────┘
    ┌─────────────────┐    │                     │    ┌────────────────────┐  ┌───────────────┐
    │  I2C            ├────┤ I2C-1        Port-3 ├────┤ tser953        CSI ├──┤ Camera sensor │
    └─────────────────┘    │                     │    └────────────────────┘  └───────────────┘
                           └─────────────────────┘

My Problem

Each stream for itself works fine, except if I try to stream 2 cameras simultaneously. I am currently not sure if this issue is still on the FPD-Link III or the Jetson side.

My FPD-Link III registers for the packet forwarding are currently configured to do Best-Effort forwarding (see 7.4.25.1 Best-Effort Round Robin CSI-2 Forwarding in the datasheet. Each RX port on the deserializer should also get its own virtual channel assigned (Port-0 = VC-0, Port-1 = VC-1, etc).

Things I have tried without success:

• boosting the clocks doesn’t make any difference

echo 1 > /sys/kernel/debug/bpmp/debug/clk/vi/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/isp/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/nvcsi/mrq_rate_locked
cat /sys/kernel/debug/bpmp/debug/clk/vi/max_rate |tee /sys/kernel/debug/bpmp/debug/clk/vi/rate
cat /sys/kernel/debug/bpmp/debug/clk/isp/max_rate | tee  /sys/kernel/debug/bpmp/debug/clk/isp/rate
cat /sys/kernel/debug/bpmp/debug/clk/nvcsi/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/nvcsi/rate

• Setting the serdes_pix_clk_hz breaks streaming completely
  • Output data rate = (sensor or deserializer pixel clock in hertz) * (bits per pixel) / (number of CSI lanes)
  • Output data rate = 1600mbps * 10 bpp / 2 lanes

Logs

test@test-desktop:~$ cat /etc/nv_tegra_release 
# R35 (release), REVISION: 5.0, GCID: 35550185, BOARD: t186ref, EABI: aarch64, DATE: Tue Feb 20 04:46:31 UTC 2024

NV/CSI logs

echo 1 > /sys/kernel/debug/tracing/tracing_on
echo 30720 > /sys/kernel/debug/tracing/buffer_size_kb
echo 1 > /sys/kernel/debug/tracing/events/tegra_rtcpu/enable
echo 1 > /sys/kernel/debug/tracing/events/freertos/enable
echo 2 > /sys/kernel/debug/camrtc/log-level
echo 1 > /sys/kernel/debug/tracing/events/camera_common/enable
echo > /sys/kernel/debug/tracing/trace
cat /sys/kernel/debug/tracing/trace

bug_logs.txt (1.9 MB)

Devicetree

Some missing values in CSI/VI and camera Devicetree are later on inserted in the top level dts file:

/* enable camera */
&rbpi_imx477_ab_p0 {
	status = "okay";
	reset-gpios = <&dummy_gpio 0 GPIO_ACTIVE_HIGH>;
};

&rbpi_imx477_ab_p1 {
	status = "okay";
	reset-gpios = <&dummy_gpio 1 GPIO_ACTIVE_HIGH>;
};

&rbpi_imx477_ab_p2 {
	status = "okay";
	reset-gpios = <&dummy_gpio 2 GPIO_ACTIVE_HIGH>;
};

&rbpi_imx477_ab_p3 {
	status = "okay";
	reset-gpios = <&dummy_gpio 3 GPIO_ACTIVE_HIGH>;
};

&rbpi_imx477_cd_p0 {
	status = "okay";
	reset-gpios = <&dummy_gpio 4 GPIO_ACTIVE_HIGH>;
};

&rbpi_imx477_cd_p1 {
	status = "okay";
	reset-gpios = <&dummy_gpio 5 GPIO_ACTIVE_HIGH>;
};

&rbpi_imx477_cd_p2 {
	status = "okay";
	reset-gpios = <&dummy_gpio 6 GPIO_ACTIVE_HIGH>;
};

&rbpi_imx477_cd_p3 {
	status = "okay";
	reset-gpios = <&dummy_gpio 7 GPIO_ACTIVE_HIGH>;
};

/* update modules */
&module0 {
	status = "okay";
	badge = "jakku_0_imx477";
};

&module0_node0 {
	status = "okay";
	devname = "imx477 9-001a";
	proc-device-tree = "/proc/device-tree/cam_i2cmux/i2c@0/rbpi_imx477_p0@1a";
};

&module1 {
	status = "okay";
	badge = "jakku_1_imx477";
};

&module1_node0 {
	status = "okay";
	devname = "imx477 9-001b";
	proc-device-tree = "/proc/device-tree/cam_i2cmux/i2c@0/rbpi_imx477_p1@1b";
};

&module2 {
	status = "okay";
	badge = "jakku_2_imx477";
};

&module2_node0 {
	status = "okay";
	devname = "imx477 9-001c";
	proc-device-tree = "/proc/device-tree/cam_i2cmux/i2c@0/rbpi_imx477_p2@1c";
};

&module3 {
	status = "okay";
	badge = "jakku_3_imx477";
};

&module3_node0 {
	status = "okay";
	devname = "imx477 9-001d";
	proc-device-tree = "/proc/device-tree/cam_i2cmux/i2c@0/rbpi_imx477_p3@1d";
};

&module4 {
	status = "okay";
	badge = "jakku_4_imx477";
};

&module4_node0 {
	status = "okay";
	devname = "imx477 10-001a";
	proc-device-tree = "/proc/device-tree/cam_i2cmux/i2c@1/rbpi_imx477_p0@1a";
};

&module5 {
	status = "okay";
	badge = "jakku_5_imx477";
};

&module5_node0 {
	status = "okay";
	devname = "imx477 10-001b";
	proc-device-tree = "/proc/device-tree/cam_i2cmux/i2c@1/rbpi_imx477_p1@1b";
};

&module6 {
	status = "okay";
	badge = "jakku_6_imx477";
};

&module6_node0 {
	status = "okay";
	devname = "imx477 10-001c";
	proc-device-tree = "/proc/device-tree/cam_i2cmux/i2c@1/rbpi_imx477_p2@1c";
};

&module7 {
	status = "okay";
	badge = "jakku_7_imx477";
};

&module7_node0 {
	status = "okay";
	devname = "imx477 10-001d";
	proc-device-tree = "/proc/device-tree/cam_i2cmux/i2c@1/rbpi_imx477_p3@1d";
};

/* update NV_CSI */
&cam_csi_in0 {
	status = "okay";
	remote-endpoint = <&imx477_vc_out0>;
};

&cam_csi_in1 {
	status = "okay";
	remote-endpoint = <&imx477_vc_out1>;
};

&cam_csi_in2 {
	status = "okay";
	remote-endpoint = <&imx477_vc_out2>;
};

&cam_csi_in3 {
	status = "okay";
	remote-endpoint = <&imx477_vc_out3>;
};

&cam_csi_in4 {
	status = "okay";
	remote-endpoint = <&imx477_vc_out4>;
};

&cam_csi_in5 {
	status = "okay";
	remote-endpoint = <&imx477_vc_out5>;
};

&cam_csi_in6 {
	status = "okay";
	remote-endpoint = <&imx477_vc_out6>;
};

&cam_csi_in7 {
	status = "okay";
	remote-endpoint = <&imx477_vc_out7>;
};

&cam_vi_in0 {
	status = "okay";
};

&cam_vi_in1 {
	status = "okay";
};

&cam_vi_in2 {
	status = "okay";
};

&cam_vi_in3 {
	status = "okay";
};

&cam_vi_in4 {
	status = "okay";
};

&cam_vi_in5 {
	status = "okay";
};

&cam_vi_in6 {
	status = "okay";
};

&cam_vi_in7 {
	status = "okay";
};

CSI/VI devicetree

/ {
    tegra-capture-vi  {
		num-channels = <8>;
		status = "okay";
		ports {
			status = "okay";
			#address-cells = <1>;
			#size-cells = <0>;
            /* CSI-AB */
            port@0 {
                status = "okay";
                reg = <0>;
                cam_vi_in0: endpoint {
                    status = "okay";
                    vc-id = <0>;
                    port-index = <0>;
                    bus-width = <2>;
                    remote-endpoint = <&cam_csi_out0>;
                };
            };
            port@1 {
                status = "okay";
                reg = <1>;
                cam_vi_in1: endpoint {
                    status = "okay";
                    vc-id = <1>;
                    port-index = <0>;
                    bus-width = <2>;
                    remote-endpoint = <&cam_csi_out1>;
                };
            };
            port@2 {
                status = "okay";
                reg = <2>;
                cam_vi_in2: endpoint {
                    status = "okay";
                    vc-id = <2>;
                    port-index = <0>;
                    bus-width = <2>;
                    remote-endpoint = <&cam_csi_out2>;
                };
            };
            port@3 {
                status = "okay";
                reg = <3>;
                cam_vi_in3: endpoint {
                    status = "okay";
                    vc-id = <3>;
                    port-index = <0>;
                    bus-width = <2>;
                    remote-endpoint = <&cam_csi_out3>;
                };
            };
            
            /* CSI-CD */
            port@4 {
                status = "okay";
                reg = <4>;
                cam_vi_in4: endpoint {
                    status = "okay";
                    vc-id = <0>;
                    port-index = <2>;
                    bus-width = <2>;
                    remote-endpoint = <&cam_csi_out4>;
                };
            };
            port@5 {
                status = "okay";
                reg = <5>;
                cam_vi_in5: endpoint {
                    status = "okay";
                    vc-id = <1>;
                    port-index = <2>;
                    bus-width = <2>;
                    remote-endpoint = <&cam_csi_out5>;
                };
            };
            port@6 {
                status = "okay";
                reg = <6>;
                cam_vi_in6: endpoint {
                    status = "okay";
                    vc-id = <2>;
                    port-index = <2>;
                    bus-width = <2>;
                    remote-endpoint = <&cam_csi_out6>;
                };
            };
            port@7 {
                status = "okay";
                reg = <7>;
                cam_vi_in7: endpoint {
                    status = "okay";
                    vc-id = <3>;
                    port-index = <2>;
                    bus-width = <2>;
                    remote-endpoint = <&cam_csi_out7>;
                };
            };
		};
	};

	host1x@13e00000 {

        nvcsi@15a00000 {
            status = "okay";
            num-channels = <8>;
            #address-cells = <1>;
            #size-cells = <0>;
            /* CSI-AB */
            channel@0 {
                status = "okay";
                reg = <0>;
                ports {
                    port@0 {
                        status = "okay";
                        reg = <0>;
                        cam_csi_in0: endpoint@0 {
                            status = "okay";
                            port-index = <0>;
                            bus-width = <2>;
                            // Insert cam out here
                        };
                    };
                    port@1 {
                        status = "okay";
                        reg = <1>;
                        cam_csi_out0: endpoint@1 {
                            status = "okay";
                            remote-endpoint = <&cam_vi_in0>;
                        };
                    };
                };
            };
            channel@1 {
                status = "okay";
                reg = <1>;
                ports {
                    port@0 {
                        status = "okay";
                        reg = <0>;
                        cam_csi_in1: endpoint@0 {
                            status = "okay";
                            port-index = <0>;
                            bus-width = <2>;
                            // Insert cam out here
                        };
                    };
                    port@1 {
                        status = "okay";
                        reg = <1>;
                        cam_csi_out1: endpoint@1 {
                            status = "okay";
                            remote-endpoint = <&cam_vi_in1>;
                        };
                    };
                };
            };
            channel@2 {
                status = "okay";
                reg = <2>;
                ports {
                    port@0 {
                        status = "okay";
                        reg = <0>;
                        cam_csi_in2: endpoint@0 {
                            status = "okay";
                            port-index = <0>;
                            bus-width = <2>;
                            // Insert cam out here
                        };
                    };
                    port@1 {
                        status = "okay";
                        reg = <1>;
                        cam_csi_out2: endpoint@1 {
                            status = "okay";
                            remote-endpoint = <&cam_vi_in2>;
                        };
                    };
                };
            };
            channel@3 {
                status = "okay";
                reg = <3>;
                ports {
                    port@0 {
                        status = "okay";
                        reg = <0>;
                        cam_csi_in3: endpoint@0 {
                            status = "okay";
                            port-index = <0>;
                            bus-width = <2>;
                            // Insert cam out here
                        };
                    };
                    port@1 {
                        status = "okay";
                        reg = <1>;
                        cam_csi_out3: endpoint@1 {
                            status = "okay";
                            remote-endpoint = <&cam_vi_in3>;
                        };
                    };
                };
            };

            /* CSI-EF */
            channel@4 {
                status = "okay";
                reg = <4>;
                ports {
                    #address-cells = <1>;
                    #size-cells = <0>;
                    port@0 {
                        reg = <0>;
                        cam_csi_in4: endpoint@0 {
                            port-index = <2>;
                            bus-width = <2>;
                            // Insert cam out here
                        };
                    };
                    port@1 {
                        reg = <1>;
                        cam_csi_out4: endpoint@1 {
                            remote-endpoint = <&cam_vi_in4>;
                        };
                    };
                };
            };
            channel@5 {
                status = "okay";
                reg = <5>;
                ports {
                    #address-cells = <1>;
                    #size-cells = <0>;
                    port@0 {
                        reg = <0>;
                        cam_csi_in5: endpoint@0 {
                            port-index = <2>;
                            bus-width = <2>;
                            // Insert cam out here
                        };
                    };
                    port@1 {
                        reg = <1>;
                        cam_csi_out5: endpoint@1 {
                            remote-endpoint = <&cam_vi_in5>;
                        };
                    };
                };
            };
            channel@6 {
                status = "okay";
                reg = <6>;
                ports {
                    #address-cells = <1>;
                    #size-cells = <0>;
                    port@0 {
                        reg = <0>;
                        cam_csi_in6: endpoint@0 {
                            port-index = <2>;
                            bus-width = <2>;
                            // Insert cam out here
                        };
                    };
                    port@1 {
                        reg = <1>;
                        cam_csi_out6: endpoint@1 {
                            remote-endpoint = <&cam_vi_in6>;
                        };
                    };
                };
            };
            channel@7 {
                status = "okay";
                reg = <7>;
                ports {
                    #address-cells = <1>;
                    #size-cells = <0>;
                    port@0 {
                        reg = <0>;
                        cam_csi_in7: endpoint@0 {
                            port-index = <2>;
                            bus-width = <2>;
                            // Insert cam out here
                        };
                    };
                    port@1 {
                        reg = <1>;
                        cam_csi_out7: endpoint@1 {
                            remote-endpoint = <&cam_vi_in7>;
                        };
                    };
                };
            };
        };
	};

	tcp: tegra-camera-platform {
		compatible = "nvidia, tegra-camera-platform";
		/**
		* Physical settings to calculate max ISO BW
		*
		* num_csi_lanes = <>;
		* Total number of CSI lanes when all cameras are active
		*
		* max_lane_speed = <>;
		* Max lane speed in Kbit/s
		*
		* min_bits_per_pixel = <>;
		* Min bits per pixel
		*
		* vi_peak_byte_per_pixel = <>;
		* Max byte per pixel for the VI ISO case
		*
		* vi_bw_margin_pct = <>;
		* Vi bandwidth margin in percentage
		*
		* max_pixel_rate = <>;
		* Max pixel rate in Kpixel/s for the ISP ISO case
		*
		* isp_peak_byte_per_pixel = <>;
		* Max byte per pixel for the ISP ISO case
		*
		* isp_bw_margin_pct = <>;
		* Isp bandwidth margin in percentage
		*/
		num_csi_lanes = <4>;
		max_lane_speed = <1600000>;
		min_bits_per_pixel = <10>;
		vi_peak_byte_per_pixel = <2>;
		vi_bw_margin_pct = <25>;
		max_pixel_rate = <7500000>;
		isp_peak_byte_per_pixel = <5>;
		isp_bw_margin_pct = <25>;

		modules {
			module0: module0 {
				status = "okay";
				position = "0";
				orientation = "1";
				module0_node0: drivernode0 {
					status = "okay";
					pcl_id = "v4l2_sensor";
				};
			};
			module1: module1 {
				status = "okay";
				position = "1";
				orientation = "1";
				module1_node0: drivernode0 {
					status = "okay";
					pcl_id = "v4l2_sensor";
				};
			};
			module2: module2 {
				status = "okay";
				position = "2";
				orientation = "1";
				module2_node0: drivernode0 {
					status = "okay";
					pcl_id = "v4l2_sensor";
				};
			};
			module3: module3 {
				status = "okay";
				position = "3";
				orientation = "1";
				module3_node0: drivernode0 {
					status = "okay";
					pcl_id = "v4l2_sensor";
				};
			};
			module4: module4 {
				position = "4";
				orientation = "1";
				module4_node0: drivernode0 {
					status = "okay";
					pcl_id = "v4l2_sensor";
				};
			};
			module5: module5 {
				position = "5";
				orientation = "1";
				module5_node0: drivernode0 {
					status = "okay";
					pcl_id = "v4l2_sensor";
				};
			};
			module6: module6 {
				position = "6";
				orientation = "1";
				module6_node0: drivernode0 {
					status = "okay";
					pcl_id = "v4l2_sensor";
				};
			};
			module7: module7 {
				position = "7";
				orientation = "1";
				module7_node0: drivernode0 {
					status = "okay";
					pcl_id = "v4l2_sensor";
				};
			};
		};
	};
};

IMX477 devicetree

/ {
	cam_i2cmux {
		i2c@0 { // AB
			rbpi_imx477_ab_p0: rbpi_imx477_p0@1a {
				status = "disabled";

				compatible = "ridgerun,imx477";
				/* I2C device address */
				reg = <0x1a>;
				reset-gpios = <&dummy_gpio 0 GPIO_ACTIVE_HIGH>;

				/* V4L2 device node location */
				devnode = "video0";

				/* Physical dimensions of sensor */
				physical_w = "3.680";
				physical_h = "2.760";

				sensor_model = "imx477";

				use_sensor_mode_id = "true";

				mode0 { /* IMX477_MODE_3840x2160 */
					mclk_khz = "24000";
					num_lanes = "2";
					tegra_sinterface = "serial_a";
					vc_id = "0";
					phy_mode = "DPHY";
					discontinuous_clk = "no";
					dpcm_enable = "false";
					cil_settletime = "0";
					lane_polarity = "6";

					active_w = "3840";
					active_h = "2160";
					mode_type = "bayer";
					pixel_phase = "rggb";
					csi_pixel_bit_depth = "10";
					readout_orientation = "90";
					line_length = "5832";
					inherent_gain = "1";
					mclk_multiplier = "80";
					pix_clk_hz = "300000000";
					// serdes_pix_clk_hz = "600000000";

					gain_factor = "16";
					framerate_factor = "1000000";
					exposure_factor = "1000000";
					min_gain_val = "16"; /* 1.00x */
					max_gain_val = "356"; /* 22x */
					step_gain_val = "1";
					default_gain = "16"; /* 1.00x */
					min_hdr_ratio = "1";
					max_hdr_ratio = "1";
					min_framerate = "2000000"; /* 2.0 fps */
					max_framerate = "30000000"; /* 30.0 fps */
					step_framerate = "1";
					default_framerate = "30000000"; /* 30.0 fps */
					min_exp_time = "13"; /* us */
					max_exp_time = "683709"; /* us */
					step_exp_time = "1";
					default_exp_time = "2495"; /* us */

					embedded_metadata_height = "2";
				};
				mode1 { /* IMX477_MODE_1920X1080 */
					mclk_khz = "24000";
					num_lanes = "2";
					tegra_sinterface = "serial_a";
					vc_id = "0";
					phy_mode = "DPHY";
					discontinuous_clk = "no";
					dpcm_enable = "false";
					cil_settletime = "0";
					lane_polarity = "6";

					active_w = "1920";
					active_h = "1080";
					mode_type = "bayer";
					pixel_phase = "rggb";
					csi_pixel_bit_depth = "10";
					readout_orientation = "90";
					line_length = "3076";
					inherent_gain = "1";
					mclk_multiplier = "80";
					pix_clk_hz = "300000000";
					// serdes_pix_clk_hz = "600000000";

					gain_factor = "16";
					framerate_factor = "1000000";
					exposure_factor = "1000000";
					min_gain_val = "16"; /* 1.00x */
					max_gain_val = "356"; /* 22x */
					step_gain_val = "1";
					default_gain = "16"; /* 1.00x */
					min_hdr_ratio = "1";
					max_hdr_ratio = "1";
					min_framerate = "2000000"; /* 2.0 fps */
					max_framerate = "60000000"; /* 60.0 fps */
					step_framerate = "1";
					default_framerate = "60000000"; /* 60.0 fps */
					min_exp_time = "13"; /* us */
					max_exp_time = "683709"; /* us */
					step_exp_time = "1";
					default_exp_time = "2495"; /* us */

					embedded_metadata_height = "2";
				};
				ports {
					status="okay";
					#address-cells = <1>;
					#size-cells = <0>;
					port@0 {
						status="okay";
						reg = <0>;
						imx477_vc_out0: endpoint {
							status="okay";
							vc-id = <0>;
							port-index = <0>;
							bus-width = <2>;
							remote-endpoint = <&cam_csi_in0>;
						};
					};
				};
			};

			rbpi_imx477_ab_p1: rbpi_imx477_p1@1b {
				status = "disabled";

				compatible = "ridgerun,imx477";
				/* I2C device address */
				reg = <0x1b>;
				reset-gpios = <&dummy_gpio 1 GPIO_ACTIVE_HIGH>;

				/* V4L2 device node location */
				devnode = "video1";

				/* Physical dimensions of sensor */
				physical_w = "3.680";
				physical_h = "2.760";

				sensor_model = "imx477";

				use_sensor_mode_id = "true";

				mode0 { /* IMX477_MODE_3840x2160 */
					mclk_khz = "24000";
					num_lanes = "2";
					tegra_sinterface = "serial_a";
					vc_id = "1";
					phy_mode = "DPHY";
					discontinuous_clk = "no";
					dpcm_enable = "false";
					cil_settletime = "0";
					lane_polarity = "6";

					active_w = "3840";
					active_h = "2160";
					mode_type = "bayer";
					pixel_phase = "rggb";
					csi_pixel_bit_depth = "10";
					readout_orientation = "90";
					line_length = "5832";
					inherent_gain = "1";
					mclk_multiplier = "80";
					pix_clk_hz = "300000000";
					// serdes_pix_clk_hz = "600000000";

					gain_factor = "16";
					framerate_factor = "1000000";
					exposure_factor = "1000000";
					min_gain_val = "16"; /* 1.00x */
					max_gain_val = "356"; /* 22x */
					step_gain_val = "1";
					default_gain = "16"; /* 1.00x */
					min_hdr_ratio = "1";
					max_hdr_ratio = "1";
					min_framerate = "2000000"; /* 2.0 fps */
					max_framerate = "30000000"; /* 30.0 fps */
					step_framerate = "1";
					default_framerate = "30000000"; /* 30.0 fps */
					min_exp_time = "13"; /* us */
					max_exp_time = "683709"; /* us */
					step_exp_time = "1";
					default_exp_time = "2495"; /* us */

					embedded_metadata_height = "2";
				};
				mode1 { /* IMX477_MODE_1920X1080 */
					mclk_khz = "24000";
					num_lanes = "2";
					tegra_sinterface = "serial_a";
					vc_id = "1";
					phy_mode = "DPHY";
					discontinuous_clk = "no";
					dpcm_enable = "false";
					cil_settletime = "0";
					lane_polarity = "6";

					active_w = "1920";
					active_h = "1080";
					mode_type = "bayer";
					pixel_phase = "rggb";
					csi_pixel_bit_depth = "10";
					readout_orientation = "90";
					line_length = "3076";
					inherent_gain = "1";
					mclk_multiplier = "80";
					pix_clk_hz = "300000000";
					// serdes_pix_clk_hz = "600000000";

					gain_factor = "16";
					framerate_factor = "1000000";
					exposure_factor = "1000000";
					min_gain_val = "16"; /* 1.00x */
					max_gain_val = "356"; /* 22x */
					step_gain_val = "1";
					default_gain = "16"; /* 1.00x */
					min_hdr_ratio = "1";
					max_hdr_ratio = "1";
					min_framerate = "2000000"; /* 2.0 fps */
					max_framerate = "60000000"; /* 60.0 fps */
					step_framerate = "1";
					default_framerate = "60000000"; /* 60.0 fps */
					min_exp_time = "13"; /* us */
					max_exp_time = "683709"; /* us */
					step_exp_time = "1";
					default_exp_time = "2495"; /* us */

					embedded_metadata_height = "2";
				};
				ports {
					status="okay";
					#address-cells = <1>;
					#size-cells = <0>;
					port@0 {
						status="okay";
						reg = <0>;
						imx477_vc_out1: endpoint {
							status="okay";
							vc-id = <1>;
							port-index = <0>;
							bus-width = <2>;
							remote-endpoint = <&cam_csi_in1>;
						};
					};
				};
			};

			rbpi_imx477_ab_p2: rbpi_imx477_p2@1c {
				status = "disabled";

				compatible = "ridgerun,imx477";
				/* I2C device address */
				reg = <0x1c>;
				reset-gpios = <&dummy_gpio 2 GPIO_ACTIVE_HIGH>;

				/* V4L2 device node location */
				devnode = "video2";

				/* Physical dimensions of sensor */
				physical_w = "3.680";
				physical_h = "2.760";

				sensor_model = "imx477";

				use_sensor_mode_id = "true";

				mode0 { /* IMX477_MODE_3840x2160 */
					mclk_khz = "24000";
					num_lanes = "2";
					tegra_sinterface = "serial_a";
					vc_id = "2";
					phy_mode = "DPHY";
					discontinuous_clk = "no";
					dpcm_enable = "false";
					cil_settletime = "0";
					lane_polarity = "6";

					active_w = "3840";
					active_h = "2160";
					mode_type = "bayer";
					pixel_phase = "rggb";
					csi_pixel_bit_depth = "10";
					readout_orientation = "90";
					line_length = "5832";
					inherent_gain = "1";
					mclk_multiplier = "80";
					pix_clk_hz = "300000000";
					// serdes_pix_clk_hz = "600000000";

					gain_factor = "16";
					framerate_factor = "1000000";
					exposure_factor = "1000000";
					min_gain_val = "16"; /* 1.00x */
					max_gain_val = "356"; /* 22x */
					step_gain_val = "1";
					default_gain = "16"; /* 1.00x */
					min_hdr_ratio = "1";
					max_hdr_ratio = "1";
					min_framerate = "2000000"; /* 2.0 fps */
					max_framerate = "30000000"; /* 30.0 fps */
					step_framerate = "1";
					default_framerate = "30000000"; /* 30.0 fps */
					min_exp_time = "13"; /* us */
					max_exp_time = "683709"; /* us */
					step_exp_time = "1";
					default_exp_time = "2495"; /* us */

					embedded_metadata_height = "2";
				};
				mode1 { /* IMX477_MODE_1920X1080 */
					mclk_khz = "24000";
					num_lanes = "2";
					tegra_sinterface = "serial_a";
					vc_id = "2";
					phy_mode = "DPHY";
					discontinuous_clk = "no";
					dpcm_enable = "false";
					cil_settletime = "0";
					lane_polarity = "6";

					active_w = "1920";
					active_h = "1080";
					mode_type = "bayer";
					pixel_phase = "rggb";
					csi_pixel_bit_depth = "10";
					readout_orientation = "90";
					line_length = "3076";
					inherent_gain = "1";
					mclk_multiplier = "80";
					pix_clk_hz = "300000000";
					// serdes_pix_clk_hz = "600000000";

					gain_factor = "16";
					framerate_factor = "1000000";
					exposure_factor = "1000000";
					min_gain_val = "16"; /* 1.00x */
					max_gain_val = "356"; /* 22x */
					step_gain_val = "1";
					default_gain = "16"; /* 1.00x */
					min_hdr_ratio = "1";
					max_hdr_ratio = "1";
					min_framerate = "2000000"; /* 2.0 fps */
					max_framerate = "60000000"; /* 60.0 fps */
					step_framerate = "1";
					default_framerate = "60000000"; /* 60.0 fps */
					min_exp_time = "13"; /* us */
					max_exp_time = "683709"; /* us */
					step_exp_time = "1";
					default_exp_time = "2495"; /* us */

					embedded_metadata_height = "2";
				};
				ports {
					status="okay";
					#address-cells = <1>;
					#size-cells = <0>;
					port@0 {
						status="okay";
						reg = <0>;
						imx477_vc_out2: endpoint {
							status="okay";
							vc-id = <2>;
							port-index = <0>;
							bus-width = <2>;
							remote-endpoint = <&cam_csi_in2>;
						};
					};
				};
			};

			rbpi_imx477_ab_p3: rbpi_imx477_p3@1d {
				status = "disabled";

				compatible = "ridgerun,imx477";
				/* I2C device address */
				reg = <0x1d>;
				reset-gpios = <&dummy_gpio 3 GPIO_ACTIVE_HIGH>;

				/* V4L2 device node location */
				devnode = "video3";

				/* Physical dimensions of sensor */
				physical_w = "3.680";
				physical_h = "2.760";

				sensor_model = "imx477";

				use_sensor_mode_id = "true";

				mode0 { /* IMX477_MODE_3840x2160 */
					mclk_khz = "24000";
					num_lanes = "3";
					tegra_sinterface = "serial_a";
					vc_id = "2";
					phy_mode = "DPHY";
					discontinuous_clk = "no";
					dpcm_enable = "false";
					cil_settletime = "0";
					lane_polarity = "6";

					active_w = "3840";
					active_h = "2160";
					mode_type = "bayer";
					pixel_phase = "rggb";
					csi_pixel_bit_depth = "10";
					readout_orientation = "90";
					line_length = "5832";
					inherent_gain = "1";
					mclk_multiplier = "80";
					pix_clk_hz = "300000000";
					// serdes_pix_clk_hz = "600000000";

					gain_factor = "16";
					framerate_factor = "1000000";
					exposure_factor = "1000000";
					min_gain_val = "16"; /* 1.00x */
					max_gain_val = "356"; /* 22x */
					step_gain_val = "1";
					default_gain = "16"; /* 1.00x */
					min_hdr_ratio = "1";
					max_hdr_ratio = "1";
					min_framerate = "2000000"; /* 2.0 fps */
					max_framerate = "30000000"; /* 30.0 fps */
					step_framerate = "1";
					default_framerate = "30000000"; /* 30.0 fps */
					min_exp_time = "13"; /* us */
					max_exp_time = "683709"; /* us */
					step_exp_time = "1";
					default_exp_time = "2495"; /* us */

					embedded_metadata_height = "2";
				};
				mode1 { /* IMX477_MODE_1920X1080 */
					mclk_khz = "24000";
					num_lanes = "2";
					tegra_sinterface = "serial_a";
					vc_id = "3";
					phy_mode = "DPHY";
					discontinuous_clk = "no";
					dpcm_enable = "false";
					cil_settletime = "0";
					lane_polarity = "6";

					active_w = "1920";
					active_h = "1080";
					mode_type = "bayer";
					pixel_phase = "rggb";
					csi_pixel_bit_depth = "10";
					readout_orientation = "90";
					line_length = "3076";
					inherent_gain = "1";
					mclk_multiplier = "80";
					pix_clk_hz = "300000000";
					// serdes_pix_clk_hz = "600000000";

					gain_factor = "16";
					framerate_factor = "1000000";
					exposure_factor = "1000000";
					min_gain_val = "16"; /* 1.00x */
					max_gain_val = "356"; /* 22x */
					step_gain_val = "1";
					default_gain = "16"; /* 1.00x */
					min_hdr_ratio = "1";
					max_hdr_ratio = "1";
					min_framerate = "2000000"; /* 2.0 fps */
					max_framerate = "60000000"; /* 60.0 fps */
					step_framerate = "1";
					default_framerate = "60000000"; /* 60.0 fps */
					min_exp_time = "13"; /* us */
					max_exp_time = "683709"; /* us */
					step_exp_time = "1";
					default_exp_time = "2495"; /* us */

					embedded_metadata_height = "2";
				};
				ports {
					status="okay";
					#address-cells = <1>;
					#size-cells = <0>;
					port@0 {
						status="okay";
						reg = <0>;
						imx477_vc_out3: endpoint {
							status="okay";
							vc-id = <3>;
							port-index = <0>;
							bus-width = <2>;
							remote-endpoint = <&cam_csi_in3>;
						};
					};
				};
			};
		};

		i2c@1 { // CD
			rbpi_imx477_cd_p0: rbpi_imx477_p0@1a {
				status = "disabled";

				compatible = "ridgerun,imx477";
				/* I2C device address */
				reg = <0x1a>;
				reset-gpios = <&dummy_gpio 4 GPIO_ACTIVE_HIGH>;	

				/* V4L2 device node location */
				devnode = "video4";

				/* Physical dimensions of sensor */
				physical_w = "3.680";
				physical_h = "2.760";

				sensor_model = "imx477";

				use_sensor_mode_id = "true";

				mode0 { /* IMX477_MODE_3840x2160 */
					mclk_khz = "24000";
					num_lanes = "2";
					tegra_sinterface = "serial_c";
					vc_id = "0";
					phy_mode = "DPHY";
					discontinuous_clk = "no";
					dpcm_enable = "false";
					cil_settletime = "0";
					lane_polarity = "0";

					active_w = "3840";
					active_h = "2160";
					mode_type = "bayer";
					pixel_phase = "rggb";
					csi_pixel_bit_depth = "10";
					readout_orientation = "90";
					line_length = "5832";
					inherent_gain = "1";
					mclk_multiplier = "80";
					pix_clk_hz = "300000000";
					// serdes_pix_clk_hz = "600000000";

					gain_factor = "16";
					framerate_factor = "1000000";
					exposure_factor = "1000000";
					min_gain_val = "16"; /* 1.00x */
					max_gain_val = "356"; /* 22x */
					step_gain_val = "1";
					default_gain = "16"; /* 1.00x */
					min_hdr_ratio = "1";
					max_hdr_ratio = "1";
					min_framerate = "2000000"; /* 2.0 fps */
					max_framerate = "30000000"; /* 30.0 fps */
					step_framerate = "1";
					default_framerate = "30000000"; /* 30.0 fps */
					min_exp_time = "13"; /* us */
					max_exp_time = "683709"; /* us */
					step_exp_time = "1";
					default_exp_time = "2495"; /* us */

					embedded_metadata_height = "2";
				};
				mode1 { /* IMX477_MODE_1920X1080 */
					mclk_khz = "24000";
					num_lanes = "2";
					tegra_sinterface = "serial_c";
					vc_id = "0";
					phy_mode = "DPHY";
					discontinuous_clk = "no";
					dpcm_enable = "false";
					cil_settletime = "0";
					lane_polarity = "0";

					active_w = "1920";
					active_h = "1080";
					mode_type = "bayer";
					pixel_phase = "rggb";
					csi_pixel_bit_depth = "10";
					readout_orientation = "90";
					line_length = "3076";
					inherent_gain = "1";
					mclk_multiplier = "80";
					pix_clk_hz = "300000000";
					// serdes_pix_clk_hz = "600000000";

					gain_factor = "16";
					framerate_factor = "1000000";
					exposure_factor = "1000000";
					min_gain_val = "16"; /* 1.00x */
					max_gain_val = "356"; /* 22x */
					step_gain_val = "1";
					default_gain = "16"; /* 1.00x */
					min_hdr_ratio = "1";
					max_hdr_ratio = "1";
					min_framerate = "2000000"; /* 2.0 fps */
					max_framerate = "60000000"; /* 60.0 fps */
					step_framerate = "1";
					default_framerate = "60000000"; /* 60.0 fps */
					min_exp_time = "13"; /* us */
					max_exp_time = "683709"; /* us */
					step_exp_time = "1";
					default_exp_time = "2495"; /* us */

					embedded_metadata_height = "2";
				};

				ports {
					status="okay";
					#address-cells = <1>;
					#size-cells = <0>;
					port@0 {
						status="okay";
						reg = <0>;
						imx477_vc_out4: endpoint {
							status="okay";
							vc-id = <0>;
							port-index = <2>;
							bus-width = <2>;
							remote-endpoint = <&cam_csi_in4>;
						};
					};
				};
			};

			rbpi_imx477_cd_p1: rbpi_imx477_p1@1b {
				status = "disabled";

				compatible = "ridgerun,imx477";
				/* I2C device address */
				reg = <0x1b>;
				reset-gpios = <&dummy_gpio 5 GPIO_ACTIVE_HIGH>;	

				/* V4L2 device node location */
				devnode = "video5";

				/* Physical dimensions of sensor */
				physical_w = "3.680";
				physical_h = "2.760";

				sensor_model = "imx477";

				use_sensor_mode_id = "true";

				mode0 { /* IMX477_MODE_3840x2160 */
					mclk_khz = "24000";
					num_lanes = "2";
					tegra_sinterface = "serial_c";
					vc_id = "1";
					phy_mode = "DPHY";
					discontinuous_clk = "no";
					dpcm_enable = "false";
					cil_settletime = "0";
					lane_polarity = "0";

					active_w = "3840";
					active_h = "2160";
					mode_type = "bayer";
					pixel_phase = "rggb";
					csi_pixel_bit_depth = "10";
					readout_orientation = "90";
					line_length = "5832";
					inherent_gain = "1";
					mclk_multiplier = "80";
					pix_clk_hz = "300000000";
					// serdes_pix_clk_hz = "600000000";

					gain_factor = "16";
					framerate_factor = "1000000";
					exposure_factor = "1000000";
					min_gain_val = "16"; /* 1.00x */
					max_gain_val = "356"; /* 22x */
					step_gain_val = "1";
					default_gain = "16"; /* 1.00x */
					min_hdr_ratio = "1";
					max_hdr_ratio = "1";
					min_framerate = "2000000"; /* 2.0 fps */
					max_framerate = "30000000"; /* 30.0 fps */
					step_framerate = "1";
					default_framerate = "30000000"; /* 30.0 fps */
					min_exp_time = "13"; /* us */
					max_exp_time = "683709"; /* us */
					step_exp_time = "1";
					default_exp_time = "2495"; /* us */

					embedded_metadata_height = "2";
				};
				mode1 { /* IMX477_MODE_1920X1080 */
					mclk_khz = "24000";
					num_lanes = "2";
					tegra_sinterface = "serial_c";
					vc_id = "1";
					phy_mode = "DPHY";
					discontinuous_clk = "no";
					dpcm_enable = "false";
					cil_settletime = "0";
					lane_polarity = "0";

					active_w = "1920";
					active_h = "1080";
					mode_type = "bayer";
					pixel_phase = "rggb";
					csi_pixel_bit_depth = "10";
					readout_orientation = "90";
					line_length = "3076";
					inherent_gain = "1";
					mclk_multiplier = "80";
					pix_clk_hz = "300000000";
					// serdes_pix_clk_hz = "600000000";

					gain_factor = "16";
					framerate_factor = "1000000";
					exposure_factor = "1000000";
					min_gain_val = "16"; /* 1.00x */
					max_gain_val = "356"; /* 22x */
					step_gain_val = "1";
					default_gain = "16"; /* 1.00x */
					min_hdr_ratio = "1";
					max_hdr_ratio = "1";
					min_framerate = "2000000"; /* 2.0 fps */
					max_framerate = "60000000"; /* 60.0 fps */
					step_framerate = "1";
					default_framerate = "60000000"; /* 60.0 fps */
					min_exp_time = "13"; /* us */
					max_exp_time = "683709"; /* us */
					step_exp_time = "1";
					default_exp_time = "2495"; /* us */

					embedded_metadata_height = "2";
				};

				ports {
					status="okay";
					#address-cells = <1>;
					#size-cells = <0>;
					port@0 {
						status="okay";
						reg = <0>;
						imx477_vc_out5: endpoint {
							status="okay";
							vc-id = <1>;
							port-index = <2>;
							bus-width = <2>;
							remote-endpoint = <&cam_csi_in5>;
						};
					};
				};
			};
			
			rbpi_imx477_cd_p2: rbpi_imx477_p2@1c {
				status = "disabled";

				compatible = "ridgerun,imx477";
				/* I2C device address */
				reg = <0x1c>;
				reset-gpios = <&dummy_gpio 6 GPIO_ACTIVE_HIGH>;	

				/* V4L2 device node location */
				devnode = "video6";

				/* Physical dimensions of sensor */
				physical_w = "3.680";
				physical_h = "2.760";

				sensor_model = "imx477";

				use_sensor_mode_id = "true";

				mode0 { /* IMX477_MODE_3840x2160 */
					mclk_khz = "24000";
					num_lanes = "2";
					tegra_sinterface = "serial_c";
					vc_id = "2";
					phy_mode = "DPHY";
					discontinuous_clk = "no";
					dpcm_enable = "false";
					cil_settletime = "0";
					lane_polarity = "0";

					active_w = "3840";
					active_h = "2160";
					mode_type = "bayer";
					pixel_phase = "rggb";
					csi_pixel_bit_depth = "10";
					readout_orientation = "90";
					line_length = "5832";
					inherent_gain = "1";
					mclk_multiplier = "80";
					pix_clk_hz = "300000000";
					// serdes_pix_clk_hz = "600000000";

					gain_factor = "16";
					framerate_factor = "1000000";
					exposure_factor = "1000000";
					min_gain_val = "16"; /* 1.00x */
					max_gain_val = "356"; /* 22x */
					step_gain_val = "1";
					default_gain = "16"; /* 1.00x */
					min_hdr_ratio = "1";
					max_hdr_ratio = "1";
					min_framerate = "2000000"; /* 2.0 fps */
					max_framerate = "30000000"; /* 30.0 fps */
					step_framerate = "1";
					default_framerate = "30000000"; /* 30.0 fps */
					min_exp_time = "13"; /* us */
					max_exp_time = "683709"; /* us */
					step_exp_time = "1";
					default_exp_time = "2495"; /* us */

					embedded_metadata_height = "2";
				};
				mode1 { /* IMX477_MODE_1920X1080 */
					mclk_khz = "24000";
					num_lanes = "2";
					tegra_sinterface = "serial_c";
					vc_id = "2";
					phy_mode = "DPHY";
					discontinuous_clk = "no";
					dpcm_enable = "false";
					cil_settletime = "0";
					lane_polarity = "0";

					active_w = "1920";
					active_h = "1080";
					mode_type = "bayer";
					pixel_phase = "rggb";
					csi_pixel_bit_depth = "10";
					readout_orientation = "90";
					line_length = "3076";
					inherent_gain = "1";
					mclk_multiplier = "80";
					pix_clk_hz = "300000000";
					// serdes_pix_clk_hz = "600000000";

					gain_factor = "16";
					framerate_factor = "1000000";
					exposure_factor = "1000000";
					min_gain_val = "16"; /* 1.00x */
					max_gain_val = "356"; /* 22x */
					step_gain_val = "1";
					default_gain = "16"; /* 1.00x */
					min_hdr_ratio = "1";
					max_hdr_ratio = "1";
					min_framerate = "2000000"; /* 2.0 fps */
					max_framerate = "60000000"; /* 60.0 fps */
					step_framerate = "1";
					default_framerate = "60000000"; /* 60.0 fps */
					min_exp_time = "13"; /* us */
					max_exp_time = "683709"; /* us */
					step_exp_time = "1";
					default_exp_time = "2495"; /* us */

					embedded_metadata_height = "2";
				};

				ports {
					status="okay";
					#address-cells = <1>;
					#size-cells = <0>;
					port@0 {
						status="okay";
						reg = <0>;
						imx477_vc_out6: endpoint {
							status="okay";
							vc-id = <2>;
							port-index = <2>;
							bus-width = <2>;
							remote-endpoint = <&cam_csi_in6>;
						};
					};
				};
			};
			
			rbpi_imx477_cd_p3: rbpi_imx477_p3@1d {
				status = "disabled";

				compatible = "ridgerun,imx477";
				/* I2C device address */
				reg = <0x1d>;
				reset-gpios = <&dummy_gpio 7 GPIO_ACTIVE_HIGH>;	

				/* V4L2 device node location */
				devnode = "video7";

				/* Physical dimensions of sensor */
				physical_w = "3.680";
				physical_h = "2.760";

				sensor_model = "imx477";

				use_sensor_mode_id = "true";

				mode0 { /* IMX477_MODE_3840x2160 */
					mclk_khz = "24000";
					num_lanes = "2";
					tegra_sinterface = "serial_c";
					vc_id = "3";
					phy_mode = "DPHY";
					discontinuous_clk = "no";
					dpcm_enable = "false";
					cil_settletime = "0";
					lane_polarity = "0";

					active_w = "3840";
					active_h = "2160";
					mode_type = "bayer";
					pixel_phase = "rggb";
					csi_pixel_bit_depth = "10";
					readout_orientation = "90";
					line_length = "5832";
					inherent_gain = "1";
					mclk_multiplier = "80";
					pix_clk_hz = "300000000";
					// serdes_pix_clk_hz = "600000000";

					gain_factor = "16";
					framerate_factor = "1000000";
					exposure_factor = "1000000";
					min_gain_val = "16"; /* 1.00x */
					max_gain_val = "356"; /* 22x */
					step_gain_val = "1";
					default_gain = "16"; /* 1.00x */
					min_hdr_ratio = "1";
					max_hdr_ratio = "1";
					min_framerate = "2000000"; /* 2.0 fps */
					max_framerate = "30000000"; /* 30.0 fps */
					step_framerate = "1";
					default_framerate = "30000000"; /* 30.0 fps */
					min_exp_time = "13"; /* us */
					max_exp_time = "683709"; /* us */
					step_exp_time = "1";
					default_exp_time = "2495"; /* us */

					embedded_metadata_height = "2";
				};
				mode1 { /* IMX477_MODE_1920X1080 */
					mclk_khz = "24000";
					num_lanes = "2";
					tegra_sinterface = "serial_c";
					vc_id = "3";
					phy_mode = "DPHY";
					discontinuous_clk = "no";
					dpcm_enable = "false";
					cil_settletime = "0";
					lane_polarity = "0";

					active_w = "1920";
					active_h = "1080";
					mode_type = "bayer";
					pixel_phase = "rggb";
					csi_pixel_bit_depth = "10";
					readout_orientation = "90";
					line_length = "3076";
					inherent_gain = "1";
					mclk_multiplier = "80";
					pix_clk_hz = "300000000";
					// serdes_pix_clk_hz = "600000000";

					gain_factor = "16";
					framerate_factor = "1000000";
					exposure_factor = "1000000";
					min_gain_val = "16"; /* 1.00x */
					max_gain_val = "356"; /* 22x */
					step_gain_val = "1";
					default_gain = "16"; /* 1.00x */
					min_hdr_ratio = "1";
					max_hdr_ratio = "1";
					min_framerate = "2000000"; /* 2.0 fps */
					max_framerate = "60000000"; /* 60.0 fps */
					step_framerate = "1";
					default_framerate = "60000000"; /* 60.0 fps */
					min_exp_time = "13"; /* us */
					max_exp_time = "683709"; /* us */
					step_exp_time = "1";
					default_exp_time = "2495"; /* us */

					embedded_metadata_height = "2";
				};

				ports {
					status="okay";
					#address-cells = <1>;
					#size-cells = <0>;
					port@0 {
						status="okay";
						reg = <0>;
						imx477_vc_out7: endpoint {
							status="okay";
							vc-id = <3>;
							port-index = <2>;
							bus-width = <2>;
							remote-endpoint = <&cam_csi_in7>;
						};
					};
				};
			};
		};
	};
};

[JerryChang]

hello TobidieTopfpflanze,

I’ve went through the code, it looks like a typo for rbpi_imx477_ab_p3 camera node.
for instance,

@TobidieTopfpflanze

num_lanes = "3";

is the problem you’re try to resolve an abnormal preview from one of camera while running 2-cam simultaneously?
let’s check whether streams were output consistently for 2-cam scenario.
for instance, you may check with fakesink to disable preview, and shows frame-rate only.
here’s a sample pipeline, please replace <num> with your camera index for testing.
$ gst-launch-1.0 nvarguscamerasrc sensor-id=<num> ! 'video/x-raw(memory:NVMM),width=1920, height=1080, framerate=30/1, format=NV12' ! nvvidconv ! fpsdisplaysink text-overlay=0 name=sink_<num> video-sink=fakesink sync=0 -v

[TobidieTopfpflanze]

Hello JerryChang,

@JerryChang

I’ve went through the code, it looks like a typo for rbpi_imx477_ab_p3 camera node.

Oh, I accidentally changed the num_lanes to "3" instead of the vc_id-property. Thanks for spotting that.

This should not affect my previous test scenario, because I only used mode 0 for testing.

---

Back to my issue with streaming more than one camera…

I am able to start one stream at the time without any issue. It doesn’t matter from what video device. My issue starts appearing as soon as I start a second pipeline that runs on the same serial interface with a different vc-id. Its not a camera in particular that causes that problem.

The pipeline that you gave me, results in the same behavior. One stream works perfectly fine. Both streams are stopping short after I start the 2nd stream on one of the other virtual channels.

test@test-desktop:~$ ls /dev/video*
/dev/video0  /dev/video1  /dev/video2  /dev/video3

Logs

./stream_imx477_fpssink.sh

#!/bin/bash

CAM_ID=$1

gst-launch-1.0 nvarguscamerasrc sensor-id=$CAM_ID ! 'video/x-raw(memory:NVMM),width=1920, height=1080, framerate=60/1, format=NV12' ! nvvidconv ! fpsdisplaysink text-overlay=0 name=sink_$CAM_ID video-sink=fakesink sync=0 -v

Cam 1

test@test-desktop:~$ ./stream_imx477_fpssink.sh 3
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
/GstPipeline:pipeline0/GstFPSDisplaySink:sink_3/GstFakeSink:fakesink0: sync = false
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstNvArgusCameraSrc:nvarguscamerasrc0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)60/1
/GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstFPSDisplaySink:sink_3.GstGhostPad:sink.GstProxyPad:proxypad0: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstFPSDisplaySink:sink_3/GstFakeSink:fakesink0.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstFPSDisplaySink:sink_3.GstGhostPad:sink: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)60/1
/GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)60/1
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 3840 x 2160 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 22.250000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 22.250000; Exposure Range min 13000, max 683709000;

GST_ARGUS: Running with following settings:
   Camera index = 3 
   Camera mode  = 1 
   Output Stream W = 1920 H = 1080 
   seconds to Run    = 0 
   Frame Rate = 59.999999 
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
/GstPipeline:pipeline0/GstFPSDisplaySink:sink_3/GstFakeSink:fakesink0: sync = false
/GstPipeline:pipeline0/GstFPSDisplaySink:sink_3: last-message = rendered: 15, dropped: 0, current: 28.19, average: 28.19
/GstPipeline:pipeline0/GstFPSDisplaySink:sink_3: last-message = rendered: 29, dropped: 0, current: 26.39, average: 27.29
/GstPipeline:pipeline0/GstFPSDisplaySink:sink_3: last-message = rendered: 43, dropped: 0, current: 26.84, average: 27.14
/GstPipeline:pipeline0/GstFPSDisplaySink:sink_3: last-message = rendered: 57, dropped: 0, current: 26.36, average: 26.95
/GstPipeline:pipeline0/GstFPSDisplaySink:sink_3: last-message = rendered: 71, dropped: 0, current: 26.35, average: 26.83
/GstPipeline:pipeline0/GstFPSDisplaySink:sink_3: last-message = rendered: 85, dropped: 0, current: 26.30, average: 26.74
/GstPipeline:pipeline0/GstFPSDisplaySink:sink_3: last-message = rendered: 99, dropped: 0, current: 26.46, average: 26.70
CONSUMER: ERROR OCCURRED
ERROR: from element /GstPipeline:pipeline0/GstNvArgusCameraSrc:nvarguscamerasrc0: TIMEOUT
Additional debug info:
Argus Error Status
Execution ended after 0:00:07.583233408
Setting pipeline to NULL ...
GST_ARGUS: Cleaning up
Freeing pipeline ...

Cam 2

test@test-desktop:~$ ./stream_imx477_fpssink.sh 1
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
/GstPipeline:pipeline0/GstFPSDisplaySink:sink_1/GstFakeSink:fakesink0: sync = false
Setting pipeline to PLAYING ...
New clock: GstSystemClock
/GstPipeline:pipeline0/GstNvArgusCameraSrc:nvarguscamerasrc0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)60/1
/GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:src: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstFPSDisplaySink:sink_1.GstGhostPad:sink.GstProxyPad:proxypad0: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstFPSDisplaySink:sink_1/GstFakeSink:fakesink0.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstFPSDisplaySink:sink_1.GstGhostPad:sink: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)60/1
/GstPipeline:pipeline0/Gstnvvconv:nvvconv0.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)60/1
/GstPipeline:pipeline0/GstCapsFilter:capsfilter0.GstPad:sink: caps = video/x-raw(memory:NVMM), width=(int)1920, height=(int)1080, format=(string)NV12, framerate=(fraction)60/1
GST_ARGUS: Creating output stream
CONSUMER: Waiting until producer is connected...
GST_ARGUS: Available Sensor modes :
GST_ARGUS: 3840 x 2160 FR = 29.999999 fps Duration = 33333334 ; Analog Gain range min 1.000000, max 22.250000; Exposure Range min 13000, max 683709000;

GST_ARGUS: 1920 x 1080 FR = 59.999999 fps Duration = 16666667 ; Analog Gain range min 1.000000, max 22.250000; Exposure Range min 13000, max 683709000;

GST_ARGUS: Running with following settings:
   Camera index = 1 
   Camera mode  = 1 
   Output Stream W = 1920 H = 1080 
   seconds to Run    = 0 
   Frame Rate = 59.999999 
GST_ARGUS: Setup Complete, Starting captures for 0 seconds
GST_ARGUS: Starting repeat capture requests.
CONSUMER: Producer has connected; continuing.
/GstPipeline:pipeline0/GstFPSDisplaySink:sink_1/GstFakeSink:fakesink0: sync = false
/GstPipeline:pipeline0/GstFPSDisplaySink:sink_1: last-message = rendered: 13, dropped: 0, current: 25.21, average: 25.21
/GstPipeline:pipeline0/GstFPSDisplaySink:sink_1: last-message = rendered: 27, dropped: 0, current: 26.43, average: 25.83
CONSUMER: ERROR OCCURRED
ERROR: from element /GstPipeline:pipeline0/GstNvArgusCameraSrc:nvarguscamerasrc0: TIMEOUT
Additional debug info:
Argus Error Status
Execution ended after 0:00:10.306560790
Setting pipeline to NULL ...
GST_ARGUS: Cleaning up
Freeing pipeline ...

NV/CSI

nvcsi_logs_fakesink.txt (1022.1 KB)

[TobidieTopfpflanze]

To verify my FPD-Link III Linux driver, I wrote a Device Tree for AVT cameras. In this setup, I receive two camera streams simultaneously on a CSI-2 interface using Virtual Channels. Any additional stream exceeds the deserializer’s maximum data rate.
AVT made some changes in V4L to properly implement their driver. From the sources, it appears the ISP of the Jetson is completely bypassed due to their integrated ISP on the camera.

GitHub - alliedvision/linux_nvidia_jetson at l4t-35.5.0

This test proves that the issue is on the Jetson CSI/VI - V4l2 side of things. Unfortunately, these cameras are too heavy, expensive, and have excessively high power consumption for our customer.

[JerryChang]

hello TobidieTopfpflanze,

@TobidieTopfpflanze

I am able to start one stream at the time without any issue. It doesn’t matter from what video device. My issue starts appearing as soon as I start a second pipeline that runs on the same serial interface with a different vc-id. Its not a camera in particular that causes that problem.

could you please evaluate the CSI bandwidth and ISP throughput,
please also include 15% overhead to the calculation.
here’s an example to check CSI date rate, 4K(3840x2160),90-fps,10-bit.
CSI data-rate = 3840 * 2160 * 10 * 90 * 1.15 ~= 8.584 Gbps
ISP throughput = 8.584 Gbps / Raw10 ~= 858 MPixel/sec

there’re several pre-built updates based-on JetPack-5.1.3/l4t-r35.5.0.
for instance,
(1) could you please refer to Topic 293597 for camera firmware update to improve CSI stability,
(2) this might be minor, it’s Topic 292219 to resolve getModuleString() always returns the values of 1st camera
(3) you should apply pre-built update from Topic 284939 to enable Infinite Timeout Support.

@TobidieTopfpflanze

This test proves that the issue is on the Jetson CSI/VI - V4l2 side of things.

may I have more details about… “Any additional stream exceeds the deserializer’s maximum data rate.”