Gstreamer cannot capture from Bayer sensor iMX415 on CSI E/F

I am working on a Jetson Nano (production version) mounted on a custom carrier board.
I have successfully developed a device driver for sensor iMX415 and I can capture from 2 sensors on CSI AB (x4) and CSI CD (x4) at the same time.
I have now changed my device tree to capture from CSI AB (x4) and CSI EF (x4).
According to dmesg everything works fine, but I don’t have any image in overlay on the screen from CSI EF (x4).
If I try to get raw data with v4l2 there isn’t any issue. So, I am pretty sure my device tree is ok.

What is strange is that CSI clock seems different when I capture with v4l2 and when I capture with gstreamer.

Here is the CSI clock when capturing RAW data with v4l2

Here is the CSI clock when capturing data with gstreamer

Here are CSI data when capturing RAW data with v4l2

These are CSI data when capturing with gstreamer

Another interesting point is that sensors connected to CSI AB and CSI EF are programmed exactly in the same way and modes on device tree contain the same data for both sensor instances (except for serial “serial_a” and “serial_e”).

Could you please help to diagnose ?
Thanks

Andrea

Any error message from the argus daemon or kernel timeout message?
Check the argus daemon message from below command.

sudo su
kill the process of nvargus-daemon
/usr/sbin/nvargus-daemon
launch camera from another console.

Hi ShaneCCC,
I did the test you recommended

Here is the log file from nvargus-daemon in case when gstreamer cannot capture from CSI EF:

nvargus-EF.log (3.9 KB)

Here is the log file from nvargus-daemon when gstreamer works fine from CSI AB

nvargus-AB.log (2.3 KB)

I attach also my device tree for sensors configuration

/ {
host1x {
	  vi {
        vi_port0: port@0 { // CSI-2 AB
            status="okay";
            reg = <0>;
            imx415_vi_in0: endpoint {
                status="okay";
                port-index = <0>;
                bus-width = <4>;
                remote-endpoint = <&imx415_csi_out0>;
            };
        };
        vi_port4: port@4 { // CSI-2 EF
            status="okay";
            reg = <1>;
            imx415_vi_in4: endpoint {
                status="okay";
                port-index = <4>;
                bus-width = <4>;
                remote-endpoint = <&imx415_csi_out4>;
            };
        };
	};

	nvcsi {
		csi_chan0: channel@0 {
			reg = <0>;
            status="okay";
			ports {
				#address-cells = <1>;
				#size-cells = <0>;
				csi_chan0_port0: port@0 {
                    status="okay";
					reg = <0>;
					imx415_csi_in0: endpoint@0 {
                        status="okay";
						port-index = <0>;
						bus-width = <4>;
						remote-endpoint = <&imx415_out0>;
					};
				};
				csi_chan0_port1: port@1 {
					reg = <1>;
                    status="okay";
					imx415_csi_out0: endpoint@1 {
                        status="okay";
						remote-endpoint = <&imx415_vi_in0>;
					};
				};
			};
		};
		csi_chan4: channel@4 {
			reg = <1>;
            status="okay";
			ports {
				#address-cells = <1>;
				#size-cells = <0>;
				csi_chan4_port0: port@0 {
                    status="okay";
					reg = <0>;
					imx415_csi_in4: endpoint@0 {
                        status="okay";
						port-index = <4>;
						bus-width = <4>;
						remote-endpoint = <&imx415_out2>;
					};
				};
				csi_chan4_port1: port@1 {
					reg = <1>;
                    status="okay";
					imx415_csi_out4: endpoint@1 {
                        status="okay";
						remote-endpoint = <&imx415_vi_in4>;
					};
				};
			};
		};
	};

    i2c@7000c400 {
        i2cmux_sens@70 {
            i2c@0 {
                imx415@1a {
                    compatible = "framos,imx415";

                    /* I2C device address */
                    reg = <0x1a>;

                    /* V4L2 device node location */
                    devnode = "video0";

                    /* Physical dimensions of sensor */
                    physical_w = "15.00";
                    physical_h = "12.50";

                    sensor_model = "imx415";

                    avdd-reg = "vana";
                    dvdd-reg = "vdig";
                    iovdd-reg = "vif";

                    /* enable CID_SENSOR_MODE_ID for sensor modes selection */
                    use_sensor_mode_id = "false";

                    mode0 { //
                        mclk_khz = "37125";
                        num_lanes = "4";
                        tegra_sinterface = "serial_a";
                        discontinuous_clk = "no";
                        cil_settletime = "0";
                        active_w = "3864";
                        active_h = "2192";

                        dynamic_pixel_bit_depth = "10";
                        csi_pixel_bit_depth = "10";
                        mode_type = "bayer";
                        pixel_phase = "gbrg";

                        readout_orientation = "0";
                        line_length = "5280";
                        inherent_gain = "1";
                        pix_clk_hz = "237600000";

                        min_gain_val = "1.0";
                        max_gain_val = "72.0";
                        min_hdr_ratio = "1";
                        max_hdr_ratio = "64";
                        min_framerate = "1.462526";
                        max_framerate = "25";
                        min_exp_time = "1";
                        max_exp_time = "65535";
                        embedded_metadata_height = "1";
                    };

                    ports {
                        #address-cells = <1>;
                        #size-cells = <0>;

                        port@0 {
                            reg = <0>;
                            status="okay";
                            imx415_out0: endpoint {
                                status="okay";
                                port-index = <0>;
                                bus-width = <4>;
                                remote-endpoint = <&imx415_csi_in0>;
                            };
                        };
                    };
                };
            };
            i2c@2 {
                imx415@1a {
                    compatible = "framos,imx415";

                    /* I2C device address */
                    reg = <0x1a>;

                    /* V4L2 device node location */
                    devnode = "video0";

                    /* Physical dimensions of sensor */
                    physical_w = "15.00";
                    physical_h = "12.50";

                    sensor_model = "imx415";

                    avdd-reg = "vana";
                    dvdd-reg = "vdig";
                    iovdd-reg = "vif";

                    /* enable CID_SENSOR_MODE_ID for sensor modes selection */
                    use_sensor_mode_id = "false";

                    mode0 { //
                        mclk_khz = "37125";
                        num_lanes = "4";
                        tegra_sinterface = "serial_e";
                        discontinuous_clk = "no";
                        cil_settletime = "0";
                        active_w = "3864";
                        active_h = "2192";

                        dynamic_pixel_bit_depth = "10";
                        csi_pixel_bit_depth = "10";
                        mode_type = "bayer";
                        pixel_phase = "gbrg";

                        readout_orientation = "0";
                        line_length = "5280";
                        inherent_gain = "1";
                        pix_clk_hz = "237600000";

                        min_gain_val = "1.0";
                        max_gain_val = "72.0";
                        min_hdr_ratio = "1";
                        max_hdr_ratio = "64";
                        min_framerate = "1.462526";
                        max_framerate = "25";
                        min_exp_time = "1";
                        max_exp_time = "65535";
                        embedded_metadata_height = "1";
                    };

                    ports {
                        #address-cells = <1>;
                        #size-cells = <0>;

                        port@0 {
                            reg = <0>;
                            status="okay";
                            imx415_out2: endpoint {
                                status="okay";
                                port-index = <4>;
                                bus-width = <4>;
                                remote-endpoint = <&imx415_csi_in4>;
                            };
                        };
                    };
                };
            };
        };
    };
  };
};

/ {
tegra-camera-platform {
	/**
	 * The general guideline for naming badge_info contains 3 parts, and is as follows,
	 * The first part is the camera_board_id for the module; if the module is in a FFD
	 * platform, then use the platform name for this part.
	 * The second part contains the position of the module, ex. "rear" or "front".
	 * The third part contains the last 6 characters of a part number which is found
	 * in the module's specsheet from the vendor.
	 */
	modules {
		cam_module0: module0 {
            status="okay";
			badge = "imx415_font_framos";
			position = "front";
			orientation = "0";
			cam_module0_drivernode0: drivernode0 {
                status="okay";
				pcl_id = "v4l2_sensor";
				devname = "imx415 7-001a";
				proc-device-tree = "/proc/device-tree/host1x/i2c@7000c400/i2cmux_sens@70/i2c@0/imx415@1a";
			};
		};
		cam_module1: module1 {
            status="okay";
			badge = "imx415_side_framos";
			position = "side";
			orientation = "1";
			cam_module1_drivernode0: drivernode0 {
                status="okay";
				pcl_id = "v4l2_sensor";
				devname = "imx415 9-001a";
				proc-device-tree = "/proc/device-tree/host1x/i2c@7000c400/i2cmux_sens@70/i2c@2/imx415@1a";
			};
		};
	};
  };
};

Did you check the v4l2-ctl --list-devices to confirm the video node were gen correctlly?

Yes, sure, the device node is in place and the sensor is properly configured as I can see from several debug messages from the device driver (in the attached log file the sensor device is imx415 9-001a)

dmesg_j13.log (95.9 KB)

Try to modify discontinuous_clk to yes

The result is unfortunately unchanged.

Does this sensor initial need much times?
Add set_mode_delay_ms to the device tree to try.
https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%20Linux%20Driver%20Package%20Development%20Guide%2Fcamera_sensor_prog.html%23

also get more log by demon
sudo su
kill the process of nvargus-daemon
export enableCamScfLogs=5
/usr/sbin/nvargus-daemon
launch camera from another console.

Hi ShaneCCC,
here are logs as per your request:

The positive case, so the sensor on CSI-AB
nvargus-AB3.log (337.1 KB)

The failure case, sensor on CSI-EF
nvargus-EF3.log (12.5 KB)

I absolutely don’t understand where such sensorId 67 comes from !

Looking forward

Andrea

What below configure for your case?

                num_csi_lanes = <8>;
                max_lane_speed = <1500000>;
                min_bits_per_pixel = <10>;
                vi_peak_byte_per_pixel = <2>;
                vi_bw_margin_pct = <25>;
                max_pixel_rate = <750000>;
                isp_peak_byte_per_pixel = <5>;
                isp_bw_margin_pct = <25>;

Here are my settings:

    num_csi_lanes = <8>;
    max_lane_speed = <2500000>;
    min_bits_per_pixel = <10>;
    vi_peak_byte_per_pixel = <2>;
    vi_bw_margin_pct = <25>;
    max_pixel_rate = <160000>;
    isp_peak_byte_per_pixel = <5>;
    isp_bw_margin_pct = <25>;

Hi ShaneCCC,
any news about where GUID and sensorId 67 comes from ?
Looking forward

Andrea

Just found below error.
Have a check the sensor programing guide.

Camera position. Values supported depend on the number of cameras in the system:
•In a two-camera system: rear and front.
•In a three-camera system: bottom, top, and center.
•In a six-camera system: bottomleft, bottomright, centerleft, centerright, topleft, and topright.

Hi ShaneCCC,
yes this was the error.
It is my fault to don’t have applied correctly your user guide recommandations.
Anyway, I strongly encourage you to implement some checks in DTS compilation in order to avoid such kind of mistakes.
Thank you for your precious help.

Andrea

1 Like

Have you connsidered using a Device Tree Schema Validation ?

https://www.kernel.org/doc/Documentation/devicetree/writing-schema.md

hi,

could I have driver for IMX415.

Thanks

Unfortunately the driver isn’t open-source.

Andrea