Capture csi camera stops on Jetson TX2

Hello,
I have a Nvidia Jetson TX2 with L4T 32.1 and also cameras IMX327 and IMX334.
I tested capturing 3 cameras IMX327 (2 lanes CSI, 1080@60) at the same time and 3 cameras IMX334 (4 lanes CSI, 4k@60). It works great!
But we will need to connect different camera modules, for example, one camera IMX327 and one camera IMX334. Now this is the only option I can check. When capturing from one of the cameras, everything works well.
I start capture from both cameras, but it stops after about 20 seconds. And yet I don’t know the solution to the problem of freezing.

tegra186-camera-imx334-a00.dtsi

    /*
 * Copyright (c) 2016-2019, NVIDIA CORPORATION.  All rights reserved.
 *
 * This program is free software; you can redistribute it and/or modify
 * it under the terms of the GNU General Public License as published by
 * the Free Software Foundation; either version 2 of the License, or
 * (at your option) any later version.
 *
 * This program is distributed in the hope that it will be useful, but WITHOUT
 * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
 * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
 * more details.
 *
 * You should have received a copy of the GNU General Public License
 * along with this program.  If not, see <http://www.gnu.org/licenses/>.
 */

/ {
    host1x {
        vi@15700000 {
            num-channels = <2>;
            ports {
                #address-cells = <1>;
                #size-cells = <0>;
                port@0 {
                    reg = <0>;
                    liimx334_vi_in0: endpoint {
                        port-index = <0>;
                        bus-width = <4>;
                        remote-endpoint = <&liimx334_csi_out0>;
                    };
                };
                port@1 {
                    reg = <1>;
                    liimx327_vi_in1: endpoint {
                        port-index = <2>;
                        bus-width = <2>;
                        remote-endpoint = <&liimx327_csi_out1>;
                    };
                };
            };
        };

        nvcsi@150c0000 {
            num-channels = <2>;
            #address-cells = <1>;
            #size-cells = <0>;
            channel@0 {
                reg = <0>;
                ports {
                    #address-cells = <1>;
                    #size-cells = <0>;
                    port@0 {
                        reg = <0>;
                        liimx334_csi_in0: endpoint@0 {
                            port-index = <0>;
                            bus-width = <4>;
                            remote-endpoint = <&liimx334_imx334_out0>;
                        };
                    };
                    port@1 {
                        reg = <1>;
                        liimx334_csi_out0: endpoint@1 {
                            remote-endpoint = <&liimx334_vi_in0>;
                        };
                    };
                };
            };
            channel@1 {
                reg = <1>;
                ports {
                    #address-cells = <1>;
                    #size-cells = <0>;
                    port@0 {
                        reg = <0>;
                        liimx327_csi_in1: endpoint@0 {
                            port-index = <2>;
                            bus-width = <2>;
                            remote-endpoint = <&liimx327_imx327_out1>;
                        };
                    };
                    port@1 {
                        reg = <1>;
                        liimx327_csi_out1: endpoint@1 {
                            remote-endpoint = <&liimx327_vi_in1>;
                        };
                    };
                };
            };
        };
    };

    i2c@3180000 {
        tca9546@70 {
            i2c@0 {
                imx334_a@1a {
                    compatible = "nvidia,imx334";
                    /* I2C device address */
                    reg = <0x1a>;

                    /* V4L2 device node location */
                    devnode = "video0";

                    /* Physical dimensions of sensor */
                    physical_w = "3.674";
                    physical_h = "2.738";

                    sensor_model = "imx334";

                    /* Define any required hw resources needed by driver */
                    /* ie. clocks, io pins, power sources */
                    avdd-reg = "vana";
                    iovdd-reg = "vif";

                    /* Defines number of frames to be dropped by driver internally after applying */
                    /* sensor crop settings. Some sensors send corrupt frames after applying */
                    /* crop co-ordinates */
                    /*post_crop_frame_drop = "0";*/

                    /* if true, delay gain setting by one frame to be in sync with exposure */
                    delayed_gain = "true";
                    
                    set_mode_delay_ms = 600;

                    has-eeprom;
                    fuse_id_start_addr = <91>;

                    /**
                    * ==== Modes ====
                    * A modeX node is required to support v4l2 driver
                    * implementation with NVIDIA camera software stack
                    *
                    * == Signal properties ==
                    *
                    * phy_mode = "";
                    * PHY mode used by the MIPI lanes for this device
                    *
                    * tegra_sinterface = "";
                    * CSI Serial interface connected to tegra
                    * Incase of virtual HW devices, use virtual
                    * For SW emulated devices, use host
                    *
                    * pix_clk_hz = "";
                    * Sensor pixel clock used for calculations like exposure and framerate
                    *
                    * readout_orientation = "0";
                    * Based on camera module orientation.
                    * Only change readout_orientation if you specifically
                    * Program a different readout order for this mode
                    *
                    * == Image format Properties ==
                    *
                    * active_w = "";
                    * Pixel active region width
                    *
                    * active_h = "";
                    * Pixel active region height
                    *
                    * pixel_t = "";
                    * The sensor readout pixel pattern
                    *
                    * line_length = "";
                    * Pixel line length (width) for sensor mode.
                    *
                    * == Source Control Settings ==
                    *
                    * Gain factor used to convert fixed point integer to float
                    * Gain range [min_gain/gain_factor, max_gain/gain_factor]
                    * Gain step [step_gain/gain_factor is the smallest step that can be configured]
                    * Default gain [Default gain to be initialized for the control.
                    * use min_gain_val as default for optimal results]
                    * Framerate factor used to convert fixed point integer to float
                    * Framerate range [min_framerate/framerate_factor, max_framerate/framerate_factor]
                    * Framerate step [step_framerate/framerate_factor is the smallest step that can be configured]
                    * Default Framerate [Default framerate to be initialized for the control.
                    * use max_framerate to get required performance]
                    * Exposure factor used to convert fixed point integer to float
                    * For convenience use 1 sec = 1000000us as conversion factor
                    * Exposure range [min_exp_time/exposure_factor, max_exp_time/exposure_factor]
                    * Exposure step [step_exp_time/exposure_factor is the smallest step that can be configured]
                    * Default Exposure Time [Default exposure to be initialized for the control.
                    * Set default exposure based on the default_framerate for optimal exposure settings]
                    * For convenience use 1 sec = 1000000us as conversion factor
                    *
                    * gain_factor = ""; (integer factor used for floating to fixed point conversion)
                    * min_gain_val = ""; (ceil to integer)
                    * max_gain_val = ""; (ceil to integer)
                    * step_gain_val = ""; (ceil to integer)
                    * default_gain = ""; (ceil to integer)
                    * Gain limits for mode
                    *
                    * exposure_factor = ""; (integer factor used for floating to fixed point conversion)
                    * min_exp_time = ""; (ceil to integer)
                    * max_exp_time = ""; (ceil to integer)
                    * step_exp_time = ""; (ceil to integer)
                    * default_exp_time = ""; (ceil to integer)
                    * Exposure Time limits for mode (sec)
                    *
                    * framerate_factor = ""; (integer factor used for floating to fixed point conversion)
                    * min_framerate = ""; (ceil to integer)
                    * max_framerate = ""; (ceil to integer)
                    * step_framerate = ""; (ceil to integer)
                    * default_framerate = ""; (ceil to integer)
                    * Framerate limits for mode (fps)
                    *
                    * embedded_metadata_height = "";
                    * Sensor embedded metadata height in units of rows.
                    * If sensor does not support embedded metadata value should be 0.

                    * num_of_exposure = "";
                    * Digital overlap(Dol) frames
                    *
                    * num_of_ignored_lines = "";
                    * Used for cropping, eg. OB lines + Ignored area of effective pixel lines
                    *
                    * num_of_lines_offset_0 = "";
                    * Used for cropping, vertical blanking in front of short exposure data
                    * If more Dol frames are used, it can be extended, eg. num_of_lines_offset_1
                    *
                    * num_of_ignored_pixels = "";
                    * Used for cropping, The length of line info(pixels)
                    *
                    * num_of_left_margin_pixels = "";
                    * Used for cropping, the size of the left edge margin before
                    * the active pixel area (after ignored pixels)
                    *
                    * num_of_right_margin_pixels = "";
                    * Used for cropping, the size of the right edge margin after
                    * the active pixel area
                    */
                    mode0 {
                        mclk_khz = "24000";
                        num_lanes = "4";
                        tegra_sinterface = "serial_a";
                        discontinuous_clk = "yes";
                        dpcm_enable = "false";
                        cil_settletime = "0";

                        dynamic_pixel_bit_depth = "12";
                        csi_pixel_bit_depth = "12";
                        mode_type = "bayer";
                        pixel_phase = "rggb";
                        pixel_t = "bayer_rggb";

                        active_w = "3864";
                        active_h = "2180";
                        readout_orientation = "0";
                        line_length = "5500";
                        inherent_gain = "1";
                        mclk_multiplier = "30";
                        pix_clk_hz = "742240215";

                        min_gain_val = "1";
                        max_gain_val = "30";
                        min_hdr_ratio = "1";
                        max_hdr_ratio = "64";
                        min_framerate = "3";
                        max_framerate = "60";
                        min_exp_time = "7.41";
                        max_exp_time = "166725";

                        embedded_metadata_height = "1";
                    };
                    ports {
                        #address-cells = <1>;
                        #size-cells = <0>;
                        port@0 {
                            reg = <0>;
                            liimx334_imx334_out0: endpoint {
                                port-index = <0>;
                                bus-width = <4>;
                                remote-endpoint = <&liimx334_csi_in0>;
                            };
                        };
                    };
                };
            };			
			
			i2c@1 {
				imx327_b@1a {
				compatible = "nvidia,imx327";
				/* I2C device address */
				reg = <0x1a>;

				/* V4L2 device node location */
				devnode = "video1";

				/* Physical dimensions of sensor */
				physical_w = "1.948";
				physical_h = "1.109";


				/* Define any required hw resources needed by driver */
				/* ie. clocks, io pins, power sources */
				//avdd-reg = "vana";
				iovdd-reg = "vif";
				/*dvdd-reg = "vdig";*/
				sensor_model ="imx327";

				/* Defines number of frames to be dropped by driver internally after applying */
				/* sensor crop settings. Some sensors send corrupt frames after applying */
				/* crop co-ordinates */
				post_crop_frame_drop = "0";

				/* if true, delay gain setting by one frame to be in sync with exposure */
				delayed_gain = "true";

				/**
				* A modeX node is required to support v4l2 driver
				* implementation with NVIDIA camera software stack
				*
				* mclk_khz = "";
				* Standard MIPI driving clock, typically 24MHz
				*
				* num_lanes = "";
				* Number of lane channels sensor is programmed to output
				*
				* tegra_sinterface = "";
				* The base tegra serial interface lanes are connected to
				*
				* discontinuous_clk = "";
				* The sensor is programmed to use a discontinuous clock on MIPI lanes
				*
				* dpcm_enable = "true";
				* The sensor is programmed to use a DPCM modes
				*
				* cil_settletime = "";
				* MIPI lane settle time value.
				* A "0" value attempts to autocalibrate based on mclk_multiplier
				*
				*
				*
				*
				* active_w = "";
				* Pixel active region width
				*
				* active_h = "";
				* Pixel active region height
				*
				* pixel_t = "";
				* The sensor readout pixel pattern
				*
				* readout_orientation = "0";
				* Based on camera module orientation.
				* Only change readout_orientation if you specifically
				* Program a different readout order for this mode
				*
				* line_length = "";
				* Pixel line length (width) for sensor mode.
				* This is used to calibrate features in our camera stack.
				*
				* mclk_multiplier = "";
				* Multiplier to MCLK to help time hardware capture sequence
				* TODO: Assign to PLL_Multiplier as well until fixed in core
				*
				* pix_clk_hz = "";
				* Sensor pixel clock used for calculations like exposure and framerate
				*
				*
				*
				*
				* inherent_gain = "";
				* Gain obtained inherently from mode (ie. pixel binning)
				*
				* min_gain_val = ""; (floor to 6 decimal places)
				* max_gain_val = ""; (floor to 6 decimal places)
				* Gain limits for mode
				*
				* min_exp_time = ""; (ceil to integer)
				* max_exp_time = ""; (ceil to integer)
				* Exposure Time limits for mode (us)
				*
				*
				* min_hdr_ratio = "";
				* max_hdr_ratio = "";
				* HDR Ratio limits for mode
				*
				* min_framerate = "";
				* max_framerate = "";
				* Framerate limits for mode (fps)
				*
				* embedded_metadata_height = "";
				* Sensor embedded metadata height in units of rows.
				* If sensor does not support embedded metadata value should be 0.
				*/
					mode0 {/*mode IMX327_MODE_1920X1080_CROP_60FPS*/
						mclk_khz = "24000";
						num_lanes = "2";
						tegra_sinterface = "serial_c";
						discontinuous_clk = "yes";
						dpcm_enable = "false";
						cil_settletime = "0";

						dynamic_pixel_bit_depth = "12";
						csi_pixel_bit_depth = "12";
						mode_type = "bayer";
						pixel_phase = "rggb";
						pixel_t = "bayer_rggb";

						active_w = "1948";
						active_h = "1096";
						readout_orientation = "0";
						line_length = "2200";
						inherent_gain = "1";
						mclk_multiplier = "25";
						pix_clk_hz = "148500000";

						min_gain_val = "1";
						max_gain_val = "31";
						min_hdr_ratio = "1";
						max_hdr_ratio = "64";
						min_framerate = "3";
						max_framerate = "60";
						min_exp_time = "14.8";
						max_exp_time = "16650";
						embedded_metadata_height = "1";
					};
					ports {
						#address-cells = <1>;
						#size-cells = <0>;
						port@0 {
							reg = <0>;
							liimx327_imx327_out1: endpoint {
								port-index = <2>;
								bus-width = <2>;
								remote-endpoint = <&liimx327_csi_in1>;
							};
						};
					};
				};
			
			};
            
        };
    };
};



/ {

    tegra-camera-platform {
        compatible = "nvidia, tegra-camera-platform";
        /**
        * Physical settings to calculate max ISO BW
        *
        * num_csi_lanes = <>;
        * Total number of CSI lanes when all cameras are active
        *
        * max_lane_speed = <>;
        * Max lane speed in Kbit/s
        *
        * min_bits_per_pixel = <>;
        * Min bits per pixel
        *
        * vi_peak_byte_per_pixel = <>;
        * Max byte per pixel for the VI ISO case
        *
        * vi_bw_margin_pct = <>;
        * Vi bandwidth margin in percentage
        *
        * max_pixel_rate = <>;
        * Max pixel rate in Kpixel/s for the ISP ISO case
        *
        * isp_peak_byte_per_pixel = <>;
        * Max byte per pixel for the ISP ISO case
        *
        * isp_bw_margin_pct = <>;
        * Isp bandwidth margin in percentage
        */
        num_csi_lanes = <6>;
        max_lane_speed = <1500000>;
        min_bits_per_pixel = <10>;
        vi_peak_byte_per_pixel = <2>;
        vi_bw_margin_pct = <25>;
        max_pixel_rate = <750000>;
        isp_peak_byte_per_pixel = <5>;
        isp_bw_margin_pct = <25>;

        /**
         * The general guideline for naming badge_info contains 3 parts, and is as follows,
         * The first part is the camera_board_id for the module; if the module is in a FFD
         * platform, then use the platform name for this part.
         * The second part contains the position of the module, ex. "rear" or "front".
         * The third part contains the last 6 characters of a part number which is found
         * in the module's specsheet from the vender.
         */
        modules {
            module0 {
                badge = "imx334_front_liimx334";
                position = "front";
                orientation = "0";
                drivernode0 {
                    /* Declare PCL support driver (classically known as guid)  */
                    pcl_id = "v4l2_sensor";
                    /* Driver v4l2 device name */
                    devname = "imx334 30-001a";
                    /* Declare the device-tree hierarchy to driver instance */
                    proc-device-tree = "/proc/device-tree/i2c@3180000/tca9546@70/i2c@0/imx334_a@1a";
                };
            };
            module1 {
                badge = "imx327_rear_liimx327";
                position = "rear";
                orientation = "1";
                drivernode0 {
                    /* Declare PCL support driver (classically known as guid)  */
                    pcl_id = "v4l2_sensor";
                    /* Driver v4l2 device name */
                    devname = "imx327 31-001a";
                    /* Declare the device-tree hierarchy to driver instance */
                    proc-device-tree = "/proc/device-tree/i2c@3180000/tca9546@70/i2c@1/imx327_b@1a";
                };
            };
        };
    };
};

tegra186-quill-camera-imx334-a00.dtsi

/*
 * Copyright (c) 2017, NVIDIA CORPORATION.  All rights reserved.
 *
 * This program is free software; you can redistribute it and/or modify
 * it under the terms of the GNU General Public License as published by
 * the Free Software Foundation; either version 2 of the License, or
 * (at your option) any later version.
 *
 * This program is distributed in the hope that it will be useful, but WITHOUT
 * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
 * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
 * more details.
 *
 * You should have received a copy of the GNU General Public License
 * along with this program.  If not, see <http://www.gnu.org/licenses/>.
 */
#include <t18x-common-modules/tegra186-camera-imx334-a00.dtsi>
#include "dt-bindings/clock/tegra186-clock.h"

#define CAM0_RST_L	TEGRA_MAIN_GPIO(R, 5)
#define CAM0_PWDN	TEGRA_MAIN_GPIO(R, 0)
#define CAM1_RST_L	TEGRA_MAIN_GPIO(R, 1)
#define CAM1_PWDN	TEGRA_MAIN_GPIO(L, 6)
#define CAMERA_I2C_MUX_BUS(x) (0x1E + x)

/* camera control gpio definitions */

/ {
	i2c@3180000 {
		tca9546@70 {
			compatible = "nxp,pca9546";
			reg = <0x70>;
			#address-cells = <1>;
			#size-cells = <0>;
			skip_mux_detect = "yes";
			vcc-supply = <&en_vdd_cam>;
			vcc_lp = "vcc";
			force_bus_start = <CAMERA_I2C_MUX_BUS(0)>;

			i2c@0 {
				reg = <0x00>;
				i2c-mux,deselect-on-exit;
				#address-cells = <1>;
				#size-cells = <0>;
//				pca9570_a@24 {
//					compatible = "nvidia,pca9570";
//					reg = <0x24>;
//					channel = "a";
//					drive_ic = "DRV8838";
//				};
				imx334_a@1a {
					/* Define any required hw resources needed by driver */
					/* ie. clocks, io pins, power sources */
					clocks = <&tegra_car TEGRA186_CLK_EXTPERIPH1>,
							 <&tegra_car TEGRA186_CLK_PLLP_OUT0>;
					clock-names = "extperiph1", "pllp_grtba";
					mclk = "extperiph1";
					reset-gpios = <&tegra_main_gpio CAM0_RST_L GPIO_ACTIVE_HIGH>;
					vana-supply = <&en_vdd_cam_hv_2v8>;
					vif-supply = <&en_vdd_cam>;
				};
			};

			i2c@1 {
				reg = <0x01>;
				i2c-mux,deselect-on-exit;
				#address-cells = <1>;
				#size-cells = <0>;
				imx327_b@1a {
					/* Define any required hw resources needed by driver */
					/* ie. clocks, io pins, power sources */
					clocks = <&tegra_car TEGRA186_CLK_EXTPERIPH1>,
							 <&tegra_car TEGRA186_CLK_PLLP_OUT0>;
					clock-names = "extperiph1", "pllp_grtba";
					mclk = "extperiph1";
					reset-gpios = <&tegra_main_gpio CAM0_PWDN GPIO_ACTIVE_HIGH>;
					vana-supply = <&en_vdd_cam_hv_2v8>;
					vif-supply = <&en_vdd_cam>;
				};
			};

		};
	};
};

What APP do you use to capture?
Have you try to boost the nvcsi/vi clocks and boost system to performance mode?

sudo nvpmodel -m 0
sudo jetson_clocks

sudo su
echo 1 > /sys/kernel/debug/bpmp/debug/clk/vi/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/isp/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/nvcsi/mrq_rate_locked
cat /sys/kernel/debug/bpmp/debug/clk/vi/max_rate
cat /sys/kernel/debug/bpmp/debug/clk/isp/max_rate
cat /sys/kernel/debug/bpmp/debug/clk/nvcsi/max_rate
echo {max_rate} > /sys/kernel/debug/bpmp/debug/clk/vi/rate echo {max_rate} > /sys/kernel/debug/bpmp/debug/clk/isp/rate
echo ${max_rate} > /sys/kernel/debug/bpmp/debug/clk/nvcsi/rate

1 Like

Thanks for the quick response!
I captured a video using gstreamer

gst-launch-1.0 nvarguscamerasrc sensor-id=0 ! nvvidconv ! xvimagesink -ev

Thanks, that helped! There were no stops in 5 minutes of capture.
I will give a final answer about the results later.

After 5 hours of continuous capture, no errors occurred.
The problem can be considered resolved

You may need to narrow down is the nvpmodel/jetson_clocks or boost the NVCSI/VI/ISP help on this.