Two CSI cameras each work individually, but not at the same time

[b]UPDATE - both cameras appear to work now, but only after running the v4l2-compliance tool. Before running the tool, the second camera doesn’t work.

Does anyone know why running v4l2-compliance would make the cameras work? And is there anything I can do to make them work at boot, besides automatically launching that process at startup?[/b]

Hello,

We have two CSI cameras from E-Con Systems (NileCAM30) that we have successfully modified to work with the Jetson TX2i on an Auvidea J90 carrier board, which has two cameras, on CSI lanes CD and EF.

Each camera works when it is the only one connected. But when booting with both cameras connected at once, although we see two nodes in /dev - video0 and video1, only one of them works correctly. Streaming from the other causes errors. Sometimes the one that works is video0, and sometimes it’s video1, but never both.

We’re not sure what is wrong, as both cameras and both CSI connections are verified to work individually. It’s just that when both cameras are connected at the same time, one of them doesn’t work. And which of the two cameras works seems to change with each boot.

Our device tree files are attached. The specific error we get when trying to use the nonworking camera is:

[ 1013.484299] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[ 1013.490807] tegra-vi4 15700000.vi: tegra_channel_error_recovery: attempting to reset the capture channel

Also, this message appears:

[  580.413760] tegra-vi4 15700000.vi: Status:  7 channel:00 frame:0000
[  580.420040] tegra-vi4 15700000.vi:          timestamp sof 605231803040 eof 605241120832 data 0x00000001
[  580.429463] tegra-vi4 15700000.vi:          capture_id 7 stream  2 vchan  0

v4l2-compliance passes for both cameras. Both cameras seem to initialize successfully in dmesg, as shown below:

[   12.996048] tegra-vi4 15700000.vi: subdev ar0330 1-0042 bound
[   12.996052] tegra-vi4 15700000.vi: subdev 150c0000.nvcsi-2 bound
[   12.996425] tegra-vi4 15700000.vi: subdev ar0330 7-0042 bound
[   12.996429] tegra-vi4 15700000.vi: subdev 150c0000.nvcsi-4 bound

Note that, although we have both cameras using the same reset GPIO, we modified the probe function to only actually toggle it once, and to free up the GPIOs after using them, so we don’t think this is the problem.

Any help would be greatly appreciated, thanks.

tegra186-quill-camera-ar0330.dtsi

/* Econ device tree for AR0330 support */
#include <dt-bindings/media/camera.h>
#include <dt-bindings/platform/t210/t210.h>
#include "dt-bindings/clock/tegra186-clock.h"
#define ECON_CAM0_RST_L	TEGRA_MAIN_GPIO(R, 1) /* reset line shared between cameras */
#define ECON_CAM_0_PWDN	TEGRA_MAIN_GPIO(R, 2) /* unconnected - assigned to unused GPIO */
#define ECON_HIM_MODE	TEGRA_MAIN_GPIO(R, 0) /* unconnected - assigned to unused GPIO */
#define ECON_ADD0_GPIO TEGRA_MAIN_GPIO(R, 3)  /* unconnected - assigned to unused GPIO */
#define ECON_ADD1_GPIO TEGRA_MAIN_GPIO(R, 5)  /* unconnected - assigned to unused GPIO */

/ {
	host1x {
		vi@15700000 {
			num-channels= <2>;
			ports {
				#address-cells = <1>;
				#size-cells = <0>;
				port@0 {
					reg = <0>;
					ecam_vi_in2: endpoint {
						csi-port = <2>; /* CSI ports C and D */
						bus-width = <4>;
						remote-endpoint = <&ecam_csi_out2>;
					};
				};
				port@1 {
					reg = <1>;
					ecam_vi_in4: endpoint {
						csi-port = <4>; /* CSI ports E and F */
						bus-width = <4>;
						remote-endpoint = <&ecam_csi_out4>;
					};
				};
			};
		};

		nvcsi@150c0000 {
			num-channels = <2>;
			channel@0 {
				reg = <0>;
				ports {
					#address-cells = <1>;
					#size-cells = <0>;
					port@0 {
						reg = <0>;
						ecam_csi_in2: endpoint@0 {
							csi-port = <2>; /* CSI ports C and D */
							bus-width = <4>;
							remote-endpoint = <&ecam_ar0330_out2>;
						};
					};
					port@1 {
						reg = <1>;
						ecam_csi_out2: endpoint@1 {
							remote-endpoint = <&ecam_vi_in2>;
						};
					};
				};
			};
			channel@1 {
				reg = <1>;
				ports {
					#address-cells = <1>;
					#size-cells = <0>;
					port@0 {
						reg = <0>;
						ecam_csi_in4: endpoint@2 {
							csi-port = <4>; /* CSI ports E and F */
							bus-width = <4>;
							remote-endpoint = <&ecam_ar0330_out4>;
						};
					};
					port@1 {
						reg = <1>;
						ecam_csi_out4: endpoint@3 {
							remote-endpoint = <&ecam_vi_in4>;
						};
					};
				};
			};
		};

	};

	/* CSI-CD is i2c-1 - c2400000 */
	/* CSI-EF is i2c-7 - c2500000 */

	i2c@c240000 {
		status = "okay";
		#address-cells = <1>;
		#size-cells = <0>;
				ar0330_c@42 {
					status = "okay";
					compatible = "nvidia,ar0330";
					reg = <0x42>;
					sensor_model ="ar0330";
					avdd-reg = "vana";
					iovdd-reg = "vif";
					post_crop_frame_drop = "0";
					clocks = <&tegra_car TEGRA186_CLK_EXTPERIPH1>,
					       <&tegra_car TEGRA186_CLK_PLLP_OUT0>;
					clock-names = "extperiph1", "pllp_grtba";
					mclk = "extperiph1";
					clock-frequency = <24000000>;
					
					reset-gpios = <&tegra_main_gpio ECON_CAM0_RST_L GPIO_ACTIVE_HIGH>; 
					pwdn-gpios = <&tegra_main_gpio ECON_CAM_0_PWDN GPIO_ACTIVE_LOW>; 
					him-mode = <&tegra_main_gpio ECON_HIM_MODE GPIO_ACTIVE_LOW>; 
					add0-gpio = <&tegra_main_gpio ECON_ADD0_GPIO GPIO_ACTIVE_HIGH>;
					add1-gpio = <&tegra_main_gpio ECON_ADD1_GPIO GPIO_ACTIVE_HIGH>;
					
					vana-supply = <&en_vdd_cam_hv_2v8>;
					vif-supply = <&en_vdd_cam>;
					mode0 { /* mode AR0330 */
						mclk_khz = "37125";
						num_lanes = "4";
						tegra_sinterface = "serial_c";	/* CSI lanes C-D */
						discontinuous_clk = "no";
						dpcm_enable = "false";
						cil_settletime = "0";
						dynamic_pixel_bit_depth = "12";
						csi_pixel_bit_depth = "12";
						mode_type = "bayer";
						pixel_phase = "rggb";
						pixel_t = "uyvy";

						active_w = "4192";
						active_h = "3120";
						readout_orientation = "0";
						line_length = "2200";
						inherent_gain = "1";
						mclk_multiplier = "2";
						pix_clk_hz = "74250000";

						min_gain_val = "0"; /* dB */
						max_gain_val = "48"; /* dB */
						min_hdr_ratio = "1";
						max_hdr_ratio = "1";
						min_framerate = "1.5";
						max_framerate = "30";
						min_exp_time = "30";
						max_exp_time = "660000";
						embedded_metadata_height = "1";
					};
					ports {
						#address-cells = <1>;
						#size-cells = <0>;
						port@0 {
							reg = <0>;
							ecam_ar0330_out2: endpoint {
				   		        csi-port = <2>; /* CSI ports C and D */
			  			        bus-width = <4>;
			              		        remote-endpoint = <&ecam_csi_in2>;
		 				 };
					};
				};
			};
	};
	i2c@c250000 {
		status = "okay";
		#address-cells = <1>;
		#size-cells = <0>;
				ar0330_e@42 {
					status = "okay";
					compatible = "nvidia,ar0330";
					reg = <0x42>;
					sensor_model ="ar0330";
					avdd-reg = "vana";
					iovdd-reg = "vif";
					post_crop_frame_drop = "0";
					clocks = <&tegra_car TEGRA186_CLK_EXTPERIPH1>,
					       <&tegra_car TEGRA186_CLK_PLLP_OUT0>;
					clock-names = "extperiph1", "pllp_grtba";
					mclk = "extperiph1";
					clock-frequency = <24000000>;
					
					reset-gpios = <&tegra_main_gpio ECON_CAM0_RST_L GPIO_ACTIVE_HIGH>; 
					pwdn-gpios = <&tegra_main_gpio ECON_CAM_0_PWDN GPIO_ACTIVE_LOW>; 
					him-mode = <&tegra_main_gpio ECON_HIM_MODE GPIO_ACTIVE_LOW>; 
					add0-gpio = <&tegra_main_gpio ECON_ADD0_GPIO GPIO_ACTIVE_HIGH>;
					add1-gpio = <&tegra_main_gpio ECON_ADD1_GPIO GPIO_ACTIVE_HIGH>;
					
					vana-supply = <&en_vdd_cam_hv_2v8>;
					vif-supply = <&en_vdd_cam>;
					mode0 { /* mode AR0330 */
						mclk_khz = "37125";
						num_lanes = "4";
						tegra_sinterface = "serial_e";	/* CSI lanes E-F */
						discontinuous_clk = "no";
						dpcm_enable = "false";
						cil_settletime = "0";
						dynamic_pixel_bit_depth = "12";
						csi_pixel_bit_depth = "12";
						mode_type = "bayer";
						pixel_phase = "rggb";
						pixel_t = "uyvy";

						active_w = "4192";
						active_h = "3120";
						readout_orientation = "0";
						line_length = "2200";
						inherent_gain = "1";
						mclk_multiplier = "2";
						pix_clk_hz = "74250000";

						min_gain_val = "0"; /* dB */
						max_gain_val = "48"; /* dB */
						min_hdr_ratio = "1";
						max_hdr_ratio = "1";
						min_framerate = "1.5";
						max_framerate = "30";
						min_exp_time = "30";
						max_exp_time = "660000";
						embedded_metadata_height = "1";
					};
					ports {
						#address-cells = <1>;
						#size-cells = <0>;
						port@0 {
							reg = <0>;
							ecam_ar0330_out4: endpoint {
				   		        csi-port = <4>; /* CSI ports E and F */
			  			        bus-width = <4>;
			              		        remote-endpoint = <&ecam_csi_in4>;
		 				 };
					};
				};
			};
	};
};
/ {
	tegra-camera-platform {
		compatible = "nvidia, tegra-camera-platform";
		modules {
			module0 {
				badge = "ecam_cu30-1";
				position = "front";
				orientation = "1";
				drivernode0 {
					pcl_id = "v4l2_sensor";
					devname = "e-CAM30_GMSL_CUTX2_CSI_CD";
					proc-device-tree = "/proc/device-tree/i2c@c240000/ar0330_c@42";
					status = "okay";
				};
			};
			module1 {
				badge = "ecam_cu30-2";
				position = "rear";
				orientation = "1";
				drivernode0 {
					pcl_id = "v4l2_sensor";
					devname = "e-CAM30_GMSL_CUTX2_CSI_EF";
					proc-device-tree = "/proc/device-tree/i2c@c250000/ar0330_e@42";
					status = "okay";
				};
			};
		};
	};

	gpio@2200000 {
		camera-control-output-low {
			gpio-hog;
			output-low;
			gpios = <ECON_CAM0_RST_L 0 ECON_CAM_0_PWDN 1 ECON_ADD0_GPIO 0 ECON_ADD1_GPIO 0>;
			label = "cam0-rst", "cam0-pwdn", "add0_gpio", "add1_gpio";
		};
	};
};

tegra186-quill-p3489-1000-a00-00-ar0330.dts

/*
 *
 * Copyright (c) 2016, NVIDIA CORPORATION.  All rights reserved.
 *
 * This program is free software; you can redistribute it and/or modify
 * it under the terms of the GNU General Public License as published by
 * the Free Software Foundation; version 2 of the License.
 *
 * This program is distributed in the hope that it will be useful, but WITHOUT
 * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
 * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
 * more details.
 *
 */

#include "tegra186-quill-p3489-1000-a00-00-ucm1.dts"
#include "quill-platforms/tegra186-quill-camera-ar0330.dtsi"

/ {
	nvidia,dtsfilename = __FILE__;

	host1x {
		vi_base: vi@15700000 {
			ports {
				ecam_vi_port0: port@0 {
					status = "okay";
					ecam_vi_in2: endpoint {
						status = "okay";
					};
				};
				ecam_vi_port1: port@1 {
					status = "okay";
					ecam_vi_in4: endpoint {
						status = "okay";
					};
				};
			};
		};
		csi_base: nvcsi@150c0000 {
			csi_chan0: channel@0 {
				status = "okay";
				ports {
					csi_chan0_port0: port@0 {
						status = "okay";
						ecam_csi_in2: endpoint@0 {
							status = "okay";
						};
					};
					csi_chan0_port1: port@1 {
						status = "okay";
						ecam_csi_out2: endpoint@1 {
							status = "okay";
						};
					};
				};
			};
			csi_chan1: channel@1 {
				status = "okay";
				ports {
					csi_chan1_port0: port@0 {
						status = "okay";
						ecam_csi_in4: endpoint@2 {
							status = "okay";
						};
					};
					csi_chan1_port1: port@1 {
						status = "okay";
						ecam_csi_out4: endpoint@3 {
							status = "okay";
						};
					};
				};
			};
		};
	};
};

Hi joshshields0,
Can you please share the kernel dmesg log when the above issue occurs (before and after running v4l2-compliance)?

Thank you for your help Kannan.m.

Here is dmesg after running gstreamer with the faulty camera (happened to be video0 this time):

[  339.822958]  mcu_stream_config(800) ISP Status = 0x0000 , Ret code = 0x00
[  339.829918]  Index = 0x0008 , format = 0x59565955, width = 2304, height = 1536, frate num = 24
[  339.838650] Skipping Previous mode set ...
[  340.933643] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[  340.940119] tegra-vi4 15700000.vi: tegra_channel_error_recovery: attempting to reset the capture channel
[  341.005846] nvcsi 150c0000.nvcsi: csi4_cil_check_status (2) CIL_INTR_STATUS 0x00000001
[  341.014096] nvcsi 150c0000.nvcsi: csi4_cil_check_status (2) CIL_ERR_INTR_STATUS 0x00000001
[  341.161519] tegra-vi4 15700000.vi: Status:  7 channel:00 frame:0000
[  341.167788] tegra-vi4 15700000.vi:          timestamp sof 366009743104 eof 366019060928 data 0x00000001
[  341.177183] tegra-vi4 15700000.vi:          capture_id 7 stream  2 vchan  0
[  346.712952] nvcsi 150c0000.nvcsi: csi4_stream_check_status (2) INTR_STATUS 0x00000004
[  346.720829] nvcsi 150c0000.nvcsi: csi4_stream_check_status (2) ERR_INTR_STATUS 0x00000004
[  346.729042] nvcsi 150c0000.nvcsi: csi4_cil_check_status (2) CIL_INTR_STATUS 0x00000135
[  346.736975] nvcsi 150c0000.nvcsi: csi4_cil_check_status (2) CIL_ERR_INTR_STATUS 0x00000135

Here it is after running v4l-compliance -d /dev/video0 :

[  383.735272] tegra-vi4 15700000.vi: =================  START STATUS  =================
[  383.743569] tegra-vi4 15700000.vi: ==================  END STATUS  ==================
[  384.398164]  mcu_stream_config(800) ISP Status = 0x0000 , Ret code = 0x00
[  384.405198]  Index = 0x0000 , format = 0x59565955, width = 640, height = 480, frate num = 60
[  384.455669]  mcu_stream_config(800) ISP Status = 0x0000 , Ret code = 0x00
[  384.462636]  Index = 0x0008 , format = 0x59565955, width = 2304, height = 1536, frate num = 24
[  384.513617]  mcu_stream_config(800) ISP Status = 0x0000 , Ret code = 0x00
[  384.520554]  Index = 0x0000 , format = 0x59565955, width = 640, height = 480, frate num = 60
[  384.569259]  mcu_stream_config(800) ISP Status = 0x0000 , Ret code = 0x00
[  384.576197]  Index = 0x0001 , format = 0x59565955, width = 1152, height = 768, frate num = 60
[  384.625230]  mcu_stream_config(800) ISP Status = 0x0000 , Ret code = 0x00
[  384.632161]  Index = 0x0008 , format = 0x59565955, width = 2304, height = 1536, frate num = 24
[  384.681204]  mcu_stream_config(800) ISP Status = 0x0000 , Ret code = 0x00
[  384.688126]  Index = 0x0000 , format = 0x59565955, width = 640, height = 480, frate num = 60
[  384.737168]  mcu_stream_config(800) ISP Status = 0x0000 , Ret code = 0x00
[  384.744074]  Index = 0x0000 , format = 0x59565955, width = 640, height = 480, frate num = 60
[  384.752628] Skipping Previous mode set ...
[  384.769760]  mcu_stream_config(800) ISP Status = 0x0000 , Ret code = 0x00
[  384.776639]  Index = 0x0008 , format = 0x59565955, width = 2304, height = 1536, frate num = 24
[  384.853106]  mcu_stream_config(800) ISP Status = 0x0000 , Ret code = 0x00
[  384.860011]  Index = 0x0008 , format = 0x59565955, width = 2304, height = 1536, frate num = 24
[  384.868730] Skipping Previous mode set ...
[  384.893868]  mcu_stream_config(800) ISP Status = 0x0000 , Ret code = 0x00
[  384.900752]  Index = 0x0008 , format = 0x59565955, width = 2304, height = 1536, frate num = 24
[  384.909462] Skipping Previous mode set ...

And here it is after running the same gstreamer command again, which succeeds this time:

[  403.623360]  mcu_stream_config(800) ISP Status = 0x0000 , Ret code = 0x00
[  403.630276]  Index = 0x0008 , format = 0x59565955, width = 2304, height = 1536, frate num = 24
[  403.639007] Skipping Previous mode set ...
[  404.118583] nvcsi 150c0000.nvcsi: csi4_cil_check_status (2) CIL_INTR_STATUS 0x00000111
[  404.126564] nvcsi 150c0000.nvcsi: csi4_cil_check_status (2) CIL_ERR_INTR_STATUS 0x00000111
[  404.174554] No vi channel is active

I was unable to fit the whole boot log in a post here. Let me know if you need it. Here are some lines that look relevant to the camera:

[    0.238074] iommu: Adding device 150c0000.nvcsi to group 34
[    3.027713] nvcsi 150c0000.nvcsi: initialized
[    8.207612] Current Firmware Version - (CU30-6e19dc3)
[    9.631278] ar0330 1-0042: Failed writing register ret = -121!
[    9.637660] ar0330 1-0042: tb_read_16b_reg:write reg error:reg=0x0
[    9.801349] 0. Initialized Control 0x00980900 - Brightness 
[    9.801351] 1. Initialized Control 0x00980901 - Contrast 
[    9.801353] 2. Initialized Control 0x00980902 - Saturation 
[    9.801356] 3. Initialized Control 0x0098090c - White Balance, Automatic 
[    9.801358] 4. Initialized Control 0x00980910 - Gamma 
[    9.801360] 5. Initialized Control 0x0098091a - White Balance Temperature 
[    9.801362] 6. Initialized Control 0x0098091b - Sharpness 
[    9.801363] 7. Initialized Control 0x00980913 - Gain 
[    9.808476] 8. Initialized Custom Ctrl 0x009a0901 - Exposure Auto 
[    9.808478] 9. Initialized Control 0x009a0902 - Exposure Time, Absolute 
[    9.808481] 10. Initialized Control 0x009a090d - Zoom, Absolute 
[    9.808483] 11. Initialized Control 0x009a0908 - Pan, Absolute 
[    9.808485] 12. Initialized Control 0x009a0909 - Tilt, Absolute 
[    9.812131] 13. Initialized Custom Ctrl 0x009a092d - Denoise 
[    9.812134] 14. Initialized Control 0x00980914 - Horizontal Flip 
[    9.812136] 15. Initialized Control 0x00980915 - Vertical Flip 
[    9.816130] 16. Initialized Custom Ctrl 0x009a0926 - ROI Exposure 
[    9.820129] 17. Initialized Custom Ctrl 0x009a0924 - ROI Window Size 
[    9.824130] 18. Initialized Custom Ctrl 0x009a0931 - Exposure Compensation 
[   11.373265] Current Firmware Version - (CU30-6e19dc3) ISP Already Initialized !! 
[   11.761231] 0. Initialized Control 0x00980900 - Brightness 
[   11.767407] 1. Initialized Control 0x00980901 - Contrast 
[   11.773343] 2. Initialized Control 0x00980902 - Saturation 
[   11.779440] 3. Initialized Control 0x0098090c - White Balance, Automatic 
[   11.786774] 4. Initialized Control 0x00980910 - Gamma 
[   11.792462] 5. Initialized Control 0x0098091a - White Balance Temperature 
[   11.799884] 6. Initialized Control 0x0098091b - Sharpness 
[   11.805925] 7. Initialized Control 0x00980913 - Gain 
[   11.833836] 8. Initialized Custom Ctrl 0x009a0901 - Exposure Auto 
[   11.840580] 9. Initialized Control 0x009a0902 - Exposure Time, Absolute 
[   11.847832] 10. Initialized Control 0x009a090d - Zoom, Absolute 
[   11.854404] 11. Initialized Control 0x009a0908 - Pan, Absolute 
[   11.860883] 12. Initialized Control 0x009a0909 - Tilt, Absolute 
[   11.871341] 13. Initialized Custom Ctrl 0x009a092d - Denoise 
[   11.877741] 14. Initialized Control 0x00980914 - Horizontal Flip 
[   11.884423] 15. Initialized Control 0x00980915 - Vertical Flip 
[   11.894604] 16. Initialized Custom Ctrl 0x009a0926 - ROI Exposure 
[   11.904895] 17. Initialized Custom Ctrl 0x009a0924 - ROI Window Size 
[   11.915457] 18. Initialized Custom Ctrl 0x009a0931 - Exposure Compensation 
[   11.938180] tegra-vi4 15700000.vi: subdev ar0330 1-0042 bound
[   11.945581] tegra-vi4 15700000.vi: subdev 150c0000.nvcsi-2 bound
[   11.952724] tegra-vi4 15700000.vi: subdev ar0330 7-0042 bound
[   11.959277] tegra-vi4 15700000.vi: subdev 150c0000.nvcsi-4 bound

Hi joshshields0,
Thanks for your logs, they were helpful.

Another quick clarification, did you probe and check whether the common RESET GPIO was toggled only once from LOW to HIGH and doesnot toggle in between or get back to LOW state after freeing it up?

Yes, I actually modified the ar0330_probe() function to make sure the reset GPIO is only toggled once. It looks like this now:

static int reset_done = 0;

/* Reset Release for MCU */
/* ensure it only happens once for multiple cameras */
if (!reset_done)
{
	reset_done = 1;
	toggle_gpio(reset_gpio, 0);
	msleep(100);	
	toggle_gpio(reset_gpio, 1);
	msleep(1000);
}

The other modification I made to that function was to free the GPIOs immediately after use, so that if initialization of one camera fails, the other one can still be initialized.

Thanks for your reply. We have found the issue and are working on it. We will update you soon.

I wanted to report that E-Con has sent me a patch which resolves the issue. Thank you E-Con for the excellent support.