Support 2 or 4 cameras working simultaneously in one MAX9286

My nvidia platform is Xavier. My GMSL ic chip is max9286-max96705, sensor is ar0231.
I want to capture from four ar0231 in one max9286 simultaneously.Now I can capture from any one of the four cameras, but I can not capture from four cameras simultaneously.I got JetPack4.2 had supportted virtual channel. And someone has capture from 2 cameras simultaneously in TX2 through virtual channel.I discribe the device-tree like it. But I can not capture from any camera. Maybe my device-tree is wrong.Follow is my device-tree

/*
 * Copyright (c) 2017-2018, NVIDIA CORPORATION.  All rights reserved.
 *
 * This program is free software; you can redistribute it and/or modify
 * it under the terms of the GNU General Public License as published by
 * the Free Software Foundation; either version 2 of the License, or
 * (at your option) any later version.
 *
 * This program is distributed in the hope that it will be useful, but WITHOUT
 * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
 * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
 * more details.
 *
 * You should have received a copy of the GNU General Public License
 * along with this program.  If not, see <http://www.gnu.org/licenses/>.
 */

/ {
	host1x {
		vi@15c10000 {
			num-channels = <2>;
			ports {
				#address-cells = <1>;
				#size-cells = <0>;
				port@0 {
					reg = <0>;
					liar0231_vi_in0: endpoint {
						vc-id = <0>;
						port-index = <0>;
						bus-width = <4>;
						remote-endpoint = <&liar0231_csi_out0>;
					};
				};
				port@1 {
					reg = <1>;
					liar0231_vi_in1: endpoint {
						vc-id = <1>;
						port-index = <0>;
						bus-width = <4>;
						remote-endpoint = <&liar0231_csi_out1>;
					};
				};
			};
		};

		nvcsi@15a00000 {
			num-channels = <2>;
			#address-cells = <1>;
			#size-cells = <0>;
			channel@0 {
				reg = <0>;
				ports {
					#address-cells = <1>;
					#size-cells = <0>;
					port@0 {
						reg = <0>;
						liar0231_csi_in0: endpoint@0 {
							port-index = <0>;
							bus-width = <4>;
							remote-endpoint = <&liar0231_ar0231_out0>;
						};
					};
					port@1 {
						reg = <1>;
						liar0231_csi_out0: endpoint@1 {
							remote-endpoint = <&liar0231_vi_in0>;
						};
					};
				};
			};
			channel@1 {
				reg = <1>;
				ports {
					#address-cells = <1>;
					#size-cells = <0>;
					port@0 {
						reg = <0>;
						liar0231_csi_in1: endpoint@2 {
							port-index = <0>;
							bus-width = <4>;
							remote-endpoint = <&liar0231_ar0231_out1>;
						};
					};
					port@1 {
						reg = <1>;
						liar0231_csi_out1: endpoint@3 {
							remote-endpoint = <&liar0231_vi_in1>;
						};
					};
				};
			};
		};
	};

	i2c@3180000 {
		tca9546@70 {
			i2c@0 {
			ar0231_a@11 {
			compatible = "nvidia,ar0231";
			/* I2C device address */
			reg = <0x11>;

			/* V4L2 device node location */
			devnode = "video0";

			/* Physical dimensions of sensor */
			physical_w = "3.674";
			physical_h = "2.738";

			sensor_model = "ar0231";

			/* Define any required hw resources needed by driver */
			/* ie. clocks, io pins, power sources */
			avdd-reg = "vana";
			iovdd-reg = "vif";

			/* Defines number of frames to be dropped by driver internally after applying */
			/* sensor crop settings. Some sensors send corrupt frames after applying */
			/* crop co-ordinates */
				post_crop_frame_drop = "0";

			/* if true, delay gain setting by one frame to be in sync with exposure */
			//delayed_gain = "true";

		       /**
			* A modeX node is required to support v4l2 driver
			* implementation with NVIDIA camera software stack
			*
			* mclk_khz = "";
			* Standard MIPI driving clock, typically 24MHz
			*
			* num_lanes = "";
			* Number of lane channels sensor is programmed to output
			*
			* tegra_sinterface = "";
			* The base tegra serial interface lanes are connected to
			*
			* discontinuous_clk = "";
			* The sensor is programmed to use a discontinuous clock on MIPI lanes
			*
			* dpcm_enable = "true";
			* The sensor is programmed to use a DPCM modes
			*
			* cil_settletime = "";
			* MIPI lane settle time value.
			* A "0" value attempts to autocalibrate based on mclk_multiplier
			*
			*
			*
			*
			* active_w = "";
			* Pixel active region width
			*
			* active_h = "";
			* Pixel active region height
			*
			* pixel_t = "";
			* The sensor readout pixel pattern
			*
			* readout_orientation = "0";
			* Based on camera module orientation.
			* Only change readout_orientation if you specifically
			* Program a different readout order for this mode
			*
			* line_length = "";
			* Pixel line length (width) for sensor mode.
			* This is used to calibrate features in our camera stack.
			*
			* mclk_multiplier = "";
			* Multiplier to MCLK to help time hardware capture sequence
			* TODO: Assign to PLL_Multiplier as well until fixed in core
			*
			* pix_clk_hz = "";
			* Sensor pixel clock used for calculations like exposure and framerate
			*
			*
			*
			*
			* inherent_gain = "";
			* Gain obtained inherently from mode (ie. pixel binning)
			*
			* min_gain_val = ""; (floor to 6 decimal places)
			* max_gain_val = ""; (floor to 6 decimal places)
			* gain_step_pitch = ""; (floor to 6 decimal places, unit is db)
			* Gain limits for mode
			*
			* min_exp_time = ""; (ceil to integer)
			* max_exp_time = ""; (ceil to integer)
			* exp_time_1h = ""; (ceil to integer)
			* Exposure Time limits for mode (us)
			*
			*
			* min_hdr_ratio = "";
			* max_hdr_ratio = "";
			* HDR Ratio limits for mode
			*
			* min_framerate = "";
			* max_framerate = "";
			* Framerate limits for mode (fps)
			*
			* embedded_metadata_height = "";
			* Sensor embedded metadata height in units of rows.
			* If sensor does not support embedded metadata value should be 0.
			*
			* num_of_exposure = "";
			* Digital overlap(Dol) frames
			*
			* num_of_ignored_lines = "";
			* Used for cropping, eg. OB lines
			*
			* num_of_lines_offset_0 = "";
			* Used for cropping, vertical blanking in front of short exposure data
			* If more Dol frames are used, it can be extended, eg. num_of_lines_offset_1
			*
			* num_of_ignored_pixels = "";
			* Used for cropping, The length of line info(pixels)
			*
			* num_of_left_margin_pixels = "";
			* Used for cropping, the size of the left edge margin before
			* the active pixel area (after ignored pixels)
			*
			* num_of_right_margin_pixels = "";
			* Used for cropping, the size of the right edge margin after
			* the active pixel area
			*
			*/
				mode0 { // AR0231_MODE_4056X3040
					mclk_khz = "24000";
					num_lanes = "4";
					tegra_sinterface = "serial_a";
					phy_mode = "DPHY";
					discontinuous_clk = "no";
					dpcm_enable = "false";
					cil_settletime = "0";

					dynamic_pixel_bit_depth = "12";
					csi_pixel_bit_depth = "12";
					mode_type = "bayer";
					pixel_phase = "grbg";
					active_w = "1928";
					active_h = "1208";
					pixel_t = "bayer_grbg12";
					readout_orientation = "0";
					line_length = "2200";
					inherent_gain = "1";
					mclk_multiplier = "200";
					pix_clk_hz = "178200000";

					min_gain_val = "1.0";
					max_gain_val = "64";
					gain_step_pitch = "0.1";
					min_hdr_ratio = "1";
					max_hdr_ratio = "1";
					min_framerate = "1";
					max_framerate = "30";
					min_exp_time = "30";
					max_exp_time = "1000000";
					exp_time_1h = "6";
					embedded_metadata_height = "0";
				};
				ports {
					#address-cells = <1>;
					#size-cells = <0>;
					port@0 {
						reg = <0>;
						liar0231_ar0231_out0: endpoint {
							vc-id = <0>;
							port-index = <0>;
							bus-width = <4>;
							remote-endpoint = <&liar0231_csi_in0>;
							};
						};
					};
				};
#if 1
			ar0231_a@12 {
			compatible = "nvidia,ar0231";
			/* I2C device address */
			reg = <0x12>;

			/* V4L2 device node location */
			devnode = "video1";

			/* Physical dimensions of sensor */
			physical_w = "3.674";
			physical_h = "2.738";

			sensor_model = "ar0231";

			/* Define any required hw resources needed by driver */
			/* ie. clocks, io pins, power sources */
			avdd-reg = "vana";
			iovdd-reg = "vif";

			/* Defines number of frames to be dropped by driver internally after applying */
			/* sensor crop settings. Some sensors send corrupt frames after applying */
			/* crop co-ordinates */
			post_crop_frame_drop = "0";

			/* if true, delay gain setting by one frame to be in sync with exposure */
			//delayed_gain = "true";

		       /**
			* A modeX node is required to support v4l2 driver
			* implementation with NVIDIA camera software stack
			*
			* mclk_khz = "";
			* Standard MIPI driving clock, typically 24MHz
			*
			* num_lanes = "";
			* Number of lane channels sensor is programmed to output
			*
			* tegra_sinterface = "";
			* The base tegra serial interface lanes are connected to
			*
			* discontinuous_clk = "";
			* The sensor is programmed to use a discontinuous clock on MIPI lanes
			*
			* dpcm_enable = "true";
			* The sensor is programmed to use a DPCM modes
			*
			* cil_settletime = "";
			* MIPI lane settle time value.
			* A "0" value attempts to autocalibrate based on mclk_multiplier
			*
			*
			*
			*
			* active_w = "";
			* Pixel active region width
			*
			* active_h = "";
			* Pixel active region height
			*
			* pixel_t = "";
			* The sensor readout pixel pattern
			*
			* readout_orientation = "0";
			* Based on camera module orientation.
			* Only change readout_orientation if you specifically
			* Program a different readout order for this mode
			*
			* line_length = "";
			* Pixel line length (width) for sensor mode.
			* This is used to calibrate features in our camera stack.
			*
			* mclk_multiplier = "";
			* Multiplier to MCLK to help time hardware capture sequence
			* TODO: Assign to PLL_Multiplier as well until fixed in core
			*
			* pix_clk_hz = "";
			* Sensor pixel clock used for calculations like exposure and framerate
			*
			*
			*
			*
			* inherent_gain = "";
			* Gain obtained inherently from mode (ie. pixel binning)
			*
			* min_gain_val = ""; (floor to 6 decimal places)
			* max_gain_val = ""; (floor to 6 decimal places)
			* gain_step_pitch = ""; (floor to 6 decimal places, unit is db)
			* Gain limits for mode
			*
			* min_exp_time = ""; (ceil to integer)
			* max_exp_time = ""; (ceil to integer)
			* exp_time_1h = ""; (ceil to integer)
			* Exposure Time limits for mode (us)
			*
			*
			* min_hdr_ratio = "";
			* max_hdr_ratio = "";
			* HDR Ratio limits for mode
			*
			* min_framerate = "";
			* max_framerate = "";
			* Framerate limits for mode (fps)
			*
			* embedded_metadata_height = "";
			* Sensor embedded metadata height in units of rows.
			* If sensor does not support embedded metadata value should be 0.
			*
			* num_of_exposure = "";
			* Digital overlap(Dol) frames
			*
			* num_of_ignored_lines = "";
			* Used for cropping, eg. OB lines
			*
			* num_of_lines_offset_0 = "";
			* Used for cropping, vertical blanking in front of short exposure data
			* If more Dol frames are used, it can be extended, eg. num_of_lines_offset_1
			*
			* num_of_ignored_pixels = "";
			* Used for cropping, The length of line info(pixels)
			*
			* num_of_left_margin_pixels = "";
			* Used for cropping, the size of the left edge margin before
			* the active pixel area (after ignored pixels)
			*
			* num_of_right_margin_pixels = "";
			* Used for cropping, the size of the right edge margin after
			* the active pixel area
			*
			*/
				mode0 { // AR0231_MODE_4056X3040
					mclk_khz = "24000";
					num_lanes = "4";
					tegra_sinterface = "serial_a";
					phy_mode = "DPHY";
					discontinuous_clk = "no";
					dpcm_enable = "false";
					cil_settletime = "0";

					dynamic_pixel_bit_depth = "12";
					csi_pixel_bit_depth = "12";
					mode_type = "bayer";
					pixel_phase = "grbg";
					active_w = "1928";
					active_h = "1208";
					pixel_t = "bayer_grbg12";
					readout_orientation = "0";
					line_length = "2200";
					inherent_gain = "1";
					mclk_multiplier = "200";
					pix_clk_hz = "178200000";

					min_gain_val = "1.0";
					max_gain_val = "64";
					gain_step_pitch = "0.1";
					min_hdr_ratio = "1";
					max_hdr_ratio = "1";
					min_framerate = "1";
					max_framerate = "30";
					min_exp_time = "30";
					max_exp_time = "1000000";
					exp_time_1h = "6";
					embedded_metadata_height = "0";
				};
				ports {
					#address-cells = <1>;
					#size-cells = <0>;
					port@0 {
						reg = <0>;
						liar0231_ar0231_out1: endpoint {
							vc-id = <1>;
							port-index = <0>;
							bus-width = <4>;
							remote-endpoint = <&liar0231_csi_in1>;
							};
						};
					};
				};
#endif
			};
		};
	};
};



/ {

	tegra-camera-platform {
		compatible = "nvidia, tegra-camera-platform";
	       /**
		* Physical settings to calculate max ISO BW
		*
		* num_csi_lanes = <>;
		* Total number of CSI lanes when all cameras are active
		*
		* max_lane_speed = <>;
		* Max lane speed in Kbit/s
		*
		* min_bits_per_pixel = <>;
		* Min bits per pixel
		*
		* vi_peak_byte_per_pixel = <>;
		* Max byte per pixel for the VI ISO case
		*
		* vi_bw_margin_pct = <>;
		* Vi bandwidth margin in percentage
		*
		* max_pixel_rate = <>;
		* Max pixel rate in Kpixel/s for the ISP ISO case
		*
		* isp_peak_byte_per_pixel = <>;
		* Max byte per pixel for the ISP ISO case
		*
		* isp_bw_margin_pct = <>;
		* Isp bandwidth margin in percentage
		*/
		num_csi_lanes = <4>;
		max_lane_speed = <1500000>;
		min_bits_per_pixel = <10>;
		vi_peak_byte_per_pixel = <2>;
		vi_bw_margin_pct = <25>;
		max_pixel_rate = <750000>;
		isp_peak_byte_per_pixel = <5>;
		isp_bw_margin_pct = <25>;

		/**
		 * The general guideline for naming badge_info contains 3 parts, and is as follows,
		 * The first part is the camera_board_id for the module; if the module is in a FFD
		 * platform, then use the platform name for this part.
		 * The second part contains the position of the module, ex. "rear" or "front".
		 * The third part contains the last 6 characters of a part number which is found
		 * in the module's specsheet from the vender.
		 */
		modules {

			module0 {
			badge = "ar0231_bottom_A6V26";
				position = "front";
				orientation = "1";
				drivernode0 {
					/* Declare PCL support driver (classically known as guid)  */
				pcl_id = "v4l2_sensor";
					/* Driver v4l2 device name */
					devname = "ar0231 30-0011";
				/* Declare the device-tree hierarchy to driver instance */
					proc-device-tree = "/proc/device-tree/i2c@3180000/tca9546@70/i2c@0/ar0231_a@11";
			};
			};
			module1 {
			badge = "ar0231_front_A6V26";
				position = "bottom";
				orientation = "0";
				drivernode0 {
					/* Declare PCL support driver (classically known as guid)  */
				pcl_id = "v4l2_sensor";
					/* Driver v4l2 device name */
					devname = "ar0231 30-0012";
				/* Declare the device-tree hierarchy to driver instance */
					proc-device-tree = "/proc/device-tree/i2c@3180000/tca9546@70/i2c@0/ar0231_a@12";
			};
			};
		};
	};
};

hello zmaqidong,

it seems you had wrong port binding configuration, each CSI brick could handle up-to 4-lane input.
for example,
there’s conflict by configure two 4-lane ar0231 into port-0.

port@0 {
					reg = <0>;
					liar0231_vi_in0: endpoint {
						vc-id = <0>;
						port-index = <0>;
						bus-width = <4>;
						remote-endpoint = <&liar0231_csi_out0>;
					};
				};
				port@1 {
					reg = <1>;
					liar0231_vi_in1: endpoint {
						vc-id = <1>;
						port-index = <0>;
						bus-width = <4>;
						remote-endpoint = <&liar0231_csi_out1>;
					};
				};

please update the port binding to follow the connection logic.
you may also refer to Sensor Driver Programming Guide, and check the port index mapping diagram for Jetson platforms.
thanks

Hi JerryChang,

Do you mean the configuration should be something like this?

port@0 {
					reg = <0>;
					liar0231_vi_in0: endpoint {
						vc-id = <0>;
						port-index = <0>;
						bus-width = <2>;
						remote-endpoint = <&liar0231_csi_out0>;
					};
				};
				port@1 {
					reg = <1>;
					liar0231_vi_in1: endpoint {
						vc-id = <1>;
						port-index = <0>;
						bus-width = <2>;
						remote-endpoint = <&liar0231_csi_out1>;
					};
				};

I can’t use virtual ID feature if I need 4lane input?

Hi rary,
I think you are right.But maybe something others are needed to configure. I configure like this,But I can not capture from the two cameras simultaneously.When I capture from the second camera, the first camera does not work.

Hi JerryChang,
I agree with rary.Maybe my devive tree is just not comprehensive.Jetpack 4.2 says ‘CSI virtual channel support for Jetson AGX Xavier and Jetson TX2’.https://developer.nvidia.com/embedded/jetpack
someone has tried it and he succeed.https://devtalk.nvidia.com/default/topic/1049464/jetson-tx2/tx2-mipi-virtual-channel-error-/post/5326163/#5326163.I followed him, But I failed.

Hi
I noticed that your device tree lacks the vc_id properties in the mode0.
If you couldn’t capture from any cameras, that would be the cause.

hello rary,

no, I mean the port-index. both of your camera module mapping to port-index=0, this means you had connect two camera as CSI-A input. this will not works with/without virtual channel.
you should check the port index mapping diagram to have more details.
thanks

Hi JerryChang,

Sorry I didn’t get it.
I understand that, without virtual channel, connecting two cameras into one CSI port won’t work.
However from my understanding, with virtual channel, the aggregator output two camera signals into one port.
So mapping two cameras to one CSI port is fine.

The sample dtsi for imx390 also maps two cameras to port-index=<0>.

hello rary,

it’s correct that without virtual channel, connecting two cameras into one CSI port won’t work.
with virtual channel, you’ll still need configuration in serdes for each camera node and there’s vc-id to distinguish each of them.
please also refer to below sample of imx390.

i2c@3180000 {
		tca9546@70 {
			i2c@0 {
			imx390_a@1b {
				...
				gmsl-link {
					src-csi-port = "b";
					dst-csi-port = "a";
					<b>serdes-csi-link = "a";</b>
					csi-mode = "1x4";
					st-vc = <0>;
					vc-id = <0>;
					num-lanes = <2>;
					streams = "ued-u1", "raw12";
					};
...

			imx390_b@1c {
				...
				gmsl-link {
					src-csi-port = "b";
					dst-csi-port = "a";
					<b>serdes-csi-link = "b";</b>
					csi-mode = "1x4";
					st-vc = <0>;
					vc-id = <1>;
					num-lanes = <2>;
					streams = "ued-u1", "raw12";
					};
				};

hello rary,

BTW, is there a frame sync configuration in your setups.
please also have a try to enable/disable that for testing.
thanks

Hi JerryChang,
Thank you for your clear explanation. Now I understand what was the problem.
I’m sorry for confusing you, but I’m not original poster of this thread.

From my experience, frame sync sometimes cause a problem so I recommend to disable it for first try.

Hi rary, JerryChang,

Do you mean disabling frame sync in the max9286? or is there a frame sync at the TX2 side?

I’m having a similar issue, I’m able to capture from all the cameras attached to the MAX9286 independently, using different Virtual Channel IDs, but whenever I try to capture from 2 cameras, the TX2 doesn’t capture. I disabled the frame sync in the MAX9286 and I started receiving buffers but the TX2 is triggering CHANSEL_SHORT_FRAME and the MAX9286 is getting a Line Buffer overflowed that might be related to that issue.

hi all,

there’s known issue we should disable frame-sync on deserializer as a workaround to enable simultaneous streaming. thanks

Hi CariosR92,JerryChang
When frame sync is disabled, I can capture from 2 cameras simultaneously. But the image is malposed.
When frame sync is enabled, I can capture from 2 cameras simultaneously for 3 seconds. 3 seconds later, frame sync failed. Maybe there is something wrong with the config of max9286.

hello zmaqidong,

could you please attach the image which shown the malposed?
please also contact with the SerDes vendor to investigate the root cause.
thanks

Hi JerryChang,
1.Today I replace the camera with another one, I can capture from the 2 cameras simultaneously for about 3 minutes when frame synchronization is enabled.And I tried different cameras, the period is different.Now the problem is that how to keep frame synchronization locked always but not just 3 minutes.
2.When frame synchronization is disabled, the image is error, maybe it is not malposed.

hello zmaqidong,

  1. just for confirmation about why frame sync specific. could you please share some details that “who’s” configure fsync in single deserializer setup.
  2. could you please share failure messages when you reproduce the issue.
  3. please double check the attachment, I cannot download your abnormal images.
    thanks

Hi JerryChang,
Thanks for your help. I have solved the problem.THe camera should work on slave mode.

Hi zmaqidong, JerryChang,

I have a similar setup and would like to ask a couple of questions:

  1. When you disabled the frame sync in the MAX9286 and you say the image is malposed, are you able to capture from the TX2 and see the images? Or is there a SHORT_FRAME error?

I disable frame sync and the TX2 keeps telling me SHORT_FRAME, but the MAX9286 triggers a Line Buffer Overflowed error (register 0x22).

Are you getting this error when frame sync is disabled?

  1. When you enable sync. Are you using automatic frame sync in the deserializer? Or are you using external frame sync?

Do you think frame sync is completely needed to be able to capture properly?

Hi CarlosR92,
1)When I disable the frame sync in the MAX9286, I can get the image.And I do not know what is SHORT_FRAME error.
2)When I use manual frame sync in the deserializer and the cameras work on slave mode, I can capture from four cameras simultaneously.
3)when only one camera works, frame sync can be disalbed.BUt if two or four cameras work, frame sync is completely needed.If cameras work on master mode, the four cameras can work about 3 minutes. the reason is frame sync lost 3 minutes later. So the cameras must work on slave mode.