Camera aggregator driver NVP6324

I have few questions about integrate cameras at TX2 NX.

I have a board with NVP6324 that is just an analogic camera aggregator. I have a driver for it that probe correctly through i2c.

The driver is a generic Linux driver that attach NVP6324 as a V4L2 subdev.

My questions:

  1. This driver doesn’t use tegra_camera_platform (tegra_v4l2_camera.h) or any customization from Nvidia. Is mandatory use tegra_camera_platform to get it work at TX2 NX? Documentation makes it appear that it is not necessary, using imx185_v1 as sample, but the corresponding dtsi (tegra186-camera-imx185_v1-a00.dtsi) has a node for tegra_camera_platform and source imx185_v1.c uses it.

  2. As NVP6324 is a camera aggregator it uses a MIPI_CSI2 with x4 lanes and virtual channel concept for connect 4 cameras. Unlike imx390 sample, that use GSML serializer and deserializer, and can control the remote sensors using a kind of virtual i2c, at NVP6324 I can’t control the remote sensors.

Connection is NVP6324 (x4 lanes) → NVCSI (Port4 and Port5, x2 lanes at each port) → VI

Could someone point me the correct way to create the DTSI at this configuration?

At VI node, I have to use only one channel and port or one for each virtual channel? The port index at this case is VI stream number or NVCSI port? For each VI input remote-endpoint is all the same NVCSI output (nvp6324_csi_out0), considering that connection is only at CSI 4 and 5?

vi@15700000 {
	num-channels = <4>;
	ports {
		#address-cells = <1>;
		#size-cells = <0>;
		port@0 {
			reg = <0>;
			nvp6324_vi_in0: endpoint {
				vc-id = <0>;
				port-index = <4>;
				bus-width = <4>;
				remote-endpoint = <&nvp6324_csi_out0>;
			};
		};
		port@1 {
			reg = <1>;
			nvp6324_vi_in1: endpoint {
				vc-id = <1>;
				port-index = <4>;
				bus-width = <4>;
				remote-endpoint = <&nvp6324_csi_out0>;
			};
		};
		port@2 {
			reg = <2>;
			nvp6324_vi_in2: endpoint {
				vc-id = <2>;
				port-index = <4>;
				bus-width = <4>;
				remote-endpoint = <&nvp6324_csi_out0>;
			};
		};
		port@3 {
			reg = <3>;
			nvp6324_vi_in3: endpoint {
				vc-id = <3>;
				port-index = <4>;
				bus-width = <4>;
				remote-endpoint = <&nvp6324_csi_out0>;
			};
		};
	};
};

At sensor I2C node, is necessary create one port to each virtual channel and all with same NVCSI input (nvp6324_csi_in0)? Mode0 is mandatory?

i2c@3180000 {		
	ahd: nvp6324@30 {
		
		compatible = "nextchip,nvp6324_mipi";
		reg = <0x30>;
		
		rst-gpios = <&tegra_aon_gpio VIDEO_DEC_RST GPIO_ACTIVE_HIGH>;

		sensor_model ="nvp6324";

		use_sensor_mode_id = "true";

		mode0 {

			mclk_khz = "27000";
			num_lanes = "4";
			tegra_sinterface = "serial_e";
			phy_mode = "DPHY";
			discontinuous_clk = "no";
			dpcm_enable = "false";
			cil_settletime = "0";
			dynamic_pixel_bit_depth = "8";
			csi_pixel_bit_depth = "8";
			mode_type = "yuv";
			pixel_phase = "uyvy";
			active_w = "1280";
			active_h = "720";
			readout_orientation = "0";
			line_length = "1080";
			inherent_gain = "1";
			mclk_multiplier = "2";
			pix_clk_hz = "23040000";
		 
		    };

		ports {
			#address-cells = <1>;
			#size-cells = <0>;
			port@0 {
				reg = <0>;
				nvp6324_nvp6324_out0: endpoint {
					//vc-id = <0>;
					port-index = <4>;
					bus-width = <4>;
					remote-endpoint = <&nvp6324_csi_in0>;
					};
				};
			port@1 {
				reg = <1>;
				nvp6324_nvp6324_out1: endpoint {
					vc-id = <1>;
					port-index = <4>;
					bus-width = <4>;
					remote-endpoint = <&nvp6324_csi_in0>;
					};
				};
			port@2 {
				reg = <2>;
				nvp6324_nvp6324_out2: endpoint {
					vc-id = <2>;
					port-index = <4>;
					bus-width = <4>;
					remote-endpoint = <&nvp6324_csi_in0>;
					};
				};
			port@3 {
				reg = <3>;
				nvp6324_nvp6324_out3: endpoint {
					vc-id = <3>;
					port-index = <4>;
					bus-width = <4>;
					remote-endpoint = <&nvp6324_csi_in0>;
					};
				};
		};


	}; /*ahd: nvp6324@30 */

};

At NVCSI node I have to use only one channel? Considering that we’re using NVCSI port 4 at x4 lanes?

nvcsi@150c0000 {
	num-channels = <1>;
	#address-cells = <1>;
	#size-cells = <0>;
	channel@0 {
		reg = <0>;
		ports {
			#address-cells = <1>;
			#size-cells = <0>;
			port@0 {
				reg = <0>;
				nvp6324_csi_in0: endpoint@0 {
					port-index = <4>;
					bus-width = <4>;
					remote-endpoint = <&nvp6324_nvp6324_out0>;
				};
			};
			port@1 {
				reg = <1>;
				nvp6324_csi_out0: endpoint@1 {
					remote-endpoint = <&nvp6324_vi_in0>;
				};
			};
		};
	};
};
  1. How many sensor connect to NVP6324 can access those sensor via i2c? If not you need to check if NVP6324 output only one streaming or multiple streaming with virtual channel.

  2. The VI port means how many sources input as your dts here there have 4 input source with virtual channels.

  3. Yes if that tell those sensor connect to NVP6324 and can control them individually.

At sensor I2C node, is necessary create one port to each virtual channel and all with same NVCSI input (nvp6324_csi_in0)? Mode0 is mandatory?

We can’t access sensors directly, we connect ready-made cameras, that communicate with NVP6324 using AHD standard. The stream output of NVP6324 is show in bellow picture:

The output is YUV422 8 bit.

DTSI samples from NVIDIA consider that each sensor has an individual node at I2C, with different address, individual ports and modes. But at NVP6324 we doesn’t have access to individual camera configs or address, we only “see” NVP6324, for this reason we only see one device at I2C 7 bit address 0x30.

In this way, how can I define the multiple virtual channels using only one I2C node? I have tried this without success:

nvp6324@30 {
			
	compatible = "nextchip,nvp6324_mipi";
	reg = <0x30>;
	
	rst-gpios = <&tegra_aon_gpio VIDEO_DEC_RST GPIO_ACTIVE_HIGH>;

	mode0 {
		num_lanes = "4";
		tegra_sinterface = "serial_e";
		phy_mode = "DPHY";
		mode_type = "yuv";
		pixel_phase = "uyvy";
		active_w = "1280";
		active_h = "720";
		readout_orientation = "0";
		line_length = "720";
		inherent_gain = "1";
		mclk_multiplier = "2";
		pix_clk_hz = "23040000";
	    };

	ports {
		#address-cells = <1>;
		#size-cells = <0>;
		port@0 {
			reg = <0>;
			nvp6324_nvp6324_out0: endpoint {
				vc-id = <0>;
				port-index = <4>;
				bus-width = <4>;
				remote-endpoint = <&nvp6324_csi_in0>;
				};
			};
		port@1 {
			reg = <1>;
			nvp6324_nvp6324_out1: endpoint {
				vc-id = <1>;
				port-index = <4>;
				bus-width = <4>;
				remote-endpoint = <&nvp6324_csi_in1>;
				};
			};
		port@2 {
			reg = <2>;
			nvp6324_nvp6324_out2: endpoint {
				vc-id = <2>;
				port-index = <4>;
				bus-width = <4>;
				remote-endpoint = <&nvp6324_csi_in2>;
				};
			};
		port@3 {
			reg = <3>;
			nvp6324_nvp6324_out3: endpoint {
				vc-id = <3>;
				port-index = <4>;
				bus-width = <4>;
				remote-endpoint = <&nvp6324_csi_in3>;
				};
			};
	};


}; 

At this moment, I’m using the attached DTSI. This tegra186-camera-lanai-nvp6324.dtsi (9.5 KB)

Of course this isn’t correct because I using nodes with same address, but it was the only one that create one /dev/video at system.

dmesg output related to nvcsi, vi4 and nvp6324

jetson@image-box:~$ dmesg | grep "nvcsi\|vi4\|nvp6324"
[    0.438860] iommu: Adding device 150c0000.nvcsi to group 24
[    3.746626] nvp6324_mipi 2-0030: nvp6324 found, address: 0x30, chip id: 0xb0, rev: 0x00
[    3.937976] nvcsi 150c0000.nvcsi: initialized
[    3.938877] tegra-vi4 15700000.vi: using default number of vi channels,15
[    3.942655] tegra-vi4 15700000.vi: initialized
[    3.943446] tegra-vi4 15700000.vi: subdev 150c0000.nvcsi--1 bound
[    3.943457] tegra-vi4 15700000.vi: subdev 150c0000.nvcsi--4 bound
[    3.943461] tegra-vi4 15700000.vi: subdev nvp6324_mipi 2-0030 bound
[    3.943914] tegra-vi4 15700000.vi: subdev 150c0000.nvcsi--3 bound
[    3.943977] tegra-vi4 15700000.vi: subdev 150c0000.nvcsi--2 bound

jetson@image-box:~$ ls -la /dev/video*
crw-rw----+ 1 root video 81, 0 May 5 07:02 /dev/video0

Trying see if this /dev/video0 works. The output here is entirely wrong, because pixel format is different from configured at driver or dtsi, I don’t know where it search this configs.

jetson@image-box:~$ v4l2-ctl --all
Driver Info (not using libv4l2):
        Driver name   : tegra-video
        Card type     : vi-output, 150c0000.nvcsi--4
        Bus info      : platform:15700000.vi:4
        Driver version: 4.9.201
        Capabilities  : 0x84200001
                Video Capture
                Streaming
                Extended Pix Format
                Device Capabilities
        Device Caps   : 0x04200001
                Video Capture
                Streaming
                Extended Pix Format
Priority: 2
Video input : 0 (Camera 4: ok)
Format Video Capture:
        Width/Height      : 1920/1080
        Pixel Format      : 'RG10'
        Field             : Any
        Bytes per Line    : 3840
        Size Image        : 4147200
        Colorspace        : Default
        Transfer Function : Default (maps to Rec. 709)
        YCbCr/HSV Encoding: Default (maps to ITU-R 601)
        Quantization      : Default (maps to Full Range)
        Flags             :

Camera Controls

                    bypass_mode 0x009a2064 (intmenu): min=0 max=1 default=0 value=0
                override_enable 0x009a2065 (intmenu): min=0 max=1 default=0 value=0
                   height_align 0x009a2066 (int)    : min=1 max=16 step=1 default=1 value=1
                     size_align 0x009a2067 (intmenu): min=0 max=2 default=0 value=0
               write_isp_format 0x009a2068 (int)    : min=1 max=1 step=1 default=1 value=1
       sensor_signal_properties 0x009a2069 (u32)    : min=0 max=0 step=0 default=0 flags=read-only, has-payload
        sensor_image_properties 0x009a206a (u32)    : min=0 max=0 step=0 default=0 flags=read-only, has-payload
      sensor_control_properties 0x009a206b (u32)    : min=0 max=0 step=0 default=0 flags=read-only, has-payload
              sensor_dv_timings 0x009a206c (u32)    : min=0 max=0 step=0 default=0 flags=read-only, has-payload
               low_latency_mode 0x009a206d (bool)   : default=0 value=0
               preferred_stride 0x009a206e (int)    : min=0 max=65535 step=1 default=0 value=0
                   sensor_modes 0x009a2082 (int)    : min=0 max=30 step=1 default=30 value=30 flags=read-only

if I try capture something:

VIDIOC_S_FMT: failed: Inappropriate ioctl for device
[ 2578.639489] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[ 2578.646035] tegra-vi4 15700000.vi: tegra_channel_error_recovery: attempting to reset the capture channel
[ 2578.656682] nvcsi 150c0000.nvcsi: csi4_stream_check_status (4) ERROR_STATUS2VI_VC0 = 0x0000000c
[ 2578.665447] nvcsi 150c0000.nvcsi: csi4_stream_check_status (4) ERROR_STATUS2VI_VC1 = 0x0000000e
[ 2578.674194] nvcsi 150c0000.nvcsi: csi4_stream_check_status (4) ERROR_STATUS2VI_VC2 = 0x0000000c
[ 2578.682913] nvcsi 150c0000.nvcsi: csi4_stream_check_status (4) ERROR_STATUS2VI_VC2 = 0x00000006
[ 2578.691624] nvcsi 150c0000.nvcsi: csi4_stream_check_status (4) INTR_STATUS 0x00016cec
[ 2578.699489] nvcsi 150c0000.nvcsi: csi4_stream_check_status (4) ERR_INTR_STATUS 0x00016cec

1.You need to find a way to handle multiple streaming configure from the same i2c address.
2. The v4l2-ctl --list-formats-ext report incorrect size and pixel format may cause by the device tree incorrect. Have a confirm the context by get the dts from run time system. You can get it by below command.

sudo dtc -I fs -O dts -o extracted_proc.dts /proc/device-tree