Xavier NX MIPI CSI-2 without I2C from FPGA

Hello NVIDIA Developers,
I would like to be able to use a Jetson Xavier NX running L4T 32.7.1 to read a 2-lane MIPI CSI-2 without I2C from a Xilinx FGPA. The video is 1920 x 1080, 30fps, YUV422, similar to what is discussed in these forum threads:
jetson-xavier-nx-mipi-csi-2-without-i2c-from-fpga
After referring to relevant examples, I have modified the ov5693 device tree file and successfully created/dev/video0, but I still cannot capture images. The process is as follows:
1.modified the ov5693 device tree file

/ {
	host1x {
		vi@15c10000 {
			num-channels = <1>;
			ports {
				#address-cells = <1>;
				#size-cells = <0>;
				port@0 {
					reg = <0>;
					e3326_vi_in0: endpoint {
						port-index = <0>;
						bus-width = <2>;
						remote-endpoint = <&e3326_csi_out0>;
					};
				};
			};
		};

		nvcsi@15a00000 {
			num-channels = <1>;
			#address-cells = <1>;
			#size-cells = <0>;
			status = "okay";		//
			channel@0 {
				reg = <0>;
				ports {
					#address-cells = <1>;
					#size-cells = <0>;
					port@0 {
						reg = <0>;
						e3326_csi_in0: endpoint@0 {
							port-index = <0>;
							bus-width = <2>;
							remote-endpoint = <&e3326_ov5693_out0>;
						};
					};
					port@1 {
						reg = <1>;
						e3326_csi_out0: endpoint@1 {
							remote-endpoint = <&e3326_vi_in0>;
						};
					};
				};
			};
		};
	};

	i2c@3180000 {
		ov5693_c@36 {
			compatible = "nvidia,ov5693";
			/* I2C device address */
			reg = <0x36>;

			/* V4L2 device node location */
			devnode = "video0";

			/* Physical dimensions of sensor */
			physical_w = "3.674";
			physical_h = "2.738";

			/* Define any required hw resources needed by driver */
			/* ie. clocks, io pins, power sources */
			// avdd-reg = "vana";		//modified
			// iovdd-reg = "vif";

			/* Sensor output flip settings */
			vertical-flip = "true";

			/* Enable EEPROM support */
			has-eeprom = "0";			//1


			mode0 { // OV5693_MODE_1920X1080
				mclk_khz = "24000";
				num_lanes = "2";
				tegra_sinterface = "serial_c";
				phy_mode = "DPHY";
				discontinuous_clk = "yes";
				dpcm_enable = "false";
				cil_settletime = "0";

				active_w = "1920";
				active_h = "1080";
				pixel_t = "yuv_yuyv16";
				readout_orientation = "0";
				line_length = "2688";
				inherent_gain = "1";
				mclk_multiplier = "6.67";
				pix_clk_hz = "160000000";

				gain_factor = "10";
				min_gain_val = "10";/* 1DB*/
				max_gain_val = "160";/* 16DB*/
				step_gain_val = "1";
				default_gain = "10";
				min_hdr_ratio = "1";
				max_hdr_ratio = "1";
				framerate_factor = "1000000";
				min_framerate = "1816577";/*1.816577 */
				max_framerate = "30000000";/*30*/
				step_framerate = "1";
				default_framerate = "30000000";
				exposure_factor = "1000000";
				min_exp_time = "34";/* us */
				max_exp_time = "550385";/* us */
				step_exp_time = "1";
				default_exp_time = "33334";/* us */
				embedded_metadata_height = "0";
			};

			ports {
				#address-cells = <1>;
				#size-cells = <0>;

				port@0 {
					reg = <0>;
					e3326_ov5693_out0: endpoint {
						port-index = <0>;
						bus-width = <2>;
						remote-endpoint = <&e3326_csi_in0>;
					};
				};
			};
		};
	};

	e3326_lens_ov5693@P5V27C {
		min_focus_distance = "0.0";
		hyper_focal = "0.0";
		focal_length = "2.67";
		f_number = "2.0";
		aperture = "2.0";
	};

	tegra-camera-platform {
		compatible = "nvidia, tegra-camera-platform";
		num_csi_lanes = <2>;
		max_lane_speed = <1500000>;
		min_bits_per_pixel = <8>;
		vi_peak_byte_per_pixel = <2>;
		vi_bw_margin_pct = <25>;
		max_pixel_rate = <160000>;
		isp_peak_byte_per_pixel = <5>;
		isp_bw_margin_pct = <25>;

		modules {
			module0 {
				badge = "e3326_front_P5V27C";
				position = "rear";
				orientation = "1";
				drivernode0 {
					/* Declare PCL support driver (classically known as guid)  */
					pcl_id = "v4l2_sensor";
					/* Driver v4l2 device name */
					devname = "ov5693 2-0036";
					/* Declare the device-tree hierarchy to driver instance */
					proc-device-tree = "/proc/device-tree/i2c@3180000/ov5693_c@36";
				};
				drivernode1 {
					/* Declare PCL support driver (classically known as guid)  */
					pcl_id = "v4l2_lens";
					proc-device-tree = "/proc/device-tree/e3326_lens_ov5693@P5V27C/";
				};
			};
		};
	};
};

2.Modified the driver files ov5693.c and ov5693_mode_tbls.h to ignore the I2C、reset/pwdn gpio and TEGRA_CAMERA_CID_GROUP_HOLD part.
ov5693.c (14.5 KB)
ov5693_mode_tbls.h (1.3 KB)

3.Compiled the kernel, Flashed the board.

Tested the data capture,

$ v4l2-ctl -d /dev/video0 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
        Index       : 0
        Type        : Video Capture
        Pixel Format: 'YUYV'
        Name        : YUYV 4:2:2
                Size: Discrete 1920x1080
                        Interval: Discrete 0.033s (30.000 fps)

        Index       : 1
        Type        : Video Capture
        Pixel Format: 'YUYV'
        Name        : YUYV 4:2:2
                Size: Discrete 1920x1080
                        Interval: Discrete 0.033s (30.000 fps)
$ v4l2-ctl -d /dev/video0 --set-fmt-video=width=1920,height=1080,pixelformat=YUYV --stream-mmap --stream-count=1 --stream-to=test.raw --verbose
Some errors occurred:

VIDIOC_QUERYCAP: ok
VIDIOC_G_FMT: ok
VIDIOC_S_FMT: ok
Format Video Capture:
        Width/Height      : 1920/1080
        Pixel Format      : 'YUYV'
        Field             : None
        Bytes per Line    : 3840
        Size Image        : 4147200
        Colorspace        : sRGB
        Transfer Function : Default (maps to sRGB)
        YCbCr/HSV Encoding: Default (maps to ITU-R 601)
        Quantization      : Default (maps to Limited Range)
        Flags             :
VIDIOC_REQBUFS: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_QUERYBUF: ok
VIDIOC_QBUF: ok
VIDIOC_STREAMON: ok
        Index    : 0
        Type     : Video Capture
        Flags    : mapped, error
        Field    : None
        Sequence : 0
        Length   : 4147200
        Bytesused: 4147200
        Timestamp: 0.000000s (Monotonic, End-of-Frame)

        Index    : 0
        Type     : Video Capture
        Flags    : mapped, error
        Field    : None
        Sequence : 0
        Length   : 4147200
        Bytesused: 4147200
        Timestamp: 0.000000s (Monotonic, End-of-Frame)

        Index    : 1
        Type     : Video Capture
        Flags    : mapped, error
        Field    : None
        Sequence : 1
        Length   : 4147200
        Bytesused: 4147200
        Timestamp: 0.000000s (Monotonic, End-of-Frame)

        Index    : 2
        Type     : Video Capture
        Flags    : mapped, error
        Field    : None
        Sequence : 2
        Length   : 4147200
        Bytesused: 4147200
        Timestamp: 0.000000s (Monotonic, End-of-Frame)

        Index    : 3
        Type     : Video Capture
        Flags    : mapped, error
        Field    : None
        Sequence : 3
        Length   : 4147200
        Bytesused: 4147200
        Timestamp: 0.000000s (Monotonic, End-of-Frame)

dmesg got the following error:

[  266.485096] [RCE] Configuring VI GoS.
[  266.485145] [RCE] VM GOS[#0] addr=0xc2100000
[  266.485157] [RCE] VM GOS[#1] addr=0xc2101000
[  266.485168] [RCE] VM GOS[#2] addr=0xc2102000
[  266.485180] [RCE] VM GOS[#3] addr=0xc2103000
[  266.485191] [RCE] VM GOS[#4] addr=0xc2104000
[  266.485202] [RCE] VM GOS[#5] addr=0xc2105000
[  266.485217] [RCE] vi5_hwinit: firmware CL2018101701 protocol version 2.2
[  266.485263] [RCE] VI GOS[#0] set to VM GOS[4] base 0xc2104000
[  269.149052] tegra194-vi5 15c10000.vi: no reply from camera processor
[  269.149211] tegra194-vi5 15c10000.vi: uncorr_err: request timed out after 2500 ms
[  269.149381] tegra194-vi5 15c10000.vi: err_rec: attempting to reset the capture channel
[  269.153169] tegra194-vi5 15c10000.vi: err_rec: successfully reset the capture channel

Here are some logs obtained:
dmesg.txt (84.4 KB)
trace_log.txt (130.5 KB)

This bothered me for several days. I tried to find solutions in the Jetson forum,Could you provide some inputs to debug further on this issue ?

Thanks,
Ice.

The trace log tell didn’t receive any validate data from sensor.
May need to probe the MIPI signal to confirm it.

Also the tegra_sinterface should be serial_a if HW connect to CSI0 aka CSIA but shouldn’t the key point.

1 Like

Thank you for your reply!
I verified that there is no problem with the MIPI signal from the FPGA, I will verify it further, as well as modify the serial_a you mentioned.

Hi, ShaneCCC
I verified the MIPI signal, it is correct.I’m confused where the problem is.
I guess there are still some problems with my device tree configuration:

  1. Could it be the configuration of power supply and clock?
  2. Is it possible that the VI port is bound?
  3. The driver source code of the ov5693 (I have hardly modified it)?
    ov5693.c (21.4 KB)
    tegra194-camera-e3326-a00.dtsi (5.4 KB)
    tegra194-p2822-0000-camera-e3326-a00.dtsi (1.8 KB)

My ultimate goal is to receive mipi signals from FPGA, configured as mipi csi dphy 1080p30, 2lane, discontinuous clock.
I have checked these configuration files, but I am not sure whether they are accurate. Could you give me a hand check whether these files are wrong?
Thanks,
Ice.

Suppose it could be HW or FPGA output signal problem.

Thank you for your patience in answering. The problem was found after detecting the FPGA signal again. Resolved.

Hi @ShaneCCC,
I got the video stream under the L4T 32.7.1 (kernel-4.9) following the above steps.
Another question, does this camera device tree and driver modification work for L4T 34.1.1 (kernel-5.10) ?

Suppose need some modification for J5.1 AKA r35.1, I would suggest don’t use 34.1.1
The vi@xxx{} change to tegra-capture-vi {}

Hi @ShaneCCC,
Thank you for your reply.
Yes, I know that Jetpack 5.0.1 is just a developer preview release. But it should be similar to the latest version 5.0.2, right?
Because I have an NX running on Jetpack 5.0.1, I’m trying to port the driver on the L4T34.1. For the latest version, I’ll consider it later.

J5.1 is r35.x release and 5.0.2 is r34.x and wouldn’t support any more.

Hi @ShaneCCC,
This is the version information I saw on the official website.
Anyway, my problem has been solved. The same method was used to get the video stream in r34.1. I will listen to your advice and deprecate this version, thank you bro .

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.