Device tree for ISL79987 video decoder MIPI CSI 2

Hi all,

I’m using a video decoder ISL79987 connected to the BUS I2C2 (gen2_i2c), this latter is being detected using the i2cdetect tool, i back ported the driver from Linux 5.18 to Linux 5.10 version, the driver probe successfully, and it registers the device to sysfs as video node without errors (Creating the /dev/video0)

The device tree was created following the scheamtic below :

Also the description of the DT below :

/ {

	tegra-capture-vi {
		status = "okay";
		num-channels = <2>;
		ports {
			#address-cells = <1>;
			#size-cells = <0>;
			port@0 {
				status = "okay";
				reg = <0>;
				isl79987_vi_in0: endpoint {
					status = "okay";
					port-index = <0>;
					bus-width = <1>;
					remote-endpoint = <&isl79987_csi_out0>;
				};
			};
			port@1 {
				status = "okay";
				reg = <1>;
				isl79987_vi_in1: endpoint {
					status = "okay";
					port-index = <0>;
					bus-width = <1>;
					remote-endpoint = <&isl79987_csi_out1>;
				};
			};

		};
	};

	host1x@13e00000 
	{
		nvcsi@15a00000 
		{
			num-channels = <2>;
			#address-cells = <1>;
			#size-cells = <0>;
			status = "okay";

			channel@0 {
				status = "okay";
				reg = <0>;
				ports {
					status = "okay";
					#address-cells = <1>;
					#size-cells = <0>;
					port@0 {
						status = "okay";
						reg = <0>;
						isl79987_csi_in0: endpoint@0 {
							status = "okay";
							port-index = <0>;
							bus-width = <1>;
							// TODO slave mode ?
							slave-mode;
							remote-endpoint = <&isl79987_out0>;
						};
					};
					port@1 {
						status = "okay";
						reg = <1>;
						isl79987_csi_out0: endpoint@1 {
							status = "okay";
							remote-endpoint = <&isl79987_vi_in0>;
						};
					};
				};
			};
			// TODO
			channel@1 
			{
				status = "okay";
				reg = <1>;
				ports 
				{
					status = "okay";
					#address-cells = <1>;
					#size-cells = <0>;
					// Define Port coming from Renesas and going to nvcsi
					port@0 
					{
						status = "okay";
						reg = <0>;
						isl79987_csi_in1: endpoint@2 {
								status = "okay";
								port-index = <0>;
								bus-width = <1>;
								remote-endpoint = <&isl79987_vin0>;
						};
					};
					// Define Port coming from nvcsi and going to vi
					port@1 {
						status = "okay";
						reg = <1>;
						isl79987_csi_out1: endpoint@3 {
								remote-endpoint = <&isl79987_vi_in1>;
						};
					};
				};
			};

		}; // end nvcsi
	}; // end host1x

	// Board is connected to I2C-2 Instance = Port Name Gen2-i2C = Bus number 1
	i2c@c240000 
	{
			// Device Tree node for first ISL79987 chip.
			// Physical address of the chip is 0x45
			isl79987_a@45 
			{
				status = "okay";
				// Device identifier. The Linux kernel uses this keyword to bind the device driver to a specific device.
				// This should be exactly the same than in cpp driver code
				compatible = "isil,isl79987";
				// I2C Slave Address
				reg = <0x45>;

				// TODO: Should we set other properties ?

				ports {
					status = "okay";
					#address-cells = <1>;
					#size-cells = <0>;
					
					// Define port for Renesas output CSI-2 (Control Renesas Chip)
					port@0 
					{
						status = "okay";
						reg = <0>;
						// Define one endpoint going from Renesas Board to CSI2
						isl79987_out0: endpoint 
						{
							status = "okay";
							
							// TODO write possibles values.
							// 1 Camera setup: 1 Channel, 1 data lane
							data-lanes = <1>;
							bus-width = <1>;
							// Renesas chip is connected to CSI_A
							port-index = <0>;

							// TODO find those values
							clock-lanes = <0>;
							link-frequencies = /bits/ 64 <108000000 216000000 432000000>;

							// Link device to NVCSI
							remote-endpoint = <&isl79987_csi_in0>;
						};
					};
					// Define sink endpoints for all analog stream coming into Renesas chip.
					port@1 {
						status="okay";
						reg = <1>;
						isl79987_vin0: endpoint 
						{
							// TODO find how to model analog input
							remote-endpoint = <&isl79987_csi_in1>;
						};
					};
					      
				}; // end ports
			}; // end isl79987_a
	}; // end i2c
};

/ {

	tegra-camera-platform 
	{
		compatible = "nvidia, tegra-camera-platform";
		status = "okay";

		// TODO Total number of CSI lanes when all cameras are active
		num_csi_lanes = <1>;
		// TODO Max lane speed in Kbit/s
		max_lane_speed = <15000000>;
		// TODO Min bits per pixelnum_csi_lanes
		min_bits_per_pixel = <10>;
		// TODO Max byte per pixel for the VI ISO case
		vi_peak_byte_per_pixel = <2>;
		// TODO Vi bandwidth margin in percentage
		vi_bw_margin_pct = <25>;
		// TODO Max byte per pixel for the ISP ISO case
		isp_peak_byte_per_pixel = <5>;
		// TODO Isp bandwidth margin in percentage
		isp_bw_margin_pct = <25>;

		modules {
			// Module for ISL79987 Front
			cam_module0: module0 
			{
				status="okay";

				// A unique name that identifies this module. The name must consist of three parts, separated by underscores:
				// - The module’s camera board ID (camera_board_id).
				// - The position of the module, for example, rear or front.
				// - The module’s part number, which you can find in the module datasheet. If the part number is not available, use a unique identifier. Only the last six characters of the part number are significant.
				// If your system has multiple identical modules, each module must have a different position, making the module name unique.
				badge = "isl79987_rear";
				
				// Camera position. Values supported depend on the number of cameras in the system:
				// - In a two-camera system: rear and front.
				// - In a three-camera system: bottom, top, and center.
				// - In a six-camera system: bottomleft, bottomright, centerleft, centerright, topleft, and topright.
				position = "rear";
				
				// Sensor orientation: a numeric index representing the orientation of the sensor. 
				// The relation between indices and orientations is arbitrary, but typically in two-camera systems 0 is “rear facing” and 1 is “front facing.”
				orientation = "0";

				// Driver node definition for sensor
				cam_module0_drivernode0: drivernode0 
				{
					status="okay";
					// Declare PCL support driver (classically known as guid)
					pcl_id = "v4l2_sensor";
					// <Driver v4l2 device name> <i2cbus>-<i2cPhysicalAddress>
					devname = "isl79987 1-0045";
					// Declare the device-tree hierarchy to driver instance
					proc-device-tree = "/proc/device-tree/i2c@c240000/isl79987_a@45";
				}; // end drivernode0
			}; // end module0

		}; // end modules
	}; // end tegra-camera-platform 
};


I tested the capture using v4l2 but this latter is failing

conti@conti-desktop:~$ GST_DEBUG=1 gst-launch-1.0 v4l2src ! 'video/x-raw,format=UYVY,with=720,height=480,interlace-mode=interleaved' ! videoconvert ! xvimagesink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
0:00:00.027313984  2409 0xaaaaf3d56e40 ERROR                   v4l2 gstv4l2object.c:1944:gst_v4l2_object_get_interlace_mode: Unknown enum v4l2_field 6
0:00:00.027347648  2409 0xaaaaf3d56e40 ERROR                   v4l2 gstv4l2object.c:1944:gst_v4l2_object_get_interlace_mode: Unknown enum v4l2_field 6
Setting pipeline to PLAYING ...
New clock: GstSystemClock
0:00:00.027612128  2409 0xaaaaf3d56e40 ERROR                   v4l2 gstv4l2object.c:1944:gst_v4l2_object_get_interlace_mode: Unknown enum v4l2_field 6
ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Device '/dev/video0' does not support interleaved interlacing
Additional debug info:
gstv4l2object.c(3828): gst_v4l2_object_set_format_full (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0:
Device wants interleaved interlacing
Execution ended after 0:00:00.000482720
Setting pipeline to PAUSED ...
Setting pipeline to READY ...
Setting pipeline to NULL ...
Freeing pipeline ...

Can you please give some suggestions about the Device tree, is it correct for a video decoder using MIPI CSI interface ?

Also I’m wondering why the gst is failing ?

Thanks

Using v4l2-ctl to capture from sensor and get the trace log to check if more clue for the failed.

https://elinux.org/Jetson/l4t/Camera_BringUp

Hi @ShaneCCC

I will try to get more debug logs with the link you provide

I’ve tried the command below to capture and there is what i get in the dmesg

v4l2-ctl --set-ctrl bypass_mode=0 --stream-mmap --stream-count=1 --stream-to=image2.raw -d /dev/video0

Do you have any idea about why it’s say that the VI channel not found for stream- 0 vc- 0 in fact that i’m not using virtual channel (I’m using only one video decoder), ** i didn’t add the property vc-id in the DT**

[  572.344657] bwmgr API not supported
[  572.348477] isl7998x 1-0045: stream ON
[  572.352126] tegra-i2c c240000.i2c: unmasked IRQ: 8c
[  572.352432] tegra-i2c c240000.i2c: transfer complete: 26 0 0
[  574.853423] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: request timed out after 2500 ms
[  574.862621] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: attempting to reset the capture channel
[  574.873750] (NULL device *): vi_capture_control_message: NULL VI channel received
[  574.881485] t194-nvcsi 13e40000.host1x:nvcsi@15a00000: csi5_stream_close: Error in closing stream_id=0, csi_port=0
[  574.892194] (NULL device *): vi_capture_control_message: NULL VI channel received
[  574.899932] t194-nvcsi 13e40000.host1x:nvcsi@15a00000: csi5_stream_open: VI channel not found for stream- 0 vc- 0
[  574.910832] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: successfully reset the capture channel
[  577.667929] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: request timed out after 2500 ms
[  577.677146] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: attempting to reset the capture channel
[  577.687200] (NULL device *): vi_capture_control_message: NULL VI channel received
[  577.694949] t194-nvcsi 13e40000.host1x:nvcsi@15a00000: csi5_stream_close: Error in closing stream_id=0, csi_port=0
[  577.705649] (NULL device *): vi_capture_control_message: NULL VI channel received
[  577.713389] t194-nvcsi 13e40000.host1x:nvcsi@15a00000: csi5_stream_open: VI channel not found for stream- 0 vc- 0
[  577.724310] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: successfully reset the capture channel
[  580.258599] tegra-camrtc-capture-vi tegra-capture-vi: uncorr_err: request timed out after 2500 ms
[  580.267817] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: attempting to reset the capture channel
[  580.278178] (NULL device *): vi_capture_control_message: NULL VI channel received
[  580.285922] t194-nvcsi 13e40000.host1x:nvcsi@15a00000: csi5_stream_close: Error in closing stream_id=0, csi_port=0
[  580.296652] (NULL device *): vi_capture_control_message: NULL VI channel received
[  580.304410] t194-nvcsi 13e40000.host1x:nvcs15a00000: csi5_stream_open: VI channel not found for stream- 0 vc- 0
[  580.315309] tegra-camrtc-capture-vi tegra-capture-vi: err_rec: successfully reset the capture channel
[  580.324935] isl7998x 1-0045: stream OFF
[  580.325001] tegra-i2c c240000.i2c: unmasked IRQ: 8c
[  580.325317] tegra-i2c c240000.i2c: transfer complete: 26 0 0

Can you please also tell me if the biding in the DT seems correct to you ?

conti@conti-desktop:~$ sudo media-ctl -p -d /dev/media0
[sudo] password for conti: 
Media controller API version 5.10.104

Media device information
------------------------
driver          tegra-camrtc-ca
model           NVIDIA Tegra Video Input Device
serial          
bus info        
hw revision     0x3
driver version  5.10.104

Device topology
- entity 5: 13e40000.host1x:nvcsi@15a00000- (2 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev0
	pad0: Sink
		<- "isl7998x 1-0045":0 [ENABLED]
	pad1: Source
		-> "vi-output, isl7998x 1-0045":0 [ENABLED]

- entity 8: 13e40000.host1x:nvcsi@15a00000- (2 pads, 0 link)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev1
	pad0: Sink
	pad1: Source

- entity 11: isl7998x 1-0045 (5 pads, 1 link)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev2
	pad0: Source
		[fmt:UYVY8_2X8/720x480 field:seq-bt]
		-> "13e40000.host1x:nvcsi@15a00000-":0 [ENABLED]
	pad1: Sink
		[fmt:UYVY8_2X8/720x480 field:seq-bt]
	pad2: Sink
		[fmt:UYVY8_2X8/720x480 field:seq-bt]
	pad3: Sink
		[fmt:UYVY8_2X8/720x480 field:seq-bt]
	pad4: Sink
		[fmt:UYVY8_2X8/720x480 field:seq-bt]

- entity 17: vi-output, isl7998x 1-0045 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video0
	pad0: Sink
		<- "13e40000.host1x:nvcsi@15a00000-":1 [ENABLED]

Thank you for your support

Even I have a similar use case, please provide more details of the working device tree and any driver code changes done for the analog video encoder ISL79987 feeding input to the NVCSI inside Jetson Orin NX

Thanks

Thanks.
I am not able to find the is79987.c driver file in Jetson BSP, how to get them from kernel repo

You can find the driver on the patch i shared earlier, btw the jetson orin BSP from nvidia does not contain this decoder driver so you have to add it and cross compile it by yourself

Thanks a lot

Still few queries to get things more clear for me.

  1. you downloaded this driver from GitHub kernel source. If yes could you provide the link?

  2. in our case I don’t think we use virtual video input so I think I need to do some changes in this same device tree?

Thanks

Hello,

  1. You can get the kernel source using the NVIDIA SDK Manager by running the script ./source_sync.sh -t tag or you can follow this link from nvidia Kernel Customization — Jetson Linux<br/>Developer Guide 34.1 documentation.

  2. Since, the patch i provided is for Jeston AGX Orin and you’re currently using an ORIN NX, you have to cusotimize your dtsi according to orin nx platform, for virtual channel, we used a tip that allows us to have 4 cameras support per ISL78897

I have already done that for latest version 36.4.3 but I could not find driver for this video decoder ISL78897

Ok
We think about this more when we start working on it after design work is finished…

Indeed you’re not gonna find the driver when downloading nvidia jetson BSP, you have to add the isl79987.c and the dtsi by yourself by configuring the linux-tegra kernel and cross compile it to get the new image with your video decoder driver

Can you explain this.

  1. I tried getting isl79987.c from linux kernel website git repositories, but could not find it. From were did you download isl79987.c

  2. Even though I have slight idea about third party driver build, could please explain the steps for cross compile of this driver

  3. Did you modify some existing dtsi file or added a new dtsi file for this isl79987

thanks

  1. There is decoder driver from the official linux github (Tag : v5.18) : linux/drivers/media/i2c/isl7998x.c at v5.18 · torvalds/linux · GitHub
    NVIDIA Jetson Linux supports Linux v5.15 kernel thats why you didnt found this driver since it was added by the linux maintainers since 5.18 kernel version
    So what you need to do is to add it to your L4T and cross compile it, if there is errors, it means you need to ajust the driver to fit the v5.15.

  2. You can follow ridgrun cross compilation guide for jetpack 5.0, since you’re using Jetpack 6.0, you will just have to change some stuff in the command lines like the L4T version and so on
    NVIDIA Jetson Xavier AGX

  3. You will need to create a new dtsi file for the video decoder and include it to the main jetson orin nx *DTS, just check the patch i shared and you will be able to understand all the modifications that was done

1 Like

Thanks a lot for elborate information.
Will refer to this thread, when we work on this camera decoder project.

1 Like

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.