ADV7481 and NVCSI media entity

Hello. I’m currently trying to port ADV7481 with NVIDIA Xavier NX.

And there’s issue on media controller framework between ADV7481’s txa entity and NVCSI.

The following is result of media-ctl -p -d /dev/media0.

A@A:~/work$ media-ctl -p -d /dev/media0
Media controller API version 0.1.0

Media device information

driver tegra194-vi5
model NVIDIA Tegra Video Input Device
serial
bus info
hw revision 0x3
driver version 0.0.0

Device topology

  • entity 1: 15a00000.nvcsi–1 (2 pads, 1 link)
    type V4L2 subdev subtype Unknown flags 0
    device node name /dev/v4l-subdev0
    pad0: Sink
    pad1: Source
    → “vi-output, 15a00000.nvcsi–1”:0 [ENABLED]

  • entity 4: adv748x 1-0070 txa (2 pads, 1 link)
    type V4L2 subdev subtype Sensor flags 0
    device node name /dev/v4l-subdev2
    pad0: Sink
    [fmt:unknown/0x0]
    ← “adv748x 1-0070 hdmi”:1 [ENABLED,IMMUTABLE]
    pad1: Source
    [fmt:unknown/0x0]

  • entity 7: adv748x 1-0070 hdmi (2 pads, 1 link)
    type V4L2 subdev subtype Sensor flags 0
    device node name /dev/v4l-subdev1
    pad0: Sink
    [fmt:YUYV8_1X16/1920x1080 field:none colorspace:smpte170m]
    [dv.caps:BT.656/1120 min:1x1@0 max:10000x10000@165000000 stds:CEA-861,DMT,CVT,GTF caps:progressive,reduced-blanking,custom]
    pad1: Source
    [dv.caps:BT.656/1120 min:1x1@0 max:10000x10000@165000000 stds:CEA-861,DMT,CVT,GTF caps:progressive,reduced-blanking,custom]
    [dv.detect:BT.656/1120 1920x1080p30 (2200x1125) stds: flags:]
    [dv.current:BT.656/1120 1920x1080p25 (2640x1125) stds:CEA-861 flags:CE-video]
    → “adv748x 1-0070 txa”:0 [ENABLED,IMMUTABLE]

  • entity 12: vi-output, 15a00000.nvcsi–1 (1 pad, 1 link)
    type Node subtype V4L flags 0
    device node name /dev/video0
    pad0: Sink
    ← “15a00000.nvcsi–1”:1 [ENABLED]

I tried to link txa and nvcsi with ‘media-ctl --link’ option but It’s not working. (invalid argument message)

The following is my dtsi.

/ {
host1x {
vi@15c10000 {
num-channels = <1>;
ports {
#address-cells = <1>;
#size-cells = <0>;
port@0 {
reg = <0>;
status = “okay”;
adv748x_vi_in0: endpoint@0 {
status = “okay”;
port-index = <0>;
bus-width = <2>;
remote-endpoint = <&adv748x_csi_out0>;
};
};
};
};

	nvcsi@15a00000 {
		num-channels = <1>;
		#address-cells = <1>;
		#size-cells = <0>;
		channel@0 {
			reg = <0>;
			status = "okay";
			ports {
				#address-cells = <1>;
				#size-cells = <0>;
				port@0 {
					reg = <0>;
					status = "okay";
					adv748x_csi_in0: endpoint@0 {
						status = "okay";
						port-index = <0>;
						bus-width = <2>;
						remote-endpoint = <&adv748x_out0>;
					};
				};
				port@1 {
					reg = <1>;
					status = "okay";
					adv748x_csi_out0: endpoint@1 {
						status = "okay";
						remote-endpoint = <&adv748x_vi_in0>;
					};
				};
			};
		};
	};
};

i2c@c240000 {	// HDMI or SDI
	status = "okay";
	#address-cells = <1>;
	#size-cells = <0>;

	adv748x@70 {
		compatible = "adv748x";
		reg = <0x70>;		// HDMI

		ports {
			#address-cells = <1>;
			#size-cells = <0>;

			port@0 {
				reg = <0>;
				adv748x_out0: endpoint@0 {
					port-index = <0>;
					bus-width = <2>;
					remote-endpoint = <&adv748x_csi_in0>;
				};
			};
			// port@1 {
			// 	reg = <1>;
			// 	adv748x_hdmi0: endpoint@1 {
			// 		port-index = <0>;
			// 		bus-width = <2>;
			// 	};
			// };
		};
	}; 
};

Due to lack of dummy hdmi_in property, adv7481’s dt-binding like reg couldn’t applied and I think should modified parse_dt to cope with.

How can I link txa’s source and nvcsi’s sink?

Thanks.

hello Linuxman123,

you may refer to tc358840, which is HDMI to CSI bridge driver for reference.
thanks

Thanks for reply Jerry.

Actually, I already checked dtsi things but nothing was special to me.

I think nvcsi’s sink is linked using endpoint things, am I right?

I don’t know why It’s not linked correctly.

hello Linuxman123,

yes, that’s using remote-endpoint to link the VI, NvCSI, and sensor modules port bindings. please check developer guide for the port bindings session.
you may see-also training video, V4L2 Sensor Driver Development Tutorial for reference of driver development.
thanks

In dt-bindings adv748x.txt, there’s remote-endpoint = <&hdmi_in> in port 8.
(link : linux/adv748x.txt at v5.10 · torvalds/linux · GitHub)
In NVIDIA, should I have to use dummy property? because there is no property like that in xavier nx + hdmi input is just a input for adv748x, not an input for xavier nx.

hello Linuxman123,

you’ll need to define the port binding for this, for example, hdmi_in => csi_in => csi_out => vi_in.

Hello Jerry, Thanks for your reply.

Do you mean device-tree should be like the following?

/ {
host1x {
vi@15c10000 {
num-channels = <1>;
ports {
#address-cells = <1>;
#size-cells = <0>;
port@0 {
reg = <0>;
status = “okay”;
adv748x_vi_in0: endpoint@0 {
status = “okay”;
port-index = <0>;
bus-width = <2>;
remote-endpoint = <&adv748x_csi_out0>;
};
};
};
};

nvcsi@15a00000 {
	num-channels = <1>;
	#address-cells = <1>;
	#size-cells = <0>;
	channel@0 {
		reg = <0>;
		status = "okay";
		ports {
			#address-cells = <1>;
			#size-cells = <0>;
			port@0 {
				reg = <0>;
				status = "okay";
				adv748x_csi_in0: endpoint@0 {
					status = "okay";
					port-index = <0>;
					bus-width = <2>;
					remote-endpoint = <&adv748x_out0>;
				};
			};
			port@1 {
				reg = <1>;
				status = "okay";
				adv748x_csi_out0: endpoint@1 {
					status = "okay";
					remote-endpoint = <&adv748x_vi_in0>;
				};
			};
		};
	};
};

};

hdmi_in {
status = “okay”;
#address-cells = <1>;
#size-cells = <0>;

ports {
	#address-cells = <1>;
	#size-cells = <0>;
	
	port@0 {
		reg = <0>;
		hdmi_in: endpoint@0 {
			remote-endpoint = <&adv748x_hdmi0>;
		};
	};
};

};

i2c@c240000 { // HDMI or SDI
status = “okay”;
#address-cells = <1>;
#size-cells = <0>;

adv748x@70 {
	compatible = "adv748x";
	reg = <0x70>;		// HDMI

	ports {
		#address-cells = <1>;
		#size-cells = <0>;

		port@0 {
			reg = <0>;
			adv748x_out0: endpoint@0 {
				port-index = <0>;
				bus-width = <2>;
				remote-endpoint = <&adv748x_csi_in0>;
			};
		};
		port@1 {
		 	reg = <1>;
		 	adv748x_hdmi0: endpoint@1 {
		 		port-index = <0>;
		 		bus-width = <2>;
				remote-endpoint = <&hdmi_in>;
		 	};
		};
	};
}; 

};

I tried this device-tree but failed because I don’t know which ‘hdmi_in’ property should be used.

Could you tell me which property should I use? or any examples?

Thanks.

hello Linuxman123,

you don’t need a really remote-endpoint as hdmi_in, please use adv748x_out to replace hdmi_in to define the port binding.
it looks like this… adv748x_out => csi_in => csi_out => vi_in.
after that, the linux kernel should recognize it and register video node to link to adv748x.

Hello Jerry. Thanks for your quick reply.

Actually, adv748x_out => csi_in => csi_out => vi_in topology is my original device-tree which I already tried.(My first question above describes about that)

As you said, video node was successfully generated but in perspective of media controller framework, txa entity and nvcsi entity pads were disconnected.

I don’t know why It’s not connected.

hello Linuxman123,

I don’t understand this part,
may I know what’s the difference between adv748x 1-0070 txa and adv748x 1-0070 hdmi,
why you’re having such port bindings.
for example,

entity 4: adv748x 1-0070 txa (2 pads, 1 link)
type V4L2 subdev subtype Sensor flags 0
device node name /dev/v4l-subdev2
pad0: Sink
[fmt:unknown/0x0]
← “adv748x 1-0070 hdmi”:1 [ENABLED,IMMUTABLE]
pad1: Source
[fmt:unknown/0x0]

entity 7: adv748x 1-0070 hdmi (2 pads, 1 link)
type V4L2 subdev subtype Sensor flags 0
device node name /dev/v4l-subdev1
pad0: Sink
[fmt:YUYV8_1X16/1920x1080 field:none colorspace:smpte170m]
[dv.caps:BT.656/1120 min:1x1@0 max:10000x10000@165000000 stds:CEA-861,DMT,CVT,GTF caps:progressive,reduced-blanking,custom]
pad1: Source
[dv.caps:BT.656/1120 min:1x1@0 max:10000x10000@165000000 stds:CEA-861,DMT,CVT,GTF caps:progressive,reduced-blanking,custom]
[dv.detect:BT.656/1120 1920x1080p30 (2200x1125) stds: flags:]
[dv.current:BT.656/1120 1920x1080p25 (2640x1125) stds:CEA-861 flags:CE-video]
→ “adv748x 1-0070 txa”:0 [ENABLED,IMMUTABLE]

Hello Jerry.

adv748x can be used as HDMI/Analog to MIPI CSI-2 converter.

In adv748x driver, it has input entity which is about HDMI/Analog(AFE) and output entity which is about output.

In my situation, I use adv748x as HDMI to MIPI CSI-2 converter. In driver code, both input entity HDMI and output entity MIPI CSI initialized by v4l2_i2c_subdev_init.

As you see, HDMI entity → TxA entity linked correctly but TxA entity → NVCSI entity didn’t linked and I don’t know why.

Thanks.

hello Linuxman123,

my meant, why don’t you have the port binding to connect adv748x 1-0070 hdmi with nvcsi directly?

Yes It can output when HDMI directly connect to NVCSI but My real goal is porting adv748x driver with their own purpose.
Such as… linking 2 entity with my flavor (Like Analog input entity → TxB output entity)

hello Linuxman123,

sorry, we don’t have such reference driver without connect HDMI to NVCSI directly.