Single camera wrongly mapping to multiple /dev/videoX

[jetsondev23]

Hello,

I have a problem with a multi-camera setup. There are two ov9782 and one imx477. Three video devices seem correctly created: /dev/video0, /dev/video1, /dev/video2.

Somehow, when using nvarguscamerasrc with gst-launch-1.0, the IMX is the one being used with both sensor-id=0 and sensor-id=2, usable at the same time.

The test pipeline is the following, with $1 being one of 0,1,2
gst-launch-1.0 nvarguscamerasrc sensor-id=$1 ! 'video/x-raw(memory:NVMM), width=1280, height=800, framerate=60/1' ! omxh264enc control-rate=2 bitrate=4000000 ! video/x-h264 ! rtspclientsink location=rtsp://127.0.0.1:8554/cam$1

I figured it should be a CSI ↔ VI mapping issue but I am completely stuck, the configuration looks correct to me.
This is a JetsonNano SOM, with a custom carrier board.

The ov9782 are on CSI A and CSI E, 2 lanes each. The imx477 is on CSI C+D, 4 lanes.

media-ctl --print-dot:

$ media-ctl -p
Media controller API version 0.1.0

Media device information
------------------------
driver          vi
model           NVIDIA Tegra Video Input Device
serial          
bus info        
hw revision     0x3
driver version  0.0.0

Device topology
- entity 1: nvcsi--3 (2 pads, 2 links)
            type V4L2 subdev subtype Unknown flags 0
            device node name /dev/v4l-subdev0
	pad0: Sink
		<- "imx477 30-0010":0 [ENABLED]
	pad1: Source
		-> "vi-output, imx477 30-0010":0 [ENABLED]

- entity 4: imx477 30-0010 (1 pad, 1 link)
            type V4L2 subdev subtype Sensor flags 0
            device node name /dev/v4l-subdev1
	pad0: Source
		[fmt:SRGGB10_1X10/1920x1080 field:none colorspace:srgb]
		-> "nvcsi--3":0 [ENABLED]

- entity 6: vi-output, imx477 30-0010 (1 pad, 1 link)
            type Node subtype V4L flags 0
            device node name /dev/video0
	pad0: Sink
		<- "nvcsi--3":1 [ENABLED]

- entity 18: nvcsi--2 (2 pads, 2 links)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev2
	pad0: Sink
		<- "ov9782 60-0010":0 [ENABLED]
	pad1: Source
		-> "vi-output, ov9782 60-0010":0 [ENABLED]

- entity 21: ov9782 60-0010 (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev3
	pad0: Source
		[fmt:SBGGR10_1X10/1280x800 field:none colorspace:srgb]
		-> "nvcsi--2":0 [ENABLED]

- entity 23: vi-output, ov9782 60-0010 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video1
	pad0: Sink
		<- "nvcsi--2":1 [ENABLED]

- entity 35: nvcsi--1 (2 pads, 2 links)
             type V4L2 subdev subtype Unknown flags 0
             device node name /dev/v4l-subdev4
	pad0: Sink
		<- "ov9782 61-0010":0 [ENABLED]
	pad1: Source
		-> "vi-output, ov9782 61-0010":0 [ENABLED]

- entity 38: ov9782 61-0010 (1 pad, 1 link)
             type V4L2 subdev subtype Sensor flags 0
             device node name /dev/v4l-subdev5
	pad0: Source
		[fmt:SBGGR10_1X10/1280x800 field:none colorspace:srgb]
		-> "nvcsi--1":0 [ENABLED]

- entity 40: vi-output, ov9782 61-0010 (1 pad, 1 link)
             type Node subtype V4L flags 0
             device node name /dev/video2
	pad0: Sink
		<- "nvcsi--1":1 [ENABLED]

The following should be the relevant dtsi files
tegra210-camera-rbpcv2-dual-ov9782_tegra.dtsi (29.0 KB)
tegra210-porg-camera-rbpcv2-dual-ov9782.dtsi (2.6 KB)

Thanks in advance for any help you can provide.

[jetsondev23]

I found some more documentation on this, do I understand correctly that only 2 camera streams can be ISP processed and any additional ones need to go through v4l2? Or what does the “presentation to system memory” from the TX1 TRM mean?

[ShaneCCC]

The port-index is incorrect, it should be 4 instead of 2.

			channel@2 {
				reg = <2>;
				ports {
					#address-cells = <1>;
					#size-cells = <0>;
					port@0 {
						reg = <0>;
						rbpcv3_ov9782_csi_in1: endpoint@2 {
							port-index = <2>;
							bus-width = <2>;
							remote-endpoint = <&rbpcv3_ov9782_out1>;
						};
					};

[jetsondev23]

Indeed, I made a mistake while reverting the files back from some testing I’ve been doing to fix it. port-index = <4> now but the imx stream still ends-up at /dev/video0 and /dev/video2.

Using only the two ov9782 works fine by the way.

Is it even possible to have three streams without additional complex manipulations? Is the “presentation to system memory” from the TRM something the nano does by itself before going back into the ISP or do I have to configure something more?

[ShaneCCC]

There’s no problem to support to 4 cameras for directly connect to Nano.
Please dump the device tree to check the configure again.

sudo dtc -I fs -O dts -o extracted_proc.dts /proc/device-tree

[jetsondev23]

extracted_proc.dts (313.6 KB)

Looks like it is the correct port now (line 6381)

			channel@2 {
				reg = <0x2>;

				ports {
					#address-cells = <0x1>;
					#size-cells = <0x0>;

					port@1 {
						reg = <0x1>;

						endpoint@3 {
							remote-endpoint = <0x82>;
							phandle = <0x61>;
							linux,phandle = <0x61>;
						};
					};

					port@0 {
						reg = <0x0>;

						endpoint@2 {
							port-index = <0x4>;
							remote-endpoint = <0x81>;
							bus-width = <0x2>;
							phandle = <0x7c>;
							linux,phandle = <0x7c>;
						};
					};
				};
			};

[ShaneCCC]

The position is incorrect.

https://docs.nvidia.com/jetson/archives/r35.3.1/DeveloperGuide/text/SD/CameraDevelopment/SensorSoftwareDriverProgramming.html?highlight=set_mode_delay_ms#module-properties

[jetsondev23]

Thank you very much for your help, correcting the position and badge indeed fixes the problem. I wrongly assumed they are only metadata accessible through v4l or similar as information to the user.