Jetson nano padd lt6911uxc hdmi to mip streaming error :video4linux video0: frame start syncpt timeout!0

lt6911uxc__fromNvida.txt (13.1 KB)

Hi
We are working on POC(proof Of Concept) of build connecting Jetson Nano interfacing lontium lt6911uxc chip convert hdmi to mipi as image sensor with SOM (similar to interfacing IMX327 image sensor).
We use CAM I2C for sensor control interface and CSI 2 ports with 2 lanes for data capture.
We used kernel driver from from nvidia lt6911uxc.c (attached as .txt since it is not allowed to upload c file)

We took JetPack 4.6 code with L4T 32.6.1.
We build Kernel Image & dtb.
We add 2 dtsi files at:ubuntu:~/Linux_for_Tegra/source/public/hardware/nvidia/platform/t210/porg/ker
nel-dts/porg-platforms
tegra210-camera-rbpcv2-lt6911uxc.dtsi,
tegra210-porg-camera-rbpcv2-lt6911uxc.dtsi
The files are simillar to imx29 dtsi attached with l4t nvidia porg-platform.

We also update the tegra210-p3448-0002-p3449-0000-b00.dts by remove imx219 and add lt6911uxc dtsi:
//include “porg-platforms/tegra210-porg-camera-rbpcv2-dual-imx219.dtsi”
include “porg-platforms/tegra210-porg-camera-rbpcv2-dual-lt6911uxc.dtsi”

We loaded the dtb and the lt6911uxc.ko.
The linux kernel start media and nvcsi according to the dtb call the probe of the lt6911uxc.ko driver.It create /dev/media0 and /dev/video0 and /dev/video1.
So, we got the driver on.

But, we encountered a problem with the streaming .
We used the v4l2-ctl and cuda sample as described at Lontium User Guide document .
While run the command:
v4l2-ctl -d /dev/video0 --set-fmt-video=width=1920,height=1080,pixelformat=UYVY–set-ctrl --stream-mmap --stream-count=300

we got the following output:
[ 293.401744] video4linux video0: frame start syncpt timeout!0
[ 293.609738] video4linux video0: frame start syncpt timeout!0

While running the command :
./camera_v4l2_cuda -d /dev/video0 -s 1920x1080 -f UVVY -n 300 -r 30

We got the screen blank and on uart debug console we got per each frame buffer the print :
video4linux video0: frame start syncpt timeout!0
and the result of the cuda sample is get blank screen for the 300 frames and got on the terminal :
./12_camera_v4l2_cuda -d /dev/video0 -s 1920x1080 -f UYVY -v
root@uvision-desktop:/usr/src/jetson_multimedia_api/samples/12_camera_v4l2_cuda# ./camera_v4l2_cuda -d /dev/video0 -s 1920x1080 -f UYVY -n 300 -r 60 -v

WARN: camera_initialize(): (line:297) >> yud:ioctl VIDIOC_G_PARM failed line 297 err= -1 errno= 25 Inappropriate ioctl for device

INFO: camera_initialize(): (line:316) Camera ouput format[devName/dev/video0]: (1920 x 1080) stride: 3840, imagesize: 4147200, frate: 0 / 0
[INFO] (NvEglRenderer.cpp:110) Setting Screen width 1920 height 1080
INFO: init_components(): (line:355) Initialize v4l2 components successfully
WARN: request_camera_buff(): (line:393) Camera v4l2 buf length is not expected
WARN: request_camera_buff(): (line:394) Camera v4l2 buf length is not expected ctxBufSz=0 bufLn= 4147200
WARN: request_camera_buff(): (line:393) Camera v4l2 buf length is not expected
WARN: request_camera_buff(): (line:394) Camera v4l2 buf length is not expected ctxBufSz=0 bufLn= 4147200
WARN: request_camera_buff(): (line:393) Camera v4l2 buf length is not expected
WARN: request_camera_buff(): (line:394) Camera v4l2 buf length is not expected ctxBufSz=-1673812816 bufLn= 4147200
WARN: request_camera_buff(): (line:393) Camera v4l2 buf length is not expected
WARN: request_camera_buff(): (line:394) Camera v4l2 buf length is not expected ctxBufSz=-1673812784 bufLn= 4147200
INFO: prepare_buffers(): (line:541) Succeed in preparing stream buffers
INFO: start_stream(): (line:558) Camera video streaming on …
----------- Element = renderer0 -----------
Total Profiling time = 0
Average FPS = 0
Total units processed = 0
Num. of late units = 0

INFO: stop_stream(): (line:724) Camera video streaming off …
App run was successful

        In parallel we see on the debug console , per each frame a line :
      [  xxxxxxxxxxx] video4linux video0: frame start syncpt timeout!0

As you can see I added the error print of ioctl() call:
WARN: camera_initialize(): (line:297) >> yud:ioctl VIDIOC_G_PARM failed line 297 err= -1 errno= 25 Inappropriate ioctl for device

and also it prints : frate 0/0

The driver load output is ok:
.487027] lt6911uxc 7-002b: probing lt6911uxc v4l2 sensor at addr 0x2b
[ 8.504034] lt6911uxc 7-002b: mclk absent,assuming sensor driven externally
[ 8.526465] lt6911uxc 7-002b: tegracam sensor driver:lt6911uxc_v2.0.6
[ 8.568867] lt6911uxc 8-002b: probing lt6911uxc v4l2 sensor at addr 0x2b

uvision@uvision-desktop:~$ v4l2-ctl -d /dev/video1 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Index : 0
Type : Video Capture
Pixel Format: ‘UYVY’
Name : UYVY 4:2:2
Size: Discrete 1920x1080
Interval: Discrete 0.017s (60.000 fps)
Size: Discrete 3840x2160
Interval: Discrete 0.017s (60.000 fps)
Size: Discrete 1280x720
Interval: Discrete 0.017s (60.000 fps)

    Index       : 1
    Type        : Video Capture
    Pixel Format: 'NV16'
    Name        : Y/CbCr 4:2:2
            Size: Discrete 1920x1080
                    Interval: Discrete 0.017s (60.000 fps)
            Size: Discrete 3840x2160
                    Interval: Discrete 0.017s (60.000 fps)
            Size: Discrete 1280x720
                    Interval: Discrete 0.017s (60.000 fps)

    Index       : 2
    Type        : Video Capture
    Pixel Format: 'UYVY'
    Name        : UYVY 4:2:2
            Size: Discrete 1920x1080
                    Interval: Discrete 0.017s (60.000 fps)
            Size: Discrete 3840x2160
                    Interval: Discrete 0.017s (60.000 fps)
            Size: Discrete 1280x720
                    Interval: Discrete 0.017s (60.000 fps)

uvision@uvision-desktop:~$ sudo media-ctl -p -d /dev/media0
Media controller API version 0.1.0

Media device information

driver vi
model NVIDIA Tegra Video Input Device
serial
bus info
hw revision 0x3
driver version 0.0.0

Device topology

  • entity 1: nvcsi–2 (2 pads, 2 links)
    type V4L2 subdev subtype Unknown flags 0
    device node name /dev/v4l-subdev0
    pad0: Sink
    ← “lt6911uxc 7-002b”:0 [ENABLED]
    pad1: Source
    → “vi-output, lt6911uxc 7-002b”:0 [ENABLED]

  • entity 4: nvcsi–1 (2 pads, 2 links)
    type V4L2 subdev subtype Unknown flags 0
    device node name /dev/v4l-subdev1
    pad0: Sink
    ← “lt6911uxc 8-002b”:0 [ENABLED]
    pad1: Source
    → “vi-output, lt6911uxc 8-002b”:0 [ENABLED]

  • entity 63: lt6911uxc 7-002b (1 pad, 1 link)
    type V4L2 subdev subtype Sensor flags 0
    device node name /dev/v4l-subdev2
    pad0: Source
    [fmt:UYVY8_1X16/1920x1080 field:none colorspace:srgb]
    → “nvcsi–2”:0 [ENABLED]

  • entity 65: vi-output, lt6911uxc 7-002b (1 pad, 1 link)
    type Node subtype V4L flags 0
    device node name /dev/video0
    pad0: Sink
    ← “nvcsi–2”:1 [ENABLED]

  • entity 79: lt6911uxc 8-002b (1 pad, 1 link)
    type V4L2 subdev subtype Sensor flags 0
    device node name /dev/v4l-subdev3
    pad0: Source
    [fmt:UYVY8_1X16/1920x1080 field:none colorspace:srgb]
    → “nvcsi–1”:0 [ENABLED]

  • entity 81: vi-output, lt6911uxc 8-002b (1 pad, 1 link)
    type Node subtype V4L flags 0
    device node name /dev/video1
    pad0: Sink
    ← “nvcsi–1”:1 [ENABLED]

What can cause such case and how can I get more debug info ?

The tegra210-porg-camera-rbpcv2-dual-lt6911.dtsi is:

/*

  • Copyright (c) 2018-2021, NVIDIA CORPORATION. All rights reserved.
  • This program is free software; you can redistribute it and/or modify
  • it under the terms of the GNU General Public License as published by
  • the Free Software Foundation; either version 2 of the License, or
  • (at your option) any later version.
  • This program is distributed in the hope that it will be useful, but WITHOUT
  • ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
  • FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
  • more details.
  • You should have received a copy of the GNU General Public License
  • along with this program. If not, see http://www.gnu.org/licenses/.
    */

include “tegra210-camera-rbpcv2-dual-lt6911.dtsi”

define CAM_BUSY TEGRA_GPIO(S, 1)
define HDMI_RESET TEGRA_GPIO(T, 0)
define CAM_BUSY1 TEGRA_GPIO(S, 0)
define HDMI_RESET1 TEGRA_GPIO(S, 7)

/ {
cam_i2cmux {
compatible = “i2c-mux-gpio”;
#address-cells = <1>;
#size-cells = <0>;
mux-gpios = <&gpio CAM_I2C_MUX GPIO_ACTIVE_HIGH>;
i2c-parent = <&i2c7>;
i2c@0 {
reg = <0>;
#address-cells = <1>;
#size-cells = <0>;
rbpcv2_lt6911uxc_a@2b {
reset-gpios = <&gpio HDMI_RESET1 GPIO_ACTIVE_HIGH>;
pwdn-gpios = <&gpio CAM_BUSY1 GPIO_ACTIVE_HIGH>;
};
};
i2c@1 {
reg = <1>;
#address-cells = <1>;
#size-cells = <0>;
rbpcv2_lt6911uxc_e@2b {
reset-gpios = <&gpio HDMI_RESET GPIO_ACTIVE_HIGH>;
pwdn-gpios = <&gpio CAM_BUSY GPIO_ACTIVE_HIGH>;
};
};
};

gpio@6000d000 {
	camera-control-output-low {
		gpio-hog;
		output-low;
		gpios = < HDMI_RESET 0  HDMI_RESET1 0>;
		label = "cam1-pwdn", "cam2-pwdn";
	};
};

};

The tegra210-porg-camera-rbpcv2-dual-imx219.dtsi is:

/*

  • Copyright (c) 2018-2021, NVIDIA CORPORATION. All rights reserved.
  • This program is free software; you can redistribute it and/or modify
  • it under the terms of the GNU General Public License as published by
  • the Free Software Foundation; either version 2 of the License, or
  • (at your option) any later version.
  • This program is distributed in the hope that it will be useful, but WITHOUT
  • ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
  • FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
  • more details.
  • You should have received a copy of the GNU General Public License
  • along with this program. If not, see http://www.gnu.org/licenses/.
    */

include “tegra210-camera-rbpcv2-dual-lt6911.dtsi”

define CAM_BUSY TEGRA_GPIO(S, 1)
define HDMI_RESET TEGRA_GPIO(T, 0)
define CAM_BUSY1 TEGRA_GPIO(S, 0)
define HDMI_RESET1 TEGRA_GPIO(S, 7)

/ {
cam_i2cmux {
compatible = “i2c-mux-gpio”;
#address-cells = <1>;
#size-cells = <0>;
mux-gpios = <&gpio CAM_I2C_MUX GPIO_ACTIVE_HIGH>;
i2c-parent = <&i2c7>;
i2c@0 {
reg = <0>;
#address-cells = <1>;
#size-cells = <0>;
rbpcv2_lt6911uxc_a@2b {
reset-gpios = <&gpio HDMI_RESET1 GPIO_ACTIVE_HIGH>;
pwdn-gpios = <&gpio CAM_BUSY1 GPIO_ACTIVE_HIGH>;
};
};
i2c@1 {
reg = <1>;
#address-cells = <1>;
#size-cells = <0>;
rbpcv2_lt6911uxc_e@2b {
reset-gpios = <&gpio HDMI_RESET GPIO_ACTIVE_HIGH>;
pwdn-gpios = <&gpio CAM_BUSY GPIO_ACTIVE_HIGH>;
};
};
};

gpio@6000d000 {
	camera-control-output-low {
		gpio-hog;
		output-low;
		gpios = < HDMI_RESET 0  HDMI_RESET1 0>;
		label = "cam1-pwdn", "cam2-pwdn";
	};
};

};

hello yehudaamar1965,

it’s report failures that VI engine cannot receive start-of-frame correctly.
you may arrange hardware resources to probe the hardware signaling, for instance, please confirm you have given correct power-supply, and this bridge device it’s sending MIPI packets to CSI channel.

furthermore,
as you’re working with lt6911uxc, which is a HDMI2CSI bridge driver. you may refer to similar driver, such as tc358840 for reference.
for example,
$public_sources/kernel_src/hardware/nvidia/platform/t210/jetson/kernel-dts/jetson-platforms/tegra210-camera-imx274-hdmi.dtsi

Hi again

Thanx for the response.
There is a question I asked and you did not reffered it, so I point to it again:
I asked regarding the using the cuda sample is get blank screen for the 300 frames and got on the terminal :
./12_camera_v4l2_cuda -d /dev/video0 -s 1920x1080 -f UYVY -v
root@uvision-desktop:/usr/src/jetson_multimedia_api/samples/12_camera_v4l2_cuda# ./camera_v4l2_cuda -d /dev/video0 -s 1920x1080 -f UYVY -n 300 -r 60 -v

I added check status of ioctl(() at the camera_initialize()
int err = ioctl (ctx->cam_fd, VIDIOC_G_PARM, &streamparm);

and got that it fails on: int err = ioctl (ctx->cam_fd, VIDIOC_G_PARM, &streamparm);

WARN: camera_initialize(): (line:297) >> yud:ioctl VIDIOC_G_PARM failed line 297 err= -1 errno= 25 Inappropriate ioctl for device

It also print frate 0/0

INFO: camera_initialize(): (line:316) Camera ouput format[devName/dev/video0]: (1920 x 1080) stride: 3840, imagesize: 4147200, frate: 0 / 0

what can it indicate?

Thanx

hello yehudaamar1965,

it’s standard control, VIDIOC_G_PARM to query the camera settings,
since v4l2_cuda it also used v4l standard IOCTL to fetch the stream, you may resolve the low level failure first.

please see-also developer guide, Verifying the V4L2 Sensor Driver for troubleshooting.

Hi JerryChang
Thanx for you answer.Indeed, we found a hardware issue, while the hdmi connected to lt6911uxc negotiate with the screen and change internally the lane to 4 instead of 2 .We solved it.
Now, we have 2 more issues :
1. We used the Jetson nano EVB as described J13 CSI0 using 2 lanes, and to J49 CSI2 using 2 lanes.
We got streaming OK over video0 which is CSI0 and video is OK…
While we connect the same mipi output cable from lontium to the second port J49 CSI2 of the
Jetson EVB,and run the same commands described above, v4l2-ctl or cuda sample 12
camera_v4l2_cuda - there is no video and we got the print: video4linux video0: frame start syncpt
timeout!0
We just passed the cable from the CSI0 port.The dtsi is above, and seems simmilar and also
compared to the imx-219 dual dtsi.
So, do you have an idea what is wrong? (We tested it on 2 different etson EVBs and same behaviour
so it is not a hardware failure).

2. We have a custome board, and use CSI2 and CSI4 (CSI 4 insteat CSI0 at EVB)…
I changed at I chaned the tegra210-camera-rbpcv2-dual-lt6911.dtsi the vi port@0 from
port-index =<0> to port-index= <2>, and tegra_sinterface = “serial_c” (from serial_e)
It did not work.
Is there any other change to do? Is tehere any hardware setting to do for CSI4 to operate.
The driver loaded properly and we got /dev/video0 and /dev/video1 and registeredc to v4l2.
Should I also change the rbpcv2_lt6911uxc_a to rbpcv2_lt6911uxc_c or any other change?
I attached the dtsi for CSI0 and CSI2 for question 1 named
tegra210-camera-rbpcv2-dual-lt6911uxc___CSI0_CSI2.dtsi.txt ,
and tegra210-camera-rbpcv2-dual-lt6911uxc___CSI2_CSI4.dtsi.txt for question 2.
tegra210-camera-rbpcv2-dual-lt6911uxc___CSI4.dtsi.txt (20.1 KB)
tegra210-camera-rbpcv2-dual-lt6911uxc___CSI4.dtsi.txt (20.1 KB)

hello yehudaamar1965,

it looks you’ve attach two dtsi files with the same naming.
one of attachment had a failure with tegra_sinterface, it’s mismatch with the CSI/VI port definition.

anyways,
please check you’re using correct CSI configuration.
you may see-also [Jetson Nano Product Design Guide] for [Table 8-3. CSI Configuration].

tegra210-camera-rbpcv2-dual-lt6911uxc___CSI0_CSI2.dtsi.txt (20.1 KB)
tegra210-camera-rbpcv2-dual-lt6911uxc___CSI2_CSI4.dtsi.txt (20.1 KB)

Hi JerryChang

  1. I attached the correct dtsi files regarding the issue with CSI4 instead CSI0 from former post
    issue number 2.
    The dtsi for this is the attached tegra210-camera-rbpcv2-dual-lt6911uxc___CSI2_CSI4.dtsi.txt

  2. You did not reffered to my question Number 1 issue of the former post regarding Jetson nano
    EVB connecting to J49 CSI2 using 2 lanes that streaming not working while J13 CSI0 using 2
    lanes working.We just took the cable from cam0 J13 and connected it to cam2 J49, so the input
    is the same one.Please reffer to the question 1 details on former post.We used the attached dtsi
    tegra210-camera-rbpcv2-dual-lt6911uxc___CSI0_CSI2.dtsi.txt

    thanx
    Yehuda

hello yehudaamar1965,

let me review device tree separately, and share my update as following.

(1) tegra210-camera-rbpcv2-dual-lt6911uxc___CSI0_CSI2.dtsi.txt (20.1 KB)

		i2c_0: i2c@0 {
			lt6911uxc_cam0: rbpcv2_lt6911uxc_a@2b {
				...
				ports {
					port@0 {
						reg = <0>;
						rbpcv2_lt6911uxc_dual_out0: endpoint {
						port-index = <2>;    <== this is incorrect. 

this is minor,
please also update the badge property as other unique strings.
or, it’ll load the default IMX219 camera settings

		modules {
			cam_module0: module0 {
				badge = "porg_front_RBPCV2";

			cam_module1: module1 {
				badge = "porg_rear_RBPCV2";

(2) tegra210-camera-rbpcv2-dual-lt6911uxc___CSI2_CSI4.dtsi.txt (20.1 KB)

this device tree settings looks okay, could you please describe the issue again?
BTW, if you have different issues, it’s better to have separate thread for tracking.

Hi
I want to inform that the video streaming is working.
CSI2 at evb - the problem was that the CSI2 has conflict of port-index<> at the tegra210-porg-plugin-manager.dtsi (the original was fine but while debug we change it and did not rollback…
The CSI4 at our custom : I changed it at the dtsi to port-index<2> and tegra_sinterface to “serial_c” BUT at egra210-porg-plugin-manager.dtsi for vi_base: vi ports port-index<0> was as at EVB
So, we had to verify that there is no overlay at the plugin-mamager

Thanx for ypur support

Yehuda