Multiple resolutions/framerates with TC358743

Robotics & Edge Computing Jetson & Embedded Systems Jetson Nano
1

Hi,

Followed the guides (mainly this) posted in another thread and managed to get my HDMI-to-CSI conversion board working quite quickly. However, I can’t get it to work with any other resolution&framerate combination other than 1080p@60FPS. The odd thing is that, when I compiled the driver module from the files here I was only able to get 720p@50FPS.

Does anyone have any tips on how to further progress here? I did a comparison between the two drivers and the main differences seem to be the EDID, but also the clocks (I think this is the problem). How would I be able to get further info on what the actual failure reason is?

Thank you!
Alex

3

hello alexx88,

may I know what’s the format dumps. $ v4l2-ctl -d /dev/video0 --list-formats-ext
please also share your capture pipeline you’re capturing the frames. thanks

4

jetson@ubuntu:~$ v4l2-ctl -d /dev/video0 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Index : 0
Type : Video Capture
Pixel Format: ‘AR24’
Name : 32-bit BGRA 8-8-8-8
Size: Discrete 1280x720
Size: Discrete 1920x1080

Index : 1
Type : Video Capture
Pixel Format: ‘UYVY’
Name : UYVY 4:2:2
Size: Discrete 1280x720
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.020s (50.000 fps)
Interval: Discrete 0.017s (60.000 fps)
Size: Discrete 1920x1080
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.020s (50.000 fps)
Interval: Discrete 0.017s (60.000 fps)

Index : 2
Type : Video Capture
Pixel Format: ‘NV16’
Name : Y/CbCr 4:2:2
Size: Discrete 1280x720
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.020s (50.000 fps)
Interval: Discrete 0.017s (60.000 fps)
Size: Discrete 1920x1080
Interval: Discrete 0.033s (30.000 fps)
Interval: Discrete 0.020s (50.000 fps)
Interval: Discrete 0.017s (60.000 fps)

and the command I use for capturing the frames (obviously adjusted for source resolution):

gst-launch-1.0 v4l2src device=/dev/video0 num-buffers=10000 ! ‘video/x-raw,format=UYVY,width=1920,height=1080’ ! nvvidconv ! omxh264enc ! mpegtsmux ! filesink location=test.ts

5

hello alexx88,

please have a try for using v4l standard IOCTL to access the stream,
for example, $ v4l2-ctl -d /dev/video0 --set-fmt-video=width=1920,height=1080,pixelformat=UYVY--set-ctrl bypass_mode=0 --stream-mmap --stream-count=100

it’ll show < for each success capture frames, and report the fps every second.
please have a try and sharing your test results.

6

Thanks for the suggestion. Much easier to test like this. Essentially any other input format different than 1080p@60 produced this:

jetson@ubuntu:~$ v4l2-ctl -d /dev/video0 --set-fmt-video=width=1080,height=720,pixelformat=UYVY --set-ctrl bypass_mode=0 --stream-mmap --stream-count=1000
New timings found

And then locked.

It sometimes is able to process other formats as well, but maybe 1/10 tries.

jetson@ubuntu:~$ v4l2-ctl -d /dev/video0 --set-fmt-video=width=1920,height=1080,pixelformat=UYVY --set-ctrl bypass_mode=0 --stream-mmap --stream-count=1000
New timings found
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 50.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 50.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 50.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 50.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 50.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 50.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 50.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<

If I were to guess, I’d say that the TC358743 driver I have in right now can’t handle anything besides 1080p@60. I am seeing a few driver errors with dmesg, but not familiar enough with the code to be able to tell what they mean (these errors appeared when trying to capture frames using gstreamer)

Error in tc358743_enum_frame_interval num framerates (3 outside 3)
Error in tc358743_enum_frame_size, 2 outside of num_frmfmt (2)
Unexpected code (4106), UYUV: 1498831189, ABGR32: 875708738

The interesting part is that going back into the driver’s kernel log, I can see that a lot of the frames are correctly detected:

[ 1833.691748] tc358743 6-000f: Trying to find frmfmt that matches fse->code, code: 8207 (UYVY: 1498831189, ARGB32: 875708738, MEDIA_BUS_FMT_UYVY8_1X16: 8207, MEDIA_BUS_FMT_RGB888_1X24: 4106)
[ 1833.691750] tc358743 6-000f: Code ok
[ 1833.691753] tc358743 6-000f: Index ok
[ 1833.691756] tc358743 6-000f: !!! tc358743_enum_frame_size() complete successfully, width: 1280, height: 720
[ 1833.691762] tc358743 6-000f: tc358743_enum_frame_interval()
[ 1833.691764] tc358743 6-000f: ----------------------------------------
[ 1833.691767] tc358743 6-000f: Trying to find frame interfval that matches fie->code, code: 8207 (UYVY: 1498831189, ARGB32: 875708738, MEDIA_BUS_FMT_UYVY8_1X16: 8207)
[ 1833.691769] tc358743 6-000f: Code ok
[ 1833.691772] tc358743 6-000f: Matched width 1280 and 1280, height 720 and 720
[ 1833.691774] tc358743 6-000f: w/h ok or end (i=0, num=2)
[ 1833.691776] tc358743 6-000f: i ok
[ 1833.691778] tc358743 6-000f: index ok
[ 1833.691781] tc358743 6-000f: !!! tc358743_enum_frame_interval() completed successfully, interval: 1/30
[ 1833.691790] tc358743 6-000f: tc358743_enum_frame_interval()
[ 1833.691792] tc358743 6-000f: ----------------------------------------
[ 1833.691795] tc358743 6-000f: Trying to find frame interfval that matches fie->code, code: 8207 (UYVY: 1498831189, ARGB32: 875708738, MEDIA_BUS_FMT_UYVY8_1X16: 8207)
[ 1833.691797] tc358743 6-000f: Code ok
[ 1833.691799] tc358743 6-000f: Matched width 1280 and 1280, height 720 and 720
[ 1833.691801] tc358743 6-000f: w/h ok or end (i=0, num=2)
[ 1833.691803] tc358743 6-000f: i ok
[ 1833.691805] tc358743 6-000f: index ok
[ 1833.691808] tc358743 6-000f: !!! tc358743_enum_frame_interval() completed successfully, interval: 1/50
[ 1833.691813] tc358743 6-000f: tc358743_enum_frame_interval()
[ 1833.691815] tc358743 6-000f: ----------------------------------------
[ 1833.691818] tc358743 6-000f: Trying to find frame interfval that matches fie->code, code: 8207 (UYVY: 1498831189, ARGB32: 875708738, MEDIA_BUS_FMT_UYVY8_1X16: 8207)
[ 1833.691819] tc358743 6-000f: Code ok
[ 1833.691822] tc358743 6-000f: Matched width 1280 and 1280, height 720 and 720
[ 1833.691824] tc358743 6-000f: w/h ok or end (i=0, num=2)
[ 1833.691826] tc358743 6-000f: i ok
[ 1833.691828] tc358743 6-000f: index ok
[ 1833.691830] tc358743 6-000f: !!! tc358743_enum_frame_interval() completed successfully, interval: 1/60
[ 1833.691835] tc358743 6-000f: tc358743_enum_frame_interval()
[ 1833.691837] tc358743 6-000f: ----------------------------------------
[ 1833.691840] tc358743 6-000f: Trying to find frame interfval that matches fie->code, code: 8207 (UYVY: 1498831189, ARGB32: 875708738, MEDIA_BUS_FMT_UYVY8_1X16: 8207)
[ 1833.691842] tc358743 6-000f: Code ok
[ 1833.691844] tc358743 6-000f: Matched width 1280 and 1280, height 720 and 720
[ 1833.691846] tc358743 6-000f: w/h ok or end (i=0, num=2)
[ 1833.691848] tc358743 6-000f: i ok
[ 1833.691851] tc358743 6-000f: Error in tc358743_enum_frame_interval num framerates (3 outside 3)

7

hello alexx88,

I’m curious about what does this New timings found message meant, and what’ll be kernel driver doing here?
is it changing the timing? if yes, this may be failure the capture pipeline.

8

I’m getting this error from ‘vi’ for each frame when trying to use 1280x720.

[ 557.712067] vi 54080000.vi: tegra_channel_error_status:error 4000 frame 0

9

It’s odd as querying the detected timings always yields the correct reply. How would I be able to get more debug data (can I get debug info from ‘vi’ ?).

10

hello alexx88,

you may check VI driver as following,
please also refer to TRM for register descriptions.
$public_sources/kernel_src/kernel/nvidia/drivers/media/platform/tegra/camera/vi/vi2_fops.c

static int tegra_channel_error_status(struct tegra_channel *chan)
{
...
  		err = tegra_csi_error(csi_chan, index);

there’re some debug prints. i.e. dev_dbg()
you may enable dynamic debug flag to gather more details.
for example,

$ sudo -i 
# cd /sys/kernel/debug/dynamic_debug/
# echo file vi2_fops.c +p > control
# echo file channel.c +p > control
11

I’m not getting any extra debugging info in ‘dmesg’ with dynamic debug. I’d assume by default there’s no debug info in the vi_fops.c driver and I need to add it?

12

hello alexx88,

may I know which Jetpack release you’re working with?
the other approach is Building the NVIDIA Kernel manually.

13

So went in a bit of a different route. As I might have mentioned earlier, I am able to get all resolutions to work, but need to use two different kernel modules compiled with different settings (both were linked in one of the threads discussing this IC). I hope I’ll get my hands on the TC358743 datasheet and timing calculator, so I can nail down the correct settings.

@JerryChang , is the TC358840 driver fully supported and working?

14

Fresh news. If I modify the device tree to force only one CSI lane, 720p works. As such I’m pretty sure the problem is actually in the number of lanes the CSI controller expects data to come through. If the device tree is configured with 2, then it won’t work with video streams that only use 1. I can see that a similar problem has been raised in this thread here CSI2 debug messages don't seem to be working - #11 by ShaneCCC , was there any conclusion?

Also, I can see that one of your colleagues said that the trace feature doesn’t work on Nano, what’s the approach to get debug data out of the vi driver?

[ 496.446293] vi 54080000.vi: tegra_channel_error_status:error 4000 frame 0
[ 496.462955] vi 54080000.vi: tegra_channel_error_status:error 4000 frame 1
[ 496.479664] vi 54080000.vi: tegra_channel_error_status:error 4000 frame 2

15

Hello Alexx,

Completely agree with your analysis.

I had encountered the same problem with a 1080x720@50 configuration.

[  197.060514] vi 54080000.vi: tegra_channel_error_status:error 4000 frame 2
[  197.074007] vi 54080000.vi: TEGRA_CSI_PIXEL_PARSER_STATUS 0x00004015

See ticket https://forums.developer.nvidia.com/t/camera-sensor-tc358743-erreur-timeout-frame-start-syncpt-timeout-0/208966

This configuration requires only one CSI line.

To solve it I forced the configuration to use a single line.

/ {
	host1x {
		vi_base: vi {
			num-channels = <1>;
			ports {
				#address-cells = <1>;
				#size-cells = <0>;
				vi_port0: port@0 {
					status = "okay";
					reg = <0>;
					tc358743_vi_in1: endpoint {
						status = "okay";
						port-index = <0>; // CSI-A
						bus-width = <2>;
						remote-endpoint = <&tc358743_csi_out0>;
					};
				};
			};
		};

		csi_base: nvcsi {
			num-channels = <1>;
			#address-cells = <1>;
			#size-cells = <0>;

			csi_chan0: channel@0 {
				reg = <0>;
				ports {
					#address-cells = <1>;
					#size-cells = <0>;
					csi_chan0_port0: port@0 {
						status = "okay";
						reg = <0>;
						tc358743_csi_in0: endpoint@0 {
							port-index = <0>;
							bus-width = <2>;
							remote-endpoint = <&tc358743_out1>;
						};
					};
					csi_chan0_port1: port@1 {
						status = "okay";
						reg = <1>;
						tc358743_csi_out0: endpoint@1 {
							remote-endpoint = <&tc358743_vi_in1>;
						};
					};
				};
			};
		};

		hdmi27m_osc: hdmi-osc {
			compatible = "fixed-clock";
			clock-output-names = "hdmi-osc";
			clock-frequency = <27000000>;
			#clock-cells = <0>;
		};

		i2c@546c0000 {
			status = "okay";
			#address-cells = <1>;
			#size-cells = <0>;
			tc358743@0f {
				status = "okay";
				compatible = "toshiba,tc358743";
				reg = <0x0f>;

				/* External clock */
				clocks = <&hdmi27m_osc>;
				clock-names = "refclk";
				
				/*reset-gpios      = <&gpio TEGRA_GPIO(S, 7) 0>;*/

				/* Physical dimensions of sensor */
				physical_w = "4.713";
				physical_h = "3.494";			
				
				/* Sensor Model */
				sensor_model ="tc358743";

				ports {
					#address-cells = <1>;
					#size-cells = <0>;

					port@0 {
						reg = <0>;
						// =============== v4l2_of_endpoint =============== //
						tc358743_out1: endpoint {
							port-index  = <0>; /* CSI A */
							bus-width   = <2>;
							phy_mode    = "DPHY";
							data-lanes  = <1 2>;
							clock-lanes = <0>;
							clock-noncontinuous;
							link-frequencies = /bits/ 64 <297000000>; // 297000000/486000000/594000000
							remote-endpoint = <&tc358743_csi_in0>;
						};
					};
				};
			};
		};
    };
};
16

I have long been using 1080p60 capture and halving the framerate, where I’d much prefer direct 1080p30 capture with the tc358743. Can either of you capture 1080p30 with your modified drivers?

@s.pastor I like your use of fixed clock in the dts over patching the driver source files as was done in the previous efforts on this forum. It does not appear to me that you have constrained the driver to use only one lane… I see bus-width = <2> and data-lanes = <1 2> in your dtsi fragment. But I do see the link-frequency halved at 297000000 vs 594000000. (I may be wrong, it’s been a year since I looked at this, and am feeling very unsure/rusty on this stuff).

17

@dave-lowe :
Yes sorry, in the example I am setting the clock to the correct frequency so that I only need one line for the desired configuration (in my case: 1080x720@50).

19

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.