TX2 CSI 4-lane with L4T r28.2

Dears
We design a TX2 extension board.
Verify camera interface MIPI CSI now.
Base on TX2 with L4T r28.2 .
We use ov5693 camera (from TX2 Dev kits) verify our CSI PortC(2 lane) and PortD(2 lane) could normal work. So the trace CSI PortC/D should be normal.
We use sony camera module that output format is YUV(uyuv), reference many forum info we could display 1080p30 on CSI PortC(2 lane), by
gst-launch-1.0 -v v4l2src device="/dev/video0" ! “video/x-raw,width=1920,height=1080, format=(string)I420” ! nvvidconv ! “video/x-raw(memory:NVMM)” ! nvoverlaysink

Currently we try to use 1080p30 4 lane(same sony camera module could setting output) on CSI PortC and PortD but get green screen in monitor.
Below is what I modify base on 2 lane(PortC) chang to 4 lane(PortC+D) in camera module dts file.
host1x {
vi@15700000 {

csi-port = <2>; // same as 2 lane setup
bus-width = <4>;

};
nvcsi@150c0000 {

csi-port = <2>; // same as 2 lane setup
bus-width = <4>;

};
};

mode0 { // OV5693_MODE_2592X1944
    .....
    num_lanes = "4";
    tegra_sinterface = "serial_c";// same as 2 lane setup
    .....
};

ports {
	.....
	csi-port = <2>; // same as 2 lane setup
	bus-width = <4>;
	.....
};

tegra-camera-platform {
	.....
	num_csi_lanes = <4>;
	.....
};

when I excute below:
gst-launch-1.0 -v v4l2src device="/dev/video0" ! “video/x-raw,width=1920,height=1080, format=(string)I420” ! nvvidconv ! “video/x-raw(memory:NVMM)” ! nvoverlaysink
got below message:
[ 102.006977] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
[ 102.006977] tegra-vi4 15700000.vi: PXL_SOF syncpt timeout! err = -11
After Ctrl-C get below:
[ 104.058920] tegra-vi4 15700000.vi: ATOMP_FE syncpt timeout!
[ 104.151026] nvcsi 150c0000.nvcsi: csi4_stream_check_status (2) INTR_STATUS 0x00010000
[ 104.158902] nvcsi 150c0000.nvcsi: csi4_stream_check_status (2) ERR_INTR_STATUS 0x00010000

Could you provide suggestion, thanks.

To enable trace for more information

https://elinux.org/Jetson_TX2/28.1_Camera_BringUp

root@tegra-ubuntu:~# cat /sys/kernel/debug/tracing/trace# tracer: nop

entries-in-buffer/entries-written: 276/276 #P:4

_-----=> irqs-off

/ _----=> need-resched

| / _—=> hardirq/softirq

|| / _–=> preempt-depth

||| / delay

TASK-PID CPU# |||| TIMESTAMP FUNCTION

| | | |||| | |

 kworker/4:1-2318  [004] ...1  5712.264592: rtos_queue_peek_from_isr_failed: tstamp:178899555062 queue:0x0b4a3c58
 kworker/4:1-2318  [004] ...1  5712.264596: rtcpu_start: tstamp:178899555920
 kworker/4:1-2318  [004] ...1  5712.264598: rtcpu_vinotify_handle_msg: tstamp:178901908851 tag:CSIMUX_FRAME channel:0x00 frame:2792 vi_tstamp:2808248987 data:0x000000a2
 kworker/4:1-2318  [004] ...1  5712.264599: rtcpu_vinotify_handle_msg: tstamp:178901908996 tag:CHANSEL_SHORT_FRAME channel:0x04 frame:2792 vi_tstamp:2808248988 data:0x00000001
 kworker/4:1-2318  [004] ...1  5712.264613: rtcpu_vinotify_handle_msg: tstamp:178901909131 tag:ATOMP_FE channel:0x00 frame:2792 vi_tstamp:2808248990 data:0x00000001
 kworker/4:1-2318  [004] ...1  5712.264614: rtcpu_vinotify_handle_msg: tstamp:178902950215 tag:CSIMUX_FRAME channel:0x00 frame:3307 vi_tstamp:2809290661 data:0x000000a2
 kworker/4:1-2318  [004] ...1  5712.264615: rtcpu_vinotify_handle_msg: tstamp:178903471048 tag:CSIMUX_FRAME channel:0x00 frame:13293 vi_tstamp:2809811497 data:0x000000a2
 kworker/4:1-2318  [004] ...1  5712.316389: rtcpu_vinotify_handle_msg: tstamp:178904512727 tag:CSIMUX_FRAME channel:0x00 frame:13806 vi_tstamp:2810853171 data:0x000000a2
 kworker/4:1-2318  [004] ...1  5712.316392: rtos_queue_peek_from_isr_failed: tstamp:178904555902 queue:0x0b4a3c58
 kworker/4:1-2318  [004] ...1  5712.368383: rtcpu_vinotify_handle_msg: tstamp:178906075230 tag:CSIMUX_FRAME channel:0x00 frame:2544 vi_tstamp:2812415682 data:0x000000a2
 kworker/4:1-2318  [004] ...1  5712.368384: rtcpu_vinotify_handle_msg: tstamp:178907116902 tag:CSIMUX_FRAME channel:0x00 frame:4083 vi_tstamp:2813457354 data:0x000000a2
 kworker/4:1-2318  [004] ...1  5712.420383: rtcpu_vinotify_handle_msg: tstamp:178907637739 tag:CSIMUX_FRAME channel:0x00 frame:12533 vi_tstamp:2813978191 data:0x000000a2
 kworker/4:1-2318  [004] ...1  5712.472388: rtcpu_vinotify_handle_msg: tstamp:178909200250 tag:CSIMUX_FRAME channel:0x00 frame:14070 vi_tstamp:2815540701 data:0x000000a2
 kworker/4:1-2318  [004] ...1  5712.472390: rtos_queue_peek_from_isr_failed: tstamp:178909556409 queue:0x0b4a3c58
 kworker/4:1-2318  [004] ...1  5712.472391: rtcpu_vinotify_handle_msg: tstamp:178909721085 tag:CSIMUX_FRAME channel:0x00 frame:14073 vi_tstamp:2816061539 data:0x000000a2
 kworker/4:1-2318  [004] ...1  5712.524384: rtcpu_vinotify_handle_msg: tstamp:178910762758 tag:CSIMUX_FRAME channel:0x00 frame:12538 vi_tstamp:2817103212 data:0x000000a2
 kworker/4:1-2318  [004] ...1  5712.576384: rtcpu_vinotify_handle_msg: tstamp:178912325269 tag:CSIMUX_FRAME channel:0x00 frame:4092 vi_tstamp:2818665722 data:0x000000a2
 kworker/4:1-2318  [004] ...1  5712.576385: rtcpu_vinotify_handle_msg: tstamp:178913887781 tag:CSIMUX_FRAME channel:0x00 frame:2559 vi_tstamp:2820228232 data:0x000000a2
 kworker/4:1-2318  [004] ...1  5712.628381: rtos_queue_peek_from_isr_failed: tstamp:178914556917 queue:0x0b4a3c58
 kworker/4:1-2318  [004] ...1  5712.628382: rtcpu_vinotify_handle_msg: tstamp:178914929454 tag:CSIMUX_FRAME channel:0x00 frame:1539 vi_tstamp:2821269906 data:0x000000a2
 kworker/4:1-2318  [004] ...1  5712.628382: rtcpu_vinotify_handle_msg: tstamp:178915450288 tag:CSIMUX_FRAME channel:0x00 frame:14597 vi_tstamp:2821790742 data:0x000000a2
 kworker/4:1-2318  [004] ...1  5712.680394: rtcpu_vinotify_handle_msg: tstamp:178917012817 tag:CSIMUX_FRAME channel:0x00 frame:16134 vi_tstamp:2823353253 data:0x000000a2
 kworker/4:1-2318  [004] ...1  5712.732388: rtcpu_vinotify_handle_msg: tstamp:178917533638 tag:CSIMUX_FRAME channel:0x00 frame:16137 vi_tstamp:2823874089 data:0x000000a2
 kworker/4:1-2318  [004] ...1  5712.732390: rtcpu_vinotify_handle_msg: tstamp:178918575311 tag:CSIMUX_FRAME channel:0x00 frame:14602 vi_tstamp:2824915763 data:0x000000a2
 kworker/4:1-2318  [004] ...1  5712.784381: rtos_queue_peek_from_isr_failed: tstamp:178919557422 queue:0x0b4a3c58
 kworker/4:1-2318  [004] ...1  5712.784382: rtcpu_vinotify_handle_msg: tstamp:178920137819 tag:CSIMUX_FRAME channel:0x00 frame:1548 vi_tstamp:2826478273 data:0x000000a2
 kworker/4:1-2318  [004] ...1  5712.836383: rtcpu_vinotify_handle_msg: tstamp:178921179495 tag:CSIMUX_FRAME channel:0x00 frame:15 vi_tstamp:2827519946 data:0x000000a2
 kworker/4:1-2318  [004] ...1  5712.836384: rtcpu_vinotify_handle_msg: tstamp:178921700329 tag:CSIMUX_FRAME channel:0x00 frame:15377 vi_tstamp:2828040783 data:0x000000a2
 kworker/4:1-2318  [004] ...1  5712.888388: rtcpu_vinotify_handle_msg: tstamp:178922742004 tag:CSIMUX_FRAME channel:0x00 frame:14866 vi_tstamp:2829082456 data:0x000000a2
 kworker/4:1-2318  [004] ...1  5712.940386: rtcpu_vinotify_handle_msg: tstamp:178924304515 tag:CSIMUX_FRAME channel:0x00 frame:1300 vi_tstamp:2830644967 data:0x000000a2
 kworker/4:1-2318  [004] ...1  5712.940387: rtos_queue_peek_from_isr_failed: tstamp:178924557930 queue:0x0b4a3c58
 kworker/4:1-2318  [004] ...1  5712.940388: rtcpu_vinotify_handle_msg: tstamp:178924825353 tag:CSIMUX_FRAME channel:0x00 frame:791 vi_tstamp:2831165803 data:0x000000a2
 kworker/4:1-2318  [004] ...1  5712.992417: rtcpu_vinotify_handle_msg: tstamp:178926387860 tag:CSIMUX_FRAME channel:0x00 frame:792 vi_tstamp:2832728313 data:0x000000a2
 kworker/4:1-2318  [004] ...1  5713.044381: rtcpu_vinotify_handle_msg: tstamp:178927429532 tag:CSIMUX_FRAME channel:0x00 frame:1307 vi_tstamp:2833769986 data:0x000000a2
 kworker/4:1-2318  [004] ...1  5713.044383: rtcpu_vinotify_handle_msg: tstamp:178927950368 tag:CSIMUX_FRAME channel:0x00 frame:14877 vi_tstamp:2834290824 data:0x000000a2
 kworker/4:1-2318  [004] ...1  5713.096384: rtcpu_vinotify_handle_msg: tstamp:178929512879 tag:CSIMUX_FRAME channel:0x00 frame:15390 vi_tstamp:2835853334 data:0x000000a2
 kworker/4:1-2318  [004] ...1  5713.096385: rtos_queue_peek_from_isr_failed: tstamp:178929558435 queue:0x0b4a3c58
 kworker/4:1-2318  [004] ...1  5713.096386: rtcpu_vinotify_handle_msg: tstamp:178930033718 tag:CSIMUX_FRAME channel:0x00 frame:13089 vi_tstamp:2836374170 data:0x000000a2
 kworker/4:1-2318  [004] ...1  5713.148391: rtcpu_vinotify_handle_msg: tstamp:178931075391 tag:CSIMUX_FRAME channel:0x00 frame:13602 vi_tstamp:2837415844 data:0x000000a2
 kworker/4:1-2318  [004] ...1  5713.200443: rtcpu_vinotify_handle_msg: tstamp:178932637903 tag:CSIMUX_FRAME channel:0x00 frame:2596 vi_tstamp:2838978354 data:0x000000a2
 kworker/4:1-2318  [004] ...1  5713.200446: rtcpu_vinotify_handle_msg: tstamp:178933159034 tag:CSIMUX_FRAME channel:0x00 frame:3111 vi_tstamp:2839499191 data:0x000000a2
 kworker/4:1-2318  [004] ...1  5713.200447: rtcpu_vinotify_handle_msg: tstamp:178933159165 tag:CHANSEL_SHORT_FRAME channel:0x04 frame:3111 vi_tstamp:2839499191 data:0x00000001
 kworker/4:1-2318  [004] ...1  5713.200448: rtcpu_vinotify_handle_msg: tstamp:178933159298 tag:ATOMP_FE channel:0x00 frame:3111 vi_tstamp:2839499193 data:0x00000001
 kworker/4:1-2318  [004] ...1  5713.252460: rtos_queue_peek_from_isr_failed: tstamp:178934558961 queue:0x0b4a3c58
 kworker/4:1-2318  [004] ...1  5713.252463: rtcpu_vinotify_handle_msg: tstamp:178934721248 tag:CSIMUX_FRAME channel:0x00 frame:3112 vi_tstamp:2841061701 data:0x000000a2
 kworker/4:1-2318  [004] ...1  5713.304463: rtcpu_vinotify_handle_msg: tstamp:178935762946 tag:CSIMUX_FRAME channel:0x00 frame:2603 vi_tstamp:2842103374 data:0x000000a2
 kworker/4:1-2318  [004] ...1  5713.304468: rtcpu_vinotify_handle_msg: tstamp:178936283780 tag:CSIMUX_FRAME channel:0x00 frame:13613 vi_tstamp:2842624211 data:0x000000a2
 kworker/4:1-2318  [004] ...1  5713.356460: rtcpu_vinotify_handle_msg: tstamp:178937325478 tag:CSIMUX_FRAME channel:0x00 frame:13102 vi_tstamp:2843665884 data:0x000000a2
 kworker/4:1-2318  [004] ...1  5713.408430: rtcpu_vinotify_handle_msg: tstamp:178938887988 tag:CSIMUX_FRAME channel:0x00 frame:3888 vi_tstamp:2845228395 data:0x000000a2
 kworker/4:1-2318  [004] ...1  5713.408439: rtos_queue_peek_from_isr_failed: tstamp:178939559499 queue:0x0b4a3c58
 kworker/4:1-2318  [004] ...1  5713.408443: rtcpu_vinotify_handle_msg: tstamp:178939929660 tag:CSIMUX_FRAME channel:0x00 frame:2355 vi_tstamp:2846270068 data:0x000000a2
 kworker/4:1-2318  [004] ...1  5713.460429: rtcpu_vinotify_handle_msg: tstamp:178940450497 tag:CSIMUX_FRAME channel:0x00 frame:13877 vi_tstamp:2846790905 data:0x000000a2
 kworker/4:1-2318  [004] ...1  5713.512428: rtcpu_vinotify_handle_msg: tstamp:178942013006 tag:CSIMUX_FRAME channel:0x00 frame:12342 vi_tstamp:2848353415 data:0x000000a2
 kworker/4:1-2318  [004] ...1  5713.512432: rtcpu_vinotify_handle_msg: tstamp:178942533842 tag:CSIMUX_FRAME channel:0x00 frame:12345 vi_tstamp:2848874252 data:0x000000a2
 kworker/4:1-2318  [004] ...1  5713.564426: rtcpu_vinotify_handle_msg: tstamp:178943575516 tag:CSIMUX_FRAME channel:0x00 frame:13882 vi_tstamp:2849915925 data:0x000000a2
 kworker/4:1-2318  [004] ...1  5713.564430: rtos_queue_peek_from_isr_failed: tstamp:178944560005 queue:0x0b4a3c58

The log show CHANSEL_SHORT_FRAME that means the output line didn’t as expect. Make sure the sensor output is 1080 lines or try modify the driver to report less to try.

Dear Shane,
Could I know what value output line system expect, and what value system got or count.
I try some value to active_h in dts, but still got CHANSEL_SHORT_FRAME message.

“try modify the driver to report less”
Did you means is adjust below setting in ov5693.c (our test driver)
#define OV5693_DEFAULT_HEIGHT 1080
I try use 1050, also got CHANSEL_SHORT_FRAME message.
And use below command check, and got same resule
v4l2-ctl --set-fmt-video=width=1920,height=1080ixelformat=“UYVY” --device=/dev/video0 --stream-count=3 --stream-mmap
<< 0.97 fps
< 0.97 fps

Currently for test MIPI CSI port the camera module is fiexd 1080p30 4 lane output.

Base on 1080p30 2 lane YUV portC experience,
In ov5693 dts I jsut modify below:
pixel_t = “uyvy”;
embedded_metadata_height = “1”;
even the dts not include 1080p30 mode, but still could normal work for 1080p30 2 lane.
So, I am not sure the setting value how to transfer in system. Where I could set the value correct.

Best regards,

Modify both below and confirm it with v4l2-ctl --list-formats-ext.

#define OV5693_DEFAULT_HEIGHT 1944

static const struct camera_common_frmfmt ov5693_frmfmt[] = {
  	{{2592, 1944},	ov5693_30fps,	1, 0,	OV5693_MODE_2592X1944},
  };

Modify and comfirm:
nvidia@tegra-ubuntu:~$ v4l2-ctl --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Index : 0
Type : Video Capture
Pixel Format: ‘UYVY’
Name : UYVY 4:2:2
Size: Discrete 1920x1080
Interval: Discrete 0.033s (30.000 fps)

Index       : 1
Type        : Video Capture
Pixel Format: 'NV16'
Name        : Y/CbCr 4:2:2

And still got “CHANSEL_SHORT_FRAME”

The size didn’t change. It still 1080.

Copy your modify, did this you want?
nvidia@tegra-ubuntu:~$ v4l2-ctl --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
Index : 0
Type : Video Capture
Pixel Format: ‘UYVY’
Name : UYVY 4:2:2
Size: Discrete 2592x1944
Interval: Discrete 0.033s (30.000 fps)

Index       : 1
Type        : Video Capture
Pixel Format: 'NV16'
Name        : Y/CbCr 4:2:2

No, What I mean is the reduce the 1080 to 107x to try. What I show just point where to modify.

I test from 1079~1070
Below is result for 1079, others 1078~1070 different just “1079”
nvidia@tegra-ubuntu:~$ v4l2-ctl --list-formats-extioctl: VIDIOC_ENUM_FMT
Index : 0
Type : Video Capture
Pixel Format: ‘UYVY’
Name : UYVY 4:2:2
Size: Discrete 1920x1079
Interval: Discrete 0.033s (30.000 fps)

Index       : 1
Type        : Video Capture
Pixel Format: 'NV16'
Name        : Y/CbCr 4:2:2

For 1079,1077,1075,1073,1071 below command will got error below and no trace message.
nvidia@tegra-ubuntu:~ gst-launch-1.0 -v v4l2src device="/dev/video0" ! "video/x-raw,width=1920,height=1079, format=(string)I420" ! nvvidconv ! "video/x-raw(memory:NVMM)" ! nvoverlaysink Setting pipeline to PAUSED ... Pipeline is live and does not need PREROLL ... Setting pipeline to PLAYING ... New clock: GstSystemClock ERROR: from element /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: Internal data flow error. Additional debug info: gstbasesrc.c(2948): gst_base_src_loop (): /GstPipeline:pipeline0/GstV4l2Src:v4l2src0: streaming task paused, reason not-negotiated (-4) Execution ended after 0:00:03.830170345 Setting pipeline to PAUSED ... Setting pipeline to READY ... Setting pipeline to NULL ... Freeing pipeline ... nvidia@tegra-ubuntu:~

For 1078,1076,1074,1072,1070
nvidia@tegra-ubuntu:~$ gst-launch-1.0 -v v4l2src device="/dev/video0" ! “video/x-raw,width=1920,height=1078, format=(string)I420” ! nvvidconv ! “video/x-raw(memory:NVMM)” ! nvoverlaysink
=> result got green screen
=> trace message same as 1080 about the tag: and data:
kworker/0:3-1279 [000] …1 238.595174: rtcpu_vinotify_handle_msg: tstamp:7849421085 tag:CSIMUX_FRAME channel:0x00 frame:3979 vi_tstamp:3554453068 data:0x000000a2
kworker/0:3-1279 [000] …1 238.595176: rtcpu_vinotify_handle_msg: tstamp:7849421222 tag:CHANSEL_SHORT_FRAME channel:0x04 frame:3979 vi_tstamp:3554453068 data:0x00000001
kworker/0:3-1279 [000] …1 238.595177: rtcpu_vinotify_handle_msg: tstamp:7849421358 tag:ATOMP_FE channel:0x00 frame:3979 vi_tstamp:3554453070 data:0x00000001
kworker/0:3-1279 [000] …1 238.595177: rtcpu_vinotify_handle_msg: tstamp:7849941614 tag:CSIMUX_FRAME channel:0x00 frame:12429 vi_tstamp:3554973904 data:0x000000a2

Currently we could saw 1080p30 4 lane image.
Due to CSI lane mapped:
Correct
camera-- TX2
CLK – CLK
D1 – C_D0
D2 – C_D1
D3 – D_D0
D4 – D_D1

Wrong
CLK – CLK
D1 – C_D0
D2 – C_D1
D3 – D_D1
D4 – D_D0

Best regards,

Currently we could saw 1080p60 4 lane image also.
Could we fixed the issue by FW?
Swap D_D0 and D_D1?

Best regards,

Sorry to tell Tegra didn’t have this support.

In the TRM,chapter “28.3.3.1 SCIL Features” shows the SCIL can support “Data lane swapping in D-PHY and trio swapping in C-PHY”. And chapter “28.4.2.2” and chapter “28.4.4” shows the details info about “Lane Merger”.

So i think this can fixed the HW issue of FoxK, but i didn’t find any source code about this function, neither in the driver nor in the dts files, how to enable and config this function ?

Thanks!

I find the register “NVCSI_PHY_0_NVCSI_CIL_LANE_SWIZZLE_CTRL_0”
I think this is the keypoint so i search “swizzle” in the kernel source, i find it!!
FoxK can try this or maybe you have modified your schematic, yes, it was more than one years ago

Below is the CSI_CIL_POLARITY_SWIZZLE_CTRL reg setting
00000 A0 A1 B0 B1 --> A0 A1 B0 B1
00001 A0 A1 B0 B1 --> A0 A1 B1 B0
00010 A0 A1 B0 B1 --> A0 B0 B1 A1
00011 A0 A1 B0 B1 --> A0 B0 A1 B1
00100 A0 A1 B0 B1 --> A0 B1 A1 B0
00101 A0 A1 B0 B1 --> A0 B1 B0 A1
00110 A0 A1 B0 B1 --> A1 A0 B0 B1
00111 A0 A1 B0 B1 --> A1 A0 B1 B0
01000 A0 A1 B0 B1 --> A1 B0 B1 A0
01001 A0 A1 B0 B1 --> A1 B0 A0 B1
01010 A0 A1 B0 B1 --> A1 B1 A0 B0
01011 A0 A1 B0 B1 --> A1 B1 B0 A0
01100 A0 A1 B0 B1 --> B0 A1 A0 B1
01101 A0 A1 B0 B1 --> B0 A1 B1 A0
01110 A0 A1 B0 B1 --> B0 A0 B1 A1
01111 A0 A1 B0 B1 --> B0 A0 A1 B1
10000 A0 A1 B0 B1 --> B0 B1 A1 A0
10001 A0 A1 B0 B1 --> B0 B1 A0 A1
10010 A0 A1 B0 B1 --> B1 A1 B0 A0
10011 A0 A1 B0 B1 --> B1 A1 A0 B0
10100 A0 A1 B0 B1 --> B1 B0 A0 A1
10101 A0 A1 B0 B1 --> B1 B0 A1 A0
10110 A0 A1 B0 B1 --> B1 A0 A1 B0
10111 A0 A1 B0 B1 --> B1 A0 B0 A1