4K30 with TC358840 on TX2 with 32.7.6

Dear Jetson Community, dear Nvidia,

I am currently upgrading a running product of one of our customers to a more recent L4T-Version. During the upgrade process, I have noticed several changes that were brought into the TC358840 driver, present in the Jetson-Linux sources.
Using this driver (in the following “original-driver”), I noticed how the system failed to stream at high lane-speeds.

I am using the following setup to replicate the issue with a proven hardware.

The TC358840 chip is connected via 8 lanes to the TX2 - streaming 1440p proves how all the 8 lanes do in fact work

nvidia@tx2-hdmi2csi:~$ v4l2-ctl -d0 --query-dv-timings
	Active width: 2560
	Active height: 1440
	Total width: 2720
	Total height: 1481
	Frame format: progressive
	Polarities: -vsync -hsync
	Pixelclock: 241699200 Hz (60.00 frames per second) // --> 30'212'400 pixels per second per lane (8 Lanes used)
	Horizontal frontporch: 0
	Horizontal sync: 160
	Horizontal backporch: 0
	Vertical frontporch: 0
	Vertical sync: 41
	Vertical backporch: 0
	Standards:
	Flags:
nvidia@tx2-hdmi2csi:~$ v4l2-ctl -d0 --stream-mmap
New timings found
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 60.00 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<^C

And here a trace, showcasing both channels being used together.

amp:2107445691 data:0x00000000
     kworker/5:1-828   [005] ....    55.601955: rtcpu_vinotify_event: tstamp:2107460274 tag:CHANSEL_PXL_SOF channel:0x00 frame:0 vi_tstamp:2107459412 data:0x00000001
     kworker/5:1-828   [005] ....    55.601958: rtcpu_vinotify_event: tstamp:2107460634 tag:CHANSEL_PXL_SOF channel:0x01 frame:0 vi_tstamp:2107459412 data:0x00000001
     kworker/5:1-828   [005] ....    55.601961: rtcpu_vinotify_event: tstamp:2107460991 tag:ATOMP_FS channel:0x00 frame:0 vi_tstamp:2107459420 data:0x00000000
     kworker/5:1-828   [005] ....    55.601963: rtcpu_vinotify_event: tstamp:2107461268 tag:ATOMP_FS channel:0x01 frame:0 vi_tstamp:2107459420 data:0x00000000
     kworker/5:1-828   [005] ....    55.601966: rtcpu_vinotify_event: tstamp:2107466216 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:0 vi_tstamp:2107465557 data:0x08000000
     kworker/5:1-828   [005] ....    55.601969: rtcpu_vinotify_event: tstamp:2107466515 tag:CHANSEL_LOAD_FRAMED channel:0x04 frame:0 vi_tstamp:2107465650 data:0x18000000
     kworker/5:1-828   [005] ....    55.657889: rtcpu_vinotify_event: tstamp:2107966860 tag:CHANSEL_PXL_EOF channel:0x00 frame:0 vi_tstamp:2107966172 data:0x059f0002
     kworker/5:1-828   [005] ....    55.657896: rtcpu_vinotify_event: tstamp:2107967710 tag:CHANSEL_PXL_EOF channel:0x01 frame:0 vi_tstamp:2107966172 data:0x059f0002
     kworker/5:1-828   [005] ....    55.657900: rtcpu_vinotify_event: tstamp:2107967987 tag:ATOMP_FE channel:0x00 frame:0 vi_tstamp:2107966944 data:0x00000000
     kworker/5:1-828   [005] ....    55.657903: rtcpu_vinotify_event: tstamp:2107968380 tag:ATOMP_FE channel:0x01 frame:0 vi_tstamp:2107966945 data:0x00000000

By lowering the resolution to 1080p60, the TC358840-Chip will only use 4 lanes, but at a higher pixel-rate per lane.

nvidia@tx2-hdmi2csi:~$ v4l2-ctl -d0 --query-dv-timings
	Active width: 1920
	Active height: 1080
	Total width: 2080
	Total height: 1111
	Frame format: progressive
	Polarities: -vsync -hsync
	Pixelclock: 138652800 Hz (60.00 frames per second) // --> 37'125'000 pixels per second per lane (4 Lanes used)
	Horizontal frontporch: 0
	Horizontal sync: 160
	Horizontal backporch: 0
	Vertical frontporch: 0
	Vertical sync: 31
	Vertical backporch: 0
	Standards:
	Flags:

Streaming with this resolution is not possible using the driver present in the sources. The driver will fail to stream, as the following trace shows.

     kworker/5:1-828   [005] ....   504.897849: rtos_queue_send_from_isr_failed: tstamp:16148014087 queue:0x0b4b2058
     kworker/5:1-828   [005] ....   504.897854: rtos_queue_send_failed: tstamp:16148015455 queue:0x0b4a7698
     kworker/5:1-828   [005] ....   504.897855: rtos_queue_send_from_isr_failed: tstamp:16148019873 queue:0x0b4a7698
     kworker/5:1-828   [005] ....   504.897857: rtos_queue_send_from_isr_failed: tstamp:16148020141 queue:0x0b4ab1a8
     kworker/5:1-828   [005] ....   504.897859: rtos_queue_send_from_isr_failed: tstamp:16148020476 queue:0x0b4acdd8
     kworker/5:1-828   [005] ....   504.897861: rtos_queue_send_from_isr_failed: tstamp:16148020746 queue:0x0b4ae958
     kworker/5:1-828   [005] ....   504.897862: rtos_queue_send_from_isr_failed: tstamp:16148021011 queue:0x0b4af718
     kworker/5:1-828   [005] ....   504.897865: rtos_queue_send_from_isr_failed: tstamp:16148021278 queue:0x0b4b04d8
     kworker/5:1-828   [005] ....   504.897866: rtos_queue_send_from_isr_failed: tstamp:16148021548 queue:0x0b4b1298
     kworker/5:1-828   [005] ....   504.897868: rtos_queue_send_from_isr_failed: tstamp:16148021815 queue:0x0b4b2058
     kworker/5:1-828   [005] ....   504.897870: rtos_queue_send_failed: tstamp:16148022877 queue:0x0b4a7698

Streaming 4K30 with the original driver is also not possible → the trace looks similar to the one above.

Trying an older version of the driver allowed me to at least bring the 1080p60 to work, apparently solving the issue with the pixel-rate per lane, however the 4K30 still failed to stream.

nvidia@tx2-hdmi2csi:~$ v4l2-ctl -d0 --query-dv-timings
	Active width: 1920
	Active height: 1080
	Total width: 2080
	Total height: 1111
	Frame format: progressive
	Polarities: -vsync +hsync
	Pixelclock: 138652800 Hz (60.00 frames per second)
	Horizontal frontporch: 0
	Horizontal sync: 160
	Horizontal backporch: 0
	Vertical frontporch: 0
	Vertical sync: 31
	Vertical backporch: 0
	Standards:
	Flags:
nvidia@tx2-hdmi2csi:~$ v4l2-ctl -d0 --stream-mmap
New timings found
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 60.39 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 60.50 fps

As we see in the trace, now the old driver is able to stream 1080p60, but only using 4 lanes (or one channel).

		 kworker/0:1-735   [000] ....   182.118454: rtcpu_vinotify_event: tstamp:6059875943 tag:CHANSEL_PXL_SOF channel:0x00 frame:0 vi_tstamp:6059875269 data:0x00000001
     kworker/0:1-735   [000] ....   182.118457: rtcpu_vinotify_event: tstamp:6059876533 tag:ATOMP_FS channel:0x00 frame:0 vi_tstamp:6059875276 data:0x00000000
     kworker/0:1-735   [000] ....   182.118459: rtcpu_vinotify_event: tstamp:6059882305 tag:CHANSEL_LOAD_FRAMED channel:0x01 frame:0 vi_tstamp:6059881755 data:0x08000000
     kworker/0:1-735   [000] ....   182.118462: rtcpu_vinotify_event: tstamp:6060382725 tag:CHANSEL_PXL_EOF channel:0x00 frame:0 vi_tstamp:6060382080 data:0x04370002
     kworker/0:1-735   [000] ....   182.118464: rtcpu_vinotify_event: tstamp:6060383644 tag:ATOMP_FE channel:0x00 frame:0 vi_tstamp:6060383064 data:0x00000000

The 4K30 still fails to stream, as the following trace shows.

nvidia@tx2-hdmi2csi:~$ v4l2-ctl -d0 --query-dv-timings
	Active width: 3840
	Active height: 2160
	Total width: 4400
	Total height: 2250
	Frame format: progressive
	Polarities: +vsync +hsync
	Pixelclock: 297000000 Hz (30.00 frames per second)
	Horizontal frontporch: 0
	Horizontal sync: 560
	Horizontal backporch: 0
	Vertical frontporch: 0
	Vertical sync: 90
	Vertical backporch: 0
	Standards:
	Flags:
     kworker/0:1-735   [000] ....   324.978269: rtos_queue_send_failed: tstamp:10523897825 queue:0x0b4a7698
     kworker/0:1-735   [000] ....   324.978288: rtos_queue_send_from_isr_failed: tstamp:10523904554 queue:0x0b4a7698
     kworker/0:1-735   [000] ....   324.978294: rtos_queue_send_from_isr_failed: tstamp:10523904821 queue:0x0b4ab1a8
     kworker/0:1-735   [000] ....   324.978299: rtos_queue_send_from_isr_failed: tstamp:10523905087 queue:0x0b4acdd8
     kworker/0:1-735   [000] ....   324.978305: rtos_queue_send_from_isr_failed: tstamp:10523905354 queue:0x0b4ae958
     kworker/0:1-735   [000] ....   324.978311: rtos_queue_send_from_isr_failed: tstamp:10523905618 queue:0x0b4af718
     kworker/0:1-735   [000] ....   324.978316: rtos_queue_send_from_isr_failed: tstamp:10523905884 queue:0x0b4b04d8
     kworker/0:1-735   [000] ....   324.978321: rtos_queue_send_from_isr_failed: tstamp:10523906150 queue:0x0b4b1298
     kworker/0:1-735   [000] ....   324.978326: rtos_queue_send_from_isr_failed: tstamp:10523906429 queue:0x0b4b2058
     kworker/0:1-735   [000] ....   324.978331: rtos_queue_send_failed: tstamp:10523907571 queue:0x0b4a7698

And this is the point, where my confusion begins. Apparently, the maximum pixel-rate works, but only when using 4 lanes. Is there some kind of bandwidth limitation in the NVCSI or VI that needs adjsustment? The hardware is proven to work with Kernel 4.4.
The plan would be, to adjust the upstream driver (“original-driver”) to work with the 4K30 resolution, as this driver seems to be the most feature rich.

I would be very happy to get some hints.

Best Regards,
wegr

hello wegr,

may I know what’s your actual use-case?
there did had limitation on TX2 series, it needs to enable 8-lane (i.e. gang mode) for 4K stream.

Hello Jerry,

The use case is a video rescaling and switching device - we have customers both in the medical and office/business space creating various products based on Jetson.

I was already able to verify that gang mode works by streaming 1440p60 video. As we are able to run the maximum pixelclock on 4 lanes but not on 8 made me wonder if there was some sort overload happening to the ISR-System, as the errors would suggest.
The main concern is, that it worked on older Jetson Linux / L4T versions.

Best regards,
wegr

hello wegr,

may I also confirm which L4T release version it worked before?

That was on 32.2.1 - also on Kernel 4.9

hello wegr,

I see, that’s such an old release version.
please check Camera Sensor Drivers Porting to examine your sensor driver.