Only each of the second frame is captured on the CSI bus

We designed custom carrier board for TX1/TX2 with TI FPDLink deserializer DS90UB940 which is connected to the MIPI-CSI2 bus of the Jetson SOM.

Video capturing works well with installed TX1. But with installed TX2 - only each of the second frame can be captured and it doesn’t depend on video timings.

Workaround for that is to increase the time between FrameEnd and FrameStart packets (CSI low level protocol) on the CSI transmitter side of DS90UB940.
FE_FS

It seems like a known issue and several related topics can be found on the forum. But solution is always workaround on the CSI transmitter side.

I suspect that it is related to some limitation of new TX2 VI architecture where RTCPU was introduced which detects, timestamps and populate frame start event to the VI driver. For some reason it misses FS for each of the second frame when time btw FE and FS less some threshold. I’m not sure if it is SW (rtcpu firmware) or HW limitation and I didn’t find anything in the spec of TX2.

Because time between FE and FS packets cannot always be changed on some of the CSI transmitters - can you share the limitations?

Also if it is SW limitation of RTCPU - is it possible to get source code so we can modify it for our needs? Or can you suggest please some other approaches - for example - to tweak RTCPU clock etc.
Is there any tegra camera driver for TX2 which does not use RTCPU like it was implemented for TX1?

Thanks

Sorry for the late response, we’re doing the investigation with our internal camera team, will do the update soon.

Thanks

@vkorshun
Have you try to boost the NVCSI/VI clocks to try.

https://elinux.org/Jetson_TX2_Camera_BringUp

@vkorshun
How did you identify the received frame and how did you know the dropped frame are interleave?
Could you dump all the VI notify?

Yes, we tried to maximize VI/ISP/NVCSI:

echo 1 > /sys/kernel/debug/bpmp/debug/clk/vi/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/isp/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/nvcsi/mrq_rate_locked
cat /sys/kernel/debug/bpmp/debug/clk/vi/max_rate |tee /sys/kernel/debug/bpmp/debug/clk/vi/rate
cat /sys/kernel/debug/bpmp/debug/clk/isp/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/isp/rate
cat /sys/kernel/debug/bpmp/debug/clk/nvcsi/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/nvcsi/rate

But it didn’t help at all

Here are the traces:
vi_traces_issue.txt (785.8 KB) vi_traces_no_issue.txt (779.5 KB)

The source of the video the same (1920x1200x60) - from the same V4l2 session (stream on). Difference only the time btw FE and FS packets on CSI bus which was set on CSI transmitter during runtime.


Last column is frame len based on traces. So when issue is happening - time btw sof is always 33 ms. VI driver just doesn’t get sof in vi_notify_wait and frame is skipped. Issue is not reproducible with TX1 with the same CSI transmitter settings and I still suspect rtcpu.

@vkorshun
Did you have apply any patch for the csi4_fops.c/vi4_fops.c

@ShaneCCC, no patches were applied to csi4_fops.c/vi4_fops.c. There are patches that we did in the camera_common.c and sensor_common.c for adding support of RGB pixel format.

Thanks for your confirm.
Any side effect for the Workaround to increase the time between FrameEnd and FrameStart packets

@ShaneCCC , thanks with quick reply.
With this workaround we cannot cover all the edge cases. For example sometimes serializer can send reset to the deserializer so csi transmitter should be reconfigured again. The second case - it is sensative to video timings/resolutions.

Main thing - we don’t understand limitations and might won’t be able to change this parameter with another csi transmitters.
As an engineer I see risks when this issue is not under control on the SOM side and that’s bad for mass production.

Could you check if can increase VSYNC blanking period to try.

@ShaneCCC , unfortunately we cannot change video timings on the video source side (serializer).

Last finding - our workaround by setting CSI registers on the 940 side does not really work because i2c multimaster is not supported by Nvidia for TX2 due some hw issue. i2c bus hangs on the arbitration after some time. So, we need real fix of initial half FPS issue. Could somebody please support us?

@vkorshun
Could you modify below the value from 0xe39c08e3 to 0xe39c08e0 in vi4_init() in the vi4_fops.c to collect the trace log for checking.

vi4_write(chan, NOTIFY_TAG_CLASSIFY_0, 0xe39c08e3);

Thanks

@ShaneCCC ,
Here is the trace file. I compared with the previous one and could not find the difference…
csi_trace3.txt (874.2 KB)

Please replace the file …/JetPack_4.5_Linux_JETSON_TX2/Linux_for_Tegra/bootloader/ with attached file then issue below command to update the camera firmware and collect the trace again.camera-rtcpu-sce.img.txt (122.3 KB)

sudo ./flash.sh -r -k sce-fw jetson-tx2-devkit mmcblk0p1

@ShaneCCC , Here are the files with new traces with issue and without. What I can see - new FE channel/FS channel traces are appeared there. In traces while issue is happening - more FE channel/FS channel pairs and less tegra_channel_capture_frame. When issue is not happening - there are same number of FE/FS pairs and tegra_channel_capture_frame.

vi_rtcpu_trace_issue.txt (604.9 KB) vi_rtcpu_trace_no_issue.txt (2.2 MB)