Only each of the second frame is captured on the CSI bus

[vkorshun]

We designed custom carrier board for TX1/TX2 with TI FPDLink deserializer DS90UB940 which is connected to the MIPI-CSI2 bus of the Jetson SOM.

Video capturing works well with installed TX1. But with installed TX2 - only each of the second frame can be captured and it doesn’t depend on video timings.

Workaround for that is to increase the time between FrameEnd and FrameStart packets (CSI low level protocol) on the CSI transmitter side of DS90UB940.

It seems like a known issue and several related topics can be found on the forum. But solution is always workaround on the CSI transmitter side.

I suspect that it is related to some limitation of new TX2 VI architecture where RTCPU was introduced which detects, timestamps and populate frame start event to the VI driver. For some reason it misses FS for each of the second frame when time btw FE and FS less some threshold. I’m not sure if it is SW (rtcpu firmware) or HW limitation and I didn’t find anything in the spec of TX2.

Because time between FE and FS packets cannot always be changed on some of the CSI transmitters - can you share the limitations?

Also if it is SW limitation of RTCPU - is it possible to get source code so we can modify it for our needs? Or can you suggest please some other approaches - for example - to tweak RTCPU clock etc.
Is there any tegra camera driver for TX2 which does not use RTCPU like it was implemented for TX1?

Thanks

[kayccc]

Sorry for the late response, we’re doing the investigation with our internal camera team, will do the update soon.

Thanks

[ShaneCCC]

@vkorshun
Have you try to boost the NVCSI/VI clocks to try.

https://elinux.org/Jetson_TX2_Camera_BringUp

[ShaneCCC]

@vkorshun
How did you identify the received frame and how did you know the dropped frame are interleave?
Could you dump all the VI notify?

[vkorshun]

Yes, we tried to maximize VI/ISP/NVCSI:

echo 1 > /sys/kernel/debug/bpmp/debug/clk/vi/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/isp/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/nvcsi/mrq_rate_locked
cat /sys/kernel/debug/bpmp/debug/clk/vi/max_rate |tee /sys/kernel/debug/bpmp/debug/clk/vi/rate
cat /sys/kernel/debug/bpmp/debug/clk/isp/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/isp/rate
cat /sys/kernel/debug/bpmp/debug/clk/nvcsi/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/nvcsi/rate

But it didn’t help at all

[vkorshun]

Here are the traces:
vi_traces_issue.txt (785.8 KB) vi_traces_no_issue.txt (779.5 KB)

The source of the video the same (1920x1200x60) - from the same V4l2 session (stream on). Difference only the time btw FE and FS packets on CSI bus which was set on CSI transmitter during runtime.

Last column is frame len based on traces. So when issue is happening - time btw sof is always 33 ms. VI driver just doesn’t get sof in vi_notify_wait and frame is skipped. Issue is not reproducible with TX1 with the same CSI transmitter settings and I still suspect rtcpu.

[ShaneCCC]

@vkorshun
Did you have apply any patch for the csi4_fops.c/vi4_fops.c

[vkorshun]

@ShaneCCC, no patches were applied to csi4_fops.c/vi4_fops.c. There are patches that we did in the camera_common.c and sensor_common.c for adding support of RGB pixel format.

[ShaneCCC]

Thanks for your confirm.
Any side effect for the Workaround to increase the time between FrameEnd and FrameStart packets

[vkorshun]

@ShaneCCC , thanks with quick reply.
With this workaround we cannot cover all the edge cases. For example sometimes serializer can send reset to the deserializer so csi transmitter should be reconfigured again. The second case - it is sensative to video timings/resolutions.

Main thing - we don’t understand limitations and might won’t be able to change this parameter with another csi transmitters.
As an engineer I see risks when this issue is not under control on the SOM side and that’s bad for mass production.

[ShaneCCC]

Could you check if can increase VSYNC blanking period to try.

About the Jetson & Embedded Systems category

Discussions relating to the Jetson DevKits and other Embedded computing devices

[vkorshun]

@ShaneCCC , unfortunately we cannot change video timings on the video source side (serializer).

[vkorshun]

Last finding - our workaround by setting CSI registers on the 940 side does not really work because i2c multimaster is not supported by Nvidia for TX2 due some hw issue. i2c bus hangs on the arbitration after some time. So, we need real fix of initial half FPS issue. Could somebody please support us?

[ShaneCCC]

@vkorshun
Could you modify below the value from 0xe39c08e3 to 0xe39c08e0 in vi4_init() in the vi4_fops.c to collect the trace log for checking.

vi4_write(chan, NOTIFY_TAG_CLASSIFY_0, 0xe39c08e3);

Thanks

[vkorshun]

@ShaneCCC ,
Here is the trace file. I compared with the previous one and could not find the difference…
csi_trace3.txt (874.2 KB)

[ShaneCCC]

Please replace the file …/JetPack_4.5_Linux_JETSON_TX2/Linux_for_Tegra/bootloader/ with attached file then issue below command to update the camera firmware and collect the trace again.camera-rtcpu-sce.img.txt (122.3 KB)

sudo ./flash.sh -r -k sce-fw jetson-tx2-devkit mmcblk0p1

[vkorshun]

@ShaneCCC , Here are the files with new traces with issue and without. What I can see - new FE channel/FS channel traces are appeared there. In traces while issue is happening - more FE channel/FS channel pairs and less tegra_channel_capture_frame. When issue is not happening - there are same number of FE/FS pairs and tegra_channel_capture_frame.

vi_rtcpu_trace_issue.txt (604.9 KB) vi_rtcpu_trace_no_issue.txt (2.2 MB)

[msmsuzuk]

Hello, vkorshun

The following issues have not been solved even in our project. The challenges are similar.
The outline is as follows.

---

To define the sensor (FPGA) that connects to CSI as dev / video0
The driver diverted ov5693 and took the same measures as in the forum below.

How to grab pre-configured csi video stream without I2C?

Hello Forum! Is there an way to grab a pre-configured CSI-2 video stream? we are using a TX2 dev board with R28.1. The input video stream is generated by a SDI-to-CSI2 adapter, it’s a 4-lane, 1920x1080@60, YUV422 signal. And I connected the input MIPI-CSI data and clk lines directly to the CSI port of tx2. What should I do to grab the video stream through the /dev/video0 device? Should I need to update device tree or modify the csi-camera drivers to register a camera device? what is the corr…

A sensor (FPGA) was connected using TX1 and TX2 respectively.
Sensor (FPGA) CSI clock frequency is 300MHz
Why you think 300MHz is right
The CSI Line Rate is set to 620Mbps, but the clock is halved to about 300MHz because the data is DDR.

Below are the results of the experiment
Jetson TX1 settings: 1920 x 1200, 60fps, lane 4
Of course, the conditions were set in the DTS file.
As a result of operation, the frame rate became 60 fps. The acquired data is normal.

Then the same experiment with TX1 replaced by TX2

Jetson TX2 settings: 1920 x 1200, 60fps, lane 4
The conditions were set in the DTS file with reference to TX1.

The TX1 and TX2 DTS files have the following differences.
・ In TX1, the I2C settings of the DTS file have not been changed.
・ In TX2, the lane and port settings were changed in the DTS file.

As a result of operation, the frame rate became 30 fps.
And it seems that the acquired data is missing data on a regular basis.

Different, different perspective experiments
We confirmed the operation with the same TX2 and a sensor (FPGA) with the CSI clock frequency changed to 150MHz.
With that setting, the frame rate was halved to 15fps.

As reported in other posts, TX2 has an SOF acquisition cycle of about 33 msec.
Originally, the cycle should be 16msec.
We think that correct capture is possible if the following notification arrives at 16msec intervals.

1. ATOMP_FS
2. CHANSEL_PXL_SOF
3. CHANSEL_LOAD_FRAMED
   Four. CHANSEL_PXL_EOF
   Five. ATOMP_FE

The DTS file settings may be insufficient.
If anyone can review it, I would like to publish the DTS file.
Also, please suggest another survey item. We will publish the necessary information.

With the TX1 board, 60fps gives the expected results, we have a track record.

We are watching the measures and results for your challenges.
Please let me know if it has been resolved.
Or, please provide useful information.

[ShaneCCC]

@vkorshun
Please have a try below patch.

diff --git a/drivers/media/platform/tegra/camera/vi/vi4_fops.c b/drivers/media/platform/tegra/camera/vi/vi4_fops.c
index 7b19eb1..ea64585 100644
--- a/drivers/media/platform/tegra/camera/vi/vi4_fops.c
+++ b/drivers/media/platform/tegra/camera/vi/vi4_fops.c
@@ -599,7 +599,7 @@ static int tegra_channel_capture_frame_single_thread(
                        vi4_channel_write(chan, chan->vnc_id[i],
                                                CHANNEL_COMMAND, LOAD);
                        vi4_channel_write(chan, chan->vnc_id[i],
-                               CONTROL, SINGLESHOT | MATCH_STATE_EN);
+                               CONTROL, MATCH_STATE_EN);
                }

                /* wait for vi notifier events */
@@ -660,7 +660,7 @@ static int tegra_channel_capture_frame_multi_thread(
                vi4_channel_write(chan, chan->vnc_id[i],
                        CHANNEL_COMMAND, LOAD);
                vi4_channel_write(chan, chan->vnc_id[i],
-                       CONTROL, SINGLESHOT | MATCH_STATE_EN);
+                       CONTROL, MATCH_STATE_EN);
        }

        /* wait for vi notifier events */
@@ -817,7 +817,7 @@ static void tegra_channel_capture_done(struct tegra_channel *chan)
                tegra_channel_surface_setup(chan, buf, i);
                vi4_channel_write(chan, chan->vnc_id[i], CHANNEL_COMMAND, LOAD);
                vi4_channel_write(chan, chan->vnc_id[i],
-                       CONTROL, SINGLESHOT | MATCH_STATE_EN);
+                       CONTROL, MATCH_STATE_EN);
        }

        for (i = 0; i < chan->valid_ports; i++) {

[vkorshun]

@ShaneCCC , thanks a lot for the patch. We could get 60 fps with it and now it is under the testing.
But I have some questions regarding SINGLESHOT. So as I understand - with SINGLESHOT bit - VI channel is disabled automatically after FE packet. For reenabling - ENABLE bit needs to be set again on the next frame. Without SINGLESHOT - VI channel is always enabled and looks like that’s why it stopped missing frames.
From other hand - there is some kind of warning in TRM:

“The SINGLESHOT bit, in almost all use cases, should be set. Continuous retriggering of a VI frame, if reprogramming does not
complete in time, could result in corrupting an already-completed frame (even though a STALE_FRAME error will be
generated). As such, STALE_FRAME errors should never be masked in continuous mode, since they always indicate that a
race between configuration and frame start has occurred.”

So corrupting frame means some artifacts or it is detected as STALE_FRAME and frame is dropped? Could you comment regarding the risks of continuos mode and maybe other possible side effects?