VI(Video Input) and VIC(Video Image Compositor) performace

[zangyichang]

Hi

we are now adapt GMSL2 camera on Orin NX 16G, Jetpack5.1.2. Found camera data end-to-end delay about 150ms (Sensor–>GMSL2–>VI–>VIC–>EGL) . I found There are 3 parts:

1. Camera signal process delay about 68ms, got from camera producter;
2. VI consume about 32 ms, got by eof_timestamp - sof_timestamp;

diff kernel/nvidia/drivers/media/platform/tegra/camera/vi/vi5_fops.c /data/base/kernel/nvidia/drivers/media/platform/tegra/camera/vi/vi5_fops.c
33,34d32
< extern u64 curr_clocksource_get_offset_ns(void);
< 
498,499d495
< 	static int cnt = 0;
< 
572,577c568
< //	vb->vb2_buf.timestamp = descr->status.sof_timestamp;
< 	vb->vb2_buf.timestamp = ktime_mono_to_real(descr->status.sof_timestamp - curr_clocksource_get_offset_ns());
<  
< 	cnt++;
< 	if(0 == (cnt & 0xff))
< 		printk("====%lld\n****%lld\n", descr->status.sof_timestamp, descr->status.eof_timestamp);
---
> 	vb->vb2_buf.timestamp = descr->status.sof_timestamp;

I got the log like

[34342.218154] ====34360323961344
               ****34360355953792
[34355.018020] ====34373123828448
               ****34373155820896
[34367.817999] ====34385923806560
               ****34385955799072
[34380.617869] ====34398723673536
               ****34398755666016
[34393.417744] ====34411523540416
               ****34411555532928
[34406.217840] ====34424323518272
               ****34424355510784
[34419.017577] ====34437123384992
               ****34437155377440
[34431.817570] ====34449923362816
               ****34449955355264

3. VIC consume about 37 ms, process image(3840x2160) YUYV to ARGB, crop, scalling to 1280x720.

Is it normal, is there anyway I can shorten delay for VI and VIC?

[zangyichang]

Add layout design as follow

[JerryChang]

hello zangyichang,

it looks normal result, we’ve evaluated the glass-to-glass latency, which is around 100ms on developer kits as well.
please see-also similar thread, Topic 278757.

however,
you may try boosting clocks, it should have better G2G results (slightly) by setting to performance mode.
for example,
here’re commands to boost all the VI/CSI/ISP clocks.

sudo su
echo 1 > /sys/kernel/debug/bpmp/debug/clk/vi/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/isp/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/nvcsi/mrq_rate_locked
echo 1 > /sys/kernel/debug/bpmp/debug/clk/emc/mrq_rate_locked
cat /sys/kernel/debug/bpmp/debug/clk/vi/max_rate |tee /sys/kernel/debug/bpmp/debug/clk/vi/rate
cat /sys/kernel/debug/bpmp/debug/clk/isp/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/isp/rate
cat /sys/kernel/debug/bpmp/debug/clk/nvcsi/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/nvcsi/rate
cat /sys/kernel/debug/bpmp/debug/clk/emc/max_rate | tee /sys/kernel/debug/bpmp/debug/clk/emc/rate

you may also check the script file to enable VIC at max clock rate.
for instance, VPI - Vision Programming Interface: Performance Benchmark