Our company is developing video system that requires very low latency screen video output.
Our installation is: JETSON-TX2 + LI-IMX264 (2448x2058x25fps) + DELL u2412mb (8ms latency). We tried to use all possible variants of gst-launch, also tried to read image via system calls libv4l2 (V4L2_MEMORY_MMAP with VIDIOC_DQBUF, VIDIOC_QBUF etc). Unfortunately, we are not able to achieve latency less than ~120ms, our goal is 30-40ms. All discussions on forum that I’ve read does not provide a solution. The latency is measured by making photo of a timer on a smartphone.
The main questions are:
What is the reason of this latency? Is it because of buffering mechanism in VI/V4l2?
Is it possible to reduce latency to 30-40ms? May be there is some way to update VI and implement consecutive video readout and screen output in some RTOS-way?
it’s almost impossible since you’re having camera sensor with 25-fps, each frame programming time takes 40-ms.
may I double confirm your use-case.
you had put your camera device target a stop-watch, render preview frames to a display monitor. in the meanwhile, you’re using another camera device (you should have high-speed camera to capture these timestamps) to take a picture to include both the timestamp of stop-watch and display monitor; then and you would like to reduce the latency of them.
so,
the roughly pipeline shown as below, and you’re asking the latency from (A) to (B).
for example,
in addition,
here’s one similar discussion thread you may also check, Topic 55327.
there’s by default to have 2~3 frame latency for glass-to-glass latency,
(i.e. the latency from (A) to (B) as I mentioned in post #3)
so, it’s close to your evaluation, ~120ms.
Well, the camera has global shutter. So, as I suggest, the delay is defined only by the exposure (small enough).
Yes, here is a snapshot with 116ms latency (using the following pipeline): gst-launch-1.0 nvarguscamerasrc aeantibanding=0 ! 'video/x-raw(memory:NVMM),width=(int)2448,height=(int)2058,framerate=(fraction)25/1,format=(string)NV12' ! nvvidconv flip-method=2 ! 'video/x-raw,width=800,height=600,format=I420' ! xvimagesink
the global shutter camera were supported,
may I also know is this camera be used in continuous capture or triggered operation?
from software point-of-view. it waits for start-of-frame and also end-of-frame to complete a single frame request.
you may also narrow down your single frame latency for evaluation your sensor capability.
please refer to Applications Using V4L2 IOCTL Directly, there’s sample pipeline by using V4L2 IOCTL to verify basic functionality during sensor bring-up.
you may also access L4T sources and refer to TX2’s VI kernel drivers.
for example, $L4T_Sources/r32.4.3/Linux_for_Tegra/source/public/kernel/nvidia/drivers/media/platform/tegra/camera/vi/vi4_fops.c
please also refer to Camera Architecture Stack, libargus only support with bayer sensor types, you may enable v4l2src to access other camera streams.
thanks
