Applying the fix for lowering the latency of V4L2 camera capture

Robotics & Edge Computing Jetson & Embedded Systems Jetson TX2
1

Hello,

We’re seeing unexpected latency (around 70ms) when doing a simple local preview (without any encoding and networking transport) of the video on our Jetson TX2 with three camera setup: Three Synchronized 4K Cameras for NVIDIA Jetson TX2. We have enabled the boost mode on Jetson already.

The issue seems to be related to previously mentioned number of buffer queues on V4L2 video driver: https://devtalk.nvidia.com/default/topic/934387/jetson-tx1/one-frame-latency-delay-in-tx1-v4l-stack/post/5241547/#5241547.

Here’s the code snippet:

$TOP/kernel/drivers/media/platform/tegra/camera/mc_common.h
#define   QUEUED_BUFFERS  4
 
$TOP/kernel/drivers/media/platform/tegra/camera/channel.c
static void tegra_channel_ring_buffer(){
...
    /* release buffer N at N+2 frame start event */
    if (chan->num_buffers >= (QUEUED_BUFFERS - 1))
    free_ring_buffers(chan, 1);

What would be the procedure to change QUEUED_BUFFERS (from 4 to 2), compile and deploy only the library touched by this change?

Any other suggestions on how to lower the capture latency are more than welcomed.

Thank you!

B.

2

hello bogdan0kg3v,

couple of questions need your feedback.
thanks

  1. may I have your confirmation that which JetPack release you’re working on?
  2. could you please share your latency criteria, what’s the end-to-end latency you expected?
    for example,
    camera init–> start camera streaming–> v4l2 user space–> rendering to display for preview.
3

Hi JerryChang,

Thanks on the quick follow up.

  1. We’re running L4T 28.2.1 on Jetson TX2
  2. Our cameras are running at 100FPS and we would like to hit 20ms target from the exposure event on camera sensor to the render event on the screen.

FYI, in order to measure a precise latency for the cameras, we’re using glass-to-glass latency measurement based on LED and photocell setup: Glass to glass video latency is now under 50 milliseconds – fpv.blue.

4

Any new insights on the problem we’re facing?

5

hello bogdan0kg3v,

  1. there are some known issues with TX2’s VI driver (for stability and buffer timestamp), you may apply kernel fixes from
    Topic 1038067 and Topic 1020202.
  2. could you please dump the buffer timestamp, to evaluate the buffer transferring latency from start of frame to user-space.
6

Hi bogdan0kg3v,

I am having the same problem.
Have you had any updates on reducing the latency?