V4L frame timing on Orin/Jetpack5

Autonomous Machines Jetson & Embedded Systems Jetson AGX Orin
1

Hello,

I’m using Orin 32GB HW+jetpack5 and developing a C++ application using the V4L interface.
C++ code to control the MIPI camera is similar to jetson-utils/camera/v4l2Camera.cpp at master · dusty-nv/jetson-utils · GitHub.

Application turns on a light, snaps an image, turns off the light and repeats a few 100’s of milliseconds later.
I’m having an issue where images seem to not be illuminated properly and can confirm that by letting light on the entire time (not possible in the real application), images all come out OK.

This tells me that I may not understand the timing of V4L frame acquisition.
The diagram below is my current understanding of timing. Is it correct, i.e. is it the case that the call to “select()” will return after the entire image was transferred and buffer populated with pixel data?

image
image1487×585 19.9 KB

Is there any information anywhere about pixel data transfer and timing for jetpack5 on orin that I can refer to?

Thanks,
Alain

4

Looks like you want a feature like flash light?
I don’t think there’s a way to get right frame by current design.
But you can check FS(Frame Start) event vi5_fops.c to enable the light in a period to get frames illuminated. But this need to implement in the kernel driver.

5

it’s not a flash light, just an LED device or a laser which gets turned on for the duration of the capture.
Capture rate is quite slow, say 5 frames per second or less.

My question really is:
after select() returns and ioctl(fd, VIDIOC_DQBUF, &bufdata) is called and returns,
are pixel data completely transferred or could the transfer still be ongoing?

I have found thread about frame timestamps but that was for argus-based SW, which is not my case since I’m using V4L:

6

So your request is knowing the time to turn on the LED?
If so you can check the vi5_capture_enqueue/vi5_capture_dequeue in vi5_fops.c

7

simply put, I want to know when pixel data are fully transfered to the buffer.

SW does the following:

    // wait for frame
    fd_set fds;
    FD_ZERO(&fds);
    FD_SET(m_fd, &fds);

    int rc = select(m_fd + 1, &fds, NULL, NULL, &tv);
    if (rc == -1) {
        return false;
    }
    // (A) are pixel data fully transfered here?

    // dequeue buffer
    struct v4l2_buffer buf;
    memset(&buf, 0, sizeof(v4l2_buffer));
    buf.memory = V4L2_MEMORY_MMAP;
    if (xioctl(m_fd, VIDIOC_DQBUF, &buf) == -1) {
        return false;
    }
    // (B) are pixel data fully transfered here?

    bufptr = get_pointer_from_buf_index(buf.index);
    process_frame(bufptr);
    // (C) are pixel data fully transfered while buffer is being processed?

    return true;

my question is: are pixel data transfered at (A), (B) or could it be they are still being transferred while SW is processing the buffer in (C)?

8

I don’t think able to know exactly time in user space. However you able to know if in vi driver like get the EOF syncpoint.