Jetpack 4.2 V4L2 Buffer Bug?

Hello,

I’m writing an app on a TX2i that uses the V4L2 API to capture a video stream from a camera. I had it functioning in Jetpack 3.2, but when I ported it to Jetpack 4.2 there were new bugs.

After some debugging, the temporary solution is to increase the number of V4L2 buffers from 1 to at least 3. The first 2 buffers to queue would never dequeue, but by the time it got to the third buffer it would dequeue and then the other buffers would free up for use as well. This also requires the DQBUF call to be non-blocking, else the program will never continue.

If the device is set to non-blocking with just 1 buffer, QBUF will constantly return an EINVAL (Invalid Argument) or each frame I try to grab. The V4L2 documentation discusses the reasons EINVAL would be returned, but none of those reasons make sense in this context.

Did something change with the implementation of V4L2 in the tegra driver code between Jetpack 3.2 and 4.2 that would cause this? I’m not satisfied with the temporary solution, so any help is appreciated.

Hi,
Please share how we can reproduce the issue. Reproduce it with an existing tegra_multimedia_api sample or attach a test sample so that we can build and run.

Unfortunately I do not have a camera that the tegra_multimedia_api samples work with at the moment (due to pixel format).

This is the code my program is based on:

#include <errno.h>
#include <fcntl.h>
#include <linux/videodev2.h>
#include <stdint.h>
#include <stdio.h>
#include <string.h>
#include <sys/ioctl.h>
#include <sys/mman.h>
#include <unistd.h>

uint8_t *buffer;

static int xioctl(int fd, int request, void *arg)
{
        int r;

        do r = ioctl (fd, request, arg);
        while (-1 == r && EINTR == errno);

        return r;
}

int init_mmap(int fd)
{
    struct v4l2_requestbuffers req = {0};
    req.count = 1;
    req.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
    req.memory = V4L2_MEMORY_MMAP;

    if (-1 == xioctl(fd, VIDIOC_REQBUFS, &req))
    {
        perror("Requesting Buffer");
        return 1;
    }

    struct v4l2_buffer buf = {0};
    buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
    buf.memory = V4L2_MEMORY_MMAP;
    buf.index = 0;
    if(-1 == xioctl(fd, VIDIOC_QUERYBUF, &buf))
    {
        perror("Querying Buffer");
        return 1;
    }

    buffer = mmap (NULL, buf.length, PROT_READ | PROT_WRITE, MAP_SHARED, fd, buf.m.offset);
    printf("Length: %d\nAddress: %p\n", buf.length, buffer);
    printf("Image Length: %d\n", buf.bytesused);

    return 0;
}

int capture_image(int fd)
{
    struct v4l2_buffer buf = {0};
    buf.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
    buf.memory = V4L2_MEMORY_MMAP;
    buf.index = 0;
    if(-1 == xioctl(fd, VIDIOC_QBUF, &buf))
    {
        perror("Query Buffer");
        return 1;
    }

    if(-1 == xioctl(fd, VIDIOC_STREAMON, &buf.type))
    {
        perror("Start Capture");
        return 1;
    }

    fd_set fds;
    FD_ZERO(&fds);
    FD_SET(fd, &fds);
    struct timeval tv = {0};
    tv.tv_sec = 2;
    int r = select(fd+1, &fds, NULL, NULL, &tv);
    if(-1 == r)
    {
        perror("Waiting for Frame");
        return 1;
    }

    if(-1 == xioctl(fd, VIDIOC_DQBUF, &buf))
    {
        perror("Retrieving Frame");
        return 1;
    }

    int outfd = open("out.img", O_RDWR);
    write(outfd, buffer, buf.bytesused);
    close(outfd);

    return 0;
}

int main()
{
        int fd;

        fd = open("/dev/video0", O_RDWR);
        if (fd == -1)
        {
                perror("Opening video device");
                return 1;
        }

        if(init_mmap(fd))
            return 1;

        if(capture_image(fd))
            return 1;

        close(fd);
        return 0;
}

It will capture a single frame from /dev/video0. On Jetpack 3.3* this program succesfully returns, but if you run it in Jetpack 4.2 it will block and never finish. Adding the O_NONBLOCK flag when opening the device on Jetpack 4.2 will prevent it from blocking, but give an error.

If you want to continuously capture frames, you just need to change

if(capture_image(fd))
    return 1;

to

while(true){capture_image(fd);}

. Obviously the program will never return at this point, but it will allow you to test with varying amounts of buffers.

If there are any missing steps let me know.

Hi,
The sample applications are

tegra_multimedia_api\samples\v4l2cuda
tegra_multimedia_api\samples2_camera_v4l2_cuda

The capture buffer number is 4 by default. If you reduce the number, it may malfunction.

I would try those sample applications, but my understanding is that they do not support the onboard camera (OV5693) that comes with the board due to pixel format.

As for the default number of buffers, even with 4 buffers the program will not work on Jetpack 4.2. It will block on the first DQBUF call unless you open the device with the O_NONBLOCK flag. I’m looking to understand what could have changed between Jetpack 4.2 and 3.3 that would cause this change in behavior.

Hi,
These samples are not for Bayer sensors. For Bayer sensors we usually go through Argus.
If the issue can be reproduced with default ov5693, we still encourage you to give a test app so that we can take a look.

The issue can be reproduced with the default camera using the code I provided earlier. Do you mean something else when you say “give a test app”? I don’t see a way to attach the compiled binary.

Hi @dylan,
Please contact us for further support. https://www.nvidia.com/en-us/contact/sales/

Allocating only one buffer is not a verified case. You see it working on jetpack 3.3, but there can be stability issues.

struct v4l2_requestbuffers req = {0};
req.count = 1;
req.type = V4L2_BUF_TYPE_VIDEO_CAPTURE;
req.memory = V4L2_MEMORY_MMAP;

I have contacted for further support (still awaiting a reply), but there is something else that I thought of that is worth mentioning.

Even with more than one buffer (I’m using 4 buffers), the software will not work unless the devices is opened with the O_NONBLOCK flag. Without it, the program will block forever because the first buffer will get stuck during the DQBUF call. This behavior did not exist in Jetpack 3.3, which is why I think it is a bug. Because O_NONBLOCK is a workaround and not necessary to my program, I want to avoid using it.

Bare in mind that I cannot test the multimedia samples with the onboard camera, so I cannot verify if the issue is present there.

Hi @dylan.george.

You can try capture video with v4l2 using the v4l2-ctl tool from the v4l2-utils package to validate capture with v4l2. Below you will find some useful commands:

  • v4l2-ctl -d /dev/video0 --list-formats-ext
  • v4l2-ctl -d /dev/video0 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=100

Below you will find the output of those commands with the default Onboard camera OV5693:

nvidia@nvidia-desktop:~$ v4l2-ctl -d /dev/video0 --list-formats-ext
ioctl: VIDIOC_ENUM_FMT
	Index       : 0
	Type        : Video Capture
	Pixel Format: 'BG10'
	Name        : 10-bit Bayer BGBG/GRGR
		Size: Discrete 2592x1944
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 2592x1458
			Interval: Discrete 0.033s (30.000 fps)
		Size: Discrete 1280x720
			Interval: Discrete 0.008s (120.000 fps)

nvidia@nvidia-desktop:~$ v4l2-ctl -d /dev/video0 --set-ctrl bypass_mode=0 --stream-mmap --stream-count=100
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.09 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.04 fps
<<<<<<<<<<<<<<<<<<<<<<<<<<<<<< 30.03 fps
<<<<<<<<