We recently moved our custom Orin AGX Industrial board from Jetpack 5.1.4 (R35.6.0) to Jetpack 6.2.0 (R36.4.3) and noticed a regression in our camera pipeline, whereby we can’t get the VI driver stack to capture frames into DMA compatible buffers for other downstream processing tasks.
We’ve been using the direct v4l2 ioctl path along with the Jetson Accelerated Gstreamer library plugins in our pipeline. We do not use the libargus / nvarguscamerasrc flow, nor the ISP within the Jetson SoC.
Specifically, we’ve been using the gst-nvv4l2camerasrc gstreamer plugin installed from apt, but this does not work anymore in JP6. The issues observed are the same for plugins built from source as well. We suspect this is because of changes in the Nvidia VI camera driver stack, which leads to issues with DMA buffer importing in the v4l2 driver stack, but we haven’t been able to find a stable patch fix that doesn’t affect other subsystems.
Although we do use a proprietary camera driver, and a custom device tree, these are functional and enumerate the camera the same way between Jetpack releases on the Orin. There are components of v4l2/gstreamer that work in both releases too. For context, using v4l2src (without DMA and zero-copy mechanisms) works fine and so do other Accelerated Gstreamer plugins like nvvidconv which use the VIC in the SoC.
# This pipeline works on both JP5 and JP6 with AGX Orin.
gst-launch-1.0 v4l2src device=/dev/video4 ! "video/x-raw, width=3840, height=2160, format=UYVY" ! nvvidconv flip-method=2 ! "video/x-raw(memory:NVMM), format=NV12" ! nvv4l2h264enc bitrate=28000000 insert-sps-pps=1 ! filesink location=output.h264
# A sample command to reproduce the issue in gstreamer with nvv4l2camerasrc on Jetpack6.
gst-launch-1.0 nvv4l2camerasrc device=/dev/video3 ! 'video/x-raw(memory:NVMM), width=3840, height=2160, format=UYVY, framerate=30/1' ! fakesink
# Original nvv4l2camerasrc plugin output on Jetpack6:
$ GST_DEBUG=3 GST_PLUGIN_PATH=. gst-launch-1.0 nvv4l2camerasrc device=/dev/video3 ! 'video/x-raw(memory:NVMM), width=3840, height=2160, format=UYVY, framerate=30/1' ! fakesink
Setting pipeline to PAUSED ...
Pipeline is live and does not need PREROLL ...
0:00:02.445469071 5916 0xaaab10562000 FIXME default gstutils.c:4025:gst_pad_create_stream_id_internal:<nvv4l2camerasrc0:src> Creating random stream-id, consider implementing a deterministic way of creating a stream-id
Pipeline is PREROLLED ...
Setting pipeline to PLAYING ...
0:00:02.445708305 5916 0xaaab10562000 WARN nvv4l2camerasrc gstnvv4l2camerasrc.cpp:684:gst_nv_v4l2_camera_fixate:<nvv4l2camerasrc0> Unable to fixate caps structure
New clock: GstSystemClock
0:00:02.611785893 5916 0xaaab10562000 ERROR nvv4l2camerasrc gstnvv4l2camerasrc.cpp:525:gst_nvv4l2camera_buffer_pool_release_buffer:<nvv4l2camerabufferpool0> VIDIOC_QBUF failed :Invalid argument
0:00:02.612032901 5916 0xaaab10562000 ERROR nvv4l2camerasrc gstnvv4l2camerasrc.cpp:525:gst_nvv4l2camera_buffer_pool_release_buffer:<nvv4l2camerabufferpool0> VIDIOC_QBUF failed :Invalid argument
0:00:02.612220974 5916 0xaaab10562000 ERROR nvv4l2camerasrc gstnvv4l2camerasrc.cpp:525:gst_nvv4l2camera_buffer_pool_release_buffer:<nvv4l2camerabufferpool0> VIDIOC_QBUF failed :Invalid argument
0:00:02.612411990 5916 0xaaab10562000 ERROR nvv4l2camerasrc gstnvv4l2camerasrc.cpp:525:gst_nvv4l2camera_buffer_pool_release_buffer:<nvv4l2camerabufferpool0> VIDIOC_QBUF failed :Invalid argument
0:00:02.612607325 5916 0xaaab10562000 ERROR nvv4l2camerasrc gstnvv4l2camerasrc.cpp:525:gst_nvv4l2camera_buffer_pool_release_buffer:<nvv4l2camerabufferpool0> VIDIOC_QBUF failed :Invalid argument
0:00:02.612791142 5916 0xaaab10562000 ERROR nvv4l2camerasrc gstnvv4l2camerasrc.cpp:525:gst_nvv4l2camera_buffer_pool_release_buffer:<nvv4l2camerabufferpool0> VIDIOC_QBUF failed :Invalid argument
I see a similar issue with the Jetson Multimedia API Samples 12_v4l2_camera_cuda, where setting
up the DMA buffers fails there as well, returning EFAULT. This sample and the gstreamer pipeline both
use the V4L2_MEMORY_DMABUF type for the underlying V4L2 buffer memory type, so the issues seem
related.
I’ve attached a simpler test program (v412_dma_test) that reproduces this EFAULT error on JP6 and uses the same sequence of ioctls.
v4l2_dma_issue_repro_jp6.zip (12.8 KB)
Here is a comparison of the output I see on JP5 and JP6 using my test script in the zip with the IMX490 attached.
# On Jetpack5:
$ ./v412_dma_test
VIDIOC_G_FMT: Current format: 3840x2160, FourCC=UYVY, Size=16588800
Created NvBufSurface w,h=(3840x2160), pitch=7680, isContiguous=0
NvBufSurfaceParams dma_fd=5, dataSize=16646144, dataPtr=0xaaab034125a0
NvBufSurfaceMappedAddr ptr=0xffffebe3db60
Requesting 1 dma buffer(s)
VIDIOC_REQBUFS: Allocated 1 buffer(s), is V4L2_BUF_TYPE_VIDEO_CAPTURE = 1, is V4L2_MEMORY_DMABUF = 1
VIDIOC_QUERYBUF returned buf with dma_fd=0! Setting to 5 again before queuing.
VIDIOC_QUERYBUF buf: index=0, length=16588800, dma_fd=5, flags=0x2000, bytesused=0
VIDIOC_QBUF succeeded.
VIDIOC_STREAMON succeeded
VIDIOC_DQBUF dequeued buffer 0
VIDIOC_STREAMOFF succeeded
# On Jetpack6:
$ ./v412_dma_test
VIDIOC_G_FMT: Current format: 3840x2160, FourCC=UYVY, Size=16588800
Created NvBufSurface w,h=(3840x2160), pitch=7680, isContiguous=0
NvBufSurfaceParams dma_fd=5, dataSize=16646144, dataPtr=0xaaaaff985230
NvBufSurfaceMappedAddr ptr=0xffffd93de310
Requesting 1 dma buffer(s)
VIDIOC_REQBUFS: Allocated 1 buffer(s), is V4L2_BUF_TYPE_VIDEO_CAPTURE = 1, is V4L2_MEMORY_DMABUF = 1
VIDIOC_QUERYBUF returned buf with dma_fd=0! Setting to 5 again before queuing.
VIDIOC_QUERYBUF buf: index=0, length=17113344, dma_fd=5, flags=0x2000, bytesused=0
VIDIOC_QBUF failed: Bad address (errno=14)
Note that the length of the v4l2_buffer that VIDIOC_QUERYBUF ioctl fills in after returning in JP5 is equal to the size of the image, whereas in JP6 it is larger than the allocated NvBufSurface – 17113344 vs 16588800 bytes. The NvBufSurface allocation size is consistent between releases. I believe this causes an internal size-check to fail when the DMA buffer is imported in the v4l2 driver stack.
I suspect the difference could be due to a change in the VI channel driver in Jetpack6. I’ve attached a snippet below, showing how it now expects extra space for embedded metadata in channel.c: tegra_channel_queue_setup.
static int
tegra_channel_queue_setup(struct vb2_queue *vq,
unsigned int *nbuffers, unsigned int *nplanes,
unsigned int sizes[], struct device *alloc_devs[])
{
...
if (*nplanes) {
if (sizes[0] < chan->format.sizeimage) {
pr_err("%s: sizes[0] = %d chan->format.sizeimage = %d ...\n"
,__func__,sizes[0],chan->format.sizeimage);
return -EINVAL;
}
} else {
# On JP5 this was just sizes[0] = chan->format.sizeimage;
sizes[0] = chan->format.sizeimage + EMBEDDED_DATA_MAX_SIZE + EMBEDDED_DATA_BUFFER_ZONE_SIZE;
}
*nplanes = 1;
....
}
Embedded metadata is not a feature we use in our camera pipeline, and it has been disabled in the device tree entry by setting “embedded_metadata_height=0", as suggested by the nvidia docs.
A second potentially related issue is that the VIDIOC_QUERYBUF ioctl sets the returned file descriptor of the struct v4l2_buffer to zero, although it returns successfully. In the v412_dma_test program, I workaround this to ensure the file descriptor exported by libnvbufsurface is passed into the subsequent VIDIOC_QBUF call. This is what the gstreamer plugin does as well after calling VIDIOC_QUERYBUF.
Temporarily, I’ve been using the v4l2src plugin without DMA instead of nvv4l2camerasrc to work around these issues, but this has a huge performance hit for the overall pipeline in terms of end-to-end latency and CPU usage.
I’ve tried patching this change in channel.c and vi5_fops.c to remove the addition of embedded metadata and revert to how JP5 VI driver works, but this crashes nvgpu driver when I load nvvidconv plugin due to unserviced DMA interrupts / errors. See v4l2_revert.patch in the uploaded zip.
$ gst-inspect-1.0 nvvidconv # makes nvgpu crash with my patch.
