Nvvidconv scaling causes horizontal tearing near the top of the frame

Autonomous Machines Jetson & Embedded Systems Jetson Xavier NX
1

Hi,
My use-case is an avionics DVR.
I am using Linux R35.5.0 (Jetpack 5.1.3) with a custom carrier board.
I have modified the device tree and ov5693 driver to support camera capture from an FPGA.

In this use-case the data path is NTSC composite camera 720x480 ->ADV7842->FPGA->Jetson MIPI.
De-interlacing to 480p is done by ADV7842. Jetson captures at 768x480 to obey 64 byte stride alignment requirement. I use nvidconv to remove the extra 48 pixels per line. This works fine but when there is horizontal motion near the top of the frame tearing occurs.

Running “/usr/bin/jetson_clocks” prior does not fix the issue.
Putting an extra “queue” before nvvidconv does not fix the issue.

Please advise on how to resolve this.

TEARING:
gst-launch-1.0 nvv4l2camerasrc device=/dev/video0
! ‘video/x-raw(memory:NVMM),width=768,height=480,framerate=60/1’
! nvvidconv left=0 right=720 top=0 bottom=480
! queue ! nvdrmvideosink conn_id=0 plane_id=1 set_mode=0
-e

NO TEARING:
gst-launch-1.0 nvv4l2camerasrc device=/dev/video0
! ‘video/x-raw(memory:NVMM),width=768,height=480,framerate=60/1’
! queue ! nvdrmvideosink conn_id=0 plane_id=1 set_mode=0
-e

3

hello alanser,

could you please check those frames sending to Jetson CSI channel are consistency?
you may examine frame-by-frame for confirmation.

4

frames from ADV7842 is consistent and correct.

When I use nvvidconv with:
v4l2src with caps “video/x-raw” as below instead of
nvv4l2camerasrc with caps “video/x-raw(memory:NVMM)” there is no tearing.
Why? Seems like memory:NVMM DMA with 768x480 is the issue?
I could go that way but we want to avoid the overhead of v4l2c memcpy to move video frames.

gst-launch-1.0 v4l2src device=/dev/video0 ! ‘video/x-raw, width=768, height=480,framerate=60/1’
! nvvidconv left=0 right=720 top=0 bottom=480
! queue !
nvdrmvideosink conn_id=0 plane_id=1 set_mode=0
-e

5

hello alanser,

it looks like a bug,
had you try nvv4l2camerasrc with caps “video/x-raw” as well?

6

nvv4l2camerasrc with “video/x-raw” gives an error as below. Does “memory:NVMM” DMA have any stride requirement?

gst-launch-1.0 nvv4l2camerasrc device=/dev/video0
! ‘video/x-raw,width=768,height=480,framerate=60/1’
! nvvidconv left=0 right=720 top=0 bottom=480
! queue ! nvdrmvideosink conn_id=0 plane_id=1 set_mode=0
-e
WARNING: erroneous pipeline: could not link nvv4l2camerasrc0 to nvvconv0, nvv4l2camerasrc0 can’t handle caps video/x-raw, width=(int)768, height=(int)480, framerate=(fraction)60/1

7

hello alanser,

my bad, nvv4l2camerasrc only support video/x-raw(memory:NVMM)
you may execute $ gst-inspect-1.0 nvv4l2camerasrc to check plugin details.

is it progressive mode or interlaced mode? could you please change sink element for testing?
for instance,
$ gst-launch-1.0 nvv4l2camerasrc device=/dev/video0 ! 'video/x-raw(memory:NVMM), ,width=768,height=480,framerate=60/1' ! interlace-mode=progressive' ! nvvidconv ! fpsdisplaysink text-overlay=0 video-sink=xvimagesink sync=0 -v

8

Jerry,
"

progressive

I do not have X, Wayland or any other display servers. Only video renderer is nvdrmvideosink.

If I encode to a file with no video sink and with nvvidconv in the path as per below I get tearing in the encoded file.

gst-launch-1.0 nvv4l2camerasrc device=/dev/video0
! ‘video/x-raw(memory:NVMM),width=768,height=480,framerate=60/1,interlace-mode=progressive’
! nvvidconv
! queue
! nvv4l2h264enc ! h264parse ! mpegtsmux ! filesink location=/media/nvme0n1p2/1.ts

10

Hi,
The frames are captured in ioctl(VIDIOC_DQBUF) in gst_nvv4l2camera_buffer_pool_acquire_buffer(). Please add NvBufSurfaceSyncForDevice() or NvBufSurfaceSyncForCpu() after the frame data is captured for a try. The source code of nvv4l2camerasrc is in

Jetson Linux | NVIDIA Developer
Driver Package (BSP) Sources

11

I added that (see below), but I am now getting these errors for both Cpu and Device:

NvMapMemCacheMaint Bad parameter
NvMapMemCacheMaint Bad parameter
e: Error(4) in sync
nvbusurface: NvBufSurfaceSyncForDevice: Error(4) in sync
nvbusurface: NvBufSurfaceSyncForDevice: Error(4) in sync

if (-1 == ioctl(pool->video_fd, VIDIOC_DQBUF, nv_mem->nvcam_buf->buffer))
goto dqbuf_error;
NvBufSurfaceSyncForCpu (nv_mem->nvcam_buf->surface, 0, 0);
//NvBufSurfaceSyncForDevice(nv_mem->nvcam_buf->surface, 0, 0);

12

Hi,
Please refer to other samples which calls NvBufSurfaceSyncForCpu()/NvBufSurfaceSyncForDevice(), and check if any parameter is wrong.

13

all examples I could find always use 0 for 2nd parameter (index)

jetson@ubuntu:~/prj/jetpack5$ grep -r --include *.c NvBufSurfaceSyncFor
gst-nvvidconv/gstnvvconv.c: ret = NvBufSurfaceSyncForCpu (surface, 0, i);
gst-nvvidconv/gstnvvconv.c: ret = NvBufSurfaceSyncForDevice (surface, 0, i);

jetson@ubuntu:~/prj/root/Jetson/build/l4t/Linux_for_Tegra$ sudo grep -r --include *.c NvBufSurfaceSyncFor
rootfs/usr/src/nvidia/graphics_demos/weston-dmabuf-formats/weston-dmabuf-formats.c: CHECK(NvBufSurfaceSyncForCpu (buf->surface, 0, 0));
rootfs/usr/src/nvidia/graphics_demos/weston-dmabuf-formats/weston-dmabuf-formats.c: NvBufSurfaceSyncForCpu (buf->surface, 0 , 1);
rootfs/usr/src/nvidia/graphics_demos/weston-dmabuf-formats/weston-dmabuf-formats.c: CHECK(NvBufSurfaceSyncForCpu (buf->surface, 0, 0));
rootfs/usr/src/nvidia/graphics_demos/weston-dmabuf-formats/weston-dmabuf-formats.c: CHECK(NvBufSurfaceSyncForCpu (buf->surface, 0, 0));
rootfs/usr/src/nvidia/graphics_demos/weston-dmabuf-formats/weston-dmabuf-formats.c: CHECK(NvBufSurfaceSyncForCpu (buf->surface, 0, 1));
rootfs/usr/src/nvidia/graphics_demos/weston-dmabuf-formats/weston-dmabuf-formats.c: CHECK(NvBufSurfaceSyncForCpu (buf->surface, 0, 0));
rootfs/usr/src/nvidia/graphics_demos/weston-dmabuf-formats/weston-dmabuf-formats.c: CHECK(NvBufSurfaceSyncForCpu (buf->surface, 0, 0));
rootfs/usr/src/nvidia/graphics_demos/weston-dmabuf-formats/weston-dmabuf-formats.c: CHECK(NvBufSurfaceSyncForCpu (buf->surface, 0, planeIdx));

I also tried with -1, -1 parameters but that also reports the same errors
NvBufSurfaceSyncForCpu (nv_mem->nvcam_buf->surface, -1, -1);

Index of a buffer in the batch. -1 refers to all buffers
in the batch.
Index of a plane in the buffer. -1 refers to all planes
in the buffer.

if this was the issue would this pipeline not also produce tearing, which it does not (only if nvvidconv is added)?

NO TEARING:
gst-launch-1.0 nvv4l2camerasrc device=/dev/video0
! ‘video/x-raw(memory:NVMM),width=768,height=480,framerate=60/1’
! queue ! nvdrmvideosink conn_id=0 plane_id=1 set_mode=0
-e

15

you may see-also Buffer Surface Management API.

16

NvBufSurfaceSyncFor… failed because NvBufSurfaceMap is needed in gst_nv_memory_allocator_alloc():

NvBufSurfaceMap(nvbuf->surface, 0, 0, NVBUF_MAP_READ_WRITE);

Anyway, both of these are now working without returning any errors but tearing issue persists:
NvBufSurfaceSyncForCpu (nv_mem->nvcam_buf->surface, 0, 0);
NvBufSurfaceSyncForDevice(nv_mem->nvcam_buf->surface, 0, 0);

Any ideas?

gstnvv4l2camerasrc.diff.txt (1.3 KB)

17

Hi,
It looks like the userspace APIs do not help. Could you check the sensor driver? Not sure if the buffers can be synchronized in kernel driver.

18

you mean VI5 driver? My ov5693 driver does not seem to have any means to do that… Nor does VI5 driver as far as I can see.

19

Hi,
I mean in ADV7842->FPGA. Is the frame data put in CPU buffer or DMA buffer? Could you please check this?

20

The data path is:
Composite video input (720x480i60)
TO: ADV7842 (deinterlaces and outputs 720x480p60)
TO: FPGA (adds 48 pixels to obey Jetson 64 byte stride alignment requirement and puts 768x480p60 on MIPI bus)
TO: Jetson MIPI.

For FPGA MIPI source we use a hacked OV5693 driver.
We use gstreamer and nvvidconv to remove added 48 pixels as per pipeline below. As for CPU or DMA buffer… I believe the answer to that is DMA buffer since we use nvv4l2camerasrc and memory:NVMM

gst-launch-1.0 nvv4l2camerasrc device=/dev/video0
! ‘video/x-raw(memory:NVMM),width=768,height=480,framerate=60/1’
! nvvidconv left=0 right=720 top=0 bottom=480
! queue ! nvdrmvideosink conn_id=0 plane_id=1 set_mode=0
-e

And as I have said before, if I remove nvvidconv from pipeline above there is no tearing.

21

Hi,
The frame data is not synchronized while VIC engine accesses it. You would need to check if it is possible to do synchronization in kernel driver. If it is not possible, please use v4l2src plugin to capture frame data to CPU buffer first and then copy to NVMM buffer(NvBufSurface).

22

Please explain to me why there is only tearing if nvvidconv is added to the pipeline?

@DaneLLL Are you referring to VI driver?

23

Hi,
I mean driver of the ADV7842 or FPGA. It may need to add some code for synchronization.