Broken picture after l4t decode+transform interlaced source

khizbulin · August 6, 2020, 12:22pm

Hi,

in l4t sdk 32.4.2 broke transcoder interlaced h264 source. After transcode picture twitching.

mag_broken

Reproduce:

gst-launch-1.0 filesrc location=mag0.mp4 ! qtdemux name=demux demux.video_0 ! queue ! h264parse ! nvv4l2decoder ! nvvidconv  ! 'video/x-raw(memory:NVMM), format=(string)I420' ! nvv4l2h264enc bitrate=5000000 ! h264parse ! queue ! matroskamux name=mux ! filesink location=mag.mkv -e

Downgrade to # R32 (release), REVISION: 2.3, GCID: 17644089, BOARD: t186ref, EABI: aarch64, DATE: Tue Nov 5 21:48:17 UTC 2019 solve problem.

DaneLLL · August 7, 2020, 12:16am

Hi,
r32.4.3 is production release. Please try this version. If the issue is still present, please share the video file so that we can reproduce the issue.

khizbulin · August 7, 2020, 9:05am

Hi,

last r32.4.3 also broken. Looks like bug from nvbuffer api, because my v4l2 code produce also twitching picture. For you fast investigate, gstreamer without nvvidconv create picture without twiching and source must be interlaced.

DaneLLL · August 12, 2020, 12:53am

Hi,
Please share the video clip so that we can reproduce the issue.

khizbulin · August 12, 2020, 7:14am

Hi,

interlaced sample https://drive.google.com/file/d/1I4KAwT-2-Wf69x7GeKgZZwW9PGerui0K/view?usp=sharing

DaneLLL · August 14, 2020, 2:02am

Hi,
We can reproduce the issue with the video file. Will investigate further and update.

khizbulin · November 5, 2020, 8:29am

r32.4.4 release don’t fix this issue

DaneLLL · November 5, 2020, 9:05am

Hi,
We don’t observe the issue in running the pipeline:

$ gst-launch-1.0 filesrc location=mag0.mp4 ! qtdemux name=demux demux.video_0 ! queue ! h264parse ! nvv4l2decoder ! nvv4l2h264enc bitrate=5000000 ! h264parse ! queue ! matroskamux name=mux ! filesink location=mag-1.mkv

Please give it a try.

khizbulin · June 23, 2021, 4:50am

Hi,

remove nvvidconv don’t solve my problem. Video broken when resize use

gst-launch-1.0 filesrc location=mag0.mp4 ! qtdemux name=demux demux.video_0 ! queue ! h264parse ! nvv4l2decoder ! nvvidconv  ! 'video/x-raw(memory:NVMM), width=(int)320, height=(int)200, format=(string)I420' ! nvv4l2h264enc bitrate=5000000 ! h264parse ! queue ! matroskamux name=mux ! filesink location=mag-1.mkv -e

I’m watch something like when use samples.

./video_decode H264 --disable-rendering --input-nalu -f 2 -v4l2-memory-cap-plane 1 -o /mnt/nfs/ts/mag0.yuv /mnt/nfs/ts/mag0.h264

Looks like NvBufferTransform produce broken image nv12 → yuv420p.

khizbulin · June 23, 2021, 4:54am

jetson tx2 4gb

# cat /etc/nv_tegra_release 
# R32 (release), REVISION: 5.1, GCID: 27362550, BOARD: t186ref, EABI: aarch64, DATE: Wed May 19 18:16:00 UTC 2021

# dpkg -l | grep l4t
ii  nvidia-l4t-3d-core                            32.5.1-20210519111140                            arm64        NVIDIA GL EGL Package
ii  nvidia-l4t-apt-source                         32.5.1-20210219084708                            arm64        NVIDIA L4T apt source list debian package
ii  nvidia-l4t-bootloader                         32.5.1-20210614115015                            arm64        NVIDIA Bootloader Package
ii  nvidia-l4t-camera                             32.5.1-20210519111140                            arm64        NVIDIA Camera Package
ii  nvidia-l4t-configs                            32.5.1-20210219084708                            arm64        NVIDIA configs debian package
ii  nvidia-l4t-core                               32.5.1-20210519111140                            arm64        NVIDIA Core Package
ii  nvidia-l4t-cuda                               32.5.1-20210519111140                            arm64        NVIDIA CUDA Package
ii  nvidia-l4t-firmware                           32.5.1-20210519111140                            arm64        NVIDIA Firmware Package
ii  nvidia-l4t-graphics-demos                     32.5.1-20210519111140                            arm64        NVIDIA graphics demo applications
ii  nvidia-l4t-gstreamer                          32.5.1-20210519111140                            arm64        NVIDIA GST Application files
ii  nvidia-l4t-init                               32.5.1-20210519111140                            arm64        NVIDIA Init debian package
ii  nvidia-l4t-initrd                             32.5.1-20210614115015                            arm64        NVIDIA initrd debian package
ii  nvidia-l4t-jetson-io                          32.5.1-20210219084708                            arm64        NVIDIA Jetson.IO debian package
rc  nvidia-l4t-jetson-multimedia-api              32.5.1-20210519111140                            arm64        NVIDIA Jetson Multimedia API is a collection of lower-level APIs that support flexible application development.
ii  nvidia-l4t-kernel                             4.9.201-tegra-32.5.1-20210505093723              arm64        NVIDIA Kernel Package
ii  nvidia-l4t-kernel-dtbs                        4.9.201-tegra-32.5.1-20210505093723              arm64        NVIDIA Kernel DTB Package
ii  nvidia-l4t-kernel-headers                     4.9.201-tegra-32.5.1-20210505093723              arm64        NVIDIA Linux Tegra Kernel Headers Package
ii  nvidia-l4t-libvulkan                          32.5.1-20210519111140                            arm64        NVIDIA Vulkan Loader Package
ii  nvidia-l4t-multimedia                         32.5.1-20210519111140                            arm64        NVIDIA Multimedia Package
ii  nvidia-l4t-multimedia-utils                   32.5.1-20210519111140                            arm64        NVIDIA Multimedia Package
ii  nvidia-l4t-oem-config                         32.5.1-20210219084708                            arm64        NVIDIA OEM-Config Package
ii  nvidia-l4t-tools                              32.5.1-20210614115015                            arm64        NVIDIA Public Test Tools Package
ii  nvidia-l4t-wayland                            32.5.1-20210519111140                            arm64        NVIDIA Wayland Package
ii  nvidia-l4t-weston                             32.5.1-20210519111140                            arm64        NVIDIA Weston Package
ii  nvidia-l4t-x11                                32.5.1-20210519111140                            arm64        NVIDIA X11 Package
ii  nvidia-l4t-xusb-firmware                      32.5.1-20210614115015                            arm64        NVIDIA USB Firmware Package

khizbulin · June 25, 2021, 12:39pm

Also, NvBufferTransform not guilty. Corrupts video dumps directly from dma_buf capture plane decoder. I’ve tested several codecs. With interlaced source decoder produce corrupt video, and progressive source works well.

DaneLLL · July 8, 2021, 1:35am

Hi,
Would like to know what your use-case is. For transcoding, you can simply link

... ! nvv4l2decoder ! nvv4l2h264enc ! ...

We have checked with teams and provided this suggestion. It is not required to convert NV12 to I420. Not sure why you have to do it in your use-case. Keeping NV12 should be a valid solution for general use-cases.

khizbulin · July 8, 2021, 8:36am

Hi,

without nvvidconv transcoder works fine, but i need resized video. Also in another project, i send decoded dma_buf from nvv4ldecoder to cuda and receive in cuda code artefacts from interlaced source too. Progressive source works ok.

Thanks for your support.

khizbulin · July 8, 2021, 9:12am

use-case is:

transcode interlaced source to resized encoded destination
decode interlaced source and send dma_buf with raw nv12 to cuda code

DaneLLL · July 13, 2021, 9:12am

Hi,
Please try the command by setting num-extra-surfaces and adding queue :

$ gst-launch-1.0 filesrc location=mag0.mp4 ! qtdemux name=demux demux.video_0 ! queue ! h264parse ! nvv4l2decoder num-extra-surfaces=4 ! queue ! nvvidconv ! 'video/x-raw(memory:NVMM), width=320,height=240' ! nvoverlaysink

khizbulin · July 13, 2021, 1:36pm

Hi,
Artifacts remain, but they appear much less frequently.

source: ao0.mp4 - Google Drive
result: a3.mkv - Google Drive

artefacts at 00:01:01, 00:02:00, 00:02:41 etc

gst-launch-1.0 filesrc location=ao0.mp4 ! qtdemux name=demux demux.video_0 ! queue ! h264parse ! nvv4l2decoder num-extra-surfaces=4 ! queue ! nvvidconv  ! 'video/x-raw(memory:NVMM), format=(string)I420' ! nvv4l2h264enc bitrate=7000000 ! h264parse ! queue ! matroskamux name=mux ! filesink location=a3.mkv -e

DaneLLL · July 14, 2021, 1:28am

Hi,
Does it happen if you do rescaling like:

nvvidconv ! 'video/x-raw(memory:NVMM), width=320,height=240'

Or it only happens in converting to I420:

nvvidconv  ! 'video/x-raw(memory:NVMM), format=(string)I420'

Converting NV12 to I420 looks not required. It is better to send NV12 buffers to hardware encoder.

DaneLLL · July 23, 2021, 6:33am

Hi,
Please also try to adjust num-extra-surfaces and min-threshold-buffers like:

... ! nvv4l2decoder num-extra-surfaces=8 ! queue min-threshold-buffers=8 ! ...