ffmpeg NVDEC/CUVID produces artifacts when decoding h264

For one particular file I have, CUDA decoding seems to produce artifacts that I can’t reproduce on software decoding or with VDPAU. There relevant ffmpeg issue is here:

Example of the artifacts:

Example clip that can reproduce artifacts:

Example command to reproduce:

ffmpeg -report -c:v h264_cuvid -ss 00:00:01.0 -i LB-test-small.mkv -vframes 1 output.png

OS: GNU/Linux 64-bit
GPU: NVIDIA Corporation GM206 [GeForce GTX 960] (rev a1)
Drivers: NVIDIA UNIX x86_64 Kernel Module 384.59 Wed Jul 19 23:53:34 PDT 2017

I’m actually not sure if this is a driver issue, or ffmpeg’s implementation of the cuda decoder, but I thought I’d report it here if that could help.

It’s normal behavior. The coded size of height in the elementary stream is 544, to give an integral number of macroblocks. But the stream specifies cropping to height 532. CUVID only reports the display size it does not actually crop the video. You have to do that yourself. So apparently ffmpeg is not honoring the stream’s display_area fields.

Your MKV container also specifies 532 but CUVID only knows about elementary streams.

ffmpeg does respect the display_area - otherwise all 1080p content would be glitched in the same way (1088 coded size, of course)

Possibly they do it only for 1088->1080. That’s pretty common, check the code.

Regardless, as I said, the OP’s stream has a display area specified and CUVID only reports it.