NVENC Quality - Blocky & Jumpy Blacks

Professional Graphics and Rendering Video Technologies Video Codec and Optical Flow SDK
#1

My company uses NVIDIA GPUs in an OEM Encoder to leverage realtime NVENC GPU encoding. When our customers play content encoded with nvenc_hevc on very bright LED walls / screens, regions of the video with a lot of black / dark content are noticeably and distractingly blocky and jumpy. Our previous generation encoder uses libx264 for encoding, and this is not a problem with videos encoded at the same bitrate. I understand there is a significant difference between CPU and GPU encoding, but the amount of blockiness and jumpiness with the blacks seems to be quite a significant problem with nvenc when watching on bright screens / in bright venues.

Oddly, the nvenc_h264 encoder seems to have this problem slightly less. I have attached some sample videos encoded with libx264, nvenc_h264, nvenc_hevc for comparison which you can download here: https://drive.google.com/file/d/1S3WZRk8RcfmZ5w9FEVyWU4KJ0HDJJzGL/view?usp=sharing.
Note: If you are viewing from a standard computer monitor, you will likely have to turn your screen brightness and contrast to 100% to see what I am talking about (what our customers see on LED walls).

The video bitrate of nvenc_hevc has to be nearly double that of libx264 to achieve a quality of blacks (i.e., steadiness) that is acceptable when compared to libx264. Is there any hope of improving the quality of the blacks with a software update, or is the hardware encoder chip pretty much set for good?

Thank you for your help!

#2

Hi paul_m,

Can you provide us with your encode API parameters (both HEVC and H264) and the command line for x264?

Thanks,
Ryan Park

#3

paul_m, it might be caused by this particular bug reported here many ages ago…

https://devtalk.nvidia.com/default/topic/958132/nvenc-hevc-with-full-range-colors-/

#4

rypark,
These are ffmpeg arguments that we are using.

-c:v hevc_nvenc -profile:v main10 -pix_fmt yuv420p -keyint_min 120 -map 0:v -b:v 12000k

-c:v libx264 -preset fast -profile:v baseline -pix_fmt yuv420p -keyint_min 60 -map 0:v -b:v 10000k

#5

Hi paul_m,

We’re filing a bug for your issue. Which operating system and GPU are you using?

Thanks,
Ryan Park

#6

Thanks, Ryan.

We are using CentOS 7.4 and the GTX 1050 Ti

#7

Hi Ryan, is there any update to this? We tested on RTX 2060 and it seems to have the same problem. Thank you!

#8

This is also an issue on Windows 10 using an nVIDIA GeForce GTX 1050 and especially bad with SD material. Info on HW encoder used:

NVEncC (x64) 4.36 (r1048) by rigaya, Apr  2 2019 13:02:42 (VC 1900/Win/avx2)
OS Version     Windows 10 x64 (17763)
CPU            Intel Core i7-4770K @ 3.50GHz [TB: 4.30GHz] (4C/8T)
GPU            #0: GeForce GTX 1050 (640 cores, 1455 MHz)[PCIe3x16][425.31]
NVENC / CUDA   NVENC API 9.0, CUDA 10.1, schedule mode: auto

Please let me know if additional info is needed. Thank you.

#9

We have same issue, most problematic is Dark Grey color.

#10

Hi.

Is it possible to share the raw data and encode command line(s) used to help us reproduce this issue internally?

#11

Mandar, I have updated the test asset folder to include the original file for testing, DarksTest_720p59.avi
I also included a comparison using RTX and GTX GPUs.
You can download the files here: https://drive.google.com/open?id=1EjMewFsuycbMyAzMjCSYJpr2Z3bvK9uL

The encoding command we use in production is slightly more complex, but this problem is visible even with FFmpeg default implementation. Here is an example with the GTX GPU:

ffmpeg -i /home/admin/Videos/DarksTest_720p59.avi -c:v hevc_nvenc -b:v 5000k -c:a aac -b:a 128k /home/admin/Videos/nvenc-hevc.mp4

Thunderm, interesting observation. We see it mostly where there is any grain in the image (due to high ISO on the camera), which is likely gray color.

Much thanks, all!

#12

Hi,
Sorry for the delay in at least providing a command line.

"NVEncC64.exe" --codec hevc --vbrhq 9623 --preset quality --qp-init 14 --qp-max 26 --qp-min 14
   --gop-len 72 --lookahead 32 --ref 4 --max-bitrate 12000 --aq --vpp-pmd threshold=200.0
   --vpp-deband range=4,thre=4,dither=31,rand_each_frame --videoformat ntsc --colormatrix bt709
   --colorprim bt709 --transfer bt709 --log ".\video.1080p.clip.PEMLOG.txt" -i "video.1080p.clip.mkv"
   -o ".\video.1080p.clip.hevc"

The source is encoded in x264 AVC. I usually get what I need using x264 2-pass with an output size of about 1.8G. When I switched to HEVC, x265 was taking a really long long time using 2-pass encoding. To get HEVC output using 1-pass HW encoding, I had to tweak the parameters to where the output, visually, is now close to AVC. Unfortunately the output files are now larger at about 2G to 2.1G as I couldn’t get the same or very close quality out of HEVC as what I normally got with the software based 2-pass AVC encoding.

I don’t have the clip handy as I’ll have to dig it out. Let me know if you still want it. Thank you.

#13

I took closer look on yours provided samples.

The main problem is that you converted yuvj422p(pc) (0-255) to same format for libx264, but when you converted this sample via NVENC you used yuv444p(tv) (16-235) which will degrade picture quality and introduce color banding artifacts.

Actualy this is not very good example of this problem on NVENC.

We are facing different issue, we have samples in yuv420(tv) limited range and when we convert them via NVENC then there are many banding artifacts in dark parts, H264 is actualy better than HEVC, if we convert samples to yuv420p10le(tv) and encode them as 10bit artifacts are gone. But libx265 doesn’t introduce any artifacts like that even in 8bit mode.

#14

Thank you for your reply, but I do not think the color range is the cause of the problem. To confirm this, I used one of the other pixel formats available for nvenc, p010le, and set the color range to full, and it still has the same problem. Video: https://drive.google.com/a/livingasone.com/file/d/1loT-Ps6Rk6srb2sHBhfat84KvRD5hWyT/view?usp=sharing.

Encoder hevc_nvenc [NVIDIA NVENC hevc encoder]:
    General capabilities: delay 
    Threading capabilities: none
    Supported pixel formats: yuv420p nv12 p010le yuv444p yuv444p16le bgr0 rgb0 cuda
#15

When i transcode this file on my 2070 RTX card I don’t see artifacts.

Can you check it:

https://drive.google.com/file/d/1UDmvU9MUR0-EQThD3yO7-_g7bMcxG1bU/view?usp=sharing

Also please post your ffmpeg command line to check what could be wrong.

For example it is well known that AQ aka Adaptive Quantitization (Temporal or Spatial) can downgrade image quality in dark parts.

Be aware that google drive will always do transcoding so download file and recheck it in your own player (eg. VLC).

COMMENT:
!!DON’T USE GTX or older GPU generation for transcoding into HEVC on any production system, as it is obsolete and new generation RTX brings around 50% quality improvments in all possible scenarios!!

#16

Is there any update on this, or a workaround?

#17 
  -c:v hevc_nvenc
  -profile:v "$vprofile"
  -threads:v 4
  -g:v "$keyframe_int"

  #-preset slow
  #-rc cbr_hq
  #-2pass 1
  -preset:v p7
  -tune:v hq
  -rc:v cbr
  -multipass:v fullres

  -rc-lookahead:v 32
  -bf:v 4 -b_ref_mode:v each
  #-bf:v 3 -b_ref_mode middle
  -spatial-aq:v 1
  -temporal-aq:v 1
)
nvenc_keyframeforce=(
  #-force_key_frames:v "expr:gte(t,n_forced*$keyframe_sec)"
  -keyint_min:v "$keyframe_int"
  -forced-idr:v 1
  -no-scenecut:v 1
)

this is the settings I use on ffmpeg master; even extraordinary amounts of bitrate thrown at it don’t seem to reduce the blocking much if at all

it’s only in areas where otherwise you get pretty bad banding in h264, but this isn’t really obviously better

if there’s a workaround using “worse” encoding settings or such that fix this even if only at high bitrate I’m very much interested