My company uses NVIDIA GPUs in an OEM Encoder to leverage realtime NVENC GPU encoding. When our customers play content encoded with nvenc_hevc on very bright LED walls / screens, regions of the video with a lot of black / dark content are noticeably and distractingly blocky and jumpy. Our previous generation encoder uses libx264 for encoding, and this is not a problem with videos encoded at the same bitrate. I understand there is a significant difference between CPU and GPU encoding, but the amount of blockiness and jumpiness with the blacks seems to be quite a significant problem with nvenc when watching on bright screens / in bright venues.
Oddly, the nvenc_h264 encoder seems to have this problem slightly less. I have attached some sample videos encoded with libx264, nvenc_h264, nvenc_hevc for comparison which you can download here: nvidia_test_files.zip - Google Drive.
Note: If you are viewing from a standard computer monitor, you will likely have to turn your screen brightness and contrast to 100% to see what I am talking about (what our customers see on LED walls).
The video bitrate of nvenc_hevc has to be nearly double that of libx264 to achieve a quality of blacks (i.e., steadiness) that is acceptable when compared to libx264. Is there any hope of improving the quality of the blacks with a software update, or is the hardware encoder chip pretty much set for good?
The encoding command we use in production is slightly more complex, but this problem is visible even with FFmpeg default implementation. Here is an example with the GTX GPU:
The source is encoded in x264 AVC. I usually get what I need using x264 2-pass with an output size of about 1.8G. When I switched to HEVC, x265 was taking a really long long time using 2-pass encoding. To get HEVC output using 1-pass HW encoding, I had to tweak the parameters to where the output, visually, is now close to AVC. Unfortunately the output files are now larger at about 2G to 2.1G as I couldn’t get the same or very close quality out of HEVC as what I normally got with the software based 2-pass AVC encoding.
I don’t have the clip handy as I’ll have to dig it out. Let me know if you still want it. Thank you.
The main problem is that you converted yuvj422p(pc) (0-255) to same format for libx264, but when you converted this sample via NVENC you used yuv444p(tv) (16-235) which will degrade picture quality and introduce color banding artifacts.
Actualy this is not very good example of this problem on NVENC.
We are facing different issue, we have samples in yuv420(tv) limited range and when we convert them via NVENC then there are many banding artifacts in dark parts, H264 is actualy better than HEVC, if we convert samples to yuv420p10le(tv) and encode them as 10bit artifacts are gone. But libx265 doesn’t introduce any artifacts like that even in 8bit mode.
When i transcode this file on my 2070 RTX card I don’t see artifacts.
Can you check it:
Also please post your ffmpeg command line to check what could be wrong.
For example it is well known that AQ aka Adaptive Quantitization (Temporal or Spatial) can downgrade image quality in dark parts.
Be aware that google drive will always do transcoding so download file and recheck it in your own player (eg. VLC).
COMMENT: !!DON’T USE GTX or older GPU generation for transcoding into HEVC on any production system, as it is obsolete and new generation RTX brings around 50% quality improvments in all possible scenarios!!