We are observing PSNR and visual quality drop in NVENC since drivers 525 (550 also tested) when using B-frames with parameter reference MIDDLE.
VMAF quality drop is around 0.7 (avg 82.7 → 82 in our samples)
Degradation is observed only on B ref mode MIDDLE and HEVC, there is no any issues with mode EACH and NONE or with H264 instead.
It looks like issue is located in part which is calculating number of reference frames as when limiting reference frames to 2 it is not observed when using 1 B-frames.
Is anyone else is facing this issue?
Do you need more info?
Is B ref mode MIDDLE still recomended or should we switch to EACH?
It looks like issue is with L0/L1 reference frames functionality, from our tests mode MIDDLE will ignore any value set in numRefL1 and will treat it as zero.
Same test with mode EACH will output different PSNR for different combinations of L0/L1.
Test with 4 B-frames: numRefL0 / numRefL1 - PSNR
--- EACH ---
auto - 34.065502
2/2 - 34.010047
4/4 - 34.073324
6/6 - 34.085693
2/0 - 33.945370
4/0 - 34.065502 <-- ref (auto)
6/0 - 34.076715
--- MIDDLE ---
auto - 33.986005
2/2 - 33.928253 <-- should be more
4/4 - 33.986005 <-- should be more
6/6 - 33.986005 <-- should be more
2/0 - 33.928253
4/0 - 33.986005 <-- ref (auto)
6/0 - 33.986005
I do not think setting multiref-l0 above six or multiref-l1 above two are allowed by the driver for middle.
With bframes and any bref-mode enabled, setting multiref-l0 above six or multiref-l1 above two results in a driver hang on my GTX 1660 Super. This makes sense to me, as according to the x265 documents (https://x265.readthedocs.io/en/default/cli.html), a) a conformant HEVC stream can have a maximum of 8 l0 references, b) enabling bframes adds 1 ref to the list of l0 references, and c) using bframes as references (which x265 calls --b-pyramid) adds another, and 6+2=8 after all.
Presumably this also means the maximum number of l1 refs is four, but I’d have to read through the HEVC specification to find out.
Would it be possible to add a mechanism to prevent the total number of l0/l1 refs from rising above 8/4 total? I can remember to avoid bad settings without issue, but I could see it being a big problem for less experienced users.
That is TRUE, but it should be handled primary in SW like FFMPEG, eg. there is code in NVENC GitHub - rigaya/NVEnc: NVENCによる高速エンコードの性能実験 project, that is limiting value as described, i don’t think that there should be limit in DRIVERs if they can handle more REF frames.
--- EACH ---
auto - 34.042956
4/0 - 34.042956
--- MIDDLE ---
auto - 34.058473
4/0 - 34.058473
Maybe issue is not in REF frames, but degradation is only observed in b_ref_mode middle, i can do more research to test if there is something different in later 525+ drivers directly in stream.