What transform does NVENC use from BGRA to YUV/YCbCr?

I am encoding video with NVENC and CUDA. The encoder is initialized with values returned from nvEncGetEncodePresetConfigEx()with NV_ENC_CODEC_H264_GUID, NV_ENC_PRESET_P1_GUID, and NV_ENC_TUNING_INFO_LOW_LATENCY. Each frame is passed to the encoder with the format NV_ENC_BUFFER_FORMAT_ABGR. The encoded images are 8-bit sRGB (e.g. they have gamma applied for display on a computer monitor).

This works and NV12 format frames are received and manually converted back to BGRA on the client but I believe there is some slight hue shifting that I could fix by better matching the encoder when transforming back to BGRA on the client.

  1. What transform does NVENC use to transform BGRA to YUV/YCbCr?
  2. Can I configure NVENC to use the BT.709 conversion (assuming this is best for sRGB)?
  3. Is there a better configuration to round-trip sRGB images?

Thanks,
Charles

  1. What transform does NVENC use to transform BGRA to YUV/YCbCr?

No transform since avcodec/nvenc: add support for gbrp rgb input · FFmpeg/FFmpeg@7555d6f · GitHub

  1. Can I configure NVENC to use the BT.709 conversion (assuming this is best for sRGB)?

sRGB does not use BT.709 transfer. It also does not use any matrix, it is rgb. Matrix means YCbCr, not RGB.

  1. Is there a better configuration to round-trip sRGB images?

What do you mean? Not unless you use 10 bit YCbCr for 8 bit rgb.

Thanks val.zapod.vz.

I’m encoding a BGRA buffer with NVENC (directly not via ffmpeg) that, after decoding, I receive as an NV12 (e.g. YCbCr) format buffer. I want to know what transform is correct to get from NV12 back to BGRA again? I couldn’t find any explicit documentation on what NVENC does wrt. transforming RGB to YCbCr.

I hadn’t looked in the ffmpeg source though so that link should be helpful. Thanks.

NV12 is not YCbCr, that is not that simple. Technically there is no garantee pixel format says anything about the colorspace. P010/P016 can be used in many cases too. Yes, in most cases NV12 is YCbCr, but it can be YCgCo or ICtCp. And there can be NV30 there.

*But it is fare to assume it is 420 YCbCr. Why? Because NV12 is 12 bit per pixel, so 8 bit for Y and CbCr both take two times less than that, so 4 bits. So 12 bit alltogether.