PyNvENC HEVC+alpha

Gaming and Visualization Technologies Video Processing & Optical Flow
1

Hi there!

I am trying to encode HEVC bitstream with alpha layer using the VideoProcessingFramework.

nvEnc = nvc.PyNvEncoder(…)
nvEnc.Capabilities()
gives me this line
<NV_ENC_CAPS.SUPPORT_ALPHA_LAYER_ENCODING: 48>: 1,

but I can’t find any info how to set NV_ENC_CONFIG_HEVC:: enableAlphaLayerEncoding = 1 from python.

If SUPPORT_ALPHA_LAYER_ENCODING was added to python bindings, it should be usable, isn’t it?

Video Codec SDK API reference

Also

  1. The availability of the feature can be queried using nvEncGetEncodeCaps() and checking for NV_ENC_CAPS_SUPPORT_ALPHA_LAYER_ENCODING. Note that only NV_ENC_BUFFER_FORMAT_NV12, NV_ENC_BUFFER_FORMAT_ARGB and NV_ENC_BUFFER_FORMAT_ABGR input formats are supported with alpha layer encoding.

But for some reason in python buffer_format ARGB and ABGR are not binded from C++ code. Why?

Thank you!

2

Also tried to write HEVC + alpha channel video using native C++ code. But no luck.
I tried to use

Video_Codec_SDK_12.1.14/Samples/AppEncode/AppEncDec

with this main changes:

NV_ENC_BUFFER_FORMAT eFormat = NV_ENC_BUFFER_FORMAT_ARGB;
…
NV_ENC_CONFIG *pConfig = initializeParams.encodeConfig;
pConfig->rcParams.alphaLayerBitrateRatio = 3;
pConfig->encodeCodecConfig.hevcConfig.enableAlphaLayerEncoding = 1;

But it writes video, which ffmpeg reads with a lot of errors:

Last message repeated 1 times

[hevc @ 0x5603d1b5fd80] missing picture in access unit with size 51
[hevc @ 0x5603d1b5fd80] missing picture in access unit with size 49
Last message repeated 1 times
[hevc @ 0x5603d1b5fd80] missing picture in access unit with size 51
[hevc @ 0x5603d1b5fd80] missing picture in access unit with size 49
Last message repeated 1 times
[hevc @ 0x5603d1b5fd80] missing picture in access unit with size 51
[hevc @ 0x5603d1b5fd80] missing picture in access unit with size 49
Last message repeated 1 times
Input #0, mpegts, from ‘out.mkv’:
Duration: 00:00:16.00, start: 0.000000, bitrate: 66 kb/s
Program 1
Metadata:
service_name : Service01
service_provider: FFmpeg
Stream #0:0[0x100]: Video: hevc (Main) (HEVC / 0x43564548), yuv420p(tv), 1080x1080 [SAR 1:1 DAR 1:1], 30 fps, 25 tbr, 90k tbn

And without using “alpha features” ffmpeg reads it successfully:

Input #0, mpegts, from ‘out_wo_alpha.mkv’:
Duration: 00:00:16.00, start: 0.000000, bitrate: 66 kb/s
Program 1
Metadata:
service_name : Service01
service_provider: FFmpeg
Stream #0:0[0x100]: Video: hevc (Main) (HEVC / 0x43564548), yuv420p(tv), 1080x1080 [SAR 1:1 DAR 1:1], 30 fps, 25 tbr, 90k tbn

What exact feature I am missed to turn on? Why video still have pix format yuv420p(tv) even when alpha features enabled?

Sadly I can’t attach full code on this forum, but I can share it if needed.

Any help appreciated.

3

Hi there @jedi.vova,

I am not sure how long alpha layer encoding is supported in NVENC yet, it might very well be that this feature did not make it to the Python bindings. I would suggest to open an issue on the github pages of VPF directly, the contributors are not necessarily reading these forums.

As to the general issue with alpha encoding, you need to be very exact wth regards to buffer formats, padding and memory assingment as well as making sure that you don’t hit any of the situations that do not allow alpha layer support as stated in the section “Alpha encoding is not supported in the following scenarios” of the API reference.

4

Yep, firstly I tried to do it via python bindings, and some features are not accessible with it.
So I tried to make an simple example in C++, setting all parameters as in this guide:

  1. frame format NV_ENC_BUFFER_FORMAT_ARGB
  2. NV_ENC_CONFIG_HEVC:: enableAlphaLayerEncoding = 1
  3. NV_ENC_RC_PARAMS::alphaLayerBitrateRatio = 3

But, for some reason, it writes raw stream with frames format NV12 without alpha layer…