HEVC with alpha layer

Professional Graphics and Rendering Video Technologies Video Codec and Optical Flow SDK
#1

I’m able to create an HEVC bitstream with alpha layer using the Video Codec SDK. What I cannot figure out is how to mux this bitstream into an MP4 file or HLS stream. FFmpeg, mp4box, and mp4mux all fail in different ways due to the alpha layer. This is fairly new territory and as far as I can tell only Apple encoders have end-to-end encode support for HEVC with alpha to MP4.

My question is this: How do I mux this HEVC bitstream - with alpha layer - into an MP4 that Quicktime can play?

Related question: How did the NVIDIA devs test their encoder?

Video Codec SDK reference: NVENC Video Encoder API Programming Guide :: NVIDIA Video Codec SDK Documentation
Feature request with FFmpeg: #9088 (HEVC bitstream with alpha layer to mp4) – FFmpeg
Code to create the bitstream: GitHub - johnhe4/nvenc_h265_transparency: Encodes a sequence of YUV frames into HEVC (h.265) with transparency, using Nvidia's nvEncodeApi

1 Like
#2

There is now Apple encoder available.

mp4box, and mp4mux all fail

How? Can you paste their problem here.

this HEVC bitstream - with alpha layer - into an MP4

This is called Annex B. Frankly speacking you will have to write you own code, IMHO, according to Apple spec. See how it was done above.

#3

Thanks for the update, it prompted me to update the FFmpeg ticket with more details.

I don’t think this will help because I need muxing-only support in FFmpeg, and I will be encoding on Linux, but it is good to see activity on this.

I don’t remember the exact issues with mp4box and mp4mux but in both cases they didn’t correctly handle the presence of the alpha layer when muxing to an MP4. You can see the commands I used at the end of the github readme which is linked above in the original ticket.