3D Video recording with NVEnc HEVC

[nunduniel]

Hi,
I need to record some original interleaved (line-by-line) 3D video with HEVC in 4Kp30.
It seems that unlike H.264 there is no really any supported interleaved formats, and in NvEncodeAPI.h the struct NV_ENC_STEREO_PACKING_MODE seems only to be used for NV_ENC_CONFIG_H264, not for NV_ENC_CONFIG_HEVC. Is there anything similar in HEVC?

As wordaround (for now) I just rearrange the interleaved video to be top-bottom, so the 4:2:0 format doesn’t mix vertically adjacent left and right lines of the image during subsampling.

The request is to generate additionally a 2D recording from the recorded top-bottom 3D HEVC video. The idea is now to extract from the already encoded HEVC bitstream somehow the upper region, which could then be reused for the 2D recording, without having to re-encode the video (to avoid quality loss and extra encoding utilization).
From what I understand of the HEVC bitstream format, on the top-level the picture will be broken up into slices and/or tiles, each of them consist of the coding tree units which are like 32x32 macroblocks.

1. I read on a forum post, that HEVC tiles are not supported by NVEnc hardware, is that correct?
2. I heard the CTU is fixed to 32x32. I am using an Ampere GPU (A4000) and am not sure if that limitation is CTU max size is 32x32 (instead 64x64 like in HEVC spec) or if it means as well that any CTU size, so e.g. minSize is as well 32x32. Can I mix 32x32 CTUs with 8x8 CTUs?
3. I’m thinking to use slices, for the top and bottom half of the image. I would use sliceMode=3 (numSlices in Picture) and specify 2 slices (or use slice mode 2 and specify the # of CTU rows). From what I understand those 2 slices for top and bottom half are then totally independent (no prediction or motion vector going across top half to bottom half) so that they can be totally independently processed and used. Is that supported by the Nvidia NVEnv HW for HEVC on Ampere?
4. How to handle heights not divisible by 32? The 3D video is 3840x2160, so top and bottom are each 1080px height. That is not divisible by 32px (if that really is only fixed CTU supported), so I would need to pad each half to 1088px. To not get a space in the middle, the top half would probably want to pad 8px at the top, and the bottom half 8px at the bottom. How can we signal to the player to “ignore” those padding lines? Are there any metadata or settings? For 4K with a single full-picture slice I see that it is encoded as 3840x2176, so padded by 16px, but I assume that all the padding is on the bottom there, and not 8px on the top and 8px on the bottom.

Any help would greatly be appreciated
Thanks

[nunduniel]

Any feedback?
Could you at least say if all CTU are always 32x32 sized by Nvidia HW (or can it be changed)?
And is there a way to set the “top crop” conformance window parameter for the 8px shift from the top, that the NVDec decoder will honor (or other video decoders)?

Greetings

[Abhijit_Patait]

1. Correct
2. CTU size is fixed at 32x32 for HEVC in all GPUs up to and including Ampere GPUs
3. Use appropriate slice mode, but to make the slices completely independent, set NV_ENC_CONFIG_HEVC::enableConstrainedEncoding = 1

[nunduniel]

Hi,
Thanks for the reply.
I can add custom padding at top and bottom (8px at top and 8px at bottom to make it divisible), but the question was revolving around how to mark it for decoder / playback, that they don’t show that black padded extra region.
On the HEVC standard I think there is a “conformance window” parameter for cropping top, left, bottom & right. I think its in the SPS (sequence parameter set).
Can I pad the top & bottom of video with black, and per conformance window parameter set top offset and bottom offset to 8px, so those black paddings won’t be shown?
Does NVDec and other decoders adhere to those parameters? I have seen that most 3840x2160 video is encoded at 2176, and had 16px padding at bottom.
But do they regard that top offset of cropping too?

Thanks for the help.
Greetings

[Abhijit_Patait]

NVDEC supports cropping. HEVC SPS has fields to indicate the display rectangle. You should set this display rectangle while creating decoder object using cuvidCreateDecoder() API.

For more details, see this link.

[nunduniel]

Thanks much, that helps. So I will set the SPS fields for conformance window (top and bottom).

Can we assume that all big video players out there (Windows media player, VLC, etc…) will regard those SPS conformance window settings (especially the top crop) ? So that the video will be played back correctly anywhere for customers?

Thanks

[Abhijit_Patait]

We cannot vouch for the behaviour of the video players in the market.