Unpacking HEVC MV structure returned by NVENC ME-only Sample App

Gaming and Visualization Technologies Video Processing & Optical Flow
1

Hello,

I have been playing around with the sample app of Motion Estimation Mode Only in Video Codec SDK. I have been able to unpack H264 MV structure but not HEVC. I am wondering if you have a documentation or utility to unpack the MV from zigzag structure to raster scan.

typedef struct _NV_ENC_HEVC_MV_DATA
{
    NV_ENC_MVECTOR    mv[4];               /**< up to 4 vectors within a CU */
    uint8_t           cuType;              /**< 0 (I), 1(P) */
    uint8_t           cuSize;              /**< 0: 8x8, 1: 16x16, 2: 32x32, 3: 64x64 */
    uint8_t           partitionMode;       /**< The CU partition mode
                                                0 (2Nx2N), 1 (2NxN), 2(Nx2N), 3 (NxN),
                                                4 (2NxnU), 5 (2NxnD), 6(nLx2N), 7 (nRx2N) */
    uint8_t           lastCUInCTB;         /**< Marker to separate CUs in the current CTB from CUs in the next CTB */
} NV_ENC_HEVC_MV_DATA;

The sample output header for HEVC is as follows

ctb, cuType, cuSize, partitionMode, MV[0].x, MV[0].y, MV[1].x, MV[1].y, MV[2].x, MV[2].y, MV[3].x, MV[3].y, lastCUInCTB

I am confused with the first column, it is an ascending list of indices that seem to be cu index but not CTB?

How do we know the CTB size? Does the sdk support all 64×64, 32×32, or 16×16 CTB size?

Thanks for your time!

2

There may be a bug in the sample where the CTBs are not completely written out. Following is my patch in AppEncME.cpp

int m = ((nWidth + 31) / 32) * ((nHeight + 31) / 32);
            fpOut << "ctb_id, cu_id, lastCUInCTB, cuType, cuSize, partitionMode, " <<
                "MV[0].x, MV[0].y, MV[1].x, MV[1].y, MV[2].x, MV[2].y, MV[3].x, MV[3].y" << std::endl;
            NV_ENC_HEVC_MV_DATA *outputMV = (NV_ENC_HEVC_MV_DATA *)mvData.data();
            bool lastCUInCTB = false;
            int cu_cnt=0;
            for (int l = 0; l < m;)
            {
                cu_cnt=0;
                do
                {
                    lastCUInCTB = outputMV->lastCUInCTB ? true : false;
                    fpOut << l << ", " << cu_cnt << ", " << lastCUInCTB << ", " << static_cast<int>(outputMV->cuType) << ", " << static_cast<int>(outputMV->cuSize) << ", " << static_cast<int>(outputMV->partitionMode) << ", " <<
                    outputMV->mv[0].mvx << ", " << outputMV->mv[0].mvy << ", " << outputMV->mv[1].mvx << ", " << outputMV->mv[1].mvy << ", " <<
                    outputMV->mv[2].mvx << ", " << outputMV->mv[2].mvy << ", " << outputMV->mv[3].mvx << ", " << outputMV->mv[3].mvy << std::endl;

                    outputMV += 1;
                    cu_cnt++;
                } while (!lastCUInCTB);
                l++;
            }
2 Likes