NvMedia video surface and NVM_SURF_ATTR_CPU_ACCESS

Autonomous Vehicles DRIVE PX 2 General
1

Hello I’m using h264 encoding, at the moment it only works with video surfaces created with NVM_SURF_ATTR_CPU_ACCESS_UNMAPPED flag (please see the code below), NvMediaVideoSurfacePutBits takes around 14ms (which is slow!). When I’m trying to use NVM_SURF_ATTR_CPU_ACCESS_UNCACHED or NVM_SURF_ATTR_CPU_ACCESS_CACHED the encoded image is completely corrupted, NvMediaVideoSurfacePutBits takes around 8ms though. Could you please point me the way how to solve this?
SDK 5.0.5.0b

NVM_SURF_FMT_DEFINE_ATTR(surfaceFormatAttributes);
NVM_SURF_FMT_SET_ATTR_YUV(
    surfaceFormatAttributes, YUV, 420, SEMI_PLANAR, UINT, 8, BL);
NvMediaSurfaceType surfaceType = NvMediaSurfaceFormatGetType(
    surfaceFormatAttributes, NVM_SURF_FMT_ATTR_MAX);
if (surfaceType == NvMediaSurfaceType_Unsupported)
    ERROR("NvMediaSurfaceFormatGetType() failed");

NvMediaSurfAllocAttr surfAllocAttrs[] = {
    { NVM_SURF_ATTR_WIDTH, IMAGE_WIDTH },
    { NVM_SURF_ATTR_HEIGHT, IMAGE_HEIGHT },
    { NVM_SURF_ATTR_CPU_ACCESS, NVM_SURF_ATTR_CPU_ACCESS_UNMAPPED }
};
const int numSurfAllocAttrs =
    sizeof(surfAllocAttrs) / sizeof(surfAllocAttrs[0]);
DEBUG("Creating video surface ...");
mVideoSurface = std::shared_ptr<NvMediaVideoSurface>(
    NvMediaVideoSurfaceCreateNew(
        mDevice.get(), surfaceType, surfAllocAttrs, numSurfAllocAttrs, 0
    ),
    [] (NvMediaVideoSurface * surface) {
        if (surface != nullptr)
        {
            DEBUG("Destroying video surface ...");
            NvMediaVideoSurfaceDestroy(surface);
        }
    });

…then later…

NvMediaVideoSurfaceMap surfaceMap;
NvMediaStatus status = NvMediaVideoSurfaceLock(
    mVideoSurface.get(), &surfaceMap);
if (status != NVMEDIA_STATUS_OK)
    ERROR("NvMediaVideoSurfaceLock() failed");

void * buffers[3];
uint32_t pitches[3];
buffers[0] = imageYUV.data;
pitches[0] = IMAGE_WIDTH;
buffers[1] = imageYUV.data + IMAGE_WIDTH * IMAGE_HEIGHT;
pitches[1] = IMAGE_WIDTH / 2u;
buffers[2] = static_cast<uint8_t*>(buffers[1]) +
    IMAGE_WIDTH * IMAGE_HEIGHT / 4u;
pitches[2] = IMAGE_WIDTH / 2u;
status = NvMediaVideoSurfacePutBits(
    mVideoSurface.get(), nullptr, buffers, pitches);
if (status != NVMEDIA_STATUS_OK)
    ERROR("NvMediaVideoSurfacePutBits() failed");

NvMediaVideoSurfaceUnlock(mVideoSurface.get());
2

Dear xynkin,

NvMediaVideoSurfaceMap has CPU accessible memory pointers of the surface inside depends on the format.

  • Video Domain API :
    NvMediaVideoDecoderRenderEx()
  • nvmvid_play in drive-t186ref-linux/samples/nvmedia/vid_play/
  • H.264 decode in Video Domain

NvMediaVideoMixerRenderSurface()

  • Video Mixer can convert YUV to RGB

NvMediaVideoSurfaceMap surfaceMap;
status = NvMediaVideoSurfaceLock(targetBuffer->videoSurface, &surfaceMap);
NvMediaVideoSurfaceUnlock(targetBuffer->videoSurface);

The section of “NvMedia → Understanding NvMedia → Using the Software Video Decoders” in Development Guide can give some hint to access the memory.

NvMedia provides two functions to support software video decoders.
•The decoder obtains a mapped YUV video surface by calling the NvMediaVideoSurfaceLock function. During the call, NvMedia waits while the surface is being used by the internal engines. Consequently, this is a blocking call. Once the function returns, the surface is used by the software decoder. The returned parameters are the mapped memory YUV pointers and the associated pitches. It also returns the width and height in terms of luma pixels.
•When the decoder finishes filling up the surfaces, it calls the NvMediaVideoSurfaceUnlock function. This tells NvMedia that the surface can be used for the internal engines. On certain Tegra platforms, the mapping returns a NULL pointer indicating that the surface is not CPU accessible.