Retrieve cv::Mat in NvMOT_Process from NvMOTFrame and NvBufSurfaceParams

[superware2]

Hello,

• Hardware Platform: Jetson Xavier NX
• JetPack Version: 5.0.2
• DeepStream Version: 6.1.1

I’m trying to retrieve an OpenCV cv::Mat from a NvMOTFrame and NvBufSurfaceParams without success.

The custom low-level tracker library query is:

NvMOTStatus NvMOT_Query(uint16_t customConfigFilePathSize, char *pCustomConfigFilePath, NvMOTQuery *pQuery)
{
    pQuery->computeConfig = NVMOTCOMP_CPU;
    pQuery->numTransforms = 1;
    pQuery->colorFormats[0] = NVBUF_COLOR_FORMAT_RGBA;
    pQuery->memType = NVBUF_MEM_SURFACE_ARRAY;
    pQuery->batchMode = NvMOTBatchMode_NonBatch;
    pQuery->supportPastFrame = false;

    return NvMOTStatus_OK;
}

And frame processing:

#include <iostream>
#include <nvbufsurface.h>
#include <opencv2/imgcodecs.hpp>
#include "NvMOTContext.hpp"

NvMOTStatus NvMOTContext::processFrame(const NvMOTProcessParams *params, NvMOTTrackedObjBatch *pTrackedObjectsBatch)
{
    if (!params || params->numFrames <= 0) {
        return NvMOTStatus_OK;
    }

    NvMOTTrackedObjList *trackedObjList = &pTrackedObjectsBatch->list[0];
    NvMOTFrame *frame = &params->frameList[0];
    NvBufSurfaceParams *bufferParams = frame->bufferList[0];
    NvBufSurface *bufSurface;

    if (NvBufSurfaceFromFd(int(bufferParams->bufferDesc), (void **)&bufSurface) != 0)
        std::cout << "NvBufSurfaceFromFd failed" << std::endl;

    if (NvBufSurfaceMap(bufSurface, -1, -1, NVBUF_MAP_READ) != 0)
        std::cout << "NvBufSurfaceMap failed" << std::endl;

    if (NvBufSurfaceSyncForCpu(bufSurface, -1, -1) != 0)
        std::cout << "NvBufSurfaceSyncForCpu failed" << std::endl;

    cv::Mat bgraFrame(bufferParams->height, bufferParams->width, CV_8UC4, bufferParams->dataPtr, bufferParams->pitch);

    cv::imwrite("out.jpeg", bgraFrame);

    ...
}

After playing the GStreamer pipeline and supplying nvtracker with video/x-raw(memory:NVMM),format=RGBA, the saved out.jpeg is just noise.

What am I doing wrong? Any help will be greatly appreciated.

Thank you!

[Fiona.Chen]

In nvtracker NvMOTProcessParams, the data is in NvBufSurfaceParams “dataPtr” and “mappedAddr.addr”. It is no use to use “NvBufSurfaceFromFd()”.

[superware2]

Hello @Fiona.Chen,

Thank you for your reply.

cv::Mat bgraFrame(bufferParams->height, bufferParams->width, CV_8UC4, bufferParams->mappedAddr.addr[0], bufferParams->pitch);

Works.

Thank you.