Recover frame data in NvMOT_Process from NvMOTFrame and NvBufSurfaceParams

Accelerated Computing Intelligent Video Analytics DeepStream SDK
1

Hi,
I am writing an implementation for nvtracker.h and in NvMOT_Process, I need to recover a frame’s pixels and make an OpenCV Mat out of it. This thread discusses the same issue but I really can’t replicate the solution which isn’t detailed enough.
I understand I can access the buffer through mapped CPU memory, but the NvBufSurfaceSyncForCpu function takes an NvBufSurface rather than an NvBufSurfaceParams object.

How do I go about doing this?

• Hardware Platform (Jetson / GPU) GTX 1070
• DeepStream Version 5.0
• JetPack Version (valid for Jetson only)
• TensorRT Version 7.0.0
• NVIDIA GPU Driver Version (valid for GPU only) 440.82

2

Here is what I was able to get so far

NvBufSurfaceParams *buffList = pParams->frameList[0].bufferList[0];                                                                    
NvBufSurface *pBufSurface;                                                                                                              
int res1 = NvBufSurfaceFromFd(int(buffList->bufferDesc), (void **)&pBufSurface);  // res1 = 0, SUCCESS          
int res2 = NvBufSurfaceSyncForCpu(pBufSurface, -1, -1); // res2 = -1, FAIL
4

Looks like NvBufSurfaceSyncForCpu didn’t work because I used NVBUF_MEM_DEFAULT.
Using NVBUF_MEM_CUDA_PINNED, I was able to get the frame with this snippet

NvBufSurfaceParams *buffList = pParams->frameList[0].bufferList[0];
cv::Mat bgrFrame(buffList->height, buffList->width, CV_8UC3, buffList->dataPtr, buffList->pitch);

This line of text from the developer guide is what solved it for me

Note that support is currently limited to the following types:

dGPU: NVBUF_MEM_CUDA_PINNED
NVBUF_MEM_CUDA_UNIFIED

Jetson: NVBUF_MEM_SURFACE_ARRAY

5

Great work.
BTW, is your tracker algorithm based on CPU or GPU, I would suggest to use Unified memory type if it’s based on GPU.

6

Thank you for the hint. It is GPU based but with some preprocessing done on the CPU. Looking to move that on the GPU in not too long, so that will then be helpful!