NvBufSurface and OpenCV

• Hardware Platform Jetson
• DeepStream Version 6
• JetPack Version 4.6
• TensorRT Version 8
• Issue Type questions

I am following the instructions from the documentation to access the frames from the NvBufSurface’s surfaceList. The documentation suggests using dataPtr, but in nvbfsurface.h we can find this comment for the field:

  /** Holds a pointer to allocated memory. Not valid for
   \ref NVBUF_MEM_SURFACE_ARRAY or \ref NVBUF_MEM_HANDLE. */
  void * dataPtr;

SInce NVBUF_MEM_SURFACE_ARRAY is the default for Tegra, how should I access the memory on a Jetson?

Then, the second suggested option for creating a cv::cuda::GpuMat uses the pitch. In that case, the documentation tells to use cv::cuda::swapChannels with a weird destination order. What are the original and the destination color formats here, and why do we have to do this?

cv::cuda::GpuMat gpuMat;
const int aDstOrder[] = {2,0,1,3};
unsigned int index = 0;   // Index of the buffer in the batch.
unsigned int width, height; // set width and height of buffer
NvBufSurface *input_buf; // Pointer to input NvBufSurface

gpuMat = cv::cuda::GpuMat(height, width, CV_8UC4,
(void *)input_buf->surfaceList[index].dataPtr);

# OR

gpuMat = cv::cuda::GpuMat(height, width, CV_8UC4,
     (void *) input_buf->surfaceList[index].dataPtr,
      input_buf->surfaceList[index].pitch);
cv::cuda::swapChannels(gpuMat, aDstOrder);

Please provide complete information as applicable to your setup.
• Hardware Platform (Jetson / GPU)
• DeepStream Version
• JetPack Version (valid for Jetson only)
• TensorRT Version
• NVIDIA GPU Driver Version (valid for GPU only)
• Issue Type( questions, new requirements, bugs)
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)
• Requirement details( This is for new requirement. Including the module name-for which plugin or for which sample application, the function description)
• The pipeline being used

I have just edited the post, but I want to emphasize that this is a question about the docs.

  1. please refer to Deepstream sample code snippet - #3 by bcao, and How to convert NvBufSurface to cv::Mat

Alright, but what about my second question?

Furthermore, I saw that the Gst-nvinfer maps EGL frames for NVBUF_MEM_SURFACE_ARRAY in gstnvinfer_allocator.cpp. When do I need to use NvBufSurfaceMapEglImage()?

I have a same question.

please find NvBufSurfaceMapEglImage in this link: NVIDIA DeepStream SDK API Reference: NvBufSurface Types and Functions

Thank you for pointing me to the docs ( which are just the header files ). However, the behavior is not clearly described.

From my own research I figured out that surface->surfaceList[i].dataPtr can be used on dGPU, while on Tegra you have to chain NvBufSurfaceMapEglImage and cuGraphicsResourceGetMappedEglFrame like described here.

1 Like

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.