# NvBuffer to OpenCV GpuMat. Does memory is directly accessible to the device? **URL:** \ **Category:** Jetson TX2\ **Tags:** opencv\ **Created:** [September 7, 2022, 8:30am UTC](https://forums.developer.nvidia.com/t/nvbuffer-to-opencv-gpumat-does-memory-is-directly-accessible-to-the-device/227245 "2022-09-07T08:30:48Z")\ **Posts on this page:** 12\ **Page:** 1
**Author:** ![MrSunshine](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/mrsunshine/32/187662_2.png) [@MrSunshine](https://forums.developer.nvidia.com/u/MrSunshine)\ **Post date:** [September 7, 2022, 8:30am UTC](https://forums.developer.nvidia.com/t/nvbuffer-to-opencv-gpumat-does-memory-is-directly-accessible-to-the-device/227245/1 "2022-09-07T08:30:48Z")
Hi, My topic is heavly based on this [post](https://forums.developer.nvidia.com/t/is-it-possible-to-directly-map-the-nvbuffer-to-a-opencv-gpumat/215332) I’m currently trying to directly map a NvBuffer to a Opencv::GpuMat with NvBufferMemMap and NvBufferMemSyncForDevice on a Jetson TX2i. I took the main from the samples available in **/usr/src/jetson\_multimedia\_api/** and took precisely the number 07. (Video convert) In the sample the NvBuffer is created with NvBufferCreateEx with the thread\_context. I inlaided a little snippet of code in the function **read\_video\_frame** (OpenCV Version 4.5.4). The following is working : ```auto void *ptr; NvBufferMemMap(src_dma_fd, 0, NvBufferMem_Read_Write, &ptr); NvBufferMemSyncForCpu(src_dma_fd, 0, &ptr); cv::Mat plane_y; static cv::Mat y_only(src_param.height[0], src_param.width[0], CV_8U); plane_y = {src_param.height[0], // 1080 src_param.width[0], // 1920 CV_8U, ptr, // y src_param.pitch[0]}; // 2048 static cv::cuda::GpuMat cuda_plane_y(src_param.height[0], src_param.width[0], CV_8U); cuda_plane_y.upload(plane_y); cuda_plane_y.download(y_only); cv::imshow("img", y_only); cv::waitKey(0); ``` The following is not working : ```auto void *ptr; NvBufferMemMap(src_dma_fd, 0, NvBufferMem_Read_Write, &ptr); NvBufferMemSyncForDevice(src_dma_fd, 0, &ptr); cv::cuda::GpuMat plane_y; static cv::Mat y_only(src_param.height[0], src_param.width[0], CV_8U); plane_y = {src_param.height[0], // 1080 src_param.width[0], // 1920 CV_8U, ptr, // y src_param.pitch[0]}; // 2048 plane_y.download(y_only); cv::imshow("img", y_only); cv::waitKey(0); ``` ```auto error: (-217:Gpu API call) invalid argument in function 'download' ``` **Changing NvBufferMemSyncForCpu does not work for me.** I’m only trying to directly map to a GpuMat to avoid memory transfer time. _What i already did to debug :_ Changing my GpuMat constructor to ```auto cv::cuda::GpuMat plane_y = cv::cuda::GpuMat(src_param.height[0], // 1080 src_param.width[0], // 1920 CV_8U, ptr); // 2048 ``` Doesn’t work either. Adding or deleting the pitch also do not change my problem. And in a final effort, changing to a non-static y\_only cv::Mat did not work. My question is, does context have to specify something when creating a NvBuffer that the memory is shared between Device and CPU ? Or i’m missing something important in my code to make it successful ? Best regards. ~S. ---
**Author:** ![DaneLLL](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/danelll/32/14043_2.png) [@DaneLLL](https://forums.developer.nvidia.com/u/DaneLLL)\ **Post date:** [September 7, 2022, 10:06am UTC](https://forums.developer.nvidia.com/t/nvbuffer-to-opencv-gpumat-does-memory-is-directly-accessible-to-the-device/227245/3 "2022-09-07T10:06:01Z")
Hi, Only NvBuffer in RGBA format can be mapped to _cv::Mat_ or _cv::gpuMat_. Please refer to the patches: [LibArgus EGLStream to nvivafilter - #14 by DaneLLL](https://forums.developer.nvidia.com/t/libargus-eglstream-to-nvivafilter/70627/14) [NVBuffer (FD) to opencv Mat - #6 by DaneLLL](https://forums.developer.nvidia.com/t/nvbuffer-fd-to-opencv-mat/83012/6) ---
**Author:** ![MrSunshine](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/mrsunshine/32/187662_2.png) [@MrSunshine](https://forums.developer.nvidia.com/u/MrSunshine)\ **Post date:** [September 7, 2022, 10:38am UTC](https://forums.developer.nvidia.com/t/nvbuffer-to-opencv-gpumat-does-memory-is-directly-accessible-to-the-device/227245/4 "2022-09-07T10:38:50Z")
Hi. Thank you for your answer. I will verify if i can map a RGBA NvBuffer to a GpuMat. But for the cv::Mat part it’s working with only one plane (Y). So actually NvBuffer can map to a cv::Mat with YUV420 format. Why does GpuMat cannot do the same operation as cv::Mat ? ~S. ---
**Author:** ![DaneLLL](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/danelll/32/14043_2.png) [@DaneLLL](https://forums.developer.nvidia.com/u/DaneLLL)\ **Post date:** [September 7, 2022, 11:13am UTC](https://forums.developer.nvidia.com/t/nvbuffer-to-opencv-gpumat-does-memory-is-directly-accessible-to-the-device/227245/5 "2022-09-07T11:13:06Z")
Hi, This would need other users to share experience. For OpenCV we are also learning from public posts. It looks like RGBA is a suitable format for hooking NvBuffer and OpenCV. ---
**Author:** ![Honey\_Patouceul](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/honey_patouceul/32/389777_2.png) [@Honey\_Patouceul](https://forums.developer.nvidia.com/u/Honey_Patouceul)\ **Post date:** [September 7, 2022, 1:56pm UTC](https://forums.developer.nvidia.com/t/nvbuffer-to-opencv-gpumat-does-memory-is-directly-accessible-to-the-device/227245/6 "2022-09-07T13:56:38Z")
For NV12 format to GpuMat, you may have a look to this post: > [@Real-time CLAHE processing of video, framerate issue. Gstreamer + nvivafilter + OpenCV](https://forums.developer.nvidia.com/t/real-time-clahe-processing-of-video-framerate-issue-gstreamer-nvivafilter-opencv/155614/5): > > I think I got it… Seems there is 256 bytes stride. This should be better (so far, not tested so much): static std::vector\ uv(2); static void cv\_process\_NV12(void\*\* pPitch, int32\_t width, int32\_t height) { //printf ("cv\_process\_NV12 %d x %d\n", width, height); const int stride = 256; int num\_strides = ((int)width)/stride; int use\_width = num\_strides\*stride; if (use\_width \< (int)width) use\_width += stride; int use\_height = height; cv::cuda::… ---
**Author:** ![MrSunshine](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/mrsunshine/32/187662_2.png) [@MrSunshine](https://forums.developer.nvidia.com/u/MrSunshine)\ **Post date:** [September 9, 2022, 11:20am UTC](https://forums.developer.nvidia.com/t/nvbuffer-to-opencv-gpumat-does-memory-is-directly-accessible-to-the-device/227245/7 "2022-09-09T11:20:41Z")
Hi, Following up my recent post, i’ve successfully mapped a ABGR32 NvBuffer to a cv::Mat. But i still struggle to map my NvBuffer to a cv::cuda::GpuMat. It looks like my HW Buffer is not allocated on the device, i’m kinda flabbergasted [ysyyork](https://forums.developer.nvidia.com/t/is-it-possible-to-directly-map-the-nvbuffer-to-a-opencv-gpumat/215332) managed to download it to a Cpu mat. My question are still pending but i kinda wanna shift my focus on the jetson multimedia api context, i’m absolutly neophyte to the subject of context creation. To make a reminder, i’m using the main inside the sample of the jetson multimedia api. ( **07\_video\_convert** ) - Do i have something to do with the context used in this sample ? - Can **OpenCV 4.5.4** cause any trouble ? Maybe some compilation flags i’ve been missing with this library ? Best regards. ~S. ---
**Author:** ![MrSunshine](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/mrsunshine/32/187662_2.png) [@MrSunshine](https://forums.developer.nvidia.com/u/MrSunshine)\ **Post date:** [September 9, 2022, 12:12pm UTC](https://forums.developer.nvidia.com/t/nvbuffer-to-opencv-gpumat-does-memory-is-directly-accessible-to-the-device/227245/8 "2022-09-09T12:12:54Z")
Hi, Thank you for your answer. I’m not trying or using anything related to the NV12 format. It looks like your solution is coming from a EGLImage produced by NvEGLImageFromFd. I cannot use this specific library. Still thank you for your time. ~S. ---
**Author:** ![DaneLLL](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/danelll/32/14043_2.png) [@DaneLLL](https://forums.developer.nvidia.com/u/DaneLLL)\ **Post date:** [September 13, 2022, 2:35am UTC](https://forums.developer.nvidia.com/t/nvbuffer-to-opencv-gpumat-does-memory-is-directly-accessible-to-the-device/227245/9 "2022-09-13T02:35:46Z")
Hi, You would need to check why _NvEGLImageFromFd()_ does not work properly in your environment. For mapping NvBuffer to cv::cuda::GpuMat, it is a must-call function. ---
**Author:** ![MrSunshine](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/mrsunshine/32/187662_2.png) [@MrSunshine](https://forums.developer.nvidia.com/u/MrSunshine)\ **Post date:** [September 14, 2022, 6:29am UTC](https://forums.developer.nvidia.com/t/nvbuffer-to-opencv-gpumat-does-memory-is-directly-accessible-to-the-device/227245/10 "2022-09-14T06:29:41Z")
Hi, Thank you for your answer. I’m currently working in a monitorless environnement. If i can recall correctly _NvEGLImageFromFd()_ needs to be used with an egl\_display if set to NULL the default egl\_display is used. I also don’t have mesa-utils in my system. Do you know how can i work around with no monitor wired ? I’ve been viewing some post about creating a virtual monitor but with no pratical example on how to do it. Best regards. ~S. ---
**Author:** ![DaneLLL](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/danelll/32/14043_2.png) [@DaneLLL](https://forums.developer.nvidia.com/u/DaneLLL)\ **Post date:** [September 21, 2022, 9:41am UTC](https://forums.developer.nvidia.com/t/nvbuffer-to-opencv-gpumat-does-memory-is-directly-accessible-to-the-device/227245/11 "2022-09-21T09:41:40Z")
Hi, You may try to run this sample: [Opencv gpu mat into GStreamer without downloading to cpu - #15 by DaneLLL](https://forums.developer.nvidia.com/t/186703/15) If it can be run successfully, it should be fine to call the function in monitorless environment: ```auto egl_display = eglGetDisplay(EGL_DEFAULT_DISPLAY); eglInitialize(egl_display, NULL, NULL); eglTerminate(egl_display); ``` ---
**Author:** ![MrSunshine](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/mrsunshine/32/187662_2.png) [@MrSunshine](https://forums.developer.nvidia.com/u/MrSunshine)\ **Post date:** [September 21, 2022, 11:29am UTC](https://forums.developer.nvidia.com/t/nvbuffer-to-opencv-gpumat-does-memory-is-directly-accessible-to-the-device/227245/12 "2022-09-21T11:29:01Z")
Hi, Thank you for your answer. I’m sorry to disappoint again, but here’s what i’m returning with commands from EGL library. ```auto /* Get default EGL display */ EGLDisplay egl_display; egl_display = eglGetDisplay(EGL_DEFAULT_DISPLAY); if (egl_display == EGL_NO_DISPLAY) printf("Failed to get EGL display connection\n"); /* Init EGL display connection */ if (!eglInitialize(egl_display, NULL, NULL)) printf("Failed to initialize EGL display connection\n"); NvEglRenderer* g_renderer = NvEglRenderer::createEglRenderer("renderer0", src_param.width[0], src_param.height[0], 0, 0); if (g_renderer) printf("Failed to create EGL renderer"); EGLImageKHR egl_image = NULL; egl_image = NvEGLImageFromFd(egl_display,src_dma_fd); NvDestroyEGLImage(egl_display,egl_image); eglTerminate(egl_display); ``` > Failed to get EGL display connection > Failed to initialize EGL display connection > [INFO] (~/Desktop/yuv422p/NvEglRenderer.cpp:110) Setting Screen width 1920 height 1080 > [ERROR] (~/Desktop/yuv422p/NvEglRenderer.cpp:198) Unable to get egl display > [ERROR] (~/Desktop/yuv422p/NvEglRenderer.cpp:154) Got ERROR closing display > NvEGLImageFromFd: No EGLDisplay to create EGLImage > NvDestroyEGLImage: No EGLDisplay to destroy EGLImage It look likes i’m also having trouble with OpenGL and drivers associated with it. > glxinfo > name of display: localhost: > X Error of failed request: BadValue (integer parameter out of range for operation) > Major opcode of failed request: 152 (GLX) > Minor opcode of failed request: 24 (X\_GLXCreateNewContext) > Value in failed request: 0x0 > Serial number of failed request: 41 > Current serial number in output stream: 42 > [@DaneLLL](#): > > Hi, > You may try to run this sample: > [Opencv gpu mat into GStreamer without downloading to cpu - #15 by DaneLLL](https://forums.developer.nvidia.com/t/186703/15) > > If it can be run successfully, it should be fine to call the function in monitorless environment: > > ```auto > egl_display = eglGetDisplay(EGL_DEFAULT_DISPLAY); > eglInitialize(egl_display, NULL, NULL); > eglTerminate(egl_display); > > ``` ---
**Author:** ![Honey\_Patouceul](https://sea2.discourse-cdn.com/nvidia/user_avatar/forums.developer.nvidia.com/honey_patouceul/32/389777_2.png) [@Honey\_Patouceul](https://forums.developer.nvidia.com/u/Honey_Patouceul)\ **Post date:** [October 2, 2022, 4:54pm UTC](https://forums.developer.nvidia.com/t/nvbuffer-to-opencv-gpumat-does-memory-is-directly-accessible-to-the-device/227245/13 "2022-10-02T16:54:33Z")
Maybe for headless you could try VirtualGL. I don’t have much experience with it, but searching this forum for that you may find some information.