How to create OpenCV cv::Mat from NvBuffer in Jetpack 5.1

Hi, what is now the updated way to populate an OpenCV Matrix using Jetpack 5.1. The method described in Using OpenCV to create cv::Mat objects from images received by the Argus yuvJpeg sample program - #4 by moren1 doesn’t seem to be valid for JP5.1, as we would need to now use NvBufSurf. Additionally, the color format flags seem to have changed. I do not see the NvBufferColorFormat_ABGR32 enum anymore.

Basically, I’m looking for JP5.1 version of this snippet

// Copy the image into an OpenCV Mat object. Note that OpenCV Matrix objects store the color
// channels in the order of blue, green, and red (hence the "bgr" naming convention).
auto *iNativeBuffer = Argus::interface_cast<EGLStream::NV::IImageNativeBuffer>(iFrame->getImage());
if (!iNativeBuffer) {
    ORIGINATE_ERROR("IImageNativeBuffer not supported by Image.");
if (m_dmabuf == -1) {
    m_dmabuf = iNativeBuffer->createNvBuffer(
    if (m_dmabuf == -1) {
        ORIGINATE_ERROR("\tFailed to create NvBuffer\n");
if (iNativeBuffer->copyToNvBuffer(m_dmabuf) != Argus::STATUS_OK) {
    ORIGINATE_ERROR("Failed to copy frame to NvBuffer.");
void *pdata = nullptr;
NvBufferParams params;
NvBufferGetParams(m_dmabuf, &params);
NvBufferMemMap(m_dmabuf, plane, NvBufferMem_Read, &pdata);
NvBufferMemSyncForCpu(m_dmabuf, plane, &pdata);
cv::Mat imgbuf = cv::Mat(streamResolution.height(), streamResolution.width(), CV_8UC4, pdata, params.pitch[0]);
cv::cvtColor(imgbuf, bgr, cv::COLOR_RGBA2BGR);

Can you please provide an example that would work with the 09_argus_camera_jpeg sample application? Thank you.

Please refer to this patch:
NVBuffer (FD) to opencv Mat - #6 by DaneLLL

And replace NvBuffer APIs with NvBufSurfaceMap()/NvBufSurfaceUnMap()

Hi @DaneLLL , I have and that is not the problem I’m currently facing. The issue is that the cv::Mat generated looks to have the wrong color channels. Could you please tell me what to replace the NvBufferColorFormat_ABGR32 and the color conversion cv::COLOR_RGBA2BGR` flags with?

I have tried a number of combinations of the following flags, and none of them seem to work. Seems like the order of the channels is no longer consistent.

NvBuffer flags tried


OpenCV color conversions tried


Which combinations of these would yield the same results as in the snippet in question? Thanks.

NvBufferColorFormat_ABGR32 is equivalent to NVBUF_COLOR_FORMAT_RGBA. Please convert to the format and map the buffer to cv::Mat

Thanks @DaneLLL . The issue was that the alpha and the red channels were swapped. Thanks for your help.

1 Like


In case you still need it, check this patch:

diff --git a/multimedia_api/ll_samples/samples/13_argus_multi_camera/main.cpp b/multimedia_api/ll_samples/samples/13_argus_multi_camera/main.cpp
index 1bd8ed9..2e83025 100644
--- a/multimedia_api/ll_samples/samples/13_argus_multi_camera/main.cpp
+++ b/multimedia_api/ll_samples/samples/13_argus_multi_camera/main.cpp
@@ -39,6 +39,8 @@
 #include <stdio.h>
 #include <stdlib.h>
+#include <opencv2/opencv.hpp>
 using namespace Argus;
 using namespace EGLStream;
@@ -267,7 +269,8 @@ bool ConsumerThread::threadInitialize()
     /* Allocate composited buffer */
     input_params.width = STREAM_SIZE.width();
     input_params.height = STREAM_SIZE.height();
-    input_params.colorFormat = NVBUF_COLOR_FORMAT_NV12;
+    input_params.colorFormat = NVBUF_COLOR_FORMAT_RGBA;
     input_params.layout = NVBUF_LAYOUT_PITCH;
     input_params.memType = NVBUF_MEM_SURFACE_ARRAY;
     input_params.memtag = NvBufSurfaceTag_VIDEO_CONVERT;
@@ -356,8 +359,8 @@ bool ConsumerThread::threadExecute()
                 batch_surf[i] = NULL;
                 m_dmabufs[i] = iNativeBuffer->createNvBuffer(iEglOutputStreams[i]->getResolution(),
-                                                          NVBUF_COLOR_FORMAT_YUV420,
-                                                          NVBUF_LAYOUT_BLOCK_LINEAR);
+                                                          NVBUF_COLOR_FORMAT_RGBA,
+                                                          NVBUF_LAYOUT_PITCH);
                 if (!m_dmabufs[i])
                     CONSUMER_PRINT("\tFailed to create NvBuffer\n");
                 if (-1 == NvBufSurfaceFromFd(m_dmabufs[i], (void**)(&batch_surf[i])))
@@ -374,7 +377,19 @@ bool ConsumerThread::threadExecute()
             /* Composite multiple input to one frame */
             NvBufSurfTransformMultiInputBufCompositeBlend(batch_surf, pdstSurf, &m_compositeParam);
-            g_renderer->render(m_compositedFrame);
+            NvBufSurfaceMap(pdstSurf, -1, 0, NVBUF_MAP_READ);
+            NvBufSurfaceSyncForCpu(pdstSurf, -1, 0);
+            cv::Mat imgbuf = cv::Mat(STREAM_SIZE.height(),
+                                     STREAM_SIZE.width(),
+                                     CV_8UC4, pdstSurf->surfaceList->mappedAddr.addr[0]);
+            cv::Mat display_img;
+            cvtColor(imgbuf, display_img, cv::COLOR_RGBA2BGR);
+            NvBufSurfaceUnMap(pdstSurf, -1, 0);
+            cv::imshow("img", display_img);
+            cv::waitKey(1);

You also need to use opencv4 instead of opencv in Makefile:

+CPPFLAGS+=`pkg-config --cflags opencv4`
+LDFLAGS+=`pkg-config --libs opencv4`

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.