Create a QByteArray with NV12 data

[aniss.louahadj]

Hello,

I’ve been tasked to create a QByteArray from data acquired by a camear using libargus.
Previously, the data acquired was in RGBA. Then opencv was used to convert it into a cv::Mat in RGB format like so :

// Get NvBuffer from native buffer
m_dmabuf_fd = m_iImageNativeBuffer->createNvBuffer(m_iEGLOutputStream->getResolution(), NvBufferColorFormat_NV12, NvBufferLayout_Pitch);

// Create and extract NvBuffer parameters
NvBufferParams params;
NvBufferGetParams(m_dmabuf_fd, &params);

// Map a pointer to the NvBuffer
void *pointer;
NvBufferMemMap(m_dmabuf_fd, 0, NvBufferMem_Read, &pointer);

// Create and fill an OpenCV matrix
cv::Mat matrix;
opencvMatrix = cv::Mat(*(params.height), *(params.width), CV_8UC4, (char*)(pointer), *(params.pitch));
cv::cvtColor(matrix, matrix, cv::COLOR_RGBA2RGB);

// Fill a QByteArray buffer with the capture informations
QByteArray buffer((char*)(matrix.data), *(params.height) * *(params.width) * 3);

However, I’d like to directly use the nv12 format instead of going with the RGBA then RGB as of now the framework used is compatible with nv12 data.
I changed the creation of the native buffer with :

m_dmabuf_fd = m_iImageNativeBuffer->createNvBuffer(m_iEGLOutputStream->getResolution(), NvBufferColorFormat_NV12, NvBufferLayout_Pitch);

But I don’t know how to create a QByteArray with the acquired data.
I understood that NV12 format is a 2 plane format one for Y and one for UV.

 // Create and extract NvBuffer parameters
    NvBufferParams params;
    NvBufferGetParams(m_dmabuf_fd, &params);

    // Map a pointer to the NvBuffer
    void* pointer_y;
    NvBufferMemMap(m_dmabuf_fd, 0, NvBufferMem_Read, &pointer_y);
    void* pointer_uv;
    NvBufferMemMap(m_dmabuf_fd, 1, NvBufferMem_Read, &pointer_uv);

Does anyone know how to create a QByteArray then ?

[DaneLLL]

Hi,
Not like RGBA, you would need to handle alignment for NV12. The size of Y plane is (pitch *height) bytes. UV plane is (0.5*pitch*height) bytes.

Here is an example of 1920x1080:
Memory for NvMap - #10 by DaneLLL

[aniss.louahadj]

I’m sorry but the link didn’t really help me.
I tried to get the buffer for the nv12 image like so :

// Create and extract NvBuffer parameters
  NvBufferParams params;
  NvBufferGetParams(m_dmabuf_fd, &params);

  // Map a pointer to the NvBuffer
  void* pointer_y;
  NvBufferMemMap(m_dmabuf_fd, 0, NvBufferMem_Read, &pointer_y);
  void* pointer_uv;
  NvBufferMemMap(m_dmabuf_fd, 1, NvBufferMem_Read, &pointer_uv);

I then tried to only display the Y plane with openCv :

cv::Mat picYV12 = cv::Mat(*(params.pitch), *(params.height), CV_8UC1, (char*)pointer_y);
cv::imshow("test.bmp", picYV12);

But the output I get is

[aniss.louahadj]

I also tried to have the Y and UV planes in a QByteArray by doing :

// Create and fill an OpenCV matrix
  cv::Mat y_matrix;
  y_matrix = cv::Mat(*(params.height), *(params.width), CV_8UC1, (char*)(pointer_y), *(params.pitch));
  cv::Mat uv_matrix =
  uv_matrix = cv::Mat(*(params.height)/2, *(params.width)/2, CV_8UC1, (char*)(pointer_uv), *(params.pitch)/2);
// Fill a QByteArray buffer and the timestamp with the capture informations
  QByteArray buffer((char*)(y_matrix.data), *(params.height) * *(params.width) * 3 / 2);
  buffer.append((char*)(uv_matrix.data));

And the output I get is :

[DaneLLL]

Hi,
Looks like it only supports NV12 in contiguous memory, so that the two planes(Y and UY planes) have to be in widthheight1.5 bytes continuously. But hardware dma buffer generally has alignment between Y and UV planes. We would suggest use RGBA sine it is more simple to map to OpenCV.

[aniss.louahadj]

As I mentioned in the first post I really need the NV12 format because of the Framework I’m working on.

[DaneLLL]

Hi,
Or you may try like this:
[Gstreamer] nvvidconv, BGR as INPUT - #4 by DaneLLL
To get I420 data in appsink

[aniss.louahadj]

I managed to find a solution of my own.
I used NvBufferColorFormat_YUV420 instead of NvBufferColorFormat_NV12
To get the data I needed to create three pointers :

unsigned char* pointer_y = NULL;
NvBufferMemMap(m_dmabuf_fd, 0, NvBufferMem_Read, (void**)&pointer_y);
unsigned char* pointer_u = NULL;
NvBufferMemMap(m_dmabuf_fd, 1, NvBufferMem_Read, (void**)&pointer_u);
unsigned char* pointer_v = NULL;
NvBufferMemMap(m_dmabuf_fd, 2, NvBufferMem_Read, (void**)&pointer_v);

And then fill a char array without taking the padding in consideration :

//Get Y data
int count = 0;
auto image = new unsigned char[params.width[0] * params.height[0] * 3 / 2];
for (unsigned int i = 0; i < params.height[0]; i++)
{
    for (unsigned int j = 0; j < params.width[0]; j++)
    {
        image [count++] = pointer_y[i * params.pitch[0] + j];
    }
}

//Get UV data
for (unsigned int i = 0; i < params.height[1]; i++)
{
    for (unsigned int j = 0; j < params.width[1]; j++)
    {
        image [count++] = pointer_u[i * params.pitch[1] + j];
        image [count++] = pointer_v[i * params.pitch[2] + j];
    }
}

// Fill a QByteArray buffer
QByteArray buffer((char*)(image), params.width[0] * params.height[0]*3/2);