Argus - Memory Leak when retrieving NV12 Buffer

Hello,

I’m currently facing a memory leak on Jetson Orin when using the Argus API to extract image data.

I’ve based my implementation in the code found in /usr/src/jetson_multimedia_api/ :

  • argus/apps/camera/modules/Dispatcher

  • samples/13_argus_multi_camera/main.cpp

The code successfully retrieves and displays NV12 data.

However, the memory consumption steadily increases over time (initially confirmed with htop over night, with a >10Gb increase in memory consumption).

Using Valgrind, I confirmed the memory leak is present and coming from:

  • The creation of the OutputStream (Producer side);
  • The creation of the NvBuffer (Consumer side).

Producer side ( threadInitialize() )

bool UpdatedProducer::initEGLOutputStreamSettings()
{
    // Sanity check
    if (!m_iSensorMode)
    {
        m_lastError = "Producer -> ISensorMode not initialized";
        return false;
    }
    if (!m_iEglOutputStreamSettings)
    {
        m_lastError = "Producer -> IEGLOutputStreamSettings not initialized";
        return false;
    }

    m_iEglOutputStreamSettings->setPixelFormat(Argus::PIXEL_FMT_YCbCr_420_888);
    m_iEglOutputStreamSettings->setMode(Argus::EGL_STREAM_MODE_MAILBOX);
    m_iEglOutputStreamSettings->setResolution(m_iSensorMode->getResolution());
    m_iEglOutputStreamSettings->setMetadataEnable(true);

    return true;
}

bool UpdatedProducer::initOutputStreamWithEGLOutputStreamSettings()
{
    // Sanity check
    if (!m_iCaptureSession)
    {
        m_lastError = "Producer -> ICaptureSession not initialized";
        return false;
    }
    if (!m_outputStreamSettings)
    {
        m_lastError = "Producer -> OutputStreamSettings not initialized";
        return false;
    }

    // Create the output stream
    m_outputStream = Argus::UniqueObj<Argus::OutputStream>(m_iCaptureSession->createOutputStream(m_outputStreamSettings.get()));
    if (!m_outputStream)
    {
        m_lastError = "Producer -> Failed to create OutputStream";
        return false;
    }

    // Extract the EGL output stream interface
    m_iEGLOutputStream = Argus::interface_cast<Argus::IEGLOutputStream>(m_outputStream);
    if (!m_iEGLOutputStream)
    {
        m_lastError = "Producer -> Failed to create EGLOutputStream interface";
        return false;
    }
    
    return true;
}

Consumer side ( threadExecute() )

bool UpdatedConsumer::getNvBuffer()
{
    // Sanity check
    if (!m_iEGLOutputStream)
    {
        std::cerr << "Consumer -> EGL output stream interface not initialized" << std::endl;
        return false;
    }
    if (!m_iImageNativeBuffer)
    {
        std::cerr << "Consumer -> Image Native Buffer interface not initialized" << std::endl;
        return false;
    }

    Argus::Status status;

    // Get NvBuffer from native buffer
    m_dmabuf_fd = m_iImageNativeBuffer->createNvBuffer(m_iEGLOutputStream->getResolution(), NVBUF_COLOR_FORMAT_YUV420, NVBUF_LAYOUT_PITCH, EGLStream::NV::ROTATION_0, &status);

    if (Argus::STATUS_OK != status) 
    {
        std::cerr << "Consumer -> Failed to create NvBuffer" << std::endl;
        return false;
    }

    return true;
}

bool UpdatedConsumer::copyBuffer()
{
    // Sanity check
    if (m_dmabuf_fd == -1)
    {
        std::cerr << "Consumer -> NvBuffer not initialized" << std::endl;
        return false;
    }

    // Create and extract NvBuffer parameters
    NvBufferParams params;
    NvBufferGetParams(m_dmabuf_fd, &params);

    #ifndef NDEBUG
    std::cout << "Consumer -> NvBuffer Params:\n";
    std::cout << "Consumer -> dmabuf_fd      : " << params.dmabuf_fd << " (Holds the DMABUF FD of the hardware buffer)" << std::endl;
    std::cout << "Consumer -> memsize        : " << params.memsize << " (size of the memory)" << std::endl;
    std::cout << "Consumer -> nv_buffer_size : " << params.nv_buffer_size << " (size of hardware buffer)" << std::endl;
    std::cout << "Consumer -> pixel_format   : " << params.pixel_format << " (0 = NvBufferColorFormat_YUV420 -> BT.601 colorspace - YUV420 multi-planar)" << std::endl;
    std::cout << "Consumer -> num_planes     : " << params.num_planes << " (number of planes of hardware buffer)" << std::endl;

    for(unsigned int i=0; i < params.num_planes; i++)
    {
        std::cout << "Consumer -> Plane " << i << " : " << std::endl;
        std::cout << "Consumer -> width          : " << params.width[i] << " (width of each plane of hardware buffer)" << std::endl;
        std::cout << "Consumer -> height         : " << params.height[i] << " (height of each plane of hardware buffer)" << std::endl;
        std::cout << "Consumer -> pitch          : " << params.pitch[i] << " (pitch of each plane of hardware buffer)" << std::endl;
        std::cout << "Consumer -> offset         : " << params.offset[i] << " (memory offset values of each video plane of hardware buffer)" << std::endl;
        std::cout << "Consumer -> psize          : " << params.psize[i] << " (size of each video plane of hardware buffer)" << std::endl;
        std::cout << "Consumer -> layout         : " << params.layout[i] << " (layout type of each plane of hardware buffer)" << std::endl;
    }
    #endif

    // Map pointers to the NvBuffer
    unsigned char* pointer_y = NULL;
    NvBufferMemMap(m_dmabuf_fd, 0, NvBufferMem_Read, (void**)&pointer_y);
    unsigned char* pointer_u = NULL;
    NvBufferMemMap(m_dmabuf_fd, 1, NvBufferMem_Read, (void**)&pointer_u);
    unsigned char* pointer_v = NULL;
    NvBufferMemMap(m_dmabuf_fd, 2, NvBufferMem_Read, (void**)&pointer_v);
    
    char* buffer;
    buffer = new char[params.width[0] * params.height[0] * 3 / 2];

    // std::cout << "----> copyBuffer() -> ##### created -> " << static_cast<void*>(buffer) << std::endl;  

    // Fill buffer with NV12 format data
    // ----------------------
    // |                    |
    // |        Y           |
    // |                    |
    // |                    |
    // ----------------------
    // |       U / V        |
    // |                    |
    // ----------------------
    int count = params.width[0] * params.height[0];
    for(unsigned int i=0; i<params.height[0]; i++) // for each row
    {
        memcpy(buffer + (i*params.width[0]),(char*)(pointer_y + (i*params.pitch[0])), params.width[0]); // copy width portion of pitch (width + padding = total row size)
        
        // UV Interleaved Portion (U0 V0 U1 V1 ...)
        if(i < params.height[0]/2)
        {
            for (unsigned int j=0; j < params.width[1]; j++)
            {
                buffer[count++] = (char)(pointer_u[i * params.pitch[1] + j]);
                buffer[count++] = (char)(pointer_v[i * params.pitch[1] + j]);
            }
        }
    }

    // Fill the timestamp with the capture informations
    int64_t ts = m_timestamp * 1e-6;

    // Send the Data and Timestamp
    sigCopyBuffer(buffer, ts);

    // Destroy the created NvBuffer
    NvBufferMemUnMap(m_dmabuf_fd, 0, (void**)&pointer_y);
    NvBufferMemUnMap(m_dmabuf_fd, 1, (void**)&pointer_u);
    NvBufferMemUnMap(m_dmabuf_fd, 2, (void**)&pointer_v);
    NvBufferDestroy(m_dmabuf_fd);

    return true;
}

Valgrind Output 2

All Argus::UniqueObj are being destructed with .reset(), upon shutdown.

My Argus version is the following → “Argus Version: 0.98.3 (multi-process)”.

Thank you in advance.

hello joao.malheiro.silva,

may I also know which Jetpack release version you’re working with?
we did resolve some memory leakage, for example, Topic 268519, the fixes has check-in to rel-35 code-line as well.
hence, you may expect changes will be included in the next Jetpack-5 release version. i.e. Jetpack-5.1.3