MMAPI R28.2.1 VideoConverter memory leak

Autonomous Machines Jetson & Embedded Systems Jetson TX2
1

Hi,

We are currently developing custom use case with MMAPI.
FYI, we use L4T R28.2.1.

We have successfully implemented an encoding pipeline WITHOUT video scaling feature (NvVideoConverter), only with video encoder (NvVideoEncoder) as follow:

.               +----------------------------+
 +----------+   |        !         !         |   +------------+
 |  Camera  |   | Output ! NvVideo ! Capture |   |  File      |
 |  Capture |-->| plane  ! Encoder ! plane   |-->|  Recording |
 |  Argus   |   |  DMA   !         !  MMAP   |   |            |
 +----------+   |        !         !         |   +------------+
                +----------------------------+

With the application above, we haven’t any problems about memory leak.

Now, we are trying to implement an encoding pipeline WITH video scaling feature (NvVideoConverter) as follow:

.               +------------------------------+    +----------------------------+
 +----------+   |        !           !         |    |        !         !         |   +------------+
 |  Camera  |   | Output ! NvVideo   ! Capture |    | Output ! NvVideo ! Capture |   |  File      |
 |  Capture |-->| plane  ! Converter ! plane   |--->| plane  ! Encoder ! plane   |-->|  Recording |
 |  Argus   |   |  DMA   !           !  MMAP   |    |  DMA   !         !  MMAP   |   |            |
 +----------+   |        !           !         |    |        !         !         |   +------------+
                +------------------------------+    +----------------------------+

During few seconds (9-10 seconds), application works well. After 9-10 seconds of execution, the application consumes all available RAM (memory leak) and crashes.

Questions:

  1. Is there a memory leak issue with VideoConverter?
  2. Is there an existing sample which implements Argus capture, video converter and video encoder?
  3. In which case should we setup MMAP instead of DMA and vice-versa?

Thanks.

2

Hi,
You should not need NvVideoConverter in your case. Please refer to 10_camera_recording:

fd = iNativeBuffer->createNvBuffer(STREAM_SIZE,
                                           NvBufferColorFormat_YUV420,
                                           (DO_CPU_PROCESS)?NvBufferLayout_Pitch:NvBufferLayout_BlockLinear);

By default it configures to capture resolution here, but you can configure it to desired resolution.

3

Hi DaneLLL,

In fact, it is a little bit more complex than directly resizing the input because in final application the input camera resolution will be fixed.

We want to capture a video stream from native resolution (for example 1920x1080) and then resize it with NvVideoConverter (apply some TNR, flip or interpolation methods, with NvVideoConverter) before encoding.

That’s why, we are trying to implement a pipeline including NvVideoConverter.

4

Please check if you call NvbufferCreate and NvBufferDestroy in pair. Probably you keep allocating new buffers but don’t free old used buffers.

Besides, we also have TNR function in Argus. Have you tried it?

5

Hi DaneLLL,

You are right, I forgot to destroy NvBuffer as mentionned in description of createNvBuffer method.

However, I’m a bit confused because in 10_camera_recording example, NvBufferDestroy is only called when input encoder queue is full and not at the end of the pipeline, is it normal?

Thanks for your help.

6

Hi Aurelien,
You are correct there is an issue in de-initialization:

CONSUMER: Released frame. 1828717843
CONSUMER: Acquired Frame. 1828717843
CONSUMER: Released frame. 1828717844
<b>CONSUMER: Acquired Frame. 1828717844</b>
CONSUMER: Released frame. 1828717841
<b>CONSUMER: Acquired Frame. 1828717841</b>
CONSUMER: Released frame. 1828717847
<b>CONSUMER: Acquired Frame. 1828717847</b>
CONSUMER: Released frame. 1828717853
<b>CONSUMER: Acquired Frame. 1828717853</b>
CONSUMER: Released frame. 1828717843
CONSUMER: Got EOS, exiting...

Please add logic to release these buffers.

7 
diff --git a/multimedia_api/ll_samples/samples/10_camera_recording/main.cpp b/multimedia_api/ll_samples/samples/10_camera_recording/main.cpp
index 6f531b8..bb277cf 100644
--- a/multimedia_api/ll_samples/samples/10_camera_recording/main.cpp
+++ b/multimedia_api/ll_samples/samples/10_camera_recording/main.cpp
@@ -41,6 +41,7 @@
 #include <stdlib.h>
 #include <iostream>
 #include <fstream>
+#include <queue>
 
 using namespace Argus;
 using namespace EGLStream;
@@ -126,6 +127,7 @@ private:
     NvVideoEncoder *m_VideoEncoder;
     std::ofstream *m_outputFile;
     bool m_gotError;
+    std::queue < int > *dmabuf_queue;
 };
 
 ConsumerThread::ConsumerThread(OutputStream* stream) :
@@ -134,6 +136,7 @@ ConsumerThread::ConsumerThread(OutputStream* stream) :
         m_outputFile(NULL),
         m_gotError(false)
 {
+    dmabuf_queue = new std::queue < int >;
 }
 
 ConsumerThread::~ConsumerThread()
@@ -147,6 +150,8 @@ ConsumerThread::~ConsumerThread()
 
     if (m_outputFile)
         delete m_outputFile;
+
+    delete dmabuf_queue;
 }
 
 bool ConsumerThread::threadInitialize()
@@ -245,6 +250,8 @@ bool ConsumerThread::threadExecute()
             // Release the frame.
             fd = v4l2_buf.m.planes[0].m.fd;
             NvBufferDestroy(fd);
+            fd = dmabuf_queue->front();
+            dmabuf_queue->pop();
             if (VERBOSE_ENABLE)
                 CONSUMER_PRINT("Released frame. %d\n", fd);
         }
@@ -302,11 +309,20 @@ bool ConsumerThread::threadExecute()
         v4l2_buf.m.planes[0].m.fd = fd;
         v4l2_buf.m.planes[0].bytesused = 1; // byteused must be non-zero
         CHECK_ERROR(m_VideoEncoder->output_plane.qBuffer(v4l2_buf, NULL));
+        dmabuf_queue->push(fd);
     }
 
     // Wait till capture plane DQ Thread finishes
     // i.e. all the capture plane buffers are dequeued
     m_VideoEncoder->capture_plane.waitForDQThread(2000);
+    while(!dmabuf_queue->empty())
+    {
+        int fd = dmabuf_queue->front();
+        NvBufferDestroy(fd);
+        dmabuf_queue->pop();
+        if (VERBOSE_ENABLE)
+            CONSUMER_PRINT("Released frame. %d\n", fd);
+    }
 
     CONSUMER_PRINT("Done.\n");

Please refer to above patch

8

Thanks for your support.