Possible memory leak using opencv VideoCapture with gstreamer backend

[test_account]

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU)
Jetson Nano Developer Kit
• Gstreamer Version
1.14.5
• Opencv Version
4.7.0
• Python Version
3.6.9
• JetPack Version (valid for Jetson only)
4.6
• Issue Type( questions, new requirements, bugs)
Possible memory leak
• How to reproduce the issue ? (This is for bugs. Including which sample app is using, the configuration files content, the command line used and other details for reproducing)

Minimal python script:

def mem_available_mb():
    with open('/proc/meminfo') as file:
        mem_available_in_kb = None
        mem_total_in_kb = None
        for line in file:
            if 'MemTotal' in line:
                mem_total_in_kb = int(line.split()[1])
                if mem_available_in_kb is not None:
                    print("used:", mem_total_in_kb / 1024)
            elif 'MemAvailable' in line:
                mem_available_in_kb = int(line.split()[1])
                break
            if mem_available_in_kb is None or mem_total_in_kb is None:
                continue

    return mem_available_in_kb / 1024

def init_cam_streams():
    cams = []
    for i in range(2):
        cams.append(cv2.VideoCapture(i, cv2.CAP_GSTREAMER))
    return cams

if __name__ == "__main__":
    file_name = "test gstreamer minimal.txt"
    cams = init_cam_streams()
    while True:
        try:
            a = mem_available_mb()
            with open(file_name, "a") as f:
                f.write(str(a) + "\n")
            time.sleep(10)
            print(len(cams))
        except KeyboardInterrupt:
            break

While running this script, available memory decreases with ± 1MB/min. Switching to FFMPEG backend causes the problem to disappear. Any suggestions?

[DaneLLL]

Hi,
Please share the gstreamer pipeline you run in cv2.VideoCapture() for reference. And the latest release is 4.6.3. If possible, please upgrade to the release and try.

[test_account]

Thanks for the reply! I am currently using

"v4l2src device=/dev/video{i} ! video/x-raw ! videoconvert ! video/x-raw, format=BGR ! appsink max_buffers=2 drop=1"

as pipeline. Upgrading is unfortunately no option at the moment.

[mhd0425]

You are measuring system memory, right? Does it begin to free itself if you run it long enough?

Check the process’s memory with something like top.

[test_account]

No if I leave it running for a long time (several hours) the memory keeps increasing gradually. When using top it is clear that it is the memory used by this process that is increasing.

[Honey_Patouceul]

You may first check with a second terminal running top as suggested by @mhd0425, then you should see (I’m testing that on AGX Orin running JP5, so your Nano case may differ) that gstreamer is not the culprit:

gst-launch-1.0 v4l2src device=/dev/video0 ! video/x-raw,framerate=30/1 ! videoconvert ! video/x-raw,format=BGR ! queue ! appsink max_buffers=2 drop=1

Not tried in a long run, but seems ok so far.

Then you may try to involve OpenCv from C++. The following shows no leak :

#include <iostream>
#include <opencv2/opencv.hpp>
#include <opencv2/videoio.hpp>
#include <opencv2/highgui.hpp>

int
main ()
{
   const char* gst =  "v4l2src device=/dev/video0 ! video/x-raw,framerate=30/1 ! videoconvert ! video/x-raw,format=BGR ! queue ! appsink max_buffers=2 drop=1";
   
   cv::VideoCapture cap(gst, cv::CAP_GSTREAMER);
   if (!cap.isOpened ()) {
      std::cout << "Failed to open camera." << std::endl;
      return (-1);
   }
   std::cout << "Video Capture opened (backend: " << cap.getBackendName() << ")" << std::endl;

   unsigned int width = (unsigned int) cap.get (cv::CAP_PROP_FRAME_WIDTH);
   unsigned int height = (unsigned int) cap.get (cv::CAP_PROP_FRAME_HEIGHT);
   double fps = cap.get (cv::CAP_PROP_FPS);
   unsigned int pixels = width * height;
   std::cout << "Framing: " << width << " x " << height << " @" << fps << " FPS (" << pixels*fps/1000000.f << " MPixels/s)" <<std::endl;
 
   cv::Mat frame_in;
   cv::namedWindow("MyCameraPreview", cv::WINDOW_AUTOSIZE);
   while (1) {
      if (!cap.read (frame_in)) {
         std::cout << "Capture read error" << std::endl;
         break;
      }
      else {
         cv::imshow("MyCameraPreview",frame_in);
         cv::waitKey(1); 

         if (cv::waitKey(1) == 'q')
	         break; 
      }
   }

   cv::waitKey(33);
   cap.release();

   return 0;
}

Now, using it from python, even without displaying, there is an increase of memory usage. So the issue may be in opencv python bindings or in python libs.

Note that this issue is not specific to Jetson, you may get further help from opencv forum or reporting an issue to opencv github.