Hardware: Jetson Xavier
- L4T 32.6.1
- GStreamer Core Library version 1.14.5
- nvvidconv: 1.2.3
We are using the
nvvidconv plugin in a pipeline that performs ROI transforms, scaling of raw images and jpeg encoding.
When using the
nvvidconv plugin, we’re noticing that it leaks GPU memory consistently and very quickly when used with an upstream
videocrop plugin that changes dimensions of the input image. We suspect the nvvidconv element leaks GPU memory everytime the crop dimensions are changed.
Here is a minimal Python script to reproduce the issue:
import asyncio from concurrent.futures import ThreadPoolExecutor import random import gi gi.require_version('Gst', '1.0') from gi.repository import GLib, Gst Gst.init(None) pipeline = Gst.parse_launch("videotestsrc ! videocrop name=crop top=16 left=16 right=16 bottom=16 ! video/x-raw, framerate=10/1 ! nvvidconv ! video/x-raw(memory:NVMM), format=NV12, width=224, height=224, framerate=10/1 ! autovideosink") crop = pipeline.get_by_name('crop') pipeline.set_state(Gst.State.PLAYING) main_loop = GLib.MainLoop() loop = asyncio.get_event_loop() task = loop.run_in_executor(ThreadPoolExecutor(), main_loop.run) # Switch videocrop upstream from nvvidconv leads to leakage async def switch_scale(crop): for _ in range(1000): x = random.randint(0, 100) crop.set_property('top', x) crop.set_property('top', x) crop.set_property('top', x) crop.set_property('top', x) await asyncio.sleep(0.1) print("frame") # Option 1: Leaks GPU memory like crazy when performing crops loop.run_until_complete(asyncio.gather(task,switch_scale(crop))) # Option 2: Does not leak GPU memory #loop.run_until_complete(asyncio.gather(task,))
Where Option 1 will leak (about 4MB/s) when monitoring jtop’s GPU MEM output, and Option 2 will not.
I’ve found these issues that might be related:
Is it possible to get an newer version of the nvvidconv plugin that may have had this memory leak addressed for Jetpack 4.6 that I can use with my version of Gstreamer?
Here are two screenshots of jtop indicating GPU usage, taken approximately 2 minutes apart:
About two minutes later:
As can be seen from the images, the GPU memory allocation has grown from 11.4MB to 181MB in a short period of time. It will continue growing.