I have been testing the deepstream-test3 sample which comes with Deepstream 4.0.1 and I could observe a raise in CPU memory usage of about 10x in something like 10 hours, while processing 10 RTSP streams at once.
I run Deepstream 4.0.1, TensorRT 5.1, CUDA 10.1, CUDNN 7.5, on a RTX2080ti.
Did you notice any similar behavior?
In Deepstream release note it is mentioned that a “small memory leak” has been observed. Could it be related to this?
I have experience the same issue. I tried just to remove the nvinfer plugging and connected the streammux directly to the sink and still the leak is there. I found the source code of the nvinfer but I could not find the nvstreammux one. There is any chance to get it to do my proper debug.
yes I’ve read that - its in the same thread we are talking in. It just says its a more widespread issue in deepstream (and therefore absolutely critical to users) and not specific to test app 3 - this is why I’meagerly awaiting an update on progress… Even a rough ETA of when a fix will be in place will help us? Currently we are totally in the dark.
So as a rough idea when will DS 5.0 be released? Are you targeting a specific month this year?
What can we do in the meantime - are there any work-arounds or specific elements we could avoid. My program needs to run all day, every day - so do we have to do stuff like routinely kill it and restart it to stop running out of memory? what do you suggest is the best practice way to handle this? Maybe others on the forum could help with how they tackle these issues?
Thankyou for the June info - that helps us plan. Is there a feature list you are working toward for this release?
I have a related question - when I try to debug my deepstream-test3 based app with valgrind - it crashes and says you’ve ‘achieved the impossible’. Do you have any tips with using valgrind to detect memory issue with deepstream/gstreamer?
HAve been through the valgrind doco but no hope - I think the issue is that I’m actually developing directly on the jetson nano and its just too resource constrained to run my app with 4 sources and 4 sinks inside of valgrind.
What is a good development workflow - build on a desktop with GPU first so that you can use the full power of tools like valgrind and then convert to run on the nano later ??