Memory will not be released after deepstream cleans the pipeline

Please provide complete information as applicable to your setup.

• Hardware Platform (Jetson / GPU)
• DeepStream Version 5.0
• JetPack Version 32.4.3
• TensorRT Version 7.1
• NVIDIA GPU Driver Version 10.2
• Issue Type questions
• Memory will not be released after deleting the pipeline.The code example is deepstream-test2,When I start the pipeline loading model to run for a period of time, set the pipeline status to null and delete the pipeline. The memory will not be released until the program exits. How can I actively release the loaded model memory?
• gst_element_set_state (pipeline, GST_STATE_NULL);
g_print (“Deleting pipeline\n”);
gst_object_unref (GST_OBJECT (pipeline));
g_source_remove (bus_watch_id);
g_main_loop_unref (loop);

The pipeline is deleted, but the loaded model will not be released。

What model?

use yolov5s model

What is your complete scenario? When and where do you want to release the loaded model memory? For what purpose?

My application scenario is to build a video processor. When I need to process the video, I will create a pipeline and load the model (yolov5) to infer the video. When I don’t need it, I will delete the pipeline and clean up the memory. This is because my main process is running all the time, so when I delete the pipeline , the memory will not be released, which will put pressure on my Jetson nx.

I also have the same issue but using yolov4. I want to be able to kill the pipeline and release the memory. Any luck in how to clear the model from memory?

Hi, curious_cat, we have the same problem and hope someone can solve it.

Unforunately, some memory will always be occupied when the process is alive. E.g. the cuda memory for tensorRT,… to make sure the process does not have to initialize tensorrt again when start/stop the operation.

I seem to be able to recreate deepstream pipeline by spawning it in a separate process, and kill it using sigint. Seems to be working. I have it in a loop, and the memory seems to go down when the deepstream process dies, and up when the deepstream process gets recreated.

Is there a way to free up memory manually? When I need to load other models (retinaface), I have to free up the original model memory (yolov5).

Why don’t you create two different pipelines? Spawn in a thread (C++) or process (python), and use the sigint to kill it.
This repo by these guys seems to have the idea with the pipeline. I have implemented something similar.

Most of these un-released memory are used by CUDA context and the CUDA kernels of TensorRT/cuDNN/cuBLAS, you can release them by cudaDeviceReset() ,but these memory are still needed and can be leveraged when you run another model, so you actually don’t need to release these memory.

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.