Memory Leak issue of Gstreamer 1.8.3 in JetPack 3.2.1

Hello,

I encountered a memory leak isuue of gstreamer 1.8.3 in Jetpack 3.2.1.

I’m using the follwoing command.

gst-launch-1.0 -v -e
rtspsrc location=rtsp://xx.xx.xx.xx:554/avpf/
! rtph265depay
! h265parse
! omxh265dec
! nvcompositor
! nvoverlaysink

I’m checking a physical memory usage by “top” command.
The “RES” column is a physical memory size.

This value is increasing with time.

Does anyone have informations about this memory leak isuue and solutions?

Hi kozo,
We will check and update.

Hello DaneLLL,

Thnak you for your reply.

If you have something new, Could you tell me ASAP.

Regards,

Hi kozo,
Please try with the attachment.
libgstnvcompositor.zip (11.8 KB)

Hello DaneLLL,

I will try the new nvcompositor.so.
It feels good with about 30 minutes.

I will excute it for a long time from now.
Maybe, 3 days.
Then, I will tell you the result.

Best Regards,

Hello DaneLLL,

The new nvcompositor.so was not so good.
The script was killed about 20 hous later.

----------- syslog ----------------------
Aug 14 16:08:40 tegra-ubuntu kernel: [109476.870946] Out of memory: Kill process 2486 (jkapp3) score 844 or sacrifice child
Aug 14 16:08:40 tegra-ubuntu kernel: [109476.882294] Killed process 2486 (jkapp3) total-vm:9134644kB, anon-rss:6767424kB, file-rss:11876kB

In a case of the original nvcompositor.so, it was killed about 3 days later.

Also in the case without nvcompositor, the following script, this memory leak was happened.

gst-launch-1.0 -v -e
rtspsrc location=rtsp://xx.xx.xx.xx:554/avpf/
! rtph265depay
! h265parse
! omxh265dec
! nvoverlaysink

However in the case without nvoverlaysink, the following script, I feel that this memory leak was NOT happened.

gst-launch-1.0 -v -e
rtspsrc location=rtsp://xx.xx.xx.xx:554/avpf/
! rtph265depay
! h265parse
! omxh265dec
! nvegltransform
! nveglglessink

BTW, the image size that I’m using is 3840x960.

Hi kozo,
Per your description, it seems to be an issue in nvoverlaysink. We will try to reproduce the issue.

Hello DaneLLL,

I tried the option “render-delay” of the nvoverlaysink.

gst-launch-1.0 -v -e
rtspsrc location=rtsp://xx.xx.xx.xx:554/avpf/
! rtph265depay
! h265parse
! omxh265dec
! nvoverlaysink render-delay=1000000

The default value of the option is 0 (ns).
It was a short time executing.
I feel the memory leak is reduced.
But I do’nt know the reason.

Hi kozo,
We have run over-weekend test with nvoverlaysink(without nvcompositor) and don’t hit the process being killed by OoM killer. You hit the issue when running ‘nvcompositor ! nvoverlaysink’ or running nvoerlaysink only?

Hello DaneLLL,

Thank you for your help and support.
This occured in both of “nvcompositor ! nvoverlaysink” and " nvoverlaysink" only.

However, I feel the following two things.

  1. The larger the image size, the easier this is occured.
    the image size that I’m using is 3840 x 2160 size.
    the frame rate is 30 fps.
    At the size is 1920x1080, the memory leak was only little.

  2. As the probability of packet loss ( and jitter) increases, this is more likely to occur.
    The rtspsrc element is received a data through the wifi network.
    There are quite a lot of packet losses and jitter.

BTW, Although the process was not killed by “out of memory”, is a physical memory usage increased?

Hi kozo,
We don’t observe obvious memory increase in the test. Will try 4K resolution.

Hello DaneLLL,

Using “rtspsrc” element may be difficult to reproduce.
I tried to using “videotestsrc” element.

A)
gst-launch-1.0
videotestsrc
! video/x-raw,format=NV12,width=3840,height=2160,framerate=30/1
! nvvidconv
! nvoverlaysink sync=false

B)
gst-launch-1.0
videotestsrc
! video/x-raw,format=NV12,width=3840,height=2160,framerate=30/1
! omxh265enc
! h265parse
! omxh265dec
! nvoverlaysink sync=false

I tried the above two scripts for 26 minutes each.

A) did NOT increase a memory usage.
B) increased 1KB memory usage.

Hello DaneLLL,

Did you reproduce this phenomenon?
And, do you have any new information about this?

Best Regrds,

Hi kozo,
We have run 4K streaming with nvoverlaysink and the process is still alive overweekend, although memory increase is observed via ‘top’

It is long run case and we will need some time for investigation.

Hello DaneLLL,

What is this status?

Best Regards,

Hi koza,
We are still checking.

You can contact NVIDIA region salesperson. Let’s see if we can have further cooperation to prioritize this issue.