gstreamer pipeline video decoding delay grows over time, and memory consumed also grows

My pipeline like this: rtspsrc ! decodebin ! nvvidconv ! nvvideosink, and i get the frames via EGL stream with mailbox mode.

As time goes on, video delay grows, and the process’s memory also grows.

The camera output 2048x1536 h264 video stream at about 25fps, and the camera and tx2 both connected to the same ethernet, the stream bitrates about 3Mbps, so the network latency can be excluded.

The console print the frame rate sometimes very low, like this:

TVMR: FrameRate = 24.459626 
TVMR: FrameRate = 25.006377 
TVMR: FrameRate = 24.955268 
TVMR: FrameRate = 25.006001 
TVMR: FrameRate = -6.213658 
TVMR: FrameRate = 4.150881 
TVMR: FrameRate = 25.005189 
TVMR: FrameRate = 24.956202 
TVMR: FrameRate = 25.006752 
TVMR: FrameRate = -2.770359 
TVMR: FrameRate = 2.267876 
TVMR: FrameRate = 24.954769 
TVMR: FrameRate = 25.006939 
TVMR: FrameRate = 25.007252 
TVMR: FrameRate = -1.782559 
TVMR: FrameRate = 1.559933 
TVMR: FrameRate = 25.007190

and what element print this?

I examine the usage via tegrastats, the CPU/GPU/NVDEC reports low usage. After running for some time, the EGL stream state often not in EGL_STREAM_STATE_NEW_FRAME_AVAILABLE_KHR state.

So, i think the problem is the decoder or elements before decoder output lower frame rate than actually is, and how can i fix this problem, thanks.

Hi SongjianSu, can you share the sample and steps so that we can reproduce the issue on r28.1?

Hi SongjianSu,

First, the decodebin is an element that links elements dynamically using an autoplug approach. This means that depending on the data you are trying to decode to raw that’s the pipeline that is going to be created. So a first good step will be knowing what elements are involved in your pipeline, you can do it by generating a graph of the pipeline :

https://developer.ridgerun.com/wiki/index.php/How_to_generate_a_Gstreamer_pipeline_diagram_(graph)

Once you have that, I will recomend you to change your pipeline to remove the decodebin and start testing removing element by element in order to determine which one causes the leak. Then you can use something like Valgrind in order to catch the memory leak :

And finally, if you want to analize performance, a tool like GstShark will be definetly be of use:

https://developer.ridgerun.com/wiki/index.php?title=GstShark

Best Regards,

Hi josejich,
I have try to use GstShark tool in gstreamer. The code like this

GST_DEBUG="GST_TRACER:7" GST_TRACERS="framerate" gst-launch-1.0 rtspsrc ! rtph264depay ! h264parse ! omxh264dec ! nvvidconv ! "video/x-raw(memory:NVMM),format=(string){I420}" ! autovideosink

I have 2 questions to ask you.
First, when I test framerate, and want to plot it, it occur error like this:

nvidia@tegra-ubuntu:~/gst-shark/scripts/graphics$ ./gstshark-plot gstshark_2017-09-13_10\:46\:34/ -p
[error] at line 75: token "''": syntax error, unexpected CHARACTER_LITERAL

[error] Error creating AST
[warning] Unable to open trace metadata for path "gstshark_2017-09-13_10:46:34".
[warning] [Context] Cannot open_trace of format ctf at path gstshark_2017-09-13_10:46:34.
[warning] [Context] cannot open trace "gstshark_2017-09-13_10:46:34" from gstshark_2017-09-13_10:46:34/ for reading.
[error] Cannot open any trace for reading.

[error] opening trace "gstshark_2017-09-13_10:46:34/" for reading.

[error] none of the specified trace paths could be opened.

Loading proctime events...
Loading interlatency events...
Loading scheduling events...
Loading cpuusage events...
Loading framerate events...
^Coctave exited with signal 11

Second, when I test proctime, I can plot it, but how can I analyze it with the graph. In https://developer.ridgerun.com/wiki/index.php?title=GstShark, it just tell you how to use gstshark, but it doesn’t teach you how to analyze performance, so do you have some suggest ?

Thanks.

This is the pipeline

Please share clear steps so that we can reproduce the issue on r28.1.

Hi Clancy,

I filed a bug regarding your results, we will take a look on the issue (but it might be related to the unusual framerate you are getting).
Regarding how to analize the results for proctime, have you seen the following example?

In the vertical axis you can see the procesing time it took for a particular element to process over the running time of the pipeline (horizontal axis).

Hi, josejich

I have read the example in https://developer.ridgerun.com/wiki/index.php?title=GstShark_-_Example:_Processing_time.

But I don’t konw how long time when a particular element to process is normal or abnormal.

Thanks