CPU usage surging when more than 8 transcoder processes execute

Robotics & Edge Computing Jetson & Embedded Systems Jetson Xavier NX
1

Jetpack ver.: 4.6
Hardware: Xavier NX

I developed a transcoder (rtsp->decoder->encoder->rtmp) program based on Gstreamer. It worked as expected. I tested it with multiple child processes via fork(). I’ve noticed that if I spawned 8 processes, each consumed only 1% CPU. But if more processes were added, like 10, each one’s consumption jumped to 4% CPU. You can expect the same phenomenon when using gst-launch method. Attached please find the relevant screenshots.

How to explain this? Any advice to mitigate the CPU usage? Thanks!

8 processes:
cpu1

10 processes:
cpu2

2

Hi, AndySimcoe

I might be able to help optimize the GStreamer pipelines. Can you please share the GStreamer pipeline descriptions that you are using in your application?

Jafet Chaves,
Embedded SW Engineer at RidgeRun
Contact us: support@ridgerun.com
Developers wiki: https://developer.ridgerun.com/
Website: www.ridgerun.com

3

Hi Jafet,

My pipeline is like this:
gst-launch-1.0 rtspsrc location=“rtsp://192.168.1.28:8554/h264_test” latency=0 ! rtph264depay ! h264parse ! nvv4l2decoder enable-max-performance=1 ! nvv4l2h264enc maxperf-enable=1 iframeinterval=25 control-rate=1 bitrate=2000000 EnableTwopassCBR=1 preset-level=4 MeasureEncoderLatency=1 profile=2 ! ‘video/x-h264,stream-format=(string)byte-stream, alignment=(string)au’ ! h264parse ! flvmux ! rtmpsink location=‘rtmp://localhost/live/livestream1’ sync=0

Thank you for your help!

4

Does it improve if running all pipelines in the same gst-launch process ?

gst-launch-1.0 \
rtspsrc location=“rtsp://192.168.1.28:8554/h264_test” latency=0 ! rtph264depay ! h264parse ! nvv4l2decoder enable-max-performance=1 ! nvv4l2h264enc maxperf-enable=1 iframeinterval=25 control-rate=1 bitrate=2000000 EnableTwopassCBR=1 preset-level=4 MeasureEncoderLatency=1 profile=2 ! ‘video/x-h264,stream-format=(string)byte-stream, alignment=(string)au’ ! h264parse ! flvmux ! rtmpsink location=‘rtmp://localhost/live/livestream1’ sync=0  \
rtspsrc location=<second_source_uri> latency=0 ! rtph264depay ! h264parse ! nvv4l2decoder enable-max-performance=1 ! nvv4l2h264enc maxperf-enable=1 iframeinterval=25 control-rate=1 bitrate=2000000 EnableTwopassCBR=1 preset-level=4 MeasureEncoderLatency=1 profile=2 ! ‘video/x-h264,stream-format=(string)byte-stream, alignment=(string)au’ ! h264parse ! flvmux ! rtmpsink location=‘<second_sink_uri>’ sync=0 \
...

Also try monitoring with tegrastats and see if there is memory/swap or else that may be cause the issue.
Also note that increased usage may come from your RTMP server. Monitor its usage with top or htop.

6

Hi,
You may also try Jetpack 5.1(r35.2.1) and check sudo tegrastats

Thank jchaves and Honey Patouceul for the quick suggestion.

7

Hi Honey_Patouceul,

Thank you for your advice. I’ll look into them.

8

Hi DaneLLL,

I’ll try JetPack 5.1 and give feedback later. Thank you!

9

Hi, Honey_Patouceul

Your suggestion about multiple pipelines in one process reminds me that I developed a program a couple of days ago that created multiple transcoder pipelines/bins in one main loop. I go back to test that program and the result is encouraging in terms of CPU and memory usage. When 10 transcoders are running, the process uses just 10% CPU.

However, I can remember the reason why I switched to multi-process, I need to create/remove transcoders pipeline/bin dynamically. It seemed the resources could not be released completely after removing a pipeline/bin which caused the memory usage to rise steadily. I think I’ve done the clean-up properly after deleting the pipeline. I’ve googled this issue and could not find a solution. Do you think there are tricks of deleting pipelines cleanly but keeping a program running? Thanks!

10

Have you tried writing a gstreamer application that replicates this memory leak and monitoring with Valgrind? Without seeing how you are actually de-allocating the pipeline and where the leak is coming from it is hard to debug further.

AFAIK you should be able to run multiple pipelines, even with dynamic creation and deletion, within the same GStreamer context without issue.

1 Like
11

Hi, pwolfe1

Thank you for your advice about Valgrind. I’ll give it a shot.

13

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.