CPU and Ram usage increase over time, especially with an internet connection

[sej]

Hi!

I see a behavior that I don’t understand on my Xavier AGX board.
I am using the board to get a stream that comes via rtsp with Gstreamer, then I modify this video and I restream with WebRTC.
Thanks to the tegrastats tool I check the CPU and RAM usage of the board. When the board isn’t running any program, I see how the ram increases over time and the cpu usage also increases, sometimes the CPU 1 reaches 95 ~ 100% and then the board reboots. I put some graphs that represents this state.
The information has been obtained by the tegrastats command as I said before.

And the RAM:

And an example with the board restart can be seen in the following image:

In the last image you can see that the last data collected before the reboot, CPU 1 is close to 100%.

After read this topic:

AGX Xavier easy to crash when ethernet network connected

I can run multiple deepstreem_test_3.py (up to 7 pipeline with each feeding 4 video files) without crash the system (or causing it self reboot) if I disconnect ethernet network on AGX Xavier (by clicking top right corner network icon and select disconnect right below wired connection 1) . However, if I reconnect the network and run the above same multiple deepstream_test_3.py then the system crash (self reboot). Steps to duplicate the crash (self reboot): turn on AGX and make sure network is…

I have done the test with and without an internet connection and I see a big difference:
Without internet:

With internet:

I don’t understand the behaviour of CPU 1 and the RAM of the board.
Is there any reason for RAM and CPU 1 increse their usage sometimes, especially with internet connection?

My Jetpack version is:

# R32 (release), REVISION: 4.4, GCID: 23942405, BOARD: T186REF, EABI: aarch64

Thanks in advance

[linuxdev]

Just a thought on this which might need to be considered…I don’t know about your memory use software, but there are different categories of memory use, and some of them are not really required so much as they are used for performance reasons when nothing else uses the memory.

Can you monitor the memory using “xosview” ("sudo apt-get install osview") and see if it tracks the same as your software? Notice that you can enlarge the xosview and see more details on memory use. The “used” memory is something which is actually being used. The parts to pay attention to are “BUFF” and “CACHE” memory. These are not actually consuming memory since they are released as soon as something needs memory.

An example is that when a hard drive (or eMMC) is read the content goes into RAM. Once an application is finished with the read one could release the buffer used for storing the initial disk read. On the other hand, if something else wants to read the same information, then reading again without using the RAM which initially stored that information would slow reading down. As Linux reads the filesystem it tends to initially be memory “used”, but after the read completes, the memory becomes “buffered” (this is a general theme for performance, but depending on task, it might be either cached or buffered). A second read of that data is fast when taken from an existing buffer or cache. There is no need to release this until either (A) something has written to this, and the cache or buffer is invalidated, or (B) we are running out of RAM for an application and the memory must be used by something else. Does xosview show actual “used” memory consumption is going up? Or does the increase in memory you are seeing correspond to “cached” or “buffered” memory in xosview?

Note: There are other applications which may also show different memory categories, but xosview is my example.

Also, the GPU uses system memory, so you might also check if tegrastats shows GPU memory use is going up.

[ShaneCCC]

Could you have step to reproduce it.

[sej]

I think my problem is another, I don’t think my program is what causes the problem, maybe it accelerates it, but it isn’t the cause.
It has already happened to me a few times that leaving the board without doing anything has restarted itself.
The temperature sure is not the problem because the board is cold when this happens and the fan is running in that moment.
And when this happens, when restarting board, the CPUs are over 20 ~ 30%, especially the CPU0 with peaks that sometimes reaches 100%. Sometimes over time the CPUs go themselves to 0 ~ 5% or hit 100% and the board is reset again.

Do you have any idea what’s going on?

[ShaneCCC]

There is no update from you for a period, assuming this is not an issue any more.
Hence we are closing this topic. If need further support, please open a new one.
Thanks

Can’t tell what going on, may need more experience and information.
Could you have reproduce steps?