The better way to logging and monitoring the system

Robotics & Edge Computing Jetson & Embedded Systems Jetson AGX Orin
1

Hi, I just want to discuss about what could be the best way to logging and monitoring system’s CPU, GPU and RAM. I currently wrote a script in python to save the system diagnostics from /proc file and then plot it using python again. I just wonder if anyone here uses better way to log and monitor the system.

4

Hi,
You can run $ sudo tegrastats to get the system status:

Tegrastats Utility — NVIDIA Jetson Linux Developer Guide 1 documentation

jtop is the GUI based on it:

GitHub - rbonghi/jetson_stats: 📊 Simple package for monitoring and control your NVIDIA Jetson [Orin, Xavier, Nano, TX] series

5

As far as I know, I can’t create a log record with tegrastats. I can only monitor the system. I don’t have the chance to watch the system while it is running. I want it to take a log record while it is running and then visualize and look at it. Isn’t something like this possible?

6

A file could become very long, but this will both display and log:

tegrastats 2>&1 | tee -a log_tegrastats.txt

To create the log without display:

tegrastats 2>&1 > log_tegrastats.txt

If you wish to add some sort of log rotation script to avoid too large of a log you could do that too. For example you can count lines with “wc -l log_tegrastats.txt | cut -d' ' -f 1”, and if over a certain size, then kill the tegrastats process, move file log_tegrastats.txt to log_tegrastats.txt.1, and starting again.

Also, don’t forget that “>>” appends, whereas “>” truncates upon restart. With “tee” it also truncates unless you use the “-a” to tell it to append.

More interesting: This can be done over ssh from a remote system so it doesn’t touch the disk on the Jetson.

7

This way I can get the log of the whole system. Do you suggest anything to visualise it later ?

8

It’s just a plain text file. I don’t know of anything set up for producing a graph. jtop essentially uses the same information that tegrastats puts out and consolidates a lot of information (jtop is recommended). However, directly looking at this graphically probably requires someone to build a script with some tool like paraview or one of the more common visualization packages.

Anyone know of a program to directly view tegrastats graphically?

1 Like
10

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.