Comparing GPU Utilization Metrics between nvidia-smi and tegrastats on Jetson Xavier

[lui.feola]

Hello NVIDIA Community,

I am currently working on a project using the Jetson Xavier and I’m encountering some challenges in comparing GPU utilization metrics across different monitoring tools. Specifically, I’m trying to understand how the GR3D_FREQ readings (both X and Y) from tegrastats relate to the utilization.gpu [%] and utilization.memory [%] metrics provided by nvidia-smi.

Here’s what I’ve been doing:

• I use the following nvidia-smi command to monitor the GPU and memory usage:

nvidia-smi --query-gpu=timestamp,gpu_name,memory.total,memory.used,memory.free,utilization.gpu,utilization.memory,pstate,compute_mode,temperature.gpu --format=csv -l 5

• Simultaneously, I monitor the output from tegrastats to get GR3D_FREQ X and Y metrics.

My question is: How can I effectively compare these metrics to get a comprehensive understanding of the GPU’s performance and utilization? Are there any known scaling factors or conversion methods to align these metrics between the two tools? Additionally, is there a recommended approach or tool that might offer a unified view of these metrics?

Any insights or suggestions would be greatly appreciated as I navigate these monitoring tools to better understand and optimize my system’s performance.

Thank you in advance for your help!

[AastaLLL]

Hi,

Here is the document for your reference:
https://docs.nvidia.com/jetson/archives/r35.4.1/DeveloperGuide/text/AT/JetsonLinuxDevelopmentTools/TegrastatsUtility.html#reported-statistics

The GR3D_FREQ X%@[Y1,Y2] value indicates the GPU active percentage.
It can be used to represent the GPU utilization percentage.

For memory, please check RAM X /Y (lfb NxZ).
Since Jetson’s physical memory is shared by CPU and GPU, it is a system-wise memory usage that is different from the nvidia-smi’s output.

Thanks.

[lui.feola]

Hi @AastaLLL ,

Thank you for your response!

To ensure I fully understand your explanation, could you confirm if the utilization.memory [%] metric from nvidia-smi is directly comparable with the X% value in the GR3D_FREQ X%@[Y1,Y2] format from tegrastats? I want to make sure I am interpreting the relationship between these metrics correctly for accurate performance analysis.

Looking forward to your clarification!

[AastaLLL]

Hi,

GR3D_FREQ is the utilization.gpu.
RAM can be considered as utilization.memory.

But Jetson RAM value contains all the system usage including CPU and GPU.
nvidia-smi only reports the GPU memory usage.

Thanks.