Measure GPU utilization on Jetson Platforms

Robotics & Edge Computing Jetson Systems Jetson Xavier NX
,
1

I’m on Jetson Xavier NX. My goal is to measure a how much GPU is utilized while running my Cuda application (% of SM utilization). One can do that using nvidia-smi dmon on Discrete GPUs:

nvidia-smi dmon -s u -d 1

But nvidia-smi isn’t available on jetson platforms. I’ve tried using jtop and tegrastats but they provide the percentage of time in which the GPU was getting used in the last second. Even if I run a single block with 1 thread to process something for 1 second, they report 100 percent utilization.

And if we go for profiling tools like ncu or nsys, they report the SM utilization in terms of how much time the actual mathematical computation was being done on the cores. So if my kernel is memory bound, it will report very less SM utilization (even though the kernel was utilizing the whole GPU but was mostly moving data rather than computing on cores).

I wanna see how much GPU hardware is actually used for performing my task.
So for example on my jetson Xavier NX (6 SMs) if only 1 block with n number of threads (where n > total available cores in 1 SM) is working on specific task, the GPU utilization should be 100/6 = 16.6% or less.

How can I measure this on jetson platforms?

2

Hello @talha.tahir!

Based on the title and content of your topic, it looks like it may receive better visibility and feedback in a different category. We took the liberty of moving it for you.

If this was an incorrect assessment, please send me a direct message.

Disclaimer: this moderation suggestion and message were generated with AI assistance.

3

Hi, Just putting things simply here; what is the correct way to see GPU utilization for some application involving GPU kernels. For instance if I were to make a simple application that involves GPU kernels and I want to see how much GPU is utilized? How much of the GPU can be used for other application such as running any AI workloads or any downstream tasks.

As @talha.tahir I also see that running even a simple kernels makes the GPU load as per tegrastats or in Jetson Power GUI close to a 100%.

image
image792×825 97.7 KB

5

Hi,

Please use the command below to get the integrated GPU utilization:

$ sudo tegrastats

Thanks

6

As mentioned earier, I’ve already tried using it. It gives me 100 percent usage even when I’m running a cuda kernel using a single thread and block. Here is the example code I’m running

__global__ void vectorAddSingleBlock(const float* A, const float* B, float* C, int N) {

   int tid = threadIdx.x;

int stride = blockDim.x;




   // Grid-stride loop so 1 block and 1 thread processes all N elements

for (int i = tid; i < N; i += stride) {

C[i] = A[i] + B[i];

   }

}
...
...
   dim3 gridDim(1);
   dim3 blockDim(1);

   while (true) {
       // Launch kernel on 1 block and 1 thread
       vectorAddSingleBlock<<<gridDim, blockDim>>>(d_A, d_B, d_C, N);
       // Ensure kernel completes before launching the next iteration
       cudaDeviceSynchronize();
}
...
...

And here are the results of sudo tegrastats:


08-05-2026 12:40:34 RAM 8092/15656MB (lfb 4x4MB) SWAP 0/16020MB (cached 0MB) CPU [0%@729,0%@729,0%@729,0%@729,2%@729,1%@729,3%@729,4%@729] EMC_FREQ 0%@3199 GR3D_FREQ 99%@[1168] NVENC off NVDEC off NVJPG off NVJPG1 off VIC off OFA off NVDLA0 off NVDLA1 off PVA0_FREQ off APE 174 cv0@51.781C cpu@55.656C soc2@50.281C soc0@53.375C cv1@52.968C gpu@55.781C tj@55.781C soc1@52.343C cv2@49C VDD_IN 9455mW/9493mW VDD_CPU_GPU_CV 2745mW/2821mW VDD_SOC 2898mW/2898mW
08-05-2026 12:40:35 RAM 8089/15656MB (lfb 4x4MB) SWAP 0/16020MB (cached 0MB) CPU [4%@729,3%@729,0%@729,2%@729,0%@729,1%@729,0%@729,1%@729] EMC_FREQ 0%@3199 GR3D_FREQ 99%@[1168] NVENC off NVDEC off NVJPG off NVJPG1 off VIC off OFA off NVDLA0 off NVDLA1 off PVA0_FREQ off APE 174 cv0@51.843C cpu@55.937C soc2@50.437C soc0@53.343C cv1@53C gpu@55.843C tj@55.937C soc1@52.375C cv2@49.062C VDD_IN 9455mW/9486mW VDD_CPU_GPU_CV 2745mW/2806mW VDD_SOC 2898mW/2898mW
...
...

As we can see in the output, GR3D_FREQ x@y (x comes out to be 99%). So it’s basically checking in the last second, how much time GPU was geting used. Even if a single core was getting used for an entire 1 second, it gives 100 percent utilization. But in actual I need to know how much GPU fabric was utilized i.e., how many cores or SMs were actually used by my kernel and how much GPU was free and available for other tasks.

7

Hi,

Do you mean the 100% usage is incorrect?

Based on your code, it launch keep launch the kerenl to the GPU.
Even it only use one thread, the code will active GPU cores as much as possible.

Thanks.

8

100 percent reported by tegrastats is not wrong but this is not the metrics I want. tegrastats gives weather the GPU was getting utilized or not in the last 1 second (not the amout of GPU which was getting utilized).

The example code will only use 1 SM (as grid size is 1) and only 1 cuda core (as block size is 1) at a given time. As I’ve used cudaDeviceSynchronize(); at the end of the kernel launch so it will not allow multiple kenrels to launch on the GPU untill the previous kenrel execution is finished.

So at a given time for Jetson Xavier NX, 5 out of 6 SMs will be free and 63/64 cuda cores will be free as well on the SM which is running the kernel.

This is the metrics I want to see i.e., how much GPU SMs and cores are getting utilized out of the all the SMs and cores.