I have few questions about nvml’s utilization.
I’m currently working with nvmlDeviceGetProcessUtilization, nvmlDeviceGetUtilizationRates APIs.
nvidia-smi -l 1 & nvidia-smi pmon -d 1 -o T
By this command, I wanted to get device utilization and process utilization during same sampling period
But I found some strange result.
Thu Jul 4 21:25:12 2024
±----------------------------------------------------------------------------------------+
| NVIDIA-SMI 550.54.14 Driver Version: 550.54.14 CUDA Version: 12.4 |
|-----------------------------------------±-----------------------±---------------------+
| GPU Name Persistence-M | Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap | Memory-Usage | GPU-Util Compute M. |
| | | MIG M. |
|=========================================+========================+======================|
| 0 NVIDIA GeForce GTX 1660 Ti Off | 00000000:01:00.0 On | N/A |
| 43% 33C P5 12W / 120W | 921MiB / 6144MiB | 2% Default |
| | | N/A |
±----------------------------------------±-----------------------±---------------------+
21:25:12 0 1148 G - - - - - - Xorg
21:25:12 0 1295 G 12 4 - - - - gnome-shell
21:25:12 0 2739 G - - - - - - chrome --type=g
21:25:12 0 4460 G - - - - - - Code --gpu-pref
21:25:12 0 23521 G - - - - - - Slack --gpu-pre
I thought that device utilization value >= sum of process utilization values
But result above : device utilization value < sum of process utilization values
Question 1) Is sampling period of device utilization and process utilization are is different?
Official NVML document says that device utilization’s sampling period is 1/6~1sec, but i couldn’t find sampling period info about process utilization. Are they the same?
Question 2) If they are the same, do the timestamps for the measurements match? In other words, do the measurements start and end at the same time?
Question 3) Is there any way to check nvmlDeviceGetUtilizationRates’s result’s timestamp? In case of nvmlDeviceGetProcessUtilization, it is able to check each sample data’s recorded timestamp.
I hope I could get clear answer from NVML developers. Thank you :)