Hi, I am using NVML for monitoring of GPU power. My understanding is that for NVML, the source of power measurement is an actual power measurement hardware unit on the GPU. I would like to know what the maximum sampling rate of the power measurement is for some common data center GPUs such as Tesla K40m, K80 and V100. This is not stated on the datasheets or anywhere else.
nvmlDeviceGetPowerUsage in a loop for 100 seconds on a Tesla K40m and counted a total of 274394 samples returned after 100 seconds, however, i don’t think this corresponds to the actual sampling rate of the hardware since the function could be returning the last measured sample for when the hardware power measurement unit is not ready. So, for example, if the actual power sampling rate of Tesla K40m is 50Hz, you’d get one actual measurement every 20ms and if you try to call the function after only 10ms, you’d receive the previously measured power as output. Kindly let me know what you think.