According to NVML release 470 API: NVML API Reference Guide :: GPU Deployment and Management Documentation (nvidia.com), ‘used’ should never be 0 because driver/GPU always sets aside a small amount of memory for bookkeeping.
However, on the following system, the used memory is always 0. Can someone share why this occurs? What’s wrong? Thanks.
GPU: Tesla K80
Driver Version: 470.82.01 CUDA Version: 11.4
GPU load: no process is running
$ nvidia-smi --query-gpu=memory.total,memory.free,memory.used --format=csv,noheader
11441 MiB, 11441 MiB, 0 MiB
NVML code (error checking is removed)
void get_used_memory(void *libhandle, unsigned int gpuid, nvmlDevice_t dev_handle)
{
nvmlMemory_t memoryInfo;
*(void**) (&pfn_nvmlDeviceGetMemoryInfo) = dlsym(libhandle, "nvmlDeviceGetMemoryInfo");
nvmlReturn_t nvmlReturn = pfn_nvmlDeviceGetMemoryInfo(dev_handle, &memoryInfo);
float mib = 1024 * 1024.0;
printf("GPU-%d memory(MiB): total=%d free=%d used=%d\n",
gpuid,
int(memoryInfo.total / mib),
int(memoryInfo.free / mib),
int(memoryInfo.used / mib));
}
results:
GPU-0 memory(MiB): total=11441 free=11441 used=0