According to latest online NVML API Reference Guide :: GPU Deployment and Management Documentation (nvidia.com)), the nvmlMemory_v2_t::used amount also includes the nvmlMemory_v2_t::reserved amount.
However, based on my testing, the ‘used’ amount doesn’t include the ‘reserved’ amount. Can someone confirm this too? Thanks.
GPU type: V100-SXM2-32GB
GPU Load: No process is running.
Driver Version: 510.85.02 CUDA Version: 11.6
$ nvidia-smi --query-gpu=memory.total,memory.free,memory.used,memory.reserved --format=csv,noheader
32768 MiB, 32510 MiB, 0 MiB, 257 MiB
NVML code: (error handling is removed)
void get_used_memory_v2(void *libhandle, unsigned int gpuid, nvmlDevice_t dev_handle)
{
nvmlMemory_v2_t memoryInfo;
memoryInfo.version = nvmlMemory_v2;
*(void**) (&pfn_nvmlDeviceGetMemoryInfo_v2) = dlsym(libhandle, "nvmlDeviceGetMemoryInfo_v2");
nvmlReturn_t nvmlReturn = pfn_nvmlDeviceGetMemoryInfo_v2(dev_handle, &memoryInfo);
float mib = 1024 * 1024.0;
printf("GPU-%d memory_v2(MiB): total=%d free=%d used=%d reserved=%d\n",
gpuid,
int(memoryInfo.total / mib),
int(memoryInfo.free / mib),
int(memoryInfo.used / mib),
int(memoryInfo.reserved / mib));
}
results:
GPU-0 memory_v2(MiB): total=32768 free=32510 used=0 reserved=257