Per the doc, cuda driver only provides some apis for allocating/freeing/mapping/unmapping/exporting/importing virtual memory. Are there any apis we can use to get the statistics for virtual memory? e.g. total virtual memory allocated in this device, and high watermark. These statistics are available for memory pools.
If I understand correctly, there are four classes of cuda memory allocation:
cudaMalloc
cudaMallocAsync
, from a memory poolcudaMallocAsync
under graph capture , from an internal graph memory poolcuMemCreate
, for virtual memory allocation
For 2 and 3, cuda runtime provides several apis to get the statistics. And we also have cudaMemGetInfo
to get the total number of memory used. But it is not easy to tell the usage from 1 and 4.