When I invoke cudaMemGetInfo() I get the following:
It seems that quite a large share of memory is unavailable for allocation. Is it always this high?
I’m running a RTX2070 max-Q 8GB GPU on a laptop. I’m using the CPU’s integrated graphics driver for the display.
Is there any way I can free up more memory?
Thank you
I am guessing this is a Windows 10 machine with a WDDM driver. If so, this is normal. Windows 10 uses the WDDM 2.x driver model, under which only 81±1% of the GPU memory is available to user apps. The WDDM 1.x driver model used by Windows 7 was more generous, making 96±1% of the GPU’s memory available to user apps.
With the WDDM driver model, the OS is in control of the GPU memory. The OS provides for virtualization and oversubscription of that memory. CUDA makes memory allocations via the OS. I am not aware of any workarounds. The available percentage of GPU memory could be higher when running under Linux, although I have not checked that recently.