GPU Memory Usage shows "N/A"

asel.kembay · February 23, 2021, 1:17pm

Hello, Nvidia team,
I have an NVIDIA GeForce GTX 1080 Ti, I installed Cuda release 10.1, V10.1.105, as well as installed cudnn for Cuda 10.1, But I am facing an error. When I type “nvidia-smi”, GPU Memory Usage shows “N/A”. What can I do? Could you help me to solve this problem, please?

What I have done till now:

The first time, I wanted to use my local GPU on WSL2, there are I also had errors.
Secondly, I tried to use it on Windows, installed it, reinstalled Cuda 11,2, 11,1, 10,2,9.2. In all versions, I am having the same error.

I would be glad for your help. Thank you.

njuffa · February 23, 2021, 5:57pm

“N/A” is not an error, it simply means “not available”. Under Windows, with the default WDDM driver model, the operating system manages GPU memory allocations, so nvidia-smi, which queries the NVIDIA driver for the data it displays, doesn’t know anything about the per-process GPU memory usage. In fact with my very recent driver it says outright “Not available in WDDM driver model” instead of “N/A”.

You should be able to see overall GPU memory utilization, though. For example, on my Windows 10 system, simply invoking nvidia-smi displays this:

Tue Feb 23 09:53:18 2021
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 461.40       Driver Version: 461.40       CUDA Version: 11.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name            TCC/WDDM | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Quadro P2000       WDDM  | 00000000:17:00.0 Off |                  N/A |
| 67%   72C    P0    45W /  75W |    432MiB /  5120MiB |    100%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   1  Quadro RTX 4000    WDDM  | 00000000:65:00.0  On |                  N/A |
| 75%   85C    P0    80W / 125W |    774MiB /  8192MiB |     88%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

I haven’t used WSL2 yet, so I don’t know what issues you may have encountered there.

asel.kembay · February 24, 2021, 12:41am

Hello, @njuffa,
Thank you for your response, but I think in my case it is an error. Here I am uploading my “nvidia-smi” output. If you can see it. All are N/A and the total can be up to only 16% or 14%. I run my training(machine learning calculation), I think for that 14% is less.
Could you look at my attached photo please and say your opinion
Thank you

asel.kembay · February 24, 2021, 12:46am

I also checked in this way, right now I am not running my calculations.

njuffa · February 24, 2021, 12:49am

The posted output looks exactly as expected on a Windows system. As I said, with the WDDM driver model (which is the default), nvidia-smi has no way of knowing the per-process memory usage. If you invoke nvidia-smi -q instead of nvidia-smi it will actually tell you so by displaying the more verbose “Not available in WDDM driver model” instead of the terse “N/A” needed to fit into the summary format.

Note that nvidia-smi does show the overall GPU memory usage, here 520 MiB / 11264 MB. if you want to track this graphically over time, I would suggest taking a look at TechPowerUp’s GPU-Z utility (free download).

asel.kembay · February 24, 2021, 12:50am

Yeap, I got exactly “Not available in WDDM driver model”. Then how can use my GeForce GTX 1080 Ti? Does it mean that I am using GeForce driver or not. To be honest, I am confused.

njuffa · February 24, 2021, 12:55am

The fact that nvidia-smi cannot display the per-process GPU memory usage has zero impact on the use of the GPU for graphics or compute applications.

If you just want to know which processes currently use the GPU(s), nvidia-smi does provide that information under “Process name”. As an example, this is what it currently shows on my system:

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A     14284      C   ...ore_22.fah\FahCore_22.exe    N/A      |
|    1   N/A  N/A      1464    C+G   Insufficient Permissions        N/A      |
|    1   N/A  N/A      2132    C+G   ...y\ShellExperienceHost.exe    N/A      |
|    1   N/A  N/A      2996    C+G   ...ge\Application\msedge.exe    N/A      |
|    1   N/A  N/A      5904      C   ...ore_22.fah\FahCore_22.exe    N/A      |
|    1   N/A  N/A      7736    C+G   ...bbwe\Microsoft.Photos.exe    N/A      |
|    1   N/A  N/A      8012    C+G   C:\Windows\explorer.exe         N/A      |
|    1   N/A  N/A      8400    C+G   ...ekyb3d8bbwe\YourPhone.exe    N/A      |
|    1   N/A  N/A      8480    C+G   ...nputApp\TextInputHost.exe    N/A      |
|    1   N/A  N/A      9092    C+G   ...artMenuExperienceHost.exe    N/A      |
|    1   N/A  N/A      9988    C+G   ...5n1h2txyewy\SearchApp.exe    N/A      |
|    1   N/A  N/A     15204    C+G   ...SupportAssistClientUI.exe    N/A      |
+-----------------------------------------------------------------------------+

asel.kembay · February 24, 2021, 1:08am

I opened Task Manager to see, right now I am running training calculation.

If we can see here python.exe “Type” is C. Why? Even though, here device = cuda

njuffa · February 24, 2021, 1:48am

Type “C” means it is a compute app. Which seems very much correct to me.

“G” means it is a graphics app, “C+G” means it is a mixed application using the GPU for both compute and graphics. Many Microsoft GUI components and applications fall into this mixed category.

In my example output above, you can see that both GPU 0 and GPU 1 are running an instance of Folding@Home Core 22, which is a compute app.

From the nvidia-smi documentation:

Type
Displayed as “C” for Compute Process, “G” for Graphics
Process, and “C+G” for the process having both Compute
and Graphics contexts.

asel.kembay · February 24, 2021, 3:16am

Thank you very much for help

asel.kembay · February 24, 2021, 3:01pm

Good evening @njuffa , since morning, I was running my calculation. Here I attached a screen of my PowerShell, if you can see running time is almost 13 hours, and the total GPU usage still 13%, Do you think is it correct and am I using my GPU?
On my code, I checked the device, so on the left side, you can see Running on “cuda”. But why only 13% of usage? I think I am not using GPU on my calculation, is it correct?

Thank you for your help
Assel

njuffa · February 24, 2021, 3:32pm

I have no knowledge how GPU-intensive your application is, but 13% is not close to zero, which is the GPU utilization I would expect if that app weren’t running.