GPU Memory Usage shows "N/A"

Accelerated Computing CUDA CUDA Setup and Installation
1

Hello, Nvidia team,
I have an NVIDIA GeForce GTX 1080 Ti, I installed Cuda release 10.1, V10.1.105, as well as installed cudnn for Cuda 10.1, But I am facing an error. When I type “nvidia-smi”, GPU Memory Usage shows “N/A”. What can I do? Could you help me to solve this problem, please?

What I have done till now:

  1. The first time, I wanted to use my local GPU on WSL2, there are I also had errors.
  2. Secondly, I tried to use it on Windows, installed it, reinstalled Cuda 11,2, 11,1, 10,2,9.2. In all versions, I am having the same error.

I would be glad for your help. Thank you.

1 Like
2

“N/A” is not an error, it simply means “not available”. Under Windows, with the default WDDM driver model, the operating system manages GPU memory allocations, so nvidia-smi, which queries the NVIDIA driver for the data it displays, doesn’t know anything about the per-process GPU memory usage. In fact with my very recent driver it says outright “Not available in WDDM driver model” instead of “N/A”.

You should be able to see overall GPU memory utilization, though. For example, on my Windows 10 system, simply invoking nvidia-smi displays this:

Tue Feb 23 09:53:18 2021
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 461.40       Driver Version: 461.40       CUDA Version: 11.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name            TCC/WDDM | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Quadro P2000       WDDM  | 00000000:17:00.0 Off |                  N/A |
| 67%   72C    P0    45W /  75W |    432MiB /  5120MiB |    100%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   1  Quadro RTX 4000    WDDM  | 00000000:65:00.0  On |                  N/A |
| 75%   85C    P0    80W / 125W |    774MiB /  8192MiB |     88%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

I haven’t used WSL2 yet, so I don’t know what issues you may have encountered there.

3

nv
nv1090×691 35.2 KB

Hello, @njuffa,
Thank you for your response, but I think in my case it is an error. Here I am uploading my “nvidia-smi” output. If you can see it. All are N/A and the total can be up to only 16% or 14%. I run my training(machine learning calculation), I think for that 14% is less.
Could you look at my attached photo please and say your opinion
Thank you

4

Cap
Cap1153×619 34.5 KB

I also checked in this way, right now I am not running my calculations.

5

The posted output looks exactly as expected on a Windows system. As I said, with the WDDM driver model (which is the default), nvidia-smi has no way of knowing the per-process memory usage. If you invoke nvidia-smi -q instead of nvidia-smi it will actually tell you so by displaying the more verbose “Not available in WDDM driver model” instead of the terse “N/A” needed to fit into the summary format.

Note that nvidia-smi does show the overall GPU memory usage, here 520 MiB / 11264 MB. if you want to track this graphically over time, I would suggest taking a look at TechPowerUp’s GPU-Z utility (free download).

7

Yeap, I got exactly “Not available in WDDM driver model”. Then how can use my GeForce GTX 1080 Ti? Does it mean that I am using GeForce driver or not. To be honest, I am confused.

8

The fact that nvidia-smi cannot display the per-process GPU memory usage has zero impact on the use of the GPU for graphics or compute applications.

If you just want to know which processes currently use the GPU(s), nvidia-smi does provide that information under “Process name”. As an example, this is what it currently shows on my system:

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A     14284      C   ...ore_22.fah\FahCore_22.exe    N/A      |
|    1   N/A  N/A      1464    C+G   Insufficient Permissions        N/A      |
|    1   N/A  N/A      2132    C+G   ...y\ShellExperienceHost.exe    N/A      |
|    1   N/A  N/A      2996    C+G   ...ge\Application\msedge.exe    N/A      |
|    1   N/A  N/A      5904      C   ...ore_22.fah\FahCore_22.exe    N/A      |
|    1   N/A  N/A      7736    C+G   ...bbwe\Microsoft.Photos.exe    N/A      |
|    1   N/A  N/A      8012    C+G   C:\Windows\explorer.exe         N/A      |
|    1   N/A  N/A      8400    C+G   ...ekyb3d8bbwe\YourPhone.exe    N/A      |
|    1   N/A  N/A      8480    C+G   ...nputApp\TextInputHost.exe    N/A      |
|    1   N/A  N/A      9092    C+G   ...artMenuExperienceHost.exe    N/A      |
|    1   N/A  N/A      9988    C+G   ...5n1h2txyewy\SearchApp.exe    N/A      |
|    1   N/A  N/A     15204    C+G   ...SupportAssistClientUI.exe    N/A      |
+-----------------------------------------------------------------------------+
9

power
power1397×898 224 KB

I opened Task Manager to see, right now I am running training calculation.

pow
pow830×532 26.5 KB

If we can see here python.exe “Type” is C. Why? Even though, here device = cuda

run
run890×193 3.47 KB

10

Type “C” means it is a compute app. Which seems very much correct to me.

“G” means it is a graphics app, “C+G” means it is a mixed application using the GPU for both compute and graphics. Many Microsoft GUI components and applications fall into this mixed category.

In my example output above, you can see that both GPU 0 and GPU 1 are running an instance of Folding@Home Core 22, which is a compute app.

From the nvidia-smi documentation:

Type
Displayed as “C” for Compute Process, “G” for Graphics
Process, and “C+G” for the process having both Compute
and Graphics contexts.

11

Thank you very much for help

12

Good evening @njuffa , since morning, I was running my calculation. Here I attached a screen of my PowerShell, if you can see running time is almost 13 hours, and the total GPU usage still 13%, Do you think is it correct and am I using my GPU?
On my code, I checked the device, so on the left side, you can see Running on “cuda”. But why only 13% of usage? I think I am not using GPU on my calculation, is it correct?

cuda_ask_gpu
cuda_ask_gpu1593×811 65 KB

Thank you for your help
Assel

1 Like
13

I have no knowledge how GPU-intensive your application is, but 13% is not close to zero, which is the GPU utilization I would expect if that app weren’t running.

14

I have the same problem, I am running a CNN, but when I start training my model, at the beginning of the first epoch it throws out of memory error I check my GPU usage in task manager but my GPU had not been used at all!! what should I do? please help me!

15

There are a number of reasons for this happening, common ones are:

  1. Your NN is large and contains many millions of parameters
  2. You have a large batch size

If you run model.summary() what do you see? Any layers with millions of parameters are likely causing the issue.

Try simplifying your model, or reducing batch size and see what effect this has.

16

If you’re running a deep learning model, also be aware that part of each batch is loading training data into GPU as well as performing the calculations for each training step.

Data transfer is not CPU intensive so depends how long this takes.