PyTorch utilize CPU instead of GPU

Accelerated Computing CUDA CUDA on Windows Subsystem for Linux
1

When I try to use CUDA for training NN or just for simple calculation, PyTorch utilize CPU instead of GPU

Python 3.8.3 (default, Jun 25 2020, 23:21:14)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> torch.cuda.is_available()
True
>>> device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
>>> device
device(type='cuda', index=0)
>>> tensor = torch.rand(1, 1, 10).to(device)
>>> tensor
tensor([[[0.1126, 0.1737, 0.9678, 0.8833, 0.6923, 0.2118, 0.9874, 0.9397,
          0.4831, 0.4274]]], device='cuda:0')
>>> tensor_two = tensor + tensor
>>> tensor_two
tensor([[[0.2252, 0.3474, 1.9356, 1.7666, 1.3847, 0.4236, 1.9747, 1.8794,
          0.9661, 0.8549]]], device='cuda:0')
>>> while True:
...   tensor_two = tensor + tensor
...

nvidia-smi output under Windows:

Sun Jun 28 10:46:36 2020
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 455.41       Driver Version: 455.41       CUDA Version: 11.1     |
|-------------------------------+----------------------+----------------------+
| GPU  Name            TCC/WDDM | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 1070   WDDM  | 00000000:01:00.0  On |                  N/A |
| 44%   39C    P2    33W / 151W |   1394MiB /  8192MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A         4      C   Insufficient Permissions        N/A      |
|    0   N/A  N/A      1268    C+G   ...lPanel\SystemSettings.exe    N/A      |
|    0   N/A  N/A      1468    C+G   C:\Windows\System32\dwm.exe     N/A      |
|    0   N/A  N/A      3784    C+G   ...bbwe\Microsoft.Photos.exe    N/A      |
|    0   N/A  N/A      8844    C+G   ...5n1h2txyewy\SearchApp.exe    N/A      |
|    0   N/A  N/A      8892    C+G   ...artMenuExperienceHost.exe    N/A      |
|    0   N/A  N/A     10380    C+G   ...ropbox\Client\Dropbox.exe    N/A      |
|    0   N/A  N/A     14504    C+G   ...y\ShellExperienceHost.exe    N/A      |
|    0   N/A  N/A     15984    C+G   ...8bbwe\WindowsTerminal.exe    N/A      |
|    0   N/A  N/A     19380    C+G   ...2txyewy\TextInputHost.exe    N/A      |
|    0   N/A  N/A     28380    C+G   ...b3d8bbwe\WinStore.App.exe    N/A      |
+-----------------------------------------------------------------------------+

Task manager:

wsl-gpu-cpu
wsl-gpu-cpu1322×732 60.8 KB

WSL version:
Linux version 4.19.121-microsoft-standard (oe-user@oe-host) (gcc version 8.2.0 (GCC)) #1 SMP Fri Jun 19 21:06:10 UTC 2020

Ubuntu 20.04 LTS

DxDiag.txt (114.6 KB)

The same code on Ubuntu PC:

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 440.100      Driver Version: 440.100      CUDA Version: 10.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX 108...  Off  | 00000000:01:00.0 Off |                  N/A |
|  0%   46C    P2    63W / 250W |    663MiB / 11178MiB |     19%      Default |
+-------------------------------+----------------------+----------------------+
|   1  GeForce GTX 108...  Off  | 00000000:02:00.0 Off |                  N/A |
|  0%   39C    P8    11W / 250W |     12MiB / 11178MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   2  GeForce GTX 108...  Off  | 00000000:04:00.0 Off |                  N/A |
|  0%   43C    P8    12W / 250W |     12MiB / 11178MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   3  GeForce GTX 108...  Off  | 00000000:05:00.0 Off |                  N/A |
|  0%   43C    P8    13W / 250W |     12MiB / 11178MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
                                                                           
+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID   Type   Process name                             Usage      |
|=============================================================================|
|    0       942      G   /usr/lib/xorg/Xorg                            25MiB |
|    0      1239      G   /usr/bin/gnome-shell                          57MiB |
|    0      5591      C   ...vlad/.pyenv/versions/pytorch/bin/python   567MiB |
+-----------------------------------------------------------------------------+
2 Likes
2

Same thing here. I use a surface book 2, linux kernel 4.19.121, ubuntu and miniconda.
While the GPU is detected by pytorch, it is not used during training.

3 Likes
3

Tensorflow docker container doesn’t utilize GPU too.

Train on 60000 samples
Epoch 1/10
60000/60000 [==============================] - 39s 642us/sample - loss: 0.4938 - accuracy: 0.8272
Epoch 2/10
60000/60000 [==============================] - 27s 447us/sample - loss: 0.3753 - accuracy: 0.8636
Epoch 3/10
60000/60000 [==============================] - 26s 436us/sample - loss: 0.3360 - accuracy: 0.8761
Epoch 4/10
60000/60000 [==============================] - 27s 458us/sample - loss: 0.3117 - accuracy: 0.8863
Epoch 5/10
60000/60000 [==============================] - 33s 547us/sample - loss: 0.2945 - accuracy: 0.8915
Epoch 6/10
60000/60000 [==============================] - 34s 571us/sample - loss: 0.2815 - accuracy: 0.8951
Epoch 7/10
60000/60000 [==============================] - 37s 615us/sample - loss: 0.2700 - accuracy: 0.8998
Epoch 8/10
60000/60000 [==============================] - 38s 626us/sample - loss: 0.2591 - accuracy: 0.9039
Epoch 9/10
60000/60000 [==============================] - 39s 643us/sample - loss: 0.2498 - accuracy: 0.9059
Epoch 10/10
60000/60000 [==============================] - 36s 595us/sample - loss: 0.2404 - accuracy: 0.9105

[W 18:12:36.300 NotebookApp] Notebook tensorflow-tutorials/classification.ipynb is not trusted
[I 18:12:38.682 NotebookApp] Kernel started: 9eb86799-5559-4d34-ba36-7edd178525c4
2020-07-09 18:12:53.102083: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libnvinfer.so.6
2020-07-09 18:12:53.103647: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libnvinfer_plugin.so.6
2020-07-09 18:13:51.119733: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcuda.so.1
2020-07-09 18:13:51.220018: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:967] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2020-07-09 18:13:51.220439: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1555] Found device 0 with properties:
pciBusID: 0000:01:00.0 name: GeForce GTX 1070 computeCapability: 6.1
coreClock: 1.683GHz coreCount: 15 deviceMemorySize: 8.00GiB deviceMemoryBandwidth: 238.66GiB/s
2020-07-09 18:13:51.220576: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2020-07-09 18:13:51.220709: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
2020-07-09 18:13:51.222465: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10
2020-07-09 18:13:51.223247: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10
2020-07-09 18:13:51.225612: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10
2020-07-09 18:13:51.228846: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10
2020-07-09 18:13:51.229013: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2020-07-09 18:13:51.230293: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:967] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2020-07-09 18:13:51.231493: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:967] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2020-07-09 18:13:51.231930: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1697] Adding visible gpu devices: 0
2020-07-09 18:13:51.240228: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 3410010000 Hz
2020-07-09 18:13:51.241999: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x56b3460 initialized for platform Host (this does not guarantee that XLA will be used). Devices:
2020-07-09 18:13:51.242058: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): Host, Default Version
2020-07-09 18:13:51.689421: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:967] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2020-07-09 18:13:51.689939: I tensorflow/compiler/xla/service/service.cc:168] XLA service 0x56465a0 initialized for platform CUDA (this does not guarantee that XLA will be used). Devices:
2020-07-09 18:13:51.689973: I tensorflow/compiler/xla/service/service.cc:176]   StreamExecutor device (0): GeForce GTX 1070, Compute Capability 6.1
2020-07-09 18:13:51.691056: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:967] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2020-07-09 18:13:51.691691: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1555] Found device 0 with properties:
pciBusID: 0000:01:00.0 name: GeForce GTX 1070 computeCapability: 6.1
coreClock: 1.683GHz coreCount: 15 deviceMemorySize: 8.00GiB deviceMemoryBandwidth: 238.66GiB/s
2020-07-09 18:13:51.691838: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2020-07-09 18:13:51.691882: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
2020-07-09 18:13:51.691952: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcufft.so.10
2020-07-09 18:13:51.691996: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcurand.so.10
2020-07-09 18:13:51.692096: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusolver.so.10
2020-07-09 18:13:51.692196: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcusparse.so.10
2020-07-09 18:13:51.692287: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudnn.so.7
2020-07-09 18:13:51.695324: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:967] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2020-07-09 18:13:51.697101: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:967] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2020-07-09 18:13:51.697587: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1697] Adding visible gpu devices: 0
2020-07-09 18:13:51.697738: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcudart.so.10.1
2020-07-09 18:13:52.656161: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1096] Device interconnect StreamExecutor with strength 1 edge matrix:
2020-07-09 18:13:52.656231: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1102]      0
2020-07-09 18:13:52.656282: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1115] 0:   N
2020-07-09 18:13:52.657540: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:967] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2020-07-09 18:13:52.657991: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1324] Could not identify NUMA node of platform GPU id 0, defaulting to 0.  Your kernel may not have been built with NUMA support.
2020-07-09 18:13:52.659207: E tensorflow/stream_executor/cuda/cuda_gpu_executor.cc:967] could not open file to read NUMA node: /sys/bus/pci/devices/0000:01:00.0/numa_node
Your kernel may have been built without NUMA support.
2020-07-09 18:13:52.660162: I tensorflow/core/common_runtime/gpu/gpu_device.cc:1241] Created TensorFlow device (/job:localhost/replica:0/task:0/device:GPU:0 with 6835 MB memory) -> physical GPU (device: 0, name: GeForce GTX 1070, pci bus id: 0000:01:00.0, compute capability: 6.1)
2020-07-09 18:14:10.154011: I tensorflow/stream_executor/cuda/cuda_driver.cc:801] failed to allocate 6.67G (7167590400 bytes) from device: CUDA_ERROR_UNKNOWN: unknown error
2020-07-09 18:14:25.532223: I tensorflow/stream_executor/cuda/cuda_driver.cc:801] failed to allocate 6.01G (6450831360 bytes) from device: CUDA_ERROR_UNKNOWN: unknown error
[I 18:14:38.780 NotebookApp] Saving file at /tensorflow-tutorials/classification.ipynb
2020-07-09 18:14:39.436606: I tensorflow/stream_executor/cuda/cuda_driver.cc:801] failed to allocate 5.41G (5805748224 bytes) from device: CUDA_ERROR_UNKNOWN: unknown error
2020-07-09 18:15:30.640365: I tensorflow/stream_executor/platform/default/dso_loader.cc:44] Successfully opened dynamic library libcublas.so.10
1 Like
4

Tensorflow docker containers should work. Do you mind to describe the exact container you tried to run (the actual docker command line) and how did you determin it wasn’t utilizing the GPU?

5

I ran commands from wsl user guide.
Docker command line:
docker run -it --gpus all -p 8888:8888 tensorflow/tensorflow:latest-gpu-py3-jupyter

CPU/GPU utilization was monitored with task manager and nvidia-smi under Windows.

2 Likes
7

I just ran the above docker command and checked my GPU usage when running the samples, it stayed at 0% throughout:

image
image1099×28 2.16 KB 

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 465.12       Driver Version: 465.12       CUDA Version: 11.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name            TCC/WDDM | Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  GeForce RTX 3080   WDDM  | 00000000:40:00.0 Off |                  N/A |
|  0%   60C    P2   111W / 340W |   9764MiB / 10240MiB |      2%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

I’m hoping to be able to use Pytorch with Cuda on Windows in the future.

Offtopic: When I run the python commands @vvodan posted in the first post I get the same results, pytorch can see the GPU and enables CUDA without actually using it even though I’m not using a docker to run python in. (Just Anaconda) Does this mean I have access to the gpu without running a docker? If not, does that mean I have to use a prebuilt docker image to eventually speed up ML?

–Chris