CUDA version & --gpus argument

I’m following the NVIDIA Container Toolkit user guide one step at a time (Ubuntu 20 LTS). However, I have quickly run into some things that don’t seem right (https://docs.nvidia.com/datacenter/cloud-native/container-toolkit/user-guide.html).

If I execute

docker run --rm --gpus all nvidia/cuda:10.0-base nvidia-smi

I get:

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 510.47.03    Driver Version: 510.47.03    CUDA Version: 11.6     |
...

Notice that it shows CUDA Version 11.6 (the version installed on the host). Shouldn’t it show 10.0?

Also, if I copy-paste from the guide and execute:

sudo docker run --rm --runtime=nvidia -e NVIDIA_VISIBLE_DEVICES=all nvidia/cuda:10.0-base nvidia-smi

I get exactly the same output (full output below). Shouldn’t it show me that the GPU is disabled (or not visible)?

Thanks!

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 510.47.03    Driver Version: 510.47.03    CUDA Version: 11.6     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  Off  | 00000000:03:00.0  On |                  N/A |
| 35%   35C    P5    25W / 260W |    666MiB / 11264MiB |     33%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    0   N/A  N/A      1640      G                                     342MiB |
|    0   N/A  N/A      3095      G                                      54MiB |
|    0   N/A  N/A      3107      G                                      36MiB |
|    0   N/A  N/A      3311      G                                      48MiB |
|    0   N/A  N/A      3528      G                                     139MiB |
|    0   N/A  N/A      7828      G                                      40MiB |
+-----------------------------------------------------------------------------+
``
1 Like