NVlink not working RTX 3090 / Ubuntu 16.04

Graphics / Linux Linux
1

I have a system with 3 RTX 3090s running Ubuntu 16.04.
I installed a 4-slot NVLink bridge for RTX 3090 between two of them. However, the system seems not to recognize the link.

Here are the outputs of nvidia-smi:

+-----------------------------------------------------------------------------+
| NVIDIA-SMI 465.19.01    Driver Version: 465.19.01    CUDA Version: 11.3     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  NVIDIA GeForce ...  Off  | 00000000:1F:00.0 Off |                  N/A |
| 30%   32C    P8    22W / 350W |      3MiB / 24268MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   1  NVIDIA GeForce ...  Off  | 00000000:67:00.0 Off |                  N/A |
| 30%   38C    P8    27W / 350W |      3MiB / 24268MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
|   2  NVIDIA GeForce ...  Off  | 00000000:68:00.0 Off |                  N/A |
| 33%   43C    P8    29W / 350W |    261MiB / 24265MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|    2   N/A  N/A      1308      G   /usr/lib/xorg/Xorg                193MiB |
|    2   N/A  N/A      3190      G   compiz                             63MiB |
+-----------------------------------------------------------------------------+

and the output of nvidia-smi nvlink -s:

GPU 0: NVIDIA GeForce RTX 3090 (UUID: GPU-a6b5deee-4f50-21b6-77b1-5fbe2da5f7ad)
         Link 0: <inactive>
         Link 1: <inactive>
         Link 2: <inactive>
         Link 3: <inactive>
GPU 1: NVIDIA GeForce RTX 3090 (UUID: GPU-9aef0a33-de26-7dda-2262-3e570b03dbd9)
         Link 0: <inactive>
         Link 1: <inactive>
         Link 2: <inactive>
         Link 3: <inactive>
GPU 2: NVIDIA GeForce RTX 3090 (UUID: GPU-70a4f649-aa93-7507-73c4-fe00026e15c5)
         Link 0: <inactive>
         Link 1: <inactive>
         Link 2: <inactive>
         Link 3: <inactive>

Is there any setup required after I installed the bridge ? Please let me know if you have any suggestion.

Thanks.

Here is the bug report file:
nvidia-bug-report.log.gz (2.2 MB)

2

Either the bridge is not seated correctly or it doesn’t work with your gpu model. NVLink on 3090s has been a nuisance before:
https://forums.developer.nvidia.com/t/two-dual-geforce-rtx-3090s-and-nvlink-ubuntu-support-at-least-blender-has-support-for-nvlink/160561/11

1 Like