Hello, I am using the 4-slot RTX NVLINK bridge along with two RTX 3090 cards. In both Windows and Linux, it seems that it’s not quite working (with CUDA 11.8). On Ubuntu 20.04, driver 520.61.05, nvidia-smi nvlink seems to indicate that the NVLink connections are present but down. The p2pBandwidt…

I have no experience with NVlink, but this Reddit thread has some comments regarding mixed PCIe width and also BIOS support for SLI.

Thanks. I’ve seen several similar threads over the last couple of years, pointing inconclusively to: a lack of motherboard PCIe bandwidth - minimum PCIe 3.0 x8 for all cards that are to be using NVLink. a lack of motherboard “SLI support”, meaning that the BIOS contains a certificate of the kind …

Another data point: This support query on a Supermicro X10DAI motherboard shows that despite having a pair of x16 slots, it is not SLI capable. That a serious, non gaming orientated motherboard, is not SLI capable, perhaps lends some credence to your reference above to Nvidia certification. Maybe m…

Right. This is a crucial “gotcha” or point of confusion; my use case has nothing to do with SLI or gaming. I only care about NVLink, and the cards come with that. For another motherboard example, the ASUS Pro WS WRX80E-SAGE SE supports 7x PCIe 4.0 x16 slots and has “4-way SLI” support, but again …

Yes, it does seem a bit of a mess. The few instances I’ve come across of people successfully using a pair of 3090s with NVlink under Windows, have required settings labeled “SLI”, to be enabled for it to work. For more mental gymnastics, check out the Supermicro X10DAX , which is SLI capable: “Suppo…

Yep, it’s almost as if the cost of discovering this the fun way is priced into those other offerings!

You’re running a card on PCIe 2.0 x4 (dual direction speed of 1.8GB/s) Your test returns unidirectional bandwidth between devices of : 1.28 and 1.43 GB/s what is output for nvlink -s is not what you can utilise with your configuration, because the data fed to the gpu on 2.0 x4 still needs to have…

RTX 3090 + NVLink + CUDA P2P - not working on Linux or Windows, in different ways?

Accelerated Computing CUDA CUDA Programming and Performance

louismichaelhowe May 24, 2023, 11:01pm 9

Well, I’m happy to report that it looks like it’s working as expected now with two RTX 3090s, at least on Ubuntu 22.04, driver 530.30.02. This is on a motherboard with no SLI support.

1 Like

Topic		Replies	Views
Making work NVLink on Windows 10 with dual 3090 CUDA Developer Tools	0	2629	April 13, 2021
Issue with P2P connection using two RTX A4500 CUDA Programming and Performance cuda , ubuntu	7	2730	March 31, 2023
P2p Bandwidth 150% higher than maximum achievable CUDA Programming and Performance cuda , ubuntu	10	3196	April 11, 2023
NV-Link Setup Troubleshooting and NV-Link Status Output Help CUDA Setup and Installation	7	13110	April 13, 2023
How to enable P2P access? CUDA Setup and Installation cuda	3	5315	February 6, 2023
Compatibility of NVLink bridges OptiX	4	3825	June 14, 2022
Two/Dual GeForce RTX 3090s and NVLink: Ubuntu support? At least Blender has support for NVLink Linux cuda , ubuntu	14	11800	April 16, 2023
Ubuntu - NVLink not working with two RTX 3090 GPU - Hardware	5	2091	February 6, 2023
Low P2P GPU bandwidth performance between GeForce GPUs CUDA Programming and Performance	20	1987	October 9, 2024
Does Titan RTX support P2P access w/o NVLink? CUDA Programming and Performance	9	4088	December 15, 2019

RTX 3090 + NVLink + CUDA P2P - not working on Linux or Windows, in different ways?

Related topics