Hi Nvidia, I’m working on a VR ray tracing title and wanted to add support for NV Link to boost performance for remote cloud rendering situations, but it appears that even your professional GPU lineup no longer has the NV Link connector:
Is NV Link truly a dead technology? I mostly wanted it to double the path tracing performance in VR, but I’m wondering to what degree NV Link is even necessary or beneficial for such workloads, as the two eyes can be rendered completely independently. In that case I may be better off simply buying two 4090s for this situation. Any caveats there?
NVLink is far from dead. If you check out the Information on our pages you will see that the technology is still in use and developing.
The fact that it is not supported on Desktop or Workstation GPUs any longer does not mean the technology is discontinued.
And basically you are correct, NVLink is most beneficial if you need to transfer huge amounts of data between GPUs, like Deep Learning tasks in HPC settings. On the other hand, for VR applications, even if there were a need to transfer whole frames between GPUs, PCIe 4 offers enough bandwidth even for 4k at 120Hz and then some.
So if your engine supports Multi-GPU rendering like alternate frames, you should be fine with a 4090. But you should take into account that Desktop GPUs are not designed for any kind of server setups.
this is a very-very unfortunate decision of Nvidia, we are scientists and fully utilizing NVlink in our duo-RTX A6000 setup for machine learning.
So, basically, no possibility for us to move to A6000 Ada L. Gen., and pool all of their 96Gb combined memory together, to train GANs, transformers, or just big ANNs of other types?
What would you recommend then? How with limited resources can we build a similar ML-PC with a big enough video RAM on board, but most importantly - pooled together?
Welcome @alex1988d to the NVIDIA developer forums.
I cannot say much about any decisions or the possible impact of it. But of course there are alternatives.
The simplest being to just make use of PCIe. While not as fast as NVLINK it still allows to utilize CUDA across multiple devices the same way that NVLINK does.
I can imagine that buying a full-fledged DGX H100 is beyond the budget of smaller companies or educational institutions. For that there are alternatives like Cloud service providers, for example the NVIDIA partner Cyxtera. Check the DGX page for DGX as a service.
Thanks for your reply Markus,
We have never tried to pool both GPUs VRAM together using PCIe x16 Gen4, but my guess is even if it works without tweaking much - it would slow down the learning rate significantly.
Cloud services we can’t use for 2 reasons - 1. data leaks and 2. constant modifications of ANNs with iterative development - in other words, we need a machine that stands just here and we are not limited by time of learning and/or amount of tasks, like in AWS or alternatives offer.
Yes, DGX is too expensive, but it would be an ideal variant for us of course.
We need smth like it was before - 2 GPUs 5k$ each + 5k$ for the rest of the machine and we have a nice 15 000$ rig with 96Gb VRAM.
I guess we will just stay with RTX A6000 further as no alternatives so far.
The next step of the upgrade, despite the price, would be dual Hopper H100 PCIe, as it was promised they do harbor NVlink.
Best,
Alex
I agree with @alex1988d, the Cloud GPU service can never be an alternative to NVLINK. I think NVidia removed that feature thinking about the general consumer market. But also, there are specific applications and setup of the GPUs where faster data transmission in between GPUs are crucial. NVidia could still keep the NVLink feature as Pro version.
@alex1988d and @_Bi2022 I had the same concerns as I wanted multiple GPUs NVLinked together for working with large models. Although NVIDIA’s DGX systems are ridiculously overpriced, you should check out Lambdalabs and Bizon-tech, they build custom multi-GPU desktop workstation machines and also servers. I have a workstation from Bizon-tech with multiple NVLinked GPUs, it works really well and is a fraction of the cost of NVIDIA’s DGX systems.
Ya but those use older generation Ampere or Turing RTX cards, right? Lovelace is too good to give up, at least for my purposes which is ray tracing. For ML those older cards in NVLink are probably a good value. Since 2x 3090s gives you 48gb of VRAM total (addressable as a contiguous block, if I understand it correctly).
I am hoping that the ability to have larger than 48 GB amounts of VRAM available on a ADA GPU [or ability to be pooled from several cards] will occur for use on a local workstation. Additionally, it does seem that a PCIe 5 bus will be needed going forward.[ not just the GPU].
Agree with @alex1988d on this. In my field, 3D and VFX NVLink has been a critical way to pool VRAM for large scenes. I currently run multiple 4090’s, but often have to pay for an expensive render farm service that still has 3090’s when I need more VRAM. Despite the exorbitant cost of the RTX 6000 Ada, it only has 48GB of VRAM with no NVLink. Thus, there is no practical solution for VRAM pooling. This will push us back to using CPUs, which are much slower but have no VRAM limitation with local systems. A great deal of 3D/VFX work is iterating on designs, simulations, and animations, which is best (or only possible) on a local machine. Given that NVLink is still actively developed and used, dropping it from consumer and workstation GPUs was a huge letdown.
Removing NVLink (or not including it) for the Ada RTX 6000 is likely because two of them linked together with full nvlink speeds would cause them to rival the H100, which would cut the entry price for H100-level performance in half. This would not maintain suffiicient segmentation.
Obviously this is why they did it, to make more money, however, I was told indirectly from Nvidia reps that VR SLI is indeed supported on Ada 6000, even without an NVLink connector, and transfers data across the PCIe bus to the other GPU at the end of each frame. It’s a driver-level solution for something you can do manually, with some elbow grease.
But, it does hurt the value of the upgrade proposition to not have double the addressable VRAM like true NVLink offers. Still, if all you need is FPS, for a gaming scenario, then doubling the VRAM from 24 to 48GB is likely not a major advantage in the sense that you can’t design your games around 48GB (or even 24GB), so even NVLink doubling the VRAM wouldn’t be much benefit, probably.
PCI-e 4 link is not available on consumer level cards including the RTX 4090. You have no p2p shared memory pool whatsoever. Everything is to the CPU and back again.
The professional cards (RTX 5000+) can pool memory in pcie-4 which is still very fast.
Do NOT buy a bunch of 4090’s for deep learning. That would be a truly disastrous purchase decision.
Yea, so let me get this straight.
You removed NVLink Support…
Forced a brand new year 2024 AI GPU to utilize year 2016 PCI-e gen 4 bus speeds…
Knowing PCI-e Gen 6 Exists, and knowing that Pooled Memory is desired for AI Processing.
And released the DGX H100, which is basically an NVlink, Mainboards at the same time?
Hello @peterjfrancoiii and a warm welcome to the NVIDIA developer forums!
GeForce GPUs have always been targeted at Gamers and in that area SLI has been deprecated for a long time. If you can play Crysis at more than 100 FPS on an integrated (AMD) GPU, who needs more than one discrete GPU in their PC?
And for actual high performance needs we rather use NVLINK which is so much faster than SLI ever was. At the same time adding NVLINK hardware bridges to consumer GeForce GPUs that will not be used in 99% of end-user scenarios does not seem like a good idea since it would make the GPUs also more expensive to the customers.
Out of curiosity, do you know which AMD consumer GPUs support Multi-GPU OpenCL through Crossfire or even PCIe?
The computer tasks wanting to explore are simply not practical and or perhaps, even possible, to proceed with doing, given the current capabilities of the GPU. Until one can either have in a single card or be able to easily pool the memory [ one contiguous /usable block of memory to e.g. load a model] of several cards so that the total amount of of memory is in the range of 96 GB [ to start] and these cards are on a PCIe-5 bus, one is hesitant to be configuring new worksatations, Fortunately, there is plenty of useful work to be done that does not require the above level of compute power. So this is the problem solve for this matter for now. One appreciates, to an extent understands, that there are alternatives for solving this challenge now. But it seems, all of them also markedly increase the total cost of ownership. Given the two paths, opting for the status quo, for the moment, seems reasonable and wise.
As someone who’s roots are in the VFX Industry I think this is a massive mistake. I’ve been a big fan of NVIDIA and and based a lot of my experience in utilising CUDA in this industry.
I get the market segmentation for AI workloads, you don’t want a cheaper high powered GPU that can have NVLink to take the competition away from H100s etc. But what about people in VFX, Graphics, Graphics + ML ?
We can’t pool memory with P2P, we have huge scenes with different graphical representations to render using the power of two RTX 6000s and now we have to fit them into either GPU’s 48GB VRAM. It’s just sad.
It’s not like there is any ‘more expensive’ option as a backup. If we could afford a H100, sure it has more memory, but it’s architecture is designed for AI workloads, not graphics + AI/ML, which is something that was so great about the A/RTX 6000 series cards.
I see people on this thread basically showing the reason why this was done, because they know 2x RTX 6000s would make cheaper inference and/or training GPU’s with a model parallel memory split. This is annoying, but it would be great if NVIDIA could remember it’s roots, in graphics, graphics + ML workloads. Otherwise I might aswell just use 2x 4090’s sure I don’t get P2P, but P2P and and extra 24GB of memory (at lower bandwidth, sure it shaves some TDP), 10 theoretical TFLOP/S of fp16/32 difference for a lower clocked GPU is not worth the massive price difference.
Just to put it in perspective the 4090 retailed in 2022 for $1599 the RTX 6000 for $6799…
There is a real need for NVlink for desktop workstation scientific applications as well.
Not every researcher has the money to spring for cloud time or to buy a H100 machine.
Nvidia is trying to push people who use Gromacs or NAMD molecular dynamics software onto its licensed cloud providers to make more money.
Unfortunately, this is what the industry is heading towards. That’s ok, I can buy graphics and compute cards from another chip provider now.
No NVlink on the desktop means Nvidia will lose out on sales.
Nvidia is getting away from its roots. It’s important to remember that scientists first started thinking and programming graphics cards for computational purposes before there was CUDA for the stupid data centers. Data centers aren’t the only market, but probably the ones that make the most money.