In the following attached image, I can see 4 A100 connected with NVLINK.
2-A100 paired with 1 set NVLINK.
I would like to know whether all 4 - RTX A6000 GPUs can be connected with NVLINK, to share memory across all 4 GPUs?
Hello @shan_8992, nice to read from you again.
If you look at the A100 specs you will see that PCIe versions of the GPU only support 2 GPUs connected with a physical NVLINK bridge. Of course you can still connect (and share memory of) all four through PCIe, but are limited to PCIe bandwidth.
Only dedicated server boards with different physical interconnect design can support full NVLINK bandwidth for four GPUs. And of course then you would want to look into NVSWITCH as well to enlarge your Multi-GPU setup even further.
But I recommend you talk to your local service provider on the topic, or our enterprise sales channels.
Thanks!
Thanks for your reply @MarkusHoHo
I was checking The Making of the NVIDIA DGX Station - YouTube time 1:40, looks like 4 GPUs are connected through a single NVLINK. I was expecting something like that supported for A6000 GPU too.
Also, thanks for sharing for full bandwidth support for NVLink is supported only on certain boards. Any NVIDIA pages to check supported hardware?
We are using SYS-740GP-TNRT, I would like to know whether we can acheive full bandwidth support by NVLink ?
There are slight differences in Board designs, all suite to specific end-user requirements.
The DGX A100 like the successor DGX H100 are proprietary, ready-made Server designs that put the highest potential performance into a workstation form-factor. After that you will need to go for Data-Center solutions like DGX SuperPODs.
The A6000 targets a different audience where high-speed multi-GPU usage is not the foremost priority.
And regarding the Server you mention, I believe it is PCIe based, so you will only be able to use max 2x2 NVLINK connections, not 4x.
1 Like
@MarkusHoHo
Currently, I use GPU Operator to setup device plugin/driver/monitoring GPUs.
Once we add NVLink do we need to perform any driver/hardware-related operation to enable NVLink to usage?
Hi, since NVlink is a HW function, the NVlink-speed of Ampere-architecure GPUs is limited to third gen, as spec’d in the table you reference.
Only Hopper GPUs have an even faster NVlink…
-Frank
ok got it, thank you.
That I cannot answer. I asked the main Repo contributors directly, let’s see if they can help us out here.
1 Like