I have a setup question about connecting two servers with H200 NVL GPUs.
In the first server, there are two H200 NVL GPUs connected to each other via an NVLink bridge.
In the second server, there are four H200 NVL GPUs: two of them are bridged together, and the other two are also bridged together.
Each server also has one ConnectX-7 OSFP single-port adapter.
I want to connect these two servers directly. Since there will only be a single port between them, I assume I won’t need a switch.
If I connect the servers via the ConnectX adapters, run the Subnet Manager on one of the servers, and establish an InfiniBand network, will the GPUs in one server be able to communicate with the GPUs in the other server over InfiniBand? Would this create a bottleneck?
Do I really need a switch in this case? And if the ConnectX-7 card had two ports instead of one, could I still connect the servers directly without a switch?
Welcome to Nvidia forum. Your question is too complex and very hard to explain in sentences. But overall, in most cases, your topology works “fine” without bottlenecks with communication in the IB link.
Based on your description, each GPU of H200 NVL pair can communicate with each other very well, because nvlink is the best solution for GPU communication. But the true challenges come from the communication between GPU pairs and OSFP cards. If GPU pairs and NIC card are not in the same NUMA node, they will use CPUs, system memory, and the QPI link to transfer data, which is a serious issue in your topology. If you use all of the GPU pairs for inference or training, you will see a very poor performance.
If your task pushes you to invest all your GPU resources in training, a big box, I mean the NVLink model, is a proper choice.
As for your second question, purchasing an IB switch in your topology is not necessary. Just connecting the CX7 200G or 400G back-to-back is enough.