QM8700 and QM8790 interconnect - advice/comments

Infrastructure & Networking Switches and Gateways InfiniBand/VPI Switch Systems
, ,
1

Greetings NVidia folks,

I have the following network configuration, and I am wondering if the network configuration can be improved or what I need to do to better the whole interconnectivity.

   ------------------                       ------------------                       ------------------                       -------------------          
   | **QM8700**'#1  |                       | **QM8790** #1  |                       | **QM8700** #2  |                       | Scratch Storage |          
   | CEPH Storage   |---4x un-split ports---| Cluster Nodes  |---4x un-split ports---| Cluster Nodes  |---6x un-split ports---| BeeGFS NVMe     |          
   | 12 un-split    | [4x2m passive cables] | 12 un-split    | [4x2m passive cables] | 12 un-split    | [6x2m passive cables] | 12 un-split     |          
   | 28 split ports |                       | 28 split ports |                       | 28 split ports |                       | 28 split ports  |          
   |                |                       |                |                       |                |                       |                 |          
   ------------------                       ------------------                       ------------------                       -------------------          


                                         ------------------
                                         | **QM8790** #2  |
                                         | Extra Switch   |
                                         | 12 un-split    |
                                         | 28 split ports |
                                         |                |
                                         ------------------

The “extra” QM8790#2 is not in use now, and I wonder how it can be used to enhance the overall performance of the IB network.
Currently, the switch QM8700#2 is running the Subnet Manager.

Any comments/advice will be greatly appreciated.

Thank you for your attention to this message.

.

2

Hi,

This would largely be dependent on traffic patterns, which are unique to each fabric. For added redundancy you might be able to reconfigure the switches into a fat tree topology utilizing the two QM8790s at the spine layer and QM8700 at the leaf layer. Please see below for some more information regarding fat tree:

Nvidia offers an online topology generator which can assist:

Nvidia also has a professional services organization who can meet with you and optimize your network configuration for it’s specific needs. If interested in pursuing that option, you should reach out to your Nvidia account team.