IS5022 Switch Fabric

Infrastructure & Networking Switches and Gateways InfiniBand/VPI Switch Systems
1

Hello,

I’m very new to infiniband, and looking to reuse some gear that already exists in my organization.

I have 6 servers with dual single port infiniband cards (MHQH19B-XTR), and currently have 1 IS5022 switch. The servers are running IPoIB on CentOS 7 and only using infiniband for IP traffic.

I’d like to remove the switch as a single point of failure, and my first thought was to add a 2nd switch and ‘stack’ them as I would in Ethernet land. From the product overviews on the IS5022 unmanaged switch it looks like its possible to connect another IS5022 switch and connect the 2nd ib adapter of each server, as well as a connection between both IS5022 switches ‘out of the box’.

I would then bond the ib0 and ib1 interfaces on each server, configured in mode 1 (active/backup).

Is this possible ‘out of the box’? At a basic level it looks as though building a mesh topology is one of the primary purposes of infiniband. If so, then my greatest concern is bonding IPoIB.

This is how I see the physical topology (small boxes on each device are ports):

Thanks,

Steve

2

Hi Steve,

The topology looks fine.Since you have IS5022 switches they are “unmanaged” and are not capable of running an SM (subnet manager) so the other step you would need to take is to start opensm on 1 or 2 (if back SM is wanted) of your hosts compute1/2.The SM is responsible for discovering all IB (Infiniband) devices and assisgning a LID to each then setting up all the possible routing paths to each and every deivce and propogating the routes to each swtich in the fabric.Hope this helps.

3

IS5022 is externally managed. I think in your case at least one switch needs to be manageable. But I might be wrong.