How can i connect 4 Server with ConnectX3 (dual port FDR) Infiniband without a switch

1

I don’t want to make LAN network, I want to connect some of servers

server01 <= ping => server02

server01 <= ping => server03

server03 <= ping => server04

like BUS network architecture this is semi BUS.

There are some Solutions on the Internet existed but did not work for me.

One of them:

connecting servers together and in the end run the application in a relationship

Server01:

n1_ib0: 10.17.163.201

n1_ib1: 10.17.164.201

Server02:

n2_ib0: 10.17.163.202

n2_ib1: X

Server03:

n3_ib0: 10.17.164.202

n3_ib1: 10.17.165.202

Server04:

n4_ib0: 10.17.165.201

n4_ib1: X

for all nodes gateway: 0.0.0.0 netmask: 255.255.255.0

n1

ib0 (=> to ib0 n2)

ib1 (=> to ib0 n3)

n2

run opensm -g GIUD --daemon

(<= to n1) ib0

(X) ib1

n3

ib0 (=> to ib1 n1)

ib1 (=> to ib0 n4)

n4

run opensm -g GIUD --daemon

(<= to ib1 n3) ib0

(X) ib1

i can connect just one link with opensm service

when i run two “opensm -g” service, all connections will lost

2

Hi arash,

When working with infiniband opensm attaches to a specific IB port on the local machine and configures only the fabric connected to it .

For the described environment you would have to use a switch for communicating all nodes.

Thank you,

Karen.

3

of course you could also go the hard way and run an OpenSM daemon on every peer-to-peer connection. when running your opensm instances, you will need to specify the port name you are binding the OpenSM to.