Connect 3 nodes without a switch

Infrastructure & Networking Adapters and Cables InfiniBand/VPI Adapter Cards
1

Hi guys,

I’m trying to configure a network of 3 nodes to comunicate over IB without using a switch.

Each node is equiped with a dual-port ConnectX-4 adapter, so I thought it might be possible to connect them directly with each other.

So this is basically the setup I thought about:

Node1----------Node2----------Node3

The other possible solution would be to setup a ring, so that every node has a direct connection to every other node:

Node1----------Node2----------Node3

|--------------------------------------|

The problem is i coudn’t figure out the correct way to set this up.

What is the correct configuration for the subnetmanager in this setup?

I hope you can help me.

2

Hi Benedikt,

Have you tried to run opesnm -B in a triangle connection?

Does the opensm starts? (sminfo)

Regards,

Sophie.

3

Hi Sophie,

thanks for your reply.

I can start them in a ring, but then I still got 3 different subnetworks.

Is there a way to connect them to one network?

Edit:

I tried setting it up as a triangle setup, but the HCA1 always suppresses the HCA2 Interface.

I always get only the hosts and the subnetmanager from the first Port wit ibhosts or sminfo.

Regards,

Benedikt

4

Hi Benedikt,

Unfortunately, without a switch, this is the expected behavior and can not be changed.

Only through switch(s) the SM can discover all ports and place them under the same subnet.

Regards,

Sophie.