I have 2 servers with 2 Infiniband ConnectX-5 single port EDR adapters in each server. All ports are connected back-to-back using IB EDR copper cable. Using ibstat command, both ports in each server are detected by the OS (Ubuntu Linux 18.04) with port state status showing INITIALIZING.
Upon running OpenSM on one of the servers, the port on the first adapter (ib0) on both server changes to become ACTIVE state. However, the other port on the second adapter on both servers (ib1) remains at INITIALIZING state. How can the state for the second port also be made ACTIVE? Do we need to launch another instance of OPENSM for this purpose?
One SM must be running for each InfiniBand subnet.
Opensm attaches to a specific IB port on the local machine and configures only the fabric connected to it. (If the local machine has other IB ports, opensm will ignore the fabrics connected to those other ports). If no port is specified, opensm will select the first “best” available port.
To start an opensm service on port2 you can run “opensm -B -g <2nd port guid>”
Another option is to create a config file. This method will start an opensm on port2 on every boot:
Go to /etc/opensm/ directory
Run “opensm --create-config opensm.conf”
Change the GUID value
Restart the opensm service via “/etc/init.d/opensmd restart”