I’ve posted this topic on other forums on the internet with mediocre results, so hopefully coming straight to the source will help me resolve this:
I am setting up a new IB storage network for my home lab and I need help establishing a connection with my Windows Server 2016 box.
The hardware is:
HP DL180 G6 server with Windows Server 2016 x64
HP Infiniband 4X DDR Connect-X PCI-e Dual Port HBA (Part number is 448397-B21 / 452372-001)
Infiniband SFF-8740 cable
Topspin 120 24-port IB switch
I have upgraded the firmware on the Topspin switch to 2.9.0/build 170 and have installed the Mellanox WinOF VPI (v5.35) on my server. I have set the MTU on both ports of the HBA to 2048 to match the switch settings.
The card detects as a Mellanox ConnectX interface card in the OS, but the network adapter appears as “cable unplugged” for both ports. I have used WinOF to configure the ports as IPoIB and as Ethernet just to see if one works over the other, but both are down. Cabling the card do the same card in another Windows box and setting both to Ethernet mode brings up the connection immediately, so the issue is definitely Infiniband related.
mst status shows:
I have flashed the cards from HP to Mellanox firmware 2.9.1 using the following command:
flint -d mt25418_pciconf0 -i fw-25408-2_9_1000-MHGH28-XTC_A2-A3.bin -nofs -allow_psid_change burn
There is a link light on the switch for both connections as well as a link light on the HBA on both ports. The ports on the switch show Up/Up. The ports are set to auto-negotiate and show up at 4x-SDR(10Gbps). When I connect the adapters to the Topspin 120 switch and put them into IPoIB mode, get get a good green light on the adapter and switch, and the amber light is lit on each adapter, indicating a good connection.
In addition, the switch itself shows the two servers as up and active nodes in the IB section of the management console, and the IB HBAs show up as active ports and functional neighbors. So the subnet manager on the switch is functioning as it should, I guess.
So everything seems good and all, but the HBAs still show as Network Cable http://skimlinks.pgpartner.com/mrdr.php?url=http%3A%2F%2Fskimlinks.pgpartner.com%2Fsearch.php%2Fform_keyword%3Dnetwork%2Bcable unplugged within Windows and I am not able to send any traffic through them.
What have I missed? How come the adapters show “cable unplugged” in the OS?