The WinOF-2 User Manual states that you can assign the IP address via the Network Adapters page under Network Settings in Control Panel in Windows. (I’m using Windows Server 2016.)
However, when I set the ports using WinMFT/mlxconfig to IB (mlxconfig -d mt4115_pciconf0 set LINK_TYPE_P1=1), the device manager then says that it can’t load the driver.
Does this mean that in order to be able to set the IP address in Windows, that the port type MUST be set to ETH (type 2)?
Section 3.1.1 of the WinOF2 user manual says to set the port type to ethernet. Section 3.1.2 sets the IP address.
Section 3.3.1 says that assigning an IP address (which refers back to Section 3.1.2) is supported (under Supported IPoIB features), but I can’t seem to get the WinOF2 driver to work/load properly so that I can use Infiniband with Windows Server 2016 WITHOUT having to change the port type from IB to ETH.
I verified that the there are no issues with the card because I swapped out the OS boot drive with another boot drive and installed SLES12 SP1 and the MLNX OFED linux driver on that and I am able to confirm a “PORT_ACTIVE” state when I run ibv_devinfo in linux.
But when I switch back to Windows Server 2016, it says that it can’t load the driver and I’ve completely uninstalled and reinstalled the driver several times.
Does this mean that the Mellanox ConnectX-4 VPI dual-port card DOESN’T actually support Infiniband given that I can’t set the IP address without changing the port type from IB to ETH? Or is there something else that I need to do to make Windows Server 2016 and the WinOF2 driver work with IB?
You don’t need to set the port type to ETH before setting it to IB. IPoIB should work intact. Not sure I understand what do you mean by “the device manager then says that it can’t load the driver”. Do you mean port status was in “PORT_INITIALIZED” mode? If so the you need to run SM (Subnet Manager) on the fabric.
was the port status in “Port_Armed” state? if it was then you may have a CX-4 firmware issue. You haven’t mention what firmware version you use but Firmware version 12.23.1020 and higher is required for implementing IPoIB you need Firmware version: 14/16.23.1020 and higher is required.
Suggesting that review & follow the instruction of chapter 3.3 (Infiniband Network) in the WinOF-2 v2.0 User manual and configure accordingly.
Use Mellanox WinOF-2 Release Notes Rev 2.0 as well to line up with the IPoIB restrictions (Pkeys usage etc…)
I’m confused by the way you wrote the message in regards to the firmware version.
The card is currently running firmware version 12.17.2050 so I’ll update it to the latest that’s available (12.24.1000) according to Mellanox’s site for firmware downloads for MCX456A-ECAT.
Please confirm that version of firmware will work for IPoIB in Windows Server 2016.
Thank you.
(To your other points though, the device manager (Start → Control Panel → Device Manager) shows that the ConnectX-4 card has a yellow exclamation point next to it, and when I double click on it, it says that the driver failed to start - error 10. I already have a SM running on the fabric, driven by node1 of my cluster, which is running SLES12 SP1. And to make sure that it isn’t a hardware issue, I used another SSD and installed SLES12 SP1 on it and was able to bring up the card/port just fine.)
The diagnostic tools (mst, mlxconfig, mlxfwmanager, etc.) in Windows Server 2016, are all able to see the card (ports are listed as PORT_DOWN state). Like I said, I’ll try updating the firmware first and I’ll report back.