Hi
Can you load an Ethernet driver for these cards? I’ve bought a couple of Sun badged ones and flashed them back to Mellanox firmware. As an IB card it looks fine but I really wanted Ethernet support so I could attach it to an Ethernet switch. There are some drivers in the package but loading them results in a “code 10” within Windows (2012). I’ve tried the 2008r2 drivers with similar results. So just asking if I am being particularly thick here before I troubleshoot more!
Thanks
Drew
Hi Mike,
Thanks but kind of know all you’ve posted already. I’ll see what happens on Windows 28kr2 / Centos I guess.
Thanks
Drew
You need the ConnectX3 Cards for the right firmware to work on Server 2012. The ConnectX2 cards are not supported.
Mike
Thanks for the reply. Of course, not supported does not necessarily mean it won’t work i.e. I’ve got a few Connect X MNEH cards working perfectly fine in Windows 2012.
Anyway, are you saying that these will work fine on Windows 2008R2? I guess I’ll check now.
Hi Justin
Many thanks for the detailed information - really appreciated. I really need my primary machine running under Windows so I guess I need to mess around with W2K8R2 (but its a shame if I have to regress the OS to get it to work - just bought a Myricom 10gb card just in case) but I have a spare card or two so I’m going to brush up on my not very good Centos skills and give your instructions a bash!
Cheers
Drew
Hi~
If you have a ConnectX-2 VPI firmware above 2.9.8350 then you can use it!
If Mellanox support ConnectX-2 EN INTERIM firmware you can use it on WS2k12.
Good luck!
Some of the ConnectX Cards will work, it depends on the firmware, but they are not supported for Windows 2012. Some of the RDMA features won’t/don’t work. I have not used them with Windows Server 2008R2.
With (for example) CentOS 6.4 using the CentOS supplied packages (yum “Infiniband Support”), the MHGH28-XTC cards can run in either Infiniband or native 10GbE mode. By default the cards are in Infiniband mode.
The file used to configure which mode is:
///mlx4
The comments in the file give decent instructions on what to do.
As a helpful reference example, this shows the PCI address for one of my MHGH28-XTC cards in a CentOS 6.4 test box here:
$ | Mellanox
01:00.0 InfiniBand: Mellanox Technologies MT25418 [ConnectX VPI PCIe 2.0 2.5GT/s - IB DDR / 10GigE] (rev a0)
That “01:00.0” address is then used in the mlx4 file as:
0000:01:00.0
After rebooting, the cards show up as native ethernet cards:
$
eth1 Link Ethernet HWaddr 00:03:BA:00:ED:BA
172.16.101.1 Bcast:172.16.101.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets0 errors0 dropped0 overruns0 frame0
TX packets0 errors0 dropped0 overruns0 carrier0
0 1000
RX bytes0 (0.0 b) TX bytes0 (0.0 b)
eth2 Link Ethernet HWaddr 00:03:BA:00:ED:B9
172.16.102.1 Bcast:172.16.102.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets0 errors0 dropped0 overruns0 frame0
TX packets0 errors0 dropped0 overruns0 carrier0
0 1000
RX bytes0 (0.0 b) TX bytes0 (0.0 b)
:
$
2: eth2: mtu 1500 qdisc state UNKNOWN qlen 1000
link/ether 00:03:ba:00:ed:b9 brd ff:ff:ff:ff:ff:ff
172.16.102.1/24 172.16.102.255 scope global eth2
3: eth1: mtu 1500 qdisc state UNKNOWN qlen 1000
link/ether 00:03:ba:00:ed:ba brd ff:ff:ff:ff:ff:ff
172.16.101.1/24 172.16.101.255 scope global eth1
The ethernet configuration of them is then done through the standard files in /etc/sysconfig/network-scripts/.
As a helpful reference these are two working config files (for the MHGH28-XTC card in the above example):
$ cat /etc/sysconfig/network-scripts/ifcfg-eth1
DEVICE=eth1
TYPE=Ethernet
ONBOOT=yes
NM_CONTROLLED=no
BOOTPROTO=none
DEFROUTE=no
PEERDNS=no
PEERROUTES=no
IPV4_FAILURE_FATAL=yes
IPV6INIT=no
NAME=“System eth1”
IPADDR=172.16.101.1
PREFIX=24
:
$ cat /etc/sysconfig/network-scripts/ifcfg-eth2
DEVICE=eth2
TYPE=Ethernet
ONBOOT=yes
NM_CONTROLLED=no
BOOTPROTO=none
DEFROUTE=no
PEERDNS=no
PEERROUTES=no
IPV4_FAILURE_FATAL=yes
IPV6INIT=no
NAME=“System eth2”
IPADDR=172.16.102.1
PREFIX=24
Hope that all gets you up and running.
Just in case anyone had the same issue as me, the Connect-X Windows 7 x64 drivers seem to work fine in Windows Server 2012 using this card. I’ve pinged and copied a few files around so hopefully when I setup my storage it keeps up!
Maybe I downloaded the wrong W2K8R2 ones but even though a comparison of versions were correct in control panel the W2K8R2 drivers were dated 2009 in the program files location whereas the Windows 7 driver set was some time in 2010.
Cheers
Drew