I have several MT25418 (“MT25418 [ConnectX VPI - 10GigE / IB DDR, PCIe 2.0 2.5GT/s]”) adapters that I am trying to get operational in ESX 6.0 in 10G Ethernet mode. Right now I’m trying to get them configured point-to-point but I’ll probably be adding a 10G Ethernet switch at some future point.
How do I make this card recognizable by ESXi 6.0?
What is the combination of firmware/drivers that will get these adapters to be recognized in ESX and allow me to switch from IB mode to Ethernet mode?
Here’s what I have attempted so far:
These HBAs were originally HP-branded HP Infiniband 4X DDR Connect-X PCI-e Dual Port HBA that I flashed to Mellanox-native v2.9.1 firmware on a Windows box.
-
Create a fresh install of ESX 6.0
-
From the VMware compatibility guide the MT25418 requires the “MLNX Native EN 3.2.0.7 NIC driver”, so I download and install the v 3.2.0.7 offline bundle:
esxcli software acceptance set --level=CommunitySupported
esxcli software vib install -d /tmp/MEL-mlnx-3.2.0.7-offline_bundle-3070178.zip
reboot
After reboot I see:
[root@esx1:~] lspci | grep Mellanox
0000:04:00.0 Serial bus controller: Mellanox Technologies MT25418 [ConnectX VPI - 10GigE / IB DDR, PCIe 2.0 2.5GT/s] [vmnic2]
[root@esx1:~]
[root@esx1:~] esxcli network nic list
Name PCI Device Driver Admin Status Link Status Speed Duplex MAC Address MTU Description
vmnic0 0000:02:00.0 e1000e Up Down 0 Half 00:30:48:f6:55:16 1500 Intel Corporation 82574L Gigabit Network Connection
vmnic1 0000:03:00.0 e1000e Up Up 1000 Full 00:30:48:f6:55:17 1500 Intel Corporation 82574L Gigabit Network Connection
[root@esx1:~] esxcli hardware pci list
0000:04:00.0
Address: 0000:04:00.0
Segment: 0x0000
Bus: 0x04
Slot: 0x00
Function: 0x0
VMkernel Name: vmnic2
Vendor Name: Mellanox Technologies
Device Name: MT25418 [ConnectX VPI - 10GigE / IB DDR, PCIe 2.0 2.5GT/s]
Configured Owner: Unknown
Current Owner: VMkernel
Vendor ID: 0x15b3
Device ID: 0x634a
SubVendor ID: 0x15b3
SubDevice ID: 0x0006
Device Class: 0x0c06
Device Class Name: InfiniBand
Programming Interface: 0x00
Revision ID: 0xa0
Interrupt Line: 0x0a
IRQ: 10
Interrupt Vector: 0x2e
PCI Pin: 0x00
Spawned Bus: 0x00
Flags: 0x0201
Module ID: 4109
Module Name: vmklinux_9
Chassis: 0
Physical Slot: 6
Slot Description: Chassis slot 6; function 0
Passthru Capable: false
Parent Device: PCI 0:0:2:0
Dependent Device: PCI 0:4:0:0
Reset Method: Function reset
FPT Sharable: false
[root@esx1:~] esxcli system module list | grep mlx4
nmlx4_core true true
mlx4_core true true
mlx4_en true true
nmlx4_en false true
[root@esx1:~]
As you can see, the nmlx4_en Mellanox Ethernet module isn’t loading. And I suspect it is due to the card defaulting to IB mode somehow somewhere.
How do I make this card recognizable by ESXi 6.0?