Hi Team,
I am trying to setup SRIOV with ConnectX-3 card using Vmware ESXi 6.0. I have followed these documents on the Mellanox website,
http://www.mellanox.com/related-docs/MFT/MFT_user_manual_4_10_0.pdf http://www.mellanox.com/related-docs/MFT/MFT_user_manual_4_10_0.pdf
http://www.mellanox.com/related-docs/prod_software/Mellanox_OFED_ESXi_User_Manual_v2.4.0.pdf http://www.mellanox.com/related-docs/prod_software/Mellanox_OFED_ESXi_User_Manual_v2.4.0.pdf
to setup SRIOV. I have installed the necessary via files, configured the ports for Ethernet, configured device for SRIOV_EN and NUM_OF_VFS but I don’t see the virtual functions when I execute lspci |grep Mellanox. Can you please advice. Thanks.
/opt/mellanox/bin/mlxconfig -d mt4099_pciconf0 set SRIOV_EN=1 NUM_OF_VFS=16
/opt/mellanox/bin/mlxconfig -d mt4099_pciconf0 set LINK_TYPE_P1=2 LINK_TYPE_P2=2
esxcli system module parameters set -m mlx4_core -p ‘num_vfs=8 port_type_array=2’
[kajens1@host1:/var/log] esxcli software vib list | grep MEL
net-ib-core 2.4.0.0-1OEM.600.0.0.2494585 MEL PartnerSupported 2018-09-17
net-ib-ipoib 2.4.0.0-1OEM.600.0.0.2494585 MEL PartnerSupported 2018-09-17
net-ib-mad 2.4.0.0-1OEM.600.0.0.2494585 MEL PartnerSupported 2018-09-17
net-ib-sa 2.4.0.0-1OEM.600.0.0.2494585 MEL PartnerSupported 2018-09-17
net-mlx-compat 2.4.0.0-1OEM.600.0.0.2494585 MEL PartnerSupported 2018-09-17
net-mlx4-core 2.4.0.0-1OEM.600.0.0.2494585 MEL PartnerSupported 2018-09-17
net-mlx4-en 2.4.0.0-1OEM.600.0.0.2494585 MEL PartnerSupported 2018-09-17
net-mlx4-ib 2.4.0.0-1OEM.600.0.0.2494585 MEL PartnerSupported 2018-09-17
nmlx4-core 3.15.5.5-1OEM.600.0.0.2768847 MEL VMwareCertified 2018-05-06
nmlx4-en 3.15.5.5-1OEM.600.0.0.2768847 MEL VMwareCertified 2018-05-06
nmlx4-rdma 3.15.5.5-1OEM.600.0.0.2768847 MEL VMwareCertified 2018-05-06
nmlx5-core 4.15.8.8-1OEM.600.0.0.2768847 MEL VMwareCertified 2018-05-06
nmst 4.11.0.103-1OEM.600.0.0.2768847 MEL PartnerSupported 2018-12-03
[kajens1@host1:/var/log] esxcli software vib list | grep mft
mft 4.11.0.103-0 Mellanox PartnerSupported 2018-12-03
[kajens1@host1:/var/log] mst status
MST devices:
mt4099_pciconf0
mt4099_pci_cr0
[kajens1@host1:/var/log] mst status -vv
PCI devices:
DEVICE_TYPE MST PCI RDMA NET NUMA
ConnectX3(rev:1) mt4099_pciconf0 01:00.0 net-vmnic5,net-vmnic1000502
ConnectX3(rev:1) mt4099_pci_cr0 01:00.0 net-vmnic5,net-vmnic1000502
[kajens1@host1:/var/log] esxcli network nic list
Name PCI Device Driver Admin Status Link Status Speed Duplex MAC Address MTU Description
vmnic0 0000:1b:00.0 tg3 Up Up 1000 Full 08:94:ef:06:fa:b8 1500 Broadcom Corporation NetXtreme BCM5719 Gigabit Ethernet
vmnic1000502 0000:01:00.0 nmlx4_en Up Up 40000 Full 00:02:c9:35:8f:c1 1500 Mellanox Technologies MT27500 Family [ConnectX-3]
vmnic5 0000:01:00.0 nmlx4_en Up Up 40000 Full 00:02:c9:35:8f:c0 1500 Mellanox Technologies MT27500 Family [ConnectX-3]
[kajens1@host1:/var/log] mlxconfig -d mt4099_pciconf0 q
Device #1:
Device type: ConnectX3
Device: mt4099_pciconf0
Configurations: Next Boot
SRIOV_EN True(1)
NUM_OF_VFS 16
LINK_TYPE_P1 ETH(2)
LINK_TYPE_P2 ETH(2)
LOG_BAR_SIZE 3
BOOT_PKEY_P1 0
BOOT_PKEY_P2 0
BOOT_OPTION_ROM_EN_P1 True(1)
BOOT_VLAN_EN_P1 False(0)
BOOT_RETRY_CNT_P1 0
LEGACY_BOOT_PROTOCOL_P1 PXE(1)
BOOT_VLAN_P1 1
BOOT_OPTION_ROM_EN_P2 True(1)
BOOT_VLAN_EN_P2 False(0)
BOOT_RETRY_CNT_P2 0
LEGACY_BOOT_PROTOCOL_P2 PXE(1)
BOOT_VLAN_P2 1
IP_VER_P1 IPv4(0)
IP_VER_P2 IPv4(0)
CQ_TIMESTAMP True(1)
[kajens1@host1:/var/log] mlxfwmanager --query
Querying Mellanox devices firmware …
Device #1:
Device Type: ConnectX3
Part Number: MCX354A-FCB_A2-A5
Description: ConnectX-3 VPI adapter card; dual-port QSFP; FDR IB (56Gb/s) and 40GigE; PCIe3.0 x8 8GT/s; RoHS R6
PSID: MT_1090120019
PCI Device Name: mt4099_pci_cr0
Port1 MAC: 0002c9358fc0
Port2 MAC: 0002c9358fc1
Versions: Current Available
FW 2.42.5000 N/A
PXE 3.4.0752 N/A
Status: No matching image found
[kajens1@host1:/var/log] lspci |grep Mellanox
0000:01:00.0 Network controller: Mellanox Technologies MT27500 Family [ConnectX-3] [vmnic5]
Thanks,
Kajen Siva