How to keep guid of HCA in vsphere  SR-IOV

Infrastructure & Networking Adapters And Cables InfiniBand/VPI Adapter Cards
#1

Hi, im my environment, the server is running ESXI 6.2 and the vm is rhel 7.2。 I cut the PF into 4 V like below

[root@localhost:~] lspci |grep MT

0000:82:00.0 Network controller: Mellanox Technologies MT27500 Family [ConnectX-3] [vmnic4]

0000:82:00.1 Network controller: Mellanox Technologies MT27500 Family [ConnectX-3 Virtual Function] [PF_0.130.0_VF_0]

0000:82:00.2 Network controller: Mellanox Technologies MT27500 Family [ConnectX-3 Virtual Function] [PF_0.130.0_VF_1]

0000:82:00.3 Network controller: Mellanox Technologies MT27500 Family [ConnectX-3 Virtual Function] [PF_0.130.0_VF_2]

0000:82:00.4 Network controller: Mellanox Technologies MT27500 Family [ConnectX-3 Virtual Function] [PF_0.130.0_VF_3]

then the vm use one of the VF, like:

[root@db1 ~]# ibstatus

Infiniband device ‘mlx4_0’ port 1 status:

default gid: fe80:0000:0000:0000:0014:0500:0000:0001

base lid: 0x8

sm lid: 0x4

state: 4: ACTIVE

phys state: 5: LinkUp

rate: 56 Gb/sec (4X FDR)

link_layer: InfiniBand

Infiniband device ‘mlx4_0’ port 2 status:

default gid: fe80:0000:0000:0000:0014:0500:0000:0005

base lid: 0xb

sm lid: 0x4

state: 4: ACTIVE

phys state: 5: LinkUp

rate: 56 Gb/sec (4X FDR)

link_layer: InfiniBand

once the sever and IB switch lose power ,the guid of the HCA in vm changes 。

Last login: Fri Mar 17 21:26:54 2017 from 192.168.0.26

[root@db1 ~]# ibstatus

Infiniband device ‘mlx4_0’ port 1 status:

default gid: fe80:0000:0000:0000:0014:0500:0000:0002

base lid: 0x8

sm lid: 0x4

state: 4: ACTIVE

phys state: 5: LinkUp

rate: 56 Gb/sec (4X FDR)

link_layer: InfiniBand

Infiniband device ‘mlx4_0’ port 2 status:

default gid: fe80:0000:0000:0000:0014:0500:0000:0001

base lid: 0xb

sm lid: 0x4

state: 4: ACTIVE

phys state: 5: LinkUp

rate: 56 Gb/sec (4X FDR)

link_layer: InfiniBand

does somebody encounter the same situation? welcome to discuss

my email is 578155689@qq.com mailto:578155689@qq.com

#2

Hi,aviap:

thanks for your response. Do U try the ESXI server and Switch both lose power? The one of them lose power may not cause vm guid change。

and I tried two vm in my lab ，one not change，one did .Maybe it’s probabilistic event.

U can have a try. thank u

#3

basically, if VM is binded to one VF only then “default GID” which derives from the base-port GUID should not change and have the same value back after switch/server reboot

Have you checked if the VM is still under the same “original” VF after switch reboot or for some reason, it may have shifted to another VF?

suggesting to approach to support@mellanox.com mailto:support@mellanox.com and open a support ticket to further investigate this