Connect-X 3 Unable to change virtual mac

benjaminbendel · November 18, 2022, 2:03pm

Hi,

I recently deployed some hp servers with hp 546flr-sfp+ adapters. During the deployment the proccess created bond interfaces for the two ports on the card and set the virtual mac address on both ports to the same address. The deployment failed. How do i delete this setting? How do I revert to the physical mac addresses?

root@ubuntu:/home/ahd# mstconfig -d 0000:04:00.0 -e q

Device #1:
----------

Device type:    ConnectX3Pro
Device:         0000:04:00.0

Configurations:                              Default         Next Boot
         SRIOV_EN                            True(1)         True(1)
         NUM_OF_VFS                          16              16
         WOL_MAGIC_EN_P1                     True(1)         True(1)
         WOL_MAGIC_EN_P2                     True(1)         True(1)
         PHY_TYPE_P1                         XFI(2)          XFI(2)
         XFI_MODE_P1                         _10G(0)         _10G(0)
         FORCE_MODE_P1                       False(0)        False(0)
         PHY_TYPE_P2                         XFI(2)          XFI(2)
         XFI_MODE_P2                         _10G(0)         _10G(0)
         FORCE_MODE_P2                       False(0)        False(0)
         LOG_BAR_SIZE                        5               5
         BOOT_OPTION_ROM_EN_P1               True(1)         True(1)
         BOOT_VLAN_EN_P1                     False(0)        False(0)
         BOOT_RETRY_CNT_P1                   0               0
         LEGACY_BOOT_PROTOCOL_P1             PXE(1)          PXE(1)
         BOOT_VLAN_P1                        1               1
         BOOT_OPTION_ROM_EN_P2               True(1)         True(1)
         BOOT_VLAN_EN_P2                     False(0)        False(0)
         BOOT_RETRY_CNT_P2                   0               0
         LEGACY_BOOT_PROTOCOL_P2             PXE(1)          PXE(1)
         BOOT_VLAN_P2                        1               1
         IP_VER_P1                           IPv4(0)         IPv4(0)
         IP_VER_P2                           IPv4(0)         IPv4(0)
         CQ_TIMESTAMP                        True(1)         True(1)
         STEER_FORCE_VLAN                    False(0)        False(0)
Device Firmware does not support reading current configurations

ipavis · November 22, 2022, 7:29am

Hi,

Thank you for contacting us!
To revert back you would need to use the physical Mac Address.
The Mac Address can be found on a sticker, on the adapter itself.

Thanks,
Ilan.

benjaminbendel · November 22, 2022, 7:45am

Hi,
thank you very much.
At what point do I have to set this. In the Bios everything is grayed out and in the OS the virtual Mac is taken for the permanent address and I am unable to change it.

Kind Regards

ipavis · November 22, 2022, 8:30am

Hi,

Please review this document of how to enable SR-IOV on ConnectX-3
https://support.mellanox.com/s/article/howto-configure-sr-iov-for-connectx-3-with-kvm--ethernet-x

You will just need to backtrack on this procedure and you should be able to delete the setting.

Thanks,
Ilan.

benjaminbendel · November 22, 2022, 3:10pm

Hi,

I am very very thankfull for your help. I installed and configured everything like the guide said, but it seems that i miss the step where I am able to configure the mac address.

root@STOR1:/home/ubuntu# mlxfwmanager
Querying Mellanox devices firmware ...

Device #1:
----------

  Device Type:      ConnectX3Pro
  Part Number:      779799-B21_Ax
  Description:      HP Ethernet 10G 2-port 546FLR-SFP+ Adapter
  PSID:             HP_2240110004
  PCI Device Name:  /dev/mst/mt4103_pci_cr0
  Port1 MAC:        e41d2d6db6b0
  Port2 MAC:        e41d2d6db6b1
  Versions:         Current        Available
     FW             2.42.5044      N/A
     CLP            8025           N/A
     PXE            3.4.0754       N/A
     UEFI           14.11.0048     N/A

  Status:           No matching image found

eno49: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet 10.13.0.2  netmask 255.255.0.0  broadcast 10.13.255.255
        inet6 fe80::e61d:2dff:fe6d:b6b0  prefixlen 64  scopeid 0x20<link>
        ether e4:1d:2d:6d:b6:b0  txqueuelen 1000  (Ethernet)
        RX packets 1638  bytes 124540 (124.5 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 1809  bytes 913958 (913.9 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eno49d1: flags=4163<UP,BROADCAST,RUNNING,MULTICAST>  mtu 1500
        inet6 fe80::609:73ff:fed4:6e30  prefixlen 64  scopeid 0x20<link>
        ether 04:09:73:d4:6e:30  txqueuelen 1000  (Ethernet)
        RX packets 238  bytes 22112 (22.1 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 5  bytes 466 (466.0 B)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

eno49d1 is the second port and it still has the wrong mac address.

I am very sorry but would it be possible to give me another hint.

Thank you very much.

benjaminbendel · November 22, 2022, 3:20pm

Hi,

after reboot it looks like this but still has the original interface the wrong mac address.

eno49d1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 04:09:73:d4:6e:30 brd ff:ff:ff:ff:ff:ff
    inet6 fe80::609:73ff:fed4:6e30/64 scope link
       valid_lft forever preferred_lft forever
10: eno49v0: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether 0a:ac:81:0f:25:33 brd ff:ff:ff:ff:ff:ff
11: eno49v1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether a6:67:41:e6:f9:d5 brd ff:ff:ff:ff:ff:ff
12: eno49v2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether e6:ad:3c:ca:e3:b1 brd ff:ff:ff:ff:ff:ff
13: eno49v3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN group default qlen 1000
    link/ether c6:63:c8:51:2d:ba brd ff:ff:ff:ff:ff:ff

root@STOR1:/home/ubuntu# mlxburn -d /dev/mst/mt4103_pciconf0 -vpd

  VPD-KEYWORD    DESCRIPTION             VALUE
  -----------    -----------             -----
Read Only Section:

  PN             Part Number             779799-B21
  EC             Revision                C-5718
  SN             Serial Number           IL2810082M
  V0             Misc Info               Alom 10GbE x8 6W
  V2             N/A                     5810
  V4             N/A                     040973D46E30
  V5             N/A                     0C
  VA             N/A                     HP:V2=MFG:V3=FW_VER:V4=MAC:V5=PCAR
  VB             N/A                     HP ConnectX-3Pro SFP+
  RV             Checksum Complement     0x51
  IDTAG          Board Id                HP Ethernet 10G 2-port 546FLR-SFP+ Adapter

Kind Regards

robertas.didziunaitis · March 23, 2023, 10:32am

What was the deployment package/utility, that set identical Virtual MACs for You ?
I am facing the same problem - got card with two identical Virtual MACs