Workaround for vlan management issue - bond vs promisc

Infrastructure & Networking Adapters and Cables Ethernet Adapter Cards
1

Continuing the discussion from ConnectX-5 in Proxmox loses connectivity to management interface on VLAN trunk. Promic on fixes it:

(upgrade of proxmox to 9.2.11 and firware upgrade did not fix)

Working fix by using a bond:

auto ens1f0np0
iface ens1f0np0 inet manual

auto bond0
iface bond0 inet manual
bond-slaves ens1f0np0
bond-miimon 100
bond-mode active-backup

auto vmbr0
iface vmbr0 inet manual
bridge-ports bond0
bridge-stp off
bridge-fd 0
bridge-vlan-aware yes
bridge-vids 2-4094

auto vmbr0.16
iface vmbr0.16 inet static
address 192.168.1.50/24
gateway 192.168.1.1

To bypass the Mellanox hardware bug completely, you need to implement the Single-Port Bond wrapper mentioned earlier. This safely hides the physical interface from Proxmox’s VLAN-aware logic, forcing the host to handle the VLAN trunk in software.

2

Hi Greg,
Glad you were able to find a workaround and thanks for sharing.

In promiscuous mode the NIC passes all the traffic that it receives to the kernel, while in non-promiscuous mode the NIC drops any frame with destination MAC address different than NIC’s MAC address, or broadcast/multicast.

Are we using a compatible or recent DOCA OFED driver? Make sure this is ok first.
We could tcpdump and confirm the incoming traffic’s dMAC and vlan tag as a sanity check.
Are we seeing any ethtool rx_steer_missed_packets (or any other drops) increase during the issue?
Otherwise as a test, explicitly config the bridge’s MAC (hwaddress), or try moving management off vmbr0.16 to ens17f0np0.16

Thanks.
Regards,
Eric