Hi,
I’ve got 2 servers, one FreeNAS 11.3 (FreeBSD) and one Debian 10 and both are using ConnectX-3 (mcx311a-xcat). Both are connected to a switch using tagged vlan.
On the Debian server everything works fine and the ConnectX-3 can send and recieve Unicast/Multicast traffic. But on the FreeNAS host only Unicast works in both direction and the ConnectX-3 can send but not receive CARP packets.
The switch shouldn’t be the problem because if I use a Intel NIC instead of the ConnectX-3 on the FreeNAS host, receiving CARP packets is working fine.
I also tried it with a direct connection (two ConnectX3 connected by DAC) without the switch between and the same problem.
I also flashed the NICs to the latest firmware and swapped them bwtween the servers to verify that this is not a hardware problem.
Are there any known Bugs that could cause this behavior?
Or any settings I need to take care of to allow receiving multicast traffic like CARP on FreeBSD?
Logs:
Feb 23 08:41:45 BM-Homeserver kernel: mlx4_core: Mellanox ConnectX core driver v3.5.1 (April 2019)
Feb 23 08:41:45 BM-Homeserver kernel: mlx4_core: Initializing mlx4_core
Feb 23 08:41:45 BM-Homeserver mlx4_core0: Unable to determine PCI device chain minimum BW
Feb 23 08:41:45 BM-Homeserver kernel: mlx4_en mlx4_core0: Activating port:1
Feb 23 08:41:45 BM-Homeserver mlxen0: Ethernet address: f4:52:14:89:23:b0
Feb 23 08:41:45 BM-Homeserver kernel: mlx4_en: mlx4_core0: Port 1: Using 8 TX rings
Feb 23 08:41:45 BM-Homeserver kernel: mlx4_en: mlx4_core0: Port 1: Using 8 TX rings
Feb 23 08:41:45 BM-Homeserver kernel: mlxen0: link state changed to DOWN
Feb 23 08:41:45 BM-Homeserver kernel: mlxen0: link state changed to DOWN
Feb 23 08:41:45 BM-Homeserver kernel: mlx4_en: mlx4_core0: Port 1: Using 8 RX rings
Feb 23 08:41:45 BM-Homeserver kernel: mlx4_en: mlx4_core0: Port 1: Using 8 RX rings
Feb 23 08:41:45 BM-Homeserver kernel: mlx4_en: mlxen0: Using 8 TX rings
Feb 23 08:41:45 BM-Homeserver kernel: mlx4_en: mlxen0: Using 8 TX rings
Feb 23 08:41:45 BM-Homeserver kernel: mlx4_en: mlxen0: Using 8 RX rings
Feb 23 08:41:45 BM-Homeserver kernel: mlx4_en: mlxen0: Using 8 RX rings
Feb 23 08:41:45 BM-Homeserver kernel: mlx4_en: mlxen0: Initializing port
Feb 23 08:41:45 BM-Homeserver kernel: mlx4_en: mlxen0: Initializing port
Feb 23 08:41:45 BM-Homeserver mlxen0: tso4 disabled due to -txcsum.
Feb 23 08:41:45 BM-Homeserver mlxen0: tso6 disabled due to -txcsum6.
Feb 23 08:41:45 BM-Homeserver kernel: mlxen0: link state changed to UP
Feb 23 08:41:45 BM-Homeserver kernel: mlxen0: link state changed to UP
Feb 23 08:49:59 BM-Homeserver mlx5en: Mellanox Ethernet driver 3.5.1 (April 2019)