Is anyone else around running CentOS 7 x64 and ConnectX (1/2/3) series cards in eth mode?
Asking because after rebooting my CentOS 7 x64 (1708) desktop this evening after a yum update,
the Mellanox ConnectX-2 card in it (set to run in ethernet mode) refused to come up correctly.
That was with the latest kernel (installed yesterday) of kernel-3.10.0-693.17.1.
Instead, it came up in Infiniband mode, with “ip addr” complaining about a potential bad address.
Had a feeling it might be kernel related (something unforeseen from recent Meldown/Spectre
patches maybe?), so tried the previous kernels to see if that’s the cause. Short answer: Yep.
My desktop has these kernels installed at the moment:
- kernel-3.10.0-693.17.1.el7.x86_64 | ConnectX-2 card not working
- kernel-3.10.0-693.11.6.el7.x86_64 | ConnectX-2 card not working
- kernel-3.10.0-693.11.1.el7.x86_64 | ConnectX-2 card works
- kernel-3.10.0-693.el7.x86_64 | ConnectX-2 card works
So, kernel-3.10.0-693.11.6 and onwards are “busted” from this point of view. Reverting to either
of the older two kernels and the card comes up fine, working as 10GbE as expected.
For reference, this is using the CentOS 7 x64 Inbox driver.
The address of the card in my desktop:
$ lspci | grep Mellanox
06:00.0 InfiniBand: Mellanox Technologies MT26428 [ConnectX VPI PCIe 2.0 5GT/s - IB QDR / 10GigE] (rev b0)
And this is the entry for it in /etc/rdma/mlx4.conf telling it to operate in ethernet mode (it’s a single port card):
$ tail -2 /etc/rdma/mlx4.conf
measure. But figured people here would probably be interested/impacted too.