Ethernet mode non-functional with recent CentOS7 kernels and ConnectX-2 cards?

1

Hi all,

Is anyone else around running CentOS 7 x64 and ConnectX (1/2/3) series cards in eth mode?

Asking because after rebooting my CentOS 7 x64 (1708) desktop this evening after a yum update,

the Mellanox ConnectX-2 card in it (set to run in ethernet mode) refused to come up correctly.

That was with the latest kernel (installed yesterday) of kernel-3.10.0-693.17.1.

Instead, it came up in Infiniband mode, with “ip addr” complaining about a potential bad address.

Had a feeling it might be kernel related (something unforeseen from recent Meldown/Spectre

patches maybe?), so tried the previous kernels to see if that’s the cause. Short answer: Yep.

My desktop has these kernels installed at the moment:

  • kernel-3.10.0-693.17.1.el7.x86_64 | ConnectX-2 card not working
  • kernel-3.10.0-693.11.6.el7.x86_64 | ConnectX-2 card not working
  • kernel-3.10.0-693.11.1.el7.x86_64 | ConnectX-2 card works
  • kernel-3.10.0-693.el7.x86_64 | ConnectX-2 card works

So, kernel-3.10.0-693.11.6 and onwards are “busted” from this point of view. Reverting to either

of the older two kernels and the card comes up fine, working as 10GbE as expected.

For reference, this is using the CentOS 7 x64 Inbox driver.

The address of the card in my desktop:

$ lspci | grep Mellanox

06:00.0 InfiniBand: Mellanox Technologies MT26428 [ConnectX VPI PCIe 2.0 5GT/s - IB QDR / 10GigE] (rev b0)

And this is the entry for it in /etc/rdma/mlx4.conf telling it to operate in ethernet mode (it’s a single port card):

$ tail -2 /etc/rdma/mlx4.conf

0000:06:00.0 eth

Note - Filed a bug report on bugs.centos.org 0014419: Mellanox ConnectX cards refuse to work in Ethernet mode with kernel kernel-3.10.0-693.11.6 onwards - CentOS Bug Tracker , and will likely file one on the RedHat Bugzilla too for good

measure. But figured people here would probably be interested/impacted too.

2

Understood Viki. That may have had something to do with why I opened this topic in the Hobby and Home Users Group, where it’s common to use gear sourced from (eg) Ebay and similar.

Regarding the problem itself, it’s been confirmed by RH staff due to someone else reporting the same problem (I guess with a newer card).

A working solution is in the CentOS bug report too, in case that’s of use to people. It also works with the older ConnectX (series 1, not 2) cards, as I also tested with that.

3

Hi Justin,

Please note that the connectX-2 adapter is reached its end of life and end of support period following that it was not certified against the latest kernel versions.

Best Regards,

Viki