ConnectX-4 ASAP2 Failed to connect to devlink Netlink

Hello dear community, I have a problem I’m trying to solve and I’m completely stuck with, I hope someone can give me a clue or has an idea.

The scenario:

I have ConnectX4 cards in my servers and I would like to enable ASAP2 offload with VLANS as outlined in this document:

https://docs.openstack.org/neutron/latest/admin/config-ovs-offload.html

Everything works to the point I do this step:

devlink dev eswitch set pci/0000:81:00.0 mode switchdev

Failed to connect to devlink Netlink

I’ve checked the path: /sys/class/net/enp129s0f1/compat/devlink/mode and indeed there is no such bit.

I’ve also seen this article here which also does the same thing and it obviously works for the author:

https://community.mellanox.com/s/article/getting-started-with-mellanox-asap-2

SRIOV is enabled in the bios and on the card, FW on the card is the latest, kernel is the latest, OS is Centos 7

lspci -nn | grep Mellanox

81:00.0 Ethernet controller [0200]: Mellanox Technologies MT27700 Family [ConnectX-4] [15b3:1013]

81:00.1 Ethernet controller [0200]: Mellanox Technologies MT27700 Family [ConnectX-4] [15b3:1013]

81:00.2 Ethernet controller [0200]: Mellanox Technologies MT27700 Family [ConnectX-4 Virtual Function] [15b3:1014]

81:00.3 Ethernet controller [0200]: Mellanox Technologies MT27700 Family [ConnectX-4 Virtual Function] [15b3:1014]

81:00.4 Ethernet controller [0200]: Mellanox Technologies MT27700 Family [ConnectX-4 Virtual Function] [15b3:1014]

81:01.2 Ethernet controller [0200]: Mellanox Technologies MT27700 Family [ConnectX-4 Virtual Function] [15b3:1014]

81:01.3 Ethernet controller [0200]: Mellanox Technologies MT27700 Family [ConnectX-4 Virtual Function] [15b3:1014]

81:01.4 Ethernet controller [0200]: Mellanox Technologies MT27700 Family [ConnectX-4 Virtual Function] [15b3:1014]

[root@node2 ~]# ip link show dev enp129s0f1

5: enp129s0f1: <BROADCAST,MULTICAST> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000

link/ether 24:8a:07:24:be:2d brd ff:ff:ff:ff:ff:ff

This output is also odd as it’s missing the vf-s despite they are present.

Any idea what am I doing wrong or what should I check?

Thank you very much!

Zoltan

Hi Zoltan,

Please note that VXLAN encapsulation / decapsulation offloading of OVS actions is supported only in ConnectX-4 Lx/ ConnectX-5 adapter cards.

Expected output of lspci

000:00:05.0 : Mellanox TechnologiesMT27710 Family [ConnectX-4 L]

000:00:05.1 : Mellanox TechnologiesMT27710 Family [ConnectX-4 Lx Virtual Function]

Note : From the above output i can see that you are using ConnectX-4.

For more information you refer to the ASAP^2 user manual:

http://www.mellanox.com/related-docs/prod_software/ASAP2_Hardware_Offloading_for_vSwitches_User_Manual_v4.4.pdf

Thanks,

Samer