Unable to add melanox nic's to ovs dpdk

Getting into below error while we try to add below melanox nic’s to ovs dpdk in rhel 9.2 with openvswitch 3.1.2 and dpdk 22.11.4

pci@0000:12:00.0 ens1f0np0 network MT27710 Family [ConnectX-4 Lx]
pci@0000:12:00.1 ens1f1np1 network MT27710 Family [ConnectX-4 Lx]

The above ones are the ones we are trying to add, getting below error in logs,

2024-04-23T10:59:43.414Z|00119|dpdk|ERR|EAL: Driver cannot attach the device (0000:12:00.0)
2024-04-23T10:59:43.414Z|00120|dpdk|ERR|EAL: Failed to attach device on primary process
2024-04-23T10:59:43.414Z|00121|netdev_dpdk|WARN|Error attaching device ‘0000:12:00.0’ to DPDK
2024-04-23T10:59:43.414Z|00122|netdev|WARN|dpdk-p2: could not set configuration (Invalid argument)
2024-04-23T10:59:43.414Z|00123|dpdk|ERR|Invalid port_id=32

** We tried also to attach to dpdk driver vfio-pci detaching from mlx5_core and then use it with ovs-dpdk → then also getting same error

** By referring to networking - Error attaching device to DPDK - Ask Ubuntu, we tried to directly use with ovs dpdk - even that gives the same error

** Where as on same server we are able to successfully add below intel NIC’s after attaching to vfio-pci and successfully use it with ovs dpdk

pci@0000:37:00.2 ens2f2 network I350 Gigabit Network Connection
pci@0000:37:00.3 ens2f3 network I350 Gigabit Network Connection

** Help is much appreciated - why is the issue seen only with mellanox? are these not supported with rhel9.2

MLX NIC DPDK driver not base on VFIO like INTEL. It base on own MLX PMD.

Pls follow below to build.



Hi @xiaofengl , my nic’s are attached to mlx5_core already

0000:12:00.0 ‘MT27710 Family [ConnectX-4 Lx] 1015’ if=ens1f0np0 drv=mlx5_core unused=vfio-pci
0000:12:00.1 ‘MT27710 Family [ConnectX-4 Lx] 1015’ if=ens1f1np1 drv=mlx5_core unused=vfio-pci

(1) but when I try to add this nic to ovs-dpdk I get below error,

[root@localhost ~]# ovs-vsctl add-port ovs-br1 dpdk-p1 – set Interface dpdk-p1 type=dpdk options:dpdk-devargs=0000:12:00.0
ovs-vsctl: Error detected while setting up ‘dpdk-p1’: Error attaching device ‘0000:12:00.0’ to DPDK. See ovs-vswitchd log for details.
ovs-vsctl: The default log directory is “/usr/local/var/log/openvswitch”.

(2) error logs at /usr/local/var/log/openvswitch/ovs-vswitchd.log are as below

2024-04-30T16:16:26.183Z|00087|dpdk|ERR|EAL: Driver cannot attach the device (0000:12:00.0)
2024-04-30T16:16:26.183Z|00088|dpdk|ERR|EAL: Failed to attach device on primary process
2024-04-30T16:16:26.183Z|00089|netdev_dpdk|WARN|Error attaching device ‘0000:12:00.0’ to DPDK
2024-04-30T16:16:26.183Z|00090|netdev|WARN|dpdk-p1: could not set configuration (Invalid argument)
2024-04-30T16:16:26.183Z|00091|dpdk|ERR|Invalid port_id=32

(3) Host and OVS details: Running on RHEL 9.2
Using open vSwitch 3.1.2 and dpdk 22.11.4

Hi @xiaofengl , are below steps recommended

(1) printf “” > /sys/bus/pci/drivers/mlx5_core/unbind
(2) echo switchdev > /sys/class/net//compat/devlink/mode
(3) printf “” > /sys/bus/pci/drivers/mlx5_core/bind

these creating VF’s and all is required for hw offloading right? it’s not a dependency to use it with ovs-dpdk?