Getting into below error while we try to add below melanox nic’s to ovs dpdk in rhel 9.2 with openvswitch 3.1.2 and dpdk 22.11.4
pci@0000:12:00.0 ens1f0np0 network MT27710 Family [ConnectX-4 Lx]
pci@0000:12:00.1 ens1f1np1 network MT27710 Family [ConnectX-4 Lx]
The above ones are the ones we are trying to add, getting below error in logs,
2024-04-23T10:59:43.414Z|00119|dpdk|ERR|EAL: Driver cannot attach the device (0000:12:00.0)
2024-04-23T10:59:43.414Z|00120|dpdk|ERR|EAL: Failed to attach device on primary process
2024-04-23T10:59:43.414Z|00121|netdev_dpdk|WARN|Error attaching device ‘0000:12:00.0’ to DPDK
2024-04-23T10:59:43.414Z|00122|netdev|WARN|dpdk-p2: could not set configuration (Invalid argument)
2024-04-23T10:59:43.414Z|00123|dpdk|ERR|Invalid port_id=32
** We tried also to attach to dpdk driver vfio-pci detaching from mlx5_core and then use it with ovs-dpdk → then also getting same error
** By referring to networking - Error attaching device to DPDK - Ask Ubuntu, we tried to directly use with ovs dpdk - even that gives the same error
** Where as on same server we are able to successfully add below intel NIC’s after attaching to vfio-pci and successfully use it with ovs dpdk
pci@0000:37:00.2 ens2f2 network I350 Gigabit Network Connection
pci@0000:37:00.3 ens2f3 network I350 Gigabit Network Connection
** Help is much appreciated - why is the issue seen only with mellanox? are these not supported with rhel9.2
Hi @xiaofengl , my nic’s are attached to mlx5_core already
0000:12:00.0 ‘MT27710 Family [ConnectX-4 Lx] 1015’ if=ens1f0np0 drv=mlx5_core unused=vfio-pci
0000:12:00.1 ‘MT27710 Family [ConnectX-4 Lx] 1015’ if=ens1f1np1 drv=mlx5_core unused=vfio-pci
(1) but when I try to add this nic to ovs-dpdk I get below error,
[root@localhost ~]# ovs-vsctl add-port ovs-br1 dpdk-p1 – set Interface dpdk-p1 type=dpdk options:dpdk-devargs=0000:12:00.0
ovs-vsctl: Error detected while setting up ‘dpdk-p1’: Error attaching device ‘0000:12:00.0’ to DPDK. See ovs-vswitchd log for details.
ovs-vsctl: The default log directory is “/usr/local/var/log/openvswitch”.
(2) error logs at /usr/local/var/log/openvswitch/ovs-vswitchd.log are as below
2024-04-30T16:16:26.183Z|00087|dpdk|ERR|EAL: Driver cannot attach the device (0000:12:00.0)
2024-04-30T16:16:26.183Z|00088|dpdk|ERR|EAL: Failed to attach device on primary process
2024-04-30T16:16:26.183Z|00089|netdev_dpdk|WARN|Error attaching device ‘0000:12:00.0’ to DPDK
2024-04-30T16:16:26.183Z|00090|netdev|WARN|dpdk-p1: could not set configuration (Invalid argument)
2024-04-30T16:16:26.183Z|00091|dpdk|ERR|Invalid port_id=32
(3) Host and OVS details: Running on RHEL 9.2
Using open vSwitch 3.1.2 and dpdk 22.11.4
Hi @xiaofengl , are below steps recommended
(1) printf “” > /sys/bus/pci/drivers/mlx5_core/unbind
(2) echo switchdev > /sys/class/net//compat/devlink/mode
(3) printf “” > /sys/bus/pci/drivers/mlx5_core/bind
these creating VF’s and all is required for hw offloading right? it’s not a dependency to use it with ovs-dpdk?
Thanks!