I am also stuck with this. Basically, the same output is received. Any Google search just drops me back here.
First of all, I think we should have to get dpdk0
to work first properly. Then, we can focus on adding the corresponding VF. Sometimes the memory (i.e., hugepage) can indeed be a problem. Verify first that you have enough FREE hugepages.
To ensure this, allocate more hugepages to the OS than you would use for OVS. I did this:
bf2@host1# echo 12288 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
bf2@host1# mountpoint -q /dev/hugepages || mount -t hugetlbfs nodev /dev/hugepages
Something eats up a lot from them, but there are at least 4096 which I will assign to OVS:
bf2@host1# cat /proc/meminfo |grep -i ^Huge
HugePages_Total: 6797
HugePages_Free: 6669
HugePages_Rsvd: 0
HugePages_Surp: 0
Hugepagesize: 2048 kB
Hugetlb: 13920256 kB
I managed to do one thing, though. Or at least I realized something when playing around with parameters.
Eventually, I can add the physical port without any issue if and only if I disable hw-offload in Open vSwitch. But, adding the corresponding VF to be able to send back and forth to the host is still failing.
So, I set the folling to OvS when initializing:
ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init=true
ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-lcore-mask=0xff
ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-socket-mem="4096"
ovs-vsctl --no-wait set Open_vSwitch . other_config:hw-offload=false
Then, I start ovs-vswitchd
manually (ovsdb-server
and ovs-vsctl
run already)
ovs-vswitchd unix:/var/run/openvswitch/db.sock --pidfile --log-file=/var/run/openvswitch/cslev-ovs-vswitchd.log --detach
Add bridge and ports, and set dpdk params:
ovs-vsctl add-br ovs_dpdk_br0 -- set bridge ovs_dpdk_br0 datapath_type=netdev
ovs-vsctl set Open_vSwitch . other_config:dpdk-extra="-a 0000:03:00.0,representor=[0,65535]"
ovs-vsctl --no-wait add-port ovs_dpdk_br0 dpdk0 -- set Interface dpdk0 type=dpdk -- set Interface dpdk0 options:dpdk-devargs=0000:03:00.0
But, when I add the VF via this command
ovs-vsctl --no-wait add-port ovs_dpdk_br0 dpdk1 -- set Interface dpdk1 type=dpdk -- set Interface dpdk1 options:dpdk-devargs=0000:03:00.0,representor=[0,65535]
it fails:
Port dpdk1
Interface dpdk1
type: dpdk
options: {dpdk-devargs="0000:03:00.0,representor=[0,65535]"}
error: "Error attaching device '0000:03:00.0,representor=[0,65535]' to DPDK"
The vswitchd logfile says this:
2023-08-31T09:28:09.060Z|00191|dpdk|INFO|EAL: Probe PCI driver: mlx5_pci (15b3:a2d6) device: 0000:03:00.0 (socket -1)
2023-08-31T09:28:09.061Z|00192|dpdk|ERR|mlx5_net: Cannot retrieve PCI address of IB device mlx5_3
2023-08-31T09:28:09.062Z|00193|netdev_dpdk|WARN|Error attaching device '0000:03:00.0,representor=[0,65535]' to DPDK
2023-08-31T09:28:09.062Z|00194|netdev|WARN|dpdk1: could not set configuration (Invalid argument)
It seems arguments are invalid.
Thanks