Error with configuring OVS-DPDK on Bluefiled-2

Infrastructure & Networking BlueField
1

I am trying to configure ovs-dpdk on Bluefield-2 in embedded mode to offload the flows on it according the Configuring OVS-DPDK Offload with BlueField-2 document (Mellanox Interconnect Community).
after executing the commands and restarting the openvswitch, the openvswitch status shows this errors:

|00018|dpdk|EMER|Unable to initialize DPDK: Invalid argument
ovs-vswitchd: Cannot init EAL (Invalid argument)
ovs|00002|daemon_unix|ERR|fork child died before signaling startup (killed (signal 6), core dumped)
ovs|00003|daemon_unix|EMER|could not initiate process monitoring

Also, it the ovs-vsctl show:

Bridge ovs_dpdk_br0
datapath_type: netdev
Port ovs_dpdk_br0
Interface ovs_dpdk_br0
type: internal
Port dpdk1
Interface dpdk1
type: dpdk
options: {dpdk-devargs=“0000:03:00.0,representor=[0,65535]”}
error: “Error attaching device ‘0000:03:00.0,representor=[0,65535]’ to DPDK”
Port dpdk0
Interface dpdk0
type: dpdk
options: {dpdk-devargs=“0000:03:00.0”}
error: “could not add network device dpdk0 to ofproto (Resource temporarily unavailable)”
ovs_version: “2.17.7-e054917”

The “systemctl restart openvswitch” command shows this error:
Failed to restart openvswitch.service: Unit openvswitch.service not found.

Are these problems related together?

I didn’t find any solution for this error. Would you please help me with this.

Also, the document didn’t mention anything about binding the interfaces to DPDK drivers. Is it necessary or not?

Thank you all

2

There is no straight answer(s)for these errors, further troubleshooting/validation needs to take place.
Though, it could be very well related to ovs-switchd having no or not enough memory allocated/reserved for vswitch to use (IE: hugepages).

I would make sure latest DPU image/FW are in place
I would verified that configuration steps were properly followed (enabling ovs-dpdk hardware offload) https://docs.nvidia.com/networking/display/BlueFieldDPUOSLatest
I would make sure to configure hugepages (grep -sri HugePages_ /proc/meminfo)
Maybe test first DPDK via testpmd on the DPU to make sure that DPDK functions as expected

Check the syslog, messages, ovs-vswitchd.log files for errors or pointers.

Useful link:
https://enterprise-support.nvidia.com/s/article/Basic-Debug-utilities-with-OVS-DPDK-offload-ASAP-Direct

systemctl status ovs-vswitchd -l

Lots to check and validate to narrow down and isolate the issue.

3

I tried everything and check all the points that you mentioned but nothing worked. OVS still cannot initialize dpdk and cannot attach the ports to the bridge.

4

I am also stuck with this. Basically, the same output is received. Any Google search just drops me back here.
First of all, I think we should have to get dpdk0 to work first properly. Then, we can focus on adding the corresponding VF. Sometimes the memory (i.e., hugepage) can indeed be a problem. Verify first that you have enough FREE hugepages.
To ensure this, allocate more hugepages to the OS than you would use for OVS. I did this:

bf2@host1# echo 12288 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
bf2@host1# mountpoint -q /dev/hugepages || mount -t hugetlbfs nodev /dev/hugepages

Something eats up a lot from them, but there are at least 4096 which I will assign to OVS:

bf2@host1# cat /proc/meminfo  |grep -i ^Huge
HugePages_Total:    6797
HugePages_Free:     6669
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
Hugetlb:        13920256 kB

I managed to do one thing, though. Or at least I realized something when playing around with parameters.
Eventually, I can add the physical port without any issue if and only if I disable hw-offload in Open vSwitch. But, adding the corresponding VF to be able to send back and forth to the host is still failing.

So, I set the folling to OvS when initializing:

ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-init=true
ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-lcore-mask=0xff
ovs-vsctl --no-wait set Open_vSwitch . other_config:dpdk-socket-mem="4096"
ovs-vsctl --no-wait set Open_vSwitch . other_config:hw-offload=false

Then, I start ovs-vswitchd manually (ovsdb-server and ovs-vsctl run already)

ovs-vswitchd unix:/var/run/openvswitch/db.sock --pidfile --log-file=/var/run/openvswitch/cslev-ovs-vswitchd.log --detach

Add bridge and ports, and set dpdk params:

ovs-vsctl add-br ovs_dpdk_br0 -- set bridge ovs_dpdk_br0 datapath_type=netdev
ovs-vsctl set Open_vSwitch . other_config:dpdk-extra="-a 0000:03:00.0,representor=[0,65535]"
ovs-vsctl --no-wait add-port ovs_dpdk_br0 dpdk0 -- set Interface dpdk0 type=dpdk -- set Interface dpdk0 options:dpdk-devargs=0000:03:00.0

But, when I add the VF via this command

ovs-vsctl --no-wait add-port ovs_dpdk_br0 dpdk1 -- set Interface dpdk1 type=dpdk -- set Interface dpdk1 options:dpdk-devargs=0000:03:00.0,representor=[0,65535]

it fails:

Port dpdk1
            Interface dpdk1
                type: dpdk
                options: {dpdk-devargs="0000:03:00.0,representor=[0,65535]"}
                error: "Error attaching device '0000:03:00.0,representor=[0,65535]' to DPDK"

The vswitchd logfile says this:

2023-08-31T09:28:09.060Z|00191|dpdk|INFO|EAL: Probe PCI driver: mlx5_pci (15b3:a2d6) device: 0000:03:00.0 (socket -1)
2023-08-31T09:28:09.061Z|00192|dpdk|ERR|mlx5_net: Cannot retrieve PCI address of IB device mlx5_3
2023-08-31T09:28:09.062Z|00193|netdev_dpdk|WARN|Error attaching device '0000:03:00.0,representor=[0,65535]' to DPDK
2023-08-31T09:28:09.062Z|00194|netdev|WARN|dpdk1: could not set configuration (Invalid argument)

It seems arguments are invalid.

Thanks