ConnectX-6 OVS DPDK ERROR

Hi,

I am testing OVS with DPDK with Nvidia ConnectX-6 LX 25Gbps. I am using below versions installed from ubuntu APT repo.

OVS 2.17.7
DPDK 21.11.4
Network Card: Mellanox Technologies MT2894 Family [ConnectX-6 Lx]
Ubuntu 20.04
Kernel: 5.4.0-170-generic
CPU: 2x Intel Gold 6430
RAM: 1TB

ethtool -i ens6f0

driver: mlx5_core
version: 5.0-0
firmware-version: 26.35.3502 (MT_0000000531)
expansion-rom-version:
bus-info: 0000:a8:00.0

I have initialized dpdk and enabled hw offload.

ovs-vsctl set Open_vSwitch . "other_config:dpdk-init=true
ovs-vsctl set Open_vSwitch . other_config:hw-offload=true

I have bind the PF with dpdk.

ovs-vsctl show

097aff34-b65e-4228-a1b7-eba4a30ac670
Bridge br0
datapath_type: netdev
Port dpdk-p0
Interface dpdk-p0
type: dpdk
options: {dpdk-devargs=“0000:a8:00.0”}
Port mgmt0
Interface mgmt0
type: internal
Port br0
Interface br0
type: internal
ovs_version: “2.17.7”

I have made this setup between two physical compute nodes and trying to test the performance via iperf3. I see below errors in ovs logs continuously.

2024-01-29T12:51:04.851Z|00119|netdev_offload_dpdk(hw_offload5)|WARN|dpdk-p0: rte_flow creation failed: 1 ((null)).
2024-01-29T12:51:04.851Z|00120|netdev_offload_dpdk(hw_offload5)|WARN|dpdk-p0: Failed flow: flow create 4 ingress priority 0 group 0 transfer pattern eth src is d8:9e:f3:bf:97:0a dst is 01:80:c2:00:00:0e type is 0x88cc has_vlan is 0 / end actions count / drop / end

Here are the results of iperf3.

iperf3 -c 10.1.1.1 --bidir

Connecting to host 10.1.1.1, port 5201
[ 5] local 10.1.1.2 port 36304 connected to 10.1.1.1 port 5201
[ 7] local 10.1.1.2 port 36314 connected to 10.1.1.1 port 5201
[ ID][Role] Interval Transfer Bitrate Retr Cwnd
[ 5][TX-C] 0.00-1.00 sec 447 MBytes 3.75 Gbits/sec 33 881 KBytes
[ 7][RX-C] 0.00-1.00 sec 473 MBytes 3.97 Gbits/sec
[ 5][TX-C] 1.00-2.00 sec 441 MBytes 3.70 Gbits/sec 9 847 KBytes
[ 7][RX-C] 1.00-2.00 sec 407 MBytes 3.41 Gbits/sec
[ 5][TX-C] 2.00-3.00 sec 469 MBytes 3.93 Gbits/sec 10 901 KBytes
[ 7][RX-C] 2.00-3.00 sec 466 MBytes 3.91 Gbits/sec
[ 5][TX-C] 3.00-4.00 sec 459 MBytes 3.85 Gbits/sec 9 933 KBytes
[ 7][RX-C] 3.00-4.00 sec 441 MBytes 3.70 Gbits/sec
[ 5][TX-C] 4.00-5.00 sec 494 MBytes 4.14 Gbits/sec 16 953 KBytes
[ 7][RX-C] 4.00-5.00 sec 408 MBytes 3.42 Gbits/sec
[ 5][TX-C] 5.00-6.00 sec 488 MBytes 4.09 Gbits/sec 2 973 KBytes
[ 7][RX-C] 5.00-6.00 sec 473 MBytes 3.97 Gbits/sec
[ 5][TX-C] 6.00-7.00 sec 462 MBytes 3.88 Gbits/sec 2 964 KBytes
[ 7][RX-C] 6.00-7.00 sec 407 MBytes 3.42 Gbits/sec
[ 5][TX-C] 7.00-8.00 sec 491 MBytes 4.12 Gbits/sec 3 1003 KBytes
[ 7][RX-C] 7.00-8.00 sec 468 MBytes 3.93 Gbits/sec
[ 5][TX-C] 8.00-9.00 sec 486 MBytes 4.08 Gbits/sec 4 1005 KBytes
[ 7][RX-C] 8.00-9.00 sec 473 MBytes 3.97 Gbits/sec
[ 5][TX-C] 9.00-10.00 sec 470 MBytes 3.94 Gbits/sec 8 1.05 MBytes
[ 7][RX-C] 9.00-10.00 sec 475 MBytes 3.98 Gbits/sec


[ ID][Role] Interval Transfer Bitrate Retr
[ 5][TX-C] 0.00-10.00 sec 4.60 GBytes 3.95 Gbits/sec 96 sender
[ 5][TX-C] 0.00-10.00 sec 4.59 GBytes 3.95 Gbits/sec receiver
[ 7][RX-C] 0.00-10.00 sec 4.39 GBytes 3.77 Gbits/sec 380 sender
[ 7][RX-C] 0.00-10.00 sec 4.39 GBytes 3.77 Gbits/sec receiver

Even line rate cannot be achieved via dpdk, which I am easily achieving via kernel interfaces (without ovs).

Also with new LTS drivers i.e 5.4 and 5.8, I am even not able to bind NIC via dpdk

ovs-vsctl add-port br0 dpdk-p0 – set Interface dpdk-p0 type=dpdk options:dpdk-devargs=0000:a8:00.0

The error say “Could Not set configuration” “invalid port_id=32” in ovs logs.

Any suggestions ?

Ammad Ali

Pls follow below guide build DPDK with MLX5 PMD.

https://doc.dpdk.org/guides/platform/mlx5.html#mlx5-common-compilation

https://doc.dpdk.org/guides/nics/mlx5.html#limitations

This topic was automatically closed 14 days after the last reply. New replies are no longer allowed.