I am trying to get DPDK 17.11.4 to run with a ConnectX-3 card on a virtual environment using Openstack.
This uses the VFIO driver which initialises correctly (though running in no-IOMMU mode). However starting DPDK via the testpmd binary fails trying to add default flows to the device.
$ /root/testpmd -c 0xf -n 4 -w 0000:00:07.0 – --rxq=8 --txq=8
EAL: Detected 12 lcore(s)
EAL: No free hugepages reported in hugepages-1048576kB
EAL: Probing VFIO support…
EAL: VFIO support initialized
EAL: WARNING: cpu flags constant_tsc=yes nonstop_tsc=no → using unreliable clock cycles !
EAL: PCI device 0000:00:07.0 on NUMA socket -1
EAL: Invalid NUMA socket, default to 0
EAL: probe driver: 15b3:1004 net_mlx4
PMD: net_mlx4: PCI information matches, using device “mlx4_1” (VF: true)
PMD: net_mlx4: 1 port(s) detected
PMD: net_mlx4: port 1 MAC address is fa:16:3e:de:c6:4b
USER1: create a new mbuf pool <mbuf_pool_socket_0>: n=171456, size=2176, socket=0
Warning! port-topology=paired and odd forward ports number, the last port will pair with itself.
Configuring Port 0 (socket 0)
PMD: net_mlx4: 0x55a3e1527200: cannot attach flow rules (code 95, “Operation not supported”), flow error type 2, cause 0x17328ec40, message: flow rule rejected by device
Fail to start port 0
Please stop the ports first
I’ve seen this error before on a bare metal setup. this was due to the value of the log_num_mgm_entry_size parameter in the mlx4_core kernel module. Setting this to -1 there worked, but it has no effect in this virtual setup. The bare metal system is running the same Linux kernel and RDMA versions as this Openstack environment.
$ more /sys/module/mlx4_core/parameters/log_num_mgm_entry_size
Tracing the code shows a call to ibv_create_flow() returning the error. Why does this report “Operation not supported” when trying to add flows to the device? Is there something else which is required here to get DPDK to work correctly here?
Thanks for any help. Can provide more info if required.