DPDK procedure for 1M/sec flow insertion rate

Infrastructure & Networking Software And Drivers Application Accelerator Software
1

The DPDK mlx5 guide lists as a feature “Flow insertion rate of more then million flows per second, when using Direct Rules”. I would appreciate any information on a test setup that supports this rate.

I have both ConnectX-5 and ConnectX-6 NICs. Using the dpdk-test-flow-perf program from DPDK 24.11.1 running on Ubuntu 24.04 with doca-roce-2.9.1-0.1.9, I see a maximum of 350K inserts/sec.

Given there is a potential 3x speedup, it would be very helpful to hear what hardware, driver, DPDK command, and DPDK code used to insert 1M/sec. That might guide me (and others) to determine if my issue is hardware (maybe only certain cards can reach that rate?) or software (driver or DPDK).

2

Hi brian

You should Use HW steering to get max performance:
HW steering:

  • WQE based high scaling and safer flow insertion/destruction.
  • Set dv_flow_en to 2 in order to enable HW steering.
  • Async queue-based rte_flow_async APIs supported only.
  • NIC ConnectX-5 and before are not supported.
  • Reconfiguring flow API engine is not supported. Any subsequent call to rte_flow_configure() with different configuration than initially provided will be rejected with -ENOTSUP error code.
  • Partial match with item template is not supported.
  • IPv6 5-tuple matching is not supported.
  • With E-Switch enabled, ports which share the E-Switch domain should be started and stopped in a specific order:
    • When starting ports, the transfer proxy port should be started first and port representors should follow.
    • When stopping ports, all of the port representors should be stopped before stopping the transfer proxy port.If ports are started/stopped in an incorrect order, rte_eth_dev_start()/rte_eth_dev_stop() will return an appropriate error code:
    • -EAGAIN for rte_eth_dev_start().
    • -EBUSY for rte_eth_dev_stop().
  • Matching on ICMP6 following IPv6 routing extension header, should match ipv6_routing_ext_next_hdr instead of ICMP6. IPv6 routing extension matching is not supported in flow template relaxed matching mode (see struct rte_flow_pattern_template_attr::relaxed_matching).
  • The supported actions order is as below:

MARK (a) *_DECAP (b) OF_POP_VLAN COUNT | AGE METER_MARK | CONNTRACK OF_PUSH_VLAN MODIFY_FIELD *_ENCAP (c) JUMP | DROP | RSS (a) | QUEUE (a) | REPRESENTED_PORT (d)

  1. Only supported on ingress.
  2. Any decapsulation action, including the combination of RAW_ENCAP and RAW_DECAP actions which results in L3 decapsulation. Not supported on egress.
  3. Any encapsulation action, including the combination of RAW_ENCAP and RAW_DECAP actions which results in L3 encap.
  4. Only in transfer (switchdev) mode.

https://doc.dpdk.org/guides/nics/mlx5.html#features

More information you can contact networking-support@nvidia.com