ConnectX6DX - rte-flow / RSS performance drop on mixed traffic

Infrastructure & Networking Adapters And Cables Ethernet Adapter Cards
, ,
#1

Dear Support,

card infos at end of email.

There appears to be a huge performance issue on mixed UDP/TCP using symmetric load-balancing accross multiple workers. We are using the option “rxq_cqe_comp_en=4”
E.g. using default test-pmd on a DPDK v20.11 or newer :

sudo ./dpdk-testpmd -n 8 -l 4,6,8,10,12,14,16,18,20 -a 0000:4b:00.0,rxq_cqe_comp_en=4 -a 0000:4b:00.1,rxq_cqe_comp_en=4 – --forward-mode=mac --rxq=8 --txq=8 --nb-cores=8 --numa -i -a --disable-rss

and configuring:

flow create 0 ingress pattern eth / ipv4 / tcp / end actions rss types ipv4-tcp end queues 0 1 2 3 4 5 6 7 end key 6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A / end

flow create 0 ingress pattern eth / ipv4 / udp / end actions rss types ipv4-udp end queues 0 1 2 3 4 5 6 7 end key 6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A / end

flow create 1 ingress pattern eth / ipv4 / tcp / end actions rss types ipv4-tcp end queues 0 1 2 3 4 5 6 7 end key 6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A / end

flow create 1 ingress pattern eth / ipv4 / udp / end actions rss types ipv4-udp end queues 0 1 2 3 4 5 6 7 end key 6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A6D5A / end

will see significant packet drops at a load > 50 Gbps on any type of mixed UDP/TCP traffic. E.g.

Whenever those packet drops occur, I see those in the xstats as “rx_phy_discard_packets”

On the other hand using a TCP-or UDP-only traffic profile perfectly scales up to 100Gbps w/o drops.

Thanks for your help!

Tobias

ConnectX6DX

<Devices>
    <Device pciName="0000:4b:00.0" type="ConnectX6DX" psid="DEL0000000027" partNumber="0F6FXM_08P2T2_Ax">
      <Versions>
        <FW current="22.31.1014" available="N/A"/>
        <PXE current="3.6.0403" available="N/A"/>
        <UEFI current="14.24.0013" available="N/A"/>
      </Versions>
      <MACs Base_Mac="b8cef6xxxxxx" />
      <GUIDs Base_Guid="b8cef6xxxxxx" />
      <Status>No matching image found</Status>
      <Description>Mellanox ConnectX-6 Dx Dual Port 100 GbE QSFP56 Network Adapter</Description>
    </Device>
    <Device pciName="0000:98:00.0" type="ConnectX6DX" psid="DEL0000000027" partNumber="0F6FXM_08P2T2_Ax">
      <Versions>
        <FW current="22.31.1014" available="N/A"/>
        <PXE current="3.6.0403" available="N/A"/>
        <UEFI current="14.24.0013" available="N/A"/>
      </Versions>
      <MACs Base_Mac="b8cef6xxxxxx" />
      <GUIDs Base_Guid="b8cefxxxxxx" />
      <Status>No matching image found</Status>
      <Description>Mellanox ConnectX-6 Dx Dual Port 100 GbE QSFP56 Network Adapter</Description>
    </Device>
</Devices>