Capturing traffic with varying MACs with DPDK + Connect-X4 Lx shows out-of-order reception with single queue

Infrastructure & Networking Software And Drivers
, ,
#1

Hello,

We’re writing a packet capture / replay tool using DPDK (18.11.8) and Connect-X 4Lx network card. Centos 8 system, 4.18 kernel, OFED 5.0. We’re experiencing a problem with out-of-order packet reception.

  • Usually packet order is OK, however when capturing a replay with rapidly changing MAC adresses, first ~20 packets are out-of-order. lt’s a simple protocol over Ethernet (no IP/TCP/UDP) so this probably rules out RSS and such.
  • We tried polling one-by-one instead of in batches, still same result.
  • NIC timestamps follow the wrong order, which makes me think the issue has to happen before timestamping.
  • Issue doesnt appear when packets go through kernel path (tcpdump/gulp) or when the time between sent packets is longer.

I found a similar older question with no answer: https://community.mellanox.com/s/question/0D51T00006RVuBySAL/mlx5connectx4-out-of-order-packets-w-a-single-queue

What can be causing this? Is there some MAC based switching / classification that could cause packets to take different paths even with 1 queue? If so, can it be turned off? Thanks!

Best regards,

Adam

#2

HI Adam,

I see you have a valid support contract with us. In order to assist further, I will create a support ticket and will send out an email requesting further details/clarifications via the support ticket.

Thanks,

Namrata.