We’re writing a packet capture / replay tool using DPDK (18.11.8) and Connect-X 4Lx network card. Centos 8 system, 4.18 kernel, OFED 5.0. We’re experiencing a problem with out-of-order packet reception.
- Usually packet order is OK, however when capturing a replay with rapidly changing MAC adresses, first ~20 packets are out-of-order. lt’s a simple protocol over Ethernet (no IP/TCP/UDP) so this probably rules out RSS and such.
- We tried polling one-by-one instead of in batches, still same result.
- NIC timestamps follow the wrong order, which makes me think the issue has to happen before timestamping.
- Issue doesnt appear when packets go through kernel path (tcpdump/gulp) or when the time between sent packets is longer.
I found a similar older question with no answer: https://community.mellanox.com/s/question/0D51T00006RVuBySAL/mlx5connectx4-out-of-order-packets-w-a-single-queue
What can be causing this? Is there some MAC based switching / classification that could cause packets to take different paths even with 1 queue? If so, can it be turned off? Thanks!