Mlx5 udp out-of-order between fragmented and non-fragmented UDP packets when sender is using mlx4


I am a user of mellanox ethernet card and have two type cards ConnectX-4 LX and ConnectX-3 Pro. And they all running on Redhat Linux.

I get a problem of udp reordering between fragmented and non-fragmented UDP packets. when the packages larger than MTU mixed with packages smaller than MTU, ConnectX-4 LX using mlx5 on receive side’s udp flow will seperate into 2 flows. So I use the below command to solve it.

ethtool -N ethx rx-flow-hash udp4 sd

the command change 4-tuple hash for UDP to 2-tuple and work perfect .Udp traffic between 2 ConnectX-4 LX is ordered no matter fragment or not.

BUT when I use the ConnectX-3 Pro use mlx4 as sender side, the ConnectX-4 LX using mlx5 as receiver become out-of-order. It seem that the command “ethtool -N ethx rx-flow-hash udp4 sd” has no effort for the udp packages sended by mlx4.

Now I got below situation for udp traffic :

CX4 —> CX4 good

CX4 —> other brand card good

other brand card → CX4 good

CX3 —> other brand card good


I think the sender side should be no problem, because other brand card can get right package order. Maybe in the module mlx5 some special function is ​on effort when sender is a mlx4?

​ So maybe someone can help me. very very think you.

I use the attach c program to test out-of-order:


Compile on Linux:

gcc udp_server_seq.c -o udp_server_seq

gcc udp_client_seq.c -o udp_client_seq


server side:

./udp_server_seq | tee a.txt

client side:

./udp_client_seq serverip 8000


./ a.txt or open a.txt

and the out-of-order ratio is 20% of my test.

udp_server_seq.c (1.74 KB)

udp_client_seq.c (2.38 KB) (584 Bytes)