High Latency using DPDK with MCX512A-ACA_Ax_Bx

I’ve a MCX512A-ACA_Ax_Bx dual port card, for which I’ve written a UDP based ping/pong app using DPDK. I bind pong server process to on of the ports and the ping to the other to send and receive udp packets and measure the latency (similar to sockperf). I seem to get very high latency when doing this (see histogram below). Interestingly all of the latency values reported (with a small degree of jitter) fall on a multiple of 4ms. I tested my ping process against sockperf running on a different PC as well (I reused the same protocol), and that gives more reasonable (but not great) latency figures (histogram at the bottom).

Is using 2 ports on the same card in this way supported?

DPDK ping to DPDK pong, same card different port

Value Percentile TotalCount 1/(1-Percentile)

11976703.000 0.000000 1 1.00

24002559.000 0.500000 168 2.00

24002559.000 0.750000 168 4.00

31997951.000 0.875000 177 8.00

32014335.000 0.937500 194 16.00

32014335.000 0.968750 194 32.00

40009727.000 0.984375 199 64.00

40009727.000 0.992188 199 128.00

40042495.000 0.996094 200 256.00

40042495.000 1.000000 200 inf

[Mean = 23625748.480, StdDeviation = 4791371.481]

[Max = 40042495.000, Total count = 200]

[Buckets = 22, SubBuckets = 2048]

DPDK ping to sockperf sr, one port to remote PC

Value Percentile TotalCount 1/(1-Percentile)

48991.000 0.000000 3 1.00

121151.000 0.500000 1000 2.00

124543.000 0.750000 1501 4.00

150399.000 0.875000 1750 8.00

169343.000 0.937500 1875 16.00

182911.000 0.968750 1938 32.00

194303.000 0.984375 1969 64.00

209535.000 0.992188 1985 128.00

255999.000 0.996094 1993 256.00

275967.000 0.998047 1997 512.00

350207.000 0.999023 1999 1024.00

362495.000 0.999512 2000 2048.00

362495.000 1.000000 2000 inf

[Mean = 114185.328, StdDeviation = 36970.092]

[Max = 362495.000, Total count = 2000]

[Buckets = 22, SubBuckets = 2048]

Hi Michael,

For best performance we recommend following our dpdk performance report

https://fast.dpdk.org/doc/perf/DPDK_20_05_Mellanox_NIC_performance_report.pdf

Please apply all tuning optimization elements and see it if helps.

Regards,

Chen