I am using Mellanox ConnectX-3, and I see packet loss when the traffic surges after idle. It can handle all the traffic most of the time, but the packet drop happens only after the traffic was idle (tens or hundreds of packets) for some time being. I suspect the card or CPU was doing some kind of power save stuff. I have intel_idle.max_cstate set to 1. How can I avoid this kind of packet loss? Thank you.
Thank you the reply. I tried everything in the document. I checked BIOS settings, CPU power setting, and RPS. But still I have same issue.
In my understanding, the Ethernet card has internal buffer (FIFO) to keep incoming packets, and the card should write the packet via PCIe interface, this would involve reading RX descriptor and writing packet data to memory. Does the packet loss means internal buffer overflow? What would prevent the card from writing packet data to memory?
In general, performance and packet loss issues is much dependent on the system variables and configuration.
I recommend to follow the steps on our Performance Tuning Guide for the system tuning recommendations and recommended BIOS settings.
Please also notice the ConnectX-3 optimized steering configuration on page 21: