32 RSS queues

Infrastructure & Networking Adapters and Cables Ethernet Adapter Cards
1

Our usage scenario is a >10Gbit/sec linux router where the forwarded traffic is loadbalanced across as much cpu cores as possible. At the moment we use Intel X520 nic’s. But they are limited to 16 hardware RX interrupts (RSS) per port. This means a dual 12 core machine is not optimally used.

According to the feature summary ( http://www.mellanox.com/related-docs/products/ethernet_nic_brochure_c_2_3.pdf http://www.mellanox.com/related-docs/products/ethernet_nic_brochure_c_2_3.pdf ) a ConnectX 3 nic supports up to 32 RSS queues.

Does this mean the hardware effectively allocates and uses 32 MSI-X RX interrupts on a 32 core system? And to be sure: a dual port nic would allocate 32 RX irq’s for each port?.

2

Thank you for the information.

We tried RPS, but our testing showed it comes with a significant penalty compared to (hardware) RSS due to the extra required inter-processor interrupts. This is especially noticeable at high packet rates.

3

About RPS (and more): https://www.kernel.org/doc/Documentation/networking/scaling.txt https://www.kernel.org/doc/Documentation/networking/scaling.txt

4

Hi,

Currently the driver supports 16 queues per port.total of 32.

You can try enable RPS which is a standard interface for any NIC and set rps_cpus mask to the remaining 8 cores.

This will allow utilization of all the cores in the system.