Unexpected Core Allocation with ConnectX-5 RSS Hash Function

Infrastructure & Networking Adapters and Cables Ethernet Adapter Cards
1

Purpose: We aim to test the functioning of the RSS hash function in ConnectX-5.

Issue: Even though the hash function is configured to consider only the destination IP, packets sent from multiple clients to a single server are being directed to different cores.

Procedure:

  1. Use the following command to configure the toeplitz hash function to consider only the destination IP:

    sudo ethtool -N ens1f0np0 rx-flow-hash tcp4 d

  2. We then confirmed that the settings were applied correctly using:

    sudo ethtool -n ens1f0np0 rx-flow-hash tcp4

    The result showed: “TCP over IPV4 flows use these fields for computing Hash flow key: IP DA”

  3. Packets were then sent from multiple clients.

  4. Upon checking the core assignments made by RSS, we found that they were allocated to different cores.

Questions: I’m puzzled by these results. Specifically, I want to understand how, through the RSS hash function, we can ensure that packets with the same destination IP are directed to the same core without having to hardcode the destination IP.

2

Hi,

Thanks for your question.
According to my check, 4-tuple hash still will be used with the mentioned configuration.
At the moment you can configure it statically per IP and steer it to a particular receive queue.

Best Regards,
Anatoly

3

Thank you for your answer.

  1. In the environment you tested, does it mean that the 4-tuple hash was used regardless of the setting, even if it was set to use only DA IP? In other words, can we consider that the command sudo ethtool -N ens1f0np0 rx-flow-hash tcp4 does not work?
  2. If all 4-tuples are considered regardless of the settings, traffic from the same client must be assigned to a different core each time because the src port changes each time.
    However, considering only the DA IP, it was confirmed that although the core allocation changes when the client changes, traffic from the same client is always allocated to the same core
  3. Ultimately, I want to apply RSS based on the SA IP. Our environment has a large number of unspecified clients, so it seems difficult to statically steer a specific IP to a specific queue. Is it impossible to apply RSS considering only the SA IP?

Best regards,
KH

4

Thanks for asking more questions.

  1. Yes, this is correct, 4-tuple hash is used regardless of ethtool configuration you applied.
  2. and 3) If unexpected behavior is observed, and a deeper investigation is needed, or in the case of a new feature, we recommend to open a support case in Nvidia portal, and we will be happy to assist.

Best Regards,
Anatoly