How to generate PFC pause packets with Mlnx CX-5

Infrastructure & Networking Adapters and Cables Ethernet Adapter Cards
1

I have two servers with CX-5 card, tx and rx. There have switchs between the servers. PFC disabled on switchs.
I am using pktgen to generate packets on tx server and PFC enabled on rx. All packets received on rx_prio0. Rx server have rx_out_of_buffer counter, but it didn’t generate tx PFC pause packets.
My question is, must PFC be enabled for directly connected switch in order for rx server to generate PFC pause packets. If not, how should I configure to generate PFC pause packets.

root@POD65-CLU01-GPU02:~#
root@POD65-CLU01-GPU02:~# ethtool -S enp94s0f0np0 | grep rx_out_of_buffer
     rx_out_of_buffer: 28152779
root@POD65-CLU01-GPU02:~#
root@POD65-CLU01-GPU02:~# ethtool -S enp94s0f0np0 | grep rx_prio0
     rx_prio0_bytes: 11367274808
     rx_prio0_packets: 132018633
     rx_prio0_discards: 0
     rx_prio0_pause: 0
     rx_prio0_pause_duration: 0
     rx_prio0_pause_transition: 0
root@POD65-CLU01-GPU02:~#
root@POD65-CLU01-GPU02:~# mlnx_qos -i enp94s0f0np0
DCBX mode: OS controlled
Priority trust state: pcp
default priority:
Receive buffer size (bytes): 130944,130944,0,0,0,0,0,0,max_buffer_size=262016
Cable len: 7
PFC configuration:
        priority    0   1   2   3   4   5   6   7
        enabled     1   1   0   0   0   0   1   0
        buffer      1   1   0   0   0   0   1   0
tc: 0 ratelimit: unlimited, tsa: strict
         priority:  0
         priority:  1
         priority:  2
         priority:  3
         priority:  4
         priority:  5
         priority:  6
         priority:  7
root@POD65-CLU01-GPU02:~#
root@POD65-CLU01-GPU02:~# ethtool -S enp94s0f0np0 | grep pause
     rx_pause_ctrl_phy: 0
     tx_pause_ctrl_phy: 0
     rx_prio0_pause: 0
     rx_prio0_pause_duration: 0
     tx_prio0_pause: 0
     tx_prio0_pause_duration: 0
     rx_prio0_pause_transition: 0
     rx_prio1_pause: 0
     rx_prio1_pause_duration: 0
     tx_prio1_pause: 0
     tx_prio1_pause_duration: 0
     rx_prio1_pause_transition: 0
     rx_prio6_pause: 0
     rx_prio6_pause_duration: 0
     tx_prio6_pause: 0
     tx_prio6_pause_duration: 0
     rx_prio6_pause_transition: 0
     tx_pause_storm_warning_events: 0
     tx_pause_storm_error_events: 0
2

Hi,

If RX server shows out_of_buffer packets but no PFC pause packets, it might indicate that the PFC is not configured properly.
I suggest following this guide to configure it: ESPCommunity
Additionally, while it is not strictly mandatory to configure PFC on the switch, it is recommended as it will ensure end-to-end propagation and might assist in debugging and understand where the issue is exactly.
Please configure the switches as well if possible.

If the issue still occurs after these steps, please open a case with enterprisesupport@nvidia.com and it will be handled based on entitlement.

Thanks,
Jonathan.