BlueField-3 DOCA PCC: Should PCC bind to SF or PF? How to test custom CC when RDMA traffic uses SF?

Infrastructure & Networking DOCA Getting Started & Resources
1

I’m trying to deploy and test my custom congestion control algorithm on BlueField-3 using DOCA PCC. I can configure IP on a Sub-Function (SF) and run perftest successfully, but I cannot bind PCC to the SF device. PCC only seems to bind to the PF device (e.g., mlx5_0). When I send RDMA traffic via the SF, PCC bound to PF doesn’t appear to take effect.

Here’s my setup and symptoms:

  • SF interface: enp3s0f0s0
  • RDMA traffic: only works via SF (perftest OK)
  • PCC binding: only works via PF (mlx5_0)
  • show_gids output:
DEV     PORT    INDEX   GID                                     IPv4            VER     DEV
---     ----    -----   ---                                     ------------    ---     ---
mlx5_2  1       0       fe80:0000:0000:0000:0039:04ff:fed3:d339                 v2      enp3s0f0s0
mlx5_3  1       0       fe80:0000:0000:0000:0045:19ff:fe2e:0276                 v2      enp3s0f1s0
n_gids_found=2

My questions:

  1. Should DOCA PCC bind to the SF or the PF for traffic that is sent via SF?
  2. If PCC must bind to PF, how can I ensure SF traffic is actually controlled by the PCC algorithm?
  3. What is the correct IP / RDMA device configuration to test a custom PCC algorithm on BF3 when traffic uses SF?

Any guidance on the correct binding model (PF vs SF) and recommended configuration steps would be really helpful.

Thanks in advance!

2

I have the exactly same problem. Looking forward to solution.

3

I run the compiled example program doca_pcc and bind PCC to mlx5_0 using the command ./doca_pcc -d mlx5_0, but I can only send RoCE traffic from the SF, which is mlx5_2. I observed that traffic sent from mlx5_2 (perftest) is not controlled by PCC bound to mlx5_0. I would like to know how to properly deploy and test whether the NVIDIA DOCA PCC example code is working correctly.

4

Hi,

Thanks for your questions.

Following to my check, at the moment PCC was tested on PF only.

If you have questions regarding PCC on VF or SF, I’d recommend you to open a new support case in Nvidia portal.

To Create a New Enterprise Support Case Please Use One of the Following Methods

  1. Log in and Create a Case in the Enterprise Support Portal
    AppHub UI
    This is recommended and will provide the fastest response

  2. Call your Regional Enterprise Support Number
    Find your regional phone numbers:
    NVIDIA Enterprise Customer Support

  3. Create a Case Without a Login
    https://enterprise-support.nvidia.com/s/create-case
    Can be used to check warranty status
    Recommended to reset login passwords
    Requires validation and may increase time to respond

Best Regards,

Anatoly