Is there any configuration that is limiting the virtual function bandwidth on connectx-5-ex adapter ? We've working with this adapter and noticed that we are limited to 50gb when using the virtual function while the same limit does not occur with the pf

Infrastructure & Networking Adapters and Cables Ethernet Adapter Cards
,
1

IB_SEND_BW result:

PF:

#bytes #iterations BW peak[Gb/sec] BW average[Gb/sec] MsgRate[Mpps]

65536 344452 0.00 60.19 0.114807

65536 344471 0.00 60.20 0.114814

Single VF:

#bytes #iterations BW peak[Gb/sec] BW average[Gb/sec] MsgRate[Mpps]

65536 275444 0.00 48.14 0.091816

65536 276957 0.00 47.92 0.091394

Both VFs:

#bytes #iterations BW peak[Gb/sec] BW average[Gb/sec] MsgRate[Mpps]

65536 126892 0.00 22.06 0.042083

65536 126602 0.00 22.13 0.042201

#bytes #iterations BW peak[Gb/sec] BW average[Gb/sec] MsgRate[Mpps]

65536 128045 0.00 22.38 0.042682

65536 126065 0.00 22.03 0.042022

2

Hi Efi ,

First of all we recommend tune the server and the adapter following the below guide so you can reach line rate on the PF .

https://community.mellanox.com/s/article/performance-tuning-for-mellanox-adapters

Regarding the VFs performance as i can see you running the performance test from the baremetal and not from VM (as pure SRIOV solution) , the performance should be a little bit lower than the PF and it also depends to how many VFs you create per PF as the PF resources are splitted between all VFs related to the same port(PF) .

Thanks,

Samer