Enablement of SR-IOV on ConnectX-5 only possible for one port (Physical Function)

Hello together,

I am currently trying to use SR-IOV with a Mellanox ConnectX-5 networking card. As part of this, it is required to create Virtual Functions, which should be done through the following command:

echo 4 > /sys/class/net/enp2s0f0np0/device/sriov_numvfs

This then creates the desired virtual functions without a problem. However, on the second port of the networking card the command fails with the following error:

echo 4 > /sys/class/net/enp2s0f1np1/device/sriov_numvfs
-bash: echo: write error: Input/output error

dmesg shows the following:

[ 2057.180802] pci 0000:02:01.2: unknown header type 7f, ignoring device
[ 2058.196440] mlx5_core 0000:02:00.1: mlx5_sriov_enable:230:(pid 2558): pci_enable_sriov failed : -5
[ 2058.196800] mlx5_core 0000:02:00.1: E-Switch: Unload vfs: mode(LEGACY), nvfs(4), necvfs(0), active vports(5)
[ 2058.212367] mlx5_core 0000:02:00.1: E-Switch: Disable: mode(LEGACY), nvfs(4), necvfs(0), active vports(1)

I have tested the same commands on a second identical card which produces the exact same error output.
OS: Ubuntu 22.04 LTS
Any hints on what is causing this problem are appreciated. Thanks :)


You need check if the PCIE bridge of CX5 installed, support ARI which limited on VF nu#.

The server is a Dell PowerEdge R7515 and ARI support is enabled by default on the machine. This has been verified.