Hi,
I am trying to setup SR-IOV as outlined in this article on the official Mellanox website - https://community.mellanox.com/s/article/howto-configure-sr-iov-for-connectx-4-connectx-5-with-kvm–ethernet-x
My Platform Setup -
Kernel 5.0.2
Ubuntu server 16.04
However, I am facing a few issues -
- When setting up SR-IOV and create VF’s using the command below -
echo 4 > /sys/class/net/ens785f0/device/sriov_numvfs (ens785f0 is the active interface on my machine) I see VF’s are created as interfaces ens785f2, ens785f3,ens785f4 and ens785f5 not as described in the article above under step 6 - where the interfaces are recognized as VF’s.
Below is the output of my ip link show command (attached is also the screenshot)-
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 65536 qdisc noqueue state UNKNOWN mode DEFAULT group default qlen 1000
link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
2: eno1: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq master br0 state UP mode DEFAULT group default qlen 1000
link/ether a4:bf:01:05:ff:af brd ff:ff:ff:ff:ff:ff
3: eno2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
link/ether a4:bf:01:05:ff:b0 brd ff:ff:ff:ff:ff:ff
4: ens785f0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc mq state UP mode DEFAULT group default qlen 1000
link/ether 98:03:9b:9b:4c:2a brd ff:ff:ff:ff:ff:ff
5: ens785f1: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
link/ether 98:03:9b:9b:4c:2b brd ff:ff:ff:ff:ff:ff
6: br0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc noqueue state UP mode DEFAULT group default qlen 1000
link/ether a4:bf:01:05:ff:af brd ff:ff:ff:ff:ff:ff
7: ens785f2: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
link/ether 02:0a:75:93:2d:99 brd ff:ff:ff:ff:ff:ff
8: ens785f3: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
link/ether f2:b3:de:bb:f7:00 brd ff:ff:ff:ff:ff:ff
9: ens785f4: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
link/ether f6:db:e2:ad:18:e4 brd ff:ff:ff:ff:ff:ff
10: ens785f5: <BROADCAST,MULTICAST> mtu 1500 qdisc noop state DOWN mode DEFAULT group default qlen 1000
link/ether 12:fd:f6:18:e0:e2 brd ff:ff:ff:ff:ff:ff
-
The other issue I am facing is that the interfaces created do not have link status up as should be expected as per the article. This is strange as the PF (physical Function) link status is indeed active (Attached is the screenshot showing the same). I do not have ibdev2netdev -v hence I have provided the status of link being up with ethtool.
-
What is the right way to attach VF to VM? I do not have or intend to use virt-manager. Right now I am using vfio-driver based commands to attach VF to VM i.e
sudo /sbin/modprobe vfio-pci
echo “15b3 1018” > /sys/bus/pci/drivers/vfio-pci/new_id
echo “0000:02:00.2” > /sys/bus/pci/devices/0000:02:00.2/driver/unbind
echo “0000:02:00.2” > /sys/bus/pci/drivers/vfio-pci/bind
/usr/local/kvm/bin/qemu-system-x86_64 \
-name ubuntu-vm \
-boot order=c \
-drive file=/root/vm_images/ubuntu_vm.img,index=0 \
-cpu host \
-machine pc-i440fx-4.1,accel=kvm \
-smp cores=20,threads=1,sockets=1 \
-m 10240 \
-netdev tap,id=tapdev,script=/root/native_script_recv/notunctl,downscript=no \
i-device e1000,netdev=tapdev \
-device vfio-pci,host=02:00.2 \
-vnc 127.0.0.1:2
Regards,
Aketh T. M
