chao
December 23, 2019, 5:33am
1
Attach the MCX4131A-GCAT 1x50G NIC to the X86 platform, setup SR-IOV to enable the DPDK testpmd testing at GuestOS.
From the testing, found that the 64B can’t achieve the same performance as baremetal environment (30% gap).
Similar results at the ARM platform.
Was it because of the NIC didn’t support the 64B processing at SR-IOV mode?
Is there any special settings should try?
Already set the inte_iommu=on, iommu=pt at Host OS.
Host Kernel: Linux 3.10.0-957.el7.x86_64
QEMU/KVM Hypervisor:
virsh # version
Compiled against library: libvirt 4.5.0
Using library: libvirt 4.5.0
Using API: QEMU 4.5.0
Running hypervisor: QEMU 1.5.3
mlxconfig -d 65:00.0 set NUM_OF_VFS=1 SRIOV_EN=1 CQE_COMPRESSION=1
lspci at Host OS:
65:00.0 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx]
65:00.1 Ethernet controller: Mellanox Technologies MT27710 Family [ConnectX-4 Lx Virtual Function]
Guest OS xml cfg:
PCI passthrough:
Guest OS: Linux vm 3.10.0-957.el7.x86_64
GCC @Guest OS: gcc version 7.3.1 20180303
DPDK @Guest OS: V19.08
testpmd: ./x86_64-native-linuxapp-gcc/app/testpmd -l 1-17 -w 00:06.0 – -i --nb-cores=4 --burst=64 --rss-ip --txq=4 --rxq=4 --mbcache=512 --rxd=4096 --txd=4096
samerka
December 23, 2019, 7:06am
2
Hi Jerry,
Please refer to the following performance report by Mellanox for DPDK 19.08
http://fast.dpdk.org/doc/perf/DPDK_19_08_Mellanox_NIC_performance_report.pdf
For DPDK 19.05
http://fast.dpdk.org/doc/perf/DPDK_19_05_Mellanox_NIC_performance_report.pdf
There are few examples for ConnectX-4/5.
Thanks,
Samer
chao
December 23, 2019, 9:12am
3
Hi, Samer
Thanks for your quick response.
I didn’t see the MCX4131A-GCAT 1x50G SR-IOV over KVM from all the published Mellanox NIC performance reports. Only saw the baremetal test results with 2 NICs at 2x40G mode.
Would like to confirm with Mellanox, does this specific NIC not support 64B well at 1x50G SR-IOV mode?
Thanks,
Jerry
samerka
December 23, 2019, 3:50pm
4
Hi Jerry,
Please review the below Release notes for all supported features by mlx5
http://doc.dpdk.org/guides/rel_notes/release_19_08.html#api-changes
Also in the user manual “SR-IOV is supported”
http://doc.dpdk.org/guides-19.08/nics/mlx5.html
Regarding the NIC review the documentation.
Thanks,
Samer
chao
December 24, 2019, 1:13am
5
Hi, Samer
Thanks for providing the inputs.
Yes. This NIC card does support the SR-IOV feature.
My question here is I have tried all the performance optimizations the user manual has listed.
32.10. Performance tuning
http://doc.dpdk.org/guides-19.08/nics/mlx5.html
But the SR-IOV performance throughput (64B) dropped 30% comparing with baremetal performance.
Is there any more settings I could try?
Thanks,
Jerry
samerka
December 24, 2019, 2:31pm
6
Hi Jerry,
I see that you have a valid support contract with Mellanox
Please open a support case for deep debug at support@mellanox.com .
Thanks,
Samer