We are running KVM with libvirt on CentOS 7.6 with 100Gb ethernet MCX515A adapter. The network on the physical host is fine but our virtual guest nodes are communicating over a virtual ethernet bridge (br0) and they need RoCE to mount /lustre. Is Soft RoCE a reasonable option in this situation?
Hello James,
You should be able to use SoftRoCE in this case. Have you tried it?
https://community.mellanox.com/s/article/howto-configure-soft-roce
How Soft-RoCE works: Soft-RoCE driver implements the InfiniBand RDMA transport over the Linux network stack. It enables a system with standard Ethernet adapter to interoperate with hardware RoCE adapter or with another system running Soft-RoCE.
Please let me know…
Many thanks,
~Steve