I have encountered a problem regarding the migration process using RDMA, which seems to have a longer total time compared to TCP.
Here are the relevant details:
- Network configuration: Mellanox ConnectX-3 Pro EN 40G NIC
- Software versions: QEMU 4.2.0, kernel 4.4.0
- Installed MLNX_OFED from NVIDIA as instructed.
- Compiled QEMU with the “–enabled-rdma” flag to ensure the presence of librdmacs and libibverbs interfaces.
- Verified bandwidth using iperf for both RDMA and TCP. RDMA achieved 26 Gbits/sec, while TCP reached 20 Gbits/sec, demonstrating the expected superiority of RDMA.
- During the migration of a 30GB VM at the maximum migration speed of 40G, I observed the following migration times:
- RDMA: 13531ms
- TCP: 13483ms
I’m uncertain about what might be causing this discrepancy. Any insights or suggestions would be greatly appreciated. Thank you.