We are trying to deploy scale out converged software defined storage in vSphere and are limited by the number of physical PCIe slots in some hosts. Therefore we have a need for SR-IOV.
We have had a nightmare and spent a tremendous amount of time trying to figure some questions out but have been unsuccessful so far.
The questions are as follows and concern mainly ConnectX-3 VPI (MCX354A-FCBT) and ConnectIB (MCB194A-FCAT) adapters:
- Which ESXi version exactly allows SR-IOV with these adapters (5.5 / 6.0 / 6.0u2 etc…) ?
- Which matching drivers exactly should be used (Native / OFED) for each version of ESXi ?
- Is there a specific recommendation on which firmware to use ?
- Where can we find precise documentation and procedure on how to configure SR-IOV on the ESXi hosts ?
- In a physical host running ESXi 6.0u2 (or eventually lower), is it possible to mix ConnectIB, ConnectX-3 and ConnectX-2 adapters where some adapters are passed-through entirely to one VM ?
- Is there by any chance a script to clean up an ESXi host of all previous Mellanox and OFED drivers completely… ? Indeed, each installation requires a manual removal of each module, which is very time consuming. This is especially true when updating the ESXi hosts and the drivers having to be uninstalled before the update and reinstalled after the update due to the strict acceptance level of the ESXi update process.
Thanks a lot for your insight.