Hi all,
Our company has 3 HP DL380p G8 and 3 DL380p G7 servers. We plan to upgrade their IO connectivity to reach 10Gbps network or even higher. The main reason is that we are building Hyper Converged Infrastructure, especially SDS (VMware VSAN, or Microsoft S2D). We already have a pair of 10Gb Switches (JG219A).
We took a deep search and found some useful information:
- We see there are 2 options: 10Gb Ethernet NIC, or 40Gb InfiniBand (IB) card. We prefer InfiniBand as it has more hardware offload features that we could benefit from, such as RDMA (RoCE).
- FlexLOM card (FLR) is not an option, as the only slot has already been occupied.
- According to HP DL380p G8 and DL380 G7 QuickSpecs, the only suitable IB card is HP Infiniband FDR/Ethernet 10Gb/40Gb 2-port 544QSFP Adapter (649281-B21). I dig deeper and found that this IB card is OEMed by Mellanox, and according to HP-Mellanox-Reference-Guide-August-2012 http://www.mellanox.com/pdf/products/oem/HP-Mellanox-Reference-Guide-August-2012.pdf the appropriate Mellanox Part # is MCX354A-FCBT.
- I took one step further and found the ConnectX®-3 VPI Single and Dual QSFP+ Port Adapter Card User Manual https://www.google.com.vn/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0ahUKEwirtPLo6a3WAhXGopQKHb8cAlcQFggnMAA&url=http%3A%2F%2Fwww.mellanox.com%2Frelated-docs%2Fuser_manuals%2FConnectX-3_VPI_Single_and_Dual_QSFP%2B_Port_Adapter_Card_User_Manual.pdf&usg=AFQjCNFtl76fwEVPHEs3sO4obw2uGo3slA , Rev 2.5. There on Section 7.6, MCX354A-FCB[T/S] Specifications, I found something:
- Protocol Support:
- Ethernet: 10GBASE-CX4, 10GBASE-R, and 1000BASE-R, 40GBASE-R4
- Data Rate: Up to 56Gb/s FDR– InfiniBand
- Protocol Support:
1/10/40/56Gb/s – Ethernet
I know that we also need QSFP to SFP+ adapter P/N 655874-B21, or MAM1Q00A-QSA in Mellanox world in order to use 10Gb Ethernet.
However, I am not so familiar with InfiniBand, and not confident to tell what else we need to take. Do we need special cables in order to work with our existing 10Gb switches? Or do we need to buy special InfiniBand switches?
Hope I explained our case clearly. Long story short, we need to verify the possibility to invest on InfiniBand (6 - 10 dual-port cards) and related items.
Actually I tried to reach both HP Sales and Presales teams in my country, but haven’t received any feedback from them for days. I have no options left but posting a question here and hope someone from the community could help me out.
Thank you very much in advanced!