Sorry if it’s a noob question. Can I connect Intel Omnipath host interface card (PCIe) to the Mellanox SX6012 switch?
I think I configured everything on the workstation, but when I plug in the cable to Mellanox switch, I never see the lights lit up.
Status of the OPA interface card shows it’s either “polling” or “disconnected”.
Thanks!
edit:
Okay, so in conclusion, the Intel OPA HFI (Omnipath Host Fabric Interface) card isn’t compatible with the Mellanox SX6012 Switch.
Going in to details, I found out that Intel OPA uses standard QSFP copper cables. The cables I have are Intel branded Hitachi IQSFP26C20, 2 meters, 26 AWG, passive, and nothing wrong with the cables. Complicating the problem, one of the OPA HFI was not making connection properly with the cable so that’s that. I’ll have to search for a different switch that’s Intel OPA compatible. Thanks for chiming in!
OPA is not an open standard, unlike Ethernet or InfiniBand. So, no interoperability with the SX6012 switch.
QSFP+ and QSFP28 are just connector types, they don’t fully specify what the cable is for. E.g. 100GbE and EDR (100Gbs) InfiniBand cables both use QSFPx connectors but the cables aren’t interchangeable—in part due to the information that’s stored in non-volatile memory in the cable electronics, and is read by the switch. (One of the differences in 100GbE vs EDR is that 100GbE requires Forward Error Correction and EDR doesn’t)
The fact that OPA chose the same connector doesn’t imply anything about interoperability or protocol support. I was on an Intel Webinar at which an ‘OPA architect’ explained that achieving low bit error rates at 100Gbs simply isn’t economical, so OPA has great link error detection and recovery to compensate. At the same time, EDR 100Gbs cables were shipping in volume at bit error rates over 1,000 times lower than what he claimed was un-economical. Another reason to say that OPA cables won’t interoperate. ☺
Hi. OmniPath (OPA) is an Intel proprietary protocol and is only implemented by OPA switches. You didn’t mention what type of cable you’re using, but OPA cables are also proprietary, and not compatible with Ethernet or InfiniBand.
Does proprietary mean that the protocol is also proprietary so that these two are not compatible?
As for the cable, I’m pretty sure it’s a QSFP+ cable, but I’d have to check again at work. I do know the ones I have are very rigid and about 2 meters. Searching for Intel OPA cable on Newegg I see there are QSFP28 cables. When I was experimenting, I plugged in both ends on the Mellanox switch, the lights came on for both ports. But when I tried Intel OPA interface card to the Mellanox switch, no lights. Also tried Intel OPA to another Intel OPA using the same cable, and no lights come on. Are you sure the cables are also proprietary? Intel OPA site lists all of it’s cable as QSFP28, which I think is a standard.
Okay, so in conclusion, the Intel OPA HFI (Omnipath Host Fabric Interface) card isn’t compatible with the Mellanox SX6012 Switch.
Going in to details, I found out that Intel OPA uses standard QSFP copper cables. The cables I have are Intel branded Hitachi IQSFP26C20, 2 meters, 26 AWG, passive, and nothing wrong with the cables. Complicating the problem, one of the OPA HFI was not making connection properly with the cable so that’s that. I’ll have to search for a different switch that’s Intel OPA compatible. Thanks for chiming in!