Connect problems with ISR 9024 switch

Infrastructure & Networking Switches and Gateways InfiniBand/VPI Switch Systems
1

I have a small test lab with 4 Windows 2012 servers and a Citrix Xenserver all system have Connect X2 VPI cards (MHQH19B-XTR) or embedded on MB. Node to node work fine using IPoIB on private subnet running 2.10.72 fw. I recently added a Voltaire ISR 9024D-M switch. The switch configuration appears to go fine however, when I connect a cable from any node to switch I get

the following error on any port. I have used two different cables from different manufactures same result :

1179×864 224 KB

My alternate to switch would be use two dual port cards in main server and connect to nodes. As switch would be

more flexible.

Thanks for any help

2

The switch indicates all ports are disconnected even with a cable in any of them. I toggle the enable/disable ports and there is no difference. If I use direct connection between nodes using IPoIB it works fine. My card are on 2.7.1 fw version so they work in Windows 2012 with RDMA. Possible firmware issue between card and switch? The switch switch is on 0.8.6 FW which is old.

Worst case I can buy a newer Mellanox 8 port IS5022 I believe they are backward compatible with X-2

Thanks

3

Hi Paul, i couldn’t see what error message you are referring to. the GUI screenshot looks clean or maybe i missed anything.

you also said you tested B2B with local OpenSM on the servers. pay attention to the fact that the Voltaire switch runs with an embedded SM. you might want to turn the Server SM off in that case.

4

Since your HCA is a VPI type, it could be that your cards are now set to Eth mode. the 9024 switch you are working is is IB only type.

Try to set the card to IB mode. you should find a tool with your HCA OFED stack: connectx_port_config

connectx_port_config -d pci bus slot -c ib,ib