sx1036 as IB-to IP gateway

Infrastructure & Networking Switches and Gateways Gateways
#1

Hi there.

Anyone having a clue on how many 10GbE ports, using all possibilities with fan-out cables, one could us on a SX1036 switch running a with the gateway option active?

As far as I have understood it there will be 8 ports available for 40GbE and the rest for 56G IB. I have seen that when using fan-out cables, some ports will be disconnected. So my question is if there are any specific restrictions when using the gateway feature limiting the total number of available 10GbE ports.

Regards, Ingvar

#2

Hi Ingvar,

The maximum ports you can achieve when using split options is 64 (with a mix of 40GbE and 10GbE).

If it is a gateway, you need allocate some of the ports to run as FDR, while others to run as Ethernet. It could be 50-50 (%), but not necessarily, it depends on the setup being used. it is possible that you have more hosts on the InfiniBand side or vice-versa.

This post may help as well.

http://community.mellanox.com/docs/DOC-1450 https://community.mellanox.com/s/article/mellanox-breakout-cables-40g--gt--4x10g-and-100g--gt--4x25g

Ophir.

#3

Also note that since the original post in this thread, the MLNX-OS version has been updated, allowing configuring any port for either Ethernet or IB. So the combination of fan-out cabling and maximum number of 10Gb Ethernet ports are not correct.

regards, /I

#4

Thanks a lot for the very informative image. A short follow-up question:

Is it so that the blocked ports are caused by using split cables?

example: port 4 uses a 4*10G split cable so port 1 is blocked

port 5 uses split cable, so port 6 gets blocked.

If this is the case, port 7 is blocked because of a split cable in the IB part (port 9?) So if the port 9 is not splitted or un used, we could use the port 7 in the ethernet part as a 10G port? (without split cable)

i.e. port 2 -10G , port 3 -10G, port 4 - split 4x10G, port 5- split 4x10G, port 8 - 10G, port 7 -?? (always blocked or can it be used for 10G)

I’m just trying to figure out the absolute maximum number of 10G Ethernet ports if used as an IB/Ether GW

#5

Does the SX1036 require a license to gain gateway and IB functionality?

#6

Hi Viper,

Yes, the SX1036 requires a license, the license type is UPGR-1036-GW ( if you are looking at purchasing one )

#7

so, the answer is: with the right cables (40G to 4X10G) you can do that.

with 1036 you can get to up to 64 10GE ports.

work with this:

1039×178 19.3 KB

as far as the above relationship with gateway functionality, at this point if you want to use gateway, you can only use ports 1-8 for Eth and the rest for IB. means, with splitter cables you can get now: 110Gb (2X4X10+3X10). all the rest are IB ports.

Thanks

#8

Hi Ingvar,

When using the SX1036 as a gateway (VPI profile): you can split the following ports to 4*10GbE:

1/4 - This would also disable port 1/1

1/5- This would also disable port 1/6:

In summary: you will have the following ports available for 10GbE:

1/2 - 40/10/1GbE

1/3 - 40/10/1GbE

1/4 - 4*10GbE

1/5 - 4*10GbE

1/7 - 40/10/1GbE

1/8 - 40/10/1GbE