How to configure Mellanox card to accept breakout cable connection (4x10Gbps)?

Hello,

We installed 2 Mellanox NIC on serverA and 2 of them on serverB.

Mellanox NIC info:

Model: MCX354A

PSID: MT_1090110019

Driver: mlx4_en

version: 2.2-1.0.1 (Jun 13 2014)

firmware-version: 2.31.5050

ServerA-B:

-Dell poweredge R720xd

-CPU: 32 CPU each 2.9 GHz (Sandy Bridge): Numa0: 0-2-4-…30,

Numa1:1-3-…-31

-6 slot PCIe 8GT/s (16 lane)

-RAM: 128 GB DDR3

-OS: ubuntu 12.04 LTS (Kernel: 3.11.0.22)

-four Intel SFP+ cards with ixgbe driver

We are going to test breakout cable (40G QSFP+ to 4x10G SFP+), in the following configuration:

-1Mellanox NIC (FCBT) in serverA is connected to 4 intel NIC (SFP+) in server B, using a breakout cable.

But when we connect the breakout cable, just one 10G SFP+ is working from total of four, and 3x10G SFP+ is off (no light). We then generate the traffic, it is normal that just 10G traffic was passed.

I think that this is due to the fact that Mellanox NIC is configured for 40G.

My question is how can i configure Mellanox to recognize 4x10G configuration for breakout scheme?

We tested this issue using Cisco switch configured to 4x10G SFP+ both breakout cable and switch ports were active and working.

Thanks for your support

Mohammad

Thanks

HI Mohammad,

The split of the port from 40GbE to 4x10GbE is applicable only on Mellanox switch side.

It is not supported on Mellanox 40GbE adapters.

Thanks,

Ophir.

Hi,

I added a post discussing the breakout cable options when using Mellanox Hardware

I hope it will help,

http://community.mellanox.com/docs/DOC-1450 https://community.mellanox.com/s/article/mellanox-breakout-cables-40g--gt--4x10g-and-100g--gt--4x25g

Ophir.