We currently using a ConnectX-3 card (MCX353A-FCB) as data sink for a Xilinx Virtex-5 FPGA based 10GBit Ethernet implementation which uses 4-lane data transfer via XAUI over a QSFP copper cable. The related Xilinx IP cores are “10-Gigabit Ethernet MAC” and “XAUI” and the 4 GTX transceivers use a line rate of 3.125 Gbps and 8B/10B encoding.
As a replacement we tried to use a ConnectX-4 card (MCX453A-FCA_Ax) but currently without success.
There seem to be only activity on the 1st of the 4 transmission lanes (Visible on FPGA side) which indicate that the X-4 card may use a wrong ethernet standard for communication. Though, we are unsure how the used ethernet standard is actually called (as 4 lanes are used 10GBASE-CX4 or 10GBASE-KX4 seem to fit). According to the documentation both standards should be supported by the X-4 card.
Do you have any suggestions? Does anyone know if this kind of protocol setup is really supported by the X-4 card? Are there configuration options of the X-4 card which can be checked/changed?
Assuming that you are using a supported cable (can be verified in the firmware release notes, in section ‘Supported Cables and Modules’), I would suggest to set the speed manually for the nic in this instance (in case there is no switch to auto-negotiate from).
As you can see in the ethtool output of the CX-3 nic, auto-negotiation is off.
according to the release notes the used cable “MC2206130-002” is supported.
When setting the speed manually to 10G and autoneg to off via ethtool the output is like below.
Now only the link mode “10000baseKR/Full” is advertised, but this one uses only one lane instead of the required 4. Do you know why the card does not report “10000baseKX4/Full*”* as supported link mode? As mentioned before, according to the documentation the card should support this mode.
We would really like to understand, why according to the documentation the card “MCX453A-FCA_Ax” supports “10000baseKX4/Full” but this mode is not advertised by ethtool.