I have a HP DL360 server with two dual-port ConnectX-3 Pro cards (4 total NIC’s). The plan was to use them in 40Gb Ethernet mode with 40GBASE-LRL4 transceivers to drive over the distance we require at our site.
The problem I’m seeing is that I can get link-up with 40GBASE-SR4 transceivers, but not the LRL4 transceivers. Is this supported on these cards? I can get a link up with the same transceiver/fiber connection in another device, but not on the ConnectX-3 Pro card in the server.
In both cases I’m using Arista brand transceivers, is that the problem?
Can you give me details of your transceiver ?
What is the distance between your server and your switch ?
What is your FEC status enable or disable on your both sides ?
The transceiver I’m trying to use is an Arista 40GBASE-PLRL4, with an MPO connector (8 fibers). The length of cable we’re using for testing right now is only 300 feet of single mode fiber, though the eventual run will be longer than that. For what it’s worth I’m able to get a link up on another server using a ConnectX-3 card using the same cable/transceiver. The connection right now is going to an Arista DCS-7050SX-64-F switch.
I do not see how to check FEC status on the ConnectX3-Pro side. When checking for error correction settings on the Arista side the most I can see is that it says “not supported on this platform”. Still, even just trying to link one NIC to another NIC on the same server to just get a light/link-up does not work.
For more info, it appears the parts are from HP: PSID HP_1370110017 and HP_1380110018. I’ve upgraded the firmware on both cards to 2.42.5004 and still am not able to get a link up.
I am wondering if the problem could be related to overall power output? The server is loaded with the FlexLOM’s being populated as well as the other PCIe card slots being populated, though it’s only using discrete/irig inputs for those slots.
Cannot see this transceiver in Validated and Supported 40GbE Cables list
But if your transceiver is spec compliant, it must works.
Compare the ethtool -m between the two servers, it must give information on the cable
See also HowTo Find Cable Info on Mellanox Adapters and Switches https://community.mellanox.com/s/article/howto-find-cable-info-on-mellanox-adapters-and-switches