We have some new Dell R640 servers with ConnectX-4 NIC running CentOS 7.8. We have been trying to get them up and running, but have failed so far. The NIC is set to Ethernet and we are using the Inbox drivers in CentOS.
The interface has been configured, but no matter what I do, I am unsuccessful bringing up the line protocol.
The NIC has the latest Dell branded firmware-version: 12.26.6000 (DEL2190110032), and we are using the mlx5_core as described. Firmware found here: https://www.mellanox.com/support/firmware/dell
ETHTOOL show light is being received and sent, but the line protocol remains down:
Laser tx bias current (Channel 1) : 39.616 mA
Laser tx bias current (Channel 2) : 39.800 mA
Laser tx bias current (Channel 3) : 39.008 mA
Laser tx bias current (Channel 4) : 39.678 mA
Transmit avg optical power (Channel 1) : 1.7866 mW / 2.52 dBm
Transmit avg optical power (Channel 2) : 1.9046 mW / 2.80 dBm
Transmit avg optical power (Channel 3) : 1.8098 mW / 2.58 dBm
Transmit avg optical power (Channel 4) : 1.8976 mW / 2.78 dBm
Rcvr signal avg optical power(Channel 1) : 1.2391 mW / 0.93 dBm
Rcvr signal avg optical power(Channel 2) : 1.3948 mW / 1.45 dBm
Rcvr signal avg optical power(Channel 3) : 1.2484 mW / 0.96 dBm
Rcvr signal avg optical power(Channel 4) : 1.2208 mW / 0.87 dBm
Here is some more technical output from ETHTOOL about the optics:
Laser wavelength : 1302.350nm
Laser wavelength tolerance : 1.030nm
Vendor name : CISCO-FINISAR
Vendor OUI : 00:90:65
Vendor PN : FTLC1151RDPL-C2
Vendor rev : A
Vendor SN : FNS23020KUW
Revision Compliance : SFF-8636 Rev 2.5/2.6/2.7
Module temperature : 49.52 degrees C / 121.14 degrees F
Module voltage : 3.2212 V
What could be causing this problem? We also have some ConnectX-5 NIC and these are working just fine with the same optics and cables?
I have now installed Windows 2019 Server on the same server. The problem is the same. No link, so this seems to be related to the ConnectX-4 card vs. Cisco Finisar optics and not driver related.