Interface Link State is DOWN Always

Infrastructure & Networking Adapters and Cables Ethernet Adapter Cards
1

Hi Team,
As I am using Mellanox MT28800 Family [ConnectX-5 Ex], I installed MLNX_OFED software in Ubuntu 22.04 PC with the kernel version 6.2.0-35-generic. In ip a, I could see the interfaces listed,

3: enp1s0p0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
link/ether 04:3d:52:a1:12:bc brd ff:ff:ff:ff:ff:ff
4: enp1s0p1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
link/ether 04:3d:52:a1:12:bd brd ff:ff:ff:ff:ff:ff

When I try to make interface up, the link shows DOWN always [ I have connected QSFP28 with loopback ].
sudo ifconfig enp1s0p0 192.168.6.1 up
sudo ifconfig enp1s0p0 192.168.6.1 up
ip a
3: enp1s0p0: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
link/ether 04:3d:52:a1:12:bc ff:ff:ff:ff:ff:ff
inet 192.168.6.1/24 brd 192.168.6.255 scope global enp1s0p0
valid_lft forever preferred_lft forever
4: enp1s0p1: <NO-CARRIER,BROADCAST,MULTICAST,UP> mtu 1500 qdisc mq state DOWN group default qlen 1000
link/ether 04:3d:52:a1:12:bd brd ff:ff:ff:ff:ff:ff
inet 192.168.6.2/24 brd 192.168.6.255 scope global enp1s0p1
valid_lft forever preferred_lft forever

Please let me know why the Link status shows DOWN always? Even after the interface is configured for UP?

2

Hello,

If MLNX_OFED is installed, please provide the following output in order to check physical connection:
mlxlink -d -e -m -c

Best Regards,
Viki

3

Hi,

The output of mlxlink -d -e -m -c.
Operational Info

State : Polling
Physical state : ETH_AN_FSM_FIX_REVERSALS
Speed : N/A
Width : N/A
FEC : N/A
Loopback Mode : No Loopback
Auto Negotiation : ON

Supported Info

Enabled Link Speed : 0xf8f1f0d3 (100G,50G,40G,25G,10G,1G)
Supported Cable Speed : 0x2024a101 (100G,56G,50G,40G,25G,10G,1G)

Troubleshooting Info

Status Opcode : 0
Group Opcode : N/A
Recommendation : No issue was observed

Tool Information

Firmware Version : 16.35.2000
MFT Version : mft 4.25.0-62

EYE Opening Info

Physical Grade : 0, 0, 0, 0
Height Eye Opening [mV] : 0, 0, 0, 0
Phase Eye Opening [psec] : 0, 0, 0, 0

Module Info

Identifier : QSFP28
Compliance : 100GBASE-SR4 or 25GBASE-SR
Cable Technology : 850 nm VCSEL
Cable Type : Optical Module (separated)
OUI : Other
Vendor Name : FS
Vendor Part Number : QSFP28-SR4-100G
Vendor Serial Number : F2020XXXXXX
Rev : 04
Wavelength [nm] : 850
Transfer Distance [m] : 50
Attenuation (5g,7g,12g) [dB] : N/A
FW Version : N/A
Digital Diagnostic Monitoring : Yes
Power Class : 3.5 W max
CDR RX : ON,ON,ON,ON
CDR TX : ON,ON,ON,ON
LOS Alarm : N/A
Temperature [C] : 62 [-10…85]
Voltage [mV] : 3242 [2900…3600]
Bias Current [mA] : 6.012,6.006,5.998,5.992 [0…15]
Rx Power Current [dBm] : 0,0,0,0 [-18…4]
Tx Power Current [dBm] : 0,0,0,0 [-10…4]

Physical Counters and BER Info

Time Since Last Clear [Min] : N/A
Effective Physical Errors : N/A
Raw Physical Errors Per Lane : N/A
Effective Physical BER : N/A
Raw Physical BER : N/A

Note: Please ignore XXXXXX
As in above info the FEC shows N/A. How I can enable RS-FECin Ubuntu OS?

4

Hi,

From the output of mlxlink it looks like there is a negotiating issue.
Looking further in the output I can see that the cable you are using has not been tested by NVIDIA and is not supported as such.

Vendor Name : FS
Vendor Part Number : QSFP28-SR4-100G

Attached is the documentation for the 3rd party cables/switches that have been tested and supported by NVIDIA.

https://docs.nvidia.com/networking/display/connectx5firmwarev16351012/firmware+compatible+products#src-99404427_FirmwareCompatibleProducts-SupportedCablesandModules

Please review this document and try using the supported hardware.

Thanks,
Ilan.

1 Like