SUSE Linux (HANA Server) - Showing Mellanox Card "No Link Detected"

We have SAP HANA Database installed on Lenovo Servers, with Mellanox 10G Cards. We want to upgrade current 1G copper link to 10G mellanox. We got Fiber ran to ports on servers, but server is showing “No Link Detected”. Mellanox drivers are showing loaded. Could someone please advise why it’s showing like that? and if we can fix it?

eth102 is mellanox, and connected to 10G switch port via Fiber:

uswix340# ethtool eth102

Settings for eth102:

Supported ports: [ TP ]

Supported link modes: 10000baseT/Full

Supports auto-negotiation: No

Advertised link modes: 10000baseT/Full

Advertised pause frame use: No

Advertised auto-negotiation: No

Speed: Unknown!

Duplex: Unknown! (255)

Port: Twisted Pair

PHYAD: 0

Transceiver: internal

Auto-negotiation: off

MDI-X: Unknown

Supports Wake-on: d

Wake-on: d

Current message level: 0x00000014 (20)

link ifdown

Link detected: no

uswix340# ifplugstatus

lo: link beat detected

usb0: link beat detected

eth4: link beat detected

eth5: unplugged

usb1: link beat detected

eth6: unplugged

eth7: link beat detected

eth8: unplugged

eth9: unplugged

eth10: unplugged

eth11: link beat detected

eth100: unplugged

eth101: unplugged

eth102: unplugged

eth103: unplugged

bond2: link beat detected

uswix340#

uswix340# lspci | grep Mellanox

31:00.0 Ethernet controller: Mellanox Technologies MT27500 Family [ConnectX-3]

a1:00.0 Ethernet controller: Mellanox Technologies MT27500 Family [ConnectX-3]

uswix340#

uswix340# /sbin/connectx_port_config -s


Port configuration for PCI device: 0000:31:00.0 is:

eth

eth



Port configuration for PCI device: 0000:a1:00.0 is:

auto (eth)

auto (eth)


uswix340# mst status

MST modules:


MST PCI module loaded

MST PCI configuration module loaded

MST devices:


/dev/mst/mt4099_pciconf0 - PCI configuration cycles access.

domain:bus:dev.fn=0000:31:00.0 addr.reg=88 data.reg=92

Chip revision is: 01

/dev/mst/mt4099_pciconf1 - PCI configuration cycles access.

domain:bus:dev.fn=0000:a1:00.0 addr.reg=88 data.reg=92

Chip revision is: 01

/dev/mst/mt4099_pci_cr0 - PCI direct access.

domain:bus:dev.fn=0000:31:00.0 bar=0xdf700000 size=0x100000

Chip revision is: 01

/dev/mst/mt4099_pci_cr1 - PCI direct access.

domain:bus:dev.fn=0000:a1:00.0 bar=0xf2700000 size=0x100000

Chip revision is: 01

uswix340#

137: PCI a100.0: 0200 Ethernet controller

[Created at pci.323]

UDI: /org/freedesktop/Hal/devices/pci_15b3_1003_0

Unique ID: nYg+.mx8SyyVVYd6

Parent ID: vcUh.B6xqWRvtaZ0

SysFS ID: /devices/pci0000:a0/0000:a0:02.0/0000:a1:00.0

SysFS BusID: 0000:a1:00.0

Hardware Class: network

Model: “Mellanox MT27500 Family [ConnectX-3]”

Vendor: pci 0x15b3 “Mellanox Technologies”

Device: pci 0x1003 “MT27500 Family [ConnectX-3]”

SubVendor: pci 0x15b3 “Mellanox Technologies”

SubDevice: pci 0x0077

Driver: “mlx4_core”

Driver Modules: “mlx4_core”

Device File: eth102

Memory Range: 0xf2700000-0xf27fffff (rw,non-prefetchable)

Memory Range: 0xf3800000-0xf3ffffff (ro,non-prefetchable)

IRQ: 272 (no events)

HW Address: 7c:fe:90:1a:b2:40

Link detected: no

Module Alias: “pci:v000015B3d00001003sv000015B3sd00000077bc02sc00i00”

Driver Info #0:

Driver Status: mlx4_core is active

Driver Activation Cmd: “modprobe mlx4_core”

Config Status: cfg=no, avail=yes, need=no, active=unknown

Attached to: #122 (PCI bridge)

Is the OS & NICs on the top of the proper mlnx_ofed driver & fw? what is the link-configuration? is it back-to-back connection or NICs plugged into a switch?

if to switch - check in the switch if the link-port is on the proper system profile (eth or VPI-single-switch

try to enforce eth on both PCI device ports: 0000:a1:00.0 (# /sbin/connectx_port_config -d 0000:a1:00.0 --conf eth,eth)

next, #flint -d /dev/mst/mt4099_pci_cr0 q to reveal the adapter’s PSID and then check in Mellanox website if cables are compatible with the adapter

seems to be a floppy connection to be fixed