Unable to use connect-X3 on ubuntu 18 with ethernet because there is no LINK_TYPE_P1 option

Hi,

I use connect-X3 on windows10 with ethernet mode, it works fine.

However, connect-X3 does not work on 18.04 LTS because there is no LINK_TYPE_P1 option.


# mlxconfig -d /dev/mst/mt4099_pciconf0 q

Device #1:

----------

Device type: ConnectX3

Device: /dev//mt4099_pciconf0

Configurations: Next Boot

BOOT_OPTION_ROM_EN_P1 True1)

BOOT_VLAN_EN_P1 False0)

BOOT_RETRY_CNT_P1 0

LEGACY_BOOT_PROTOCOL_P1 PXE1)

BOOT_VLAN_P1 1

BOOT_OPTION_ROM_EN_P2 True1)

BOOT_VLAN_EN_P2 False0)

BOOT_RETRY_CNT_P2 0

LEGACY_BOOT_PROTOCOL_P2 PXE1)

BOOT_VLAN_P2 1

IP_VER_P1 IPv40)

IP_VER_P2 IPv40)


#sudo mstconfig -d 01:00.0 s LINK_TYPE_P1=ETH

Device #1:

----------

Device type: ConnectX3

PCI device: 01:00.0

Configurations: Next Boot New

-E- Device doesn't support LINK_TYPE_P1 configuration


# mlxburn -d /dev/mst/mt4099_pciconf0 -vpd

VPD-KEYWORD DESCRIPTION VALUE

----------- ----------- -----

Read Only Section:

PN Part Number 00W0038

EC Revision A2

SN Serial Number 00W0038YK5020000FD2

V0 Misc Info PCIe Gen3 x8

RV Checksum Complement 0x6

IDTAG Board Id CX353A - ConnectX-3 QSFP

How to fix this problem?

Can I see the following output:

mst version

flint -d /dev/mst/mt4099_pciconf0 q

lspci -s 01:00.0 -vvvxxx | grep -i part

This card (00W0038) comes up as IBM MELLANOX INFINIBAND CONNECTX-3 VPI CX353A HCA SINGLE PORT FDR14 56GB/S OR 40/56GB/S.

odd that it shows parameters for port 2

Dear Sophie,

Thank you for your reply.​


$ sudo mst version

mst, mft 4.12.0-105, built on Apr 29 2019, 14:34:01. Git SHA Hash: a197f32

$ sudo flint -d /dev/mst/mt4099_pciconf0 q

Image type: FS2

FW Version: 2.40.5030

FW Release Date: 4.1.2017

Device ID: 4099

Description: Node Port1 Port2 Sys image

GUIDs: f4521403001ab390 f4521403001ab391 f4521403001ab392 f4521403001ab393

MACs: f452141ab390 f452141ab391

VSD:

PSID: IBM1100110019

​

#sudo lspci -s 01:00.0 -vvvxxx | grep -i part

[PN] Part number: 00W0038

I think this problem caused by ubuntu 18.04 LTS with kernel 5.0.0-29-generic which cannot install mlnx-en-dkms of ethernet driver.


DKMS make.log for mlnx-en-4.6 for kernel 5.0.0-29-generic (x86_64)

cc1: some warnings being treated as errors

scripts/Makefile.build:284: recipe for target '/var/lib/dkms/mlnx-en/4.6/build/$

make[3]: *** [/var/lib/dkms/mlnx-en/4.6/build/drivers/net/ethernet/mellanox/mlx$

make[3]: *** Waiting for unfinished jobs....

scripts/Makefile.build:515: recipe for target '/var/lib/dkms/mlnx-en/4.6/build/$

make[2]: *** [/var/lib/dkms/mlnx-en/4.6/build/drivers/net/ethernet/mellanox/mlx$

Makefile:1606: recipe for target '_module_/var/lib/dkms/mlnx-en/4.6/build' fail$

make[1]: *** [_module_/var/lib/dkms/mlnx-en/4.6/build] Error 2

make[1]: Leaving directory '/usr/src/linux-headers-5.0.0-29-generic'

makefile:136: recipe for target 'kernel' failed

make: *** [kernel] Error 2

​Sincerely,

Sincerely,

I am not sure if it correlates to DKMS. You installed the MFT package 4.12 manually which should allow you to change the parameters from this card.

This card is an Mellanox ConnectX-3 VPI Single-port QSFP FDR14/40GbE HCA OEM’d by IBM.

Why is it showing then some parameters for port 2? This is a single port card. Was the FW burned previously forcing a different PSID?

Is it a different card or the same card that was installed on Windows?

If different, what is the FW version on the card on Windows?

Our latest GA FW is 2.42.5000, can you flash this FW (downloadable from IBM website).

You can then validate parameters via the mlxconfig.

I would recommend flashing the FW using the correct image for that PSID.

Sophie.

I would also recommend to contact IBM directly if further technical assistance would be needed.