mlx5_core enable hca failed, mlx5_load_one failed with error code -22

I have two physical cards each with two ports but only one Physical card is getting enabled and getting following error on the dmesg

mlx5_core enable hca failed, mlx5_load_one failed with error code -22

for i in $(lspci | grep Mellanox | awk ‘{print $1}’); do echo $i; lspci -vv -s $i | grep “Part number” -A 3; done

03:00.0

[PN] Part number: MCX456A-ECAT

[EC] Engineering changes: A4

[SN] Serial number: MT1514X00006

[V0] Vendor specific: PCIeGen3 x16

03:00.1

[PN] Part number: MCX456A-ECAT

[EC] Engineering changes: A4

[SN] Serial number: MT1514X00006

[V0] Vendor specific: PCIeGen3 x16

90:00.0

[PN] Part number: MCX456A-ECAT

[EC] Engineering changes: A8

[SN] Serial number: MT1621K00548

[V0] Vendor specific: PCIeGen3 x16

90:00.1

[PN] Part number: MCX456A-ECAT

[EC] Engineering changes: A8

[SN] Serial number: MT1621K00548

[V0] Vendor specific: PCIeGen3 x16

$ lspci | grep Mellanox

03:00.0 Infiniband controller: Mellanox Technologies MT27700 Family [ConnectX-4]

03:00.1 Infiniband controller: Mellanox Technologies MT27700 Family [ConnectX-4]

90:00.0 Infiniband controller: Mellanox Technologies MT27700 Family [ConnectX-4]

90:00.1 Infiniband controller: Mellanox Technologies MT27700 Family [ConnectX-4]

[pasokan@sn18 ~]$

$ ibstat

CA ‘mlx5_0’

CA type: MT4115

Number of ports: 1

Firmware version: 12.22.1002

Hardware version: 0

Node GUID: 0x248a0703001cdfd4

System image GUID: 0x248a0703001cdfd4

Port 1:

State: Active

Physical state: LinkUp

Rate: 56

Base lid: 128

LMC: 0

SM lid: 1

Capability mask: 0x2651e848

Port GUID: 0x248a0703001cdfd4

Link layer: InfiniBand

CA ‘mlx5_1’

CA type: MT4115

Number of ports: 1

Firmware version: 12.22.1002

Hardware version: 0

Node GUID: 0x248a0703001cdfd5

System image GUID: 0x248a0703001cdfd4

Port 1:

State: Active

Physical state: LinkUp

Rate: 56

Base lid: 129

LMC: 0

SM lid: 1

Capability mask: 0x2651e848

Port GUID: 0x248a0703001cdfd5

Link layer: InfiniBand

$

Hi Pharthiphan,

I am not sure if your issue is still relevant as it was posted on 6/11, however what Mellanox OFED Drivers did you installed and have you validated the FW version/compatibility?

You can download the MFT package from the following link:

Mellanox Firmware Tools (MFT) Mellanox Firmware Tools (MFT)

To query the FW:

#mst start

#mst status -v

#flint -d q

Note: Check based on the RN of the Drivers that the FW is supported/compatible. If not, I would suggest to align the FW to a supported version.

Sophie.