установка OFED на Ubuntu 14.10 (проблемы с прошивкой сетевого адаптера)

zlodeynext · March 12, 2015, 8:05pm

Доброго времени суток! Возникла проблема с установкой библиотеки OFED на Ubuntu 14.10.

На сервере установлена сетевая карточка Mellanox Technologies MT25204 [InfiniHost III Lx HCA].

Скачал пакет MLNX_OFED_LINUX-2.4-1.0.4-ubuntu14.10-x86_64.tgz. Распаковал, установил все зависимости и запустил установку.

Установка прервалась на следующей операции: “Querying Mellanox devices firmware …”

В консоль вывелось следующее сообщение об ошибке: “Failed to query 0000: 07 : 00.0 device, error: File exists. MFE_OLD_DEVICE_TYPE”

Не могу понять, почему устройство не ответило на запрос. Буду рад любым советам.

alekseys1 · March 25, 2015, 1:12pm

I’m sorry, there is no serial number in the output and I afraid it will be very difficult to figure out what type it is. Try to use other MFT tools versions available from Mellanox Technologies Mellanox Firmware Tools (MFT)

zlodeynext · March 20, 2015, 10:53am

user@ivan-X7DWT:~$

user@ivan-X7DWT:~$ lspci |grep Mellanox | awk ‘{print $1}’ | xargs -i -r lspci -s {} -xxxvvv

07:00.0 InfiniBand: Mellanox Technologies MT25204 [InfiniHost III Lx HCA] (rev 20)

Subsystem: Mellanox Technologies MT25204 [InfiniHost III Lx HCA]

Control: I/O- Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-

Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- SERR- <PERR- INTx-

Interrupt: pin A routed to IRQ 30

Region 0: Memory at d8a00000 (64-bit, non-prefetchable) [size=1M]

Region 2: Memory at d8000000 (64-bit, prefetchable) [size=8M]

Capabilities:

00: b3 15 74 62 02 00 10 00 20 00 06 0c 08 00 00 00

10: 04 00 a0 d8 00 00 00 00 0c 00 00 d8 00 00 00 00

20: 00 00 00 00 00 00 00 00 00 00 00 00 b3 15 74 62

30: 00 00 00 00 40 00 00 00 00 00 00 00 0b 01 00 00

user@ivan-X7DWT:~$

zlodeynext · March 17, 2015, 7:34am

I tried . But faced with a problem.

I downloaded mft-2.7.1-7.tar, unpacked , remove the old version mft with a script , which was part of the package.

And when he tried to zaputsit ./install.sh script error occurred : “./install.sh: 30 : ./install.sh: Syntax error: " (” unexpected".

I just do not have long started to learn Linux, so can something wrong to do . Prompt …

erezf · March 17, 2015, 7:50am

What OS are you using ?

zlodeynext · March 16, 2015, 7:08am

Thanks for the advice . Tried …

root@ivan-X7DWT:~# lspci | grep Infini

07:00.0 InfiniBand: Mellanox Technologies MT25204 [InfiniHost III Lx HCA] (rev 20)

root@ivan-X7DWT:~#

root@ivan-X7DWT:~# mstflint -d 07:00.0 q

-E- Cannot open Device: 07:00.0. MFE_OLD_DEVICE_TYPE

root@ivan-X7DWT:~#

root@ivan-X7DWT:~# mstflint -d 07:00.0 qq q

-E- Cannot open Device: 07:00.0,qq. MFE_OLD_DEVICE_TYPE

root@ivan-X7DWT:~#

Are there any other options for something ?

erezf · March 14, 2015, 2:58pm

Can you try using mstflint with the physical function name ? for example :

mstflint -d 82:00.0 q

or

mstflint -d 82:00.0 qq q

alekseys1 · March 19, 2015, 1:19pm

Could you post the output of the command?

lspci |grep Mellanox | awk ‘{print $1}’ | xargs -i -r lspci -s {} -xxxvvv

alekseys1 · March 12, 2015, 8:44pm

Here are the steps that might solve the issue

Before installing MOFED

Download Mellanox Firmware tool

Run

mst start

mst status

flind -d /dev/mst/PATH_TO_DEVICE q

Obtain PSID

Find the latest firmware on mellanox site coresponing your to PSID

Burn the firmware like this

flint -d /dev/mst/PATH -i IMAGE_ANEM --nofs burn

This HCA probably is old type card that their flash size is smaller

zlodeynext · March 13, 2015, 7:25am

Я выполнил все указанные действия:

root@ivan-X7DWT:~#

root@ivan-X7DWT:~# mst start

Starting MST (Mellanox Software Tools) driver set

Loading MST PCI module - Success

Loading MST PCI configuration module - Success

Create devices

root@ivan-X7DWT:~# mst status

MST modules:

MST PCI module loaded

MST PCI configuration module loaded

MST devices:

/dev/mst/mt25204_pciconf0 - PCI configuration cycles access.

domain:bus:dev.fn=0000:07:00.0 addr.reg=88 data.reg=92

Chip revision is: A0

/dev/mst/mt25204_pci_cr0 - PCI direct access.

domain:bus:dev.fn=0000:07:00.0 bar=0xd8800000 size=0x100000

Chip revision is: A0

root@ivan-X7DWT:~# flint -d /dev/mst/mt25204_pciconf0 q

-E- Cannot open Device: /dev/mst/mt25204_pciconf0. No such file or directory MFE_OLD_DEVICE_TYPE

root@ivan-X7DWT:~# flint -d /dev/mst/mt25204_pci_cr0 q

-E- Cannot open Device: /dev/mst/mt25204_pci_cr0. No such file or directory MFE_OLD_DEVICE_TYPE

root@ivan-X7DWT:~#

не понимаю в чем дело…

alekseys1 · March 13, 2015, 1:28pm

Could you try to run ‘mstflint’ command instead of ‘flint’?

zlodeynext · March 14, 2015, 6:26am

Попробовал:

root@ivan-X7DWT:~# mst start

Starting MST (Mellanox Software Tools) driver set

Loading MST PCI module - Success

Loading MST PCI configuration module - Success

Create devices

root@ivan-X7DWT:~#

root@ivan-X7DWT:~# mst status

MST modules:

MST PCI module loaded

MST PCI configuration module loaded

MST devices:

/dev/mst/mt25204_pciconf0 - PCI configuration cycles access.

domain:bus:dev.fn=0000:07:00.0 addr.reg=88 data.reg=92

Chip revision is: A0

/dev/mst/mt25204_pci_cr0 - PCI direct access.

domain:bus:dev.fn=0000:07:00.0 bar=0xd8800000 size=0x100000

Chip revision is: A0

root@ivan-X7DWT:~#

root@ivan-X7DWT:~# mstflint -d /dev/mst/mt25204_pci_cr0 q

Unable to parse device name /dev/mst/mt25204_pci_cr0

-E- Cannot open Device: /dev/mst/mt25204_pci_cr0. Invalid argument MFE_CR_ERROR

root@ivan-X7DWT:~#

root@ivan-X7DWT:~# mstflint -d /dev/mst/mt25204_pciconf0 q

Unable to parse device name /dev/mst/mt25204_pciconf0

-E- Cannot open Device: /dev/mst/mt25204_pciconf0. Invalid argument MFE_CR_ERROR

root@ivan-X7DWT:~#

Есть ли еще какие-либо способы узнать PSID и обновить прошивку?

erezf · March 16, 2015, 10:00am

Hi,

I think the issue is that you’re using a recent version of MFT package against an old card. I guess what you need to do in order to upgrade is to use an older MFT version (below 3.0)

Can you user an older version from this archive and see if helps ? remember to uninstall your existing MFT package:

Mellanox Technologies Mellanox Firmware Tools (MFT)

zlodeynext · March 17, 2015, 8:21am

Ubuntu 14.10 (3.16.0-23-generic)

zlodeynext · March 18, 2015, 8:54am

What do you think , what could be the problem?

alekseys1 · March 20, 2015, 1:22pm

Run as root, please

zlodeynext · March 25, 2015, 7:41am

root@ivan-X7DWT:~#

root@ivan-X7DWT:~# lspci | grep Mellanox | awk ‘{print $1}’ | xargs -i -r lspci -s {} -xxxvvv

07:00.0 InfiniBand: Mellanox Technologies MT25204 [InfiniHost III Lx HCA] (rev 20)

Subsystem: Mellanox Technologies MT25204 [InfiniHost III Lx HCA]

Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+

Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- SERR- <PERR- INTx-

Latency: 0, Cache Line Size: 32 bytes

Interrupt: pin A routed to IRQ 30

Region 0: Memory at d8a00000 (64-bit, non-prefetchable) [size=1M]

Region 2: Memory at d8000000 (64-bit, prefetchable) [size=8M]

Capabilities: [40] Power Management version 2

Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)

Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-

Capabilities: [48] Vital Product Data

No end tag found

Capabilities: [90] MSI: Enable- Count=1/32 Maskable- 64bit+

Address: 0000000000000000 Data: 0000

Capabilities: [84] MSI-X: Enable+ Count=32 Masked-

Vector table: BAR=0 offset=00082000

PBA: BAR=0 offset=00082200

Capabilities: [60] Express (v1) Endpoint, MSI 00

DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 unlimited

ExtTag+ AttnBtn- AttnInd- PwrInd- RBE- FLReset-

DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-

RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-

MaxPayload 128 bytes, MaxReadReq 512 bytes

DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-

LnkCap: Port #8, Speed 2.5GT/s, Width x8, ASPM L0s, Exit Latency L0s unlimited, L1 unlimited

ClockPM- Surprise- LLActRep- BwNot-

LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk-

ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-

LnkSta: Speed 2.5GT/s, Width x8, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-

Kernel driver in use: ib_mthca

00: b3 15 74 62 06 04 10 00 20 00 06 0c 08 00 00 00

10: 04 00 a0 d8 00 00 00 00 0c 00 00 d8 00 00 00 00

20: 00 00 00 00 00 00 00 00 00 00 00 00 b3 15 74 62

30: 00 00 00 00 40 00 00 00 00 00 00 00 0b 01 00 00

40: 01 48 02 00 00 00 00 00 03 90 00 80 ff ff ff ff

50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

60: 10 00 01 00 20 0e 00 00 00 20 00 00 81 f4 03 08

70: 00 00 81 00 00 00 00 00 00 00 00 00 00 00 00 00

80: 00 00 00 00 11 60 1f 80 00 20 08 00 00 22 08 00

90: 05 84 8a 00 00 00 00 00 00 00 00 00 00 00 00 00

a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

root@ivan-X7DWT:~#

praetzel · May 6, 2016, 4:05pm

I’ve run into the exact same problem.

I started with fully patched CentOS 7 on Asus and SuperMicro motherboards using the stock InfiBand software. mst start would not create devices on CentOS 7 - although lsci showed them to be there and it looked like all appropiate drivers were installed

The InfiniHost III boards (I’ve tested 4) are taken from Isilon IQ6000 and IQ12000 systems.

A friend experienced with IB hardware said that the firmware should be reflashed but I’ve not managed to do that. He’s been unable to get the cards to work in a server which also has newer Mellanox cards.

[root ~]# lspci | grep Mellanox

02:00.0 InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex (Tavor compatibility mode) (rev 20)

[root ~]# mst start

Starting MST (Mellanox Software Tools) driver set

Loading MST PCI module - Success

Loading MST PCI configuration module - Success

Create devices

[root ~]# mst status

MST modules:

MST PCI module loaded

MST PCI configuration module loaded

MST devices:

/dev/mst/mt25208_pciconf0 - PCI configuration cycles access.

bus:dev.fn=02:00.0 addr.reg=88 data.reg=92

Chip revision is: A0

/dev/mst/mt25208_pci_cr0 - PCI direct access.

bus:dev.fn=02:00.0 bar=0xfea00000 size=0x100000

Chip revision is: A0

/dev/mst/mt25208_pci_ddr0 - PCI direct access.

bus:dev.fn=02:00.0 bar=0xdf800000 size=0x800000

[root ~]# ibv_devinfo

No IB devices found

[root ~]# flint -d /dev/mst/mt25208_pci_cr0 query

Received CFI query from addr 0x10: [▒&]

Received CFI query from addr 0x20: [Z]

-E- Can not open /dev/mst/mt25208_pci_cr0: MFE_CFI_FAILED

[root MLNX_OFED_LINUX-1.5.3-4.0.42-rhel5.10-x86_64]# mstflint -d 0:02.0 -i fw-25208-4_8_200-MHEL-CF128.bin b

Warning: memory access to device 0:02.0 failed: No such device or address.

Warning: Fallback on IO: much slower, and unsafe if device in use.

-E- Can not open 0:02.0: No such file or directory MFE_CR_ERROR

[root MLNX_OFED_LINUX-1.5.3-4.0.42-rhel5.10-x86_64]# mstflint -d /dev/mst/mt25209_pciconf0 -i fw-25208-4_8_200-MHEL-CF128.bin b

Unable to parse device name /dev/mst/mt25209_pciconf0

-E- Can not open /dev/mst/mt25209_pciconf0: Invalid argument MFE_CR_ERROR

[root ~]# lspci |grep Mellanox | awk ‘{print $1}’ | xargs -i -r lspci -s {} -xxxvvv

02:00.0 InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex (Tavor compatibility mode) (rev 20)

Subsystem: Device 4953:0008

Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-

Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- SERR- <PERR- INTx-

Latency: 0, Cache Line Size: 64 bytes

Interrupt: pin A routed to IRQ 169

Region 0: Memory at fea00000 (64-bit, non-prefetchable) [size=1M]

Region 2: Memory at df800000 (64-bit, prefetchable) [size=8M]

Region 4: Memory at a0000000 (64-bit, prefetchable) [size=512M]

Capabilities: [40] Power Management version 2

Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)

Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-

Capabilities: [90] MSI: Enable- Count=1/32 Maskable- 64bit+

Address: 0000000000000000 Data: 0000

Capabilities: [84] MSI-X: Enable- Count=32 Masked-

Vector table: BAR=0 offset=00082000

PBA: BAR=0 offset=00082200

Capabilities: [60] Express (v1) Endpoint, MSI 00

DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 unlimited

ExtTag+ AttnBtn- AttnInd- PwrInd- RBE- FLReset-

DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-

RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-

MaxPayload 128 bytes, MaxReadReq 4096 bytes

DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-

LnkCap: Port #8, Speed 2.5GT/s, Width x8, ASPM L0s, Latency L0 unlimited, L1 unlimited

ClockPM- Surprise- LLActRep- BwNot-

LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+

ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-

LnkSta: Speed 2.5GT/s, Width x8, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-

Kernel modules: ib_mthca

00: b3 15 78 62 06 01 10 00 20 00 06 0c 10 00 00 00

10: 04 00 a0 fe 00 00 00 00 0c 00 80 df 00 00 00 00

20: 0c 00 00 a0 00 00 00 00 00 00 00 00 53 49 08 00

30: 00 00 00 00 40 00 00 00 00 00 00 00 0a 01 00 00

40: 01 90 02 00 00 00 00 00 03 90 ff 7f 11 11 00 00

50: 00 00 00 00 00 00 00 00 14 00 0f 00 78 62 a0 02

60: 10 00 01 00 20 0e 2c 01 00 50 00 00 81 f4 03 08

70: 40 00 81 10 00 00 00 00 00 00 00 00 00 00 00 00

80: 00 00 00 00 11 60 1f 00 00 20 08 00 00 22 08 00

90: 05 84 8a 00 00 00 00 00 00 00 00 00 00 00 00 00

a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

Topic		Replies	Views
mlnx_add_kernel_support.sh and magic version errors. Mellanox OFED	5	1419	April 21, 2016
Dell M1000e blade server, InfiniBand QDR subnet issue, OFED 4.4, opensm initialization error! InfiniBand/VPI Switch Systems infiniband , port , opensm , winof-driver , ifconfig	22	1507	December 12, 2018
Mixing OFED 1.5.3 and 2.2 in the same network? Mellanox OFED	4	432	June 18, 2017
Mlnx_ofed install error Mellanox OFED kernel	6	1338	August 23, 2023
Error /hca_self_test.ofed: line 165: [: too many arguments Mellanox OFED	13	455	August 28, 2014
Kernel Modules from Mellanox OFED Stack Won't Load Mellanox OFED	13	604	August 9, 2013
Trouble with ConnectX-3 VPI adapter card over XenServer 6.2 (Service Pack 1)	11	377	June 3, 2014
Can not set CX6 DX to switchdev mode InfiniBand/VPI Adapter Cards hw	10	1636	July 18, 2024
MLNX_OFED_LINUX installtion Error: Error: The current MLNX_OFED_LINUX is intended for rhel6.4 Mellanox OFED ibv_devices	12	2287	January 13, 2014
MHGH28-XTC not working InfiniBand/VPI Adapter Cards	52	644	May 25, 2013

установка OFED на Ubuntu 14.10 (проблемы с прошивкой сетевого адаптера)

Related topics