установка OFED на Ubuntu 14.10 (проблемы с прошивкой сетевого адаптера)

Доброго времени суток! Возникла проблема с установкой библиотеки OFED на Ubuntu 14.10.

На сервере установлена сетевая карточка Mellanox Technologies MT25204 [InfiniHost III Lx HCA].

Скачал пакет MLNX_OFED_LINUX-2.4-1.0.4-ubuntu14.10-x86_64.tgz. Распаковал, установил все зависимости и запустил установку.

Установка прервалась на следующей операции: "Querying Mellanox devices firmware …"

В консоль вывелось следующее сообщение об ошибке: "Failed to query 0000: 07 : 00.0 device, error: File exists. MFE_OLD_DEVICE_TYPE"

Не могу понять, почему устройство не ответило на запрос. Буду рад любым советам.

I’m sorry, there is no serial number in the output and I afraid it will be very difficult to figure out what type it is. Try to use other MFT tools versions available from Mellanox Technologies http://www.mellanox.com/page/management_tools

user@ivan-X7DWT:~$

user@ivan-X7DWT:~$ lspci |grep Mellanox | awk ‘{print $1}’ | xargs -i -r lspci -s {} -xxxvvv

07:00.0 InfiniBand: Mellanox Technologies MT25204 [InfiniHost III Lx HCA] (rev 20)

Subsystem: Mellanox Technologies MT25204 [InfiniHost III Lx HCA]

Control: I/O- Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx-

Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- SERR- <PERR- INTx-

Interrupt: pin A routed to IRQ 30

Region 0: Memory at d8a00000 (64-bit, non-prefetchable) [size=1M]

Region 2: Memory at d8000000 (64-bit, prefetchable) [size=8M]

Capabilities:

00: b3 15 74 62 02 00 10 00 20 00 06 0c 08 00 00 00

10: 04 00 a0 d8 00 00 00 00 0c 00 00 d8 00 00 00 00

20: 00 00 00 00 00 00 00 00 00 00 00 00 b3 15 74 62

30: 00 00 00 00 40 00 00 00 00 00 00 00 0b 01 00 00

user@ivan-X7DWT:~$

user@ivan-X7DWT:~$

user@ivan-X7DWT:~$

I tried . But faced with a problem.

I downloaded mft-2.7.1-7.tar, unpacked , remove the old version mft with a script , which was part of the package.

And when he tried to zaputsit ./install.sh script error occurred : “./install.sh: 30 : ./install.sh: Syntax error: " (” unexpected".

I just do not have long started to learn Linux, so can something wrong to do . Prompt …

What OS are you using ?

Thanks for the advice . Tried …

root@ivan-X7DWT:~# lspci | grep Infini

07:00.0 InfiniBand: Mellanox Technologies MT25204 [InfiniHost III Lx HCA] (rev 20)

root@ivan-X7DWT:~#

root@ivan-X7DWT:~#

root@ivan-X7DWT:~#

root@ivan-X7DWT:~# mstflint -d 07:00.0 q

-E- Cannot open Device: 07:00.0. MFE_OLD_DEVICE_TYPE

root@ivan-X7DWT:~#

root@ivan-X7DWT:~#

root@ivan-X7DWT:~# mstflint -d 07:00.0 qq q

-E- Cannot open Device: 07:00.0,qq. MFE_OLD_DEVICE_TYPE

root@ivan-X7DWT:~#

Are there any other options for something ?

Can you try using mstflint with the physical function name ? for example :

mstflint -d 82:00.0 q

or

mstflint -d 82:00.0 qq q

Could you post the output of the command?

lspci |grep Mellanox | awk ‘{print $1}’ | xargs -i -r lspci -s {} -xxxvvv

Here are the steps that might solve the issue

Before installing MOFED

Download Mellanox Firmware tool

Run

mst start

mst status

flind -d /dev/mst/PATH_TO_DEVICE q

Obtain PSID

Find the latest firmware on mellanox site coresponing your to PSID

Burn the firmware like this

flint -d /dev/mst/PATH -i IMAGE_ANEM --nofs burn

This HCA probably is old type card that their flash size is smaller

Я выполнил все указанные действия:

root@ivan-X7DWT:~#

root@ivan-X7DWT:~# mst start

Starting MST (Mellanox Software Tools) driver set

Loading MST PCI module - Success

Loading MST PCI configuration module - Success

Create devices

root@ivan-X7DWT:~# mst status

MST modules:


MST PCI module loaded

MST PCI configuration module loaded

MST devices:


/dev/mst/mt25204_pciconf0 - PCI configuration cycles access.

domain:bus:dev.fn=0000:07:00.0 addr.reg=88 data.reg=92

Chip revision is: A0

/dev/mst/mt25204_pci_cr0 - PCI direct access.

domain:bus:dev.fn=0000:07:00.0 bar=0xd8800000 size=0x100000

Chip revision is: A0

root@ivan-X7DWT:~# flint -d /dev/mst/mt25204_pciconf0 q

-E- Cannot open Device: /dev/mst/mt25204_pciconf0. No such file or directory MFE_OLD_DEVICE_TYPE

root@ivan-X7DWT:~# flint -d /dev/mst/mt25204_pci_cr0 q

-E- Cannot open Device: /dev/mst/mt25204_pci_cr0. No such file or directory MFE_OLD_DEVICE_TYPE

root@ivan-X7DWT:~#

не понимаю в чем дело…

Could you try to run ‘mstflint’ command instead of ‘flint’?

Попробовал:

root@ivan-X7DWT:~# mst start

Starting MST (Mellanox Software Tools) driver set

Loading MST PCI module - Success

Loading MST PCI configuration module - Success

Create devices

root@ivan-X7DWT:~#

root@ivan-X7DWT:~#

root@ivan-X7DWT:~# mst status

MST modules:


MST PCI module loaded

MST PCI configuration module loaded

MST devices:


/dev/mst/mt25204_pciconf0 - PCI configuration cycles access.

domain:bus:dev.fn=0000:07:00.0 addr.reg=88 data.reg=92

Chip revision is: A0

/dev/mst/mt25204_pci_cr0 - PCI direct access.

domain:bus:dev.fn=0000:07:00.0 bar=0xd8800000 size=0x100000

Chip revision is: A0

root@ivan-X7DWT:~#

root@ivan-X7DWT:~#

root@ivan-X7DWT:~# mstflint -d /dev/mst/mt25204_pci_cr0 q

Unable to parse device name /dev/mst/mt25204_pci_cr0

-E- Cannot open Device: /dev/mst/mt25204_pci_cr0. Invalid argument MFE_CR_ERROR

root@ivan-X7DWT:~#

root@ivan-X7DWT:~#

root@ivan-X7DWT:~#

root@ivan-X7DWT:~# mstflint -d /dev/mst/mt25204_pciconf0 q

Unable to parse device name /dev/mst/mt25204_pciconf0

-E- Cannot open Device: /dev/mst/mt25204_pciconf0. Invalid argument MFE_CR_ERROR

root@ivan-X7DWT:~#

Есть ли еще какие-либо способы узнать PSID и обновить прошивку?

Hi,

I think the issue is that you’re using a recent version of MFT package against an old card. I guess what you need to do in order to upgrade is to use an older MFT version (below 3.0)

Can you user an older version from this archive and see if helps ? remember to uninstall your existing MFT package:

Mellanox Technologies http://www.mellanox.com/page/management_tools

Ubuntu 14.10 (3.16.0-23-generic)

What do you think , what could be the problem?

Run as root, please

root@ivan-X7DWT:~#

root@ivan-X7DWT:~#

root@ivan-X7DWT:~# lspci | grep Mellanox | awk ‘{print $1}’ | xargs -i -r lspci -s {} -xxxvvv

07:00.0 InfiniBand: Mellanox Technologies MT25204 [InfiniHost III Lx HCA] (rev 20)

Subsystem: Mellanox Technologies MT25204 [InfiniHost III Lx HCA]

Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR- FastB2B- DisINTx+

Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- SERR- <PERR- INTx-

Latency: 0, Cache Line Size: 32 bytes

Interrupt: pin A routed to IRQ 30

Region 0: Memory at d8a00000 (64-bit, non-prefetchable) [size=1M]

Region 2: Memory at d8000000 (64-bit, prefetchable) [size=8M]

Capabilities: [40] Power Management version 2

Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)

Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-

Capabilities: [48] Vital Product Data

No end tag found

Capabilities: [90] MSI: Enable- Count=1/32 Maskable- 64bit+

Address: 0000000000000000 Data: 0000

Capabilities: [84] MSI-X: Enable+ Count=32 Masked-

Vector table: BAR=0 offset=00082000

PBA: BAR=0 offset=00082200

Capabilities: [60] Express (v1) Endpoint, MSI 00

DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 unlimited

ExtTag+ AttnBtn- AttnInd- PwrInd- RBE- FLReset-

DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-

RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-

MaxPayload 128 bytes, MaxReadReq 512 bytes

DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-

LnkCap: Port #8, Speed 2.5GT/s, Width x8, ASPM L0s, Exit Latency L0s unlimited, L1 unlimited

ClockPM- Surprise- LLActRep- BwNot-

LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- CommClk-

ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-

LnkSta: Speed 2.5GT/s, Width x8, TrErr- Train- SlotClk- DLActive- BWMgmt- ABWMgmt-

Kernel driver in use: ib_mthca

00: b3 15 74 62 06 04 10 00 20 00 06 0c 08 00 00 00

10: 04 00 a0 d8 00 00 00 00 0c 00 00 d8 00 00 00 00

20: 00 00 00 00 00 00 00 00 00 00 00 00 b3 15 74 62

30: 00 00 00 00 40 00 00 00 00 00 00 00 0b 01 00 00

40: 01 48 02 00 00 00 00 00 03 90 00 80 ff ff ff ff

50: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

60: 10 00 01 00 20 0e 00 00 00 20 00 00 81 f4 03 08

70: 00 00 81 00 00 00 00 00 00 00 00 00 00 00 00 00

80: 00 00 00 00 11 60 1f 80 00 20 08 00 00 22 08 00

90: 05 84 8a 00 00 00 00 00 00 00 00 00 00 00 00 00

a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

root@ivan-X7DWT:~#

root@ivan-X7DWT:~#

root@ivan-X7DWT:~#

I’ve run into the exact same problem.

I started with fully patched CentOS 7 on Asus and SuperMicro motherboards using the stock InfiBand software. mst start would not create devices on CentOS 7 - although lsci showed them to be there and it looked like all appropiate drivers were installed

The InfiniHost III boards (I’ve tested 4) are taken from Isilon IQ6000 and IQ12000 systems.

A friend experienced with IB hardware said that the firmware should be reflashed but I’ve not managed to do that. He’s been unable to get the cards to work in a server which also has newer Mellanox cards.

[root ~]# lspci | grep Mellanox

02:00.0 InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex (Tavor compatibility mode) (rev 20)

[root ~]# mst start

Starting MST (Mellanox Software Tools) driver set

Loading MST PCI module - Success

Loading MST PCI configuration module - Success

Create devices

[root ~]# mst status

MST modules:


MST PCI module loaded

MST PCI configuration module loaded

MST devices:


/dev/mst/mt25208_pciconf0 - PCI configuration cycles access.

bus:dev.fn=02:00.0 addr.reg=88 data.reg=92

Chip revision is: A0

/dev/mst/mt25208_pci_cr0 - PCI direct access.

bus:dev.fn=02:00.0 bar=0xfea00000 size=0x100000

Chip revision is: A0

/dev/mst/mt25208_pci_ddr0 - PCI direct access.

bus:dev.fn=02:00.0 bar=0xdf800000 size=0x800000

[root ~]# ibv_devinfo

No IB devices found

[root ~]# flint -d /dev/mst/mt25208_pci_cr0 query

Received CFI query from addr 0x10: [▒&]

Received CFI query from addr 0x20: [Z]

-E- Can not open /dev/mst/mt25208_pci_cr0: MFE_CFI_FAILED

[root MLNX_OFED_LINUX-1.5.3-4.0.42-rhel5.10-x86_64]# mstflint -d 0:02.0 -i fw-25208-4_8_200-MHEL-CF128.bin b

Warning: memory access to device 0:02.0 failed: No such device or address.

Warning: Fallback on IO: much slower, and unsafe if device in use.

-E- Can not open 0:02.0: No such file or directory MFE_CR_ERROR

[root MLNX_OFED_LINUX-1.5.3-4.0.42-rhel5.10-x86_64]# mstflint -d /dev/mst/mt25209_pciconf0 -i fw-25208-4_8_200-MHEL-CF128.bin b

Unable to parse device name /dev/mst/mt25209_pciconf0

-E- Can not open /dev/mst/mt25209_pciconf0: Invalid argument MFE_CR_ERROR

[root ~]# lspci |grep Mellanox | awk ‘{print $1}’ | xargs -i -r lspci -s {} -xxxvvv

02:00.0 InfiniBand: Mellanox Technologies MT25208 InfiniHost III Ex (Tavor compatibility mode) (rev 20)

Subsystem: Device 4953:0008

Control: I/O- Mem+ BusMaster+ SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-

Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- SERR- <PERR- INTx-

Latency: 0, Cache Line Size: 64 bytes

Interrupt: pin A routed to IRQ 169

Region 0: Memory at fea00000 (64-bit, non-prefetchable) [size=1M]

Region 2: Memory at df800000 (64-bit, prefetchable) [size=8M]

Region 4: Memory at a0000000 (64-bit, prefetchable) [size=512M]

Capabilities: [40] Power Management version 2

Flags: PMEClk- DSI- D1- D2- AuxCurrent=0mA PME(D0-,D1-,D2-,D3hot-,D3cold-)

Status: D0 NoSoftRst- PME-Enable- DSel=0 DScale=0 PME-

Capabilities: [90] MSI: Enable- Count=1/32 Maskable- 64bit+

Address: 0000000000000000 Data: 0000

Capabilities: [84] MSI-X: Enable- Count=32 Masked-

Vector table: BAR=0 offset=00082000

PBA: BAR=0 offset=00082200

Capabilities: [60] Express (v1) Endpoint, MSI 00

DevCap: MaxPayload 128 bytes, PhantFunc 0, Latency L0s <64ns, L1 unlimited

ExtTag+ AttnBtn- AttnInd- PwrInd- RBE- FLReset-

DevCtl: Report errors: Correctable- Non-Fatal- Fatal- Unsupported-

RlxdOrd- ExtTag- PhantFunc- AuxPwr- NoSnoop-

MaxPayload 128 bytes, MaxReadReq 4096 bytes

DevSta: CorrErr- UncorrErr- FatalErr- UnsuppReq- AuxPwr- TransPend-

LnkCap: Port #8, Speed 2.5GT/s, Width x8, ASPM L0s, Latency L0 unlimited, L1 unlimited

ClockPM- Surprise- LLActRep- BwNot-

LnkCtl: ASPM Disabled; RCB 64 bytes Disabled- Retrain- CommClk+

ExtSynch- ClockPM- AutWidDis- BWInt- AutBWInt-

LnkSta: Speed 2.5GT/s, Width x8, TrErr- Train- SlotClk+ DLActive- BWMgmt- ABWMgmt-

Kernel modules: ib_mthca

00: b3 15 78 62 06 01 10 00 20 00 06 0c 10 00 00 00

10: 04 00 a0 fe 00 00 00 00 0c 00 80 df 00 00 00 00

20: 0c 00 00 a0 00 00 00 00 00 00 00 00 53 49 08 00

30: 00 00 00 00 40 00 00 00 00 00 00 00 0a 01 00 00

40: 01 90 02 00 00 00 00 00 03 90 ff 7f 11 11 00 00

50: 00 00 00 00 00 00 00 00 14 00 0f 00 78 62 a0 02

60: 10 00 01 00 20 0e 2c 01 00 50 00 00 81 f4 03 08

70: 40 00 81 10 00 00 00 00 00 00 00 00 00 00 00 00

80: 00 00 00 00 11 60 1f 00 00 20 08 00 00 22 08 00

90: 05 84 8a 00 00 00 00 00 00 00 00 00 00 00 00 00

a0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

b0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

c0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

d0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

e0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

f0: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00