MLNX_OFED_LINUX installtion Error: Error: The current MLNX_OFED_LINUX is intended for rhel6.4

Infrastructure & Networking Software And Drivers Mellanox OFED
1

Hi

Just try to install MLNX_OFED_LINUX-2.0-3.0.0-rhel6.4-x86_64.iso on Oracle Enterprise Linux 6.4. The command ./mlnxofedinstall gives error: The current MLNX_OFED_LINUX is intended for rhel6.4

uname -r is 2.6.32-358.el6.x86_64 (change to rehl kernel)

Tried also to run new installation package:

./mlnx_add_kernel_support.sh -m /mnt --make-tgz -k 2.6.32-358.el6.x86_64 -s /lib/modules/2.6.32-358.el6.x86_64/build -n MLNX-NEW

creation success, but when I try to install the same error:

error: The current MLNX_OFED_LINUX is intended for rhel6.4

Any Idea ?

Many thanks

2

did execute the commands, please see below:

[root@xxx]# lspci -d 15b3:

03:00.0 Network controller: Mellanox Technologies MT27500 Family [ConnectX-3]

[root@xxx]# lsmod | grep mlx

mlx5_ib 78913 0

mlx5_core 72388 1 mlx5_ib

mlx4_en 78738 0

mlx4_ib 155891 0

ib_sa 24010 5 rdma_ucm,rdma_cm,ib_ipoib,ib_cm,mlx4_ib

ib_mad 43177 4 ib_cm,ib_umad,mlx4_ib,ib_sa

ib_core 81943 12 rdma_ucm,ib_ucm,rdma_cm,iw_cm,ib_ipoib,ib_cm,ib_uverbs,ib_umad,mlx5_ib,mlx4_ib,ib_sa,ib_mad

mlx4_core 240441 2 mlx4_en,mlx4_ib

compat 22986 17 rdma_ucm,ib_ucm,rdma_cm,iw_cm,ib_addr,ib_ipoib,ib_cm,ib_uverbs,ib_umad,mlx5_ib,mlx5_core,mlx4_en,mlx4_ib,ib_sa,ib_mad,ib_core,mlx4_core

[root@xxx]# ibv_devices

device node GUID

…still no device visible, any other idea ? Many thanks in advance

3

Hi here is the output:

ls -l /dev/infiniband/

total 0

crw-rw-rw-. 1 root root 10, 56 Jan 1 10:33 rdma_cm

and this:

cat /etc/udev/rules.d/90-ib.rules

KERNEL==“umad*”, NAME=“infiniband/%k” MODE=“0666”

KERNEL==“issm*”, NAME=“infiniband/%k”

KERNEL==“ucm*”, NAME=“infiniband/%k”, MODE=“0666”

KERNEL==“uverbs*”, NAME=“infiniband/%k”, MODE=“0666”

KERNEL==“ucma”, NAME=“infiniband/%k”, MODE=“0666”

KERNEL==“rdma_cm”, NAME=“infiniband/%k”, MODE=“0666”

What to do next ?

4

Stop openibd

Turn debug on for udev.

udevadm control --log-priority debug

udevadm control --reload-rules

Start openibd

And search for 90-ib.rules in /var/log/messages

These are logs when it works, but you should see errors that can give use some clues

Jan 6 23:08:40 my-host udevd-work[11143]: MODE 0666 /etc/udev/rules.d/90-ib.rules:3

Jan 6 23:08:40 my-host udevd-work[11143]: NAME ‘infiniband/ucm0’ /etc/udev/rules.d/90-ib.rules:3

Jan 6 23:08:40 my-host udevd-work[11143]: create db link (infiniband/ucm0 char/231:224)

Jan 6 23:08:40 my-host udevd-work[11143]: creating device node ‘/dev/infiniband/ucm0’, devnum=231:224, mode=0666, uid=0, gid=0

Jan 6 23:08:40 my-host udevd-work[11143]: preserve file ‘/dev/infiniband/ucm0’, because it has correct dev_t

Jan 6 23:08:40 my-host udevd-work[11143]: preserve permissions /dev/infiniband/ucm0, 020666, uid=0, gid=0

Jan 6 23:08:40 my-host udevd-work[11143]: creating symlink ‘/dev/char/231:224’ to ‘…/infiniband/ucm0’

5

Ok did install OEL OFED.

Still have the issue that I dont see the ib0

Could it be that I have already configured eth1 as ethernt ?

see output (is empty):

#ibv_devices

device node GUID

6

Hi,

Since you’re using OEL 6.4 ,I would recommend downloading the version for OEL 6.4

Here the Iso link:

Mellanox Technologies: MLNX_OFED EULA http://www.mellanox.com/page/mlnx_ofed_eula?mtag=linux_sw_drivers&mrequest=downloads&mtype=ofed&mver=MLNX_OFED-2.0-3.0.0&mname=MLNX_OFED_LINUX-2.0-3.0.0-oel6.4-x86_64.iso

Or TGZ file"

MLNX_OFED_LINUX-2.0-3.0.0-oel6.4-x86_64.tgz http://www.mellanox.com/page/mlnx_ofed_eula?mtag=linux_sw_drivers&mrequest=downloads&mtype=ofed&mver=MLNX_OFED-2.0-3.0.0&mname=MLNX_OFED_LINUX-2.0-3.0.0-oel6.4-x86_64.tgz

7

Hi

Thanks that helped out with --skip-distro-check

My question now, I think hardware is working. But I dont see any Ib0 interface ? How to get the infiniband interface name ?

Thanks

./hca_self_test.ofed

---- Performing Adapter Device Self Test ----

Number of CAs Detected … 1

PCI Device Check … PASS

Kernel Arch … x86_64

Host Driver Version … MLNX_OFED_LINUX-2.0-3.0.0 (OFED-2.0-3.0.0): 2.6.32-358.el6.x86_64

Host Driver RPM Check … PASS

Firmware on CA #0 VPI … v2.30.3200

Firmware Check on CA #0 (VPI) … PASS

NOTE: The found fw version is higher than the fw included in this package (v2.30.3110)

Host Driver Initialization … PASS

Number of CA Ports Active … 0

Kernel Syslog Check … PASS

Node GUID on CA #0 (VPI) … NA

------------------ DONE ---------------------

8

You can skip the distro check with --skip-distro-check

You might also need --without-depcheck

Cheers,

sorin

9

As Rian pointed out, you should install OEL OFED (I had no idea it is available)

ibdev2netdev should show something like this

mlx4_0 port 1 ==> ib0 (Up)

mlx4_0 port 2 ==> eth1 (Up)

10

ibv_devices should show something like this, regardless how the ports are configured

device node GUID

mlx4_0 0002c9030…

Do you see the adapter with this command?

lspci -d 15b3:

If yes, can you check if mlx modules are loaded? lsmod | grep mlx

11

Can you check this dir?

ls -l /dev/infiniband/

total 0

crw-rw---- 1 root root 231, 64 Jan 6 18:57 issm0

crw-rw---- 1 root root 231, 65 Jan 6 18:57 issm1

crw-rw-rw- 1 root root 10, 56 Dec 28 00:17 rdma_cm

crw-rw-rw- 1 root root 231, 224 Dec 28 00:17 ucm0

crw-rw-rw- 1 root root 231, 0 Jan 6 18:57 umad0

crw-rw-rw- 1 root root 231, 1 Jan 6 18:57 umad1

crw-rw-rw- 1 root root 231, 192 Dec 28 00:17 uverbs0

Those are created by udev and the rules should look like this.

If those rules are missing or have wrong permissions, that might be the cause.

cat /etc/udev/rules.d/90-ib.rules

KERNEL==“umad*”, NAME=“infiniband/%k” MODE=“0666”

KERNEL==“issm*”, NAME=“infiniband/%k”

KERNEL==“ucm*”, NAME=“infiniband/%k”, MODE=“0666”

KERNEL==“uverbs*”, NAME=“infiniband/%k”, MODE=“0666”

KERNEL==“ucma”, NAME=“infiniband/%k”, MODE=“0666”

KERNEL==“rdma_cm”, NAME=“infiniband/%k”, MODE=“0666”

12

I did plug in the 2 infiniband cables and checked if they are correct plugged in into port 1 and 2.

Now output looks like this:

[root@xxx ~]# ibv_devices

device node GUID

mlx4_0 0002c90300345f50

[root@xxx ~]# lspci -d 15b3:

03:00.0 Network controller: Mellanox Technologies MT27500 Family [ConnectX-3]

[root@xxx ~]# ls -l /dev/infiniband/

total 0

crw-rw----. 1 root root 231, 64 Jan 7 10:10 issm0

crw-rw----. 1 root root 231, 65 Jan 7 10:10 issm1

crw-rw-rw-. 1 root root 10, 56 Jan 7 10:10 rdma_cm

crw-rw-rw-. 1 root root 231, 224 Jan 7 10:10 ucm0

crw-rw-rw-. 1 root root 231, 0 Jan 7 10:10 umad0

crw-rw-rw-. 1 root root 231, 1 Jan 7 10:10 umad1

crw-rw-rw-. 1 root root 231, 192 Jan 7 10:10 uverbs0

[root@xxx ~]# ls /sys/class/infiniband

mlx4_0

[root@ewsexaobi01 ~]# ibv_devinfo

hca_id: mlx4_0

transport: InfiniBand (0)

fw_ver: 2.30.3200

node_guid: 0002:c903:0034:5f50

sys_image_guid: 0002:c903:0034:5f53

vendor_id: 0x02c9

vendor_part_id: 4099

hw_ver: 0x1

board_id: HP_0230240019

phys_port_cnt: 2

port: 1

state: PORT_ACTIVE (4)

max_mtu: 4096 (5)

active_mtu: 4096 (5)

sm_lid: 2

port_lid: 13

port_lmc: 0x00

link_layer: InfiniBand

port: 2

state: PORT_ACTIVE (4)

max_mtu: 4096 (5)

active_mtu: 4096 (5)

sm_lid: 2

port_lid: 14

port_lmc: 0x00

link_layer: InfiniBand

Is this ok now ? How to get now network configured ip ?

13

Hi ,

To configure ip , and for configuration in general i’ll recommend using the Mellanox OFED user manual:

http://www.mellanox.com/related-docs/prod_software/Mellanox_OFED_Linux_User_Manual_v2.1-1.0.0.pdf http://www.mellanox.com/related-docs/prod_software/Mellanox_OFED_Linux_User_Manual_v2.1-1.0.0.pdf

Section 4.3.3