Freebsd 11.1 ConnectX-4 VPI

Infrastructure & Networking Software And Drivers Mellanox OFED
1

Hello I just have strange behaviour I’m using modules from Freebsd 11.1 I compiled and build using buildworld etc… and I can load mlx5 without no issue but I don’t see Infiniband port on system on dmesg I have

mlx5_core0: <mlx5_core> mem 0xf4000000-0xf5ffffff irq 16 at device 0.0 on pci1

mlx5_core0: INFO: firmware version: 12.21.1000

mlx5_core0: INFO: Module 0, status: plugged

mlx5_core1: <mlx5_core> mem 0xf2000000-0xf3ffffff irq 17 at device 0.1 on pci1

mlx5_core1: INFO: firmware version: 12.21.1000

I have Infiniband switch and ports are in Infiniband mode , I have loaded mlx5 module (core module) but why I can’t see the ports ??

Building this drivers from OFED 3.0 from Mellanox site end with :

…/drivex/mlx5/generated/freebsd/core/mlx5_alloc.c:28:10: fatal error: ‘linux/errno.h’ file not found

#include <linux/errno.h>

^~~~~~~~~~~~~~~

1 error generated.

1 27 0xffffffff80200000 1f67a88 kernel

2 1 0xffffffff82169000 316708 zfs.ko

3 2 0xffffffff82480000 cb78 opensolaris.ko

4 1 0xffffffff8248d000 3d908 mlx5.ko

5 3 0xffffffff824cb000 1c960 linuxkpi.ko

6 1 0xffffffff824e8000 b1770 ibcore.ko

7 1 0xffffffff82621000 3650 ums.ko

8 1 0xffffffff82625000 3c93f linux64.ko

9 1 0xffffffff82662000 7b0f linux_common.ko

Maybe someone can help with this ? I don’t know why there is not mlx5_ib module ?? It is not implemented yet ?? Maybe Infiniband for ConnectX-4 VPI cards is not ready ??

Please let me know.

BR

Adam

2

What do you mean by inbox drivers?

“Inbox drivers” is a weird term that some vendors (mostly Mellanox it seems) use to describe the drivers provided by the OS.

My suspicion is that it’s a shortened way to say “the drivers that came in the box”. I’ve not personally heard anyone other than Mellanox use the term until recently, where it was used by (from memory) a Red Hat engineer. Hopefully the phrase dies out though, and people use the industry standard phrases instead. eg “OS provided driver”

3

Hi Marc

Does this solve the directory reference problems like “linux/errno.h” ? Because if you pass this problem others come in the line. I think mellanox driver instructions on github page is old.Header file directory references must be organized in order to compile mellanox driver properly.Besides RDMA-ISER is not supported except initiator(there is another mellanox github page). Why target is not supported but initiator is something I wonder.After installation, freebsd-11-1 lists mellanox NIC properly.I dont get what the use of those drivers if rdma-iser is not supported

thnx

4

Hi Adam,

You can try to build the inbox drivers

cd /usr/src/sys/modules/mlx5

make all install

cd /usr/src/sys/modules/mlx5en

make all install

kldload mlx5

ifconfig (must show your adapter)

BR

Marc

5

Hi,

No, it doesn’t solve the build error.

The build error is related to the mellanox driver sources that you got from git.

What i explained compiles the native driver provided with FreeBSD 11.1

For now, there’s no GA release for FreeBSD 11.1 it is certainly the reason the build failed.

I 've tried to workaround this problem by adding in the CFLAGS the path to the linx/errno.h CFLAGs = -I, but the compilation failed at a later stage.

Conclusion:

For now, there’s only this FReeBSD 11 build that is supported : FreeBSD 11 Current (v289420)

Marc

6

Hey Adam, as a thought… if you don’t get a useful response here it’s a good idea to ask on the FreeBSD Infiniband mailing list:

freebsd-infiniband Info Page FreeBSD Mailing lists: subscription for freebsd-infiniband

It’s not a high traffic mailing list, but generally the people there are responsive to on-topic questions (like this).

7

Hi

So I should skip mellanox driver sources on github.What do you mean by inbox drivers? Freebsd11.1 supports rdma-iser for both target and initiator?

thnx

8

“inbox driver” is a term that Mellanox uses for the drivers that comes with ubuntu centos debian rhel etc linux distributions.it is generic for all types of linux distros.this is the way of doing their business.

My problem is that even if I dont install anything , freebsd11 lists connectx-4 VPI drivers.My question is that what the use of installing mellanox drivers on github is if freebsd inbox drivers are already there.

Furthermore if ISER-RDMA is supported only for initiator but not target, I just concluded this because I did not get any response on this issue.initiator without target is like coffee without cookie.