Does ConnectX-2 support latest Ceph RDMA code?

1

I followed this doc:

Bring Up Ceph RDMA - Developer’s Guide https://community.mellanox.com/s/article/bring-up-ceph-rdma---developer-s-guide

https://community.mellanox.com/s/article/bring-up-ceph-rdma---developer-s-guide

Bring Up Ceph RDMA - Developer’s Guide

…

Support matrix

OS CentOS7.2, Ubuntu 16.04

NIC ConnectX-4, ConnectX-4 Lx

MLNX OFEDMLNX OFED 4.

…

Now I have a question, Does the NIC version have to be ConnectX-4 or ConnectX-4 Lx?

The NIC in my device is ConnectX-2 or ConnectX-3, are they supported? What about OS?

2

Hi,

CEPH RDMA can be used with following cards - ConnectX-3 (RoCEv1 and IB only), ConnectX-3 Pro , ConnectX-4, ConnectX-5, ConnectX-4 Lx

I guess ConnectX-2 can be used only with IB.

If you use MLNX OFED and compile CEPH, the only limitation on OS is generic CEPH OS support matrix.

We specifically tested CEPH RDMA on CentOS 7.2/7.4 and UB 16.04.

Regards,

Vladimir

3

Hi,

CEPH RDMA code is not complete in 12.2.5

We suggest to take Mellanox branch that is based on 12.2.4 and includes all fixes.

Please see updated post:

https://community.mellanox.com/s/article/bring-up-ceph-rdma---developer-s-guide

Regards,

Vladimir

4

Few important bug fixes are missing in Luminous 12.2.5 release.

you can clone MLNX branch and add patches that were added to CEPH between 12.2.4 and 12.2.5.

5

This one just bit us recently. Be aware that Mellanox OFED no longer supports CX-2. They’ve had notes in MOFED since 4.0.x but they were still compiling in actual card support for a few versions. The latest versions (>=4.1) no longer have support at all.

6

Thank you very much!

May I ask another question, Does Luminous 12.2.5 version support RDMA?

I find the code about RDMA and DPDK in the packet of Luminous 12.2.5.

So I guess this version has already support RDMA, Can I use Luminous 12.2.5 to test RDMA directly follow this this doc?

Bring Up Ceph RDMA - Developer’s Guide https://community.mellanox.com/s/article/bring-up-ceph-rdma---developer-s-guide

Best regards to you !

7

Thanks!

CEPH RDMA code is only not complete in 12.2.5?

Is that means basic features is availiable? or just hava a few bugs not fixed?

Our project is based on Luminous 12.2.5, so I want use Luminous 12.2.5 to test basic RDMA features , does it work properly by this way?

Regards,

feihong