im testing OFED 2 on Ubuntu. its not tested on 13.04 but it works quite well except a few probs

But does anyone know if SDP is supported, I cant find anything in the docs about it, except a mention of SDP after page 150 in the manual, but it doesnt talk about weather its supported.

If i modprobe ib_SDP it comes up with “FATAL: Module ib_sdp not found.” which doesnt look positive.

the probs with 13.04 is that the infiniband diags dont work corectly. ie. a ibstat works, but ib switches, or ibhosts fails with

/usr/sbin/ibhosts: line 50: /usr/local/sbin/ibnetdiscover: No such file or directory

the ubuntu packages for RDMA srptools and infiniband-diags seem to be vers 1.5.4 and not updated, so ill compile them from source when i get a chance.

Why SDP well its faster than IB_IPOIB has less CPU overhead, and low latency, so makes sense for replication data, And also for IB MySQL transport. you can deploy servers over IB and connect your dedicated SQL serves over SDP making them much faster than IPOIB!!

and yes i added “SDP_LOAD=yes” to > /etc/infiniband/openib.conf

it says loading ib_sdp [FAILED] when you restart IB /etc/init.d/openibd restart

FYI OFED 2 with RDMA SRP test is faster than 1.5.4! much faster, still to test with iSCSI over IPOIB.

i compiled 13.04 with custom drivers for SCST + OFED 2. which all works well. And want to use SDP with DRBD.

Any advice?


yes after hours of going throug manuals i found that answer also.

so will roll over to OFED driver.


BTW, you may want to try Mellanox OFED 2.0 ipoib - the guys reports on very good numbers for ipoib-ud with very good stability. (for certain scenarios, ipoib-ud is now better then ipoib-CM)

Check it out and see if it is comparable to what you were looking to do with SDP.

Good luck.

SDP Support has been deprecated by Mellanox.

I believe OFA is supporting it still though.


yes we tried ipoib in CM and UD modes, using ip drbd replication set to 20Gb/s but it only managed 200~300Mb/s.

whereas using SRP with MDADM we can achieve 900Mb/s to 1400Mb/s easy!

It was OFED 2 on the target only, not on the initiator as that is ESXi5.1 with the esxi driver which i dont think is based on OFED 2, as i suspect target and initiator have to be the same level to see the best performance. Though we did see a jump in SRP banddwidth ? unless others have seen otherwise.?

lets hope Mellanox update the vmware driver 1.8.1 to OFED2? plz!

bmac20 https://community.mellanox.com/s/profile/0051T000008EZfFQAW - Maybe you could compile in SDP support to Mellanox OFED 2.x yourself?

http://git.openfabrics.org/git?p=~amirv/libsdp.git;a=tree http://git.openfabrics.org/git?p=~amirv/libsdp.git;a=tree