Cannot send masked atomic compare-and-swap RDMA requests


I’m having trouble sending a masked atomic compare-and-swap request with the RDMA verbs API. I’ve included the header file “verbs_exp.h” which has added support for these masked atomic requests.

My client is facing a segmentation fault upon calling “ibv_exp_post_send” to send this request type. Using the same RDMA verb, I am able to send one sided READ and WRITE and regular atomic compare-and-swap requests, however, I’m facing issues with sending masked atomic requests.

gdb points to the following, which hasn’t been particularly helpful:

Program received signal SIGSEGV, Segmentation fault.
0x00007ffff78441a8 in perror_internal (fp=fp@entry=0x607260,
s=s@entry=0xfffffffff797d156 <error: Cannot access memory at address 0xfffffffff797d156>, errnum=errnum@entry=22)
at perror.c:32
32 perror.c: No such file or directory.

I initially thought this may be a hardware limitation. However, according the the RDMA Programming User Manual (v1.7), these extended atomic operations are supported by ConnectX®-2 and subsequent hardware.

My testbed uses ConnectX-4 Lx 40G NICs and MLNX OFED 4.9- run Ubuntu 18.04 and kernel 5.4.0-126-generic.

I’ve attached my source code and Makefile, in case it assists with debugging. I’ve also included instructions on how to run the program at the top of the source code file.

I’d greatly appreciate any advice anyone could provide!

Thank you for your time.


Makefile (99 Bytes)
rdma_rw-RC.c (37.1 KB)

Hi Hamed,

It appears you are trying to use the experimental verbs ibv_exp_post_send.
Please note, that we are no longer provide support to experimental verbs.

Is there any specific reason for not using rdma-core library instead?
I suggest to install the latest MLNX_OFED driver and use rdme-core library, where you need to port experimental verbs to extended analog.


Hi Chen,

Thank you very much for your response! There is a prior forum post regarding rdma-core supporting extended atomics and it seems that extended atomics are not and will not be supported with rdma-core. Are you aware whether this has changed since then?

Here is the reference to the forum post: How to use extended atomics? - #3 by jjn217

Best regards,