The “RDMA Aware Networks Programming User Manual” has an example with multicast UD. I was able to compile it, but I could not get it to run successfully. But I don’t think that multicast is what I need. With my current setup, I am able to run some of the test programs successfully, but not others. For example, I was able to run ib_send_bw with a UD connection fine. When I tried to run udaddy, the test exits with a RDMA_CM_EVENT_UNREACHABLE error. I don’t know how to go about debugging this problem.
For udaddy run
On receiver (server)
#udaddy -b
On the sender
#udaddy -s
For example of source program, here is a great resource
Better resource to ask and search for generic questions related to RDMA to ask on linux-rdma mailing news group.
Hi,
Thanks for the response and the links.
WRT to your response on my question on udaddy, using the -b flag allowed
me to get past the error I was hitting. Now the program runs, but it does not
finish. On the server side I see:
$ udaddy -b 10.10.10.3
udaddy: starting server
receiving data transfers
On the client side:
$ udaddy -b 10.10.10.7 -s 10.10.10.3
udaddy: starting client
udaddy: connecting
initiating data transfers
receiving data transfers
They never get past the output shown above.
On both the server and the client computers, top shows that udaddy
is consuming 100% CPU. I was expecting the test to finish similar
to the output shown in the Mellanox "How to Enable,
Verify, and Troubleshoot RDMA" page.
I tried rerunning the test with a message_count of 100 but got the same
result. udaddy is doing something, but it never finishes. Any thoughts?
Thanks,
Terry
That might be some issue with this specific application. There are plenty of other tools. For example:
rping
ib_send_bw, ib_read_bw, ib_write_bw and other (part of perftest)
You can choose connection type with ib_send_bw using ‘-c’ flag
On receiver
ib_send_bw -c UD
On the sender
ib_send_bw -c UD
If you have multiple ports of devices, it might be necessary to use ‘-d’ option to specify under layer device . For example ‘-d mlx5_1’ . The lower devices can be obtained using ibstat, ibv_devinfo utilities. If you have Mellanox OFED installed, you might use ‘ibdev2netdev -v’ command