I have a 100GbE BF2 device (A) MBF2M516A-CENOT that is configured in separated mode. I have another two hosts (B and C) with a ConnectX-5 100GbE. All connected to a Mellanox 100GbE switch.
Using perftest ib_send_bw, B-to-C I see the full ~90Gbps b/w (expected). When I run A-B or A-C I only see < 1 Gbps.
I know this is not the usual bump in the wire mode of operation but I thought moving data in the ARM’s memory from the net interface would be more.
Can someone indicate whether this is about right and expected or not?