Bonding/teaming over multiple adapters.

Infrastructure & Networking Adapters and Cables
1

Is bonding possible over multiple cards? Every KB I find on this topic mentions a dual port adapter.

We’d like to get over 63Gbps PCIe3 limit per card and thus would like to spread the traffic over multiple cards. (running connect-x 3). If not possible, how about connectx-4 LX cards?

What if we need bonded RoCE?

I’m amazed no one has asked this before.

Thanks.

2

Hello,

RoCE LAG is a feature meant for mimicking Ethernet bonding for IB devices and is available for dual port cards only.

Any change of bonding configuration that negates one of the above rules (i.e, bonding mode is not 1, 2 or 4, or both Ethernet interfaces that belong to the same card are not the only slaves of the bond interface), will result in exiting RoCE LAG mode and the return to normal IB device per port configuration.

Please review the following for more information:

https://docs.mellanox.com/pages/viewpage.action?pageId=58757626#RDMAoverConvergedEthernet(RoCE)-RoCELAG(ConnectX-3/ConnectX-3Pro)

https://community.mellanox.com/s/article/How-to-Configure-RoCE-over-LAG-ConnectX-4-ConnectX-5-ConnectX-6

Thank you,

-Hilary

3

We’ve got two dual port cards. It’s unclear whether bonding is possible over them (over multiple cards). Or is it over ports of a single card only?

4

The statement “both Ethernet interfaces that belong to the same card are not the only slaves of the bond” refers to two interfaces belonging to the same adapter; i.e., it is not possible to utilize multiple adapters’ ports within the LAG configuration.

-Hilary