If I have only 2 processes launching RDMA operations. Is it enough to bind each process to a different port since I have a dual-port RNIC to guarantee isolation among both processes (at least Bandwidth wise) or will using virtual lanes/service levels still have other advantages over using 2 ports? Of course, this is given that I will limit the Gbps for each port.
I think maybe the only thing the virtual lanes may offer is controlling traffic priority level because rate limiting can effectively be done by using different physical ports, right?
Thanks for posting your inquiry to the NVIDIA Developer Forums.
SL/VL configuration is a QoS consideration. When you configure applications to:
Run on specific SLs
Configure those SLs to map to a specific VL, and
Weight of those VLs is configured,
Traffic running on a prioritized VL will take precedence over traffic on an unprioritized VL. It won’t really isolate this traffic, aside from segregating it into a different SL/VL. Suppose one could assign equal weights to SL/VL pairs, and use different SLs for each stream of traffic, but that’s not really necessary. Regarding SL/VL and QoS configuration, recommend reading the following writeup from our Application Performance director: https://hpcadvisorycouncil.atlassian.net/wiki/spaces/HPCWORKS/pages/1178075141/Understanding+Basic+InfiniBand+QoS
(Sent my previous response too early by accident, continuing… :) )
Use of different physical adapter ports is effective enough isolation at the host level. Just be aware that you’re using the same physical PCIe slot for both ports on the single adapter - PCIe bus resources are shared.
Hypothetically, let’s say for some reason you have backpressure on the PCIe bus due to issues with one process whereas the other does not (IE - slow receiver), both processes could theoretically be impacted (IE - by bus latency spike or credit exhaustion) if the situation is extreme enough.