Accelerating NVSHMEM 2.0 Team-Based Collectives Using NCCL

Originally published at: Accelerating NVSHMEM 2.0 Team-Based Collectives Using NCCL | NVIDIA Developer Blog

NVSHMEM 2.0 is introducing a new API for performing collective operations based on the Team Management feature of the OpenSHMEM 1.5 specification. A team is a subset of processing elements (PEs) in an OpenSHMEM job. The concept is analogous to communicators in MPI. The new Teams API is a replacement for the active-set-based API for…