Typically folks use the MPI driver, in this case mpic++ or mpicxx, instead of the compiler driver (i.e. pgc++/nvc++) which will take care adding the appropriate libraries and libraries paths. Though the MPI driver does need to be configured to use the NVHPC compilers.
The easiest thing to do is set your PATH to include one of the MPI builds that we ship. Now we did move the install directory where these are located a few years ago during the rebranding of PGI to NVHPC, so the location will be dependent on which version of the compiler you’re using. Pre-20.5, they’ll be in “$PGI//<release_year>/mpi/”, and post 20.5 they moved from the common directory to under the specific release directory: $NVHPC//<release_number>/comm_libs/".
Now the “-Mmpi” flag is legacy, and we no longer ship MPICH with the compilers. Though it should still be functional if you set the environment variable “MPIDIR” to the base install directory of you MPICH installations.
Alternatively, you can manually add the MPICH library path, “-L/path/to/mpich/lib”, and libraries, example: “-lmpich”, to the link line.