HPC SDK 21.9 is now available

Accelerated Computing HPC Compilers Legacy PGI Compilers
1

This update delivers important bug fixes and improvements to the HPC compilers and libraries included in the HPC SDK.

Please refer to the Release Notes for full details.

Download the current release at NVIDIA HPC SDK Current Release Downloads | NVIDIA Developer.

View the current documentation.

2

Hello!

I’m using Ubuntu 20.04 with HPC SDK 21.9 as docker container from NVIDIA HPC SDK | NVIDIA NGC.
But, I need use SLURM with MPI.

Dear colleagues, tell me please the ways do it. I seeing two workarounds :

  1. Build custom OpenMPI and replace HPC SDK one with it. How to change it in the HPC SDK package?
  2. Configure HPC SDK Open MPI. How do it?

Thank All for any assistance)

3

I asked some folks here and they said that you should look at using Pyxis which is a SLURM plugin allowing unprivileged cluster users to run containerized tasks through the srun command. It what we use on NVIDIA’s Selene system and would allow for the NVHPC SDK container to work as-is with pmix.

1 Like
4

And if in another way, via a building a custom configured openmpi? With ompi_info we seen that NVIDIA HPC SDK Open MPI package was configured with --with-slurm but without --with-pmi:

--prefix=/proj/nv/libraries/Linux_x86_64/dev/hpcx/205181-dev-clean-2/ompi' '--with-libevent=internal' '--enable-mpi1-compatibility' '--without-xpmem' '--with-cuda=/proj/cuda/11.0/Linux_x86_64' '--with-slurm' '--with-platform=../contrib/platform/nvhpc/optimized-x86_64' '--with-hcoll=/proj/nv/libraries/Linux_x86_64/dev/hpcx/205181-dev-clean-2/hcoll' '--with-ucx=/proj/nv/libraries/Linux_x86_64/dev/hpcx/205181-dev-clean-2/ucx' 'LDFLAGS=-Wl,-rpath-link=/proj/nv/libraries/Linux_x86_64/dev/hpcx/205181-dev-clean-2/ucx/lib

Therefore, I cannot run my mpi program with SLURM via an interactive command srun.
I need to add only --with-pmi[x] and to recompile another Open MPI. Can I do this? If so, how do I specify the correct paths for it within an installed HPC SDK package?:
$ ompi_info | grep pmi

           `MCA pmix: isolated (MCA v2.1.0, API v2.0.0, Component v4.1.2)
            MCA pmix: flux (MCA v2.1.0, API v2.0.0, Component v4.1.2)
            MCA pmix: pmix3x (MCA v2.1.0, API v2.0.0, Component v4.1.2)
            MCA ess: pmi (MCA v2.1.0, API v3.0.0, Component v4.1.2)`

Thank very much!