Bundled OpenMPI library and headers

Accelerated Computing HPC Compilers Legacy PGI Compilers
#1

The User’s Guide in section 6.8 states that PGI products for Linux ship with a PGI-built version of Open MPI that includes everything required to compile, execute and debug MPI programs using Open MPI.

How well would this version work with the PBS/Torque scheduler? I believe that Open MPI has to be explicitly configured to do so.

#2

Hi KurtMcCall31738,

We explicitly do not build with Torque support in the OpenMPI we ship since doing so would require all uses to have Torque available.

For this case, you’ll need to build OpenMPI yourself with the PGI compilers. If needed, I can ask the folks here who build our OpenMPI for the configuration line we use to build, which you can modify to include Torque support.

-Mat

#3

Mat,

Yes, please supply supply the config line that you use to build OpenMpi.

Thanks,
Kurt

#4

Hi,

Here is the configure line we currently use for building Open MPI 3.1.3:

./configure --enable-shared --enable-static --without-tm --enable-mpi-cxx --disable-wrapper-runpath --with-cuda=/path/to/cuda/dir --prefix=/path/to/install/dir CC=pgcc CXX=pgc++ FC=pgfortran CPP=cpp CFLAGS="-O1 -tp p7-64 -c99" CXXFLAGS="-O1 -tp p7-64" FCFLAGS="-O1 -tp p7-64"

Obviously to include Torque support, you will want to change --without-tm to --with-tm above.

We are in the process of updating to Open MPI 4.0.2 here, and we are following the Open MPI developers’ advice to link Open MPI against the UCX library with this next update. If you would prefer to use Open MPI 4.0.2, here is the configuration we are currently testing:

./configure --enable-shared --enable-static --without-tm --enable-mpi-cxx --disable-wrapper-runpath --with-ucx=/path/to/ucx/dir --with-cuda=/path/to/cuda/dir -enable-mpirun-prefix-by-default --enable-mpi1-compatibility --enable-mca-no-build=btl-uct --prefix=/path/to/install/dir CC=pgcc CXX=pgc++ FC=pgfortran CPP=cpp CFLAGS="-O1 -tp p7-64 -c99" CXXFLAGS="-O1 -tp p7-64" FCFLAGS="-O1 -tp p7-64"

Again, change --without-tm to --with-tm above for Torque support.

Hope this helps.

+chris

#5

Thanks Chris. PGI also bundles the MPICH library and headers. Would you supply the config line for it
as well?

Thanks,
Kurt