This issue also reported at the OpenMPI forums. It’s unclear whether this is nvcc in NVHPC 23.1 and 23.11 or the OpenMPI 5.x configuration method for getting the compiler to report undeclared builtins.
Background information
5.0.0 and 5.0.1 configure test for PMIx no longer works with NVHPC, appears to be the compiler check for undeclared builtins. The associated stub compiles and runs but does not produce output, which is interpreted as inability to get the compiler to report undeclared builtins, halting the configure since pmix is required as of OMP 5.x. GCC through 13.2.0 does not have this issue. OMPI 4.x pmix test stub differs (orted vs prrte?) and does not have this issue for GCC 12.2.0 / 13.2.0 or NVHPC 23.1 / 23.11. Currently unable to try OMP 5.0.2 or external pmix/hwloc/libevent recent enough for pmix>=4.2 (policy). Can anyone please verify working OMP 5.x build with NVHPC 23.11 (cuda 12.3) or 23.1 (cuda 12.0)?
Versions
- NVHPC 23.11, Cuda compilation tools, release 12.0, V12.0.76, Build cuda_12.0.r12.0/compiler.31968024_0
- NVHPC 23.1, Cuda compilation tools, release 12.3, V12.3.52, Build cuda_12.3.r12.3/compiler.33281558_0
OMPI build
Source build using NVHPC 23.1 (CUDA 12.0) and 23.11 (CUDA 12.3), CC=nvcc, FC=nvfortran, CXX=nvc++
Configure is scripted, script block is:
...<archive copy/unroll, builddir create, set $distro and $basearch, etc>...
module purge
module load nvhpc/23.11
...<setup ./pbs-config for --with-tm>...
export CFLAGS=''
export FCFLAGS=''
../configure \
--prefix=/opt/soft/$distro/$basearch/openmpi/5.0.1/nvhpc/23.11 \
--x-includes=/usr/include \
--x-libraries=/usr/lib64 \
--enable-branch-probabilities \
--enable-dependency-tracking \
--enable-mpi-ext=all \
--with-pmix=internal \
--enable-pmix-timing \
--with-package-string="Open MPI 5.0.1 with NVHPC 23.11" \
--with-ident-string="NRLDC CCS" \
--enable-ipv6 \
--enable-heterogeneous \
--enable-hwloc-pci \
--with-hwloc=internal \
--with-ofi \
--with-verbs \
--with-tm="$PBS_EXEC" \
--enable-sparse-groups \
--enable-peruse \
--enable-mpi-fortran=all \
CC=nvcc \
FC=nvfortran \
CXX=nvc++
# --enable-mpi-cxx \ # (C++ bindings no longer supportd)
# --enable-mpi-cxx-seek \ # (C++ bindings no longer supportd)
System
- Operating system/version: Rocky Linux 8.8 aarch64
- Computer hardware: HPE Apollo80 gen 2 (Fujitsu A64FX armv8 FX700)
- Network type: 10Gb ether, 100Gb ether, HDR infiniband
Details
Configure fails at PMIx checking:
checking for nvcc options needed to detect all undeclared functions... cannot detect
configure: error: in `/tmp/openmpi-nvhpc/openmpi-5.0.1/build/3rd-party/openpmix':
configure: error: cannot make nvcc report undeclared builtins
The outer config.log for this failure is:
configure:5795: *** Configuring PMIx
configure:63782: ===== configuring 3rd-party/openpmix =====
configure:63971: running /bin/sh ../../../3rd-party/openpmix/configure --disable-option-checking '--prefix=/opt/soft/el8/aarch64/openmpi/5.0.1/nvhpc/23.11' --without-tests-examples --enable-pmix-binaries --disable-pmix-backward-compatibility --disable-visibility --disable-hwloc-lib-checks --with-hwloc-extra-libs="/tmp/openmpi-nvhpc/openmpi-5.0.1/build/3rd-party/hwloc-2.7.1/hwloc/libhwloc.la" '--x-includes=/usr/include' '--x-libraries=/usr/lib64' '--enable-branch-probabilities' '--enable-dependency-tracking' '--enable-mpi-ext=all' '--enable-pmix-timing' '--with-package-string=Open MPI 5.0.1 with NVHPC 23.11' '--with-ident-string=NRLDC CCS' '--enable-ipv6' '--enable-heterogeneous' '--enable-hwloc-pci' '--with-ofi' '--with-verbs' '--with-tm=' '--enable-sparse-groups' '--enable-peruse' '--enable-mpi-fortran=all' 'CC=nvcc' 'CFLAGS=' 'CPPFLAGS=-I/tmp/openmpi-nvhpc/openmpi-5.0.1/build/3rd-party/hwloc-2.7.1/include -I/tmp/openmpi-nvhpc/openmpi-5.0.1/3rd-party/hwloc-2.7.1/include' 'CXX=nvc++' 'FC=nvfortran' 'FCFLAGS=' 'CPP=cpp' 'PKG_CONFIG_PATH=/opt/soft/el8/aarch64/ucx/1.13.1/lib/pkgconfig:/opt/soft/el8/aarch64/openssl/1.1.1s/lib/pkgconfig' --cache-file=/dev/null --srcdir=../../../3rd-party/openpmix
configure:63991: ===== done with 3rd-party/openpmix configure =====
configure:65532: error: Could not find viable pmix build.
The inner config.log (<build_dir>/3rd-party/openpmix/config.log) for this failure is:
...snip snip...
| /* end confdefs.h. */
| #include <float.h>
| #include <limits.h>
| #include <stdarg.h>
| #include <stddef.h>
| extern void ac_decl (int, char *);
|
| int
| main (void)
| {
| (void) ac_decl (0, (char *) 0);
| (void) ac_decl;
|
| ;
| return 0;
| }
configure:18028: result: cannot detect
configure:18032: error: in `/tmp/openmpi-nvhpc/openmpi-5.0.1/build/3rd-party/openpmix':
configure:18034: error: cannot make nvcc report undeclared builtins
The stub compiles and runs but does not produce output under nvcc in NVHPC 23.1 or 23.11.