PGI 15.4 and OpenMPI 1.8.4 Errors

Accelerated Computing HPC Compilers Legacy PGI Compilers
1

Good Day,
I am having problems compiling OpenMPI 1.8.4 with PGI 15.4.
I have successfully compiled the same 1.8.4 code with same configure with PGI 14.6 but it fails when using 15.4.
I have experienced this on two separate CentOS 6.6 servers, one haswell and the other sandybridge.

export MYINSTALLDIR=/share/apps/mpi/openmpi/1.8.4-pgi-15.4
export CC=pgcc
export CXX=pgcpp
export FC=pgfortran
export CFLAGS='-fast'
export CXXFLAGS=${CFLAGS}
export FCFLAGS=${CFLAGS}
../configure --prefix=${MYINSTALLDIR} --with-verbs=no --disable-ipv6 --enable-static --with-gnu-ld --disable-io-romio 2>&1 | tee configure-`date +%y%m%d-%H%M`.log

When making fails with following error:

Making all in profile
make[3]: Entering directory `/home/src/install/openmpi/openmpi-1.8.4/buid154/ompi/mpi/fortran/mpif-h/profile'
  FC       psizeof_f.lo
PGF90-S-0081-Illegal selector - KIND value must be non-negative  (psizeof_f.f90: 17)
  0 inform,   0 warnings,   1 severes, 0 fatal for pmpi_sizeof_complex128_scalar
PGF90-S-0081-Illegal selector - KIND value must be non-negative  (psizeof_f.f90: 27)
  0 inform,   0 warnings,   1 severes, 0 fatal for pmpi_sizeof_complex128_r1
PGF90-S-0081-Illegal selector - KIND value must be non-negative  (psizeof_f.f90: 37)
  0 inform,   0 warnings,   1 severes, 0 fatal for pmpi_sizeof_complex128_r2
PGF90-S-0081-Illegal selector - KIND value must be non-negative  (psizeof_f.f90: 47)
  0 inform,   0 warnings,   1 severes, 0 fatal for pmpi_sizeof_complex128_r3
PGF90-S-0081-Illegal selector - KIND value must be non-negative  (psizeof_f.f90: 57)
  0 inform,   0 warnings,   1 severes, 0 fatal for pmpi_sizeof_complex128_r4
PGF90-S-0081-Illegal selector - KIND value must be non-negative  (psizeof_f.f90: 67)
  0 inform,   0 warnings,   1 severes, 0 fatal for pmpi_sizeof_complex128_r5
PGF90-S-0081-Illegal selector - KIND value must be non-negative  (psizeof_f.f90: 77)
  0 inform,   0 warnings,   1 severes, 0 fatal for pmpi_sizeof_complex128_r6
PGF90-S-0081-Illegal selector - KIND value must be non-negative  (psizeof_f.f90: 87)
  0 inform,   0 warnings,   1 severes, 0 fatal for pmpi_sizeof_complex128_r7
PGF90-S-0081-Illegal selector - KIND value must be non-negative  (psizeof_f.f90: 577)
  0 inform,   0 warnings,   1 severes, 0 fatal for pmpi_sizeof_real128_scalar
PGF90-S-0081-Illegal selector - KIND value must be non-negative  (psizeof_f.f90: 587)
  0 inform,   0 warnings,   1 severes, 0 fatal for pmpi_sizeof_real128_r1
PGF90-S-0081-Illegal selector - KIND value must be non-negative  (psizeof_f.f90: 597)
  0 inform,   0 warnings,   1 severes, 0 fatal for pmpi_sizeof_real128_r2
PGF90-S-0081-Illegal selector - KIND value must be non-negative  (psizeof_f.f90: 607)
  0 inform,   0 warnings,   1 severes, 0 fatal for pmpi_sizeof_real128_r3
PGF90-S-0081-Illegal selector - KIND value must be non-negative  (psizeof_f.f90: 617)
  0 inform,   0 warnings,   1 severes, 0 fatal for pmpi_sizeof_real128_r4
PGF90-S-0081-Illegal selector - KIND value must be non-negative  (psizeof_f.f90: 627)
  0 inform,   0 warnings,   1 severes, 0 fatal for pmpi_sizeof_real128_r5
PGF90-S-0081-Illegal selector - KIND value must be non-negative  (psizeof_f.f90: 637)
  0 inform,   0 warnings,   1 severes, 0 fatal for pmpi_sizeof_real128_r6
PGF90-S-0081-Illegal selector - KIND value must be non-negative  (psizeof_f.f90: 647)
  0 inform,   0 warnings,   1 severes, 0 fatal for pmpi_sizeof_real128_r7
make[3]: *** [psizeof_f.lo] Error 1
make[3]: Leaving directory `/home/src/install/openmpi/openmpi-1.8.4/buid154/ompi/mpi/fortran/mpif-h/profile'
make[2]: *** [all-recursive] Error 1
make[2]: Leaving directory `/home/src/install/openmpi/openmpi-1.8.4/buid154/ompi/mpi/fortran/mpif-h'
make[1]: *** [all-recursive] Error 1
make[1]: Leaving directory `/home/src/install/openmpi/openmpi-1.8.4/buid154/ompi'
make: *** [all-recursive] Error 1
2

Hi James,

Sorry you ran into this. Support for the Fortran 2008 storage_size intrinsic was introduced with PGI 15.4, and OpenMPI 1.8.4 is now detecting this. Unfortunately, OpenMPI 1.8.4 also assumes that PGI 15.4 supports 128-bit floating-point types as part of this, which is not the case. We have reached out to the OpenMPI developers on this issue, and they have already corrected it in their development branch for a future release.

For now, you can get OpenMPI 1.8.4 to compile with PGI 15.4 by applying the following patch. Copy and paste the following snippet into a file, called no_real128.patch:

--- openmpi-1.8.4/ompi/mpi/fortran/base/gen-mpi-sizeof.pl.orig  2014-10-31 17:29:50.000000000 -0700
+++ openmpi-1.8.4/ompi/mpi/fortran/base/gen-mpi-sizeof.pl       2015-03-20 10:45:42.123850606 -0700
@@ -140,7 +140,7 @@
 for my $size (qw/8 16 32 64/) {
     queue_sub("integer(int${size})", "int${size}", "int${size}");
 }
-for my $size (qw/32 64 128/) {
+for my $size (qw/32 64/) {
     queue_sub("real(real${size})", "real${size}", "real${size}");
     queue_sub("complex(real${size})", "complex${size}", "real${size}");
 }

Next, unpack the OpenMPI 1.8.4 tar file somewhere, and then apply this patch before you cd into the directory and run configure:

$ patch -p0 < no_real128.patch

Now OpenMPI 1.8.4 should build with PGI 15.4.

Hope this helps.

Best regards,

+chris

3

Hi Chris,
Thanks for the patch. I have applied it to source and it now successfully compiles.

Thank you for your help.

4

chris,

FYI: I tried to apply your patch and I got this:

(280) $ patch -p0 < ~/no_real128.patch
patching file openmpi-1.8.4/ompi/mpi/fortran/base/gen-mpi-sizeof.pl
Hunk #1 FAILED at 140.
1 out of 1 hunk FAILED -- saving rejects to file openmpi-1.8.4/ompi/mpi/fortran/base/gen-mpi-sizeof.pl.rej

Obviously, I could read the patch and just did what it wanted manually, but do you know why the patch failed for me (RHEL 6.5 box)? I looked at the patch file in Vim with set list and saw no fancy copy-and-paste errors.

Matt

5

Hi Matt,

I would have to look at that .rej file to see if I could tell why the patch failed.

Could you send it to trs@pgroup.com, and mention “For Chris” in the subject line?

Thanks!

+chris

6

Done and sent, though as the email says, the .rej file is your patch!

Matt

7

Just to confirm what Matt said, the patch failed for me as well, but I just manaully made the change to the gen-mpi-sizeof.pl file.

8

Note, I’m pretty sure that Open MPI 1.8.5 does not have this issue. At least I built it with PGI 15.4 and no patch was needed.

Matt

9

1.8.5 built for you? What was your configure line?

I tried this:
env CC=pgcc FC=pgfortran F77=pgfortran CXX=pgcpp CFLAGS=-fast FCFLAGS=-fast
FFLAGS=-fast CXXFLAGS=-fast ./configure -prefix=/share/apps/openmpi-1.8.5/pgi-2015

I’m using PGI 15.4.

I get:
PGC-S-0039-Use of undeclared variable INT64_T (ompi_datatype_module.c: 278)
PGC-S-0039-Use of undeclared variable AINT (ompi_datatype_module.c: 278)

Ah, the fix is export CPP=cpp and rerunning configure.

10

Sorry for the late reply. I guess my notifications weren’t on!

I compiled it with:

 ./configure --enable-shared --disable-wrapper-rpath CC=pgcc CXX=pgcpp F77=pgfortran FC=pgfortran CFLAGS='-fpic -m64' CXXFLAGS='-fpic -m64' FCFLAGS='-fpic -m64' FFLAGS='-fpic -m64' --prefix=/ford1/share/gmao_SIteam/MPI/openmpi-1.8.5-pgi-15.5.0 | & tee configure.pgi15.5.log

And, yeah, I guess I forgot all of my PGI module files have “unsetenv CPP”!