Error Building Open MPI 2.0.1 with PGI 16.10 macOS

Accelerated Computing HPC Compilers Legacy PGI Compilers
1

All,

Can someone help with this. I recently tried building Open MPI 2.0.1 with the new Community Edition of PGI on macOS. My first mistake was I was configuring with a configure line I’d cribbed from Linux that had -fPIC. Apparently -fPIC was removed from the macOS build. Okay, I can remove that and I configured with:

./configure --disable-wrapper-rpath CC=pgcc CXX=pgc++ FC=pgfortran CFLAGS='-m64' CXXFLAGS='-m64' FCFLAGS='-m64' --without-verbs --prefix=/Users/mathomp4/installed/Compiler/pgi-16.10/openmpi/2.0.1 | & tee configure.pgi-16.10.log

But, now, when I try to actually build, I get an error pretty quick inside the make:

  CC       printf.lo
  CC       proc.lo
  CC       qsort.lo

PGC-F-0249-#error --  "No way to define ev_uint64_t" (/Users/mathomp4/src/MPI/openmpi-2.0.1/opal/mca/event/libevent2022/libevent/include/event2/util.h: 126)
PGC/x86-64 OSX 16.10-0: compilation aborted
  CC       show_help.lo
make[3]: *** [proc.lo] Error 1
make[3]: *** Waiting for unfinished jobs....
make[2]: *** [all-recursive] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all-recursive] Error 1

This was done with -j2, so if I remake with ‘make V=1’ I see:

source='proc.c' object='proc.lo' libtool=yes \
	DEPDIR=.deps depmode=pgcc /bin/sh ../../config/depcomp \
	/bin/sh ../../libtool  --tag=CC   --mode=compile pgcc -DHAVE_CONFIG_H -I. -I../../opal/include -I../../ompi/include -I../../oshmem/include -I../../opal/mca/hwloc/hwloc1112/hwloc/include/private/autogen -I../../opal/mca/hwloc/hwloc1112/hwloc/include/hwloc/autogen -I../../ompi/mpiext/cuda/c   -I../.. -I../../orte/include   -I/Users/mathomp4/src/MPI/openmpi-2.0.1/opal/mca/hwloc/hwloc1112/hwloc/include -I/Users/mathomp4/src/MPI/openmpi-2.0.1/opal/mca/event/libevent2022/libevent -I/Users/mathomp4/src/MPI/openmpi-2.0.1/opal/mca/event/libevent2022/libevent/include  -O -DNDEBUG -m64  -c -o proc.lo proc.c
libtool: compile:  pgcc -DHAVE_CONFIG_H -I. -I../../opal/include -I../../ompi/include -I../../oshmem/include -I../../opal/mca/hwloc/hwloc1112/hwloc/include/private/autogen -I../../opal/mca/hwloc/hwloc1112/hwloc/include/hwloc/autogen -I../../ompi/mpiext/cuda/c -I../.. -I../../orte/include -I/Users/mathomp4/src/MPI/openmpi-2.0.1/opal/mca/hwloc/hwloc1112/hwloc/include -I/Users/mathomp4/src/MPI/openmpi-2.0.1/opal/mca/event/libevent2022/libevent -I/Users/mathomp4/src/MPI/openmpi-2.0.1/opal/mca/event/libevent2022/libevent/include -O -DNDEBUG -m64 -c proc.c -MD -o proc.o
PGC-F-0249-#error --  "No way to define ev_uint64_t" (/Users/mathomp4/src/MPI/openmpi-2.0.1/opal/mca/event/libevent2022/libevent/include/event2/util.h: 126)
PGC/x86-64 OSX 16.10-0: compilation aborted
make[3]: *** [proc.lo] Error 1
make[2]: *** [all-recursive] Error 1
make[1]: *** [all-recursive] Error 1
make: *** [all-recursive] Error 1

I guess my question is whether this is an issue with PGI or Open MPI or both? I’m not too sure. I thought I’d ask here and I’ll probably ping the Open MPI list as well. But, no matter what, does anyone have thoughts on how to solve this?

(Note: I know that the macOS distro includes MPICH3 it’s just that the infrastructure for the model I work on doesn’t know MPICH3. I’m going to try and extend that now (need to figure out how to link, etc., as well as get beta versions of libraries) but Open MPI is “built-in” so I tried that first.)

Thanks,
Matt

2

There was some response to my query on the Open MPI list: users

and they point to stdint.h not being found. As I detail in my post (should appear at the above link soon; quoted below) stdint.h is found by most of Open MPI’s configure step, but not when libevent2022 is configured. This is weird.

If I check my configure output:

============================================================================
== Configuring Open MPI

*** Startup tests
checking build system type… x86_64-apple-darwin15.6.0

checking for sys/types.h… yes
checking for sys/stat.h… yes
checking for stdlib.h… yes
checking for string.h… yes
checking for memory.h… yes
checking for strings.h… yes
checking for inttypes.h… yes
checking for stdint.h… yes
checking for unistd.h… yes

>
> So, the startup saw it. But:
>
> ```text
--- MCA component event:libevent2022 (m4 configuration macro, priority 80)
checking for MCA component event:libevent2022 compile mode... static
checking libevent configuration args... --disable-dns --disable-http --disable-rpc --disable-openssl --enable-thread-support --d
isable-evport
configure: OPAL configuring in opal/mca/event/libevent2022/libevent
configure: running /bin/sh './configure' --disable-dns --disable-http --disable-rpc --disable-openssl --enable-thread-support --
disable-evport  '--disable-wrapper-rpath' 'CC=pgcc' 'CXX=pgc++' 'FC=pgfortran' 'CFLAGS=-m64' 'CXXFLAGS=-m64' 'FCFLAGS=-m64' '--w
ithout-verbs' '--prefix=/Users/mathomp4/installed/Compiler/pgi-16.10/openmpi/2.0.1' 'CPPFLAGS=-I/Users/mathomp4/src/MPI/openmpi-
2.0.1 -I/Users/mathomp4/src/MPI/openmpi-2.0.1 -I/Users/mathomp4/src/MPI/openmpi-2.0.1/opal/include   -I/Users/mathomp4/src/MPI/o
penmpi-2.0.1/opal/mca/hwloc/hwloc1112/hwloc/include -Drandom=opal_random' --cache-file=/dev/null --srcdir=. --disable-option-che
cking
checking for a BSD-compatible install... /usr/bin/install -c
<snip>
checking for sys/types.h... no
checking for sys/stat.h... no
checking for stdlib.h... no
checking for string.h... no
checking for memory.h... no
checking for strings.h... no
checking for inttypes.h... no
checking for stdint.h... no
checking for unistd.h... no
3

This seems to be working

setenv CC pgcc
setenv FC pgfortran
setenv CXX pgc++
setenv FCFLAGS “-Mpreprocess -Mbackslash”

./configure --enable-mpi-fortran --enable-mpi-cxx --prefix=/path/to/openmpi

make
make check
make install

Something I also find useful is an siterc file that has switches not
normally defined by PGI compilers.

siterc for gcc commands PGI does not support

switch -ffast-math is hide;

switch -pipe is hide;

switch -fexpensive-optimizations is hide;

switch -pthread is
append(LDLIB1= -lpthread);

switch -qversion is
early
help(Display compiler version)
helpgroup(overall)
set(VERSION=YES);

switch -Wno-deprecated-declarations is hide;

switch -flat_namespace is hide;

you can make sure the file is read by running
pgfortran -dryrun foo.f

and see what rc files are read.

dave

4

Hmm. I just cannot get it to work. I’ll keep trying, but even your options didn’t work for me!

5

I did it again just to make sure. I use cshell on a Yosemite mac.

set path=(/opt/pgi/osx86-64/16.10/bin $path)
Make sure you have the PGI compilers in your $path

which pgfortran; which pgcc; which pgc++

which all point to /opt/pgi/osx86-64/16.10/bin

I described a siterc file to use with builds that cover some gcc switches
that are not handled by the PGI compilers - here is a siterc to handle them.


% more ~/siterc //the comments below are added to tell you the effect.
The do not belong in the rcfile.
switch -ffast-math is hide; //ignore this

switch -pipe is hide; // ignore

switch -fexpensive-optimizations is hide; / /ignore

switch -pthread is
append(LDLIB1= -lpthread); // means -lpthread

switch -qversion is
early
help(Display compiler version)
helpgroup(overall)
set(VERSION=YES); // means -V

switch -Wno-deprecated-declarations is hide; //ignore

switch -flat_namespace is hide; //ignore






If you want everyone to handle them, store it as
$PGI/osx86-64/16.10/bin/siterc

If you want only YOU to handle the new changes store it in your
home directory. ~/siterc (world readable)

to make sure it works, use -dryrun to display the driver instructions
without executing them


% pgfortran -dryrun -pipe -fexpensive-optimizations -pthread -Wno-deprecated-declarations -flat_namespace foo.f90
Reading rcfile /proj/pgi/osx86-64/16.10/bin/.pgfortranrc
Reading rcfile /proj/pgi/osx86-64/16.10/bin/nativerc
Reading rcfile /proj/pgi/osx86-64/16.10/bin/fnativerc
Reading rcfile /proj/pgi/osx86-64/16.10/bin/x86rc
Reading rcfile /proj/pgi/osx86-64/16.10/bin/x8664rc
Reading rcfile /proj/pgi/osx86-64/16.10/bin/osxrc
Reading rcfile /proj/pgi/osx86-64/16.10/bin/osx64rc
Reading rcfile /proj/pgi/osx86-64/16.10/bin/iparc
Reading rcfile /proj/pgi/osx86-64/16.10/bin/pgf90rc
Reading rcfile /proj/pgi/osx86-64/16.10/bin/pgftnrc
Reading rcfile /proj/pgi/osx86-64/16.10/bin/acc1rc
Reading rcfile /proj/pgi/osx86-64/16.10/bin/accosx8664rc
Skipping accxrc (not found)
Skipping nolocalrc (not found)
Skipping localrc (not found)
Reading rcfile /proj/pgi/osx86-64/16.10/bin/localrc.yosemite <===one for everyone
Reading rcfile /proj/pgi/osx86-64/16.10/bin/siterc
Reading local rcfile /home/tull/siterc <==============mine!!!
Skipping .mypgirc (not found)
Skipping .mypgfortranrc (not found)
Skipping $MYLOCALRC (not found)
Export PGI=/proj/pgi
Export PGIBIN=/proj/pgi/osx86-64/16.10/bin

If your siterc file is read you will see it. The rcfiles can define things multiple
times, but it is the last rc file that sets something that counts.

If not, all the switches defined in the siterc file will fail…




To build openmpi (make this work first, then add other switches)
setenv PGI /opt/pgi
setenv FC pgfortran
setenv CC pgcc
setenv CXX pgc++
setenv FCFLAGS “-Mpreprocess -Mbackslash”
will handle many of the compiler issues with how we handle '' in source.


cd /path/to/openmpi/package/openmpi-2.0.1

./configure --enable-mpi-fortran --enable-mpi-cxx --prefix=/path/to/final64/package

make
make check
make install

and things should work.


dave

6

Dave,

I used your posted configuration with a newly installed PGI Community edition, and it still fails at all the same places. Specifically,

checking for ANSI C header files… no
checking for sys/types.h… no
checking for sys/stat.h… no
checking for stdlib.h… no
checking for string.h… no
checking for memory.h… no
checking for strings.h… no
checking for inttypes.h… no
checking for stdint.h… no
checking for unistd.h… no
checking for dlfcn.h… no

then, a bit later,

checking zlib.h presence… yes
configure: WARNING: zlib.h: present but cannot be compiled
configure: WARNING: zlib.h: check for missing prerequisite headers?
configure: WARNING: zlib.h: see the Autoconf documentation
configure: WARNING: zlib.h: section “Present But Cannot Be Compiled”
configure: WARNING: zlib.h: proceeding with the compiler’s result
configure: WARNING: ## ---------------------------------- ##
configure: WARNING: ## Report this to http://libevent.org ##
configure: WARNING: ## ---------------------------------- ##
checking for zlib.h… no


Then, when it gets to the make command, it fails at

CC proc.lo
PGC-F-0249-#error – “No way to define ev_uint64_t” (/opt/openmpi-2.0.1/opal/mca/event/libevent2022/libevent/include/event2/util.h: 126)
PGC/x86-64 OSX 16.10-0: compilation aborted


This is on OS X 10.11.6. The gcc version is displaying as

[stentorian@bennet openmpi-2.0.1]$ gcc --version
Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/usr/include/c++/4.2.1
Apple LLVM version 8.0.0 (clang-800.0.42.1)
Target: x86_64-apple-darwin15.6.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin


and the Xcode version from About is 8.1 (8B62). I am using the following build script:


#!/bin/bash
CC=pgcc
FC=pgfortran
CXX=pgc++
FCFLAGS=“-Mpreprocess -Mbackslash”
export CC FC CXX FCFLAGS
./configure --enable-mpi-fortran --enable-mpi-cxx --prefix=/tmp/local
make


I had a prior installation of Xcode command line compilers only, but I updated to the latest Xcode just prior to trying this.

Any thoughts on why yours might work and mine and Matt’s do not?

7

Have you installed the PGI compilers since updating the Xcode?

I thing you should re-install if you did not, with the new Xcode parts
being the default when you call gcc.

Xcode updates should be checked to make sure they are installed
and you have assented to the license agreement. The most often
occurring problem is the Xcode command line tool not being installed
or assented.

Make sure the Xcoe install is complete and default, and make sure you
reinstalled the PGI compilers with the new Xcode the default.

dave

8

I also tried removing Xcode, downgrading to Xcode 7, and it still failed.

I am running the PGI compilers by using

$ open PGI.terminal

then compiling from there, in case that matters for replicability.

9

Okay, jtull, what black magic do you do? Your siterc does seem to solve this for me.

Do you have any idea what might be the thing that gets this working? Is it qversion? I can’t imagine hiding options should make a difference.

Still, it seems to work! I’m going to really stress OpenMPI now. I’ll let you know what happens.

Matt

10

Agreed. The siterc file is required to make this work.

11

Well, nuts. I just tried building Open MPI 2.1.0 with:

./configure --disable-wrapper-rpath CC=pgcc CXX=pgc++ FC=pgfortran FCFLAGS='-Mpreprocess -Mbackslash' --prefix=/Users/mathomp4/installed/Compiler/pgi-16.10/openmpi/2.1.0 | & tee configure.pgi-16.10.log

and jtull’s siterc file but when I do, the resulting mpifort does not actually work. It seems to, but when you try to simply mpirun a hello world it dies.

I’m doing some investigating now…