Building (or not) OpenMPI on a Mac

Accelerated Computing HPC Compilers Legacy PGI Compilers
1

After failing with every combination of flags to run the configure script provided with OpenMPI-1.2.5, I did a little Googling for the recurring error message and found this:


Re: Portland Group cpp fails sanity check
Paul Eggert
Wed, 09 Jan 2008 17:04:23 -0800

“Brad Larsen” <[EMAIL PROTECTED]> writes:

configure:4274: /opt/pgi/linux86-64/7.1-1/bin/pgcpp conftest.c
“conftest.c”, line 16: error: identifier “Syntax” is undefined
Syntax error
^

Apparently the Portland group preprocessor refuses to preprocess
input files unless it can also fully parse them as C programs.
This behavior is weird and will break Autoconf-generated scripts.
Try substituting a C preprocessor that is not so all-fired picky:
you can use GCC’s preprocessor for that.

Now admittedly I’m running Mac OS X 10.4.11 on a dual-quadcore machine, with PGI Workstation 7.1-5, so not the same environment as the guy I ran across, but is Eggert correct? Am I doomed to failure anytime I throw the PGI tools at a GNU autoconf-driven build?

(And if anyone can pull off an OpenMPI build on OS X with PGI, I’d love to hear how you did it.)

2

Hi,

I do not see source version 1.2.5 for Mac OS X, so I assume they use the same source for all platform.

Try this(you may add more options):

./configure -prefix=/home/me/myopenmpi CC=pgcc CXX=pgcpp F77=pgf77 FC=pgf90 --enable-mpirun-prefix-by-default

Please use our latest release. I did it with 7.1-5 release, Openmpi 1.2.5. You might need to adjust some flags(multiply_defined) in opal/tools/wrappers/Makefile and a couple other places.

I have not run it to see if it works.

Hongyon

3

Alas, this results in:

*** C++ compiler and preprocessor
checking whether we are using the GNU C++ compiler… no
checking whether pgcpp accepts -g… yes
checking dependency style of pgcpp… none
checking how to run the C++ preprocessor… pgcpp -E
checking for the C++ compiler vendor… portland group
checking if C++ compiler works… no

  • It appears that your C++ compiler is unable to produce working
  • executables. A simple test application failed to properly
  • execute. Note that this is likely not a problem with Open MPI,
  • but a problem with the local compiler installation. More
  • information (including exactly what command was given to the
  • compiler and what error resulted when the command was executed) is
  • available in the config.log file in this directory.

configure: error: Could not run a simple C++ program. Aborting.

Looking in config.log, I see this:


configure:21643: pgcpp -c -DNDEBUG conftest.cpp >&5
configure:21649: $? = 0
configure:21904: result: portland group
configure:21926: checking if C++ compiler works
configure:22003: pgcpp -o conftest -DNDEBUG conftest.cpp >&5
/usr/bin/ld: fatal error in /usr/bin/ld64
configure:22006: $? = 2
configure: program exited with status 2
configure: failed program was:
| /* confdefs.h. /
| define PACKAGE_NAME “Open MPI”
| define PACKAGE_TARNAME “openmpi”
| define PACKAGE_VERSION “1.2.5”
| define PACKAGE_STRING “Open MPI 1.2.5”
| define PACKAGE_BUGREPORT "> 2. Getting help — Open MPI main documentation> "
| define OMPI_MAJOR_VERSION 1
| define OMPI_MINOR_VERSION 2
| define OMPI_RELEASE_VERSION 5
| define OMPI_GREEK_VERSION “”
| define OMPI_VERSION “5”
| define ORTE_MAJOR_VERSION 1
| define ORTE_MINOR_VERSION 2
| define ORTE_RELEASE_VERSION 5
| define ORTE_GREEK_VERSION “”
| define ORTE_VERSION “5”
| define OPAL_MAJOR_VERSION 1
| define OPAL_MINOR_VERSION 2
| define OPAL_RELEASE_VERSION 5
| define OPAL_GREEK_VERSION “”
| define OPAL_VERSION “5”
| define OMPI_ARCH “i386-apple-darwin8.11.1”
| define OMPI_ENABLE_MEM_DEBUG 0
| define OMPI_ENABLE_MEM_PROFILE 0
| define OMPI_ENABLE_DEBUG 0
| define OMPI_WANT_MPI_CXX_SEEK 1
| define MPI_PARAM_CHECK ompi_mpi_param_check
| define OMPI_WANT_PRETTY_PRINT_STACKTRACE 1
| define OMPI_WANT_PERUSE 0
| define OMPI_ENABLE_PTY_SUPPORT 1
| define OMPI_ENABLE_HETEROGENEOUS_SUPPORT 1
| define OPAL_ENABLE_TRACE 0
| define OPAL_ENABLE_IPV6 1
| define ORTE_WANT_ORTERUN_PREFIX_BY_DEFAULT 1
| define OMPI_CC “pgcc”
| define STDC_HEADERS 1
| define _GNU_SOURCE 1
| define OMPI_C_HAVE_BUILTIN_EXPECT 0
| define OMPI_C_HAVE_BUILTIN_PREFETCH 0
| define HAVE_SYS_TYPES_H 1
| define HAVE_SYS_STAT_H 1
| define HAVE_STDLIB_H 1
| define HAVE_STRING_H 1
| define HAVE_MEMORY_H 1
| define HAVE_STRINGS_H 1
| define HAVE_INTTYPES_H 1
| define HAVE_STDINT_H 1
| define HAVE_UNISTD_H 1
| define HAVE_LONG_LONG 1
| define HAVE_LONG_DOUBLE 1
| define HAVE_INT8_T 1
| define HAVE_UINT8_T 1
| define HAVE_INT16_T 1
| define HAVE_UINT16_T 1
| define HAVE_INT32_T 1
| define HAVE_UINT32_T 1
| define HAVE_INT64_T 1
| define HAVE_UINT64_T 1
| define HAVE_INTPTR_T 1
| define HAVE_UINTPTR_T 1
| define HAVE_MODE_T 1
| define HAVE_SSIZE_T 1
| define HAVE_PTRDIFF_T 1
| define SIZEOF_CHAR 1
| define SIZEOF_SHORT 2
| define SIZEOF_INT 4
| define SIZEOF_LONG 8
| define SIZEOF_LONG_LONG 8
| define SIZEOF_LONG_DOUBLE 8
| define SIZEOF_FLOAT 4
| define SIZEOF_DOUBLE 8
| define SIZEOF_VOID_P 8
| define SIZEOF_SIZE_T 8
| define SIZEOF_SSIZE_T 8
| define SIZEOF_PTRDIFF_T 8
| define OMPI_ALIGNMENT_CHAR 1
| define OMPI_ALIGNMENT_SHORT 2
| define OMPI_ALIGNMENT_WCHAR 4
| define OMPI_ALIGNMENT_INT 4
| define OMPI_ALIGNMENT_LONG 8
| define OMPI_ALIGNMENT_LONG_LONG 8
| define OMPI_ALIGNMENT_FLOAT 4
| define OMPI_ALIGNMENT_DOUBLE 8
| define OMPI_ALIGNMENT_LONG_DOUBLE 8
| define OMPI_ALIGNMENT_VOID_P 8
| define OMPI_NEED_C_BOOL 1
| define OMPI_ENABLE_MPI_PROFILING 1
| define OMPI_HAVE_WEAK_SYMBOLS 0
| define OMPI_CXX “pgcpp”
| / end confdefs.h. */
| include
| int
| main ()
| {
| std::string foo = “Hello, world”
| ;
| return 0;
| }
configure:22037: result: no
configure:22051: error: Could not run a simple C++ program. Aborting.

I appreciate your help with this very much; my frustration’s running high because this should be a simple step towards doing what I /really/ want to do, and that’s compile my own code.

4

Which PGI version do you use? Please use 7.1-5 release.

If you use latest version, check your installation or your OS by running a simple hello c++ program to see if it pgcpp works.

Hongyon

5

If the hello program does not work.

Also make sure you set your PATH correctly.

Path should be set to:

setenv PATH /opt/pgi/osx86-64/7.1-5/bin:$PATH


Please post what is in /opt/pgi/osx86-64/7.1-5/bin/localrc.

Hongyon

6

Hongyon:

I’m using 7.1-5. I tried a simple “Hello, world” program in C++ and got the same /usr/bin/ld64 error from the configure script. A little Googling and some trial-and-error revealed that I needed to pass “-tp=p7” through the various *FLAGS options to the configure script. Now it’s breezing through the C tests, the C++ tests, the F77 tests, and I’ve run into a known issue involving OpenMPI, PGI compilers, and the location of modules when -g is used.

I’m making solid progress now and you definitely got me going in that direction. Thanks so much for your help!

7

Something is not quite right. Is your machine 32-bit only? Or you tried to compile 32-bit? Do you have /opt/pgi/osx86-64/7.1-5/bin directory? If yes, what is in localrc file in that directory. I assume you were admin/root when you installed compilers?

Hongyon

8

I had an admin do the installation; he put in the 32 and 64 bit versions. The localrc file you referenced is:

OPUS> cat /opt/pgi/osx86-64/7.1-5/bin/localrc

set GCCDIR=/usr/lib/gcc/i686-apple-darwin8/4.0.1;
set GCCINC=/usr/lib/gcc/i686-apple-darwin8/4.0.1/include;
set OEM_INFO=64-bit target on Apple OS/X ;
set LOCALRC=YES;
set LC=$if(-Bstatic,-lgcc -lgcc_eh -lc -lgcc -lgcc_eh -lc, -lgcc -lc -lgcc) -lSystem;
export PGI=“/opt/pgi”;

makelocalrc executed by root Tue Mar 4 15:30:10

The machine is running Mac OS 10.4.11, which has a blend of 32-bit and 64-bit code on it. My software, the stuff I need a functioning OpenMPI for, is all 64-bit.

I don’t have anything explicitly set in my environment to parse the “localrc” file – is that something I should have done somewhere?

(My apologies for being so generally clueless.)

9

Hi,

How do you set your PATH?
Can you post your output of echo $PATH? I am trying to check why you get an error when using 64-bit compiler.
What is output of pgf90 -V or pgcpp -V?

Using -tp=p7 will give you 32-bit compiler.

Please make sure to clean up your configuration(OpenMPI) before reconfigure with 64-bit compilers. If there is any -tp=p7 or -tp=??? (anything) in your configuration/makefile, make sure to remove them.

There is no need to parse localrc. The compiler driver should handle it as long as you set the PATH correctly.

Hongyon

10

I got more information from our engineer.

PGI compilers use system linker. There is a problem involving debugging information with a linker (ld64). The problem occurs in Tiger. They have fixed problem in Leopard.

You don’t see an error in 32-bit because there is not an issue with 32-bit linker.

I would recommend to run configuration without -g on Tiger.

Hongyon

11

Sure thing:

OPUS> echo $PATH

/opt/pgi/osx86-64/7.1/bin:packages/lampi-1.5.16/bin:/usr/sbin:/sbin:/usr/local/bin:/opt/local/vendor:/bin:/usr/X11R6/bin:/usr/lanl/bin:/opt/local/bin:/opt/local/ensight/bin

The compiler tests:

OPUS> pgf90 -V nofile.f

pgf90 7.1-5 64-bit target on Apple OS/X
Copyright 1989-2000, The Portland Group, Inc. All Rights Reserved.
Copyright 2000-2007, STMicroelectronics, Inc. All Rights Reserved.
PGF90/x86-64 OSX 7.1-5
Copyright 1989-2000, The Portland Group, Inc. All Rights Reserved.
Copyright 2000-2007, STMicroelectronics, Inc. All Rights Reserved.
PGF90-F-0002-Unable to open source input file: nofile.f
PGF90/x86-64 OSX 7.1-5: compilation aborted



OPUS> pgcpp -V nofile.cpp

pgcpp 7.1-5 64-bit target on Apple OS/X
Copyright 1989-2000, The Portland Group, Inc. All Rights Reserved.
Copyright 2000-2007, STMicroelectronics, Inc. All Rights Reserved.
Edison Design Group C/C++ Front End, version 3.6 (Jan 23 2008 12:37:02)
Copyright 1988-2006 Edison Design Group, Inc.

Catastrophic error: could not open source file “nofile.cpp”

1 catastrophic error detected in the compilation of “nofile.cpp”.
Compilation terminated.

I started with a clean source tree of OpenMPI and the most basic configure:

cd openmpi-1.2.5 && ./configure CC=pgcc CXX=pgcpp F77=pgf77 FC=pgf90

And this failed with:

*** C++ compiler and preprocessor
checking whether we are using the GNU C++ compiler… no
checking whether pgcpp accepts -g… yes
checking dependency style of pgcpp… none
checking how to run the C++ preprocessor… pgcpp -E
checking for the C++ compiler vendor… portland group
checking if C++ compiler works… no

  • It appears that your C++ compiler is unable to produce working
  • executables. A simple test application failed to properly
  • execute. Note that this is likely not a problem with Open MPI,
  • but a problem with the local compiler installation. More
  • information (including exactly what command was given to the
  • compiler and what error resulted when the command was executed) is
  • available in the config.log file in this directory.

configure: error: Could not run a simple C++ program. Aborting.

In the config.log file the relevant error was:

configure:21643: pgcpp -c -DNDEBUG conftest.cpp >&5
configure:21649: $? = 0
configure:21904: result: portland group
configure:21926: checking if C++ compiler works
configure:22003: pgcpp -o conftest -DNDEBUG conftest.cpp >&5
/usr/bin/ld: fatal error in /usr/bin/ld64
configure:22006: $? = 2

Does this help?

12

They look fine.

Can you try this? Copy file below into some file, say, abc.cpp.

Compile with pgcpp -v abc.cpp -o abc

Then run abc if it compiles.

----------------------------start abc.cpp------------------------------
#include
int
main ()
{
std::string foo = “Hello, world”
;
return 0;
}


Hongyon

13

Here you are:

vikingale:~> cat hello.cpp
include
int main () {
std::string foo = “Hello, world”;
return 0;
}



vikingale:~> pgcpp -v hello.cpp

/opt/pgi/osx86-64/7.1-5/bin/pgcpp1 --nollalign -Dunix -D__unix -D__unix__ -D__LITTLE_ENDIAN__ -D__inline__= -Dosx86 -D__osx86 -D__osx86__ -D__APPLE__ -D__LP64__ -D__x86_64__ -D_M_IX86 -D__NO_MATH_INLINES -D__amd64__ -D__extension__= -D__SSE__ -D__MMX__ -D__SSE2__ -D__SSE3__ -D__SSSE3__ -D__PGI -D__signed__=signed -I/opt/pgi/osx86-64/7.1-5/include/CC -I/opt/pgi/osx86-64/7.1-5/include -I/usr/local/include -I/usr/lib/gcc/i686-apple-darwin8/4.0.1/include -I/usr/lib/gcc/i686-apple-darwin8/4.0.1/include -I/usr/include -q -o /var/tmp/pgcppZqcczEXD4Iv-.il hello.cpp

/opt/pgi/osx86-64/7.1-5/bin/pgcpp2 hello.cpp -opt 1 -x 59 4 -x 119 0x10000010 -x 129 0x80 -x 19 0x400000 -x 119 0x40610400 -x 119 0x1000 -x 59 4 -x 129 0x10 -x 70 0x8000 -x 122 1 -x 120 0x80 -x 122 0x40 -x 123 0x1000 -x 127 21 -tp core2-64 -y 80 0x40000000 -x 120 0x400 -astype 1 -fn hello.cpp -il /var/tmp/pgcppZqcczEXD4Iv-.il -x 123 4 -x 2 0x400 -x 123 0x80000000 -alwaysinline /opt/pgi/osx86-64/7.1-5/lib/libintrinsics.il 4 -x 120 0x200000 -cmdline ‘+pgcpp /var/tmp/pgcppZqcczEXD4Iv-.il -v’ -asm /var/tmp/pgcpplqccHrDXWiwS.s
PGCC/x86 OSX 7.1-5: compilation successful

/usr/bin/as -arch x86_64 /var/tmp/pgcpplqccHrDXWiwS.s -o hello.o
Action(ReadTIFile(./hello.ti))

/usr/bin/ld /usr/lib/crt1.o /opt/pgi/osx86-64/7.1-5/lib/pgimainxx.o -arch x86_64 -L/opt/pgi/osx86-64/7.1-5/lib -L/usr/lib -L/usr/lib/gcc/i686-apple-darwin8/4.0.1/x86_64 -L/usr/lib/gcc/i686-apple-darwin8/4.0.1 hello.o -lstd -lcpp -lc -lnspgc -lpgc -lm -lgcc -lc -lgcc -lSystem
/usr/bin/ld: fatal error in /usr/bin/ld64
pgcpp-Fatal-linker completed with exit code 1

Unlinking /var/tmp/pgcppZqcczEXD4Iv-.il
Unlinking /var/tmp/pgcpplqccHrDXWiwS.s

(And thanks for your continuing help; I’ll probably add that every 3-4 messages!)

14

Hi,

What is your xcode version?

It looks like your system only support 32-bit but I am not sure. Can you please try this again?

% cat > abc.c
main() { }
%Control-C

%gcc -m64 abc.c
%a.out


If it gives you Bad CPU type an error, then the system does not support 64-bit.

Try it with g++ too. Just rename abc.cpp.

Hongyon

15

XCode is 2.4.1.

I followed your abc instructions with gcc/g++ and both programs ran (of course they didn’t do anything but they executed properly).

16

Hi,

There is an issue with ld64 and currently our system similar to your configuration is down and I can’t test on it yet.

I have file a TPR # 14620 for this. We will let you know progress as soon as possible.

If you want to the status of the TPR, please e-mail trs@pgroup.com. I will also keep you updated on user forum here.

Thanks for you patience.
Hongyon

17

Thank you for all the help with this. I have the luxury of time since this platform port is something I’m doing “for fun” and not because my boss ordered it – so when it works, it works. Thanks again,


Jim