We need an OpenMPI with PGI installed on our cluster.
We are doing successful installations with 18.5 using the following:
export CC=pgcc
export FC=pgfortran
export F90=pgf90
export F77=pgf77
export CXX=pgc++
export CPP=cpp
export CFLAGS="-O2"
export FFLAGS="-O2"
export FCFLAGS="$FFLAGS"
export CXXFLAGS="$CFLAGS"
../configure --with-lsf=$lsf --with-lsf-libdir=$lsf/lib \
--with-verbs --prefix=$prefix --enable-mpirun-prefix-by-default \
--enable-shared
However, when using the same build script and 19.1 we get an error:
CC gds_dstore.lo
pgcc-Fatal-/zdata/groups/nfsopt/pgi/linux86-64-nollvm/19.1/bin/pgc TERMINATED by signal 11
Arguments to /zdata/groups/nfsopt/pgi/linux86-64-nollvm/19.1/bin/pgc
/zdata/groups/nfsopt/pgi/linux86-64-nollvm/19.1/bin/pgc ../../../../../../../../../../opal/mca/pmix/pmix2x/pmix/src/mca/gds/ds12/gds_dstore.c -opt 2 -terse 1 -inform warn -x 119 0xa10000 -x 122 0x40 -x 123 0x1000 -x 127 4 -x 127 17 -x 19 0x400000 -x 28 0x40000 -x 120 0x10000000 -x 70 0x8000 -x 122 1 -x 125 0x20000 -quad -vect 56 -y 34 16 -x 34 0x8 -x 32 26214400 -y 19 8 -y 35 0 -x 42 0x30 -x 39 0x40 -x 39 0x80 -x 59 4 -x 129 2 -tp sandybridge -x 120 0x1000 -astype 0 -x 121 1 -stdinc /zdata/groups/nfsopt/pgi/linux86-64-nollvm/19.1/include-gcc48:/zdata/groups/nfsopt/pgi/linux86-64-nollvm/19.1/include:/usr/lib/gcc/x86_64-redhat-linux/4.8.5/include:/usr/local/include:/usr/include -def unix -def __unix -def __unix__ -def linux -def __linux -def __linux__ -def __NO_MATH_INLINES -def __LP64__ -def __x86_64 -def __x86_64__ -def __LONG_MAX__=9223372036854775807L -def '__SIZE_TYPE__=unsigned long int' -def '__PTRDIFF_TYPE__=long int' -def __extension__= -def __amd_64__amd64__ -def __k8 -def __k8__ -def __MMX__ -def __SSE__ -def __SSE2__ -def __SSE3__ -def __SSSE3__ -predicate '#machine(x86_64) #lint(off) #system(posix) #cpu(x86_64)' -idir . -idir ../../../../../../../../../../opal/mca/pmix/pmix2x/pmix/src/mca/gds/ds12 -idir ../../../../src/include -idir ../../../../include -idir /zdata/groups/common/<>/special-courses/s141388/openmpi-3.1.3/build-tmp/opal/mca/pmix/pmix2x/pmix -idir /zdata/groups/common/<>/special-courses/s141388/openmpi-3.1.3/opal/mca/pmix/pmix2x/pmix -idir /zdata/groups/common/<>/special-courses/s141388/openmpi-3.1.3/opal/mca/pmix/pmix2x/pmix/src -idir /zdata/groups/common/<>/special-courses/s141388/openmpi-3.1.3/build-tmp/opal/mca/pmix/pmix2x/pmix/include -idir /zdata/groups/common/<>/special-courses/s141388/openmpi-3.1.3/opal/mca/pmix/pmix2x/pmix/include -idir /zdata/groups/common/<>/special-courses/s141388/openmpi-3.1.3 -idir /zdata/groups/common/<>/special-courses/s141388/openmpi-3.1.3/build-tmp -idir /zdata/groups/common/<>/special-courses/s141388/openmpi-3.1.3/opal/include -idir /zdata/groups/common/<>/special-courses/s141388/openmpi-3.1.3/build-tmp/opal/include -idir /zdata/groups/common/<>/special-courses/s141388/openmpi-3.1.3/build-tmp/opal/mca/event/libevent2022/libevent/include -idir /zdata/groups/common/<>/special-courses/s141388/openmpi-3.1.3/opal/mca/event/libevent2022/libevent -idir /zdata/groups/common/<>/special-courses/s141388/openmpi-3.1.3/opal/mca/event/libevent2022/libevent/include -idir /zdata/groups/common/<>/special-courses/s141388/openmpi-3.1.3/build-tmp/opal/mca/hwloc/hwloc1117/hwloc/include -idir /zdata/groups/common/<>/special-courses/s141388/openmpi-3.1.3/opal/mca/hwloc/hwloc1117/hwloc/include -def HAVE_CONFIG_H -def NDEBUG -def NDEBUG -def __PIC__ -def PIC -cmdline '+pgcc ../../../../../../../../../../opal/mca/pmix/pmix2x/pmix/src/mca/gds/ds12/gds_dstore.c -DHAVE_CONFIG_H -I. -I../../../../../../../../../../opal/mca/pmix/pmix2x/pmix/src/mca/gds/ds12 -I../../../../src/include -I../../../../include -I/zdata/groups/common/<>/special-courses/s141388/openmpi-3.1.3/build-tmp/opal/mca/pmix/pmix2x/pmix -I/zdata/groups/common/<>/special-courses/s141388/openmpi-3.1.3/opal/mca/pmix/pmix2x/pmix -I/zdata/groups/common/<>/special-courses/s141388/openmpi-3.1.3/opal/mca/pmix/pmix2x/pmix/src -I/zdata/groups/common/<>/special-courses/s141388/openmpi-3.1.3/build-tmp/opal/mca/pmix/pmix2x/pmix/include -I/zdata/groups/common/<>/special-courses/s141388/openmpi-3.1.3/opal/mca/pmix/pmix2x/pmix/include -I/zdata/groups/common/<>/special-courses/s141388/openmpi-3.1.3 -I/zdata/groups/common/<>/special-courses/s141388/openmpi-3.1.3/build-tmp -I/zdata/groups/common/<>/special-courses/s141388/openmpi-3.1.3/opal/include -I/zdata/groups/common/<>/special-courses/s141388/openmpi-3.1.3/build-tmp/opal/include -I/zdata/groups/common/<>/special-courses/s141388/openmpi-3.1.3/build-tmp/opal/mca/event/libevent2022/libevent/include -I/zdata/groups/common/<>/special-courses/s141388/openmpi-3.1.3/opal/mca/event/libevent2022/libevent -I/zdata/groups/common/<>/special-courses/s141388/openmpi-3.1.3/opal/mca/event/libevent2022/libevent/include -I/zdata/groups/common/<>/special-courses/s141388/openmpi-3.1.3/build-tmp/opal/mca/hwloc/hwloc1117/hwloc/include -I/zdata/groups/common/<>/special-courses/s141388/openmpi-3.1.3/opal/mca/hwloc/hwloc1117/hwloc/include -DNDEBUG -DNDEBUG -O2 -Mvect=sse -Mcache_align -Mpre -c -MD -fpic -DPIC -o .libs/gds_dstore.o' -outfile .libs/gds_dstore.o -x 123 0x80000000 -x 123 4 -preinclude _c_macros.h -x 119 0x20 -def __pgnu_vsn=40805 -autoinl 10 -x 168 400 -x 174 128000 -x 14 0x200000 -x 14 0x400000 -x 123 8 -x 120 0x200000 -x 70 0x40000000 -x 164 0x800000 -y 163 0xc0000000 -x 189 0x10 -y 189 0x4000000 -x 9 1 -x 72 0x1 -x 136 0x11 -quad -x 119 0x10000000 -x 129 0x40000000 -x 164 0x1000 -x 62 8 -asm /tmp/pgccX8qitgI9Bv8C.s
It always fails at said source code
gds_dstore.lo
.
I want to build it ourselves and not use the shipped MPI. :)
Have anybody seen this, and are there work-arounds?