MVAPICH2 and PGI 12.2

Accelerated Computing HPC Compilers Legacy PGI Compilers
1

When trying to build MVAPICH2 1.8a2 with PGI 12.2, I ran into an error. First, I configured with:

env CC=pgcc CXX=pgCC FC=pgfortran F77=pgfortran ./configure --prefix=/home/user/mvapich2-18a2-nemesis --with-device=ch3:nemesis | & tee configure.log

and upon running make I get:

 make[5]: Entering directory `/home/user/src/mvapich2-1.8a2/src/pm/hydra/tools/topo/hwloc/hwloc/src'
   CC     topology.lo
   CC     traversal.lo
   CC     distances.lo
 PGC-S-0039-Use of undeclared variable __FLT_MAX__ (distances.c: 325)
 PGC-S-0039-Use of undeclared variable __FLT_MIN__ (distances.c: 325)
 PGC/x86-64 Linux 12.2-0: compilation completed with severe errors

It’s like it’s not seeing the include directory for PGI 12.2 which has float.h.

When I run pgcc -show:

(203) > pgcc -show | grep -i include
pgcc-Warning-No files to process
COMPINCDIR          Compiler include subdirectory          =include
DEFCPPINC           =/opt/pgi/linux86-64/12.2/include/CC
DEFSTDINC           =/opt/pgi/linux86-64/12.2/include  /usr/local/include /usr/lib/gcc/x86_64-redhat-linux/4.1.2/include /usr/lib/gcc/x86_64-redhat-linux/4.1.2/include /usr/include
DRYSTDINC           Display standard include directories and exit
GCCINC              =/usr/lib/gcc/x86_64-redhat-linux/4.1.2/include
GPPDIR              =/usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2 /usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/x86_64-redhat-linux /usr/lib/gcc/x86_64-redhat-linux/4.1.2/../../../../include/c++/4.1.2/backward
I8STORAGE           Indicates whether to include pgfi8st.o object
MPI1INC             =/opt/pgi/linux86-64/2012/mpi/mpich/include
MPIHPINC            =/opt/hpmpi/include
MPILIBINC           =/opt/pgi/linux86-64/2012/mpi/mpich/include
STDINC              =/opt/pgi/linux86-64/12.2/include  /usr/local/include /usr/lib/gcc/x86_64-redhat-linux/4.1.2/include /usr/lib/gcc/x86_64-redhat-linux/4.1.2/include /usr/include

/opt/pgi/linux86-64/12.2/include is in DEFSTDINC.

Is there an extra flag I need to pass? Or something slightly off with the 12.2 install?

Thanks,
Matt

2

Okay, a bit more investigating seems to indicate that this is due to the inclusion of a new ifndef in float.h:

(214) > diff /opt/pgi/linux86-64/11.10/include/float.h /opt/pgi/linux86-64/12.2/include/float.h
15a16
> #if !defined(_GNU_SOURCE)
50a52
> #endif

Indeed, this ifndef is new from 12.1 to 12.2!

So, I tried to configure with an additional CPPFLAGS=-D_GNU_SOURCE, but the make still fails (maybe hwloc manipulates _GNU_SOURCE as well).

Any ideas on how to get around this? Obviously I could remove the ifndef in float.h, but I’m assuming that’s there for a good reason (I tried looking at the TPRs, but there aren’t any publicly reported for 12.2 yet.)

3

I encountered the same problem. If you look closer at float.h you’ll see it checks if_GNU_SOURCE is not defined, so defining it won’t do any good. My workaround was to wrap the include as follows:

#if (defined (_GNU_SOURCE))&&(__PGIC__==12)&&(__PGIC_MINOR==2)
#undef _GNU_SOURCE
#include <float>
#define _GNU_SOURCE
#else
#include <float>
#endif

Not pretty, and if this change in float.h carries over into future versions it will need to be modified.

edit: Something funny going on with the BBCode, it keeps removing the .h extension in the include statements. But you probably get the idea.

4

The “_GNU_SOURCE” line was added for our new g++ compatibility feature in the our C++ compiler (i.e. the pgc++ driver, or pgcpp --gnu). Though, our engineers may broken something on the C side because of it. Let me investigate and get back to you.

  • Mat
5

Unfortunately, the correction did not make it into the 12.2 release

In the two files listed below,

change
#if !defined(_GNU_SOURCE)
to
#if !defined(_GNU_SOURCE) || !defined(__cplusplus)

/opt/pgi/linux86-64/12.2/include/float.h
/opt/pgi/linux86-64/12.2/include/limits.h

6

Thank you for the quick reply. I came here considering submitting a problem report but first searched the forums to see if anyone else encountered the same issue.

7

I found that the limits.h file already had your suggested #if statement. It was only float.h that required correction. That solved my problems building MVAPICH2, but OpenMPI is having related problems. In the C++ framework, I am getting:

"/opt/pgi/linux86-64/12.2/include/CC/stl/_limits.h", line 153: error:
          identifier "__CHAR_BIT__" is undefined
    ((int)((sizeof(_Int) * (CHAR_BIT))) - ((__imin == 0) ? 0 : 1))

on a C++ file being compiled with pgCC. I am guessing in this case that __cplusplus will be defined and thus the CHAR_BIT macro in limits.h is getting #if’ed out. I don’t have time to debug this the rest of the day but will get back to it next week.

8

OpenMPI mus tbe defining _GNU_SOURCE .

As a workaround, the best thing to do is to remove that #ifdef, and the corresponding #endif from
limits.h and float.h.

This has turned out to b a popular flag. Our mistake. We’ll have it fixed in 12.3.

9

For what’s it’s worth. I’m running into close but similar issue related to float.h vars not being found when compiling OpenMPI in OFED 1.5.3.2 using PGI v12.2

libtool: compile:  pgcc -DHAVE_CONFIG_H -I. -I../../opal/include -I../../orte/include -I../../ompi/include -I../../opal/mca/paffinity/linux/plpa/src/libplpa -I../.. -D_REENTRANT -O -DNDEBUG -c cmd_line.c  -fpic -DPIC -o .libs/cmd_line.o

PGC-S-0039-Use of undeclared variable __LDBL_MANT_DIG__ (../../opal/util/arch.h: 271)
PGC-S-0039-Use of undeclared variable __LDBL_MAX_EXP__ (arch.c: 70)
PGC/x86-64 Linux 12.2-0: compilation completed with severe errors

I’m sorta new to PGI so when will 12.3 be available or is there a way to get beta release or hot fix?

10

Hi S. Fishback,

I’m sorta new to PGI so when will 12.3 be available

Baring delays, it’s currently scheduled for March 8th.

is there a way to get beta release or hot fix?

The workaround is to remove the _GNU_SOURCE ifdef in these two files.

Hope this helps,
Mat

11

Encountered the same issue with MPICH2 1.4.1p1. It would be nice if this was posted on the MPICH page ( Porting and Tuning Guides | PGI). It would have saved me some time and head banging.