Probelm in building HDF5

Dr_TinghaiOu24301 · October 4, 2017, 2:48pm

I have pgfortran 17.4-0 64-bit, pgcc 17.4-0 64-bit, and pgc++ 17.4-0 64-bit installed on Red Hat Enterprise Linux Server release 7.3 (Maipo). (3.10.0-514.16.1.el7.x86_64)
I try to build NetCDF for Fortran following the guidance on the page http://www.pgroup.com/resources/netcdf/netcdf43_pgi2014.htm.
I received error during configuring.

configure:4645: checking whether we are using the GNU C compiler
configure:4664: pgcc -c -O tp=p7-64 -DpgiFortran conftest.c >&5
PGC-S-0040-Illegal use of symbol, choke (conftest.c: 16)
PGC-W-0156-Type not specified, ‘int’ assumed (conftest.c: 16)
PGC/x86-64 Linux 17.4-0: compilation completed with severe errors
configure:4664: $? = 2
configure: failed program was:
| /* confdefs.h */

I received the following error when building HDF5,

Making install in hl
make[1]: Entering directory /data/Util/WORK_DIR/hdf5/hdf5-1.8.19/hl' Making install in src make[2]: Entering directory /data/Util/WORK_DIR/hdf5/hdf5-1.8.19/hl/src’
make[3]: Entering directory `/data/Util/WORK_DIR/hdf5/hdf5-1.8.19/hl/src’
/usr/bin/mkdir -p ‘/data/Util/INSTALL_DIR/hdf5/lib’
/bin/sh …/…/libtool --mode=install /usr/bin/install -c libhdf5_hl.la ‘/data/Util/INSTALL_DIR/hdf5/lib’
libtool: warning: relinking ‘libhdf5_hl.la’
libtool: install: (cd /data/Util/WORK_DIR/hdf5/hdf5-1.8.19/hl/src; /bin/sh “/data/Util/WORK_DIR/hdf5/hdf5-1.8.19/libtool” --silent --tag CC --mode=relink pgcc -c99 -Minform=inform -fast -s -O tp=p7-64 -version-info 12:0:2 -o libhdf5_hl.la -rpath /data/Util/INSTALL_DIR/hdf5/lib H5DO.lo H5DS.lo H5IM.lo H5LT.lo H5LTanalyze.lo H5LTparse.lo H5PT.lo H5TB.lo …/…/src/libhdf5.la -lz -ldl -lm )
/data/Util/WORK_DIR/hdf5/hdf5-1.8.19/libtool: line 10545: pgcc: command not found
libtool: error: error: relink ‘libhdf5_hl.la’ with the above command before installing it

Could you please help me with this?

tull · October 4, 2017, 5:51pm

I just built hdf5 with 17.4. Let me take you through the process.

First I downloaded hdf5-1.8.19.tar.gz and untarred it.
Then I cd into the top directory, and I run

./configure --help
to see what switches I might have to set and any ENV variables.
I see I have to enable fortran and c++ support in hdf5.

I put PGI 17.4 into my path.

export CC=pgcc
export CXX=pgc++
export FC=pgfortran

./configure --prefix=/path/to/final64/installation
–enable-fortran2003 --enable-fortran --enable-cxx

make
make check
make install

and it all seems to work. I have the same installation as you,
but there is a difference. I have this siterc file I use to define
some gcc switches that PGI does not, so that any that slip into the
build don’t break it. I see I have to enable fortran and c++ in hdf5.

% cat siterc
switch -ffast-math is hide;

switch -pipe is hide;

switch -fexpensive-optimizations is hide;

switch -fno-strict-aliasing is hide;

switch -Wall is hide;

switch -pthread is
append(LDLIB1= -lpthread);

switch -qversion is
early
help(Display compiler version)
helpgroup(overall)
set(VERSION=YES);

switch -Wno-deprecated-declarations is hide;

I put the file (world readable) in my home directory, and I run

pgcc -dryrun x.c ! x.c need not exist

to see if the siterc file is read.

From that things work.

Setting tp=p7-64
is using a really old chip, and possibly a newer chip with
more performance should be your platform. But you do want to
build to support your oldest cpu.

dave

Dr_TinghaiOu24301 · October 5, 2017, 8:32am

Dear Dave,
Thank you for your quick reply and the instruction. I will have a test based on your suggestions.
Before working with it, I would like to ask one question about Specify the type of the target processor.

I have a new server with 40 CPUs with Intel® Xeon® CPU E5-2640 v4 @ 2.40GHz (x86_64, 32-bit, 64-bit).

which ‘tp’ setting do you recommend?

Best regards
Tinghai

Dr_TinghaiOu24301 · October 5, 2017, 1:53pm

As a follow-up.

I had a try following the instructions.
./configure --prefix=/data/Util/INSTALL_DIR/hdf5
–enable-fortran2003 --enable-fortran --enable-cxx

make
make check
sudo make install

I got same error as before.

Below please find some basic information about the compiling.
Compiling Options:

Compilation Mode: production
C Compiler: /appl/pgi/linux86-64/17.4/bin/pgcc ( pgcc 17.4-0 64-bit target on x86-64 Linux -tp haswell )
CFLAGS: -O tp=p7-64
H5_CFLAGS: -c99 -Minform=inform -fast -s
AM_CFLAGS:
CPPFLAGS: -DpgiFortran
H5_CPPFLAGS: -D_GNU_SOURCE -D_POSIX_C_SOURCE=200112L -DNDEBUG -UH5_DEBUG_API
AM_CPPFLAGS:
Shared C Library: yes
Static C Library: yes
Statically Linked Executables: no
LDFLAGS:
H5_LDFLAGS:
AM_LDFLAGS:
Extra libraries: -lz -ldl -lm
Archiver: ar
Ranlib: ranlib
Debugged Packages:
API Tracing: no

Languages:

Fortran: yes
Fortran Compiler: /appl/pgi/linux86-64/17.4/bin/pgfortran
Fortran 2003 Compiler: yes
Fortran Flags: -O tp=p7-64
H5 Fortran Flags:
AM Fortran Flags:
Shared Fortran Library: yes
Static Fortran Library: yes

C++: yes
C++ Compiler: /appl/pgi/linux86-64/17.4/bin/pgc++
C++ Flags: -O tp=p7-64
H5 C++ Flags:
AM C++ Flags:
Shared C++ Library: yes
Static C++ Library: yes

Features:

Parallel HDF5: no
High Level library: yes
Threadsafety: no
Default API Mapping: v18
With Deprecated Public Symbols: yes
I/O filters (external): deflate(zlib)
MPE: no
Direct VFD: no
dmalloc: no
Clear file buffers before write: yes
Using memory checker: no
Function Stack Tracing: no
Strict File Format Checks: no
Optimization Instrumentation: no

tull · October 5, 2017, 4:20pm

You do not have to install the compilers on each platform
you intend to run your code on. On any platform, you can run

pgf90 -V

and it will return the version of the compiler, and the version of
CPU you have. The ‘best’ choice for that system would be the oldest
cpu in your system.

Please send the output of

pgf90 -dryrun x.f
to see if the siterc file I sent is being read.

There should be a make log or config.log file with information
about what failed.

dave

Dr_TinghaiOu24301 · October 5, 2017, 10:54pm

I got error message after pgf90 -V if I enable siterc
RC file /appl/pgi/linux86-64/17.4/bin/.pgf90rc line 48: Bad RC file:
ffast - math is hide
pgf90 17.4-0 64-bit target on x86-64 Linux -tp haswell
PGI Compilers and Tools
Copyright © 2017, NVIDIA CORPORATION. All rights reserved.

I got the same error information after make,
Makefile:264: *** missing separator. Stop.

in Magefile,

262 CCDEPMODE = depmode=pgcc
263 CC_VERSION = /appl/pgi/linux86-64/17.4/bin/pgcc ( RC file /appl/pgi/linux86-64/17.4/bin/.pgccrc line 48: Bad RC file:
264 pgcc 17.4-0 64-bit target on x86-64 Linux -tp haswell )
265 CFLAGS = -O tp=p7-64

output message for ‘pgf90 -dryrun x.f’
Reading rcfile /appl/pgi/linux86-64/17.4/bin/.pgf90rc
Reading rcfile /appl/pgi/linux86-64/17.4/bin/nativerc
Reading rcfile /appl/pgi/linux86-64/17.4/bin/fnativerc
Reading rcfile /appl/pgi/linux86-64/17.4/bin/x86rc
Reading rcfile /appl/pgi/linux86-64/17.4/bin/x8664rc
Reading rcfile /appl/pgi/linux86-64/17.4/bin/lin86rc
Reading rcfile /appl/pgi/linux86-64/17.4/bin/lin8664rc
Reading rcfile /appl/pgi/linux86-64/17.4/bin/iparc
Reading rcfile /appl/pgi/linux86-64/17.4/bin/pgf90rc
Reading rcfile /appl/pgi/linux86-64/17.4/bin/pgftnrc
Reading rcfile /appl/pgi/linux86-64/17.4/bin/acc1rc
Reading rcfile /appl/pgi/linux86-64/17.4/bin/acclin8664rc
Skipping accxrc (not found)
Reading rcfile /appl/pgi/linux86-64/17.4/bin/localrc
Skipping localrc.casmoc-lyk-1 (not found)
Reading local rcfile /home/tinghai/siterc
RC file /appl/pgi/linux86-64/17.4/bin/.pgf90rc line 48: Bad RC file:
ffast - math is hideSkipping siterc.casmoc-lyk-1 (not found)
Skipping .mypgirc (not found)
Skipping .mypgf90rc (not found)
Skipping .mypgx86rc (not found)
Skipping $MYLOCALRC (not found)
Export PGI=/appl/pgi

/appl/pgi/linux86-64/17.4/bin/pgf901 x.f -opt 1 -terse 1 -inform warn -nohpf -nostatic -x 19 0x400000 -quad -x 59 4 -x 15 2 -x 49 0x400004 -x 51 0x20 -x 57 0x4c -x 58 0x10000 -x 124 0x1000 -tp haswell -x 57 0xfb0000 -x 58 0x78031040 -x 47 0x08 -x 48 4608 -x 49 0x100 -x 120 0x200 -stdinc /appl/pgi/linux86-64/17.4/include-gcc48:/appl/pgi/linux86-64/17.4/include:/usr/lib/gcc/x86_64-redhat-linux/4.8.5/include:/usr/local/include:/usr/include -def unix -def __unix -def unix -def linux -def __linux -def linux -def __NO_MATH_INLINES -def LP64 -def __x86_64 -def x86_64 -def LONG_MAX=9223372036854775807L -def ‘SIZE_TYPE=unsigned long int’ -def ‘PTRDIFF_TYPE=long int’ -def __THROW= -def extension= -def amd_64__amd64 -def __k8 -def k8 -def SSE -def MMX -def SSE2 -def SSE3 -def SSSE3 -def STDC_HOSTED -nofreeform -vect 48 -y 54 1 -x 70 0x40000000 -y 163 0xc0000000 -x 189 0x10 -modexport x.cmod -modindex x.cmdx -output x.ilm

/appl/pgi/linux86-64/17.4/bin/pgf902 x.ilm -fn x.f -opt 1 -terse 1 -inform warn -x 51 0x20 -x 119 0xa10000 -x 122 0x40 -x 123 0x1000 -x 127 4 -x 127 17 -x 19 0x400000 -x 28 0x40000 -x 120 0x10000000 -x 70 0x8000 -x 122 1 -x 125 0x20000 -quad -x 59 4 -tp haswell -x 120 0x1000 -x 124 0x1400 -y 15 2 -x 57 0x3b0000 -x 58 0x48000000 -x 49 0x100 -x 120 0x200 -astype 0 -x 70 0x40000000 -x 124 1 -y 163 0xc0000000 -x 189 0x10 -y 189 0x4000000 -asm x.s

/usr/bin/as x.s -o x.o

/appl/pgi/linux86-64/17.4/bin/pgappend -noerror x.o -name .IPDINFO x.cmod -name .IPEINFO x.cmdx

/usr/bin/ld /usr/lib64/crt1.o /usr/lib64/crti.o /appl/pgi/linux86-64/17.4/lib/trace_init.o /usr/lib/gcc/x86_64-redhat-linux/4.8.5/crtbegin.o /appl/pgi/linux86-64/17.4/lib/initmp.o /appl/pgi/linux86-64/17.4/lib/f90main.o --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 /appl/pgi/linux86-64/17.4/lib/pgi.ld -L/appl/pgi/linux86-64/17.4/lib -L/usr/lib64 -L/usr/lib/gcc/x86_64-redhat-linux/4.8.5 x.o -rpath /appl/pgi/linux86-64/17.4/lib -rpath /usr/lib/gcc/x86_64-redhat-linux/4.8.5/…/…/…/…/lib64 -L/usr/lib/gcc/x86_64-redhat-linux/4.8.5/…/…/…/…/lib64 -lpgf90rtl -lpgf90 -lpgf90_rpm1 -lpgf902 -lpgf90rtl -lpgftnrtl -lpgmp -lnuma -lpthread -lnspgc -lpgc -lrt -lpthread -lm -lgcc -lc -lgcc -lgcc_s /usr/lib/gcc/x86_64-redhat-linux/4.8.5/crtend.o /usr/lib64/crtn.o
Keeping /tmp/pgf90lWfHu6Px6Eg.stb
Keeping ./x.dbg

Dr_TinghaiOu24301 · October 6, 2017, 2:18pm

Dear Dave,

Just for a updating.

I found one error in my siterc when coping your files.
It seems that I successfully installed the netCDF. however, I got the same error when configuring netCDF.

configure: failed program was:
| /* confdefs.h */

I don’t know whether this will lead to some other problem or not.
for example, I got no information about hdf5 and nc4 in the netCDF-FORTRAN lib.

nc-config --all

This netCDF 4.4.1.1 has been built with the following features:

–cc -> pgcc
–cflags -> -I/data/Util/INSTALL_DIR/netCDF/include -DpgiFortran
–libs -> -L/data/Util/INSTALL_DIR/netCDF/lib -lnetcdf

–has-c++ -> no
–cxx ->

–has-c++4 -> no
–cxx4 ->

–has-fortran-> yes
–fc -> pgfortran
–fflags -> -I/data/Util/INSTALL_DIR/netCDF-Fortran/include
–flibs -> -L/data/Util/INSTALL_DIR/netCDF-Fortran/lib -lnetcdff -lnetcdf
–has-f90 -> no
–has-f03 -> yes

–has-dap -> no
–has-nc2 -> yes
–has-nc4 -> yes
–has-hdf5 -> yes
–has-hdf4 -> no
–has-logging-> no
–has-pnetcdf-> no
–has-szlib ->

–prefix -> /data/Util/INSTALL_DIR/netCDF
–includedir-> /data/Util/INSTALL_DIR/netCDF/include
–libdir -> /data/Util/INSTALL_DIR/netCDF/lib
–version -> netCDF 4.4.1.1

nf-config --all

This netCDF-Fortran 4.4.4 has been built with the following features:

–cc -> pgcc
–cflags -> -I/data/Util/INSTALL_DIR/netCDF-Fortran/include -DpgiFortran

–fc -> pgfortran
–fflags -> -I/data/Util/INSTALL_DIR/netCDF-Fortran/include
–flibs -> -L/data/Util/INSTALL_DIR/netCDF-Fortran/lib -lnetcdff -lnetcdf
–has-f90 -> no
–has-f03 -> yes

–has-nc2 -> no
–has-nc4 -> no

–prefix -> /data/Util/INSTALL_DIR/netCDF-Fortran
–includedir-> /data/Util/INSTALL_DIR/netCDF-Fortran/include
–version -> netCDF-Fortran 4.4.4

Have a nice weekend.
Tinghai

tull · October 6, 2017, 6:31pm

looks like you copied the text without the semicolons at the end of lines,
which will fail. This could be my fault because of the UF not copying
everything. I have emailed you the siterc file that works, but here it
is again, with semicolons.

switch -ffast-math is hide;

switch -pipe is hide;

switch -fexpensive-optimizations is hide;


switch -fno-strict-aliasing is hide;

switch -Wall is hide;

switch -pthread is 
	append(LDLIB1= -lpthread);

switch -qversion is
        early
        help(Display compiler version)
        helpgroup(overall)
        set(VERSION=YES);

switch -Wno-deprecated-declarations is hide;

Dr_TinghaiOu24301 · October 8, 2017, 11:11pm

Thank you for the detailed instruction.

I successfully installed HDF5 and netCDF based on your instruction.
However, I found one problem in the installed netCDF-FORTRAN library.
The netCDF4 is not supported in the netCDF-FORTRAN library, which has been successfully included in netCDF-C library.
I can’t find any information like ‘enable netcdf4’ in the configure file of netCDF-Fortran 4.4.4.

Do you have any suggestions related to this?

Detailed information can be found below.

nf-config --all

This netCDF-Fortran 4.4.4 has been built with the following features:

–cc -> pgcc
–cflags -> -I/data/Util/netCDF44/include -DpgiFortran

–fc -> pgfortran
–fflags -> -I/data/Util/netCDF44/include
–flibs -> -L/data/Util/netCDF44/lib -lnetcdff -lnetcdf
–has-f90 -> no
–has-f03 -> yes

–has-nc2 -> no
–has-nc4 -> no

–prefix -> /data/Util/netCDF44
–includedir-> /data/Util/netCDF44/include
–version -> netCDF-Fortran 4.4.4

nc-config --all

This netCDF 4.4.1.1 has been built with the following features:

–cc -> pgcc
–cflags -> -I/data/Util/netCDF44/include -DpgiFortran
–libs -> -L/data/Util/netCDF44/lib -lnetcdf

–has-c++ -> no
–cxx ->

–has-c++4 -> yes
–cxx4 -> pgc++
–cxx4flags -> -I/data/Util/netCDF44/include -I/data/Util/netCDF44/include
–cxx4libs -> -L/data/Util/netCDF44/lib -lnetcdf_c++4 -lnetcdf

–has-fortran-> yes
–fc -> pgfortran
–fflags -> -I/data/Util/netCDF44/include
–flibs -> -L/data/Util/netCDF44/lib -lnetcdff -lnetcdf
–has-f90 -> no
–has-f03 -> yes

–has-dap -> no
–has-nc2 -> yes
–has-nc4 -> yes
–has-hdf5 -> yes
–has-hdf4 -> no
–has-logging-> no
–has-pnetcdf-> no
–has-szlib ->

–prefix -> /data/Util/netCDF44
–includedir-> /data/Util/netCDF44/include
–libdir -> /data/Util/netCDF44/lib
–version -> netCDF 4.4.1.1

Dr_TinghaiOu24301 · October 9, 2017, 9:33am

A following-up.

I added CPPFLAGS and LDFLAGS (as shown below) in the configure and then it seems than netCDF4 is included in the netCDF-FORTRAN library.

CPPFLAGS=-I/data/Util/netCDF44/include LDFLAGS=-L/data/Util/netCDF44/lib
./configure --prefix=/data/Util/netCDF44 >& config.log

http://www.pgroup.com/resources/netcdf/netcdf43_pgi2014.htm
I don’t know why ‘unstenv LDFLAGS’ is included in the above instruction link.

Hope my problem has been resolved as it looks now.
Thank you for your great support, Dave!

nf-config --all

This netCDF-Fortran 4.4.4 has been built with the following features:

–cc -> pgcc
–cflags -> -I/data/Util/netCDF44/include -I/data/Util/netCDF44/include

–fc -> pgfortran
–fflags -> -I/data/Util/netCDF44/include
–flibs -> -L/data/Util/netCDF44/lib -lnetcdff -L/data/Util/netCDF44/lib -lnetcdf -lnetcdf
–has-f90 -> no
–has-f03 -> yes

–has-nc2 -> yes
–has-nc4 -> yes

–prefix -> /data/Util/netCDF44
–includedir-> /data/Util/netCDF44/include
–version -> netCDF-Fortran 4.4.4

tull · October 9, 2017, 7:28pm

I have informed engineering of your comments.

The porting guidelines for HDF5 and NetCDF are 4 years old, and
really need updating.

dave

Dr_TinghaiOu24301 · October 10, 2017, 8:08am

Thank you very much for your help, Dave.

Tinghai

Probelm in building HDF5

switch -Wno-deprecated-declarations is hide;

Below please find some basic information about the compiling. Compiling Options:

Languages:

Features:

Below please find some basic information about the compiling.
Compiling Options: