Problem in compiling WRFV2.1.1

Accelerated Computing HPC Compilers Legacy PGI Compilers
1

Hi all,
I’m getting “undefined reference to” errors when i compile the WRFV2.1.1 from UCAR with PGI compiler 6.1. I have successfully compile netcdf with PGI compiler and i’m using a machine with Redhat 9 and the following info:
Linux 2.4.21-4.ELsmp #1 SMP Fri Oct 3 17:31:21 EDT 2003 i686 athlon i386 GNU/Linux.
Below is the errors i got after issue ./compile em_real

ranlib libwrflib.a
pgf90 -o real.exe -fast -w -byteswapio -Mfree -tp p6 -byteswapio real_em.o …/dyn_em/module_initialize_real.o libwrflib.a -L…/external/io_netcdf -lwrfio_nf -L/tmp/final/lib -lnetcdf -L…/external/io_grib1 -lio_grib1 …/frame/module_internal_header_util.o …/frame/pack_utils.o -L…/external/esmf_time_f90 -lesmf_time
/tmp/final/lib/libnetcdf.a(ncx.o)(.text+0xb14): In function ncx_get_float_uchar': : **undefined reference to __mth_i_fixux’**
/tmp/final/lib/libnetcdf.a(ncx.o)(.text+0xd23): In function ncx_put_float_uchar': : **undefined reference to __mth_i_floatux’**
/tmp/final/lib/libnetcdf.a(ncx.o)(.text+0xfc4): In function ncx_get_double_uchar': : undefined reference to __mth_i_dfixux’
/tmp/final/lib/libnetcdf.a(ncx.o)(.text+0x1203): In function ncx_put_double_uchar': : undefined reference to __mth_i_dfloatux’
make[1]: [em_real] Error 2 (ignored)
pgf90 -o ndown.exe -fast -w -byteswapio -Mfree -tp p6 -byteswapio ndown_em.o …/dyn_em/module_initialize_real.o libwrflib.a -L…/external/io_netcdf -lwrfio_nf -L/tmp/final/lib -lnetcdf -L…/external/io_grib1 -lio_grib1 …/frame/module_internal_header_util.o …/frame/pack_utils.o -L…/external/esmf_time_f90 -lesmf_time
/tmp/final/lib/libnetcdf.a(ncx.o)(.text+0xb14): In function ncx_get_float_uchar': : undefined reference to __mth_i_fixux’
/tmp/final/lib/libnetcdf.a(ncx.o)(.text+0xd23): In function ncx_put_float_uchar': : undefined reference to __mth_i_floatux’
/tmp/final/lib/libnetcdf.a(ncx.o)(.text+0xfc4): In function ncx_get_double_uchar': : undefined reference to __mth_i_dfixux’
/tmp/final/lib/libnetcdf.a(ncx.o)(.text+0x1203): In function ncx_put_double_uchar': : undefined reference to __mth_i_dfloatux’
make[1]: [em_real] Error 2 (ignored)
make[1]: Leaving directory `/tmp/WRFV2/main’
( cd test/em_real ; /bin/rm -f wrf.exe ; ln -s …/…/main/wrf.exe . )
if [ 0 -eq 1 ] ; then
( cd test/em_real ; /bin/rm -f wrf_ESMFApp.exe ; ln -s …/…/main/wrf_ESMFApp.exe . ) ;
fi
( cd test/em_real ; /bin/rm -f real.exe ; ln -s …/…/main/real.exe . )
( cd test/em_real ; /bin/rm -f ndown.exe ; ln -s …/…/main/ndown.exe . )
( cd test/em_real ; /bin/rm -f README.namelist ; ln -s …/…/run/README.namelist . )
( cd test/em_real ; /bin/rm -f ETAMPNEW_DATA RRTM_DATA ;
ln -sf …/…/run/ETAMPNEW_DATA . ;
ln -sf …/…/run/RRTM_DATA . ;
if [ 4 -eq 8 ] ; then
ln -sf …/…/run/ETAMPNEW_DATA_DBL ETAMPNEW_DATA ;
ln -sf …/…/run/RRTM_DATA_DBL RRTM_DATA ;
fi )
( cd test/em_real ; /bin/rm -f GENPARM.TBL ; ln -s …/…/run/GENPARM.TBL . )
( cd test/em_real ; /bin/rm -f LANDUSE.TBL ; ln -s …/…/run/LANDUSE.TBL . )
( cd test/em_real ; /bin/rm -f SOILPARM.TBL ; ln -s …/…/run/SOILPARM.TBL . )
( cd test/em_real ; /bin/rm -f VEGPARM.TBL ; ln -s …/…/run/VEGPARM.TBL . )
( cd test/em_real ; /bin/rm -f tr49t67 ; ln -s …/…/run/tr49t67 . )
( cd test/em_real ; /bin/rm -f tr49t85 ; ln -s …/…/run/tr49t85 . )
( cd test/em_real ; /bin/rm -f tr67t85 ; ln -s …/…/run/tr67t85 . )
( cd test/em_real ; /bin/rm -f gribmap.txt ; ln -s …/…/run/gribmap.txt . )
( cd run ; /bin/rm -f real.exe ; ln -s …/main/real.exe . )
( cd run ; /bin/rm -f ndown.exe ; ln -s …/main/ndown.exe . )
( cd run ; /bin/rm -f namelist.input ; ln -s …/test/em_real/namelist.input . )
-------------------------------------------END------------------------


I have also put my configure.wrf file just incase some one want to have a look at it

------------------configure.wrf------------------------------------------

Architecture specific settings

Settings for PC Linux i486 i586 i686, PGI compiler (Single-threaded, no nesting)

FC = pgf90
LD = pgf90
CC = gcc
SCC = $(CC)
RWORDSIZE = $(NATIVE_RWORDSIZE)

\

Architecture specific settings

Settings for PC Linux i486 i586 i686, PGI compiler (Single-threaded, no nesting)

FC = pgf90
LD = pgf90
CC = gcc
SCC = $(CC)
RWORDSIZE = $(NATIVE_RWORDSIZE)
SFC = $(FC)
PROMOTION = -r$(RWORDSIZE) -i4
CFLAGS =
FCOPTIM = -fast
FCDEBUG = #-g
#FCBASEOPTS = -w -byteswapio -Ktrap=fp -Mfree -tp p6 $(FCDEBUG)
FCBASEOPTS = -w -byteswapio -Mfree -tp p6 $(FCDEBUG) # -Mlfs
FCFLAGS = $(FCOPTIM) $(FCBASEOPTS)
ARCHFLAGS = -DDEREF_KLUDGE -DIO_DEREF_KLUDGE -DIWORDSIZE=4 -DDWORDSIZE=8 -DRWORDSIZE=$(RWORDSIZE) -DLWORDSIZE=4
-DNETCDF -DGRIB1
-DTRIEDNTRUE
-DLIMIT_ARGS
INCLUDE_MODULES = -module …/main -I…/external/io_netcdf -I…/external/io_int -I…/external/esmf_time_f90
-I…/frame -I…/share -I…/phys -I…/chem -I…/inc
PERL = perl
REGISTRY = Registry
LIB = -L…/external/io_netcdf -lwrfio_nf -L/tmp/final/lib -lnetcdf
-L…/external/io_grib1 -lio_grib1
…/frame/module_internal_header_util.o …/frame/pack_utils.o -L…/external/esmf_time_f90 -lesmf_time
LDFLAGS = $(FCFLAGS) -byteswapio
ENVCOMPDEFS =
WRF_CHEM = 0
CPP = /lib/cpp -C -P -traditional
POUND_DEF = $(COREDEFS) -DNONSTANDARD_SYSTEM -DF90_STANDALONE -DCONFIG_BUF_LEN=$(CONFIG_BUF_LEN) -DMAX_DOMAINS_F=$(MAX_DOMAINS)
CPPFLAGS = -I$(LIBINCLUDE) -C -P $(ARCHFLAGS) $(ENVCOMPDEFS) $(POUND_DEF)
AR = ar ru
M4 = m4
RANLIB = ranlib
NETCDFPATH = /tmp/final
CC_TOOLS = $(CC)

externals : wrf_ioapi_includes wrfio_nf wrfio_grib1 wrfio_int module_dm.F esmf_time

module_dm.F :
( /bin/cp module_dm_warning module_dm.F ; cat module_dm_stubs.F >> module_dm.F )

wrfio_nf :
( cd …/external/io_netcdf ;
make NETCDFPATH=/tmp/final RANLIB=“$(RANLIB)” CPP=“$(CPP)” FC=“$(FC) $(PROMOTION) $(FCFLAGS) -w” TRADFLAG=“-traditional” )

wrfio_grib1 :
( cd …/external/io_grib1 ; \ make CC=“$(SCC)” CFLAGS=“$(CFLAGS)” RANLIB=“$(RANLIB)” CPP=“$(CPP)” FC=“$(SFC) $(PROMOTION) -I. $(FCDEBUG) $(FCBASEOPTS) -w” TRADFLAG=“-traditional” )

wrfio_int : ( cd …/external/io_int ; \ make CC=“$(CC)” RANLIB=“$(RANLIB)” CPP=“$(CPP)” SFC=“$(SFC) $(PROMOTION) $(FCDEBUG) $(FCBASEOPTS)” FC=“$(SFC) $(PROMOTION) $(FCDEBUG) $(FCBASEOPTS) -w”
TRADFLAG=“-traditional” all )

esmf_time :
( cd …/external/esmf_time_f90 ; \ make FC=“$(FC) $(PROMOTION) $(FCDEBUG) $(FCBASEOPTS)” RANLIB=“$(RANLIB)” CPP=“$(CPP) -I…/…/inc -I. $(POUND_DEF)” )

compile these without high optimization to speed compilesolve_interface.o : solve_interface.F

shift_domain_em.o : shift_domain_em.F
module_io_mm5.o : module_io_mm5.F
module_si_io.o : module_si_io.F
module_io_wrf.o : module_io_wrf.Fmodule_domain.o : module_domain.F
module_start.o : module_start.F
module_initialize.o : module_initialize.F
module_initialize_b_wave.o : module_initialize_b_wave.F
module_initialize_hill2d_x.o : module_initialize_hill2d_x.Fmodule_initialize_quarter_ss.o : module_initialize_quarter_ss.F
module_initialize_squall2d_x.o : module_initialize_squall2d_x.F
module_initialize_squall2d_y.o : module_initialize_squall2d_y.F
module_initialize_real.o : module_initialize_real.F
module_dm.o : module_dm.Fstart_domain.o : start_domain.Fmediation_integrate.o : mediation_integrate.F
module_configure.o : module_configure.F

solve_interface.o
shift_domain_em.o
module_io_mm5.o module_si_io.o module_io_wrf.o module_domain.o \module_start.o module_initialize.o module_initialize_b_wave.o
module_initialize_hill2d_x.o module_initialize_quarter_ss.o
module_initialize_squall2d_x.o module_initialize_squall2d_y.o
mediation_integrate.o
module_configure.o
module_initialize_real.o module_dm.o start_domain.o :
$(RM) $@
$(SED_FTN) $.F > $.b
$(CPP) -I…/inc $(CPPFLAGS) $.b > $.f
$(RM) $.b
$(FC) -c $(PROMOTION) $(FCBASEOPTS) $(MODULE_DIRS) $.f

if [ ! -e $@ ] ; then \

sleep 10 ; $(FC) -c $(PROMOTION) $(FCBASEOPTS) $(MODULE_DIRS) $*.f ; \

fi

if [ ! -e $@ ] ; then \

sleep 30 ; $(FC) -c $(PROMOTION) $(FCBASEOPTS) $(MODULE_DIRS) $*.f ; \

fi

if [ ! -e $@ ] ; then \

sleep 300 ; $(FC) -c $(PROMOTION) $(FCBASEOPTS) $(MODULE_DIRS) $*.f ; \

fi

###########################################################

Macros, these should be generic for all machines

LN = ln -sf
MAKE = make -i -r
RM = rm -f

There is probably no reason to modify these rules

wrf_ioapi_includes :
( cd …/external/ioapi_share ; \ $(MAKE) NATIVE_RWORDSIZE=“$(NATIVE_RWORDSIZE)” RWORDSIZE=“$(RWORDSIZE)” )

.F.i:
$(RM) $@
$(CPP) -I…/inc $(CPPFLAGS) $.F > $@
mv $.i $(DEVTOP)/pick/$.f
cp $.F $(DEVTOP)/pick

.F.o:
$(RM) $@
$(SED_FTN) $.F > $.b
$(CPP) -I…/inc $(CPPFLAGS) $.b > $.f
$(RM) $.b
$(FC) -c $(FCFLAGS) $(MODULE_DIRS) $(PROMOTION) $.f

if [ ! -e $@ ] ; then \

sleep 10 ; $(FC) -c $(FCFLAGS) $(MODULE_DIRS) $(PROMOTION) $*.f ; \

fi

if [ ! -e $@ ] ; then \

sleep 10 ; $(FC) -c $(FCFLAGS) $(MODULE_DIRS) $(PROMOTION) $*.f ; \

fi

.F.f:
$(RM) $@
$(SED_FTN) $.F > $.b
$(CPP) -I…/inc $(CPPFLAGS) $.b > $@ $(RM) $.b

.f.o:
$(RM) $@
$(FC) -c $(FCFLAGS) $(PROMOTION) $*.f

if [ ! -e $@ ] ; then \

sleep 10 ; $(FC) -c $(FCFLAGS) $(PROMOTION) $*.f ; \

fi

if [ ! -e $@ ] ; then \

sleep 10 ; $(FC) -c $(FCFLAGS) $(PROMOTION) $*.f ; \

fi

.c.o:
$(RM) $@
$(CC) -c $(CFLAGS) $*.c

--------------END configure.wrf--------------------------------------

Thanks,
peterr0

2

Hi Petero,

The undefined references are for the SSE2 math functions. To fix, either compile NetCDF with “-tp p6” or WRF without it.

We’ve changed the default for 32-bits to use SSE2 floating point instructions on systems that support them. Previously the default was to use x87 and only use SSE2 if the flags “-Mscalarsse” or “-fastsse” were used. “-tp p6” is the architecture flag for a Pentium III which does not support SSE2.

Hope this helps,
Mat

3

Hi Matt
Thanks for the reply,
I still could not compile the wrf. I edited the configure.wrf file as you suggested and issue
./compile em_real
And got the warning"alignment 8 of symbol `module_configure_9_’ in libwrflib.a(module_configure.o) is smaller than 16 in real_em_ipa8_real.exe.oo"
Any suggestion?
Petero

See below:

ranlib libwrflib.a
pgf90 -o real.exe -fast -Mscalarsse -Mcache_align -w -byteswapio -Mfree -Mipa=fast,inline,safe -byteswapio real_em.o …/dyn_em/module_initialize_real.o libwrflib.a -L…/external/io_netcdf -lwrfio_nf -L/tmp/final/lib -lnetcdf -L…/external/io_grib1 -lio_grib1 …/frame/module_internal_header_util.o …/frame/pack_utils.o -L…/external/esmf_time_f90 -lesmf_time
IPA: no IPA optimizations for 1 source files
IPA: Recompiling real_em.o: new IPA information
IPA: Recompiling …/dyn_em/module_initialize_real.o: new IPA information

/usr/bin/ld: Warning: alignment 8 of symbol module_configure_9_' in libwrflib.a(module_configure.o) is smaller than 16 in real_em_ipa8_real.exe.oo /usr/bin/ld: Warning: alignment 8 of symbol module_configure_9_’ in libwrflib.a(module_configure.o) is smaller than 16 in libwrflib.a(module_configure.o)
/usr/bin/ld: Warning: alignment 8 of symbol module_configure_9_' in libwrflib.a(module_configure.o) is smaller than 16 in libwrflib.a(module_configure.o) /usr/bin/ld: Warning: alignment 8 of symbol module_configure_9_’ in libwrflib.a(module_configure.o) is smaller than 16 in libwrflib.a(module_configure.o)
libwrflib.a(module_configure.o) is smaller than 16 in libwrflib.a(module_configure.o)
/usr/bin/ld: Warning: alignment 8 of symbol module_configure_9_' in libwrflib.a(module_configure.o) is smaller than 16 in libwrflib.a(module_configure.o) /usr/bin/ld: Warning: alignment 8 of symbol esmf_stubs_12_’ in …/external/esmf_time_f90/libesmf_time.a(ESMF_Stubs.o) is smaller than 16 in real_em_ipa8_real.exe.oo
pgf90 -o ndown.exe -fast -Mscalarsse -Mcache_align -w -byteswapio -Mfree -Mipa=fast,inline,safe -byteswapio ndown_em.o …/dyn_em/module_initialize_real.o libwrflib.a -L…/external/io_netcdf -lwrfio_nf -L/tmp/final/lib -lnetcdf -L…/external/io_grib1 -lio_grib1 …/frame/module_internal_header_util.o …/frame/pack_utils.o -L…/external/esmf_time_f90 -lesmf_time
IPA: no IPA optimizations for 1 source files
IPA: Recompiling ndown_em.o: new IPA information

IPA: Recompiling …/dyn_em/module_initialize_real.o: new IPA information

/usr/bin/ld: Warning: alignment 8 of symbol module_configure_9_' in libwrflib.a(module_configure.o) is smaller than 16 in ndown_em_ipa9_ndown.exe.oo /usr/bin/ld: Warning: alignment 8 of symbol module_configure_9_’ in libwrflib.a(module_configure.o) is smaller than 16 in libwrflib.a(module_configure.o)
/usr/bin/ld: Warning: alignment 8 of symbol module_configure_9_' in libwrflib.a(module_configure.o) is smaller than 16 in libwrflib.a(module_configure.o) /usr/bin/ld: Warning: alignment 8 of symbol module_configure_9_’ in libwrflib.a(module_configure.o) is smaller than 16 in libwrflib.a(module_configure.o)
/usr/bin/ld: Warning: alignment 8 of symbol esmf_stubs_12_' in ../external/esmf_time_f90/libesmf_time.a(ESMF_Stubs.o) is smaller than 16 in ndown_em_ipa9_ndown.exe.oo make[1]: Leaving directory /tmp/WRFV2/main’
( cd test/em_real ; /bin/rm -f wrf.exe ; ln -s …/…/main/wrf.exe . )
if [ 0 -eq 1 ] ; then
( cd test/em_real ; /bin/rm -f wrf_ESMFApp.exe ; ln -s …/…/main/wrf_ESMFApp.exe . ) ;
fi
( cd test/em_real ; /bin/rm -f real.exe ; ln -s …/…/main/real.exe . )
( cd test/em_real ; /bin/rm -f ndown.exe ; ln -s …/…/main/ndown.exe . )
( cd test/em_real ; /bin/rm -f README.namelist ; ln -s …/…/run/README.namelist . )
( cd test/em_real ; /bin/rm -f ETAMPNEW_DATA RRTM_DATA ;
ln -sf …/…/run/ETAMPNEW_DATA . ;
ln -sf …/…/run/RRTM_DATA . ;
if [ 4 -eq 8 ] ; then
ln -sf …/…/run/ETAMPNEW_DATA_DBL ETAMPNEW_DATA ;
ln -sf …/…/run/RRTM_DATA_DBL RRTM_DATA ;
fi )
( cd test/em_real ; /bin/rm -f GENPARM.TBL ; ln -s …/…/run/GENPARM.TBL . )
( cd test/em_real ; /bin/rm -f LANDUSE.TBL ; ln -s …/…/run/LANDUSE.TBL . )
( cd test/em_real ; /bin/rm -f SOILPARM.TBL ; ln -s …/…/run/SOILPARM.TBL . )
( cd test/em_real ; /bin/rm -f VEGPARM.TBL ; ln -s …/…/run/VEGPARM.TBL . )
( cd test/em_real ; /bin/rm -f tr49t67 ; ln -s …/…/run/tr49t67 . )
( cd test/em_real ; /bin/rm -f tr49t85 ; ln -s …/…/run/tr49t85 . )
( cd test/em_real ; /bin/rm -f tr67t85 ; ln -s …/…/run/tr67t85 . )
( cd test/em_real ; /bin/rm -f gribmap.txt ; ln -s …/…/run/gribmap.txt . )
( cd run ; /bin/rm -f real.exe ; ln -s …/main/real.exe . )
( cd run ; /bin/rm -f ndown.exe ; ln -s …/main/ndown.exe . )
( cd run ; /bin/rm -f namelist.input ; ln -s …/test/em_real/namelist.input . )
[pettl@xxxx1 WRFV2]#

4

Hi Petero,

I think this is because you added “-Mcache_align” which aligns data on 16 byte boundaries. This can aid performance but you need to make sure that all your objects, including libraries, were compiled with it as well. Otherwise, you get inconsistant alignments. Try recompiling without “-Mcache_align” or move the flag from FCOPTIM in configure.wrf to FCBASEOPTS (I’m assuming this is where you added it). Not all WRF objects get compiled with th FCOPTIM options, but all use FCBASEOPTS.

Although very little has changed, I updated our WRF tips and techniques page today to include WRF 2.1 and PGI 6.1.

Hope this helps,
Mat

Note that in 64-bits “-Mcache_align” is on by default.

5

Matt,
I have compile the WRFV2 successfully.
thanks,
Petero.