relocation truncated to fit: R_X86_64_PC32 and -Mrecursive

brockp · October 13, 2009, 9:19pm

We have a user, who is running CAM, we are compiling with openmpi/1.3.2 built with pgi/8.0.
This user increased his problem size such that when he linked we would get errors about: “relocation truncated to fit: R_X86_64_PC32”

Normally we solved this problem by adding -mcmodel=medium at compile and link.

In this case though, the code would compile with the ‘debug’ options on, those options were:

-g -Ktrap=fp -Mrecursive -Mbound

In our case we found that adding -Mrecursive to the optimized flags, allowed the code to compile.

What we want to know is, what did Mrecursive do to effect this (I was surprised it worked), as it was not needed before the problem size was increased.

Any insight into what is going on would be helpful.

MatColgrove · October 14, 2009, 8:40pm

Hi BrockP,

My best guess is that the user didn’t compile their library with “-fpic”. The “R_X86_64_PC32” relocation is used with function calls. So if you are calling a function in static library from within a mcmodel=medium program, the distance to the function is too far. Compiling the libraries with -fpic is the only work-around until the large memory model is supported.

As to why “-Mrecursive” works, again just a guess, but -Mrecursive will place local variables on the heap instead of the stack. So if the user has a very large local static array, by moving it to the heap, the data size may have gotten small enough the -mcmodel=medium is no longer required.

Hope this helps,
Mat

brockp · October 16, 2009, 6:09pm

Thank you for that information, I will pass it onto the user.

Brock Palen

Surya · October 16, 2009, 11:53pm

Hi Mat,

I’ve a similar problem with MM5 MPP run. But I didn’ get an error when I ran with smaller domain.

Configure:

RUNTIME_SYSTEM = "linux"
MPP_TARGET=$(RUNTIME_SYSTEM)
# edit the following definition for your system
LINUX_MPIHOME = /opt/pgi/linux86-64/9.0/mpi/mpich
MFC = $(LINUX_MPIHOME)/bin/mpif90
MCC = $(LINUX_MPIHOME)/bin/mpicc
MLD = $(LINUX_MPIHOME)/bin/mpif90

#FCFLAGS = -O2 -Mcray=pointer -Mnoframe -byteswapio
#LDOPTIONS = -O2 -Mcray=pointer -Mnoframe -byteswapio
#LOCAL_LIBRARIES = -L$(LINUX_MPIHOME)/build/LINUX/ch_p4/lib -lfmpich -lmpich

FCFLAGS = -fast -Mcray=pointer -Mnoframe -byteswapio -DDEC_ALPHA
#FCFLAGS = -fast -Mcray=pointer -fpic -Mnoframe -byteswapio -DDEC_ALPHA  
#LDOPTIONS = -fast -Mcray=pointer -mcmodel=medium -Mnoframe -byteswapio -DDEC_ALPHA
#LDOPTIONS = -fast -Mcray=pointer -fpic -Mnoframe -byteswapio -DDEC_ALPHA
LDOPTIONS = -fast -Mcray=pointer -Mnoframe -byteswapio -DDEC_ALPHA    
LOCAL_LIBRARIES = -L$(LINUX_MPIHOME)/lib -lfmpich -lmpich 

MAKE = make -i -r 
AWK = awk 
SED = sed 
CAT = cat 
CUT = cut 
EXPAND = expand 
M4 = m4 
CPP = /lib/cpp -C -P -traditional 
CPPFLAGS = -DMPI -Dlinux -DSYSTEM_CALL_OK -DDEC_ALPHA 
CFLAGS = -DMPI -I$(LINUX_MPIHOME)/include -DDEC_ALPHA 
ARCH_OBJS = milliclock.o 
IWORDSIZE = 4 
RWORDSIZE = 4 
LWORDSIZE = 4

Error Message:

.......
/opt/pgi/linux86-64/9.0/mpi/mpich/bin/mpicc -c -I../../MPP -I../../MPP/RSL -I../../pick -I../../MPP/debug -I../../MPP/RSL/RSL -DMPP1 -DIOR=2 -DIWORDSIZE=4  -DRWORDSIZE=4  -DLWORDSIZE=4  -DASSUME_HOMOGENEOUS_ENVIRONMENT=1 -DMPI -I/opt/pgi/linux86-64/9.0/mpi/mpich/include -DDEC_ALPHA  milliclock.c
PGC-W-0156-Type not specified, 'int' assumed (milliclock.c: 34)
PGC/x86-64 Linux 9.0-1: compilation completed with warnings
/opt/pgi/linux86-64/9.0/mpi/mpich/bin/mpif90 -o mm5.mpp addall.o addrx1c.o addrx1n.o bdyin.o bdyrst.o bdyten.o bdyval.o cadjmx.o coef_diffu.o condload.o consat.o convad.o couple.o date.o dcpl3d.o dcpl3dwnd.o decouple.o define_comms.o diffu.o diffth.o diffthd.o diffmoi.o diffintp.o dm_io.o dots.o dtfrz.o fillcrs.o fkill_model.o gamma.o gauss.o hadv.o init.o initsav.o initts.o kfbmdata.o kill_model.o lb_alg.o lbdyin.o mhz.o mm5.o mp_equate.o mp_initdomain.o mp_shemi.o mparrcopy.o mpaspect.o nconvp.o nudge.o output.o outsav.o outtap.o outts.o outts_c.o param.o paramr.o rdinit.o rho_mlt.o savread.o settbl.o setvegfr.o sfcrad.o shutdo.o slab.o solar1.o solve.o sound.o subch.o trans.o transm.o upshot_mm5.o vadv.o vadv2.o vecgath.o write_big_header.o write_fieldrec.o write_flag.o                                exmoiss.o                                             cupara2.o cup.o cupara3.o maximi.o minimi.o     	           blkpbl.o hirpbl.o   mrfpbl.o tridi2.o            slab_col.o sfcrad_col.o transm_col.o        initnest.o chknst.o                  nstlev1.o nstlev2.o nstlev3.o nstlev4.o nstlev5.o                  mp_stotndt.o smt2.o bcast_size.o                  merge_size.o mp_feedbk.o                  rdter.o            lwrad.o swrad.o            milliclock.o  ../../MPP/RSL/RSL/librsl.a -fast -Mcray=pointer -Mnoframe -byteswapio -DDEC_ALPHA     -L/opt/pgi/linux86-64/9.0/mpi/mpich/lib -lfmpich -lmpich 
init.o: In function `init_':
/home/mark/Download/MM5V3/MM5_MPP/MM5/MPP/build/./init.f:1351: relocation truncated to fit: R_X86_64_PC32 against symbol `zdiffu_' defined in COMMON section in init.o

I understood from previous post that compiling libraries with “-fpic” is the only work-around. Can you plz elaborate on how to compile the libraries with “-fpic” (Sorry, I’m not a tech savvy). Or is there is any other solution to fix this.

Thanking you in advance

Surya

MatColgrove · October 19, 2009, 9:44pm

Hi Surya,

Shared libraries (.so) are required to be built with position independent code (-fpic) so what you should do is simply add “-Bdynamic” to your link. If you absolutely need to statically link your libraries, the you’ll need to build your own MPICH (See http://www.pgroup.com/resources/mpich/mpich127_pgi90.htm) and add “-fpic” to each of the “FLAG” variables.

Hope this helps.
Mat

Surya · October 20, 2009, 11:23pm

Hi Matt,

Thanks for your suggestion. Now I’m trying to build MPICH with “-fPIC”. While doing so I got stuck with an error message as follow:

** Testing if Fortran77 application can be linked with logging library
/home/mark/Download/mpich-1.2.7p1/bin/mpif77  -I/home/mark/Download/mpich-1.2.7p1/include  -c fpi.f
/home/mark/Download/mpich-1.2.7p1/bin/mpif77 -O2 -fPIC  -o fpilog fpi.o -L/home/mark/Download/mpich-1.2.7p1/lib -lfmpich -llmpe -lmpe   -lm
** Fortran77 application can be linked with logging library

** Testing if Fortran77 application can be linked with graphics library
/home/mark/Download/mpich-1.2.7p1/bin/mpif77  -I/home/mark/Download/mpich-1.2.7p1/include  -c fxgraphics.f
/home/mark/Download/mpich-1.2.7p1/bin/mpif77 -O2 -fPIC  -o fxgraphics fxgraphics.o -L/home/mark/Download/mpich-1.2.7p1/lib -lmpe -lX11   -lm
** Fortran77 application can be linked with graphics library

Copying SLOG2SDK's lib
Copying SLOG2SDK's doc
Copying SLOG2SDK's logfiles
Creating SLOG2SDK's bin
Installed SLOG2SDK in /home/mark/Download/mpich-1.2.7p1
/home/mark/Download/mpich-1.2.7p1/sbin/mpeuninstall may be used to remove the installation
Installed MPE in /home/mark/Download/mpich-1.2.7p1
/home/mark/Download/mpich-1.2.7p1/sbin/mpeuninstall may be used to remove the installation
Finished making MPE Profiling Libraries

make --no-print-directory mpi-utils
pgcc -O2 -Msignextend -V -fPIC -DUSE_SOCKLEN_T -DUSE_U_INT_FOR_XDR -c /home/mark/Download/mpich-1.2.7p1/mpid/server/serv_p4.c -I./ -o serv_p4.o

pgcc 9.0-4 64-bit target on x86-64 Linux -tp penryn-64 
Copyright 1989-2000, The Portland Group, Inc.  All Rights Reserved.
Copyright 2000-2009, STMicroelectronics, Inc.  All Rights Reserved.
PGC/x86-64 Linux 9.0-4
Copyright 1989-2000, The Portland Group, Inc.  All Rights Reserved.
Copyright 2000-2009, STMicroelectronics, Inc.  All Rights Reserved.
PGC-S-0137-Incompatible prototype declaration for function getline (/home/mark/Download/mpich-1.2.7p1/mpid/server/serv_p4.c: 204)
PGC/x86-64 Linux 9.0-4: compilation completed with severe errors
make[2]: *** [serv_p4.o] Error 2
bin/mpicc -o /home/mark/Download/mpich-1.2.7p1/bin/mpichversion /home/mark/Download/mpich-1.2.7p1/util/mpichversion.c
rm -f /home/mark/Download/mpich-1.2.7p1/bin/mpireconfig.dat
rm -f /home/mark/Download/mpich-1.2.7p1/bin/mpireconfig.dat
creating util/mympiinstall

In addition, I’m attaching my enviroment settings

#PATH=$PATH:$HOME/bin:/usr/local/bin:$NCARG_ROOT/bin:$PGI/linux86-64/9.0/mpi/mpich/bin:/opt/pgi/linux86-64/9.0/bin
PATH=$PATH:$HOME/bin:/usr/local/bin:$NCARG_ROOT/bin:$PGI/linux86-64/bin:/opt/pgi/linux86-64/9.0/bin
#:$PGI/linux86-64/9.0/mpi/mpich/bin:/opt/pgi/linux86-64/9.0/bin
export PATH 

#MANPATH=$MANPATH:/opt/pgi/linux86-64/9.0/man:$PGI/linux86-64/9.0/mpi/mpich/man
MANPATH=$MANPATH:/opt/pgi/linux86-64/9.0/man:$PGI/linux86-64/man
export MANPATH

LM_LICENSE_FILE=$LM_LICENSE_FILE:/opt/pgi/license.dat
export LM_LICENSE_FILE

export CC=pgcc
export CFLAGS="-O2 -Msignextend -V -fPIC"
export CPPFLAGS="-DNDEBUG -DpgiFortran -fPIC"
export CXX=pgcpp
export CXXFLAGS="-O2 -g -fPIC"
export F90=pgfortran
export FC=pgfortran
export F77=pgfortran
export FFLAGS="-O2 -w -V -fPIC"
export F90FLAGS="-O2 -fPIC"
export LDFLAGS="-O2 -fPIC"
export OPTFLAGS="-O2 -fPIC"
export CPP="pgCC -E"

Hoping there is a fix for this ! Waiting for your response.

Thanks

Surya

MatColgrove · October 21, 2009, 4:21pm

Hi Surya,

The error “Incompatible prototype declaration for function getline” means that some how the getline function is already defined with a different, incompatible, prototype. To determine where this second prototype is coming from, compile with “-P” so only the pre-processing step is performed, and search the resulting “.i” file for instances of “getline”.

What OS are you using?

Mat