compile error with pgf90 with mpi

Hi All,

I am getting the following error when I compile my code using pgf90 (mpif90) (PGI ver. 6.0) with mpich ver 1.2.6. This error shows up only when I increase my domain size beyond a certain point. I tried using -mcmodel=medium as a compile option with mpif90, but this did not solve the problem. I will shortly write to the mpich forum/contact, but I thought I’ll just share it with the PGI group to see if anyone has come across such an error and what may be causing it.

Any pointers/workaround/suggestions highly appreciated!
-Madhu


/opt/mpich.pgi-1.2.6/lib/libmpich.a(isend.o)(.text+0x21b): In function PMPI_Isend': : relocation truncated to fit: R_X86_64_PC32 MPIR_shandles /opt/mpich.pgi-1.2.6/lib/libmpich.a(irecv.o)(.text+0x21c): In function PMPI_Irecv’:
: relocation truncated to fit: R_X86_64_PC32 MPIR_rhandles
/opt/mpich.pgi-1.2.6/lib/libmpich.a(mperror.o)(.text+0x125): In function MPIR_Errors_are_fatal': : relocation truncated to fit: R_X86_64_PC32 MPID_MyWorldRank /opt/mpich.pgi-1.2.6/lib/libmpich.a(mperror.o)(.text+0x25c): In function MPIR_Errors_warn’:
: relocation truncated to fit: R_X86_64_PC32 MPID_MyWorldRank
/opt/mpich.pgi-1.2.6/lib/libmpich.a(commreq_free.o)(.text+0xd7): In function `PMPI_Request_free’:
: relocation truncated to fit: R_X86_64_PC32 MPIR_shandles
/opt/mpich.pgi-1.2.6/lib/libmp

Hi Madhu,

You might want to check with the MPICH experts but I don’t beileve MPICH-1 allows for large data objects. In the past, I’ve worked around this limit by breaking-up my large matrix and sending them in pieces. Hopefully others have better solutions.

  • Mat

I see the same problems compiling my code with pgf90, but with no MPICH stuff involved. The arrays seem to be limited to around 100MB, which doesn’t get me very far on a 32GB machine.

pgf90 -w -Mlfs -Bstatic -mcmodel=medium -s -fast -tp p7-64 -o runfmm_pgf90_xeon runfmm.o setup.o legendre.o direct.o fmm1.o buildtree.o octree.o mod_octree.o boxcoev.o boxcoes.o interaction.o search.o multipole.o fmm2v.o fmm2s.o influence.o kernel.o diffuse.o vrm.o cottet.o input.o matrix.o quadrature.o output.o dynamic.o lump_vortons.o split_vortons.o vary_core_size.o streamline.o timer.o
/opt/pgi/linux86-64/6.1/libso/libpgf90.a(initpar.o)(.text+0xf2): In function __hpf_initarg': : relocation truncated to fit: R_X86_64_PC32 .bss /opt/pgi/linux86-64/6.1/libso/libpgf90.a(initpar.o)(.text+0x121): In function __hpf_initarg’:
: relocation truncated to fit: R_X86_64_PC32 .bss
…more of the same…

…addition:

I can’t make a static executable, at least. It builds without “-Bstatic”. Which, still, doesn’t help me much.

Hi mstock,

Objects compiled with “-mcmodel=medium” can not be statically linked. This is a Linux limitation rather than PGI limitation.

Instead, try compiling with just “-Mlarge_arrays”, which can be statically linked. While you’ll be using the small memory model, arrays larger than 2GB can be still used so long as they are dynamically allocated. Also, make sure that your array index variables are declared as INTEGER*8.

Hope this helps,
Mat

Unfortunately, that doesn’t help. The code is in F77, so dynamic allocation is out of the question. And the problem doesn’t seem to be with any one array, but the total of all allocated arrays. When that total exceeds about 2GB, I get the errors.