Hi,
As I haven’t tried this before, I would like to ask if anyone has tried using icc, icpc and pgfortran to compile for his/her application.
If yes, how do you go about doing that using cmake?
Regards,
Chee Choung
Hi Chee Choung,
I would like to ask if anyone has tried using icc, icpc and pgfortran to compile for his/her application.
Yes, and it works fine. You’ll have the normal inter-operable language issues C/C++ mixing with Fortran, and adding the appropriate runtime libraries at link, but those issues are not compiler specific.
If you are mixing an OpenMP applications, please use PGI’s LLVM back-end compilers (i.e. add -Mllvm to your PGI compilation). Our LLVM back-end uses the runtime that’s compatible with Intel’s KMPC runtime, while the native back-end is not compatible.
If yes, how do you go about doing that using cmake?
I don’t know cmkae well enough to say specifically, but would assume that you can set the “FC=pgfortran” or “F90=pgfortran” in you environment to have cmake pick-up the correct compiler. Depending on how you link your application, you’ll need to add the compiler runtime libraries to your link. If you’re linking using icc, you’d need to add PGI’s runtime libraries, or if you’re using pgfortran to link, add Intel’s.
To get the correct PGI libraries to add, you can run pgfortran with the “-dryrun” command which will show you the steps, including the link, the compiler would issue, but wont execute them. Be sure to add the PGI compiler flags that you’re using to compile since these flags will effect which libraries are linked.
For example:
% pgfortran -dryrun -fast x.o -V18.10
... cut ...
/usr/bin/ld /usr/lib64/crt1.o /usr/lib64/crti.o /proj/pgi/linux86-64/18.10/lib/trace_init.o /crtbegin.o /proj/pgi/linux86-64/18.10/lib/initmp.o /proj/pgi/linux86-64/18.10/lib/f90main.o --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib/ld-linux.so.2 /proj/pgi/linux86-64/18.10/lib/pgi.ld -L/proj/pgi/linux86-64/18.10/lib -L/usr/lib64 x.o -rpath /proj/pgi/linux86-64/18.10/lib -rpath /../../../../lib64 -L/../../../../lib64 -lpgf90rtl -lpgf90 -lpgf90_rpm1 -lpgf902 -lpgf90rtl -lpgftnrtl -lpgmp -lnuma -lpthread --start-group -lpgmath -lnspgc -lpgc --end-group -lrt -lpthread -lm -lgcc -lc -lgcc /crtend.o /usr/lib64/crtn.o
From this, you’d add “-L/proj/pgi/linux86-64/18.10/lib -lpgf90rtl -lpgf90 -lpgf90_rpm1 -lpgf902 -lpgf90rtl -lpgftnrtl -lpgmp -lnuma -lpthread -Wl,–start-group -lpgmath -lnspgc -lpgc -Wl,–end-group” to you’re link.
Note that if you’re using OpenACC, I would suggest using PGI to link since there’s an additional device object link step that is performed. If you can’t use PGI to link, compile you’re OpenACC code with “-ta=tesla:nordc”. RDC stands for relocatable device code which requires the device link step. With “nordc” you can skip the device link but then can’t make calls to device routines from device code or access global device data.
Hope this helps,
Mat
Hi Mat,
Thank you for your comprehensive reply.
To add on my challenges in compiling, for both icc and pgfortran, I am using MPI.
For icc, I am using mpicc from mpich ( which is compiled by icc).
For pgfortran, I am using pgfortran with -Mmpi switch.
Currently, I am linking toward mpicc. pgfortran displays error like "undefined reference to ‘mpi_*’ ".
My mpich directory is in “/opt/mpich” while my pgi directory is in “/opt/pgi”.
How do I link pgfortran mpi library to intel compiler ?
Regards,
Chee Choung
I believe you could do the same with MPI as Mat described with OpenMP. Specify the MPIDIR as the base path to the mpich directory and use -dryrun to see which flags you should add to the link line.
$ MPIDIR=/path/to/mpich/ pgfortran -Mmpi=mpich -dryrun x.o
...
/usr/bin/ld /usr/lib/x86_64-linux-gnu/crt1.o /usr/lib/x86_64-linux-gnu/crti.o /proj/pgi/linux86-64/18.10/lib/trace_init.o /usr/lib/gcc/x86_64-linux-gnu/5/crtbegin.o /proj/pgi/linux86-64/18.10/lib/initmp.o /proj/pgi/linux86-64/18.10/lib/f90main.o --eh-frame-hdr -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 /proj/pgi/linux86-64/18.10/lib/pgi.ld -L/proj/pgi/linux86-64/18.10/lib -L/usr/lib64 -L/usr/lib/gcc/x86_64-linux-gnu/5 x.o -rpath /proj/pgi/linux86-64/20XX-dev/mpi/mpich-3.2.1//lib -rpath /proj/pgi/linux86-64/18.10/lib -rpath /usr/lib/gcc/x86_64-linux-gnu/5/../../../../lib64 -L/usr/lib/gcc/x86_64-linux-gnu/5/../../../../lib64 -lmpich -lopa -lmpl -lpthread -L/proj/pgi/linux86-64/20XX-dev/mpi/mpich-3.2.1//lib -lfmpich -lmpichf90 -lpgf90rtl -lpgf90 -lpgf90_rpm1 -lpgf902 -lpgf90rtl -lpgftnrtl -lpgmp -lnuma -lpthread --start-group -lpgmath -lnspgc -lpgc --end-group -lrt -lpthread -lm -lgcc -lc -lgcc -lgcc_s /usr/lib/gcc/x86_64-linux-gnu/5/crtend.o /usr/lib/x86_64-linux-gnu/crtn.o
Should be something like
-L/proj/pgi/linux86-64/20XX-dev/mpi/mpich-3.2.1//lib -lfmpich -lmpichf90 -lpgf90rtl -lpgf90 -lpgf90_rpm1 -lpgf902 -lpgf90rtl -lpgftnrtl -lpgmp -lnuma -lpthread
Hi Mat,
I would like to compile OpenMP pgfortran mixing with intel icc and icpc. The “void main (void)” is in icpc.
I have followed your instruction to use -Mllvm for pgfortran compiling options. And also link the flags as described earlier on.
I used:
“pgfortran -mp=allcores -O3 -Mlarge_arrays -llapack -lblas -Mcuda -ta=tesla:cc60,nordc”
to compile my code.
I can successfully compile and execute the code but single thread. What am I missing here?
When you linked, did you use the PGI LLVM runtime libraries, i.e. the one under “$PGI/linux86-64-llvm/18.10/lib”?
Did you include the Intel OpenMP compilation flag (-qopenmp) for the Intel compilation and link?
If not, the Inel KMP OpenMP runtime wont be present and/or initialize the PGI portions of the runtime so the code wont go parallel.
If so, then I’m not sure. I’d ask that you give more details on the exact step you used to build and if possible, examples.
Note, that I have not tried mixing different MPI builds and am not sure how they will work together. Maybe if they are the same MPICH versions it will work, but I would recommend that you instead build your own MPICH using icpc, icc, and pgfortran. I’ve not done this either so don’t know if you’ll encounter issues.
-Mat