My code is parallelised using OpenMP. Even if I make sure I have enabled OpenMP in the compiler options, and the compile-time diagnostics show me that the code is parallelised, and I ensure that there is more than one core available at run-time, my run-time diagnostics reveal that the code is still only running on one core.
Is there anything else I need to do to make sure that OpenMP is fully enabled?
Hi Eos Pengwern,
How are you setting the number of OpenMP threads?
This can be done either via the shell’s environment variable OMP_NUM_THREADS or in your program via a call to “omp_set_num_threads”.
By default, if you do not set the number of threads, only one will be used. Though, if you the flag “-mp=allcores”, the default is to use all available cores.
Hope this helps,
Mat
Thanks Mat!
I think I see the problem; I haven’t explicitly set the number of threads, since I’m used to Intel Fortran where the default is to have as many threads as there are cores available. Hence the PGI compiler has been defaulting to one core, it seems. That’s easy to fix!
Regards,
Stephen.
As expected, setting:
nthreads = omp_get_num_procs()
call omp_set_num_threads(nthreads)
… before the first of my OpenMP regions solves the issue and gives me full parallelism on all cores.
I run into problems, though, when I try to parallelise other sections in other modules of my application. In another module, if I put the above two lines before the first OpenMP region, then it runs very well the first time the code is executed but there is always a hard crash (i.e. GUI suddenly disappears) the second time that same section of code is run.
If, on the other hand, I try to put the above two lines right at the beginning of the program, in the separate ‘initialisation’ module, then the hard crash occurs just before the section of code that had previously been running well when the two lines had been in its own module.
So, where is the correct place to set the number of cores for OpenMP to use, and once it has been set how wide is its scope?
You can put in at the beginning of a program before parallel section. Once set, the scope is throughout the program. I am not sure why it crashes as we don’t have enough information to determine the cause, please check that nthreads is more than 1. If you don’t use omp_lib module, please make sure omp_get_num_procs() is of integer type.
Hongyon
Setting the number of threads once at the beginning of the program definitely doesn’t work. If, in the initialisation section of the program, I put the code:
nthreads = omp_get_num_procs()
call omp_set_num_threads(nthreads)
open (12, file='threadcount.txt', status='REPLACE')
!$omp parallel
nthreads = omp_get_num_threads()
!$omp end parallel
write (12,*) 'In omp_init, nthreads = ', nthreads
close(12)
…then it is clear from the resulting file that the correct number of threads has been set. If, before the first parallelised loop, I then put the code:
open (12, file='threadcount.txt', status='REPLACE')
write (12,*) 'In Make_Incident_Image...'
close(12)
!$omp parallel
nthreads = omp_get_num_threads()
!$omp end parallel
open (12, file='threadcount.txt', status='REPLACE')
write (12,*) 'In Make_Incident_Image, nthreads = ', nthreads
close(12)
…then the application crashes, and the file contents indicate that it was at the “omp_get_num_threads” that it crashed.
Furthermore, if I then try to insert:
nthreads = omp_get_num_procs()
call omp_set_num_threads(nthreads)
…into the routine before the open(12…), then it becomes clear that that the file is never even opened. In other words, any OpenMP command at all, following the first initialisation, causes a crash.
Could this behvaiour be connected with the fact that the Fortran code here is all in a DLL called from a C++ GUI, and each execution of a Fortran routine is in a discrete call from C++? [/code]
Hi,
You are correct that the crash is due to the fact that Fortran is called from C++. Currently it is required that you call pghpf_init from your main program if you are going to use it with Fortran. It is required only on windows.
Here is example:
main.c:
extern “C” void foo_ ( int * );
extern “C” void pghpf_init(int *);
static int zz = 0;
main(int argc, char **argv)
{
int i;
pghpf_init(&zz);
i = 19;
foo_(&i);
}
foo.f90:
subroutine foo (i)
integer i
print *, i
end subroutine foo
Let me know if this helps.
Hongyon
Thanks Hongyon;
However, I already have a call to pghpf_init at the beginning of my program; the C++ program calls it from its ‘main’ routine, immediately before the first call to the Fortran routine where the OpenMP settings are initialised.
Stephen.
Which version of PGI you are using?
Which C++ compiler do you use?
How do you link with openmp library?
What is the command line? What is your OS?
Here is example with pgcpp and pgf90:
%pgf90 foo.f90 -mp -c
%pgcpp myproc.cpp foo.obj -mp -pgf90libs
Hongyon
Thanks Hongyon;
I’m using PGU Visual Fortran V10.8 with Microsoft VC++ 2008. I link with the OpenMP library just by selecting the “Process OpenMP directives” option in the Project property menu. The complete command line is:
(compiler)
-Mpreprocess -DPGI -DUSE_MKL -Bdynamic -Mbackslash -mp -I"C:\Program Files\Intel\MKL\10.2.2.025\include" -I"c:\program files\pgi\win64\10.8\include" -I"C:\Program Files\PGI\Microsoft Open Tools 10\include" -I"C:\Program Files\PGI\Microsoft Open Tools 10\PlatformSDK\include" -fastsse -tp=nehalem-64 -Minform=warn
(linker)
-Mmakedll -Bdynamic -mp -o “D:\Documents\Visual Studio 2008\Builds\Release_x86-64\nCore_PGI_x86-64.dll” -Wl,/libpath:“C:\Program Files\Intel\MKL\10.2.2.025\em64t\lib” mkl_solver_lp64_sequential.lib mkl_intel_lp64_dll.lib mkl_sequential_dll.lib mkl_core_dll.lib
All this is in indows & Ultimate 64-bit on a Lenovo W510 with an Intel i7 processor.
Stephen.
Hi,
Do you have OpenMP directive in C/C++ source code at all? If that is the case, this could be a problem. Since your C/C++ is compiled with Visual C++ and it has its own Openmp implemenation. While your Fortran code is compiled with PGI and it also has its own implementation.
Let say, you don’t have OpenMP code in C/C++, you will need to make sure that your main program/project is linked with PGI OpenMP library instead of Visual C++ OpenMP library.
Hongyon
Thanks Hongyon;
No, I don’t use any OpenMP in the C++ code; all the computationally-intensive stuff is kept within the Fortran DLL.
That being the case, is there a library I should explicitly link with, over and above those already listed in my linker command line (see my previous post)?
Stephen.
In your Visual C++ main project, this is for a Release configuration:
Open up property page and set following properties:
Configuration Properties → C/C++ → Code Generation → Runtime Library → Multi-threaded(/MT)
Configuration Properties → General → Common Language Runtime support ->No Common Language Runtime support
Configuration Properties → Linker → General → Additional Library Directories
Added following 2 lines,
the first line depends on which version your PVF is.
the second line depends on where you put your Fortran library.
C:\Program Files\PGI\win64\10.9\lib
C:\Users\myname\Documents\Visual Studio 2008\Projects\openmpf\openmpf\x64\Release
openmpf.lib
-defaultlib:pgmp
-defaultlib:pg
-defaultlib:pgf90
-defaultlib:pgf90_rpm1
-defaultlib:pgf902
-defaultlib:pgf90rtl
-defaultlib:pgftnrtl
-nodefaultlib:msvcrt
-nodefaultlib:msvcmrt
-defaultlib:libcmt
-defaultlib:pgc
-defaultlib:libnspgc
-defaultlib:oldnames
Also, I would make it static library instead of dll library. I am not sure it create DLL properly in PVF. If it works for you, go ahead.
Hongyon
I also noticed that you use MKL library. I don’t have experience using them but it may have some specific library that you might use with PGI.
Hongyon
I posted above that you should build static library instead of DLL.
All the dynamic library build in PVF is now should be as good as static library.
Hongyon