I have two FORTRAN codes that use openMP to run parallel loops. Both codes call the same subroutines that do most of the computation. Both codes seem to work in that the single-CPU versions give the same results as the parallel versions.
I noticed some odd behavior with the RAM. In one code, there are two parallel loops, with one run after the other, as in
!$omp parallel firstprivate(lots of variables) shared(other variables) !$omp do do i=1,Nloops lots of computations enddo !$omp enddo compute statistics, set up next loop !$omp parallel firstprivate(lots of variables) shared(other variables) !$omp do do i=1,Nloops more computations enddo !$omp enddo
When using a large number of threads, the code uses about 40% of the RAM (according to the top command) when executing the first loop. When it enters the second loop, the RAM usage roughly doubles. This behavior is not seen when compiled with gfortran.
Here are the compile commands:
pgf90 -O3 -Mextend -mcmodel=medium -mp -o fred_pgi fred.for
gfortran -fopenmp -mcmodel=medium -O2 -ffixed-line-length-132 -o fred_gfort fred.for
In my other code, there are 5 sets of parallel loops, with three of them having fewer iterations. In this case, the RAM usage increases by a bit over a factor of 4 as the program progresses. With gfortran, the RAM usage stays at a fixed value.
It seems like the PGI-compiled code does not want to give back the RAM that was used in a loop. In most cases the RAM usage does not exceed the machine capacity, but I have had a case or two where large arrays are needed and the RAM limit was exceeded, causing a system crash.
Is there a way to make the PGI-compiled codes stay at a fixed RAM usage, like the gfortran-compiled codes do? I tried to put
after the first parallel loop, and nothing different happened.