[SOLVED] Non-working OpenMP

Hi all,

I’m currently interested in compiling the WRF-ARW meteorological model in smpar (i.e.: multithreaded method), but I seem to have some issues with OpenMP.

The machine I’m working on has an 8 core intel CPU i7, therefore should be fully capable of multithreading. I’m testing such capability by compiling and running the example program at page 55 of the PGI Compiler User’s Guide (Release 2013) on an x64 Ubuntu 12.10 OS (bash shell).

I’ve created a FORTRAN file (main.f) with the example program and I’m proceeding this way:

 
modelstation@modelstation:~$ export OMP_NUM_THREADS=8
modelstation@modelstation:~$ pgf90 -mp main.f

Now, by running the executable, I’m expecting to see 8 threads, but the output is this:

Thread             0  local sum:     500500.0000000000     
Global Sum:     500500.0000000000     
FORTRAN STOP

I don’t know exactly why and I do not know exactly how to proceed (it’s my first attempt at multithreaded computing).

If it may be helpful for diagnostics: while looking for solutions I’ve also tried testing the OpenMP suggested test-program (http://openmp.org/wp/openmp-compilers/) and I received a “missing predefs.h” error, which I solved by installing libc6-dev-i386 (following this suggestion http://ubuntuforums.org/showthread.php?t=1877944). Even in that case I could see only one thread instead of the expected 8.

Thanks for your help and Best Regards!

Hmm. Well, I just took that same program and tried it here:

$ pgfortran -mp -Minfo main.f90
main:
     13, Parallel region activated
     17, Parallel loop activated with static block schedule
     20, Barrier
     23, Begin critical section (__cs_unspc)
     24, End critical section (__cs_unspc)
         Parallel region terminated
$ setenv OMP_NUM_THREADS 1
$ ./a.out
 Thread             0  local sum:     500500.0000000000     
 Global Sum:     500500.0000000000     
FORTRAN STOP
$ setenv OMP_NUM_THREADS 8 
$ ./a.out
 Thread             2  local sum:     39125.00000000000     
 Thread             4  local sum:     70375.00000000000     
 Thread             6  local sum:     101625.0000000000     
 Thread             0  local sum:     7875.000000000000     
 Thread             7  local sum:     117250.0000000000     
 Thread             1  local sum:     23500.00000000000     
 Thread             5  local sum:     86000.00000000000     
 Thread             3  local sum:     54750.00000000000     
 Global Sum:     500500.0000000000     
FORTRAN STOP

and similarly for the OpenMP test case.

One thing you can try is to hardwire the number of threads and see if that works (note the num_threads):

#include <omp.h>
#include <stdio.h>
int main() {
#pragma omp parallel num_threads(4)
printf("Hello from thread %d, nthreads %d\n", omp_get_thread_num(), omp_get_num_threads());
}

This should run with 4 threads regardless of your environment. If it works, then something in your environment is setting OMP_NUM_THREADS back to 1 somehow.

Matt

Thanks for the input!

Weird thing is that, before seeing your answer, I’ve also tried a different couple of programs, both fortran

C******************************************************************************
C FILE: omp_hello.f
C DESCRIPTION:
C   OpenMP Example - Hello World - Fortran Version
C   In this simple example, the master thread forks a parallel region.
C   All threads in the team obtain their unique thread number and print it.
C   The master thread only prints the total number of threads.  Two OpenMP
C   library routines are used to obtain the number of threads and each
C   thread's number.
C AUTHOR: Blaise Barney  5/99
C LAST REVISED: 
C******************************************************************************

      PROGRAM HELLO
     
      INTEGER NTHREADS, TID, OMP_GET_NUM_THREADS,
     +        OMP_GET_THREAD_NUM
  
C     Fork a team of threads giving them their own copies of variables
!$OMP PARALLEL PRIVATE(NTHREADS, TID)


C     Obtain thread number
      TID = OMP_GET_THREAD_NUM()
      PRINT *, 'Hello World from thread = ', TID

C     Only master thread does this
      IF (TID .EQ. 0) THEN
        NTHREADS = OMP_GET_NUM_THREADS()
        PRINT *, 'Number of threads = ', NTHREADS
      END IF

C     All threads join master thread and disband
!$OMP END PARALLEL

      END

and C

#include <omp.h>
#include <stdio.h>
int main() {
#pragma omp parallel
printf("Hello from thread %d, nthreads %d\n", omp_get_thread_num(), omp_get_num_threads());
}

and they work fine! They return multiple threads if OMP_NUM_THREADS is exported…it’s only main.f that keeps having a single thread issue…

…can’t understand why!

Hi AZaldei,

Since the same source works for Matt, and you can run other programs under OpenMP, I’m wondering if there is a cut and paste issue in your main.f? Can you please post the source you’re using, as well as the compilation line and the output from your run?

Thanks,
Mat

Sure! Here is the program:

PROGRAM MAIN
       INTEGER I, N, OMP_GET_THREAD_NUM
       REAL*8 V(1000), GSUM, LSUM
       GSUM = 0.0D0
       N = 1000
       DO I = 1, N
       V(I) = DBLE(I)
       ENDDO
       !$OMP PARALLEL PRIVATE(I,LSUM) SHARED(V,GSUM,N)
       LSUM = 0.0D0
       !$OMP DO
       DO I = 1, N
       LSUM = LSUM + V(I)
       ENDDO
       !$OMP END DO
       !$OMP CRITICAL
       print *, "Thread ",OMP_GET_THREAD_NUM()," local sum: ",LSUM
       GSUM = GSUM + LSUM
       !$OMP END CRITICAL
       !$OMP END PARALLEL
       PRINT *, "Global Sum: ",GSUM
       STOP
       END

Here is the compiling options:

modelstation@modelstation:~/Software$ export OMP_NUM_THREADS=8
modelstation@modelstation:~/Software$ pgfortran -mp main.f

And here’s the output:

modelstation@modelstation:~/Software$ ./a.out
 Thread             0  local sum:     500500.0000000000     
 Global Sum:     500500.0000000000     
FORTRAN STOP

It’s a formatting issue. Since your file’s suffix is “.f”, fixed form is used. Hence, the directives need to start in column 1. Matt used a “.f90” suffix where free format was used.

You have three options, compile with “-Mfree”, move the “!$OMP” directives to the first column, or rename your file to “main.f90”.

Hope this helps,
Mat

% pgf90 -mp main.f -Minfo 
% a.out
 Thread             0  local sum:     500500.0000000000     
 Global Sum:     500500.0000000000     
FORTRAN STOP

% pgf90 -mp main.f -Minfo -Mfree
main:
      9, Parallel region activated
     12, Parallel loop activated with static block schedule
     15, Barrier
     17, Begin critical section (__cs_unspc)
     18, End critical section (__cs_unspc)
         Parallel region terminated
% a.out
 Thread             0  local sum:     7875.000000000000     
 Thread             7  local sum:     117250.0000000000     
 Thread             6  local sum:     101625.0000000000     
 Thread             4  local sum:     70375.00000000000     
 Thread             2  local sum:     39125.00000000000     
 Thread             3  local sum:     54750.00000000000     
 Thread             5  local sum:     86000.00000000000     
 Thread             1  local sum:     23500.00000000000     
 Global Sum:     500500.0000000000     
FORTRAN STOP

I had a reply just like mat’s but mat beat me to it.

And I learned that PGI supports ! as a comment character in column 1! (My edit used c$OMP).

Whooops! You’re absolutely right! I didn’t notice!

Thanks! Addressing the formatting issue worked perfectly!