Segmentation fault error when running wrf.exe (WRFv3.8.1)

Accelerated Computing HPC Compilers Legacy PGI Compilers
#1

Dear all,

When I ran the wrf.exe with the following command:

mpiexec --hostfile ${PBS_NODEFILE} -np 32 -ppn 32 ./wrf.exe

I got the segmentation fault (11) as soon as wrf.exe starts. The real.exe works without any problems. I compiled WRF with the followings options, compiler, and library:

  1. Option: 54. (dmpar) PGI (pgf90/pgcc)
  2. compiler pgi-19.5
  3. Netcdf-3.6.1
  4. Mpich-3.0.4

export FC=pgf90
export F90=pgf90
export F77=pgf90
export CC=pgcc
export CXX=pgc++
export FFLAGS="-O2 -w -V"
export CFLAGS="-O2 -Msignextend -V"
export CPPFLAGS="-DNDEBUG -DpgiFortran"


Before compiling WRF I have taken all the tests suggested in the following link:

https://www2.mmm.ucar.edu/wrf/OnLineTutorial/compilation_tutorial.php

And confirm that my system has passed all the tests required and both WRF and WPS have been compiled successfully

When I used intel-compiler and intel-mpi to compile WRF, the segmentation fault did not occur. So, I think the problem was because of the pgi-compiler but I am not sure what it is exactly. I would appreciate it very much if you could help me on how to solve the problem. If you need any other information, please let me know.

Thank you in advance.

#2

Hi Hungnt,

Not to say that this isn’t a compiler issue, but first let’s first check that you’re not running into a stack overflow. Can you please try running again, this time setting your stack size to “unlimited” and setting the OpenMP environment variable “OMP_STACKSIZE=196M”?

-Mat

#3

Dear Mat,

Thank you very much for your advice. I have done as you suggested by adding “ulimit -s unlimited” and “export OMP_STACKSIZE=196M” to the bash_profile, bashrc and reran the model but the same error still occured. I would like to ask if there are other reasons which could cause the error or not?

Thanks again for your help.

#4

I would like to ask if there are other reasons which could cause the error or not?

While it’s a been awhile, I have built WRF v3.8.1 before and didn’t see any issues, and why I’m leaning towards something specific to your system. Though exactly what that reason is, I’m not sure.

Though, WRF v3.8.1 is a bit old, so you might try using the newest version 4.1. Also if you can, try updating the compiler to the latest PGI release, 20.4.

-Mat