I lately compared the cpu time between Xenon and AMD64 in running MM5 codes (compiled on PGI 5.2), and found that the speed up was merely 1.4 times. Although the Xenon chip has a clock rate 2.8GHz faster than the AMD64 at 1.8GHz, I would expect the benefit of using 64 bits will still triumph a faster 32bit chips by a large margin.
I wonder if there’s any problem with my optimization setting or the environment? I am using the configure.user file provided in the “tips and resource” for compiling the AMD64/MM5 on Federa Linux. Do I need to use other Linux favor such as SuSE?
Any input will be greatly appreciated!
PS I also have other problems with compiling other modules of MM5 on AMD64/PGI fortran. May post my questions later …
The only thing a 64-bit system can guarantee you is that it can handle larger problem sizes. A 64-bit system should at the very least have the same performance as a 32-bit executable running on the same system. Some codes will see a performance benefit, and while rare, others actually degrade.
I just ran MM5 on a 1600 Mhz Dual CPU AMD Opteron running SuSE9.1 using both a 64-bit and 32-bit executable. The 64-bit executable took 144 seconds while the 32-bit took 148 seconds. Some performance benefit, but not much.
To better compare the Xeon and the AMD64, I’d run the Xeon’s MM5 executable on the AMD64. Comparing this result to the 64-bit executable’s result should give you a better idea of what amount of the performance increase can be attributed to 64-bits. Comparing the two 32-bit results will show what portion of the speed-up is due to differences between the two systems.
Please feel free to ask questions about the problems your having with the MM5 modules. I’m not an expert in building these modules, but we should be able to work through each problem. I’m sure if your encountering a problem, others are as well.
I tried your suggestions but failed to run the codes. For example, I compiled MM5 on Xenon and transported them to opteron for a trial run, which failed. The Xenon machine runs on redHat Linux 9 whilst the opteron sits on federa linux. Both have pgi 5.2-4 fortran.
While it is interesting to compare xenon and amd, but I am most concerned with compiling and running mm5 codes. I encountered quite a bit of problems. First, I have some problem with NCAR Graphics. Then I tried to compile the MM5/TERRAIN module without ncargraphics. The compilation apparently succeeded but the codes always fail with segmentation fault. Can you help?
MM5 uses a lot of stack space, so usually when MM5 seg faults it’s because of a stack overflow. To correct, set your stack size to unlimited (check your shell docs for the specific command to increase your stack space. On csh its ‘unlimit’).
The second cause is forgetting to add “-Mbyteswapio -Mnosgimp” and in 64-bits “-DDEC_ALPHA”. Adding “-DDEC_ALPHA” in 32-bits will also cause a seg fault.
I’ve only built the NCAR Graphics library and TERRAIN once but don’t remember the issues (its been awhile). I’ll try it again and log what I do.
Yes, please post your log after coming back from testing ncarg and terrain. In particular, I want to know if one should compile these modules as 32bit or 64bit on opteron?
My compilation problems are with the other MM5 modules, not MM5 itself. Since there are so many modules, I want to resolve them one at a time. Your experience with ncarg and terrain will be helpful.
I was able to work on the NCAR Graphic library and was able to get the test case to work in 32-bits (on a P4 and AMD64). I did have to make some modifications to the configuration, but not much. To get it to work in 64-bits will require some porting which I have not done. I may have the time later but for now the 32-bit version should work fine. I’ll try and fit in TERRAIN today or tommorrow.
In both cases I used pgf90 and pgcc release 5.2-4 and ncag release 4.3.1. Disclamier: I’ve only done a minimal amount of testing. The accompanying test case worked but I have not tested beyond that. Also, I have not tried increasing the optimization.
Add “-lpgf90 -lpgf90_rpm1 -lpgf902” to the “CtoFLibraries”
copy config/LINUX to config/LINUX.org
copy config/LINUX.pgi to config/LINUX
Edit config/Template by either removing or commenting out
lines 833 and 835. Change line 834 from “.f” to “.F”.
The resulting preprocesser “.f” files and not formatted properly, so I
removed the preprocessing since pgf90 can handle “.F” files directly.
Run “./Configure -v” and follow the steps. Be sure to build the
F90 libraries and set the f90 compiler to ‘pgf90’. I did not use
the HDF libraries.
Run “make Everything >& make.log”. Inspect the log for errors.
set your PATH to the location you installed the ncar executables.
Also set NCARG_ROOT.
Test the configuration by running
% ncargex cpex08
% ctrans -d X11 cpex08.ncgm
32-bits on AMD64
Download and untar the package
set NCARG to the “nacag-4.3.1” directory.
Set your compiler path to the 32-bit compilers. These
are located in “$PGI/linux86/5.2/bin”. The 64-bit compilers
are located in “$PGI/linux86-64/5.2/bin”.
Modify the file config/LINUX.pgi
Add “-lpgf90 -lpgf90_rpm1 -lpgf902” to the “CtoFLibraries”
Add “x86_64:” to line 357 of config/ymake. This will allow the
configure script to recognize the system.
Edit config/Template by either removing or commenting out
lines 833 and 835. Change line 834 from “.f” to “.F”.
The resulting preprocesser “.f” files and not formatted properly, so I
removed the preprocessing since pgf90 can handle “.F” files directly.
Run “./Configure -v” and follow the steps. Be sure to build the
F90 libraries and set the f90 compiler to ‘pgf90’. I did not use
the HDF libraries.
Run “make Everything >& make.log”. Instpect the log errors.
set your PATH to the location you installed the ncar executables.
Also set NCARG_ROOT
Test the configuration by running
% ncargex cpex08
% ctrans -d X11 cpex08.ncgm
Let me know if you have problems or if something is unclear.
I have successfully compiled TERRAIN in 32bit on AMD64. As said, modified the PGI path pointing to 32bit pgi version. Then, edit TERRAIN/Makefile. Add “-L/usr/X11R6/lib” to LOCAL_LIBRARIES; else, the linker cannot find libX11.
May you kindly take a loot at REGRID/pregrid… As before I used 32bit pgi fortran to compile the codes. What happens is that a c program: uti/cio.c is compiled by gnu c which produces 64bit output. What should I do to get around this problem? Thanks!
Either add “-m32” to the CCFLAGS or add a line that sets CC to pgcc. “-m32” is gcc’s flag to tell the compiler to create a 32-bit object.
FYI, I’ve been able to compile the 32-bit TERRAIN with the NCAR Libraries and in 64-bits without the libraries. To compile in 64-bits change the “LOCAL_LIBRARIES” to “-L/usr/X11R6/lib64 -lX11 -L/usr/lib64 -g77libs” and set “NCARGRAPHICS” to “NONCARG”. On 64-bit systems 32-bit libraries are still in “lib” while 64-bit libraries are in “lib64”. However, I’m not sure if it’s really worth it to compile this 64-bits. But that’s one of the great thing about AMD64, if can’t or don’t want to port your app to 64-bits, you don’t have to!
I’m still downloading data so I haven’t tested it yet. I’ll update you if I see the seg fault you reported.
The seg fault your seeing with the 64-bit terrain.exe is caused by a subtle coding error in the file src/terdrv.F. The subroutine TERDRV calls WATERCHECK using 10 actual arguments while the WATERCHECK subroutine is expecting 11. Adding a 11th argument “0” at line 477 and 481 fixes the problem.
The Makefile that comes with the REGRID package from UCAR has a “Linux” section in it which is configured for use with pgf90. The selection of the “Linux” section is done automatically in the Makefile by grepping the output of the “uname -a” command.
Most likely, your Makefile has been modified and the “-DDEC -DBIT32” define flags were removed from the “CPPFLAGS” and “CCFLAGS” variables. To fix, you can add these define flags back in to both the regridder and pregridder sections, or re-download the REGRID.tar.gz package.
I am also having problems getting ictrans working on opteron - I was trying to follow your instructions given previously - but can’t work out which lines to edit because my files don’t have the same line numbers as yours.
Alternatively - are there any binaries out there for ictrans and idt ?
I downloaded the latest NCAR graphics library (4.4.1) this morning and was able to get “ictrans” and “idt” to compile. I first moved the “config/LINUX.64.PGI” to “config/LINUX” and then ran “./Configure” as normal. I think this config file is new as of the 4.4.0 release.
As for binaries, I did see a binary available for Red Hat 9.0 using the PGI compilers, but I don’t know if these were for 32 or 64-bit.
I’m trying to compile ncarg-4.4.1 using pgf90 (version 6.0) and gcc (3.3.5). The ncarg package comes with LINUX.GNU and LINUX.PGI config files but no config file is available to work with pgf90 and gcc.
Trying to create a $NCARG_INSTALL/config/LINUX file that uses pgf90 and gcc I always get unresoveld symbols.
Does anybody have a “…/config/LINUX” file that works with pgf90 and gcc ?
Thanks,
Víctor.
PS: By the way, I’m running debian 32bits on an AMD64 system.
The problem is that they use gcc to link, but the PGI runtimes are not being included on the link line. I tried changing the “CtoFLibraries” in the LINUX file to include these libraries, but for some reason, Configure just ignores them.
Instead, either edit the Makefile’s “CTOFLIBS” variable to include the following libraries, or simply pass them via the command line:
Please change the PGI library path if different. I was able to successfully compile all objects using pgf90 and gcc when I added the correct runtime libraries.