Compiling CUDA Edit into Existing f90

I’m interning with NASA for the summer and I’m editing a preexisting CFD solver to implement CUDA. I’ve read all of the appropriate literature, but this is my first time working with CUDA. I had a question on compilation.

I have edited an existing .f90 file to use cudafor, determine if devices are present and, if so, transfer control to a .cuf file I’ve coded, which includes a module and four subroutines. I have one host subroutine to transfer all necessary imports to the device and calls the two global subroutines, which are parallelizing two original loops. Both of the global subroutines also call a device subroutine.

There is an existing Makefile.am, which I am also fairly new to, having only worked with small chunks of code at my university and no projects large enough to merit a Makefile. Unfortunately, because of the nature of this project I cannot post source code, so I will try and be as descriptive as possible. The Makefile also handles MPI, which is also implemented in the software. When I try and run the configure script (to generate the actual Makefile from the Makefile.am) I get the following error:
config.status: creating src/Makefile
Warning: unable to locate module “cudafor” used in “././structured_inviscid_flux.f90”.
Warning: unable to locate module “cuda_drv” used in “././structured_inviscid_flux.f90”.

For simplicity, I have made sure that the cuda_drv.cuf file is in the same directory as the rest of the main files. I can see that all of the .f90 files are creating .mod, something the .cuf isn’t, or at least isn’t doing yet. I’m compiling using the pgfortran compiler, with options -pg and -Mcuda to enable the .f90 to use cudafor.

Okay, I believe that is the best description I can give. Does anyone have any advice on how to link this new file in with the rest and how to edit the Makefile. If the contents of the Makefile are necessary, I may be able to post that much, just let me know.

Thanks.
R M Sivley

Hi R M Sivley,

Warning: unable to locate module “cudafor” used in “././structured_inviscid_flux.f90”.
Warning: unable to locate module “cuda_drv” used in “././structured_inviscid_flux.f90”.

My best guess is that your configure script uses a Fortran dependency generator utility since these warnings are not coming from a PGI tool.

For the first warning, the utility most likely doesn’t know where to look for modules not contained within your project. Provided that a make rule is generated, sans the dependency, I think you should be able to ignore this one.

For the second warning, the utility probably doesn’t know to look in “.cuf” files for modules. Try renaming “cuda_drv.cuf” to “cuda_drv.f90” to see if it takes care of the warning. Since you’re compiling with “-Mcuda”, the suffix doesn’t matter.

Side note, with “-pg” (gprof style profiling) you will only be profiling the CPU code, not the GPU code. For GPU profiling, please use the PGI utility “pgcollect” instead of instrumenting your code (i.e. remove -pg). In 10.5 and earlier the GPU profiling information is fairly limited but will improve in the up coming 10.6 release.

Hope this helps,
Mat

I moved the cuda directory nearer to my src directory and set up a path for it. Solved the problem. You were right with the Mcuda suffix, simply changing it to .f90 solved that issue as well. Spot on.

Thanks for clearing that up before I got there. I certainly would’ve been hung up on it. Will pgcollect also analyze non-gpu optimized code? For the purpose of comparing the cpu-code to the cpu/gpu-code? The superiors like to see statistics.

Will pgcollect also analyze non-gpu optimized code? For the purpose of comparing the cpu-code to the cpu/gpu-code?

Sure. Pgcollect use Oprofile hardware base sampling on a CPU. As of the 10.6 release, it will begin use NVIDIA’s cudaprof profiler to obtain GPU measurements.

  • Mat

Sounds great. Any chance you’d mind hopping up to my other post and checking out that issue. Your input is very informative.