mpif90 and -ta=nvidia,time

This is another “might be a bug, might be a feature” question. In my ongoing testing (viz. this forum and my current near-spamming of it), a driver-kernel I made up seemed to show an interesting oddity. Usually, in my code, I tend to use MPI_Wtime as a Fortran timer since it’s fairly portable, looks at wall time, and is one of those few MPI bits that doesn’t care about MPI that much.

But, I noticed that when I compile my driver with FC=mpif90 rather than FC=pgifortran, the -ta=nvidia,time timing information isn’t outputted. My compile string is:

$(FC) -fast -Kieee -Minfo=all,accel -r4 -Mextend -Mpreprocess -Ktrap=fp -ta=nvidia,time

I then run my code, which is uniprocessor, without mpirun or anything, just ./command.exe. With pgfortran, I get the Accelerator Kernel Timing data, with mpif90, I do not. I’ve even removed all the bits of MPI from ‘use mpi’ to ‘MPI_Finalize(ierr)’ and this is consistent even then.

I suppose my question now is, is this expected behavior? That is, because of the nature of mpif90, the wrapper is shuttling the timing data to /dev/null or some other stream? Or should mpif90 output this data, albeit without any guarantee of when and where and how many processes do so if you run on >1 process (like most WRITEing with MPI)?

Hi Matt,

Looks like a driver configuration issue where mpif90 isn’t pulling in the “kvprint.o” object which is used to print the profiling information. To work around the issue, either use “pgf90 -Mmpi” instead of “mpir90” or add the file “/opt/pgi/linux86-64/9.0-3/lib/kvprint.o” to you link. (update the path to match your system).

I’ll send a report to engineering and see if we can get this fixed.

Thanks,
Mat

Yep, that fixed it. Just for my own edification, if I’m using the built-in mpich, is there any difference between “pgf90 -Mmpi” and “mpif90”, bug aside? (I can see a possibility if it was OpenMPI with its more complex set of wrappers.)

“-Mmpi” will use the pre-built MPI libraries that are included with the PGI products. Note, I just built OpenMPI 1.3 and found that the kvprint.o object does correctly get add to the link when using the OpenMPI mpif90 driver. So using openMPI is another work around.

  • Mat

I have the same problem using pgcc (10.0-0 64-bit target on x86-64 Linux -tp penryn-64). Using -ta=nvidia,time no timing information is given.
However, linking manually with “/opt/pgi/linux86-64/10.0/lib/kvsetta.o” prints the timing information when the program is executed.

Richard

Hi Richard,

With 10.0, kvprint was replaced with the kvsetta object. We’ll send a note to our contact at OpenMPI and see if we can get this object added to the link when -ta=time is used.

Thanks,
Mat