libnuma and pgf77 on 12.9-0

Hi,

We’ve recently upgraded to v12.9-0 from 10.4. I’ve noticed that if I promote executables to machines without that version of the PGI compiler, I also need to promote libnuma.so. I am not sure why the code needs to dynamically link this library as none of my code uses MPI or multi-threading. Is there anyway to disable this? I’ve tried using -Mconcur=nonuma and -mp=nonuma. Neither seem to do anything. The error message I receive is :

error while loading shared libraries: libnuma.so: cannot open shared object file: No such file or directory.

Hi Eric,

The “-mp=nonuma” flag should have done it. From the output of the “-dryrun” flag (below), you can see that libnuma doesn’t get linked in so I’m not sure why the dependency would still exist.

Another thing to try is to not link with the OpenMP library at all (i.e. add “-nomp”). We started linking with the OpenMP library by default in the 11.0 release so that users would be able to use CPU binding even with serial code.

  • Mat


% pgf90 -dryrun -mp=nonuma x.o
...
/usr/bin/ld /usr/lib64/crt1.o /usr/lib64/crti.o /usr/pgi/linux86-64/12.9/lib/trace_init.o /usr/lib/gcc/x86_64-redhat-linux/4.1.2/crtbegin.o /usr/pgi/linux86-64/12.9/lib/initmp.o /usr/pgi/linux86-64/12.9/lib/f90main.o -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 /usr/pgi/linux86-64/12.9/lib/pgi.ld -L/usr/pgi/linux86-64/12.9/lib -L/usr/lib64 -L/usr/lib/gcc/x86_64-redhat-linux/4.1.2 x.o -rpath /usr/pgi/linux86-64/12.9/lib /usr/pgi/linux86-64/12.9/lib/nonuma.o -lpgmp -lpthread -lpgf90 -lpgf90_rpm1 -lpgf902 -lpgf90rtl -lpgftnrtl -lnspgc -lpgc -lrt -lpthread -lm -lgcc -lc -lgcc /usr/lib/gcc/x86_64-redhat-linux/4.1.2/crtend.o /usr/lib64/crtn.o

I am still getting this error even using both flags.

/usr/local/pgi/linux86/12.9/bin/pgf77 -c -O2 -byteswapio -mp=nonuma -nomp simstat.F
/usr/local/pgi/linux86/12.9/bin/pgf77 -o …/exe//simstat.exe simstat.o

If I do an nm on the exe:

nm simstat.exe | grep numa
U numa_alloc_local
U numa_available
000000000a6dcd18 B numa_exit_on_error
U numa_max_node
U numa_set_preferred

Numa is still there.

Eric

Hi Eric,

You need to add the flag to the link options not just the compile.

  • Mat

Mat,

Thanks for your help on this.

Using LDFLAGS=-Wl,-nomp=nonuma yields the error message

/usr/bin/ld: cannot find -lnuma

Not sure what I am doing wrong.

Thanks,
Eric

Hi Eric,

The “-Wl,ldflags” says to pass the “ldflags” to the linker, which doesn’t know what “-nomp” is. Also, the compiler flags are “-nomp” or “-mp=nonuma”, you’ve mixed them together. Try:

LDFLAGS=-nomp

  • Mat

Thank you. That did it.

Eric