Libnuma and pgcc?

Is it possible to use libnuma?

With gcc I compile my app with:
gcc -o pstream -O4 -DNUMA pstream.c -lpthread -lm -lnuma

With pathscale I compile with:
pathcc -o pstream -O4 -DNUMA pstream.c -lpthread -lm -lnuma

With portland group’s cc I get:
$ pgcc -o pstream -O4 -DNUMA pstream.c -lpthread -lm -lnuma
pstream.c:
pstream.o: In function stream_thread': pstream.c:(.text+0x3af): undefined reference to numa_run_on_node’
pstream.c:(.text+0x625): undefined reference to numa_free' pstream.c:(.text+0x635): undefined reference to numa_free’
pstream.c:(.text+0x644): undefined reference to `numa_free’

$ pgcc -V

pgcc 7.0-7 64-bit target on x86-64 Linux
Copyright 1989-2000, The Portland Group, Inc. All Rights Reserved.
Copyright 2000-2007, STMicroelectronics, Inc. All Rights Reserved.

I do have an include in my source:
#include <numa.h>

I also have the libnuma libraries installed.
~/pstream$ ldd ./pstream_gcc | grep -i numa
libnuma.so.1 => /usr/lib/libnuma.so.1 (0x00002b46f37d0000)

Hi,

You should be able to link with -lnuma. At install time we determine where to look for system libraries and if libnuma exists and put in localrc. What is in your $PGI/linux86-64/7.0-7/bin/localrc file? Is there /usr/lib64/libnuma*? What is the output at link time when replacing -lpthread -lm -lnuma with -mp -v?

Hongyon

Ah, so what line should I put in /opt/pgi/linux86-64/7.0-7/bin/localrc to be able to use /usr/lib/libnuma.so.1 instead of
/opt/pgi/linux86-64/7.0-7/lib/libnuma.so.1?

$ cat /opt/pgi/linux86-64/7.0-7/bin/localrc | egrep -v “^#”
set LDSO=/lib64/ld-linux-x86-64.so.2;
set GCCDIR=/usr/lib/gcc/x86_64-linux-gnu/4.1.2;
set GCCINC=/usr/lib/gcc/x86_64-linux-gnu/4.1.2/include;
set G77DIR="";
set OEM_INFO=64-bit target on x86-64 Linux;
set LOCALRC=YES;
set THROW=__THROW=;
set EXTENSION=extension=;
set LC=$if(-Bstatic,-lgcc -lgcc_eh -lc -lgcc -lgcc_eh -lc, -lgcc -lc -lgcc);
export PGI=/opt/pgi;

I do not need -mp (it’s already threaded), but:
pgcc -o pstream -O4 -DNUMA pstream.c -mp -v
/opt/pgi/linux86-64/7.0-7/bin/pgc pstream.c -opt 4 -x 119 0xa10000 -x 122 0x40 -x 123 0x1000 -x 127 4 -x 127 16 -x 19 0x400000 -x 28 0x40000 -x 70 0x8000 -x 122 1 -quad -x 59 4 -x 59 4 -tp gh-64 -astype 0 -stdinc /opt/pgi/linux86-64/7.0-7/include:/usr/local/include:/usr/lib/gcc/x86_64-linux-gnu/4.1.2/include:/usr/lib/gcc/x86_64-linux-gnu/4.1.2/include:/usr/include -def unix -def __unix -def unix -def linux -def __linux -def linux -def __NO_MATH_INLINES -def x86_64 -def LONG_MAX=9223372036854775807L -def ‘SIZE_TYPE=unsigned long int’ -def ‘PTRDIFF_TYPE=long int’ -def __THROW= -def extension= -def amd64 -def SSE -def MMX -def SSE2 -def SSE3 -def SSE4A -def ABM -predicate ‘#machine(x86_64) #lint(off) #system(posix) #cpu(x86_64)’ -def NUMA -cmdline ‘+pgcc pstream.c -o pstream -O4 -DNUMA -mp -v’ -x 123 4 -x 123 0x80000000 -alwaysinline /opt/pgi/linux86-64/7.0-7/lib/libintrinsics.il 4 -mp -x 69 0x200 -asm /tmp/pgcclwNgHmT7-iF7.s
PGC-I-0222-Redundant definition for symbol __THROW (/usr/include/sys/cdefs.h: 63)
PGC-I-0222-Redundant definition for symbol extension (/usr/include/sys/cdefs.h: 287)
PGC/x86-64 Linux 7.0-7: compilation completed with informational messages

/usr/bin/as /tmp/pgcclwNgHmT7-iF7.s -o /tmp/pgccJwNgPW2kYM8Q.o

/usr/bin/ld /usr/lib64/crt1.o /usr/lib64/crti.o /opt/pgi/linux86-64/7.0-7/lib/trace_init.o /usr/lib/gcc/x86_64-linux-gnu/4.1.2/crtbegin.o /opt/pgi/linux86-64/7.0-7/lib/initmp.o -m elf_x86_64 -dynamic-linker /lib64/ld-linux-x86-64.so.2 /opt/pgi/linux86-64/7.0-7/lib/pgi.ld -L/opt/pgi/linux86-64/7.0-7/lib -L/usr/lib64 -L/usr/lib/gcc/x86_64-linux-gnu/4.1.2 -rpath /opt/pgi/linux86-64/7.0-7/lib /tmp/pgccJwNgPW2kYM8Q.o -o pstream -lpgmp -lpgbind -lnuma -lpthread -lnspgc -lpgc -lm -lgcc -lc -lgcc /usr/lib/gcc/x86_64-linux-gnu/4.1.2/crtend.o /usr/lib64/crtn.o
/tmp/pgccJwNgPW2kYM8Q.o: In function stream_thread': pstream.c:(.text+0x40f): undefined reference to numa_run_on_node’
pstream.c:(.text+0x685): undefined reference to numa_free' pstream.c:(.text+0x695): undefined reference to numa_free’
pstream.c:(.text+0x6a4): undefined reference to `numa_free’
pgcc-Fatal-linker completed with exit code 1

Unlinking /tmp/pgcclwNgHmT7-iF7.s
Unlinking /tmp/pgccJwNgPW2kYM8Q.o

Hi,

What distribution of Linux is this?

A quick and dirty solution is make a symlink from /usr/lib to /usr/lib64. Then run makelocalrc again to update localrc file and as it finds /usr/lib64/libnuma.so.1 it should remove /opt/pgi/linux86-64/7.0-7/lib/libnuma.so.1.

Hongyon