LAM/MPI and GNU Libtool

Greetings.

I am the lead developer for the LAM/MPI project.

I would very much like to see support for PGI compilers in GNU Libtool (which would allow, among other packages, LAM/MPI to compile nicely with the PGI compilers). I’ve even submitted 2 patches – both of which have been accepted – to the GNU Libtool project to enable PGI support in Libtool.

The problem is that I clearly don’t know enough about the PGI compilers to make the Libtool patches correctly – even with my latest patch, Libtool still doesn’t make some kinds of shared libraries properly.

My question is: does anyone at PGI care? Over the past several months, I’ve e-mailed trs@pgroup.com directly, asking specific questions, and CC’ed trs@pgroup.com on all my posts to libtool-users@gnu.org. Despite that, I’ve gotten only one e-mail response from the support group which was vague and non-technical. I’ve resorted to this public web forum because I can’t seem to get a reply via e-mail.

I’d really like some help in getting this Libtool patch right – which would only broaden the reach of the PGI compilers (it would certainly make compiling LAM/MPI with your compilers a trivial task: “./configure; make all install” – like it is with all other compilers!).

Does PGI care? Or should we drop PGI support for LAM/MPI?

Thanks for your time.

Mr. Squyres,

We appreciate that you submitted patches to the libtool development team and were are able have these patches accepted. As far as we can tell libtool is correct, however it has exposed a bug in our driver (see Linker option) where we are not preserving the order of position sensitive flags when passed to the linker. We have submitted a techinal problem report (TPR#3477) to our engineers and hope to have the issue resolved in the near future.

The correct work around here would be to either call ld directly or use gcc. In this case, both pgcc and gcc simply envoke ld, pgcc just puts some of the flags in the wrong place.

Thank you for your efforts!
Mat

In later e-mails (cc’ed to trs@pgroup.com), we worked around the --whole-archive issue and have Libtool properly building C libraries (both static and shared) with the PGI compilers. So the proposed workarounds are unnecessary.

Remember: the goal here is to get Libtool to do the Right Things – not to make PGI users have weird/complicated workarounds. So we can change Libtool do whatever we want.

It also builds static/shared C++ libraries properly, but only if the C++ libraries do not contain templated functions. This is the last issue that I am aware of. Are there special flags that must be given to pgCC to link C++ libraries that include templated functions?

I have a trivial project that shows this problem – it builds a trivial set of C++ libraries (one of which contains a templated function) and then tries to link a trivial C++ application against those libraries. The final application link fails, complaining about issues with the templated function. The failure is slightly different depending on whether the C++ libraries were built as static or shared libraries, but the gist of it remains the same (i.e., I’m guessing it’s a templated function issue).

I don’t see a way to attach a tarball here in the web forums – I’ve sent a copy of it to trs@pgroup.com. To replicate the problem:


$ gunzip -c lt-pgi-test-1.2.3.tar.gz | tar xf -
$ cd lt-pgi-test-1.2.3.tar.gz
$ ./configure CC=pgcc CXX=pgCC
$ make

Here’s the output from the C++ portion:


[8:32] vogon:/home/jsquyres/lt-pgi-test/cxx % make
Making all in one
make[1]: Entering directory /home/jsquyres/lt-pgi-test/cxx/one' source='cxx_one_1.cc' object='cxx_one_1.lo' libtool=yes \ DEPDIR=.deps depmode=none /bin/sh ../../depcomp \ /bin/sh ../../libtool --tag=CXX --mode=compile pgCC -DPACKAGE_NAME=\"\" -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\" -DPACKAGE_STRING=\"\" -DPACKAGE_BUGREPORT=\"\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -I. -I. -g -c -o cxx_one_1.lo cxx_one_1.cc pgCC -DPACKAGE_NAME=\"\" -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\" -DPACKAGE_STRING=\"\" -DPACKAGE_BUGREPORT=\"\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -I. -I. -g -c cxx_one_1.cc -o cxx_one_1.o source='cxx_one_2.cc' object='cxx_one_2.lo' libtool=yes \ DEPDIR=.deps depmode=none /bin/sh ../../depcomp \ /bin/sh ../../libtool --tag=CXX --mode=compile pgCC -DPACKAGE_NAME=\"\" -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\" -DPACKAGE_STRING=\"\" -DPACKAGE_BUGREPORT=\"\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -I. -I. -g -c -o cxx_one_2.lo cxx_one_2.cc pgCC -DPACKAGE_NAME=\"\" -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\" -DPACKAGE_STRING=\"\" -DPACKAGE_BUGREPORT=\"\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -I. -I. -g -c cxx_one_2.cc -o cxx_one_2.o /bin/sh ../../libtool --tag=CXX --mode=link pgCC -g -o liblt_pgi_cxx_one.la cxx_one_1.lo cxx_one_2.lo mkdir .libs ar cru .libs/liblt_pgi_cxx_one.a cxx_one_1.o cxx_one_2.o ranlib .libs/liblt_pgi_cxx_one.a creating liblt_pgi_cxx_one.la (cd .libs && rm -f liblt_pgi_cxx_one.la && ln -s ../liblt_pgi_cxx_one.la liblt_pgi_cxx_one.la) make[1]: Leaving directory /home/jsquyres/lt-pgi-test/cxx/one’
Making all in two
make[1]: Entering directory /home/jsquyres/lt-pgi-test/cxx/two' source='cxx_two_1.cc' object='cxx_two_1.lo' libtool=yes \ DEPDIR=.deps depmode=none /bin/sh ../../depcomp \ /bin/sh ../../libtool --tag=CXX --mode=compile pgCC -DPACKAGE_NAME=\"\" -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\" -DPACKAGE_STRING=\"\" -DPACKAGE_BUGREPORT=\"\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -I. -I. -g -c -o cxx_two_1.lo cxx_two_1.cc pgCC -DPACKAGE_NAME=\"\" -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\" -DPACKAGE_STRING=\"\" -DPACKAGE_BUGREPORT=\"\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -I. -I. -g -c cxx_two_1.cc -o cxx_two_1.o source='cxx_two_2.cc' object='cxx_two_2.lo' libtool=yes \ DEPDIR=.deps depmode=none /bin/sh ../../depcomp \ /bin/sh ../../libtool --tag=CXX --mode=compile pgCC -DPACKAGE_NAME=\"\" -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\" -DPACKAGE_STRING=\"\" -DPACKAGE_BUGREPORT=\"\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -I. -I. -g -c -o cxx_two_2.lo cxx_two_2.cc pgCC -DPACKAGE_NAME=\"\" -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\" -DPACKAGE_STRING=\"\" -DPACKAGE_BUGREPORT=\"\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -I. -I. -g -c cxx_two_2.cc -o cxx_two_2.o /bin/sh ../../libtool --tag=CXX --mode=link pgCC -g -o liblt_pgi_cxx_two.la cxx_two_1.lo cxx_two_2.lo mkdir .libs ar cru .libs/liblt_pgi_cxx_two.a cxx_two_1.o cxx_two_2.o ranlib .libs/liblt_pgi_cxx_two.a creating liblt_pgi_cxx_two.la (cd .libs && rm -f liblt_pgi_cxx_two.la && ln -s ../liblt_pgi_cxx_two.la liblt_pgi_cxx_two.la) make[1]: Leaving directory /home/jsquyres/lt-pgi-test/cxx/two’
Making all in .
make[1]: Entering directory /home/jsquyres/lt-pgi-test/cxx' /bin/sh ../libtool --tag=CC --mode=link pgcc -g -o liblt_pgi_cxx.la -rpath /usr/local/lib one/liblt_pgi_cxx_one.la two/liblt_pgi_cxx_two.la mkdir .libs rm -fr .libs/liblt_pgi_cxx.lax mkdir .libs/liblt_pgi_cxx.lax rm -fr .libs/liblt_pgi_cxx.lax/liblt_pgi_cxx_one.a mkdir .libs/liblt_pgi_cxx.lax/liblt_pgi_cxx_one.a (cd .libs/liblt_pgi_cxx.lax/liblt_pgi_cxx_one.a && ar x /home/jsquyres/lt-pgi-test/cxx/one/.libs/liblt_pgi_cxx_one.a) rm -fr .libs/liblt_pgi_cxx.lax/liblt_pgi_cxx_two.a mkdir .libs/liblt_pgi_cxx.lax/liblt_pgi_cxx_two.a (cd .libs/liblt_pgi_cxx.lax/liblt_pgi_cxx_two.a && ar x /home/jsquyres/lt-pgi-test/cxx/two/.libs/liblt_pgi_cxx_two.a) ar cru .libs/liblt_pgi_cxx.a .libs/liblt_pgi_cxx.lax/liblt_pgi_cxx_one.a/cxx_one_1.o .libs/liblt_pgi_cxx.lax/liblt_pgi_cxx_one.a/cxx_one_2.o .libs/liblt_pgi_cxx.lax/liblt_pgi_cxx_two.a/cxx_two_1.o .libs/liblt_pgi_cxx.lax/liblt_pgi_cxx_two.a/cxx_two_2.o ranlib .libs/liblt_pgi_cxx.a rm -fr .libs/liblt_pgi_cxx.lax creating liblt_pgi_cxx.la (cd .libs && rm -f liblt_pgi_cxx.la && ln -s ../liblt_pgi_cxx.la liblt_pgi_cxx.la) source='foo.cc' object='foo.o' libtool=no \ DEPDIR=.deps depmode=none /bin/sh ../depcomp \ pgCC -DPACKAGE_NAME=\"\" -DPACKAGE_TARNAME=\"\" -DPACKAGE_VERSION=\"\" -DPACKAGE_STRING=\"\" -DPACKAGE_BUGREPORT=\"\" -DSTDC_HEADERS=1 -DHAVE_SYS_TYPES_H=1 -DHAVE_SYS_STAT_H=1 -DHAVE_STDLIB_H=1 -DHAVE_STRING_H=1 -DHAVE_MEMORY_H=1 -DHAVE_STRINGS_H=1 -DHAVE_INTTYPES_H=1 -DHAVE_STDINT_H=1 -DHAVE_UNISTD_H=1 -DHAVE_DLFCN_H=1 -I. -I. -g -c -o foo.o foo.cc /bin/sh ../libtool --tag=CXX --mode=link pgCC -g -o foo -L. foo.o -llt_pgi_cxx pgCC -g -o foo foo.o -L/home/jsquyres/lt-pgi-test/cxx /home/jsquyres/lt-pgi-test/cxx/.libs/liblt_pgi_cxx.a foo.o(.text+0x5e): In function main’:
/home/jsquyres/lt-pgi-test/cxx/foo.cc:12: undefined reference to cxx_one_2(int)' /home/jsquyres/lt-pgi-test/cxx/.libs/liblt_pgi_cxx.a(cxx_one_1.o)(.text+0x1d): In function cxx_one_1’:
/home/jsquyres/lt-pgi-test/cxx/one/cxx_one_1.cc:8: undefined reference to q__tm__2_i__FZ1Z_Z1Z' make[1]: *** [foo] Error 2 make[1]: Leaving directory /home/jsquyres/lt-pgi-test/cxx’
make: *** [all-recursive] Error 1

It also builds static/shared C++ libraries properly, but only if the C++ libraries do not contain templated functions. This is the last issue that I am aware of. Are there special flags that must be given to pgCC to link C++ libraries that include templated functions?

This has always been a messy process. The problem being that templates need to be pre-linked before they can be included in a library. This process is described in Section 11.5 of the PGI user guide but I’ll paraphrase below. Note that the following is only valid for releases prior to 6.0. With 6.0, which should be available shortly, templates will be generated as weak symbols, and the GNU linker will throw out duplicates. None of the following will be necessary (or allowed for that matter) and your current process should just work. In order to do this, however, we had to break compatabiliy between pgCC 6.0 and prior versions so users will need to recompile all objects with 6.0 before they can be used.

For pgCC version 5.2-4 and before:

For each source file in the library, compile it:

pgCC -c --one_instantiation_per_object $(YOUR_FLAGS) file.cc

You must now do an extra pre-link step to instantiate all the templates used in this library, putting all the library .o’s on this line, and no -c flag.

pgCC --one_instantiation_per_object --prelink_objects $(YOUR_FLAGS) file1.o file2.o etc.

Achive or build the shared library as before, except you must include the new hidden templates in the Template.dir directory:

ar qv lib_mylib.a Template.dir/*.o file1.o file2.o etc.

I looked through our TRS mail and talked with Dave, but as of noon today we have not recieved your example so I’ve been unable to confirm that this does indeed solve your issue. However, I see nothing in the log that would lead me to believe that it wouldn’t.

Does libtool have a way to invoke different build methods for different versions of the compiler?

Thanks,
Mat

My mail server logs seem to show that the mail was sent yesterday morning:

Mar 3 08:24:51 milliways sendmail[20300]: j23DORFW020293: to=<trs@pgroup.com>, ctladdr=<jsquyres@lam-mpi.org> (26147/3000), delay=00:00:23, xdelay=00:00:08, mailer=esmtp, pri=425149, relay=pacific.pgroup.com. [192.124.124.8], dsn=2.0.0, stat=Sent (Ok: queued as CFA6381F0)

I unfortunately had a client failure after that and lost my copy of the sent mail itself. :-( Although all my accompanying text is now lost, I can easily re-send the tarball. I’ll answer your other questions in another followup…

For pgCC version 5.2-4 and before:

For each source file in the library, compile it:

pgCC -c --one_instantiation_per_object $(YOUR_FLAGS) file.cc

>
> You must now do an extra pre-link step to instantiate all the templates used in this library, putting all the library .o's on this line, and > **no** > -c flag.
>
> ```text
pgCC --one_instantiation_per_object --prelink_objects $(YOUR_FLAGS) file1.o file2.o etc.

Achive or build the shared library as before, except you must include the new hidden templates in the Template.dir directory:

ar qv lib_mylib.a Template.dir/*.o file1.o file2.o etc.

>

Yoinks.  Ok.

> I looked through our TRS mail and talked with Dave, but as of noon today we have not recieved your example so I've been unable to confirm that this does indeed solve your issue.  However, I see nothing in the log that would lead me to believe that it wouldn't.

I just re-sent it.

> Does libtool have a way to invoke different build methods for different versions of the compiler?

Libtool is just a bunch of shell code -- it can be made to do whatever is necessary.

So -- some followup questions for you:

- How does one reliably tell the difference between versions of pgcc (and friends)?  A quick-n-easy shell test would be preferred.

- Just so that we're absolutely 100% clear, is none of the above procedure necessary for pg* 6.x -- just a normal compile and link (per what libtool was already doing) will work fine?
  • How does one reliably tell the difference between versions of pgcc (and friends)? A quick-n-easy shell test would be preferred.


pgCC -V | awk  ' /pgCC/  { print $2 }' | cut -d. -f1



  • Just so that we’re absolutely 100% clear, is none of the above procedure necessary for pg* 6.x – just a normal compile and link (per what libtool was already doing) will work fine?

After receiving your test program I was able confirm that release 6.0 “just works”.

Thanks,
Mat