hpl-2.0_v15 compile Problem - undefined reference to stuffs within libhpl.a

Accelerated Computing CUDA CUDA Programming and Performance
1

Hi forks,
I am getting problem compiling hpl-2.0_FERMI_v15.
I DO have search on web page related and result nothing.

Installed:
MKL, IMPI, XE2013_sp1, CUDA tool kit 6, openmpi(not in use here), fermi card and working driver of course.

The Make.CUDA is here

Error report:
make[2]: Entering directory /root/t/build/hpl-2.0_FERMI_v15/testing/ptest/CUDA' mpiicc -DAdd__ -DF77_INTEGER=int -DStringSunStyle -DCUDA -I/root/t/build/hpl-2.0_FERMI_v15/include -I/root/t/build/hpl-2.0_FERMI_v15/include/CUDA -I/opt/intel/mkl/lib/intel64/include -I/opt/intel/impi/4.1.3.049/intel64/include -I/usr/local/cuda/include -I/opt/intel/mkl/include -fomit-frame-pointer -O3 -funroll-loops -W -Wall -fopenmp -lpthread -openmp -mt_mpi -nocompchk -o /root/t/build/hpl-2.0_FERMI_v15/bin/CUDA/xhpl HPL_pddriver.o HPL_pdinfo.o HPL_pdtest.o /root/t/build/hpl-2.0_FERMI_v15/lib/CUDA/libhpl.a -L /root/t/build/hpl-2.0_FERMI_v15/src/cuda -ldgemm -L/opt/cuda6/lib64 -lcuda -lcudart -lcublas -L/opt/intel/mkl/lib/intel64 -lpthread /opt/intel/impi/4.1.3.049/intel64/lib/libmpi_mt.so /root/t/build/hpl-2.0_FERMI_v15/lib/CUDA/libhpl.a(HPL_idamax.o): In function HPL_idamax’:
…/HPL_idamax.c:(.text+0x18): undefined reference to idamax_' /root/t/build/hpl-2.0_FERMI_v15/lib/CUDA/libhpl.a(HPL_dgemv.o): In function HPL_dgemv’:
…/HPL_dgemv.c:(.text+0x90): undefined reference to dgemv_' ../HPL_dgemv.c:(.text+0xe5): undefined reference to dgemv_’
/root/t/build/hpl-2.0_FERMI_v15/lib/CUDA/libhpl.a(HPL_dcopy.o): In function HPL_dcopy': ../HPL_dcopy.c:(.text+0x1f): undefined reference to dcopy_’
/root/t/build/hpl-2.0_FERMI_v15/lib/CUDA/libhpl.a(HPL_daxpy.o): In function HPL_daxpy': ../HPL_daxpy.c:(.text+0x36): undefined reference to daxpy_’
/root/t/build/hpl-2.0_FERMI_v15/lib/CUDA/libhpl.a(HPL_dscal.o): In function HPL_dscal': ../HPL_dscal.c:(.text+0x26): undefined reference to dscal_’
/root/t/build/hpl-2.0_FERMI_v15/lib/CUDA/libhpl.a(HPL_dtrsv.o): In function HPL_dtrsv': ../HPL_dtrsv.c:(.text+0xc7): undefined reference to dtrsv_’
/root/t/build/hpl-2.0_FERMI_v15/lib/CUDA/libhpl.a(HPL_dger.o): In function HPL_dger': ../HPL_dger.c:(.text+0x75): undefined reference to dger_’
…/HPL_dger.c:(.text+0xaf): undefined reference to dger_' make[2]: *** [dexe.grd] Error 1 make[2]: Leaving directory /root/t/build/hpl-2.0_FERMI_v15/testing/ptest/CUDA’
make[1]: *** [build_tst] Error 2
make[1]: Leaving directory `/root/t/build/hpl-2.0_FERMI_v15’
make: *** [build] Error 2

please help me,
dye

2

and what may dscal_, dger_, d_copy_, and others be?

seems that you are experiencing a library/ linking problem; are your includes correct?

3

Thanks for your fast reply jimmy,
there only exist -lcuda -lcudart as default in Make.CUDA,
and I think it shouldn’t add something like BLAS or GOTOBLAS2 library.

FYI, the library or included in my Make.CUDA:

MPinc        = -I/opt/intel/impi/4.1.3.049/intel64/include
MPlib        = /opt/intel/impi/4.1.3.049/intel64/lib/libmpi_mt.so
LAinc        = -I/opt/intel/mkl/lib/intel64/include
LAlib        = -L $(TOPdir)/src/cuda  -ldgemm -L/opt/cuda6/lib64 -lcuda -lcudart -lcublas -L/opt/intel/mkl/lib/intel64 -lpthread

INCdir       = $(TOPdir)/include
HPLlib       = $(TOPdir)/lib/$(ARCH)/libhpl.a 
HPL_INCLUDES = -I$(INCdir) -I$(INCdir)/$(ARCH) $(LAinc) $(MPinc) -I/usr/local/cuda/include
HPL_LIBS     = $(HPLlib) $(LAlib) $(MPlib)

I have searched for that(including this forum),
It sounds like the cuBLAS is not compatible with HPL, which needs a wrapper to work with
and that thread didn’t solved this error.
link I found is here

4

Isn’t HPL the HPC library?

What specific device are you using?

5

Thanks for your ninja respon jimmy that’s unbelievable!
what a HPC library is? I managed to compiled HPL with MKL, GOTOBLAS2 with that one unknown.

I am using a normal entry fermi card - GT520m, yeah, it only have 1G. But it should have this kind of compile time error!

Please help,
dye

6

“I put -lmpi_mt everywhere to make sure no lib is NOT compiled with libmpi.so”

“I put an extra -mt_mpi flag to CCFLAGS”

I think the extra flags/ arguments are now confusing cuda

“I used to succeed in compiling & running the (pure) hpl with MKL, part of Make.Linun_PII_CBLAS here”

If you managed to compile with the cuda part excluded, I would think that a purely hypothetical option would be something along the lines of attempting to a) separately compile the cuda part, and b) draw it in as a (pre-compiled) library
The cuda part, as a library, should compile if it is not cluttered by the additional mpl flags; you then just need to find a way to draw it into the main program

Also, just make sure your device is of sufficient compute capability, for the (cuda) functions you are using, and the (cuda) work you are doing on it

7

Hi dye
I came across the same issue as you did. Have you fixed this?