Hi,
I’ve been trying to benchmark a system with NVIDIA Tesla K40c (driver version: 390.30 + cuda version 9.1).
Both nvidia driver and cuda are working fine.
# /usr/local/cuda-9.1/bin/nvcc -V
nvcc: NVIDIA (R) Cuda compiler driver
Copyright (c) 2005-2017 NVIDIA Corporation
Built on Fri_Nov__3_21:07:56_CDT_2017
Cuda compilation tools, release 9.1, V9.1.85
[b]# nvidia-smi
Thu Mar 8 12:36:43 2018
±----------------------------------------------------------------------------+
| NVIDIA-SMI 390.30 Driver Version: 390.30 |
|-------------------------------±---------------------±---------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 Tesla K40c Off | 00000000:C4:00.0 Off | 0 |
| 23% 43C P0 66W / 235W | 0MiB / 11441MiB | 64% Default |
±------------------------------±---------------------±---------------------+
±----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| No running processes found |
±----------------------------------------------------------------------------+[/b]
I’ve installed:
- OpenBLAS
- mpich
- hpl-2.0_FERMI_v15
i’ve already got compiled binaries for mpich and openBLAS and also got binaries and libraries for CUDA as well.
On trying to compile HPL with the following Make.CUDA file, i’m getting error of unavailability of mpi.h file:
make[1]: Leaving directory /root/hpl/hpl-2.0_FERMI_v15_latest' make -f Make.top build_src arch=CUDA make[1]: Entering directory /root/hpl/hpl-2.0_FERMI_v15_latest’
( cd src/auxil/CUDA; make TOPdir=/root/hpl/hpl-2.0_FERMI_v15_latest )
make[2]: Entering directory /root/hpl/hpl-2.0_FERMI_v15_latest/src/auxil/CUDA' /root/hpl/mpich/bin/mpicc -o HPL_dlacpy.o -c -DAdd__ -DF77_INTEGER=int -DStringSunStyle -DCUDA -I/root/hpl/hpl-2.0_FERMI_v15_latest/include -I/root/hpl/hpl-2.0_FERMI_v15_latest/include/CUDA -I-I/root/hpl/mpich/include64 -I/usr/local/cuda-9.1/include -fomit-frame-pointer -O3 -funroll-loops -W -Wall -fopenmp ../HPL_dlacpy.c In file included from /root/hpl/hpl-2.0_FERMI_v15_latest/include/hpl.h:80:0, from ../HPL_dlacpy.c:50: /root/hpl/hpl-2.0_FERMI_v15_latest/include/hpl_pmisc.h:54:17: fatal error: mpi.h: No such file or directory #include "mpi.h" ^ compilation terminated. make[2]: *** [HPL_dlacpy.o] Error 1 make[2]: Leaving directory /root/hpl/hpl-2.0_FERMI_v15_latest/src/auxil/CUDA’
make[1]: *** [build_src] Error 2
make[1]: Leaving directory `/root/hpl/hpl-2.0_FERMI_v15_latest’
make: *** [build] Error 2
Make.CUDA file
- shell --------------------------------------------------------------
----------------------------------------------------------------------
SHELL = /bin/sh
CD = cd
CP = cp
LN_S = ln -fs
MKDIR = mkdir -p
RM = /bin/rm -f
TOUCH = touch
----------------------------------------------------------------------
- Platform identifier ------------------------------------------------
----------------------------------------------------------------------
ARCH = CUDA
Set TOPdir to the location of where this is being built
ifndef TOPdir
TOPdir =/root/hpl/hpl-2.0_FERMI_v15_latest
endif
INCdir = $(TOPdir)/include
BINdir = $(TOPdir)/bin/$(ARCH)
LIBdir = $(TOPdir)/lib/$(ARCH)
HPLlib = $(LIBdir)/libhpl.a
----------------------------------------------------------------------
- Message Passing library (MPI) --------------------------------------
----------------------------------------------------------------------
MPinc tells the C compiler where to find the Message Passing library
header files, MPlib is defined to be the name of the library to be
used. The variable MPdir is only used for defining MPinc and MPlib.
MPdir = /root/hpl/mpich
MPinc = -I$(MPdir)/include64
#MPlib = $(MPdir)/lib64/libmpi.a
MPlib = $(MPdir)/lib/libmpich.so
----------------------------------------------------------------------
- Linear Algebra library (BLAS) -----------------------------
----------------------------------------------------------------------
LAinc tells the C compiler where to find the Linear Algebra library
header files, LAlib is defined to be the name of the library to be
used. The variable LAdir is only used for defining LAinc and LAlib.
#LAdir = $(TOPdir)/…/…/lib/em64t
LAdir = /root/hpl/openblas
LAinc =
CUDA
#LAlib = -L /home/cuda/Fortran_Cuda_Blas -ldgemm -L/usr/local/cuda/lib -lcublas -L$(LAdir) -lmkl -lguide -lpthread
LAlib = -L $(TOPdir)/src/cuda -ldgemm -L/usr/local/cuda-9.1/lib64 -lcuda -lcudart -lcublas -L$(LAdir)/libopenblas.so
----------------------------------------------------------------------
- F77 / C interface --------------------------------------------------
----------------------------------------------------------------------
F2CDEFS = -DAdd__ -DF77_INTEGER=int -DStringSunStyle
----------------------------------------------------------------------
- HPL includes / libraries / specifics -------------------------------
----------------------------------------------------------------------
HPL_INCLUDES = -I$(INCdir) -I$(INCdir)/$(ARCH) $(LAinc) $(MPinc) -I/usr/local/cuda-9.1/include
HPL_LIBS = $(HPLlib) $(LAlib) $(MPlib)
- Compile time options -----------------------------------------------
-DHPL_COPY_L force the copy of the panel L before bcast;
-DHPL_CALL_CBLAS call the cblas interface;
-DHPL_DETAILED_TIMING enable detailed timers;
-DASYOUGO enable timing information as you go (nonintrusive)
-DASYOUGO2 slightly intrusive timing information
-DASYOUGO2_DISPLAY display detailed DGEMM information
-DENDEARLY end the problem early
-DFASTSWAP insert to use DLASWP instead of HPL code
By default HPL will:
*) not copy L before broadcast,
*) call the BLAS Fortran 77 interface,
*) not display detailed timing information.
HPL_OPTS = -DCUDA
----------------------------------------------------------------------
HPL_DEFS = $(F2CDEFS) $(HPL_OPTS) $(HPL_INCLUDES)
----------------------------------------------------------------------
- Compilers / linkers - Optimization flags ---------------------------
----------------------------------------------------------------------
next two lines for GNU Compilers:
CC = /root/hpl/mpich/bin/mpicc
CCFLAGS = $(HPL_DEFS) -fomit-frame-pointer -O3 -funroll-loops -W -Wall -fopenmp
next two lines for Intel Compilers:
CC = mpicc
CCFLAGS = $(HPL_DEFS) -O3 -axS -w -fomit-frame-pointer -funroll-loops -openmp
CCNOOPT = $(HPL_DEFS) -O0 -w
On some platforms, it is necessary to use the Fortran linker to find
the Fortran internals used in the BLAS library.
#LINKER = mpif77
LINKER = /root/hpl/mpich/bin/mpif77
#LINKFLAGS = $(CCFLAGS) -static_mpi
#LINKFLAGS = $(CCFLAGS)
ARCHIVER = ar
ARFLAGS = r
RANLIB = echo
----------------------------------------------------------------------
MAKE = make TOPdir=$(TOPdir)
Some suggesstions which I got were:
- install mpich-devel package (done and can see /usr/include/mpi.h file on my local file system but still the error persists, also tried copying the entire include folder into cpmpile mpich folder under hpl folder but still the issue remains)
Can someone help me with this? I’ve been trying to debug this for long.
Thanks,
Karan