nvcc and googletest

Accelerated Computing CUDA CUDA Programming and Performance
1

Hi,

I’m trying to get nvcc working together with googletest (GitHub - google/googletest: GoogleTest - Google Testing and Mocking Framework).

My simple test that does not compile looks like this:

#define GTEST_HAS_TR1_TUPLE 0

#include <gtest/gtest.h>

int main(void) {

}

$ nvcc -I<path/to/gtest>/include test.cu

gives me some warnings and two errors I don’t know how to fix.

Specifically:

<path/to/gtest>/include/gtest/gtest.h(515): warning: attribute “warn_unused_result” ignored

<path/to/gtest>/include/gtest/gtest.h(1098): warning: support for trigraphs is disabled

<path/to/gtest>/include/gtest/gtest.h(515): warning: attribute “warn_unused_result” ignored

<path/to/gtest>/include/gtest/gtest.h(1098): warning: support for trigraphs is disabled

[b]<path/to/gtest>/include/gtest/internal/gtest-internal.h:364: error: `&’ cannot appear in a constant-expression

<path/to/gtest>/include/gtest/internal/gtest-internal.h:364: error: `&’ cannot appear in a constant-expression[/b]

I’m using:

$ nvcc --version

nvcc: NVIDIA ® Cuda compiler driver

Copyright © 2005-2009 NVIDIA Corporation

Built on Thu_Jul_30_09:24:36_PDT_2009

Cuda compilation tools, release 2.3, V0.2.1221

$ g++ --version

g++ (Debian 4.3.2-1.1) 4.3.2

Copyright © 2008 Free Software Foundation, Inc.

This is free software; see the source for copying conditions. There is NO

warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

Any ideas?

Regards

2

nvcc isn’t anywhere near a competent enough C++ compiler to handle something like gtest (you wouldn’t be able to use gtest on the device side anyway). If you want to use gtest in host code, you are going to have to keep it separate from device code.

This works:

avid@cuda:~$ /opt/cuda/bin/nvcc -v -c -I /home/avid/sw/include -o gtest.cc.o gtest.cc

#$ _SPACE_= 

#$ _MODE_=DEVICE

#$ _HERE_=/opt/cuda/bin

#$ _THERE_=/opt/cuda/bin

#$ _TARGET_SIZE_=64

#$ TOP=/opt/cuda/bin/..

#$ LD_LIBRARY_PATH=/opt/cuda/bin/../lib:/opt/cuda/bin/../extools/lib:

#$ PATH=/opt/cuda/bin/../open64/bin:/opt/cuda/bin/../bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games

#$ INCLUDES="-I/opt/cuda/bin/../include" "-I/opt/cuda/bin/../include/cudart"  

#$ LIBRARIES=  "-L/opt/cuda/bin/../lib64" -lcudart

#$ CUDAFE_FLAGS=

#$ OPENCC_FLAGS=

#$ PTXAS_FLAGS=

#$ gcc -c -x c++ "-I/opt/cuda/bin/../include" "-I/opt/cuda/bin/../include/cudart"   -I. -I"/home/avid/sw/include" -m64 -o "gtest.cc.o" "gtest.cc" 

avid@cuda:~$ cat gtest.cc

#define GTEST_HAS_TR1_TUPLE 0

#include <gtest/gtest.h>

int main(void) {

	return 0;

}

where as this doesn’t:

avid@cuda:~$ /opt/cuda/bin/nvcc -v -c -I /home/avid/sw/include -o gtest.cu.o gtest.cu

#$ _SPACE_= 

#$ _MODE_=DEVICE

#$ _HERE_=/opt/cuda/bin

#$ _THERE_=/opt/cuda/bin

#$ _TARGET_SIZE_=64

#$ TOP=/opt/cuda/bin/..

#$ LD_LIBRARY_PATH=/opt/cuda/bin/../lib:/opt/cuda/bin/../extools/lib:

#$ PATH=/opt/cuda/bin/../open64/bin:/opt/cuda/bin/../bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games

#$ INCLUDES="-I/opt/cuda/bin/../include" "-I/opt/cuda/bin/../include/cudart"  

#$ LIBRARIES=  "-L/opt/cuda/bin/../lib64" -lcudart

#$ CUDAFE_FLAGS=

#$ OPENCC_FLAGS=

#$ PTXAS_FLAGS=

#$ gcc -D__CUDA_ARCH__=100 -E -x c++ -DCUDA_NO_SM_12_ATOMIC_INTRINSICS -DCUDA_NO_SM_13_DOUBLE_INTRINSICS -DCUDA_FLOAT_MATH_FUNCTIONS -DCUDA_NO_SM_11_ATOMIC_INTRINSICS  "-I/opt/cuda/bin/../include" "-I/opt/cuda/bin/../include/cudart"   -I. -D__CUDACC__ -C  -I"/home/avid/sw/include" -include "cuda_runtime.h" -m64 -o "/tmp/tmpxft_00007ed7_00000000-4_gtest.cpp1.ii" "gtest.cu" 

#$ cudafe --m64 --gnu_version=40303 --diag_error=host_device_limited_call --diag_error=ms_asm_decl_not_allowed -tused  --gen_c_file_name "/tmp/tmpxft_00007ed7_00000000-1_gtest.cudafe1.c" --stub_file_name "/tmp/tmpxft_00007ed7_00000000-1_gtest.cudafe1.stub.c" --stub_header_file_name "/tmp/tmpxft_00007ed7_00000000-1_gtest.cudafe1.stub.h" --gen_device_file_name "/tmp/tmpxft_00007ed7_00000000-1_gtest.cudafe1.gpu" --include_file_name "/tmp/tmpxft_00007ed7_00000000-3_gtest.fatbin.c" "/tmp/tmpxft_00007ed7_00000000-4_gtest.cpp1.ii" 

/home/avid/sw/include/gtest/gtest.h(948): warning: attribute "warn_unused_result" ignored

/home/avid/sw/include/gtest/gtest.h(1641): warning: support for trigraphs is disabled

#$ gcc -D__CUDA_ARCH__=100 -E -x c -DCUDA_NO_SM_12_ATOMIC_INTRINSICS -DCUDA_NO_SM_13_DOUBLE_INTRINSICS -DCUDA_FLOAT_MATH_FUNCTIONS -DCUDA_NO_SM_11_ATOMIC_INTRINSICS  "-I/opt/cuda/bin/../include" "-I/opt/cuda/bin/../include/cudart"   -I. -D__CUDACC__ -C  -I"/home/avid/sw/include" -m64 -o "/tmp/tmpxft_00007ed7_00000000-5_gtest.cpp2.i" "/tmp/tmpxft_00007ed7_00000000-1_gtest.cudafe1.gpu" 

#$ cudafe --m64 --gnu_version=40303 --c  --gen_c_file_name "/tmp/tmpxft_00007ed7_00000000-6_gtest.cudafe2.c" --stub_file_name "/tmp/tmpxft_00007ed7_00000000-6_gtest.cudafe2.stub.c" --stub_header_file_name "/tmp/tmpxft_00007ed7_00000000-6_gtest.cudafe2.stub.h" --gen_device_file_name "/tmp/tmpxft_00007ed7_00000000-6_gtest.cudafe2.gpu" --include_file_name "/tmp/tmpxft_00007ed7_00000000-3_gtest.fatbin.c" "/tmp/tmpxft_00007ed7_00000000-5_gtest.cpp2.i" 

#$ gcc -D__CUDA_ARCH__=100 -E -x c -DCUDA_NO_SM_12_ATOMIC_INTRINSICS -DCUDA_NO_SM_13_DOUBLE_INTRINSICS -DCUDA_FLOAT_MATH_FUNCTIONS -DCUDA_NO_SM_11_ATOMIC_INTRINSICS  "-I/opt/cuda/bin/../include" "-I/opt/cuda/bin/../include/cudart"   -I. -D__CUDABE__  -I"/home/avid/sw/include" -m64 -o "/tmp/tmpxft_00007ed7_00000000-7_gtest.cpp3.i" "/tmp/tmpxft_00007ed7_00000000-6_gtest.cudafe2.gpu" 

#$ filehash -s " " "/tmp/tmpxft_00007ed7_00000000-7_gtest.cpp3.i" > "/tmp/tmpxft_00007ed7_00000000-8_gtest.hash"

#$ nvopencc  -TARG:compute_10 -m64  "/tmp/tmpxft_00007ed7_00000000-7_gtest.cpp3.i"  -o "/tmp/tmpxft_00007ed7_00000000-2_gtest.ptx"

#$ ptxas --key="56b2f5d681b54543"  -arch=sm_10  "/tmp/tmpxft_00007ed7_00000000-2_gtest.ptx"  -o "/tmp/tmpxft_00007ed7_00000000-9_gtest.sm_10.cubin" 

#$ fatbin --key="56b2f5d681b54543" --source-name="gtest.cu" --usage-mode=" " --embedded-fatbin="/tmp/tmpxft_00007ed7_00000000-3_gtest.fatbin.c" "--image=profile=sm_10,file=/tmp/tmpxft_00007ed7_00000000-9_gtest.sm_10.cubin" "--image=profile=compute_10,file=/tmp/tmpxft_00007ed7_00000000-2_gtest.ptx" 

#$ cudafe++ --m64 --gnu_version=40303 --diag_error=host_device_limited_call --diag_error=ms_asm_decl_not_allowed --parse_templates  --gen_c_file_name "/tmp/tmpxft_00007ed7_00000000-1_gtest.cudafe1.cpp" --stub_file_name "/tmp/tmpxft_00007ed7_00000000-1_gtest.cudafe1.stub.c" --stub_header_file_name "/tmp/tmpxft_00007ed7_00000000-1_gtest.cudafe1.stub.h" "/tmp/tmpxft_00007ed7_00000000-4_gtest.cpp1.ii" 

/home/avid/sw/include/gtest/gtest.h(948): warning: attribute "warn_unused_result" ignored

/home/avid/sw/include/gtest/gtest.h(1641): warning: support for trigraphs is disabled

#$ gcc -D__CUDA_ARCH__=100 -E -x c++ -DCUDA_NO_SM_12_ATOMIC_INTRINSICS -DCUDA_NO_SM_13_DOUBLE_INTRINSICS -DCUDA_FLOAT_MATH_FUNCTIONS -DCUDA_NO_SM_11_ATOMIC_INTRINSICS  "-I/opt/cuda/bin/../include" "-I/opt/cuda/bin/../include/cudart"   -I. -I"/home/avid/sw/include" -m64 -o "/tmp/tmpxft_00007ed7_00000000-10_gtest.ii" "/tmp/tmpxft_00007ed7_00000000-1_gtest.cudafe1.cpp" 

#$ gcc -c -x c++ "-I/opt/cuda/bin/../include" "-I/opt/cuda/bin/../include/cudart"   -I. -I"/home/avid/sw/include" -m64 -o "gtest.cu.o" "/tmp/tmpxft_00007ed7_00000000-10_gtest.ii" 

/home/avid/sw/include/gtest/internal/gtest-internal.h:359: error: `&' cannot appear in a constant-expression

/home/avid/sw/include/gtest/internal/gtest-internal.h:359: error: `&' cannot appear in a constant-expression

# --error 0x100 --

avid@cuda:~$ cat gtest.cu

#define GTEST_HAS_TR1_TUPLE 0

#include <gtest/gtest.h>

int main(void) {

	return 0;

}

See what a different in compilation the source filename extension makes?

3

So how do I write test code then? I do not know how to separate. I write some kernel in a cu file and some host code to execute that kernel. But I think I have to compile the host code with nvcc as well (because of the kernel call). So let’s say I compile this into a library and then I have a cc file that calls the function (that starts the kernel) from the host part. Is this expect to work?

Regards,
Jens

4

I think you have the basic idea. Segregate code that must pass through nvcc and code that must not (ie. that containing the test code) and link the whole lot together. You can usually get away with writing a minimal host side wrapper function (or template) that encapsulates the CUDA kernel and compile that with nvcc as a .cu file, and the rest of your application with the host compiler and use the host linker to put everything together. The only catch point can be C++ function name mangling, so you need to declare your nvcc side functions as extern “C” {}. After than it should be straightforward.

Another alternative, although one I don’t have much experience with, is to compile the kernel code to a cubin file, and the use the CUDA driver API to load and setup the kernel launch at runtime. This is a lot more involved that using the runtime API, but it has the advantage that everything, including the host code surrounding the kernel call itself should be able to be instrumented with google test without interference from nvcc.

5

Thanks very much. I have gtest working now. In general the insight of keeping things separate is very useful. nvcc should only do what it is made for.

6

Can GTEST (google test for c++ )be used for testing device code as well? Anybody having experience of this using cuda version 3.0 and above? I saw in forums that it works for cuda version 2.3 emulation mode, but not sure for above 3.0 (with no emulation mode support).

Please reply.