Cuda Runtime library and AddressSanitizer incompatibilty

Accelerated Computing CUDA CUDA Programming and Performance
1

Linking and compiling against AddressSanitizer with -fsanitize=address causes cudaRuntimeGetVersion(), and other related cuda functions to not work at all.

// test_cuda.cpp
#include <iostream>
#include <cuda_runtime.h>

int main()
{
        int cudaRuntimeVersion = -1;
        int cudaDriverVersion = -1;
        cudaRuntimeGetVersion(&cudaRuntimeVersion);
        cudaDriverGetVersion(&cudaDriverVersion);
        int numGpusFound = 0;
        cudaGetDeviceCount(&numGpusFound);
        std::cout << "CUDA runtime version: " << cudaRuntimeVersion << std::endl;
        std::cout << "CUDA toolkit version: " << cudaDriverVersion << std::endl;
        std::cout << "Number of cuda GPU(S) found: " << numGpusFound << std::endl;
}

[brock@localhost ~]$ g++ -I/usr/local/cuda-8.0/targets/x86_64-linux/include -L/usr/local/cuda-8.0/targets/x86_64-linux/lib -lcudart test_cuda.cpp -o testcuda.exe

[brock@localhost ~]$ ./testcuda.exe
CUDA runtime version: 8000
CUDA toolkit version: 8000
Number of cuda GPU(S) found: 1

[brock@localhost ~]$ g++ -I/usr/local/cuda-8.0/targets/x86_64-linux/include -fsanitize=address -L/usr/local/cuda-8.0/targets/x86_64-linux/lib -lcudart test_cuda.cpp -o testcuda.exe
[brock@localhost ~]$ ./testcuda.exe
CUDA runtime version: -1
CUDA toolkit version: 8000
Number of cuda GPU(S) found: 0

=================================================================
==2291==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 152 byte(s) in 1 object(s) allocated from:
#0 0x7f5386776800 in __interceptor_malloc …/…/…/./libsanitizer/asan/asan_malloc_linux.cc:86
#1 0x7f5381caeeaf (/home/brock/workspace/lib64/libasan.so.5.0.0+0x8cbeaf)

SUMMARY: AddressSanitizer: 152 byte(s) leaked in 1 allocation(s).

Details:

CUDA 8.0
g++ (GCC) 8.1.0
Driver Version: 375.51 (Same problem, different leak errors with 390.67)

Does anyone know what the problem is? A similar issue has been opened on the sanitizers github: asan: problem calling NVIDIA CUDA libraries · Issue #629 · google/sanitizers · GitHub

2

If you follow the link to a related issue (asan: problem calling NVIDIA CUDA libraries · Issue #629 · google/sanitizers · GitHub), using ASAN_OPTIONS=protect_shadow_gap=0, seems to make a difference:

[brock@localhost ~]$ ASAN_OPTIONS=protect_shadow_gap=0 ./testcuda.exe
CUDA runtime version: 8000
CUDA toolkit version: 9010
Number of CUDA GPU(s) found: 1

I can’t explain why that helps though. (It reports 9010 instead of 8000 as the toolkit version because I’m using the newer driver now)