CMake is not interacting properly with the installed CUDA?

Accelerated Computing CUDA CUDA NVCC Compiler
1

I’ve been pulling my hair out over this one. My company sysadmins are at a loss.

Ever since I’ve upgraded to CUDA 13 I’ve been unable to compile my code base with CMake. The error seems to originate at the CMake build phase, where this error appears:

CMake Error at /usr/share/cmake-4.2/Modules/CMakeDetermineCompilerId.cmake:928 (message):
Compiling the CUDA compiler identification source file
“CMakeCUDACompilerId.cu” failed.

Compiler: /usr/bin/nvcc

Build flags:

Id flags: --keep;–keep-dir;tmp -v

The output was:

255

#$ NVVM_BRANCH=nvvm

#$ SPACE=

#$ CUDART=cudart

#$ HERE=/usr/lib/nvidia-cuda-toolkit/bin

#$ THERE=/usr/lib/nvidia-cuda-toolkit/bin

#$ TARGET_SIZE=

#$ TARGET_DIR=

#$ TARGET_SIZE=64

#$ NVVMIR_LIBRARY_DIR=/usr/lib/nvidia-cuda-toolkit/libdevice

#$
PATH=/usr/lib/nvidia-cuda-toolkit/bin:/usr/local/cuda-13.1/bin:/home/david-cerutti/anaconda3/bin:/home/david-cerutti/anaconda3/condabin:/usr/local/cuda-13.1/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin:/snap/bin

#$ LIBRARIES= -L/usr/lib/x86_64-linux-gnu/stubs -L/usr/lib/x86_64-linux-gnu

#$ rm tmp/a_dlink.reg.c

#$ gcc -D__CUDA_ARCH_LIST__=520 -E -x c++ -D__CUDACC__ -D__NVCC__
-D__CUDACC_VER_MAJOR__=12 -D__CUDACC_VER_MINOR__=0
-D__CUDACC_VER_BUILD__=140 -D__CUDA_API_VER_MAJOR__=12
-D__CUDA_API_VER_MINOR__=0 -D__NVCC_DIAG_PRAGMA_SUPPORT__=1 -include
“cuda_runtime.h” -m64 “CMakeCUDACompilerId.cu” -o
“tmp/CMakeCUDACompilerId.cpp4.ii”

#$ cudafe++ --c++17 --gnu_version=130300 --display_error_number
–orig_src_file_name “CMakeCUDACompilerId.cu” --orig_src_path_name
“/home/david-cerutti/StormmBuild/CMakeFiles/4.2.1/CompilerIdCUDA/CMakeCUDACompilerId.cu”
–allow_managed --m64 --parse_templates --gen_c_file_name
“tmp/CMakeCUDACompilerId.cudafe1.cpp” --stub_file_name
“CMakeCUDACompilerId.cudafe1.stub.c” --gen_module_id_file
–module_id_file_name “tmp/CMakeCUDACompilerId.module_id”
“tmp/CMakeCUDACompilerId.cpp4.ii”

#$ gcc -D__CUDA_ARCH__=520 -D__CUDA_ARCH_LIST__=520 -E -x c++
-DCUDA_DOUBLE_MATH_FUNCTIONS -D__CUDACC__ -D__NVCC__
-D__CUDACC_VER_MAJOR__=12 -D__CUDACC_VER_MINOR__=0
-D__CUDACC_VER_BUILD__=140 -D__CUDA_API_VER_MAJOR__=12
-D__CUDA_API_VER_MINOR__=0 -D__NVCC_DIAG_PRAGMA_SUPPORT__=1 -include
“cuda_runtime.h” -m64 “CMakeCUDACompilerId.cu” -o
“tmp/CMakeCUDACompilerId.cpp1.ii”

#$ cicc --c++17 --gnu_version=130300 --display_error_number
–orig_src_file_name “CMakeCUDACompilerId.cu” --orig_src_path_name
“/home/david-cerutti/StormmBuild/CMakeFiles/4.2.1/CompilerIdCUDA/CMakeCUDACompilerId.cu”
–allow_managed -arch compute_52 -m64 --no-version-ident -ftz=0 -prec_div=1
-prec_sqrt=1 -fmad=1 --include_file_name “CMakeCUDACompilerId.fatbin.c”
-tused --module_id_file_name “tmp/CMakeCUDACompilerId.module_id”
–gen_c_file_name “tmp/CMakeCUDACompilerId.cudafe1.c” --stub_file_name
“tmp/CMakeCUDACompilerId.cudafe1.stub.c” --gen_device_file_name
“tmp/CMakeCUDACompilerId.cudafe1.gpu” “tmp/CMakeCUDACompilerId.cpp1.ii” -o
“tmp/CMakeCUDACompilerId.ptx”

#$ ptxas -arch=sm_52 -m64 “tmp/CMakeCUDACompilerId.ptx” -o
“tmp/CMakeCUDACompilerId.sm_52.cubin”

ptxas fatal : Value ‘sm_52’ is not defined for option ‘gpu-name’

–error 0xff –

Call Stack (most recent call first):
/usr/share/cmake-4.2/Modules/CMakeDetermineCompilerId.cmake:8 (CMAKE_DETERMINE_COMPILER_ID_BUILD)
/usr/share/cmake-4.2/Modules/CMakeDetermineCompilerId.cmake:53 (__determine_compiler_id_test)
/usr/share/cmake-4.2/Modules/CMakeDetermineCUDACompiler.cmake:163 (CMAKE_DETERMINE_COMPILER_ID)
CMakeLists.txt:279 (enable_language)

The final lines there suggest what I think is the problem: CUDA 13.x no longer supports Maxwell (sm_52), but CMake is testing for CUDA by penning a tiny program (CMakeCUDACompilerId.cu) to have the installed NVCC compile, so that it can then identify the NVCC version and location. However, it’s trying to telegraph -arch sm_52, which is not going to fly with CUDA 13.x. How can I tell CMake to pass a different architecture, e.g. sm_75 (the lowest common denominator as of CUDA 13.0)?

Note 1:

I have tried feeding CMake the explicit CUDA install location and NVCC path, via placing at the end of the cmake build line:
```
-DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-13.1 -DCUDAToolkit_ROOT=/usr/local/cuda-13.1 -DCMAKE_CUDA_COMPILER=/usr/local/cuda-13.1/bin/nvcc
```
That will allo wmy code to begin to compile, but I start to get lots of bizarre errors relating to the deprecations in CUDA 13.x, e.g. double4. I have code in my header files that is intended to change everything to double4_16a, and for legacy CUDA 11.x - 12.x compilers I have part of the code as such:

#  if (CUDART_VERSION < 13000)
typedef double4 double4_16a;
typedef longlong4 longlong4_16a;
typedef ulonglong4 ulonglong4_16a;
#  else
...

Despite the fact that I have CUDA 13.1 installed (and this is now my only version of CUDA, having forcibly removed other versions and after checking various $PATH, $LD_LIBRARY_PATH environment variables), it seems to think that CUDART_VERSION is something below 13000. If I compile my very own test program, putting aside CMake and calling nvcc manually, I get nvcc version 13.1 to compile the program and I can then run it to verify that the CUDART_VERSION it sees is 13010 (the CUDA driver version is also reported as 13.1, which I expect). Somewhere, it seems that CMake has some sort of post-hypnotic suggestion to keep referring to some CUDA 12.

Note 2:
I have reinstalled CMake, updating all the way from 3.28 (installed via sudo apt install, which seems to think that 3.28 is still the latest), where the above behavior first appeared, to 4.2.1 (installed via Kitware). Blew away the build directory for my code base and rebuilt from scratch, the error is manifested exactly the same.

Note 3 (Update):
I have reinstalled the NVIDIA Driver, rolling back to version 580, and reinstalled the CUDA Toolkit (rolling back to 12.9). I can once again compile my simple test program with nothing more than nvcc cuda_ver.cu -o cuda_ver on the command line, and the program reports the correct CUDA toolkit version and CUDART_VERSION value (12090). For good measure, I made a typedef of typedef double4 double4_16a; to test backwards compatibility of my fix for the double4 deprecation. It works fine–with the typedef I can make std::vector<double4_16a> when compiling with CUDA 12.9. Still, CMake isn’t cooperating. I’ve blown away all other CUDA installations to make way fro Driver version 580 and cuda-12.9, but now CMake complains:

CMake Error at /usr/share/cmake-4.2/Modules/Internal/CMakeCUDAFindToolkit.cmake:148 (message):
  Couldn't find CUDA library root.
Call Stack (most recent call first):
  /usr/share/cmake-4.2/Modules/CMakeDetermineCUDACompiler.cmake:109 (cmake_cuda_find_toolkit)
  CMakeLists.txt:278 (enable_language)

I looked around and found suggestions to explicitly tell CMake where to find the CUDA root directory during the cmake build command. I found two similar ideas (not sure if one is a misstatement while the other is correct):
```

* Add -DCUDAToolkit_LIBRARY_ROOT=/usr/local/cuda-12.9 and
      -DCUDAToolkit_LIBRARY_DIR=/usr/local/cuda-12.9
-- or --
* Add -DCUDA_TOOLKIT_ROOT_DIR=/usr/local/cuda-12.9

Still hammering on this…

2

Hi there. I believe you may or may not be encountering something related to this post. So I’d suggest to maybe check if you’re on GCC15 and try 14 or 13? Cheers.