How to enable rdc in optix program

I’m trying to enable the -rdc option in my optix project, but I don’t know why I get a compilation error after enabling the rdc option.
Below is my CMakelists.txt:

set(CMAKE_CUDA_STANDARD 11)
set(CMAKE_CUDA_SEPARABLE_COMPILATION ON)
set(CUDA_NVCC_FLAGS “${CUDA_NVCC_FLAGS} -rdc=true”)
set_source_files_properties(${CMAKE_CURRENT_SOURCE_DIR}/merge.cu
PROPERTIES CUDA_SOURCE_PROPERTY_FORMAT OBJ
)
OPTIX_add_sample_executable( optixSphere target_name
optixSphere.cu
optixSphere.h
optixSphere.cpp
merge.h
merge.cu
)
target_link_libraries( ${target_name} ${CUDA_LIBRARIES} cudadevrt)
#target_sources(optixSphere PRIVATE ${objects})
#target_link_libraries(optixSphere ${CUDA_LIBRARIES})

I hope to use the rdc option in merge.cu.
As a beginner, any help you can give me is appreciated.

And what was the compilation error you got?

If that is a direct copy of the CMakeLists.txt, you’re using the incorrect quotation marks!

set(CUDA_NVCC_FLAGS “${CUDA_NVCC_FLAGS} -rdc=true”)
# See the difference?
set(CUDA_NVCC_FLAGS "-rdc=true;${CUDA_NVCC_FLAGS}")
# Add  this message after you changed your CUDA_NVCC_FLAGS and see if there are non-ASCII characters printed.
message("CUDA_NVCC_FLAGS = " "${CUDA_NVCC_FLAGS}")

Also CUDA_NVCC_FLAGS is a semicolon separated list.

I’m not sure that will help though. You’re enabling separable compilation and relocatable-device-code for the whole project but only want to do that for a single file and it’s unclear what’s included in there. That might run into linking issues with CUDA runtime calls. I tried this with the optixRaycasting example and it is not working like this.

I would recommend to not use the OptiX SDK application framework for such things but migrate to your own standalone OptiX application framework as soon as possible if you ever want to ship your programs.

Please read these two posts and all links inside the whole threads about options for such OptiX application frameworks, especially since you seem to be trying to build an application with CUDA native kernels and OptiX device code.

There are better ways to set that up with the native CMake LANGUAGE CUDA feature described here.
That CMake example project has the separable compilation and relocatable device code set for the OptiX device code and it should be possible to do the same for the native CUDA code.
https://forums.developer.nvidia.com/t/converting-vs-property-sheet-into-cmake-settings/287159/5
https://forums.developer.nvidia.com/t/why-am-i-getting-optix-dir-notfound/279085/4

Thanks for your quick reply, I modified the quotation marks but it has no effect.
Here are my compilation errors:

CUDA_NVCC_FLAGS = -rdc=true;-arch;sm_50;–use_fast_math;-lineinfo;-Wno-deprecated-gpu-targets
sutil OPTIXIR
ptx_files = /home/ahcrt/SDK/build/lib/ptx/./sutil_generated_camera.cu.optixir;/home/ahcrt/SDK/build/lib/ptx/./sutil_generated_geometry.cu.optixir;/home/ahcrt/SDK/build/lib/ptx/./sutil_generated_shading.cu.optixir;/home/ahcrt/SDK/build/lib/ptx/./sutil_generated_sphere.cu.optixir;/home/ahcrt/SDK/build/lib/ptx/./sutil_generated_whitted.cu.optixir
/usr/bin/ld: …/lib/ptx/optixSphere_generated_merge.cu.o: in function __sti____cudaRegisterAll()': /tmp/tmpxft_00059ae1_00000000-6_merge.cudafe1.stub.c:2: undefined reference to __cudaRegisterLinkedBinary_c9dbf6b3_8_merge_cu_618b2975’
collect2: error: ld returned 1 exit status
make[2]: *** [optixSphere/CMakeFiles/optixSphere.dir/build.make:549: bin/optixSphere] Error 1
make[1]: *** [CMakeFiles/Makefile2:198: optixSphere/CMakeFiles/optixSphere.dir/all] Error 2
make: *** [Makefile:84: all] Error 2[quote=“droettger, post:2, topic:294325, full:true”]

Yes, undefined reference to __cudaRegisterLinkedBinary is what I see inside the optixRaycasting example as well, when trying your idea.

It’s probably not finding the cudadevrt library either.
target_link_libraries( ${target_name} ${CUDA_LIBRARIES} cudadevrt)

Add a message("${CUDA_LIBRARIES} =" "${CUDA_LIBRARIES}") to see what’s inside that.
I would expect some path to your local CUDA toolkit installation for all libs inside that and you didn’t provide that for the cudadevrt.

Also please search the internet for this error.
There seem to be quite some hits for it which mention that this needs a separate link step pointing to this CUDA documentation:
https://docs.nvidia.com/cuda/cuda-compiler-driver-nvcc/index.html#using-separate-compilation-in-cuda

Again, I don’t understand why you do that. What in merge.cu requires this?

Also I think you should be able to get this working when using a standalone CMakeLists.txt with the CMake LANGUAGE CUDA feature.
If you want to develop a program using native CUDA kernels and OptiX device code, that is the better approach.
I consider it a waste of time to try getting this to work inside the OptiX SDK example framework.

Thanks again for your help, I will seriously consider your suggestions.