Incidentally, I have tried putting the included file in a *.cu file with the same result.
Ok, then you’re not compiling the *.cu file as device code to *.ptx source correctly.
That could actually be a problem inside the your CMake script which might not generate the right custom build rule.
LocalGeometry.h is included in the two examples optixWhitted and optixRaycasting and these should work.
The latter should even work on your headless machine. See below.
The easiest method to change an existing OptiX SDK example would be to copy the whole folder of an example you’re interested in, then rename that folder, then change the top-level CMakeLists.txt to contain add_subdirectory(your_new_example_name).
Then change the copied CMakeLists.txt inside your new folder to have a different project name with
OPTIX_add_sample_executable(your_new_example_name <list_of_src_files>)
Configure and Generate the solution with CMake.
Open the solution, (maybe unload all projects except your new one), rebuild the solution, check if the new example builds without errors.
If that worked, you have effectively duplicated a working OptiX SDK example under a different name.
Start to change it at will.
The optixRaycasting example works offscreen only and just writes a *.ppm image. (But that example is rather untypical for OptiX usage, because it only does ray-triangle intersection testing with OptiX and everything else (ray generation, shading) with native CUDA kernels.)
Anyway, the main() function in there contains no sign of any GUI. You can see there what of the GLFW code needs to be removed to make your newly copied example work offscreen only.
If you want to see how to program OptiX examples without using anything from the SDK but the OptiX API headers, check out my more advanced examples.
I’m also using a different, easier to change script for the custom build rules of the OptiX device code *.cu to *.ptx source translation: nvcuda_compile_ptx.cmake
Find how that is called in each of my examples’ CMakeLists.txt.
Same story to build a new example: Copy an example folder, rename it, add_subdirectory() in the CMakeLists.txt one level higher (in folder “apps”), rename all occurrences of the old example name to your new example name.
Important: Do that replacement in all *.txt and *.cpp files and with partial match. The generated *.ptx files get placed in a relative folder with <app_name>_core name which is defined inside the CMakeLists.txt and used inside the *.cpp files to load the *.ptx files
The more advanced examples rtigo3 and nvlink_shared have their rendering resolution independent of the window client area, which would make it easily possible to convert that mode into an offscreen renderer without any window and GUI.
Both contain a benchmark code path (command line option -m 1 or --mode 1) which is already only rendering offscreen and dumping the resulting rendered image.
That is pure raytracing and much faster than with all the display stuff.
The screenshot functionality is slow though because I do the tonemapping on the CPU, but that could be easily done with CUDA.
(Insider: The shortcut rtigo actually stands for Ray Tracing Image Generator OptiX, so I planned it to be offscreen capable initially.)
You would only need to comment out a lot of stuff which isn’t needed anymore starting here: runApp