A problem when i want to createModule

COMPILE ERROR: Optimized debugging is not supported. Module is built with full debug info, but requested debug level is not “OPTIX_COMPILE_DEBUG_LEVEL_FULL”

That happens because in Debug x64 target builds the OptiX SDK examples are translating the *.cu files to *.ptx or *.optixir module input data with the NVCC command line option --device-debug (-G).
See nvcc.exe --help :
--device-debug (-G) Generates debug information for device code. If --dopt is not specified, then turns off all optimizations. Don't use for profiling; use -lineinfo instead.

On the other hand you have set the OptixModuleCompileOptions optlevel = OPTIX_COMPILE_OPTIMIZATION_DEFAULT (which is the same as all optimizations enabled) and you set debugLevel = OPTIX_COMPILE_DEBUG_LEVEL_NONE, which means no debug code is maintained.

That’s why the compiler complains about the full debug input code to optixModuleCreate.

The solution would be to either not compile the *.cu files with the -G option to not generate debug device code (which is also horrendously slow).

Or easier but a lot slower at runtime, you change the OptixModuleCompileOptions to handle debug information when building debug targets. The OptiX SDK example code shows how to do that. Search for #if !defined( NDEBUG ).

That -G option is added to the nvcc command line in OptiX SDK 8.0.0/SDK/CMakeLists.txt line 269:

    foreach( config ${CMAKE_CONFIGURATION_TYPES} )
      if( ${config} STREQUAL "Debug" )
        optix_add_cuda_flag_config( _${config} "-G" )
        optix_add_cuda_flag_config( _${config} "-O0" )
      optix_add_cuda_flag( "-G" )
      optix_add_cuda_flag( "-O0" )

resp. for the NVRTC (NVIDIA CUDA Runtime Compiler) in line 328:

set( CUDA_NVRTC_FLAGS_DEBUG ${SAMPLES_NVRTC_CXX} -arch ${CUDA_MIN_SM_COMPUTE_TARGET} -G -use_fast_math -default-device -rdc true -D__x86_64 CACHE STRING "List of NVRTC options just for the samples" FORCE )

You could also remove it from the command line in the actual cuda_execute_process macro inside OptiX SDK 8.0.0/SDK/CMake/FindCUDA/run_nvcc.cmake if you’re using that. That filters for the -G option anyways to remove the --line-info option which lets nvcc warn when both are given.

Similarly if you’re using a different method to translate the OptiX device code *.cu files in your own application framework yourself.

For example, in my own OptiX application framework I’m using none of the above OptiX SDK CMake scripts but have my own which unconditionally translate the *.cu files without debug information to always have fast ray tracing code.
That is done with this CMake function which generates custom build rules per OptiX device code *.cu file:
with this command line:
and I only set the module compile options depending on some own debug define:

Note that I also would not recommend to set the maxRegisterCount = 50 but you already commented that out.
The better default is maxRegisterCount = OPTIX_COMPILE_DEFAULT_MAX_REGISTER_COUNT (0).

(Please don’t attach screenshots of source code. Just post the text in a code box. That is easier to manage and makes answering faster because one can copy from the text directly.)