OptixHello embeded in new application run in Release but not in Debug mode

Please provide a minimal and complete reproducer in failing state. The given information is insufficient to analyze what you’ve done exactly.

Other than that, if you’re not working inside the OptiX SDK example application framework itself, I would not recommend using the OptiX SDKsutil library in own applications. It’s meant to simplify the OptiX SDK examples. None of its code is strictly required for own projects and you also incur some issues with it, like hardcoded folder names for resources. Check what the sampleDataFilePath(), sampleFilePath() and getSampleDir() functions do. You simply don’t want that in your own applications.

glad and glfw3 are just 3rd party libraries used inside the OptiX SDK. You can include them into your own OptiX application framework from wherever you like. There is not really a need to use the ones built by the OptiX SDK example framework. You could even use completely different libraries to build your own OptiX application framework. This is all independent of the OptiX API itself.

Then the question is how you built your new application’s solution.

If you’re using CMake to build the solution for that, than it’s a matter of finding the 3rd party library include folders, add them to the additional includes and copy their DLLs over to your executable’s module directory.

Compiling the OptiX device code CUDA sources could be done different ways, with custom build rules or using the built-in CMake CUDA language support.
The OptiX SDK examples generate custom build rules on one way, my OptiX 7 examples (link in this sticky thread) do it in a different way with a custom macro NVCUDA_COMPILE_MODULE.

If you’re building the Microsoft Visual Studio solution completely by hand, then it’s basically a matter of adding the required *.cpp and *.h files of your application’s code, then adding the OptiX device CUDA code *.cu files and setting up the CUDA Visual Studio integration options for them exactly how it’s required!
https://raytracing-docs.nvidia.com/optix7/guide/index.html#program_pipeline_creation#program-input