OptiX still doesn’t handle PTX files translated with the NVCC device code debug information --device-debug or -G.
That is being worked on. Please compile all OptiX PTX targets without these NVCC debug options.
For Nsight Compute profiling, use OPTIX_COMPILE_OPTIMIZATION_LEVEL_3 and OPTIX_COMPILE_DEBUG_LEVEL_LINEINFO.
For Nsight Visual Studio Edition or cuda-gdb debugging use OPTIX_COMPILE_OPTIMIZATION_LEVEL_0 and OPTIX_COMPILE_DEBUG_LEVEL_LINEINFO.
The OptiX input PTX source is not really representative of what the final SASS microcode is actually doing.
OptiX contains a PTX parser and compiler and effectively rewrites the PTX input. After that there is the CUDA PTX assembler and microcode generator inside the CUDA driver.
The line-info is working good enough to see what SASS code comes from what CUDA line inside the Nsight Compute profiler.
As said, better debugging support for OptiX 7 is being worked on.
BTW, which CUDA Toolkit version are you using?
Generally the newer the OptiX SDK version and display driver, the better. Maybe upgrade to OptiX 7.2.0 which support CUDA 11.1 and requires 456.71 display drivers under Windows resp. 455.28 under Linux.