If this is with earlier OptiX versions before 7.0.0, then restarting the app is probably the only solution.
You would need to be able to destroy the underlying CUDA device contexts because they are probably in an erroneous state after that failure. But that is not an option inside the OptiX API before 7.0.0.
OptiX 1 - 6 use the existing primary CUDA context on the device like the CUDA runtime API to allow OptiX/CUDA interoperability and that doesn’t get destroyed with the OptiX context because it’s owned by the application process.
In OptiX 7 all CUDA context and resource management is explicit and fully under your control.
Means you can create and destroy CUDA contexts per device at will if needed (at least with the CUDA Driver API).
It’s the more modern OptiX API, explicit, multithreading safe, always faster, and more future proof. It’s highly recommended to port older OptiX applications over to that.