If you want real-time ray tracing on VR which Engine/API/SDK you would use?

If you want to develop real-time ray tracing for VR, which framework you would use and how?

  • Unity and unreal does not support ray-tracing for VR because dx11 attachment, until rtxgi plugin on unreal engine (I do not know how does that work)

  • APIs like OpenGL, DXR(Falcor), VKR, OptiX (OWL) might work. DXR, especially VKR is not beginner friendly (my personal opinion), and OpenGL could be a bit naive to maintain SBT. OptiX (OWL-CUDA) could be better choice for NVIDIA GPUs.

  • If rendering with an API above mentioned is okay, how to connect with the VR headset? I have HTC Vive Pro Eye, so it would be connected through SteamVR. Now both OpenVR (deprecated) and OpenXR might work.

  • What about VRWorks? Does it support Real-Time Ray-Tracing? The examples show DX11, DX12, VK11, VK12, OpenGL but no OptiX example. That could be easier to start with.

I am inexperienced in both ray tracing, and VR. I would like to ask, as an experience programmer, which framework you would chose and why? I am personally giving preference to OptiX-OWL-CUDA, but after practicing some sample codes, I cannot find way how to do it for the VR. Please help.