This topic was originally posted as part of the “Meet the Experts” session on Raytracing Gems II by @mechamania.
@ttcheblokov would you be able to help with the NRD related questions?
We are developing a full ray tracing engine for VR, using SDKs like DLSS, RTXDI and NRD. By using our own variable resolution technique, we have been able to render two eyes of 3k x 3k pixels at 90Hz using two RTX 3090 GPUs. Our goal is to eventually get these same results on a single RTX 4090. This technique however, causes some problems with NRD.
What we do:
When shooting the primary rays we transform them non linearly so that areas we want to render at lower resolution are compressed. We adjust our motion vectors so that they correctly represent the non-linear pixel movement and then we compute diffuse/specular using RTXDI. At the end of the pipeline, we upscale the image while transforming it back to a linear view; stretching the lower resolution areas back. The current use for this technique is a form of fixed foveated rendering. In the future we’ll want to extend this into dynamic foveated rendering.
The problem:
To make this technique work with NRD we tried using the compensated motion vectors, this works perfect except for one thing; The specular reflections are not correct when moving the camera. A part of NRD does its own form of temporal reprojection without considering our motion vectors, which causes visual errors since it does not account for the non-linear transformation. We’ve tried modifying NRD ourselves, but we did not succeed in doing so.
Do you have suggestions how to cope with this?
And would it be possible for future versions of NRD to have a user-defined shader function that allows for non-linear transformations to be applied?