Problem with motion vectors with NRD

We are developing a full ray tracing engine for VR, using SDKs like DLSS, RTXDI and NRD. By using our own variable resolution technique, we have been able to render two eyes of 3k x 3k pixels at 90Hz using two RTX 3090 GPUs. Our goal is to eventually get these same results on a single RTX 4090. This technique however, causes some problems with NRD.

What we do:

When shooting the primary rays we transform them non linearly so that areas we want to render at lower resolution are compressed. We adjust our motion vectors so that they correctly represent the non-linear pixel movement and then we compute diffuse/specular using RTXDI. At the end of the pipeline, we upscale the image while transforming it back to a linear view; stretching the lower resolution areas back. The current use for this technique is a form of fixed foveated rendering. In the future we’ll want to extend this into dynamic foveated rendering.

The problem:

To make this technique work with NRD we tried using the compensated motion vectors, this works perfect except for one thing; The specular reflections are not correct when moving the camera. A part of NRD does its own form of temporal reprojection without considering our motion vectors, which causes visual errors since it does not account for the non-linear transformation. We’ve tried modifying NRD ourselves, but we did not succeed in doing so.

Do you have suggestions how to cope with this?

And would it be possible for future versions of NRD to have a user-defined shader function that allows for non-linear transformations to be applied?

This is a neat use case that you should definitely talk to the NRD team about. They should be able to help you (better than I can) to get your motion vectors playing nice with the denoiser(s) in the short term.

I don’t know specifics about RTXDI’s feature roadmap, but you can ask the RTXDI team about non-linear transform support by posting in the forum Raytracing - NVIDIA Developer Forums

Ok, I will do. How can I contact the NRD team?
Thx.

Hi mechamania, we can just copy this post into the Forum category Adam suggested and I will add one of the NRD team to the thread.

Ok, Thx a lot!