We are developing a full ray tracing engine for VR, using SDKs like DLSS, RTXDI and NRD. We’ve achieved a reliable 90 FPS for two eyes at 3k x 3k using two RTX 3090 GPUs (one per eye). But there is a problem with RTXDI in combination with VR. In some light conditions there is some flickering/noise of the specular reflections. This is acceptable as long as this flickering is the same in both eyes. But light selection is based on a random generator that uses a seed based on the pixel index, which naturally results in different 3D positions being lit differently between the two (slightly offset) eyes. As a result of this the flickering of specular light is different in both eyes, which can ruin depth perception in VR.
Do you have suggestions how we could solve this problem?
Will there be a RTXDI release in the near future suitable for VR?
Sounds like an interesting project!
If the random number generator is based on pixel index (not the world-space position of the surface at the pixel) and the texels for the left and right eyes are contained in a single texture, then you should be able to remap the pixel indices of the right eye to the equivalent indices of the left eye (or vice versa) before feeding the pixel index to the random number generator.
A bit of modulus math to the rescue! e.g. PixelIndexForRNG.xy = PixelIndex.xy % SingleEyeResolution.xy
I don’t know specifics about RTXDI’s feature roadmap, but you can ask the RTXDI team by posting in the forum Raytracing - NVIDIA Developer Forums
Thanks for the answer.
This modulus math is to prevent float rounding errors result in different random seed?
We want for each pixel on the surface that for left and right eye the same light are selected.
But pixels for left and right eye will be slightly different and the the number of pixels for both eyes may even be not the same. So you can not prevent that for one of the eyes some light is choosen which is not choosen for the other eye, Which can result in a flickering on only one eye.
I do not see how to prevent this.
This modulus math is to prevent float rounding errors result in different random seed?
The modulus math is to remap pixel IDs from a second eye into the first eye’s space - so you get the same seed values for both eyes.
for each pixel on the surface that for left and right eye the same light are selected
Right! To accomplish this, you’ll need the random number generator to produce the same results for pixel locations on both eyes. If the pixel resolution differs for each eye, then you won’t be able to use pixel index as a random seed. An idea you could try is to use world-space positions for seed IDs - but you’ll need to quantize world-space into chunks so that both eyes use similar seed IDs.
Hi, thank you for doing this AMA! I’m part of this project as well, I’d like to clarify that the problem doesn’t have to do with a large offset like two eyes on a single texture would have. Both eyes have their own individual texture, so they both use pixel coordinates of e.g. [0…3000], but since VR eyes are slightly offset in 3D space, the world positions that the pixels represent tend to differ slightly per eye.
This difference means that different seeds are used for the same world positions, which results in different noise on 3D surfaces on the left and right eye, which can be a bit off-putting.
I’ve briefly attempted a world-space seed solution like the one you’re suggesting before, the biggest issue was maintaining stability and handling floating point errors with it, but perhaps it deserves another attempt with a closer look at balancing chunk sizes