Request for Motion Confidence Buffer for OptiX Temporal denoise

Gaming and Visualization Technologies Visualization OptiX
1

Hi,

This question already got asked but got no respose: What should I set the motion vector to after disocclusion? - #3 by atilkurtulmus

Basically “what do we set the motion vector to after a disocclusion”, basically when there’s 0 previous frame data to point back to?

I imagine that its not gonna like me inputting NaN, and (0,0) just means that surface has not moved.

This is also a problem for edges, since multiple samples (which give AA) blend to the same pixel and two objects can be moving in fastly different directions, leading to their blended Motion AOV containing garbage.

While I guess in games (but those are 1 spp) for methods like TAAU you’d set the disoccluded pixel to point “off screen” this is clearly not an option here due to the above.

For example the NVidia Video Flow SDK spits out not only a 2D motion field but also a confidence score for each pixel.

This is possible to do for us by instead of accumulating the Motion Vectors together, we’d accumulate their covariance as well.

Then then our counterpart to the Video Flow Confidence would be some form of a rescaled/ramped Covariance of Motion vectors.

I actually do this for Normals when feeding the OptiX non-temporal denoiser, I don’t renormalize them and whenever I’m not confident (due to blending of reflections or refractions) the accumulated normals get closer to 0,0,0

Given that the OptiX SDK only wants the motion field, how do we let it know of the confidence?

2

Given that the OptiX SDK only wants the motion field, how do we let it know of the confidence?

That already exists since OptiX SDK 7.7.0 in form of the OptixDenoiserGuideLayer::flowTrustworthiness field.
https://raytracing-docs.nvidia.com/optix8/api/struct_optix_denoiser_guide_layer.html

The OptiX SDK optixDenoiser example is using it.

Please read the OptiX SDK 8.0.0 Release Notes (link directly below the download button for each individual OptiX SDK version) which lists all API changes. It’s mentioning the addition of the flowTrustworthiness field as well.

The OptiX Programming Guide isn’t mentioning it inside the denoiser chapter, yet. We’ll update that.

The OptiX denoiser accepts null vectors for the flow when you don’t have better data available.
That’s what it wants the first frame to be when there isn’t any motion information available. Just don’t use NaN. ;-)
If the renderer can provide motion vectors from camera movement or transformed geometry, use that data for the disoccluded areas instead of null vectors.
There is always potential for ghosting if there is no better motion value available. The denoiser tries to detect and reduce that already.

1 Like
3

Thanks ever so much!