This question already got asked but got no respose: What should I set the motion vector to after disocclusion? - #3 by atilkurtulmus
Basically “what do we set the motion vector to after a disocclusion”, basically when there’s 0 previous frame data to point back to?
I imagine that its not gonna like me inputting NaN, and (0,0) just means that surface has not moved.
This is also a problem for edges, since multiple samples (which give AA) blend to the same pixel and two objects can be moving in fastly different directions, leading to their blended Motion AOV containing garbage.
While I guess in games (but those are 1 spp) for methods like TAAU you’d set the disoccluded pixel to point “off screen” this is clearly not an option here due to the above.
For example the NVidia Video Flow SDK spits out not only a 2D motion field but also a confidence score for each pixel.
This is possible to do for us by instead of accumulating the Motion Vectors together, we’d accumulate their covariance as well.
Then then our counterpart to the Video Flow Confidence would be some form of a rescaled/ramped Covariance of Motion vectors.
I actually do this for Normals when feeding the OptiX non-temporal denoiser, I don’t renormalize them and whenever I’m not confident (due to blending of reflections or refractions) the accumulated normals get closer to 0,0,0
Given that the OptiX SDK only wants the motion field, how do we let it know of the confidence?