With DLSS Ray Reconstruction & NvRTX 5.3 coming soon I have a few questions that hopefully could be cleared up if anyone is able to answer.
Will the Ray Reconstruction be supported in the Path Tracer and/or what kinds of raytracing will be supported with it
I’ve heard that using Ray Reconstruction also has to be used alongside normal DLSS and the two cannot be separated, is this true and if so can DLAA also be used with it?
Could this overhaul other denoisers in engine such as ones used for Lumen, RTXDI, OpenImageDenoise, Optix, etc
In the video presentation they showed footage of Cyberpunk 2077 with the RT Overdrive mode and some dynamically changing emissive lighting. One thing that is apparent is that the lighting reacts quickly and doesn’t have a sort of “prolonged temporal history” for lack of a better term. I’m fairly certain this is caused by the denoiser and with DLSS 3.5 Ray Reconstruction it reacts a lot more quickly. Is this effect something that would also occur within Unreal. I notice that this is a common theme amongst most RT denoisers and it can ruin scenes that have a lot of dynamically changing lighting.
With NvRTX there is both the standard NvRTX branch and the NvRTX_Caustics branch. Why are the two split. Wouldn’t it be more productive to have all the features in a single branch? Though don’t think I’ve seen any missing features from the NvRTX branch in the Caustics one, but is this actually the case?
First of all welcome to the NVIDIA developer forums @buse.simon.
DLSS 3 for UE 5.3 will come as soon as our QA department is satisfied it is good enough to be released.
With respect to the very detailed questions from @mglade30:
Since this is part of the whole DLSS/RTX package, it should be possible to enhance any ray-traced content this way. But right now only path-traced content is officially supported and will likely yield the best results. Bryan gives a very good explanation of this in his YT video.
DLSS is more or less the name of the “package” which encompasses Super Resolution, Frame Generation, DLAA and Ray Reconstruction. As such you will need to enable DLSS to use any of these features. But using DLAA on top of Ray Reconstruction does not make much sense, since one of the points of RR is to avoid AA effects due to noise in the first place.
RTXDI does not use a denoiser, Optix is a different league for professional applications where you are not limited by real-time rendering constraints and the others are not under NVIDIA control. Integrating RR into third-party solutions is not in our scope right now.
When RR is part of the UE plugin in the future and ray-traced lighting is used, we expect to be able to get the same effect, yes.
If I recall correctly than some part of the Caustics branch requires certain patches to run correctly with UE which we do not want to integrate into the main RTX branch. But I don’t exactly know what differences those are. Maybe @jackwang1 knows more.
Basically, NvRTX Caustics branch always rebased on the latest NvRTX branch, Caustics branch has additional feature like Water/Mesh Caustics, Restir GI, as well as Enhanced Translucency stuff. For real game project, NvRTX branch is much easier for developer to integrate the corresponding committee number in order to resolve some specific issues.
Thanks for the responses. I wrote a follow up but it was unnecessarily long and convoluted so I’ll revise it to make it more clear though it will probably still be just as long. 😛
I understand that DLSS encompasses a bunch of different things, that is why I mentioned DLAA. My understanding is that DLAA and DLSS-SR at 100% render scale are effectively the same thing. If I am wrong please correct me but either way the point is to basically have Ray Reconstruction without necessarily downscaling and then upscaling the image via Super Resolution. This mod for Cyberpunk 2077 allows you to modify the DLSS render scales or force DLAA which all work with DLSS-RR.
Only focusing on how it looks and not paying attention to performance, Cyberpunk at 1080p with DLSS-SR Quality (66.6%) doesn’t look all that great. I just don’t think DLSS-SR at any resolution compares to its native resolution without TAA counterpart in terms of clarity, that being said DLSS-SR shines when used at much higher starting resolutions like 4K. Going back to that Cyberpunk mod using DLSS-RR with DLAA undoubtedly looks a lot better than using DLSS-RR with DLSS-SR Quality on 1080p. I do not want be scaling down before using DLSS-RR which is going to cause a loss of quality.
I was under the impression that RTXDI makes use of the ReLAX denoiser from NRD though maybe I’m mistaken. My question basically was would DLSS-RR serve as a replacement for current denoisers that are used for Ray Tracing and Path Tracing inside of Unreal. From the way DLSS-RR has been advertised, I get the impression that it overhauls all the denoisers that are used for RT/PT for a more unified approach to cleaning the image via DLSS-RR.
If this is the case and it seems to work better than previous denoising methods then it sounds like it would be good for rendering. After seeing this DLSS 3.5 NVIDIA Ramen Shop Tech Demo it looks like it cleans the Lumen noise up quite nicely, albeit not perfect, it looks much better from my experience than what’s available to clean up the image that you can currently do inside of Unreal.
This is because when rendering in Unreal with Lumen I feel as if it is borderline impossible to get a noise free image even with all the settings cranked up to their maximum quality. For gaming this might not be the biggest issue but for offline rendering it’s very unfortunate that Lumen seems to not be capable of getting a clean image out. If you chose to use the Temporal and Spatial Anti-aliasing methods and don’t use TAA/TSR then you get the best quality image in terms of anti aliasing and other effects that benefit from it like motion blur or RT shadows though Lumen seems to heavily rely on some accumulation/denoising through TAA/TSR in order to resolve the image.
If you were to only use TAA/TSR while rendering then it’s not a great trade off to make because of the clarity it removes from the image while still not cleaning up all of Lumen’s noise on top missing the massive (I would say essential) improvements you also get from the Temporal and Spatial Anti-aliasing. If you use the Temporal and Spatial Anti-aliasing alongside TAA(TSR) this can also break Lumen even more and cause more noise for some strange reason (I don’t know why this is). I think in the Unreal documentation they say that using the two is not recommended unless absolutely necessary. If we could use the Temporal and Spatial Anti-aliasing alongside DLSS-RR to clean up Lumens noise while rendering that would be great.
When it comes to Pathtracing you can use OpenImageDenoise which honestly is not great for temporal consistency and tends to remove a lot of microdetail but does clean up noise well or you can use Optix which handles better temporally but I would say does a worse job than OpenImage at cleaning up noise. The third option is just increasing the samples until you get an acceptable image which is the preferred option but of course the longest. I haven’t seen what DLSS-RR is capable of yet in terms of the Unreal Pathtracer though.
Here’s some comparisons I did in Cyberpunk 1 & 2 & 3
a quick reply only, so as not to seem as if I leave you hanging.
I had to correct one thing (the italic part in my original post), currently RR is only officially supported with the path-traced version of CyberPunk 2077. But in the future this might change.
Also
you are correct, NRD can/should be applied to RTXDI results. I misunderstood you there, as RTXDI itself works independent of denoising.
Beyond that I think I need to talk to some DLSS experts first and maybe have them chip in to give you a comprehensive answer since you touch on quite a few topics.
Actually I need to amend that, the Caustics branch has not been updated yet to 5.3, there were some other urgent priorities. But it is still planned to come some time soon.
Hi Markus (happy new year btw). I was wondering if you were able to give us path tracing devs any ETA for DLSS-RR’s public SDK release, or how one might go about requesting early access. It’s really been quite a long time coming, and it’s holding up my game’s production (NRD’s image quality isn’t ideal in stereo/VR, and still doesn’t work through Streamline SDK. I have it on authority that that is coming, eventually, but TBH NRD is something I intend to use as a fallback for AMD GPUs, with DLSS-RR for Nvidia GPU owners).
Any info you could share would be greatly appreciated.
Any News on this?
The fact that RR is flagged as coming soon (as Part of DLSS3.5) since a year now is very frustrating.
(DLSS - Download and Get Started | NVIDIA Developer)
I think we all can relate to things taking sometimes longer in development. But if it does you have to own it.
And in this situation the consequence would be to take down the misleading information on the site and to set realistic expectations like: RR wont be released before UE5.5 or DLSS 4.0 as of reasons…or to not mention it at all, if any commitment isn’t possibile. This would at least make it possibile to plan feature roadmap for us.
@MarkusHoHo can you please provide more information as we are also waiting and hoping on RR as we are unhappy with unreal 5’s current offerings. Will it be possible for users to enable RR on games by using the NVidia App? I notice that at present the game needs to support RR and then the NVidia App will be able to optimize for it, and I’m wondering if that would change in the future so that users can optimize games that weren’t built/released for RR.
Oh also, is there any 5.5 branch in progress? I couldn’t find one anywhere. Thanks